Column 2 : Software design

{ Category Archives } Software design

Salesforce Releases Force.com Visual Process Manager

A couple of months back, there was a private discussion amongst the Enterprise Irregulars about who Salesforce.com was going to buy next, and there was a thought in the back of my mind that it might be a BPM vendor. Since that time, two BPM vendors have been acquired, but not by Salesforce: instead, they launched their own Force.com Visual Process Manager for designing and running processes in the cloud.

However, they seem determined to keep it a secret: first, the Visual Process Manager Demo video on YouTube has been made private (that’s just a screen snapshot of the cached video below), and second, I was unable to get a call back in response to the technical questions that I had during the demo.

For those of you unfamiliar with options for Salesforce application development ( as I mostly was before this briefing), Force.com is the platform originally built for customizing the Salesforce CRM offering, which became a necessity for larger customers requiring customization of data, UI and business logic. Customers started using it as a general business application development and delivery platform, and there are now 135,000 custom applications on Force.com, ranging from end-user-created databases and analytics, to sophisticated order management and e-commerce systems that link directly to customers and trading partners, and can update data from other Salesforce applications. In the past four years, they’ve gone from offering transactional applications to entire custom websites, and are now adding collaboration with Chatter.

As you might guess, there are processes embedded in many applications; classic software development might view these as screen flows, that is, the process for a person to move from one screen to another within an application. Visual Process Manager came about for exactly that purpose: customers were building departmental enterprise applications applications with process (screen flow) logic, but were having to use a lot of code in order to make it happen.

Salesforce acquired Informavores for their process design and execution engine, and that became Visual Process Manager. This is primarily human-centric BPM; it’s not intended as a system-centric orchestration platform, since most customers already have extensive middleware for integration, usually on-premise and already integrated with their Force.com apps so don’t need that capability. That means that although a process step can call a web service or pretty much anything else within their existing Force.com platform, asynchronous web service calls are not supported; this would be expected to be done by that middleware layer.

The process designer allows you to create a process map, then create a form that is tied to each human-facing step in the process map. Actions are bound to the buttons on the forms, where a form may be a screen for internal use, or a web page for a public user to access. You can also add in automated steps and decisions, as well as calling subprocesses and sending emails. It uses a fairly simple flowchart presentation for the process map, without swimlanes. There isn’t a lot of event handling that I could see, such as handling an external event that cancels an insurance quote process. There’s a process simulator, although that wasn’t demonstrated.

Visual Process Manager is priced at $50/user/month for Force.com Enterprise and Unlimited Edition customers, although it’s not clear if that’s just for the application developers, or if there’s a runtime licensing component as well.

Similar to what I said about SAP NetWeaver BPM, this isn’t the best BPMS around – in fact, in the case of Force.com, it’s little more than application screen flow – but it doesn’t have to be the best in class: it only has to be the best BPMS for Force.com customers.

Designing compelling customer-facing user experiences #BTF09

For the last breakout of the day before the final keynote, I attended Mike Gualtieri’s session on designing customer-facing user interfaces. He started with the idea that application developers have to be involved in user experience design, and not just leave it to the designers (which is, of course, exactly what we did in the bad old days of development when there was no such thing as a user experience designer). Forrester defines user experience as “users’ perceptions of the usefulness, usability, and desirability of a Web application based upon the sum of all their direct and indirect interactions with it”, and propose that a great UX is useful, usable and desirable.

User experience impacts how your customers feel about you, and it’s also not just about the interfaces that the customer works with directly: a second-hand interface can also impact the customer experience, as you know if you’ve ever waited ages while a hotel desk clerk clicks their way through a complex interface in order to check you in. A good UX can increase purchases, retain customers and attract more customers; leaving it to chance hurts your conversion rates, alienates customers and increases your development costs due to redesign and redevelopment.

Gualtieri argues that UX design is Lean (although you could argue that only good UX design is Lean), and sets out best practices for good UX design:

Become your users, by listening to their needs, observing them in their natural habitat, creating personas, and empathizing with them. Users typically don’t articulate their needs fully or accurately so it’s not sufficient to just listen to them, but they will demonstrate them if you watch how they do their work. This type of user research is not the same as gathering requirements from business stakeholders; remember the Henry Ford quote: “If I asked people what they wanted, they would have said faster horses”. Forrester uses personas in their own materials – for example, representing an application development manager, complete with picture and name – and I’m seeing some companies such as Global 360 use these for BPMS user interface design.
Design first, and understand constraints and potential areas of change as well as the different personas that you discovered in your user research. Keep in mind that you have to serve business goals by serving user goals. Create rough prototypes first, and don’t rush into development or lock into a design too soon. There is some amount of art UX design, so don’t assume that tools can do it for you. Keep the basic principles in mind: useful, usable and desirable.
Trust no one: test your designs. It doesn’t matter how many experts review the designs, there is no better review of some features than testing the UX with a range of intended users. Remember that this is not just about usability, it’s also about usefulness and desirability.
Inject UX design into your software development life cycle. Everyone on the team should understand why UX design is important, and be incented to help create great UX. UX design should be part of your development process, and requires someone on the team to own the UX design efforts. You still need to use the same techniques as discussed in the other best practices, not just do the design in isolation from the users, but having it integrated into the development team will improve the overall software design.

He finished with the ideas that your development efforts are essentially wasted if the user experience isn’t done right, but it doesn’t have to add a lot of time or money to your project. Good UX design is the mark of a great application development team.

2009 10 08 Sandy Kemsley Forrester Comments (0)

Can packaged applications ever be Lean? #BTF09

Chip Gliedman, George Lawrie and John Rymer participated in a panel on packaged applications and Lean.

Rymer argued that packaged apps can never be Lean, since most are locked down, closed engines where the vendor controls the architecture, they’re expensive and difficult to upgrade, they use more functions than customers use, they provide a single general UI for all user personas, and each upgrade includes more crap that you don’t need. I tend to be on his side in this argument about some types of appls (as you might guess about someone who used to write code for a living), although I’m also a fan of buy over build because of that elusive promise of a lower TCO.

Gliedman argued the opposite side, pointing out that you just can’t build the level of functionality that a packaged application provides, and there can be data and integration issues once you abandon the wisdom of a single monolithic system that holds all your data and rules. I tend to agree with respect to functionality, such as process modeling: you really don’t want to build your own graphical process modeler, and the alternative is hacking your own process together using naked BPEL or some table-driven kludge. Custom coding also does not guarantee any sort of flexibility, since many changes may require significant development projects (if you write bad code, that is), rather than a package app that may be more configurable.

It’s never a 100% choice between packaged apps and custom development, however: you will always have some of each, and the key is finding the optimal mix. Lean packaged apps tend to be very fit-to-purpose, but that means that they become more like components or services than apps: I think that the key may be to look at composing apps from these Lean components rather than building Lean from scratch. Of course, that’s just service-oriented architecture, albeit with REST interfaces to SaaS services rather than SOAP interfaces to internal systems.

There are cases where Lean apps are completely sufficient for purpose, and we’re seeing a lot of that in the consumer Web 2.0 space. Consider Gmail as an alternative to an Exchange server (regardless of whether you use Outlook as a desktop client, which you can do with either): less functionality, but for most of us, it’s completely sufficient, and no footprint within an organization. SaaS, however, doesn’t not necessarily mean Lean. Also, there are a lot of Lean principles that can be applied to packaged application deployment, even if the app itself isn’t all that Lean: favoring modular applications; using open source; and using standards-based apps that fit into your architecture. Don’t build everything, just the things that provide your competitive differentiation where you can’t really do what you need in a packaged apps; for those things where you are doing the same all every other company, suck it up and consider a packaged app, even if it’s bulky.

Clearly, Gliedman is either insane or a secret plant from [insert large enterprise vendor name here], and Rymer is an incurable coder who probably has a ponytail tucked into his shirt collar. Nonetheless, an entertaining discussion.

2009 10 08 Sandy Kemsley Forrester
SOA Comments (5)

How Can Lean Software Enable You To Better Serve The Business? #BTF09

John Rymer and Dave West presented a breakout session in the application development track on how Lean software development practices can be applied in your business. This obviously had a big focus on Agile, and how it can be used within large organizations. Unlike what some people think, Agile isn’t cowboy coding: it is quite disciplined, but it is optimized for delivering the right thing (from a business standpoint) in the minimal time. It’s all based on four principles: deliver the right product, provide hard value, simplify the platform, and allow efficient evolution. An optimal strategy depends on all four of those elements, but Agile projects may deliver on two or three of them, proving the value of Agile before a full Agile strategy is in use.

In order to apply these principles across your entire application development portfolio, you need a strategy that addresses these elements, and provides some way to measure the impact of such a strategy. Delivering the right product requires a focus on people and talent, and the industrial concepts of mass customization rather than mass production; providing hard value requires linking your development process to value streams with their focus on investment return; simplifying the platform requires a focus on tools and technology; and allowing efficient evolution requires optimizing work processes both within development teams and across the organization. I especially liked their chart comparing today’s practices in tools and technologies against Lean practices:

Today’s practices	Lean practices
Install for today and tomorrow	Install for today, architect for tomorrow
Configure a general UI for many users	Design for people in their work roles
Adopt integrated suites	Adopt narrow-purpose modules and services
No component substitution is allowed	Component substitution is allowed
Architectural evolution is slow by design	Architectural evolution is constant by design

There are ways to bring Agile into an organization, even when budgets are flat and there is the perception that legacy systems just can’t be replaced without yet another huge project expense. Likely, your developers are already practicing some Agile methods already, and you could easily gain permission to prove these out in non-critical systems development.

Good session, with a high-speed tag team between Rymer and West. Unfortunately, the logistics aren’t quite as good as the general sessions: too-small meeting rooms requiring elevator access from the main conference area, no tables and no wifi coverage (at least in the room that I was in at this time).

2009 10 08 Sandy Kemsley Forrester Comments (1)

Software AG partner frameworks

I had lunch today at Innovation World with a Software AG partner that will be releasing one of the industry vertical frameworks (I didn’t ask if I could use their name, so am withholding it for now). They see the frameworks as a necessity to even demonstrate BPM to a vertical business, as well as providing a base on which to create a custom solution. A few bits about the partner frameworks:

The partner that I spoke with does not plan to productize the framework; rather, it will be used as the starting point for a custom solution for a client. This is how I see most companies implementing frameworks on top of any BPMS, with a mixed degree of success: although it’s difficult to turn some or all of a framework into a product rather than a service, especially for the services companies who normally create them, a productized framework can have advantages when it comes to maintenance and support. Customers need to be aware that a non-productized framework is really just another piece of custom code, and the long-term maintenance costs will reflect that.
This partner plans to retain the intellectual property of the framework and any custom code that they build on it, allowing them to roll the code developed for any customer back into the framework for resale. This is great for the industry in general, and future customers in particular, but customers would need to ensure that they specify any processes to which they do not want to give up IP rights.
Software AG does not provide guidelines or rules for what should or should not be in a framework, or how to create one. In their online partner forum, however, they describe the existing frameworks so that partners can get an idea of what should be in one.
Software AG is not certifying the partner frameworks, so customers need to do their own due diligence on the strength of the solution. Some sort of certification program would likely improve customer confidence in the third-party frameworks.

Vertical industry frameworks are definitely the new black in BPM these days: in addition to Software AG’s program of mixed internal and third-party frameworks, Savvion announced a fairly ambitious framework program with one tier of components built by Savvion and one by their partners, and TIBCO provides some vertical frameworks as a vertical marketing tool.

I’m all for providing a leg up for customers to start working with a BPMS in their industry, but we need to be clear about whether something is a true framework or a set of unsupported templates, understand the value that a framework can bring, and know the pitfalls of a framework approach. I’ve seen some pretty big BPMS implementations that went totally off track because of the use of a non-productized framework: the framework became brittle legacy custom code before it was even in production, was seriously impacted by a minor upgrade to the underlying BPMS platform, and did not allow access to recent modeling and optimization functions provided in the BPMS since it was designed and built for a previous version.

In general, I think that most “frameworks” that overlay BPMS’ are actually templates, providing marketing and sales support for the underlying product in that vertical, but not providing a lot of value in terms of a code base. Those that do have a significant code base are usually not productized, hence need to be evaluated as a big chunk of custom code: although the initial purchase price is likely lower than having all that code written for you, you have to consider the ongoing maintenance costs.

2008 11 04 Sandy Kemsley BPM Comments (1)

Business Rules Forum: Pedram Abrari on MDA, SOA and rules

Pedram Abrari, founder and CTO of Corticon, did a breakout session on model-driven architecture, SOA, and the role that rules play in all of this. I’m also in the only room in conference center that’s close enough to the lobby to pick up the hotel wifi, and I found an electrical outlet, so I’m in blogger heaven.

It’s a day for analogies, and Abrari uses the analogy of car for a business application: the driver representing business, and the mechanic representing IT. A driver needs to have control over where he’s going and how he gets there, but doesn’t need to understand the details of how the car works. The mechanic, on the other hand, doesn’t need to understand where the driver is going, but keeps the car and its controls in good working order. Think of the shift from procedural to declarative development concepts, where we’ve moved from stating how to do something, to what needs to be done. A simple example: the difference between writing code to sum a series of numbers, and just selecting a range of cells in Excel and selecting the SUM function.

The utopia of model-driven architecture (MDA) is that business applications are modeled, not programmed; they’re abstract yet comprehensive, directly executable (or at least deployable to an execution environment without programming), the monitoring and analytics are tied directly to the model, and optimization is done directly on the model. The lack of programming required for creating an executable model is critical for keeping the development in the model, and not having it get sucked down into the morass of coding that often happens in environments that are round-trippable in theory, but end up with too much IT tweaking in the execution environment to ever return to the modeling environment.

He then moved on to define SOA: the concept of reusable software components that can be loosely coupled, and use a standard interface to allow for platform neutrality and design by contract. Compound/complex services can be built by assembling lower-level services in an orchestration, usually with BPM.

The key message here is that MDA and SOA fit together perfectly, as most of us are aware: those services that you create as part of your SOA initiative can be assembled directly by your modeling environment, since there is a standard interface for doing so, and services provide functionality without having to know how (or even where) that function is executed. When your MDA environment is a BPMS, this is a crystal-clear connection: every BPMS provides easy ways to interrogate and integrate web services directly into a process as a process step.

From all of this, it’s a simple step to see that a BRMS can provide rules/decisioning services directly to a process; essentially the same message that I discussed yesterday in my presentation, where decision services are no different than any other type of web services that you would call from a BPMS. Abrari stated, however, that the focus should not be on the rules themselves, but on the decision service that’s provided, where a decision is made up of a complete and consistent set of rules that addresses a specific business decision, within a reasonable timeframe, and with a full audit log of the rules fired to reach a specific decision in order to show the decision justification. The underlying rule set must be declarative to make it accessible to business people.

He ended up with a discussion of the necessity to extract rules out of your legacy systems and put them into a central rules repository, and a summary of the model-driven service-oriented world:

Applications are modeled rather than coded
Legacy applications are also available as web services
Business systems are agile and transparent
Enterprise knowledge assets (data, decisions, processes) are stored in a central repository
Management has full visibility into the past, present and future of the business
Enterprises are no longer held hostage by the inability of their systems to keep up with the business

Although the bits on MDA and SOA might have been new to some of the attendees, some of the rules content may have been a bit too basic for this audience, and/or already covered in the general keynotes. However, Abrari is trying to make that strong connection between MDA and rules for model-driven rules development, which is the approach that Corticon takes with their product.

2008 10 29 Sandy Kemsley BRM
BusinessRulesForum
SOA Comments (2)

PegaWorld: Paul Kompare on JPMorgan Chase’s agile methodology

Paul Kompare, SVP of commercial banking technology at JPMorgan Chase discussed their Pega implementation of a straight-through processing infrastructure for commercial loans. He gave us a brief view of the current environment and the proposed solution, then moved on to discuss their agile implementation approach. Although they refer to this as Agile, it’s still a bit waterfall-like: the sprints don’t result in released code, but in checkpoint demos, and these are the points when the business representatives interact with the development team rather than being a co-located team (which likely would not have been possible since they rely heavily on offshore development resources). However, it’s a big improvement over their old waterfall methodology.

They delivered the project in two phases, each with three iterations:

Happy paths and primary flows
Exception paths and secondary flows
UI, integration and reports

In each iteration, they establish and sign off on the criteria, then use Pega to directly capture objectives and model the processes. With this relatively agile process, they improved project sponsorship as well as getting early buy-in from the business, since they were able to demonstrate something earlier and more frequently. Using PRPC also gives the business managers more visibility into the business processes and their underlying logic, rather than having those processes locked up inside opaque application code. They found that the tools gave them more agility and flexibility during implementation, greater reusability and faster time to market, as well as allowing potential changes to be identified earlier.

They did have some challenges with adapting to an agile approach: this type of approach assumes that changes to the design and functionality of the system will occur during development, and relies on rolling out a phased series of small, self-contained components. From a funding standpoint, it’s almost impossible to issue fixed-price contracts for agile development, since there is not a really fixed statement of work on which to base the proposed price. I’ve seen cases where a third-party services firm doesn’t really get agile methodology, and there is a huge amount of overhead as they attempt to shoehorn their waterfall deliverables into each iteration of the agile development, or they just abandon the agile approach and go back to waterfall.

There’s also major changes to roles and responsibilities: the business participants have much greater responsibility during design and as the system rolls out, and having them trained in the design tools is critical.

He concluded that adopting agile development methodologies has been a challenge for them, but that it’s definitely helping them to achieve shorter development cycles. There’s very little here that’s specific to commercial loans, Pega implementations, or even BPM; these same factors would be seen in any organization shifting to an agile approach. However, Kompare made the point that they were driven to consider an agile approach because the Pega tools tend to work better in that environment than with a traditional development methodology.

2008 10 21 Sandy Kemsley Pegasystems Comments (0)

PegaWorld: Meryl Stewart and Kelly Karlen on Business-IT Collaboration at BlueCross BlueShield

Last session of the day, and Meryl Stewart and Kelly Karlen of BlueCross BlueShield of Minnesota talked about maximizing BPM value through business and IT collaboration. They established a shared business-IT objective of enabling the business to manage their frequently-changing business rules to provide agility, while still maintaining environmental stability by following the necessary change management procedures.

They’ve wrapped some procedures around their projects to explicitly call this out, as well as explicit governance layers for processes and rules. Some of this — a big part — is about well-defined roles and responsibilities, such as a business rules steward. They categorize these procedures and methods by collection, execution and optimization stages, and walked us through each of the roles in each of the stages.

In the collection stage, they have a pretty structured way to create business rules and shore them in an enterprise repository; this is independent of their BPM technology, since not all processes end up being automated.

They wanted to make execution more efficient, so combined their RUP methodology with Pega’s RUP-like methodology and lightened it up to create a more agile “RUP Lite” (although as they walk through it, it doesn’t feel so light but it does have fairly frequent code releases). Within that methodology, they have a number of additional roles to work on the business to technology transformation of the execution phase, and definite rules about who can do what types of changes and who does the associated testing. There’s a level of semi-technical analyst who can do a lot of the non-coding changes.

The optimization stage is where business agility happens, but this was addressed pretty quickly and seemed to be some sort of standard change management procedure.

This definitely shows some good software development practices, but there’s nothing particularly innovative here that can’t be replicated elsewhere as long as you can get the collaboration part to work; collaboration is primarily a function of finding people on both sides of the business-IT divide who can see over the wall to the other side, and maybe even straddle the divide somewhat with their skills.

They’ve applied the methodology to a couple of projects and have seen positive ROI, and very few coding changes since most of the process tuning can be done by business users or the semi-technical analysts. In one process, they’ve had 11 rule changes in 4 months with resultant savings of $820k in the improved processes; if IT were to have been involved in these changes, only $126k of the savings could have been realized in the same timeframe due to IT project schedules — a good measure of the value of agility provided by allowing the business to change business rules. Fundamentally, they changed an 8-week IT build cycle to 10 days or less by allowing the business to change the rules, but still following a test and deploy cycle that keeps lT happy.

That’s it for today; there’s a reception, then dinner and a cruise on the Potomac to view the monuments by night. The esteemed Dr. Michael zur Muehlen will not be joining us in spite of being right across the river in Arlington; when I invited him, he gave some lame excuse about just getting back from Seoul.

2008 10 20 Sandy Kemsley BPM
Pegasystems Comments (0)

TUCON: Using BPM to Prioritize Service Creation

Immediately after the Spotfire-BPM session, I was up to talk about using BPM to drive top-down service discovery and definition. I would have posted my slides right away, but one of the audience members pointed out that the arrows in the two diagrams should be bidirectional (I begged forgiveness on the grounds that I’m an engineer, not a graphic artist), so I fixed that up before posting to Slideshare:

| View | Upload your own

My notes that I jotted down before the presentation included the following:

SOA should be business focused (even owned by the business): a top-down approach to service definition provides better alignment of services with business needs.
The key is to create business-granular services corresponding to business functions: a business abstraction of SOA. This requires business-IT collaboration.
Build thin applications/processes and fat services to enable agile business processes. Fat services may have multiple operations for different requirements, e.g., retrieving/updating just the customer name versus the full customer record in an underlying system.
Shared business semantics are key to identifying reusable business services: ensure that business analysts creating the process models are using the same terminology.
Seek services that have the greatest business value.
Use cases can be used to identify candidates for services, as can boundary crossings activity diagrams.
Process decomposition can help identify reusable services, but it’s not possible to decompose and reengineer every process: look for ineffective processes with high strategic value as targets for decomposition.
Build the SOA roadmap based on business value.
SOA isn’t (just) about creating services, it’s about building business processes and applications from services.
Services should be loosely-coupled and location-independent.

There were some interesting questions arising from this, one being when to put service orchestration in the services layer (i.e., have one service call another) and when to put it in the process layer (i.e., have a process call the services). I see two facets to this: is this a business-level service, and do you want transparency into the service orchestration from the process level? If it’s not a business-level service, then you don’t want business analysts having to learn enough about it to use it in a process. You can still do orchestration of technical services into a business service using BPM, but do that as a subprocess, then expose the subprocess to the business analyst; or push that down to the service level. If you’re orchestration business-level services into coarser business-level services, then the decision whether to do this at the service or process level is about transparency: do you want the service orchestration to be visible at the process level for monitoring and process tracing?

This was the first time that I’ve given this presentation, but it was so easy because it came directly out of my experiences. Regardless, it’s good to have that behind me so that I can focus on the afternoon sessions.

2008 05 01 Sandy Kemsley BPM
SOA
TIBCO Comments (0)

BPEL for Java Developers Webinar

Active Endpoints is hosting a webinar this Thursday on BPEL Basics for Java Developers, featuring Ron Romano, their principal consulting architect. From their information:

A high-level overview of BPEL and its importance in a web-services environment is presented, along with a brief discussion of the basic BPEL activities and how they relate to Java concepts. The following topics will be covered:

Parsing the Language of SOA with Java as a guide

Breaking out of the VM: evolving from RPC to Web Services

BPEL Activities – Receive, Reply, Invoke

BPEL Facilities – Fault Handling and Compensation (“Undo”)

The VP of Marketing assures me that he was allowed only two slides at the end of the presentation, and that otherwise this is focused on the technical goodies.

You need to register in advance at the link above.

Tagged BPEL

2008 04 15 Sandy Kemsley BPM standards
SOA Comments (1)

BPM and Model-Driven Development, SaaS and the economy

It’s been a slow week for blogging due to a lot of billable client work, which takes precedence, and I’ve also missed several webinars that I wanted to attend. However, an article that I wrote for Intelligent Enterprise was picked up on TechWeb and published on the Yahoo! News Tech page (thanks to Bruce Williams of Software AG for tipping me off), which has resulted in quite a bit of traffic this week. I wrote the article at the end of the Gartner BPM summit last month, sifting through the wide variety of information that I saw there and distilling out some common themes: model-driven architecture/development, BPM and software-as-a-service, and the impact of the slowing economy on BPM.

The part on BPM and model-driven development was written prior to the Great BPMN Debate, but there’s an obvious tie-in, since BPMN is the modeling language that’s typically used for MDD in BPM. One of the webinars that I missed, but have just played back, is one from PegaSystems and OMG on Five Principles for Success with Model Driven Development (playback available, registration required), which touches on a number of the ideas of using (usually graphical) models to express artifacts across the entire software development lifecycle. Richard Soley of OMG and Setrag Khoshafian of Pega went through these principles in detail:

Directly capture objectives through executable models and avoid complex mappings between tools
Make a BPM suite the core layer of your MDD: model-driven development is achieved through BPM
Build and manage an enterprise repository of your modeling assets using a complete BPM suite
Leverage the platform and architecture pattern independence
Adopt a BPM suite methodology, center of excellence, best practices and continuous improvement lifecycle

The principles presented by Khoshafian were rather suspiciously aligned with Pega’s way of doing things — I have the feeling that Soley would have produced a somewhat different list of principles on his own — but the entire webinar is still worth watching, especially if you’re trying to haul your organization out of a waterfall development model or trying to understand how BPM and MDD interrelate.

To my new visitors arriving here because of the TechWeb syndication of the article: browse the archives by month or category (including the conference subcategories), or use the search feature to find topics of interest. I have several mostly-finished blog posts waiting for some final touches, so stay tuned for more content.

2008 03 27 Sandy Kemsley BPM
cloud Comments (4)

A chance encounter with jBPM

In one of those weird coincidences, JBoss World is happening simultaneously in the same conference center as ProcessWorld; Tom Baeyens noticed that I was blogging from ProcessWorld and contacted me, and we had a chance to meet up today.

We had a great discussion about model-driven applications, and finding the dividing line between what works and doesn’t work in a model-driven paradigm. Tom’s premise is that model-driven development can work in simple cases, but that it’s not possible to generalize: at some point, enough technical underpinnings need to be added to the model that it’s no longer understandable by a non-technical business analyst. Although the business analyst can still create a model and pass it for translation to a developer for augmentation, it’s going to be a one-way trip in most cases.

I definitely see some validity to this position; what typically ends up happening in today’s model-driven development in BPM is that either the developers just take over maintenance of the models at some point (even if the tool allows the models to be shared), or the business analysts learn enough of the technical side to be able to understand the fully-executable model which is, unfortunately, no longer really a business model.

jBPM is not at all about model-driven development, but provides a BPM engine that can be embedded directly in your Java code. Tom shared some of his vision of the future for the project with respect to expanding modeling capabilities and other areas; suffice it to say that total BPM world domination is on the plan.

2008 02 15 Sandy Kemsley BPM Comments (5)

Gartner BPM: The New Agile BPM Method, David Norton

This morning, I attended with David Norton’s session on integrating BPM and Agile software development methods. In BPM, we always talk about how BPM brings agility to business processes, but what facilitates that agility? Although a lot of this talk is about Agile, it’s definitely valid to look at how to apply Agile methods to BPM projects, since there’s still some amount of software development in almost every BPM project.

He started with a review of software development methods: architected model-driven (including BPM, where you draw an executable process model rather than writing code to handle work routing), architected RAD, and Agile. He then drilled in on Gartner’s 10 principles of NeoRAD (an example of Agile), such as close involvement of the customer, peer review and an iterative approach.

BPM needs to be a mix of agility and discipline, but not a waterfall methodology that we so often see used by old-style development teams when they take on a BPM project; BPM is predominantly architected model-driven because of the executable process models, but also uses some aspects of architected RAD and Agile since there are integration and UI components that require development beyond just the process modeling stage.

He described the principles of Extreme Programming as a coding methodology, and Scrum as a way to manage agile development projects; Scrum, in particular, has a lot of useful concepts that could be applied to BPM projects. He also covered Dynamic Systems Development Method, a type of RAD framework that includes the concept of turning an operational prototype into the end product to reduce waste and time in the development cycle, and Lean Software Development, focused on reducing waste and defects in the same sort of way as Lean works in the manufacturing sector.

He then looked at how BPM release cycles — continuous cycle of design and optimization, iterations of under six weeks, new policies and rules in a day — and how they are much more aligned with Agile methodologies than the traditional waterfall approach, which typically sees the first release in 4-6 months (in the best possible case). Unfortunately, most development teams inside organizations are still stuck in waterfall methodologies, and third-party professional services firms are more motivated to suggest long development cycles with large development teams. That means that even though the process model might be done as zero-code architected model-driven, the (often excessive) customization that happens in the component/service layer and the user interface drags down the project schedule.

This presentation was really much more about Agile than BPM — sometimes Gartner makes a bit of a stretch when bringing in their analysts from other areas and trying to make them BPM-ish — but if you’re not already looking at Agile development for any BPM-related project, you definitely should be.

2008 02 07 Sandy Kemsley Gartner Comments (4)

Business analyst mindmap

This is my day for having friends post amazing business-related graphics. This one (you’ll need to click through to the full-size version in order to see the detail) is by Bryce Johnson:

2008 01 29 Sandy Kemsley Comments (2)

Forrester Day 1: Packaged Applications panel

We finished up the day with a panel of Forrester analysts addressing the issue of whether packaged applications (i.e., ERP software) will ever be designed for people and built for change — that is, can these apps ever be agile. This was structured as a debate monitored by Merv Adrian, pitting Sharyn Leaver on the business side against John Rymer on the IT development side.

Adrian started by repeating the four principles of dynamic business applications that Connie Moore discussed this morning, then framed some questions for the packaged application version:

Will packaged applications contextualize work using unitary but dynamic workplaces?
Will packaged applications enable up-front customization to meet unique customer needs? (The feeling is that apps will be customizable “at the edges” only in order to allow for future upgrades/agility)
Will packaged applications enable ongoing adaptation for continuous business evolution? (Yeah, right)
Will packaged applications allow development and change by business people and IT professionals in cooperation?

Leaver and Rymer duked it out over each question, providing some really interesting viewpoints. It wasn’t exactly impromptu; each question had at least one backup presentation slide from one of them to make their point. They ended up with a slide comparing the four application biggies: Oracle, SAP, Microsoft and IBM.

From my standpoint, this was the least interesting presentation of the day, since I don’t focus on packaged applications, although it was an interesting “he said, she said” debate format. The general agreement is that the gap is narrowing between build and buy+customize, but that most customers will still require customization for competitive differentiation.

Tom Pohlmann popped back up at the end of the day with some closing remarks, then directed us off to the vendor showcase reception and the pool party. Given that it’s only 21C here in Carlsbad — compared to a balmy 28C back in Toronto (even though it’s already after dark there) — I don’t think that anyone’s going swimming tonight. Since I’m still on Eastern time, I’ll probably be asleep by 9pm again tonight.

2007 09 25 Sandy Kemsley Forrester Comments (0)

Generalists and specialists

Great graphic by Dave Gray of XPLANE, showing the difference between generalists and specialists. Click through on the image to his Flickr page to see in full resolution, or view it directly on his blog, there’s a lot of other great material there.

His key point: generalists are best at defining the problem or goal, while specialists are best at solving the problem or executing the plan.

I met Dave at VizThink3 in Toronto this summer, where he led us through some exercises in visual thinking; I’m more of a written word sort of person, so these exercises were good at stretching my brain a bit; I even drew a visual representation of crowdsourcing. I think that my drawing skills need a bit of work.

BPM is one of those areas where a structured type of visual thinking is used: most often, people draw a process flow to represent their business process, and that visual representation (now standardized in BPMN) has become the primary way for specifying processes for automation. Although Dave’s work focusses in a great part on visuals for marketing and training communications, process visualization is part of what they do; check out their website for the Standard & Poor’s example, where the process map looks like a subway route map.

2007 08 20 Sandy Kemsley BPA Comments (0)

BEAParticipate: Building your own UI

First session of the last morning, Eduardo Chiocconi of BEA and Rob Wald of JPMorgan Chase talked about the ALBPM UI: what comes out of the box, and what you can build yourself.

Out of the box, ALBPM has three user interfaces alternatives:

HiPer WorkSpace, a full user workspace with menus based on their permissions, a view of their inbox, and views of any instances that the user opens. It uses CSS if you want to change styles, and can be customized further in terms of removing buttons and other controls.
JSR-168 portlets for any standard portal environment, such as WebLogic.
WorkSpace Extensions WorkList Portlets that can be plugged into ALI and provide additional integration functionality over the standard portal interface, such as integration with the ALI Collaboration environment.

They’re working on consolidating these interfaces in order to reduce the user learning curve.

If those don’t work for you, then you can create your own user interface, either browser-based or rich client, using the available APIs. For building web clients, many of their customers have used JSP to re-code the entire user interface, then used servlets to access the engine. Alternatively, you can use Struts or JSF/AJAX. All of these can use their Java-based process API, PAPI, or the web services version, PAPI-WS, to retrieve instances from the engine, or WAPI (a servlet-based API) to execute interactive activities such as screen flows.

For rich clients, they’re seeing a lot of .Net development that uses PAPI-WS to retrieve instances, then create the UI. It’s also possible to use Eclipse or Swing to build rich user interfaces that call PAPI directly. This is more complex for interactive activities, but there are ways to work around that.

To sum up, there are three public APIs:

PAPI (process API), which is a Java implementation. If you’re working in a Java environment, this provides the tightest integration and best functionality. It manages multiple process engines transparently, and does instance caching on the client side to reduce latency in connecting to the engine — a critical performance factor.
PAPI-WS is a subset of PAPI that operates as web services (SOAP), although this is being extended in the near future to provide the full functionality of PAPI. There may be a .Net version of PAPI in the future, but for now you have to use PAPI-WS if you’re developing in .Net (e.g., ASP.Net), and can also be used from any web service client. Right now, PAPI-WS is part of the HiPer WorkSpace, but will be decoupled as a self-contained web application in the future. It’s also possible to expose processes directly as web services, as we heard in an earlier session, which provides another point of integration from a web service or .Net development environment.
WAPI, which is a servlet API that can be used to launch the UI of an interactive activity at a point in the process, which can’t be done in PAPI or PAPI-WS.

With any custom UI, there’s always the question of single sign-on. With the WorkSpace Extensions WorkList Portlets in ALI, that’s handled natively; in the HiPer WorkSpace and JSR-168 portlet implementations it requires some customization, although there is a single sign-on login servlet provided with the JSR-168 portlets to make this easier.

Getting to the specific JPMorgan case study, they created a custom user interface since, like many large companies, they want integration with other applications in their environment and want more control over the look and feel of the interface. It’s possible to just create custom JSPs and use them in the standard work portal framework, which provides a great deal of control over the UI without completely rewriting it, but this wasn’t sufficient for many of their applications. What they ended up doing was creating a completely custom inbox using Struts/JSP/GWT with PAPI: one example that he showed was using Struts and AJAX via the Google Web Toolkit to manage financial reconciliation processes. They’re also using IceFaces, an open source RenderKit implementation of JSF (as a replacement for Struts) that supports AJAX to create a visual drag-and-drop components for use in an IDE such as Eclipse. Since JPMorgan is dedicated to the use of open source, they’re doing some innovative development that’s not seen in most corporate environments, but maybe should be. They’re also using the JSR-168 portlets in a more standard portal implementation, and building rich clients with Eclipse.

On the back end of their implementation, they’ve found that some of the PAPI protocols don’t work well over wide-area networks, such as between their US and Japan operations, so they do quite a bit of preloading of the PAPI cache.

JPMorgan has implemented ALBPM as a centralized shared service in order to provide efficient use of both human and server resources: centralized code and best practices on the human side, and a single ALBPM server handling 10 applications without difficulty.

2007 05 09 Sandy Kemsley BEAparticipate
BPM Comments (0)

BEAParticipate: Best Practices for Succeeding with BPM

I’m jumping around between tracks (and hence rooms): I started the afternoon in the ALUI Experience track, then on to the ALBPM Technical/Developer track, and now I’m in the ALBPM Experience track for a discussion of best practices for managing BPM projects with Dan Atwood of BEA (another former Fuego employee) and Karl Djernal of Citigroup. It’s a bit difficult to pick and choose sessions when you’re interested in multiple tracks: this session and the one after it in the same track are both around best practices, although appear to cover different aspects of BPM implementations and I’d like to sit through both. This one is categorized as “beginner” and the next as “intermediate”, so I’m hoping that someone’s actually checked to ensure that there’s not too much overlap between them. I’d also like to see the next technical track on how BPM and ESB work together, but think that I can probably get a briefing on that directly from BEA as required.

Atwood started the session with seven key practices for BPM success:

Fundamentals of process-based competition: understanding the competitive advantage of being a process-oriented company, and the business case for BPM.
BPM and its value to the corporation: understanding what BPM is and how it differs from other management and technology approaches.
From functional management to process-oriented thinking: how the shift from functional management must permeate through the ranks of middle management in order to disperse the fiefdoms within an organization.
Getting hands-on BPM experience, with the help of mentors.
Foundations for process practitioners: BPM as the capability for implementing and extending other management practices such as Six Sigma.
Business process modelling and methods: learn about process-oriented architectures and development methods, and how they differ from traditional approaches.
Human interactions and their roles within BPM: while system-to-system automation is often a BPM focus, the human-facing parts of the process are critical. In other words, you can’t think of these as being “human-interrupted” processes, as a customer of mine did long ago.

Obviously a big fan of BPM books, Atwood references Peter Fingar, Howard Smith, Andrew Spanyi, John Jeston, Mike Jacka, Paulette Kellerin and Keith Harrison-Broninski, as well as a raft of BPM-related sites (although not, unfortunately, www.column2.com). Also a fan of lists, he finishes up with his top five success factors:

Executive sponsorship
Correct scoping
Start with the end in mind
Framework
Engage stakeholders

Hmmm, that seems to make 12 best practices in total…

Djernal then discussed the Agile methodology that they used for BPM implementation at Citigroup, starting with a description of Agile and Scrum as the anti-waterfall approach: providing incremental deliveries based on changing, just-in-time requirements, and involving the end users closely during the development cycle to provide feedback on each iteration. Just as important as delivery mechanisms is the Agile team structure: the team’s not managed in the traditional sense, but works closely with the customer/end-user to create what they want. There’s a 15-minute team meeting every day, and a delivery (sprint) every 30 days. Many teams vary the sprint length slightly while sticking to the Agile methodology, although there’s danger in increasing it too much or you slip back to months-long delivery cycles. Initiated by the original prioritized set of product features, the user feedback on each iteration can impact both the features and the priorities. There’s basically three roles in Agile: a product owner who represents the stakeholders, the team that implement everything, and the ScrumMaster who provides mentoring on the Agile process and helps to sort out external roadblocks.

The interesting thing is how they brought together BPM and Agile, since I’m convinced that these are two things that belong together. Process diagrams fill in a lot of the documentation gap and are a naturally agile form of creating a functional specification; they form a good basis for communication between the business and IT. Changes in requirements that cause changes to the business process can be done easily in a graphical process modelling environment. In fact, in many BPM environments, the processes can be prototyped and an initial executable version developed in a matter of days without writing any code, which in turn helps to set priorities on the functions that do require coding, such as developing web services wrappers around legacy systems.

They’ve learned some things from their experiences so far:

Get training on using the BPM products, and on BPM in general.
Use some external resources (like me) to help you get started.
Since BPM involves integration, setting up the development, testing and production environments can be time-consuming and require specialized resources.
Spend some time up front putting together a good test environment, including automated testing tools.
Create a centre of excellence for BPM.
Start something small for your first BPM project.

There’s a lot of arguments about how Agile can’t really handle large-scale development projects, but it’s my belief that most BPM projects lend themselves well to Agile. The worst disasters that I’ve seen in BPM implementation have been the product of the most non-Agile development processes imaginable, with months of requirements writing followed by many more months of development, all of which resulted in something that didn’t match the users’ requirements and was much too costly to change. As I’ve said many times before, if you can’t get something up and running in BPM in a matter of a couple of months, then you’re doing something really wrong.

2007 05 07 Sandy Kemsley BEAparticipate
BPM Comments (0)

The New Software Industry: David Messerschmitt

David Messerschmitt, a prof at UC Berkeley and the Helsinki University of Technology, finished the formal presentations for the day with a talk on how inter-firm cooperation can be improved in the software industry. This is an interesting wrap-up, since we’ve been hearing about technology, applications and business opportunities all day, and this takes a look at how all these new software industry companies can cooperate to the benefit of all parties.

He started out by proposing a mission statement: focus the software industry’s attention and resources on providing greater value to the user and consumer. This has two aspects: do less harm, and do more direct provision of value to the customer rather than the computational equivalent of administrivia.

In general the software industry has a fairly low customer satisfaction rate of around 75%, whereas specific software sectors such as internet travel and brokerage rank significantly higher. In general, services provided by people have a lower satisfaction rate (likely due to the variability of service levels), and the satisfaction rates are decreasing each year. Complaints are focussed on gratuitous change (change due to platform changes rather than anything that enhances user value) and security, and to some extent on having to change business processes to match an application’s process rather than having the system adapt to their business process. Certainly, there are lessons here for BPM implementations.

Messerschmitt raised the issue of declining enrolment of women in computer science, which he thinks is in part due to the perception that computer science is more about heads-down programming rather than about dealing with users’ requirements. He sees this as a bit of a canary in a coal mine, indicating some sort of upcoming problem for the computing industry in general if it is driving away those who want to deal with the user-facing side of software development. Related to that, he recommends the book Democratizing Innovation by Eric von Hippel, for its study of how customers are providing innovation that feeds back into product design and development, not just in software but in many areas of products.

He ended up by discussing various ways to improve inter-firm cooperation, such as the Global Environment for Networking Innovations (GENI) initiative, ways to accomplish seamless operation of enterprise systems, and referring to a paper that he recently wrote and will be published in July’s IEEE Proceedings, Rethinking Components: From Hardware and Software to Systems. He then listed elements of collective action that can be pursued by industry players, academia and professional organizations to help achieve this end:

Systematically look at knowledge gaps and ensure that the research is addressing those gaps
Create/educate the human resources that are needed by the industry
Understand and encourage complementarities, like broadband and certain types of software
Structures and processes: capture end-user innovations for incorporation into a product, and achieve a more orderly evolution of technology with the goal of leaving behind many fewer legacies in the future

He’s definitely of the “a rising tide lifts all boats” mindset.

2007 04 30 Sandy Kemsley NewSoftwareIndustry Comments (0)

The New Software Industry: Bob Glushko and Shelley Evenson

Bob Glushko, a prof at UC Berkeley, and Shelley Evenson, a prof at CMU, discussed different views on bridging the front stage and back stage in service system design. As a side note, I have to say that it’s fun to be back (temporarily) in an academic environment: many of these presentations are much more like grad school lectures than standard conference presentations. And like university lectures, they cover way too much material in a very short time by speaking at light speed and flipping slides so fast that there’s no time to even read what’s on the slide, much less absorb or document it. If I had a nickel for every time that a presenter today said “I don’t have time to go into this but it’s an important concept” while flipping past an interesting-looking slide, I could probably buy myself the drink that I need to calm myself after the information overload.

Glushko posits that greater predictability produces a better experience, even if the average level of service is lower, using the example of a self-service hotel check-in versus the variability of dealing with a reception clerk. Although he doesn’t mention it, this is exactly the point of Six Sigma: reducing variability, not necessarily improving service quality.

He goes on to discuss the front stage of services, which is the interaction of the customer or other services with the services, and the back stage, which is the execution of the underlying services themselves. I love his examples: he uses an analogy of a restaurant, with the front stage being the dining room, and the back stage being the kitchen. Front stage designers focus on usability and other user interface factors, whereas the back stage designers focus on efficiency, standardization, data models and the like. This tends to create a tension between the two design perspectives, and begs the question if these are intrinsic or avoidable.

From a design standpoint, he feels that it’s essential to create information flow and process models that span both the back and front stages. The focus of back stage design is to design modular and configurable services that enable flexibility and customization in the front stage, and to determine which back stage services you will perform and which you will outsource/reuse from other service providers. Front stage design, on the other hand, is focussed on designing the level of service intensity (the intensity of information exchange between the customer and the service, whether the service is human or automated), and to implement model-based user interfaces and use these models to generate/configure/specify the APIs of user interfaces for the services. By exposing back stage information in front stage design, more back stage information can improve the immediate experience for a specific customer, and can improve subsequent experiences. Data mining and business intelligence can also improve service for future customers.

Evenson, who specializes in interaction design, has a very different perspective than Glushko, who focusses on the back stage design, but rather than being opposing views, they’re just different perspectives on the same issues of designing service systems.

She started out with a hilarious re-rendering of Glushko’s restaurant example by making the point that she applied colour to make the division of the co-production between front and back stage more visible.

Her slides really went by so fast that I was only able to capture a few snippets: sensors will improve the degree of interaction and usefulness of web-based services; technology influences our sense of self; services are activities or events that form a product through interaction with a customer; services are performances: choreographed interactions manufactured at the point of delivery; services are the visible front end of a process that co-produces value. A service system is a framework that connects service touchpoints so that they can sense, respond and reinforce one another. The system must be dynamic enough to be able to efficiently reflect the expectations people bring to the experience at any given moment. Service systems enable people to have experiences and achieve goals.

She discussed the difficulties of designing a service system, such as the difficulty of prototyping and the difficulty of representing the experience, and pointed out that it requires combining aspects of business, technology and experience. She feels that it’s helpful to create an integrated service design language: systems of elements with meanings (that designers use to communicate and users “read”) plus sets of organizing principles.

2007 04 30 Sandy Kemsley NewSoftwareIndustry
SOA Comments (0)

Column 2