Skip to content

Upcoming Webinars with Progress Software

Blogging around here has been sporadic, to say the least. I have several half-finished posts about product reviews and some good BPM books that I’ve been reading, but I have that “problem” that independent consultants sometimes have: I’m too busy doing billable work to put up much of a public face, both with work with vendors and some interesting end-customer projects.

Today, I’ll be presenting the second in a series of three webinars for Progress Software, focused on how BPM fits with more traditional application development environments and existing custom applications. Progress continues to integrate the Savvion and Corticon acquisitions into their product set, and wanted to put forward a webinar series that would speak to their existing OpenEdge customers about how BPM can accelerate their application development without having to abandon their existing custom applications. I really enjoyed the first of the series, because Matt Cicciari (Progress product marketing manager) and I had a very conversational hour – except for the part where he lost his voice – and this time we’ll be joined by Ken Wilmer, their VP of technology, to dig into some of their technology a bit more. My portion will focus on generic aspects of combining BPM and traditional application development, not specific to the Progress product suite, so this may be of use even if you’re not using Progress products but want to understand how these seemingly disparate methodologies and technologies come together.

We’re doing today’s webinar twice: once at 11am Eastern to cover Europe and North America, then a repeat at 7pm ET (that’s 11AM tomorrow in Sydney) for the Asia Pacific region or those of you who just didn’t get enough in the first session. It will be live both times, so I will have the chance to think about what I said the first time around, and completely change it. ;-)

You can sign up for today’s session here, plus the next session on February 29th that will include more about business rules in this hybrid environment.

2012 01 25 Sandy Kemsley BPM
Software design Comments (0) Permalink

Q&A From Making Social Mean Business

We had a few unanswered questions left from our webinar on Tuesday, so I’ve included the ones that were not related to Pega’s products below, with answers from both Emily Burns and myself:

There’s a lot of discussion about the readiness of an org before social features are introduced to its employees. What would be a way to assess maturity/readiness of an org for such features with regards to BPM?

Emily: Boy, I guess I am on the more liberal side of that discussion and would err on the side of providing access to these features and seeing how they evolve—collective intelligence is pretty impressive, and can take things in many positive directions that a designer just wouldn’t think of. It’s hard for me to see the downside to fostering better communication and collaboration between people who are already working on the same cases, but may not currently be aware of who the other people are.

Sandy: There is a lot of work being done on social business readiness by organizations such as the Social Business Council (http://council.dachisgroup.com/) that can serve as a reference for how that will work with social features in BPM. In assessing readiness, you can look at the use of other social tools within the organization, such as wikis for documentation, or instant messaging for chat, to get an idea of whether the users have been provided with tools such as this in the past. However, just because they haven’t used these in the workplace before is no reason to avoid social BPM functions since users may be using similar capabilities in consumer applications, and as Emily points out, the best thing is to provide them with the tools and see what emerges.

Emily: For features that impact the application more, such as design-by-doing, that I think is an area that does need careful consideration. In the case of design-by-doing, more often than not, that is something that is limited only to certain roles, and even then, while the default is to allow the new type of case to be instantly in production, in reality, most of our clients use it more as a way of gathering suggestions for application improvements. As it becomes more widely used, and best practices developed around governance, I expect this type of thing to be used more aggressively to foster the kind of real-time adaptation for which it was conceived.

Sandy: Although many organizations are worried about users “going wild” with collaborative and social tools, the opposite is often true: it is more difficult to get users to participate unless they can see a clear personal benefit, such as being able to get their job done better or more efficiently. This may require creating some rewards specifically geared at users who are taking advantage of the social tools, in order to help motivate the process.

While the knowledge that we can glean from social networking sites is indeed powerful, and allows us to serve up tailored offers, it can also irritate some customers, or seem “creepy” like it’s a bit of an invasion of privacy.

Emily: I totally agree, and am just such a customer. In fact, I won’t go to a company’s Facebook page unless I am logged out of Facebook, because I don’t want them to know anything about me, nor do I want my friends to know about my interactions with different companies. In order to get around this sort of stone-walling, there are a few things that organizations can do.

  1. Make the content and actions that can be performed from your Facebook page sufficiently compelling that you overcome this resistance.
  2. DON’T BE SNEAKY! Do not default settings to “post to my wall” so that all of a client’s friends see that she just applied for a new credit card. Be frank and up front about any information that might be broadcast, and about how you are using the information that they have so graciously allowed you to access by virtue of logging in via Facebook. If you want to give people the option of posting something, make sure they are forced to make the choice. And make it transparent and easy to change settings in the future. This will help you gain trust and increase the uptake of these low-cost, highly viral channels.

Sandy: I completely agree – transparency is the key here for organizations starting on a social media path. Anything less than complete transparency about what you’re doing with the consumer’s information – including their actions on your site – will be exposed in the full glare of public scrutiny on the web when people discover it. Accept, however, that there is a wide range of social behavior for customers: some want to be seen to be associated with your product or brand, and will “Like” your Facebook page or check-in on Foursquare at your location, whereas others will not want that information to be publicized in any way.

Do you think there is a trust built up yet for customers to interact with companies via social as yet?

Emily: See my response above. I think that in many cases, organizations have started out on the wrong foot, taking advantage of how easily available the information is to really milk it for all its worth. The fact that many of the social networking sites had low-granularity privacy settings initially made it so that this wasn’t entirely the fault of the different organizations, either. Because of this, and in light of continually improving granularity and control over privacy settings, I think now is a time to try to re-establish trust, and establish what it means to be a good “social” corporate citizen.

Sandy: Social media is becoming a powerful channel for customer interaction, particularly in situations where the company is monitoring Twitter and Facebook updates to track any problems that customers are experiencing. From my own personal experience (and in part because I have a large Twitter following and use my real name on Twitter), I have had near-immediate responses to problems that I Tweeted about hotels, car rentals and train travel. In some cases, the social media wasn’t necessarily well-integrated with the rest of their customer service channel, but when it is well-integrated, it’s a very satisfying customer experience for someone like me with a strong social media focus. There are initiatives to create the type of trusted online behavior that we would all like to see, such as the Respect Trust Framework; early days for these, but we’ll see more organizations adopt this as customers insist on their online rights.

I’ve also included my slides below, although not Emily’s deck. I’ll update this post with the link to the webinar replay when it is available.

Making Social BPM Mean Business
2011 12 08 Sandy Kemsley BPM
social Comments (0) Permalink

Making Social BPM Mean Business

When I owned a boutique consulting firm in the 1990’s, our catchphrase was “Making Technology Mean Business”, and when we were coming up with a title for the webinar that I’m doing with Pegasystems next week, an updated version of that phrase just seemed to fit. We’ll be discussing the social aspects of business processes, particularly in the context of case management. I’ll be expanding on a discussion point from my Changing Nature of Work keynote at BPM 2011 to discuss the social dimension and how that correlates with structure (i.e., a priori modeling), triggered in part by some of the discussion that arose from that presentation. As with the spectrum of structure, I believe that there’s a spectrum of socialness in business processes: some processes are just inherently more social than others (or can benefit from social features).

Interested? The webinar is on Tuesday at 11am Eastern, and you can register here.

2011 12 02 Sandy Kemsley BPM
social Comments (0) Permalink

Emerging Trends in BPM – Five Years Later

I just found a short article that I wrote for Savvion (now part of Progress Software) dated November 21, 2006, and decided to post it with some updated commentary on the 5th anniversary of the original paper. Enjoy!

Emerging trends in BPM
What happened in 2006, and what’s ahead in 2007

The BPM market continues to evolve, and although 2006 has seen some major events, there will be even more in 2007. This column takes a high-level view of four areas of ongoing significant change in BPM: the interrelationship between SOA and BPM; BPM standards; the spread of process modeling tools; and the impact of Web 2.0 on BPM.

SOA and BPM, together at last. A year ago, many CIOs couldn’t even spell SOA, much less understand what it could do for them. Now, Service-Oriented Architecture and BPM are seen as two ends of the spectrum of integration technologies that many organizations are using as an essential backbone for business agility.

SOA is the architectural philosophy of exposing functionality from a variety of systems as reusable services with standardized interfaces; these, in turn, can be orchestrated into higher-level services, or consumed by other services and applications. BPM systems consume the services from the SOA environment and add in any required human interaction to create a complete business process.

As with every year for the last several years, 2006 has seen ongoing industry consolidation, particularly with vendors seeking to bring SOA and BPM together in their product portfolios. This trend will continue as SOA and BPM become fully recognized as being two essential parts of any organization’s process improvement strategy.

There has certainly been consolidation in the BPM vendor portfolios, especially the integration vendors adding better human-centric capabilities through acquisitions: Oracle acquired BEA in 2008, IBM acquired Lombardi in 2009, Progress acquired Savvion in 2010, and TIBCO acquired Nimbus in 2011. Although BPM is being used in some cases to orchestrate and integrate systems using services, this is still quite a green field for many organizations who have implemented BPM but are still catching up on exposing services from their legacy applications, and orchestrating those with BPM.

BPM standards. 2006 was the year that the Business Process Modeling Notation (BPMN), a notational standard for the graphical representation of process models, went mainstream. Version 2 of the standard was released, and every major BPM vendor is providing some way for their users to make use of the BPMN standard, whether it’s through a third-party modeling tool or directly in their own process modelers.

But BPMN isn’t the only standard that gained importance this year. 2006 also saw the widespread adoption of XPDL (XML Process Definition Language) by BPM vendors as an interchange format: once a process is modeled in BPMN, it’s saved in the XPDL file format to move from one system to another. A possible competitor to XPDL, the Business Process Definition Metamodel (BPDM) had its first draft release this year, but we won’t know the impact of this until later in 2007. On the SOA side, the Business Process Execution Language (BPEL), a service orchestration language, is now widely accepted as an interchange format, if not a full execution standard.

The adoption of BPM standards is critical as we consider how to integrate multiple tools and multiple processes to run our businesses. There’s no doubt that BPMN will remain the predominant standard for the graphical representation of process models, but 2007 could hold an interesting battle between XPDL, BPDM and BPEL as serialization formats.

The “Version 2” that I referred to was actually the second released version of the BPMN standard, but the actual version number was 1.1. That battle for serialization formats still goes on: most vendors support XPDL (and will continue to do so) but are also starting to support the (finally released) BPMN file format as well. BPDM disappeared somewhere in the early days of BPMN 2.0. BPEL is used as a serialization and interchange format primarily between systems that use BPEL as their core execution language, which are a minority in the broader BPMS space.

Modeling for the masses. In March of 2006, Savvion released the latest version of their free, downloadable process modeler: an application that anyone, not just Savvion customers, could download, install and run on their desktop without requiring access to a server. This concept, pioneered by Savvion in 2004, lowers the barrier significantly for process modeling and allows anyone to get started creating process models and finding improvements to their processes.

Unlike generic modeling tools like Microsoft Visio, a purpose-built process modeler can enforce process standards, such as BPMN, and can partially validate the process models before they are even imported into a process server for implementation. It can also provide functionality such as process simulation, which is essential to determining improvements to the process.

2006 saw other BPM vendors start to copy this initiative, and we can expect more in the months to come.

Free or low-cost process modelers have proliferated: there are web-based tools, downloadable applications and Visio BPMN add-ons that have made process modeling accessible – at least financially – to the masses. The problem continues to be that many people using the process modeling tools lack the analysis skills to do significant process optimization (or even, in some cases, representation of an event-driven process): the hype about having all of your business users modeling your business processes has certainly exceeded the reality.

Web 2.0 hits BPM. Web 2.0, a set of technologies and concepts embodied within the next generation of internet software, is beginning to impact enterprise software, too.

Web 2.0 is causing changes in BPM by pushing the requirement for zero-footprint, platform-independent, rich user interfaces, typically built using AJAX (Asynchronous Java and XML). Although browser-based interfaces for executing processes have been around for many years in BPM, the past year has seen many of these converted to AJAX for a lightweight interface with both functionality and speed.

There are two more Web 2.0 characteristics that I think we’re going to start seeing in BPM in 2007: tagging and process syndication. Tagging would allow anyone to add freeform keywords to a process instance (for example, one that required special handling) to make it easier to find that instance in the future by searching on the keywords. Process event syndication would allow internal and external process participants to “subscribe” to a process, and feed that process’ events into a standard feed reader in order to monitor the process, thereby improving visibility into the process through the use of existing feed technologies such as RSS (Really Simple Syndication).

Bringing Web 2.0 to BPM will require a few changes to corporate culture, especially those parts that require different – and more creative – types of end-user participation. As more people at all levels in the organization participate in all facets of process improvement, however, the value of this democratization of business processes will become clear.

I’ve been writing and presenting about the impact of social software on BPM for over five years now; adoption has been slower than I predicted, although process syndication (subscribing to a process’ events) has finally become mainstream. Tagging of processes is just starting to emerge; I’ve seen it in BonitaSoft but few other places.

I rarely do year-end prediction posts, but it was fun to look back at one that I did five years ago to see how well I did.

2011 11 21 Sandy Kemsley BPM
SOA
social Comments (2) Permalink

BPM Glossary

Business Process Incubator has a new toy to play with: a BPM Glossary widget. I’ve embedded it below, and I believe that its available for any site to use.

So far, I can only figure out how to search for a term (and submit a new term if it’s not found in the glossary), but can’t work out how to browse the glossary, if there is such a function.

2011 11 16 Sandy Kemsley BPM Comments (6) Permalink

Enterprise BPM Webinar Q&A Followup

I know, two TIBCO-related posts in one day, but I just received the link to the replay of the Enterprise BPM webinar that I did for TIBCO last week, along with the questions that we didn’t have time to answer during the webinar, and wanted to summarize here. First of all, my slides:

Enterprise BPM

These were the questions that came in during the webinar via typed chat that are not related to TIBCO or its products; I think that we covered some of these during the session but will respond to all of them here.

Is it possible to implement BPM (business process management) without a BPMS?

How to capture process before/without technology?

These are both about doing BPM without a BPMS. I wrote recently about Elevations Credit Union (the fact that they are an IBM customer is completely immaterial in this context) that gained a huge part of their BPM success long before they touched any technology, Basically, they carved out some high-level corporate goals related to quality, modeled their value streams, then documented their existing business processes relative to those value streams. Every business process had to fit into a value stream (which was in turn related to a corporate goal), or else it didn’t survive. They saw how processes touched various different groups, and where the inefficiencies lay, and they did all of this using manual mapping on white boards, paper and sticky notes. In other words, they used the management discipline and methodology side of BPM before they (eventually) selected a tool for collaborative process modeling, which then helped them to spread the word further in their organization. There is a misperception in some companies that if you a buy a BPMS, your processes will improve, but you really need to reorient your thinking, management and strategic goals around your business processes before you start with any technology, or you won’t get the benefits that you are expecting.

In enterprises that do not have SOA implemented horizontally across the organization, how can BPM be leveraged to implement process governance in the LOB silos, yet have enterprise control?

A BPM center of excellence (CoE) would be the best way to ensure process governance across siloed implementations. I wrote recently about a presentation that I was at where Roger Burlton spoke about BPM maturity; there was some advice that he had at the end of that about organizations that had only a level 1 or 2 in process maturity (which, if you’re still very siloed, you’re probably at): get a CoE in place and target it more at change initiatives than governance. However, you will be able to leverage the CoE to put standards in place, provide mentoring and training, and eventually build a repository of reusable process artifacts.

I work in the equipment finance industry. Companies in this space are typically classified as banks/bank-affiliates, captives and independents. With a few exceptions it’s my understanding that this particular industry has been rather slow at adopting BPMS. Have you noticed this in other industries and, if so, what do you see as being the “tipping point” for greater BPMS adoption rates? Does it ultimately come down to a solid ROI, or perhaps a few peer success stories?

My biggest customers are in financial services and insurance, so are also fairly conservative. Insurance, in particular, tends to adopt technology at the very end of adoption tail. I have seen a couple of factors that can impact a slower-moving adoption of any sort of technology, not just BPMS: first, if they just can’t do business the old way any more, and have to adopt the new technology. An example of this was a business process outsourcer for back-office mutual fund transactions that started losing bids for new work because it was actually written into the RFP that they had to have “imaging and workflow” technology rather than paper-based processes. Secondly, if they can’t change quickly enough to be competitive in the market, which is usually the case when many other of their competitors have already started using the technology. So, yes, it does come down to a solid ROI and some peer success stories, but in many cases, the ROI is one of survival rather than just incremental efficiency improvements.

Large scale organizations tend to have multiple BPM / workflow engines. What insights can you share to make these different engines in different organizational BUs into an enterprise BPM capability?

Every large organization that I work with has multiple BPMS, and this is a problem that they struggle with constantly. Going back to the first question, you need to think about both sides of BPM: it’s the management discipline and methodology, then it’s the technology.  The first of these, which is arguably the one with the biggest impact, is completely independent of the specific BPMS that you’re using: it’s about getting the organization oriented around processes, and understanding how the end-to-end business processes relate to the strategic goals. Building a common BPM CoE for the enterprise can help to bring all of these things together, including the expertise related to the multiple BPM products. By bringing them together, it’s possible to start looking at the target use cases for each of the systems currently in use, and selecting the appropriate system for each new implementation. Eventually, this may lead to some systems being replaced to reduce the number of BPMS used in the organization overall, but I rarely see large enterprises without at least two different BPMS in use, so don’t be fanatical about getting it down to a single system.

Typically what is the best order to implement ; first BPM and last SOA or vice versa.

I recommend a hybrid approach rather than purely top-down (BPM first) or bottom-up (SOA first). First, do an inventory in your environment for existing services, since there will almost always be some out there, even if just in your packaged applications such as ERP. While is this happening, start your BPM initiative by setting the goals and doing some top-down process modeling. Assuming that you have a particular process in mind for implementation, do the more detailed process design for that, taking advantage of any services that you have discovered, and identifying what other services need to be created. If possible, implement the process even without the services: it will be no worse from an efficiency standpoint than your current manual process, and will provide a framework both for adding services later and for process monitoring. As you develop the services for integration and automation, replace the manual steps in the process with the services.

Re: Enterprise BPM Goals – Develop, Execute, but what about Governance?

This was in response to the material on my agenda for the webinar. Yes, governance is important, but I only had 40 minutes and could barely cover the design/develop/execute parts of what we wanted to cover. Maybe TIBCO will have me back for another webinar on governance. ;-)

Data/content centric processes vs. people-centric vs. EAI/integration centric re: multiple BPMS platforms. Any guidelines for when and where to demarcate?

These divisions are very similar to the Forrester divisions of the BPMS landscape from a few years ago, and grew mostly out of the different types of systems that were all lumped together as “BPMS” by the analysts in the early 2000’s. Many of today’s products offer strength in more than one area, but you need to have a good understanding of your primary use cases when selecting a product. Personally, I think that content-centric and human-centric isn’t the right way to split it: more like unstructured (case management) versus structured; even then, there is more of a spectrum of functionality in most cases than purely unstructured or purely structured. So really, the division is between processes that have people involved (human-centric) or those that are more for automated integration (system-centric), with the latter having to accommodate this wider spectrum of process types. If you have mostly automated integration processes, then certainly an integration-centric BPMS makes sense; if you have human-facing processes, then the question is a bit more complex, since you’re dealing with content/documents, process types, social/collaborative capabilities and a host of other requirements that you need to look at relative to your own use cases. In general, the market is moving towards the full range of human-facing processes being handled by a single product, although specialist product companies would differ.

Thoughts on the role of the application/solution architect within an LOB or COE vs. that of the enterprise architect assigned to the BPM domain?

An enterprise architect assigned to the BPM CoE/domain is still (typically) part of the EA team, therefore involved with the broader scope of enterprise architecture issues. An application/solution architect tends to be more product and technology focused, and in many some that is just a fancy term used for a developer. In other words, the EA should be concerned with overall strategy and goals, whereas the solution architect is focused on implementation.

Role of the COE in governance? How far does/should it extend?

The CoE is core to governance: that’s what it’s there for. At the very least, the CoE will set the standards and procedures for governance, and may rely on the individual projects to enforce that governance.

Is it really IT giving up control? In many cases, the business does whatever they do — and IT has little (or aged) information about the actual processes.

This was in reference to slide #11 in my deck about cultural issues. Certainly business can (and often do) go off and implement their own processes, but that is outside the context of enterprise-wide systems. In order to have the business be doing that within the enterprise BPMS, IT has to ensure that business can access the process discovery and modeling tools that become the front-end of process design. That way, business and IT share models of the business processes, which means that what gets implemented in the BPMS might actually resemble what is required by the business. In some cases, I see a company buy a BPMS but not allow the business users to use the business-level tools to participate in process modeling: this is usually the result of someone in IT thinking that this is beyond the capability of the business people.

Is following of any BPM notation standards part of BPM development? I saw that there was no mention of it.

There was so much that I did not have time to address with only 40 minutes or so to speak, and standards didn’t make the cut. In longer presentations, I always address the issue of standards, since a common process modeling notation is essential to communication between various stakeholders. BPMN is the obvious front-runner there, and if used properly, can be understood by both business and IT. It’s not just about process models, however: a BPMS implementation has to also consider data models, organizational models and more, around which there is less standardization.

Regarding Common UI: shouldn’t it be Common Architecture, accessed by different UIs that fit the user’s roles, knowledge, etc?

In the context of slide #6, I did mean a common UI, literally. In other words, using the BPMS’ composite application development and forms environment to create a user interface that hides multiple legacy applications behind a single user interface, so that the user deals with this new integrated UI instead of multiple legacy UIs. Your point seems to be more about persona-based (or role-based) interfaces into the BPMS, which is a valid, but different, point. That “single UI” that I mention would, in fact, be configurable for the different personas who need to access it.

How does a fully fledged BPM tool stack up against workflow tools part of other COTS application, e.g. workflow in a document management tool or in a trouble ticketing tool?

A full BPMS tends to be much more flexible than what you will find in the embedded workflow within another platform, and is more of an application development platform than just a way to control processes within that application. On the other side, the workflow within those applications are typically already fully integrated with the other business objects within them (e.g., documents, trouble tickets), so the implementation may be faster for that particular type of process. If the only type of process management that you need to do is document approvals within your document management system, it may make sense to use that rather than purchase a full BPMS; if you have broader process management needs, start looking at a more general BPMS platform that can handle more of your use cases.

How do u see BPM tools surviving when CRM tools with more or less same capability is getting widely accepted by enterprises with out-of-box processes defined?

Similar to my response to the previous question, if the processes are related only to the business objects within the CRM, then you may be better off using the workflow tools within it. However, as soon as you want to integrate in other data sources, systems or users, you’ll start to get beyond the functional capabilities of the simpler workflow tools within the CRM. There’s room in the market for both; the trick is, for customers, to understand when to use one versus the other.

What are the reasons you see for BPM tools not getting quickly and widely accepted and what are the solutions to overcome that?

There are both cost and complexity components with BPMS adoption, but a big reason before you even start looking at tools is moving your organization to a process-driven orientation, as I discussed above. Once people start to look at the business as end-to-end processes, and those processes as assets and capabilities that the business offers to its customers, there will be a great pull for BPMS technologies to help that along. Once that motivation is in place, the cost and complexity barriers are still there, but are becoming less significant: first of all, more vendors are offering cloud-based versions of their software that allow you to try it out – and even do your full development and testing – without capital expenditures. If they offer the option, you can move your production processes on-premise, or leave them in the cloud to keep the total cost down. As for complexity, the products are getting easier to use, but are also offering a lot more functionality. This shifts the complexity from one of depth (learning how to do a particular function) to breadth (learning what all the functions are and when to use which), which is still complex but less of a technological complexity.

Is it possible to start introducing and implementing BPM in one department or module only and then extending the BPM to other departments or modules? Or this should be the enterprise wide decisions since it involves heavy cost to bring BPM technologies.

Almost every organization that I work with does their BPM implementation in one department first, or for one process first (which may span departments): it’s just not possible to implement everything that you will ever implement in BPM at the same time, first time. There needs to be ROI within that first implementation, but you also have to look at enterprise cost justification as with any horizontal technology: plan for the other projects that will use this, and allocate the costs accordingly. That might mean that some of the initial costs come from a shared services or infrastructure budget rather than the project budget, because they will eventually be allocated to future projects and processes.

How difficult would it be to replace legacy workflow system with BPM?

It depends (that’s always the consultant’s answer). Seriously, though, it depends on the level of integration between the existing workflow system and other systems, and how much of the user interface that it provides. I have seen situations where a legacy workflow system is deeply embedded in a custom application platform, with fairly well-defined integration points to other systems, and the user interface hiding the workflow system from the end user. In this case, although it’s not trivial, it is a straightforward exercise to rip out the workflow system since it is being used purely as a process engine, replace it with a new one, refactor the integration points so that the new system calls the other systems in the environment (usually easier since modern BPMS’ have better integration capabilities) and refactor the custom UI so that it calls the new BPMS (also usually easier because of updated functionality). That’s the best case, and as I said, it’s still not trivial. If the legacy workflow system also provides the user interface, then you’re looking at redeveloping your entire UI either in the new BPMS or in some other UI development tool, plus the back-end systems integration work. A major consideration in either case is that you don’t just want to replace the same functionality of the old workflow system, since the new BPMS will have far greater functionality: you need to think about how you are going to leverage capabilities such as runtime collaboration that never existed in the old system, in order to see the greatest benefit from the upgrade.

Is it possible to switch between BPM vendors without having pain?

No. Similar to the previous answer, this is a non-trivial exercise, and depending on how much of the functionality of the BPMS that you were using, could be pretty much a complete redevelopment. If the BPMS was used primarily for orchestration of automated processes, it will be much easier, but as soon as you get into custom integration/orchestration and user interfaces, it gets a lot more complicated (and painful).

Do we really need to go for BPM in a situation where we need only integration orchestration only?

One end of the BPMS market is integration-centric systems, which primarily do just integration orchestration. The advantage of using a BPMS for this instead of orchestrating directly in application code is that you get all of the other stuff that comes with the BPMS “for free”: graphical process modeling, execution monitoring, process governance and whatever other goodies are in the BPMS. It’s not really free, of course, but it’s valid to consider a comparison of all of that functionality against what parts of it you would have to custom-build if you were to do the orchestration in code.

That’s it for the Q&A. If you listen to the replay, or were on the live broadcast, my apologies for the rushed beginning: I got off on the wrong foot out of the gate, but settled down after the first few minutes.

Tagged
2011 11 14 Sandy Kemsley BPM
SOA Comments (1) Permalink

TIBCO Spotfire 4.0

I had a briefing with TIBCO on their Spotfire 4.0 release, announced today and due to be released by the end of November. Spotfire is the analytics platform that TIBCO acquired a few years back, and provides end-user tools for dimensional analysis of data. This includes both visualization and mashups with other data sources, such as ERP systems.

In 4.0, they have focused on two main areas:

  • Analytic dashboards for monitoring and interactive drilldowns. This seems more like the traditional BI dashboards market, whereas Spotfire is known for its multidimensional visualizations, but I expect that business customers find that just a bit too unstructured at times.
  • Social collaboration around data analysis, both in terms of finding contributors and publishing results, by allowing Spotfire analysis to be embedded in SharePoint or shared with tibbr, and by including tibbr context in an analysis.

I did get a brief demo, starting with the dashboards. This starts out like a pretty standard dashboard, but does show some nice tools for the user to change the views, apparently including custom controls that can be created without development. The dynamic visualization is good, as you would expect if you have ever seen Spotfire in full flight: highlighting parts of one visualization object (graph or table) causes the corresponding bits in the other visualizations on the screen to be highlighted, for example.

Spotfire 4.0 - tibbr in sidebar of dashboard

There’s also some built-in collaboration: a chart on the Spotfire dashboard can be shared on tibbr, which has a static snapshot of the chart shared in a discussion thread but links back directly to the live visualization, [Insert obligatory iPad demo here] whereas sharing in SharePoint embeds the live visualization rather than a static shot. Embedding a tibbr discussion as context within an analysis is really less of an integration than just a side-by-side complementary viewing: you can have a tibbr discussion thread viewed on the same page as part of the analysis, although the tibbr thread is not itself being analyzed.

I found that the integration/collaboration was a bit lightweight, some of it no more than screen sharing (like the difference between a portal and a composite application). However, the push into the realm of more traditional dashboards will allow Spotfire to take on the more traditional BI vendors, particularly for data related to other TIBCO products, such as ActiveMatrix BPM.

[Update: All screenshots from briefing; for some reason, Flickr doesn't want to show them as an embedded slideshow]

Tagged ,
2011 11 14 Sandy Kemsley BI Comments (0) Permalink

Taking Time To Remember

296354_10150930653380305_645435304_21318425_2030905227_nToday is Remembrance Day in Canada (Veterans’ Day if you are in the US), which marks the anniversary of the signing of the armistice in World War I on November 11, 1918. Today, this day is used to honor soldiers of all wars.

I started a little project last year, after finding my grandfather’s WWI journals and my father’s WWII journal: I have been blogging the journals on a daily basis, with each day’s entry on the same day, just shifted by 94 years and 67 years, respectively. My grandfather’s journal covers the entire period from the day he shipped out in 1916 until when he arrived home in 1919; we’re at November 1917 right now, so have a year and half to go. My father’s journal, unfortunately, only covers the period from January-September 1944, so is already finished, but considering that he was in the navy, and took troops into the beaches of Normandy during the invasion, there’s some pretty interesting reading.

For the most part, these are just a few sentences each day written by small-town boys who were not recognized war heroes: not usually the kind of stories that we read about the wars. My grandfather’s sense of melancholy and my father’s sense of adventure are interesting contrasts. Feel free to follow along, and help out with the handwriting when I can’t decipher the journal myself.

2011 11 11 Sandy Kemsley off topic Comments (0) Permalink

SAP NetWeaver Business Warehouse with HANA

Continuing in the SAP World Tour in Toronto today, I went to a breakout innovation session on NetWeaver Business Warehouse (BW) and HANA, with Steve Holder from their BusinessObjects center of excellence. HANA, in case you’ve been hiding from all SAP press releases in the past two years, is an analytic appliance (High-performance ANalytic Applicance, in fact) that includes hardware and in-memory software for real-time analysis of non-aggregated information (i.e., not complex event processing). Previously, you would have had to move your BW data (which had probably already been ETL’d from your ERP to BW) over to HANA in order to take advantage of that processing power; now, you can actually make HANA be the persistence layer for BW instead of a relational database such as Oracle or DB2, so that the database behind BW becomes HANA. All the features of BW (such as cubes and analytic metadata) can be used just as they always could be, and any customizations such as custom extractors already done on BW by customers are supported, but moving to an in-memory provides a big uplift in speed.

Previously, BW provided data modeling, an analytical/planning engine, and data management, with the data storage in a relationship database. Now, BW only provides the data modeling, and everything else is pushed into HANA for in-memory performance. What sort of performance increases? Early customer pilots are seeing 10x faster data loading, 30x faster reporting (3x faster than BW Accelerator, another SAP in-memory analytics option), and a 20% reduction in administration and maintenance (no more RDBMS admins and servers). This is before the analytics have been optimized for in-memory: this is just a straight-up conversion of their existing data into HANA’s in-memory columnar storage. Once you turn on in-memory InfoCubes, you can eliminate physical cubes in favor of virtual cubes; there are a lot of other optimizations that can be done by eventually refactoring to take advantage of HANA’s capabilities, allowing for things such as interfacing to predictive analytics, and providing linear scaling of data, users and analysis.

This is not going to deprecate BW Accelerator, but provides options for moving forward that include a transition migration path from BWA to BW on HANA. BWA, however, provides performance increases for only a subset of BW data, so you can be sure that SAP will be encouraging people to move from BWA to BW on HANA.

A key message is that customers’ BW investments are completely preserved (although not time spent on BWA), since this is really just a back-end database conversion. Eventually, the entire Business Suite ERP system will run on top of HANA, so that there will be no ETL delay in moving operational data over to HANA for analysis; presumably, this will have the same sort of transparency to the front-end applications as does BW on HANA.

Tagged
2011 11 10 Sandy Kemsley BI
SAP Comments (0) Permalink

SAP World Tour Toronto: Morning Keynotes

There was a big crowd out for SAP’s only Canadian stop in its World Tour today: about 900 people in the keynote as Mark Aboud took the stage to discuss how SAP helps companies run their business, and look at the business trends in Canada right now: focus on the customer to create an experience; improve employee engagement by providing them with better tools and information to do their job better, increase speed in operations, managing information and distributing information. He moved on to talk about three technology trends, which echo what I heard at CASCON earlier this week: big data, cloud and mobility. No surprises there. He then spoke about what SAP is doing about these business and technology trends, which is really the reason that we’re all here today: cloud, analytics and mobility. Combined with their core ERP business, these “new SAP” products are where SAP is seeing market growth, and where they seem to be focusing their strategy.

He then invited CBC business correspondent Amanda Lang to the stage to talk further about productivity and innovation. It’s not just about getting better – it’s about getting better faster. This was very much a Canadian perspective, which means a bit of an inferiority complex comparing ourselves to the Americans, but also some good insights into the need to change corporate culture in order to foster an atmosphere of innovation, including leaving room for failure. Aboud is also providing some good insights into how SAP is transforming itself, in addition to what their customers are doing. SAP realized that they needed to bring game-changing technology to the market, and now see HANA as being as big for SAP as R/3 was back in the day. As Lang pointed out, service innovation is as important (or even more so) than product innovation in Canada, and SAP is supporting service businesses such as banking in addition to their more traditional position in product manufacturing companies.

Next up was Gary Hamel, recently named by the Wall Street Journal as the world’s most influential business thinker. Obviously, I’m just not up on my business thinkers, because I’ve never heard of him; certainly, he was a pro at business-related sound bytes.  He started off by asking what makes us inefficient, and talking about how we’re at an inflection point in terms of the rate of change required by business today. Not surprisingly, he sees management as the biggest impediment to efficiency and innovation, and listed three problematic characteristics that many companies have today:

  • Inertial (not very adaptable)
  • Incremental (not very innovative)
  • Insipid (not very inspiring)

He believes that companies need to foster with initiative, creativity and passion in their employees, not obedience, diligence and intellect. I’m not sure that a lot of companies would survive without intellect, but I agree with his push from feudal “Management 1.0” systems to more flexible organizations that empower employees. Management 1.0 is based on standardization, specialization, hierarchy, alignment, conformance, predictability and extrinsic rewards. Management 2.0 is about transparency (giving people the information that they need to do their job), disaggregation (breaking down the corporate power structures to give people responsibility and authority), natural hierarchies (recognizing people’s influence as measured by how much value they add), internal markets (providing resources inside companies based on market-driven principles rather than hierarchies, allowing ideas to come from anyone), communities of passion (allowing people to work on the things for which they have passion in order to foster innovation), self-determination (allowing freedom to move within corporate control structures based on value added), and openness (external crowdsourcing). Lots of great ideas here, although guaranteed to shake up most companies today.

The only bad note of the morning (aside from having to get up early, rent a Zipcar and drive through morning rush hour to an airport-area conference center far from downtown) was on the Women’s Leadership Forum breakfast. Moderated by a Deloitte partner, the panel included a VP of Marketing from Bell and Director of Legal for Medtronic. Where are the women in technology? Where are the women entrepreneurs? The woman from Bell, when asked about lessons that she could share, started with “work harder, every day – just that extra half hour or so”. That is so wrong. We need to be working smarter, not longer hours, and we need to take time away from work so that we’re not focused on it every day of our life if we expect to show true innovative leadership. About 20 minutes into the conversation, when the moderator turned the talk away from business and started asking about their children, horseback riding and the dreaded “work-life balance”, I left. What other business leadership forum that didn’t have the word “women” in the title would have included such topics? Quite frankly, this was an embarrassment.

Tagged
2011 11 10 Sandy Kemsley SAP Comments (0) Permalink

Aligning BPM and EA Tutorial at BBCCon11

I reworked my presentation on BPM in an enterprise architecture context (a.k.a., “why this blog is called ‘Column 2’”) that I originally did at the IRM BPM conference in London in June, and presented it at the Building Business Capability conference in Fort Lauderdale last week. I removed much of the detailed information on BPMN, refined some of the slides, and added in some material from Michael zur Muehlen’s paper on primitives in BPM and EA. Some nice improvements, I thought, and it came in right on time at 3 hours without having to skip over some material as I did in London.

Aligning BPM and EA

Here are some of the invaluable references that I used in creating this presentation:

That should give you plenty of follow-on reading if you find my slides to be too sparse on their own.

Tagged
2011 11 08 Sandy Kemsley BPM
EA Comments (0) Permalink

Sal Vella on Technologies for a Smarter Planet at CASCON2011

I attended the keynote at IBM’s CASCON conference in Toronto today, where Judy Huber, who directs the IBM Canada software lab, kicked off the session by reminding us that IBM software development has been happening in Canada since 1967 and continues to grow, and of the importance of collaboration between the research and industry communities. She introduced Joanna Ng, who is the head of research at the lab, to congratulate the winners of the most influential paper from CASCON 2001 (that date is not a typo, it’s a 10-year thing): Svetlana Kiritchenko and Stan Matwin for “Classification with Co-Training” (on email classification).

The main speaker was Sal Vella, VP of architecture and technology within the IBM software group, talking about technologies to build solutions for a smarter planet. Fresh from the IOD conference two weeks ago, I was all primed for this; there was a great booth at IOD that highlighted “smarter technology” with some interesting case studies. IBM’s smarter planet initiative is about technologies that allow us to do things that we were never able to do before, much of which is based on the immeasurable volume of data constantly produced by people, devices and systems. Consider electricity meters, like the one that you have in your home: it used to be that these were read once per month (if you were lucky) by a human, and the results entered into a billing system. Now, smart meters are read every 15 minutes to allow for time of use billing that rewards people for shifting their electricity usage away from peak periods. Analytics are being used in ways that they were never used before, and he discussed the popular Moneyball case of building a sports team based on player statistics. He also spoke about an even better use of analytics to create “solutions for a partying planet”: a drinks supplier predicting sports games outcomes to ensure that the pubs frequented by the fans of the teams most like to win had enough alcohol on hand to cover the ensuing parties. Now that’s technology used for the greater good. Winking smile

There are a lot of examples of big data and analytics that were previously unmanageable that are now becoming reasonable targets, most of which could be considered event-based: device instrumentation, weather data, social media, credit card transactions, crime statistics, traffic data and more. There are also some interesting problems in determining identity and relationships: figuring out who people really are even when they use different versions of their name, and who they are connected to in a variety of different ways that might indicate potential for fraud or other misrepresentation. Scary and big-brotherish to some, but undeniably providing organizations (including governments) with deeper insights into their customers and constituents. If those who complain about governments using this sort of technology “against” them would learn how to use it themselves, the tables might be turned as we gain insight into how well government is providing services to us.

We heard briefly from Charles Gauthier, acting director at the institute for information technology at National Research Council (NRC) Canada. NRC helped to create the CASCON conference 21 years ago, and continue to sponsor it; they support research in a number of areas that overlap with CAS and the other researchers and exhibitors presenting here.

The program chairs, Marin Litoiu of York University and Eleni Stroulia of University of Alberta presented awards for the two outstanding papers from the 22 papers at the conference:

  • “Enhancing Applications Robustness in Cloud Data Centres” by Madalin Mihailescu, Andres Rodriguez and Cristiana Amza of University of Toronto, and Dmitrijs Palcikovs, Gabriel Iszlai, Andrew Trossman and Joanna Ng of IBM Canada
  • “Parallel Data Cubes on Multi-Core Processors with Multiple Disks” for best student paper, by Hamidreza Zaboli and Frank Dehne of Carlton University

We finished with a presentation by Stan Matwin of University of Ottawa, co-author of the most influential paper presentation on email classification from the CASCON of 10 years past (his co-author is due to give birth on Wednesday, so decided not to attend). It was an interesting look at how the issue of email classification has continued to grow in the past 10 years; systems have become smarter since then, and we have automated spam filtering as well as systems for suggesting actions to take (or even taking actions without human input) on a specific message. The email classification that they discussed in their paper was based on classification systems where multiple training sets were used in concert to provide an overall classification for email messages. For example, two messages might both use the word “meeting” and a specific time in the subject line, but one might include a conference room reference in the body while the other references the local pub. Now, I often have business meetings in the pub, but I understand that many people do not, so I can see the value of such a co-training method. In 2001, they came to the conclusion that co-training can be useful, but is quite sensitive to its parameters and the learning algorithms used. Email classification has progressed since then: Bayesian (and other) classifiers have improved drastically, data representation is richer (through the use of meta formats and domain-specific enrichment) to allow for easier classification. social network and other information can be correlated, and there are specific tailored solutions for some email classification applications such as legal discovery. Interesting to see this sort of perspective on a landmark paper in the field of email classification.

I’m not sticking around for any of the paper presentations, since the ones later today are a bit out of my area of interest, and I’m booked the rest of the week on other work. However, I have the proceedings so will have a chance to look over the papers.

Tagged
2011 11 07 Sandy Kemsley BI Comments (0) Permalink

NSERC BI Network at CASCON2011 (Part 2)

The second half of the workshop started with Renée Miller from University of Toronto digging into the deeper database levels of BI, and the evolving role of schema from a prescriptive role (time-invariant, used to ensure data consistency) to a descriptive role (describe/understand data, capture business knowledge). In the old world, a schema was meant to reduce redundancy (via Boyce-Codd normal form), whereas the new world schema is used to understand data, and the schema may evolve. There are a lot of reasons why data can be “dirty” – my other half, who does data warehouse/BI for a living, is often telling me about how web developers create their operational database models mostly by accident, then don’t constrain data values at the UI – but the fact remains that no matter how clean you try to make it, there are always going to be operational data stores with data that needs some sort of cleansing before effective BI. In some cases, rules can be used to maintain data consistency, especially where those rules are context-dependent. In cases where the constraints are inconsistent with the existing data (besides asking the question of how that came to be), you can either repair the data, or discover new constraints from the data and repair the constraints. Some human judgment may be involved in determining whether the data or the constraint requires repair, although statistical models can be used to understand when a constraint is likely invalid and requires repair based on data semantics. In large enterprise databases as well as web databases, this sort of schema management and discovery could be used to identify and leverage redundancy in data to discover metadata such as rules and constraints, which in turn could be used to modify the data in classic data repair scenarios, or modify the schema to adjust for a changing reality.

Sheila McIlraith from University of Toronto presented on a use-centric model of data for customizing and constraining processes. I spoke last week at Building Business Capability on some of the links between data and processes, and McIlraith characterized processes as a purposeful view of data: processes provide a view of the data, and impose policies on data relative to some metrics. Processes are also, as she pointed out, are a delivery vehicle for BI – from a BPM standpoint, this is a bit of a trivial publishing process – to ensure that the right data gets to the right stakeholder. The objective of her research is to develop business process modeling formalism that treats data and processes as first class citizens, and supports specification of abstract (ad hoc) business processes while allowing the specification of stakeholder policies, preferences and priorities. Sounds like data+process+rules to me. The approach is to specify processes as flexible templates, with policies as further constraints; although she represents this as allowing for customizable processes, it really just appears to be a few pre-defined variations on a process model with a strong reliance on rules (in linear temporal logic) for policy enforcement, not full dynamic process definition.

Lastly, we heard from Rock Leung from SAP’s academic research center and Stephan Jou from IBM CAS on industry challenges: SAP and IBM are industry partners to the NSERC Business Intelligence Network. They listed 10 industry challenges for BI, but focused on big data, mobility, consumable analytics, and geospatial and temporal analytics.

  • Big data: Issues focus on volume of data, variety of information and sources, and velocity of decision-making. Watson has raised expectations about what can be done with big data, but there are challenges on how to model, navigate, analyze and visualize it.
  • Consumable analytics: There is a need to increase usability and offering new interactions, making the analytics consumable by everyone – not just statistical wizards – on every type of device.
  • Mobility: Since users need to be connected anywhere, there is a need to design for smaller devices (and intermittent connectivity) so that information can be represented effectively, and seamless with representations on other devices. Both presenters said that there is nothing that their respective companies are doing where mobile device support is not at least a topic of conversation, if not already a reality.
  • Geospatial and temporal analytics: Geospatial data isn’t just about Google Maps mashups any more: location and time are being used as key constraints in any business analytics, especially when you want to join internal business information with external events.

They touched briefly on social in response to a question (it was on their list of 10, but not the short list), seeing it as a way to make decisions better.

For a workshop on business intelligence, I was surprised at how many of the presentations included aspects of business rules and business process, as well as the expected data and analytics. Maybe I shouldn’t have been surprised, since data, rules and process are tightly tied in most business environments. A fascinating morning, and I’m looking forward to the keynote and other presentations this afternoon.

Tagged
2011 11 07 Sandy Kemsley BI
BPM
BRM Comments (0) Permalink

NSERC BI Network at CASCON2011 (Part 1)

I only have one day to attend CASCON this year due to a busy schedule this week, so I am up in Markham (near the IBM Toronto software lab) to attend the NSERC Business Intelligence Network workshop this morning. CASCON is the conference run by IBM’s Centers for Advanced Studies throughout the world, including the Toronto lab (where CAS originated), as a place for IBM researchers, university researchers and industry to come together to discuss many different areas of technology. Sometimes, this includes BPM-related research, but this year the schedule is a bit light on that; however, the BI workshop promises to provide some good insights into the state of analytics research.

Eric Yu from University of Toronto started the workshop, discussing how BI can enable organizations to become more adaptive. Interestingly, after all the talk about enterprise architecture and business architecture at last week’s Building Business Capability conference, that is the focus of Yu’s presentation, namely, that BI can help enterprises to better adapt and align business architecture and IT architecture. He presented a concept for an adaptive enterprise architecture that is owned by business people, not IT, and geared at achieving measurable business success. He discussed modeling variability at different architectural layers, and the traceability between them, and how making BI an integral part of an organization – not just the IT infrastructure – can support EA adaptability. He finished by talking about maturity models, and how a closed loop deployment of BI technologies can help meet adaptive enterprise requirements. Core to this is the explicit representation of change processes and their relationship to operational processes, as well as linking strategic drivers to specific goals and metrics.

Frank Tompa from University of Waterloo followed with a discussion of mapping policies (from a business model, typically represented as high-level business rules) to constraints (in a data model) so that these can be enforced within applications. My mind immediately went to why you would be mapping these to a database model rather than a rules management system; his view seems to be that a DBMS is what monitors at a transactional level and ensures compliance with the business model (rules). His question: “how do make the task of database programming easier?” My question: “why aren’t you doing this with a BRMS instead of a DBMS?” Accepting his premise that this should be done by a database programmer, the approach is to start with object definitions, where an object is a row (tuple) defined by a view over a fixed database schema, and represents all of the data required for policy making. Secondly, consider the states that an object can assume by considering that an object x is in state S if its attributes satisfy S(x). An object can be in multiple states at once; the states seem to be more like functions than states, but whatever. Thirdly, the business model has to be converted to an enforcement model through a sort of process model that also includes database states; really more of a state diagram that maps business “states” to database states, with constraints on states and state transitions denoted explicitly. I can see some value in the state transition constraint models in terms of representing some forms of business rules and their temporal relationships, but his representation of a business process as a constraint diagram is not something that a business analyst is ever going to read, much less create. However, the role of the business person seems to be restricted to “policy designer” listing “states of interest”, and the goal of this research is to “form a bridge between the policy manager and the database”. Their future work includes extracting workflows from database transaction logs, which is, of course, something that is well underway in the BPM data mining community. I asked (explicitly to the presenter, not just snarkily here in my blog post) about the role of rules engines: he said that one of the problems was in vocabulary definition, which is often not done in organizations at the policy and rules level; by the time things get to the database, the vocabulary is sufficiently constrained that you can ensure that you’re getting what you need. He did say that if things could be defined in a rules engine using a standardized vocabulary, then some of the rules/constraints could be applied before things reached the database; there does seem to be room for both methods as long as the business rules vocabulary (which does exist) is not well-entrenched.

Jennifer Horkoff from University of Toronto was up next discussing strategic models for BI. Her research is about moving BI from a technology practice to a decision-making process that starts with strategic concerns, generates BI queries, interprets the results relative to the business goals and decide on necessary actions. She started with the OMG Business Motivation Model (BMM) for building governance models, and extended that to a Business Intelligence Model (BIM), or business schema. The key primitives include goals, situations (can model SWOT), indicators (quantitative measures), influences (relationships) and more. This model can be used at the high-level strategic level, or at a more tactical level that links more directly to activities. There is also the idea of a strategy, which is a collection of processes and quality constraints that fulfill a root-level goal. Reasoning that can be done with BIMs, such as whether a specific strategy can fulfill a specific goal, and influence diagrams with probabilities on each link used to help determine decisions. They are using BIM concepts to model a case study with Rouge Valley Health System to improve patient flow and reduce wait times; results from this will be seen in future research.

Each of these presentations could have filled a much bigger time slot, and I could only capture a flavor of their discussions. If you’re interested in more detail, you can contact the authors directly (links to each above) to get the underlying research papers; I’ve always found researchers to be thrilled that anyone outside the academic community is interested in what they’re doing, and are happy to share.

We’re just at the md-morning break, but this is getting long so I’ll post this and continue in a second post. Lots of interesting content, I’m looking forward to the second half.

Tagged
2011 11 07 Sandy Kemsley BI
BPM
BRM
EA Comments (0) Permalink

Catch Me Twice On “Webinar Week”

I’m presenting on two webinars this week. First, on Tuesday (tomorrow), I will be joining Jeremy Westerman of TIBCO to discuss the BPM issues and challenges specific to large enterprises. It’s at 11am Eastern (8am Pacific) on Tuesday, and you can sign up here.

Then, on Wednesday, I’ll be presenting with Matt Cicciari of Progress on how BPM can work within an application development environment. Since this is targeted at Progress OpenEdge developers who may not know a lot about BPM, I’ll be covering some BPM background plus why you want to do certain things with a BPMS, such as explicit process modeling. This is at 11am Eastern on Wednesday, and you can sign up here.

These two gigs are sandwiched between IBM’s CASCON today, where I am attending the NSERC Business Intelligence workshop in the morning and the keynote presentations in the afternoon, and SAP’s World Tour on Thursday. Both of these, although not requiring me to get on an airplane, do require me to get in a Zipcar and drive to the far reaches of the Toronto suburbs and beyond.

2011 11 07 Sandy Kemsley BPM Comments (0) Permalink

Improving Process Quality with @TJOlbrich

My last session at Building Business Capability before heading home, and I just had to sit in on Thomas Olbrich’s session on some of the insights into process quality that he has gained through the Process TestLab. Just before the session, he decided to retitle it as “How to avoid being mentioned by Roger Burlton”, namely, not being one of the process horror stories that Roger loves to share.

According to many analyst studies, only 18% of business process projects achieve their scope and objectives while staying on time and on budget, making process quality more of an exception than the rule. In the Process TestLab, they see a lot of different types of process quality errors:

  • 92% have logical errors
  • 62% have business errors
  • 95% have dynamic defects that would manifest in the environment of multiple processes running simultaneously, and having to adapt to changing conditions
  • 30% are unsuited to the real-world business situation

Looking at their statistics for 2011 to date, about half of the process defects are due to discrepancies between models and the verbal/written description – what would typically be considered “requirements” – with the remainder spread across a variety of defects in the process models themselves. The process model defects may manifest as endless loops, disappearing process instances, missing data and a variety of other undesired results.

He presented four approaches for improving process quality:

  • Check for process defects at the earliest possible point in the design phase
  • Validate the process before implementing, either through manual reenactment, simulation, the TestLab approach which simulates the end-user experience as well as the flow, or a BPMS environment such as IBM BPM (formerly Lombardi) that allows playback of models and UI very early in the design phase
  • Check for practicability to determine if the process will work in real life
  • Understand the limits of the process to know when it will cease to deliver when circumstances change

Olbrich’s approach is based on the separation of business-based modeling of processes from IT implementation: he sees that these sort of process quality checks are done “before you send the process over to IT for implementation”, which is where their service fits in. Although that’s still the norm in many cases, as model-driven development becomes more business-friendly, the line between business modeling and implementation is getting fuzzier in some situations. However, in most complex line-of-business processes, especially those that use quite a bit of automation and have complex user experience, this separation is still prevalent.

Some of his case studies certainly bear this out: a fragment of the process models sent to them by a telecom customer filled an entire slide, even though the activities in the processes were only slightly bigger than individual pixels. The customer had “tested” the process themselves already, but using the typical method of showing the process, encouraging people to walk through it as quickly as possible, and sign off on it. In the Process TestLab, they found 120 defects in process logic alone, meaning that the processes would never have executed as modeled, and 20 process integration defects that determine how different processes related to each other. Sure, IT would have worked around those defects during implementation, but then the process as implemented would be significantly different from the process as modeled by the business. That means that the business’ understanding and documentation of their processes are flawed, and that IT had to make changes to the processes – possibly without signoff from the business – that may actually change the business intention of the processes.

It’s necessary to use context when analyzing and optimizing processes in order to avoid verschlimmbesserung, roughly translated as “improvements that make things worse”, since the interaction between processes is critical: change is seldom limited to a single process. This is where process architecture can help, since it can show the relations between processes as well as the processes themselves.

Testing process models by actually experiencing them, as if they were already live, allows business users and analysts to detect flaws while they are still in the model stage by standing in for the users of the intended process and seeing if they could do the assigned business task given the user interface and information at that point in the process. Process TestLab is certainly one way to do that, although a sufficiently agile model-driven BPMS could probably do something similar if it were used that way (which most aren’t). In addition to this type of live testing, they also do more classic simulation, highlighting bottlenecks and other timing-related problems across process variations.

The key message: process quality starts at the very beginning of the process lifecycle, so test your processes before you implement, rather than trying to catch them during system testing. The later that errors are identified, the more expensive it is to fix them.

Tagged
2011 11 03 Sandy Kemsley BPM Comments (0) Permalink

What Analysts Need to Understand About Business Events

Paul Vincent, CTO of Business Rules and CEP at TIBCO (and possibly the only person at Building Business Capability sporting a bow tie), presented a less technical view of events that you would normally see in one of his presentation, intended to have the business analysts here at Building Business Capability understand what events are, how they impact business processes, and how to model them. He started with a basic definition of events – an observation, a change in state, or a message – and why we should care about them. I cover events in the context of processes in many of the presentations that I give (including the BPM in EA tutorial that I did here on Monday), and his message is the same: life is event-driven, and our business processes need to learn to deal with that fact. Events are one of the fundamentals of business and business systems, but many systems do not handle external events well. Furthermore, many process analysts don’t understand events or how to model them, and can end up creating massive spaghetti process models to try and capture the result of events since they don’t understand how to model events explicitly.

He went through several different model types that allow for events to be captured and modeled explicitly, and compared the pros and cons of each: state models, event process chain models, resources events agents (REA) models, and BPMN models. The BPMN model is the only one that really models events in the context of business processes, and relates events as drivers of process tasks, but is really only appropriate for fairly structured processes. It does, however, allow for modeling 63 different types of events, meaning that there’s probably nothing that can happen that can’t be modeled by a BPMN event. The heavy use of events in BPMN models can make sense for heavily automated processes, and can make the process models much more succinct. Once the event notation is understood, it’s fairly easy to trace through them, but events are the one thing in BPMN that probably won’t be immediately obvious to the novice process analyst.

In many cases, individual events are not the interesting part, but rather a correlation between many events; for example, fraud events may be detected only have many small related transactions have occurred. This is the heart of complex event processing (CEP), which can be applied to a wide variety of business situations that rely on large volumes of events, and distinguishes between simple process patterns and business rules that can be applied to individual transactions.

Looking at events from an analyst’s view, it’s necessary to identify actors and roles, just as in most use cases, then identify what they do and (more importantly) when they do it in order to drive out the events, their sources and destinations. Events can be classified as positive (e.g., something that you are expecting to happen actually happened), negative (e.g., something that you are expecting to happen didn’t happen within a specific time interval) or sets (e.g., the percentage of a particular type of event is exceeding an SLA). In many cases, the more complex events that we start to see in sets are the ones that you’re really interested in from a business standpoint: fraud, missed SLAs, gradual equipment failure, or customer churn.

He presented the EPTS event reference architecture for complex events, then discussed how the different components are developed during analysis:

  • Event production and consumption, namely, where events come from and where they go
  • Event preparation, or what selection operations need to be performed to extract the events, such as monitoring, identification and filtering
  • Event analysis, or the computations that need to be performed on the individual events
  • Complex event detection, that is, the event correlations and patterns that need to performed in order to determine if the complex event of interest has occurred
  • Event reaction, or what event actions need to be performed in reaction to the detected complex event; this can overlap to some degree with predictive analytics in order to predict and learn the appropriate reactions

He discussed event dependencies models, which show event orderings, and relate events together as meaningful facts that can then be used in rules. Although not a common practice, this model type does show relationships between events as well as linking to business rules.

He finished with some customer case studies that include CEP and event decision-making: FedEx achieving zero latency in determining where a package is right now; and Allstate using CEP to adjust their rules on a daily basis, resulting in a 15% increase in closing rates.

A final thought that he left us with: we want agile processes and agile decisions; process changes and rule changes are just events. Analyzing business events is good, but exploiting business events is even better.

Tagged
2011 11 03 Sandy Kemsley CEP Comments (2) Permalink

Process and Information Architectures

Last day of the Building Business Capability conference, and I attended Louise Harris’ session on process and information architectures as the missing link to improving enterprise performance. She was on the panel on business versus IT architecture that I moderated yesterday, and had a lot of great insight into business architecture and enterprise architecture.

Today’s session highlighted how business processes and information are tightly interconnected – business processes create and maintain information, and information informs and guides business processes – but that different types of processes use information differently. This is a good distinction: looking at what she called “transactional” (structured)  versus “creative” (case management) versus “social” (ad hoc), where transactional processes required exact data, but the creative and social processes may require interpretation of a variety of information sources that may not be known at design time. She showed the Burlton Hexagon to illustrate how information is not just input to be processed into output, but also used to guide processes, inform desisions and measure process results.

This led to Harris’ definition of a business process architecture as “defining the business processes delivering results to stakeholders and supported by the organization/enterprise showing how they are related to each other and to the strategic goals of the organization/enterprise”. (whew) This includes four levels of process models:

  • Business capability models, also called business service models or end-to-end business process models, which is the top level of the work hierarchy that defined what business processes are, but not how they are performed. Louise referenced this to a classic EA standpoint as being row 1 of Zachman (in column 2).
  • Business process models, which provide deeper decomposition of the end-to-end models that tie them to the KPIs/goals. This has the effect of building process governance into the architecture directly.
  • Business process flow models, showing the flow of business processes at the level of logistical flow, such as value chains or asset lifecycles, depending on the type of process.
  • Business process scope models (IGOEs, that is, Inputs, Guides, Outputs, Enablers), identifying the resources involved in the process, including information, people and systems.

She moved on to discuss information architecture, and its value in defining information assets as well as content and usage standards. This includes three models:

  • Information concept model with the top level of the information related to the business, often organized into domains such as finance or HR. For example, in the information domain of finance, we might have information subject areas (concepts) of Invoicing, capital assets, budget, etc.
  • Information relationship model defines the relationships between the concepts identified in the information concept model, which can span different subject areas. This can look like an ERD, but the objects being connected are higher-level business objects rather than database objects: this makes it fairly tightly tied to the processes that those business objects undergo.
  • Information governance model, which defines that has to be done to maintain information integrity: governance structure, roles responsible, and policy and business standards.

Next was bringing together the process and information architectures, which is where IGOE (business process scope models) come into play, since they align information subject areas with top level business processes or business capabilities, allowing identification of gaps between process and information. This creates a framework for ensuring alignment at the design and operational levels, but does not map information subject areas to business functions since that is too dependent on the organizational structure.

Harris presented these models as being the business architecture, corresponding to rows 1 and 2 of Zachman (for example), which can then be used to provide context for the remainder of the enterprise architecture and into design. For example, once these models are established, the detailed process design can be integrated with logical data models.

She finished up by looking at how process and information architectures need to be developed in lock step, since business process ensures information quality, while information ensures process effectiveness.

Tagged
2011 11 03 Sandy Kemsley BPM
EA Comments (3) Permalink

Assessing BPM Maturity with @RogerBurlton

Roger Burlton held a joint session across several of the tracks on assessing BPM maturity, starting with the BPTrends pyramid of process maturity, which ranges from a wide base of the implementation level, to the middle tier of the business process level, up to the enterprise level that includes strategy and process architecture. He also showed his own “Burlton Hexagon” of the disciplines that form around business process and performance: policy and rules, human capital, enabling technologies, supporting infrastructure, organizational structure, and intent and strategy. His point is that not everyone is ready for maturity in all the areas that impact BPM (such as organizational structure), although they may be doing process transformation projects that require greater maturity in many of these other areas. At some level, these efforts must be able to be traced back to corporate strategy.

He presented a process maturity model based on the SEI capability maturity model, showing the following levels:

  1. Initial – zero process organizations
  2. Repeatable – departmental process improvement projects, some cross-functional process definition
  3. Defined – business processes delivered and measurements defined
  4. Managed – governance system implemented
  5. Optimizing – ongoing process improvement

Moving from level 2 to 3 is a pretty straightforward progression that you will see in many BPM “project to program” initiatives, but the jump to level 4 requires getting the high-level management on board and starting to make some cultural shifts. Organizations have to be ready to accept a certain level of change and maturity: in fact, organizational readiness will always constrain achievement of greater maturity, and may even end up getting the process maturity team in trouble.

He presented a worksheet for assessing your enterprise BPM gap, with several different factors on which you are intended to mark the current state, the desired future state, and the organizational management (labeled as “how far will management let you go?”). The factors include enterprise context, value chain models, alignment of resources with business processes, process performance measurement system, direct management responsibility for value chains, and a process CoE. By marking the three states (as is, to be, and what can we get away with) on each of these as a starting point, it allows you to see not just the spread between where you are and where you need to be, but adds in that extra dimension of organizational readiness for moving to a certain level of process maturity.

Depending on whether your organization is ready to crawl, walk or run (as defined by your organizational readiness relative to the as-is and to-be states), there are different techniques for getting to the desired maturity state: for those with low organizational readiness, for example, you need to focus on increasing that first, then evolve the process capabilities together with readiness as it increases. Organizational readiness at the executive level manifests as understanding, willingness and ability to do their work differently: in many cases, executives don’t want to change how they do their work, although they do want to reap the benefits of increased process maturity.

He showed a more detailed spreadsheet of a maturity and readiness assessment for a large technology company, color-coded based on which factors contribute most to an increase in maturity, and which hold the most risk since they represent the biggest jump in maturity without necessarily having the readiness.

With such a focus on readiness, change management is definitely a big issue with increasing process maturity. In order to address this, there are a number of steps in a communication plan: understand the stakeholders’ concerns, determine the messages, identify the media for delivering the messages, identify timetables for communication and change, identify the messengers, create/identify change agents (who is sometimes the biggest detractor to start), and deliver the message and handle the feedback. In looking at stakeholder concerns as part of the communication plan, you need to look at levels from informational (“what is it”), personal (“how will it impact my job”), management (“how will the change happen”), consequences (“what are the benefits”) and on into collaboration where the buy-in really starts to happen.

Ultimately, you’re not trying to sell business process change (or BPMS) within the organization: you’re trying to sell improvements in business performance, particularly for processes that are currently painful. Focus on the business goals, and use a model of the customer experience to illustrate how process improvements can improve that experience and therefore help meet overall business goals.

Finishing up with the maturity model, if you’re at level 1 or 2, get an initial process steering committee and CoE in place for governance, and plan a simple process architecture targeted at change initiatives rather than governance. Get standards for tools and templates in place, and start promoting the process project successes via the CoE. This is really about getting some lightweight governance in place, showing some initial successes, and educating all stakeholders on what process can do for them.

If you’re at level 3 or 4, you need to be creating your robust process architecture in collaboration with the business, and socialize it across the enterprise. With the Process Council (steering committee) in place, make sure that the process stewards/owners report up to the council. Put process measurements are in place, and ensure that the business is being managed relative to those KPIs. Expand process improvement out to the related areas across the enterprise architecture, and create tools and methods within the CoE that make it easy to plan, justify and execute process initiatives.

Tagged
2011 11 02 Sandy Kemsley BPM Comments (1) Permalink

Accepting The Business Architecture Challenge with @logicalleap

Forrester analyst Jeff Scott presented at Building Business Capability on what business architecture is and what business architects do. According to their current research, interest in business architecture is very high – more than half of organizations consider it “very important”, and all organizations survey showed some interest – and more than half also have an active business architecture initiative. This hasn’t changed all that much since their last survey on this in 2008, although the numbers have crept up slightly. Surprisingly, less than half of the business architecture activities are on an enterprise-wide level, although if you combine that with those that have business architecture spanning multiple lines of business, it hits about 85%. When you look at where these organizations plan to take their business architecture programs, over 80% are planning for them to be enterprise-wide but that hasn’t changed in 3 years, meaning that although the intention is there, that may not actually be happening with any speed.

He moved on to a definition of business architecture, and how it has changed in the past 15 years. In the past, it used to be more like business analysis and requirements, but now it’s considered an initiative (either by business, EA or IT) to improve business performance and business/IT alignment. The problem is, in my opinion, that the term “business/IT alignment” has become a bit meaningless in the past few years as every vendor uses it in their marketing literature. Process models are considered a part of business architecture by a large majority of organizations with a business architecture initiative, as are business capability models and business strategy, application portfolio assessments, organizational models and even IT strategy.

Business architecture has become the hot new professional area to get into, whether you’re a business analyst or an enterprise architecture, which means that it’s necessary to have a better common understanding of what business architecture actually is and who the business architects are. I’m moderating a panel on this topic with three business/IT/enterprise architects today at 4:30pm, and plan to explore this further with them. Scott showed their research on what people did before they became (or labeled themselves as) business architects: most held a business analyst role, although many also were enterprise architects, application architects and other positions. Less than half of the business architects are doing it full time, so may still be fulfilling some of those other roles in addition. Many of them are resident in the EA group, and more than half of organizations consider EA to be responsible for the outcomes of business architecture.

It’s really a complex set of factors in figuring out what business architects do: some of them are working on enterprise-wide business transformation, while others are looking at efficiency within a business unit or project. The background of the business architect – that is, what they did before they became a business architect – can hugely impact (obviously) the scope and focus of their work as a business architect. In fact, is business architecture a function performed by many players, or is it a distinct role? Who is really involved in business architecture besides those who hold the title, and where do they fit in the organization? As Scott pointed out, these are unanswered questions that will plague business architecture for a few years still to come.

He presented several shifts to make in thinking:

  • Give up your old paradigms (think different; act different to get different results)
  • Start with “why” before you settle on the how and what
  • “Should” shouldn’t matter when mapping from “what is” to “what can be”
  • Exploration, not standardization, since enterprise architecture is still undergoing innovation on its way to maturity
  • Business architecture, not technology architecture, is what provides insight, risk management and leadership (rather than engineering, knowledge and management)
  • Stress on “business” in business architecture, not “architecture”, which may not fit into the EA frameworks that are more focused on knowledge
  • Focus on opportunity rather than efficiency, which is aligned with the shift in focus for BPM benefits that I’ve been seeing in the past few years
  • Complex problems need different solutions, including looking at the problems in context rather than just functional decomposition.
  • Solve the hard “soft” problems of building business architect skills and credibility, leveraging local successes to gain support and sponsorship, and overcome resistance to change
  • Think like the business before applying architectural thinking to the business problems and processes

He finished up with encouragement to become more business savvy: not just the details of business, but innovation and strategy. This can be done via some good reading resources, finding a business mentor and building relationships, while keeping in mind that business architecture should be an approach to clarify and illuminate the organization’s business model.

He wrote a blog post on some of the challenges facing business architects back in July, definitely worth a read as well.

Tagged
2011 11 02 Sandy Kemsley EA Comments (0) Permalink