Next week at at Dreamforce, Active Endpoints’s new Cloud Extend for Salesforce will officially go live. I had a briefing a few months back when it hit beta, and an update last week where I saw little new functionality from the first briefing, but some nice case studies and partner support.

Cloud Extend for Salesforce is a helper layer that integrates with Salesforce that allows business users to create lightweight processes and guides – think screenflows with context-sensitive scripting – to help users through complex processes in Salesforce. In Salesforce, as in many other ERP and CRM systems, achieving a specific goal sometimes requires a complex set of manual steps. Adding a bit of automation and a bit of structure, along with some documentation displayed to the user at the right time, can mean the difference between a process being done correctly or having some critical steps missed. If you look at the Cloud Extend case study with PSA Insurance & Financial Services covered in today’s press release, a typical “first call” sales guide created with Cloud Extend includes such actions as recording the prospect’s current policy expiration date, setting reminders for call-back, sending out collateral and emails to the prospect, and interacting with other PSA team members via Salesforce Chatter. This will mean that less follow-up items are missed, and improve the overall productivity of the sales reps since some of the actions are automated or semi-automated. Michael Rowley, CTO of Active Endpoints, wrote about about Cloud Extend at the time of the beta release, covering more of the value proposition that they are seeing by adding process to data-centric applications such as Salesforce. Lori MacVittie of F5 wrote about how although data and core processes can be commoditized and standardized, customer interaction processes need to be customized to be of greatest value. Interestingly, the end result is still a highly structured pre-defined process, although one that can be created by a business user using a simple tree structure.

When I saw a demo of Cloud Extend, I was reminded of similar guides and scripts that I’ve seen overlaid on other enterprise software to assist in user interactions, usually for telemarketing or customer service to be prompted on what to say on the phone to a customer, but this is more interactive than just scripts: it can actually update Salesforce data as part of the screenflow, hence making it more of a BPM tool than just a user scripting tool. Considering that the ActiveVOS BPMS is running behind the scenes, that shouldn’t come as a surprise, since it is optimized around integration activities. Yet, this is not an Active Endpoints application: the anchor application is Salesforce, and Cloud Extend is a helper app around that rather than taking over the user experience. In other words, instead of a BPMS silo in the clouds as we’re seeing from many BPMS cloud vendors, this is using a BPMS platform to facilitate a functionality integrated into another platform. A cloud “OEM” arrangement, if you please.

The Guide Designer – a portal into the ActiveVOS functionality from within Salesforce – allows a non-technical user to create a screen flow, add more screens and automated steps, call subflows, and call Salesforce functions. The flow can be simulated graphically, stepping forwards and backwards through it, in order to test different conditions; note that this is simulation in order to determine flow correctness, not for the purpose of optimizing the flow under load, hence is quite different from simulation that you might see in a full-featured BPA or BPMS tool. Furthermore, this is really intended to be a single-person screen flow, not a workflow that moves work between users: sort of like a macro, only more so. Although it is possible to interrupt a screen flow and have another person restart it, that doesn’t appear to be the primary use case.

There are a few bits that likely a non-technical user couldn’t do without a bit of initial help, such as creating automated steps and connecting up the completed guides to the Salesforce portal, but it is pretty easy to use. It uses a simple branching tree structure to represent the flow, where the presence of multiple possible responses at a step creates the corresponding number of outbound branches. In flowcharting terms, that means only OR gates, no merges and no loopbacks (although there is a Jump To Step capability that would allow looping back): it’s really more of a decision tree than what you might thing of as a standard process flow.

Creating a guide, or a “guidance tree” as it is called in the Guide Designer consists of adding an initial step, specifying whether it is a screen (i.e., a script for the user to read), an automated step that will call a Salesforce or other function, a subflow step that will call a predefined subflow, a jump to step that will transfer execution to another point in the tree, or an end step. Screen steps include a prompt and up to four answers to the prompt; this is the question that the user will answer at this point in response to what is happening on their customer call. One outbound path is added to the step for each possible answer, and a subsequent step automatically created on that path. The branches keep growing until end steps are defined on each branch.

A complex tree can obviously get quite large, but the designer UI has a nice way of scrolling up and down the tree: as you select a particular step, you see only the connected steps twice removed in either direction, with a visual indicator to show that the branch continues to extend in that direction.

Regardless of the complexity of the guidance tree, there is no palette of shapes or anything vaguely BPMN-ish: the user just creates one step after another in the tree structure, and the prompt and answers create the flow through the tree. Active Endpoints see this tree-like method of process design, rather than something more flowchart-like, to be a key differentiator. In reality, under the covers, it is creating BPMN that is published to BPEL, but the designer user interface just limits the design to a simple branching tree structure that is a subset of both BPMN and BPEL.

Once a flow is created and tested, it is published, which makes it available to run directly in the Salesforce sales guides section directly on a sales lead’s detail page. As the guide executes, it displays a history of the steps and responses, making it easy for the user to see what’s happened so far while being guided along through the steps.

Obviously, the Active Endpoints screen flows are executing in the cloud, although as of the April release, they were using Terremark rather than hosting it on Salesforce’s platform. Keeping it on an independent platform is critical for them, since there are other enterprise cloud software platforms with which they could integrate for the same type of benefits, such as Quickbooks and SuccessFactors. Since there is very little data persisted in the process instances within Cloud Extend, just some execution metrics for reporting and the Salesforce object ID for linking back to records in Salesforce, there is less concern about where this data is hosted, since it will never contain any personally identifiable information about a customer.

We’re starting to see client-side screen flow creation from a few of the BPMS vendors – I covered TIBCO’s Page Flow Models in my review of AMX/BPM last year – but those screen flows are only available at a step in a larger BPMS model, whereas Cloud Extend has encapsulated that capability for use in other platforms. For small, nimble vendors who don’t need to own the whole application, providing embeddable process functionality for data-centric applications can make a lot of sense, especially in a cloud environment where they don’t need to worry about the usual software OEM problems of installation and maintenance.

I’m curious about whatever happened to Salesforce’s Visual Process Manager and whether it will end up competing with Cloud Extend; I had a briefing of Visual Process Manager over a year ago that amounted to little, and I haven’t heard anything about it since. Neil Ward-Dutton mentions these two possibly-competing offerings in his post on the beta release of Cloud Extend, but as he points out, Visual Process Manager is more of a general purpose workflow tool, while Cloud Extend is focused on task-specific screen flows within the Salesforce environment. Just about the opposite of what you might have expected to come out of these respective vendors.

I just found my notes from a Salesforce.com lunch event that I went to in Toronto back in April, where Peter Coffee spoke enthusiastically while we ate three lovingly-prepared courses at Bymark, and was going to just pitch them out but found that there was actually quite a bit of good material in there. Not sure how I managed to write so much while still eating everything in front of me.

This came just a few days after the SF.com acquisition of Radian6, a move that increased the Canadian staff to 600. SF has about 1,500 customers in Canada, a few of whom where in the room that day. Their big push with these and all their customers is on strategic IT in the cloud, rather than just cost savings. One of the ways that they’re doing this is by incorporating process throughout the platform, allowing it to become a global user portal rather than just a collection of silos of information.

Coffee discussed a range of cloud platform types:

• Infrastructure as a service (IAAS) provides virtualization, but persists the old IT and application development models, combining the weaknesses of all of them. Although you’ve outsourced your hardware, you’re still stuck maintaining and upgrading operating systems and applications.
• Basic cloud application development, such as Google apps and their add-ons.
• SF.com, which provides a full application development environment including UI and application support.

The old model of customization, that most of us are familiar with in the IT world, has led to about 1/3 of all enterprise software running on the current version, and the rest stuck with a previous version, unable to do the upgrade because the customization has locked it in to a specific version. This is the primary reason that I am so anti-customization: you get stuck on that old version, and the cost of upgrading is not just the cost of upgrading the base software, but of regression testing (and, in the worst case, redeveloping) all the customization that was done on top of the old version. Any wonder that software maintenance ends up costing 10x the original purchase cost?

The SF.com model, however, is an untouchable core code base sitting on managed infrastructure (in fact, 23 physical instances with about 2,000 Dell servers), and the customization layer is just an abstraction of the database, business logic and UI so that it is actually metadata but appears to be a physical database and code. In other words, when you develop custom apps on the SF.com platform, you’re really just creating metadata that is fairly loosely coupled with the underlying platform, and resistant to changes therein. When security or any other function on the core SF.com platform is upgraded, it happens for all customers; virtualization or infrastructure-as-a-service doesn’t have that, but requires independent upgrades for each instance.

Creating an SF.com app doesn’t restrict you to just your app or that platform, however: although SF.com is partitioned by customer, it allows linkages between partners through remapping of business objects, leveraging data and app sharing. Furthermore, you can integrate with other cloud platforms such as Google, Amazon or Facebook, and with on-premise systems using Cast Iron, Boomi and Informatica. A shared infrastructure, however, doesn’t compromise security: the ownership metadata is stored directly with the application data to ensure that direct database access by an administrator doesn’t allow complete access to the data: it’s these layers of abstraction that help make the shared infrastructure secure. Coffee did punt on a question from the (mostly Canadian financial services) audience about having Canadian financial data in the US: he suggested that it could be encrypted, possibly using an add-on such as CipherCloud. They currently have four US data centers and one in Singapore, with plans for Japan and the EU; as long as customers can select the data center country location that they wish (such as on Amazon), that will solve a lot of the problem, since the EU privacy laws are much closer to those in Canada. However, recent seizures of US-owned offshore servers brings that strategy into question, and he made some comments about fail-overs between sites that makes me think that they are not necessarily segregating data by the country specified by the customer, but rather picking the one that optimizes performance. There are other options, such as putting the data on a location-specific Amazon instance, and using SF.com for just the process parts, although that’s obviously going to be a bit more work.

Although he was focused on using SF.com for enterprises, there are stories of their platform being used for consumer-facing applications, such as Groupon using the Force.com application development platform to power the entire deals cycle on their website. There’s a lot to be said for using an application development environment like this: in addition to availability and auto-upgrading, there’s also built-in support for multiples mobile devices without changing the application, using iTunes for provisioning, and adding Chatter for collaboration to any application. Add the new Radian6 capabilities to monitor social media and drive processes based on social media interactions and mentions, and you have a pretty large baseline functionality out of the box, before you even start writing code. There are native ERP system and desktop application connectors, and a large partner network offering add-ins and entire application suites.

I haven’t spent any time doing evaluation specifically of Salesforce or the Force.com application development platform (except for a briefing that I had over a year ago on their Visual Process Manager), but I’m a big fan of building applications in the cloud for many of the reasons that Coffee discussed. Yes, we still need to work out the data privacy issues; mostly due to the potential for US government intervention, not hackers. More importantly, we need to get over the notion that everything that we do within enterprises has to reside on our own servers, and be built from the metal up with fully customized code, because that way madness lies.

We finished the cloud case studies with Abhishek Agrawal of psHEALTH. He used to work for Appian, so likely had a bias in that direction already, but they selected Appian since they wanted a complete cloud solution, plus for the strong process modeling capabilities, data security infrastructure, scalability and robustness. Lastly, they liked that they could create solutions without writing code: they wanted to be a case management solutions provider without being a software company, something that a lot of outsourcing companies struggle with.

That doesn’t mean that they haven’t built their own components, rather that they haven’t done that using the usual “lines of code”; Agrawal stated that their total lines of code = 0. They built a library of process application components within Appian, then can easily assemble those into custom case management solutions for their clients.

Looking at the before and after of one of their clients in deploying the Appian-based case management solution, they doubled the number of cases that a case worker could handle (from 40 to 80) by making it much easier to access, work with and transfer case files. They’re also working on some mobile applications, including support for their clients’ case workers and also for the end-customers (patients) with things such as a smartphone-based medication diary.

Data security was key for them, being in the healthcare industry, and they gained ISO27001 and SAS-70 Type II certifications for their Appian-based applications, which says a lot about the potential for high security in the cloud. They also were able to go to the market with a complete product solution that required only minor tweaks for each client, rather than a complete custom build each time, making it much faster to onboard a new client. For them, a cloud-based solution and the easy ability to build new applications from a library of components have been key to their success.

Next up was John Cowles from Clayton Holdings, which does risk analysis for the mortgage industry.

Clayton has 240 users across 5 business units on Appian Cloud BPM, and they have only 5 primary resources for building and maintaining the 100+ processes that they have in production. They had limited IT resources and limited budget, and found that software-as-a-service fit their budget and resources well. Initially, they had no IT involvement at all: it was all operations, business analysts and process efficiency people. They found that Appian was easy enough to build applications without IT support, although now that they are undergoing some large back-end system changes, they do have a bit more technical input. They’ve seen an improvement in their BPM cultural maturity, and an increase in adoption rates as well as demand for new applications. Cowles now wants to do everything with Appian: he sees it as a general-purpose application assembly tool, not just a BPM tool. Interestingly, they did what I always recommend: limited or no system integration for the first implementations, then add that later on once they figure out what they really need, and start to see some process efficiencies. This lines up with their Agile philosophy of prototyping everything, and having frequent releases with incremental new functionality.

They have experienced huge efficiency gains due to their BPM implementation and other process efficiency efforts: 38% reduction in headcount in spite of a 6% increase in workload, time saved not doing manual gathering of user performance data, and process improvements in moving from email and Excel to BPM. The focus on change management and process management early on were important for their success. He also recommended collecting the reporting requirements up front to ensure that the necessary data are being collected by the process. Good points, and a nice success story for cloud BPM.

I had a chance for an advance briefing of Appian’s Tempo release last week; this is a new part of the Appian product suite that focuses on mobility, cloud and social aspects of BPM for social collaboration. This isn’t a standalone social collaboration platform, but includes deep links into the Appian BPM platform through events, alerts, tasks and more. They’ve included Twitter-like status updates and RSS feeds so that you can publish and consume the information in a variety of other forms, offering a fresh new alternative to the usual sort of process monitoring that we see in a BPMS. The free app for the iPhone and iPad requires an account on Appian Forum (the Appian user community site) or access to an Appian BPM installation (not sure if this is both an on-premise system and the cloud-based offering) in order to do anything so I wasn’t really able to check it out, but saw it on an emulator in the demo.

Their goal for Tempo is to provide a zero-training interface that allows people to track and participate in processes either from a browser or from a mobile device. You can think of it as a new user interface for their BPM and information management systems: in some cases to provide an alternative view to the portal for occasional/less-skilled users, and in some cases as the only interface for more collaborative or monitoring applications. It doesn’t replace the information- and feature-rich portal interface used by heads-down workers, but provides a simpler view for interacting with processes by executives or mobile workers. Users can interact with tasks that are assigned to them in a structured process within BPM, such as approving a purchase request, but can also create a new case from any event, whether that original event was related to BPM or not. For example, I had a view of the internal Appian instance of Tempo (I’ve redacted everything from this screenshot except this event, since some of the other events included internal information) where a “Marketing” stream included RSS feeds from a number of BPM news and blog sources. Swiping on any given event in the iPhone/iPad app – say an external blog post – allowed a new case to be opened that linked to that external event. At this point, the case opening functionality is pretty rudimentary, only allowing for a single process type to be created, which would then be manually reassigned to a specific person or subprocess, you can see the future value of this when the case/process type can be selected.

As this scenario highlights, Tempo can include process and information from a variety of other sources, internal and external, that may have nothing to do with Appian BPM, in addition to providing visibility into core business processes. Anything with an RSS feed can be added; I saw Salesforce.com notifications, although not sure if they were just from an RSS feed or if there is some sort of more direct integration. Considering the wide adoption of RSS as a publication method for events, this is likely not an issue, but there are also some more direct system connections: an SAP event appearing in Tempo can be expanded to retrieve data directly from the corresponding SAP item, such as invoice details. This turns Tempo into a sort of generalized business dashboard for monitoring tasks and events from many different business sources: collaboration within a business information context.

The browser interface will be familiar if you’ve ever used Facebook: it has a big panel in the center for events, with the event filters in a panel on the left, and the event actions in a panel on the right. Users can subscribe to specific event types, which automatically creates filters, or can explicitly filter by logical business groupings such as departments. Individual events can be “starred” for easy retrieval, as you would with messages in Gmail. The user’s BPM inbox is exposed in the filter panel as “My Tasks”, so that their interaction with structured business processes is seen in the same context as other information and events with which they interact. The action panel on the right allows for the user to initiate new processes, depending on their role; this is more comprehensive than the “Open a case” functionality that we saw on the iPad: this is a full BPM process instantiation based on a user input form, such as creating a new IT change request. The actions available to a user are based on their role and permissions.

Access to certain event classes can be restricted based on user and role permissions, but a user can comment on any event that they can see in their event stream. This form of collaboration is very similar to the Facebook model: you comment on someone an item that someone else posts, then are notified when anyone else adds a comment to the same event.

There’s been some nice optimization for the iPhone and iPad apps, such as one-click approvals without having to open the item, and rendering of Appian forms natively in the app. Although I’ve seen many iPad demos in the past year – it seems impossible to visit a vendor or go to a conference without seeing at least one – this offers significant value because of the deep integration to business processes and information. It’s easy to envision a mobile worker, for example, using the app to update information while at their client site, rather than filling out paper forms that need to be transcribed later. The app can directly access documents from the Appian content management system, or link to anything in a browser via a URL. It also allows for multiple user logins from the same device, which makes it good for testing but also useful in cases where a mobile device might be passed from worker to worker, such as for healthcare workers where a single device would support rotating shifts of users.

This certainly isn’t the first mobile app for BPM – you can see a few more listed at David Moser’s blog post on process apps – and the expected demand for mobile BPM will continue to drive more into this marketplace. This is, however, a very competent offering by a mainstream BPM vendor, which helps to validate the mobile BPM market in general.

This also isn’t the first BPM vendor to come out with a social media-style collaborative event stream interface (for lack of a better term), but this is a good indication of what we can expect to see as standard BPM functionality in the future.

Back in May, I wrote about a cool new cloud-based service called RAVEN Cloud, which translated natural language text into process maps. As I wrote then:

You start out either with one of the standard text examples or by entering your own text to describe the process; you can use some basic text formatting to help clarify, such as lists, indenting and fonts. Then, you click the big red button, wait a few seconds, and voilà: you have a process map. Seriously.

They’re releasing RAVEN Cloud for general availability today (the beta sticker is still on the site as of the time of this writing), and I had an update demo with Dave Ruiz a couple of days ago. There are two major updates: UI enhancements, particularly the Business Process Explorer for process organization and categorization, and exporting to something other than JPEG.

The Business Process Explorer, in the left sidebar, looks like a set of folders containing processes although the “folders” are actually categories/tags, like in Google Docs: a process can be in more than one of these folders simultaneously if it relates to multiple categories, and the categories become metadata on the processes “contained” within them. This become more obvious when you look at the attributes for a process, where the Process Category drop-down list allows multiple selections. There is context menu support in the explorer to take actions on a selected process (open, rename, delete, move, save as), and the Process Explorer can be collapsed to provide more screen real estate for the process itself.

The Process Explorer contains a few standard categories, including process examples and tutorials; there is a separate administration panel for managing the process examples, which can then be used by any user as templates for creating  a new process. The tutorials highlight topics such as writing nested conditionals, and can be used in conjunction with the writing guide and their YouTube videos. I liked this one on correcting errors; I saw a bit of this in the demo when Dave accidentally misspelled a role name, resulting in an unwanted break in the flow, and didn’t specify the “else” clause of an “if” statement, resulting in an incomplete conditional:

Another feature that I saw in this version, which also brings them closer to BPMN compliance, is the inclusion of explicit start and end nodes in a process model. There can be multiple end nodes, but not multiple start nodes.

In addition to exporting as a JPEG image – useful for documentation but not for importing to another tool for analysis or execution – RAVEN Cloud now supports export to Visio or a choice of three XML formats: XMI 2.1, XPDL 2.0 and XPDL 2.1. The process model imported to Visio looked great, and the metadata at the process and shape level were preserved. Importing the XPDL into the BizAgi Process Modeler didn’t create quite as pretty a result: the process model was topographically correct, but the formatting needed some manual cleanup. In either case, this demonstrates the ability to have a business analyst without process modeling skills create a first version of a model, which can then be imported into another tool for further analysis and/or execution.

This still creates only simple process models: it supports unadorned activities, simple start and end events, sequence flows, OR gateways and swimlanes. It also isn’t BPMN compliant, although it’s close. They’re planning to add link events (off-page connectors) and AND gateways, although it’s not clear what natural language constructs would support those, and they may have to use keywords instead, which weakens the natural language argument.

There will still be a free version, which does not support user-created categories or Visio/XPDL exports, and the paid version will be available for subscription for $25/user/month with volume discounts plus a 10% discount for an annual versus monthly subscription. An account can be either single-user or multi-user; by default, all models created within an account are visible for read-only access to all other users in that account, although access can be restricted further if required. A future version will include process model versioning and more access control options, since you can’t really have multi-user editing of a single process model unless you’re keeping some past versions. I think there’s also an opportunity for hybrid pricing similar to Blueworks Live, where a lower-cost user could have read-only permissions on models that were created by others, possibly with some commenting capabilities for feedback. It’s all self-provisioned: you just set up your account, enter your credit card details if you’re going for the paid version, and add users by their name and email address; they’ll receive an email invitation to create their account login and profile. I didn’t ask if one RAVEN Cloud login/profile can be shared across multiple accounts; that would be interesting for people like me, who work with multiple organizations on their process models, and I’ve seen something like this in Freshbooks, an online time tracking and invoicing applications, so that Freshbooks customers can easily interact since a single login (authentication) can have access to multiple accounts (authorization).

They’re also working on hosting RAVEN Cloud in a private cloud environment, so keep watching for that.

My verdict: still cool, but they need to re-work their subscription model a bit, and bring their notation in line with BPMN. They also have some challenges ahead in defining the language for new element types, but I’m sure that they’re up to it.

Kristof Kloeckner, IBM’s VP of Strategy & Enterprise Initiatives System and Software, & CTO of Cloud Computing, delivered today’s keynote on the theme of a smarter planet and IBM’s cloud computing strategy. Considering that this is the third IBM conference that I’ve been to in six months (Impact, IOD and now CASCON), there’s not a lot new here: people + process + information = smarter enterprise; increasing agility; connecting and empowering people; turning information into insights; driving effectiveness and efficiency; blah, blah, blah.

I found it particularly interesting that the person in charge of IBM’s cloud computing strategy would make a comment from the stage that he could see audience members “surreptitiously using their iPads”, as if those of us using an internet-connected device during his talk were not paying attention or connecting with his material. In actual fact, some of us (like me) are taking notes and blogging on his talk, tweeting about it, looking up references that he makes, and other functions that are more relevant to his presentation than he understands.

I like the slide that he had on the hype versus enterprise reality of IT trends, such as how the consumerization of IT hype is manifesting in industrialization of IT, or how the Big Switch is becoming reality through multiple deployment choices ranging from fully on-premise to fully virtualized public cloud infrastructure. I did have to laugh, however, when he showed a range of deployment models where he labeled the on-premise enterprise data center as a “private cloud”, as well as enterprise data centers that are on-premise but operated by a 3rd party, and enterprise infrastructure that is hosted and operated by a 3rd party for an organization’s exclusive use. It’s only when he gets into shared and public cloud services that he reaches what many of us consider to be “cloud”: the rest is just virtualization and/or managed hosting services where the customer organization still pays for the entire infrastructure.

It’s inevitable that larger (or more paranoid) organizations will continue to have on-premise systems, and might combine them with cloud infrastructure in a hybrid cloud model; there’s a need to have systems management that spans across these hybrid environments, and open standards are starting to emerge for cloud-to-enterprise communication and control.

Kloeckner feels that one of the first major multi-tenanted platforms to emerge(presumably amongst their large enterprise customers) will be databases; although it seems somewhat counterintuitive that organizations nervous about the security and privacy of shared services would use them for their data storage, in retrospect, he’s probably talking about multi-tenanted on-premise or private hosted systems, where the multiple tenants are parts of the same organization. I do agree with his concept of using cloud for development and test environments – I’m seeing this as a popular solution – but believe that the public cloud infrastructure will have the biggest impact in the near term on small and medium businesses by driving down their IT costs, and in cross-organization collaborative applications.

I’m done with CASCON 2010; none of the afternoon workshops piqued my interest, and tomorrow I’m presenting at a seminar hosted by Pegasystems in downtown Toronto. As always, CASCON has been a great conference on software research of all types.

Bruce SPicer of Keystar Consultancy presented on a project that he did with Surrenda-link Investment Management to implement Appian cloud-based BPM for the process around procuring US life settlement assets (individual life insurance policies) to become part of their investment funds. They were specifically looking at a software as a service offering for this, in order to reduce cost and risk (considering the small size of their IT group), since SaaS allows them to scale up and down seamlessly without increasing costs significantly. They’ve built their own portal/user interface, using Appian Anywhere as the underlying process and analytics engine; it surprises me a bit that they’re not using more out of the box UI.

They were overtime and over budget, mostly because they (admittedly) screwed up the process mapping due to immature processes, inexperience with process analysis, and inexperience with gathering requirements versus just documenting the as-is state. Even worse, someone senior signed off on these incorrect process models, which were then used for initial development in the proof of concept before corrections were made. They made some methodology corrections after that, improving their process analysis by looking at broad processes before doing a detailed view of a functional silo, and moving to Agile development methodologies. Even with the mistakes that were made, they’re in production and on track to achieve their three-year ROI.

This should be a compelling case study, but maybe because it was just after lunch, or maybe because his presentation averaged 120+ words per slide, I had a hard time getting into this.

We started after the break with Jeremy Westerman, head of BPM product marketing for TIBCO, presenting on AMX BPM. The crowd is a bit slow returning, which I suspect is due more to the availability of Wii Hockey down the hall than to the subject matter. Most telling, Westerman has the longest timeslot of the day, 45 minutes, which shows the importance that TIBCO is placing on marketing efforts for this new generation of their BPM platform. As I mentioned earlier, I’ve had 3+ hours of briefing on AMX BPM recently and think that they’ve done a good job of rearchitecting – not just refactoring – their BPM product to a modern architecture that puts them in a good competitive position, assuming that they can get the customer adoption. He started by talking about managing business processes as strategic assets, and the basics of what it means to move processes into a BPMS, then moved on to the TIBCO BPM products: Business Studio for modeling, the on-premise AMX BPM process execution environment, and the cloud-based Silver BPM process execution environment. This built well on their earlier messages about integration and SOA, since many business processes – especially for the financial-biased audience here today – are dependent on integrating data and messaging with other enterprise systems. Business-friendly is definitely important for any BPM system, but the processes also have to be able to punch at enterprise weight.

His explanation of work management also covered optimizing people within the process: maximizing utilization while still meeting business commitments through intelligent routing, unified work lists and process/work management visibility. A BPM system allows a geographically distributed group of resources to be treated as single pool for dynamic tunable work management, so that the actual organizational model can be used rather than an artificial model imposed by location or other factors. This led into a discussion of workflow patterns, such as separation of duties, which they are starting to build into AMX BPM as I noted in my recent review. He walked through other functionality such as UI creation, analytics and event processing; although I’ve seen most of this before, it was almost certainly new to everyone except the few people in the room who had attended TUCON back in May. The BPM booth was also the busiest one during the break, indicating a strong audience interest; I’m sure that most BPM vendors are seeing this same level of interest as organizations still recovering from the recession look to optimize their processes to cut costs and provide competitive advantage.

Ivan Casanova, director of cloud marketing for TIBCO, started with some pretty simple Cloud 101 stuff, then outlined their Silver line of cloud platforms: Silver CAP for developing cloud services, Silver Fabric for migrating existing applications, Silver BPM for process management, and Silver Spotfire for analytics. Some portion of the IT-heavy audience was probably thinking “not in my data centre, dude!”, but eventually every organization is going to have to think about what a cloud platform brings in terms of speed of deployment, scalability, cost and ability to collaborate outside the enterprise. Although he did talk about using Fabric for “private cloud” deployments that leverage cloud utility computing principles for on-premise systems, he didn’t mention the most likely baby step for organizations who are nervous about putting production data in the cloud, which is to use the cloud for development and testing, then deploy on premise. He finished with a valid point about how they have a lot of trust from their customers, and how they’ve built cloud services that suit their enterprise customers’ privacy needs; IBM uses much the same argument about why you want to use an large, established, trusted vendor for your cloud requirements rather than some young upstart.

We then heard from Greg Shevchik, a TIBCO MDM specialist, for a quick review of the discipline of master data management and TIBCO’s Collaborative Information Manager (CIM). CIM manages the master data repositories shared by multiple enterprise systems, and allows other systems – such as AMX BPM – to use data from that single source. It includes a central data repository; governance tools for validation and de-duplication; workflow for managing the data repository; synchronization of data between systems; and reporting on MDM.

Last up for the Toronto TIBCO Now was Al Harrington (who was the mystery man who opened the day), giving us a quick view of the new generation of TIBCO’s CEP product, BusinessEvents. There’s a lot to see here, and I probably need to get a real briefing to do it justice; events are at the heart of so many business processes that CEP and BPM are becoming ever more intertwined.

My battery just hit 7% and we’re after 5pm, so I’ll wrap up here. The TIBCO Now roadshow provides a good overview of their updated technology portfolio and the benefits for customers; check for one coming your way.

Back in May at TIBCO’s TUCON conference, I had a chance for a briefing on their new ActiveMatrix BPM, although not a full demo. Since then, however, I’ve had about three hours of demo sessions with Roger King to see more of what they’re doing with ActiveMatrix (which I will refer to as AMX) BPM and Silver BPM, the cloud version. King has been at the core of TIBCO’s BPM products for a long time: he was at Staffware long before their acquisition by TIBCO, and now heads up BPM product strategy and product management, so is in a position to not only give a thorough demo, but to provide context for where they’ve come from and where they’re going in BPM. He also does a mean Austin Powers imitation, which is kind of funny when you consider that he’s a Brit imitating a Canadian who is imitating a Brit, all for the amusement of a Canadian.

AMX BPM Architecture

The AMX BPM platform has five components – Workspace, Work Manager, Event Collector, Process Manager, and Openspace – wired together using public services, which allows anyone to use those same services to replace or augment the components. In fact, the BPM platform and AMX BPM applications are all just AMX composite applications.

There is a common administration interface for all AMX products, and when we’re in the AMX Admin interface, the common nature of the platform becomes more evident: you can see the AMX platform, AMX BPM, sample applications and organizational models that we’ll be looking at in the demo. Selecting the AMX BPM application, you’ll see the underlying five components as well as all public services available.

The AMX service governance and management tools share common components with AMX SOA, but are not included in AMX BPM.

What this new architecture means is that Business Studio can be used to create services using combinations of components, services and other object types, which allows processes to be embedded as part of any application: process is not a separate thing used to orchestrate applications, but part of the application itself.

Organizational Models

Business Studio, as the design/modeling environment for composite model-driven process applications, now supports five model types: process, page flow, organizational, data and form. Organizational models are considered a key starting point: an organizational model becomes a functional overlay on your corporate LDAP, and can include multiple organizations such as trading partners in order to define organizational scope during runtime. Metadata is defined on a per-organization level, which can be used to partition visible information if people from multiple organizations participate in a process at runtime.

Organizational models allow you to define Positions, which represent roles that can be used for work assignment in a process, e.g., all CSRs or all Life CSRs in the insurance claims scenario that’s provided as their sample organizational model. A position can include capabilities such as language skills, and privileges such as ability to authorize claims, scoped to an organization and assigned at an instance level to a process model. You can also define groups, e.g., Managers, and assign capabilities and privileges with groups, as well as access to system actions. Resources in the organization model can be mapped to LDAP attributes at runtime using queries. Later, during process modeling, positions are used to assign participants to a step in a process model, and to perform capability-based routing.

On the end-user side, the model appears in the Organization Browser within the ActiveMatrix Workspace if the user has permissions to see this: an admin, for example, might be able to see all of the organization models for multiple organizations, see the participants that are part of any position, and view or create the queries that bind the resources from the underlying LDAP. A regular user may see only their own organization’s structure, without the ability to modify the participants in a position or any other resources.

Business Object (Data) Models

TIBCO isn’t the first BPM vendor to jump on the process data model bandwagon, and they won’t be the last: the link between MDM and process instance data needs to be firmly established so that you don’t end up with data definitions within your BPMS that don’t match up with the other data sources in your organization.

In Business Studio, you can define a business object model – a sort of simplified UML data model – that allows structured data models to be shared within or across packages and applications The business object model can have subclasses as well as data relationships, and supports a hierarchy of data models. Even better, you can import it from UML, XSD, WSDL or directly from a database, and can export it to XSD or WSDL.

A business object model is then attached via an external reference to a process model to become part of the process instance data. Access control (in/out/bidirectional) can be set for that business object at each step in the process, as with any other instance parameter, in order to control whether the process step can read and write to the instance data defined by the business object. If you create scripts (which are Javascript in AMX BPM) to manipulate the instance data, the script editor will look up attributes and type directly from the business object model, which means that it’s fully-accessible instance data, not a blob that you need to decode manually.

Page Flow Models

One optimization that BPM vendors are starting to learn from application development environment products is the idea of screen flow: a stateless mini process model that runs in the web tier to handle a series of screens/pages for a single user in a monolithic operation. Think of it as the underpinnings for a wizard-type interface, where the user is stepped through multiple pages of information and data entry in order to complete a single task in a process. This is similar to the capabilities emerging in web application platforms such as NetSuite’s SuiteFlow, Salesforce’s Visual Process Manager and the process modeling within OutSystems, except in this case, the page flow is what happens at a single step in a larger process model. It contains a lot of the functionality of a process model, including gateways, conditional logic, data access, service calls, etc., but when the page flow completes, it returns to process engine to be sent on to the next step in the orchestration.

Page flows are another important efficiency improvement in BPM systems architecture; otherwise, you end up doing a lot of dequeuing and enqueuing of work items to and from the process engine, which has a lot of overhead. Some systems have had ways to kludge this for many years by binding together several sequential steps in the process model, such that the work item was dequeued at the beginning of the first step in the group, then not enqueued again until the end of the last step. In the case of TIBCO AMX BPM, the page flow looks almost like a subprocess call, with the monolithic step shown in the process map, then the steps in the page flow shown in a limited version of what looks like the process modeler. The alternative to using something like a page flow in any BPM environment is to build that capability into the user interface for that process step, which tends to make the UI very complex, and embeds far too much of the business logic (including routing logic) into the UI rather than in the process and data flow models.

An interesting use case that we discussed was to create a page flow for the start node of a process and expose that as a business service: since the start node page flow can include steps, forms and data access/validation before instantiating the (stateful) process, it provides an environment for initial data gathering at the start of a process without having to use some other web form to gather the data necessary to kick off a process instance. This can then be published as a business service to call from other appslications and environments, wherever you want the process to be invoked.

User Interface Form Models

User interface forms can be assigned to each step in a process model, and will be auto-generated for the specific task based on the process instance data, which could be a business object model. The default form that is generated – and will be rendered in General Interface to preview the user interface at that step – isn’t that pretty, but it’s fairly straightforward to then pretty it up in the forms editor, adding master-detail structures and different control types, and associating objects with CSS classes for styling.

Although you may not end up using these auto-generated forms without some amount of customization, they remain invaluable for rapid prototyping of processes.

It’s also possible (although only feasible in simple cases) to specify no form URL attached to a process or page flow step, which will generate a form at runtime based on the process instance data exposed at that step.

Process Patterns in Process Models

We didn’t spend much time on the Business Studio process modeler, since there is a lot of common functionality with most other process modelers (although Roger will cringe to hear me say that), but just focused on a few new features.

One of these features is a set of predefined process patterns, similar in functionality to what you might do with business rules, but more visually associated with the process model. We looked at two of these: chaining, where subsequent human tasks for same item are worked by same person if they are capable, but can be assigned to others if the first participant can’t perform all of the tasks; and separation of duties, where subsequent tasks (e.g., work and QA) must be done by different people.

At the time that I saw the demo, chained events were visualized as embedded subprocesses, and patterns couldn’t be combined; these need to be fixed in order to make the patterns more visually accurate as well as more functional.

ActiveMatrix Workspace

The ActiveMatrix Workspace is the web-based end-user environment, with views of the work available to the current user as well as views of the organizational model as discussed above. Taking a look at the work views, however, is where we see some of the first major changes to the underlying process engine: there are no more physical queues; rather, what appear as logical queues (or work lists) to users are created with dynamic queries. In the old days – and in the old architecture of iProcess and some of its competitors – segregating work items into explicit queues was necessary for performance reasons; today, faster hardware and in-memory software allows this to be done dynamically. When I think of all the times that I’ve have to work around queuing mechanisms in BPM systems, both for work assignment and reporting purposes, this is huge since it provides much easier slicing and dicing of the work items. For example, a standard work view for a supervisor would be “Supervised Work”, which shows all work items that are both available but not yet assigned to a user, plus those that have been allocated to specific users: in effect, a view of the unassigned and assigned work in progress, regardless of its logical queue location. Now, you could argue that queues in most BPMS are usually implemented as database constructs anyway, which is mostly true; however, access to work except by queue is often not very efficient in these queue-oriented systems.

In addition to the Work Views and Organization Browser, the Workspace includes the following user views:

• Business Services, which is a user view of process-based composite applications, so that user doesn’t think about kicking off a process, but about doing a business function. In the sample insurance scenario that we were looking at, this included functions such as “First Notice of Loss Notification”.
• Process Views, which shows a view of the executing processes.
• Event Views, which shows an administrator-like view of login/logout and other system events, plus events related to process instances such as access and update. This is tied closely to the Process Views, which links back to the Event Views in order to show instance events.

We walked through the invocation of a business service in the Business Services view, which essentially kicked off a process instance. However, the first two steps of the “process” were actually screens in a page flow: the simple data entry screens were auto-generated, and the process wasn’t actually instantiated until after the Submit button was clicked on the second screen. We followed the process through the remainder of the steps, which showed off most of the capabilities that we had seen in Business Studio during design:

• Looking at the Work View, the new claims show up in an OFFERED state (meaning that they aren’t yet claimed for work by a participant) in all CSRs’ work lists. Each user’s view of the work list depends on their group membership, so it completely dynamic.
• Data is entered by the CSR for the work item to record the claim; the UI in this case was generated from the business object, including multi-entry field types such as multiple claimants for one claim.
• A language filter based on capability is used to assign work for the Customer Interview step; this step also used a 2-screen page flow to gather information while on the phone with the customer.
• The work item is processed through the Assess Claim step, and that work item is excluded from same user for the Pay Claim task using the Separation of Duties process pattern.

TIBCO Silver and Silver BPM

TIBCO’s big splash at the TUCON conference this year was around Silver, their cloud environment based on Amazon EC2, and specifically Silver BPM, their cloud-based BPM offering. This is not a cloud-based process discovery or modeling tool; this is the actual processes executing in the cloud.

Setting Silver BPM apart from some of the other cloud BPM environments is the shared design tool, Business Studio, that can connect simultaneously to both on-premise AMX environments and Silver environments. The same models, applications and other services built in Business Studio can be deployed to either environment without having to export and import them into another design tool. This immediately suggests an obvious use case of Silver for development and testing of process applications, even if they are going to be deployed on premise, since each member of the development and test teams could have their own virtual AMX servers, if desired. Although there is no guarantee of the ActiveMatrix and Silver BPM platforms providing exactly the same functionality at any point in time, I would guess that TIBCO will have to keep them fairly closely aligned in order to support this dev/test use case. IBM uses exactly this strategy for marketing their Smart Business Dev/Test environments (although not for any of their BPM products), and the potential cost reductions for these non-production environments is starting to resonate with enterprise clients, not just smaller businesses. That’s not to say that you can’t use Silver BPM for production environments, just that I am imagining that many larger enterprises won’t yet trust the cloud for core production capabilities.

Although EC2 is underneath Silver, the customer only needs to set up their Amazon EC2 account, enable the EC2, S3 and RDS services, and pay their Amazon bill at the end of the month: all other provisioning and administration is done through the Silver Center console. Silver customers are on a one-year or three-year subscription license with TIBCO (as usual, I forgot to ask about pricing) that is an additional cost above the Amazon services.

From within Silver Center, you can create a new Silver BPM instance using the following steps:

1. Name the instance, then select the product group (e.g., Silver BPM) and software version.
2. Enter the name and passwords for the database administrator, user and sample user accounts.
3. Specify the underlying EC2 properties: size/memory configuration, geographic zone, some security parameters and whether to use an elastic (fixed) IP address.

This will start a new EC2 instance that combines a basic Linux image with the selected Silver BPM software image, which takes about 20 minutes to provision and boot the virtual server.

Once up and running, the Silver Administrator and Silver Workspace look amazingly similar to their on-premise ActiveMatrix counterparts: if I didn’t look at the top banner or the URL, I would not have been able to distinguish them. Even from within Business Studio, once you set up the connection parameters, a Silver BPM server looks pretty much like an AMX server, and you can connect to multiple of each type from Business Studio. Then, you deploy models to any of the servers by dragging and dropping them onto the server.

There are some obvious holes in the current Silver BPM offerings. First, they have cloud-based execution but no cloud-based discovery or modeling: Business Studio is an Eclipse-based desktop application, with models stored in a local workspace and shared primarily through the use of a Subversion repository (more commonly used by developers for source code control). This is not process discovery/modeling for the masses, and doesn’t contain the collaborative features such as interactive sharing of process models that we see emerging in many other tools.

Second, there is no multi-tenancy, but only a 1:1 relationship between Silver BPM and EC2: an EC2 instance can contain only one Silver BPM instance, and a Silver BPM instance cannot span multiple EC2 instances. In the future, they will allow for 1:many and many:1, but for now, this is really more like dedicated cloud resources rather than multi-tenancy hosting.

Third, there is an LDAP server created within the Silver BPM instance, but you need to manually upload your enterprise LDAP and use appropriate tools in order to keep it in sync. You can, however, use your corporate LDAP if you use Amazon Virtual Private Cloud.

TIBCO’s not the first to provide process execution in the cloud – Appian, Fujitsu, Intalio, Cordys and others have come before them – but having a big BPM name like TIBCO enter the cloud BPM market helps to further validate the use of the cloud for process execution. In fact, I’ve had a discussion with one of the more established cloud BPM vendors since my briefing with TIBCO, and I advised them not to think of TIBCO only as a competitor, but as a collaborator helping to expand this market. I don’t imagine that we’ll see many Silver customers before next year’s TUCON, but TIBCO will be out there building the cloud mindshare.

openspace

We finished our marathon demo sessions with a look at AMX openspace, TIBCO’s gadget-based container application for building composite user interfaces. In short: a mashup builder.

The standard AMX/Silver Workspace is fine for some users, while others need a highly-customized UI created by developers for advanced functionality; there is, however, a third category that prefer to roll their own user experience. I first blogged about the potential for enterprise mashups in early 2006, and attended the first MashupCamp in 2006 (and the second one in 2007), so it’s no surprise that I’ve been impatiently tapping my foot waiting for many of the larger BPM/enterprise software vendors to catch up with this trend. True, the customers aren’t there yet, but they’re not going to get there on their own, they need to see mashup/portal builders in action to recognize the value that these tools can bring to experienced users and administrators.

The basics: openspace supports a number of different gadget standards, including Shindig, OpenSocial, iGoogle and GWT, and uses a publish-subscribe model for inter-gadget communication. Gadgets are arranged on tabs/pages within the openspace application, and can be added from the predefined palette of gadgets, from the gadget store hosted on the server, or by specifying the URL and any required parameters for external gadgets such as Google Maps. Each gadget can be themed independently, although that just seems like a recipe for a graphic design nightmare. Once created, openspace pages can be published for other users to use.

This ties back into AMX BPM with the availability of BPM gadgets: process participation gadgets such as Business Services and ToDo List, basic monitoring and reporting gadgets for end users and supervisors, and Spotfire gadgets for more advanced analytics and visualization. You have to be licensed for all the underlying software such as AMX BPM and Spotfire; this just provides a new way to visualize and interact with those tools.

Summary

If you read my previous post on AMX BPM, you already know that I was impressed with what I saw. There are a number of BPM products launched around the same time as Staffware’s product (which became TIBCO’s iProcess) in the early 2000’s, and although most of these need a complete architectural overhaul to bring them up to date, this is the first that I’ve seen of a vendor biting the bullet: AMX BPM is not an iProcess upgrade; it’s a completely new next-generation BPM product. Although they gain immensely in product capabilities, they are leaving behind their existing iProcess customers, although I’m sure that they’ll offer some sort of migration assistance. There are unlikely to be many enhancements to iProcess in the future, but there are no plans for end of life, and it is expected that many current customers will keep their legacy applications on iProcess while looking to AMX BPM for new applications and functionality.

With AMX and Silver, TIBCO have leapfrogged many of their newer competitors in terms of architecture and functionality. Their challenges lie in shifting their existing customers’ new development onto AMX or Silver, and convincing potential customers that they’re no longer part of the BPM old guard.