Apparently the replay information for the BPM-BRM webinar that I did on April 24th went out last week to all the registrants, but not to me; you can replay it here. I’ve uploaded the slides to SlideShare and embedded them below, but as one person told me, I tend to put up a few words on a slide then talk at about 300 words per minute, so probably better to listen to the replay.
It was a busy week last week at TUCON and I completely forgot about the questions from the Business Rules Forum Q&A from the webinar that I did on the 24th. I’m not sure if the replay is available yet, I’ll post a link when I hear about it.
Here’s my answers to the questions that came up during the presentation, although I responded to some of these at the end. Where the question wasn’t clear, I took a stab at an interpretation; if I missed the point, please add a comment to this post with your clarification and I’ll follow up.
Explain the relationship between business models and BPM.
Not sure of the exact intent, but I think that this is asking about the relationship between business (process?) modeling and BPM. Business models of various sorts, including business process models, are often created by an organization to provide a high-level, business-oriented view of their operation. From an enterprise architecture standpoint, these are the models in the highest level of the architecture that may be created by, and are always understandable by, a non-technical business analyst. In the case of business process models, these are created to model the flow of a business process, usually in a flow-chart or swimlane type of diagram. In many cases, these are created in a standalone modeling tool — either a simple desktop application like Microsoft Visio, or a more comprehensive tool such as IDS Scheer’s ARIS — but may also be modeled directly in the process modeling environment of a BPM suite (BPMS). In this latter case, a process model can be directly translated to an executable process.
Is there any benefit to implementing a BRM without a BPM?
Yes, there are many cases of using a BRMS separately from BPMS: the rules/decisions may be accessed directly as part of a manual process, where a user enters in the required parameters and is given a decision back in return, or they may be called from other applications such as a CRM.
Please mention vendors or products by name, even if caveats apply.
and
Can you name products that support what has been presented?
and
What are the methods & technology tools used for BRM & BPM?
I can’t recall if we were talking about BPMS or BRMS vendors here, so I’ll try to cover both. To hit the major vendors, take a look at which ones are included in the reports by the big analysts. Gartner includes the following BPMS vendors in its Magic Quadrant for BPMS, published in December 2007: Adobe, Appian, Ascentn, AuraPortal, BEA, Captaris, EMC, Fujitsu, Global 360, IBM, Intalio, Lombardi, Metastorm, Microgen, Oracle, Pegasystems, Savvion, Singularity, Software AG, SunGard, TIBCO and Ultimus. Forrester splits up the market into four categories with several vendors in each, which I’ve listed in a previous post.
On the BRMS side, Forrester recently issued a report on BRMS vendors in which they evaluated CA, Corticon Technologies, Experian, Fair Isaac, Haley Limited, ILOG, Innovations Software Technology, InRule Technology, Intelligent Results, Pegasystems, and SAP.
There are other vendors of both types, but this covers the major players. Also notice that Pegasystems plays in both markets — and in fact is a leader in both — since its BPMS is based on a rules engine.
Who are some of the vendors with tight integration between BPM and BRM?
Pegasystems is the obvious starting point, since they use a rules engine as an underlying platform for their BPMS. Many BPMS vendors don’t want to talk about a tight integration with a third-party BRM since that implies a weakness in their own rules capabilities. All BPMS vendors, through their support for invoking web services, can integrate loosely with BRM.
In your opinion, have any BRMS suites achieved robust BPM capabilities?
Only Pegasystems, to my knowledge. It’s more likely that a BPMS will achieve BRM capabilities rather than the other way around, in my opinion.
How could you change a business rule and have it only affect new BPM processes and not in flight process instances?
There are two ways to do this. First, if all parameters that drive the rules are known at the beginning of the process, the process instance could invoke the rules immediately after it is created, and store the decision results until they are required; since the rules are executed at the time that the process instance is created, the instance will not be affected by any changes to rules while it is in progress. Second, a process can call a specific version of a rule, assuming that the BRMS supports rules versioning. That way, any process instances created from a specific version of a process definition can call a specific version of a rule, even if the rule has changed since then. Newer process definitions could be changed to call a later version of the rule.
You said that a BPMS will call a BRMS (typical scenarios). How would the BRMS know the scope of what needs to be checked? For example, if you have the rule "some applicant of each loan application must have a credit score of 600". When the business process for loan applications calls the BRMS, how does it determine the set of applicants that need to checked?
I think that the question is about where the BRMS gets its information that is used as parameters for the decisions. This would typically be passed to the BRMS from the calling application, in this case, the BPMS. The BPMS may need to make calls to other systems in order to get this information, then forward it to the BRMS: remember that part of the role of the BPMS is to orchestrate multiple systems and pass data along between them, including the BRMS.
Do you normally see that the same business users are maintaining both the processes and rules or are they normally different business users?
If you’re talking about the business analysts that would be designing the processes or rules, it is best if they are the same — so that they can decide what happens in a process versus what happens in a rule — but often are different people due to the training requirements. If these are separate roles, then the process analysts need to learn enough about rules to know when to request that a rule set be created for them to call from their processes.
How would you define the relationship between BPM, BRM and RBA (Run Book Automation)?
I’m not that familiar with RBA, but it is focused on IT and systems processes, not business processes. At TUCON last week, however, one of the presentations was on how they used TIBCO products for IT processes, although he didn’t refer to it as RBA.
Do you agree that BRM and BPM have to be married with the SME from the business side and the SME from the IT side to be successful?
I’m unclear on what this question means. "SME from the business side" means, to me, someone who is an expert on the business being performed; I’m not sure what "SME from the IT side" means. Both BRM and BPM are most successful when there is collaboration between business and IT: the business analysts doing the high-level modeling of the rules and processes to ensure that they meet the business requirements, and IT making sure that the technical underpinnings (such as calls to web services) are in place.
Do you have a list of feature set for BRM and BPM for product evaluation?
and
Could we get a list of recommended BPMS and Rules Management systems and why they are recommended?
Gartner and Forrester both publish comparative reports on BPMS and BRMS, I suggest that you start there.
What is the difference between BRM and procedures / governing document / policies? Give examples of BRM.
Policies, procedures and governing documents are the "rules" by which the business operates, but may not be automated in any way: many organizations just have people refer to a policies and procedures guide to tell them what to do in a manual process. BRM allows you to codify those policies and procedures so that they can be automated, and are executed the same way every time.
What about open source offerings? Have you worked or reviewed any of those?
Drools (from jboss) and NxBRE are two open source BRMS offerings. jBPM (also from jboss) and Intalio are both open source BPMS offerings. I recently did a review of Intalio but haven’t yet published what I saw; I haven’t worked with any of the other products. Many open source offerings don’t have the full functionality of their commercial counterparts so may not be included in the analysts’ comparative reports; the recent Gartner Magic Quadrant for BPMS is the first one in which Intalio has been included, for example.
Can you provide a simple example of how BPM and BRM are applied in practice?
A typical example that I’ve seen is in claims processing. There are many specific policies and procedures that must be followed to process a claim, and many BPM implementations just leave the decisions based on those policies to a person who is participating in the process: for example, give the work step to a person and have them decide the type of the claim and what region should be processing it. By adding BRM, these decisions can be automated based on data that is already known
Do you feel BPM can be used as a tool to integrate compliance management systems? e.g. OSH, Environment, Quality etc.
I’m not a compliance specialist, but I see many organizations using both BPM and BRM to help with their compliance efforts, since both can help to standardize processes and allow for easy auditing of what was actually done. As for integration with compliance management systems, that would depend on those systems and whether they provide a web services interface or other API.
What are some of the software packages you can purchase for extracting business rules?
The major BRMS products typically include tools for mining business rules from existing systems; you’ll need to check their functionality against the particular systems and platforms from which you want to extract rules to see if they’ll work for you.
How are the BRMS incorporated with any testing tools?
Many of the BRMS vendors have simulation and testing tools as part of their suite, specifically to test if the rules are complete and non-contradictory.
This fall, I’ll be back at the Business Rules Forum to make a presentation on business rules and BPM, but next week you can catch me online on a Business Rules Forum webinar speaking on the same subject:
Process improvement is a top priority for executives today, but business process management (BPM) alone doesn’t provide the whole answer. Although BPM does enable process improvement, it often doesn’t provide sufficient agility for today’s business processes.
To build for change, it is necessary to integrate business rules with BPM. This integration allows you to manage the decisions within your business processes, and easily modify those rules without recoding or changing the business processes.
In this webinar, you’ll learn about the business process management lifecycle, and how business rules can be integrated within it to greatly improve process agility. It also discusses how you can apply business rules consistently across multiple business processes and other applications.
Regular readers know that I’m a big fan of mixing business rules and BPM for maximum agility in your processes, and this webinar is an introduction to why you would want to do that, and how it works.
I’m doing a webinar tomorrow with David Straus of Corticon and Jim Sinur of Global 360 called Change the Rules without Changing your Business Processes: Using Business Rules Management to Turbo-Charge your BPM Solution. Tune in and hear how I sound after my run-in with the flu that ate Las Vegas.
Marc Kerremans’ presentation on business rules management started out looking at where rules exist in an organization, and how they are used. In many cases, rules are still embedded within applications — such that changing a rule requires changing the underlying code — or are implemented in a manual and sometimes ad hoc manner. Gartner defines business rules as “implicit and explicit business policies that define and describe a business action”, where implicit rules are those embedded within applications.
He discussed how rules can differ greatly within a single organization based on factors such as geography and the associated local regulations, and how a business rules management system — which manages rules explicitly and externally from business applications — can add value in managing all of the complex rules across an organization.
He also looked at who owns and manages business rules:
Since 2004, BRE and BPM have evolved from distinct and separate markets to an integration of BRE in many BPMS, to the current state of business rules management systems (BRMS) that go beyond simple business rules engine functionality. Typically, a BRMS contains:
As with BPMS, not all vendors will cover all of the full range of functionality equally well. There are some open source BREs that provide only the engine functionality, such as JBOSS and NxBRE. Most of the commercial vendors, such as Corticon, Fair Isaac and ILOG, are either migrating to the full BRMS functionality or are already there. There’s also an overlap of BPMS that provide BRE functionality: Pegasystems is the most commonly-cited player here since their BPMS is actually built on a rules platform, but other BPMS vendors provide rules that can be separated from the processes to at least allow reuse across multiple processes, although not across non-BPMS applications.
Agility is the primary reason that organizations look to BRM, which can manifest as awareness (accessing and presenting the right information through rules-based event monitoring), flexibility (rule modeling and simulation to handle expected change), adaptability (rapid rule modification to handle unexpected change) and productivity (executing the right policies and procedures).
There are other reasons for implementing BRM besides agility, of course: improving the quality and consistency of business decisions, improving revenue opportunities by fine-tuning pricing rules on the fly, improving customer satisfaction through greater customization, and better regulatory compliance and governance through the use of audited rules and increased visibility into how decisions are made.
Kerremans went through a number of best practices for getting started with BRM, such as analyzing rule volatility, and establishing a process for making changes to rules. As much as possible, try to work with natural language representations of rules so that business managers are comfortable with the authoring environment, although some level of structure to the language (”rules speak”) will be necessary.
He also discussed the different types of rules technology, including inference-based and event-based rules engines, before finishing up with some recommendations on developing a business rules management strategy. As with BPMS, many larger organizations will end up with multiple BRE tools to cover their entire strategy, so don’t assume that you’ll be working with only a single vendor in this space.
For you business rules aficionados, note that I’ve change the business rules category on this blog to BRM, instead of BRE.
I finished the day listening to Michael zur Muehlen discuss business processes and rules, a topic that I spoke about a few weeks ago at the Business Rules Forum. Michael, who I know from the BPM Think Tanks, is responsible for BPM courses at Stevens Institute of Technology. You can see his presentation slides online here.
He started out with the bottom line on why you want to integrate process and rules:
Who wouldn’t want this? However, he points out that users don’t like processes, since they find them abstract (or possibly requiring a more analytic view of the organization) and restrictive (that is, not able to capture all the actual business cases). He also points out the obvious problem with Eclipse-based process modelling tools: they’re not friendly to business types. Became of that, we end up with technical people maintaining business processes, which usually results in a lot of code and the next generation of legacy systems.
He went though an example of an insurance company with 12 process steps and 5000 business rules, and it became obvious why rules change faster than processes. He highlighted three places where rules and processes come together: control flow, work assignment, and cross-process policy enforcement. I still think that the key issue is the boundary: when is something done as a decision tree in a rules system, and when it is done as control flow directly in the BPMS. Michael suggests that you might want to first model the rules in the BPMS, then extract the rules, although I don’t think that the rules experts would consider that a best practice. The challenge, then, comes with the modelling that’s done by the business analysts: how much do they need to know about rules, and what does their modelling environment need to look like in order to support that?
He had some good suggestions about mining rule criteria from previously executed processes, determining what the automated rules should be based on prior manual processes. From an insurance standpoint, this can result in auto-underwriting on standard cases.
He talked about the links between process management, business rules and compliance: whereas BPMS can enforce process compliance, rules are used to enforce contextual compliance for all the things around the business processes that aren’t really part of process compliance.
Michael and a colleague did a fascinating study of which BPMN symbols are actually used, and found that there’s 6 or 7 symbols that are used in most of the diagrams — the rest are strictly long-tail usage. See page 39 of the slide deck that I link to above for the chart.
He had some practical advice on how business rules and business processes interact:
He also had a chart for classifying change, and showing where it made more sense to use business rules or business process for a particular decision/activity; for example, use rules if it’s rapidly changing, company-wide and less predictable.
My flight home leaves tomorrow mid-day, so this is likely the end of my IIR/Shared Insights BPM conference coverage. Next year, maybe they’ll spring for more than 2 nights of hotel…
Most of my customers are large financial institutions, and I help them to select and implement BPM within their organizations. The technology is only one part of that, however: I’m almost always helping them with their business processes as well. Since policies and procedures drive processes, I often end up in the thick of their policies and procedures, and that’s where the confusion starts.
First of all, there’s the definition of terms. What’s the difference between a policy and a procedure, when many people lump them together as if PoliciesAndProcedures were one word? I like this definition:
Policy is a mandate and directive from the top of the organization. Its purpose is to influence behaviour. From it, management provide the overarching principles under which the business operates. It should not vary in its message or enforcement model.
Procedures are process specific and detail the steps taken to achieve an objective. Procedures include operations manual, user manual, and all manner of process documentation.
I see policies as the rules, or laws, of an organization, whereas the procedures are the processes used to enact the policies. The problem is, however, that many companies see policies and procedures as belonging to the Legal/Compliance department, and create another set of processes — usually referred to as an "operational guide" — that are created and maintained by the operational area that executes the actual processes. If you throw in a BPMS, then some (but rarely all) of these operational procedures may be further documented in the process descriptions within the BPMS or a BPA tool.
What’s the distinction between a policy and a procedure? Is there a difference between a procedure and the operational description of a business process? What about between the business process and the process model in a BPMS?
Secondly, there’s the responsibility issue that I referred to above: who’s responsible for each of these essential bits of corporate documentation? Legal/Compliance is almost always handed policies and procedures, but what about the case when the procedures are actually descriptions the operational business processes? Should policies be left with Legal, and procedures given to the operational areas, with Compliance there to make sure that everything matches up? Or are the operational process descriptions a separate, more fine-grained version of the procedures, leaving the procedures with Legal and the operational processes with Operations?
If process maps are created within a BPMS, do they become part of the business process documentation, replace part of it, or stay as a separate "implementation view" of the processes? I’ve definitely seen cases where the process maps in a BPMS bear little resemblance to what the business perceives as its processes, either due to limitations in the BPMS environment or to the business having an incomplete view of the process.
And if there’s four separate types of documentation — policies, procedures, business processes and BPMS process definitions — who’s responsible for keeping them all in synch?
Third is the whole technology issue: how is all of this information captured, published and synchronized? There are a tools such as RulesArts and RuleBurst (both of which I saw last week at the Business Rules Forum) that help to capture policies as high-level non-executable rules — an approach that makes more sense than just trying to document them free-form in a word processor while praying for consistency. Check out the Flash demo on the RuleBurst site to see what this can look like. Some of this systems are also business rules engines, that is, they execute the rules and can be called from other applications; some are just platforms for non-technical users to document policies, detect gaps and exceptions, and help to ensure compliance.
As we move into procedures, operational guides and process definitions, it’s all about processes. Processes based on rules (and what process isn’t?), but processes nonetheless. Those organizations documenting their policies in a word processor are likely also documenting their procedures in the same way — in fact, possibly in the same document — using descriptive text and a few diagrams. At some level of detail, someone starts drawing process maps, although these are usually as illustrations to the descriptive text rather than a replacement for it.
The two biggest issues in all of this technology are synchronization (usually manual, and therefore almost certainly out of date) and publishing (ditto). From the synchronization standpoint, there needs to be something that links the policies (rules) with the various granularities of process descriptions (both text and graphical) and either keeps them in synch or alerts someone when related pieces are modified. For publication, none of this information is of any use unless it’s in the hands of the people who need it; that means that there needs to be an easy (or automated) way for all of this information to be published within an enterprise and accessed with nothing more than a browser and network authentication.
What starts to become shockingly apparent as you dig into the technology is that policies are about rules, and procedures are about processes. Yeah, I know, I said that at the start of this post, but it’s not just some abstract concept, it’s about how you need to document and implement policies and procedures. The crux of the issue is in the crossover from rules to process, since a rule (policy) usually doesn’t dictate the operational procedure required to enact it, hence there’s not a clear technology path to map from policies to procedures. If policies are maintained in a high-level rule repository and procedures are maintained in some combination of descriptive text and process maps, what’s the missing link between them?
Policy and procedure documentation is just one place where business rules and business processes intersect (they touch again at the point of process execution), and I’m interested in exploring the ideas around this. I’ve put forward more questions than answers — feel free to join the conversation by commenting on this post, tracking back from your own post, or dropping me an email.
After attending the Business Rules Forum last week, either I’m more aware of related events or I’m on more mailing lists for rules/decisioning vendors. In either case, Fair Isaac is putting on an Introduction to Enterprise Decision Management webinar tomorrow at 2pm Eastern. From their description of the event:
Learn how your organization can automate and improve decisions by:
- Taking control of your decisions and make them a corporate asset
- Separating operational decisions from processes and systems for maximum agility
- Using business rules management systems to ensure decision consistency and speed
- Applying different kinds of analytics – descriptive, predictive and decision – to make more precise and profitable decisions
It’s free, just sign up online.
I’m sure that James Taylor has almost given up on me ever writing a book review of Smart Enough Systems: I wrote a brief advance review back in April that’s printed in the book, but nothing since it was released. This week, I’ve been immersed in business rules and decisioning, and had a chance to finally meet James face-to-face after a couple of years of emailing back and forth. Also, James’ situation has changed since the book was released: he’s left Fair Isaac and is now an independent, working (I think) with his co-author, Neil Raden. Neil, who I met very briefly this week, is an independent consultant and analyst who’s been focussed on business intelligence for quite a while; James refers to his work as “BI 2.0″ (a term that I think that I invented here in early 2006). The two of them met through James’ blog and started the conversation about how someone needed to write a book about this crossover area between business rules and business intelligence.
Just to get started, here’s my pre-release review:
Taylor and Raden’s central manifesto highlights that it’s critical to embody more intelligence in today’s business decision-making and have consistent, automated decisioning built into business processes in order to remain agile and competitive in today’s fast-moving market. They take you through the core concepts of enterprise decision management (EDM), dive into the underlying technologies, then address how to integrate EDM into your business processes to create your own Smart (Enough) Systems.
By focusing on operational decisions that contribute to corporate strategy, Smart (Enough) Systems provide the ability not only to create agile business processes, but to have these processes be self-learning based on historical results. Instead of simply capturing operational process statistics in a data warehouse for later analysis, Smart (Enough) Systems use that knowledge to inform the business rules and allow them to adapt their guidance of the decision-making process. By extracting the decisions from legacy applications, static enterprise applications and manual procedures, and managing them within a shared enterprise decision management system, operational decisions can be applied consistently — and modified easily for processes in flight — across the enterprise.
What I like about the book is that it provides an overview that will get business people interested in the topic, as well as providing a practical guide to getting started. There’s a lot of books focussed on analytics and business rules already, but most of them assume that you already know what you’re going to do with these technologies; in many cases, it’s hard to discover the right decisions on which to focus, since some things are more important than you may have originally perceived when starting a project.
As I heard this week, there’s still a strong tendency to sell rules technology to IT as a way to implement systems fast and cheaper, instead of selling decisioning to the business as a way to solve business problems. For the most part, decisions in business processes are unconsciously relegated either to a developer coding them into an application, or to a front-line worker executing them on an ad hoc basis. Decisions, and the rules that underlie them, need to be made explicit: therein lies the path to both compliance and agility. I joked yesterday about Jan Venthienen’s presentation that he’d like to reduce all processes to a single activity and have all business logic in a rule set, but there’s definitely value in what he’s saying: we need to seek out the decisions in processes, capture the ever-changing business rules that are embedded within them, and move these out of the processes and into rules/decisioning systems. The reality is that process maps don’t change all that often if the decisions and business rules are stripped out of the processes: most business agility is based on rule changes, not process changes, especially for high-volume core transactional processes. However, since we often code rules directly into the business processes — making these processes some combination of processes and decision trees — we perceive the need to be that of changing processes rather than changing rules. And so (and I’ll be totally blackballed in the BPM community for this particular blasphemy), the entire BPM industry has focussed on building tools that allow for easy modification of process maps by the business, when maybe they really should have been focussed on pushing their customers towards the integration of business rules for decisioning in order to greatly reduce the need to modify process maps.
Climbing off my soapbox for a moment and returning to James and Neil’s book, they focus on a key benefit of this sort of smart decisioning in operational processes, which is the ability to replace the “irreplaceables”, such as insurance underwriters, before all the boomers take their knowledge and retire to Palm Beach. This greatly controls risk: not just the risk of the workers leaving, but the risk of bad or inconsistent decision-making. By allowing decisions to become more personalized based on the customer and scenario, this can also provide the ability to be more customer-centric, as well as agility in the face of changing regulations and market conditions.
They see a number of roadblocks to this sort of smart decisioning at the operational level:
The ideas that James and Neil put forward in their book are a great place for business and IT to start collaborating on how to make smart enough systems.
You may also want to listen to the interview that Scott Sehlhorst did with James back in June (I know, I know, when the book was actually released).
For the last session today, I attended Jan Venthienen’s session; he’s a professor at Katholieke Universiteit Leuven. He talked about different representations of rules, particularly decision tables (at length, although in an interesting way). He talked about the problems with maintaining decision trees, then as he moved on to business processes, he showed how a business process with the rules encoded in the process as routing logic was really just a form of decision tree, and therefore difficult to maintain from a rules integrity standpoint. As rules are distilled out of and separated from the processes, the processes become thinner and thinner, until you have a single branch straight-through flow. I have the feeling that he’d like to reduce the process to a single activity, where everything else is done in a complex rule called from that step. I’m not sure that I agree with that level of stripping of logic out of the process and into the rules; there’s value in having a business process that’s understandable by business users, and the more that the logic is encapsulated in rules, the harder it is to understand how the process flow works by looking at the process map. The critical thing is knowing which rules to strip out of the business process, and which to leave in.
He’s doing research now to determine if it’s possible to specify business rules, then automatically derive the business process from the rules; an interesting concept. In order to do this, there must be rules that constrain the permission and obligations of the actors in the process, e.g., an order must be accepted before the product is shipped. This presents two possible architectural styles: process first, or rules first. In either case, what is developed is an architecture of rules, events and services, with a top layer of business rules and processes, a middle layer of services and components, and a bottom layer of enterprise applications.
I’m eager to learn more about the relationship between policies, procedures and rules, and how they relate to compliance, so I sat in on a presentation by Peter Still of RuleBurst. There’s a pretty high percentage of vendors on the speaker roster, but so far the quality has been good so no complaints.
The theme of Still’s talk is that the business rules approach will only gain critical mass if it stops being a technical implementation tool and starts being a business problem-solving tool. The current pitch from the business rules vendors is that this is a way to implement systems faster and cheaper, while allowing the business to access some tuning parameters, but this is really focussed on the technological capabilities and not the business value of business rules. This is such a perfect mirror of the BPM field, where BPM has just barely moved from a purely technical sell to something that’s now being sold more and more to the business side of an organization, so I can completely understand where the business rules market is and the challenges that lie ahead in shifting the focus of their marketing message. Worldwide market for business rules product revenue is $250M — not a lot when you consider the size of related markets — and it could be a lot larger if there was greater recognition of the business benefits of business rules.
A perfect business case for re-targeting the business rules message is compliance: it’s an enterprise-wide initiative with executive support where business rules can be included in the decisioning at key points of the process. Although business rules aren’t the complete answer to compliance since compliance is a very process-focussed initiative, rules can be a significant contributor to compliance efforts. One of the difficulties with compliance is that many regulations, such as Sarbannes Oxley, are pretty vague since they have to deal with such a broad range of companies, and it’s difficult to determine precise business rules to implement them. Compliance at a transactional level is a mostly automated application of BPM and business rules, but as you move up to risk management and higher-level compliance factors, there’s less automation although still opportunities for business rules to be wrapped in a compliance framework, such as using business rules to classify a risk although the management of that risk may be done manually. Still maintains that there’s a link between transactional and operational compliance, and believes that business rules can help with that link although that’s not recognized by most business rules vendors.
As with most other complex applications of technology, you can solve this with an integrated compliance and rules solution from a single vendor, or go for a best-of-breed approach. Still recommends the former approach, and invited us to drop by his booth to check out what RuleBurst has to offer in this area.
Paul Armborst of the Westfield Group presented on his experiences in implementing business rules for their various insurance applications over the past 6 years. He sees one of the key problems is in documenting business rules, with two main choices for how to do it:
He sees that the best way would be a combination of both approaches: a throw-away repository to document rules as they are discovered, and an extract from the BRE to provide ongoing documentation.
He pointed out one of the problems with introducing business rules: “real” developers want to write Java code, not this airy-fairy business rules stuff, which is an argument that I see against BPM as well. As IT budgets get squeezed, however, development teams will start to look for ways to reduce the amount of code that they have to write and pass some of the tuning capabilities directly over to the business; both BRE and BPM provide these types of capabilities.
He discussed various methods of implementing business rules:
He also discussed stateful and stateless implementations of business rules: although stateless is simpler to implement, stateful allows for running only the rules that use data that has changed since the last evaluation of each rule.
There were some last comments on end user rule maintenance: all of their rules are written by developers, but they’re thinking about how to offer some rule creation and modification capabilities to end users. It’s important to have a BRE that allows some sort of restricted view for less technical users, but it’s also necessary for the techies to do some helpful things like naming objects in a manner that users can understand rather than, say, reverse Hungarian notation. Users who have access to create rules need to have some notion of the logic required to do so, and there needs to be some provision for testing.
I moved over to the über-geeky “chief architect” track to hear Rik Gerrits of RuleArts and Petr Choteborsky of Microsoft, but 10 minutes into the session, Gerrits is still giving some fairly basic definitions of business rules management: where rules live, how they’re developed, and how they’re managed and used. He does make a point that business processes consume business rules but that they should be distinct in terms of methodology and implementation, as well as other motivations for business rules management such as compliance and efficiency improvements.
Choteborsky took over with a case study about Microsoft’s internal development (as in applications for their own internal use, like software licence authorization), and instantly endeared himself to the audience by saying that he was in corporate IT at Microsoft, and was just as much a victim of the Microsoft product groups as we were. They had issues with software development lifecycle documents and the rules that were embedded within those documents: multiple, conflicting instances of rules in different documents; rules not explicitly defined hence less agile; no common vocabulary leading to inconsistency and miscommunication. Over time, the business logic is lost, and the business requirements documentation becomes completely out of sync with the application and the user manual, so that the only true representation of the business logic is embedded within the application as coded.
He stepped through an example, showing how to break down the prose in a requirements document to determine what is a rule set (a group of related rules), what’s a rule, what’s a fact (unchangeable, therefore may be hard-coded), what is usability behaviour (which may include hidden rules and facts), and what is contextual information that describes capability without being something that will be explicitly coded. Very cool example, since he shows the tendency for the prose in what we think of as a fairly well-written requirements document to actually be a confusing mix of facts, rules, behaviour and context that doesn’t really provide adequate information about what should be written to be easily changeable versus what can be hard-coded into an application.
He went on to show how the same paragraph should be restructured as facts and rules (describe the pure essence of how business must be conducted, independent of implementation detail), requirements (UI and application requirements to implement the rules) and context (information that makes it easier to understand the facts, rules and requirements; redundant information that is not coded). The rules mantra (which I’m just learning today) is “rules build on facts, facts build on terms”, and he shows the terms sprinkled throughout the facts and rules.
They’re attempting to change their requirements documents to this form of structured requirements using business rules (for going-forward documents, not retrofitting the existing ones), but it’s a painful process: there needs to be some common vocabulary and a significant amount of training in some cases to have people start thinking in this way. There was a comment from the audience that once the vocabulary — particularly standardization of terms — was established, there’s usually a pretty good uptake from the business community since they really like language that can help them to define their business more precisely and unambiguously.
There was another comment from the audience that what he is calling a requirement is actually a specification, which is an argument that I’ve had about a zillion times in my years of software development: I completely agree with the comment, as did Choteborsky, but he stated that this was the common terminology in use at Microsoft today and he wasn’t trying to fix everything at once. I have to see the pragmatism in that, although there should likely be some sort of migration of terminology to be more accurate.
He went into more detail on terms, facts and rules, including descriptions of each, and the use of graphical term models and fact models. He also made a distinction between a rule and a policy: a rule can produce an action or decision, whereas a policy is more general but might sound rule-like. He stepped through the before and after of a fact model, where he went through and marked each object and relationship in the model as correct, sort of incorrect, or outright wrong, then found new relationship pathways and defined new terms in the model to make it a better reflection of the actual facts and provide a more logical structure for developing rules. He’s just using Visio for creating the fact models, although I’m sure that some more comprehensive modeling tools could make this process a bit easier. They’re starting to use RuleXpress (the RuleArts product) for terms, facts and rules, although the rules themselves are actually encoded within applications: rules management without a rule engine. As he pointed out, although some business rules may end up in a business rules engine, some end up directly in the code of an application, and some are never codified but become part of an operational manual. We see exactly the same thing in BPM, where a process model may include steps that are transferred to a BPMS, but also ones that are completely manual and never represented within a BPMS. Having a modelling tool separate from the execution environment provides greater flexibility in what can be modelled, but I suspect that the same issues of synchronization and round-tripping occur in rules modelling environments as exist in process modelling.
Choteborsky was a great speaker: knowledgeable, able to explain some fairly complex concepts, and funny (when one slide came up, he said “I don’t know why PowerPoint made the font on this slide bold and ugly, but I’ve learned that I don’t need to win every battle”). The great thing is that he presented a methodology for developing business specifications that everyone in the room involved in software development could take away and start examining for their own use.
David Straus of Corticon gave an engaging presentation about BR and BI, starting with the Wikipedia definitions about each, then characterizing BI as “understanding” and BR as “action” (not unlike my statement that BI in BPM is about visibility and BR in BPM is about agility). He started with the basic drivers for a business rules management system – agility (speed and cost), business control while maintaining IT compliance, transparency, and business improvement (reduce costs, reduce risk, increase revenue) — and went on to some generalized use cases for rules-driven analysis:
He walked through a simplified claims scenario, where the claims agent is not replaced with rules but still makes a decision in the process, but their decision is compared against a decision made by a rules system and any discrepancies are investigated. In other words, although there’s still a person making the decision in the process, the rules system is acting as a watchdog to ensure that their decisions are compliant with the corporate policy. After some time, there can be some analysis of the results to detect pattens in non-compliance: is it an individual agent that’s causing non-compliance, or a particular product, or are the rules not aligned with the requirements? In some cases, the policies given to the agents are actually in conflict, so that they have two different “right” answers in some cases; in other cases, agents may have information that’s just not represented in the rules. By modeling the policies in a business rules system, these conflicts can be driven out to establish integrity across the entire set of rules. This can also be used in cases where an organization just isn’t ready to replace a human decision with a business rules system, in order to validate the rules and compare them to the human decisions; this can establish some trust of the decisioning system that may eventually lead them to replace some of the human decisions with automated ones to create more consistent and compliant decisions.
David had a number of case studies for this combination of rules and analytics, such as investment portfolio risk management, where mergers and acquisitions in the portfolio holdings may drive the portfolio out of compliance with the underlying risk profile: information about the holdings is fed back through the rules on a daily basis to establish if the portfolio is still in compliance, and trigger a (manual) rebalancing if it is out of compliance.
By combining business intelligence (and the data that it’s based on) and business rules, it’s also possible to analyze what-if scenarios for changes to rules, since the historical data can be fed through the new version of the rules to see what would have changed.
He’s challenged the BI vendors to do this sort of rules-based analysis; none of them do it now, but it would provide a hugely powerful tool for providing greater insight into businesses.
There was a question from the audience that led to a discussion about the iterative process of discovering rules in a business, particularly the ones that are just in people’s heads rather than encoded in existing systems; David did take this opportunity to make a plug for the modeling environment in their product and how it facilitates rules discovery. I’m seeing some definite opportunities for rules modeling tools when working with my customers on policies and procedures.
For the last presentation today, I listened to John Lucker of Deloitte discuss what they’ve developed in the area of predictive pricing models for property and casualty insurance. Pricing insurance is a bit trickier than pricing widgets: it’s more than just cost of goods sold plus a profit factor, there’s also the risk factor, and calculating these risks and how they affect pricing is what actuaries do for a living. However, using predictive models can make this pricing more accurate and more consistent, and therefore provides insurance companies with a way to be more competitive and more profitable at the same time.
I know pretty much nothing about predictive modeling, although I think that the algorithms are related to the pattern recognition and clustering stuff that I used to do back in grad school. There’s a ton of recent books on analytics, ranging from pop culture ones like Freakonomics to the somewhat more scholarly Competing on Analytics. I’m expecting Analytics for Dummies to come out any time now.
Predictive modeling is used heavily in credit scoring — based on your current assets, spending habits and debt load, how likely are you to pay on time — and in the personal insurance business, but it hasn’t really hit the commercial insurance market yet. However, the insurance industry recognizes that this is the future, and all the big players are at least dabbling in it. Although a lot of them have previously considered this in order to just do more consistent pricing, what they’re trying to do now is have the predictive models integrate together with business rules in order to drive results. This is helping to reduce the number of lost customers (by providing more competitive pricing), reducing expenses (by providing straight-through processing), increasing growth (by targeting new business areas), and profitability (by providing more accurate pricing).
He talked about how the nature of targeting insurance products is moving towards micro-segmentation, such as finding the 18-year-old male drivers who aren’t bad drivers or the roofing companies with low accident rates, then selling to them at a better price than most insurance companies would offer to a broader segment, such as all 18-year-old male drivers or all roofers. He didn’t use the words long tail, but that’s what he’s talking about: this is the long tail of insurance underwriting. There’s so much data about everything that we do these days, both personal and business, that it’s possible to do that sort of micro-segmentation by gathering up all that data, applying some predictive modeling to extract many more parameters of the data than would have been done in a manual evaluation, and develop the loss predictive model that allows a company to figure out whether you’re a good risk or not, and what price to charge you in order to mitigate that risk. Violation of privacy? Maybe. Good insurance business? Definitely.
The result of all this is a segmented view of the market that allows a company to decide which parts they want to focus on, and how to price any of those parts. Now it gets really interesting, because now these models can be fed into the business rules in order to determine the price for any given policy: a non-negotiable price, much like Saturn does with its cars. This disintermediates both the agents and the underwriters in the sales process, since all of the decisions about what risks to accept and how to price the policies is automated based on the predictive models and the business rules. Rules can even be made self-optimizing based on emerging trends in the data, which I discussed in my presentation this morning, although this practice is not yet mainstream.
Lucker’s message is that business rules are what leverages the power of the predictive models into something that makes a difference for a business, namely, improving business processes: reducing manual processes and associated costs, enhancing service and delivery channels, targeting sales on profitable niches (that long tail), and improving point-of-sale decision-making at an agency.
He ended up describing a top-down approach for designing business rules, starting with organizational strategy, decomposing to the functional areas (business operations, sales, customer service, distribution), then developing the business rules required to help meet the objectives of each of the areas.
Seriously, that was the name of Giovanni Diviacchi’s session that I attended this afternoon, which looked at his experience as a business analyst at both Freddie Mac and Fannie Mae (the two big government-backed mortgage companies in the US). He had a number of good pointers on how to extract rules from the business and document them in a way that will be properly implemented by the developers.
They developed a “business action language” for the business analysts to communicate with the developers in an unambiguous way, including statements such as “present” (i.e., >0 and not null), “mutually exclusive”, “is required”, and my personal fave, “read my mind”.
He pointed out that the old axiom “rules are meant to be broken” is true even for business rules, in that you can’t ever plan for all the ways in which a rule might need to be overridden; he discussed one case of a woman who was born prior to 1936, never worked and never had a Social Security Number, which meant having to override the rule that SSN is required for a mortgage. There’s a lot that can be learned from this one example: I see so many rules embedded directly in applications — especially web applications — that make some assumptions that aren’t necessarily true, such as assuming that all people have a US address.
I often work through issues of policies, procedures and processes with my customers, and it was interesting to hear his comments on the relationship between policies and rules. He said that if the policies are well-written, the rules pretty much write themselves, and by spending more time on the policy behind the rules, you end up with a better set of rules. That definitely caused an “aha” moment for me in my emerging role as an evangelist for business rules in a BPM world, and will help to form some of my ideas on how all these things come together.
After a brief intro by Gladys Lam, the executive director of the Business Rules Forum, the conference kicked off with a keynote from Ron Ross, the driving force behind this event and a big name in the business rules community. A couple of things are distracting my attention from his talk: I’m up directly after him, and I’m presenting in this room, which is the main (read: big) conference hall. Let me make my ever-present complaint about passworded wifi in the meeting room and no free wifi or wired internet in the hotel, since I know that my regular readers would be disappointed without that news from the front lines. 
Ron and I have exchanged email over the years, but this is our first opportunity to meet face-to-face; I’ll also have the chance to meet James Taylor and a few others who I only know from afar. Today, Ron’s talking about the shift from business rules to enterprise decisioning. This is the first business rules conference that I’ve ever attended, which means that most of the attendees likely know a lot more about the subject matter than I do, and most of the sessions will be new material for me.
Ron predicted that no one will be talking about SOA at a major conference in 15 years, but they will be talking about business rules or decisioning; I certainly agree with the first point, and the second makes a lot of sense.
When he said “we want our business processes to be smarter”, it was like music to my ears, and a direct lead-in to my presentation. He talked about three trends in the decisioning space:
He finished up with some pointers on where to think about applying decisioning in your business through a few use cases, such as creating “rules of record” for compliance purposes.
Like every other technology-specific conference (especially the BPM ones that I typically attend), this one has at the heart of it that its subject matter is the most important technology in an enterprise, and that herein lies the key to business perfection. I’m being a bit facetious, but we really do need to start getting a bit more cross-over between some of these conferences and technologies.
I spoke briefly last week at the Forrester Technology Leadership Forum about BPM, BI and BR, and had a great response from a couple of the business rules vendors who were in attendance. I’ll be expanding on that topic in a few weeks at the Business Rules Forum in Orlando, where the conference focus this year is on enterprise decisioning, especially as it relates to BPM and BI. I’ll be talking about how BPM, BR and BI can be combined to make a process improvement platform that?s greater than the sum of its parts, by:
Organizations are embracing business process management to improve their business processes. However, automation of processes isn?t the whole picture: processes must be both agile and transparent to reap the full benefits of BPM, which makes business rules and enterprise decisioning important topics for BPM practitioners.
The Forum Conference has offered a 10% registration discount code to readers of my blog: enter the promotional code 7PGRSP on your registration for 10% off your conference fees. I don’t get a referral fee, this is just a favour to you as my readers.
You can get the full schedule and abstracts (and printable registration) here, and register for the conference here.
I’ll be around for most of the conference, so be sure to look me up if you’re there.
I ended up skipping the session after mine at the end of the morning, but had some great hallway conversations with some of the business rules vendors who indicated that they think that I’m on track with what I’m saying about BPM and BR.
For the first of the afternoon sessions, I’m attending a panel discussion on the convergence of the three B’s — BI, BPM and BR — featuring Mike Gilpin (EA and application development), Boris Evelson (BI) and Colin Teubner (BPM). I covered a tiny bit of this topic in slides 22-24 of my presentation this morning, and will be doing a full-length presentation on this same topic at the Business Rules Forum next month in Orlando, so I’m interested to see if the Forrester analysts have the same thoughts on this subject as I do.
They start with the statement that “design for people, build for change” will drive the convergence of the three B’s. Interestingly, although a few people in the room stated that they use BPM and BI together, almost no one raised their hand to the combination of BPM and BR — a combination that I feel is critical to process agility. Gilpin went through a few introductory slides, pointing out that almost no business rules are explicitly defined, but are instead buried within processes and enterprise applications. He sees BI as driving effectiveness in businesses, and the combination of BPM and BR as driving efficiency.
Forrester will be publishing some reports about the convergence of the three B’s, and although there are some two-way combinations in vendor products now, there are no vendors that combine all three in a single product. I’m not sure that this is a bad thing: I don’t think that we necessarily want to see BR or BI become a part of BPM because it ultimately limits the usefulness of BR and BI. Instead, I see BR and BI as services to be consumed by BPM, with BI having the additional role of combining process execution statistics generated by the BPMS with other business data. An explicit question was asked about when to use the BR and BI included in the BPMS versus when to use a third-party best-of-breed BR or BI system; Teubner and Gilpin offered some guidelines for this as well as some examples of each situation, but it’s not completely clear if there’s a distinct boundary between when to use the BPMS’ in-built functionality versus the third-party specialist product.
My message on this topic is that BR is the key to process agility, and BI is the key to process visibility as well as feeding back into BR in order to further increase agility. By using the BR and BI functionality within your BPMS, however, you’re typically not getting full BR or BI functionality, but some limited subset that the BPMS vendor has selected to implement. Furthermore, you can’t reuse that functionality outside the BPMS, and in the case of business rules, a change to the BPMS’ rules often requires retesting and redeploying the process models, and does not apply to in-flight processes. However, if you’re not sure if you need BI or BR (hint: you do), then using the in-built functionality in the BPMS gives you an easy-to-integrate and lower cost way to get started. Moving to a separate third-party business rules system gives you a couple of key advantages: you can reuse the same rules across different processes and across other applications in your enterprise, and changes to the rule impacts in-flight processes since the rule is not executed from the BRE until that point in the process is reached. Moving to a separate third-party business intelligence system also provides the advantage of being able to analyze the process data in the context of other business data, and potentially feed back the results of complex analytics to inform the business rules, that in turn drive the business processes. The bottom line: BR and BI are used for many applications in the enterprise that are not explicitly process-related, or combine data from many systems of which the BPMS is just one source. For example, although there are processes embedded within your ERP system, your BPMS may not have direct access to all the information that’s in those processes and hence the BI that’s part of your BPMS can’t (easily) include that data in its analytics and reporting; a general-purpose BI platform may be much more suited to combining your BPMS statistics with your ERP statistics.
A lot of the conversation in this session, which was very interactive with the audience members, was around whether to use converged products versus separate products. It’s not a completely simple answer, and I’ll definitely be thinking about the use case boundaries between converged and separate products before I show up at the Business Rules Forum to continue this discussion.
Evelson and Teubner will be publishing an initial paper in this area in the next few weeks, using the concepts that they’ve presented here today, but see it as a springboard for more discussion in this area rather than an end-point.
John Rymer, who I’ve read before on business rules topics, talked to us about why we should care about business rules software; Forrester’s position is that business rules are a key enabler of design for people, build for change.
He started with definitions of business rules and a business rules platform, then went on to state that business rules are an alternative to conventional programming: with business rules you don’t have to translate business terms into geek speak, and you don’t have to specify every possible combination of rules works. The implications are that business people are most likely to get what they need since they can actually understand the “language” in which the rules are written, and more complex problems can be more easily solved with much less time required to change systems. Software can even adapt based on the results of the rules; BPM is just one example of this, but business rules can be applied in the same way in other types of software.
Rymer showed how business rules can be applied in the areas of the “perfect storm” that Connie Moore mentioned this morning as causing the transformation that’s underway now: design evolution (rules add adaptation to context and design), process evolution (rules enable decisioning, auto processes, closed loops), workforce evolution (business people contribute to rules) and software evolution (rules enable global policies for SOA, service selection). He went on with a great list of how organizations benefit from business rules:
He gave some good examples from financial services, showing how business rules have been applied to core tasks such as credit scoring and underwriting, then expanded to areas such as fraud detection and call center programs.
He highlighted that business rules allow organizations to divide change management responsibilities, where business people take responsibility for maintaining the more volatile rules and processes, whereas IT remains responsible for maintaining the core rules and processes.
He ended up addressing the issues of why business rules haven’t really caught on; I see so little acceptance of this with many of my financial services customers and I’m not surprised, although it seems strange that a technology that can offer so much benefit is being ignored by so many companies. The top reason for not implementing business rules? “We don’t do things that way”, that is, they like to write their rules in Java code instead. He also cited lack of standards, high product cost (which I still don’t think is more than writing and maintaining that Java code), lack of participation from the big vendors, and a still-shifting landscape as other reasons for resistance to business rules.
Like Rymer, I believe that business rules will be a significant part of any agile organization. In some cases, organizations already have business rules software but it’s hidden away in one department, but you need to pull it out of the closet and put it to work. Forrester has a Wave (product comparison) for business rules, but he admitted that it’s a bit out of date; it sounds like a new one might be coming out shortly.
