Category Archives: CEP

complex event processing

Conference Within A Conference, Part Two: Big Fast Data World

John Bates, who I know from his days at Progress Software, actually holds the title SVP of Big Fast Data at Software AG. Love it. He led off the Big Fast Data World sub-conference at Innovation World to talk about real-time decisioning based on events, whether that is financial data such as trades, or device events from an oil rig. This isn’t just simple “if this event occurs, then trigger this action” sort of decisions, but real-time adaptive intelligence that might include social media, internal business systems, market information and more. It’s where events, data, analytics and process all come together.

The goal is to use all of the data and events possible to appear to be reading your customer’s mind and offering them the most likely thing that they want right then (without being too creepy about it), using historical patterns, current context and location information. For example, a customer is in the process of buying something, and their credit card company or retail partner uses that opportunity to upsell them on a related product or a payment plan, directly to their mobile phone and before they have finished making their purchase. Or, a customer is entering a mall, they are subscribed to a sports information service, there are available tables at a sports bar in the mall, so they are pushed a coupon to have lunch at the sports bar right then. Even recommendation engines, such as we see every time that we visit Amazon or Netflix, are examples of this. Completely context sensitive, and completely personalized.

On the flip side, companies have to use continuous monitoring of social media channels for proactive customer care: real-time event and data analysis for responding to unhappy customers before situations blow up on them. People like Dave Carroll and Heather Armstrong (and sometimes even me, on a much smaller scale) can strike fear in the hearts of customer service organizations who are unable to respond appropriately and quickly, but can cause big wins for these companies when they do the right things to fix things in an expedient manner for their customers.

What do you need to do to make this happen? Not much, just low-latency universal messaging, in-memory unstructured data, real-time predictive analytics, intelligent actions via real-time integration to operational systems, and real-time visual analytics. If you’re a Software AG customer, they’re bringing together Terracotta, Apama and JackBe into a unified platform for this sort of adaptive intelligence, producing intelligent actions from big data, in real time, to/from anywhere.

Software AG Big Fast Data

We then got a bit of a lesson on big data from Nathaniel Rowe, a research analyst at Aberdeen Group: how big is big, what’s the nature of that data, and some of the problems with it. The upshot: the fact that there’s a lot of data is important, but it’s the unstructured nature of it that presents many of the difficult analytical problems. It’s about volume, but also variety and velocity: the data could be coming from anywhere, and you don’t have control over a lot of it such as the social media data or that from business partners. You have to have a clear picture of what you want out of big data, such as better customer insights or operational visibility; Rowe had a number of use cases from e-commerce to healthcare to counterterrorism. The ability to effectively use unstructured data is key: those companies that are best in class are doing this better than average, and it translates directly to measures such as sales, customer satisfaction and net promoter score. He finished up with some of the tools required – automatic data capture, data compression, and data cleansing – and how those translate directly to employees’ ability to find data, particularly from multiple sources at once. Real-time analytics and in-memory analytics are the two high-speed technologies that result in the largest measurable benefits when working with big data, making the difference between seconds (or even sub-second) to see a result or take an action, versus minutes or hours. He ended up with the correlation between investing in big data and various customer experience measures (15-18% increases) as well as revenue measures (12-17% increases). Great presentation, although I’m pretty sure that I missed 75% of it since he is a serious speed-talker and zipped through slides at the speed of light.

And we’re done for the day: back tomorrow for another full day of Innovation World. I’m off to the drinks reception then a customer party event; as always, everything is off the record as soon as the bar opens. Smile

Kicking Off @SoftwareAG @InnovationWorld

For the first time in a few years, I’m at Software AG’s Innovation World conference in San Francisco (I think that the last time I was here, it was still the webMethods Integration World), and the focus is on the Digital Enterprise. At the press panel that I attended just prior to this evening’s opening keynote, one journalist made the point that “digital enterprise” is kind of a dumb term (I paraphrase here) because everything is digital now: we need a more specific term to mean what Software AG is getting at with this. Clay Richardson of Forrester, who I dragged along to the press session, said that his colleagues are talking about the post-digital age, which I take to mean is based on the assumption that all business is digital so that term is a bit meaningless, although “post-digital” isn’t exactly descriptive either.

Terminology aside, Software AG’s heart is in the right place: CEO Karl-Heinz Streibich took the stage at the opening keynote to talk about how enterprises need to leverage this digital footprint by integrating systems in ways that enable transformation through alignment and agility. You can still detect the schisms in the Software AG product portfolio, however: many of the customer case studies were single-product (e.g., ARIS or webMethods), although we did hear about the growing synergy between Apama (CEP and analytics) and webMethods for operational visibility, as well as Apama and Terracotta (in-memory big data number crunching). As with many of the other large vendors that grow through acquisitions,

We heard briefly from Ivo Totev, Software AG’s CMO; saw presentations of two of their customer innovation awards; then had a lengthier talk on the power of mobile and social from Erik Qualman, author of Socialnomics and Digital Leader. Unlike the usual pop culture keynote speaker, Qualman’s stuff is right on for this audience: looking at how successful companies are leveraging online social relationships, data and influence to further their success through engagement: listening, interacting and reacting (and then selling). He points out that trying to sell first before engaging doesn’t work online because it doesn’t work offline; the methods of engagement are different online and offline, but the principles from a sales lead standpoint are the same. You can’t start the conversation by saying “hey, I’m great, buy this thing that I’m selling” (something that a *lot* of people/companies just starting with Twitter and/or blogging haven’t learned yet).

Qualman took the popular Dave Carroll’s “United Breaks Guitars” example from a couple of years ago, and talked about not just how United changed their policies on damage as a result of this, but the other people who leveraged the situation into increased sales: Taylor Guitars; a company that created a “Dave Carroll” travelling guitar case; and Carroll himself through sales of the song and his subsequent book on the power of one voice in the age of social media. He looked at companies that have transformed their customer experience through mobile (e.g., Starbucks mobile app, which has personally changed my café loyalty) by giving the customer a way to do what they want to do – which hopefully involves buying your product – in the easiest possible way; and how a fast and slightly cheeky social media presence can give you an incredible boost for very little investment (e.g., Oreo’s “dunk in the dark” tweet when the lights went out during the Superbowl). I gave a presentation last year on creating your own process revolution that talked about some of these issues and the new business models that are emerging because of it.

Great to see John Bates here, who I know from his tenure at Progress Software and came on at Software AG with the Apama acquisition, as well as finally meet Theo Priestley face to face after years of tweeting at each other.

Disclosure: Software AG is a customer (I’m in the middle of creating some white papers and webinars for them), and they paid my travel expenses to be at this conference. However, what I write here is my own opinion and I have not been financially compensated for it.

Can BPM Save Lives? Siemens Thinks So

My last session at Gartner BPM 2013 is a discussion between Ian Gotts of TIBCO and their customer Tommy Richardson, CTO of Siemens Medical Solutions. I spoke with Siemens last year at Gartner and TUCON and was very interested in their transition from the old iProcess BPM platform (which originally came from TIBCO’s Staffware acquisition) to the newly-engineered AMX platform, which includes BPM and several other stack components such as CEP. Siemens isn’t an end-user, however: they OEM the TIBCO products into their own Soarian software, which is then sold to medical organizations for what Richardson refers to as “ERP for hospitals”. If you go to a hospital that uses their software, a case (process instance) is created for you at check-in, and is maintained for the length of your stay, tracking all of the activity that happens while you’re there.

With about 150 customers around the world, Seimens offers both hosted and on-premise versions of their software. Standard processes are built into the platform, and the hospitals can use the process modeler to create or modify the models to match their own business processes. These processes can then guide the healthcare professionals as they administer treatment (without forcing them to follow a flow), and capture the actions that did occur so that analytics can determine how to refine the processes to better support patient diagnosis and treatment. This is especially important for complex treatment regimes such as when an unusual infectious disease is diagnosed, which requires both treatment and isolation actions that may not be completely familiar to the hospital staff. Data is fed to and from other hospital systems as part of the processes, so the processes are not executing in isolation from all of the other information about the patient and their care.

For Siemens, BPM is a silver bullet for software development: they can make changes quickly since little is hard-coded, allowing treatment processes to be modified as research and clinical results indicate new treatment methods. In fact, the people who maintain the flows (both at Siemens and their customers) are not developers: they have clinical backgrounds so that they are actually subject matter experts, although are trained on the tools and in a process analyst role rather than medical practitioner role. If more technical integration is required, then developers do get involved, but not for process model changes.

The Siemens product does a significant amount of integration between the executing processes and other systems, such as waiting for and responding to test results, and monitoring when medications are administered or the patient is moved to another location in the hospital. This is where the move to AMX is helping them, since there’s a more direct link to data modeling, organizational models, analytics, event handling from other systems via the ESB, and other functionality in the TIBCO stack, replacing some amount of custom software that they had developed as part of the previous generations of the system. As I’ve mentioned previously, there is no true upgrade from iProcess to AMX/BPM since it’s a completely new platform, so Siemens actually did a vendor evaluation to see if this was an opportunity to switch which product OEMed into their product, and decided to stay with TIBCO. When they roll out the AMX-based version in the months ahead, they will keep the existing iProcess-based system in place for each existing client for a year, with new patient cases being entered on the new system while allowing the existing cases to be worked in place on the old system. Since a case completes when a patient is discharged, there will be very few cases remaining on the iProcess system after a year, which can then be transferred manually to the new system. This migration strategy is far beyond what most companies do when switching BPM platforms, but necessary for Siemens because of the potentially life-threatening (or life-saving) nature of their customers’ processes. This also highlights how the BPMS is used for managing the processes, but not as a final repository for the persistent patient case information: once a case/process instance completes on patient check-out, the necessary information has been pushed to other systems that maintain the permanent record.

Modernizing the healthcare information systems such as what Siemens is doing also opens up the potential for better sharing of medical information (subject to privacy regulations, of course): the existence of an ESB as a basic component means that trusted systems can exchange information, regardless of whether they’re in the same or different organizations. With their hosted software, there’s also the potential to use the Siemens platform as a way for organizations to collaborate; although this isn’t happening now (as far as I can tell), it may be only a matter of time before Siemens is hosting end-to-end healthcare processes with participants from hospitals, speciality clinics and even independent healthcare professionals in a single case to provide the best possible care for a patient.

TIBCO Corporate and Technology Analyst Briefing at TUCON2012

Murray Rode, COO of TIBCO, started the analyst briefings with an overview of technology trends (as we heard this morning, mobile, cloud, social, events) and business trends (loyalty and cross-selling, cost reduction and efficiency gains, risk management and compliance, metrics and analytics) to create the four themes that they’re discussing at this conference: digital customer experience, big data, social collaboration, and consumerization of IT. TIBCO provides a platform of integrated products and functionality in five main areas:

  • Automation, including messaging, SOA, BPM, MDM, and other middleware
  • Event processing, including events/CEP, rules, in-memory data grid and log management
  • Analytics, including visual analysis, data discovery, and statistics
  • Cloud, including private/hybrid model, cloud platform apps, and deployment options
  • Social, including enterprise social media, and collaboration

A bit disappointing to see BPM relegated to being just a piece of the automation middleware, but important to remember that TIBCO is an integration technology company at heart, and that’s ultimately what BPM is to them.

Taking a look at their corporate performance, they have almost $1B in revenue for FY2011, showing growth of 44% over the past two years, with 4,000 customers and 3,500 employees. They continue to invest 14% of revenue into R&D with a 20% increase in headcount, and significant increases in investment in sales and marketing, which is pushing this growth. Their top verticals are financial services and telecom, and while they still do 50% of their business in the Americas, EMEA is at 40%, and APJ making up the other 10% and showing the largest growth. They have a broad core sales force, but have dedicated sales forces for a few specialized products, including Spotfire, tibbr and Nimbus, as well as for vertical industries.

They continue to extend their technology platform through acquisitions and organic growth across all five areas of the platform functionality. They see the automation components as being “large and stable”, meaning we can’t expect to see a lot of new investment here, while the other four areas are all “increasing”. Not too surprising considering that AMX BPM was a fairly recent and major overhaul of their BPM platform and (hopefully) won’t need major rework for a while, and the other areas all include components that would integrate as part of a BPM deployment.

Matt Quinn then reviewed the technology strategy: extending the number of components in the platform as well as deepening the functionality. We heard about some of this earlier, such as the new messaging appliances and Spotfire 5 release, some recent releases of existing platforms such as ActiveSpaces, ActiveMatrix and Business Events, plus some cloud, mobile and social enhancements that will be announced tomorrow so I can’t tell you about them yet.

We also heard a bit more on the rules modeling that I saw before the sessions this morning: it’s their new BPMN modeling for rules. This uses BPMN 1.2 notation to chain together decision tables and other rule components into decision services, which can then be called directly as tasks within a BPMN process model, or exposed as web services (SOAP only for now, but since ActiveMatrix is now supporting REST/JSON, I’m hopeful for this). Sounds a bit weird, but it actually makes sense when you think about how rules are formed into composite decision services.

There was a lot more information about a lot more products, and then my head exploded.

Like others in the audience, I started getting product fatigue, and just picking out details of products that are relevant to me. This really drove home that the TIBCO product portfolio is big and complex, and this might benefit from having a few separate analyst sessions with some sort of product grouping, although there is so much overlap and integration in product areas that I’m not sure how they would sensibly split it up. Even for my area of coverage, there was just too much information to capture, much less absorb.

We finished up with a panel of the top-level TIBCO execs, the first question of which was about how the sales force can even start to comprehend the entire breadth of the product portfolio in order to be successful selling it. This isn’t a problem unique to TIBCO: any broad-based platform vendor such as IBM and Oracle have the same issue. TIBCO’s answer: specialized sales force overlays for specific products and industry verticals, and selling solutions rather than individual products. Both of those work to a certain extent, but often solutions end up being no more than glorified templates developed as sales tools rather than actual solutions, and can lead to more rather than less legacy code.

Because of the broad portfolio, there’s also confusion in the customer base, many of whom see one TIBCO product and have no idea of everything else that TIBCO does. Since TIBCO is not quite the household name like IBM or Oracle, companies don’t necessarily know that TIBCO has other things to offer. One of my banking clients, on hearing that I am at the TIBCO conference this week, emailed “Heard of them as a player in the Cloud Computing space.  What’s different or unique about them vs others?” Yes, they play in the cloud. But that’s hardly what you would expect a bank (that uses very little cloud infrastructure, and likely does have some TIBCO products installed somewhere) to think of first when you mention TIBCO.

TIBCO TUCON2012 Day 1 Keynotes, Part 2: Big Honking Data

Back from the mid-morning break, CMO Raj Verma shifted gears from customer experience management to look at one of the other factors introduced in the first part of the session: big data.

Matt Quinn was back to talk about big data: in some ways, this isn’t new, since there has been a lot of data within enterprises for many years. What’s changed is that we now have the tools to deal with it, both in place and in motion, to find the patterns hiding within it through cleansing and transformation. He makes a sports analogy, saying that a game is not just about the final score, but about all of the events that happen to make up the entire game; similarly, it is not sufficient any more to just measure outcomes in business transactions, you have to monitor patterns in the event streams and combine that with historical data to make the best possible decisions about what is happening right now. He referred to this combination of event processing and analytics as closing the loop between data in motion and data at rest. TIBCO provides a number of products that combine to handle big data: not just CEP, but ActiveSpaces (the in-memory data grid) to enable realtime processing, Spotfire for visual analytics and integration with Hadoop.

We saw a demo of LogLogic, recently acquired by TIBCO, which provides analytics and event detection on server logs. This might sound like a bit of a boring topic, but I’m totally on with this: too many companies just turn off logging on their servers because it generates too many events that they just can’t do anything with, and it impacts performance since logging is done on the operational server. LogLogic’s appliance can collect enormous amounts of log data, detect unusual events based on various rules, and integrate with Spotfire for visualization of potential security threats.

Mark Lorion, CMO for TIBCO Spotfire, came up to announce Spotfire 5, with a complete overhaul to the analytics engine, and including the industry’s first enterprise runtime for the R statistical language, providing 10 times the performance of the open source R project for predictive analytics. Self-service predictive analytics, ftw. They are also going beyond in-memory, integrating with Teradata, Oracle and Microsoft SQL Server for in-database analysis. With Teradata horsepower behind it – today’s announcement of Spotfire being optimized for in-database computation on Teradata – you can now do near-realtime exploration and visualization of some shocking amounts of data. Brad Hopper gave us a great Spotfire demo, not something that most TUCON attendees are used to seeing on the main stage.

Rob Friel, CEO of PerkinElmer, took the stage to talk about how they are using big data and analytics in their scientific innovations in life sciences: screening patient data, environmental samples, human genomes, and drug trials to detect patterns that can improve quality of life in some way. They screened 31 million babies born last year (one in four around the globe) through the standard heel-prick blood test, and detected 18,000 with otherwise undiagnosed disorders that could be cured or treated. Their instrumentation is key in acquiring all the data, but once it’s there, tools such as Spotfire empower their scientists to discover and act on what they find in the data. Just as MGM Grand is delivering unique experiences to each customer, PerkinElmer is trying to enable personalized health monitoring and care for each patient.

To wrap up the big data section, Denny Page, TIBCO’s VP of Engineering, came on stage with his new hardware babies: a FTL Message switch and an EMS appliance, both to be available by the end of November 2012.

For the final part of the day 1 keynotes, we heard from an innovators’ panel of Scott McNealy (founder of Sun Microsystems, now chairman of Wayin), Tom Siebel (founder of Siebel Systems, now at C3 Energy where they are using TIBCO for energy usage analytics), Vivek Ranadivé, and KR Sridhar (CEO of Bloom Energy), chaired by David Kirkpatrick. Interesting and wide-ranging discussion about big data, analytics, sentiment analysis, enterprise social media, making data actionable, the internet of things and how a low barrier to platform exit drives innovation. The panel thinks that the best things in tech are yet to come, and I’m in agreement, although those who are paranoid about the impact of big data on their privacy should be very, very afraid.

I’ll be blogging from the analyst event for the rest of the day: we have corporate and technology briefings from the TIBCO execs plus some 1:1 sessions. No pool time for me today!

TIBCO TUCON2012 Day 1 Keynotes, Part 1

The keynotes started with TIBCO’s CEO, Vivek Ranadivé, talking about the forces driving change: a massive explosion of data (big data), the emergence of mobility, the emergence of platforms, the rise of Asia (he referenced the Gangnam Style video, although did not actually do the dance), and how math is trumping science (e.g., the detection and exploitation of patterns). The ability to harness these forces and produce extreme value is a competitive differentiator, and is working for companies like Apple and Amazon.

Raj Verma, TIBCO’s CMO, was up next, continuing the message of how fast things are changing: more iPhones were sold over the past few days than babies were born worldwide, and Amazon added more computing capacity last night than they had in total in 2001. He (re)introduced their concept of the two-second advantage – the right information a little bit before an event is worth infinitely more than any amount of information after the event – enabled by an event-enabled enterprise (or E3, supported by, of course, TIBCO infrastructure). Regardless of whether or not you use TIBCO products, this is a key point: if you’re going to exploit the massive amounts of data being generated today in order to produce extreme value, you’re going to need to be an event-enabled enterprise, responding to events rather than just measuring outcomes after the fact.

He discussed the intersection of four forces: cloud, big data, social collaboration and mobility. This is not a unique message – every vendor, analyst and consultant are talking about this – but he dug into some of these in detail: mobile, for example, is no longer discretionary, even (or maybe especially) in countries where food and resources are scarce. The four of these together all overlap in the consumerization of IT, and are reshaping enterprise IT. A key corporate change driven by these is customer experience management: becoming the brand that customers think of first when the product class is mentioned, and turning customers into fans. Digital marketing, properly done, turns your business into a social network, and turns customer management into fan management.

Matt Quinn, CTO, continued the idea of turning customers into fans, and solidifying customer loyalty. To do this, he introduced TIBCO’s “billion dollar backend” with its platform components of automation, event processing, analytics, cloud and social, and hosted a series of speakers on the subject of customer experience management.

We then heard from a customer, Chris Nordling, EVP of Operations and CIO of MGM Resorts and CityCenter, who use TIBCO for their MLife customer experience management/loyalty program. Their vision is to track everything about you from your gambling wins/losses to your preferences in restaurants and entertainment, and use that to build personalized experiences on the fly. By capturing the flow of big data and responding to events in realtime, the technology provides their marketing team with the ability to provide a zero-friction offer to each customer individually before they even know that they want something: offering reduced entertainment tickets just as you’re finishing a big losing streak at the blackjack tables, for example. It’s a bit creepy, but at the same time, has the potential to provide a better customer experience. Just a bit of insight into what they’re spending that outrageous $25/day resort fee on.

Quinn came back to have a discussion with one of their “loyalty scientists” (really??) about Loyalty Lab, TIBCO’s platform/service for loyalty management, which is all about analyzing events and data in realtime, and providing “audience of one” service and offerings. Traditional loyalty programs were transaction-based, but today’s loyalty programs are much more about providing a more holistic view of the customer. This can include not just events that happen in a company’s own systems, but include external social media information, such as the customer’s tweets. I know all about that.

Another customer, Rick Welts of the Golden State Warriors (who, ironically, play at the Oracle stadium) talked about not just customer loyalty management, but the Moneyball-style analytics that they apply to players on a very granular scale: each play of each game is captured and analyzed to maximize performance. They’re also using their mobile app for a variety of customer service initiatives, from on-premise seat upgrades to ordering food directly from your seat in the stadium.

Mid-morning break, and I’ll continue afterwards.

As an aside, I’m not usually wide awake enough to get much out of the breakfast-in-the-showcase walkabout, but this morning prior to the opening sessions, I did have a chance to see the new TIBCO decision services integrated into BPM, also available as standalone services. Looked cool, more on that later.

IBM Impact Day 2: Engage. Extend. Succeed.

Phil Gilbert spoke at the main tent session this morning, summarizing how they announced IBM BPM as a unified offering at last year’s Impact, and since then they’ve combined Business Events and ILOG to form IBM ODM (operational decision management). Business process and decision management provide visibility and governance, forming a conduit to provide information about transactions and data to people who need to access it. IBM claims to have the broadest, most integrated process portfolio, having taken a few dozen products and turned them into two products; Phil was quick to shoot down the idea that this is a disjointed, non-integrated collection of tools, referring to it instead as a “loosely coupled integration architecture”. Whatever.

Around those two core products (or product assemblies) are links to other enterprise tools – Tivoli, MDM, ECM and SAP – forming the heart of business processes and system orchestration. In version 8 of BPM and ODM, they’ve added collaboration, which is the third key imperative for business alongside visibility and governance.

We saw a demo of the new capabilities, most of which I talked about in yesterday’s post. For ODM, that included the new decision console (social activity stream, rules timeline) and global rules search. For BPM, there’s the new socially-aware process portal, which has been created on their publicly-available APIs so that you can roll your own portal with the same level of functionality. There’s searching in the process portal to find tasks easily. The new coach (UI form) designer allows you to create very rich task interfaces more easily, including the sidebar of task/instance details, instance-specific activity stream, and experts available for collaboration. They’ve incorporated the real-time collaboration capabilities of Blueworks Live into the BPM coaches to allow someone to request and receive help from an expert, with the user and the expert seeing each other’s inputs synchronously on the form in question. Lastly, Approve/Reject type tasks can be completed in-line directly in the task list, making it much faster to move through a long set of tasks that require only simple responses. He wrapped up with the obligatory iPad demo (have to give him credit for doing that part of the live demo himself, which most VPs wouldn’t consider).

The general session also included presentations of some innovative uses of BPM and ODM by IBM’s customers: Ottawa General Hospital, which has put patient information and processes on an iPad in the doctors’ pockets, and BodyMedia, which captures, analyzes and visualizes a flood of biometric data points gathered by an armband device to assist with a weight loss program.