Column 2

Josh Ross did a lot of video interviews here at FASTforward, including this one with me talking about search in the enterprise as part of a content management strategy.

Time out for a video interview (which will appear on the FASTforward blog sometime today or tomorrow), then back to the business productivity track to hear Auli Ellä of Orion, a Finnish pharmaceutical, discuss their enterprise search implementation. As a research-based company, internal users need to be able to find information both internally – in a Documentum document management system, eRooms and the intranet – and externally.

As she put it, it’s not about searching, it’s about finding. Prior to implementing FAST, users didn’t know which system to look in or what metadata to use, and failed searches (where they didn’t find what they needed) took an average of an hour each, which tells you how them cost-justified their enterprise search project.

They started with a pilot enterprise search project that used a small selection of internal and external information sources accessed by a select group of R&D and business users, and used that to prove their business case and gain management acceptance.

She outlined two major benefits that they’re seeing from enterprise search: it supports decision-making, and supports innovation. It saves them a lot of time spent searching internal and external resources, but also adds value to search results through increased accuracy and reliability, and the ability to filter, categorize and drill-down into the results. Information is re-used in place, not copied between systems, using the existing metadata from the source systems both for indexing and for filtering results. This provided an easier and faster search that is at least as reliable as a single source search, and although search results are federated, the access rights of the original information source are respected such that a user can’t see search results for content that they would not have access to in the source system. Users could create and save searches, making it even faster to locate frequently-accessed information.

The user feedback was impressive: in fact, some users thought that it couldn’t be real because it was too fast.

She had some lessons learned for implementing enterprise search:

• It’s essential to start with a pilot, doing a concrete proof of concept, then implementing iteratively
• It’s important to have a diverse group of pilot users, not just highly-skilled knowledge workers
• Keep it simple with the filters and navigation: too much sophisticated can just confuse things
• Be prepared for resistance, and combat that by offering tips and tricks for using the system more effectively

Going back to the goal of supporting decision-making, it’s helping them by allowing users to see all relevant information available on one screen when, for example, gathering materials for a meeting. On the innovation side, they’re starting to have some unexpected results – which drive innovation – becoming visible more easily by using filters such as content authors.

They’re now adding more internal and external information sources, and improving functionality based on feedback from the users for things such as selecting a single information source when the user does know where the content is – which is an interesting request, when you consider that that means that the users would rather search in FAST than in the underlying system due to speed, ease of use and functionality. Reliability and speed are critical for search, of course: without both of those, the users will reject it.

In the future, they’ll be developing new filters and search profiles, adding structured content, and incorporating search behind the scenes in portals to display relevant information (like the company cafeteria menu, which is currently the top search in their system). As this happens, however, they are aware that this continuous development can make it easier for the user, but increases the amount of work in the background by the information management people.

Her conclusions: if done well, search can become a personalized desktop where discovering, re-using and refining information is business as usual. It’s all about turning information into knowledge and action.

I’m a fan of Charlene Li (formerly a Forrester analyst on social media, now independent), so was excited to see her kicking off the second half of the morning by talking about the connection between search and social technologies. Her main topic is how companies are being transformed by social technologies, starting with the statement that social media isn’t about the specific sites that are hot right now, but about engaging and forming relationships with your customers. This requires learning from the community of your customers and prospective customers by listening to what they want to do:

• Instead of one-way marketing, start a conversation with your customers on Facebook, like H&R Block does.
• Instead of reactive customer service, watch for references to problems with your brand on Twitter, and engage the Twitterers directly, like Comcast does.
• Instead of a well-hidden email link on your website for customer feedback, create a part of your own site that allows customers to make public suggestions for how to do things better, and allow people to vote on the suggestions, like Starbucks does on MyStarbucks.com.
• Instead of waiting for prospective employees to come to you, find the new graduates on Facebook, like Ernst and Young does.
• Instead of faceless press releases, have executives write a public blog, like Beth Israel’s CEO does.

Companies will be transformed from the outside in, by listening to their customers and communities.

She then brought this back to how social technologies are impacting search: search now has to be able to reach all of this information on different platforms and sites, as well as using your social graph to assist with search and prioritizing results.

From an enterprise standpoint, companies are integrating external search into their support site search results in order to enrich the results. Out on the web in general, however, there’s been an explosion of web tools to search social media, and categorize the results: not just plain old Google searches, but Techmeme and Technorati to categorize and rank the results. Delicious, a social bookmarking site, tells you how many other people have tagged a specific link, and allows you to traverse any of those people’s links to see if they have other relevant links for the information that you’re trying to find. Twitter allows you to search for tags that people include in the content of their tweets (or for any other term), and FriendFeed provides you with much richer social search features such as filtering certain people in your social graph into or out of the results.

It’s not just search, it’s a combination of search, using your social graph, and active trends on the particular social media site at that moment: searching enhanced with social profiles, even if the profile that you’re using in the context of your search isn’t even in your social graph but has sufficient social capital to influence your decisions. However, most of the social searching is still siloed: if you’re looking for your friends’ opinions on a book, you might have to search on Amazon, Facebook, Twitter and a variety of blogs and personal websites. This is no different than the silos of information that we have within enterprises, and federated search could provide the same degree of benefit as we’re seeing with the implementation of enterprise search to aggregate multiple sources and platforms.

We’re just starting to look at search on the social web: I agree with Charlene that social media is having a big impact on search, and search will have a big impact on how we find things in social media.

Valentin Richter, CEO of Raytion (a FAST partner), talked about a recent study showing that only 10% of users is satisfied with their current intranet search capabilities, and what can be done about it. He spoke about several of the myths associated with enterprise search.

Although he went through 11 myths (without slides!), a key one is one that I’ve heard around here from other people: enterprise search is only a search box on a web page. We had a bit of a glimpse in the previous session about how enterprise search is much more than a search box, and I recall case studies from last year’s conference that showed that some enterprise search applications aren’t even recognized by the users as search. Search can be a tool that’s part of the infrastructure used to build applications within an enterprise, aggregating and integrating data, and presenting different views of data for different internal audiences.

It’s almost 4:30 (7:30 in for those of us in the Eastern Standard Tribe), and for some reason Dan Rasmus, Director of Business Insights at Microsoft, is quoting Schopenhauer to us. We’re still on the big picture part of the conference, but that seems a bit…philosophical.

His point, however, is looking at the new world of business and how companies can navigate through it: globalization versus resetting borders, and centralization versus distributed work networks being two major axes of interest. Add to that the current economic climate, consumer pressure for green products and myriad other factors, and you start to get an idea of what’s facing companies today. Business models are changing, at least in part in response to some of these forces.

Globalization, and what will happen if the upcoming power economies – India and China – decide that they want to pull back and build their own infrastructure instead of servicing the world. Outsourcing, and the impact of managing a freelance workforce. Blending of languages, and the impact on collaboration. The blend of work and personal, such as we see in the online lifestream of our Facebook, LinkedIn, Twitter and blog updates. The blend of data between on-premise, personal and cloud-based. The blend of place, like when read your RSS feeds from your iTouch in bed (okay, guilty), allowing people to set their own boundaries or (if they’re dumb) let their boss set them for them. Blended enterprises through mergers and acquisitions, and the challenges in terms of people and processes.

Lots of changes. Lots of challenges. Not a lot of answers.

He did list some of the skills required for the new world of business:

• Information and media literacy
• Complex communication
• Critical thinking and systems thinking
• Problem identification, formulation and solution
• Creativity and intellectual curiosity
• Interpersonal and collaborative skills
• Self-direction
• Accountability and adaptability
• Social responsibility

I had to laugh when he painted a scenario where a Net Gener goes into his job and whips a monster Vista notebook out of his backpack to compare to the crappy old desktop machine that his employer offers. I have news for Rasmus: that notebook’s going to be a Mac.

Final session of the conference, and I’m in the financial services breakout track to hear Lee Atkinson, SVP at iDNA, and Marc Hebert, CMO of Virtusa, discuss a real-world example of using enterprise search to build a true 360-degree customer view. They started with some background on the problems with enterprise applications and how dynamic business applications (Forrester’s term for composite applications; he’s quoting from a Forrester report) are starting to take on some of the functionality. Enterprise search applications provide a layer on top of the enterprise applications, files and databases to create search solutions such as corporate search or intelligence solutions. They see enterprise search as a key enabler of dynamic business applications in order to provide both speed and agility of those applications.

The case study that they presented is the implementation of enterprise search in a top ten global bank, providing fast retrieval of structured and unstructured data, and integration with business process management (not a term that I expected to hear at this conference!) to manage events being generated by the system. Their technology platform included enterprise search, BPM, SOA and enterprise portals, and they addressed the 360-degree customer view and a GRC (governance, risk and compliance) event management system. They didn’t use search because of its inherent properties — in fact, the users don’t even see this as a search application, and the applications look like standard structured data extracted from operational systems — they use it to extract information quickly from heterogeneous data sources. Prior to this, the bank had point solutions, multiple data warehouses, and no commonality between systems and databases to provide any sort of consolidated customer view. They needed to integrate data from multiple sources, and also meet their compliance regulations.

They started by providing a compliance and risk view, plus product and customer profiling, in 2005, then expanded to include event-driven business processes and exception management in 2006. In 2007, they brought in the single customer view and more advanced business processes and analysis. Of course, there is no such thing as a single view of a customer in a large organization: different business areas (compliance, operations) have different needs, and each user type’s customized view of a customer is actually a subset of the entire customer model, with other supplementary information pulled in and additional analysis added appropriate to the task at hand. The result is a number of different end-to-end processes, such as know-your-client, anti-money laundering and sanction lists in account opening.

For them, search technology is a way to integrate legacy systems — not an application that I thought of when I considered search — although it requires a deep knowledge of the business domain and the nature of the underlying data. Once the core integration structure has been created, additional data sets and applications can be added quickly and at a fairly low cost. The focus on event-driven business applications based on search results is where search really contributes value to their applications.

General sessions are done for the conference, but there’s still some interesting breakout sessions. I’ve moved to the financial services track to hear Ilkka Korkiakoski, a VP at TietoEnator (a European IT services firm) talk about the challenges for the financial services industry as it moves to real-time multichannel customer interaction. Korkiakoski was previously with the OP Bank Group, so he understands this not just from the vendor side, but from the customer side as well; TietoEnator prides themselves on their depth of vertical experience in banking and other specific industries.

He started with a perspective on what’s happening in the financial services industry, particularly in Europe:

• Consumer behavior is fundamentally changing
• The expected response time of a request is now minutes, not days
• The number of customer requests and transactions sent to the back office is 10x that of 1998
• The “Business to Consumer” model will die, and be replaced with “Consumer to Business”
• There has been a shift in strategy within banks from product leadership (1995-2000) to operational excellence (2000-2005) to customer intimacy (2005-now)

The first generation of digital services were created for the digital immigrants, whereas the second generation of digital services are being created for digital natives: those young enough that web technology has always been a part of their life. The first generation included (at the time) innovative — for the time — services such as self service and online access to information and services while still maintaining silos by channel, whereas the second generation will mix business and social components and destroy the barriers between channels for a multichannel consumer experience. To support this, the people within the bank must adapt, and new tools must be created.

Looking at the high-level trends:

• Digital service penetration: basic banking services continue moving to digital channels
• Customer retention: each interaction counts and service experience will be a critical differentiation factor
• Industrialization: growing digitalization and complexity of products and services drive industrialization
• Regulation puts pressure on margins: many initiatives, including SEPA, Basel II, MiFiD, IFRS, Solvency II and SOX
• Industry consolidation: a lot of issues related to integration, economies of scale and differentiation
• Service and product innovations: smaller segments, new types of offerings and bundling will grow rapidly

Moving on to look at opportunities for enterprise search in financial services:

• Customer knowledge, to augment the information in a CRM system
• Sales and customer service desktops, to assist with answering queries
• Service experience in real-time multichannel processes, to supplement the structured data offered to a customer
• Market and business intelligence
• Regulatory and compliance
• Fraud detection, where search and rules combine in complex event processing

Electronic banking has evolved from the basic ATMs of the early 1980s to web banking in the mid 1990s to personalized services in the early 2000s to interactive multichannel services today. As Europe moved to its reliance on internet banking — over 80% of retail transactions are conducted online, with the remainder split between in-branch and the call center — a single platform is required to support the multiple channels, such that the same information and functionality is available regardless of whether the customer is interacting with the bank on the web, in person or over the phone.

The result for the web-based user interaction is a customized portal view that includes supporting information personalized for that customer via contextual search, in addition to the expected structured transactional data.

And now a speaker who we’ve all been waiting for: J.P. Rangaswami, CIO of British Telecom, but likely better known for his blog about information, confused of calcutta. He spoke on the new polarization: how the customer is gaining control, and the impact on corporate IT departments.

We’ve changed from the corporate IT environment of 20 years ago, which was completely permission-based (as in, you had to ask permission to get access to information and applications) and created an arrogant, ivory tower attitude within IT departments. Today, the inmates are gradually taking over the asylum, and the rise in user freedom — and the desire for still more freedom — is sending a chill through traditional IT types. Users are finding ways to work around corporate restrictions to make applications from Skype to Facebook a part of their work life.

Part of a driver for this is the increase in the amount of unstructured information, since many traditional enterprise applications focused almost exclusively on structured information. The first step in dealing with unstructured information in the enterprise came through the spread of desktop applications; the next step, which we’re undergoing now, is focused on dealing with unstructured information on the web. I’m old enough to remember dedicated word processing departments within companies, which centralized control of content creation, and the pain of transitioning to decentralized desktop word processing; someday, we’ll reminisce about the days when the IT department blocked access to Facebook from inside the enterprise.

Another driver for the user revolution that we’re undergoing is the rise of peer-to-peer rather than hub-and-spoke as a model for content creation and distribution. We’re all content authors now. We all recommend content to our friends.

A major hindrance to advancing the user revolution — the polarization that Rangaswami is referring to — is the discounting of youth due to lack of experience, when expertise isn’t actually connected to age in the rise of the technology and communities behind the revolution. A second dimension of the polarization is the democratization of participation, with some people rejecting the idea that good content can be created by amateurs, even in the face of success stories like Wikipedia and the whole open source community. The third dimension of polarization is about time, specifically the speed of implementation, with barriers against the use of products that are proudly still in alpha or beta release, such as Gmail and Google Apps.

The new generation of users — and the new generation of Web 2.0 applications — ignore documentation in the form of manuals, but focuses on self-evident functionality and just knowing what the next step to take.

This genie isn’t going back into the bottle: the power is now in the hands of the consumer, and it’s time to get over the polarization and embrace the user-centered world.

Timo Teimonen of StarCut, a mobile media publisher, was up to discuss how the mobile internet is transforming. The incredible growth in mobile internet access over the past few years means that people are no longer happy with seeing full-format web pages on their small screen, but want content tailored to the smaller form factor. A huge portion of the market, mostly in Asia, experience the internet only on their mobile, and almost never on a PC except perhaps in a workplace environment. This creates challenges not just for content presentation, but also for search and monetization, with the added issues of localization and contextualization: creating search results and ad content, for example, that’s based on the user’s current location.

The heart of mobile value is based on contextuality: mobile users are willing to trade a bit of their location and preferences privacy in order to get more targeted product and service information. The mobile phone is the bit of information access that people carry with them anywhere, providing the perfect platform for extremely local advertising based on relationships and information about the user. Traditional advertising is extremely inefficient; mobile advertising can be much more efficient at delivering relevant content to people based on where they are and what they’re looking for. In turn, mobile content providers have an incredible amount of information about how people user their content: the Zagat mobile site, which is based on StarCut’s platform, can gather information about what type of restaurant that a customer searches for, be able to provide recommendations based on location and preferences, and even allow for online reservations through the mobile device. In this way, mobile allows advertising and other content to be directly connected to physical retail: a true bridging from the virtual to physical worlds.

I can definitely relate to this, since I live within close proximity of my Blackberry at all time: a device that I use primarily for data access (email, web access, and specialized services such as Google Maps) rather than voice. Mobile content is a big part of my life, and the lives of many of the people with whom I interact, and I believe the quality of this content will greatly improve as localization and contextualization improves.

Sue Feldman, VP of search and digital marketplace technologies at IDC, spoke about taking the pulse of the digital marketplace. She started by defining the digital marketplace as an online gathering place of buyers and sellers, with particular relationships and dynamics between the types of participants. Users interact with 3 types of sites: gateways (a starting point or index to the web, funded by ad revenue; these are portals and search engines such as Yahoo and Google), hubs (aggregators with high-value tools and trusted content, funded by high-value ad networks because of their trusted position as selectors of content, such as eBay or Amazon), and nodes (communities with few outward connections, such as direct consumer-facing corporate websites).

They’ve created a model of the dynamics of feedback in the digital economy: why people visit certain sites, and why they move from one site to another. Along the way, they found a few surprises:

• The web is fragmenting, not consolidating
• More queries are happening on the web than are reported
• Only a third of queries go through web search engines (and hence are monetized); the rest are directly on destination and speciality sites
• More queries go to Baidu than to Google, Yahoo and Microsoft combined, but all are in Chinese (I have no idea why IDC would find this surprising, except she does admit that prior to hiring an analyst covering China, they had an “embarrassingly North American focus” — what I take to actually mean a US focus)

Looking at the trends that are emerging, they see increasing user sophistication, with users participating in social applications and expecting personalized and localized information. There’s a greater demand for selection on trust, as information overload drives users to speciality sites that pre-select information targeted at their needs. They’re seeing a rise in ad networks that provide context-specific and relevant ads, and a rise in social networks that filter the overwhelming amount of content through a trusted network of friends.

There’s some powerful new technology and infrastructure that’s underlying these trends in the digital marketplace, from personalization and recommendation engines  to user data mining tools. To conduct business on the web, you need to exploit this infrastructure, as well as understanding the long tail effects that impact what you offer to your customers. For traditional businesses, the web is an additional channel, not a replacement for other media.

The dynamics of the digital marketplace are changing quickly, but the roles are becoming defined between the gateways, aggregators and nodes. The central role of the user cannot be overestimated, both on an individual level and as a group. Audiences are attracted not just by useful content, but also by compelling tools and usefulness. Selection and trust are key components for the aggregators and nodes, and targeted communities are replacing generic sites.