Is it time to rethink the operating model for search applications?

What search application users want.

Search applications have existed in the enterprise since the 1990s in the form of text search for document or content management systems. The bad to mediocre search experience was tolerated by users. Until Google came along. Google delivered such a simple, reliable and consistent search experience that the single mandate from users was simple, whether you were talking about search on an intranet, a website, or a search application.

“I just want search to work like Google.”

But delivering on this goal has become increasingly difficult for enterprises. Let us explore why this is the case, and why it is time to rethink existing operating models for delivering search application services.

The challenge of consistently delivering excellent search experiences.

Enterprise IT departments, committed to optimal search experiences, spend millions on commercial search engines, and often just as much to implement a solution. Open-source search engines like Solr and Elasticsearch revolutionized and now dominate the space. However, they still require hard-to-find search engine experts to build and run search applications. Without constant care and feeding of the applications, the user experience invariably degrades over time.

Google (plus others including Microsoft and Amazon) are also constantly redefining what it means for search to “work like Google”. The search experience is no longer just about typing in a few keywords to get a list of results, hopefully ranked by relevancy. Today’s users have been conditioned to have search applications behave essentially like a Question Answering system, taking natural language queries and delivering results in the form of featured snippets, related images and entity cards. Users also value “People Also Ask” features, which suggest related alternative queries.

In this first example of a common question-like query, “How old is George Clooney”, Google not only provides an exact answer (60 years), but also “People also searched for”, “People also ask”, and a knowledge panel on the subject.

These results – now the expected search engine norm – were presented in seconds and reflect Google’s use of Natural Language Processing (NLP) (to interpret the query), AI / Machine Learning (to infer related questions through the behavior of similar users), and knowledge graphs (to quickly provide the exact answer as well as the information card of related facts).

Make it work like Google search results example with question answering, snippets, People Also Ask, and answer card

Perhaps an even more subtly impressive example is the query “How good is Michael Jordan”.  Rather than a specific fact (like age), the user is asking Google for a more subjective opinion.

While Google has not formed its own opinion, it has examined its vast repository of indexed Michael Jordan pages to try to answer the question using a “snippet” from what it considers a reliable source. And again, Google provides “People also ask” content to help the user further explore the topic.

This experience effectively transforms the traditional search experience for users into an insight or knowledge discovery experience. And while there continues to be much room for improvement and personalization, it is the new experience that users want.

The challenge for companies, then, is to deliver on this expectation with limited expertise and resources.  One approach has been to leverage the increasing portfolio of cloud-hosted services offered by the major cloud providers.

Make it work like Google search results example with qualitative question answering

Cloud-hosted search solutions are not enough.

Enterprises have been trying to find shortcuts around the scarcity of search expertise, leveraging cloud-hosted search solutions from vendors such as Amazon (AWS CloudSearch, Kendra), Microsoft (Azure Cognitive Search),  Google Cloud Search, and Elastic (Elastic Cloud).  The problem with this approach is that it only addresses the problem of outsourcing the hosting and maintenance of underlying search application infrastructure.

Optimized search applications still need search experts familiar not just with search, but with the latest developments in natural language processing, AI / machine learning, and knowledge graphs.  For example, a search application development team must be adept at:

  • Content ingestion and processing (from a wide variety of data source connectors) to enrich content of various types to facilitate indexing.
  • Implementing the latest NLP and machine learning technologies to better interpret natural language queries that are spoken or typed into a search application.
  • Implementing NLP, entity extraction and machine learning technologies to “understand” and classify documents, and to extract “answers” to queries from them.
  • Implementing and populating knowledge graph databases to facilitate “question and answer” user interactions.

Different use cases require further expertise.

Cloud-hosted search solutions may also work fine for website search, but specific domain expertise may be required in the development of knowledge graph models, dictionaries (acronyms, jargon), ontologies, and taxonomies for the following web search use cases.

  • Website search for e-commerce
  • Website search for support portals
  • Website search for government portals
  • Website search for information publishers
  • Search-based applications (e.g. Uber, Google Maps, etc.)
  • Enterprise search for general intranets
  • Enterprise search for niche intranets (research portals etc.)

Behind-the-firewall use cases may highlight such as document-level security.  And hosted search providers cannot help implementers face design questions such as whether or not to implement a “Question Answering” interface, or a more traditional faceted search interface (or some combination of both).

Evolution of “Search as a Service” to a full-service operating model

 As with any other decision about IT infrastructure or business applications, enterprises have to ask themselves these questions when it comes to search applications:

  • How critical is search to our business?
  • Is search one of our organization’s core competencies?
  • Do we have, or can we afford to retain the expertise to implement and maintain state-of-the-art search?

Depending on your answers to these questions, perhaps it is time to consider an operating model where “search as a service” means entrusting your search bar – from ingestion, to query, to answer – to a technology and services firm that can deliver the user experience you want for your customers and employees.

While that idea may seem far-reaching, consider that:

  • Cloud-based infrastructure providers have made private data centers more or less obsolete.
  • Software as a Service exploded as an industry because companies were willing to entrust providers like Salesforce to operate their entire CRM and sales automation applications.
  • Many enterprises – regardless of size – now fully entrust third-party service providers for payroll, benefits, and accounting services.

In summary, Pureinsights believes Search as a Service is not just about solving the problem with a “hosted search solution”.  It should include:

  • A “full service” solution which includes people who know search to run the software, content ingestion, tuning, search quality analysis, and constant improvements to the user experience.
  • A flexible, modular architecture that allows you to leverage the technologies of your choice, whether it be a new or existing investment.
  • Experts that understand not only the latest in search technologies, but also natural language processing, AI / machine learning, and knowledge graphs.
  • All wrapped into a single monthly fee with metrics for performance measurement. That is different to the regular “search as a service” provider.

Pureinsights believes this is the evolution of Search as a Service.  We are ready to partner in this new operating model with companies for whom search is as a core business function, but not a core long-term competency. We also understand that it may take some time for your organization to assess whether or not you are ready to adopt a new trend in how you manage your search applications.

You can get started with us with a Search Application Assessment, where we can examine the current state of your search applications and help you develop a roadmap for improving them. If and when that roadmap leads you to our complete vision of Search as a Service, Pureinsights will be ready.

If you would like to learn more, please CONTACT US  today for a risk-free consultation.