Enterprise Search – Information Accessibility by Design or Technology?

flickr.com/allaboutgeorge

You’re the user experience designer hired in to give shape to a new enterprise search solution. Your client has already decided upon a technology vendor, and has purchased a license. The responsibility now rests with you to provide employees or customers with the best possible information access. Will you design the solution around the assumption that IR algorithms are able to directly deliver the relevant documents in response to most queries, without much further involvement from the user? Or will you design for optimal communication, making the user more responsible for retrieving relevant documents through engaging interaction? Since I don’t believe in technological silver bullets, I argue that good design takes you further for less money when you’re aiming for higher information accessibility.

This post is a continuation of my previous posts on a topology of search concepts, and mediation in information retrieval. I have received a lot of valuable feedback, and one reader commented specifically on my use of the term information accessibility as one of the dimensions for classifying approaches to search. Here’s a snippet from one of the emails I received (reproduced with permission):

When we came up with the definition of accessibility, we also very clearly spoke about the cost required to reach the resource (i.e., user effort to find the document). You would need to compare the costs that users associate with looking through a linear ranked list, versus reading the facet names and then interpreting them and then making a choice and then potentially not finding the document. This across a wide range of potential queries. Only then can you say that one form makes something more accessible than the other.

We were at this point discussing how the definition of accessibility in information retrieval applies to pure best-first (linearly ranked) search vs. faceted search. Put in other words, will a clever ranking algorithm generally provide a shorter path to the document sought by the user, or is the user generally a better judge of relevance? What I’m trying to understand from this discussion as a user experience designer, is how my design considerations may effect the final outcome to the better for businesses and users. I’m also aware that my chain of reasoning does not qualify as proper scientific argumentation.

Technology – What to Reasonably Expect
Many enterprise search customers acquire a search platform for the purpose of implementing search for a web site, an e-commerce portal, or several company systems like email and CRM. Whether the brand is Google, FAST/Microsoft, Autonomy, Endeca or Lucene, these platforms and their ranking algorithms must be tailored to the specific information needs of the company’s employees of customers. If you spent a lot of money on that software license, it’s reasonable to have high expectations for what this magical search box can conjure up. But how much effort will it take to fine-tune the relevance ranking, and can you expect it to perform top-notch across a wide range of potential queries?

There’s little doubt that web search engines like Google, Microsoft Live and Yahoo spend an awful lot of time and money perfecting their ranking algorithms, simply because the credibility of their brands depend on always delivering the best results first. But chances are that your enterprise search client is not able to pour that kind of money into all kinds of relevance trickery. With that in mind, can we reasonably expect ranking algorithms to outperform humans in terms of efficient information seeking? I believe that faceted search holds the upper hand on linear ranked lists (possibly combined with successive query reformation) when accessibility is considered.

Design – How Far Can You Get
I have to admit that I’m partial to set retrieval and faceted search. I believe strongly in solving information seeking problems with good user experience research and design, and have what I like to believe is a healthy skepticism towards technological silver bullets. It’s my experience that some up-front design gives a better return on investment than excessive technical tinkering later on.

A critical step in the design of faceted search is to find meaningful facets that convey a strong scent of information, and the only sure path to well-design facets goes through extensive user research. The set of facets and their values must correspond to the mental model of the users, and it must be immediately understandable to them what results they should expect upon making a selection. Well-designed facets also provide more support to the user as compared to multiple queries / query reformulation, which is more of a guessing game.

When a user reads and interprets the names of facet values, makes a choice and not finds the document he or she was looking for, I think one of 2 things may have happened:

1. Either the facets are poorly mapped to the mental model of the user.
2. Or the sought-after document simply doesn’t exist.

In the first case, a linear ranked best-first list would’ve been less misleading and would probably have served the user better. That could possibly have provided a shorter path, resulting in higher accessibility. In the second case, the user got early feedback that the search was futile, and that his or her time could be better spent looking somewhere else. In any case, I believe well-designed faceted search is very likely to provide a high level of information accessibility.

A Case of Ingenious vs. Diligent Search?
I have previously written about 4 different approaches to search user experience, where Ingenious and Diligent Search are descriptions of concepts and technologies that bring about high accessibility, with the difference that some are more algorithmically powered, while others are more user powered. I consider the Ingenious approach to include Google’s Universal Search, clustering, LSI, question answering / NLP, search intent analysis and more, where algorithms primarily make the judgment about what is relevant. With the Diligent approach, on the other hand, it’s the user who primarily makes this judgment, which includes concepts like set retrieval and faceted search.

I’m a search user experience designer, and I believe that a well-designed (and possibly low-tech) enterprise search can provide a better return on investment for businesses aiming to provide employees or customers with the best possible information access.