We shall discover the best solutions together.

Enterprise search is a wicked problem – a problem that can’t be solved merely with clever algorithms, beautiful code and lots of data. You need a proper process for search user experience design when dealing with high ambitions, diverging goals, and tight budgets and deadline – not to forget about potentially weak corporate IT infrastructure, faulty data and convoluted security mechanisms. It takes dialog and cross-diciplinary collaboration with stakeholders, users and techies to explore all the requirements for a possibly great enterprise search solution. My main goal with search user experience design is to facilitate, contribute to, and document this dialog throughout the entire timespan of enterprise search projects.

I was recently invited by Tony Russell-Rose to speak at the ECIR 2010 conference about my work with search user experience, and the process methodology I have developed together with my colleagues as Comperio. Tony himself held a tutorial on search user experience design which I regret not being able to attend. I admire the work he and his collegues at Endeca are pouring into their UI Design Pattern Library, which I hope they will open up to the public sometime soon.

I also had the pleasure to befriend Stefan Olafsson and Tyler Tate at TwigKit, two excellent guys with a passion and talent for great search design. Their software is worth checking out if you’re considering to implement a search application frontend in Java, as it will certainly speed up your development process.

Tyler and Stefan arranged for us to have a meetup with the Enterprise Search London group the evening before ECIR, at a local pub in London. My presentation was the alibi of the evening, and after that we had a great discussion about our own experiences with search design in the enterprise. Thank you everybody who joined us there.

I have published the presentation slides on SlideShare, in case you want to have a look. Feel free to re-distribute and re-use whole or parts of the presentation, provided that you respect the conditions of the Creative Commons license.

Enjoy!

Workflow

On April 7th, 2010 (1pm Pacific Time) I will be presenting at a Microsoft TechNet webcast together with a colleague. The topic at hand is: How can we use search engines as a source of content for internet facing portals.

TechNet Webcast: SharePoint 2010: Search-Driven Portals (Level 200)

Websites today are often statically authored, meaning the components on the page have been put there by a person, and they show the same content to all visitors.

What if you could leverage the user’s context and intent in a better fashion together with the features of an enterprise search engine? Then package this in reusable business components in order to author better content to the end-user?

In this webcast we will propose FAST together with SharePoint as a starting platform to achieve this. By combining your business logic in workflows you can reuse logic and content across pages in a more dynamic way. Think of search as a more intelligent CMS tool, and define search as lookups to the source necessary to achieve a task, not just queries against the search engine.

(This post has is cross-posted at Tech And Me)

If you found this post interesting or helpful, you may want to read FASTforward’09 – Day 3: Which Path to The Future Will You Take?

flickr.com/adamcrowe

From UIE Brain Sparks: When you kick off a project right, everything is much easier. When that doesn’t happen, the team pays the price. We’ve all seen projects where, part way in, a well-intentioned executive derailed the team by changing the direction. To prevent this, we want to put everyone with the power to take the project off course, on to the same course.

Tamara Adlin has developed a great technique to make that alignment happen, which she calls Ad Hoc Personas. Her method, borrowed from research-based personas, creates characters out of information the organization already has at their fingertips. They’re inexpensive and easy to create, ensuring a customer focus from the very start of the project. This remix of tweets from “The Power of Ad-Hoc Personas”, a UIE virtual seminar by Tamara herself, tells you how.

I missed out on Tamara’s webinar, unfortunately, and decided once again to compile a summary from what others have tweeted about it using the hashtag #uievs. Last time I did a similar remix of Steve Portigal’s Deep Dive Interviewing Secrets. It’s a great opportunity for me to learn, working with the material this way. I don’t intend to make a habit of it though, so don’t expect a 3rd remix anytime soon.

But for now, here’s my remix of tweets by those attending Tamara Adlin’s UIE Webinar on “The Power of Ad Hoc Personas: Truly Practical Methods to Get Your Organization On the Same Page”. Enjoy!

The Rules of Persona Work

• Rule 1: If you don’t have clear goals, the persona effort won’t work. The dirty little secret: no one knows what the company’s goals are.
• Rule 2: Get “where the buck stops” stakeholders involved from the get-go. The executives must be involved from the start.
• Rule 3: You must align personas with goals.
• Rule 4: Use @tamaraadlin’s magical persona template.
• Rule 5: Use tools that force people to use the personas.
• Rule 6: Create stories, not solutions.

The Business Value of Ad-Hoc Personas

• Personas are a valuable part of the process whether you’re working on the inside or the outside of an organization, big or small.
• Promises practical ways to get the whole organization on the same page, hopefully avoiding/preventing seagull management. “Swoop and poop” or “Seagull Management” = when executives fly over the project and crap all over it, then fly away.
• “Myth: If you don’t have data, personas are worthless.”
• Ad-hoc personas are the same as data-driven personas except they’re created in direct collaboration with high-level stakeholders. Unlike data-drive personas, ad-hoc personas are a focus and communication tool first, and a product design tool second.
• If you don’t have clear goals, don’t bother!!
• Ad-hoc personas can (and should) be created before you collect any more data about your users or customers. Most companies aren’t suffering from lack of data. They’re suffering from lack of focus. “Most companies are so confused, the last thing they need is more data.” Creating ad-hoc personas is a useful exercise to untangle an executive team and refocus language on users – instead of internal speak.
• Why care? Personas are already floating around your organization, and they’re probably wrong. Personas can solve unclear business objectives. Disorganized assumptions are like ghosts. They will haunt your project. Use ad-hoc personas to bring clarity and focus to your organization.
• When developing rich experiences across multiple channels with a large team, few people have the same users in mind when making decisions. People think about people, so the team has “hidden personas” in their head. Hidden personas wreak havoc on products. People are making small compromises that end up becoming huge problems for lots of people.
• Ad hoc personas can be used when time and money doesn’t allow for data to be collected. Collecting more data is expensive and often not very helpful. Ad-hoc personas can be created quickly and inexpensively. And chances are you’re not sending people to the moon. Not everything has to be super-complex. Ad-hoc personas solve basic problems.
• So even without data, getting everyone on the same page about who the users are is better than the hidden personas that already exist. Shared, prioritized, well-communicated personas align the whole team – inside and out. Even without data, the picture is much better. Not having data is a lesser evil than not having focus.
• “Barnacle-based” design happens. Over the years, you add more features and clog up the works of something that was once well designed. Product starts out one way, and ends up a totally different animal. A lot of tiny compromises add up to big compromises.
• Corporate Underpants: when your organization structure shows on your website. “Are your corporate underpants showing?”
• Marketing vs. design personas: marketing drives eyeballs to your product, product design moves those eyeballs around once they arrive. A Website is like a party where it’s loud and people are yelling at you saying “look over here”, “no look over here”. Marketing gets consumers to show up at the front door, now how do we answer their questions? Product design is about what happens once people are inside.

Starting the Ad-Hoc Persona Process

• Define goals, value propositions and list of all alternative words for “user” in first meeting. USER is a 4-letter word. Decide and align on your top three business objectives before moving into personas – otherwise they won’t work.
• Identify top three measurable business objectives for the company/project. “What does success look like from a business perspective?” Make sure that each objective is measurable in a specific way. “Increase revenue” and “decrease costs” are not clear, measurable goals. “Increase revenue by X% through online tools” is, however.
• Write down your guesses for the goals and make sure you get them approved by the execs before you move forward with the project. The dirty little secret: No one remembers corporate and brand objectives fully – so make them write them down AGAIN. Don’t confront executives with a need for a list of goals. Give them something to react to.
• Executives need to link personas to goals. If executives create the personas, they will defend them.
• As business goals change, previously developed personas can still hang around but you might need to re-prioritize them accordingly. Personas tend to be evergreen because users’ goals don’t change that much over time.

Design Mapping – Yellow Sticky Exercise

1. Get everyone in the room to write all their goals on post-its, then categorize.
2. Each sticky gets a person and a situation written on it.
3. Quick Sort using the “words for user” as categories.
4. Hard Sort: Cross out old language and reform each sticky into a “I want / I need” statement. Let executives use their own language, then help them adapt it to “I want / I need” statements. This is hard, but it will be a major sea change.
5. Write all the “I want / I need” statements on the whiteboard and look for patterns.
6. Find patterns across the user goals. (You’re doing all this together in a room.) Put like items together, and there are your personas.
7. Sort out the statements and group them, collapsing like-minded goals.
8. Collect needs under each type of person.

Ad-hoc personas = testable hypotheses. It will give you a place to start gathering more data. This process will give you roles and goals which is the first step to developing personas. This exercise is less about accuracy, and more about the team sharing information and the same goals.

Q: Why we should call these “ad-hoc personas” instead of “target audiences”?

• Don’t name the personas until you are comfortable with them. If you’re planning data-driven personas, it’s good not to give ad-hoc personas names/titles in case you need to revise them. If the data changes them drastically you could be set back.
• You really should get everyone in the same room when working on sketch personas. Best to have the conversation together. But calling them “target audiences” might create a turf war with marketing, so instead consider calling them “sketch personas”.

Working With Stakeholders

Ad-hoc personas are less about getting to the right answers, and more about getting the business executives to share how they’re thinking. Get the executives to barf out their assumptions and organize them. Let them prove to themselves that their assumptions are limiting. Get the old ways of thinking out first to get to new – “You’ve gotta get the scribbles out before you can draw.” Let them “bark” now, rather than later. Prove to them who their personas are.

Potential Seagulls/Key Stakeholders:

• Your boss, grand-boss and great-grand-boss. @tamaraadlin refers to your boss’s boss and your boss’s boss’s boss as “grand boss” and “great grand boss”.
• The right stakeholders to work with are the ones with the power to “poop” on your ideas.

The Prioritizing Game

Persona Prioritization Slide from Tamara's Presentation

Use prioritizing game to find out where executives really want the direction of the company to go. Your executives aren’t hiding their goals from you…they don’t even really know them! They’re your #1 user – guide them through the process. Ifexecutive  team can’t agree on the priority of the personas, how are you ever going to design a product they believe in?

1. Each person has points to give to each persona. Ask them: What is the value if you don’t please this user?
2. No two personas can get the same score, as it makes the priorities of team members clearer. Disparate score among the team drives conversation.
3. Answer the following about each persona: “If we don’t make _______ ridiculously happy, we have failed.” All personas are important, but is it more important that Sally be super happy or Carl? That really gets people talking.

Personas Template

• Use a descriptive name for your persona: Carl Quickcash for an ATM user for example.
• Persona elements: Alliterative, descriptive name + Photo + Weighted score/priority + Category (admin, end user etc) + Key Quote + Story + Key Questions + Answers we want to give.
• Make sure the persona’s questions are framed in their terms, not yours. I really like seeing the persona include questions from his/her perspective AND how the company will answer those questions.
• Use quotes to connect the person’s voice to their goals.
• Oh, by the way statements: Statements to meet a goal the user has, but maybe isnt asking us to help with. What we (the organization) can do for her that she wouldn’t even think to ask for.
• Don’t forget to hold onto that weighted score. It can change over time, but it’s important to remind everyone who has priority.

Persona-weighted feature matrix

Persona-Weighted Feature Matrix Slide from Tamara's Presentation

• Persona-weighted features are a nice way to keep the personas alive in a project and ongoing – aligns persona to business priorities.
• Features listed on left, personas as column heads each with a weight.
• Take each feature and plot it against each persona, then score & add up. The weighted score will help you to help an executive make the right decision.
• Score each feature per persona on this scale: 2(LOVE) to -1 (HATE). This will leave you will each feature having a weighted priority/score:
  • 2 = persona will love this feature.
  • 1 = sure, it’s fine, expected.
  • 0 = doesn’t affect the persona.
  • -1 = persona will hate this.
• In the end, the persona that generates the most revenue is the most important in the vast majority of cases. (rule of thumb). Pet features fall to the bottom like rocks. No more flashing widgets that spin! Remember a pile of features, a product does not make.
• Now you have the feature priorities, based on persona priorities, based on business or brand goals, which were approved by the executives. They will still try to poop on you, but at least you can tell them how they helped you get to that solution.

Communicating Persona Prioritization Slide from Tamara's Presentation.

Design Maps

• Personas are the characters, now tell the story: From the user’s logical start to their end – regardless of the bonds of our system. This is similar to scenarios and use cases, but focus on end-to-end stories. Not just little snippets of the experience. Users see it as one experience and you need to understand it and plan for it that way too.
• Break story into: Story Title + User’s Steps + Questions at each step + Assumptions about each step + Design Ideas for each step.
• Using Post-It notes, these stories can be mapped visually to explain design thinking.
• Design Maps flow nicely into wireframes. Each step can be considered as it relates to the system being designed.
• Data is great, but it’s not the only tool in the toolbox. Ad-hoc personas get executives on same page, and help us design a story of the experience. This story crosses functional areas, product descriptions, politics, etc. Personas get you out of your own heads, talking to each other.

Thank you Tamara, and thanks to everybody who tweeted during Tamara’s webinar, and who tagged their tweets with #uievs. I hope you enjoyed this remix

Preview of The Power of Ad-Hoc Personas on  SlideShare

If you found this post interesting or helpful, you may want to read Remixing Deep Dive Interviewing Secrets

http://xkcd.com/125/

Everybody knows that you should ask open and unbiased questions when interviewing users for UX research. The real skill to master for the seasoned researcher/designer is not so much that, but to know what kind of open question to ask, and when to ask them. Some questions may prove to be dead ends, and other may explode in a frenzy of possible follow-up questions. The best questions are intended to reveal information unsaid and undiscovered. But how do you know what’s missing from the interview? What have you not been told so far, and how do you probe into that? This remix of tweets from “Deep Dive Interviewing Secrets”, a UIE virtual seminar by Steve Portigal, gives you some of the answers.

I missed out on Steve’s webinar, unfortunately, and decided to check out what others had tweeted about it using the hashtag #uievs. Luckily, there had been lots of activity and discussion, and I felt that Twitter provided me with quite a comprehensive summary of Steve’s stunning insights in to interview techniques. For my own sake and for future reference, I decided to compile that Twitter timeline in to a short document. But why not share the fruits of knowledge?

So with a little extra effort, here’s my remix of tweets by those attending Steve Portigal’s UIE Webinar on “Deep Dive Interviewing Secrets: Making Sure You Don’t Leave Key Information Behind”. Enjoy!

Tips for Establishing Contact

1. Try to achieve Question > Story, instead of Question > Answer. Lots of question asking can be replaced by letting them tell you. Good moderation means not giving away the answers you are looking for. Your personal style is important to find in developing your methodology and tactics.
2. Beware the temptation of synthesize and build a model on the fly. Remain open to the subject’s model. If you want to learn about a person’s mental model, don’t expose yours first.
3. Talk like your subject talks. If a user mispronounces something, repeat it to keep them comfortable. Avoid them having to disagree with you.
4. Silence is golden – after you ask a question, be silent to allow for a response. You don’t know their style of response. Avoid the “Errrr” nervous trail off if you aren’t confident in the question, wait for a response.
5. Listening: don’t interupt, be silent (even the “hmmm mmmm”), maintain eye contact, nod, and acknowledge e.g. “I see”. If they need the “hmmm mmmm” keep um, but be aware of the interruption potential. After they’ve answered you, be silent another beat so they keep the story going.
6. Don’t let them off the hook when words don’t come easily. Make them make the label instead of agreeing with yours. The person you’re interviewing will “want to do what you want them to do”. If you move on, they move on.
7. Signal your transitions. “I’d like to move on”, “Earlier you mentioned…”, “I’d like to go back to…”.
8. Tactics you may try:
   1. Use the user’s language. E.g. if they say TiVo as “tye-vo” instead of “tee-vo”, follow along.
   2. Ask questions without answers. Don’t suggest answers. Don’t guide the responses or suggest responses. Use open-ended questions: “Can u tell me about your experience using X?” not “What are 3 things you liked about using X?”
   3. Silence. Ask your question and then be silent. Leave time for an answer to form.
9. Steps to letting go of your own perspective:
   1. Admit what you think will happen as a team.
   2. Go to the user’s natural space.
   3. Don’t dress for work.
   4. Make the interview about the subjects – you are their to learn about them.
   5. Be comfortable asking the questions that you already know the answer to, and learn what their answer is. “I wanna know how you do it”.
10. Q: If you want to learn their terminology, how do you do it without using terms? A: point at the object, introduce topic broadly and then “harvest” the terms to use later. Turn it from a test into a conversation with the purpose of learning. “Do you understand these terms” is a test and the person mail fail. “Can you tell me about THIS” (pointing) isn’t a test.

Tips for Building Rapport

1. Building Rapport with Participants: the energy that underlies great interviews.
2. Listening is the rapport engine. It is your responsibility to establish rapport.
3. If you’re going to people’s homes, it helps to be a cat/dog person. Bonding with a pet demonstrates rapport.
4. Some small talk is appropriate, to little is rude, too much is distracting and turns your interview into a social call. This interview isn’t about you – even if something familiar happens. And don’t share about yourself. Avoid the “OMG ME TOO”. Sometimes there are exceptions, when they need permission to continue or you want them to feel more comfortable.
5. Watch your body language – think about the emotional state your position puts you in. Sit to listen. Lean in. Make eye contact. Be interested.
6. Asking follow ups really makes the connections and makes the customers at ease as well. Demonstrate that you are listening by building questions on what they have said.
7. Q: Is rapport building different by phone? A: recreate the “walk down the hall” moment with virtual participants – use small talk but focus on the task.

Interview Guide Tips

1. Field Guide: The more-than-you-can-possibly-ask set of questions.
2. A guide helps you to pre-visualize the session. The act of creating an interview guide can mean that you are so prepared that you don’t need the interview guide. You may not actually use the guide you create – but if you need it, it is there.
3. The interview guide is also a good prop: “Let me see – what else did I want to ask you…”
4. An interview guide would have a list of questions, grouped together.
5. Topics Guide Approach: Identify the headings, the questions are not always needed. Start with a list of topics you’d like to learn about.
6. A more robust guide has questions as they would be asked, the motivation for asking, and any follow-ups.
7. Make it natural, not overwhelming to glance at. A interview guide needs to be something that you can refer to quickly – keep it simple.
8. A version Steve uses most has subhead list of questions as they would be asked, as well as time/order.
9. An interview guide should include Non-Verbal notes like stage directions for the moderator (probe for …, reference the screening call…), and a list of things to be sure you’ve seen and/or photographed: get a face shot, an action shot.
10. Q: How much prep time is needed pre-interview? A: depends on the test, but planning for field work takes at least 2 weeks to define screening, recruit, and define field guide.

Tips for Asking Questions

1. Use Tasks, Participation, Demonstration, and Role Playing – mix it up:
   1. Tasks – Can you draw me a map of your computer network?
   2. Participation – Can you show me how I should make a Whopper?
   3. Demonstration – Show us how you update your playlist.
   4. Role Playing – I’ll be the customer, you be the receptionist…
2. Categorizing types of questions: Sequence, Specific Examples, Peer Comparison, Project Ahead, Look Back, Quantity, Suggestive Opinion, Clarification, Other Viewpoint Comparison, Native Language, Exhaustive List, Relationships, Org. Structure, Naive Outsider Perspective, Product Comparison, Activity Comparison, “Why”, and (how would you) explain it.
3. Sequence Questions: walk me through something.
4. Specific Example Questions:
   1. Ask them to compare something to their peers.
   2. Ask them to look back or project ahead.
   3. Ask them for clarification that you have it right.
   4. Ask them how they think someone else feels.
   5. Ask them to explain Native language.
   6. Ask them to explain something to someone totally naive.
   7. Ask them to compare stuff.
5. Ask them Why? – give them permission to further explain their initial answer.
6. “You won’t get the answer to your question just by asking”. You’ve got to try a few questions to get the answer. User research isn’t as simple as asking a question, getting an answer, and moving on.
7. How do you do with the total overload and explosion of follow-up questions that come to you? Wait. Let the conversation continue. “We are constrained, unless we’re Stephen Hawking, by the linearity of time.” Prepare for exploding questions – jot down probes and take it one at a time when they organically make sense. Make quick notes for follow-up, circle. Triage: which Q is most pressing. Or which Q builds best rapport?

Practice, Practice, Practice

1. Practice writing questions.
2. If you can’t get into the lab, practice in the hallways.
3. Clarify the second interviewer position, and make sure they stay on the same page when breaking in.
4. Train non-specialists if they will be coming into the field.
5. Practice, debrief, coach – this is a skill you learn over time. Do “micro-interviews” when you meet people.
6. During de-briefs, give feedback to your team.
7. Screeners and Surveys can have bad user experience too – think about the experience you are creating.
8. Collect and share war stories with as many people as possible, it will help you learn.

Thank you Steve, and thanks to everybody who tweeted during Steve’s webinar, and who tagged their tweets with #uievs. I hope to return the favor one day

Update! Other useful resources for good interviewing:

• What every usability test moderator ought to know about good listening.
• A handout Steve uses to brief clients before going out on interviews (PDF).

Update! Preview of Steve’s presentation on SlideShare:

If you found this post interesting or helpful, you may want to read Remixing The Power of Ad-Hoc Personas

When I started this blog a year ago, I set myself some mighty ambitious goals. I dreamt of writing 2-3 blog posts a week, to have 1000+ subscriber by this time, to arrange a small search user experience conference here in Oslo (!), and to write a short e-book.

Now – a year later, I have written 26 blog posts (2 per month), there are 164 comments (roughly 1:6 post/comment ratio), I got 153 RSS subscribers, 61 followers on Twitter, and I started writing the e-book. Conference… forget about it

I say it’s a modest success, and it makes me happy. Happy to share my passion with you, but not complacent. I intend to continue blogging about search user experience, in my own slow pace. Why – because I love it! I love to write down my thoughts (helps me remember), and I love the conversations it sparks with you – the engaged and enlightened reader.

Hoping to make this blog even more useful for you, I did a small redesign over the holidays. I’ve added a more prominent Twitter feed (still working on it), and the footer now contains fresh RSS feeds for other great sites about search & user experience. I hope you find something you like there, in addition to future blog posts coming from yours truly.

Here are my blogger’s New Years resolutions:

• Tweet more – I hope to fill @thingsontop with useful links to search & UX related articles, for your pleasure.
• Upload a few presentations to SlideShare.
• Keep writing in-depth articles on search user experience – the fastest growing quiz sensation on the Interwebz.

Thank you all for sharing this experience with me!

Cheers,
Vegard

flickr.com/roxycraft

At the heart of every great enterprise search user experience lies a clear and concise information architecture. Why – because search without structure is nothing but a disheartening solitary struggle to make sense of chaos. In order to implement a successful and sustainable search information architecture, you need to identify and design around relevant information entities, categories and facets. Categories in particular have a special role to play in the forming of search information architecture, one which reaches far into the following concept development and interaction design.

Why am I so geared-up about categories? I believe that design transcends technology, and that a well-designed (and possibly low-tech) enterprise search provides a better return on investment for businesses aiming to provide employees or customers with the best possible information access. And dividing by categories is one of the most effective ways to design coherent and noise-free search results, addressing problems that can’t be solved with relevancy alone.

Entities are the information archetypes we are seeking, pieces of information that documents are merely containers for. In a corporate setting examples of such entities may be employees, customers, projects, case files etc. Categories are natural and meaningful groupings of these entities, ideally satisfying several criteria that we’ll be looking into shortly. Facets are useful dimensions for refining the selection of entities, and groups of facets usually cluster around categories. Following our corporate examples, facets for employees may be years of employment, department, professional skills etc.

When you’re working to discover a suitable information architecture for an enterprise search solution, be on the lookout for categories that satisfy the following 5 criteria (in decreasing order of importance).

1. Matching the Users Mental Model

Categories in Google Search Options

As humans we’re masters of categorization. We do it all the time, and it’s how we make sense of the world. When I see a dog, I instantly know that it is also a mammal, that it’s name is Fido, that it has an owner who’s name is Bob and that Bob is also my neighbor. I know this because previous experience has allowed me to form the categories dog, mammal, Fido, dog owner, Bob and neighbor, as well as the meaningful relationships between them, together making up a mental model. (I realize that it may be more correct to call this model an ontology, but I’ll stick to mental model for now.)

Mental models hold true for digital information as well, and search result categories should match the mental model of the user. An information scent is then provided to the user, helping her anticipate the consequence of choosing to see search results from just one particular category, and building her confidence in the effectiveness of doing so. Card sorting is one way to discover with the help from users, what the natural and meaningful categories should be for a particular domain and enterprise search solution.

One good example of search categories have recently been unofficially launched by Google, as an improvement to their also-quite-recent search options. Categories like images, news, books, and maps are easy to comprehend, and it’s a good fit to our mental model of what is out there on the Internet.

2. Effective Disambiguation of Queries

Query Disambiguation by Bing

Search terms may be ambiguous, meaning that the user hasn’t provided enough information about the exact intent of the query. Does dolphins refer to Miami Dolphins (an American football team), or marine mammals? In the spirit of HCIR and exploratory search, it makes sense to simply ask the user to clarify the confusion – to disambiguate the query – rather than just guessing. Effective disambiguation is then the starting point for further refinements of the search results.

Bing provides a dynamic set of category refinement for most queries, capturing some of the different word senses for the search terms. The categories are semantically related (mined from search logs I believe) and therefore give rise to meaningful disambiguation.

3. Unique Facet Combinations

Facets describe what is typical about a type of documents or information entities, and which attributes they have in common. All cars have a make, a model, a color, a mileage etc. All employees have a title, a department, a supervisor etc. These attributes represent facets of that it means to be a car or an employee, and they are effective tools for search results refinement and clustering.

Entities described by a distinct set of facets should be separated from other entities described by different sets of facets. Using categories is a natural way to achieve this separation in a search user interface. Cars don’t have a department like employees do, and employees don’t have a color like cars do (it would be highly unethical to filter employees by color, at least). What happens then, if cars and employees are mixed together in the search result, and the user refines by department? All cars disappear, contrary to the impression that cars and employees alike are governed by the same department facet.

It’s better to avoid this possible confusion by separating the categories, and to show unique facets only when the user has chosen a particular category. Common facets can be shown all along, of course.

4. No Need for Cross-Comparison Between Categories

Categories, like any form of tabbed navigation, has at least one major drawback. By physically separating entities in the user interface, comparison across categories becomes difficult. If the user needs to compare features or prices between different makes and models of washing machines, do not organize these items in separate tabs. Items must be shown side-by-side if effective comparisons are to be made. Otherwise the user is forced to pogo-stick between tabs or dialogs.

Same applies for categories in search results. Do not place entities into tabs that need to be compared, like employees working in different departments or customers from different countries. God says that the only right use of tabs in user interfaces are for alternative views of the same information. That makes it sound like tabs should be great for alternative list views and map views in classifieds search results. I won’t dive into that problem right now, and I hope to write more about it some other time.

5. Even Distribution Between Categories

The utility of search result refinements (including categories) increase with even distribution of documents/entities across categories. With an even distribution, each category holds roughly an equal number of items. That makes each category an effective segmentation of the result set. Two categories with a distribution of 99%-1% is not very effective. If you select the first category you still got 99% of the search results to sort through. A 50%-50% (or any other even distribution) is generally preferable, provided that this constraint does not violate the previously mentioned criteria of good match with mental models, effective query disambiguation, unique facet combinations and no need for cross-comparison between categories.

If you found this post interesting or helpful, you may want to read Enterprise Search – Information Accessibility by Design or Technology?

Facets for used car classifieds like I would design them.

Facet design may seem like a trivial task at first. After all, it’s just a list of links representing a range of facet choices. By clicking on one choice after another users simply refine and narrow down the search results. Sort them by popularity (frequency) and show those neat hit counts on each facet choice, and you collect instant bonus points for improved usability and information scent. It’s not a lot more to it, right?

Wrong. You can and should put a lot more consideration into how you give shape to facets, in terms of both information design and interaction design. Poor facet design hurts. It alienates and confuses users by disregarding established mental models and overloading them with information. Good facet design on the other hand, makes use of web design conventions and familiar vocabulary to increase affordance and help users anticipate the effects of their actions.

It’s probably true that real knowledge of what works and what doesn’t in design should be based on empirical studies and hard evidence. But I also believe that best practices and reasoning takes us a long way towards something that works better. Design patterns, evidence-based or not, is a great asset for us all, and great things should be shared. For what it’s worth, I would like to share 3 of my own favorite design patterns for faceted search. I have good experience with these, and I hope they may serve you just as well.

Pattern 1: Multi-select with check boxes
A facet design with check boxes is ideal for multiple selection within a facet, like when you’re looking to buy a car and want your search to target a few chosen manufacturers. A facet with check boxes lets you choose all these in one go, saving you both time and frustration.

You should use check boxes for multi-selection facets because:

• It’s a strong visual design element that tells users by convention to “select as many of these as you like”. Choices are implicitly combined with a logical OR, finding items matching one or more choices.
• They work well for disjunctive (mutually exclusive) categories when it’s necessary to cross-compare items from different categories. Tabs or links, on the other hand, will not let you cross-compare.
• They support concept composition, allowing users to form idiosyncratic concepts like “Japanese cars” or “Reliable used cars” without a need for prior classification by the content editors.
• You avoid tiresome pogo-sticking between categories like you would have to with regular link list facets.

You should not use check boxes when:

• Don’t use check boxes for facet that are very large (50+) or hierarchical. Multi-selection with check boxes doesn’t scale with size.

I also add a few extra details to my multi-selection facet designs:

• I include an extra set of radio buttons (Show All/Choose Makes) to avoid confusion when all check boxes are initially empty (saving the diligent user who concernedly checks all the boxes for unnecessary work).
• I initially present a short teaser list (7±2) sorted by hit count. A link to “More choices” reveals a longer list of facet values sorted alphabetically (if another sorting criteria isn’t more suitable).
• I let the facet value hit counts indicate how many search results you add/remove by checking/un-checking an option.

Pattern 2: Single-select with radio buttons
Sometimes it’s better to think of facets as a way to toggle between different views of the result set. Examples could be a facet for low, mid and high-priced cars, or a facet for mileage ranges. These options are mutually exclusive (a car can’t have both a low and a high price, or both low and high mileage) and the number of options are few. With radio buttons it’s painfully obvious to the user that just one choice can be made at a time, and it’s reassuring to know that radio button choices can easily be changed.

You should use radio buttons for single-selection facets because:

• They work well for a small number of successive inclusive or overlapping choices, like price, date and other numerical ranges.
• It’s a strong visual design element that tells users by convention to “select either this or that”.
• Undo is really easy, since all options are visible all the time.

You should not use radio buttons when:

• Don’t use radio buttons for facets of even moderate size (9+), since long lists of options take up a lot of screen real-estate. You may also run the risk of choice overloading the users.
• Don’t use radio buttons when items need to be compared across categories. Use multi-selection with check boxes instead.

A few details I make sure to include in my single-selection facet designs:

• I include a default “Show All” option in the list that implies not having made a choice.
• Even options with zero results are visible, but disabled.
• I show facet value hit counts that indicate how many search results each choice will produce.

Pattern 3: Repeated selection with link lists
The dominant design for facets is by far the link list. In this example we have a list of features you may want in a used car. Each successive click on a facet value adds a constraint to the search, helping you find cars that match all the criteria that are important to you. Each selection also reduces the number of options left to choose from. On the upside, this interaction style takes you quickly from a larger general result set to a smaller specific one. The downside is pogo-sticking.

You should use links for repeated selection facets because:

• They work well for large facets with overlapping categories, both flat and hierarchical. Choices are implicitly combined with a logical AND, finding items matching all choices.
• Links are safe – everybody understands what a link is (not everybody understands what facets are, however, but that’s a different discussion).

You should not use links when:

• Don’t use links for facets if check boxes or radio buttons (or drop-down lists or tabs) suit your problem better.
• Facets with links have one major drawback – undo is less obvious as options disappear. Choices you make eliminate choices you haven’t made, since those options are no longer available and visible.

Some details I make sure to add to my link list facets:

• I include a “Show All” link that removes all choices made for the facet. A positive wording may encourage users to explore this feature, as opposed to a more negative “Remove Choices”.
• I provide clear visual feedback on chosen values (bold face and check mark graphics).
• Hit counts for chosen values are usually equal to the number of search results, so I simply hide those.
• I sort the teaser (7±2) on hit count, and will perhaps change the sorting to alphabetical for the full list.

More search design patterns
Peter Morville has a nice collection of search pattern screenshots. Marti Hears has published her book Search User Interfaces online for free. Rumours has it that Endeca is planning to release their UI Design Pattern Library sometime soon.

If you found this post interesting or helpful, you may want to read Concept Composition with Faceted Search

flickr.com/kevinomara

Facets add value to the search user experience by helping users refine the usual ranked, best-first list of documents on a search page. The quality of faceted search, however, is at risk when search result precision is traded for recall. Users should ideally be able to find all documents relevant for a given query (high recall), and nothing other than these relevant documents (high precision). Balancing precision and recall is a constant challenge for enterprise search practitioners, since it’s notoriously difficult to achieve enough of both at the same time. When a trade-off has to be made, it may seem safer to err on the side of high recall. Then nothing of potential interest is left out, at least. But we shall see that high recall plays tricks with faceted search, forcing us to reconsider this assumption.

By design, facets and facet values summarize the entire search result set, assigning hit counts to each unique facet value. A Brand facet may tell you that the search result contains 15 Canon cameras, 14 Nikon and 7 Sony cameras. If you want to refine you search to target a particular brand, you simply choose the corresponding facet value.

flickr.com/morville

It was while reading a recent book on faceted search by Daniel Tunkelang that I came across the answer to a question that has troubled me for some time. What does facet values and their hit counts really say about the corresponding documents and their relevance to your query? How can you know up-front whether you’re making a good refinement choice or not? It defeats their purpose if facets and facet values are poor indicators of relevance, and I would like to know how facet ranking and presentation effects the user experience. Are facet values with high counts always more relevant than those with low counts?

With ranked retrieval (as opposed to set retrieval) documents are scored according to how well they match the user’s query, and the documents in the search result are ranked on this score, sorted from highest to lowest. A query that favors recall may cast a wide net, allowing the user to search freely through all parts of the document, perhaps with linguistic processing and synonym expansion applied to the query. Such a query will hopefully retrieve most of the interesting documents, but it may also dig up a lot of documents that are not particularly relevant.

It’s usually safe to assume that ordinary relevance ranking will banish the lesser relevant documents to the dark depths of the search result, well hidden from all but the most insistent users. Faceted search is played by a different set of rules, however, and low ranking documents may very well contribute significantly to the facets seen by the user. Facet values are usually sorted on hit count (if not alphabetically or hierarchical), but this ranking does not necessarily reflect the relevance ranking of the documents themselves. On the contrary, a facet value with a high count is not more relevant if it represents many irrelevant documents, as the case may be with a query that favors recall.

Knowing about the hidden menace of mindless recall, we can take some measures to ensure a satisfactory faceted search user experience:

1. Increase precision by restricting queries to specific document fields, and limit the use of linguistic processing. It may not be necessary to search everywhere with full-blown synonym expansion and phonetic normalization of the query. In short, err on the side of high precision instead of recall if possible.
2. Restrict facets to summarize only the most relevant part of the search result. Computing facets from e.g. just the 4000 highest ranked documents may dampen the noise introduced by lesser relevant documents. In FAST ESP, this technique is referred to as shallow navigators. Others may call it hedging.
3. Rank facet values according to a relevance score based on document relevancy. I imagine it would be possible to compute a utility measure similar to tf-idf suitable for ranking facets, and that this measure would favor facet values originating mainly from high ranking documents. I admit this is speculation on my part, and mostly off the top of my head, and some readers may even tell me that this feature is out-of-the-box in their favorite enterprise search software.

Finally, Faceted Search: The Book is in my opinion a must-read for all enterprise search practitioners. I give it my warmest recommendations.

If you found this post interesting or helpful, you may want to read 3 Quick Design Patterns for Better Faceted Search

flickr.com/jkim1

Okay, so the website is up, I started blogging, and I’m promoting my blog on Twitter, which again updates my Facebook status. I connect with interesting and like-minded people on LinkedIn, who all spend too much time online connecting with interesting and like-minded people on LinkedIn, just like I do. Now, in the interest of further self-promotion and personal development, what more can I do to increase my web presence?

I want to write an e-book about search user experience, based on some of my latest blog posts (and all the great discussions they have sparked). I started writing this summer, and that is partly why blogging has been a bit slow for me. I hope to get the e-book done in a few weeks time. Meanwhile, I would like to give you a little LOOK INSIDE! the text I have titled 4 Kinds of Search User Experience.

The text should have some real-world use cases as examples of the 4 kinds of search user experience. E-commerce is a good, but obvious example of Diligent Search. Please leave a comment if you want to make a suggestion.

Chapter Outline – 4 Kinds of Search User Experience

The Search User Experience Manifesto
Search is about more than fast algorithms, beautiful code and large indexes. Relevance is in the eye of the beholder, and user experience design is the ultimate frontier. I believe that design transcends technology, and that a well-designed (and possibly low-tech) enterprise search provides a better return on investment for businesses aiming to provide employees or customers with the best possible information access.

A Model of Search User Experience
Part of my job as an information architect is to correctly align user needs and business goals with available technology and resources. A poor choice of technologies or design patterns may very well cripple the entire user experience. We’ll see how 4 kinds and 2 characteristics help us understand the particular strengths and weaknesses of various search concepts and technologies, as well as their applicability to real-world use cases.

Kind 1: Plain Search
The Plain Search experience is well-suited for known-item search, where information needs are easily resolved with basic query/response behavior. Sometimes all you have are a few keywords, and a ranked best-first list of 10 blue web page links is just what you need. Less can be more, but expect difficulties with precision/recall trade-offs and ambiguous queries, as well as excessive query re-formulation if the user is unsuccessful at completing the search task.

Kind 2: Superficial Search
The wisdom of the crowds is something of a holy grail, and tapping into the behavioral patterns of millions of users have brought fame and fortune to Amazon and YouTube, and recently also to Twitter. Recommendations and real-time search leverages user behavior to feel the pulse of communities, and to surface popular and current material, even as events unfold. Superficial Search is very efficient when you don’t need to dig deep into the information space, but it’s vulnerable to feedback loops and other kinds of popularity bias.

Kind 3: Ingenious Search
The Ingenious Search experience improves upon the weaknesses of Plain Search with a model-knows-best approach. Like rubbing on Aladdin’s lamp, sophisticated algorithms automatically disambiguate query intent and extract semantic structures from content. Direct answers are provided instead of documents in an attempt to increase precision. Clever algorithms can be said to be cost effective, but they’re comparably more difficult to implement and execute well. Also be aware of awkward query formulation and high expectations on behalf of the user.

Kind 4: Diligent Search
It’s sometimes easier to ask than to guess. Diligent Search regards the human intellect as a better judge of relevance, and empowers users to increase precision by refining both query and content themselves. Faceted search, a common feature on modern e-commerce sites, enables efficient exploration of the information space (shopping) by assisting users in iteratively expressing their information needs. Choice is great, but don’t get carried away. Having too many options can be paralyzing and counter-productive for the user.

Objections
I expect some readers to disapprove of how I have grouped, named and described search-related concepts and technologies in this text. That’s alright, and I’ll try to meet you half-way by reproducing and commenting on some of the constructive feedback and criticism I have received already from engaged and enlightened readers.

Resources
This is a collection of blogs, books, e-books and pattern libraries I find useful in my work with search user experience. I hope it may serve you just as well.

flickr.com/waisian

When working with search user experience, it’s often necessary to discover a suitable information strategy for an organization or business. An information strategy can potentially have a huge influence on how the search for information is facilitated. Are the users in question generally seeking precise answers to quick questions, or are they seeking to explore a broader selection of opportunities? Question answering will generally require a more ambitious use of technology, while exploration and discovery places greater focus on communicating with the users. Perhaps an early decision to acquire software with semantic analysis capabilities will lead the project towards greater use of technology, while users studies may reveal that users are keen on spending time researching the cheapest flights to their holiday destination. Understanding the integration of business goals, user needs, and technological possibilities is in any case vital for a successful and sustainable information strategy.

Two recent articles in Technology Review reflect some of my latest writing on 4 different approaches to search user experience. The article by Daniel Tunkelang entitled “To Search, Ask” is very much in line with what I like to call Diligent Search, while the article on Wolfram Alpha is describing something closer to Ingenious Search. I find these articles very interesting, and I believe you will think so too.

Daniel Tunkelang (CTO, Endeca), a strong believer in faceted search and set retrieval, and an advocate for HCIR, prefers search engines that are more like conversational librarians. Rather than guessing what the users need, these systems provide users with opportunities to clarify and elaborate their intent. If the engine isn’t sure what users what, it just asks them.

Read Daniel’s article at MIT Technology Review →

Stephen Wolfram, the physicist and maker of Mathematica, has recently launched the computational knowledge engine Wolfram Alpha. In response to questions, Alpha is meant to compute answers rather than list Web pages, and pushes technology to the limit in order to anticipate what the users need. It attempts to achieve this by means of a constantly expanding collection of curated data sets, an elaborate calculator, and a natural-language interface for queries.

Read the article about Wolfram Alpha at MIT Technology Review →

You may also want to read more about my views on mediating information and information accessibility.

If you found this post interesting or helpful, you may want to read Wolfram Alpha – Re-Inventing the Command Line

What is your name?

The hype has passed, and it’s clear that Wolfram Alpha isn’t the new Google killer as initially anticipated. But what is this new computational knowledge engine, and how shall we use it? It seems to me that we have been given some sort of a command line tool for a comprehensive information database. The search box acts as a peeking hole into Wolfram Alpha’s wonderful inner world of facts and figures, giving us a sense of what’s hidden inside there, while at the same time making everything slightly inaccessible. Like with a command line, you have to know what you’re asking for, and you must know how to phrase that question in a way Wolfram Alpha understands, in order to get a decent answer.

Wolfram Alpha is perhaps less of a search engine, and more like a question answering engine. Being something other than a search engine may be a strength in the fight for survival alongside Google, Yahoo and Bing, but it also means that the general Internet population must learn how to use this new tool. And that doesn’t happen overnight, especially since we’re already so accustomed to the quick and dirty web search. Wolfram Alpha can be both tedious and puzzling to interact with, and must therefore provide some serious extra value if it expects us to make the extra effort.

Your wish is my command.

I’m comparing Wolfram Alpha to a command line, and if you’re not a geek, you may not know what a command line is. In short, it’s a way of giving instructions to the computer, like deleting a file, by typing text commands into a special window, instead of clicking and dragging the file icon to the trashcan with your mouse. The command line is loved by many for it’s efficiency (I do!), but it’s also an interaction paradigm hampered with some serious poor usability issues. Jono DiCarlo of Humanized does a great job at summarizing these issues in his post about the problem with language-based interfaces. Here’s why he believes command lines are inherently difficult to use:

• Not discoverable: There’s no guidance given to a first-time user. You type some letters and nothing happens: it feels like shouting into a void. If you don’t already have the basic commands memorized, there’s no way to figure out what they are.
• Cryptic names: Whether for historical reasons or for brevity, the standard names of commands, programs, and locations are all called stuff like ‘tar’ and ‘mkdir’ and ‘/usr/local/bin’. Because these names are unnatural and unfamiliar, they have to be learned by rote.
• No feedback: I just entered a command and all I got back was a blank line! It worked, but what did I just do?
• Options are hard to remember: Does the ln command take the source file first or the destination file first? What does the -z option on tar do again?
• Really easy to make mistakes: One wrong character and your innocent command is transformed into a ruthless file murderer. And there’s no undo!

Let’s look at each of these misfeatures in turn, and see if any of them apply to Wolfram Alpha.

Not Discoverable
Not entirely true for Wolfram Alpha, since the welcome page provides a multitude of helpful hints on how to get started using the engine. There’s examples of mathematical formulas, geographical locations, dates, stock quotes, demographic data and other computations Wolfram Alpha is capable of handling. But even with this collection of well-chosen examples, Wolfram Alpha’s special flavor of knowledge computing feels a bit like a guessing game. And that makes it difficult to learn how to type in the right questions.

The normal rules of web search (like typing in whatever is on your mind) don’t apply here, and feeding Wolfram Alpha the same kind of search input as you would to any other search engine seldom works. Considering that Google sees about 20-25% new and unique (never seen before) queries every day, it’s natural to assume that Wolfram Alpha’s natural language processing will have a hard time providing sufficient query formulation flexibility to searchers and their diverse information needs.

Cryptic Names
Wolfram Alpha is riddled with scientific abbreviations and mathematical notation, which are rather unfamiliar to most people. These specialized features of the input language reinforces my impression that Wolfram Alpha is a search engine for specialist (in various domains). You have to possess some prior expert knowledge in order to ask the right questions, without which you can’t get access to possibly valuable information.

No Feedback
If you successfully managed to execute a query, Wolfram Alpha gives you a lot more back than just a blank line with a blinking cursor. But if you don’t succeed, it’s usually the case that “Wolfram|Alpha isn’t sure what to do with your input.” Not very helpful, unfortunately, but I guess it’s hard to do more, especially while the engine is still in its infancy.

Instead of just dismissing unlucky searchers, Wolfram Alpha could start to back-fill unsuccessful queries with web search results delivered by eg. Google. That would avoid creating a dead-end feeling. Sure, there’s a link to web search in the sidebar, but it’s well hidden. Back-filling with web results could make Wolfram Alpha more appealing to the general audience.

Options Are Hard To Remember
Not really transferable to Wolfram Alpha, but as with cryptic names, search operators like “/” must be taught to the user before they provide any value.

Ruthless file murderer?

Really Easy To Make Mistakes
Relax, there’s no risk of you accidentally erasing Wolfram Alpha, or the Internet for that matter. But it is really easy to make mistakes when you’re typing in a query. Wolfram Alpha seems to be equipped with a decent spell checker (Levenshtein, perhaps), but it doesn’t work as well for various abbreviations. For example, gbp norway gives a different result altogether than gdp norway. Sure, no harm done, except perhaps to the user’s sense of pride and confidence in the engine.

5 Things Wolfram Alpha Does Better
So much for my Wolfram Alpha bashing. Stan Schroeder of Mashable tells us what he thinks are the bright sides of Wolfram Alpha, the computational knowledge engine. He makes a list of 5 things Wolfram Alpha does better (and vastly different) than Google:

1. Complex Queries – Like finding information about our solar system by typing in the names of a few planets. Wolfram Alpha gives individual information about each planet together with different comparisons, while Google simply returns the best matching web pages.
2. Localization – Currencies and units of measure are easily converted when treated as first-class information objects, like Wolfram Alphawho’s mostly restricted to returning whatever is on the web page, localized or not. does. It’s not so easy for Google,
3. Precision – Wolfram Alpha surely can’t answer all questions. But it tries to be as precise as possible with the answers it does give, and refuses to deliver anything half-baked. I would say that precision is nice, but it’s not everything. Exploration and discovery usually starts with a few good alternatives as basis for further research and analysis.
4. Calculation – Google provides direct answers to some specialized queries, like paris hilton height and 100 USD in EUR, but nothing close to what Wolfram Alpha does.
5. Comparison – Comparing information across web pages does not work that well for document-centric search engines (like Google). But with Wolfram Alpha it works like a charm.

Wolfram Alpha is a bit like a box of chocolate – you never know what you gonna get. If at first you don’t succeed, try again. If you happen to love command line interfaces above everything else, you may want to check out Goosh – the unofficial and curious Google command line. And if you you’re looking for some light entertainment, check out these Wolfram Alpha easter eggs.

If you found this post interesting or helpful, you may want to read What Is Your Approach to Information Retrieval?

flickr.com/allaboutgeorge

You’re the user experience designer hired in to give shape to a new enterprise search solution. Your client has already decided upon a technology vendor, and has purchased a license. The responsibility now rests with you to provide employees or customers with the best possible information access. Will you design the solution around the assumption that IR algorithms are able to directly deliver the relevant documents in response to most queries, without much further involvement from the user? Or will you design for optimal communication, making the user more responsible for retrieving relevant documents through engaging interaction? Since I don’t believe in technological silver bullets, I argue that good design takes you further for less money when you’re aiming for higher information accessibility.

This post is a continuation of my previous posts on a topology of search concepts, and mediation in information retrieval. I have received a lot of valuable feedback, and one reader commented specifically on my use of the term information accessibility as one of the dimensions for classifying approaches to search. Here’s a snippet from one of the emails I received (reproduced with permission):

When we came up with the definition of accessibility, we also very clearly spoke about the cost required to reach the resource (i.e., user effort to find the document). You would need to compare the costs that users associate with looking through a linear ranked list, versus reading the facet names and then interpreting them and then making a choice and then potentially not finding the document. This across a wide range of potential queries. Only then can you say that one form makes something more accessible than the other.

We were at this point discussing how the definition of accessibility in information retrieval applies to pure best-first (linearly ranked) search vs. faceted search. Put in other words, will a clever ranking algorithm generally provide a shorter path to the document sought by the user, or is the user generally a better judge of relevance? What I’m trying to understand from this discussion as a user experience designer, is how my design considerations may effect the final outcome to the better for businesses and users. I’m also aware that my chain of reasoning does not qualify as proper scientific argumentation.

Technology – What to Reasonably Expect
Many enterprise search customers acquire a search platform for the purpose of implementing search for a web site, an e-commerce portal, or several company systems like email and CRM. Whether the brand is Google, FAST/Microsoft, Autonomy, Endeca or Lucene, these platforms and their ranking algorithms must be tailored to the specific information needs of the company’s employees of customers. If you spent a lot of money on that software license, it’s reasonable to have high expectations for what this magical search box can conjure up. But how much effort will it take to fine-tune the relevance ranking, and can you expect it to perform top-notch across a wide range of potential queries?

There’s little doubt that web search engines like Google, Microsoft Live and Yahoo spend an awful lot of time and money perfecting their ranking algorithms, simply because the credibility of their brands depend on always delivering the best results first. But chances are that your enterprise search client is not able to pour that kind of money into all kinds of relevance trickery. With that in mind, can we reasonably expect ranking algorithms to outperform humans in terms of efficient information seeking? I believe that faceted search holds the upper hand on linear ranked lists (possibly combined with successive query reformation) when accessibility is considered.

Design – How Far Can You Get
I have to admit that I’m partial to set retrieval and faceted search. I believe strongly in solving information seeking problems with good user experience research and design, and have what I like to believe is a healthy skepticism towards technological silver bullets. It’s my experience that some up-front design gives a better return on investment than excessive technical tinkering later on.

A critical step in the design of faceted search is to find meaningful facets that convey a strong scent of information, and the only sure path to well-design facets goes through extensive user research. The set of facets and their values must correspond to the mental model of the users, and it must be immediately understandable to them what results they should expect upon making a selection. Well-designed facets also provide more support to the user as compared to multiple queries / query reformulation, which is more of a guessing game.

When a user reads and interprets the names of facet values, makes a choice and not finds the document he or she was looking for, I think one of 2 things may have happened:

1. Either the facets are poorly mapped to the mental model of the user.
2. Or the sought-after document simply doesn’t exist.

In the first case, a linear ranked best-first list would’ve been less misleading and would probably have served the user better. That could possibly have provided a shorter path, resulting in higher accessibility. In the second case, the user got early feedback that the search was futile, and that his or her time could be better spent looking somewhere else. In any case, I believe well-designed faceted search is very likely to provide a high level of information accessibility.

A Case of Ingenious vs. Diligent Search?
I have previously written about 4 different approaches to search user experience, where Ingenious and Diligent Search are descriptions of concepts and technologies that bring about high accessibility, with the difference that some are more algorithmically powered, while others are more user powered. I consider the Ingenious approach to include Google’s Universal Search, clustering, LSI, question answering / NLP, search intent analysis and more, where algorithms primarily make the judgment about what is relevant. With the Diligent approach, on the other hand, it’s the user who primarily makes this judgment, which includes concepts like set retrieval and faceted search.

I’m a search user experience designer, and I believe that a well-designed (and possibly low-tech) enterprise search can provide a better return on investment for businesses aiming to provide employees or customers with the best possible information access.

If you found this post interesting or helpful, you may want to read What Is Your Approach to Information Retrieval?

I’m very pleased with the discussions sparked by my last post about designing a topology of search concepts. Several people have raised questions about the 2 dimensions I selected as a way of characterizing search concepts in terms of business goals, user needs and technological capabilities. The post was intended to solicit discussion, and perhaps to function as a framework for increasing our own awareness about the capabilities of search technology, and how they are best applied to solve particular information seeking problems. Such a framework can be a great asset for pre-sale engineers, user experience designers, information architects and interactions designers about to build a search solution.

Especially the comments by Gene Golovchinsky helped reveal some short-comings of the dimension I had named “Algorithm vs. User Powered”. Following our discussion, I have come to believe this dimension can be better described as “Algorithmically vs. User Mediated”. Let us find out by exploring what mediation means in the context of search and information retrieval.

The goal of searching is to resolve an information need, and that resolution process can be mediated (or facilitated/supported) by algorithms, users (alone or in groups), or a mix of both. Put into other words, the mediator (or facilitator) is the entity responsible for taking actions to retrieve information that will fulfill the user’s information need. If those actions are taken by an algorithm, like extracting latent structures from the data, or analysing query intent, the search approach can be thought of as “algorithmically mediated”. On the contrary, if the actions are taken by users, like navigating facets, or sharing search results with a partner, the approach can be thought of as “user mediated”.

It’s difficult to imagine information seeking that isn’t a symbiosis between algorithm and user. Almost every search starts with a user action, whether it’s to submit a few keywords, or to just produce something that can be the starting point of a query. And unless I’m asking a question directly to someone else, an algorithm has to pull up a set of possibly relevant documents for me to investigate further. But if we now accept this pre-condition, who plays the role as primary mediator for the information seeking process?

In an attempt to break it down, I have compiled a list of search concepts that I believe are primarily mediated by either algorithms or users. Whether you agree or disagree, please share your thoughts.

Algorithmically mediated (primarily):

• Keyword Search (best-first)
• PageRank (authority-base search)
• Related Searches
• Search Suggestions (type-ahead, auto-completion)
• Universal Search (structured results)
• Clustering
• Implicit Collaborative Search
• Questions Answering (natural language processing)

User mediated (primarily):

• Collaborative Filtering
• Best Bets
• Social Search (whatever that is – Twitter?)
• Explicit Collaborative Search
• Set Retrieval
• Faceted Search

A Topology of Search Concepts

Search is a wicked problem, with no apparent universal solution in sight. Different technologies and approaches to search exist side by side, serving a multitude of business goals and user needs. In my work with search user experience I find it important to understand the particular strengths and weaknesses of search concepts like Best Bet and Faceted Search, since part of my job is to correctly align goals and needs with available technology. A poor choice of technologies or design patterns could very well cripple the entire user experience. But how do I know which concepts will work?

To answer my own question, I’ve spent some time the last few days putting together a topology of search concepts, which I’m thrilled to share with you now. I’ve made a scatter plot of a handful of search technologies and patterns, with the purpose of revealing some basic structure and similarities. The main elements of this topology are the set of search concepts, the 2 dimensions of the scatter plot, and the descriptions of each quadrant. You can see the result so far in the illustration at the top of this page.

This is still work in progress, so I’m eager to get feedback from you. I may have left things out, plotted things the wrong way, or described things poorly. Whatever you have to say, please leave a comment on this post.

Here is how I define the 2 dimensions of the scatter plot for the purpose of this topology:

• Algorithm vs. User Powered – In the absence of a better name, positioning along the algorithm vs. user powered dimension reflects to what extent human or machine intelligence is responsible for retrieving precise and accurate information in response to the query.
• Information Accessibility – By implementing a search concept for a given information space, information accessibility is a measure of how much easier it becomes to find any document of interest within that space. If the time it takes (and/or the number of steps required) to retrieve a particular document goes down, the general information accessibility goes up. Read more about accessibility in information retrieval.

The scatter plot is divided into 4 quadrants, so that the concepts positioned within each quadrant share a common set of characteristics related to business value, user experience and technological capabilities. Here is how I describe these quadrants:

1. Simple Search – When you know what you want, and you can express that need with a few keywords, the simple approach to search fits the bill. It’s by no means trivial to create simple experiences, but the general information accessibility is quite low for these concepts. Google, FAST/Microsoft, Exalead and Lucene are some of the champions of simple search.
2. Superficial Search – The superficial approach to search leverages user behavior to feel the pulse of communities, and to surface popular and current material. Superficial search is often very efficient when you don’t need to dig deep into the information space. Amazon, Twitter, PostRank and Twingly are some of the champions of superficial search.
3. Ingenious Search – With a model-knows-best approach, ingenious search relies on sophisticated algorithms to determine user intent and content semantics. Clever algorithms can be said to be cost effective, but are comparably mode difficult to implement and execute well. Autonomy, Powerset, Grokker and Wolfram Alpha are some of the champions of ingenious search.
4. Diligent Search – The diligent approach to search favors human intellectual effort over clever algorithms. Given an initial search result, users are asked to further disambiguate their queries in order to effectively explore the information space. Endeca, eBay, Freebase and Apache Solr are some of the champions of diligent search.

I won’t get crossed if you say my matrix looks a bit like Gartner’s Magic Quadrant, but there’s at least one big difference. Contrary to Gartner’s quadrants, this matrix does not suggest any better-than/worse-than relationships between the data points. Each quadrant is just different from the others, and the search technologies and patterns found within simply serve different purposes, having their own particular strengths and weaknesses.

Whether you agree or disagree with my way of mapping the world of search concepts, please share your thoughts. I’ll make sure to give credit to everybody who contributed significantly when I publish the final result. Cheers!

You may also download a poster with both the diagram and the quadrant descriptions.

flickr.com/netliferesearch

Gulltaggen claims “Norway’s premier event for digital marketing” (presentations by Seth Godin and Chris Anderson to mention a few) and it was held on the 28th and 29th of April. There’s also a steady stream of #gulltaggen tweets. We’ll give a quick recap on two talks: building successful websites through understanding what users want to achieve on your site and how marketing changes from a mass focus to the spreading of ideas.

“The Long Neck” with Gerry McGovern
Gerry’s presentation focused on page views and content productions as our traditional view of site effectiveness. But what about user satisfaction?

Microsoft’s Excel web site had around 11 000 pages. All pages were searchable in major search engines and they were adding more and more content to ensure that they cover every aspect of Excel. A long tail focus, but what about the majority of users: The long neck?

The majority of users might be interested in how to do a basic sum of values in excel, but queries on Google would present a page on the INSUM function, a far more specialized aspect of Excel. Thus, content was beginning to clog down their web site. Page views were rising but user satisfaction was declining as users were constantly running into dead ends within the enormous amounts of pages.

The solution was to simply start removing content; a concept highly unpopular in many organizations where countless hours have been placed in content production. The web site has to focus on its core message – what does the majority of users need to do online when entering the your web pages? Looking through support records and behavior analysis to find out the most frequent actions users want to take on your site. If you are not covering your users basic needs – how can you start focusing on the long tail?

As a result page views went down drastically, another unpopular concept with senior management – but user satisfaction went up! Shouldn’t we be measuring satisfaction over of page views?

Other examples were campaign pages (Hillary Clinton, John McCain) where the pages are focused around the single most important aspect – contribution.

“Marketing, Leadership & Being Remarkable” with Seth Godin
Seth’s presentation circled around the concept of the “purple cow”. How can organizations keep on making average products for average users? People have seen cows and “cows are boring” Seth continues, but a purple cow would gets everyones attention.

Seth claims that something is fundamentally wrong with todays mass marketing model on line – where organizations are constantly trying to interrupt users in order to get their attention, examples being banner and popup ads. The Internet is about attention giving , not an attention taking, and only by getting your users permission can you successfully push your ideas at them, avoid the spam filters and make the stories spread.

The Internet is not TV on a laptop but a tool for the spreading of ideas.
- Seth Godin

The ideas are what made recent Apple products so successful. The ideas that people want to talk about, and show their peers that they are using Apple products. Apple started out with a small group of engaged mac users. Apple had permission to tell these people (aka the “fan boys“) about their new products. These people would in turn acts as marketeers for Apple products, thus spreading the idea.

Seth had several other good stories in a frantic hour of a web-streamed talk both amusing and enlightening. However, I must confess that I occasionally find cows quite amusing.

Some links:
- Gulltaggen and on twitter: #gulltaggen
- Seth Godin’s blog
- Gerry McGovern

If you found this post interesting or helpful, you may want to read Topic Pages – Content re-use and news site usability (part 1)