socialsignal.com
Some choice good – excessive choice bad. That is the (condensed) Paradox of Choice, according to Barry Schwartz. We need some choice in order to exercise our free will, but the abundance of options we’re facing today (when shopping for groceries, entertainment, education and more) is actually quite overwhelming and paralyzing. No matter how much research we do, no matter how carefully we consider our options – we always risk feeling slightly disappointed and unsure if we could have made a better choice.
A good recommender will help us decide by dramatically reduce the number of options we have to explore, into a small subset of what is available and potentially relevant. A better recommender will cut the crap, and just present the options we need to make a choice we’re happy with. Nothing more, nothing less. Because too many recommendations can create a second problem much like the initial Paradox of Choice, especially when we our goal is to reach a (possibly very important) decision.
Exploration is often not a goal in itself. It’s a painstaking process we have to endure, in order to feel confident enough to make a choice. As an example, if I want to learn more about user experience design, I can go to Amazon and shop for books on the topic. A quick search returns a list of 1,089 books for me to choose from, and for each product page there’s several more recommendations, user reviews and reading lists I can base my decision on. Amazon is not giving me the answers I need, just endless opportunities for exploration. I feel my blood pressure rising.
In a world with perfect recommendation technology, Amazon would just tell me which three books I should read to learn more about user experience design. From all the 1,089 available options, Amazon would recommend me three well-written and thoughtful books, covering most bases of the topic. Perhaps I get to choose between a set of cheaper books and a set of more expensive books, depending on how much money I’m planning to invest in my knowledge upgrade. In any case, I’ll be happily making a simple choice, free from the torments of exploration. It doesn’t matter if my choice is sub-optimal (there may not even be a single optimal option for me). I’m ready to satisfice, just as long as the recommendations are sufficiently relevant.
Current technology is not capable of giving perfect recommendations (partly due to ambiguous user queries and changing user preferences), so we cover up by design systems that encourage exploration and discovery. We strive for transparency and diversity, designing an experience for the eager and motivated user. But for some of us, the Paradox of Choice returns and takes it’s terrible revenge, as our options for exploration grow and overwhelm us again. Somehow I feel that efficiency (making faster decisions) and satisfaction are better design goals for many recommendation systems.
Exploring music is, on the other hand, something I enjoy doing. My music preferences are strongly connected to my personal identity, and that changes somehow my attitude towards choice and exploration. Choosing music becomes a meaningful act of self-determination, and music exploration becomes a goal in itself. So I cherrish the many options for exploration I’m given by Last.fm.
Kirsten Swearingen and Rashmi Sinha gives us a neat breakdown of different user needs a recommender should accommodate. Here is their list of recommendation types:
- Reminder recommendations, mostly from within genre (“I was planning to read this anyway, it’s my typical kind of item”)
- “More like this” recommendations, from within genre, similar to a particular item (“I am in the mood for a movie similar to GoodFellas”)
- New items, within a particular genre, just released, that they / their friends do not know about
- “Broaden my horizon” recommendations (might be from other genres)
I feel that my music exploration needs are well represented in this list, but I wish for a stronger focus on efficient and satisfactory decision-making.
To find out more about the Paradox of Choice, check out Barry Schwartz’ video talk on TED or read his book.
How about populating music, movies, poetry, books, in general everything that we might like and has a sentimental value (which is just to say the things that imply preference based on interpretation) with a semantically rich recommendation system? Simply put, if you wanted a recommendation on a particular thing you should start by describing it.
Take del.icio.us for example. In order for the service to be of any value I first have to tag the different websites I want to bookmark, and then I can see websites that match those bookmarks.
Or better yet, Wikipedia, it’s the best music recommendation system I’ve used. Not because every artist I search for is connected to other artists I like, but simply because it directs me towards artists I _might_ like.
It’s an interesting abstraction, the notion of a recommendation not as a finality in on itself, that is, not implying a choice, but rather as a tool that helps us make that choice.
Interesting read! I have no doubt that keeping the number of options down is beneficiary in several web site scenarios. However, I’m a bit troubled by this model within a larger commercial site as it would inherently limit competition.
@Thomas Kjelsrud
I haven’t given the competition perspective much thought so far. I was more worried about how choice effects the consumer emotionally. But what you’re saying seems to be important.
Exactly how do you think this bare-bone recommendation scheme will limit competition? Will it allow producers to raise the prices on their goods? Or do you think it may favorize some producers at the expense of others?
I think it will limit the chances of getting notices within a specific field or topic, as the well-established dominate these search & recommended results.
One could of course ask if it is “noble” to e.g, publish yet another book on a popular topic in order to draw sales from “the hype”, or should they be “forced” to publish & write books on more novel areas in order to dominate their own clusters of knowledge (and keywords being searched).
Although limiting any result to the top “N” can be beneficiary for users, the ones struggling to get noticed further down in the results should still have a chance to be seen.
I guess to wrap it up: how do you monetize on the long tail while limiting the choices to the most popular/most recommended? (which is not really part of the long tail at all)
Guillermo’s comment about populating music, movies etc. with a semantically rich recommendation system is very interesting. This is exactly what we do at Jinni (http://www.jinni.com), the first semantic discovery engine for movies and TV shows.
One advantage of the semantic approach is that it’s better for the long tail than statistical approaches that rely on what’s already popular with other people.
It makes sense to me to limit the choices to make the act of choosing less of a cognitive headache. To address Thomas Kjelsrud’s point about the long tail, one possible answer is that the long tail doesn’t belong on the same body as the big head.
If the person’s information need (rather than the query that loosely approximates it) is taken into account, it may be possible to present results in a more appropriate manner than a single ranked list. Various clustering approaches may be used, for example, to identify categories of documents that are popular, unusual, detailed, technical, etc. Searchers could then use these categories (or many other possible ones) to help them make sense of search results in a more task-oriented manner.
Another (related) approach is to think about aspects, and to factor search results that way. Some of these approaches are more difficult computationally, but for many domains there may be useful metadata that can be leveraged, or reasonable heuristics may be applied.
The disadvantage of considering search results this way is that one size doesn’t necessarily fit all users, and the search provider must do more work to make the results more useful. Of course if you _can_ do it well, you’ve got a competitive advantage.