Remixing Deep Dive Interviewing Secrets

http://xkcd.com/125/

Everybody knows that you should ask open and unbiased questions when interviewing users for UX research. The real skill to master for the seasoned researcher/designer is not so much that, but to know what kind of open question to ask, and when to ask them. Some questions may prove to be dead ends, and other may explode in a frenzy of possible follow-up questions. The best questions are intended to reveal information unsaid and undiscovered. But how do you know what’s missing from the interview? What have you not been told so far, and how do you probe into that? This remix of tweets from “Deep Dive Interviewing Secrets”, a UIE virtual seminar by Steve Portigal, gives you some of the answers.

I missed out on Steve’s webinar, unfortunately, and decided to check out what others had tweeted about it using the hashtag #uievs. Luckily, there had been lots of activity and discussion, and I felt that Twitter provided me with quite a comprehensive summary of Steve’s stunning insights in to interview techniques. For my own sake and for future reference, I decided to compile that Twitter timeline in to a short document. But why not share the fruits of knowledge?

So with a little extra effort, here’s my remix of tweets by those attending Steve Portigal’s UIE Webinar on “Deep Dive Interviewing Secrets: Making Sure You Don’t Leave Key Information Behind”. Enjoy!

Tips for Establishing Contact

1. Try to achieve Question > Story, instead of Question > Answer. Lots of question asking can be replaced by letting them tell you. Good moderation means not giving away the answers you are looking for. Your personal style is important to find in developing your methodology and tactics.
2. Beware the temptation of synthesize and build a model on the fly. Remain open to the subject’s model. If you want to learn about a person’s mental model, don’t expose yours first.
3. Talk like your subject talks. If a user mispronounces something, repeat it to keep them comfortable. Avoid them having to disagree with you.
4. Silence is golden – after you ask a question, be silent to allow for a response. You don’t know their style of response. Avoid the “Errrr” nervous trail off if you aren’t confident in the question, wait for a response.
5. Listening: don’t interupt, be silent (even the “hmmm mmmm”), maintain eye contact, nod, and acknowledge e.g. “I see”. If they need the “hmmm mmmm” keep um, but be aware of the interruption potential. After they’ve answered you, be silent another beat so they keep the story going.
6. Don’t let them off the hook when words don’t come easily. Make them make the label instead of agreeing with yours. The person you’re interviewing will “want to do what you want them to do”. If you move on, they move on.
7. Signal your transitions. “I’d like to move on”, “Earlier you mentioned…”, “I’d like to go back to…”.
8. Tactics you may try:
   1. Use the user’s language. E.g. if they say TiVo as “tye-vo” instead of “tee-vo”, follow along.
   2. Ask questions without answers. Don’t suggest answers. Don’t guide the responses or suggest responses. Use open-ended questions: “Can u tell me about your experience using X?” not “What are 3 things you liked about using X?”
   3. Silence. Ask your question and then be silent. Leave time for an answer to form.
9. Steps to letting go of your own perspective:
   1. Admit what you think will happen as a team.
   2. Go to the user’s natural space.
   3. Don’t dress for work.
   4. Make the interview about the subjects – you are their to learn about them.
   5. Be comfortable asking the questions that you already know the answer to, and learn what their answer is. “I wanna know how you do it”.
10. Q: If you want to learn their terminology, how do you do it without using terms? A: point at the object, introduce topic broadly and then “harvest” the terms to use later. Turn it from a test into a conversation with the purpose of learning. “Do you understand these terms” is a test and the person mail fail. “Can you tell me about THIS” (pointing) isn’t a test.

Tips for Building Rapport

1. Building Rapport with Participants: the energy that underlies great interviews.
2. Listening is the rapport engine. It is your responsibility to establish rapport.
3. If you’re going to people’s homes, it helps to be a cat/dog person. Bonding with a pet demonstrates rapport.
4. Some small talk is appropriate, to little is rude, too much is distracting and turns your interview into a social call. This interview isn’t about you – even if something familiar happens. And don’t share about yourself. Avoid the “OMG ME TOO”. Sometimes there are exceptions, when they need permission to continue or you want them to feel more comfortable.
5. Watch your body language – think about the emotional state your position puts you in. Sit to listen. Lean in. Make eye contact. Be interested.
6. Asking follow ups really makes the connections and makes the customers at ease as well. Demonstrate that you are listening by building questions on what they have said.
7. Q: Is rapport building different by phone? A: recreate the “walk down the hall” moment with virtual participants – use small talk but focus on the task.

Interview Guide Tips

1. Field Guide: The more-than-you-can-possibly-ask set of questions.
2. A guide helps you to pre-visualize the session. The act of creating an interview guide can mean that you are so prepared that you don’t need the interview guide. You may not actually use the guide you create – but if you need it, it is there.
3. The interview guide is also a good prop: “Let me see – what else did I want to ask you…”
4. An interview guide would have a list of questions, grouped together.
5. Topics Guide Approach: Identify the headings, the questions are not always needed. Start with a list of topics you’d like to learn about.
6. A more robust guide has questions as they would be asked, the motivation for asking, and any follow-ups.
7. Make it natural, not overwhelming to glance at. A interview guide needs to be something that you can refer to quickly – keep it simple.
8. A version Steve uses most has subhead list of questions as they would be asked, as well as time/order.
9. An interview guide should include Non-Verbal notes like stage directions for the moderator (probe for …, reference the screening call…), and a list of things to be sure you’ve seen and/or photographed: get a face shot, an action shot.
10. Q: How much prep time is needed pre-interview? A: depends on the test, but planning for field work takes at least 2 weeks to define screening, recruit, and define field guide.

Tips for Asking Questions

1. Use Tasks, Participation, Demonstration, and Role Playing – mix it up:
   1. Tasks – Can you draw me a map of your computer network?
   2. Participation – Can you show me how I should make a Whopper?
   3. Demonstration – Show us how you update your playlist.
   4. Role Playing – I’ll be the customer, you be the receptionist…
2. Categorizing types of questions: Sequence, Specific Examples, Peer Comparison, Project Ahead, Look Back, Quantity, Suggestive Opinion, Clarification, Other Viewpoint Comparison, Native Language, Exhaustive List, Relationships, Org. Structure, Naive Outsider Perspective, Product Comparison, Activity Comparison, “Why”, and (how would you) explain it.
3. Sequence Questions: walk me through something.
4. Specific Example Questions:
   1. Ask them to compare something to their peers.
   2. Ask them to look back or project ahead.
   3. Ask them for clarification that you have it right.
   4. Ask them how they think someone else feels.
   5. Ask them to explain Native language.
   6. Ask them to explain something to someone totally naive.
   7. Ask them to compare stuff.
5. Ask them Why? – give them permission to further explain their initial answer.
6. “You won’t get the answer to your question just by asking”. You’ve got to try a few questions to get the answer. User research isn’t as simple as asking a question, getting an answer, and moving on.
7. How do you do with the total overload and explosion of follow-up questions that come to you? Wait. Let the conversation continue. “We are constrained, unless we’re Stephen Hawking, by the linearity of time.” Prepare for exploding questions – jot down probes and take it one at a time when they organically make sense. Make quick notes for follow-up, circle. Triage: which Q is most pressing. Or which Q builds best rapport?

Practice, Practice, Practice

1. Practice writing questions.
2. If you can’t get into the lab, practice in the hallways.
3. Clarify the second interviewer position, and make sure they stay on the same page when breaking in.
4. Train non-specialists if they will be coming into the field.
5. Practice, debrief, coach – this is a skill you learn over time. Do “micro-interviews” when you meet people.
6. During de-briefs, give feedback to your team.
7. Screeners and Surveys can have bad user experience too – think about the experience you are creating.
8. Collect and share war stories with as many people as possible, it will help you learn.

Thank you Steve, and thanks to everybody who tweeted during Steve’s webinar, and who tagged their tweets with #uievs. I hope to return the favor one day

Update! Other useful resources for good interviewing:

• What every usability test moderator ought to know about good listening.
• A handout Steve uses to brief clients before going out on interviews (PDF).

Update! Preview of Steve’s presentation on SlideShare: