For the past few months, we’ve been demoing our new Cloud Address Book to a variety of alpha testers, using phone calls and in-person chats. For those who are beginning the process of gathering user feedback, here are some ground rules for conducting user interviews. We’ve learned a few of these the hard way, so you won’t have to.
Like any set of commandments, following is the hard part. We still screw up one or two things every time, so give yourself a break as you get started.
1. It’s not about winning the discussion.
User interviews are like serious conversations with your spouse or significant other. The immediate goal is not to defend yourself or fix the problem (although far too many people try). Rather, the immediate goal of this sort of user call is to make the other person feel heard. Listen quietly and show them you understand their feelings. Make them feel like part of the team.
When you’re listening to someone provide negative feedback, your pride will make it tempting to defend design choices, explain away problems, or make counter-suggestions. Instead, your product should persuade the user on its own. Your interface will scale, but your personal arguments won’t. Ask questions, take notes, and show gratitude.
2. S#@& will happen, so have a backup.
My military buddies always use the phrase, “Two is one, and one is none.” If you only have a primary option, it will break when you need it most. For example, if you plan to call a user over Skype, expect your Internet connection to die. I often have three or four options for completing the call, such as Skype, traditional phone, Google Hangout, and GoToMeeting.
Just yesterday morning, I sat down to call a user, looked at the user’s account, and realized an account glitch was about to mess up the entire experience. Which brings me to the next commandment . . .
3. Always check the user’s account ahead of time.
At least a few hours before the user interview, inspect the user’s account. You might see some issues that will come up during the discussion. If they’re big, like problems syncing data, have your engineers fix it. If they’re minor issues, file them away as potential questions (“I noticed you haven’t connected your Twitter account. Can you tell me about that?”).
4. Check your marketing before the call.
We use a marketing automation system for our emails to prospective and current customers. It’s easy to forget that these emails can influence a user’s behavior or opinion of a product. Before the call, take a quick look at what correspondence the user has seen recently. Later, this information might help you evaluate the source of problems. For example, was your messaging about a new feature misleading?
5. You have no role in designing or building the product.
Users are much more likely to provide honest feedback if they know the product isn’t your baby. At the beginning of every interview, I go out of my way to tell the user that I had nothing to do with creating the product – I’m just an interested third party who wants to get the user’s honest feedback. I’ll also agree with negative feedback as much as possible to show the user that I’m on their side. If that means admitting something like, “Yeah, I said the same thing to our engineers the other day,” then do it. Honest feedback is the holy grail.
6. Go where the user takes you.
It’s tempting to want to keep user interviews structured. Instead, in the early part of the discussion, adjust your questions to whatever the user brings up. Asking them to provide stream-of-conscious narration while they play around is particularly helpful. You can ask specific questions at the end. So why abandon structure?
It’s important to understand how users behave when they’re on their own. At home, nobody opens a new app and methodically tries all the features in order of importance. If the user wanders off on a tangent, that might represent an issue with your design. Plus, if you indulge tangents, you might learn something about a related issue. For example, I recently let a user wax philosophic about our marketing emails. It unexpectedly helped me understand a potential flaw in our messaging.
7. Remember, every problem is your fault.
These days, with so many apps, browsers, and other programs in the mix, a problem in your app can be caused by an outside source. For example, a bug in Chrome messes up how your webapp displays.
Is that your fault?
Of course it is . . . at least for purposes of the discussion. Assume the user will blame you for any problem that happen while your app is open (and sometimes when it is not), so listen carefully to their response during the interview. Knowing how they react will help you make changes to account for extraneous influences.
8. Provide a roadmap.
Near the end of the call, I give users an idea of our endgame for the product. I discuss what we envision the product will be in a few years, and I also give them a preview of upcoming features. The idea is to show the user that the product is still in its infancy and needs months of feedback. The user will come away with the impression that they got an early look at an important product, and they will also feel like part of the team.
9. The most valuable part is after it ends.
If you’ve conducted the call properly, you’ve likely recruited a permanent member of your advisory team. That person will continue using the product and giving you feedback–whether they like the product or not–because they feel like they’re in on something cool. Stay in contact with the user and schedule follow-up discussions.
Note: Just like job interviews and holiday gifts, always send a thank you card or gift after completing a user interview. We use fellow Denver Startup Printfection to send redemption codes for a free FullContact t-shirt to all our user interviewees. Who doesn’t like t-shirts? Plus, t-shirts are mobile billboards!
10. Move on.
After a particularly negative discussion with a user, let it go. Remember that some people, for whatever reason, are just looking to vent. If you catch anyone on the wrong day, they might be predisposed to ripping your product a new one – even if the product is functional and well-designed. I make fun of Facebook every single day . . . but I keep signing in the next. Is Facebook poorly designed? Or am I just blaming it for how much wasted time I spend on the Internet?
As you conduct more and more user discussions, you’ll get a feel for how your product is doing in the aggregate. You’ll also get a handle on some of the issues that might cause a user to trash your product, such as misunderstanding its purpose, preferences for different design schemes (Mac vs. Windows anyone?), or scheduling the discussion at the wrong time of the week.