My user testing template and favorite resources from YC & others.

January 7, 2022

7 Views 0

SaveSavedRemoved 0

[ad_1]

If you’re just here for the template, it’s at the bottom of the page . No signup required.

I’ve worked as a product manager for a while. As a result, I’ve had the pleasure of observing and conducting quite a few user tests. Once I began building my own startup, I realized how instrumental those skills are. I’ve used them extensively to help guide the roadmap for Deft in the right direction.

There’s a lot of great information out there on “why” talking directly with your customers is important, so I won’t spend time lecturing on that subject [I’ll link a few posts in the notes]. Just know that it’s absolutely essential to the success of a startup. That being said, I will focus on the “how” as it applies to the template.

Top problems observed in user tests

Being a part of Pioneer, Indiehackers, and other early stage startup communities has made me realize that a lot of people don’t come from my same background and have a lot of trouble user testing. I’ll address some of the major themes I’ve seen.

Feel free to post some of your own challenges in the comments and I’ll answer them in the thread!

Uncomfortable talking to customers

This one is difficult to overcome. Usually, there are a lot of mental fallacies involved. Founders might be fooling themselves into thinking how much more important it is to engineer than it is to talk to users. Maybe they think that since they are their own customer, they know exactly what others will want. Perhaps they are a perfectionist and they feel they just aren’t ready to show the product yet.
I usually find these to be deflection from the root issue: discomfort talking to users. My recommendation here would be to try simple, short calls at first. No video required, no in-person. Just hop on a call, give someone your product link over text or email, and ask for feedback. Don’t spend more than 5 minutes.

Note: This is going to give you the wrong type of information back (I’ll talk about this later in the post), but it does help people realize how painless talking to users is.

It gets easier over time and as you iterate and eventually see the delight of your customers, you might even get a little excited to do them!

Asking questions the wrong way

You’re all jacked up on coffee and have everything setup for a great test. The user pops on and you tell them all about the thing you’re building. You ask them things like “Would you use SuperCoolApp?”, “What features do you want in my SuperCoolApp?”. These are the wrong types of questions and the wrong way to introduce a user test.

Here’s one of my favorite quotes from Henry Ford:

“If I had asked people what they wanted, they would have said faster horses.”

Why does this happen? Because you’re asking leading questions and giving the tester too much context. In real life, users are probably going to experience your product without a helping hand, so your user test should reflect that.

The right way to ask questions is to keep them open ended. When a user hits a sticky point on your app, you should be asking things like “What did you think would happen?” Resist the urge to tell the user what should happen or your own thoughts. Lack of bias, keen observation, and the ability to ask timely open ended questions is what makes these tests superior to watching screen recordings from things like Fullstory or Hotjar.

Testing too many people and wasting time

We often get two conflicting pieces of information in the startup world: Talk to users and build fast. However, talking to users takes time, so at what point do I stop talking to users before I build the thing?

From my experience, you can get amazing results from talking with as few as 5 people. With the caveat being that you were precise with who you’re testing with (e.g. If the person doesn’t fit one of your personas, you might get some misleading patterns).

Jakob Nielsen, PHD and former VP of research at Apple popularized the idea with a study that showed diminishing returns after 5 users. Saying that “after the fifth user, you are wasting your time by observing the same findings repeatedly but not learning much new” [source]

The cycle I generally follow here is:

Test with 5 users or potential users
Analyze the patterns
Build an MVP from those patterns
Test with 5 new people
Rinse and repeat for eternity

I do this for small features all the way up to entirely new products or businesses. Also important is there’s no wrong time to user test. I’ve seen people get useful feedback from testing with nothing more than a napkin drawing and some imagination.

How to conduct a user test

The Setup

Getting someone in person gives some unique advantages over doing it remotely. Especially if your prototype is rough around the edges. That being said, it’s often more difficult to get someone to commit to it. Here are some solid guidelines for either option from the team at Google Ventures.

The Customer Interview Process

I usually default to Google Venture’s process for this.

You’re going to want to get them in the right mindset. The steps are:

Welcome
Context Questions
Introduce the prototype
Scenarios (Tasks)
Quick Debrief

Tip: Don’t take notes during the interview if you’re the conductor. Delegate it to someone else watching or save it for when you’re recording later. You’ll miss really key insights otherwise.

The Analysis

Once you’re done with the user tests, you’ll notice some pretty obvious patterns. Time to turn those into a roadmap. Generally, any thing observed in 3/5 testers I consider a definitive insight and group it into a category. An example for Deft would be “Users who didn’t understand what this icon was” or “Users who loved the image search!”. Whether positive, neutral, or negative, you’ll know what to do.

Things at 2/5 or 1/5 may warrant additional investigation, but likely aren’t the most important things to focus on. I keep a record of these somewhere for a rainy day.