Your Mom is Not a Valid Test Market

mom
Bart Lorang

Author: Bart Lorang

@bartlorang

I recently asked a startup founder if he’d gotten some real feedback on his product.

His response: “I’ve shown it to all my friends and family, and they said they really like it.”

I threw up in my mouth a little bit. Not good. His app sucked (and I told him that), and this poor guy was getting grinfucked by his friends and family.

When soliciting feedback on your product, GET AS FAR AS HUMANLY POSSIBLE AWAY FROM FRIENDS AND FAMILY!

Not a test market
Shirt on the back of @heyrichu
by @mattemmi and @andrewhyde

The truth is, outside your friends and family, nobody really gives a shit about you or your product.  And that’s a good thing.  Why?  I’ve found that they’re more likely to give you honest feedback.

The sooner you realize this, the sooner you’ll be on your way to building something other people actually care about.

My mom managed to remind me of this the other day.

As part of our ongoing product development process, I send out weekly cohort surveys and ask our users to give us honest, direct feedback. As usual, no grinfucking allowed.

I ask 3 questions:

  1. On a scale of 0-10, how likely are you to recommend us to a friend or colleague?
  2. How disappointed would you be if you could no longer use us?
  3. What is one thing we can improve?

I was perusing this week’s cohort results and the average responses seemed much better than last week’s. Initially, I was thrilled. But then I drilled deeper.

Turns out, I received a survey response that looked something like this:

  1. 10 – Extremely Likely to Recommend
  2. Very Disappointed
  3. “Nothing honey, you’re doing great. We’re so proud of you. Love, Mom.”

#FACEPALM

I also received 3 more responses that looked something like this:

  1. 10 – Extremely Likely to Recommend
  2. Very Disappointed
  3. “It didn’t really work at all, but we’re trying to work with you guys and hope you guys will help us get into TechStars, so I’m going to give you high ratings anyway.”

#DOUBLEFACEPALM

I hate these types of responses. They skew my survey results and they are basically useless to me.

I don’t need praise. I have my Mom for that.

As a startup founder, you should want criticism.

Biting, scathing, painful criticism.

It doesn’t even need to be constructive.

Just real opinions. It’s just data.

Want to know the best thing you can do for a fellow startup founder? Give them your honest opinion. Don’t be a grinfucker.

  • trapezium

    I nearly always test my web interfaces on my Mum.

    Seems like a no-brainer. Digital natives know what to do when they’re presented with a button that says “don’t process any more dialogues on this page?” or similar. Any feedback they give is skewed.

    I want to test on real users. My mum’s a real user. I can put you in touch with her if you want?

    • http://fullcontact.com/ Bart Lorang

      valid point. mom’s can be good test case of “real users” – but the point was that friends and family often will recommend your product, or tell you it’s a great idea, even if it’s not.

  • Dianne Lorang

    I’ll do more than like this on facebook, I’ll comment that I like this because I am this blogger’s mom!

  • SallyBroom

    Surely it depends who your market is? If you’re building a consumer product for people like your Mum, the biggest concern is if she *doesn’t* use/love it.

    As a startup, the best place to start getting your product ‘to market’ and getting feedback is starting with the people who can get your hands on, without spending marketing cash trying to connect with people who are further out of reach with a lesser tester product.

    Totally agree that just being told by your family ‘honey you’re doing a great job’ is a big risk, but getting them and their friends to part with hard-earned cash and give you feedback is actually a very valid test market if relevant to your product.

    You mention Techstars and that makes me think of a more interesting behaviour: startups obsessing over the opinions of their tech peers who are actually not their core target market, only to find they go bust because they spend so much time doing what their tech peers told them to do instead of what their paying customer wanted.