Creating valuable products requires getting valuable feedback. Who you talk to early on counts for a whole lot.
A few weeks ago my co-founder and I started privately beta testing our company’s second product.
Our first product, Exist, has been in public beta for almost a year. We made lots of mistakes which, thankfully, we’ve learned from.
This time around, we know what to do differently—so we can make fresh mistakes this time, no doubt. Here are a few lessons I’d love to share.
Choose your users carefully
Our biggest mistake with beta testing was choosing the wrong users. We tried to protect our fragile egos and only invited people we knew. Our hope was to get honest (but polite) feedback to help us improve the product without being thrown into a pit of despair before we even launched.
The reality hit even harder: we heard crickets.
We couldn’t get the feedback we needed because we weren’t talking to users who cared. Our “users” were people who had five minutes to spare and no real inclination to tell us what they wanted to do with our product.
Why? Because they weren’t people who needed our product. We weren’t solving any pains for them. They didn’t see the value and didn’t hang around long. Most of them vanished after one login or came back once a month if we were lucky.
Don’t waste time (on both sides) with users you aren’t serving. You’ll never get the feedback you need from people who don’t need you.
For a new product you must look for users who feel the pain, see a need, and are willing to invest to get value.
As you build up your business over time this becomes a lot easier. With our second product we’re focusing on the same type of customer as we did with Exist, so we’ve already got a targeted group of people from which to pull beta testers. When you build a relationship with people who use your products, they’re often more likely to be interested in other projects you work on.
You can see this in other businesses, too. When Basecamp was still known as 37 Signals, it had several products targeting the same user base. It’s much easier to find beta testers, and eventually paying users, from a pool of people who already know and trust you.
If you’re starting out with your first product, you obviously won’t have the benefit of an existing user base, but you can build an audience in other ways. More than six months before we launched Exist to the public, I started a regular content series on our blog that focused on news and new products in the Quantified Self space. As I continued this series every week, we built up an archive of content relevant to our target market and developed an audience of people who were interested in those topics.
We created an email waiting list for Exist to keep our potential customers updated with our progress, and through regular email updates and creating content they were interested in, we were able to build a relationship with lots of our customers before they ever used our product.
In later rounds of beta testing, we surveyed our waiting list to find people who were interested in our product and understood its value. In retrospect, we should have taken this approach much earlier.
Asking for feedback
The quicker you can get feedback the better. Good ideas get to speak and the duds are silenced.
Misunderstanding your users is dangerous. You won’t know which direction is best for your product, and you won’t know why customers leave (or why they stay). Feedback is the fix.
It’s easy to feel like you’re bugging people, especially early on. But like anything in life, you won’t know until you ask, and often people are much more willing to help than you think.
Expecting customers to send you unsolicited feedback is expecting too much. You’re responsible for starting the conversation.
Talking to my customers in real-time (either in person or via Skype) is one of the best ways I’ve found to uncover useful feedback. When you’re talking to a user one-on-one this way, you can dig deeper into the little things they say and let them go off on tangents. It’s the best way to get a well-rounded picture of who this user is and how your product fits into their life.
But sometimes that’s not the kind of feedback you need. When we wanted to know which parts of our product were most popular, we sent out a survey to all our users via email. It wasn’t as personal as a one-on-one call, but we were able to gather information from lots of our users in a short period of time, and we could ask exactly what we wanted to know.
Sometimes you just want a clear, overall rating of your product or service. In those cases you can use a simple question like, “Would you recommend our product to a friend?” or you can ask your customers to rate your customer service after you’ve solved their problem.
The way you ask for feedback should depend on what type of feedback you’re looking for.
Stick to one major feedback approach at a time in the early days. Dilution of responses and user annoyance can occur if it feels like you are bombarding people. The same month you’re making a concerted effort to do development interviews, don’t send out a survey bugging people for similar information. Ongoing, focused feedback creates the most meaningful insight.
Listening to the vocal minority
Ah, the vocal minority. When you’re struggling to get feedback, any little bit seems like solid gold.
This makes it all too easy to fall into the vocal minority trap. You hear a handful of users ask for a feature that you’ve already considered building and suddenly you think every user must want it. Clearly, they just haven’t got around to telling us! Let’s ship it already!
Worse yet, you hadn’t considered building a feature but six or seven users ask for it on the same day. The next thing you know you’re rolling something out without nearly enough context. It’s easy to jump the gun from a sense of urgency and demand that isn’t really there. Sounds silly, but we’re all susceptible.
To make things people want, first prove they actually want it.
Feedback is best used to create a hypothesis for what the majority of your users might want. As Des Traynor says, “Treat every clustering of feedback that you see as a hypothesis, and then don’t build it, verify it.”
You can then do further customer development to see if your hypothesis holds up.
If it holds up that the majority of users do feel the same way, that’s when you can start digging in deeper to find out why they want this particular feature and how you can solve that problem for them (it’s about the friction, not the feature).
For Exist feedback, we use Help Scout and tags to keep an always-updated count of requests for a particular feature. It’s quick and easy to add a tag when we’re replying to a customer, and seeing the number of requests per tag makes it easier to choose what to work on next without being overwhelmed by the vocal minority.
Feedback is a multi-faceted part of building a company. Whether you’re getting too much, you have confusing or inconsistent feedback, or you’re just hearing crickets, you’re not alone.
Choose customers carefully, ask for feedback often, and always test a hypothesis before implementing what you hear from customers.
It’s easier said than done, but every improvement I make in handling customer feedback has made me better at building the best product for my users.