There is a saying among sales reps that “buyers are liars.” This saying comes from sales reps who have fulfilled various customer requests in order to make a sale, only to find that customers still didn’t buy. Customers don’t mean to lie. In fact, they feel they are telling you what they honestly believe. The problem is two-fold: 1) customers are good at describing features they want, but are lousy product managers and innovators and 2) a single voice within a company does not speak for the needs of the entire organization. Many startups have failed by building exactly what a single customer told them to build without validating the solution with the entire buying panel. Wearing your product manager hat, you need to arm yourself with tools that will allow you to accurately listen to the right customer, cut to the core issues of the customer buying panel, and capture a true picture of what really solves the customer pain. Here are three tools we have used to find out what customers really want.
Rating System - A great method to help focus the conversation is to use a rating system. You may consider asking your customers to rate which features are of most value to them. When having customers rate features and products, it is important to keep them focused on a single market and keep customer comments within the context of 1) the market they are in and 2) the problem they are trying to solve within that market. If you take the highest rated features from different customer groups (e.g., corporate customer, retail customer, etc.) and put them all together, you may end up building a product that almost meets several customers’ needs and meets no specific customer group.
Feature Testing - Another way to determine where to focus when showing a prototype to customers is through feature testing. This approach includes tactics such as A/B testing, surveys, and feature tracking. The use of these metrics can help you identify the features customers really care about. Many companies use A/B testing, where they show one version of a product to one set of customers and another version of the product to a second set of customers, then determine which version customers prefer.
The $100 Game - If you show a prototype to customers and their responses seem inconclusive, or they give you a long list of features they want, try the $100 game to focus the conversation. This method is slightly better than the rating system because it not only gives you a rating, but also a relative weighting for each level. To do this, ask your customer, “If you had $100 to invest in any features of this product, what features would you invest your money in?” Customers will often vote for the features they most care about.
For example, when ClassTop first took their prototype out to customers, they had about 20 key features which they believed customers would value. The initial customer reaction to their prototype was moderately positive, but not stellar. In one meeting with a university official, Jared Allgood asked how much the official might expect to pay for the product. The answer was only about $200 a month - not a great margin to build a business on. However, as ClassTop continued their roadshow they used the $100 game and discovered something staggering. On average, customers allocated about $80 on a drag-and-drop file-transfer feature and $20 on two other features. ClassTop soon realized they could remove the other seventeen features (and drop the future time and development costs) they thought were part of the Minimum Feature Set. More importantly, customer interest in this simpler product skyrocketed. In fact, when the team returned to the very same university official with the simplified product (three features instead of twenty), he was not only more enthusiastic than before, but when asked how much he might pay for such a product, he responded that he would easily pay $1,000 or more a month - a 5x increase!
If you haven't already done so, we encourage you to download the following interview guide, which will help you with customer conversations as you go through the Nail It Then Scale It process.
(See Nail It Then Scale It, pgs. 124-126)