Lean Analytics

Lean startup methodology is a way to mitigate/ reduce risk by prioritizing learning. Learn what you can sell, then make it.

The Lean Startup movement is galvanizing a generation of entrepreneurs. It helps you identify the riskiest parts of your business plan, then funds ways to reduce those risks in a quick, iterative cycle of learning. Most of its insights boil down to one sentence: Don’t sell what you can make; make what you can sell. And that means figuring out what people want to buy

Customer development: using continuous feedback from potential customers/ the market to guide product/ business development

Customer development—a term coined by entrepreneur and professor Steve Blank—took direct aim at the outdated, “build it and they will come” waterfall method of building products and companies. Customer development is focused on collecting continuous feedback that will have a material impact on the direction of a product and business, every step of the way.

Lean analytics focuses on measuring better so you can learn faster

One of Lean Startup’s core concepts is Build→Measure→Learn—the process by which you do everything, from establishing a vision to building product features to developing channels and marketing strategies. Within that cycle, Lean Analytics focuses on the measure stage. The faster your organization iterate through the cycle, the more quickly you’ll find the right product and market. If you measure better, you’re more likely to succeed

Qualities of a good metric

A good metric is comparative. Being able to compare a metric to other time periods, groups of users, or competitors helps you understand which way things are moving.“Increased conversion from last week” is more meaningful than “2% conversion.

A good metric is understandable. If people can’t remember it and discuss it, it’s much harder to turn a change in the data into a change in the culture9

Ratios are usually the best metrics

There are several reasons ratios tend to be the best metrics:

• Ratios are easier to act on. Think about driving a car. Distance travelled is informational. But speed—distance per hour—is something you can act on, because it tells you about your current state, and whether you need to go faster or slower to get to your destination on time.
• Ratios are inherently comparative. If you compare a daily metric to the same metric over a month, you’ll see whether you’re looking at a sudden spike or a long-term trend. In a car, speed is one metric, but speed right now over average speed this hour shows you a lot about whether you’re accelerating or slowing down.
• Ratios are also good for comparing factors that are somehow opposed, or for which there’s an inherent tension. In a car, this might be distance covered divided by traffc tickets. The faster you drive, the more distance you cover—but the more tickets you get. This ratio might suggest whether or not you should be breaking the speed limit.

At early stages, you can only look for qualitative information/ data. Ask specific questions without leading the witness. Prepare for your interviews.

Initially, you’re looking for qualitative data. You’re not measuring results numerically. Instead, you’re speaking to people—specifcally, to people you think are potential customers in the right target market. You’re exploring. You’re getting out of the building. Collecting good qualitative data takes preparation. You need to ask specifc questions without leading potential customers or skewing their answers. You have to avoid letting your enthusiasm and reality distortion rub off on your interview subjects. Unprepared interviews yield misleading or meaningless results.

Leading vs lagging indicators

Both leading and lagging metrics are useful, but they serve different purposes. A leading metric (sometimes called a leading indicator) tries to predict the future. For example, the current number of prospects in your sales funnel gives you a sense of how many new customers you’ll acquire in the future

A lagging metric, such as churn (which is the number of customers who leave in a given time period) gives you an indication that there’s a problem—but by the time you’re able to collect the data and identify the problem, it’s too late. The customers who churned out aren’t coming back

In an enterprise software company, quarterly new product bookings are a lagging metric of sales success. By contrast, new qualifed leads are a leading indicator, because they let you predict sales success ahead of time. But as anyone who’s ever worked in B2B (business-to-business) sales will tell you, in addition to qualifed leads you need a good understanding of conversion rate and sales-cycle length. Only then can you make a realistic estimate of how much new business you’ll book

Numbers don’t beat knowing your customer

Know your customer. There’s no substitute for engaging with customers and users directly. All the numbers in the world can’t explain why something is happening. Pick up the phone right now and call a customer, even one who’s disengaged.

Test assumptions but don’t over-experiment. Use qualitative data to make progress more rapidly.

Make early assumptions and set targets for what you think success looks like, but don’t experiment yourself into oblivion. Lower the bar if necessary, but not for the sake of getting over it: that’s just cheating. Use qualitative data to understand what value you’re creating and adjust only if the new line in the sand refects how customers (in specifc segments) are using your product

Testing is about comparing two options to find the better one

Testing is at the heart of Lean Analytics. Testing usually involves comparing two things against each other through segmentation, cohort analysis, or A/B testing.

Segmentation can work for products just as well as markets

A segment is simply a group that shares some common characteristic. It might be users who run Firefox, or restaurant patrons who make reservations rather than walking in, or passengers who buy first-class tickets, or parents who drive minivans. On websites, you segment visitors according to a range of technical and demographic information, then compare one segment to another. If visitors using the Firefox browser have signifcantly fewer purchases, do additional testing to fnd out why. If a disproportionate number of engaged users are coming from Australia, survey them to discover why, and then try to replicate that success in other markets. Segmentation works for any industry and any form of marketing, not just for websites. Direct mail marketers have been segmenting for decades with great success

Cohorts are like segments but segmented by time

A second kind of analysis, which compares similar groups over time, is cohort analysis. As you build and test your product, you’ll iterate constantly. Users who join you in the frst week will have a different experience from those who join later on. For example, all of your users might go through an initial free trial, usage, payment, and abandonment cycle. As this happens, you’ll make changes to your business model. The users who experienced the trial in month one will have a different onboarding experience from those who experience it in month fve. How did that affect their churn? To find out, we use cohort analysis

Longitudinal studies vs cross-sectional studies vs A/B testing

Cohort experiments that compare groups like the one in Table 2-2 are called longitudinal studies, since the data is collected along the natural lifespan of a customer group. By contrast, studies in which different groups of test subjects are given different experiences at the same time are called cross-sectional studies. Showing half of the visitors a blue link and half of them a green link in order to see which group is more likely to click that link is a cross-sectional study. When we’re comparing one attribute of a subject’s experience, such as link color, and assuming everything else is equal, we’re doing A/B testing

Don’t waste your life building something nobody wants

The reason you want to be lean and analytical about the process is so that you don’t waste your life building something nobody wants. Or, as Netscape founder and venture capitalist Marc Andreesen puts it, “Markets that don’t exist don’t care how smart you are"

Your product should be the intersection of what you’re good at, what you’re interested in and what you can make money doing

Strategic consultant, blogger, and designer Bud Caddell has three clear criteria for deciding what to spend your time on: something that you’re good at, that you want to do, and that you can make money doing

Can I do this thing I’m hoping to do, well? This is about your ability to satisfy your market’s need better than your competitors, and it’s a combination of design skill, coding, branding, and myriad other factors. If you identify a real need, you won’t be the only one satisfying it, and you’ll need all the talent you can muster in order to succeed. Do you have a network of friends and contacts who can give you an unfair advantage that improves your odds? Do you have the talent to do the things that matter really well? Never start a company on a level playing feld— that’s where everyone else is standing. These same rules apply to people working in larger organizations. Don’t launch a new product or enter a new market unless your existing product and market affords you an unfair advantage. Young competitors with fewer legacies will be fghting you for market share, and your size should be an advantage, not a handicap

Be sure you can make money doing it.* This is about the market’s need. You have to be able to extract enough money from customers for the value you’ll deliver, and do so without spending a lot to acquire those customers—and the process of acquiring them and extracting their money has to scale independent of you as a founder. For an intrapreneur, this question needs to be answered simply to get approval for the project, but remember that you’re fghting the opportunity cost—whatever the organization could be doing instead, or the proftability of the existing business. If what you’re doing isn’t likely to have a material impact on the bottom line, maybe you should look elsewhere. This is by far the most important of the three; the other two are easy, because they’re up to you. But now you have to fgure out if anyone will pay you for what you can and want to build.* Chter 3: Decng Wt to Do with Your Lie 35

3 questions to answer

The early stages of a startup, you’ll be dealing with a lot of data. You’re awash in the tides of opinion, and buffeted by whatever feedback you’ve heard most recently. Never forget that you’re trying to answer three fundamental questions:

• Have I identifed a problem worth solving?
• Is the solution I’m proposing the right one?
• Do I actually want to solve it?

Use data as a tool, don’t let it use you. Be data-informed, not data-driven.

Data is a powerful thing. It can be addictive, making you overanalyze everything. But much of what we actually do is unconscious, based on past experience and pragmatism. And with good reason: relying on wisdom and experience, rather than rigid analysis, helps us get through our day. After all, you don’t run A/B testing before deciding what pants to put on in the morning; if you did, you’d never get out the door. One of the criticisms of Lean Startup is that it’s too data-driven. Rather than be a slave to the data, these critics say, we should use it as a tool. We should be data-informed, not data-driven. Mostly, they’re just being lazy, and looking for reasons not to do the hard work. But sometimes, they have a point: using data to optimize one part of your business, without stepping back and looking at the big picture, can be dangerous—even fatal.

Humans vs machines. May not be as true with latest advances in AI. Machines can now be creative, whereas increasingly humans are doing verification.

Humans do inspiration; machines do validation

Change vs innovation. Don’t just change, innovate.

change favors local maxima; innovation favors global disruption

Try to find a more global maxima than those given by machines

Machine-only optimization suffers from similar limitations as evolution. If you’re optimizing for local maxima, you might be missing a bigger, more important opportunity. It’s your job to be the intelligent designer to data’s evolution

Definition of customer breakeven

The two knobs on this machine are customer lifetime value (CLV) and customer acquisition cost (CAC). Making more money from customers than you spend acquiring them is good, but the equation for success isn’t that simple. You still need to worry about cash fow and growth rate, which are driven by how long it takes a customer to pay off. One way to measure this is time to customer breakeven—that is, how much time it will take to recoup the acquisition cost of a customer

Survey question to measure product-market fit

how do you know if you’ve achieved product/market ft? Sean devised a simple survey that you can send customers (available at survey.io) to determine if you’re ready for accelerated growth. The most important question in the survey is “How would you feel if you could no longer use this product or service?” In Sean’s experience, if 40% of people (or more) say they’d be very disappointed to lose the service, you’ve found a ft, and now it’s time to scale

The “long funnel” of the modern internet

In the early days of the Web, transactional websites had relatively simple conversion funnels. Visitors came to the home page, navigated to the product they wanted, entered payment information, and confrmed their order. No more. Today’s funnel extends well beyond the front door of a website, across myriad social networks, sharing platforms, affliates, and pricecomparison sites. Both offine and online factors infuence a single purchase. Customers may make several tentative visits prior to a conversion. We call this the Long Funnel

Lean Analytics: a new framework that draws from other product growth frameworkds

There are a number of good frameworks that help you think about your business.

• Some, like Pirate Metrics and the Long Funnel, focus on the act of acquiring and converting customers.
• Others, like the Engines of Growth and the Startup Growth Pyramid, offer strategies for knowing when or how to grow.
• Some, like the Lean Canvas, help you map out the components of your business model so you can evaluate them independent of one another.

We’re proposing a new model called the Lean Analytics Stages, which draws from the best of these models and puts an emphasis on metrics. It identifes five distinct stages startups go through as they grow

3 engines that drive business growth. Importance of focusing on one engine at a time and then optimizing one metric for that engine.

Eric Ries talks about three engines that drive company growth: the sticky engine, the viral engine, and the paid engine. But he cautions that while all successful companies will ultimately use all three engines, it’s better to focus on one engine at a time. For example, you might make your product sticky for its core users, then use that to grow virally, and then use the user base to grow revenue. That’s focus. In the world of analytics and data, this means picking a single metric that’s incredibly important for the step you’re currently working through in your startup. We call this the One Metric That Matters (OMTM)

Focus on a single or a very small number of KPIs

Interestingly, when Moz raised its last round of financing, one of its lead investors, the Foundry Group’s Brad Feld, suggested that it track fewer KPIs. “The main reason for this is that as a company, you can’t simultaneously affect dozens of KPIs,” Joanna says. “Brad reminded us that ‘too much data’ can be counterproductive. You can get lost in strange trends on numbers that aren’t as big-picture as others. You can also lose a lot of time reporting and communicating about numbers that might not lead to action. By stripping our daily KPI reporting down to just a few metrics, it’s clear what we’re focused on as a company and how we’re doing.

Let’s look at four reasons…

Let’s look at four reasons why you should use the One Metric That Matters.

• It answers the most important question you have. At any given time, you’ll be trying to answer a hundred different questions and juggling a million things. You need to identify the riskiest areas of your business as quickly as possible, and that’s where the most important question lies. When you know what the right question is, you’ll know what metric to track in order to answer that question. That’s the OMTM.
• It forces you to draw a line in the sand and have clear goals. After you’ve identifed the key problem on which you want to focus, you need to set goals. You need a way of defning success.
• It focuses the entire company. Avinash Kaushik has a name for trying to report too many things: data puking. Nobody likes puke. Use the OMTM as a way of focusing your entire company. Display your OMTM prominently through web dashboards, on TV screens, or in regular emails.
• It inspires a culture of experimentation. By now you should appreciate the importance of experimentation. It’s critical to move through the build→measure→learn cycle as quickly and as frequently as possible. To succeed at that, you need to actively encourage experimentation. It will lead to small-f failures, but you can’t punish that. Quite the opposite: failure that comes from planned, methodical testing is simply how you learn

Marketing = sell more stuff (product) to more people (distribution) more often (frequency) for more money (price) more efficiently (cost)

Sergio Zyman, Coca-Cola’s CMO, said marketing is about selling more stuff to more people more often for more money more efficiently.

Different kinds of users

Business models are about getting people to do what you want in return for something. But not all people are equal. The plain truth is that not every user is good for you. Some are good—but only in the long term. Evernote’s freemium model works partly because users eventually sign up for paying accounts, but it can take them two years to do so. Some provide, at best, free marketing, and while they may never become paying users, they may amplify your message or invite someone who will pay. Some are downright bad—they distract you, consume resources, spam your site, or muddy your analytics

Products and business models

Product is more than the thing you buy. It’s the mix of service, branding, fame, street cred, support, packaging, and myriad other factors you pay for. When you purchase an iPhone, you’re also getting a tiny piece of Steve Jobs’s persona. In the same way, a business model is a combination of things. It’s what you sell, how you deliver it, how you acquire customers, and how you make money from them. Many people blur these dimensions of a business model. We’re guilty of it, too. Freemium isn’t a business model—it’s a marketing tactic. SaaS isn’t a business model—it’s a way of delivering software. The ads on a media site aren’t a business model—they’re a way of collecting revenue

12 types of revenue models

The team at Startup Compass, a startup dedicated to helping companies make better business decisions with data, identifes 12 revenue models: advertising, consulting, data, lead generation, licensing fee, listing fee, ownership/ hardware, rental, sponsorship, subscription, transaction fee, and virtual goods

Freemium is a GTM tactic

While freemium gets a lot of visibility, it’s actually a sales tactic, and one you need to use carefully.

• In SaaS, churn is everything. If you can build a group of loyal users faster than they erode, you’ll thrive.
• You need to measure user engagement long before the users become customers, and measure customer activity long before they vanish, to stay ahead of the game
• Many people equate SaaS models with subscription, but you can monetize on-demand software in many other ways, sometimes to great effect.</p></blockquote>

Freemium may be unsustainable for many businesses

Proponents of a free model point out that adoption and attention are the most precious of currencies. Twitter waited until it had millions of active users before introducing advertising, and despite the outcry over promoted tweets, growth has continued. Chris Anderson, former editor-in-chief of Wired and author of The Long Tail (Hyperion), observes that King Gillette pioneered the idea of giving something away (handles) to make money on something else (razor blades).* But in many ways, online users have strong expectations that the Internet should be free, which means it’s hard to charge even for valuable things. Detractors of freemium models observe that for every success like Dropbox or LinkedIn, there’s a deadpool of others who went out of business giving things away. In one example cited by the Wall Street Journal, billingmanagement software frm Chargify was on the brink of failure in 2010— but then it switched to a paid model, and in July 2012, became proftable with 900 paying customers. † Neil Davidson is concerned with the popularity of freemium, particularly among startups. “I think that for most people the freemium model is unsustainable,” he says.“It’s very hard to create something good enough that people will want to use, but with enough of a feature gap to the paid version so that people will upgrade. ” Neil believes that too many startups charge too little, and undervalue themselves.“If you’re creating something that your customers value, then you shouldn’t shy away from asking them to pay for it. If you don’t, you haven’t got a business.

Recognize that in a free-to-play multiplayer game, most users are just “fodder” for paying users. Early on in the user’s lifecycle, identify a leading indicator in her behavior—like time played per day, number of battles, or areas explored—that suggests whether she’s a non-payer, minnow, dolphin, or whale. Then provide different kinds of in-game monetization for these four segments—adapting your marketing, pricing, and promotions according to that behavior—selling bling to minnows, content to dolphins, and upgrades to whales (for example

Two-sided marketplaces

Two-sided marketplaces face a unique problem: they have to attract both buyers and sellers. That looks like twice as much work. As we’ll see in some of the case studies ahead, companies like DuProprio/Comfree, Etsy, Uber, and Amazon found ways around this dilemma, but they all boil down to one thing: focus on whomever has the money. Usually, that’s buyers: if you can fnd a group that wants to spend money, it’s easy to fnd a group that wants to make money. While there’s technically only one stakeholder in a dating site—someone who wants to date—many of the sites that focus on heterosexual relationships treat men and women diferently (for example, free enrollment for female users). We mention it here because the technique has been used to break the chicken-and-egg problem from which marketplaces suffer

You need to seed the marketplace and then measure engagement

The first step of a two-sided marketplace—and the first thing to measure—is your ability to create an inventory (supply) or an audience (demand). DuProprio looked for “for sale by owner” signs and classified listings to build its initial set of listings, and the seller’s lawn sign then drove buyer traffic, so its metrics were listings and lawn signs. The metrics you’ll care about first are around the attraction, engagement, and growth of this seed group

3 categories of marketplace metrics

Josh Breinlinger, a venture capitalist at Sigma West who previously ran marketing at labor marketplace oDesk, breaks up the key marketplace metrics into three categories: buyer activity, seller activity, and transactions.“I almost always recommend modeling the buyer side as your primary focus, and then you model supply, more in the sense of total inventory,” he says.“It’s easy to fnd people that want to make money; it’s much harder to fnd people that want to spend money.

Measure activity that’s core to your business model

Josh cautions that just tracking buyer, seller, and inventory numbers isn’t enough: you have to be sure those numbers relate to the actual activity that’s at the core of your business model.“If you wanted to juice those numbers you could do so quite easily by tweaking algorithms, but you’re not necessarily providing a better experience to users,” he says.“I believe the better focus is on more explicit marketplace activity like bids, messages, listings, or applications. ”

Marketplaces involve a lot of data

There’s a lot of data to track here, because you’re monitoring both buyer e-commerce funnels and seller content creation, as well as looking for signs of fraud or declining content quality. Which metrics you focus on will depend on what you’re trying to improve: inventory, conversion rate, search results, content quality, and so on. For example, if you’re not getting enough click-through from search results to individual listings, you can show less information in initial search results to see if that encourages more click-through. So the metrics you’ll want to watch include:

• Buyer and seller growth: The rate at which you’re adding new buyers and sellers, as measured by return visitors.
• Inventory growth: The rate at which sellers are adding inventory—such as new listings— as well as completeness of those listings.
• Search effectiveness: What buyers are searching for, and whether it matches the inventory you’re building

You can always buy supply, but you can’t buy demand

Long-term, you can always buy supply, but you can’t buy demand. In an attention economy, having an engaged, attentive user base is priceless. It’s the reason Walmart can coerce favorable terms from suppliers and that Amazon can build a network of merchants even though it’s a seller itself. When it comes to sustainable competitive advantage, demand beats supply

Shared marketplaces are regulates by its users

Shared marketplaces are often regulated by the users themselves—users rate one another based on their experience with a transaction. The easiest way to implement this system is to let users flag something that’s wrong, or that violates the terms of service. Users can also rank one another, and sellers work hard to earn a good reputation when the ratings system works well

Keeping the transaction within network is a challenge for marketplaces

One more major issue is keeping the transaction within the network. In the case of a sailboat or house marketplace, the transaction may be tens or even hundreds of thousands of dollars. That’s not really suitable for a PayPal transaction, and it’s hard to stop “leakage”—buyers and sellers fnd one another through your marketplace, and then conclude their business without you getting a transaction fee

Types of marketplaces

Two-sided markets come in all shapes and sizes.

• Early on, the big challenge is solving the “chicken and egg” problem of fnding enough buyers and sellers. It’s usually good to focus on the people who have money to spend frst.
• Since sellers are inventory, you need to track the growth of that inventory and how well it fits what buyers are looking for.
• While many marketplaces take a percentage of transactions, you may be able to make money in other ways, by helping sellers promote their products or charging a listing fee</p></blockquote>

Sometimes the solution to slow growth is to find a new market for an existing product

There’s a natural progression of metrics that matter for a business that change over time as the business evolves. The metrics start by tracking questions like “Does anyone care about this at all?” and then get more sophisticated, asking questions like “Can this business actually scale?” As you start to look at more sophisticated metrics, you may realize your business model is fundamentally flawed and unsustainable. Don’t just start from scratch: sometimes what you need is a new market, not a new product, and that market may be closer than you think.

For early product development: identify and claim your beachhead

That’s an important lesson around business models and Lean Startup—you bring an early version of your product to the market, test its usage, and look for where it’s got the highest engagement among your customers. If there’s a subsection of users who are hooked on your product—your early adopters—figure out what’s common to them, refocus on their needs, and grow from there. Claim your beachhead. It will allow you to iterate much more quickly on a highly engaged segment of the market.

Be aware of the technology adoption lifecycle if your tech is very new

If you’re building something genuinely disruptive, you need to consider the technology adoption lifecycle, from early to mainstream. Hybrid cars, Linux servers, home stereos, and microwaves were frst adopted by a small segment of their markets, but took years of evangelism and millions of marketing dollars to be considered conventional. In the frst stages of your company, you typically have a small, devoted, unreasonably passionate following. This happens because new products initially appeal only to early adopters comfortable with change, or to that segment of the market so desperate for your solution that it’s willing to tolerate something that’s still rough around the edges. Those early adopters will be vocal, but beware. Their needs might not refect those of the bigger, more lucrative mainstream.

Conversion funnels

Conversion funnels: The conversion rates for items sold, and any segmentation that reveals what helps sell items—such as the professional photographs of a property mentioned in the Airbnb case study in Chapter 1. Ratings and signs of fraud The ratings for buyers and sellers, signs of fraud, and tone of the comments. Pricing metrics If you have a bidding method in place (as eBay does), then you care whether sellers are setting prices too high or leaving money on the table.

The conversion funnel will have several stages, starting with the number of searches done by visitors. You should also measure the number of satisfed transactions, because a spike in transactions where one party is unsatisfed suggests that the site is focused on short-term gain (more sales) for longterm pain (a bad reputation, demands for refunds, and so on)

You should also look at the search terms themselves. By looking at the most common search terms that yield nothing, you’ll fnd out what your buyers are after. A dominant search term—say, “Nintendo”—might suggest a category you could add to the site to make navigation easier, or a keyword campaign you could undertake to attract more buyers. You’ll want to know what the most lucrative search terms are, too, because that tells you what kind of seller you should attract to the site. The ratio of searches to clicked listings is also an important step in your conversion funnel

Business plans are for bankers; business models are for founders

In a startup, your business model—and proof that your assumptions are reasonably accurate—is far more important than your business plan. Business plans are for bankers; business models are for founders. Deciding what business you’re in is usually quite easy. Deciding on the stage you’re at is complicated. This is where founders tend to lie to themselves. They believe they’re further along than they really are. The reality is that every startup goes through stages, beginning with problem discovery, then building something, then fnding out if what was built is good enough, then spreading the word and collecting money. These stages—Empathy, Stickiness, Virality, Revenue, and Scale—closely mirror what other Lean Startup advocates advise

Customer retention = Stickiness

The fundamental KPI for stickiness is customer retention. Churn rates and usage frequency are other important metrics to track.Long-term stickiness often comes from the value users create for themselves as they use the service. It’s hard for people to leave Gmail or Evernote, because, well, that’s where they store all their stuff. Similarly, if a player deletes his account from a massively multiplayer online game (MMO), he loses all his status and in-game items, which he’s worked hard to earn.

Viral coefficient = Number of users each user brings on = Virality

The key metric for this engine is the viral coeffcient—the number of new users that each user brings on. Because this is compounding (the users they bring, in turn, bring their own users), the metric measures how many users are brought in with each viral cycle. Growth comes from a viral coeffcient of greater than one, but you also have to factor in churn and loss. The bigger the coeffcient, the faster you grow

Problem discovery before product discovery

At the outset, you’re spending your time discovering what’s important to people and being empathetic to their problems. You’re searching through listening. You’re digging for opportunity through caring about others. Right now, your job isn’t to prove you’re smart, or that you’ve found a solution. Your job is to get inside someone else’s head. That means discovering and validating a problem and then fnding out whether your proposed solution to that problem is likely to work.

The goal of the first Lean stage is to decide whether the problem is painful enough for enough people and to learn how they are currently trying to solve it. Let’s break down what that means: The problem is painful enough People are full of inertia. You want them to act, and you want them to do so in a way that helps your business. This requires enough discomfort with their situation that they actually do what you want—signing up, paying your price, etc. Enough people care Solving a problem for one person is called consulting. You need an addressable market. Marketers want audiences that are homogeneous within (that is, members of the segment have things in common to which you can appeal) and heterogeneous between (that is, you can segment and target each market segment in a focused manner with a tailored message). They’re already trying to solve it If the problem is real and known, people are dealing with it somehow. Maybe they’re doing something manually, because they don’t have a better way. The current solution, whatever it is, will be your biggest competitor at frst, because it’s the path of least resistance for people.

We suggest that you speak with 15 prospective customers to start. After the first handful of interviews, you’ll likely see patterns emerging already. Don’t stop talking to people. Once you get to 15 interviews, you should have the validation (or invalidation) that you need to help clarify the next steps. If you can’t fnd 15 people to talk to, well, imagine how hard it’s going to be to sell to them. So suck it up and get out of the offce. Otherwise, you’re wasting time and money building something nobody wants.

Signs to look for in problem discovery interviews

The key to qualitative data is patterns and pattern recognition. Here are a few positive patterns to look out for when interviewing people:

• They want to pay you right away.
• They’re actively trying to (or have tried to) solve the problem in question.
• They talk a lot and ask a lot of questions demonstrating a passion for the problem.
• They lean forward and are animated (positive body language).

Here are a few negative patterns to look out for:

• They’re distracted.
• They talk a lot, but it’s not about the problem or the issues at hand (they’re rambling).
• Their shoulders are slumped or they’re slouching in their chairs (negative body language)

Understand the problem clearly

Connect with the subject by walking her through how you identifed the problems you’re hoping to solve, and why you think these problems matter. If you’re scratching your own itch, this will be a lot easier. If you don’t understand the problems clearly, or you don’t have good hypotheses for the problems you’re looking to solve, it’s going to show at this point.

Qualitative metrics are all about trends. But you have to be intellectually honest about what those trends are.

Qualitative metrics are all about trends. You’re trying to tease out the truth by identifying patterns in people’s feedback. You have to be an exceptionally good listener, at once empathetic and dispassionate. You have to be a great detective, chasing the “red threads” of the underlying narrative, the commonalities between multiple interviewees that suggest the right direction.Ultimately, those patterns become the things you test quantitatively, at scale. You’re looking for hypotheses. The reality of qualitative metrics is that they turn wild hunches—your gut instinct, that nagging feeling in the back of your mind—into educated guesses you can run with.Unfortunately, because they’re subjective and gathered interactively, qualitative metrics are the ones that are easiest to fake. While quantitative metrics can be wrong, they don’t lie. You might be collecting the wrong numbers, making statistical errors, or misinterpreting the results, but the raw data itself is right. Qualitative metrics are notoriously easy for you to bias. If you’re not ruthlessly honest, you’ll hear what you want to hear in interviews. We love to believe what we already believe— and our subjects love to agree with us

Look for signs that prospects have tried solving the problem already

One of the telltale signs that a problem is worth solving is when a lot of people are already trying to solve it or have tried to do so in the past. People will go to amazing lengths to solve really painful problems that matter to them.Typically, they’re using another product that wasn’t meant to solve their problem, but it’s “good enough,” or they’ve built something themselves.

Don’t underestimate market inertia. Beware of “good enough” solutions.

Too often, idealistic startups underestimate a market’s inertia. They attack market leaders with features, functionality, and strategies that aren’t meaningful enough to customers. Their MVP has too much “minimum” to provoke a change. They assume that what they’re doing—whether it’s a slicker UI, simpler system, social functionality, or something else—is an obvious win. Then “good enough” bites them in the ass. The bar for startups to succeed at any real scale is much higher than that of the market leaders. The market leaders are already there, and even if they’re losing ground, it’s generally at a slow pace. Startups need to scale as quickly as possible. You have to be 10 times better than the market leader before anyone will really notice, which means you have to be 100 times more creative, strategic, sneaky, and aggressive

Even if you find a painful problem, you still need to validate that enough people care about it

If you find a problem that’s painful enough for people, the next step is to understand the market size and potential.Remember, one customer isn’t a market, and you have to be careful about solving a problem that too few people genuinely care about. If you’re trying to estimate the size of a market, it’s a good idea to do both a top-down and a bottom-up analysis, and compare the results. This helps to check your math

Surveys can be a useful next step after one-on-one interviews

LikeBright chose Mechanical Turk to reach people at scale, but there are plenty of other tools. Surveys can be effective, assuming you’ve done enough customer development already to know what questions to ask. The challenge with surveys is fnding people to answer them. Unlike the one-to-one interviews you’ve been conducting so far, here you need to automate the task and deal with the inevitable statistical noise

3 kinds of survey questions

Your survey should include three kinds of questions:

• Demographics and psychographics you can use to segment the responses, such as age, gender, or Internet usage.
• Quantifable questions that you can analyze statistically, such as ratings, agreement or disagreement with a statement, or selecting something from a list.
• Open-ended questions that allow respondents to add qualitative data.

S-shaped growth, or market saturation, is natural and inevitable

Frank Bass, one of the founders of marketing science, described how messages propagated out in a marketplace. † His 1969 paper, “A New Product Growth Model for Consumer Durables,” explained how messages trickle out into a market through word of mouth. At frst, the spread starts slowly, but as more and more people start talking about it, spread accelerates. However, as the market becomes saturated with people who’ve heard the message, spread slows down again. This model is represented by a characteristic S-shape known as the Bass diffusion curve

Artificial virality = incentivizing users to spread the message. Product led growth = marketing activity.

While inherent virality is best, artifcial virality can be bought. Parts of Dropbox are inherently viral—users share files with colleagues and friends—but the company isn’t afraid to compensate its users. It offers additional storage for tweeting or liking the product, and rewards users for helping it to acquire new customers. The rapid growth of the service happened because of existing users trying to convince friends to sign up so they can grow their free online storage capacity. Artifcial virality comes from incentivizing existing users to tell their friends. Done right, it can work well—as Dropbox has shown—but it can also be awkward and feel forced if done poorly. You’re essentially building self-funded marketing activities into the product itself, sometimes at the expense of legitimate functionality.

Build a machine that produces more money than it consumes. Seems simple/ stupid but is ultimately fundamental.

Startup CEOs seeking venture capital would do well to remember the penny machine. It’s a good way to ensure you’re thinking like a venture capitalist. Every time your pitch strays from the simplicity of this meeting, it’s a warning sign that you need to go back and tighten it up. This isn’t just an entertaining metaphor for entrepreneurs preparing to pitch. Think of your company as a machine that predictably generates more money than you put into it. Measuring the ratio of inputs to outputs tells you whether you have a good machine or a broken one

If the result is below 0.75, you have a problem. When you pump money into the machine, less money comes out. That’s a bad thing for this stage of your business, because it means there’s a fundamental faw in your business model. If the result is better than 1, you’re doing well—you can fund your growth with the proceeds, funneling revenue increases back into the machine to increase sales and marketing spend

Per Porter, there are only 3 business strategies: be different (differentiation), target a different market (segmentation) or be more efficient (cost)

Harvard professor Michael Porter describes a variety of generic strategies by which companies compete. Firms can focus on a niche market (a segmentation strategy), they can focus on being efficient (a cost strategy), or they can try to be unique (a differentiation strategy)

Core idea behind Lean Analytics

The core idea behind Lean Analytics is this: by knowing the kind of business you are, and the stage you’re at, you can track and optimize the One Metric That Matters to your startup right now. By repeating this process, you’ll overcome many of the risks inherent in early-stage companies or projects, avoid premature growth, and build atop a solid foundation of true needs, well-defned solutions, and satisfed customers.

Great quote from Churchill

Success is not final, failure is not fatal: it is the courage to continue that counts.

• Sir Winston Churchill

Revenue maximization

What if you are charging? Christopher O’Donnell of Price Intelligently points out that startups are trying to balance revenue optimization (making the most money possible) with unit sales maximization (encouraging wide adoption as the business grows) and value perception (not pricing so low you make buyers suspicious).* Sellers also have to understand how to bundle several features or services into a package, and how to sell these bundles as tiers in order to reach several markets with different price points. Even if you’re charging every customer, you can still experiment with pricing in the form of promotions, discounts, and time-limited offers. Each of these is a hypothesis suitable for testing across cohorts (if you use timelimited offers) or A/B comparisons (if you offer different pricing to different visitors). Alex Mehr, the founder of online dating site Zoosk, understands the “optimal revenue” curve. But he argues that startups should err on the side of charging a bit too little. † “I prefer to make 10% less money but have 20% more customers. You want to stay a little bit to the left side of the peak. It is around 90% of the revenue maximization point. ” Alex overlooks the issues of elasticity, value perception, and strategic discounting in his model, however

Characteristics of enterprise markets

Enterprise buyers tend to be more regulated. They can’t make decisions on gut or emotion—or rather, they can, but it has to be justifed with a business case. Big companies are often public companies with checks and balances. The person who pays for the product (fnance) isn’t the person who uses it (the line of business). Understanding this dichotomy is critical for product development and sales.Initially, you may target early adopters, where the buyer is much closer to the user (they may be the same person at this point), but as you move past early adopters, the buyer and user diverge

The one thing that makes enterprise-focused startups different is this: B2C customer development is polling, B2B customer development is a census. In most cases, enterprise sales involve bigger-ticket items, sold to fewer customers. That means more money from fewer sources. If you’re selling a big-ticket item, this changes the game dramatically. For starters, you can afford to talk to every customer. The high sale price offsets the cost of a direct sales approach, particularly in the early stages of the sale. The small number of initial users makes an even bigger difference. You aren’t talking to a sample of 30 people as a proxy for the market at large.Instead, you’re talking to 30 companies who may well become your frst 30 customers.

Organizations are averse to change, and love the status quo. If you’re trying to sell to them, and your product is still in the early stages of the technology adoption cycle, you’re penalized simply for being new. Consumers love novelty; businesses just call it risk.

A guiding principle for applying innovation to enterprise products: no feature may add to training costs

A central theme to this new wave of innovation is the application of core product tenets from the consumer space to the enterprise. In particular, a universal lesson that I keep sharing with all entrepreneurs building for the enterprise is the Zero Overhead Principle: no feature may add training costs to the user

B2B involves both technology and domain expertise

For all these reasons, most B2B-focused startups consist of two people: a domain expert and a disruption expert.

• The domain expert knows the industry and the problem domain. He has a Rolodex and can act as a proxy for customers in the early stages of product defnition. Often this person is from the line of business, and has a marketing, sales, or business development role.
• The disruption expert knows the technology that will produce a change on which the startup can capitalize. She can see beyond the current model and understand what an industry will look like after the shift, and brings the novel approach to the existing market. This is usually the technologist

Different approaches to starting a start-up

Startups begin in many ways. Over the years, however, we’ve seen a recurring pattern in how B2B startups grow. It usually happens in one of three ways:

• The enterprise pivot: In this pattern, the company creates a popular consumer product, then pivots to tackle the enterprise. This is what Dropbox did, and to some extent it’s the way BlackBerry circumvented enterprise IT by targeting mercenary salespeople. It’s not trivial, though: enterprises have very different expectations and concerns from consumers.
• Copy and rebuild: Another approach is to take a consumer idea and make it enterpriseready. Yammer did this when it rebuilt Facebook’s status update model and copied Facebook’s feed interface.
• Disrupt an existing problem: There are plenty of disruptions that happen to an industry, from the advent of mobile data, to the Internet of Things,* to the adoption of the fax machine, to location-aware applications. Any of them can offer a big enough advantage to make it worth discarding the old way of doing things. Taleo did this to the traditional business of human resources management

Many bootstrapped startups begin their lives as consulting organizations. Consulting is a good way to discover customer needs, and it helps pay the bills. It also gives you a way to test out your early ideas, because while every customer has needs, the only needs you can build a business on are those that are consistent across a reasonably large, addressable market. Having said that, consulting companies struggle a great deal to transition from service providers to product companies because they need to, at some point, abandon service revenues and focus on the product

If you are a services-based company, you need to “burn the boats” to transition to a product-based company

It’s also necessary to “burn the boats” of the services business to ensure that you commit to the product. After all, you’re going to neglect some of your most-loved customers in order to deliver a product the general market wants instead, and it’ll be tempting to do custom work to keep them happy. You can’t run a product and a services business concurrently. Even IBM had to split itself in two; what makes you think you can do it as a fedgling startup?

In enterprise B2B, the biggest risk is often: Will it integrate with existing workflows?

In the B2C world, startups worry less about “Can I build it?” and more about “Will anyone care?” In the enterprise market, the risk is more, “Will it integrate?” Integration with existing tools, processes, and environments is the most likely source of problems, and you’ll wind up customizing for clients—which undermines the standardization you fought so hard to achieve earlier. Managing this tension between customization and standardization is one of the biggest challenges of an early-stage enterprise startup. If you can’t get the client’s users to try the product, you’re doomed. And while your technology might work, if it doesn’t properly integrate with legacy systems, it’ll be seen as your fault, not theirs

Disentangle/ segment high touch customers (services based) from low touch customers (product based)

As you transition from a high-touch consulting business to a standardized one with less customer interaction, you need to focus on disentanglement. Your goal is to not have “anchor” customers that represent a disproportionate amount of your revenue or your support calls, because you need to scale. Put your high-touch customers that you acquired early on into a segment and compare them to the rest of your customers. How do they differ? Do they consume a fair proportion of your support resources? Do their feature requests match those of all your customers and prospects? Don’t ignore the companies that made you who you are—but do realize they’re not in a monogamous relationship with you anymore. Zach Nies suggests going even further, segmenting customers into three groups.“‘A customers’ are your really big customers who negotiated a big discount and expect the world from you.‘B customers’ are customers who are fairly low maintenance, didn’t get a big discount, see themselves as partners with you, and provide useful insights. ‘C customers’ cause trouble, are a pain to deal with, and demand things from you that you feel will damage your business,” he explains. “Don’t spend too much time on the A’s—they sound good but aren’t the best for your business. Bring as many Bs on as customers as possible. And try to get your ‘C customers’ to be customers of your competitors.

Lanuching a new product or a productized service within an existing company can tricky

What’s more, as people start using your MVP, you have to manage the beta process carefully. You may be interfering with existing deals in the sales pipeline, or creating more work for customer support. If so, you need to have approval for the rollout and the buy-in of stakeholders. If you’re launching an entirely new product line, you may even have to camoufage it so you don’t cannibalize existing markets until you know it’s successful. This, of course, undermines your ability to use unfair advantages like an existing customer base.

Closing quotes on analytics and building successful startups/ products

. We started by saying that if you can’t measure something, you can’t manage it. But there’s a contrary, perhaps more philosophical, observation we need to consider. It’s a line by Lloyd S.Nelson, who worked at Nashua Corporation.“The most important fgures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them. ” This smacks of Donald Rumsfeld’s “unknown unknowns,” and as your company grows and achieves a degree of operational consistency, fguring out what you don’t know becomes a key task of management. Nelson’s point was that we often do things without knowing they’ll work. That’s called experimentation. But experimentation—for companies of any size—succeeds only if it’s part of a process of continuous learning, one we hope to have instilled in you whatever the size or stage of your business

Your customers leave a trail of digital breadcrumbs with every click, tweet, vote, like, share, check-in, and purchase, from the frst time they hear about you until the day they leave you forever, whether they’re online or off. If you know how to collect those breadcrumbs, you have unprecedented insight into their needs, their quirks, and their lives. This insight is forever changing what it means to be a business leader.Once, a leader convinced others to act in the absence of information. Today, there’s simply too much information available. We don’t need to guess—we need to know where to focus. We need a disciplined approach to growth that identifes, quantifes, and overcomes risk every step of the way. Today’s leader doesn’t have all the answers.Instead, today’s leader knows what questions to ask. Go forth and ask good questions