Startups Unplugged

Category: lean startup

Posts that include lean startup techniques.

  • A college level class in growth hacking?

    In a few days I’m teaching the first class in Growth Hacking at USC. I created this class because no matter what you feel about the “Growth Hacker” term, I find the role to be sought after by graduating students and the skill set to be appreciated by businesses. As far as I know, this is the first college class focused totally on Growth Hacking, so I thought I’d share the experience along the way. This is the first post.

    Framework

    I take Growth Hacking as understanding business fundamentals and then being able to make informed decisions using data, centered around orders-of-magnitude growth.

    Growth Hackers love data, are creative, and curious. Some are developers, some are designers, some are business oriented. Let’s dive into a way of looking at the world as a giant opportunity for growth.

    I call the framework I use to guide my thoughts CCARR: Collect, Clean, Analyze, Run, Repeat. That is, as a starting point, Collect data that is helpful, Clean up the data, Analyze the data to pull out insights, Run new experiments based on what you have learned, and Repeat the whole process as long as it makes sense.

    Tools

    Expanding on this framework, I use the following tools.

    Collect data that is helpful. This could be internal data you control, even using something as universal as Google Analytics. Data collection could also come from external sources, perhaps by web scraping other sites to save effort. I use Outwit Hub and SEOTools for Excel for basic off-the-shelf web scrapers, and Tamper Monkey’s Chrome extension for making and finding user scripts to save time.

    Clean up the data. The data you want, especially if it is from external sources, may not be in a format that you can easily use. I use Google Refine if there is a significant amount of work to do. Otherwise, I might just live with it and open up Excel.

    Analyze the data to pull out insights. This is business plus creativity. For this I use an expert level of Excel, which is just easier for me than using a database. This step is guided by knowing what business fundamental we want to impact. Eventually it all comes back to fundamentals like Life Time Value, Customer Acquisition Cost and cycle time (see next section). However, you may want to focus on one specific element of those.

    Run new experiments based on what you have learned. Here you might use ads on Facebook or Google, experiment with referrals, go to social platforms like Reddit, Product Hunt, or Twitter, or try something more guerilla. Just make sure you track what you are doing and your predictions for what will happen.

    Repeat the process as long as it makes sense.

    There are many tools that do these things. These above just happen to be the ones I’m using now.

    Fundamentals

    Adding more customers if you lose money on each one doesn’t make sense, at least not in the long-run. Eventually, your work has to come back to the fundamentals. That means if you’re trying to growth hack a business, you work on increasing Life Time Value (LTV), decreasing Customer Acquisition Cost (CAC), and speeding up cycle time. Let’s take those one by one. Simple to remember, but figuring out how to do it is the hard part.

    Ad hoc, I use a simple LTV formula: LTV = (Price per Unit – per Unit Costs) * Repeat Purchases. Those are the three triggers you have to increase monetary value to the business. As a growth hacker, the factors you can usually impact the most are Repeat Purchases and to a lesser degree, Price. It’s usually more interesting to also look at inputs to LTV over time. To do that, you can do a time series of what a customer generates in revenue and costs per time period.

    CAC is another area of focus for growth hackers. If you can figure out how to decrease your CAC, then you have more budget available to acquire more customers. The difficult thing about CAC is that different channels for customer acquisition constantly change in their efficacy. What worked yesterday may not work today, or may be too expensive to be worthwhile. That’s why many of the historical growth hacks you hear about are good for ideas, but may not be repeatable. The also means that another difficult thing is realizing that what’s working for you right now may stop working. To keep yourself honest, remember that there is no single CAC – every business has multiple channels to reach customers, each of them with their own associated cost and LTV. To make it easier to focus on growth, think of what will happen to each channel when you try to increase your numbers by 10 times. Some of the channels will not scale at all, some will scale, and some will only scale at a much higher cost.

    Rule of thumb: keep a healthy distance between your LTV and CAC, perhaps 3X or 4X, unless you are trying to take market share and have the reserves to play that game for a while.

    Cycle time is the delay between when you acquire customers and when you see the impact in the form of initial purchases. If this cycle is too long, even if you can decrease CAC and increase LTV, you’ll be out of business before you can benefit.

    The best tip is also the hardest: build a great product that your customers love. If you can do that, then growth hacking becomes a lot easier. Growth usually does not just happen all by itself or in a sustainable way. Keep the above CCARR framework in mind when you growth hack.

    (This post originally appeared in Jumpstart Magazine.)

  • Startup Weekend Toolkit

    Here’s a Startup Weekend Toolkit I put together after mentoring and judging. It’s a lot of info, so it’s best to read this before you go since you will be busy enough (and sleep deprived) during the weekend, but these resources will save you lots of pain.

    Startup Weekend isn’t a hackathon or a business plan competition… So work on these activities instead:

    1. Customer Development. “Get out of the building” and talk to people. Learn from them. What hypotheses do you want to test? Do your potential customers support and reinforce your assumptions? What qualifies success or failure?

    2. Business Model. Express what you learned in customer discovery: Who is your customer? What is your core value proposition? What are your key activities? What are your revenue streams? What is your cost structure? Who/what are your key partners/resources? What are your distribution channels? What is your roll-out strategy?

    3. Execution. What did your team build during the weekend?

    4. Your presentation. Not being able to express what you did is the unspoken failure point of many teams. Start to practice on Saturday, not Sunday.

    Are you really building an MVP?

    Working with lots of startups, the most common error I see people who think they are working on an MVP is that they do not understand what an MVP is. An MVP is focused on getting validated learning. Write down your hypothesis and what will qualify as success or failure for you. See more about that below. Otherwise, how do you know if you’re learning anything?

    A sample hypothesis format:
    I believe that [customer segment] has [problem / need] and [a specific number / percentage of them] will do [action / use this solution].

    Hypothesis Formation to guide you as you build an MVP and then collect data. Read this brief article for guidelines.
    Questions to consider: What are your riskiest assumptions? What MVP will enable you to test your riskiest assumptions? What will you accept as validation that you are solving your customers’ problem?

    MVP case studies:
    Think about how Dropbox, Aardvark, The Ladders etc tested their services before building them out.

    Some techniques to build an MVP:
    – Landing Page — put up a landing page that leads the visitor to believe that the product has been built out. Then test user actions — do they try to sign up, do they try to buy, etc;
    – Mockups / Wireframing — these can be on paper, using tools as simple as powerpoint (where I like to hyperlink fields on the page and connect them to other pages to make it feel like a website), or more sophisticated tools like InvisionApp or Balsamiq. Give the user the feeling of using your product before you write any code;
    – Video — make a video to demonstrate what your service does before you build it. This could be as simple as text or basic images or screenshots with a voice-over;
    – “Concierge” — use humans to emulate what the software would do later on, if you were to build it;
    – Prototype — actually build a working version.
    – “Parasite” — this is the name I use for riding on top of another network (Twitter, FB, Skype) and using the users there as test subjects rather than trying to drive new users to your new service.

    Doing Customer Discovery Interviews:

    1) Ask about situations in which potential customers might be aware of the problem you’re trying to solve and then ask questions around that. This means that if you’re trying to solve the problem of finding cheap vacation travel packages, I’d ask “Do you like traveling?” and ask them to talk about that, instead of directly asking “Do you want to find cheap flights for your vacations?” (which is a question that forces them to say “yes, of course” — not that helpful to you).
    2) How do you solve this problem currently? or What do you do to make it less painful? In the above example, they might have signed up for notifications for cheap flights from online discount travel agencies. This will tell you about the competitors who are already involved in solving this problem. Not just the existing business competitors, but also ways that people deal with this problem by themselves.
    3) Can you describe this problem to me in your own words? They may have already told you about this without you having to ask them. Directly asking this also only works if you didn’t already force the issue by asking them if they have the problem. Otherwise they’ll just repeat what you asked them already. This may also give you keywords to put on your landing page and in ads.
    4) Are there other solutions out there that you’ve tried in the past? This will make you aware of competitors or other services addressing similar problems.
    5) What do you wish you could do to solve this problem (even if it isn’t practical)? This can be really powerful, especially if you find out that something they think is not practical is now practical for you.

    Cool things you could show:

    • Number of people you spoke to;
    • What you learned that was unexpected;
    • Actual purchase orders;
    • How you hacked the system to get it all done.


    Watch out for these things. I made this list by surveying past participants:

    • Mentor whiplash (getting pulled in different directions by different mentors);
    • Replacing customer interviews with online surveys (because you run out of time or are too hesitant to find interview subjects);
    • Hesitating to “get out of the building;”
    • Underestimating the time it takes to talk to people;
    • Building too much, testing the problem too little;
    • Not taking risks, not being creative;
    • Not practicing the pitch enough;
    • Thinking too much about the judging;
    • Team fighting, worrying about IP protection, getting distracted, no team balance, team too large.


    Practice your pitch. Pitch structure ideas:

    • Introduce yourself and team;
    • Introduce the problem, Show your solution;
    • Talk about how you got there: Customer Discovery work you did, your Business Model.

    And for what happened when I tried to run my own one-person Startup Weekend before I was a judge for the first time, read this.

  • Going Gonzo to Build a Better Bootcamp

    Hunter S. Thompson practiced Gonzo journalism, a format where the reporter participates in the story. Here’s the start of the Gonzo startup advisory (what I used to run as a “bootcamp”).

    In my experience, if you’re an early-stage entrepreneur and working with someone like me to learn Customer Development techniques, you will have some common troubles. One of the most common ones is to use customer interviews as a way to learn about your target customers, their problems and what they value. It will take you time to become comfortable interviewing or even with the idea of going out to talk to people. You’ll delay doing it entirely. And when I think you’ve understood it I won’t really know how it goes out in the wild.

    I’m convinced that practicing Customer Development is a set of skills that cannot be learned without active practice, so I’m going gonzo. Salim Virani brought up the participatory idea over dinner and I’m making it a part of my work. I will go on customer interviews with startups in the early days of working together to observe, take notes and when needed, contribute to the discussions. I’ll give direct feedback right after the interviews so that the founders can improve more quickly without me needing to guess how well it’s going.

    I’ll report back later on results from going Gonzo.

  • Startup Weekend To Dos

    This weekend there will be more than the usual number of #lean hashtags on Twitter. Startup Weekend is holding its 54-hour events around the world, including in Hong Kong. I recently gave a pre-Startup Weekend talk on “Lean Startup in 54 Minutes” and I thought I’d share the major parts of the talk here.

    This isn’t a hackathon (where you focus on building) or a business plan competition (where you focus on untested projections of fantasy). Instead, work on Customer Development, your Business Model, and building an MVP to validate your idea.

    Get familiar with the radical idea of talking to other human beings. I know it’s more comfortable in front of the glow of your laptop screen, but when you get out of the building and speak to real people, the sunlight of reality will help you learn a lot. In my experience, people wait too long to start doing this.

    Learn about the Business Model Canvas. It will help organize your thinking and open you up to new opportunities.

    Most people who say they’re building an MVP are not. They’re really just building a basic version of their product. That’s not an MVP. An MVP is focused on getting you validated learning. An MVP doesn’t need to be coded, either. You could make a video, use paper mockups, or do a “concierge” service where people perform all the behind-the-scenes among other techniques. Just remember that the point is to learn, not to just start building.

    For more on all this, I hope this toolkit will be helpful to you. Good luck!

  • Can You Do An MVP For Your Life?

    I had to come to Hong Kong to go to my first Barcamp. The timing was perfect — I arrived less than a week before to run Startup Bootcamp so it was a perfect time to meet more people. And everyone in the bootcamp attended for at least part of the day.

    After attending a few Barcamp talks and meeting a ton of people I gave a talk of my own, titled “Can You Do an Minimum Viable Product For Your Life?” While we spend a lot of time talking about work, there’s obviously a lot more to life than that. If anything, we talk about work so much because it’s easier than all the other things that comprise our years on earth.

    Since we’re getting used to applying lean methodology to tech startups, could we do the same for ourselves? I spoke for about 10 minutes on the idea and introduced something that I did years ago as a way of validating which option I should take in a major life decision: I wrote in great detail about the decision and what it was like living with it. That process helped me understand what I really wanted and what it would be like to be in that role.

    Ideas flew in the discussion. Among those talking were Jeff Smith, Andrea Livotto, Andrew Tipton, and Nic Wang (forgive me if you spoke but I don’t know your name), giving the following list of ideas that I recall (in no particular order):

    • Play dress-up (Cool idea that really needs a better name): Go somewhere outside your network and practice “being” the person you want to be. That could include telling people that you are an architect / pastry chef / chaos-theorist etc and getting used to living with that persona.
    • During a vacation, try to work temporarily as the role that you seek.
    • Try to bring elements of the job / activities / life that you want into your life in small chunks.
    • Track your happiness, productivity, usefulness throughout the day to test what you most like doing. We get so absorbed in day-to-day activities that it’s easy to lose track.

    If you have other ideas or try one of the above, let me know the results.

  • Fear, Waiting and LaGuardia

    If there was ever a situation in which ride sharing apps would be used, it was at LaGuardia Airport, 12:30am after the airport had been closed for a storm. When I arrived, delayed for several hours with many other flights, I found a taxi line that was like nothing I had ever seen. In fact, the line I first saw — already the longest line I’d ever seen — turned out to be only the first quarter of its entire length, the remainder hidden in a slow snake behind a large pillar.

    I was tired after the long delayed flight but initially thought that the line was moving along nicely and I’d be home soon enough. On my walk out I’d already noted that there were none of the usual gypsy cab drivers running around offering their services — they had already all been snapped up. I was actually not alert enough at that hour to think of calling a private car myself, though I’m sure the wait would have been hours long if I had. Instead, I started to think about how I could get home faster than waiting out the entire line.

    Fortunately for me, a guy two spots ahead started to wonder the same thing aloud. He was clearly not from New York and wondered why no groups were forming to speed the line up. His destination was nowhere near mine and on a normal day I wouldn’t have considered sharing a taxi, but I talked to him anyway and started to teach him about how to refer to NYC neighborhoods, a crucial skill at this time. We decided to help each other out.

    This was when I realized that ride sharing services really have a tough slog. If no one was trying to use them on a night like this, or if the line was this long in spite of them, or if there was no one walking around promoting one of them and signing new customers up, there was clearly something wrong with the way they’re run. Or waiting behavior was so ingrained that they had to make a serious push to change it.

    While a few people in line vocally complained, the vast majority accepted their fate with complacency, steadily plodding forward a few steps a minute, not even responding to my new, non-New Yorker friend’s inquires as he started to work the line about sharing a ride. Many people were traveling solo, back from business trips, coming home tired and after their delayed flights, unable to think about creatively fixing their wait time problem. Contrast that attitude with the planning for the MTA strike of 2005 where people shared rides in their own cars. They had time to plan. They were wide awake. And no one had iPhone apps back then.

    What LaGuadia needed that night was a syndicator — someone to walk up and down the line, find out where peole were going and pair them up. Such roles don’t exist at LaGuardia. And it’s hard for individuals in the lines to do that themselves, especially when travleing solo. Will the guy behind you watch your bags and hold your spot? And anyway, after getting blank stares after asking 10 people (as my new friend did), who will continue to ask more? What drives that behavior? I don’t know if fear is the right word, but there’s a certain lack of willingness to break the unspoken rules about strangers forming groups to travel together.

    To initiate the process you someone who doesn’t know any better, my new friend from Denver. To close the process, you also need someone on the receiving end who doesn’t know any better. After 10 minutes he found a woman hours ahead of us in line. And no surprise, she wasn’t from New York either. And you need the guy who enables them to communicate, which happened to be me (instructing my new Denver friend about how to refer to neighborhoods). She agreed to let us share her taxi. How many people followed our lead? Zero.

    So an hour after getting in line I was on my way. Here’s how the economics of the trip broke down:
    We paid for the woman’s taxi ride (which she hadn’t asked for, but I had no problem doing so). We dropped her off first, even though her stop was farther than either of ours. Next came the guy who was willing to work the line, a well-deserved break for him. He paid $30 at that point, probably about what he would have paid if he had gone solo, but still he saved hours of wait time. When I was dropped off last, even though my stop was closest I was happy. I had saved hours of waiting and paid $20, about what I would’ve paid had I waited and gone myself, but again saving hours in the process.

    Everyone won, but unevenly. The process was messy, random, and required someone to break the rules. Thank God for that. What could the ride share services do to make sure no one ever has to go through that kind of wait again?

  • Simplicity

    I saw two useful and simple new services yesterday: Delight.io (by Thomas Pun, who I know) and PopTip.com (from a new TechStars grad). Delight.io allows developers to add a line of code to their iOS app in order to see how users interact with the apps. PopTip allows any Twitter user to run a poll within their Twitter account.

    What struck me about both of these services is how they have simplicity built in for the user. Delight.io users only need to add one line of code to their app. PopTip users can run their polls right within Twitter, which gets respondents to promote the poll itself. I’d like to know how much went into building the services themselves.

    Looking forward to using them both in the future.

  • Too experienced to learn

    Is it possible to be too experienced to learn?

    I think it is.

    A historian told me about an example from US new recruits in WWII and how marksmanship ability varied among them. Supposedly, recruits from the US countryside performed worse than those from the city. This seems counter-intuitive. After all, men who grew up in the countryside were more likely to have experience using rifles to hunt. Those from the city were probably handling guns for the first time.

    But here’s what is supposed to have happened. The more rifle-experienced countryside soldiers already had their established ways to shoot and resisted new training in the skill. Meanwhile, those who grew up in the city had little choice but to follow the instructor’s teaching in order to gain proficiency. They were more open to trying new techniques and being criticized for their novice skill level.

    That’s sometimes what I see with businesspeople. It seems like more experienced people can sometimes be less open to trying new things or hearing information that counters their established way of working.

    One very experienced individual I met last year came to me with an idea for a new financial product which would be sold to institutional investors. He wanted to go out and hire a mathematician and developer to construct an algorithm to describe the model he wanted to build. Never mind first going to talk to people in the financial services niche he wanted to target and figure out how purchasing decisions are made, or finding out if there was a market for the service, or looking at other options that were out there. He was fascinated by the potential of the product, not the people who would buy and use it.

    That itself is risky enough. But, he also had no interest in experimenting first before going out to do the development — which would initially easily cost $50K. It’s not that $50K is a large or small amount of money, it’s that I think he could have learned more by spending a fraction of that buying lunch for his target users and getting to know them.

    I’m not against spending money, I’m just against being closed to learning.

  • More MBAs More Problems

    A lot has been said about MBA programs’ lack of preparation for entrepreneurial work. I’m even going as far as to say that these programs can unintentionally mislead students by rewarding the wrong things. (I’m allowed to make these comments because I have an MBA and ran a startup.)

    Here are a few examples from MBA students I interacted with recently and how I thought they were misled.

    Misled into requiring a grand vision to be interesting. 
    One student I met during a business plan competition had a grand vision for a new fashion company based on cool designs and incorporating the idea of giving a piece of clothing away to someone in a developing country for every piece purchased. Millions of people would bond and communicate through this social exchange.

    When I asked the founder what she thought she needed to do first she responded “I need to find a CTO.” Confused, I wondered why a CTO was needed so early for a new fashion company. The response floored me: “because our website will have to be really sophisticated in order to match people around the world in the clothing gifting and communications game.” And no, there was no previous experience working in the fashion industry.

    I started to flip out. Calming myself down and asking some more questions about whether she had made any pieces to show potential buyers. No. How about just showing them designs? No… And now you want to hire a CTO? Yes, and also start production. Have you worked in the fashion industry before or can you produce the clothes yourself? No, we’re going to outsource that….

    Still bewildered, I suggested this. Go home now and set up an online store. Put a few of your designs up and see if you can get a few people to try to buy them. If yes, then you may be on to something and can decide if you want to continue. If no, then at least you save yourself from spending tens of thousands of dollars too soon. Forget about the social entrepreneurship angle until you figure out if you even have a desirable product.

    But the response came back. Well, I can’t do that because setting up an e-commerce store takes a lot of time and also, won’t I destroy my brand if people can’t really buy the things I post?

    Serenely, I said this. Spend two hours tonight setting up a Shopify account with your designs and writing the copy. It won’t take you more than that. As for destroying your non-existent brand, use a different name if you want. But do this today so you can have the experience of trying to sell a piece of clothing.

    I wonder what happened to her.

    Misled into thinking of 5-year projections as fact.
    I talked to a team of students working on an education service. They of course had a 5-year plan showing their growth.

    “Where do these numbers come from,” I asked?
    “These are industry projections for online education.”
    “But you haven’t even had one person use your new service. Will these projections hold true for you?”
    “According to everything that’s known about this market, yes.”
    “But, you don’t know anything about the market yet. You haven’t even spoken to a single user. Why should these industry numbers, which are just forecasts by the way, apply to your service?”
    “Well, we kind of need to show this growth in order to make a case for funding.”

    Thank you for the honesty. Now the healing can begin.

    I heard an entrepreneur talk about getting questions on where the numbers for his 5-year projections came from and then asking the VC about his own experience running a company.

    “Where do my numbers come from… When you ran a company did you have your own projections you showed investors?”
    “Of course.”
    “Well, my numbers come from the same place yours came from.”

    Misled into thinking that you don’t need to talk to real people and can buy success.
    One MBA was working on a game that let people play finance games.

    His plan: outsource all development (already priced out at $60K), sell it to corporate clients, get a ton of students to sign up and use it.

    My counter plan: Do no development yet, put together a basic version of this game now that you can run manually, get 20 classmates to try it out and talk to them and look at the results. Then decide if you want to start to talk to potential corporate subscribers.

    His resistance: I can’t do any of that without first developing the full game. And there’s no way I can get 20 people to do this.

    My thought: There’s money to be made as a developer, designer or “focus group” leader preying on the market of MBAs spending their student loan money.

    Then again, I’m critical because I have a high standard for MBAs. The problems above are not that unique to their background — I see that behavior in a lot of places. I just hope for better from the students and their programs.

  • How to observe your way to insight

    When we were starting out in my last startup, even before we had heard of customer development, we had a method for learning from users. It wasn’t great but it was better than what a lot of startups were doing.

    This is what we did (after we had exhausted our friends as beta testers):
    – We took our buggy early version out to a cafe or college campus
    – By asking “Do you want to check out our new startup?” we got people to stop and test it for us (at about a 20% acceptance rate)
    – We then observed the people as they used our service
    – When they finished, we interviewed them to gain additional knowledge into what they thought
    – We did this over 100 times at about 20 – 30 minutes per individual tester.When you observe people use your service, you’ll learn a lot. For us, most of the takeaways at that stage were design-related (we were at that point oblivious to the question “should we even build this service?”). But as for design learning for example, after seeing a high percentage of testers start to sign up and then stop and sit there, we realized that our sign up process was not intuitive. The words on the screen went unread. The design had to tell a story visually as well as in text.

    Here’s what we missed:

    – One-off tests didn’t give us insight into what repeat user behavior might look like. I think of this as the “Pepsi Challenge” problem. If you remember, Pepsi invited people to take a sip of 2 unmarked cups of soda. Most people chose the soda that was Pepsi. But in reality, people preferred Coke instead. The reason behind the difference in peoples’ choice? When you have a one-off small sample, people prefer sweeter drinks; when they have to drink a whole can, they prefer less sweetness.
    Poor Pepsi. And us. We learned only afterward, that one-off user behavior (long interaction time, for example) looked nothing like what people would do when they got home (low repeat usage).- We missed the ability to quickly iterate on designs based on what we learned. While design is not the complete answer in building a good product, we could have at least used the feedback we were getting rather than sitting on it for larger, infrequent updates.

    – Instead of focusing on building the service, we should have mocked up more of it to more easily test. Then based on what we learned we could decide what to build.

    – We lost good data when people didn’t collect it. You have to know what to look for, write it down and compare results or you’ll lose what you learn.

    Still, we did some things right.

    – Our later data analysis revealed what we missed in the one-off trials: that people happily engaged for long time periods but didn’t return frequently. Cohort analysis was key here. We were then able to redesign the service as one geared to themed, timed events, where it worked much better.
    – And in retrospect, we beat a competitor that had 15 people and $500K in seed money (compared to our 3 people and minimal $4K cost to launch). As far as I saw, they did no customer development or even field tests. Instead, they went for PR, which didn’t result in any lasting results for them.People often can’t tell you what they want, but you can learn about their problems, how they solve them today and then devise ways to solve those problems.

Get new posts by email A few times per year. Deep dives into tech, unit economics, timing, and more.