Recently, a couple people in the Startups Unplugged class asked me to take a more formal descriptive approach and talk about the general principles that help you learn how to improve your startup. I’m including some personal experiences from when I was building my startup Chatfe (these are mostly examples of things that didn’t go well).
Here are some recommended major activities for you that will positively impact your work.
Customer Discovery activities (do you really know your customers?)
– Talk to people in your target market. What is their daily life like? What are their biggest problems? These are extremely powerful things to understand. You can learn from customers without asking them outright what to do (they often can’t tell you anyway).
– If they could change anything about what they do, what would it be?
– When you first talk to people, focus on their problems, not your proposed features (which may need to completely change).
– Observation of users is needed and totally different from looking at analytics. Go to friends, other startup people and best, total strangers to observe them using your service. Take notes, ask questions and use what you learn to improve.
– You can learn a lot before you build anything. So don’t build until needed; use wireframes to save time.
– Be flexible enough to change when what you learn is different from your initial vision. You do need to start with a vision, but sometimes your starting vision will be revealed as a hallucination.
– Be open to pivoting (major change in your service or direction), which is something most startups do at least once. It’s tough, but better to do this earlier than when it’s too late.
– Build-Measure-Learn methodology. Think of this in reverse order. First figure out what you want to learn, then figure out what you have to measure in order to learn that, then what you must build in order to measure. For example, if you want to learn if people want your new service you don’t need to build it first to test that. Instead, you could see what % of visitors sign up on a test landing page. And Build-Measure-Learn is not a one-time event, but something you keep doing.
My experience:
– I learned a lot about user behavior through direct observation of first-time users, but didn’t always capture or apply the data. Apart from myself, I had multiple people using the same form to gather data but not everyone was detail oriented or comfortable asking questions, which meant I lost data and wasted time.
– We tied ourselves to a single way of doing things because we had already spent time building it. Based on what we learned it would have been easier in the long-run to start over and abandon some of the work.
– We didn’t apply Build-Measure-Learn methodology, which meant we wasted a lot of time as we had to rework pieces of the service bit by bit. We could have learned much faster by using a complete mockup and testing that with users. No major code development of bug fixes needed in the beginning, yet that was the first focus — not the best use of time.
– After showing friends, we got the first 100+ testers by going to college campuses and cafes and asking people to try out our new startup. We didn’t pay people, had about a 20% hit rate and average time of 20 minutes with the person. Direct observation led us to improve our UI, but we also needed to ask the right questions. I realized afterward that while feedback from these sessions was positive and people engaged for a long time, we did not understand repeat usage, which was critical to our first version’s growth.
– In the first business sale, I was able to partially apply customer discovery and charge before we had done the code development. But this wasn’t true customer discovery because I didn’t identify a large enough target group to make this piece of development into a sustainable business.
Customer Validation activities
– Develop a repeatable sales process; something that can passed on to others or automated online.
My experience:
– My sales came from word of mouth, referrals, and random meetings – not repeatable actions that I could control. This meant that I had difficulty growing the business.
Your goal is to get to a repeatable sales process. Can you adequately reach (paying) customers online? Or if you use a sales force for business customer sales, what kind of business, what title/function should they target, how do they reach these people?
Metrics
Don’t use vanity metrics because while they look good to those who don’t know better, they don’t help you learn what is improving your startup. Here are some things to try and vanity metrics I wrongly used until I started to figure it out.
One-off experiments that help you collect metrics and learn
Split test (A/B test) different designs and content. Then compare how the different versions perform.
We used Google Optimizer and a $5/day budget of Adwords to experiment with different designs.
Doing this can teach you a lot about signup rates but is not enough, as shown from the examples below.
Tracking user growth
User growth looking good… But does it matter? (don’t worry that there’s no scale shown)
Now we add another metric: the active user (usually defined as the number of users that use your service at least monthly). Uh oh…
I saw growth in users, long single-use engagement times, but noticed that people did not frequently return. This made active user growth impossible and an unsustainable user network on which our service depended at that time.
Getting publicity
You may think that you just need to increase your marketing and get higher visitor numbers. So how do things change when you get publicity?
We got written up in the New York Times.
Hooray? This is what really happened.
And this is what happens to a lot of startups. I didn’t expect to be covered in NYT, but getting press coverage isn’t your goal if your product still needs work. The benefit was some initial attention and words of congratulations (vanity metrics to be sure, but nice for morale).
My overall experience:
– Having these experiences served as great experiments for us. Since we were looking at user data we got to understand how people were truly using our service. First time use was positive and had long engagement times (what we expected from initial experiments). Repeat use was very different from what we thought (definitely not daily) and required us to change our product. We ended up moving to run themed events where people would come together at a specific time. When we eventually pivoted to a business product, this knowledge was key.
Cohorts
A way to show whether your product is changing for the better or worse over time is to track cohorts of users (in this example I track active users over time). First I break users into cohorts based on month they joined and then show what percentage of each cohort return month by month. This is a really important metric for most businesses.
Line up data to start each cohort together
Graph the data. Something is changing the user experience. In this example, users are more likely to stay active over time. This example shows that something you are doing (track your actions) is improving usage rates over time.
No one metric tells the whole story and you can learn a lot by looking at data.
Team
– Often said to be the most important part of the startup. If the current vision doesn’t pan out, a good team can regroup and do something else. A good team can learn what it needs to do to be successful. A good team knows its limitations.
My experience:
– I like to have the team work in the same place, if not every day, at least for most of the week.
– Arguments are ok, it’s how you handle yourself during and afterward that makes a big difference.
– If someone isn’t working out, they must be put in another role or moved out. Postponing that difficult discussion will only bring you more headache later.
– If someone is a talented worker but a negative force in the group, get rid of them. They are most likely not worth the pain they will create for the others.
Legal
– When you start out, it’s a good idea to put some basic things in writing. People change over time, what you remember will be different from what your partners remember; fights will happen and you will waste a ton of time thinking about these things.
– For partners and employees with equity, a vesting schedule (e.g. your equity vests over 3 or 4 years) and a one-year cliff (anyone who leaves without working at least one year gets nothing) are both good ideas.
My experience:
– I got pro bono legal help. Even without that, there are standard legal templates you can use for many common needs.
– We had all employees, interns and contractors sign contracts.
– I personally know several startups who have had founders leave with nothing in writing. This is a common problem in the startup world (no one thinks it’s going to happen to them) but it will make your life miserable later on because of the uncertainty.
Investors
– The best way to raise money is to show that you have a good business and traction. Investors invest in “lines, not dots.” E.g. show progress over time; fundraising is not a one-time activity.
– Demos are important, but not as important as having a good product and team. Practice pitching (this is more difficult than it seems).
– Meet investors at startup events, pitch/demo events, referrals from other startups in their portfolio, Angel List (angel.co). Also look at thefunded.com for info on the investors themselves.
– Raising money can easily take all your time for months. Only do it if needed. Nothing is guaranteed. Investors move in herds – it can be worse for you if you try but then fail to raise money.
– The team is often said to be the most important thing an investor looks at.
My experience:
– I knew a bunch of other startups that tried to raise money too early and failed. That time would have been better spent working on their product.
– Showing that you have a good product, users, people paying, are good forms of validation. Knowing why you’re succeeding (see Metrics section) shows you probably know how to duplicate this success.
– I made an art of bootstrapping and keeping costs low where we could. Some inventive things we did: got interns in exchange for college credit, got pro bono legal services, used college campuses and Starbucks as places to test with users. Using the limited techniques we had back then, we had better results than a competitor with a 15 person team and $500K in seed money (compared to our 3 person team and $4K spend to get to launch).
Mentors / Advisors
– Incredibly important to your development.
– Build in forms of accountability.
– Find people who have done it before, not people who have brand name recognition.
My experience:
– I found informal mentors and advisors in people in the investment community and other startup people who had run a startup before, but wish I had done more.
– Some people just wanted to build their status as advisors. I didn’t accept people who sought me out to advise me but didn’t have experience in the areas my startup needed.
Be out there:
– Be a part of startup community events, meet others, talk about your work.
– Be a part of your user community.
– Get covered: present, pitch, meet or pitch bloggers.
My experience:
– Over the last 2 years I met mentors, investors, new hires, and customers just from being out in the startup community.
– I also spent too much time at events that weren’t really relevant for me and could have sought specialty events more.
– I tried to choose events where I could talk to people, rather than sit in an audience and watch people.
Well, that’s a long list. I hope it helps as you work on your own startup.
