Side Effects. Mean Business.
or Why I Recreate Before Every Startup Program I Run and Company I Advise
It’s easy to spend money when it’s not your own.
When business people plan and spend a budget (not their own money) I rarely see evaluations of the efficacy of the activities the budget will support. In new product development, innovation management, startup programs, or whatever other types of programs you are involved in centered around creation, here’s an approach that’s a little different. It’s based on the last six years I’ve worked with companies around the world.
First, let’s introduce a concept from medicine called Number Needed to Treat (NNT) and Number Needed to Harm (NNH). NNT is a statistical measure of how many people must take a treatment in order for one person to see the benefit. Those medications and procedures don’t help 100% of the people who take them — that’s something easily forgotten.
Let’s see how NNT is applied in medical treatments.
Example 1: Antibiotics for Hand Lacerations.
What do you guess is the NNT? Seems reasonable to believe that using an antibiotic in this situation could only help the patient? That’s what most people say when I present this case. But actually, there is no benefit found.
Here’s another one.
Example 2: Aspirin to prevent a first heart attack or stroke.
What do you guess is the NNT? We’ve heard about aspirin used in this way for a long time. Seems reasonable to believe that using a drug that has been on the market for 100 years would help the patient and have no downside? Again, that’s what most people say when I present this case. But again, there is no benefit found. For the one person helped in this case, another 1,666 took aspirin daily with no impact and some now suffered side effects as well.
I bet you’re getting more skeptical, but I’ll keep asking you to guess.
Example 3: CT Scanning for lung cancer screening in high-risk smokers.
What’s the NNT? It’s beneficial to preemptively identify and treat cancer in a high-risk individual. But how is the cancer to be identified? Here again, the procedure offers some benefit but also side effects. For the one person helped out of 217, we find that one in four (the NNH) suffer from the false positive (everything from the stress of thinking you have cancer when you don’t to taking treatments for it unnecessarily).
Last one.
Example 4: Mediterranean Diet for 5 Years for Heart Disease Prevention (Without Known Heart Disease).
I’m all for the Mediterranean diet. But what’s NNT and NNH? Finally, some good news. The NNT is one in 61 and there is no NNH (no side effects from eating olive oil, fish, pasta and other good food).
How does this affect decision-making?
I was thinking about why I do certain activities and avoid others.
For example, one point of contention is why I don’t do startup demo days in the program I currently run at USC. This is an Incubator that has a 10% acceptance rate, has taken in 70+ companies over three years, sees about 40% raise capital, and other good results. Why not do a demo day when everyone else does? It’s easy. The NNT and NNH don’t warrant the demo day.
My Estimates: Demo Day for a University Incubator to Solve Funding Problem
Benefits in NNT: 1 in 10? / 1 in 50? Meaning, that number of people would raise after the demo day but would not be able to raise without the demo day.
Harms in NNH: 1 in 5? Why is the NNH so high? To prepare for a demo day, founders will do little work on their business while preparing for the event. I’ll also do less to help them on their business because I’ll be so busy organizing all the visitors and the whole production.
There’s also a further NNH of 1 in 5? because those who do not immediately raise capital therefore send a signal that they are a bad investment.
Throughout all of this, I however would look like the hero. In other words, the demo day is bad for the startups but good for me (the person running the Incubator). So I don’t do demo days. Here’s another one.
My Estimates: Many Group Meetings to Teach New Knowledge and Best Practices
Another question I get a lot is why I no longer to lots of group meetings with the startups in the Incubator throughout the week. Instead, the program has evolved to be mostly founder-driven with one-on-one office hours with me and other people from industry who I bring in. This change happened as the founder mix became half USC alumni (e.g. older, more experienced, and not living on campus) and broader (I have a wide range of company industries and tech focus areas from software, hardware, food, apparel, consumer products…).
Many group meetings benefits in NNT: 1 in 10? were helped mostly from networking with others in attendance.
Harms in NNH: 1 in 10? did little work on their business since they’re in so many meetings. Actual behavior change and application of new knowledge is limited.
If I were to just copy what worked in other programs, I’d risk harming people with side-effects. If you don’t yet know the effect, be biased toward non-interventionism.
I’m exploring these concepts in a new project on unintended consequences.