Startup Experiments: Evidence Doesn't Care About Your Job Title

Inside one of the biggest technology companies in the world, an employee had a small idea. The people in charge looked at it and decided it didn't matter much. So it sat on a list. For months. Hold on to that idea. I'll tell you what happened to it at the end.

I'm Dr. Greg Watson. I've founded fourteen companies. And I've learned that the most expensive words in business are, "I'm sure." I'm sure customers will love it. I'm sure this will work. I'm sure that idea isn't worth our time.

Here's why we say it. Being sure feels strong. Admitting "I don't know yet" feels weak. So we defend our ideas instead of testing them. But your idea doesn't care how confident you are. It either works in the real world, or it doesn't. An experiment is how you find out before it costs you everything. And remember: people liking your idea is not the same as your idea working. That's why we test.

Three moves. Predict. Measure. Decide.

Move one: Predict. Before you test anything, write down what you expect to happen. With a number. And with a reason. "People will love it" is a wish. "At least eight out of ten people can find the sign-up button in under two minutes, because we cut the steps in half" is a prediction. A wish can't be wrong. A prediction can. That's what makes it useful. And test the scary assumption first. The one that sinks everything if you're wrong.

Move two: Measure what matters. It's easy to measure the wrong thing. A thousand views. Three hundred clicks. Those numbers feel great. But did anyone actually sign up, show up, or come back? And always ask: out of how many? Thirty sign-ups out of fifty people is amazing. Thirty out of five thousand is a very different story. Same thirty people. Totally different answer.

Move three: Decide. When the results come in, you have four choices. Keep going. Change direction. Pause. Or stop. Here's the twist. A good result can still mean stop. Customers might love your service... and it still costs more to deliver than they'll pay. And a bad result doesn't always mean quit. Sometimes it shows you exactly what to fix.

And this is the part that takes real courage. Your hypothesis is not you. When the evidence says you were wrong, you didn't fail. You learned something your competitors are still guessing about. Stay committed to the purpose. Stay flexible about the method.

Now, back to that idea sitting on a list. The company was Microsoft. The idea was a small change to how ad headlines appeared on Bing. For months, nobody tested it, because experienced people were sure it didn't matter. Then an engineer finally ran the test. The result was so big, the team's first thought was that something must be broken. So they checked. And it was real. That's the lesson. The idea didn't need a more important person behind it. It needed a fair test. Evidence doesn't care about your job title.

So here's my challenge. The next time you hear yourself say, "I'm sure"... or your boss says, "We already know that won't work"... stop. Ask one question: "How would we know if we're wrong?" That question is where real learning begins. I'm Greg Watson. Welcome to Chapter 5. Let's go find out.