Welcome to the Experimenter’s Edge. Each month I share what I’m seeing in rapid experimentation and consulting work, what’s shipping inside Rapidly, and something practical you can use.
This October I said the same thing in nearly every workshop and talk I ran: we save a million dollars every time we don’t build something.
People look at me sideways when I say it. Fair enough. I made it up.
A few months into running innovation at a previous company, I was asked what return the program would deliver. I’d just explained that most of our experiments would fail, so I said somewhere between a trillion dollars and a cent. As you can imagine, that’s not a great way to keep your job! I was told to go away and come back with a better answer.
So I went back to the idea backlog. Every idea in it already had a rough cost in sprints, people and time. We ran experiments, and when customers didn’t care, we didn’t build it and banked the savings.
Three months later, I went back with the numbers. The ideas we’d stopped added up to millions in that quarter.
Why am I still telling this story?
AI has made it very cheap to start things.
Speed without structure becomes chaos, so you need speed with control.
100 ideas become 1,000, pushed from the top and the bottom of the company at the same time, and someone has to decide which ones stop.
In October, I ran a workshop with a product team whose experiment log was full of tests stuck with no decision. Nobody had agreed up front what a pass looked like, so they had nothing to stop against.
A few weeks earlier, a product operations team at a large digital company told me their discovery work takes about three months on average. By the time it finishes, the team has usually committed to building, and the budget rules make it painful to write off the discovery effort. They don’t track the ideas they kill, so stopping one looks like waste.
If it fails or you stop it, you bank that money.
How do you count it?
One team I’m working with sizes every sprint at roughly $65,000. It’s a T-shirt estimate, but it’s close enough to start.
Put a rough build cost against every idea before it’s tested, in sprints, people or dollars.
Agree on the pass / fail before you see any data: at least X% of Y customers will do this specific thing (Z).
When an idea fails or you stop it, move its cost into a column called money saved.
Report that number every month, right next to what you shipped.
The first time you add it up, the number will look made up. Mine did. But once leadership sees it every month, a stopped idea gets reported as money kept in the budget instead of a failed project, and teams start stopping more often.
One client, an ASX-listed company, killed the CEO’s pet project in a week. It had been pushed for two years. In the experiment, customers wouldn’t use it even when they were offered free money. That week saved $250,000.
What we shipped: Next, the next best action in Rapidly
On 1 October we moved everyone across to a rebuilt Rapidly.
The big change is that Rapidly now starts with one question: what should you do next?
Next shows you one idea and the one step that will move it forward. In the clip, the step is to complete a live test. Once the results go in (91 of 240 people acted, against a 15% target), the test passes, confidence moves from Even Odds to Very Likely, and the next step becomes setting up the next test.
Most of the stalled experiments I see are waiting on a small next step that nobody has picked up. Next puts that step in front of you every time you log in.
If you’re new to Rapidly, Next starts by asking what idea you want to test. Describe it in a sentence or two, select Draft my first test, and Rapidly drafts the Lean Canvas, a hypothesis with a target and a first test for you to check.
Try it at rapidly.co and let me know what you think.
Spotted in the wild
DX’s Q2 2026 report on AI in engineering covers more than 500 teams. AI wrote 52% of code in the second quarter, up from 34% in the first. Median quarterly spend on AI went from about $1.5K to about $44K over four quarters. The share of time spent on new features, rather than maintenance and overhead, stayed flat.
I heard the same numbers discussed at a State of AI SDLC session in September.
My read is that teams can now build far more than they can decide on, and a costed backlog with pass marks is built for exactly that problem.
Tool of the month: Boardy
Boardy is an AI that networks for you. You tell it on a call what you’re working on, and it suggests introductions that both people have to accept first. In the last week of October Boardy set up two calls that were more relevant than most of my LinkedIn inbox. It’s only as good as the brief you give it.
How much did your team save by stopping ideas last quarter?
Pick the last five ideas your team stopped or dropped without a decision. Put a rough build cost against each one, in sprints or dollars, and add them up.
If you can’t find five, check whether ideas are being stopped at all, or just not recorded anywhere.
Reading
Quit, by Annie Duke: why we keep going on things we should stop, and how to set kill criteria before you start.
An AI bot modded my racing sim while I watched: what four hours of unattended agent work looked like across 144 cars and tracks.
Find the best ideas to invest in
If your backlog is growing faster than your team can decide where to invest, I can help you work out which ideas to stop and what that saves you.
Want to talk it through? Reply and we’ll set up a short call. Or try the free Idea Validator with an idea you’re about to fund.
Until next month, happy innovating!
Leslie