Welcome to the Experimenter’s Edge. Each month I share what I’m seeing in rapid experimentation and consulting work, what’s shipping inside Rapidly, and something practical you can use.
Last week during our weekly sim racing session, we ended up talking about the value of vibe-coded software.
We were discussing why so many vibe-coded products appear to have customers, even when people like us look at them and wonder who would pay for that. We also wondered why vibe-coding platforms like Lovable, often dismissed as AI slop generators, can attract hundreds of millions of dollars in investment.
Maybe lots of people have always wanted to make software but couldn’t. Now they can, and customers are buying what they make. You can roll your eyes at some of the output, but the behaviour is real.
Vibe-coded products are evidence of previously blocked demand, not just rough software.
That stayed with me because the same thing is happening inside companies.
A few days later I was talking with a client about AI ideas appearing in small pockets across the business. Someone finds an annoying piece of work, opens an AI tool, and builds an agent or rough application to deal with it. Someone else does the same thing in another team. Some of these prototypes look polished enough to feel like products, even though they’re still ideas with a user interface.
I think companies are missing a useful signal here.
I think that effort is admirable. Someone has gone beyond talking and built something to fix their work, often on top of the job they were already doing. They’ve put some skin in the game, which is much stronger evidence than another suggestion on a workshop wall.
But what does that evidence actually prove?
It proves one person cared enough to act. It doesn’t prove that their agent is the right solution, that ten other people have the same problem, or that the company should now fund it as a product.
The current conversation about shadow AI often starts with governance. I would start with the builder. Ask them to show you what they made, why they made it, and what support would help them test it properly.
The prototype tells you where to look.
Start by finding the unofficial tools people have already made. Ask what was irritating enough for them to build something, what they were doing before, and whether anyone else has the same problem.
Then group the tools by the underlying job rather than the proposed technology. Three different agents may turn out to be three attempts to remove the same piece of manual work. That repeated pain is more interesting than any one demo.
From there, treat the strongest problem like any other idea:
Write down who has the problem and what they’re trying to do.
Agree on the behaviour that would show the problem is worth solving.
Put the smallest credible version in front of the likely users.
Measure what they do rather than asking whether they like the demo.
Decide whether to fund it, change it, or stop.
This is the same basic path we use in Rapidly.
A polished AI prototype doesn’t skip validation. It enters the process with a better starting signal than a Post-it note.
Companies still need to know what data and models people are using. A central AI register is more likely to work when it gives the builder something useful in return.
The central team can offer something useful instead: access to better models, tokens, security help, an expert who can sit beside the builder, and a fast route to testing or funding. Bringing the work into one place should help the person move faster.
I want to test this approach with a few companies over the next month.
What we shipped: Rapidly through your AI client
On 3 September we switched on the new Rapidly MCP customer journey in production.
In plain English, you can now ask Claude or ChatGPT/Codex to test an idea before you build it. Rapidly saves the idea, works through the Lean Canvas and hypothesis, designs the experiment options, recommends the easiest credible test for the riskiest assumption, and gives you a copy-ready prompt to build it.
The work stays in Rapidly, so the next conversation can continue from what was already saved rather than starting again with another chat. The AI client becomes another way into the same idea-to-evidence process, not another place for the work to disappear.
It’s live now. You can connect Rapidly to your AI client and try it.
Spotted in the wild
Lenny Rachitsky shared a useful line from the product lead for ChatGPT Work: as agents take on more of the execution, the human role moves from rowing to steering.
I think that’s right, but it raises the standard for the person steering. Faster building means you can travel a long way in the wrong direction before anyone notices. If the quality of the idea and hypothesis doesn’t keep pace with the execution speed, the agent just compounds the error faster.
Someone still has to choose the problem, set the evidence threshold, and stop when the evidence doesn’t arrive.
Tool of the month: Aqua Voice
Aqua recently told me I’d dictated 398,744 words and was in its top one per cent of users. Apparently it is saving me about two hours a week.
I use it to talk into any text box on my Mac: prompts, emails, documents, Slack, messaging, and plenty of the rough notes that eventually become useful. Speaking gets me past the editing we do when typing and creates more context and clarity.
It doesn’t remove the editing. You still have to decide what you mean and clean up the result. But it has made the keyboard optional for a surprising amount of my week.
What has somebody already built to avoid a piece of work?
Ask one team to show you the unofficial tools they are using. Pick one and write down the job it removes, how the work happened before, and who else has the same problem.
If the answer is nobody else, it may still be a useful personal automation. If the pain repeats across the company, you have found an idea worth testing.
Reading
How We’re Compressing Three Hours of Workshop Time Per Idea: how the Rapidly MCP is giving workshops a faster starting point without removing the humans from the work.
Keeping people at the centre of ideation: what happened when five human teams and one AI team put their ideas under the same pressure at GembaCon.
Find the best ideas to invest in
If AI ideas and homemade tools are multiplying faster than you can evaluate them, I can help you turn the strongest problems into experiments and clear investment decisions.
Want to talk it through? Reply and we’ll set up a short call. Or try the free Idea Validator with one of the ideas already circulating inside your company.
Until next month, happy innovating!
Leslie