I’m running a new online Leading the Product Lab on 27 October: Rapid Validation in the Age of AI.

Bring a real product or AI idea. We’ll identify its riskiest assumption, choose a small test with real customers, and set a success threshold before collecting data. You’ll leave with an experiment you can run within seven days.

Tuesday 27 October, 12:00–1:30pm AEDT, online via Teams. Details and tickets.

Welcome to the Experimenter’s Edge. This week, a product conversation got me thinking about what we’re asking customers to do.

Last week I was talking with a founder who wanted people to switch to his product. He’d built something useful, but potential customers were comfortable with what they already used.

He’d found me through our Idea Validator. He entered his idea and received a starting point for testing it before we spoke, so some of the work I normally walk people through had already disappeared.

That changed the question we asked about his product. Instead of listing everything it could do, we looked at what it could take out of the customer’s day.

Customers buy work that disappears. That includes the work of using the product.

When companies talk about AI optimisation, they often start by mapping a workflow, finding the slow steps and using AI to speed them up. That’s useful, but it assumes the workflow should survive.

Take a weekly report. If its purpose is to support a decision, perhaps the report doesn’t need to be assembled faster. The answer could arrive where the decision is made, with the right context and evidence. The preparation, transferring, handoffs, waiting and maintenance disappear with it.

The customer doesn’t need to see the model or the machinery. They notice that there’s less work between needing something and having a useful result.

Count the whole job

Imagine a product that summarises a document in seconds. The customer may still have to find the file, upload it, explain what matters, check the answer and move it somewhere useful. The AI step is fast, but the whole job may not be any easier.

I notice the same thing with a voice tool I use. What I want is to hold a button, speak and carry on. If I have to operate the software or rewrite its output, some of the typing it saved has returned as editing.

Maintenance counts as work too. In July, I wrote about building two sites with Ploy. I could have assembled scripts to do something similar, but I’d then have to look after them. That was part of the job I wanted the service to remove.

When you evaluate any product, including an AI product, start when the need appears and stop when the result is usable. Count the setup, explaining, moving information, waiting, checking, correcting, getting help and ongoing maintenance. The measure isn’t how fast the product responds. It’s how much less the customer has to do.

Test what can disappear

Choose one task a customer already does and watch the whole thing. Pick one piece of work to remove, then deliver the simpler experience. You can do the work manually behind the scenes if needed. That’s a Mechanical Turk pretotype.

Set the pass threshold before the test. Measure time and effort alongside accuracy, rework, help needed and trust. Then watch what happens the next time the same need appears. Do they choose the simpler route again? What still sends them back to the old process?

Some work should remain. People need to see uncertainty, correct mistakes and approve consequential actions. Removing those controls can create more work later.

Rapidly can generate a hypothesis and experiment options, but someone still has to check the assumptions, run the test and see what customers do. The software can’t turn its own suggestion into market evidence.

The aim is to remove the work that adds no value while keeping the work people need to make a good decision and trust the result.

This week, take one workflow being considered for AI optimisation. Ask whether AI should make each step faster, or make a large part of the workflow unnecessary. Test the smallest useful removal and count what remains.

Working on an AI product? Reply with the work you’re trying to remove, or put the idea through the free Idea Validator.

Leslie

exponentially.com

Welcome to the Experimenter’s Edge. Each month I share what I’m seeing in rapid experimentation and consulting work, what’s shipping inside Rapidly, and something practical you can use.

Last week during our weekly sim racing session, we ended up talking about the value of vibe-coded software.

We were discussing why so many vibe-coded products appear to have customers, even when people like us look at them and wonder who would pay for that. We also wondered why vibe-coding platforms like Lovable, often dismissed as AI slop generators, can attract hundreds of millions of dollars in investment.

Maybe lots of people have always wanted to make software but couldn’t. Now they can, and customers are buying what they make. You can roll your eyes at some of the output, but the behaviour is real.

Vibe-coded products are evidence of previously blocked demand, not just rough software.

That stayed with me because the same thing is happening inside companies.

A few days later I was talking with a client about AI ideas appearing in small pockets across the business. Someone finds an annoying piece of work, opens an AI tool, and builds an agent or rough application to deal with it. Someone else does the same thing in another team. Some of these prototypes look polished enough to feel like products, even though they’re still ideas with a user interface.

I think companies are missing a useful signal here.

I think that effort is admirable. Someone has gone beyond talking and built something to fix their work, often on top of the job they were already doing. They’ve put some skin in the game, which is much stronger evidence than another suggestion on a workshop wall.

But what does that evidence actually prove?

It proves one person cared enough to act. It doesn’t prove that their agent is the right solution, that ten other people have the same problem, or that the company should now fund it as a product.

The current conversation about shadow AI often starts with governance. I would start with the builder. Ask them to show you what they made, why they made it, and what support would help them test it properly.

The prototype tells you where to look.

Start by finding the unofficial tools people have already made. Ask what was irritating enough for them to build something, what they were doing before, and whether anyone else has the same problem.

Then group the tools by the underlying job rather than the proposed technology. Three different agents may turn out to be three attempts to remove the same piece of manual work. That repeated pain is more interesting than any one demo.

From there, treat the strongest problem like any other idea:

1. Write down who has the problem and what they’re trying to do.

2. Agree on the behaviour that would show the problem is worth solving.

3. Put the smallest credible version in front of the likely users.

4. Measure what they do rather than asking whether they like the demo.

5. Decide whether to fund it, change it, or stop.

This is the same basic path we use in Rapidly.

A polished AI prototype doesn’t skip validation. It enters the process with a better starting signal than a Post-it note.

Companies still need to know what data and models people are using. A central AI register is more likely to work when it gives the builder something useful in return.

The central team can offer something useful instead: access to better models, tokens, security help, an expert who can sit beside the builder, and a fast route to testing or funding. Bringing the work into one place should help the person move faster.

I want to test this approach with a few companies over the next month.

What we shipped: Rapidly through your AI client

On 3 September we switched on the new Rapidly MCP customer journey in production.

In plain English, you can now ask Claude or ChatGPT/Codex to test an idea before you build it. Rapidly saves the idea, works through the Lean Canvas and hypothesis, designs the experiment options, recommends the easiest credible test for the riskiest assumption, and gives you a copy-ready prompt to build it.

The work stays in Rapidly, so the next conversation can continue from what was already saved rather than starting again with another chat. The AI client becomes another way into the same idea-to-evidence process, not another place for the work to disappear.

It’s live now. You can connect Rapidly to your AI client and try it.

Spotted in the wild

Lenny Rachitsky shared a useful line from the product lead for ChatGPT Work: as agents take on more of the execution, the human role moves from rowing to steering.

I think that’s right, but it raises the standard for the person steering. Faster building means you can travel a long way in the wrong direction before anyone notices. If the quality of the idea and hypothesis doesn’t keep pace with the execution speed, the agent just compounds the error faster.

Someone still has to choose the problem, set the evidence threshold, and stop when the evidence doesn’t arrive.

Tool of the month: Aqua Voice

Aqua recently told me I’d dictated 398,744 words and was in its top one per cent of users. Apparently it is saving me about two hours a week.

I use it to talk into any text box on my Mac: prompts, emails, documents, Slack, messaging, and plenty of the rough notes that eventually become useful. Speaking gets me past the editing we do when typing and creates more context and clarity.

It doesn’t remove the editing. You still have to decide what you mean and clean up the result. But it has made the keyboard optional for a surprising amount of my week.

What has somebody already built to avoid a piece of work?

Ask one team to show you the unofficial tools they are using. Pick one and write down the job it removes, how the work happened before, and who else has the same problem.

If the answer is nobody else, it may still be a useful personal automation. If the pain repeats across the company, you have found an idea worth testing.

Reading

• How We’re Compressing Three Hours of Workshop Time Per Idea: how the Rapidly MCP is giving workshops a faster starting point without removing the humans from the work.

• Keeping people at the centre of ideation: what happened when five human teams and one AI team put their ideas under the same pressure at GembaCon.

Find the best ideas to invest in

If AI ideas and homemade tools are multiplying faster than you can evaluate them, I can help you turn the strongest problems into experiments and clear investment decisions.

Want to talk it through? Reply and we’ll set up a short call. Or try the free Idea Validator with one of the ideas already circulating inside your company.

Until next month, happy innovating!

Leslie

exponentially.com

I’ve been working on codifying how we go from high-quality, company-relevant idea generation, rather than just general AI ideas, through to creating and defining the full Lean Canvas, Pretotyping capability, hypothesis, and experiment design. It’s now working really well.

I tested it live with Gemba last week during a workshop for pre- and post-workshop setup, and I just wanted to share what it’s producing.

We can now compress three or four hours of workshop time per idea into a quick start for high-quality idea generation and high-quality experiment generation.

I’ll be sharing more on this soon. This was all generated using the Rapidly MCP for Claude or ChatGPT/Codex, which is now live, and I’ll make it available soon so it can be deployed at scale.

This is really helping us move faster from ideation to validation and focus more on working with the humans in the room to manage our exploding portfolio of product ideas.

** These are not real ideas from the workshop. They are simulated ideas just to demo how this works.

🙏 I’m looking for beta testers for the MCP. If you’re interested, just drop a message below. Thanks.

Andrew Condon, MD APAC of Gemba, invited me to run an ideation and experimentation workshop at QT Melbourne with around 50 people as part of GembaCon and the company’s 20th anniversary celebrations.

The Melbourne and Sydney teams worked on one question: what does the next wave of passion look like for fans?

They looked ahead to 2029, generated ideas and excitedly voted on the winners.

That’s normally where these sessions end.

Then we asked whether fans or customers would actually be interested in the ideas. We often pick the ones WE love, but of course we’re not the customer.

Each team ran its winning idea through a quick, AI-assisted Pretotyping session to get to the bigger question: should we build this?

They did that by thinking through and designing the first experiment they could test with customers.

We also ran an AI “sixth team” through the same exercise and compared the output at the end. Useful, but deliberately not the centre of the session.

The AI outputs were pretty good. The comparison reinforced the value of people building on one another’s thinking, with the technology supporting the work rather than taking it over.

Gemba did a great job of bringing marketing, creative, events, strategy, insights and data into the same conversation. The voting got competitive, the failure award was taken in the right spirit, and the energy kept building through the afternoon.

I liked the deliberate pause between generating an idea and building it. AI makes both faster, so teams need a point where they ask: is this the right idea for these customers, right now, in this context?

The Gemba team is willing to put its ideas under pressure and find out.

Thanks to Andrew, Elaine, Eliza and the whole Gemba team for having me.

Welcome to the Experimenter’s Edge.
Each month I share what I’m seeing in rapid experimentation training and consulting work, what’s shipping inside Rapidly, and a technique or two you can use this week. Glad to have you here, alongside 1,000+ pretotypers, product leaders, and experimenters.

Last month I said I was building this. It shipped.

Last month I told you I was working on closing the gap between an AI-built idea and the first real experiment. It’s now live, and in week one it generated 46 ideas for real companies. The most useful thing it did was stop more than half of them.

The stopping is what matters to me. Generating ideas is easy now and every tool does it, but working out which ones to bin is the harder and more valuable job, and it’s where the money sits: a valid idea makes money, a failed one saves it.

You already know the setup, because I’ve been banging on about it for months: building is cheap, a demo isn’t a validated product, and the only way to know is to put it in front of a real person and measure what they do. What’s changed is that validation can finally keep pace with the build, with human judgement kept in the loop. That’s what drove the biggest update we’ve shipped in Rapidly, our software for brainstorming, training, and consulting work.

What we shipped: the AI Idea Generator in Rapidly

Here’s what it does: it generates high-quality ideas specific to what your company is doing, then moves them straight into experimentation with the rigour of a lean canvas, a pretotype, and a hypothesis.

You choose how to work.

• Create an idea manually for workshops.

• Generate up to 20 ideas for your company, deduped against what you’re already doing, each with a rationale.

• Drop in one idea and have it build everything out like the validator.

• Or take an idea you’re working on and augment just the parts you want.

Yes, you could ask Claude, Fable, or Codex to do the same. The difference is that we’ve embedded the proven pretotyping method and refined it against our own IP and real experiments at scale, so the AI works as an enabler rather than an “AI!” button bolted onto a form. As I put it when we launched, the leverage isn’t in generating more ideas; it’s in generating better-context ideas and moving them into evidence faster.

It’s a first pass and it’s running live, so give it a go at rapidly.co and tell me what you think.

Spotted in the wild

Validate before the build decision. Teams whose engineering capacity has jumped are watching their backlog grow faster than they can test. The fix is to run a handful of real experiments before the big build call rather than after it, which is about the cheapest insurance you can buy.

Turn a backlog into decisions. Some of the most useful work is taking a stack of plausible ideas and producing a prioritised shortlist with a proceed, sequence, or stop call on each one. Being able to say what you won’t do is worth as much as the green lights.

Tool of the month: Ploy.ai

I handed Ploy.ai a PDF (drafted by Nous Research Hermes from Gbrain and Granola data) and it spun up two live sites built for AEO, roughly six hours of effort for what would usually take months. Rapidly.co and Exponentially.ai are both live off the back of it.

Fair warning: you probably won’t like those sites, and that’s fine, because they’re built for agents to find and consume, not for you. That’s the whole point of the experiment. The offers and products behind them are real, so they aren’t the pretotype; the pretotype is whether you can hand this kind of work to a service that keeps optimising, or whether it just turns into AI slop.

I’d call ploy.ai an inflection point rather than just another web builder because of the choice it represents. You could script this yourself in Claude and then maintain it forever, or you could lean on a curated set of skills with the AI judgement already built in. I’ll take the second every time.

Which of your half-built ideas has touched a real customer?

If the honest answer is none, pick one this week, define the smallest behaviour that would prove it, and test it with the simplest pretotyping method that fits: a Fake Door for intent, a Mechanical Turk to deliver the outcome by hand, a Pinocchio for a non-operational version, or a One Night Stand for a strictly limited live window.

Reading

• Idea Generators, by Reid Hoffman

“Do you know what to build, how to test it, and when to kill it?” Having a strategy to rapidly test ideas, identify the promising ones, and ruthlessly eliminate the rest isn’t just valuable—it’s essential. And that’s exactly what this book gives you

• Life at the Speed of Play by Mark Pincus, Zynga Founder

Find the best ideas to invest in

We help teams go from ideas to evidence in weeks, building rapid experimentation into their work so they can validate fast, stop the wrong ideas early, and back the winners with confidence.

Want to talk it through? Reply and we’ll set up a short call. Or just try the new AI Idea Generator or the free Idea Validator

Until next month, happy innovating!

Leslie

exponentially.com

• Accelerate how we add new ideas and generate high-quality ideas for companies based on what they're actually doing right now.

• Take those ideas and move quickly into experimentation while still using the rigour of a lean canvas, a pretotype, and a hypothesis.
  
  Our users now get a seamless choice of which path to follow. This is a first pass, and it's running live right now, so please give it a go and let me know what you think.
  
  1. You can create a new idea manually, which is really useful for running workshops.
  2. You can generate up to 20 ideas for your company, and it will dedupe them based on what you're already doing, with context about your business. It builds some context around what makes a good idea and gives you a rationale for why it chose those.
  3. You can add an idea like a validator that generates everything.
  4. You can pick an idea you're working on or have worked on and selectively augment it.
  
  And of course, you can just ask Claude or Fable or Codex to do something similar.
  
  But we've done a lot of refinement based on our internal IP, real ideas, and real experiments at scale. We've embedded the method in the tool to give you a really high-quality experience and make the AI useful as an enabler, not just an 'AI!!!' button.

If you are building innovation capability, this is where the leverage is: not just generating more ideas, but generating better-context ideas and moving them into evidence faster.

This is live now. Give it a go and let me know what you think.

Another inflection point. I asked Nous Research Hermes to use data and insights from Gbrain (from ) and Granola , etc., to draft a PDF for ploy.ai to spin up 2 new sites today that I’ve been wanting to use as experiments to optimise for AEO.

Ploy nailed the build, brand, etc. Super easy to use.

We’ll see what the results show (DATA>OPINION), but this is how we learn rapidly. 6 hours or so of effort to do a few months’ work. I thought I was wasting time/‘learning’ while wrestling with Hermes and gbrain to ingest my IP and data, etc but it paid off big time today.

Rapidly and Exponentially.ai are live as I write this. Incredible.

Just a warning: if you look at them, you’re probably not going to like them. But that’s not the point, right? They’re not designed for you. They’re designed for agents to find and consume. It’s an experiment to see how this works.

There are multiple layers of experiments built into this. Both offers and products are real and live, so that’s not the prototype. The pretotype is whether you can hand off this work to a service that constantly optimises and tweaks—or does it just turn into AI slop?

I’m prepared to experiment and take the risk. I’ll keep you posted on this.

Why do I call it an inflection point?

Surely it’s just another web builder AI tool. Well, not exactly.

You could build scripts and ask Claude to do this, and it would do a reasonable job, but then you’d have to maintain it for the rest of your life. Or you can work with curated experts who’ve taken the time and judgment to optimise for the right things and find that balance. I always prefer the second option.

It’s about using a curated set of knowledge, IP, and skills that already leverage AI, rather than constantly deciding which model to use, how much to spend, and worrying about tokens and the rest.

I’m happy to share more details about how this was set up and, more importantly, what doesn’t work. Ask away…

(zero tokens were used for this post)

put it well on ‘s Podcast. His view is that AI’s real value for experimentation is being squandered.
“So the way we should be using AI is as a testing machine, a failure machine and a way to vibe code... but build the lowest possible cycled version of your product that you can get signal back on. How are you testing a 100 ideas a day instead of one in three months? I think AI is being used more to build one idea in three months than a hundred ideas in a day.”

So how do we validate this ‘product’?

I’m shifting my thinking towards how we can accelerate validation and compress the time from idea to experimentation.

This is necessary because there’s an explosion of AI-built apps and half-solved problems. I’m sure you’ve all got five or ten half-finished projects, things you’ve spun up an initial version of and then abandoned because it got hard.

And that’s exactly the point at which we need to start figuring out how we test and experiment. How do we get data from customers and potential users to know whether this is a “hell yeah, let’s go” or an “absolutely not, stop it immediately” moment?

That’s what I’m working on right now.
If the AI-generated product IS the idea, then how do we use the proven Pretotyping and rapid experimentation as the underlying method?

It should be embedded and automated, with human-in-the-loop judgement and expertise, so we move quickly from an idea or AI-generated demo to the first experiment worth running. Then use Alberto Savoia‘s modified Bayesian probability to tell us whether it’s passed the pass/fail metric you set.

Maybe it’s time to think more about validating what AI has helped you build. Spend less time relearning how to experiment and innovate the pre-AI way, so you can focus your limited time, resources, and tokens on the right ideas.

I’d love to hear what you think and what you’re seeing in your own business.

And remember, there’s only upside in validating ideas!
Valid idea = make money
Failed idea = save money

A leader at a large retailer described it to me last month as “a long tail of ideas we can’t get to”. That’s not a strategy problem; it’s a validation capacity problem. And it’s the one AI is genuinely well-suited to solve.

That’s why we built the Idea Validator and made it a free public tool, no account needed to try it. Drop an idea in, give it a couple of minutes, and see what comes out the other end.

It produces a Lean Canvas, an XYZ-form hypothesis, and a recommended first experiment. The three hours of room time we used to spend on this becomes about two minutes, and the quality holds because the model is grounded in workshop patterns and real customer interactions, not generic frameworks. It’s a quick way to sample the pretotyping method.

It’s not trying to replace the human judgment work, just to remove the bottleneck that stops most teams from validating more than a handful of ideas a year.

A few practical notes. It’s also exposed as an MCP server, so other agents can call it as a service inside whatever workflow you’re already running. If you want to scale this across the entire backlog, that’s where I help embed this in your teams.

I’ve had great feedback on this so far, and it’s a fun starting point to get that flywheel spinning…

This is the first of two letters from me this month. The second will go deeper on the platform side: the pipeline rebuild, the Rapidly v2026 release, and where AI still falls over.

People > AI

I have lots to share about platform work, a rebuilt validation and experimentation pipeline, and where AI has taken some of the friction out. Before we get there, let’s talk about people.

What moves an organisation isn’t a tool. It’s the people inside it making better decisions, faster, with less noise. The teams who already understand the business, who carry the relationships, who feel the friction in the work, who know which ideas have legs and which ones the leadership team is quietly attached to for the wrong reasons.

The most valuable work I do isn’t deploying a platform. It’s helping those teams use what they already have, in a way that compounds.

What I do is knowledge transfer for experimentation and innovation capability inside existing teams. A way of thinking about product decisions that survives after I’ve left the room.

AI is a useful accelerator on top of that, not a substitute for it.

The clients who get the biggest return aren’t the ones with the slickest stack; they’re the ones whose teams can hold a hypothesis, run a real test, and change their minds based on what came back.

I keep meeting leaders who want to skip that part. They want the platform without the practice, the pipeline without the coaching, the AI without the judgement. It almost never works.

Teams who are already good at experimentation get better with AI; teams who aren’t mostly generate more polished bad decisions, faster.

A conversation that captured this well

I had a conversation last week with the product lead of a fast-growing consumer tech company. Engineering capacity had jumped, ideas were stacking up faster than the team could validate them, and the worry on the table wasn’t really about tooling. It was about trust.

How do they make sure ideas don’t disappear into a black hole the way they have at every other company those leaders had worked at? How do they get the rest of the business to believe that “we ran an experiment, here’s what we learned, here’s what we’re stopping” is a credible sentence rather than a polite no?

My answer was the same as it’s always been:

• Acknowledge every idea the moment it lands in the system.

• Validate against customer data, not against whoever raised it. Disconnect the idea from its owner so the team is reviewing the idea, not judging the person.

• Bring in a facilitator whose only job is to ask whether the thing actually moves the needle for a customer.

• Treat this as people work, not AI work. It’s where most of the gain actually comes from.

Two schools of thought on AI and teams

Two ways this is playing out right now:

• Cut headcount. Run leaner, hand work to agents, smaller teams shipping more.

• Keep your people, give each of them five to ten times the leverage. Take the dumb tax off the top of their day so they spend more time on the choices that compound. I’m firmly in the second camp. The best engineers I know don’t want to write yet another login screen or auth flow. They want to be unleashed on the work that actually requires judgement. The best designers and product managers don’t want to slow code production; they want to feed cleaner direction into it.

Celebrate the failures

The most powerful moment I’ve seen in an experimentation team was a senior person standing up in front of their peers and saying “I’m the biggest failure on this team”, and meaning it as a credential. It’s the opposite of how most companies are wired.

Teams who get good at this stop being afraid of the no-result experiment and start treating it as the cheapest possible insurance against the wrong investment.

The thing that gets missed: rhythm

The most powerful piece of any operating model isn’t the tool or the framework. It’s the rhythm:

• A 45-minute prioritisation session every Monday.

• A short midweek check-in.

• A Thursday review of what shipped, what stopped, what we learned. Those standing ceremonies are what make the discipline survive contact with a busy roadmap. Skip them and the whole thing reverts to vibes.

The unlock is people; the tools just remove the blockers.

AI accelerates people

I’ve spent the last few months going deep on tooling, automation, and compression. Work that used to take days now takes hours, and the gains are real.

What hasn’t changed is who does the work. AI accelerates and streamlines; the judgement, the relationships, and the read on what actually matters still come from your people. That’s where my focus sits.

The companies who need me are the ones that already have good people, good systems, and a real business to run, and who want all of those to get better.

Innovation and experimentation drive growth

Innovation and experimentation are fine words inside the practitioner community, but in the boardroom they read as long-horizon and abstract. Leaders are asking more direct questions:

• How do we get more growth this quarter?

• How do we take cost out without breaking things?

• How do we get confidence about where to invest before we commit?

**The **ingredients to innovate and see results rapidly already live inside your organisation. The teams, the relationships, the customer signal, the unused data. My job is helping you use what’s already there, with a small amount of structure and a useful amount of AI on top.

How we’re packaging work this year

Short note on the offer, because we’ve simplified it. Three tiers, all attached to a Rapidly licence so the discipline survives after we leave the room.

Tier 1. Rapid Experimentation Operating Model. A 90-day multi-squad install: discovery, Rapidly deployment, methodology training, a coached experiment cadence, and an internal experimentation team trained to sustain it. Same shape as the engagement that ran 130+ experiments and saved an ASX-listed enterprise group $12M by killing the wrong investments early.

Tier 2. Rapid Experimentation Sprint. A four-week, single-team engagement on one high-stakes idea or one team’s capability lift. Workshop, working sessions, synthesis playback, decision-ready output.

Tier 3. Half-Day Workshop. A focused session for product, innovation, or leadership teams. The fastest way to align a group on what good looks like before any further investment.

If one of those sounds useful, reply and we’ll work out which one fits.

Spotted in the wild

Wrapping a seven-session coaching arc

I closed out a seven-session pretotyping engagement with a CTO at a technology startup last week. The measure of success at the end of every engagement I run is whether they can say out loud, “we ran an experiment, here’s what we tested, here’s what we learned, here’s what we’re doing next”. If they can say that sentence on their own, the work landed.

Three questions worth sitting with

1. What’s stopping your teams from changing their minds based on the evidence?

If the honest answer is politics, or the person attached to the idea, that’s a facilitation problem more than a data problem. Disconnecting the idea from its owner is usually the first unlock.

2. When did you last celebrate a no-result experiment in front of your team?

If you can’t remember, the team is probably still optimising for the appearance of being right. The cheapest insurance against the wrong investment is making “we tested it and it didn’t work” a normal sentence.

3. Does your experimentation rhythm hold up when the roadmap gets busy?

A 45-minute Monday prioritisation, a midweek check-in, a Thursday review of what shipped, what stopped, what we learned. If those three slip the first time things get tight, the discipline isn’t installed yet.

Reading from the blog

• I Gave AI Real Access to My Business. Here’s Why

• My AI Lied to Me. Multiple Times.

Find the best ideas to invest in

We work with teams to go from ideas to evidence in weeks, embedding rapid experimentation using pretotyping as a core capability. Validate fast, stop the wrong ideas early, and back the winners with confidence.

👉 If you’d like to talk through what the Rapid Experimentation Operating Model could look like in your organisation, happy to do a free 15 or 30 minute call. Just reply.

“Data beats opinion. Every time. Across every resource type.”

Until next month, happy innovating!

Leslie

The Talk That Changed the Conversation

I was invited to speak with the VicRoads Product Management team last month about pretotyping and AI. About 30 people in the room, lunch-and-learn format, part demo, part honest conversation.

Halfway through, someone asked the question I hear in every session: “This makes sense, but how do we actually start?”

It’s the right question. And it’s the one most AI conversations skip entirely.

Here’s what I’ve noticed over 4,000 experiments. The organisations that waste the most money aren’t the ones with bad ideas. They’re the ones that build before they test. And AI is making that problem worse, not better.

Before AI, most teams could manage four to eight ideas a year. The failure rate was 80 to 90 percent. Astro Teller from Google X has talked about this publicly. But with only a handful of bets, the losses were survivable.

Now AI can generate and build at 10 to 100 times the speed. That’s genuinely exciting. But the failure rate hasn’t changed. You’re still going to get it wrong 80 to 90 percent of the time. The difference is that you can now get it wrong much faster and at a much greater scale.

That’s where pretotyping comes in.

The 4-Minute Demo

In another session, I ran a live demo. Idea to Lean Canvas to XYZ hypothesis to experiment design. All in about four minutes, using the Idea Validator.

The room went quiet. Not because it was flashy, but because it removed every excuse. The process that used to take a team the better part of a week now takes minutes.

But here’s the thing. The experiments still run in the real world, with real customers, measuring real behaviour. AI compresses the obvious. Humans apply judgement. Together, you get to evidence faster without skipping the parts that matter.

The SMS Story

I walked through a real example from a previous engagement. The original plan was a full integration project. Business case. Approvals. Six months of development. Significant cost.

Instead, the team bought 10 mobile phones and sent manual text messages to customers to test whether they’d engage with a new service. Two days. Roughly zero dollars.

Same answer. 60 times faster. 180 times cheaper.

That’s not a shortcut. That’s better decision-making.

Three Questions I Left the Room With

1. Are your decisions based on data or opinion? Check your last three product or project decisions. Were they backed by customer behaviour data, or by someone’s conviction in a meeting?

2. What’s your experiment velocity? How many experiments did your team run on real customers, measuring real actions, last quarter? If the answer is fewer than five, you’re guessing at scale.

3. What should you stop? Every organisation has at least one project that everyone quietly suspects won’t work but nobody can kill. That’s your first experiment. Find the $1 million save.

Spotted in the Wild

It’s been a busy few weeks. Here’s what’s been happening in the world of experimentation and AI.

Three talks in one month

I presented “Know Which Ideas Will Win: Pretotyping and AI for Smarter Innovation” to VicRoads, a gaming company, and walked a product team through the rapid experimentation approach. Each audience was different, but the core message landed the same way every time: AI solves “how will we build it?” but it doesn’t answer “should we build it?”

$5M to under $1M

I’ve met with another client that I worked with previously, and they’ve already demonstrated something powerful. By applying lean canvas thinking to a planned $5 million vendor implementation, they reduced the project scope to under $1 million. Same business outcome. A fraction of the risk. They’re now building experimentation into their core approach, and we’re looking at how AI can supercharge that process even further.

GenAI Advanced Professionals session

I ran a session with a group of senior professionals covering the practical AI toolkit. We walked through setting up an autonomous AI agent, the real costs involved ($200/month for Claude Pro Max is the sweet spot), and how to get started without burning thousands on API usage. The standout moment was Chris demonstrating how he transformed a basic pricing document into a comprehensive three-way financial model with full P&L, cash flow, and sensitivity analysis. All with AI.

Exponentially Platform updates

The platform has had a significant refresh. We’ve built out full subscription billing, redesigned the single idea page, and continued optimising the IdeaValidator tool. The focus is on making it easier and faster for teams to get from idea to experiment.

Tool of the Month

OpenClaw

This is the one that’s changed how I work. OpenClaw is an open-source AI agent framework that went from zero to one of the most-starred GitHub repositories in history in four months. It lets you set up an autonomous AI agent that works across your calendar, email, CRM, Google Drive, and files.

I’ve been running my agent, “Blue,” since January. It has full access to my business systems (except customer data) and does things like:

• Proactively gathers context from my calendar, emails, and CRM before meetings

• Summarises deals, drafts follow-ups, and flags stale opportunities

• Self-learns from interactions and retains memory across sessions

• Runs research and content tasks while I sleep

The key difference from standard AI tools is that it doesn’t just respond when you ask. It builds context over time, learns your preferences, and acts proactively. It’s the difference between having a chatbot and having a team member who’s always on top of everything.

Setup tips:

• Pair it with a Claude Pro Max subscription or OpenAI. The Sonnet model is fine and has a better personality than OpenAI in my experience.

• Run it on a dedicated machine (Mac Mini is a popular choice), isolated from personal data or on a VPN. To save you time, I’ve tried both methods and found the VPN much easier, more cost-effective, and with less overhead. I’ve got a guide to set it up easily, so just reply to this email if you want it.

• Start with read-only access and expand as you build trust

• The agent does its own security hardening and penetration testing if set up correctly.

If you’re only going to try one new AI tool this quarter, make it this one.

Reading

A few recent posts from the blog:

• I Gave AI Real Access to My Business. Here’s Why

• Phase 1: Rebuilding a Platform With AI

• My AI Lied to Me. Multiple Times.

• I Rebuilt It in 71 Hours. That’s the Problem.

Find the Best Ideas to Invest In

I work with teams to go from ideas to evidence in weeks. We embed rapid experimentation using pretotyping as a core capability to validate fast, stop the wrong ideas early, and invest in the winners.

👉 If you want to hear more about how we do this, happy to do a 15 or 30 min free call. Just reply.

👉 If you’d like to try the Exponentially Platform or any of the tools I’ve mentioned, reach out, and I’m happy to share access.

“AI compresses the obvious. Humans apply judgement. Together, you get to evidence faster without skipping the parts that matter.”

Until next month, happy innovating!

Leslie

Go from AI and product ideas to evidence in weeks. We help you validate fast, stop the wrong ideas early, and make decisions with data, not opinions.

Join 3,000+ pretotypers, product leaders, and experimenters getting hands-on tactics for rapid experimentation. Not theory, real techniques you can use this week.

The Real Goal: Finding the Best Ideas to Invest In

Experimentation is great, but what you’re really looking for is the best ideas to invest in. The goal is to run as many experiments as possible as fast as possible to reduce your time to data, while still keeping the human element and diversity of thinking. Gather ideas, prioritize, design experiments, run them, get feedback, and make decisions.

We’ve been exploring how to compress each of these steps using new AI capabilities, drawing on thousands of experiments we’ve run and the body of knowledge based on pretotyping.

Compressing Each Step of the Innovation Process

Let’s break this down step by step to see how we can speed up each stage.

1. Compress the brainstorming

We’ve been experimenting with a tool to generate quality, reasonably unique ideas based on understanding your company, your competitors, and your strategic goals. It can generate 10, 20, 50, or even 100 reasonably good ideas, which then go straight into the platform for evaluation.

2. Augment the prioritisation step

This is still a judgment call for your teams to prioritise, but we built a tool to quickly assess the idea. it’s still in the pretotype phase, but you can try it here. I’d love your feedback.

3. Speed up the Exploration phase

This is where the AI tools have really been effective. We’ve used our knowledge from thousands of experiments and Lean Canvases we’ve built to embed best practices for doing pretotyping and rapid experimentation at scale in the real world, while considering risk, governance, and legal constraints, of course with fully anonymised data. We’re able to take a first pass at evaluating your idea, generate a Lean Canvas, and suggest a reasonable XYZ hypothesis. It’s still a work in progress, but it’s exciting to see how we can make this go faster for you.

4. Get to experimenting as fast as possible

We’re also experimenting with a tool to recommend and suggest experiments for your particular business, its stage, and what you’re trying to learn. And we’re getting really good results from this. So watch this space.

5. Build and run the experiments

Up until now, with the variety and complexity of customers we’ve worked with, we haven’t run experiments on their behalf and we might still not do that. And that is often a bottleneck on how to get access to resources and engineering talent. But with the advent of AI we’ve been testing how you can spin up quality safe experiments within a day or two. They conform to brand, risk, and compliance, and we get them out the door fast.

Spotted in the Wild

I’ve been eating my own dog food. Here’s what I built over the past 8 weeks experimenting with Claude Code:

1. Exponentially Platform re-platforming (37 hours)

Consolidated and rebuilt the entire platform with AI-accelerated features.

2. Custom time management system (22 hours)

100% tailored to my workflow instead of settling for 80% SaaS solutions.

3. Property research automation (2 hours)

Built on a Friday night. Auto-generates research summaries for property inspections, tracks preferences, and surfaced pre-inspection details we never would have found manually.

4. Pipedrive sales bot

Connects Claude to our CRM. Summarises deals, drafts follow-ups, flags stale opportunities. Replaced manual pipeline reviews.

5. Claude Ralph (open source)

Autonomous AI development loop for Claude Code. Runs overnight and builds features without manual intervention. github.com/LeslieCBarry/claude-ralph

The So What: $80,000 to $90,000 Saved

By my estimate, I’ve saved about $80,000 to $90,000 by doing this. That’s real money that would have gone to SaaS subscriptions and development costs, and ongoing cost-avoidance for 2026.

How to Get Started

So how do you activate this? What’s your next experiment?

Have a look at all the services that you use and ask AI to brainstorm with you on the parts that you actually use, not the 80% that you don’t. See if you can build that yourself.

The space is moving fast, so get your hands on the tools, whatever your role. You’ll be amazed.

A quick example: I’ve fully replaced Zapier, Rebrandly, and multiple other tools by building exactly what I needed.

Tool of the Month

Claude Code

After 20 years away from coding, Claude Code has reminded me that everything is code. If you’re not doing this, you’re missing a superpower to accelerate your understanding of what’s possible. I’ve shipped more working tools and platform updates this month than we did in the last year.

https://github.com/LeslieCBarry/claude-ralph

Here’s my best tip from the past few weeks: I found and tested a way to build features autonomously with a self-improvement loop. The tool is called Ralph, and it’s been incredibly useful. I modified it to use Claude Code instead of AMP, and it ran flawlessly, building a new feature in just a few hours.

Thanks to Greg Isenberg and Ryan Carson for getting this walkthrough out there so fast. Follow them if you want actionable, practical tips and tools in this space.

Find the Best Ideas to Invest In

I work with teams to go from ideas to evidence in weeks. We embed rapid experimentation using pretotyping as a core capability to validate fast, stop the wrong ideas early, and invest in the winners.

👉 If you want to hear more about how we do this, happy to do a 15 or 30 min free call. Just reply.

👉 If you’d like to try any of these tools I’ve mentioned in their current pretotyping state, reach out and I’m happy to share the links with you.

“Run as many experiments as possible as fast as possible to reduce your time to data.”

Until next month, happy innovating!

Leslie

Back then, that obsession helped Microsoft beat Apple and dominate the market.

Today, the stakes are even higher — because "developers" isn't a niche anymore.
Product managers, marketers, ops, founders, analysts. Anyone who can prompt an AI and ship workflows, scripts, and tools is effectively a developer. The developer market is 100–1000x bigger than it was in Ballmer's era.

I was writing code and attending Tech-ed conferences in the early 2000s. What I'm seeing now feels like that same inflection point — except the tent is massively bigger.

And I'm not just observing it — over the past 8 weeks, Claude Code has become my dev team. I've shipped more working tools and platform updates this month than we did in the last year.

That's why Claude Code is winning right now:

• Removes friction from making things that actually work

• Opinionated about real workflows, not generic chat

• Built for people who ship, not people who "explore AI"
  
  OpenAI is still very much in the game, but it's currently too general-purpose. Codex is fine, but Claude Code feels built for the new reality where everything is code and everyone is writing it — whether they realise it or not.
  

  The next platform winners won't just serve developers. They'll create them.
  

Two Adams Walk Into a Bar (Well, Almost)

Last month we pretotyped a live LinkedIn webinar with two Adams.

Adam Murphy, a human-centered strategy expert. Adam Norris, a design and prototyping specialist. And me—eight years deep into pretotyping at scale.

The format was simple: let’s talk about what makes pretotyping different from prototyping, and why the distinction actually matters.

What I didn’t expect was how much the conversation would clarify something I’ve been wrestling with for years: why do smart teams keep jumping straight to “can we build it?” before answering “should we build it?”

My Job Is to Stop People Getting to Your Job

About 15 minutes in, Adam Norris asked me how I got into pretotyping.

I told him the truth: through building startups, failing at most of them, and getting really frustrated that we kept spending time and money building things nobody wanted.

Adam’s a prototyper. He makes physical products real. He helps teams test whether they can build something, whether it works, what it costs.

My job is to stop people taking bad ideas to Adams job.

But if I can help teams figure out should we even bother before they get to Adam, we save time, money, and a lot of heartbreak.

That’s the difference:

• Prototyping asks: “Can we make this thing?”

• Pretotyping asks: “Should we bother to make this in the first place?”

And here’s the kicker: most organizations skip pretotyping entirely. They go straight from idea to prototype to MVP—without ever testing if customers actually care.

↓ Watch this 2-minute clip to hear how I went from failed startups to falling in love with Alberto Savoia’s approach.

The MVP Trap

Adam Murphy asked the question everyone asks:

“Is a pretotype the same as an MVP?”

Absolutely not.

Here’s what’s happening right now in innovation: someone has a brain wave, they vibe-code a thing using AI, and they put it out there. They’ve skipped the most important step: does anybody care?

An MVP is build-measure-learn. You build something minimal, measure how people use it, learn from that.

Pretotyping is learn-measure-build. You learn if there’s a problem worth solving, you measure actual customer behavior (not opinions), and then you build.

I gave them a recent example: I worked with a crypto exchange last month. Their first pretotype wasn’t code—it was Intercom. We put a value proposition in front of 100 customers, asked them a question, measured if they engaged.

Fast. Cheap. Data over opinion.

And here’s the uncomfortable truth I shared: Sadly for everybody listening, mostly nobody cares. No one’s interested. All your ideas are bad. Nobody cares about your stupid idea.

It’s very difficult to get any level of interest. So you need to test fast and cheap to find out which ideas actually have legs.

↓ Watch this 3-minute clip for the full breakdown of pretotyping vs prototyping vs MVP (including the crypto exchange example).

Spotted in the wild:

Three insights from the conversation that you can apply immediately:

1. Disconnect the idea from the person

Traditionally, we pitch ideas like this: “I’ve got this awesome idea, I’m putting my career on it, give me money.” Then if it fails, someone’s in trouble.

Instead: test ideas as a team, from the customer perspective. Run structured experiments where you agree upfront what “good” looks like. When customers don’t engage, it’s not personal failure—it’s rapid learning.

2. Low fidelity gets better feedback

Adam Norris made a great point: when you show people a polished prototype and ask “what do you think?” they say “that’s pretty good.”

But when you show them a pencil sketch or Play-Doh wrapped around another product, they give you real feedback: “It’s too tall,” “Why purple?” “We don’t need another soft drink.”

Lo-fi isn’t a weakness—it’s a feature. It invites honest critique.

3. The right question defines the right prototype

Not every test needs to be high fidelity. If you’re testing whether bar layout improves customer flow, build it out of Corflute (which I did for Anheuser-Busch InBev in Boston).

You don’t need feedback on aesthetics. You need feedback on flow. Match your fidelity to your question.

The 60-Day Product Launch

Adam Norris shared one of my favorite examples: Carlton United Breweries launching their first seltzer in 60 days—from idea to shelf.

How? They ran a Design Sprint. Marketing, agency, decision-makers all in one room. Adam’s team brought prototypes to life in real-time—3D printing different can shapes, printing new labels, testing with consumers iteratively over three days.

Meanwhile, the technical team was in the kitchen changing flavor proportions based on live feedback. “Too much vodka? Add less.” “Lime needs mint? Let’s try it.”

They didn’t answer every final detail in those three days. But by iterating so quickly, they compressed months of work into 60 days.

The breakthrough? Asking “what would have to be true?” instead of “why can’t we?”

↓ Watch this 3-minute clip to hear Adam walk through the full Carlton United Breweries story.

Why Innovation Leaders Last 18 Months

Toward the end, I shared something I’ve researched on LinkedIn:

The average lifespan of an innovation leader is 18 months.

Why? Because they don’t build sustainable systems. They rely on champions pushing upstream against an organization designed for stability and repeatability.

If that’s true, you can’t fight the system. You need to work with it. Create something the system recognizes.

I learned this firsthand when I took four customers on what became the “Global Pretotyping Summit” in 2019. We visited CES in Vegas, then toured Google, Facebook, and other Silicon Valley companies. Seeing Facebook run 10,000+ experiments concurrently—releasing features automatically based on data—was a wake-up call. The companies that scale innovation don’t rely on heroes. They have systems.

↓ Watch this 2-minute clip about how the “Global Pretotyping Summit” accidentally became real.

That’s why we built Rapidly—a platform to manage ideas and experiments at scale. Because running 5–10 experiments a year is a waste of time. If 90% of ideas fail, you need to run hundreds of experiments to find the winners.

And that only works when you have a system.

How to Get Started

Both Adams had great advice for people wanting to build this muscle:

From Adam Norris (prototyping):

Get hands-on. Buy a competitor product, mold Play-Doh onto it, scan it with your phone’s free 3D scanning app, change the label, reprint it. You can get a rough prototype in under an hour.

Prototyping has never been more accessible. The tools are there. The question is: can you make time for it?

From me (pretotyping):

Read Alberto Savoia’s book The Right It. Take my free pretotyping course on YouTube. Download Rapidly and try it. Join our Slack community (850+ people from around the world).

But most importantly: just do it.

You’ve got that idea that’s been bothering you for a year. Whip up a website. Put it out there. See what happens. Learn.

Don’t be scared to fail. You’re going to fail 100%. But the outcome of failure is learning.

↓ Watch this 4-minute clip for practical tips on getting started with prototyping—even with zero tech skills.

The Bottom Line

If you knew it would take 20 failures to make a $100 million product, how fast would you want to fail?

As fast as you possibly can.

That’s pretotyping. That’s prototyping. That’s rapid experimentation.

The only way to beat the default outcome—which is failure—is to fail faster, cheaper, and smarter than everyone else.

Full conversation: ↓ Watch the uncut version, including stories about the accidental Global Pretotyping Summit, how Facebook runs 10,000 experiments a day, and why getting bad feedback on a prototype is actually the goal.

Tool of the Month

Claude Code + Windsurf Editor

I’ve been experimenting with Claude paired with Windsurf Editor, and it’s changing how fast I can prototype digital products. Think of it as having an AI pair programmer who can write, test, and iterate code in real-time—perfect for rapid pretotyping of software ideas. If you’re testing digital concepts, this combo lets you go from idea to working prototype in hours, not days.

Build the System to Test Ideas Fast

Stop wasting months on AI and Product ideas nobody uses. I work with teams to embed rapid experimentation using pretotyping as a core capability to test ideas fast, kill the losers early, and invest in the winners—in weeks, not quarters.

I run complimentary executive briefings and team sessions online. It’s a quick 30-minute intro to Pretotyping, real case studies, and how to test which ideas need validation before investment.

Book your free briefing

“Prototyping asks ‘can we make it?’ Pretotyping asks ‘should we bother?’ Most teams skip the second question—and that’s why most ideas fail.”

I can’t share the company name, but I can share the playbook.

Inside a multi-billion-dollar, highly regulated organisation, a small innovation team faced the classic challenge: how do you move fast when every decision needs approval, when compliance is non-negotiable, and when “just try it” isn’t an option?

They didn’t ask for permission to move faster. Instead, they partnered with Legal and Risk from day one. They embedded Pretotyping as the method for turning opinion into evidence. And they built safe-to-fail guardrails that could be reused across multiple experiments.

The transformation was remarkable.

What used to be “opinion shipping” — building features based on what stakeholders thought customers wanted — became an evidence-led Innovation Engine. Those retail touchpoint screens that everyone assumed were just “noise”? They became measurable conversion channels. Time from idea to first experiment? Down to about 8 days.

In roughly 12 months, they ran over 130 live experiments.

But the real win wasn’t the volume. It was the shift in decision quality. Leaders started trusting the process because experiments consistently answered “Should we build this?” before anyone wasted time on “How do we build this?”

What made it work

Three things stood out:

1. They partnered with Legal and Risk early — not as gatekeepers, but as experiment design partners. Getting permission frameworks upfront meant velocity later.

2. They ran concurrent experiments — patterns across multiple tests improved their selection quality. What worked in one channel informed tests in three others.

3. They prioritised speed over perfection — imperfect experiments that shipped in 8 days beat perfect plans that took months.

Read the full Case Study

The takeaway: Velocity without validated ideas is just fast failure. Pretotyping gives you both.

What happens when AI runs your customer interviews?

An unexpected experiment with voice AI that changed how we capture customer stories — and a case study from inside a regulated giant.

The AI Interviewer Experiment

Last month, I tried something I wasn’t sure would work: I let ChatGPT’s voice mode conduct a customer case study interview.

I’ve done hundreds of these interviews over the years, and I know the patterns: you ask a question, the interviewee starts answering, you’re already thinking about your next question, and suddenly you’ve missed the most important thing they just said. Then you’re scrambling to redirect the conversation back to what mattered.

But when I handed the facilitation to ChatGPT, something shifted.

The AI asked one question at a time. It held context perfectly. When someone gave a surface-level answer, I could prompt the AI mid-conversation: “Ask for a concrete example” or “That’s interesting, dig deeper there.” The AI adapted instantly, without the awkward conversational resets that happen when humans try to course-correct.

The result? A coherent, quote-ready narrative in one sitting. Less “wait, where were we?” and more focused storytelling from the person being interviewed.

Why this matters

Better interviews mean faster case studies, cleaner proof points for executives, and more useful enablement content. The quality of your customer stories directly impacts how quickly you can share proof of value across your organisation.

How to try it yourself

1. Brief the AI on your goal and any guardrails.

2. Ask it to keep single-question pacing (one question, wait for answer, then next)

3. Stay in the conversation to prompt follow-ups: “ask for a concrete example” or “summarise what they’ve said and move on”

Read the full AI-facilitation playbook

Spotted in the wild:

Three things from this month that you can apply immediately:

1. Time to first experiment beats time to perfect experiment

Stop debating the perfect test design. Ship something in 8 days. Learn from real data. Adjust. Repeat.

2. Partner with Legal and Risk before you need them

The teams that move fastest in regulated environments aren’t the ones who avoid compliance — they’re the ones who build reusable approval frameworks early.

3. Concurrent experiments compound learning

One test gives you one data point. Three tests running in parallel reveal patterns. Those patterns improve your ability to choose what to build next.

Building experiment velocity in your company? I work with leadership teams to embed rapid experimentation as a core capability, increasing confidence in investing in the right products. Reply if you want to explore what this looks like for your business.

One thing

Running a retrospective this month? Try this:

Copy your sticky notes into Claude or ChatGPT and prompt: “Group and label these into themes. Output a concise summary with next-step recommendations.”

I’ve been using this after team sessions, and it’s a 10x lift in clarity and speed. What used to take 20 minutes of manual clustering now happens in seconds, and the AI often spots patterns I missed.

Tool of the Month

Notion AI

I’m late to the Notion party. Now that everything lives here, Notion AI is the tool I didn’t know I needed. It has context across my whole workspace — case studies, client notes, past experiments, drafts — so it can spot patterns, pull the right snippet, or resurface that one insight without me copy‑pasting. The ability to edit and iterate in real-time on a document is 10 times faster. It’s smarter because it knows what I know. This is what Gemini should be doing in Docs, Sheets, and Gmail.

Get Started with Your Innovation Engine

I run complimentary executive briefings and team sessions online, so you can join in wherever your team is based. It is a quick 30-minute talk to introduce Pretotyping, share real case studies, and show how to test which AI and innovation ideas need validation before investment.

👉 Just reply to this email if you’d like to set one up.

Last week, I had the chance to speak at Mantel Group’s AI Innovation Day, and I opened with a provocation:

If AI lets us go straight from idea to vibe-coded proof of concept, does that mean experimentation is dead?

The answer is no. AI accelerates the build — but it doesn’t replace the fundamentals. We still have to ask the right questions, test with real customers, and let evidence guide our bets. That’s the heart of innovation.

What I Shared

• Evidence beats opinion every time. We’ve seen companies save millions by shelving the wrong ideas early, while generating new value from the right ones.

• AI is a game-changer. It accelerates everything: idea generation, coding, prototyping, testing. The real advantage isn’t just faster builds, but faster learning.

• The Innovation Engine matters more than ever. Start by learning the methods, accelerate with 2–4 week sprints, and sustain by embedding the capability into your teams.

Proof in Action

In the session I shared some real examples:

• An early project at SwiftX that was shelved → hundreds of thousands saved

• A failed AI concept that could have cost millions → stopped before launch

• An AI initiative validated with strong ROI → significant upside

• Another scaled AI experiment with a large customer → multi-million dollar value created

The message landed: winning with AI isn’t about having the boldest ideas, it’s about testing more ideas, faster, and only scaling what customers actually prove they want.

My Takeaway

Standing in front of a room of leaders grappling with AI’s potential, I was reminded that the fundamentals haven’t changed: innovation equals experimentation equals results. AI just makes it possible to do this at a speed and scale we’ve never had before.

👉 If you’re curious about how this applies in your context, the best place to start is with our Innovation Engine Playbook.

Now Free: Pretotyping Course on YouTube

When I first learned Pretotyping from Alberto Savoia, it completely changed how I thought about innovation. It’s the foundation of data-driven experimentation, and I believe it should be a baseline skill for every innovator.

That’s why I’ve made my complete $400 Pretotyping course free on YouTube. Not as a marketing play, but because I want more people using Pretotyping to test ideas with evidence, not assumptions.

Here’s what’s inside:

• Foundations of Pretotyping – what it is and why it matters

• The Innovator’s Mindset – how to escape “Thoughtland” and kill bad ideas early

• The Pretotyping Toolkit – practical methods like Fake Door and Mechanical Turk

• From Idea to First Experiment – structuring assumptions and launching your first test

• Running Experiments & Getting Data – designing pretotypes and getting the evidence you need

Hopefully, this is a small step toward more teams building what customers actually want.

👉 Start with Module 1 on YouTube

Building experiment velocity in your company? I work with leadership teams to embed rapid experimentation as a core capability, increasing confidence in investing in the right products. Reply if you want to explore what this looks like for your business.

Tool of the Month

Aqua Voice
I have been using Aqua Voice on Mac to work with Claude, ChatGPT, and Gemini without typing. It is context-aware, so it adapts whether you are in email, Slack, or Asana. After trying WhisprFlow and Talktastic, this is the one that stuck.

Get Started with Your Innovation Engine

I run complimentary executive briefings and team sessions online, so you can join in wherever your team is based. It is a quick 30-minute talk to introduce Pretotyping, share real case studies, and show how to test which AI and innovation ideas need validation before investment.

👉 Book your free session here

The results blew me away.

24 ideas captured. 12 experiments designed and run. 500+ customers engaged.

But the real transformation wasn't in the numbers; it was watching a team shift from "let's build this" to "let's test if anyone wants this."

Let me share what made the difference.

Leslie

Spotted in the wild:

Three things about building experiment velocity that you can apply immediately:

1. Time to first experiment beats time to perfect experiment

The team's first experiment was clunky. Tracking wasn't perfect. The message could have been clearer.

They launched it anyway.

Result: 20% engagement rate on their very first try.

The insight? Getting data from real customers in week one taught them more than another month of planning would have. Speed creates learning momentum that perfectionism kills.

2. "Data over opinion" becomes cultural DNA

By week 3, something shifted. Instead of debating whether customers would want a feature, they'd say "let's test it."

One experiment failed at 15% when they added pricing. Instead of defending the idea, they immediately started designing the next test.

The magic happens when teams stop being precious about their ideas and start being curious about customer behaviour.

3. Concurrent experiments compound learning

Weeks 1-2: One experiment at a time Weeks 3-4: 2-3 experiments running simultaneously

The breakthrough wasn't just running more tests—it was seeing patterns across experiments. What worked for monthly subscriptions also worked for security features. Customer psychology insights from one test informed three others.

Parallel experimentation creates a learning flywheel that single tests can't match.

The bottom line:

Most teams I work with are stuck in analysis paralysis. They want the perfect experiment, the perfect message, the perfect everything.

This team proved something different: imperfect experiments that ship fast beat perfect experiments that ship slow.

The 4-Week Rapid Experimentation Sprint:

Here's exactly how we transformed a team from zero experiments to running 3 concurrent tests:

Week 1: Foundation training and first experiment launch

- Rapid Experimentation using Pretotyping One-Day Workshop

- Lean Canvas workshops to capture customer perspective

- First Pretotyping hypothesis designed and live within 2 days

- Team learns "data over opinion" mindset

Week 2-3: Scale to multiple experiments

- 2-3 experiments running simultaneously

- Daily check-ins and experiment design workshops to replace endless planning meetings

- Focus shifts from perfection to velocity

Week 4: Results and sustainable process

- Retrospective on what worked (and what didn't)

- Framework for ongoing experimentation

- Team equipped to continue without external support using Rapidly software.

The outcome? A sustainable innovation engine that runs experiments as business-as-usual, not special projects.

Want to build this muscle in your team?

The framework is simple:

1. Start with customer problems, not solutions

2. Test one variable at a time

3. Set clear pass/fail criteria upfront

4. Launch in 8 days maximum

5.Learn, adjust, repeat

But simple doesn't mean easy. It requires shifting from "build first, validate later" to "validate first, build smart."

That's the cultural transformation that creates sustainable innovation velocity.

One thing

I used Ideaboardz for the retro, copied it into Claude.ai during the live session and prompted

‘we're running a brainstorm session to retro the rapid sprint project at [clientname]. Group and cluster these ideas’.

Instant 10x gain on time and quality of output.

Tool of the month:

After years of task management overload, I've finally found my solution in Akiflow.

I was drowning in Sunsama, Asana, Slack, Trello, starred emails, Pipedrive, and countless task lists. Akiflow brings everything into one view with tasks and calendar unified. This single source of truth has dramatically reduced my overwhelm and brought clarity to my daily planning. Game changer for my workflow!

Test your ideas using Rapidly.

Get the Playbook

Two weeks ago, Patrick Smith and I ran an experiment to see if we could use ChatGPT in voice mode to facilitate a case study discussion, preloaded with all the context, frameworks, and goals.
It has completely changed how I think about running sessions.

Imagine a brainstorming session or case study interview where no one is stuck facilitating, and the conversation flows seamlessly.

The results?

Within 15 minutes, we had forgotten we were even talking to an AI.

AI guided the discussion, allowing for natural meandering and bringing us back on track when needed.

By the end, it had generated three tailored case study drafts (focused on change management, risk, and outcomes).

This approach not only saved time but also preserved valuable organisational knowledge and created actionable artifacts in real-time.

Practically, I'm finding that AI is most useful in the experimentation process, as it helps facilitate collaboration on improving the quality of thinking at each stage and removes friction from the actual running of experiments.
*Case study coming soon.

How are you leveraging it to rethink your experiment workflows?

Subscribe now

For the past two years, I've been using Fireflies.ai to transcribe my MS Team's, Google Meet and Zoom calls. I have hundreds of transcripts BUT Fireflies has no bulk download capability. I often run these transcripts through ChatGPT to summarise project discussions or mine them for workshop improvements that I can make. My solution was to ask my VA to download the files manually and work from there.

I’m working on a project that requires 40 to 50 files, so I needed an alternative to my VA.

Enter Claude.AI

To solve this problem, I asked Claude to suggest options. Claude suggested a Python script 👀 (despite having zero Python experience). Working on my Mac, Claude guided me through multiple iterations over about 45 minutes until we had a working solution for automatic bulk downloads. While this automation has made my virtual assistant's transcription management role redundant, it's solved my problem, saved me money and paid for my Claude, ChatGPT and Fireflies subscription!

The script has been performing excellently, and I wanted to share it with you. Try it out and let me know if you discover any bugs or potential improvements.

Here are the complete step-by-step instructions to download your Fireflies.ai transcripts:

1. Initial Setup (one-time only):

bash
# Create directory 
mkdir fireflies_downloader 
cd fireflies_downloader
 
# Create the Python script 
nano fireflies_downloader.py 

# Get your API key from Fireflies.ai: 
# 1. Log into Fireflies.ai 
# 2. Go to Settings → Developer Settings → API 
# 3. Copy your API key 
# Set your API key in Terminal 
export FIREFLIES_API_KEY="your_api_key_here"
 
# Install required Python package 
pip3 install requests

2. Script Content:

• Open fireflies_downloader.py

• Press Control + K repeatedly to clear any existing content

• Paste the full script into fireflies_downloader.py

import requests
import json
import os
from datetime import datetime, timedelta
import time

class FirefliesDownloader:
    def __init__(self, api_key):
        self.api_key = api_key
        self.base_url = "https://api.fireflies.ai/graphql"
        self.headers = {
            "Authorization": f"Bearer {self.api_key}",
            "Content-Type": "application/json"
        }
    
    def get_transcript_content(self, transcript_id):
        """
        Fetch detailed content for a specific transcript
        """
        query = """
        query GetTranscriptContent($id: String!) {
            transcript(id: $id) {
                title
                id
                transcript_url
                duration
                date
                participants
                sentences {
                    text
                    speaker_id
                    start_time
                }
                summary {
                    keywords
                    action_items
                }
            }
        }
        """
        
        variables = {
            "id": transcript_id
        }
        
        try:
            print(f"Fetching content for transcript {transcript_id}...")
            response = requests.post(
                self.base_url,
                headers=self.headers,
                json={"query": query, "variables": variables}
            )
            
            if response.status_code != 200:
                print(f"Error getting transcript content: {response.status_code}")
                print(response.text)
                return None
                
            data = response.json()
            if "errors" in data:
                print("API returned errors:", json.dumps(data["errors"], indent=2))
                return None
                
            return data["data"]["transcript"]
            
        except requests.exceptions.RequestException as e:
            print(f"Error fetching transcript content: {e}")
            return None

    def get_transcripts(self, limit=25, to_date=None):
        """
        Fetch list of transcripts with date filtering
        """
        query = """
        query GetTranscripts($limit: Int, $toDate: DateTime) {
            transcripts(limit: $limit, toDate: $toDate) {
                title
                id
                transcript_url
                duration
                date
                participants
            }
        }
        """
        
        variables = {
            "limit": min(limit, 25),
            "toDate": to_date.strftime("%Y-%m-%dT%H:%M:%S.000Z") if to_date else None
        }
        
        try:
            print(f"Making API request for transcripts up to date {to_date if to_date else 'now'}...")
            response = requests.post(
                self.base_url,
                headers=self.headers,
                json={"query": query, "variables": variables}
            )
            
            data = response.json()
            if "errors" in data:
                print("API returned errors:", json.dumps(data["errors"], indent=2))
                return None
                
            return data
            
        except requests.exceptions.RequestException as e:
            print(f"Error fetching transcripts: {e}")
            return None

    def save_transcripts(self, output_dir="transcripts", to_date=None):
        """
        Save all transcripts with their content
        """
        if not os.path.exists(output_dir):
            os.makedirs(output_dir)
        
        # Get existing transcripts to avoid duplicates
        existing_files = set()
        existing_ids = set()
        for filename in os.listdir(output_dir):
            if filename.endswith('.json'):
                existing_files.add(filename)
                try:
                    with open(os.path.join(output_dir, filename)) as f:
                        data = json.load(f)
                        existing_ids.add(data['id'])
                except:
                    pass
        
        last_date = to_date or datetime.now()
        while True:
            result = self.get_transcripts(limit=25, to_date=last_date)
            if not result or "data" not in result or "transcripts" not in result["data"]:
                print("No more transcripts found or error in API response")
                break
                
            transcripts = result["data"]["transcripts"]
            if not transcripts:
                print("No more transcripts found")
                break
                
            print(f"\nFound {len(transcripts)} transcripts")
            
            earliest_date = None
            for transcript in transcripts:
                # Skip if we already have this transcript
                if transcript['id'] in existing_ids:
                    print(f"Skipping existing transcript: {transcript['title']}")
                    continue
                
                print(f"\nProcessing: {transcript['title']}")
                
                # Get detailed content
                content = self.get_transcript_content(transcript['id'])
                if content:
                    transcript = content  # Replace with full content
                
                # Save the file
                date_obj = datetime.fromtimestamp(int(transcript["date"]) / 1000)
                safe_title = ''.join(c for c in transcript['title'] if c.isalnum() or c in (' ', '-', '_', '.'))
                filename = f"{date_obj.strftime('%Y-%m-%d')}_{safe_title[:50]}.json"
                
                filepath = os.path.join(output_dir, filename)
                
                with open(filepath, 'w', encoding='utf-8') as f:
                    json.dump(transcript, f, indent=2, ensure_ascii=False)
                
                print(f"Saved transcript: {filename}")
                
                # Track earliest date we've seen
                if earliest_date is None or date_obj < earliest_date:
                    earliest_date = date_obj
                
                time.sleep(1)  # Be nice to the API
            
            if earliest_date:
                # Subtract 1 second to avoid getting the same transcript again
                last_date = earliest_date - timedelta(seconds=1)
                time.sleep(1)  # Extra pause between batches
            else:
                break

def main():
    api_key = os.getenv("FIREFLIES_API_KEY")
    if not api_key:
        print("Please set your FIREFLIES_API_KEY environment variable")
        return
    
    print(f"Using API key: {api_key[:5]}...{api_key[-5:]}")
    
    # Find the earliest date from existing files
    earliest_date = None
    if os.path.exists("transcripts"):
        files = os.listdir("transcripts")
        if files:
            try:
                for file in files:
                    if file.endswith('.json'):
                        with open(os.path.join("transcripts", file)) as f:
                            data = json.load(f)
                            date = datetime.fromtimestamp(int(data["date"]) / 1000)
                            if earliest_date is None or date < earliest_date:
                                earliest_date = date
                
                if earliest_date:
                    print(f"Starting from before date: {earliest_date}")
            except Exception as e:
                print(f"Error reading files: {e}")
    
    downloader = FirefliesDownloader(api_key)
    downloader.save_transcripts(to_date=earliest_date)

if __name__ == "__main__":
    main()

4. Save with Control + X, then Y, then Enter

3. Running the Script:

bash
# Navigate to the directory (if you're not already there) 
cd fireflies_downloader 

# Run the script 
python3 fireflies_downloader.py

4. What to Expect:

• First run will download the most recent transcripts

• Each subsequent run will fetch older transcripts

• Script will automatically:

  • Skip duplicates

  • Show progress

  • Create a 'transcripts' folder

  • Save each transcript as a JSON file named with date and title

5. To Get More Transcripts:

• Just run the script again

• Keep running until you see "No more transcripts found"

• Each run will go further back in time

6. Files Location:

• All transcripts are saved in the transcripts folder

• Files are named like: YYYY-MM-DD_Title.json

• Each file contains:

  • Full conversation text

  • Speaker information

  • Summary and keywords

  • Meeting metadata

7. Troubleshooting:

• If you get "API key not found" error:

bash
export FIREFLIES_API_KEY="your_api_key_here"

• If you close Terminal and come back later:

bash
cd fireflies_downloader 
export FIREFLIES_API_KEY="your_api_key_here" 
python3 fireflies_downloader.py

8. Best Practices:

• Run during off-peak hours

• Let each run complete

• Keep the script and files for future use

• Make backups of your transcripts folder

The script is designed to be safe to run multiple times - it won't duplicate downloads and will automatically continue where it left off.