Understanding vs Optimisation
Not everything that can be counted counts, and not everything that counts can be counted
I've noticed something odd about how we build products now. We've become extraordinarily good at optimisation.
We can measure everything: how long users spend on a page, where they click, when they leave, what they buy. We can A/B test every button color, every headline, every image. We're masters of incremental improvement.
And yet, many of the products we use daily are getting worse, not better.
This seems like a paradox until you realise that optimisation and understanding are different things. Optimisation is about getting better at what you're already doing. Understanding is about knowing what you should be doing in the first place.
The Optimisation Trap
I once were quite close in a discussions with a video streaming service where the team ran exhaustive A/B tests on their "Continue Watching" prompts.
They tested every variation of timing, positioning, and messaging. After hundreds(Alex?) of tests, they'd increased their autoplay acceptance rate by 60%.
The metrics looked fantastic. But they discovered 6 months later that their subscriber churn had actually increased by a very small number, but on they scale it was about 5M$ profits and I don’t know how much valuation.
When they finally dug into the qualitative feedback, they found that while more people were letting videos autoplay, they were also feeling more anxious and guilty about their viewing habits. They'd optimised for continuation at the cost of user wellbeing.
The A/B tests had given them a local maximum that was actually hurting their long-term success.
This same pattern plays out in product development every day, sometimes at large scale. Teams get caught up in optimising metrics because metrics are comfortable. They're concrete. They make us feel scientific. Most importantly, they give us the illusion of control.
But there's a deeper issue here that nobody talks about: not all products should be optimised the same way.
Two Kinds of Products
Products roughly fall into two categories, which we might call "intent-flexible" and "intent-fixed."
Intent-flexible products are like e-commerce sites or social media apps. Here, the user's intent is malleable. If I'm browsing Amazon, I might come looking for books but leave with a new coffee maker. That's fine - sometimes even beneficial. The product can create or modify my intent without necessarily doing harm.
Intent-fixed products are different. Think about a banking app, or software for emergency services. When I open my banking app to send rent money, no amount of "you might also …" suggestions should try to make me send a second payment, I don’t have a need, neither the money. (May be donations implemented brilliantly could generate a good intent…and increase your retention?)
When someone calls 911, they don't need engagement features.
This distinction matters because it determines how we should think about metrics and optimization.
The Right Tool for the Job
For intent-flexible products, heavy optimization makes sense. If you're running an e-commerce site, you should absolutely be measuring and optimizing your conversion funnel. You should be testing different layouts, recommendations, and checkout flows. The metrics can drive the development because the user's intent is flexible enough to accommodate that.
But for intent-fixed products, this same approach becomes anywhere from useless to actively harmful. Imagine if 911 dispatch software started optimizing for call length or trying to increase call volume. Imagine if your banking app tried to maximise time spent in the app.
This might sound absurd, but I've seen similar things happen. I once consulted for a company that made software for medical professionals. They were proud of their increasing "engagement metrics." But when I talked to users(doctors), I discovered that higher engagement usually meant doctors were struggling to complete basic tasks.
First Principles
The way out of this trap is to go back to first principles. Before you look at any metrics, ask:
1. What is the fundamental job users need to do?
2. Is their intent flexible or fixed?
3. What constitutes success from their perspective?
4. What are the ethical implications of trying to influence their behavior?
This sounds obvious, but it's surprising how rarely teams actually do this. It's easier to look at dashboards and optimise what's measurable than to think deeply about what should be built in the first place. And what problem we actually solving?
The Story Matters
The best products I've seen weren't built by optimising metrics first. When you start something new → there is nothing to optimise at the beginning. And when you start building → it built by deeply understanding a problem and then using metrics to measure progress toward solving it.
Stripe didn't succeed by optimising acquiring flows (though they did that too). They succeeded because they deeply understood the frustration developers faced with existing payment systems and built something fundamentally better.
Beyond Metrics
This doesn't mean metrics aren't important - they absolutely are. But they should serve the story, not define it. They should measure progress toward a well-understood goal, not determine what the goal should be.
Think of it this way: metrics are like a compass, not a map. They can tell you if you're heading in the right direction, but they can't tell you where you should be going.
The best products emerge from a deep understanding of user needs, clear ethical principles, and careful thought about fundamental user intent. Get that right first, then let the metrics help you execute better.
The Future
As our ability to measure and optimise continues to advance, this distinction will become even more important. We'll need to be increasingly thoughtful about when to let metrics drive development and when to let them merely inform it.
The companies and teams that grasp this will build products that don't just have good metrics, but actually make users lives better. And isn't that what we're really trying to do?