TLDR: Changing everything at once feels decisive. It is the least decisive move a business can make, because when the numbers move you cannot tell which change did it. I watched a replatform drop conversion by 20 percent, and the culprit was not the platform, rather a UX change everyone was certain was an improvement. It took rebuilding that one piece and testing it to prove it. The cost of the big-bang approach is not only the drop, it is how long and how expensive it is to find.
Click to discuss this article with your favourite AI
Every so often a business decides it is time for a refresh. New look, new navigation, new copy, new checkout. Months of work, a real budget, and then one big launch where all of it goes live together. It feels like progress. It feels bold.
It is also the most expensive way to learn nothing.
The least decisive thing you can do
When you ship fifty changes in one release, you have made fifty bets and fused them into a single result. If the numbers go up, you do not know which change earned it. If they go down, you do not know which change cost you. The good decisions and the bad ones are welded together, indistinguishable. A move that feels like conviction is the exact moment you give up the ability to tell whether you were right.
I watched this happen, and the culprit was the obvious improvement
Years ago I joined an experience-gifting business just as it was finishing a full replatform onto Salesforce. As part of it the team had redesigned more or less everything, including the checkout, which went from two pages to a single page. Checkout conversion rate had dropped by around 20 percent, and the business could not work out why. That is the signature of a big-bang change. So much had moved at once that nobody could point at the cause.
On paper the choice to move to a single page checkout was not even a suspect. A single-page checkout is the thing everyone knows is better, fewer steps, less friction, less to abandon. But when we enabled proper conversion tracking and worked through the funnel, the evidence kept leading back to it. My hypothesis, which nobody in the building liked, was that the old two-page checkout had actually been the better design.
They let me test it, and it was not a small test. We had to rebuild the previous two-page approach and feature flag the two versions against each other. The first iteration closed almost the entire gap, from a 20 percent loss down to 2 percent. A second iteration, with clearer signposting than the original, beat the old checkout outright. The change everyone was sure was an improvement had been quietly costing the checkout a fifth of its conversions. The only reason we could ever prove it was that we pulled that one decision out of the pile and tested it on its own.
It is not just me, and it is not just small companies
If that sounds like one business getting unlucky, consider Basecamp, a software company about as proud of building on taste and conviction as any in the industry. In early 2014 they rebranded and redesigned their marketing site in one move, and quietly removed the signup form from the homepage. They did not test it. Conversion fell, but it hid inside the noise of a rebrand, new traffic, and normal seasonality. It took them more than a year to conclude the decline was real and to run a test putting the form back. Signups rose 16 percent, worth millions at their scale. Their own words: with everything changed at once, “it’s really hard to identify which of the many things that contribute to the ultimate number of signups had what impact.”
The point is not that the cause is unknowable
Here is the honest version, because it matters. In both stories the cause was found. A rebuilt checkout and a feature flag found mine. One test found theirs. Neither answer was unknowable. What the all-at-once launch did was make the answer expensive. We spent months proving something a clean test at the time would have shown in weeks. Basecamp spent eighteen months and millions before they went looking. The big-bang release does not hide the answer forever. It just puts a very large price on the question.
You cannot remove a mistake you cannot locate
That is the trap underneath it. When a bad decision is welded into a release with forty good ones, you cannot pull it out, because you cannot see which one it is. Your options are to keep the whole thing, damage included, or bin the whole thing, wins included. Neither is a real choice. This is what it looks like to prioritise the output over the outcome. You shipped the redesign, which was the visible goal, and in the same move you gave up any way of knowing which parts of it were worth shipping.
You do not have to choose between a bold redesign and knowing what it did
None of this is an argument against redesigning. Redesign whenever you need to. It is an argument against shipping it as one indivisible lump.
Some of a rebrand genuinely cannot be split. You cannot show half your customers the old logo and half the new one, and you should not try. But most of what goes into a redesign is not the brand moment. It is a stack of smaller, reversible decisions: the checkout, the signup, the pricing, the navigation. Those can be sequenced and instrumented so each one is separable, and where you have the traffic, held against a control. That business got its conversions back by doing exactly that, a year later than it needed to. Doing it before you ship is the same work, minus the year of losses.
And if your traffic is too low to hold a control at all, the more honest question is not how to test the redesign but why you are pouring budget into one for an audience that small. I will come back to that in a future post.
What these stories have that most do not
Both of those businesses were lucky in one specific way. The number visibly dropped, which is what sent someone looking, and the culprit was eventually caught. The more common version, and the more expensive one, is where nothing looks broken at all. The decline is slow, there is no launch to blame and no single event to point at. You can see the wound. You cannot see the knife. That is where this goes next.
Want to prevent a costly mistake?
If you are about to spend real money on a redesign, the cheapest thing you can do is structure it so you can measure what each change did, so that if it underperforms you know why in a fortnight, not a year. I show business how to do that.