Why CRO best practices are weak evidence

Updated Steve New

In professional training, I have sometimes made the difficult parts of an offer more explicit even when I expected fewer people to buy. We have brought live-attendance requirements, workload, assessment standards and the culture of the programme forward rather than let those realities emerge after payment.

That matters because CRO advice usually arrives attached to a visible step: reduce friction, shorten the page, simplify the choice, strengthen the call to action. Those can all be sensible moves. The harder question is whether the mechanism they improved elsewhere is the one causing trouble here, and whether a better local conversion rate would improve the result this business actually cares about.

The local conversion event is not the whole system

I have seen easier purchasing or more persuasive messaging increase sales while the resulting customers were a poorer fit, less likely to participate properly or more demanding to support. We have also kept changes that reduced headline conversion because the people who did buy were better aligned with what the programme required.

These were not clean experiments, and several things often changed between launches. The useful lesson is narrower: buyer count does not tell you whether a conversion change improved the business when the change can alter who buys and what they expect.

That becomes a transfer problem as soon as evidence comes from somewhere else. If a tactic was proven to increase completed checkouts in another business, I still need to know whether completed checkout is the result I am trying to optimise. A professional programme may care more about suitable students who understand the commitment and go on to participate successfully. The local metric can be the same while the commercial objective is not.

The visible feature is not the mechanism

“Reduce friction” is a good example because sometimes the advice is exactly right. An unnecessary field, a confusing instruction or an awkward payment step can block somebody who already wants to continue. Removing it may simply remove waste.

But not every difficult step is accidental. An application, eligibility check or live-attendance requirement can help a buyer understand what they are committing to or help the business prevent an obvious mismatch before payment. In those cases the effort has another function. Removing it may improve progression through that step while also removing something useful from the decision.

Page length works the same way. A long page can be repetitive and confusing, or it can be long because a consequential purchase genuinely requires more explanation. We have made sales material more demanding to read when serious buyers needed more information before deciding.

Choice simplification can also help or harm depending on what the choices are doing. I have seen cases where different buyers genuinely needed different routes, so removing those distinctions made the page easier to scan and the decision worse to make.

We have also made marketing less persuasive when accurately representing difficulty, workload or culture mattered more than maximising immediate desire. Urgency and stronger calls to action can be useful, but a tactic that increases response by creating the wrong expectation has not necessarily improved the journey.

Page length, number of fields, number of choices and strength of the CTA are visible features. The evidence transfers more usefully when I understand the mechanism the change affected.

Evidence travels badly when the context changes

I have seen standard CRO tactics work well for one offer or audience and be inappropriate for another. Advice that makes sense for a low-ticket ecommerce purchase may transfer poorly to professional training involving live participation, assessment, reputation or a longer working relationship.

A well-run external experiment may give strong causal evidence that a change produced an uplift in its original setting while still saying little about whether the same result matters here.

Local evidence has the opposite trade-off. Support conversations, transactions, interviews and previous launches are directly about this journey, but they are often noisier and less controlled. I do not need to choose one category of evidence as inherently superior. I need enough confidence that the same mechanism is operating here and that the outcome being improved is one I actually want.

This is also why a tactic can transfer cleanly. We have removed genuinely pointless friction and seen the expected improvement. When the mechanism matches, there is no benefit in becoming contrarian for sport.

Best practices are useful priors

I use CRO best practices as evidence about where problems commonly occur. Unnecessary effort, weak proof, confusing choice and unclear calls to action are all sensible places to inspect because they frequently matter.

Once I am looking at the actual journey, though, I want to know what problem the advice solved elsewhere, what the comparable step is doing here and what would happen if the local metric improved. Sometimes that leads straight to the conventional fix. Sometimes the step that looks inefficient is carrying information, selection or expectation-setting that the original recommendation did not need to account for.

So I use best practices to decide where to look first, then check whether the mechanism they assume is actually present in the journey.