Why CRO best practices are weak evidence

I worked on a professional training offer that moved to a higher price and more professional positioning. Fewer people bought afterwards, while revenue stayed broadly similar and the cohort appeared better aligned. The obvious CRO response could have been to lower the price and recover conversion. That would have treated buyer count as the objective before establishing that it was the result the business actually needed to improve.

This is where CRO best practices get overextended. A recommendation can be sensible, and the evidence behind it can be good, without showing that the same problem exists in this journey or that the same change would improve the result this business actually cares about.

Every recommendation carries assumptions

Take a familiar piece of advice such as “reduce friction”.

There is a sensible mechanism behind it. If customers want to continue but an unnecessary field, confusing instruction or awkward payment step gets in the way, removing that difficulty can help.

The advice becomes much weaker when “friction” starts to mean anything that makes continuing slightly harder.

A serious training programme may need prospective students to understand that they have to attend live sessions, practise with other people or meet an assessment standard. A service business may need enough information to decide whether an enquiry is suitable. A financial product may have verification requirements that cannot simply disappear.

Those steps can reduce progression at the point where they appear without being mistakes. Sometimes the requirement is there for a reason, such as making sure somebody understands what they are committing to before they buy.

So if a journey appears to have too much friction, I want to know what the supposedly unnecessary step is actually doing.

The visible feature is not the mechanism

The same problem appears with advice about page length.

A long page can perform badly because it is repetitive, confusing or full of information the buyer does not need. Shortening it may help.

Another long page may be answering questions that matter before somebody makes an expensive or consequential purchase. Removing those answers can make the page shorter while leaving the buyer with more uncertainty.

The useful question is not how many words are on the page. It is what the customer still needs to understand before making the decision.

Price works in much the same way. Lowering it may reduce resistance and increase purchases, but it can also change who buys, the economics of the offer and what level of delivery the business can support.

The same visible recommendation can therefore make sense in one journey and be poor advice in another because the underlying problem is different.

Evidence does not transfer automatically

Good conversion practices usually come from some combination of experiments, repeated observations and practitioner experience. That is useful. I do not want to approach every customer journey as if nobody has learned anything before.

If a checkout contains unnecessary complexity, I will pay attention to it. If the main action is difficult to find, that is worth investigating. If an expensive offer provides almost no credible proof, I would want to understand why.

Existing knowledge helps decide where to look. It does not establish that the thing I am looking at is causing the problem.

A result is more transferable when the customer is making a similar decision and the recommendation is solving the same kind of problem. It also matters whether the business cares about the outcome the intervention improved.

A shorter form may have converted better somewhere else. Before carrying that lesson across, I would want to know what was removed, who was completing the form and what happened after submission.

By the time the original result becomes “use fewer form fields”, much of the evidence that made it meaningful has disappeared.

Evidence from your own journey is more relevant, but not automatically more reliable

There is an opposite mistake here too.

Three customer interviews can mislead you. Support complaints can make a rare problem look common because unhappy customers are more likely to contact the business. A before-and-after comparison can look convincing while several other things changed at the same time.

Evidence from the actual journey has an important advantage: it is directly about the customers and decisions you are trying to understand. That does not automatically make it trustworthy.

A repeated support question may be strong evidence that some customers are confused, but weak evidence about how common the confusion is. Transaction data may show where people stop without telling you why. An external experiment may provide strong causal evidence while coming from a business whose customers are making a rather different decision.

I would rather look at both questions: how relevant is this evidence to the journey, and how much confidence should we place in what it appears to show?

Best practices are useful hypotheses

This is where I find CRO best practices most valuable. They point to mechanisms that are common enough to investigate.

Unnecessary effort is often worth checking. Weak proof can make a consequential purchase harder to trust. Those are good reasons to inspect a journey, but they are not enough to conclude that you have found its main problem.

Before changing something because it is considered best practice, I want to know what problem the advice solved elsewhere, why the same mechanism should apply here, and what evidence from this journey supports that diagnosis.

If I cannot answer those questions, the advice may still be useful as a place to look. It is not yet a reason to change the journey.