Experimentation & CRO
Qualitative research for CRO: find the real friction
Use qualitative research for CRO to identify friction and build stronger hypotheses. Plan interviews, task observations and a traceable evidence record.
Analytics shows a fall between two steps. It does not tell you whether people misunderstood a term, lacked information, encountered a technical barrier or simply decided the offer was unsuitable. Qualitative research for CRO helps you investigate those explanations before committing traffic to a variant.
The result should be a clearer problem, supported observations and a testable change. It should not be a pile of quotations used to decorate an idea the team had already chosen. This guide covers research questions, recruitment, sessions and the path from evidence to experimentation. The CRO and A/B testing guide provides the wider programme.
Turn the business concern into a research question
“We need more leads” is a business objective. “Can likely buyers understand what happens after they send this enquiry?” is a research question. The second gives you an activity to observe and a boundary for the findings.
Write the assumptions behind the proposed intervention. If you want to remove a field, you may assume that people find it unnecessary, do not know the answer or distrust the reason for asking. These explanations imply different changes. Research should distinguish them rather than confirm that people prefer a shorter form.
Choose the method for the question. Interviews can reveal circumstances, past decisions and vocabulary. Task observation can show how people interact with an existing journey or prototype. Support logs can identify recurring expressed problems, with the limitations of who contacts support.
Use analytics to locate a question, not to prewrite its answer. An exit is a behaviour. “They leave because the price is too high” is an explanation requiring evidence. Keep alternatives in the research plan so the session can reveal something unexpected.
Recruit people relevant to the task
Define the characteristics that matter for the journey: familiarity, decision responsibility, task experience, device use and relevant constraints. Avoid demographic profiles that sound precise but do not explain why a person can answer the research question.
Include people who have completed, abandoned or avoided the task where they are relevant and reachable. Researching only satisfied customers can hide the conditions that prevent a decision. Researching only support users can overrepresent difficult experiences. Describe the recruitment route and its likely limitations.
Accessibility belongs in the plan. W3C's guidance on involving users in evaluation explains why user involvement complements technical assessment. Include relevant access needs without treating one participant as representative of every disabled user or as proof of conformance.
Do not claim a universal participant count. The useful scope depends on the question, variation in the audience and findings. Plan iterative rounds, examine what remains unclear and recruit accordingly. A small qualitative sample can reveal a problem without estimating its population prevalence.
Write tasks that do not teach the answer
A task should state the person's goal. “Find the enterprise plan and click the demo button” tells them the interface path. “Decide whether this service could support your team's situation and explain your next step” leaves room to observe whether the page makes that path understandable.
Avoid forcing a purchase decision that the participant has no reason to make. Use plausible context, with enough detail to create the task without preloading the result. If the situation is hypothetical, keep that limitation in the interpretation.
The GOV.UK guidance on moderated usability testing describes observing participants completing tasks. In a CRO study, pay attention to hesitation, repeated searches, misinterpretation, workarounds and the information people seek before proceeding.
For interviews, ask about a specific recent episode rather than a general ideal. What happened? What did they compare? What information was missing? What did they do next? A prediction that someone “would buy” is different from an account of what they actually did.
Protect consent, context and participant dignity
Explain the purpose, recording, observers and use of findings in a way participants can understand. Follow the consent and data-handling requirements appropriate to the study. Only collect information you need, and provide a route to withdraw where the process allows it.
Keep observers from interrupting or steering the participant. Agree how questions are passed to the moderator. A sales stakeholder defending the offer in the middle of a session changes the evidence and can make the person reluctant to disagree.
Use a test environment where tasks could create real orders, send messages or expose private information. If live data is necessary, establish the authorised process beforehand. Do not let convenience decide whether a participant's information appears in a shared recording.
Watch for conditions that affect the session: poor connection, unfamiliar hardware, fatigue, artificial time pressure or a prototype that behaves differently from production. Record them. They can explain an observation without erasing the need to improve the interface.
Separate observation, interpretation and action
An observation records what happened. An interpretation proposes why. An action proposes what to do. Keeping them separate prevents a confident explanation from being mistaken for raw evidence.
GOV.UK's guide to analysing a research session distinguishes observations, findings and actions. The worksheet below adapts that discipline for experimentation. It is an editorial evidence record, rather than a claim to discover a psychological cause from one session.
Swipe or use the arrow keys to see every column.
| Layer | Example in an enquiry journey | What to verify |
|---|---|---|
| Observation | Participant searches for what happens after submission | Task context and actual behaviour |
| Context | Participant must obtain internal approval first | Whether that context appears elsewhere |
| Interpretation | Uncertainty about next steps may block action | Alternative explanations |
| Proposed change | Explain follow-up and information needed | Whether it addresses the observed problem |
| Expected behaviour | More eligible people complete a suitable enquiry | Outcome and audience definition |
| Guardrail | Enquiry qualification should not deteriorate | Agreed sales-review window |
| Next evidence | Prototype observation, then feasible A/B test | Method matched to the question |
Preserve contradictory observations. Some people may understand the process already, or seek a different detail. A finding should describe its scope instead of smoothing every session into one persona. Contradictions often identify audiences that need different explanations.
Hypothetical example: the form is blamed too early
Consider a hypothetical B2B services page with an enquiry form. The team proposes removing fields because submissions are low. This scenario is illustrative and does not report client research or conversion results.
In task observation, likely buyers hesitate before reaching the form. They cannot tell whether the service accepts projects at their stage. In an interview about a recent purchase, a participant describes needing an internal brief before contacting a supplier. The form length is visible, but it is not the only plausible barrier.
The team separates the findings. One concerns fit information; another concerns preparation; a third concerns whether particular fields can be answered. It creates a prototype that explains fit and follow-up, then checks whether people can make a suitable next-step decision.
A later experiment can test a defined page change if the audience and sample permit it. The research does not establish that the revised page will increase conversion, nor that every field should remain. It gives the team a more specific intervention and a reason to measure enquiry quality.
The psychology-to-test guide helps frame uncertainty as a mechanism to investigate, without treating an interpretation as a demonstrated cause.
Convert findings into hypotheses with a decision attached
For each promising finding, describe the observation, proposed intervention, expected behaviour and metric. Add the decision that follows from a positive, negative or uncertain result. If the team will ship regardless, the task is not an experiment designed to inform that choice.
Use the A/B test hypothesis guide to make the prediction falsifiable. Explain which audience is eligible and what they should experience differently. “Improve clarity” is a design intention until you define how the task changes.
Move entries through the experiment backlog with evidence links attached. Avoid copying a quotation into a card and losing the context that gave it meaning. The team should be able to inspect the observation behind the proposal.
Check whether the result needs guardrail metrics. More submissions can be a poor outcome if sales receives unsuitable enquiries. Qualitative research can identify that consequence before the test, but its frequency still needs appropriate measurement.
Before closing the study, review a case that does not fit the strongest finding. Ask whether the participant had different information, used another route or understood a term that others missed. A counterexample can narrow the finding and reveal the condition under which the proposed change is useful.
Record what would make the team revise its recommendation. For example, if a clearer label fixes understanding but people still cannot complete the task, the next investigation concerns the process or available option. This keeps research tied to a decision rather than defending the first explanation.
Measure the usefulness of the research programme
Research should change a decision, sharpen a question, identify a defect or remove a weak idea. Track those outcomes rather than rewarding the number of sessions, quotations or presentation pages.
Keep a record of which hypotheses came from findings and how later evidence affected them. A test that does not support the proposed explanation is useful feedback. Return to the observation and ask whether the intervention, audience or interpretation needs revision.
Do not turn a session count into a causal claim. Nor should a favourable A/B result prove every psychological explanation attached to it. The experiment estimates the effect of an intervention under its design; several mechanisms may remain plausible.
Where traffic is limited, use research for the decisions it can support and report uncertainty about population effects. Start with one important journey question, recruit relevant people and preserve the evidence from observation to action. To make that cycle part of your programme, tell me about the customer journey.
Apply this to your project
Tell me the decision you are facing, the constraints and what you have already tried.
Keep exploring
A/B test guardrail metrics: protect the whole journey
Choose A/B test guardrail metrics that protect quality, performance and downstream outcomes. Define thresholds and decisions before launching a test.
Experiment backlog prioritisation: choose better tests
Prioritise your experiment backlog using evidence, traffic and decision value. Build a practical queue that turns observations into useful A/B tests.
Low-traffic A/B testing: choose a question you can answer
Plan low-traffic A/B testing around useful effect sizes, eligible audiences and uncertainty. Learn when to test, research, simplify or defer.