Strategy
Part of Customer research: how journey maps expose backstage constraints
Catching Customer Research Mistakes: Pretest and Correct
Customer research mistakes are cheapest to catch in a wording pretest and costliest after a report ships: a pretest routine, what it misses, and how to correct.
What to take away
- Pretest wording before fieldwork, and watch what people take each question to mean.
- A pretest finds interpretation faults only. It cannot fix a vague decision, a thin sample, or a purpose nobody agreed to.
- When a mistake touches participant rights or a major decision, pause, correct, and document the restart.
- A report needs an owner, a hypothesis, a guardrail, and a review date, or it changes nothing.
How one pretest round runs
Pretest the wording on a handful of people who were not involved in writing it. Customer research mistakes in question wording are cheap to catch before fieldwork and expensive once the transcripts already carry the fault.
The CDC's cognitive interviewing resource calls cognitive interviewing a popular method for evaluating survey questions, one that examines respondents' thought processes. That is narrower than a commercial concept test, and it suits wording faults.
- Give each person the question in the form fieldwork will use.
- Ask what they took it to mean, in their own words.
- Note every place where the meaning differs from what you intended.
- Change the wording, then repeat with different people.
- Test the full decision task separately, once the questions read the same to everyone.
Keep the notes from each round. They show which words people stumbled over and which changes fixed the reading, so the next study starts with better wording instead of repeating the same faults.
What a pretest misses
A pretest answers one question: do people understand this wording as intended. It says nothing about whether the study should exist, who was asked, or what happens to the answers afterward. Those faults need their own fix.
| Fault | Does a pretest catch it? | Where the fix lives |
|---|---|---|
| Wording read differently by different people | Yes | Change the wording and repeat |
| No decision behind the study | No | Name the options and the threshold |
| Sample that leaves out leavers and non-customers | No | Define the population first |
| Purpose the participant never agreed to | No | Consent and ethics review |
Each of these starts before the interview. Name the decision before you write a single question, and cut any question that cannot move the choice. Sampling and method traps belong to primary market research mistakes.
When a mistake touches people or a decision
Pause the fieldwork the moment a mistake touches participant rights, sensitive data, or a major decision. Keep going and you add more flawed records to a study you may have to withdraw.
Then preserve the records, correct or withdraw the finding, tell the owners, and document the restart. A correction reads better than a confident story built on an unsafe study. If the problem is the purpose participants were given, the consent and ethics guide explains the duties.
A follow-up contact is one common trigger. The ICO's direct marketing guidance says direct marketing covers promoting aims and ideals as well as selling products and services. It is a U.K. source, not a global research rulebook, but a promotional message to participants can cross that line. Get qualified local review first.
Getting the report acted on
A report is not an outcome. Assign an owner, a hypothesis, a change, a guardrail, and a review date to every finding that reaches a decision-maker.
The review date matters because it forces the comparison. Record why evidence was rejected, then set the later result beside the prediction and write down the gap, whether the report turned out right or wrong.
- Wording was pretested with people outside the writing team.
- Misread questions got new wording and a second test.
- The decision, its owner, and its threshold are written down.
- Any follow-up contact has its own purpose, team, and data store.
- The report names an owner, a hypothesis, a guardrail, and a review date.
Common questions
Is cognitive interviewing the same as a concept test?
No. It examines how respondents understand and answer a question. A concept test judges the idea itself, so run them as separate steps.
What should happen when a finding is withdrawn?
Tell the owners, correct or retire the finding, preserve the records, and document the restart so the next study starts from what changed.
Why record the gap at the review date?
The comparison of prediction and result is what the review date is for. It shows whether the report was right and what to fix next time.
