Rules
Part of Market research surveys beyond the obvious
Which questions market research surveys should ask
Common market research surveys questions for 2027 cover survey fit, sample size, response rate, anonymity, incentives, weighting, fraud review, and reporting.
What to take away
- Which questions a survey asks comes down to eight item families: screening, behavior, attitude, brand, purchase intent, open-ended, demographic and weighting, and quality check.
- Sample size follows a written precision target, not a universal number. A margin of plus or minus 5 points at 95 percent confidence needs about 385 completed responses before any design effect.
- Response rate is a diagnostic with a defined denominator, not a quality score.
- Anonymous and confidential are different promises. Name the one the data path can honor.
- Weighting adjusts measured gaps on known variables. It cannot repair missing groups, bad measurement or fraud.
Item families every questionnaire needs
Write the item list first. Every questionnaire that survives review fills eight families, and each family earns its place only when a decision needs it. For the wider map of modes, frames and fielding cost, read market research surveys.
| Item family | Job it does | Wording example |
|---|---|---|
| Screening | Sets eligibility and quota cells | 'Which of these have you bought in the past 30 days?' |
| Behavior | Records what happened, not what is felt | 'When did you last buy it?' |
| Attitude | Measures evaluation on a labelled scale | 'How easy was it to use?' |
| Brand | Captures unaided then aided recall | 'Which brands come to mind?' |
| Purchase intent | Rates stated likelihood over a fixed window | 'How likely are you to buy in the next 6 months?' |
| Open-ended | Collects the reason behind a choice | 'What would make you switch?' |
| Demographic and weighting | Supplies benchmark variables | 'What is your ZIP code?' |
| Quality check | Tests attention and consistency | 'Please select Somewhat agree to show you are reading.' |
Example: a core draft in field order
A category purchase study with a 400 response target could field this sequence.
A core draft in field order
- Screening'Are you 18 or older?' then 'Do you work in this category?'
- Behavior'When did you last buy it?' and 'How much did you pay?'
- Brandunaided recall first, then a shown list.
- Attitude'How easy was it to use?' on a five point labelled scale.
- Purchase intentstated likelihood over six months.
- Open-ended'What would make you switch?'
- Quality check and consentthe attention item, then 'May we quote your response with your first name and company?'
Anonymous means no response can reasonably be linked to a person. Unique links, account data, incentive records and open text usually create that link.
That consent item decides the label. When the operation can separate identity from answers but keep a link for follow-up, call the study confidential and describe the controls.
How large should the sample be
The Census Bureau's sample-design quality standard ties frame development and selection to the required detail and precision of key estimates. A business study needs the same written assumptions at a smaller scale.
Worked precision. For a proportion near 50 percent, the standard formula gives about 385 completed responses for a margin of plus or minus 5 points at 95 percent confidence. Assume a design effect of 1.4 and that target rises to about 540. At a 40 percent completion rate you would invite about 1,350 people.
Sample Size Assumptions
- Population and frame
- Selection and subgroups
- Precision and confidence
- Variability and design effect
- Expected response and attrition
- Weighting and multiplicity
Subgroups multiply the bill. Four segments reported at the same precision need roughly four times the completes, or a wider margin on each.
Report the achieved sample and its uncertainty. Those assumptions feed the benchmarks tracked in primary market research benchmarks, which compare planned coverage against what was fielded.
What counts as a good response rate
AAPOR publishes standardized response rate definitions with several formulas that differ in how partial interviews and unknown eligibility enter the denominator. Choose one, name it, and hold it steady across waves.
No figure proves accuracy. Panel completion rates vary widely; a commonly reported band runs from roughly 5 to 30 percent depending on recruitment, incentive and panel management. Treat that as a typical range, not a target.
Compare frame information on responders and nonresponders, then stress test the headline number. The sensitivity checks follow the routine set out in market research analysis, which shows how to report them without overclaiming.
Anonymity, consent and incentives
Anonymous and confidential are separate promises. Say anonymous only when no response can reasonably be linked to a person. Unique links, incentives, device data and open text can each break that.
Teams fielding in Canada work under PIPEDA, which sets consent, purpose, retention and access duties for the panel operator. The label on the survey has to match what the operator can hold.
Incentives change who takes part. A recruitment item such as 'Would you like to receive a $10 gift card?' shifts entry. Amount, eligibility, framing and payment rules all matter, so publish terms and avoid conditions that make refusal unrealistic.
Weighting and fraud checks
Weighting can close measured gaps on known variables when the benchmarks and model suit the estimate. It cannot repair unobserved coverage error, measurement error or fraud. Report weights, assumptions and variance effects.
Benchmark variables often exist already. Census profiles and earlier secondary market research can supply age, region and income bands without adding a single demographic item.
Use layered signals and predeclared rules for fraud. Response speed, straightlining, duplicate device data and implausible open text join the attention item. One failed check does not prove fraud, and rigid cutoffs can strip out older respondents, second language speakers and people using assistive devices.
