Bad instruments waste money quietly. Enumerators work hard, households give time, and the dataset still cannot answer the research question. These are the failure modes I look for when reviewing questionnaires before a pilot — and again after the first HFC cycle.
Ten design failures (and what to do)
- Double-barreled questions — Ask one thing per item.
- Leading wording — If the “right” answer is obvious, you are measuring social desirability.
- Unrealistic recall periods — “How much did you spend on health last year?” is often fiction; diaries and shorter windows exist for a reason.
- Broken skip logic — Test every path on a device, not only on paper.
- Overlapping numeric ranges — 0–5 and 5–10 both claim 5.
- Missing refuse / don’t know — Forced answers create noise that looks like data.
- Internal jargon — Acronyms that make sense in Nairobi HQ may not in a rural interview.
- Too many open text fields — Open text is expensive to code and easy to abandon.
- No unique IDs — Household, visit, and person IDs must be stable before the first interview.
- Translation as an afterthought — Back-translation and field pilot in the interview language are part of design, not polish.
The pilot is not optional
A pilot with a few real respondents and the actual enumerator team catches more than a committee meeting. I want:
- Timing per module
- Places where people stall or refuse
- Constraint failures that fire too often
- Questions that produce nonsense when taken literally
Then the form version is locked for the wave — or versioned explicitly if it must change.
After the form ships
Design does not end at deploy. HFC should watch for:
- Modules with sudden missingness spikes
- Enumerators with outlier interview lengths
- Options never selected (dead branches)
- Duplicate IDs
That is how survey design and research data systems stay connected.