Most survey design advice is about wording. Question type matters at least as much, because the wrong type makes a question impossible to answer honestly and the data looks fine either way.
Here is each type, what it is for, and the specific way it goes wrong.
Choosing between options
The workhorses. Easy to answer, easy to count, and the place where most survey bias is accidentally introduced.
- Single choice. One answer from a list. Fails when the list is not exhaustive, so always include an escape option unless you are certain.
- Multiple choice. Any number of answers. Fails when people tick everything mildly true, which is why "select up to three" produces better data than "select all that apply".
- Dropdown. A long list in little space. Fails because the options are hidden, so it suppresses answers people would have picked if they had seen them. Fine for country, bad for anything you are measuring.
- Yes or no. Fast and unambiguous. Fails when the honest answer is "it depends", which is more often than designers assume.
- Image choice. Picking between visuals. Right for design and brand questions, misleading for anything where the image quality varies between options.
Measuring intensity
Scales are how you get a number out of a feeling. They are also where comparability lives, which is why changing one between waves is more costly than it looks.
- Rating, 1 to 5. The default. Fast, easy to interpret, enough granularity for most purposes.
- Rating, 1 to 10. More granularity, more noise. Worth it when you need to detect small movements in a large sample, not otherwise.
- Net Promoter, 0 to 10. A specific scale with a specific calculation. Do not modify it, or you lose the only thing it has: comparability with everyone else using it.
- Agreement scale. Strongly agree to strongly disagree. Fails through acquiescence bias: people agree with statements by default, so mix positive and negative wording.
- Slider. Looks modern, answers worse. People park it near the middle and it is harder to use on a phone than tapping a number.
- Star rating. Familiar and heavily skewed. Everybody knows four stars means disappointed, which is useful only if you want consumer-review conventions.
Comparing and ordering
These extract priorities rather than opinions, which is more useful and more expensive to answer.
- Ranking. Put these in order. Excellent for forcing a priority out of people who would rate everything important. Fails past about six items, where it becomes genuinely hard work.
- Matrix. Several things rated on the same scale, in a grid. Efficient on a desktop and close to unusable on a phone, which is where most people will meet it. Keep it to four rows.
- Constant sum. Allocate 100 points across options. The most honest way to measure priority and the most effortful to answer. Use once per survey at most.
Open text
Where everything you did not think to ask about lives. Also where response rate goes to die if you use more than two.
- Short text. One line. For names, references, specifics. Fails when used for something that needed a paragraph.
- Long text. A box for an explanation. The single most valuable question type and the most expensive to answer. Two per survey, and make the second one optional.
- Why, after a rating. The follow-up that makes a score actionable. Worth more than the score itself.
Facts
Dates, numbers, uploads. The failure mode is the same for all of them: validation that rejects something legitimate.
- Number. For quantities and amounts. Set a sensible range and allow decimals where they make sense.
- Date. Use a picker rather than a text field, and be clear whether you want a date in the past or the future.
- File upload. For evidence, documents and photographs. Always optional unless the form cannot proceed without it, because a required upload loses everybody on a phone with a bad connection.
A matrix is efficient on a desktop and close to unusable on a phone, which is where most people will meet it.
Four rules that apply regardless of type
These matter more than the choice between any two question types above.
- Never put the answer in the question. "How useful was our improved checkout?" assumes two things and gets agreement to both.
- Give people an exit. "Prefer not to say", "not applicable" and "other" prevent forced-choice answers that are simply false.
- Keep the wording identical between waves. A rewritten rating question makes this quarter incomparable with last quarter, and the trend was the point.
- Ask about behaviour over intention. What somebody did last month predicts. What they say they would do flatters.
How many questions, and in what order
Type choice matters less than length, and length matters less than order.
Put cheap questions first. A choice from four options costs almost nothing to answer; a paragraph about what went wrong costs real effort. Starting with the expensive one means most people never reach the cheap ones.
Put demographics last, always. They are the least interesting questions to answer and the most likely to make somebody wonder what you are doing with this, which is a bad thought to have at question two.
And put the one question you would keep if you could keep only one at around a third of the way in. Early enough that most people reach it, late enough that they have committed to finishing.
Working on a phone
Most surveys are now answered on a phone, and several question types that are fine on a desktop are close to unusable there.
- Matrices need horizontal space they do not have. Four rows maximum, or split them into separate questions.
- Sliders are fiddly with a thumb and people park them in the middle.
- Long dropdowns mean scrolling inside a scroll. Use radio buttons under about eight options.
- Ranking by dragging is hard on a small screen. Keep it to five items.
- Long text is fine, but say roughly how much you want. "A sentence or two" gets answers; an empty box gets skipped.