Skip to content
crispforms

The question that loses you half your responses

Completion rates don't decay evenly. Almost always, one question is doing the damage — and it's rarely the one you'd guess.

· 5 min read · 1,186 words

Share

A form with a 40% completion rate doesn't lose 6% of people at each of ten questions. It loses a handful along the way and then falls off a cliff at one particular question. Find that question and you have found almost all of your missing responses.

This is worth saying out loud because most people respond to a poor completion rate by shortening the whole form. That works, in the narrow sense that removing the offending question removes the problem — but it usually also removes four questions that were doing no harm at all, and you end up with less data and no idea which cut helped.

The distribution matters more than the average. If you only ever look at the single completion-rate number, every fix looks equally plausible and you have no way to tell whether the one you made worked for the reason you think it did.

Why the damage concentrates

People abandon forms for two quite different reasons, and they behave differently.

The first is drift: they were never that invested, something else came up, the tab got closed. Drift is roughly uniform. It costs you a small, steady percentage at every step, it correlates with length, and shortening the form genuinely does reduce it.

The second is a wall. Someone is filling in your form, intending to finish, and then hits a question they can't or won't answer. They don't decide to abandon the form; they decide to deal with that question later, and later never arrives. A wall costs you a large percentage at exactly one step, and shortening the form elsewhere does nothing about it.

Almost every form with a disappointing completion rate has a wall in it. The two failure modes look identical in the summary number and completely different the moment you plot the steps.

Drift costs you a little everywhere. A wall costs you a lot in one place. They need opposite fixes.

What makes a question a wall

In rough order of how much damage they do in practice:

  • Asking for a phone number. People give an email address without thinking and a phone number only with a reason. A phone field with no explanation next to it is the most reliable way to lose a third of your respondents.
  • Asking for something they'd have to go and find. An order number, a policy number, an invoice date, an employee ID. The moment someone has to leave your form to answer a question, the form is competing with everything else in the tab they left for.
  • Asking something that feels intrusive with no stated reason. Salary, company size, budget, job title at a personal address. The question itself is often fine; asking it cold is what does the damage.
  • An open text box early on. It signals effort before they have invested anything. The same box at question eight, after seven cheap taps, gets answered far more often.
  • A required question they cannot answer honestly. If none of the options fit and they can't skip it, the only remaining choices are to lie or to leave. A good proportion choose to leave, and the ones who lie have poisoned your data anyway.
  • A file upload on mobile. It is a genuinely awkward interaction on a phone, and phones are most of your traffic. Sometimes unavoidable; never free.

Finding it

The number you want is per-question drop-off: of the people who reached this question, what proportion never reached the next one. Not completion rate, not time on page, not bounce.

It is the single most useful figure a form builder can show you, and a surprising number don't. If yours doesn't, you can approximate it by making questions optional one at a time and watching the total move, but that takes weeks and confounds itself. Ours shows it on the analytics tab, and it is free.

One caveat when you read it. The first question always shows the highest drop-off, because that is where everyone who never intended to answer leaves — people who clicked out of curiosity, bots that render the page, someone who realised it wasn't what they wanted. That number is mostly noise. Look at the second-highest, and at any question whose drop-off is more than about double its neighbours.

If you have time-per-question, use it as a cross-check. A question with high drop-off and low time on it was refused — they read it and left. High drop-off with high time is different: they tried, and something about answering defeated them. The first is a question to justify or remove; the second is usually a broken input, an unclear label, or a validation rule rejecting something reasonable.

Fixing it, in order of what usually works

Make it optional and see what happens. This is the cheapest experiment available and the results are consistently surprising: a large share of people answer anyway, and the ones who don't are precisely the ones who would otherwise have left. You trade complete data from 40% of respondents for slightly incomplete data from 70%, which is almost always the better trade.

If it has to stay required, explain it. One line under the question, saying why you are asking. "So we can call you back rather than emailing" recovers more responses than any amount of visual design. People are not unwilling to give you things; they are unwilling to give them to a form that won't say what for.

Move it later. A question asked after someone has answered six others is asked of someone who has already decided to finish. Sunk cost is not a fallacy when you are the one collecting the form.

Replace free text with choices where you honestly can. "What's your role?" as an open box is work; as six options and an "other" it is a tap. Do not do this where the free text was the point — an open box asking what went wrong is the most valuable question on most feedback forms.

Split the form. If four of your questions are walls and all four are necessary, the problem isn't any single question, it's that you are asking for a lot. Ask for the minimum, then ask the rest on a second screen after they have committed, or in a follow-up email.

What to expect

Moving a phone number from required to optional, on a form that had it at question two, is routinely worth ten to twenty percentage points of completion. Adding a one-line reason to an intrusive question is worth less but costs nothing. Reordering is usually worth a few points and occasionally a lot.

None of these are large pieces of work. That is rather the point: the reason completion rates stay bad is not that fixing them is hard, it is that the summary number doesn't tell you where to start, so people either guess or redesign the whole thing.

Look at the per-question numbers once. You will almost certainly find that one question is responsible for most of what you're losing, and you will probably be surprised by which one it is.

formsanalytics
Share

Build a form that people finish

Free, with the per-question drop-off analytics this piece keeps going on about.

Start building free