Writing neutral questions — Examples

Example 1 — The question, the scale and the two biases

A live survey asks the question below. Show the leading words it carries, the neutral replacement, and the answer scale that covers the whole answer space.

Question: "How easy was it to find what you needed?"
Options:  easy / very easy / extremely easy

Solution

Leading words: "easy" — asks for a judgement and hints at the wanted one
Neutral:       "How long did it take you to find what you needed?"

Options: under 30 seconds / 30 seconds to 2 minutes /
         2 to 5 minutes / over 5 minutes / did not find it

Two biases sit in the original. “Easy” evaluates, so agreement becomes the easy answer, and the three positive options force a choice among flattering answers with nowhere to report a bad experience. The duration scale carries no judgement and can be checked against analytics. The usual mistake is assuming that polite wording makes a question neutral.

Example 2 — Splitting a double-barrelled question

The survey asks one question about two ideas and then reports a single average. Split it into two questions and give each half its own answer scale.

Question: "Was the checkout clear and quick?"
Options:  yes / no

Solution

"How clear were the steps?"       very / fairly / not very / not at all
"How long did the payment take?"  under 10 seconds / 10 to 30 seconds /
                                  30 seconds to 1 minute / over 1 minute

A single “yes” cannot distinguish a checkout that was clear but slow from one that was quick but baffling, so the average describes neither group. Two questions with separate scales report both facts and can each be acted on. The usual mistake is a forced choice between yes and no, which manufactures a result and leaves no answer for a respondent who did not get far enough to judge.