How many survey responses do you need? Everything that decides the number

How many survey responses do you need? Everything that decides the number

A researcher plans a staff survey for a 1,000-person organisation and asks the only question that really matters before printing: how many completed forms do we need? Someone in the room says 100. Someone else says 10%, which is also 100. A third person, who has been burned before, says "as many as we can get." Nobody says 278, and 278 is the answer.

The gap between those guesses is not a rounding error. At 100 responses the results carry a margin of error of about ±9.3%, which means a reported 45% could really be anything from 36% to 54%. That is wide enough to make two different decisions look equally justified. At 278 the same survey reports ±5.0%, and the picture stops moving under your feet.

Sample size has a reputation for being fussy statistics. It is not. It is one formula with a handful of inputs, and once you know what each input does to the number, you can plan a survey in a couple of minutes. This guide walks through every factor that genuinely moves the figure, the ones that do not move it as much as people think, and the fastest way to land on a number you can defend. If you would rather skip straight to the arithmetic, the free Sample Size Calculator will do it now.

Start from the decision, not from a round number

Before any formula, answer a different question: what will you do differently depending on the result?

If leadership will act when "considering leaving" moves by 10 points, a margin of error of ±5% is comfortably good enough. If you need to detect a 3-point shift between this year and last, ±5% is useless, because the noise is larger than the signal you are hunting. Precision is not a badge of rigour to be maximised. It is a budget item, and you should buy exactly as much as your decision requires.

This is also why "10% of the population" is such a persistent bad rule. It scales the wrong thing. Ten percent of 500 people is 50 responses and ±13.2% precision. Ten percent of 100,000 people is 10,000 responses and ±0.9%, which is far more than almost any decision needs and a great deal of money spent on nothing.

The five things that actually move the number

Precision, and it dominates everything. The margin of error you can tolerate is the single biggest lever, and it is brutally nonlinear. Because precision improves with the square root of the sample, halving your margin of error costs four times the responses. At 95% confidence on a large population: ±10% needs 97 responses, ±5% needs 385, ±3% needs 1,068, and ±1% needs 9,604. Anyone asking for ±1% "to be safe" is asking for twenty-five times the fieldwork of ±5%.

Confidence level, which matters less than people expect. Confidence is how often a repeated survey would land inside that margin. Moving from 95% to 99% sounds like a large upgrade in rigour, but it only takes you from 385 responses to 664. Moving down to 90% saves you a little, at 271. Ninety-five percent is the convention in survey research for a reason, and there is rarely a good argument for leaving it.

Population size, but only when the population is small. This is the input people most expect to matter and it mostly does not. The finite population correction shrinks the required sample when you are surveying a meaningful slice of a closed group. For a ±5% target at 95% confidence: a population of 1,000 needs 278, of 2,000 needs 323, of 10,000 needs 370, and of 100,000 needs 383. Above roughly 20,000 people the number stops moving, because precision depends on how many people answered, not on how many could have. Below about 5,000 it matters a great deal, and a survey of a single school, clinic or company should always enter it. The Finite Population Correction Calculator shows the effect directly.

How the answers split. The formula includes the expected proportion, and it needs the largest sample when opinion divides 50/50. If you already know from a previous wave that roughly 20% pick a given option, the required sample drops from 385 to 246. This is a real saving, but treat it carefully: it only applies to the question you based the estimate on, and most surveys have many questions. Using 50% is the honest default, and it is what every calculator here assumes unless you say otherwise.

Subgroups, which is where most survey plans quietly fail. This is the factor that costs people their analysis, and it is almost never in the plan. Your sample size is calculated for the whole sample. The moment you say "and we will compare night shift to day shift," or "we will break this down by department," the arithmetic applies again inside each group. Three hundred responses split across five departments is sixty per department, and sixty responses carry a margin of error of about ±12.7%. Every subgroup comparison you intend to make will look inconclusive, not because there is no difference but because you did not buy enough precision to see it.

The fix is to plan from the smallest group you intend to report on. Decide which breakdowns matter before fielding, size each of those groups to the precision you need, and add them up. It is common for this to double or triple a naive whole-sample target, and it is far cheaper to discover that before printing than after.

Clustering: when your responses are not independent

If you sample whole classrooms, clinics, branches or villages rather than individuals scattered at random, the responses inside each cluster resemble each other more than they resemble the wider population. Students in one class share a teacher; patients at one clinic share a waiting room. Statistically, those responses carry less independent information than the raw count suggests.

The correction is the design effect, and it inflates the sample you need. With a modest intracluster correlation of 0.05 and clusters of 30 people, the design effect is about 2.45, so a target of 385 becomes about 944 real responses to carry the same precision. This is not an exotic edge case; it applies to most education, health and multi-site field research. The Design Effect Calculator turns the two inputs into the multiplier, and multiplying it by your baseline target is the whole method.

Completions are not print runs

Every number so far is completed responses back in your hands. It is not how many forms to print or how many people to invite, and confusing the two is the most common practical mistake in survey planning.

Divide the target by the response rate you expect. If you need 278 completions and a previous wave of the same survey achieved about 30%, you need to reach roughly 930 people. With a workforce of 1,000, the practical answer is to invite everyone. If the last wave achieved 60%, printing 500 forms is enough.

Two notes on that estimate:

  • Use your own history, not a benchmark. Response rates vary enormously by channel and audience. Paper forms handed out in person routinely reach 70% or more, while cold email surveys often fall below 15%. Your own previous wave with the same population is worth more than any published average.
  • Round up, always. Spoiled forms, undelivered mail and people who start and stop are a normal part of fieldwork, not a failure of planning.

Once responses come in, the Response Rate Calculator and the Completion Rate Calculator turn the raw counts into the percentages you will report.

When your headline number is an average

Everything above sizes a survey for a percentage: the share who agree, the share considering leaving. If your headline metric is an average instead, such as mean hours of sleep or mean spend, the formula changes and needs a different input, the expected standard deviation of the answers.

You will not know that before fielding, and the honest way to get it is a small pilot. Run twenty or thirty forms, read the standard deviation off the results, and feed it into the Sample Size Calculator for a Mean. The relationship is worth internalising: noisy questions need bigger samples for the same precision. A question with a standard deviation of 1.09 needs about 203 responses to pin the mean within ±0.15, but only about 74 to pin it within ±0.25.

How to get to the number in about a minute

The arithmetic above is not hard, but doing it in a spreadsheet for every survey, then tracking whether you have hit it, is exactly the kind of task that quietly stops happening.

In PaperSurvey.io this lives in the survey itself. Open a survey, go to Settings, and fill in the Response plan: your population size, the confidence level, and the target margin of error you can live with. The plan returns the number of completed responses you need, and if you add the response rate you expect, the print run likely to produce them. For our 1,000-person organisation at 95% and ±5%, that is 278 responses and about 930 copies.

From there it stops being a planning document and becomes a live measurement:

  • The print modal suggests the run. When you generate copies, the count is already filled in with the number your plan calls for, and once fieldwork is under way it suggests only what is still outstanding rather than the whole run again.
  • A progress widget tracks the target. On the Analysis view, a Response plan widget shows responses collected against the target and, underneath, the margin of error those responses actually deliver right now. At 142 of 278 it reads ±7.6%. At 300 it reads ±4.7%, inside the ±5% you asked for.
  • The data-quality banner warns you while you are short. Instead of a vague "collect more responses," it names the gap: 142 of the 278 your ±5% target needs.

The formulas behind all of it are Cochran's with the finite population correction, the same ones the free calculators run, and they are verified against SciPy on every change. You get the same 278 either way. The difference is that the target stays attached to the survey instead of living in a spreadsheet someone closed. The help centre walks through the whole flow in How Many Survey Responses Do You Need.

What no calculator can tell you

A sample size formula answers one question: how much random sampling noise will your results carry. It says nothing at all about whether the people who answered resemble the people who did not.

That is nonresponse bias, and it is not fixed by collecting more responses from the same easy-to-reach group. If your staff survey reaches 90% of office workers and 20% of night shift, doubling the sample mostly doubles the office workers and leaves the picture just as skewed, now with tighter confidence intervals around the wrong number. A smaller sample that covers everyone is worth more than a large one that quietly excludes a group, and this is a large part of why paper still works where digital does not: a form handed to someone in person reaches people that an emailed link never will.

So treat the number as a floor for precision, and treat coverage as a separate problem with separate tactics. Both have to be right.

The Bottom Line

Sample size is not guesswork and it is not a percentage of your population. Decide what difference you need to detect, pick a margin of error that supports that decision, enter your population if the group is closed and reasonably small, and then, before you commit, check the smallest subgroup you intend to report on and size for that instead. Divide by the response rate you expect to get a print run, round up, and remember that no amount of extra responses fixes a group you never reached. For most single-site surveys the honest answer lands between 278 and 385 completions, not the 100 that gets suggested in the room. Start your free trial, set a response plan on your next survey, and let the target track itself while you collect.

References

  • Bartlett, J. E., Kotrlik, J. W., & Higgins, C. C. (2001). Organizational research: Determining appropriate sample size in survey research. Information Technology, Learning, and Performance Journal, 19(1), 43-50.
  • Cochran, W. G. (1977). Sampling Techniques (3rd ed.). Wiley.
  • Dillman, D. A., Smyth, J. D., & Christian, L. M. (2014). Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method (4th ed.). Wiley.
  • Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646-675.
  • Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: A meta-analysis. Public Opinion Quarterly, 72(2), 167-189.
  • Kish, L. (1965). Survey Sampling. Wiley.
  • Killip, S., Mahfoud, Z., & Pearce, K. (2004). What is an intracluster correlation coefficient? Crucial concepts for primary care researchers. Annals of Family Medicine, 2(3), 204-208.
  • Krejcie, R. V., & Morgan, D. W. (1970). Determining sample size for research activities. Educational and Psychological Measurement, 30(3), 607-610.

Get Started with PaperSurvey.io

Start your 14-day free trial now, no credit card required.

Get Started