Range of Error

The most precise way to understand a group would be to conduct a census (interview everyone). Statistical science shows we can sample instead, and get answers very close to what we’d get if we did a census.  Range of error is the measure of how close. (We think it should be  "range of closeness"... not "error" as in mistakes).

Actually, an answer from a sample is the midpoint of a range. For example; if a survey had a +/-5% range, an answer of 50% would mean that the census’ answer would fall in the range from 45% to 55%.

 The larger the sample, the smaller the range. Note that while a larger sample’s answer is more precise, it isn’t “better” per se. If a larger range is tolerable for the decisions being made, then a smaller sample gives answers as accurate as a large one.

The chart below shows samples from 100 to 1,000 having precision that improves from about 10% to about 3%.

To Zoom, click on icon of magnifying glass 

FAQ:  What does the .95 Confidence Limit mean?

A: If a survey were conducted 100 times, 95 of the answers would fall within the range.

Technically, the Range of Error for a sample can vary in relation to the Confidence Limit chosen. By convention, market researchers use the .95 CL. So, the full expression includes CL (e.g. "The maximum Range of Error of a sample of 1,000 at the .95 CL is +/- 3.1%.)

>Download a PDF of this Witan Wisdom article

Witan Wisdom: 

Choosing Sample Size

Precision vs Budget:

Increasing sample size improves precision and increases cost.  Note that precision improves at a diminishing rate while there is a linear per-interview cost increase.

How important is precision?

Consider the scale of the investment at stake hinging on decisions from the survey results. What is the opportunity and the downside risk?

FYI: Many corporate surveys are based on samples of 400 (<5% error). Experts consider polls of 1,100 (<3% error) sufficient to represent national opinion.

Use  larger samples when:

-Small changes must be measured (e.g. political polls, awareness tracking).

-Sub-group differences must be analyzed (e.g. men vs women, customers vs noncustomers).

Oversamples of small percentage sub-groups may save budget.