The Scottish Government

« Previous | Contents | Next »

Listen

Annex A: Statistical reliability

The respondents to the questionnaire are only a sample of the total 'population'. We cannot therefore be certain that the figures obtained are exactly those we would have if everybody had been interviewed (the 'true' values). However, we can predict the variation between the sample results and the 'true' values from a knowledge of the size of the samples, on which the results are based and the number of times that a particular answer is given.

The confidence with which we can make this prediction is usually chosen to be 95% - that is, the chances are 19 in 20 that the 'true' value will fall within a specified range. The table below illustrates the predicted ranges for different sample sizes and percentages results at the '95% confidence interval'; based on a random sample.

Table A.1: Predicted ranges for different sample sizes at the 95% confidence interval

*For example, on a question where 50% of the people in a sample of 200 respond with a particular answer, the chances are 95 in 100 that this result would not vary by more than two percentage points, plus or minus from a complete coverage of the entire population using the same procedures. However, while it is true to conclude that the "actual" result (95 times out of 100) lies anywhere between 48% and 52%, it is proportionately more likely to be closer to the centre of this band (i.e. at 50%).

Tolerances are also involved in the comparison of results from different parts of a sample. A difference, in other words, must be of at least a certain size to be considered statistically significant. The following table is a guide to the sampling tolerances applicable to comparisons.

Table A.2: Sampling tolerances

Table A.3: Demographic sub-group comparisons

« Previous | Contents | Next »