« Previous | Contents | Next »
Listen
APPENDIX G: GUIDE TO STATISTICAL RELIABILITY
The respondents in the omnibus survey are only a sample of the total 'population'. We cannot therefore be certain that the figures obtained are exactly those we would have if everybody had been interviewed (the 'true' values). However, we can predict the variation between the sample results and the 'true' values, from a knowledge of the size of the samples on which the results are based, and the number of times that a particular answer is given.
The confidence with which we can make this prediction is usually chosen to be 95% - that is, the chances are 19 in 20 that the 'true' value will fall within a specified range. The table below illustrates the predicted ranges for different sample sizes and percentages results at the '95% confidence interval'; based on a random sample.
Table G.1: Predicted ranges for different sample sizes at the 95% confidence interval
Size of sample on which survey result is based | Approximate sampling tolerances applicable to percentages at or near these levels |
|---|
10% or 90% | 30% or 70% | 50% |
|---|
± | ± | ± |
|---|
100 interviews | 5.9 | 9.0 | 9.8 |
|---|
200 interviews | 4.2 | 6.4 | 6.9 |
|---|
300 interviews | 3.4 | 5.2 | 5.7 |
|---|
500 interviews | 2.6 | 4.0 | 4.4* |
|---|
1,000 interviews | 1.9 | 2.8 | 3.1 |
|---|
1,077 interviews | 1.8 | 2.7 | 3.0 |
|---|
Source: Ipsos MORI
*For example, on a question where 50% of the people in a sample of 500 respond with a particular answer, the chances are 95 in 100 that this result would not vary by more than 4 percentage points, plus or minus from a complete coverage of the entire population using the same procedures. However, while it is true to conclude that the "actual" result (95 times out of 100) lies anywhere between 46% and 54%, it is proportionately more likely to be closer to the centre of this band ( i.e. at 50%).
Tolerances are also involved in the comparison of results from different parts of a sample. A difference, in other words, must be of at least a certain size to be considered statistically significant. The following table is a guide to the sampling tolerances applicable to comparisons.
Table G.2: Sampling tolerances
Size of samples compared | Differences required for significance at or near percentage levels |
|---|
10% or 90% | 30% or 70% | 50% |
|---|
± | ± | ± |
|---|
100 and 100 | 8.4 | 12.8 | 13.9 |
|---|
200 and 200 | 5.9 | 9.0 | 9.8 |
|---|
200 and 400 | 5.1 | 7.8 | 8.5 |
|---|
200 and 500 | 4.9 | 7.5 | 8.2 |
|---|
500 and 500 | 3.7 | 5.7 | 6.2 |
|---|
1,000 and 500 | 3.2 | 4.9 | 5.4 |
|---|
Source: Ipsos MORI
« Previous | Contents | Next »