« Previous | Contents | Next »
Listen
Statistical Reliability
The respondents are samples of the total "population" so we cannot be certain that the figures obtained are exactly those we would have obtained if everybody had been interviewed (the "true" values). As a random sampling approach was taken, we can, however, predict the variation between the sample results and the "true" values from knowledge of the size of the samples on which the results are based and the number of times that a particular answer is given. The confidence with which we can make this prediction is usually chosen to be 95% - that is, the chances are 95 in 100 that the "true" value will fall within a specified range. The table below illustrates the predicted ranges for different sample sizes and percentage results at the "95% confidence interval", taking into account the effect of weighting the data.
Size of sample on which survey results is based | Approximate sampling tolerances applicable to percentages at or near these levels |
|---|
10% or 90% | 30% or 70% | 50% |
|---|
+ | + | + |
100 interviews | 5.9 | 9 | 9.8 |
|---|
500 interviews | 2.6 | 4 | 4.4 |
|---|
1,000 interviews | 1.9 | 2.8 | 3.1 |
|---|
1,714 interview (Pre-survey) | 1.4 | 2.2 | 2.4 |
|---|
1,691 interviews (Post-survey) | 1.4 | 2.2 | 2.4 |
|---|
Source: MORI
For example, with a sample of 1,714 where 30% give a particular answer, the chances are 19 in 20 that the "true" value (which would have been obtained if the whole population had been interviewed) will fall within the range of plus or minus 1.4 percentage points from the sample result.
When results are compared between separate groups within a sample, different results may be obtained. The difference may be "real", or it may occur by chance (because not everyone in the population has been interviewed). To test if the difference is a real one - i.e. if it is "statistically significant" - we again have to know the size of the samples, the percentage giving a certain answer and the degree of confidence chosen. If we assume the "95% confidence interval", the differences between the two sample results must be greater than the values given in the table below.
Size of sample compared | Differences required for significance at or near these percentage levels |
|---|
10% or 90% | 30% or 70% | 50% |
|---|
100 and 100 | 8.4 | 12.8 | 13.9 |
|---|
500 and 250 | 4.6 | 7.0 | 7.6 |
|---|
500 and 500 | 3.7 | 5.7 | 6.2 |
|---|
1,000 and 500 | 3.2 | 4.9 | 5.4 |
|---|
1,000 and 1,000 | 2.6 | 4 | 4.4 |
|---|
1,500 and 1,000 | 2.4 | 3.7 | 4.0 |
|---|
1,714 and 1,691 | 2.0 | 3.1 | 3.4 |
|---|
Source: MORI
« Previous | Contents | Next »