« Previous | Contents | Next »
Listen
APPENDIX A: GUIDE TO STATISTICAL RELIABILITY
The young people who took part in the survey are only a sample of the total secondary school population in Scotland, and as such, we cannot be certain that the figures obtained are exactly those we would have if everybody had been interviewed (the 'true' values). For a random probability survey we can, however, predict the variation between the sample results and the 'true' values from a knowledge of the size of the samples on which the results are based and the number of times that a particular answer is given. The confidence with which we can make this prediction is usually chosen to be 95% - that is, the chances are 95 in 100 that the "true" value will fall within a specified range. The survey reported on here was not a random probability survey, being limited to a sample of school pupils across the country. However, for information only, the table below illustrates the predicted ranges for different sample sizes and percentage results at the 95% confidence interval:
Approximate sampling tolerances applicable to percentages at or near to these levels | Actual Sample Size | 10% or 90% ± | 30% or 70% ± | 50% ± |
|---|
Overall | 1,762 | 1.4 | 2.1 | 2.3* |
|---|
*For example, if 50% of all young people were to give a particular answer, the chances are 19 in 20 that the 'true' value will fall within the range of ±2.3 percentage points from the sample results.
Comparing percentages between sub-groups and overall total
When results are compared between separate groups within a sample, different results may be obtained. The difference may be 'real', or it may occur by chance (because not everyone completed a questionnaire). To test if the difference is a real one - i.e. if it is statistically significant - we again have to know the size of the samples, the percentages giving a certain answer and the degree of confidence chosen. If we assume 95% confidence interval, the difference between two sample results must be greater than the values given in the table below:
| Actual Sample Size | 10% or 90% ± | 30% or 70% ± | 50% ± |
|---|
Overall (1,762) vs: | | | | |
|---|
Sub-groups of: | 200 | 4.4 | 6.7 | 7.3 |
|---|
500 | 3.0 | 4.6 | 5.0 |
1000 | 2.3 | 3.6 | 3.9 |
1500 | 2.1 | 3.2 | 3.4 |
*For example, if 50% of the total sample (1,762) gives a particular answer, and 53% of young people in a sub-group of 500 give the same answer, there is not a statistically significant difference between the responses of the two groups.
Looking at the fifth column of the above table shows that there needs to be a difference of ±5 percentage points between the two results in order for the difference to be statistically significant. Therefore, if 56% of the latter group give the same answer, then this is a statistically significant difference (since there is a 6 point difference between the two).
Comparing percentages between sub-groups
The following table indicates differences required for significant comparisons between sub-groups.
Approximate sampling tolerances applicable to percentages at or near to these levels | 10% or 90% ± | 30% or 70% ± | 50% ± |
|---|
Size of samples compared: | | | |
|---|
100 and 100 | 8.4 | 12.8 | 13.9 |
|---|
100 and 250 | 7.0 | 10.7 | 11.6 |
|---|
250 and 250 | 5.3 | 8.0 | 8.8 |
|---|
500 and 250 | 4.6 | 7.0 | 7.6 |
|---|
500 and 500 | 3.7 | 5.7 | 6.2 |
|---|
1,000 and 500 | 3.2 | 4.9 | 5.4 |
|---|
« Previous | Contents | Next »