« Previous | Contents | Next »
Listen
APPENDIX 3: QUANTITATIVE SURVEY - SAMPLE PROFILE AND STATISTICAL RELIABILITY
Statistical Reliability
When interpreting the findings it is important to remember that the results are based on a sample of the participant population, and not the entire population. Consequently, results are subject to sampling tolerances. In other words, not all differences between sub-groups are statistically significant and there is a calculated margin of error for all findings. The sample tolerances that apply to the percentage results in this report are given in the table below. This table shows the possible variation that might be anticipated because a sample, rather than the entire population, was interviewed. As indicated, sampling tolerances vary with the size of the sample and the size of the percentage results.
For example, on a question where 50% of the people in a sample of 196 respond with a particular answer, the chances are 95 in 100 that this result would not vary more than 5.0 percentage points, plus or minus, from a complete coverage of the entire participant population using the same procedures.
Approximate sampling tolerances applicable to percentages at or near these levels |
|---|
Size of sample on which survey result is based | Approx size of the relevant population | 10% or 90%
± | 20% or 80%
± | 30% or 70%
± | 40% or 60%
± | 50%
± |
|---|
196 (participants) | 400 | 3.0 | 4.0 | 4.6 | 4.9 | 5.0 |
|---|
43 (line managers) | 125 | 5.4 | 7.2 | 8.3 | 8.8 | 9.0 |
|---|
Tolerances are also involved in the comparison of results from different parts of the sample. A difference, in other words, must be of at least a certain size to be considered statistically significant. The following table is a guide to the sampling tolerances applicable to comparisons.
Differences required for significance at or near these percentage levels* |
|---|
Size of samples compared | 10% or 90% | 20% or 80% | 30% or 70% | 40% or 60% | 50% |
|---|
Participants |
|---|
196 and 196 | 4.3 | 5.7 | 6.5 | 7.0 | 7.1 |
|---|
100 and 100 | 7.2 | 9.7 | 11.1 | 11.8 | 12.1 |
|---|
30 and 30 | 14.9 | 19.8 | 22.7 | 24.3 | 24.8 |
|---|
Line Managers |
|---|
43 and 43 | 10.4 | 13.9 | 15.9 | 17.0 | 17.3 |
|---|
20 and 20 | 17.5 | 23.4 | 26.8 | 28.6 | 29.2 |
|---|
As the table above demonstrates the smaller the sample or sub-group size the higher the percentage differences between the groups has to be for the difference to be significant.
Due to the small sample sizes within this sample the report concentrates on statistically significant findings although also refers to those where large differences occur.
In tables where percentages do not add up to 100% this is due to multiple answers, to rounding, or to the exclusion of 'Don't know' or 'No response' categories. Throughout the tables an asterisk (*) denotes a value greater than zero, but less than 0.5%.
« Previous | Contents | Next »