« Previous | Contents | Next »
Listen
C Comparisons of SHS and Census results for local authority areas
C.1 This section describes the results of comparing SHS and Census figures for each local authority area. The comparisons suggest that the survey's figures should provide reasonable indications for most Council areas, but they are not precise: the SHS's results may well differ by more than a couple of percentage points from the true values. For example, the SHS correctly identifies which Council areas have a high percentage of households with a car/van, and which have a low percentage. However, it will not necessarily give the correct value for a particular Council area, provide the correct difference between the figures for any two Council areas, or rank the areas in the correct order. The charts in this section show that, in general, there is a strong correlation between the SHS value and the Census value, indicating that the SHS values are, in general, reliable. However, the charts also show that there are cases for which the SHS values may be misleading (e.g. the SHS may show a greater value for Council X than Council Y, but the Census values for the two are similar). Therefore, the SHS values for individual Councils should only be assumed to be accurate to within a few percentage points.
C.2 The SHS is a sample survey. As is made clear in Sections 2 and 10, the results may be affected by sampling variability and by non-response bias. Every year, in order to see whether such factors have a "significant" effect, figures produced from the SHS are compared with results from other sources, such as the Census, the National Travel Survey and the Labour Force Survey. Chapter 4 of SHS Fieldwork Outcomes describes such comparisons. While the SHS's figures may not agree exactly with those from other sources, there is sufficient broad agreement that there is no cause for concern (some differences are inevitable, given that the Census figures represent the position in April 2001 whereas the SHS data are more up-to-date, and that other surveys' results are themselves subject to sampling and other errors).
C.3 The comparisons reported in SHS Fieldwork Outcomes are for Scotland as a whole, so provide no information about the reliability (or otherwise) of SHS figures for Council areas. Therefore, this section gives the results of comparing SHS and Census figures for individual local authority areas. It is assumed that the Census figures represent the true position in 2001, because the Census covered (almost) the whole population, and so should provide an (almost) ideal "benchmark": any inaccuracies in the Census figures are likely to be very small relative to the SHS's sampling and other errors.
C.4 Ideally, one would compare the Census results with the SHS's figures for individual local authority areas for 2001. However, as mentioned in Sections 2 and B, the SHS is designed to provide figures for most local authority areas only for its "two-year sweep" periods, such as 1999 and 2000 combined. One cannot produce SHS results for most local authorities for a single year (such as 2001) because the sample is only designed to be representative, in terms of the MOSAIC groups of areas within each local authority, over the whole of the two-year period. The Census day was 29 April 2001. The "nearest" SHS two-year sweep is for 2001/2002, whose "midpoint" is 31 December 2001/1 January 2002. This is over eight months after the Census day. However, combining the SHS data from the two-year sweeps for 1999/2000 and 2001/2002 gives a four year period whose "midpoint" is 31 December 2000/1 January 2001 - less than four months before the Census day, and the "closest" to it that is possible for SHS results for individual local authority areas. That is why the comparisons reported here were made using SHS data for the four year period 1999 to 2002, inclusive. The comparisons assume that any underlying trends over the four year period are "balanced" around its "midpoint" (e.g. that lower values in the first two years will be offset by higher values in the final two years), so that the SHS's overall result for the four year period will represent the position around the middle of the period, which (being under four months from the Census date) should be very close to the position at the time of the Census.
C.5 As mentioned in Sections 2 and 10, the likely scale of sampling errors depends upon the size of sample: all else being equal, the smaller the sample, the greater the sampling errors may be. Therefore, the use of SHS data for the four years 1999 to 2002, inclusive, will produce figures which will tend to have smaller sampling errors than if data for only two years had been used - so the comparisons reported here will tend to understate slightly the scale of the likely errors in the SHS figures for a two-year period. However, using data for four years has a benefit: because the sampling variability is reduced, any bias in the SHS results should be seen more clearly. This is important because there is no other way of estimating the scale of any bias.
Work at home
Walk to work
Car driver to work
C.6 The comparisons reported here use the Census's only "transport" information: the number of cars/vans available to the household; and the usual means of travel to work or study (which also identifies those who work at/from home). The charts in this section show the relationship between the SHS and Census figures. In each case, the SHS value is plotted on the vertical axis and the Census value on the horizontal axis. Each chart has 32 points (one per local authority), although some may overlap. In cases where the lowest values are well above zero, the two axes start at a higher value, in order to show better the relationship between the two sets of figures. For example, the axes for the first chart run from 15 to 60, enabling the points to be shown more clearly than would be possible if the axes ran from 0 to 60. In general, the points all fall around the "lower left to upper right" diagonal, on which they would lie if each area's SHS value equalled its Census value. Clearly, the closer the points are to that diagonal the better, since that means smaller differences between the SHS and Census values.
C.7 In the notes that follow, the size of the Scottish figure determined the thresholds which were used to count differences of more than a particular number of percentage points. Because the "car availability" figures for the smallest local authority areas are based on samples of only 1,100 or so (in the four year period), and therefore have "95% confidence limits" of up to +/- 3-4%-points, there is no need to count differences of less than a couple of percentage points. In the case of the "travel to work" figures, the SHS's results for the smallest Council areas are based on samples of only 400 or so (over the four years), so could have confidence limits as wide as +/- 5-6%-points. However, in cases where a typical local authority's value is (say) 10%, apparent differences of only 1-2%-points may be of interest (even though they may not be statistically "significant"), so smaller "percentage point" thresholds were used in some cases. Percentages with two decimal places were used throughout, in order that the comparisons were not affected by rounding.
C.8 The first three charts in this section compare the SHS and Census figures for the percentages of households in each Council area who had: (i) no car/van; (ii) one car/van; and (iii) two or more cars/vans. The Census figures were obtained from Table KS17 in the Census publication Key Statistics for Council areas and Health Board areas Scotland. A number of points arise from consideration of the charts and the underlying numbers (the latter do not appear in this bulletin):
- households with no car/van - for Scotland as a whole, the overallSHS value for the four year period (35.3%) is greater than the Census value (34.2%). For individual local authority areas, the SHS value is greater than the Census value in 21 cases. The pattern of the points (relative to the diagonal) suggests that the SHS tends to underestimate slightly the percentage for local authorities which have low values, and to overestimate slightly the percentage for local authorities which have high values. The magnitude of the difference between the SHS and Census values is more than 2%-points for eight local authorities. The largest difference is 4.8%-points.
- households with one car/van - there is little difference in the overall values for Scotland as a whole ( SHS: 43.7%; Census: 43.4%). The SHS value is greater than the Census value for 21 local authorities. Again, the SHS tends to underestimate slightly the low values, and to overestimate slightly high values. The magnitude of the difference is more than 2%-points for ten local authorities. The largest difference is 4.0%-points.
- households with two or more cars/vans - the overallSHS value (21.0%) is less than that of the Census (22.4%). The SHS value is the greater for only six local authorities. The SHS tends to underestimate the percentage, particularly for local authorities with "middling" values (20-30%). Twelve differences are more than 2%-points. The largest difference is 6.1%-points.
Conclusion: the SHS tends to overestimate the percentage of households without a car/van, and to underestimate the percentage with two or more cars/vans. However, most of the SHS values are within a couple of percentage points of the true (Census) value, and therefore there appears to be only a slight bias in the SHS's figures for the availability of cars/vans.
C.9 The remaining charts compare the percentages who work at/from home, and who travel to work by various modes of transport ( NB: these comparisons do not use the Census data for travel to school or other place of study). For the purpose of these comparisons, "100%" represents everyone in employment (who is not a full-time student), so these figures are on a different basis from those in (e.g.) Table 19 (which covers only those people who travel to work, so excludes those who work at/from home). The Census figures were obtained from Census table CAS224, which is available on the Scotland's Census Results On Line website, and cover "all people aged 16-74 in employment who are not full-time students". A number of points arise from the charts and the underlying numbers (the latter do not appear in this bulletin):
- working at/from home - the overall SHS value is proportionately much higher (8.3% compared with 5.7% from the Census). The SHS value is the greater for all but one local authority, so the SHSoverestimates in almost all cases. 22 of the differences are more than 2%-points, the largest being 8.0%-points.
- walking to work - the SHS's overall value is the greater ( SHS: 12.4%; Census: 11.7%). The SHS value is the greater for 23 local authorities. The SHS tends to overestimate the percentage across the range of values. Ten differences are more than 1%-point, the largest being 2.8%-points.
- driving a car/van to work - the SHS's overall value is lower ( SHS: 52.3%; Census: 56.1%). The SHS value is the greater for 2 local authorities. The SHS tends to underestimate the percentage across the range of values. 26 differences are more than 2%-points, the largest being 12.5%-points.
- car/van passenger to work - the SHS's overall value is higher ( SHS: 10.0%; Census: 8.4%). The SHS value is the greater for 24 local authorities. The SHS tends to overestimate the percentage across the range of values. 19 differences are more than 1%-point, the largest being 10.2%-points.
- bus to work - the SHS's overall value is slightly lower ( SHS: 11.3%; Census: 11.8%). The SHS value is the greater for 14 local authorities. The magnitude of the difference tends to be bigger for the 18 areas for which the SHS value is less than the Census value, so (overall) the SHS appears to tend to underestimate slightly the percentage across the range of values. 12 differences are more than 1%-point, the largest being 5.9%-points.
- train to work - the SHS's overall value is lower ( SHS: 2.4%; Census: 3.4%). The SHS value is the greater for 5 local authorities. The SHS tends to underestimate the percentage across the range of values. 16 differences are more than _%-point, the largest being 3.1%-points.
- bicycle to work - the SHS's overall value is slightly higher ( SHS: 1.5%; Census: 1.4%). The SHS value is the greater for 16 local authorities. The SHS appears to tend to underestimate the percentage for very low values, and to tend to overestimate it for higher values. 9 differences are more than _%-point, the largest being 2.0%-points.
Car passenger to work
Bus to work
Bicycle to work
Conclusions:
1. The SHSoverestimates the percentages who work at home and who travel to work as a passenger in a car/van, and underestimates the percentages who drive a car/van to work and who commute by train. In some cases, there are large differences between the SHS value and the true (Census) value, as can be seen from the numbers of points which are "far" from the "lower left to upper right" diagonal in the relevant charts. Therefore, there appear to be definite biases in the SHS's figures for working at home, and for travel to work as a car/van driver, as a car/van passenger, and by train.
2. The SHS's results for the other main modes of travel to work appear to be only slightly biased: they tend to overestimate slightly the percentage who walk to work, to underestimate slightly the percentage who go by bus, and to underestimate low percentages but overestimate high percentages who cycle to work.
3. However, despite the biases, the SHS does distinguish correctly between those local authority areas which have "large" values and those with "small" values, and so provides a reasonable indication of the general position for each local authority, without necessarily giving a precise value. For example, the "work at home" chart shows that areas which had high values in the SHS also had high values in the Census, but the two values sometimes differ markedly (the worst case being an area for which the SHS value is about 18% but the Census value is only around 10%). Therefore, the SHS will identify correctly which local authorities are at (or near) each end of the range of values, but will not necessarily identify correctly which ones have the largest values or the smallest values.
Bicycle to work
« Previous | Contents | Next »