« Previous | Contents | Next »
Listen
CHAPTER TWO COMPARISON WITH OTHER SURVEYS AND EXAMINATION OF THE VOLUNTEERING QUESTIONS IN THE 2005 SHS
Summary
2.1 In this chapter, we have suggested that the reason behind the differences in rates of volunteering reported in the Scottish Household Survey ( SHS) and other surveys lies in the format and structure of the questions used to identify a volunteer. It is our sense that asking individuals to firstly specify the organisations they have helped - rather than the more general activity they have participated in, for example raising money - may partly account for the differences in rates of volunteering across the three surveys. Further research into this issue is recommended.
2.2 We also suggest changes to the format and structure of the volunteering suite of questions in the SHS, for example:
- removing the 'no regular hours' category when asking about length of time spent volunteering
- asking non-volunteers and current volunteers about attractions and barriers to volunteering
- asking all respondents about the qualifications they hold, rather than only those aged from 16 and not retired, to gain an accurate picture of the qualifications held by volunteers in Scotland.
2.3 The 'no regular hours' category was removed from the 2006 suite of questions and further changes - such as asking non-volunteers and current volunteers, as well as former volunteers, about attractions and barriers to volunteering, as well as all random adults about qualifications - have been made to the 2007 suite.
Introduction
2.4 In this chapter we look first at the definitions of volunteering and then at the rates of volunteering in three surveys - SHS, the Volunteer Development Scotland ( VDS) Annual Digest of Statistics, and the Department for Communities and Local Government ( DCLG) Citizenship Survey. We already know that the rates reported in the VDS digest and the DCLG survey are significantly higher than those reported in the SHS.
2.5 We look at the profile of volunteers, in terms of socio-economic background, place of residence, working status, sector, and activity of volunteering across the SHS and the VDS digest to determine whether or not there are substantial differences in the type of people who say they volunteer. We find that there are no broad differences between the surveys.
2.6 We then examine in detail the initial questions, which ascertain the rate of volunteering in Scotland, in the SHS survey over the period 1999 - 2006.
2.7 We also examined the initial questions asked in each of the comparator surveys for the year 2005, the only year for which we have relatively comparative data.
2.8 Based on that examination we pose some potential answers as to why there are such different reported rates of volunteering across the three surveys.
2.9 We also examine structure and format of the questions in the SHS 2005 and we examine the methodological performance of the remaining questions.
2.10 Finally in this chapter we make some suggestions as to issues the Executive might consider in relation to the current content, structure and format of the suite of volunteering questions in the SHS.
Definitions - Scottish Executive and VDS
2.11 The definition of volunteering used by the Scottish Executive and VDS is:
"the giving of time and energy through a third party, which can bring measurable benefits to the volunteer, individual beneficiaries, groups and organisations, communities, environment, and society at large. It is a choice undertaken of one's own free will, and is not motivated primarily for financial gain or for a wage or salary."
Formal and Informal Volunteering
2.12 This definition fits with a broad understanding of what is formal volunteering - where unpaid work is undertaken through an organisation, group or club to help other people or to aid a cause such as the improvement of the environment, rather than informal volunteering - where unpaid help is given as an individual to people who are not relatives.
Department for Communities and Local Government ( DCLG) Citizenship Survey
2.13 The Department for Communities and Local Government's ( DCLG) Citizenship Survey 2005 defines formal volunteering simply as:
"Giving unpaid help through groups, clubs or organisations to benefit other people or the environment."
2.14 The DCLG uses the same definition of informal volunteering as the Scottish Executive and VDS.
Policy Interest
2.15 An important area of policy interest in Scotland is that the SHS survey consistently shows a lower percentage of volunteers in the population than other surveys in Scotland and in England. Some questions as to why this might be include:
- the content of the questions are substantially different across different surveys
- the structure and format of the suite of questions in different surveys is a factor
- the barriers to or encouragement of volunteering in one country may be different from another
- awareness of volunteering may vary from one country to another
- cultural views of volunteering may vary from one country to another.
2.16 There is a lack of consensus regarding definitions of volunteering ( see Annex 1) and exactly which activities should and should not be regarded as volunteering.
2.17 Consequently, estimates of the prevalence of volunteering in the population as a whole, and among certain groups in society, gathered through various surveys are often different and incomparable.
2.18 Because different surveys vary in terms of questions, definitions and so on, it is difficult to make true comparisons but in the research we have aimed, through close investigation of the questions asked and answered in different surveys, to identify research questions which may, through further work, begin to shed light on reported differences in the prevalence of volunteering.
Prevalence of Volunteering
2.19 We have set out in detail in Chapter 3 - Prevalence of Volunteers and Comparison with VDS Digest of Statistics, the comparison of the profile of volunteers between the SHS and the VDS survey and we summarise the outcomes below.
2.20 Overall, in terms of age, gender, socio-economic status, and area of residence the profile of the volunteers in the two surveys is broadly similar.
2.21 Both surveys found that three of the most common volunteering activities in Scotland are raising money, helping to organise or run an event and committee work.
Comparator Surveys
2.22 Apart from the SHS, the other important source of data on volunteering in Scotland is the Annual Digest of Statistics on Volunteering in Scotland, compiled by Volunteer Development Scotland.
2.23 Both the Scottish Executive and VDS use the same definition of volunteering, however the questions about volunteering used by each body are different. Overall therefore, in terms of methods for data gathering, the VDS digest is not comparable to the SHS.
2.24 In addition, the VDS 2006 Digest (Volunteer development Scotland, 2006) provides a digest of data taken from a number of sources, including opinion polls, a survey of Voluntary Service Managers and its own database. Within the digest, VDS reports on the outcomes of the TNS Global Omnibus Survey November 2005 which asked specific questions about formal volunteering. It is this element of the VDS digest which we have used as the basis for comparison with the SHS in terms of percentage of volunteers, socio-economic background, place of residence, working status, sector, and activity of volunteering.
2.25 The other survey which we have examined is the Citizenship Survey (Department for Communities and Local Government ( DCLG) - formerly implemented by the Home Office).
Rates of Volunteering
SHS and VDS Annual Digest ( TNS System Three Survey 2005)
2.26 For the purposes of statistical comparison of the rates of volunteering across different surveys, we have used the year 2005. This is because it is the only year for which we have comparable data. However, we have to bear in mind that the question in the SHS used to identify rates of volunteering was different in 2005 from the question used in 1999 to 2004, and then again in 2006. The rate of volunteering in the 2005 survey appeared to be significantly lower than in any of the other years, from 1999 to 2006.
2.27 The key difference between data relating to volunteering in 2005 reported by SHS (Scottish Executive, 2005b), the VDS Annual Digest of Statistics on Volunteering in Scotland (Volunteer Development Scotland, 2006) and the Home Office Citizenship Survey (Kitchen et al, 2006) is in terms of the percentage of people who say they have volunteered.
2.28 The SHS data shows that 19.8% of survey participants say they have volunteered formally in the past 12 months. The VDS Annual Digest (Volunteer Development Scotland, 2006) estimates that 38% of Scottish adults volunteered in the previous year, and the Home Office survey produces a figure of 44% of people participating in formal voluntary activities in the twelve months before interview.
2.29 Even allowing for the fact that the 2005 figure in Scotland was apparently lower than in other years, the highest average rate of volunteers in Scotland over the period 1999 to 2006, is 26.4% which is still lower than either of the comparator surveys.
Trend in reported rates of volunteering in SHS
2.30 Figure 2.1 shows the SHS time series trend in reported rates of volunteering from 1999 to 2006. The full data analysis of trends over time is given in Chapter 9. The rate has changed over time, but is consistently lower than in comparator surveys.
Figure 2.1: Percentage of Population Engaged in Volunteering 1999 - 2006, with error band
2.31 There appear to be some statistically significant variations between the annual rates of volunteering reported between 1999 and 2006. For example, an average of 23.1% was identified as a volunteer in 2004, but this, following a change of question, appeared to decline significantly to 19.8% in 2005. Most of the variations between years, however, are not statistically significant.
2.32 Alterations made to the volunteering suite of questions between 1999 and 2006 make it difficult to compare volunteering data over the years. For example, the rate of volunteering among those aged 25 to 34 declines significantly between 2000 and 2006 from 26.2% to 18%, but the methodology used to identify those respondents involved in volunteering was altered during this time.
2.33 Similarly the significant increase in the rates of volunteering reported from 1999 to 2000 is probably due to a change in the question to make it more open. The percentages increased from an average of 19.3 to an average of 26.1.
2.34 From 2000 to 2002 reported rates of volunteering remained relatively consistent. Percentages fell in 2003 to 23.8% and continued to fall (but not significantly) in 2004. 2005 saw a dip to 19.8%, again possibly due to a change in the question, and in 2006 after another change to the question, the figure rose again to an average of 24.5%.
The content of the questions asked
2.35 A core question for the Scottish Executive remains - compared with other surveys why does the SHS produce such a relatively low percentage of participants who say they have volunteered? Is it to do with the questions asked?
Scottish Household Survey
2.36 This issue is not new. In 2004, the Scottish Executive commissioned RDS Consulting (later Hexagon) to carry out a study into "Improving the Collection of Volunteering Data in Scotland".
2.37 The study was commissioned because:
"There is a strong feeling amongst key stakeholders that the current approach taken in the SHS may be under reporting formal volunteering due to problems with the format and content of the questions currently being used. The key aim of this research was, therefore, to make recommendations about potential revisions to the current module of questions included in the SHS to ensure that the survey recognises the full volunteering contribution made by the people of Scotland and provide more accurate evidence about the characteristics, skills, attitudes and behaviour of such individuals". (Scottish Executive, 2005a)
2.38 The study took place to tackle two problems identified in the volunteering suite of questions:
- a fear that the focus of the preamble of the questions may be too narrow
- the wording of the questions themselves.
2.39 This piece of research found that the complexity of volunteering as an issue and different interpretations of its scope make it difficult to reach a correct or definitive figure for the number of people involved in volunteering activity in Scotland (Scottish Executive, 2005a).
2.40 The research led to changes to the volunteering suite of questions in the SHS. These changes intended to address the need to produce a more accurate picture of the prevalence of volunteering in Scotland that could be used as benchmark to measure future changes, using methodology agreed by key stakeholders.
2.41 As a result of the Hexagon study, in SHS 2005 the first question in the volunteering suite of questions was changed from:
"Thinking back over the last 12 months have you given up any time to help any clubs, charities, campaigns or organisations, I mean in an unpaid capacity?
to:
Vol1: Show card: Have you undertaken any work or activities on a voluntary basis for any of these types of groups or organisations at any time in the past 12 months?
2.42 Respondents were then shown a card with a list of types of organisations (the options included on the showcard used with Vol1 can be seen in Annex 2, which consists of the volunteering suite of questions from SHS 2005).
2.43 Respondents were then asked this question:
Vol2:"And have you undertaken any work or activities on a voluntary basis for any other groups or organisations in the past 12 months?"
2.44 By asking this follow up question it was intended that those volunteers engaged in voluntary activity with organisations that did not fit into the categories on the showcard could be identified.
2.45 Changing the question and using a prompt card did not have the desired outcome since the percentage stating they volunteered actually dropped in the 2005 survey.
2.46 Therefore in 2006 the survey reverted to using the question in the 2000 - 2004 surveys which was ( RF11A):
"Thinking back over the last 12 months have you given up any time to help any clubs, charities, campaigns or organisations, I mean in an unpaid capacity?
but,
in an attempt to reach those people who do not realise, or have forgotten that they have actually taken part in formal volunteering, (and therefore potentially increase the response rate) a new prompt card (below) was introduced for those who said No to RF11A.
Vol1A said:
Show Card: We often find that people forget about some of the things they have done because they only do them occasionally or wouldn't normally think of it as helping people or their community. Have you undertaken any work or activities on a voluntary basis for any of these types of groups or organisations at any time in the past 12 months?
2.47 The showcard included the following options:
Voluntary organisation/charity/community group
Church, religious or faith-based group
Sports club
Caring organization
Youth group (e.g. scouts, guides, youth clubs etc)
Children's group (e.g. playgroup, mothers and toddlers groups
Public service ( e.g. school, hospital, police or local government service)
School board or PTA
Arts or cultural group
Community council, SIP or Community Planning Partnership
Professional society or organization
Tenants group, housing association or residents' association
Social club
Trade union
None
Don't know
2.48 By introducing this additional question, the response rate in 2006 appeared to rise from the low of 19.8% in 2005 to an average of 24.5%. Of these, 90% answered positively to the original question, RF11A and a further 10% were "screened back in" when they saw the Vol1A showcard. However the total was still lower than the comparator surveys.
2.49 Since it seems that changing the SHS questions in that way does not of itself bring about dramatic improvement in the rate of volunteering we turned to the comparator surveys to look more closely at the questions they ask and the order they ask them in.
Volunteer Development Scotland
2.50 The first question in the 2005 TNS Global Omnibus Survey, used to compile the VDS Annual Digest Statistics, is:
Showcard 1 Look at this showcard, which gives a list of unpaid activities people do to help others. In the past year have you done any of these activities, unpaid to help others?
Yes, no, don't know
2.51 As we know, 38% of people who looked at the show card responded positively.
2.52 For those who responded positively, the first of the multi-choice options on the showcard is:
Helping to raise money
2.53 Sixty-eight per cent of those who responded positively identified this as the activity they had undertaken (more than twice as many as in the next category - organising or helping to run an event).
2.54 It is impossible to be categoric on this issue since we do not know how people interpret the option of "helping to raise money". However, it might be the case that a number of people might interpret this as for example, buying a raffle ticket, contributing to a disaster fund, or sponsoring a colleague or friend to run a marathon for charity, rather than acts of organisation, such as taking part in a sponsored activity, writing grant applications, etc.
2.55 It may be therefore that in the VDS survey, the question on the showcard about "helping to raise money" is a possible explanation as to why the response rate is higher than that in the SHS survey.
2.56 A question about raising money is asked in the SHS but only in Vol6, by which time participants are firmly established as having been involved in helping specific types of organisations over a specific time period. The percentage in the SHS who say they raise money 27.8%.
2.57 The next most popular activity - organising or helping to run an event could be a one-off activity such as helping out at a school fair, which took up around two hours in one twelve month period.
2.58 This is legitimately called volunteering but it does raise the issue of length of time spent volunteering, and whether in one survey we are counting lower numbers of people volunteering for longer periods and in another higher numbers volunteering for very short periods or one-off occasions.
2.59 In the SHS survey 24.4% of respondents say they spend fewer than 5 hours a month volunteering. This figure is higher for VDS, at 53%. This may imply that the respondents in the SHS survey spend longer on volunteering activities than the respondents in the VDS survey. However there is a "no regular hours" category in the SHS which accounts for 21.4% of respondents and it is not possible to say how many of those individuals volunteer for less than 5 hours per month.
2.60 This category really does not tell us anything, in fact it militates against an accurate picture of time spent on volunteering. It should be noted that the 'no regular hours' category has now been removed from the Scottish Household Survey.
Home Office/Department for Communities and Local Government Citizenship Survey
2.61 In looking at the Home Office Citizenship Survey 2005, ambiguity around giving or raising money does not on the face of it seem to be the explanation for the difference in response rates between SHS and that survey. This is because the Citizenship Survey asked respondents about their participation in formal volunteering in the previous 12 months and specifically rules out giving money as an option.
Show card: "which of the following groups, clubs or organisations have you been involved with during the last 12 months? That's anything you've taken part in, supported or that you've helped in any way, either on your own or with others. Please exclude giving money and anything that was a requirement of your job."
2.62 However, one of the options for "taking part or supporting" on the show card is:
(4) Sports/exercise (taking part, coaching, or going to watch)
Another option is:
(14) Hobbies/Recreation/Arts/Social Clubs
2.63 Clearly, we do not know how people interpret the options included in the VDS survey and the Citizenship Survey identified above. However, it might be the case that a number of people might interpret "helping to raise money" as, for example, buying a raffle ticket or sponsoring a friend to run a marathon.
2.64 It is possible, therefore, that some respondents who were identified as volunteers by the VDS survey and the Citizenship Survey were thinking of activities which, whilst voluntary in the sense that they were not compelled to do them, may be difficult to describe as formal volunteering.
2.65 Again, as with the VDS survey, when participants are asked, as the second question in the survey:
In the last 12 months have you given unpaid help to any groups, clubs or organisations in any of the following ways?
2.66 And given the options of:
Raising or handling money/taking part in sponsored events
and
Organising or helping to run an activity or event
2.67 These - potentially one-off activities - are the most popular for those who participate in voluntary activity (51% and 47% respectively).
Format and Structure of Questions
2.68 In addition to the above differences in the content of the survey questions, there is another significant difference between the SHS survey and both other surveys in terms of the format and structure of the important opening questions.
2.69 The SHS showcard asks about types of groups or organisations which the individual may have volunteered with, whereas the other two surveys ask about the type of activity the individual might have undertaken.
2.70 Asking individuals firstly to specify the organisation they have helped - rather then the more general activity they have participated in, for example raising money, may also partly account for the differences in rates of volunteering across the three surveys.
2.71 There is also a structural problem associated with the show card used in the SHS survey in that it confuses a description of the structure of an organisation - that is, voluntary organisation, charity or community group, (in other words third sector organisations) with a description of the function of the organisation eg a school board, arts or cultural group etc. This is being addressed by changes implemented for the 2007 Scottish Household Survey.
2.72 All the organisations listed on the show card are either third sector organisations or public sector organisations.
2.73 It would be more accurate and avoid confusion and probably double counting if Third Sector and Public Sector were used as category headings and then examples of each type of organisation were listed on the show card.
Methodological Performance of remaining questions in SHS 2005
2.74 From our analysis in Chapters 7 and 8, it appears that a small number of current volunteers were asked questions at Vol8 and Vol9, although the wording of Vol8 and Vol9 suggests they are only intended for those respondents who are not current volunteers but who had volunteered in the past.
2.75 There is no direction given after Vol6 and therefore it is not clear from the directions given in the questionnaire script that Vol7 is only intended for people who have not volunteered in the past.
2.76 We suggest that if the aim is to avoid those who have volunteered in the past twelve months from accidentally being asked this question then there needs to be an instruction at Vol7 as follows:
Vol7 onwards is to be asked only of people who have not taken part in voluntary activity the past 12 months
2.77 As a belts and braces approach, the question at Vol7 might be changed to something along the lines of:
Although you have not been engaged in voluntary activity in the past twelve months, have you ever taken part in work or activities on a voluntary basis?
2.78 Questions about attractions and barriers to volunteering were asked only of former volunteers. This is a potential gap in SHS questioning, as to determine a complete picture of attractions and barriers to volunteering, current and non-volunteers as well as former volunteers should be asked about these issues. This could be done either by adding questions to the SHS (although it is already a crowded survey) or through another survey. This is an issue that has been addressed for the 2007 suite of questions, where all non-volunteers will be asked about potential attractions and barriers.
2.79 Questions relating to qualifications held were asked only of those people aged from 16 and not retired and therefore provide no information about the qualifications held by those volunteers who have retired.
2.80 As the 60-74 age group accounts for on average 22.4% of volunteers, and considering that volunteers who retired were not asked about qualifications they hold, data regarding qualifications has not been gathered from a large proportion of volunteers. Therefore, this analysis does not provide an accurate picture of the qualifications held by volunteers. The issue has been addressed for the 2007 suite
« Previous | Contents | Next »