Scottish Household Survey 2022: methodology and fieldwork outcomes

Details of the methodology and survey fieldwork outcomes relating to the 2022 Scottish Household Survey


Survey response

Introduction

The final number of social survey interviews achieved was 10,553. This is above the target of 10,450 interviews and represents an overall response rate of 44.0%. Of the interviews conducted 70% were face-to-face interviews and 30% were remote.

Survey response is an important indicator of survey quality as non-response can introduce bias into survey estimates. Standardised outcome codes (based on an updated version of those published in Lynn et al (2001)[4]) for survey fieldwork were applied across the SHS, SHeS and SCJS. The outcome codes paper includes guidance on the appropriate categorisation of interview outcomes. This will allow consistent reporting of fieldwork performance and effective comparison between the performance of the surveys.

Scotland level

Table 3 shows that the overall household adjusted response rate was 44% per cent. The response rate of 44% was lower lower than pre-pandemic years, which exhibited a long-term mean adjusted response rate of 67% (1999/2000 – 2019). Adjusted response rates (response rates where sample is net of unreachable addresses such as unoccupied houses) were not possible to calculate for 2020 and 2021, therefore unadjusted response rates for 2019 and 2022 are presented alongside them in Table 3 for comparison.

This drop in response rate could be attributed to remaining effects of the COVID19 pandemic. All large population-based surveys in Scotland have been affected by the recent pandemic in terms of recent lower response rates. However, this downward trend had already been identified pre-2020. The mean response rate from 2015 to 2019 was 64%, 3 percentage points below the long-term average. Response rates will be continually monitored as a part of assessing survey quality.

It should be noted that from 2012 the calculation was changed slightly and a portion of the addresses of unknown eligibility are now considered to be eligible, whereas previously they would all have been classed as ineligible.

Table 3: Overall response rates by year

Year

2011

2012

2013

2014

2015

2016

Adjusted response rate

69%

67%

67%

67%

64%

64%

Unadjusted response rate

-

-

-

-

-

-

Achieved sample

14358

10644

10652

10633

10325

10470

Year

2017

2018

2019

2020

2021

2022

Adjusted response rate

64%

64%

63%

NA

NA

44%

Unadjusted response rate

-

-

57%

20%

11%

40%

Achieved sample

10683

10532

10577

3031

9952

10553

Local authority performance

Table 4 below shows levels of ineligible addresses, response rate and random adult conversion.

Table 4: Response rate by local authority

Local authority

Response rate - overall

Household interviews

Random adult interviews

Aberdeen City

30%

289

250

Aberdeenshire

43%

323

307

Angus

42%

247

218

Argyll and Bute

52%

273

252

Clackmannanshire

44%

243

213

Dumfries and Galloway

61%

329

319

Dundee City

50%

257

234

Edinburgh City

52%

1,054

1,011

East Ayrshire

37%

240

203

East Dunbartonshire

41%

237

230

East Lothian

51%

276

250

East Renfrewshire

43%

248

229

Falkirk

42%

215

200

Fife

44%

517

449

Glasgow City

36%

929

866

Highland

45%

344

282

Inverclyde

32%

204

181

Midlothian

49%

281

264

Moray

38%

210

195

Na h-Eileanan Siar

69%

355

352

North Ayrshire

32%

197

157

North Lanarkshire

36%

435

378

Orkney Islands

63%

309

295

Perth and Kinross

44%

252

218

Renfrewshire

36%

271

238

Scottish Borders

58%

324

299

Shetland Islands

68%

347

329

South Ayrshire

36%

207

173

South Lanarkshire

37%

445

409

Stirling

51%

260

234

West Dunbartonshire

37%

211

203

West Lothian

43%

224

197

TOTAL

43.8%

10,553

9,635

Argyll and Bute, Dundee City, Highland, Na h-Eileanan Siar and Orkney Islands were the areas where the highest levels of ineligible addresses were recorded. For Argyll and Bute and Na h-Eileanan Siar, high levels of ineligible addresses were expected as both areas contain a high number of holiday and second homes which are not eligible for the survey. Expected levels of ineligible addresses for Highland, Orkney and Shetland Islands were all near or above 10%.

The three lowest household response rates in 2022 were in Aberdeen City (30%), Inverclyde and North Ayrshire (both 32%).In total 8 out of 32 local authorities had a household response rate lower than 60%

The conversion from household interview to random adult completion was 91% in 2022.

Monitoring and reducing the respondent burden

The Code of Practice for Statistics states that "Statistics producers should be transparent in their approach to monitoring and reducing the burden on those providing their information, and on those involved in collecting, recording and supplying data." Furthermore it states that "The burden imposed should be proportionate to the benefits arising from the use of the statistics."

The following steps are/have been taken to reduce respondent burden.

  • Addresses selected for any of the surveys (SHS, SHeS, SCJS) are removed from the sample frame for a minimum of 4 years so that they cannot be re-sampled for another survey.
  • Samples sizes are no greater than required to obtain robust local authority estimates.
  • The size of the SHS questionnaire was not allowed to increase following the recent consultation on its contents. Where there was a strong policy need for new questions, the space/time required for these new questions was created by dropping existing questions or moving them to a biennial basis.
  • The SHS social interview is carried out using Computer Aided Personal Interviewing (CAPI). Routing is built into the CAPI script which ensures that respondents are only asked questions that are relevant to them, e.g. respondents are only asked questions on mortgage repayments if they have a mortgage.

Interviewer briefings are held annually. These events provide interviewers with an opportunity to feedback on improvements that could be made to the questionnaire to assist them conducting interviews and reduce the burden on respondents.

Physical survey

As described above, just under half of the SHS sample was assigned to the physical survey module. For completion of the physical survey, respondents had to agree to make an appointment for a surveyor to make a follow-up visit and to complete the appointment. Table 5 below shows the number of households assigned to the physical module which responded to the main SHS and the conversion to completion of the physical survey.

There was a target of at least 80 completed physical surveys for each local authority along with a target of 3,004 surveys for Scotland. The results of continuing lower than estimated levels of conversion from household survey to physical survey meant that there were fewer than 80 physical survey responses for 19 local authorities in 2022. The 2,982 surveys achieved for Scotland in 2022 was 22 fewer than the target of 3,004.

Table 5: Interview conversion rate to physical survey

Local Authority

Conversion rate from social survey to physical survey

Number of physical surveys completed

Aberdeen City

46%

66

Aberdeenshire

56%

79

Angus

60%

69

Argyll and Bute

54%

85

Clackmannanshire

50%

64

Dumfries and Galloway

54%

90

Dundee City

57%

79

Edinburgh City

61%

306

East Ayrshire

52%

69

East Dunbartonshire

49%

68

East Lothian

55%

85

East Renfrewshire

52%

76

Falkirk

48%

57

Fife

58%

127

Glasgow City

44%

201

Highland

50%

77

Inverclyde

41%

44

Midlothian

55%

96

Moray

63%

72

Na h-Eileanan Siar

50%

106

North Ayrshire

48%

52

North Lanarkshire

53%

119

Orkney Islands

66%

103

Perth and Kinross

58%

78

Renfrewshire

48%

68

Scottish Borders

53%

105

Shetland Islands

79%

137

South Ayrshire

61%

76

South Lanarkshire

55%

119

Stirling

61%

81

West Dunbartonshire

48%

61

West Lothian

49%

67

Scotland

54%

2,982

Contact

Email: shs@gov.scot

Back to top