The Scottish Government

« Previous | Contents |

Listen

Scrutiny and the Public: Qualitative Study of Public Perspectives on Regulation, Audit, Inspection and Complaints Handling of Public Services in Scotland

The Independent Review of Regulation, Audit, Inspection and Complaints Handling of Public Services in Scotland was undertaken between June 2006 and September 2007. To inform its work, the Review Secretariat commissioned Ipsos MORI and a network of academic experts to undertake a qualitative study of public perspectives on the external scrutiny of public services. A series of reconvened focus groups was held across Scotland in April and May 2007.

Main Findings

• Four roles for scrutiny were identified by participants, the first two of which are the main priorities: ensuring that service delivery meets minimum standards and adheres to statutory requirements; promoting the continual improvement of services; ensuring value for money; and providing information to inform choice of service.
• Independence from the service provider was seen as a very important quality for scrutineers . More specifically, the involvement of independent 'experts' - who are usually professionals with experience of the relevant field but with no connection to the service being scrutinised - was seen as crucial.
• Participants felt that mixed teams of scrutineers, involving a range of experts and stakeholders, including service users, were the best way to ensure effective scrutiny.
• It was felt that scrutineers should: focus on benefiting service users; focus on poorer performing service providers and higher risk situations; facilitate the sharing of best practice; conduct unannounced inspections; scrutinise internal processes as well as outcomes; scrutinise complaints information and complaints handling procedures; produce accessible reports; and follow-up on their requirements and recommendations.
• The dominant view was that the costs of external scrutiny should be met from the public purse.
• The role of service users and the broader public can be placed on a spectrum of increasingly active models from mere information provision to representation in scrutiny work. The consensus among most groups was that involvement should be towards the more active end of the spectrum and that, for example, service users should be involved in inspection teams.

Background

The Independent Review of Regulation, Audit, Inspection and Complaints Handling of Public Services in Scotland ("the Scrutiny Review"), chaired by Professor Lorne Crerar, will make recommendations to Ministers in September 2007 on a framework for the future external scrutiny of public services.

To inform the work of the Review, the Scrutiny Review Secretariat commissioned Ipsos MORI, in collaboration with a network of academic experts co-ordinated by Michael Adler, Professor of Socio-Legal Studies at the University of Edinburgh, to undertake a qualitative study of public perspectives of regulation, audit, inspection and complaints handling of public services.

The specific objectives of this study were to explore:

• public awareness and understanding of current external scrutiny arrangements
• public experience of external scrutiny
• public expectations of what the purpose and role of external scrutiny should be and what it should deliver
• views on public involvement in scrutiny e.g. through inspection visits
• views on how the current external arrangements could be changed or improved to better reflect public /service user interests.

Method

A series of reconvened focus groups was held across Scotland in March and April 2007. Groups of participants met twice ( i.e. the same participants were brought together on two occasions). The groups were segmented to reflect a variety of direct and indirect service user experiences: people with recent experience of NHS inpatient or outpatient services; users of council services; young people who had been through the care system; 6th year secondary school pupils; relatives of care home residents; parents of a child in a nursery; and parents of a child in secondary school. In addition, one group considered scrutiny from a 'public interest' or 'citizen' perspective rather than a 'service user' perspective. This group discussed the scrutiny of the coastguard service because it was a service they had never used.

Main Findings

Awareness, understanding and experience of current scrutiny arrangements

The participants were more familiar with, and more knowledgeable about, inspection and complaints handling than about regulation and audit. This resulted from their greater experience and involvement in these processes, with many participants having been consulted as part of an inspection, used the complaints procedures or known someone else who had done so. Audit was the aspect of scrutiny with which participants were least familiar.

There was a general assumption, however, that things were being checked, that they 'must be', even when they did not know how or by whom. People only really acquired more detailed information when they, or their relative, began to use a particular service.

While some service users wished they had known a bit more about scrutiny, particularly about scrutiny outcomes, when they were choosing a service, in general participants seemed content to get the information they needed ( e.g. to make a complaint) when they needed it. It appears that people do not want or need to be overloaded with information about services in general. What they need is accessible and useful information at the right time. An important time for all services is the point at which users first need them.

The purpose of scrutiny

Participants identified four roles for scrutiny, the first two of which are the main priorities:

• ensuring that service delivery meets minimum standards and adheres to statutory requirements
• promoting the continual improvement of services
• ensuring value for money
• providing information to inform choice of service.

Who scrutineers should be

Independent and objective Independence from the service provider was seen as a very important quality for scrutineers. Independence enabled scrutineers to be objective about the service and provide an honest assessment of its quality.

People with expert knowledge More specifically, in relation to standard setting but also in relation to scrutiny (participants were mainly referring to inspection), the involvement of independent 'experts' - who are usually professionals with experience of the relevant field but with no connection to the service being scrutinised - was seen as crucial.

Service users and service providers It was also seen as important, however, that service users and service providers were involved in standard setting. Service users can provide input based on their needs and expectations. Service providers can provide input based on their knowledge of service delivery and of practical issues and it was felt that their involvement was likely to encourage their commitment to the enforcement of standards.

In relation to inspection teams, a range of independent experts from different fields was deemed necessary e.g. professionals with experience of fire safety, other health and safety experts, plus education or health or social work professionals as appropriate. Again, there was strong support for service user involvement âˆ' assuming careful selection, appropriate training, and guidance from the experts. Less commonly, there was a view that trained or knowledgeable members of the general public who were not service users had a useful role to play. It was felt that service staff should not be members of scrutiny teams because they have a vested interest in the outcome. However, it was also felt that they should have the opportunity to provide evidence to the scrutineers. Participants felt that mixed teams of standard setters and scrutineers, involving a range of experts and stakeholders, were the best way to ensure effective scrutiny.

What scrutineers should do

Focus on benefiting the service user Service users (both direct and indirect) were seen as the most important beneficiaries of an effective system of external scrutiny and, because they were directly affected by service provision, external scrutiny should focus primarily on their needs.

Focus on poorer performing service providers and higher risk situations It was generally agreed that scrutiny resources, e.g. inspections, should focus on poorly performing services or high risk situations. These might be identified from the results of previous inspections, or from 'triggers' such as a change of management, a large number of complaints, unusually high staff turnover, or 'whistle blowing' by staff. That said, a basic level of scrutiny should apply to all services at all times.

Facilitate the sharing of best practice In order to promote the continual improvement of services, scrutiny should facilitate the sharing of best practice between service providers.

Conduct unannounced inspections The importance to participants of unannounced inspections cannot be over-emphasised. There was universal support for this approach, and it was the issue that came up more than any other in discussions and in recommendations. The main reason was the perception that an unannounced inspection was the only way for scrutineers to obtain a true picture of service delivery: with an announced inspection (even if the service provider is given only a few days notice), participants thought problems could be covered up and things could be put in place 'for show'. In other words, they felt it would be easy to 'pull the wool over inspectors' eyes'.

Scrutinise internal processes as well as outcomes It was felt that only by looking at internal processes could scrutineers effectively identify areas where practice could be improved and assess value for money.

Consider complaints information There was widespread agreement that services receiving a large number of complaints should be investigated. No participants suggested that a large volume of complaints could be seen in a more positive way e.g. as an indicator of a successful complaints procedure and confidence in that procedure.

Scrutinise complaints handling procedures While participants were generally comfortable with current complaints procedures, when prompted, they identified five types of barriers which might prevent or deter service users from making a complaint:

• a lack of knowledge about how to complain
• a lack of ability to complain e.g. being unable to express oneself in writing
• a concern about repercussions of the complaint
• a fear of appearing to 'nag'
• scepticism about whether anything would happen as a result of the complaint.

Participants provided several suggestions as to how such barriers could be overcome. There was a consensus that, where possible, complaints should be made to the service involved in the first instance and that only if the complaint was seen to be very serious should the complainant go directly to an external body. As noted above, it was generally agreed that scrutineers should examine the effectiveness of complaints handling procedures.

Participants were introduced to the idea of a single complaints phone line that dealt with all public services in Scotland. Overall, the perceived problems associated with it outweighed the perceived benefits.

Produce accessible reports If scrutiny reports (inspection reports in particular) are to be accessible to service users and the general public they need to:

• be kept very short ( i.e. one or two pages) or there should be a summary report, which would be sufficient for most people, and a readily available fuller report for those who want more detail
• be in plain English and avoid abbreviations/jargon with which most service users/members of the public will not be familiar
• be very clear about how any summary measures or quantitative measures are calculated and what they mean (and the explanation needs to be right next to the measure, not in an introduction or an appendix)
• contain both narrative descriptions and summary/quantitative measures. In terms of whether narrative, descriptive reports were preferable to reports based mainly on quantitative measures (ratings or percentages) or summary measures (such as 'excellent' or 'adequate'), the consensus was that elements of both were required.

Follow up on requirements and recommendations Scrutiny was seen as pointless unless action was taken to sort out any problems identified. Unannounced inspections should be undertaken to check whether service providers had carried out actions they had claimed were 'in hand' or which had been required or recommended by scrutineers. However, while participants thought that service providers should constantly be striving to raise standards (as discussed above, they felt that one of the main purposes of scrutiny was to improve services) they were in favour of a supportive climate where providers are normally given a reasonable amount of time, advice and support to improve. They identified practical problems with more serious actions such as the imposition of new management or closing down services. The latter, in particular, was seen as a last resort which should happen very rarely if there is an effective scrutiny regime in place.

Views on the system as a whole

Costs of scrutiny met by the public purse The dominant view was that the costs of external scrutiny should be met from the public purse. There were two main reasons for this. The first was that it was the only feasible option. The second, equally important reason, was that it was the only fair way âˆ' because everyone benefits from the service and from the scrutiny of the service.

The disadvantages of over-scrutiny While there was universal support among participants for at least some degree of external scrutiny, it was recognised that there were some potential disadvantages. The main problems were thought to be the effect on staff time, increased bureaucracy, undermining trust in professionals, costs, and private sector providers opting out of provision if it became too 'difficult'.

A sensible limit to scrutiny There was widespread agreement that an additional level of scrutiny, to scrutinise the scrutineers, was going too far.

The role of the public in external scrutiny

The role of service users and the broader public in external scrutiny can be placed on a spectrum of increasingly active models from mere information provision to representation in scrutiny work. The consensus among most groups was that service user/general public involvement should be towards the more active end of the spectrum and that, for example, service users should be involved in inspection teams.

The benefits of service user involvement identified by participants fell into three categories: independence from those who run the service; understanding of the service; and the importance to service users of high quality service delivery.

Despite the perceived benefits and the general consensus in favour of public involvement, there were still some concerns about it and a certain lack of support for the lay model of involvement. The main concerns mirrored two of the perceived benefits and were about lack of expertise/understanding (which might apply to some service users but applies to a greater extent to the general public) and lack of independence (in the case of service users). There was also a concern that the types of people who would get involved would not be representative.

Regardless of the perceived benefits or concerns about public involvement, two main barriers to public involvement in scrutiny were identified: lack of interest and lack of time.

Various suggestions were made about how some of the concerns about public involvement and the barriers to it could be overcome. In terms of involvement in scrutiny teams (mainly as lay assessors), the dominant view was that teams should be mixed, with input from experts (independent from the service provider), service users and, in some cases, members of the public who were not service users. The experts could then provide guidance for the service users/members of the public and, to a certain extent at least, compensate for their lack of knowledge or expertise in particular areas. Appropriate training was also viewed as essential for anyone involved in inspections, as was payment of lay assessors in order to compensate people for loss of earnings.

A number of suggestions were also made in relation to better publicising scrutiny outcomes and thereby encouraging more service users to read them (and, perhaps, to become more involved in the scrutiny process). Suggestions included readily accessible reports at the service itself e.g. in a care home or at a day nursery, on the internet and in libraries. All of these methods are currently used, at least sometimes, by scrutiny bodies.

Views on how the current external arrangements might be changed to better reflect public/service user interests

Participants' comments and suggestions on the following issues were in line with what many scrutiny bodies are already doing:

• what should be scrutinised
• scrutinising both internal processes and outcomes for the service user
• the qualities of a good scrutineer
• a prominent, often lead, role for trained experts
• collecting evidence from service users and service staff
• the powers of scrutineers
• where scrutiny reports should be made available
• who should pay for scrutiny ( i.e. the public)
• the role of the public.

There were relatively few innovative suggestions âˆ' this is perhaps not surprising, given participants lack of prior awareness and experience of scrutiny arrangements âˆ' and few suggestions that are at odds with (some) current practice. The most notable were:

• the shared perception that service users and their relatives should be seen as the most important beneficiaries of an effective system of external scrutiny. This constitutes a major challenge for scrutiny bodies
• the suggestion that a service should have the opportunity to feed back on the experience of an inspection and on the report
• producing very short (one or two page) summary inspection reports
• ensuring that reports are written in plain English and avoiding abbreviations/jargon with which most service users/members of the public are not familiar
• being very clear about how any summary measures or quantitative measures are calculated and what they mean (with the explanation appearing right next to the measure, not in an introduction or an appendix)
• in relation to the scrutiny of council services, a more stringent 'test' for independence which would require scrutineers to be from a different local authority area
• a stronger emphasis on unannounced inspections including inspections at all hours of the day and night for round-the-clock services
• inspections triggered by 'alarm bells' such as a change of management, an unusually high volume of staff turnover, an unusually high volume of complaints, and staff reports of incidences of non-compliance or bad practice.

If you have any enquiries about social research, please contact us at:

Tel: 0131 244 4943
Fax: 0131 244 7567
Email: socialresearch@scotland.gsi.gov.uk
Website: www.scotland.gov.uk/socialresearch

The full report " Scrutiny and the Public: Qualitative Study of Public Perspectives on Regulation, Audit, Inspection and Complaints Handling of Public Services in Scotland" which is summarised in this research findings is a web only document and is available on the publications page of the Scottish Government website at: www.scotland.gov.uk/publications/recent

Other documents relating to the Independent Review of Scrutiny are available at http://www.scrutinyreview.org

This document (and other Research Findings and Reports) and information about social research in the Scottish Government may be viewed on the Internet at: http://www.scotland.gov.uk/socialresearch

The site carries up-to-date information about social and policy research commissioned and published on behalf of the Scottish Government. Subjects covered include transport, housing, social inclusion, rural affairs, children and young people, education, social work, community care, local government, civil justice, crime and criminal justice, regeneration, planning and women's issues. The site also allows access to information about the Scottish Household Survey.

« Previous | Contents |