The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER 1 INTRODUCTION AND BACKGROUND

1.1 The Independent Review of Regulation, Audit, Inspection and Complaints Handling of Public Services in Scotland ("the Scrutiny Review"), chaired by Professor Lorne Crerar, will make recommendations to Ministers in September 2007 on a framework for the future external scrutiny of public services.

1.2 To inform the work of the Review, the Scrutiny Review Secretariat commissioned Ipsos MORI, in collaboration with a network of academic experts co-ordinated by Michael Adler, Professor of Socio-Legal Studies at the University of Edinburgh, to undertake a qualitative study of public perspectives of regulation, audit, inspection and complaints handling of public services in Scotland. The Secretariat requested that the study should focus on:

• knowledge and expectations
  • awareness and understanding of current external scrutiny mechanisms
  • expectations of how external scrutiny mechanisms should work
• experience
  • first-hand experience of external scrutiny mechanisms purpose views on the purpose and role of external scrutiny and what it should deliver
  • views on public involvement in scrutiny
• possible improvements
  • views on how the current external arrangements might be changed to better reflect public/service user interests.

The overall framework

1.3 A principal concern of government policy in Scotland, as elsewhere in the UK, is to improve the quality of public services ( i.e. services which are wholly or largely financed out of taxation). In particular, government wishes to improve the ways in which services are experienced by the service user, the accessibility of those services, and their ability to meet user needs.

1.4 A second, related concern addresses the methods by which such services are delivered. In this area, there are several inter-related but distinguishable policy goals including:

• achieving 'Best Value', comprising a mix of economy, efficiency, effectiveness, and other elements
• making appropriate use of new technology (for example, to improve the accessibility of services to users and to share data between organisations more effectively)
• generating systematic savings under the Scottish Executive's current 'Efficient Government' policy
• modernising procurement practice through better use of existing resources, re-investment of savings won, and the use of 'e-procurement'
• encouraging agencies to work in partnership through, for example, community planning partnerships.

1.5 A third concern in the 'Improving Public Services' agenda is improved accountability i.e. more effective scrutiny by those bodies whose roles include planning, funding, regulating, delivering or commissioning/contracting public services. Here too, policy has a multiple focus, which includes both how bodies demonstrate, through their internal governance arrangements, that they are accountable, and their openness to external comment and examination. Examples of the former include:

• the issues which have been the focus of the work of the Committee on Standards in Public life (originally the 'Nolan' Committee)
• the associated new 'Standards In Public Life' legislation for non-departmental public bodies and local authorities
• encouragement of public bodies to create internal scrutiny mechanisms ( e.g. complaints and audit committees) and develop new citizen and/or user participations mechanisms ( e.g. lay representatives and citizens' panels)
• new public performance reporting duties on public bodies (in addition to the new openness created by the freedom of information legislation).

1.6 The latter - external comment and examination - is central to the Executive's present Scrutiny Review. Since the early 1990s, arrangements for regulation, audit, inspection and complaints handling - all four of the elements identified in the Review announced in June 2006 - have been updated and strengthened.

The policy context for the four scrutiny elements

Regulation

1.7 The enterprise of regulating public service delivery has grown in size considerably over the last 15 years or so. Indeed, the regulation of the public sector is now of roughly the same scale as the regulation of the private sector. As a complement to the traditional constitutional checks of the courts and Parliament, regulation now plays an important role in supervising the workings of government and public agencies.

1.8 Interpreted broadly, such regulation would include systems of oversight, audit, and complaints handling, as well as systems of licensing, rule-making, rule-enforcement and the harnessing of the market. Indeed, it is not uncommon for individual institutions to combine regulatory roles such as complaints-handling, audit, inspection and enforcement. As we discuss further below, the Scottish experience reflects this wider UK picture, although regulatory bodies with a more targeted focus have also been created since devolution. These have included:

• Communities Scotland
• NHS Quality Improvement Scotland
• the Scottish Social Services Council
• Office of the Scottish Charities' Regulator
• Regulator of Community Interest Companies
• the Drinking Water Quality Regulator for Scotland
• Scottish Environmental Protection Agency.

1.9 In addition to traditional constitutional checks, the proliferation of regulatory bodies, and the variety of sets of regulatory powers conferred on them since devolution, has created a complex scrutiny landscape relating to public service provision. One of the consequences of this complexity is that various things are being asked of public service provision at the same time. There is a real sense in which public service agencies are subject to multiple scrutiny regimes with, at times, competing demands.

Audit

1.10 Throughout the UK, audit arrangements have been strengthened by a variety of methods:

• the focus of audit - in particular external statutory audit - has been widened, in law and in practice, from traditional financial 'regularity' audit to also embrace new non-financial elements including initially 'value for money' (which introduced scrutiny of 'efficiency') and then, from the late 1990s, the much wider criteria associated with 'Best Value' (including effectiveness, achievement of agency and national targets, and soundness of governance arrangements)
• public bodies have been encouraged to make more use of self-audit - e.g. by strengthening and widening internal audit in ways analogous to those for external audit, and also by support for much stronger performance review arrangements
• external audit bodies have been strengthened by:
  • giving them additional scrutiny roles ( e.g. to specify the public performance information local authorities must report; to examine the 'Best Value' arrangements public bodies have put in place, encourage those bodies to develop action plans to address revealed deficiencies and follow this up with further audit scrutiny)
  • strengthening the audit bodies themselves - e.g. in Scotland, following devolution, merging the former Accounts Commission (whose role focused on audit of Scotland's local government and health bodies) with the former Scottish office of the National Audit Office (whose role focused on the remaining public bodies including central government and non-departmental public bodies/quangos)
  • strengthening the arrangements and introducing new requirements for the public reporting of both data and arrangements which have been subject to external audit, not only to improve accountability but also to make audit work more accessible to users and citizens.

Inspection

1.11 From the New Public Management policies of the 1980s, through the profound changes of the 1990s, national inspectorates have long been identified as key bodies for holding public services to account and maintaining standards.

1.12 Inspectorates have acted as regulators of massive cultural and political changes in the running, financing and management of local services, where not only the probity and efficiency of financial systems are considered but also the effectiveness of the management systems. While inspectorates have pointed the way for service quality to improve, they have also enforced the regulations and evaluated the work of the local services for efficiency and best value. This context has encouraged both the expansion of the number of inspectorates and of their range of functions. Such expansion has, in turn, led the inspectorates to develop a range of different types of inspection. ¹

1.13 This context has also encouraged inspection, regulation and audit to draw closer together as they have moved centre stage in the drive to achieve service improvement and accountability. This has resulted in inspection, regulation and audit proving to be elastic concepts, ever expanding in their purviews and methodologies, with the result that the distinctions and boundaries between them have become increasing blurred. As more bodies have been created, an increasing number have made formal moves to work together, for example the former Joint Reviews conducted by the Social Work Services Inspectorate and the Audit Commission, now known as Joint Area Reviews conducted by the new National Care Standards Commission.

1.14 Recent developments in inspection in Scotland have included:

• changes in the ways in which inspections are conducted, with manuals of performance indicators showing their standards and criteria now becoming the norm
• an increased emphasis on the independence of inspectorates as they move away from being internal to the bodies under inspection, for example the creation of the Inspectorate of Prosecution in Scotland
• an expansion in the scope of inspections conducted, for example:
  • the combining of a range of services as in the inspections of services protecting children and young people
  • the drawing together of multi-disciplinary teams of inspectorates
  • single inspectorates coming together with other regulatory bodies such as Audit Scotland or with NHSQIS
• an extension in the remit of inspection to cover local authorities as well as single institutions
• changes in the ways in which findings are reported, with some publishing different reports addressed to different audiences
• an expansion of the use made of service users both as informants during an inspection and as lay assessors.

Complaints handling

1.15 Complaints about public services have been the subject of several recent policy developments, including:

• the establishment of the Scottish Public Services Ombudsman in 2002 to deal with complaints about most public services in Scotland
• the establishment of the Scottish Commission for the Regulation of Care in 2002, giving it responsibility for dealing with complaints about regulated care services
• a revised procedure in the NHS to deal with health complaints in 2005
• at the UK level, the 2004 White Paper on 'Transforming Public Services: Complaints, Redress and Tribunals' which proposed a reformed tribunal system and emphasised the value of 'proportionate dispute resolution' ², including complaints mechanisms as a means of obtaining redress. At the time of writing, the Tribunals, Courts and Enforcement Bill (which is based on the White Paper) had almost completed its legislative passage.

1.16 Below the level of the Scottish Public Services Ombudsman, which brought together the various Scottish ombudsmen under one body, there remains a potentially confusing range of complaints procedures for different public services. Some of these are statutory and are governed by standard guidelines, such as the health complaints procedure and the social work complaints procedure, but others are left to public bodies to devise. Some bodies, such as the Scottish Commission for the Regulation of Care, combine complaints and regulatory functions, while others, such as the Social Work Inspection Agency do not have any direct responsibility for dealing with complaints.

The stakeholders for the four scrutiny elements

1.17 These scrutiny arrangements beg the central question 'whose interest do they serve?' The short answer appears to be that - apart from central government and the public bodies themselves - there are three major stakeholders: service users, carers/guardians of service users (or indirect service users) and citizens.

1.18 The focus of this research is the service user, since the delivery of public services, and scrutiny arrangements for those services, place the user centre stage. At the same time, public policy and the scrutiny arrangements are increasingly sensitised to acknowledging that there are others who are affected by 'user' issues. Although they are often not always themselves users of the service - e.g. in social care, many vulnerable adults and children have carers or guardians (who may also use services in their own right) and in education, pupils have parents or guardians - they nevertheless have an obvious interest in the services. This group is referred to as indirect service users. Finally, there is also a public interest where the stakeholder is the citizen. That interest is, of course, distinguishable from that of service users (and 'carers').

1.19 There may be occasions when there are sharp differences in stakeholder viewpoints, for example, where the user may favour the highest quality service and the most robust accompanying scrutiny arrangements, while the citizen (who may not be a direct or indirect user of the service) may have reservations over the cost of the service and/or scrutiny arrangements, and may even question the need for the service in the first place.

Focus of the four elements of scrutiny

1.20 In short, these stakeholder perspectives differ - and those differences are reflected in the four different scrutiny arrangements associated with regulation, audit, inspection and complaints handling.

1.21 It is clear that at one extreme, audit has been focused more on addressing 'citizen' than 'user' or 'carer' interests. In part, this reflects the origin of audit in providing the citizen with a safeguard against improprieties in spending by public bodies. The developments described above have blurred this effect a little ( e.g. the new emphasis on public reporting of performance information which has been audited to some extent also informs users), but even here the original policy focus for the relevant legislation, not only in Scotland but throughout the UK, was the Major government's 'Citizen's Charter'.

1.22 Conversely, it is obvious that complaints procedures focus more on user (and carer) needs and interests, with the aim of making services more responsive to user needs.

1.23 Between these extremes, the arrangements for both regulation and inspection ( i.e. the policy and legislative framework, the regulatory bodies, the codes and standards guidance documents, the professional practices of those being scrutinised and those scrutinising, the reporting mechanisms, the sanctions) are intended, at least to some degree, to address the needs of all three groups of stakeholders.

1.24 In short, all four scrutiny mechanisms ultimately contribute to meeting the interests all three stakeholder groups, but with the balance given to each interest differing for each mechanism.

1.25 In turn, the knowledge and the experience that stakeholders have of these four elements of scrutiny can be expected to be uneven - with complaints procedures likely to score 'high', regulation and inspection 'medium' and audit 'low'. These ratings are of course relative and absolute levels are likely to be low. For the majority of members of the public, users and carers, knowledge and/or experience of any of the four scrutiny mechanisms is likely to be superficial.

'Joined up' scrutiny

1.26 A more recent policy issue has been the extent to which the new scrutiny arrangements form a coherent whole. This has become an increasingly important issue because the proliferation of scrutiny activity has triggered a debate about issues such as:

• the resource consequences of the burden of increased scrutiny
• the extent to which there is good co-ordination of scrutiny (between in-house and external scrutiny, and, in particular, between different external scrutineers), including co-ordination of scrutiny work and also joint initiatives between scrutiny bodies
• the relative importance of service delivery and scrutiny.

1.27 At the same time, this often heated debate has been between public and scrutiny bodies - and to date has not been so salient for users, carers and the general public. This is ironic, in that scrutiny is intended to meet the needs of those stakeholders - yet it is also understandable. It is natural that stakeholders should care more that their interests are safeguarded than how this is achieved. The present Review, however, offers an opportunity to address both the extent of safeguarding and the associated methods.

Research Aims

1.28 The overall aim of the research was to inform the work of the Scrutiny Review by increasing its understanding of public perspectives of the external scrutiny regime.

1.29 The specific objectives of this study were to explore:

• public awareness and understanding of current external scrutiny arrangements - i.e. audit, regulation, inspection and complaints handling arrangements, and the use made of outputs such as inspection reports
• public experience of external scrutiny
• public expectations of what the purpose and role of external scrutiny should be and what it should deliver
• views on public involvement in scrutiny e.g. through inspection visits
• views on how the current external arrangements could be changed or improved to better reflect public /service user interests.

Methodology

1.30 We conducted a series of focus groups across Scotland. To give participants the chance to become familiar with the topics and to reflect on the issues raised, each focus group met twice ( i.e. the same participants were brought together on two occasions). Seven pairs of focus groups were carried out, with the first group taking place approximately four weeks before the second. Due to the tight timescale of the project and the experience gained through conducting focus groups with school pupils, who covered the material very quickly and did not use the time in between to think about the issues, the group of young people who had been through the care system covered all the issues in one session. The focus groups were conducted between 6 ^th March and 25 ^th April 2007. On the whole, the use of reconvened groups worked well. Although, it was unclear how much the discussion in the second focus groups was enriched by the fact that participants had been given time to think about the issues, there were clear benefits associated with this method, such as having sufficient time to cover all of the relevant issues and encouraging a good rapport among participants. Only two participants in total failed to return for the reconvened group.

1.31 To help facilitate the discussions and make the concepts more concrete for participants, for seven of the eight groups, participants were sampled to reflect specific service user experiences. The segmentation of the groups is shown in the table below. The services were chosen to reflect experiences over the lifecourse and to cover different types of service. Within the scope of the project, it was clearly not possible to include all types of public service: the aim was to use experiences of particular services to draw more general conclusions about external scrutiny, not to explore views on scrutiny of specific areas of public service provision.

1.32 We also wanted to have one group who would consider scrutiny from a 'public interest' or 'citizen' perspective rather than a 'service user' perspective. This group was recruited on the basis of age and social class. At the start of the first session with this group, usage of various public services was discussed in order to identify a service that none of the participants had used or had any particular knowledge. On this basis, the coastguard service was selected and for the remainder of the first session and the reconvened session, the discussion focused mainly on the coastguard service.

1.33 While we brought together people with experience of particular services/areas, we aimed to achieve breadth across the different groups in terms of socio-economic status, age, gender and geography but relative homogeneity in terms of age and social class within each group. This was important to ensure that participants felt relaxed and uninhibited in the focus group discussions.

1.34 All focus groups took place in local hotels, with the exception of the groups of school pupils which took place in the school and the group of young people who had been through care which took place in the head office of 'Who Cares? Scotland'. The groups took place in the following locations: Crieff, Dumfries, Edinburgh, Glasgow and Inverness. Participants were recruited from those locations and the surrounding areas. The three Ipsos MORI researchers who have authored this report moderated all the groups.

1.35 The participants themselves dictated the general content and flow of the discussions, within the framework of a series of topics introduced by the moderators. The first set of groups was designed to gather contextual information and to introduce key aspects of scrutiny to the participants. The initial discussion centred on definitions of quality of service and awareness of scrutiny. After these issues had been covered, the moderators introduced the scrutiny review and issued participants with an information sheet giving a very brief introduction to the scrutiny of public services in Scotland. The moderators read through the information and gave participants the chance to ask any questions or request clarification on anything they were unsure about. Finally, the participants were issued with a set of seven questions, relating to the purpose of external scrutiny and the ways in which it should be carried out, and were asked to think about them before the next session.

1.36 The reconvened focus groups were designed to give participants the chance to give their own views on the ways in which public services in Scotland s hould be externally scrutinised. The discussion was structured around the questions that were issued to participants at the first session which asked for their views on what the purpose and nature of external scrutiny should be. In addition, participants were shown excerpts from relevant scrutiny reports and asked to comment on how informative and user-friendly they found them. At the end of the groups, they were asked to make recommendations for improving the ways in which public services in Scotland are scrutinised and to give reasons for their recommendations.

1.37 During the first group with the 6 ^th year secondary pupils, it became obvious that a more structured approach to the reconvened groups would be beneficial in order to encourage all members of the group to become involved and to make the issues more concrete. Therefore, a different format was adopted whereby the pupils were given group tasks to complete instead of simply discussing the issues. The first task involved participants dividing into two groups and reading through a series of statements relating to the external scrutiny of schools. They were asked to discuss each statement and place it on a spectrum ranging from 'a very good idea' at one end to 'a very bad idea' at the other end. After doing so, each group presented their assessments to the moderators and to the other group, giving their reasons as to why they had assessed each statement as they had done. The second task required the groups to decide who should be responsible for setting the standards that schools must follow. Each group was provided with a blank pie chart and a list of the groups who could be involved in setting standards for schools. They were asked to fill in the pie chart, indicating how much influence they thought each group should have. Once more, they fed back their decisions to the rest of the group. As this method worked well for the 6 ^th year pupils, it was used again for the group of young people who had been through the care system, where it also proved effective.

1.38 With the permission of participants, all discussions were recorded and then transcribed for analysis. XSight (a qualitative analysis software package) was used to organise, code, search and retrieve the text.

A note on the interpretation of qualitative research

1.39 Qualitative research is often compared and contrasted with quantitative research. Qualitative research is less concerned with measurement ("how many?", "how often?" etc.) and more concerned with understanding motivations, attitudes and feelings ("why?", "how?", etc).

1.40 This study, like most qualitative research, involves many fewer people than quantitative research but it explores their attitudes and experiences in much more depth. The aim is not to generalise to the wider population in terms of the prevalence of attitudes or behaviours ( e.g. 'three quarters of Scottish adults have inadequate pension provision' or 'women are more likely than men to worry about health issues') but to identify and explore the different issues and themes relating to the subject being researched. The assumption is that issues and themes affecting the participants are a reflection of issues and themes in the wider population. Although the extent to which they apply to the wider population or specific sub-groups cannot be quantified, the value of qualitative research is in identifying the range of different issues involved and the way in which they can impact on people.

1.41 So, for example, one of the findings from this study was that service providers should be subject to unannounced inspections as this was the only way that scrutineers could obtain a true picture of service delivery. Although we cannot extrapolate from this and say that this issue affects X% of people in Scotland, the implication is that this is likely to be an issue in the general population.

Structure of the report

1.42 Chapter Two puts the research findings in context. It looks at sources of information on service quality and what builds trust in a service - exploring the place of scrutiny outcomes in relation to personal experience, word of mouth, visits and the media. Chapter Three provides further contextual background in the form of an overview of knowledge and awareness of current scrutiny methods. Chapter Four discusses participants' expectations of the purpose and intended beneficiaries of external scrutiny. Chapter Five looks at views on what should be scrutinised, who should undertake scrutiny, what methods should be used and what powers scrutineers should have. Chapter Six explores views on scrutiny reports and what might make them more accessible. Chapter Seven considers the role of complaints in scrutiny and ways in which complaints procedures could be improved. Chapter Eight focuses on the role of the public in external scrutiny. Finally, Chapter Nine draws conclusions from the research and sets out the implications for the Scrutiny Review.

« Previous | Contents | Next »