« Previous | Contents | Next »
Listen
1. Introduction
1.1 Background
In November 2007, ODS Consulting was commissioned by the Scottish Government 1 to evaluate the audit and accreditation process for the Scottish National Standards for Information and Advice.
1.2 Aims of the Research
The aim of the study is to assess the impact the audit and accreditation process has had on improving the quality of information and advice to consumers. Specifically:
to examine the contribution accreditation has made to improving quality in information and advice services; and
to identify any added value that accreditation offers compared with working towards self-assessment against the Standards.
1.3 Our Approach to the Study
Our research methodology included four stages:
Stage One: Desktop and Literature Review
Stage Two: Consulting with Information and Advice Providers
Stage Three: Stakeholder Consultations
Stage Four: Reporting
Stage One: Desktop and Literature Review
We undertook a desktop review of the background relating to the development of the audit and accreditation process. We conducted a tracking exercise to establish what has happened since the pilot was launched in terms of the number and nature of organisations participating. This included the number of organisations that have been preparing for, and undergoing audit; those that had not proceeded to audit; those that been unsuccessful in the audit process and those that have been re-audited.
We also conducted a review of a number of different models of accreditation in operation across different sectors in the UK. This identified their key features and any apparent advantages or disadvantages. Where reviews or evaluations had been undertaken of their effectiveness, we highlighted the findings and lessons that had been identified.
Stage Two: Consulting with Information and Advice Providers
We wanted to compare the experiences and views of a sample of two groups of organisations by undertaking semi-structured telephone interviews. Firstly, those that had been accredited, not yet fully compliant or had been through the audit process, including during the pilot. Secondly, those that were "known to be self-assessing but (to date) had not taken a decision to seek accreditation".
Our original sampling framework - based on information held by HomePoint - identified 20 agencies in each category. These were selected to try and include a range of different types of organisations and services provided.
In the event, we spoke to 16 organisations in the former category, although it transpired that two had yet not been audited. Attempts to engage with a further four proved unsuccessful when the fieldwork was being undertaken in mid December.
Of the latter group, we spoke with 11 agencies on our original list. Due to difficulties in getting a response from the remaining nine, we extended the list to include a further eight organisations.
Whilst consulting with organisations that had not been through the audit process (and were not preparing for audit), it soon became apparent that none were self-assessing their performance against the Standards in a systematic way. Only organisations that have been audited or intending to undertake a future audit were systematically working to the Standards.
As such, the interviews did not represent a straight contrast between audited organisations and self-assessing organisations. Nevertheless, the interviews represented a cross-section of organisations at different stages in the process. Discussions focused on motives and intentions of organisations that: had been accredited; had been audited but not accredited; were working towards a possible future audit; and those that were not intending to apply for audit. The interviews explored the rationale for the decisions taken to date (with regard to adopting the National Standards), how they had progressed, their perception and any evidence of benefits that had accrued, and any lessons learnt so far.
We then selected 12 organisations as case study examples. Of these, nine organisations had been through the audit process (one had given up accreditation and one was not yet fully accredited) and three had shown interest in the Standards by attending some training but had not taken a decision to undergo audit.
The case studies explored in greater detail the contribution adopting the National Standards had made to improving the quality of services, and any added value accreditation had offered, compared with self-assessment. In addition to gaining the perceptions of staff and members of the governing body, where appropriate, we spoke to other relevant stakeholders such as funders.
Stage Three: Stakeholder Consultations
While undertaking the case studies, we also consulted with a wider range of stakeholders. These included a number of representative and umbrella organisations within the information and advice sectors, government and local authority representatives. The interviews explored perceptions from a wider perspective of the impact and any added value accreditation had brought.
A workshop was also held with members of the Scottish National Standards for Information and Advice Providers Panel.
Stage Four: Reporting
Our report:
- describes the process undertaken to date in terms of auditing and accrediting information and advice organisations;
- sets the context in terms of quality management frameworks;
- provides an assessment of the number and nature of organisations working towards, undergoing or having been audited;
- identifies the motives of organisations that had been audited (and, in some cases, accredited);
- assesses the impact that audit and accreditation has had on service quality from the perspective of a range of stakeholders;
- draws out lessons that have been learnt from the experiences to date; and
- recommends ways in which the accreditation framework could be strengthened.
1.4 Structure of the Report
The next section outlines the development of the audit and accreditation process - and considers the lessons from other accreditation models.
Section Three outlines the views of the advice providers that we have spoken to, while Section Four incorporates the results of discussions with stakeholders. Section Five gives conclusions and recommendations.
A list of consultees is included as Appendix 1, literature and online sources as Appendix 2. Our twelve case studies are available as a separate document.
« Previous | Contents | Next »