The Scottish Government

Next »

Listen

Linda Nicholson
The Research Shop

ISBN 0 7559 6291 5 (Web only publication)

This document is also available in pdf format (692k)

CONTENTS

ACKNOWLEDGEMENTS

EXECUTIVE SUMMARY

PART 1: INTRODUCTION, BACKGROUND AND METHODS

CHAPTER 1: EVALUATING CONSULTATIONS
1.1 State of Play: Overview
1.2 State of Play: Scottish Executive
1.3 Why Evaluation of Consultation is Important
1.4 Why Evaluate the Best Value Consultation?
1.5 Summary of Key Points

CHAPTER 2: THE CONSULTATION PROCESS
2.1 Aims of the Consultation
2.2 Background to the Consultation
2.3 Elements of the Consultation Process
2.3.1 Informal consultation
2.3.2 Formal stakeholder engagement seminars
2.3.3 External reference group
2.3.4 Conference with wider stakeholders
2.4 Decisions taken on the Design of the Consultation
2.4.1 Decision against a written consultation
2.4.2 Decision taken on early one-to-one discussions
2.4.3 Decisions taken on stakeholder seminars
2.4.4 The external reference group
2.5 Summary of Key Points

CHAPTER 3: EVALUATION OBJECTIVES AND METHODS
3.1 The Evaluation Objectives
3.2 Timing of the Evaluation
3.3 Scale of the Evaluation
3.4 Evaluation Method
3.4.1 Briefing with consultation commissioners
3.4.2 Stakeholder seminars: observation
3.4.3 Stakeholder seminars: participant self-completion questionnaire
3.4.4 Stakeholder seminars: de-brief with seminar facilitators
3.4.5 Stakeholder seminars: follow-up of non-attendees
3.4.6 Informal consultation: telephone interviews with informal consultees
3.4.7 In-depth interview with consultation commissioners
3.4.8 External reference group: observation
3.4.9 External reference group: final questionnaire
3.4.10 Evaluation report
3.5 Evaluation Model
3.6 Summary of Key Points

PART 2: EVALUATION FINDINGS

CHAPTER 4: EVALUATOR REFLECTIONS ON THE MODEL OF PARTICIPATION CHOSEN FOR THE BEST VALUE CONSULTATION
4.1 Broad Framework
4.2 Deliberative Process
4.3 Location within the Range of Scottish Executive Activities
4.4 Innovation in Activity
4.5 Summary of Key Points

CHAPTER 5: INPUTS
5.1 Planning and Running the Consultation
5.2 Support from Scottish Executive Colleagues
5.3 Policy Skills Team
5.4 Facilities
5.5 Participants
5.6 Evaluation
5.7 Summary Of Key Points

CHAPTER 6: PROCESSES
6.1 Representation
6.2 Attendance Levels
6.3 Communication of Aims
6.4 Organisation of Events
6.5 Facilitation
6.6 Resolution of Conflict
6.6.1 Preparation for conflicting views
6.6.2 Evaluator's observation
6.6.3 External reference group views
6.7 Other Process Issues Raised
6.7.1 Opportunity for networking
6.7.2 Inclusion at early stage of process
6.7.3 Process flow between consultation elements
6.7.4 Leadership and management
6.7.5 General tenor
6.8 Summary of Key Points

CHAPTER 7: OUTPUTS - BI-LATERALS AND GROUP MEETING
7.1 Consultation Team's Views
7.1.1 Tangible outputs
7.1.2 Less tangible outputs
7.1.3 Unexpected outputs
7.1.4 Limits of the approach
7.1.5 Other efficiency and effectiveness issues
7.1.6 Group meeting
7.2 Informal Consultees' Views
7.2.1 Understanding of the aims of the bi-laterals
7.2.2 Particular benefits of the bi-laterals
7.2.3 Particular drawbacks of the bi-laterals
7.2.4 Personal benefits of taking part in the bi-laterals
7.3 Summary of Key Points

CHAPTER 8: OUTPUTS - STAKEHOLDER SEMINARS
8.1 Consultation Team's Views
8.1.1 Successful elements of the stakeholder seminars
8.1.2 Less effective elements of the stakeholder seminars
8.2 Participants' Views
8.2.1 Views on whether seminars met their aims
8.2.2 Views on the most successful aspects of the seminars
8.2.3 Views on the least successful aspects of the seminars
8.2.4 Overall view of the seminars
8.2.5 Summary of participants' views of the seminars
8.3 Facilitators' Perspective
8.3.1 Views on whether the seminars met their aims
8.3.2 What worked well
8.3.3 What did not work well
8.4 External Evaluator's Observations
8.4.1 What worked well
8.4.2 What did not work well
8.5 Summary of Key Points

CHAPTER 9: OUTPUTS - EXTERNAL REFERENCE GROUP
9.1 Findings Based on Observation and Meeting Minutes
9.2 Findings to Emerge from the Questionnaire
9.3 Summary of Key Points

CHAPTER 10: SELECTED OUTCOMES
10.1 Views on Achievement of Aims
10.2 Views on the Overall Running of the Consultation
10.2 1 Best feature
10.2.2 Worst feature
10.3 Views on Whether the Consultation Represented Genuine Participation and Influence
10.4 Changes in Views Between the Start and the End of the Consultation
10.5 Longer Term Gains from the Consultation
10.5.1 Organisational gains
10.5.2 Personal gains
10.6 Most Important Benefit to Emerge
10.7 Unexpected Benefits/Drawbacks
10.8 Summary of Key Points

PART 3: LESSONS LEARNED AND RECOMMENDATIONS

CHAPTER 11: DISCUSSION OF FINDINGS AND OVERARCHING RECOMMENDATIONS
11.1 Modifications to Policy
11.2 Representation of Good Value
11.3 Getting Past the Usual Suspects
11.4 Participants were Pleased to have been Involved
11.5 The Activity was Well Structured
11.6 A Very Wide Range of Views was Obtained
11.7 It helped to get "Buy-In"
11.8 It Worked Well with our Other Activities
11.9 Did the Consultation Meet its Objectives?
11.10 Recommendations

CHAPTER 12: SPECIFIC GOOD PRACTICE AND LESSONS LEARNED
12.1 Good Consultation Practice
12.1.1 Using existing expertise
12.1.2 Preparation for conflict management
12.1.3 Facilitation of "networks of dialogue"
12.1.4 Early inclusion of participants in the context of "difficult" topics
12.1.5 Capitalise on findings from each stage
12.1.6 Sharing the load
12.1.7 Sector-specific seminars
12.2 Lessons Learned for Future Consultations
12.2.1 Contribution of internal facilitators
12.2.2 Promoting representation
12.2.3 Promoting higher attendance levels
12.2.4 Frequency of meetings
12.2.5 Providing a variety of consultation formats
12.2.6 Allowing consultees a say in the agenda
12.2.7 Duration of events
12.2.8 Importance of small group discussions
12.2.9 Attention to detail
12.2.10 Proactivity in generating practical solutions
12.2.11 Parameters for presentations at events
12.2.12 Take care not to trivialise the process
12.2.13 Agreeing roles for Scottish Executive officials at consultations
12.3 Reflections on the Evaluation
12.3.1 Need for the evaluation to be flexible and accommodating
12.3.2 Benefits of running the evaluation alongside the consultation
12.3.3 Evaluations to aim for holistic view
12.3.4 Establishing benchmarks
12.3.5 Maintaining records of research tools for future use

REFERENCES
ANNEX 1: BEST VALUE STAKEHOLDER ENGAGEMENT SEMINARS PARTICIPANT QUESTIONNAIRE
ANNEX 2: INFORMAL CONSULTATION TELEPHONE INTERVIEW SCHEDULE
ANNEX 3: CONSULTATION COMMISSIONERS' GROUP INTERVIEW SCHEDULE
ANNEX 4: EXTERNAL REFERENCE GROUP FINAL QUESTIONNAIRE
ANNEX 5: SUMMARY OF IAP2 PUBLIC PARTICIPATION SPECTRUM
ANNEX 6: SEMINAR OUTLINE PROGRAMME

TABLES

Table 1: Ways in which external reference group members reported that intended outcome and aims had been communicated to them

FIGURES

Figure 1: Model of the consultation process
Figure 2: Timing of the evaluation in relation to the elements of the consultation exercise
Figure 3: Application of findings on purpose of activity to IAP2 model of public participation

The views expressed in this report are those of the researcher and
do not necessarily represent those of the Department or Scottish Ministers.

Next »