« Previous | Contents | Next »
Listen
CHAPTER 10: SELECTED OUTCOMES
In common with many previous evaluations of consultations, the extent to which longer-term outcomes can be identified is curtailed by the timing of the research. An evaluation running alongside the consultation process, as in this case, can report on immediate outputs and process issues, but is unable to report at this stage on the longer term impacts of the consultation findings and outputs. This would require a follow-up investigation at a later date.
However, whilst the evaluation cannot as yet comment on the utility of the updated Best Value Guidance and Tools, since they were only published in May 2006, it is possible to pick up on some of the other wider benefits of the exercise for organisations and individuals who participated. The evaluation identified a number of these by means of questions placed in the questionnaire issued to external reference group members. Many of these individuals had taken part in the consultation from the earliest informal consultation stage with most attending one of the stakeholder seminars. Thus, they were in a useful position to give a considered view on the overall consultation and its impact.
10.1 VIEWS ON ACHIEVEMENT OF AIMS
External reference group members were asked to what extent they considered that the consultation was meeting its intended outcome and aims. Using a 10-point rating scale with 1 representing "not at all" and 10 representing "totally met" respondents provided an average rating of 7 with a range of 6 to 9.
10.2 VIEWS ON THE OVERALL RUNNING OF THE CONSULTATION
The questionnaire asked what the single best feature and worst feature of the consultation had been for external reference group members.
10.2.1 Best Feature
A variety of aspects of the consultation were identified. Most were process issues:
- A genuine feel that views had been listened to and participants had been truly involved
- A lack of a judgemental approach by the Scottish Executive
- The apparent openness and sincerity of the Scottish Executive officials involved
- The open, relaxed and frank discussions
- The opportunity to work with colleagues and learn about other approaches and sectors
- Enabling the views of senior practitioners to shape the overall process
One consultee identified the likely outcome as the best feature:
- The opportunity to produce something useful to my organisation
10.2.2 Worst Feature
Although all consultees who completed a questionnaire were able to identify what they saw as the worst feature of the consultation, it was remarkable that most of them qualified their response by stating that the feature they had named was, in their view, unavoidable.
Included amongst the worst features of the consultation were:
- Lack of direction at the start of the process - it was rather sluggish
- Starting the process of examining the guidelines within the full reference group
- Lack of Ministerial direction on whether Best Value will become statutory
- Lack of direct senior Scottish Executive and Ministerial involvement in the consultation
- Doing "development by committee"
- Too many external reference group meetings in quick succession
- Short timeframe
- Having external constraints curtailing fuller involvement in the consultation
10.3 VIEWS ON WHETHER THE CONSULTATION REPRESENTED GENUINE PARTICIPATION AND INFLUENCE
Views were canvassed on whether the consultation overall had met the Scottish Executive's promise of genuine participation and influence. Respondents used a 10-point rating scale where 1 represented "not met at all" and 10 represented "totally met".
The average rating was 7.6 with a range of 6 to 9. This rating was higher than those generated by other questions and was supported by several comments. Many consultees highlighted their view that the final outputs would reflect the views of the participants. One participant remarked that although the Scottish Executive had ultimately steered the consultation process:
" I feel they have been genuinely responsive and have actively sought advice/views throughout the process".
Another commented that participation latterly in the external reference group had felt like a " very democratic process".
Related to the issues of participation and influence, participants were asked what they considered to be their most significant input to the consultation process. Responses to this question reflected views on individual impact and longer term influence. Participants provided examples of their personal influence which demonstrated their perception that they had a valuable input to make and had been empowered to make this. Their responses fell into the following categories:
- Improved the quality of the outcome of the consultation
- Able to represent the position of other similar bodies and ensure their perspective taken into account
- Ensured that the outcome would be practical, simple and useful
- Used their extensive experience of the issues and the context
10.4 CHANGES IN VIEWS BETWEEN THE START AND THE END OF THE CONSULTATION
Another indicator of longer term impact of the consultation on participants is evidence that their involvement effected a change in their view between the start and end of their participation.
Of the 8 external reference group members who responded, 5 reported a change in view during the consultation process. In particular, they reported a shift from a relatively cynical view of the usefulness of the consultation, to a confidence in the process and satisfaction that practical outcomes would result. One view was that the process had been more inclusive than had been expected.
10.5 LONGER TERM GAINS FROM THE CONSULTATION
External reference group members were asked what, anything, their organisation had gained from the consultation and also what, if anything, they had gained personally from their participation.
10.5.1 Organisational Gains
All respondents stated that their organisation had gained from their participation in the consultation. A recurring theme was that the organisation as a whole was now better informed about Best Value and indeed about wider public sector debate. Another theme was that their organisation had formed new networks on account of their participation. Finally, one consultee remarked that their organisation stood to gain from the useful Best Value products which would emerge over the year.
10.5.2 Personal Gains
All respondents reported gaining personally from their involvement in the consultation. The most common gain identified was the new networks and contacts established between individuals. In addition, one respondent reported gaining new friendships.
Other gains included an increase in knowledge of Best Value in a wider context, and gaining an appreciation of the perspectives of others on this. One participant felt that they had gained from witnessing the challenges of committee working in practice.
10.6 MOST IMPORTANT BENEFIT TO EMERGE
Consultees were asked to identify the single most important benefit to emerge from the consultation. Interestingly, many respondents could not limit themselves to one item but documented several benefits. Responses ranged from personal benefits to longer term common gains.
Included in the responses were:
- Ensuring a useful product to assist public sector organisations
- A longer-term increase in good performance management across the public sector
- Ongoing new networks
- Greater insights into public policy
- Greater understanding of Best Value issues
- Buy-in from the public sector
10.7 UNEXPECTED BENEFITS/DRAWBACKS
Very little response was generated from asking external reference group members whether the consultation had produced any unexpected benefits or drawbacks. Only 2 substantive points were made. Firstly, 2 respondents commented that the drain on their time had been more than they had expected. Secondly, one consultee remarked on their pleasant surprise that the reference group was willing to meet in a variety of locations, with this reducing their overall travel time.
10.8 SUMMARY OF KEY POINTS
- The extent to which longer term outcomes could be identified was curtailed by the timing of the evaluation. A follow-up investigation would be required to ascertain longer term outcomes of the consultation which drew to a close several months before the publication of the Guidance in May 2006.
- It was possible, however, to identify some of the wider and possibly longer term benefits of the exercise for organisations and those involved from the responses to the external reference group questionnaire.
- Consultees identified a variety of best features of the consultation which suggested longer term benefits such as the willingness of participants to engage with future consultation exercises.
- Consultees anticipated that useful tools would emerge from the consultation process which they had helped to shape.
- Although many "worst features" of the consultation process were also identified by participants, many of these were seen as inevitable, eg their external commitments curtailing their fuller involvement in the consultation.
- Overall, external reference group members were consistent in rating the exercise highly on genuine participation and influence. Participants were able to provide examples of their personal input, demonstrating perceptions of personal empowerment and involvement.
- Most of those who responded reported a change in their view between the start and the end of the consultation process, indicating longer term impacts at an individual level. A recurring response was to report a shift from a relatively cynical view of the usefulness of the consultation to one of confidence in the process and satisfaction that useful outcomes would result.
- All respondents considered that both their organisation and they themselves personally had gained from their taking part in the consultation. In particular, their engagement had enabled new and valuable networks to form and allow them to appreciate the relative perspectives of other public bodies.
« Previous | Contents | Next »