The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER 6: PROCESSES

One problem with attempting any guide to participatory techniques is that similar approaches delivered differently are likely to produce varying levels of success by virtue of the different delivery processed involved. Chapter 2 described the processes adopted for delivering the Best Value consultation approaches. Here, aspects of these processes are evaluated to assess their level of success. The data collected can inform 6 key process themes of representation, attendance levels, communication of aims, organisation of events, facilitation and resolution of conflict. In addition, participants raised a number of process issues in their responses to the stakeholder seminar questionnaire and the external reference group questionnaire. These themes and issues are examined below.

6.1 REPRESENTATION

The need for consultation participants to be "representative" of their wider organisations was raised as important by members of the consultation team in their group interview. They expressed the need to get a sense of views from " across all sectors" to find out " to what extent the organisations were in any way engaged with the agenda already". Informal interviewees were selected to represent each Ministerial portfolio and invites to the stakeholder seminars were pitched at a senior official level to attract representation from senior management.

Implicit in the team's decision-making was that participants in the consultation process would be representative of their wider bodies and would engage in dialogue with their organisations about the consultation and the issues under discussion. The evaluation cast light on the extent to which this representative function manifested in practice.

Interestingly, the consultation team highlighted their concerns regarding the degree of representation displayed by participants. One member commented:

" It did become clear …that people were saying that they were speaking quite often as the champion (of Best Value ) within their organisation but they did not necessarily have the buy-in of the Board or their Chief Executive to the agenda. So the person may not be speaking with a representative, corporate view of how their body was behaving."

Differing views were presented by participants involved in the informal consultation. Whereas one consultee raised their concern (unsolicited) that they may not have represented the views of the entire organisation and that perhaps they should have undertaken a consultation exercise within their body to ascertain internal views, another spoke of presenting not just their own body's opinions, but providing a feel for the general mood across public bodies.

Although not specifically asked about the issue of representation, a few respondents to the stakeholder seminar questionnaire considered that the events should have included more representation from public body sponsor departments (although the policy team reported that they had taken a decision to deliberately limit this representation, to remove any possibility of perceived Scottish Executive "dominance" of the discussion.) . A small number of other consultees were disappointed at what they perceived to be the paucity in attendance of senior Scottish Executive officials representing related policy initiatives. One external reference group member cited " lack of evidence of senior Scottish Executive/Ministerial involvement" in the consultation as the poorest feature of the consultation process. Another remarked that it may have been useful to have had " coal face" officials represented in the consultation in addition to " corporate/policy type people".

External reference group members were asked to comment on the degree to which they thought that they were genuinely representing the views of their organisation. They provided a score on a scale of 1 to 10 where 1 had the value of "not at all representative of their organisation" and 10 stood for "totally provided their organisation's view and received its full support". Scores ranged from 3 to 10 with an average of 7 indicating that representation levels varied significantly across participants.

6.2 ATTENDANCE LEVELS

A relatively tangible aspect of evaluation evidence on the effectiveness of the process of the consultation are the attendance levels witnessed at each of the consultation stages and an examination of reasons given for failure to attend.

In relation to the informal interviews, the consultation team reported no reluctance to take part and all interviews went as planned.

In their analysis report of the consultation, the Best Value and Performance Team state that over 50% of the public bodies invited to attend the stakeholder seminars did so. For the purposes of the evaluation, contact details were obtained from the consultation team for all potential participants who had been expected to attend a seminar but had failed to turn up or had indicated that they would not be attending the seminar to which they were invited. Twenty-seven were contacted by email to seek reasons for their non-attendance.

Fourteen responses (52% response rate) were received which revealed a range of reasons for non-attendance including:

• Difficulties for smaller organisations to fit additional events into their diaries
• Competing priorities on the day
• Feel that have already contributed to previous written consultation on this topic
• Illness on the day
• Did not think that there would be benefit to be gained from attending
• The lack of pre-event briefing was off-putting and there was not enough information on which to base a decision on whether to attend

The external reference group had met 5 times at the time of writing. Apologies for absence were noted in the minutes of each meeting and indicate an overall attendance level at these meetings of 72%.

Thus, of the various elements of the Best Value consultation, the smaller group dialogues between Scottish Executive officials and selected stakeholders demonstrated a higher engagement rate than the larger seminar events. The attendance rates produced for this evaluation provide useful process indicators for future benchmarking purposes.

6.3 COMMUNICATION OF AIMS

The Best Value and Performance policy team set firm aims for the consultation prior to commencing the consultation process. The aims were re-iterated throughout the consultation process in a variety of ways including initial briefing material, Powerpoint presentations at the stakeholder seminar, printed materials for the external reference group and indeed, in the stakeholder seminar participant questionnaire.

To help to identify how well these methods communicated the aims of the consultation to participants, the final questionnaire for the external reference group posed the questions:

• To what extent are you aware of the intended outcome and aims (of the consultation)?
• How were the intended outcome and aims communicated to you?

Using a 10 point scale with 1 representing total lack of awareness of the intended outcome and aims and 10 representing total awareness, participants at the final stage of consultation rated their awareness of aims at an average of 8.1. The range was between 7 and 10. Whilst this relatively high rating is encouraging, it is likely that external reference group respondents will have had more exposure to the emphasis on aims and purpose of the consultation than consultees who took part in the earlier stages of the consultation only and a different result may have emerged had perhaps the same question been asked at the stakeholder seminars.

Four key ways in which the intended outcome and aims were communicated were noted by respondents to the external reference group questionnaire. These are displayed in Table 1 overleaf.

Table 1: Ways in which external reference group members reported that intended outcome and aims had been communicated to them

Although numbers are small, the findings nonetheless are useful and are perhaps surprising in demonstrating that some participants reached the stage of the final consultation element - the external reference group before fully becoming aware of the purpose of the consultation. Overall, the one-to-one dialogue and small group settings appeared to be more effective vehicles for communicating strategic aims than the larger, less intimate stakeholder seminars.

6.4 ORGANISATION OF EVENTS

One aspect of process is the logistical dimension of the consultation events - how easy were they to attend, were there any access issues? The previous Chapter outlined the decision taken to hold events at existing Scottish Executive venues. The views of participants attending the stakeholder seminars were probed by asking them to rate the venue chosen for the seminar in terms of its convenience of access and ease of transport. All of the seminars had been held in either Victoria Quay in Edinburgh or Meridian Court in Glasgow. A ten-point rating scale was provided with 1 representing "very poor" and 10 representing "excellent".

Overall, the average rating provided for Meridian Court (7.8) was slightly higher than that for Victoria Quay (7.3). Only 2 substantive comments were made in relation to convenience of venue. Firstly, participants attending one of the Meridian Court seminars remarked that as most of the delegates had travelled from Edinburgh, perhaps the event should have been held in Edinburgh. One comment made regarding the Victoria Quay location was that it appeared to be difficult to access by public transport.

Two further issues relating to the organisation of events were raised by an external reference group member in their questionnaire response. Firstly, the holding of several group meetings in quick succession was seen as problematic and had resulted in apologies being sent by the participant for some of these. Secondly, it was suggested that by staging such an intense grouping of meetings, the timing could clash with significant resource intensive activities amongst the consultee bodies and result in reduced input from them.

6.5 FACILITATION

One key process issue which can influence the success or otherwise of a technique is the effectiveness of the facilitator charged with delivering the approach. Scottish Executive trained facilitators were deployed for each of the stakeholder seminars. Participants attending these seminars were asked to rate the quality of the facilitation in the plenary and the break-out group they attended respectively. A 10-point rating scale was used with 1 representing "very poor" and 10 representing "excellent".

Overall, participants rated the quality of facilitation at the plenary sessions at an average of 7.2, with facilitation in break-out groups averaging slightly higher at 7.4. It is of note that these average ratings compared well with the other ratings provided by participants, suggesting that in general, people were happy with the standard of facilitation provided.

Very few qualifying comments were recorded to support the ratings made. Those comments that were annotated suggest that the following aspects of facilitation had been particularly welcomed by participants:

• Facilitators who encouraged participation by all
• Those who listened as well as contributed/led
• Facilitators who kept discussion going
• Those who kept discussion focused
• Those who kept people interested/awake (!)

6.6 RESOLUTION OF CONFLICT

This element of process is included here as an often overlooked aspect of consultation processes in general. Various commentators have raised resolution of conflicting views as a challenging feature of the written consultation which, by its nature, can generate relatively entrenched and opposing views. By the same token, a consultation process which has been specifically designed to offer face-to-face interaction and encourage an " open dialogue" will encounter opposing views head-on and presumably will require a robust approach to handling such differing perspectives.

The way in which conflicting views were handled in the current consultation was examined by means of interview with the consultation team, evaluator observation at the consultation events and, indirectly, by asking external reference group participants to what extent they felt the final outcome would reflect their input.

6.6.1 Preparation for Conflicting Views

The consultation team's approach to handling conflicts included much preparation and awareness raising amongst themselves of the issues which may produce opposing views. For example, they invested time in bilateral discussions in the informal consultation stage, from which they were able to identify potentially contentious issues. These were logged systematically onto a spreadsheet and the information shared between the team. Their approach was one of listening and " fact finding" at this early stage in order to " shake out the issues" on Best Value which were of importance to the public service bodies. The consultation team felt that this approach enabled them to start to attract "buy-in" to the exercise and their approach to the consultation.

Such preparatory work had the effect of alerting the team to the main issues on which views may clash and enabled them to be ready for sensitive handling of such conflicts should they arise in the subsequent stages.

A significant comment made by one of the informal consultees suggested that this early work by the consultation team had indeed averted a degree of conflict:

" The early stage felt like a warm up for later - it made people less defensive and more open as the consultation progressed"

6.6.2 Evaluator's Observation

Within the stakeholder seminars, a few recurring themes and areas of conflict did arise (for example, conflicting views on what Best Value constituted; on how Best Value would be audited; on issues of proportionality) and were addressed in a variety of ways by the consultation and facilitation teams. In the main, 3 different approaches to conflict resolution were observed:

• Prescriptive approach - for example, the consultation team outlining firmly what was within scope for discussion and what had already been decided on
• Permitting all views be aired without any intervention and/or attempt to bring views towards a resolution
• Identifying what appeared to be the most important issues of conflict (eg issues of audit) and explicitly tabling them for future discussion in the consultation process

In addition, the technique of inviting topics for discussion at the stakeholder seminars and holding a group vote to identify the burning issues to follow up pre-empted, to a great extent, possible conflicts and disgruntlement over the topics for discussion at the stakeholder seminars.

6.6.3 External Reference Group Views

The posing of a question on the extent to which the final outcome of the consultation was likely to reflect the individual input of respective external reference group members was intended to flush out any residual dissatisfaction that the process perhaps had not allowed for conflicts to be resolved and individual contributions to be incorporated. The responses indicated however, that in general consultees were content that the final outcome would be reflective of their input. On a rating scale of 1 to 10 with 1 representing "will not reflect my input at all" and 10 representing "my input will be fully reflected", the average respondent rating was 7.0 with a range of between 5 and 9. One consultee, however, highlighted their concern that the terms of reference for the consultation were overly restrictive, and for them, the outcome of the exercise would not promote better delivery of Best Value.

6.7 OTHER PROCESS ISSUES RAISED

The questionnaires distributed to participants at the stakeholder seminars and the external reference group provided the opportunity for consultees to document their views on other consultation process issues. A summary of those they highlighted is presented below.

6.7.1 Opportunity for Networking

One recurring theme to emerge was the opportunity that the process provided for networking between public bodies. In their analysis of the Best Value consultation (Loveridge, 2006), the consultation team stated:

" The policy team recognised that encouraging PSOs (public service organisations ) to speak to one another was equally as important as them speaking to the Scottish Executive. It was hoped that this approach would maximise the impact of the consultation by establishing a dialogue with PSOs, allowing the policy to develop in a more dynamic fashion" (p6)

Many consultees felt that they had benefited from the networking opportunities made available by the consultation and provided comments such as:

" very helpful…. I found lots of commonality with others"

" good to hear other points of view"

" enabled me to meet people I may not have had the chance to otherwise"

" I did find the networking opportunity of the seminar useful. It may be good for facilitating future benchmarking"

Indeed, a few of the external reference group members stated that in their opinion, the single best feature of the consultation process had been the networking opportunities it provided. Particularly appreciated was:

" the ability to work with colleagues and learn about other approaches and sectors"

6.7.2 Inclusion at Early Stage of Process

A few of those consultees who had been involved in the consultation from the informal stage appreciated what they saw as the benefits of inclusion from the early stages of the process:

" It was good for me to be in the loop at an early stage. It gave me confidence going into the later stages of the consultation. We tend to have a sense of isolation from the Scottish Executive and this helped to bridge that gap."

6.7.3 Process Flow between Consultation Elements

Another comment related to the perception that outcomes from each stage of the consultation fed into and informed the subsequent stages in a positive manner:

" Going from the one-to-one then onto the informal consultation group meeting prior to the seminars was good - it provided a link from one (part of the process) to the next… it flowed"

6.7.4 Leadership and Management

There were mixed views on whether the consultation process leadership style was conducive to effective consulting. This topic has been included here as an element affecting the process of the overall consultation.

Participants who commented tended to perceive the leadership style adopted by the consultation team to be "open" and "listening". However, for a few consultees, the drawback of this style was what they saw to be a " lack of direction", a " sluggish process" with one consultee suggesting that:

" if this was led by someone from, say, Quality Scotland, then there would be better leadership"

It was proposed that the consultation process could have been improved by treating it as a project with project management techniques adopted.

6.7.5 General Tenor

Finally, a few consultees summed up what they saw as the overall tenor of the consultation process:

" the overall process so far has been very positive. We have been very encouraged"

" as a process, I feel that this consultation has worked well"

" People have welcomed the Scottish Executive approach…….they have said we are not experts and we want your views"

6.8 SUMMARY OF KEY POINTS

• The representativeness of participants was raised as important by the consultation team. In reality, however, the extent to which participants wholly represented their respective organisations varied.
• It was possible to identify reasons for unexpected absence from stakeholder seminars with smaller organisations reporting particular resource-related difficulties of attending.
• Overall, the smaller group dialogues between Scottish Executive officials and selected stakeholders demonstrated a higher engagement rate than the larger seminar events.
• The attendance rates produced for this evaluation provide useful process indicators for future benchmarking purposes.
• Participants rated the communication of the aims of the consultation relatively highly.
• Overall, the one-to-one dialogue and small group settings appeared to be more effective vehicles for communicating strategic aims than the larger, less intimate stakeholder seminars.
• The venues for the stakeholder events were rated relatively highly for convenience of access and ease of transport.
• Holding a series of consultation meetings in quick succession was highlighted as potentially problematic in terms of resourcing participation.
• Participants reported being content with the standard of facilitation at the stakeholder seminars.
• The consultation team handled conflicting views in several ways: by being well prepared for conflicts; by providing firm guidance on what could be debated and what was outwith scope; by allowing views to be aired but making no attempt to bring conflicting opinions to a resolution; and by tabling controversial issues for discussion at a later date.
• A recurring theme to emerge from participants was that the consultation process had provided them with valuable networking opportunities.
• Consultees involved from an early stage identified benefits from such early inclusion in the process including building their confidence, so that they could contribute effectively later on.
• There were mixed views on whether the consultation had benefited from what was seen as the open and listening approach adopted by the consultation team, or whether this had served to slow up the consultation process.

« Previous | Contents | Next »