« Previous | Contents | Next »
Listen
CHAPTER 5: INPUTS
The paucity of previous robust evaluation of consultation exercises has been commented on previously. However, even amongst those which have been conducted, the weakest element of analysis appears to have been in ascertaining the inputs which supported the consultation, and in particular, the costs. Indeed, in its literature review of "The True Costs of Participation" (2005), Involve remarks:
" While the number of large scale evaluations of participatory working in major UK public policy areas has grown in the last couple of years…and the benefits are increasingly fully articulated, the costs are not covered in any detail at all" (p8)
There are many reasons suggested in past literature to explain the difficulties associated with establishing accurate and useful financial costs of consultation. For example, it is muted that the complex issues and often intangible benefits produced by the participatory process do not lend themselves easily to a cost-benefit framework; detailed data on participation costs are not readily accessible in a form suitable for evaluators; many of the true costs of participation remain hidden, or are in an intangible form, or only become evident only over longer time scales. There also appears to have been little previous interaction between economists and those involved in running consultations. Another argument is that the rapid expansion in participatory approaches has outstripped developments in evaluation which now need to innovate and re-work to catch up.
The approaches used in this evaluation certainly do not claim to solve these universal issues and challenges facing cost-benefit evaluators. What is attempted here is the drawing together of the limited data collected by the commissioners of the consultation, and the presentation of some new data on costs generated specifically for the evaluation. This process of identifying the costs available is useful in informing where the gaps currently lie, what is likely to work in the collection of cost data and what needs to change to do this in a better way.
From interviews with the commissioners of the consultation, feedback from facilitators of the stakeholder seminars and questionnaire responses from participants, the main inputs to the Best Value consultation were:
- Planning and running the consultation
- Support from Scottish Executive colleagues
- Scottish Executive Policy Skills team
- Facilities
- Participants
- Evaluation
Each of these is examined in turn below.
5.1 PLANNING AND RUNNING THE CONSULTATION
From discussions with the Best Value and Performance Team a picture emerged of the approach used to cost the planning and running of the consultation.
The relative costs of different participatory techniques were considered by the team but in broad terms only, without detailed cost guidance to go on. Although not formally required to estimate the costs of different approaches before finalising its consultation design, the team attempted an assessment of resource and quality trade-offs between, for example, a written consultation (" less resource intensive in terms of staff time") against what was predicted to be a much greater resource drain, but likely to produce the benefits required: face-to-face methodologies.
An accurate aggregation of the consultation team's input to the planning and running of the consultation (which only finished in May 2006) is not readily available. However, based on the details that are available, the most significant staff input was a dedicated team member " more or less 100% focused on Best Value" funded by the Scottish Executive central staff budget. In addition, the team had a leader and an administrator, again almost totally attached to the Best Value consultation, and again funded by core Government monies.
Various other Scottish Executive staff inputs are also relevant. A Head of Division attended each stakeholder seminar to present, listen and to demonstrate commitment and leadership. Further administrative support was required for the stakeholder seminars in order to cover scribing and other administrative duties.
The staff input to the consultation appeared to be resource intensive, although no timesheet data was readily available (nor stipulated as required as a prerequisite to undertaking the consultation) to calculate precisely the time input of this expertise.
5.2 SUPPORT FROM SCOTTISH EXECUTIVE COLLEAGUES
The Best Value and Performance team reported much pre-planning of the consultation, and sought advice extensively from colleagues across the Scottish Executive. The involvement of these colleagues is perhaps one of the hidden costs of the consultation. Again, an accurate calculation of this expertise is not readily available, but the key bodies consulted by the team are noted here in order to indicate the extent of this "hidden" input.
Advice was sought from:
- Outward Focus Team in the Changing to Deliver Division
- Civic Participation and Consultation Research Team
- Public Bodies Unit
- Policy Skills Team
5.3 POLICY SKILLS TEAM
The most sophisticated costings produced for the consultation were those developed to inform the deliberations over whether to buy in facilitation services or whether to run with in-house expertise on this front.
All 5 stakeholder seminars were supported by a lead facilitator and 2 associate facilitators who were all existing Scottish Executive staff (although not all seminars required the services of these facilitators on the day). Costs of using internal staff for these duties were not readily available. However, the Policy Skills Team calculated the likely costs had external facilitators been commissioned to undertake this role, at £7,875 (excluding any T&S and VAT).
Set against any input of the Scottish Executive facilitators were the benefits of the learning transfer process between facilitators. Indeed, the consultation team was keen to stress that rather than, " economising", the use of internal facilitators meant " getting benefits to the organisation" (the Scottish Executive).
5.4 FACILITIES
Again, costs of the facilities used to stage the various consultation events have not been detailed. However, in every case, an existing Scottish Executive venue was used, largely in the interests of keeping venue costs down and limiting the travel and subsistence budget for the consultation team.
Lunch and refreshments were provided at all consultation events and should be considered as part of the overall inputs. Overall, £1,000 was spent on lunches and refreshments.
5.5 PARTICIPANTS
Participants were employed in various public service capacities and had to take time out to take part in the informal consultation stage and/or stakeholder seminars and/or the external reference group. In addition, many participants inputted further to assist the evaluation of the consultation.
Further preparation has been required by participants in relation to the external reference group particularly, and it is likely that most participants will have been involved in feedback and discussion related to the consultation with colleagues in their respective organisations.
Direct costs of travel and subsistence will have been incurred by all participants as they travelled to the various Government buildings (usually in Edinburgh or Glasgow) to attend consultation events.
All of these costs incurred by the participants fell to their respective organisations and were not met by the Scottish Executive.
Obviously, the input of each participant will vary depending on circumstances of their organisation, their commitment to the Best Value consultation and the stages at which they have been involved. One attempt by the evaluator to capture new data on time input was made by inserting a question on time input into the final questionnaire issued to members of the external reference group. It was considered that many members of this group may be people who had committed the most input to the consultation by virtue of having been involved from the informal stage, through to stakeholder seminars and onto to self-selecting for the external reference group.
Members of the external reference group were asked to provide a broad estimate of the number of days input they had spent (so far) contributing to the Best Value consultation process. Their reported input ranged from 3 days to 15 days to date with an average of 6 days. The total time input of the 12 members of the group from their initial involvement up until March 2006 is estimated therefore at approximately 72 days.
5.6 EVALUATION
As stated previously, a decision was taken early on by the Best Value and Performance Team to commission an external evaluation of the consultation. The evaluation ran alongside the consultation process, from the stage of stakeholder seminars until the latter stages of the external reference group.
The budget for the external evaluation was £13,500 (excluding VAT) and this constitutes another input to the overall consultation.
5.7 SUMMARY OF KEY POINTS
- It has been argued that the weakest element of previous evaluations of consultations has been in ascertaining the inputs, and in particular, the costs.
- The current evaluation has used costs which were readily available and added new data on time input generated from questionnaires. However, significant gaps in cost data remain.
- Inputs to the consultation included: planning and running the consultation; support from Scottish Executive colleagues; Policy Skills Team contribution; facilities; participants; and the evaluation.
- The staff input to the consultation appeared to be resource intensive, although no timesheet data was readily available to calculate precisely the overall input of this expertise.
- The involvement of a range of Scottish Executive colleagues in advising on the consultation could be seen as one of the hidden costs of the exercise.
- Costs of lunches and refreshments at the consultation events amounted to £1,000.
- The input of the members of the external reference group was calculated using data provided by these participants. Based on this information, the total time input of the 12 members of the group from their initial involvement up until March 2006 was estimated at approximately 72 days.
- The external evaluation budget was £13,500 (excluding VAT) with the evaluation constituting another input to the overall consultation.
« Previous | Contents | Next »