The Scottish Government

Listen

OVERVIEW

This is the first time the Scottish Executive (SE) has conducted a survey of its stakeholders. Looking at the findings overall the Executive's key stakeholders have delivered a generally positive assessment of the way the Executive engages with them.

The majority (72%) are satisfied with their dealings with the SE in the previous year and over 40% would speak highly of the SE. Two thirds feel the Executive's approach to working with other organisations has got much better since devolution. Looking at stakeholders' experiences of working with an Executive department, the SE is performing well on:

• being committed to achieving its objectives
• listening to others
• being focused on delivery
• understanding stakeholder organisations

Encouragingly, the SE departments were perceived to be performing well on virtually all the 'critical success factors' measuring SE performance on the process of stakeholder engagement. Areas of particular strength were:

• having staff who treat stakeholders with courtesy and respect
• having staff who are easy to contact
• providing accurate and reliable information

However, there are also significant lessons to be drawn from the survey. Across each of the survey measures, there is an important minority of respondents who are dissatisfied with their experience of dealing with the Executive (11%) and 19% state they would speak critically of the SE. There are some stakeholders (15%) who feel that since devolution there has been no change in the Executive's approach to working with other organisations and 8% feel the approach has got worse.

A lack of joined-up working is of concern to many of the stakeholders and over a third believe the SE is unclear about its priorities. Many stakeholders believe the SE is less good at being innovative and taking risks, limiting staff churn, and informing stakeholders of how its thinking is developing. However, willingness to innovate/take risks and staff churn were rated as the least important of the 'critical success factors'. Looking at the perceived importance and rated performance, some key areas for improvement can be identified:

• understanding what stakeholders are trying to achieve
• understanding constraints facing stakeholders
• involving stakeholders at an early stage in developing policy
• being clear about how much is open to change
• openly sharing information
• keeping stakeholders informed about how thinking is developing
• promoting a consistent approach to dealing with stakeholder organisations

INTRODUCTION

MORI Scotland was commissioned by the Office of Chief Researcher (OCR) on behalf of the Permanent Secretary to undertake a comprehensive, corporate-level, survey of the Executive's stakeholders to:

• identify key aspects of stakeholder relationships with the SE
• demonstrate how stakeholders engage with the SE and what stakeholders feel about these relationships
• assess which critical success factors are considered most important by stakeholders and how SE performance rates against them

Between 9th September and 8th October 2004, MORI conducted an on-line and telephone survey with stakeholders ¹ across all sectors, based on lists provided by SE departments. MORI attempted to contact 1,293 stakeholders and responses were received from a total of 824 stakeholders. The response rate of 64% was high for a survey of this kind. The stakeholders surveyed included non-departmental public bodies (NDPBs), local government, voluntary groups, registered charities, private sector, central government departments and academic institutions. This paper also refers to some benchmarking findings (conducted by ORC International) comparing results from this survey with those from customer satisfaction surveys conducted by other government departments and agencies.

The stakeholder organisations ranged in size from less than 10 people to over 500. Almost half (49%) stated that their organisation delivers services on behalf of the Executive; while a large number advises the Executive (36%) or lobbies on specific issues (30%). Three quarters said that their organisation was funded wholly or in part by the Executive (23% and 52% respectively). 41% of the respondents had been with their organisation for less than 5 years, while 29% had been there for 5-10 years and 30% for more than 10 years.

GENERAL IMPRESSIONS OF THE SE

Since Devolution

The majority (64%) of stakeholders felt that the Executive's approach to working with and involving other organisations has improved since devolution (fig. 1). However, 15% believe there has been no change and 8% think it has got worse.

Three types of stakeholder were significantly more likely than other groups to say that the SE has got better at engaging with stakeholders, these were:

• voluntary organisations, interest groups and registered charities
• local authorities
• academics

Figure 1: Perceived progress since Devolution

Satisfaction with Dealings in the Previous 12 Months

The majority (72%) are satisfied with the dealings they have had with the SE in the previous 12 months, but 11% are dissatisfied and a further 14% are neutral. Satisfaction did not vary according to the main method of communication with the SE (telephone, meetings or email etc). The SE satisfaction score is lower than the average level of satisfaction (88%) for other government departments and agencies benchmarked.

Advocacy

The survey sought to assess the extent to which stakeholders went further than being 'satisfied' and would act as 'advocates' for the SE, by asking respondents the extent to which they are willing to speak highly of the SE with their peers.

Almost 1 in 5 of respondents were 'advocates' of the SE, stating that they would speak highly of the organisation without being prompted to comment. A further 22% said that they would speak highly of the SE if asked about it. Meanwhile, 14% said they would be critical if asked and 5% said they would be critical without being asked.

EXPERIENCES OF WORKING WITH THE SE

Almost half of the respondents dealt with the Executive on a weekly or daily basis (47%), and 33% on a monthly basis. The stakeholders reported that they communicate with the SE predominantly by e-mail (87%), closely followed by telephone (73%) and face-to-face meetings (69%). A large proportion of respondents had frequent contact with the SE for: seeking information or advice; responding to a consultation over a specific policy; receiving information; or providing advice to the SE. Receiving information was the most common type of contact, although almost all of the organisations had contact for each of the reasons at some time.

To gauge respondents' views on specific experiences of working with the SE, stakeholders were asked to comment on their work with just one of the departments with whom they had had contact. Respondents were asked if various statements matched their impressions of the department. Of the 6 'positive' statements presented to stakeholders, there was most agreement that the department is committed to achieving objectives and least agreement that it is joined-up. The levels of agreement were (in descending order):

• committed to achieving objectives (77%)
• listens to others (67%)
• focused on delivery (62%)
• understands my [stakeholder] organisation (61%)
• has sensible policies (57%)
• joined-up (25%)

Of these findings, SE performance on understanding stakeholder organisations can be benchmarked approximately with other government departments and agencies who have asked their customers how valued they feel and the level of interest staff show in them - the average satisfaction is 87%, which is greater than the 61% of SE stakeholders who feel the SE understands them.

Of the 5 'negative' statements presented to stakeholders, there was most agreement that the department is unclear about priorities and least agreement that the SE is remote and impersonal. The levels of agreement were (in descending order):

• unclear about priorities (35%)
• reluctant to innovate (23%)
• lacks strong leadership (20%)
• out of touch (18%)
• remote and impersonal (13%)

It was found that respondents who were generally dissatisfied with the SE had a greater propensity for scoring their key department negatively (see table 1).

Table 1: Perceived characteristics of departments by overall satisfaction with their engagement with Executive in past 12 months

CRITICAL SUCCESS FACTORS

Much of the assessment of departments will reflect the outcomes for stakeholders and the extent to which policies, funding and delivery mechanisms adopted by the SE reflect the priorities, aspirations and needs of different stakeholder groups. However, while it might be inevitable that some are disappointed by outcomes, there should be more unanimity about the Executive's performance in terms of the process of stakeholder engagement. The SE has identified a number of 'critical success factors' against which it wishes to measure its performance in terms of process. For each of these factors, respondents were asked to indicate first, how important it was to them in their dealings with their selected department, and, second how they felt the department they have most contact with is currently performing against that factor.

Perceived Importance of the Critical Success Factors

Each of the success factors was regarded as important by almost all respondents. However, there was some variation in the level of importance attached to each. The most highly valued factors related to the Executive's general approach to engagement and the extent to which it understands and seeks to assist stakeholder organisations. Factors seen to be least important were staff churn and being innovative and willing to take risks. The relative importance of each factor is shown in figure 2. There was very little variation according to stakeholders' key department - the prioritising of the different factors was found to be consistent across SE departments.

Figure 2: Relative importance of the critical success factors

The level of importance given by each respondent was given a score where 4 is essential, 3 is very important etc. The mean of the scores given by all respondents was then converted to a z-score to show the relative position of each success factor in relation to the average of the scores given to all the factors.

Perceived Performance on Key Critical Success Factors

The stakeholders were asked to rate the performance of the department with which they have most dealings. The departments were seen to be performing well on virtually all the success factors (table 2). Areas of particular strength were:

• having staff who treat stakeholders with courtesy and respect (91% rated good to excellent)
• having staff who are easy to contact (83% rated good to excellent)
• providing accurate and reliable information (74% rated good to excellent)

Departments' willingness to engage with stakeholders and understanding of stakeholder organisations - in terms of both what these organisations are trying to achieve and the constraints they face - were also highly rated.

Areas where departments were seen to be performing less well relate to:

• their willingness to innovate and take risks;
• their ability to limit staff movement; and
• informing stakeholders of both how their thinking is developing and how much of the policy is open to change.

However, as previously noted, the first two of these (innovation/risks and limiting staff movement) were rated the least important factors by respondents.

Table 2: Performance of SE departments on the 'critical success factors'

I'm now going to read out the same list of attributes and I'd like you to tell me how you would rate the performance of the [department] in your dealings with it. Could you tell me whether your experience has been excellent, very good, good, average, poor or very poor?

There were no differences by department identified. The broad pattern, of performance on factors related to staff professionalism being rated highly, was found to be consistent across departments. However, it was found that those stakeholders who were generally dissatisfied with the Executive tended to give their key department poorer ratings on the critical success factors than those who were satisfied. Four of the factors were found to be particularly related to satisfaction ratings (contributing to 37% of overall satisfaction of relationships with the SE) these were:

• understanding what stakeholders are trying to achieve
• understanding the constraints facing stakeholders
• providing accurate and reliable information
• being willing to take stakeholders' views on board

It is possible to benchmark SE performance on 4 of these critical success factors with other government departments and agencies. Overall these reveal that SE performance is similarly highly rated as other government departments and agencies, but that the SE falls below the average:

• an average of 88% expressed satisfaction with the information received from other government departments and agencies, compared to 74% of SE stakeholders
• an average of 86% expressed satisfaction with ease of contacting staff at other government departments and agencies, compared to 83% of SE stakeholders
• an average of 96% expressed satisfaction with the politeness/courtesy of staff at other government departments and agencies, compared to 91% of SE stakeholders
• an average of 83% expressed satisfaction with the prompt handling of queries, compared to 73% of SE stakeholders

Areas for Improvement

When analysed together, the perceived importance and the actual performance of the Executive identifies areas of correlation between performance and expectations. Seven areas are identified where performance is at or below average compared with the importance attributed to the characteristic. Hence, these 7 characteristics detailed below, are those identified by stakeholders as areas for improvement in the Executive's outward focus.

Seven areas for improvement

• Understands what stakeholders are trying to achieve - 2nd most important attribute, but performance is just above average
• Understands constraints facing stakeholders - 3rd most important attribute, but performance below average
• Involves you at an early stage in developing policy - 4th most important attribute, but performance below average
• Is clear about how much is open to change - importance above average, performance below average
• Openly shares information - importance above average, performance below average
• Keeps you informed about how its thinking is developing - importance is average but performance is below average
• Promotes consistent approach to dealing with organisations - both importance and performance are below average

CONCLUSION

This survey of stakeholders, along with the internal staff surveys, is a valuable tool to help gauge the strengths of the Executive and the challenges faced. It also provides a baseline against which future progress can be measured. The stakeholder survey demonstrates stakeholders' fairly high satisfaction levels with the Executive, and the process of engagement between the Executive and stakeholders is assessed positively. However, there is a minority of respondents who are dissatisfied with the Executive, especially with the process of engagement. The stakeholder survey also illustrates areas for driving the change process forward according to the critical success factors identified by the SE. Although the SE is performing well on these factors, stakeholders identified seven areas where the SE might concentrate its efforts to improve the scoring to 'very good' or even 'excellent' ratings.

^Footnote

¹ Stakeholders were defined as individuals working for organisations which were involved in joint-working with the SE on a regular basis. Individual members of the public and people who have experienced very limited or one-off communication were excluded.