The Scottish Government

Listen

Main Findings

• Eleven baseline indicators could be derived from 2 Executive data sources: the Direct Funding Database and the Consultation Registration and Evaluation System;
• These covered 7 of the Compact's 11 Activity Areas and measured such things as Consultation, Partnership, Policy Proofing, Mutual Understanding and Confidence;
• Indicators to cover the remaining 4 Activity Areas (Awareness, Use, Commitment and Co-operation) were derived from surveys of government and the voluntary sector;
• Awareness of the Compact amongst the voluntary sector was low, with a third of respondents (34%) being aware of the Compact. The larger the size of the voluntary organisation then the greater the likelihood that it had heard of the Compact;
• There was a greater level of awareness of the Compact in government (72%);
• About a third of government respondents had a member of staff who was the Compact contact point and 17% had a Compact dissemination strategy. The figures for the sector were far lower (13% and 4% respectively);
• 38% of government respondents had made use of the Compact in the last 18 months, as had 11% of the sector;
• Of those who had made use of the Compact the majority felt that this had been "very" or "quite" useful; and
• A majority of government respondents felt that this use had resulted in the quality of service provided being "better". A lower percentage of the sector (43%) held this view. However, these figures should be treated with caution due to the small absolute number of respondents.

Non-Survey Baseline Indicators

A number of data sources were analysed to see if they were capable of generating data suitable for Compact monitoring. The 2 that were most suitable were the Executive's Direct Funding Database (which gives details of grant support to voluntary organisations) and the Consultation Registration and Evaluation System. The System became operational in December 2003 and collects information on consultations undertaken by the Executive. An earlier study for the Executive (undertaken by GEN) had identified 11 key factors (or Activity Areas) that it was felt a monitoring and evaluation framework for the Compact needed to capture. These 2 data sources were felt to cover 7 of these.

In addition the Executive were in the process of setting up other systems that, once operational, could generate data that would be useful for monitoring purposes. The main one was a Compliance System that is to record, and perhaps resolve, grievances relating to non-compliance with Compact principles. Once operational the data it generates needs to be fed into any monitoring system.

Survey Based Indicators

Indicators to cover the remaining 4 Activity Areas were generated through surveys of government and the sector. In devising the surveys it was felt that the focus should be on Compact outcomes, rather than processes and outputs. It was also felt that there should only be a limited number of questions asking directly about the Compact. Not only was it felt that such questions would generate a poor response but they would also emphasise outputs, rather than the all important outcomes.

Two questionnaires were drafted. As far as possible these were symmetrical: that is they were asking identical, or similar, questions of government and of the sector. Each contained 16 questions and covered:-

• Awareness of the Compact and of the systems set up to disseminate information about it;
• The extent to which the Compact had been used and, if used, how useful this was;
• The intended outcomes of Compact implementation, looking at perceptions of such things as mutual recognition and partnership working; and
• Perceptions of co-operation between government and the sector.

The Survey Responses

Questionnaires were sent to a representative sample of the voluntary sector in Scotland, derived from the Scottish Council for Voluntary Organisations (SCVO) database and all Executive departments, agencies and NDPBs. A total of 1,023 were distributed and 287 (28%) responses received.

Awareness of the Compact and the associated Good Practice Guides was greater in government than in the sector. The larger voluntary organisations were more likely to have heard of the Compact. The likelihood of an organisation having a Compact staff contact point, and an implementation strategy, was greater in government than in the sector. Although limited numbers of respondents had made use of the Compact, those that had found it to be useful.

Respondents were asked to give a view on a number of statements that looked at the relationship between government and the sector. A significant number of sector respondents "neither agreed nor disagreed" with the statements or "did not know". This seems likely to reflect both a lack of awareness of the Compact and limited interaction with central government and its agencies. A fifth of respondents did not feel that there was collaborative working and that they were not provided with opportunities for policy development by government.

The Government respondents were generally more positive, around half "agreeing" with each of the relationship statements: for example feeling that the sector promotes partnership working with government and that it recognised the constraints that government works under.

The Focus Groups

Three focus groups were held in contrasting areas (Arbroath, Glasgow and Inverary). These explored issues of awareness and impact in greater detail. Before being contacted by GEN, the level of awareness of the Compact was generally limited. The exception was Arbroath where a local compact had been drawn up which had resulted in participants becoming aware of the National Compact.

A strong theme to emerge was the relevance of the National Compact to local groups. Most interfaced directly with local councils. It was felt that, whilst the principles of the National Compact were laudable, they needed to be interpreted locally.

Attendees had concerns over such things as partnership working and consultation. Given that these are covered by compact protocols it would seem that establishing local compacts would meet a need from the sector.

Recommendations

A number of Recommendations arising from the research were made. These included:-

• The Executive's Consultation, Registration and Evaluation system needs to be used to produce the baseline indicators;
• The baseline survey should be repeated in 18 months time, using the same questions and methodology;
• Awareness of the Compact needs to be raised amongst smaller voluntary sector organisations, government agencies and NDPBs; and
• The Executive should promote the establishment of local compacts.

Research Methods

The study involved:

• Desk research and analysis of the changes to the Compact between the 1998 and 2003 versions;
• Analysis of the suitability of various data sources for monitoring purposes;
• A postal/email survey of 700 voluntary organisations;
• A postal/email survey of 323 Executive Departments, agencies and NDPBs; and
• Convening 3 focus groups.