The Scottish Government

« Previous | Contents | Next »

Listen

EXECUTIVE SUMMARY

1. This is the report of a study commissioned by the Finance & Central Services Department of the Scottish Executive to evaluate the Scotland in the Netherlands Programme, 2004. The study was conducted by SQW Limited during October 2004 to April, 2005.

2. The nature of this and similar overseas promotional events have differed in size and duration, but the higher-level objectives have remained the same throughout. Individual elements of programmes have their own, complementary objectives. The higher-level objectives have been:

• to promote Scotland, post devolution - its new politics and its new governance
• to position Scotland as a modern knowledge-based economy, keen to learn from others and to contribute; and
• to promote Scotland as a tourist and visitor destination, and as a place to live and work.

3. The decision to choose the Netherlands as the location for this "Scotland in…" programme was made within the Executive at a ministerial level. The year 2004 was the 60th anniversary of the contribution made by Scottish troops to the liberation of Dutch towns during World War Two and this presented an opportunity to develop a programme of events linking the two countries. This was supported by the Scottish International Forum. Officials in the Executive were tasked with assembling an appropriate programme of events.

4. The Scotland in the Netherlands Programme was developed and co-ordinated by the International Division of the Scottish Executive. It consisted of a diverse set of events involving participants from Scotland and the Netherlands during September to October, 2004. These events were:

1. Public lecture and research seminar on Brain Science

2. Seminar on cancer and radiotherapy (unlike the other events, held in Edinburgh)

3. "Learning journey" on drugs/substance abuse and on prostitution issues

4. Study trip and Access to Justice conference

5. Evening of Scottish Culture

6. Wee Stories Theatre Group's performance of Tam O'Shanter

7. East of Glasgow Youth Theatre and the ISH¹ Youth Theatre exchange

8. National Arts Education Network ( NAEN) study trip and seminar on culture entitlement for young people

9. Landforms Exhibition and urban regeneration study tour.

Outcomes

5. In all cases, those involved in each event believed that their activities had achieved event- level objectives. These results were not always formally set out and generally did not include the achievement of specific targets. However, the outcomes delivered have met and often exceeded the expectations of those involved:

• the Brain Science events have helped form a longer term relationship with Dutch partners, led to consideration of collaborative initiatives and more generally raised the profile of Scottish science and the university sector
• the Cancer Seminar enabled an exchange of views and information, strengthened relationships and generated new contacts. Specifically, its proceedings will contribute to a strategic report on cancer patient needs in Scotland. These benefits were achieved without the event having to take place in the Netherlands
• the learning journey on drugs/substance abuse and on prostitution issues helped develop valuable contacts and importantly helped to inform Scottish Executive thinking on these policy areas. This was primarily about learning from Dutch experience and building relationships rather than promotion of Scotland. In this respect, the initiative has been very successful
• the study trip and conference on Access to Justice helped improve understanding of the Scottish legal structure and permitted those from Scotland to learn from the Dutch in areas relevant to the strategic review of the Scottish Legal Aid system. There were fewer opportunities identified for future collaboration
• the Evening of Scottish Culture was the centrepiece of the Scotland in the Netherlands Programme. Research has demonstrated significant shifts in awareness and perceptions of Scotland among an influential audience as a result. All the research points to a well run and successful event
• among the arts-related activities, organisers also believed that objectives have been fully met. The Wee Stories Theatre Company's visit strengthened existing links with Dutch counterparts and made further appearances at Festivals more likely. The theatre exchange between the ISH street dance/theatre group and the East of Glasgow Youth Theatre ( EGYT) also strengthened organisational links, proved inspirational for the young people involved and will provide new ideas for the Scottish Arts Council's Youth Arts Strategy. The NAEN study trip has similarly strengthened relationships and will help contribute to Scottish arts policy making in the area of cultural entitlement
• the landforms exhibition and urban regeneration study trip were particularly influential in bringing together the Scottish Better Cities group and encouraged exchange of practice with Dutch counterparts on waterfront developments. It also generated ideas that will be used in future projects in Scotland. The exhibition promoting Scottish architecture attracted just fewer than 300 Dutch visitors.

6. A summary of the benefits of each event is presented in the table on the following page. There are also some important operational lessons to learn from the conduct of the Programme, many of which were found across different events. These were around increasing the opportunities for networking; building in discussion on follow up; allowing sufficient lead time in the pre-event or set-up phase; debriefing meetings for organisers post-event; and a more strategic, Programme-level approach to press and promotion issues.

7. Most events in the Programme were about developing cooperative links, exchanging practice and learning. With the exception of the Evening of Scottish Culture, there was less emphasis on the "promotion" of Scotland in the generally understood sense of the word.

Table 1: Summary of Benefits

8. The Scottish Executive allocated a total budget of £230,000 to the Scotland in the Netherlands Programme. The majority of events were small, held at relatively little cost and have strengthened relationships and generated new learning that in many cases will influence policy. The larger Evening of Scottish Culture was a cost-effective tool for engaging an audience that it would be difficult to influence through other means.

9. In summary, we are confident in concluding that at an individual project level there have been some valuable benefits and that there are sufficient benefits to justify the expenditure made. However, it could also be argued that the individual events could have been carried out independently, assuming that the funding was available. It is therefore appropriate to consider how bringing together these individual events under the Scotland in the Netherlands umbrella has changed or enhanced delivery and outcomes at a programme level.

Programme level

10. We found that there was only a limited sense of the events being part of a coherent programme and therefore, potential benefits from the synergy of this type of programme-based support may have been lost. Clearly, in arguing for the continuation of a programme approach, it is imperative to demonstrate that it delivers more than the sum of its parts.

11. The point here is not that there could have been very much more synergy between the events in the Scotland in the Netherlands Programme, but that this should be a stronger principle in designing and implementing future programmes.

12. Our views on a desirable approach to programme design, monitoring and evaluation are less concerned about achieving significantly greater levels of quantification of outcomes - this is difficult given the present broad objectives of current strategy - but rather about ensuring that design, monitoring and evaluation takes place consistently within an overarching, programme-level structure. This will help inform the design and selection of individual activities and ensure that they are aligned to a more explicit rationale and strategy.

13. We recommend that the approach used to develop similar programmes be reviewed to ensure that the events chosen work together in a more coherent form.

14. There are at present limited monitoring or measuring mechanisms in place. This may be in part because the overarching objectives of the international strategy are fairly broad, but also because many of the outcomes cannot be anticipated. This is understandable in the first few Scotland in ... programmes, but with more experience it should become easier to anticipate the types of outcome that are achievable. Each event should set out clearly how their activity will contribute to the goals of the programme. This should be monitored through a programme wide monitoring framework.

15. This evaluation piloted a framework developed by the Analytical Services Division of the Scottish Executive. It adopted a three-phase structure - pre-event, event and post-event. We found this to provide an effective basis for evaluating programmes of this type, but make two recommendations for its future use:

• for programmes with many and diverse events, consideration should be given in the pre-event phase to the level of evaluation resource appropriate to smaller scale elements
• the framework should include an explicit assessment of how opportunities for developing and exploiting "synergy" between individual events have been identified and exploited within a programme

16. While this evaluation has focused on the achievements of the Scotland in the Netherlands Programme, we feel that it raises issues that are likely to go beyond this specific initiative. We would point to the following wider issues emerging during this research:

• importance of clarity of rationale and objectives
• the significance of changes in perception
• issues of sustainability
• in-country versus Scotland based activity

17. There is further discussion of these in the final chapter of this report.

« Previous | Contents | Next »