The Scottish Government

« Previous | Contents | Next »

Listen

CHAPTER 3 DISCUSSION

This section of the briefing paper considers the learning experiences drawn from SAHPM in the policy community; practice community; and academic community-three very different environments.

"[Research is under-utilised in policy development because]…there are fundamental differences between the research and policy communities that sometimes act as barriers to effective communication between the two, such as different time frames for results; different languages for communication; different priorities for knowledge; and lack of understanding about the realities of each other's work environments."

(Nova Scotia Health Research Foundation, Knowledge Translation: A Review of the Literature, October 2003
http://www.nshrf.ca/news/reports.shtml)

An important part of SAHPM pilot success and learning was the ability of the SAHPM team to draw on experiences of challenging exchanges with external stakeholders, as well as examples of successful KT and KB. SAHPM invited frank discussions around 'them and us' on the part of policy makers and academics. As a result, the work of the team has illuminated areas of cultural misunderstanding; common prejudices and misconceptions; and a number of areas where more work is required to develop a working confidence, and relationships of trust with the academic community and other evidence providers.

Some of the issues raised in the course of the project are outlined below. It should be emphasised that both the academic and policy communities contributed views on areas for improvement with the intention of creating a platform for future positive collaboration; and the process surrounding the exchange of views on these issues was viewed as positive, constructive and productive by both communities.

3.1 The Policy Community

Perceptions of the Policy Community in Scotland

The work of the SAHPM team has invited honest and robust debate on the policy, practice and research interfaces between the academic and policy communities. These discussions revealed a number of areas where academics felt that Scottish Executive and the wider policy community could do more to improve their interactions with academic evidence providers.

In discussions, academics raised a number of issues which they suggested presented their community with difficulty. These included assertions that the policy community:

• worked to unrealistic, short term timescales
• frequently asked academics to 'dumb down' their research findings
• sometimes 'moved the goal posts' half way through a research project
• apparently ignored some research outcomes from projects
• did not always take the time to read and analyse full academic outputs
• sometimes lost interest in projects or events when policy priorities changed
• did not appear to be interested in conceptual or blue sky thinking
• usually adopted a formal style in email exchanges
• seemed unable to commit to engagement without extensive liaison with colleagues in SE and elsewhere
• requested too much detailed background information for events weeks or months in advance of events
• provided too many briefing notes for speakers and chairs at events, and appeared to be more concerned with process than content or ideas
• seemed to be repeatedly asking academics to take part in events or engagements of which academics did not understand the significance

In general, the academics identified civil servants as having a singularity of identity, background and purpose, and academics and their administrative staff (like their civil service counterparts) often lapsed into discussion of 'us' versus 'them' stereotypes when discussing bureaucratic processes and civil servants. Likewise the policy community used similar stereotypes when discussing Universities and academics.

Regarding knowledge of the role of research in the policy cycle, even those academics who represent the leading edge of 'knowledge transfer' in Scottish academic circles demonstrated limited understanding of the working environment and real politik of the civil service and/or Ministers' requirements and timescales. This situation is well recognised and documented in academic and funding council literature, and also reflects the findings of the SAHPM consultation exercise and the SAHPM feasibility study.

NHSS colleagues were also candid in their criticisms of SEHD's (and the wider civil service) apparent shortcomings and academics and wider stakeholders expressed a degree of scepticism about whether change programmes within Scottish Executive (such as Change to Deliver) would bring about a culture change within the civil service. However, the SAHPM pilot was viewed as welcome example of potential culture and practice change and widely supported by the academic and practice communities.

Perceptions of Government Analysts

It was significant that the academic and other external stakeholders engaging with SAHPM had no or very little awareness of the presence or role of government analysts-statisticians, economists or social researchers-in Scottish Executive or other Government Departments. Whilst academics regularly recognised the need to foster links and working relationships with government intermediaries, they did not seem to be aware that there was a professional group of government analysts (Social Researchers, Economists and Statisticians) whose work focused on acting as intermediaries between the academic and policy worlds.

Likewise, the ESRC publication Influencing the UK policymaking process provides a useful broad-brush overview. However, in the section 'communicating with policymakers', for example, does not mention Government Social Researchers as potential brokers, or different mechanisms for engaging with analysts in devolved administrations.

It was apparent that academics closely involved with SEHD (including those with posts funded by SEHD) routinely misinterpreted the role of analysts as individuals who, rather than bringing funding or analytical support to a steering or advisory group would be bringing 'political interference' to a project. In the process of KT/ KB it also became apparent that analysts were seen by many academics as people to bypass to obtain access to those holding funds or those with 'real influence' on the policy process.

Academic Perceptions of the Civil Service Working Environment

Academic perceptions of the civil service working environment were relevant in the context of collaborative discussion of secondments (below); proposed external research on knowledge transfer in government; and joint PhD schemes. In discussions, academics assumed that it would be possible to parachute PhD students and interested academics or external researchers into SEASDs or policy teams, and revealed their assumptions about their perception of a civil service office environment with limitless access space, computers and administrative resources; and of civil servants with short working days and low work loads.

Improving understanding through secondments?

Current academic thinking on interaction between the academic and policy worlds suggests that communication is limited because of fundamental differences in working cultures and practices, and academics with an interest in this area, for example Research Unit for Research Utilisation ( RURU), have proposed that a more interaction between academic and policy worlds would promote improved utilisation of research.

Bearing these perceptions in mind, in the course of the SAHPM project the 'simple solution' offered by many academics to address misunderstanding and misconception between policy and academic communities was to second civil service administrators into academic units or Departments to enable civil servants to understand the pressures and rhythms of academic life; and to second academics in to policy teams in SE so that SE could understand and utilise their expert knowledge. However, in all discussions on secondments, academics did not appear to be aware that they would benefit from learning about working in the civil service, or that civil servants, or government analysts would have skill sets which would be of use in their Units or Departments.

Academics were also only positive about secondments which would enable them to take an 'expert' or 'external adviser' role in a policy team, and were not enthusiastic about taking up opportunities to carry out more routine work experience with analysts or policy teams. Where academics had an interest in analysis of the knowledge transfer process they appeared to be reluctant to consider adopting a methodological strategy of 'walking a mile in a civil servants shoes'-using participation as an end in itself-and instead were more interested in gathering and recording observational 'data' which they would interpret, elsewhere, after their placement experience.

Discussions with academics seemed to suggest that in the past secondments have simply served to illuminate the differences between policy and academic worlds rather than improve knowledge transfer mechanisms, or research utilisation. However, many academics still appeared to think that secondments offering academic expertise would offer a 'quick fix' to understanding the civil service, and did not think it was necessary to gain an understanding of the complexity of the civil service through experiential learning within the system.

The work of the SAHPM team suggested that effective KT and KB between worlds, rather than in-depth ethnographic understanding of alien worlds by individuals, would be the most cost effective and efficient way to improve the utility and quality of evidence available for use in the policy cycle and meet the needs of Ministers.

Discussions between SAHPM, academics, OCR and ESRC have established that further negotiation is required at an institutional level to take forward a 'people transfer' scheme, and that there is a need to consider appropriate location and support for 'people transfer' to ensure that those transferred gain 'system awareness' and an appreciation of the real politik of KM, KT and policy development, rather than an simply an awareness of cultural and organisational difference. OCRKT team is now (autumn 2005) taking forward negotiations on this with ESRC and identifying options which would provide real and lasting widespread benefits rather than short term insights for individuals.

Speaking the same language - avoiding the potential for misinterpretation in translation; and the potential for coaching and influencing

On a practical level it would be possible to avoid cultural misunderstandings between policy and academic communities if both sides checked on use of language, and confirmed at early stages of projects what policy and academic communities mean in the use of particular terms. For example, the term 'briefing' appears to have different meaning in the policy and academic worlds: at one meeting policy administrators discussed 'briefing' on the understanding that it comprised of one or two pages of A4 containing key facts and data which could be produced by an academic expert in a matter of hours. However academics at the same meeting interpreted 'briefing' as a 12-16 page dense academic review paper which would be produced following the intensive effort of two academics over a fortnight.

At a strategic level, academics welcomed the potential to receive coaching on influencing skills; advice on how to 'start a conversation' and 'keep the conversation going' with Scottish Executive; and 'navigation training' on effective liaison routes with the Executive.

Getting the research message across: future opportunities

The work of the SAHPM team suggested that it was not clear whether academics across disciplines and institutions were familiar with the implications of devolution and the different evidence and analysis requirements of devolved administrations. Where academics were interested in policy-relevant research discussions suggested that an engagement in UK level (notably England oriented) policy research was considered to be more attractive to academics and to peer reviewers than what could be interpreted as a more (in some UK academic circles) 'provincial' Scottish focus. Some academics could not distinguish between the work and roles of the Parliament and the Executive, and it was clear that SE has further work to do in raising awareness of the policy environment post-devolution-critically on opportunities available to academics for new or innovative research; and the benefits of engagement with the devolved administration.

Throughout the SAHPM project, the team have assessed what work the policy community can take forward to improve understanding of the type of information which would be useful in the policy cycle. Critically for SE, meetings with academics revealed that the academics were not aware of the SE Partnership Agreement which drives the work of the Scottish Executive, and therefore were unable to suggest connections between their ongoing research activity and possible SE interest.

In addition, discussions with SAHPM also showed that academics had little or no awareness of:

• how to identify relevant policy teams or officials with an interest in topical issues
• how to request to face meetings, or to send material to brief officials in SEHD on ongoing research activity
• how to propose SE engagement on steering groups for academic research
• how to inform Ministers or officials of research outcomes
• how to connect EU funding opportunities to policy priorities in Scotland
• how to seek to broker funding discussions between potential funders to seek support for cross cutting research

Where academics did hold this knowledge it appeared that they were reluctant to share this knowledge brokerage 'know how' with colleagues and competitors because they felt that the access to, and engagement of, SE derived from their KB work gave them a competitive edge over other academics.

These are issues where the policy community has the potential to provide guidance and easy-to-understand 'how to' information to the academic community to improve the potential utility of current and developing research projects. It is also incumbent on the policy community to be more proactive in inviting, and following through with, the engagement the academics (in most cases) currently do not appear to know how to solicit.

3.2 The Practice Community

The work of the SAHPM team suggested that the practice world considers that much of the health research carried out in Scotland does not translate or cross over readily into NHSS to make an impact on policy development or service delivery. NHSS stakeholders recognised that most academics are passionate and committed to their research areas, but suggested that more could be done to make a connection with the NHSS managers and health care professionals who would benefit from their work. NHSS managers emphasised that there was a major gap in the research market for utilitarian, meaningful, relevant and timely work to inform management priorities in NHSS and the creation of an evidence base for NHSS managers to parallel the comprehensive evidence base available to NHSS health care professionals.

SAHPM and NHSS stakeholders also identified the potential for knowledge to be drawn from 'internal consultants' in the NHSS- NHSS employees with vast public sector experience and knowledge which has not yet been captured or shared.

From an NHSS perspective the work of the SAHPM team provided the potential to provide a solid foundation for future joint working on evidence gathering, action research and organisational learning. However, NHSS staff emphasised the pressures faced by NHSS employees, and that engagement by NHSS staff in research activity must be able to provide immediate benefits to NHSS. From an evidence provider and research funder perspective it would appear that there is a need for future positive movement in this area to tackle the current discontinuity between knowledge and evidence generated by the academic community and use of this information by NHSS.

3.3 The Academic Community

Over the course of the SAHPM project the team discussed a wide range of issues with a large number of academics in Scotland, Europe and further afield. Key issues relating to learning experiences drawn from, and for, the academic world are outlined below.

The Overarching/Overwhelming Influence of the RAE

Arguments about the influence of the Research Assessment Exercise ( RAE) are well rehearsed, and were discussed in all meetings held between SAHPM and academics. The RAE rewards peer reviewed academic excellence (in the form of published journal articles), and in the RAE there is no reward for, or recognition of the value of, work carried out for the policy community. Although the policy community, academic community, and funding councils agree that recognition should be given to policy research, it would appear unlikely that the RAE will change significantly before the middle of the 2010s. The impact of the RAE is a clear risk factor in engaging academics in policy and practice relevant work, and although academics were very keen to engage in the proposed KT and KB activity, it was flagged repeatedly that the KT and KB activity proposed was work which the RAE does not reward.

The SAHPM team also discussed with academics and funding councils processes for assessing the value or utility of policy relevant research, and it was agreed that the 'impact' and 'value' of policy research reports is contingent on a wide range of contributing factors. Critically, evidence 'does not speak for itself', and within SE for example, the role of professional research managers in Analytical Service Divisions and the engagement of policy administrators is critical to research making an impact.

In support of developing strategies to 'make an impact' with research SAHPM reviewed the potential for academics to engage in creative dissemination or 'influencing' through presentations to Ministers and policy administrators; participation in roundtable meetings; and/or informal discussions. However, academics at all stages of their careers were understandably reluctant to devote time to engaging in time-consuming dissemination processes which would draw them even further away from activity which the RAE rewards.

Perceptions of Government Funding

Academic engagement with the SAHPM team revealed a range of attitudes towards 'government funding' and the status of funding provided to the academic community. Academics showed a tendency to view government funding, even for short term policy projects, as 'grant' money, and of 'lower status' than funding provided by funding councils. In some cases similar ambivalence was shown towards funding obtained for academic research from the European Union.

It was clear that academic enthusiasm for future engagement with SE was, to some extent, based on the initial assumption by academics that the main outcome of their engagement would be 'no strings attached' grants for further academic work on their specialist area of interest. When the SAHPM team pressed academics on the need for utilitarian as well as academic output (in line with current ESRC and SHEFC guidance), several academics suggested that such an approach would encroach on academic freedom of expression and place them in a compromised situation.

When discussing future scenarios for joint working with Scottish Executive, some academics expressed reservations about accepting funding from 'the Government' ( i.e. Scottish Executive). It appeared that direct funding from Departments was perceived to be different, and in some instances viewed in a negative light, when compared to 'Government Funding' issued via funded units or independent Funding Councils.

The work of the SAHPM team in engaging with CSO funded units also suggested that there may be scope for SEHD to consider whether it would be possible for improved links to be made between academic units funded by HD and relevant policy divisions who would be able to use their research to inform policy development and service delivery.

Increasing the Range and Scope of Expert Engagement

The SAHPM team explored the potential of increasing the range and scope of expert engagement with SEHD and NHSS through a variety of possible mechanisms. Discussions with academics revealed a willingness amongst the academic community to engage at short notice on their own terms, on their speciality topic, but understandably a reluctance to turn their attention to intellectual consideration of a issue (for example a priority policy issue) which did not directly relate to their own area of academic expertise.

Mechanisms for engagement discussed included placements, exchanges and secondments. The lead in time to arrange such schemes to enable academics to contribute to policy development or implementation was estimated by academics as between 12 to 24 months. However, a number of academics provided the caveat that much more flexibility could be provided where there was a convergence of their personal research interest and SE's policy interest.

Some academics contributing to the SAHPM team's work suggested that CSO had the opportunity to test increasing the scope of expert engagement by encouraging SEHD funded academics to focus on SEHD short term policy priority issues as well as their longer term academic objectives.

Regarding academic participation in wider KB/ KT events, discussions revealed academics and other evidence providers to be discriminatory about the invitations they were willing to accept from SE to attend round table or discussion meetings or other opportunities for networking. Academics focused on networking at events with each other, but placed much less significance on networking in government circles. In particular, awareness of opportunities to influence or inform administrators or Ministers at government events appeared to be low, or perhaps of no interest, to a wide range of academics who engaged with SAHPM.

Collaboration on Evidence-for-Policy Projects

Academics had their own specific preferences for academic and policy collaborators based on positive and negative experiences of previous interactions, and in the course of the project SAHPM team discussed ground rules for future possible collaborations with SE. Conditions set by academics for collaboration included: SE would have to act as an honest, disciplined and even handed broker; the rules of engagement would have to be clearly articulated prior to project commencement; SE would have to be aware of academic rivalries, divisions and fault lines (and be explicit in how these would affect collaboration); and SE would arbitrate and monitor output to ensure that agreed dissemination strategies were adhered to (specifically in relation to single authored journal articles).

Joint Policy and Academic Knowledge Brokerage Events: expectations and outcomes

Offering the academic community the opportunity to engage closely with SE on event planning has enabled SE to learn important lessons about the priorities of academics and the extent of their operational boundaries. Lessons learned included:

• Feedback from a wider stakeholder group (and academics themselves) suggested that academics were more experienced at organising events for academics, or the voluntary sector rather than events targeting senior government administrators, Ministers, or public or private sector Chief Executives
• Management of Knowledge Transfer events which will achieve policy community objectives, and/or involve Ministers (at a UK or devolved level) require a high level of detailed co-ordination and planning with Departments and officials, and some academics and universities were reluctant to engage in the level of detailed planning required
• When considering preferences for the format of events, academics demonstrated a preference for fluid events without consideration of targeted outcomes, and/or concluding agreements on 'next steps'. Formal meeting or seminar facilitation was also resisted by some academic collaborators. Academics have suggested that this may be due to the individualistic nature of the academic community, and again, the academic focus on peer review journal articles or books as their target output.
• Existing divisions in the academic community created challenges for academics who wish to act as 'honest brokers' in event organisation-particularly events which would cut across networks, institutions or disciplines.
• The academic community tended to prioritise academic perspectives on issues, and draw evidence from particular projects. This contrasted with the SE prioritisation of inclusive Scotland-wide discussion on key issues for the Scottish Executive
• At a European network level academic world colleagues from Europe appeared to be unfamiliar with developing policy relevant outputs from academic meetings and events; and were unfamiliar with the concept of including grass roots organisations or frontline public sector staff in events attended by academics

As a result of lessons learned:

• The SAHPM team suggested that a productive way forward would be to use the ESRC as a disinterested 'honest broker' to host public policy seminars. This would make best use of academic's intellectual assets in combination with complementary policy oriented, strategic skill sets. The OCRKT team is now taking forward plans for a Scottish seminar series.
• The SAHPM team discussed improving KT through effective facilitation provided by ESRC or other stakeholders. There is the potential for KT events to have improved facilitation, and follow up, by KT specialists asking participants to identify agreed next steps, and then at a set time after the event, for example, to identify 'what they have done' and compare that to 'what they intended to do' on leaving the event.
• SE needs to conduct further outreach work to clarify advance planning requirements for Chief Executives, Scottish and UK Ministers. Whilst strategic contacts in the Universities appreciated the opportunities provided by Ministerial engagement in events, academics were less interested in attracting Ministers or (in the case of this pilot) senior NHSS officials to their events.

3.4 The Private Sector Community

Much of the work of the SAHPM team on knowledge brokerage focused on establishing relationships and laying the foundations for future collaboration with the academic sector. However, it was clear that a number of other potential evidence providers were in a state of readiness for engagement with SE. These were primarily hybrid individuals crossing between the world of academia and private sector consultancy, or private sector consultants who 'converted' the work of academics into utilitarian products.

Interest was also expressed by a number of private sector consultancy firms in future engagement with SE. However, there were few opportunities to take forward joint work with private sector consultancy as part of testing the theory and practice of KB/ KT due to the limited capacity of private sector to engage in loss-leading activity. This was an interesting contrast to SAHPM academic collaborators who had more capacity to engage in testing the theory and practice of KT and KB.

The SAHPM consultation responses endorsed the suggestion that SAHPM should endeavour to work towards providing an ethical means of establishing constructive and positive relations with major suppliers to the NHSS. NHSS is a multi-billion pound organisation and major purchaser of goods and services, and as a result many suppliers seek to influence the NHSS through a range of marketing activities which have been open to misinterpretation and public criticism. The SAHPM team opened a dialogue with health industry representatives, resulting in interest from industry in SEHD creating a forum for future discussions between industry and NHSS.

3.5 Effective Knowledge Transfer and Knowledge Brokerage between research, policy and practice communities

The project also revealed a number of areas of cultural misunderstandings between the policy community and the academic community. To create a collaborative working environment based on relationships of trust and respect, further work is required on improving understanding of the different protocols and practice in the academic and government sector.

The SAHPM team demonstrated that it is possible to apply the theory and practice of KB and KT in Scotland, however, in doing so SAHPM has revealed a number relational issues between those involved in the KT and KB process that need to be addressed if future interaction is to be effective.

« Previous | Contents | Next »