« Previous | Contents | Next »
Listen
Section Five: Opportunities for and Barriers to Action and Change
Introduction
5.1 This section explores the opportunities for and barriers to action and change arising from the implementation of the variety of CitiStat models in different contexts across the case studies. Opportunities and barriers are often intertwined. Teasing out the risk factors requires sensitivity to the context and local judgement about what is likely to work in particular circumstances.
Opportunities for and barriers to the effective implementation of CitiStat
5.2 The evaluation evidence does not suggest that any one model is better than others. However, there are a range of factors which have affected the implementation of the CitiStat Pilot and which should be taken into account in the specific context in which any future adoption of the CitiStat approach is being considered.
Flexibility and diversity
5.3 The Pilot has shown the potential diversity and value of the model where it has been allowed to evolve to suit local circumstances and priorities and crucially, reflect local governance, accountability and partnership arrangements. The CitiStat model appears to be flexible as it can fit into existing structures and be organised in different ways. For example, it can be a partnership model as in NHS Ayrshire and Arran; an executive process holding service managers to account as in Edinburgh; a similar model working across Neighbourhoods teams as in Aberdeen; or as in NHS Tayside, a two tiered structure where non-executives hold senior and other executives to account. There may be other structures or hybrids that could be operated effectively in different contexts.
Respective roles of executives and non-executives
5.4 Experience from the Pilot suggests that the CitiStat meetings have provided a useful forum where executives can seek the support of non-executives and the Chief Executive for actions which require their backing. NHS Tayside has seen more active involvement from non-executive Board members and this has helped them to be better informed about the work of the Board and has supported scrutiny and governance. There is some wider interest in greater involvement in the future. The benefits from the involvement of non-executives are discussed more fully in section 4.
5.5 However, clearly there are political sensitivities concerning possible role conflict between non-executives or elected members and executives.
"We need to be clear about public accountability: there are distinctions to be drawn between political leadership and the performance review role of elected members, and on the other hand the operational management role of Council managers. In Scotland there needs to be a partnership between local politicians and Council managers. We need to make sure the relationship is right in order to deal with the potential conflict."
5.6 There has been limited involvement of elected members in the two local authority case studies and some ambiguity about the role of elected members in the Edinburgh case study. Participants had mixed views about the value of their greater involvement, but did feel that they would benefit from the rigour of the CitiStat process and that it would be a good model for improving the existing accountability processes. This has implications for decisions about appropriate structures for CitiStat panels in other public sector agencies and the fit with existing accountability structures.
Governance and accountability arrangements
5.7 CitiStat has raised issues about the understanding and clarity of accountability across departmental, corporate and partnership responsibilities. The process has shown the need for greater clarity of responsibilities for action, especially where service managers are held accountable for performance and where other departments are expected to provide data or other information.
5.8 Within partnerships, the process raised issues of accountability between the NHS and the local authorities. In the NHS Ayrshire and Arran evaluation process it was suggested that an NHS Chief Executive could not hold local authorities to account or vice-versa. Participants realised that this had been a tacit, but common assumption. However, it was also recognised that this should be part of the performance management of the established partnership agreements; in effect the relevant mechanisms did exist but had not been operationalised. This need for clarity extends also to the mechanisms for joint accountability of the NHS and local authorities for performance reporting targets, as highlighted later in NHS Ayrshire and Arran in relation to SEHD delayed discharge targets.
Resources for the CitiStat process
5.9 Effective data analysis and clear briefing of the Panel has been an indispensable element of the CitiStat process and in the process of ongoing adaptation.
"The back office side of CitiStat is also very important in terms of getting your data right, getting your analysis right, getting your turn around right and all the other supporting activities that make and support the Panel because without a well supported Panel the process won't work and the thing will sort of dissolve into a sort of general debating chamber".
5.10 This process has been very resource intensive both in terms of the data analysis and briefing process, attendance at meetings by large numbers of staff and the high level input from Panel members and is one of the clear barriers to effective implementation of the CitiStat process.
5.11 The analysis process has also been quite a technical process with corporate staff undertaking the analysis often having to develop expertise in operational matters in order to be able to ask the right questions. This suggests that it would be useful to consider shadowing or secondments of staff from service departments to act as 'critical friends' to the analytical team. The use of an action learning or rapid cycle change model in NHS Tayside helped to adapt the model to fit locally; supported the speed of the process in implementing and following through actions; and reinforced accountability within the wider system.
5.12 The case studies have been managed by staff working on this project in addition to their usual responsibilities. Existing scrutiny and performance management processes have also remained ongoing. Staff from the Scottish Executive Performance and Innovation Unit ( PIU) have also played a key role in compiling the Panel brief and advising on the process; this was seen in the case studies as very different to the more conventional role of the Scottish Executive of monitoring and inspection. This type and level of support was felt to be essential and unique in making the case studies work.
"[They'd say] 'I'm working on this and I'm not part of a policy Department so I'm not going to report back on your performance to anybody. I'm not going to hold you to account for your performance; I'm free to assist you in developing the system and providing advice and support as required'. That's very new and seriously it's an important point because every time the Executive came before, we've all been very cautious, we've been very careful what we do and say, and I think that is a very new concept. Their support was seen differently this time by us all."
5.13 Whilst the costs of this process have not been directly measured, there is a very clear sense that the whole data analysis stage has been crucial, has allowed the development of clear links between data and action and is therefore an essential element of the CitiStat process. Therefore, CitiStat needs a dedicated resource if it is to be sustained.
Using CitiStat as a driver for organisational change
5.14 The CitiStat Pilot has taken place within a context of on-going organisational change in several of the case studies which is likely to have presented barriers to the most effective implementation of the Pilot. It has taken place within a context of major organisational re-structuring, service re-design and competitive tendering of services. However, at the same time, CitiStat has itself been part of wider organisational changes which impact on performance management; in some of the case studies, this has been seen as an opportunity and has been part of explicit attempts to promote a stronger performance management culture within specific services.
5.15 The CitiStat process has illustrated differences in attitudes and competencies amongst managers. Some individuals have been seen to relish the process of challenge and the clear sense of priorities for action that it provided. This appears to principally relate to the contrast with other performance management and accountability processes, discussed in section 4 above.
5.16 In relation to the desire to use CitiStat as a driver for wider organisational change, this raises issues about the broader ownership of CitiStat amongst those that do not attend the Panel meetings, particularly since the process has been almost entirely paper and office based, rather than involving the public or front-line staff in investigation.
"It's also important that, right down to the guys sweeping the street, everyone has some idea of what's going on. You know there's no point in keeping CitiStat at a high level; you've got to bring it right down to the guys who actually do the work."
5.17 As discussed in section 3, there are clearly opportunities to make greater use of the views of a wider range of staff, the views of the public and to bring in more qualitative data into the process to counter the observation made by one participant that; "it's remarkable that we've not been out of the office!"NHS Tayside plan to issue an organisational newsletter to promote wider dialogue.
5.18 Most participants took the view that it is essential that the 'ownership' of CitiStat is as broad as possible and that this has important motivational benefits and helps to make the process sustainable.
"I think it would be a huge win if they [staff] actually understood that what they're actually doing is scrutinised, up through all those levels."
"There's definitely mileage in inviting other people that are involved also as observers - all the way down….it's an open process that way. We want to encourage that so that people can sit in a session…and understand it a bit more."
"You know in the past one of the main complaints was that 'nobody ever tells us anything, why should I bother when nobody thanks me for it?'…and it goes a long way just to say occasionally 'well done guys'."
5.19 This may challenge 'top-down' models of leadership, rather than a model that believes that change occurs most effectively when responsibility is diffused through all levels of the services. Section 4.24 discusses the risks of causalities of this process as some people find it difficult to adapt to organisational and cultural change of this nature.
Strategic relationship with other performance management models and service planning processes
5.20 Section 4 has discussed the CitiStat Panel process and a perception of artificiality, partly due to some overlap with existing performance management and accountability processes. In relation to existing performance management models and service planning processes, CitiStat does bring important and distinctive elements to the process, although clearly it does need to fit with existing accountability processes and become embedded in prevailing organisational structures.
5.21 CitiStat also needs to link more clearly with strategic plans and priorities, rather than simply starting with the data that is available. This could include clearer linkage with Community Plans, for example. The model could also encourage improvements in service planning: it provides a framework for service planning decisions and assessing the impact of service redesign by considering the likely impact on the key CitiStat indicators.
Other barriers to implementation during the Pilot
5.22 Other factors which have affected the implementation of the Pilot or which have been under-developed have been;
- the limited time for the Pilot process
- the perceived relevance of the focus of the Pilot to small authorities
- inadequate buy-in to the process from key stakeholders and patchy attendance at some meetings
- artificial restriction on the scope of indicators during the Pilot phase
- resistance to change and new ways of working
- departmental restructuring and reorganisation
- inadequate corporate IT systems and reliance on manual production of data
- lack of ownership of the indicators
- limited involvement of observers from external agencies
- use of few outcome measures (such as customer satisfaction) to assess the impact of the processes
- no patient or service user involvement in the process
- paper and office based processes - little direct investigation of issues in the public realm.
5.23 The proposed case study in Orkney Islands Council did not proceed. This has highlighted a number of lessons about establishing Pilot areas and also for any future wider implementation of CitiStat. One issue was that focus of the Pilot was established at the outset as being environmental quality of life issues. These were not key issues for Orkney Islands Council and whilst an alternative issue of local importance was explored, the question of whether there would be sufficient volume of data to test the CitiStat model meant that this did not proceed.
5.24 Key lessons from Orkney are that there needs to be clarity about what a Pilot is expected to achieve across an organisation and both political and managerial buy-in. Sufficient lead in time for proposed Pilot areas is important to establish these conditions and to allow the adoption of a more integrated approach to performance management from the start.
5.25 This raises some important issues about terminology and applicability of the CitiStat approach. The term 'CitiStat' itself may imply that the process is only applicable to areas where there is a large volume of statistical data (which may tend to be urban rather than rural); as discussed this is clearly not the case. The Pilot has shown its widespread and diverse applicability across the four case studies. It is undoubtedly of wider relevance and retains considerably scope for continued adaptation, including greater use of qualitative data. Participants at the CitiStat Exchange Inquiry event took the view that the spirit of the CitiStat accountability process should be retained whilst adopting appropriate locally meaningful terminology.
« Previous | Contents | Next »