The Scottish Government

« Previous | Contents | Next »

Listen

Annex 3: Case Study of NHS Tayside Board

Context of the Pilot

NHS Tayside Board covers a population of nearly 400,000. Dundee City is the largest population centre, accounting for over 140,000 of the population. Perth and Kinross covers the western part of Tayside. This includes the city of Perth and a large area of sparsely populated countryside. The eastern part of Tayside is Angus, a substantially rural area which includes a number of smaller population centres.

NHS Tayside Board has overall accountability for the planning and delivery of health care services and the lead role in improving the health of the resident population in Tayside. This involves the assessment of health and health care needs; initiating and maintaining effective measures for health protection, health promotion and health improvement; development and delivery of health service provision; and allocating and evaluating the utilisation of resources controlled by the Board. NHS Boards are expected to work in partnership with a diverse range of public, private and voluntary bodies to develop a range of community initiatives, which will impact on health in the widest sense. The Board is also expected to take forward the health aspects of the Government's social inclusion and modernisation agendas as well as NHS priorities.

Prior to the CitiStat Pilot, NHS Tayside Board used the Performance Assessment Framework ( PAF). The report went to the Board on a six monthly basis and used data validated by the Information and Statistics Division. The report was based upon the actions being taken to achieve the PAF/Health Plan/Corporate objective targets and whether or not these actions were being successful on having the desired effect. The report was descriptive and detailed, and did not address whether or not actions were achieving the higher level outcomes. The Board also did not at that time have a group or committee that specifically addressed performance.

In early 2005 as part of an internal review, the Board considered how they were going to performance manage the corporate objectives. They built on a best practice example of a data driven methodology from Avon, Gloucester and Wiltshire Health Authority. NHS Tayside adapted the performance report, also taking into consideration the high level indicators that apply in Scotland and developed the High Level Scrutiny Report. However, prior to CitiStat no significant thought had been given to the style of meetings, who should attend, or to the analysis, briefings and follow up of agreed actions.

NHS Tayside was also influenced by the Boston Institute of Health Improvement's work on patient safety ¹, which advocates measuring trends over a period of time ( IHI "Plot your Dots"). NHS Tayside generated run charts in a number of areas and the ensuing discussion of time series data helped benchmark some of these areas.

The Pilot

The CitiStat Performance Improvement Pilot started in June 2005, and operated at two levels in NHS Tayside. At the strategic level the Pilot focussed on the governance and scrutiny functions of the Board, developing a data driven accountability process led by the Chairman of NHS Tayside and supported by the non-executives which focuses on the key Ministerial priorities. Figure A3.1 shows the structure of the High Level Scrutiny Panel.

Figure A3.1: NHS Tayside High-Level Scrutiny Panel.

At the operational level the focus was on performance improvement in areas of service delivery, which was identified as important by senior executives. This study is of the operational level CitiStat Pilot, although reference is made to the strategic level Pilot, particularly where there are direct linkages.

The Pilot was supported internally by a CitiStat Team, which comprised senior and middle managers and had input from the Scottish Executive Performance and Innovation Unit through their Health CitiStat Manager.

NHS Tayside has an extensive management information database which informs the delivery of both national and local level priorities. The aim of the Pilot was to draw on this existing data set and to focus on the most important management information for senior executives.

The data was put into a simple template which provided basic trends, variability by Acute Division and Community Health Partnerships and performance against targets. The data was analysed by a team which had both statistical capability and operational knowledge. This analysis was provided to the Chief Executive who led on driving forward performance improvement through the CitiStat session, which NHS Tayside initially referred to as Bedstat.

The focus of the Bedstat ² Pilot was on access and treatment, which are two HEAT (Health Improvement, Efficiency and Governance and Access and Treatment) targets for Health Boards required by the Scottish Executive Health Department ( SEHD). It was acknowledged that waiting times, which are a high priority under access and treatment, were already under extensive scrutiny and that to avoid duplication they would not be included in the first phase of Bedstat.

The management information selected for Bedstat had the following specific objectives:

• to improve bed utilisation across Tayside Health Sector and use of services more efficiently,
• to increase at all levels staff accountability,
• to improve the quality and range of services to citizens of Tayside,
• to assist staff to recognise and identify problems and have a methodology to monitor and review actions taken to resolve them.

There were eight indicator topics: bed complement - acute and primary care divisions; bed occupancy, primary care (excluding mental health) and bed occupancy, acute services (excluding paediatrics, obstetrics and communicable diseases); delayed discharges (overall Tayside, acute and primary care), including health and social reasons for delay); private sector capacity; day case activity; pharmacy; nursing skill mix (acute); and nursing skill mix (primary care).

These topics or areas for improvement were then further disaggregated into performance indicators, for example, delayed discharges are further broken down into:

• Overall Tayside
• Overall Tayside> 6 weeks
• Acute total
• Acute> 6 weeks
• Acute< 6 weeks
• Primary Care Total
• Primary Care> 6 weeks
• Primary Care< 6 weeks
• Other

The Bedstat session was led by the Chief Executive supported by two non-executives as shown in Figure A3.2. They hold a Podium of senior executives accountable for performance against the agreed targets, using the management information and briefing provided by the CitiStat Team. Actions identified through the session are the responsibility of the senior executives on the Podium; their follow up of delivery is key to ensuring that the process retains its drive for continuous improvement.

Figure A3.2 The NHS Tayside Bedstat Panel

The CitiStat Process

Roles of the CitiStat/ Bedstat team

The structure, roles and responsibilities of the CitiStat team are outlined in Table A3.1. The Director of Change and Innovation has played a pivotal role in managing the Pilot; the tasks associated with this role have included:

• Preparing the Pilot information pack for all participants;
• Managing the CitiStat/ Bedstat team;
• Raising questions in the evaluation;
• Facilitating cross system working;
• Resolving issues between meetings.

The balance of the team has been critical to the progress of the Pilot, with a mix of performance and financial management experience and a depth of operational knowledge. The CitiStat team have acted as a conduit for information and communication; for example there have been significant contacts within the organisation and with other Health Boards.

Most members of the CitiStat team are directly involved in the development of services related to access and treatment. Engagement of managers at this level has ensured that there has been wider organisational diffusion of the Pilot's aims and objectives, rather than just amongst the senior management team.

Several members of the team have benefited from visiting other case studies to see how they are applying the CitiStat model.

Table A3.1: Structure, roles and responsibilities of the CitiStat team in NHS Tayside

Management of Papers and Actions between Panel meetings

The CitiStat process is as much about what happens between sessions to improve performance as the sessions themselves. The focus of the briefing for the Chief Executive is to enable an action orientated discussion of priority issues. The actions determined at the CitiStat session are owned by the relevant operational division(s) and their representative(s) on the Podium.

To ensure that there is rigorous follow up of identified actions the administration processes need to be clear. This function has evolved over the period of the Pilot with a dedicated resource identified to act as a hub for information exchange in the form of a dedicated part-time administrator.

An update of progress on all actions is incorporated into the briefing for the next session. More importantly, the ongoing analysis of performance takes into account the projected impact of actions and identifiable change.

To ensure that the Bedstat session can focus on performance, the data flows and action updates need to happen to a well defined timetable:

• The data report is populated on the 15 ^th of every month, unless an alternative reporting timescale has been agreed with the Panel;
• Analysis of the data happens within 3 working days;
• A briefing is then produced within 2 working days;
• The session occurs within 10 working days of the 15 ^th;
• An action note goes out within 2 working days of the session;
• Action updates are provided by the 15 ^th of the relevant month.

Continuous Improvement

At the outset of the Pilot it was intentionally decided that only existing data and targets should be used to avoid over burdening the system. However, the constraints imposed by this decision became apparent to the CitiStat Team, Panel members and respondents. The basic problem was that the data in use was designed for performance reporting, not driving forward continuous performance improvement. This is an important distinction that emerged early on in the Pilot.

After three sessions, the focus moved beyond the existing data template to identifying 'hot spot' areas within the delivery of access and treatment services. The following were selected for inclusion in the second phase of the Bedstat process:

• Delayed discharges;
• Inpatient and day case waits;
• Medical boarders;
• Acute medical and surgical bed usage;
• Medical admissions/ assessment;
• Absence;
• Community hospital beds;
• Surgical High Delivery Unit;
• Outpatient and Diagnostic Service.

It was considered that focussing on these areas would promote whole system actions for improvement. A hotspot data template was designed and presented to the fourth Bedstat meeting in December 2005. This asked for the following information to be supplied in the form of a report for the January meeting:

NHS Tayside Hot Spot/ Improvement Template

1. The Issue (What is the problem we are trying to fix?)

2. The Plan for Improvement (What are we going to do to fix the problem?)
2.1 Community Health Partnership General Managers and Acute Sector Clinical General Managers to provide details of plans in the space below
2.2 Planned performance - give planned target by month in the table below

3. Measurement for Improvement (how will we know change is an improvement?)
3.1 Set out below measures for improvement (How will we know change is an improvement?)
3.2 Process indicators - this section also includes the Local Delivery Plan trajectories

4. Risks

What are the five key risks with achieving this target and what are your plans to address/ minimise these?

The measures for improvement and the process indicators were collated into a new data report which replaced the existing information and provided the background for analysis, briefing and ensuring accountability for delivery. Throughout this transitional phase the existing data report was used to ensure that potentially challenging issues were not overlooked.

The evaluation process

The evaluator benefited from an induction visit in September 2005 to meet key personnel involved with the Pilot over one and a half days; this was arranged by the Scottish Executive CitiStat Manager. The Pilot was evaluated using action research: the CitiStat model could then be tested and adapted between each session. In order to promote ownership of the findings at local level, the evaluator used a co-facilitation model and worked closely with the Associate Director, Modernisation and Development, and a Development Consultant from the same team. The Co-Facilitation Team liaised between meetings and normally met for one to two hours on the afternoon preceding the Panel meeting; after a short break the evaluation sessions followed the Panel, and lasted 45 minutes to one hour, with slightly longer for the launch. Sessions were recorded using conference digital sound recording equipment and the sound files later transcribed. Key quotes from the evaluation sessions were extracted from the transcriptions and presented in the form of a thematically generated learning grid after each session. The evaluator also met from time to time with members of the CitiStat team to answer queries and to collect further information. All evaluation sessions were very well attended.

There have been a total of five evaluation sessions, with the December meeting being co-facilitated by the two internal members of the team. For the initial launch a visual inquiry exercise was used with the group to create their vision of success for the project and to identify challenges. This led to the development by the group of eight success factors, which formed the framework for local evaluation. In response to a request for a de-brief following the Panel meeting, the second, third and fourth sessions were more discursive to allow time for the issues to be aired and to agree actions. The final session was conducted as a focus group to which members of the CitiStat team and other key players were invited. The aim of this session was to generate a consensus view on the added value of CitiStat as compared to other performance management models, and to gather any additional data or learning points for the final evaluation report.

One of the internal facilitators videoed short interviews ³ with the Vice Chair of the CitiStat and Bedstat Panels, the Chief Executive (Acute Division) and a Community Health Partnership General Manager in January. The interviews were incorporated into a short PowerPoint presentation to represent the Tayside perspective at the CitiStat Exchange Inquiry event; other feedback was collected for display from participants at the January evaluation session.

Success Factors

The success factors were generated from an analysis of the transcript in which participants introduced their vision of success for the Pilot and the challenges to the group. They were circulated and agreed by the group at the next evaluation session.

The success factors were:

• Using the CitiStat process constructively.
• Identifying and responding to the challenges.
• Improving the quality and analysis of the data.
• Evidence of improvement: better understanding, realistic goals, interim measures and clear actions.
• Breaking down the barriers between acute and primary care.
• Improved communication and working together.
• Being clear about our finishing line and measuring the stages along the way.
• Keeping people at the centre of the vision.

Table A3.2 below, drawn from more comprehensive feedback to the group after the first session, gives an idea of the thinking at the early stage of the project. The success factors are not listed in any particular priority.

Table A3.2: Vision of success and challenges

Short extracts were taken from the quotes and matched to the images in order to generate a visual representation of the group's success factors and challenges. This was then brought to the second session. The visual representation was also used for display purposes at the CitiStat Exchange Inquiry event.

Findings

The progress and findings of the Pilot are analysed using the eight success factors, which have been thematically grouped to allow for discussion where there are linkages.

Using the CitiStat Process Constructively and Identifying and Responding to the Challenges

There was clear evidence at the outset of NHS Tayside's commitment to performance improvement, which was enhanced as they engaged with and adapted the CitiStat model for their own purposes. After two Bedstat Panels and at the first evaluation session many of the challenges became apparent: shifting from explanation to accountability; focusing on improvement not judgement; ownership of actions; and cultural change issues associated with the perceived confrontational nature of the model. Room layout and the physical distance and division between Panel members and Respondents posed some difficulties early on, especially when the Panel was held off-site in a local hotel conference room:

"Central to what we've been doing today and the other CitiStat meetings is confrontation. Now I've spent the best part of five years pulling together people that work as a team, that rely on each other, that depend on each other. We know our own limitations and we support each other...now to then be put in a situation where you're saying 'you will do that, you will do it by then' doesn't feel right".

The venues for subsequent meetings were changed to internal seminar/ conference rooms, and this led to a less confrontational meeting and promoted inclusion from those sitting at the edges:

"Are people comfortable with the physical layout? . . . I've no problem with [the notion of non-executive members holding staff to account] . . . If I had personal preference it would be working in a round table context, still asking the questions and if necessary asking them very pointedly but I just feel this: the managers up one end and the board members at this end creates a body language that I must admit I was a wee bit uncomfortable with at the first meeting."

By the end of the Pilot those at the Focus Group considered that the format of the meeting was perhaps less important than "an absolute style for the meeting". NHS Tayside had been able to identify and support what worked at the two different levels (High Level Scrutiny and Bedstat):

"I'm very comfortable that we have two completely different levels of meeting, because they both work. And they work because they work for the person who chairs it . . . As well as the team having had a chance to influence how it's all been, I think people out there have also had a chance."

The role of the Director of Change and Innovation and the CitiStat team, which meets weekly, cannot be underestimated in the wider process of ongoing adaptation. An action learning or rapid cycle change model was used to adapt the model to fit locally: "At the meeting we say 'did that work, how we can change the report?' We've been evolving and changing what we're doing." Thus a cycle of continuous monitoring, careful analysis, follow up of actions, and ongoing reflection was generated. Throughout the Pilot there was a willingness to change things that did not sit comfortably or were not productive.

Towards the end of the Pilot, praise for particular achievements figured more prominently. These included meeting the inpatient and day case waits December 2005 target of no patients waiting over six months and generating a benchmark hot spot/improvement template for medical assessment. Praise from the Chief Executive sat alongside some more penetrating comments from the Panel such as: " I'm more interested in actions" and "a lot of activity but will we make the target?"

Respondents at the Focus Group considered that a distinct advantage of the model was its ability to build a corporate memory of achievement and non-delivery:

"If you don't deliver on something it won't go away, because it's monthly, it's institutional knowledge . . . it's not that people will be knocked on the head for poor performance; rather it's that they won't be let off."

There was also a timely reminder at the January evaluation session, after generating the Hot Spot/ Improvement templates that the Panel and Respondents needed to be more focused in the future:

"To put today's meeting in context I think the fact we went from the previous ones of looking at a purely statistical report, which is the norm for CitiStat meetings, to this development of how we put together a more focussed, more appropriate report is questionable on whether it's part of the CitiStat process or whether it's just part of the support process for CitiStat."

"I don't think enough actions came out of today, there were specific areas that [we] could focus on, but there's a hell of a lot of work in progress."

"Today is probably the most high level transitional period we should be, we should be much more focused in February."

Over time there has been a change in the information given to the non-executive members of the Board; this in turn has led to enhanced understanding of key issues and more penetrating questioning of the executives. As one respondent at the Focus Group put it:

"The non-execs seem to understand the business much better than they did before . . . that's been a big cultural change for non-execs and execs, [who are] not used to non-execs asking hard questions on that kind of regular basis".

A previous 'culture of politeness', where non-executives " were asked to be supportive about something or front things in a nice way, not being asked to ask difficult questions" has been replaced by a new culture and rules of engagement which combine respect with an absolute focus on scrutiny, accountability, and improvement. Another positive is that non-executive Board members, who were not part of the Pilot, have expressed an interest in having some future involvement.

A respondent at the Focus Group said that CitiStat had given performance improvement/ management a higher profile:

"If what we do (plans, strategies etc) can't be couched in terms of performance improvement then . . . drop them. It's raising the game of planning in the organisation, now seeing planning in action around performance improvement."

One of the potential challenges of CitiStat is that the more challenging culture will generate some 'casualties' as one respondent at the Focus Group said:

"This will influence change and the speed of change. People who can't adapt will drop off or do something else . . .This is bringing it right to the doorstep; previously managers might have hidden behind a door or [thought] it's somebody else's issue or problem . . . We need to manage this exceptionally well to get the outcomes we need.."

A clear advantage of the CitiStat process is the quality and timeliness of the data analysis, which frequently combines operational knowledge with the data:

"What we've seen is more getting your hands dirty, analytical work, rather than numerical analysis."

This analysis can be passed back to managers to respond to in a pro-active way; they can also draw on members of the Modernisation Team, who are able to offer tools and techniques to solve a problem.

Improving the Quality and Analysis of the Data, Evidence of Improvement: Better Understanding, Realistic Goals, Interim Measures and Clear Actions; and Breaking down the Barriers between Acute and Primary Care

During the early stages of the Pilot it was recognised that there were some problems with the data: waiting times and delayed discharge data were robust, but other data sources were more patchy. The data also needed to be expanded to cover a wider range of delivery issues to ensure that the Panel session drove forward service improvements and avoided becoming repetitive. One Manager commented on the superficiality of the questioning and the limitations of the model in the early stages:

"The reason why I found it superficial was, because I knew I wasn't being asked the right questions because the information people were sitting with wasn't giving them the right questions. So I had green boxes when I knew there was all sorts of problems behind it, but this system does not allow me to say: And by the way can I have a chat with you about the green box, and you really need to be asking and agreeing about this?"

One of the features of this performance improvement model is its reliance on real-time unvalidated (rather than validated) data. Lack of validation was seen as a trade off against quicker turnaround times. At the same time data needs to be useful to the executive team to help monitor high level performance and to the non-executives for scrutiny purposes. There is a distinction to be made between data for improvement and data for performance.

There is ongoing commitment to ensuring that analysis is undertaken and informs the agenda for Panel sessions. This focus on improved data quality and breaking down the barriers between acute and primary care are evidenced in the Hot Spot/ Improvement template, although it is far too early to see changes in the higher level targets as a result of all the work around the template hot spots/ improvement areas.

The generation of the Hot Spots was a process which merits closer attention; it began with some fairly frank comments at the November evaluation session in response to the preceding Panel session:

"It didn't do what it says on the tin for me, I'm afraid."

"There's a fairly fundamental issue for me about the data . . . so it's a bit of a rethink for me around the dataset and the improvement process we're driving."

"I can't assume in my role that the data is the appropriate currency for measuring improvement and measuring benefit to patients and measuring effective use of resources."

This then led into a discussion on data for improvement: identifying the core improvements; the data that can be accepted as improvement measures; and ownership of the improvements. A point was made that we tend to measure what is easiest to measure or is more readily available. Closer collaboration across the whole system was required to generate better data. Equally important was the need to move down a level closer to the patient, and promote ownership by managers of the key targets:

"The local systems are going to be the thing that fundamentally address the major problem that we have . . . the work needs to go on at local level. We need to take ownership of a set of very key targets that contribute to meeting those high level ones, and we need to see the detail about how that is progressing improvement."

The evaluation was then used to clarify the process through which hot spots would be identified: " Let's get a consensus about what those hot spots are and then look at how we can constructively use this meeting to explore [them]." Once again, a willingness to go back to the drawing board was evident, which involved changing the data set in response to the hot spots. Thus a more individualised and focused approach was taken to data collection, analysis, actions and outcomes:

"It's almost like a drill down thing. Start at the high level, then you've isolated the hot spots, then you're looking at each of them individually, and against each of these you have a number of actions and outcomes that are measured . . . and the actions may differ from locality to locality."

The generation of the hot spots spanned acute and primary care thus promoting collaboration across the system and was closer to potential improvements in the patient journey as the following conversation illustrates:

"If a patient unfortunately has to come in as an emergency . . . they want to come in and be assessed rapidly; they don't want harm when they're there; they want to have the right information at the right time so they can deal with it. They don't want to be punted around from ward to ward and they want discharge no sooner and no later than they should be discharged. That's what they want."

"And not catch anything".

"Exactly and so those are the measures and we need to improve on every single one of them."

"Everybody round this table [needs to ask], what can I do to assist with that hot spot in terms of the context I'm working in? And the answer in Perth and Kinross will be different from the answer in Angus and in Dundee."

Work continued on the hot spots in December and January with the templates on in patient and day case waits, medical boarders, acute medical and surgical bed usage, medical admissions, absence and surgical high delivery unit ( HDU) coming to January Panel.

The work of the CitiStat team has been a core part of improving the quality and analysis of the data. The analysis has been cumulative with data being analysed in relation to specific areas, and the team asking: "What are the numbers telling us and what do we know operationally?" Moreover, the four re-design projects around accident and emergency waiting times, outpatient and diagnostic service, community hospitals and unscheduled care, will all use a template-like approach to generate the measures and targets that will come to a future CitiStat meeting.

The December evaluation session highlighted the need for one source for the data, but with many uses. This was further discussed in the Focus Group:

"We're doing a lot of work around a single information team, which would mean that you would have a pyramid of data; at each level of the organisation you would have a greater level of details. We've been asked for the analysis at the next level . . . it's interesting that people are asking for that."

This is encouraging and suggests that the model is promoting ownership and accountability amongst service managers. Another advantage of a single data source is that everyone is reporting on the same data.

In the January evaluation session issues were raised which related directly to the templates, how to complete them, and how higher level targets and measures could be generated from them:

"I'm just wondering how we help people to be much more aware of just what we need in a template and what information and detail we need to populate it with, because we had quite a variation in the templates that we saw today, didn't we?"

"It helped having one that fulfilled the needs very well . . . as the benchmark . . . but there's still work to be done afterwards on how that actually gets transcribed into a statistical report, and what that looks like and tolerances and all the rest."

It was acknowledged that every item on a hot spot template would be like a sub project with a lead who has to deliver. Given the level of activity related to each hot spot there was a debate and problem solving about what should come to the Panel in the future:

"Or you'd be reporting target and it would be a breach analysis . . . so that you can now say: So what's the actions around those breaches?"

"But if the aspiration is not to have medical boarders and the initiatives are not working then the requirement from here is to have new refined actions that do make that difference. We're not into the process of actually how you sort that, it's actually the high level part of that."

"What information goes in and what it looks like, what it means, how you interpret it, cross reference with other indicators are all very different . . . The only way we can do it is by dialogue."

"It's quite interesting that the acute and primary care division are discussing how to collate data, monitor and report . . . we're not there yet but that's what we're striving towards. I think that's a very positive thing that's come out of the last three months."

This suggests that the process of refining the data set to generate the hot spots both demanded and promoted improved dialogue across NHS Tayside and will require ongoing collaborative problem solving in the future. The vignette in Figure A3.3 offers an example of how a Hot Spot template on bed occupancy was generated.

Figure A3.3: Developing a Hot Spot Improvement Template

The model can also identify when a plan is not working, as one respondent at the Focus Group said about the target of 4.5% staff absence at the High Level Scrutiny Panel:

"It had become clear over a period of time that no progress towards the target had been made. The Chairman said, 'take this all away and come back with another plan, because the plan didn't achieve what it set out to achieve and you set your own target and haven't made it.' In a longer term frame, we're beginning to see when something doesn't shift."

The role of the model in enhancing corporate memory works for plans that might not be reaching their target, but also in relation to forward planning and learning from the intensive activity that went on to meet a target ( e.g. day case and inpatient waiting times).

Whilst there is evidence that there have been improvements in prospective performance management, it is too early to see any statistically significant trends. It should also be acknowledged that this Pilot covered performance across a complex system, where each part is inter-related and improvements demand whole system change. Over the past six months the process for determining forward actions and ensuring that they are delivered has been enhanced by:

• Improved analysis of the data and clear briefing;
• Relentless follow through of actions by the CitiStat Team;
• Clear expectations and leadership from the Chief Executive and Non-Executives.

The next challenge is to take CitiStat outwards and downwards in the organisation; and a newsletter is planned to promote wider dialogue.

Improved Communication and Working Together

Prior to the Pilot, the Acute and Primary Care Divisions did not use the same data set and analysis, and therefore it was very difficult for the two divisions to work to an overall service or patient-focused improvement. CitiStat has helped break down these barriers to communication. The process of discussing the limitations of the data set in the November evaluation session; the generation of the hot spots for the December meeting; and the completion of the templates in January have all led to improved dialogue and working together within NHS Tayside. This was confirmed in the transcripts and in the feedback to the Learning Exchange in January. A more questioning approach by managers was also evidenced in the feedback to the Focus Group:

" People in middle and junior management are now understanding why they're being asked to do things . . . it also promotes a more questioning approach by managers: 'why is the target that?'"

Progress over the course of the Pilot has been made in the partnership arrangements with local authorities. During the November evaluation session several issues were raised in this regard as the following dialogue illustrates:

"I think I'm the only local authority person here today and I'm asking where is the partnership in this? Because really the people you're talking about are our service users so maybe we're delivering services before they come into hospital and we're part of the prevention of admission . . . or the rapid response to get them back out, and we're the people who've got the data for delayed discharges, so where is our voice round the table?"

"The ethos of this is hold people to account. I can't hold you to account."

"If you identify the hot spots and we ask individuals round this table to identify things that they can do to assist, surely we can also ask some local government representatives the same questions . . ."

"But with respect, what's happening is your information, the way it's getting parachuted in is isolating us rather than bringing us along within the partnership and that's unfortunate."

"The only way we can achieve [shared accountability] is by the way the targets are set and agreed and taken ownership of and a [shared] response coming back . . . "We've then got to sign up to that accountability and to be part of that mechanism . . . and all three councils would have to do that."

By January the Chief Executive and the Strategic Clinical Care and Discharge Manager were able to report to the Panel that a productive meeting had taken place with local authority partners on delayed discharge. CitiStat was seen to have accelerated this process; better information allowed a higher quality conversation to happen as a spin-off. Figure A3.4 gives more detail.

Figure A3.4 Delayed Discharges

Being Clear about our Finishing Line and Measuring the Stages along the Way

The deadlines for the Board papers on a single structure have acted as a catalyst to clarify Tayside's thinking on what their scrutiny, accountability and performance improvement infrastructure should look like:

"Whatever we end up doing with this process it's got to be consistent with the process we have for setting our overall corporate objectives. Different scales and whatever else, it's all got to be linked. There's a real danger we're doing a piece of work here and then we're forgetting about it, and we're doing another piece of work here and there's no crossover."

Further work needs to be done in this area if duplication of effort and bureaucracy are to be avoided, as this comment from the January evaluation suggests:

"We have to start developing some thoughts about where this exercise sits within the overall governance responsibilities of the Board . . . and another point is the relationship of this grouping, this committee and . . . the scrutiny responsibilities the Board has . . . Clearly by April we have to come up with some proposal about how we think scrutiny, the responsibility of the Board, will be discharged in the future; and that has to provide links with what happens here and other parts of the Board as well. It can't be free-standing."

The finishing line comprises the HEAT measures; the aspirational targets within the hot spots have a clear role to play in collectively contributing to the high level targets over time. Feedback to the Focus Group also underlined the importance of the incorporation of CitiStat within the formal governance structure, "otherwise where does it fit?" This formal link permits both internal and external auditors to look at the Health Board's performance management processes. Health Boards need governance structures to be "lean, mean and driving forward the organisation rather than all about dealing with plans and reports." Governance structures needed to fit the faster delivery plans:

"I'm not sure other Boards have clicked yet. They will after twelve months of going to the Ministerial meetings and [high-level SEHD] meetings and being held accountable."

In this regard a parallel was drawn between the CitiStat type accountability meetings with the Minister, who has been given a briefing by the senior team on NHS Tayside's performance; and the Chair of the Health Board who comes along with five or six of his senior team to respond: "I hadn't really thought about how closely it mirrored the accountability . . . that's why it's useful to have that level of meeting here."

Keeping People at the Centre of the Vision

It became apparent that during the Pilot this would be the hardest success factor to evidence: the focus on quantitative data does not lend itself easily to finding the patient's voice; and this was arguably the highest level success factor, which was unlikely to be evidenced in such a short period of time. However, several participants brought the discussion back to patients and their needs in the evaluation sessions. Not least during the discussion in November on the hot spots and what patients wanted. The focus was also brought back to patients during a discussion in January about medical boarders in the period after the Christmas and New Year, whether boarding could be reduced to zero and what needed to change at a cultural level. Public and patient involvement has also figured in service redesign. Figure A3.5 gives more detail.

Figure A3.5: CitiStat and Service Re-design: Consultation with Stakeholder Groups

The more direct involvement of Clinical Group Managers, who represent the front end of delivery in the organisation, should help bring the patient's experience more centre stage; this would also improve performance in the hot spots. Although patients are represented through non-executive Board members, how they engage as partners and how that is then fed into the CitiStat process needs further consideration.

The evidencing of this success factor was raised at the Focus Group and a discussion ensued about how qualitative data on patients' views might be incorporated within the analysis and the brief:

"We have lots of sources of information about the work we're doing with the public, but we're not currently managing to get that into the briefing and that's something we could reasonably do . . . we need a section on public/ patient involvement."

A suggestion was made to incorporate NHS Tayside public/patient involvement information into the single data system in order to relate customer feedback to performance. Another included how the Scottish Household Survey, which contains information on perceptions, might be used at local level by disaggregating the areas. This was the information that Ministers use. Both suggestions were taken on board and should help keep people at the centre of the vision over the next phase of CitiStat in NHS Tayside.

Effect of External Factors and Unanticipated Outcomes

The Focus Group identified the HEAT targets and what they would look like in their final form as an external factor: "Even with the list we produced in the first place, we weren't far off the mark." They also stressed CitiStat's role in the new governance structure: "People have said we like what CitiStat provides in terms of a governance model and we can build this into the new structure."

In terms of unanticipated outcomes the CitiStat team had not foreseen the sheer volume of internal and external communication required of them during the course of the Pilot. They also remarked that there had been a consensus within the team about the CitiStat model and underlying messages. This was seen as important when the ongoing findings from the case study were discussed by individual team members with other health boards.

Key Lessons from NHS Tayside

The following conclusion offers a review of the learning from the NHS Tayside CitiStat Pilot. As well as summarising the benefits and outcomes of the model, the factors that need to be considered in any potential further implementation of CitiStat are also included.

Linkages to the Formal Governance Structure

• It is important to find a way of integrating the CitiStat model within the formal governance structure.
• Informal discussion with non-executives indicates that they feel better informed about the work of the Board and therefore able to ask more penetrating questions.
• The CitiStat process supports scrutiny and governance: plans that fail to work are identified earlier and new plans submitted.

Leadership and Organisational Culture

• The leadership provided at the highest level in NHS Tayside was essential to make the process work, thus ensuring that actions were proposed, agreed and followed through.
• A pre-existing commitment to performance improvement should not be underestimated in any wider implementation of CitiStat.
• The model successfully focused the agenda at senior level on improvement, accountability and scrutiny.
• The model generates a more prospective management approach and provides an operational and high level management tool for key hot spots or targets.
• An infrastructure that promotes and supports change, the need for dialogue, ownership of the direction of travel, and the leadership qualities of key individuals should not be underestimated.

Managing the CitiStat Process

• There is a need to avoid a blame culture.
• Importance of incorporating praise for improvement alongside scrutiny and accountability within the model.
• Keeping improvement focused; strategic not operational; management of risk has to occur outside of the Panel meetings.
• If there is an opportunity for frank dialogue, the model can be successfully adapted to the local context.
• Pivotal role of the Senior Manager responsible for project management.
• The work of the CitiStat team is central. If CitiStat is to become embedded within the organisation then this needs to be acknowledged.
• Within existing resources, work on CitiStat could not continue in the medium to long term as an add-on to current positions.

Data Quality

• Data needs to be relevant, and if it is not then it is necessary to discuss why the data is not generating an improved understanding of a management problem.
• Data needs to be timely, but also sufficiently accurate to be fit for purpose - a balance has to be struck.
• The model generates action-oriented meetings to address poor performance in particular hot spots and provides an overview of performance across the system.

Data Analysis

• The use of time series data facilitates discussion about trends; there is also a ability to link data sets when making decisions. The use of time series data, charts and graphs have also contributed to a more prospective management approach to capacity and demand.
• The role of the CitiStat Team and relevant operational knowledge of the hot spot areas have been essential in improving the quality of the analysis and briefing over the period of the Pilot

Ownership and Actions

• Ownership of the data set important: this has strengthened the link between the data, performance and agreed actions
• Improved focus on taking action and relentless follow up by the CitiStat Team: the model is action-oriented
• Importance of managers setting own targets
• There is a balance being played out between ambitious targets or easier targets, but ownership means that managers can be held to account.
• Need to engage others in the organisation to understand the model and what it is trying to achieve

Cross System Working

• Consideration needs to be given to the additional expectations placed on CitiStat team members as the model becomes embedded within the organisation
• The model enhances corporate memory and increases awareness of issues across the system, actions, and outcomes
• Closer working between primary and secondary care

People at the Centre of the Vision

• This was the highest level success factor and the most difficult to evidence directly in a short Pilot: the focus on quantitative data does not lend itself easily to finding the patient's voice.
• Public/patient involvement information is being used in service re-design projects which will come to a future CitiStat Panel.
• The direct involvement of Clinical Group Managers, who represent the front end of delivery in the organisation, in the CitiStat process should help bring the patient's experience more centre stage.
• Information on NHS Tayside public/patient involvement needs to be incorporated into the single data system in order to relate customer feedback to performance.
• Public perceptions, included in the Scottish Household Survey, which is used by Ministers, might be used to generate local views on health provision and improvement by disaggregating the areas.

« Previous | Contents | Next »