« Previous | Contents | Next »
Listen
Executive Summary
What is CitiStat?
CitiStat is a performance management system which originated in Baltimore, Maryland, USA and which uses recent operational data on key performance indicators as the basis for discussions between the Leader of an organisation and service directors. CitiStat meetings are intensive and formal; service directors are asked to account for their Department's performance and offered support to improve performance where necessary.
CitiStat in Scotland
The CitiStat model has been piloted by four case study organisations during 2005-06 within the Scottish public sector; the City of Edinburgh Council and Aberdeen City Council focused on environmental quality of life; NHS Tayside Health Board addressed the delivery of access and treatment services and NHS Ayrshire and Arran Health Board focused on delayed discharge from hospital.
What is distinct and different about CitiStat?
CitiStat brings a number of distinctive elements to performance management. It is clear that there is no single element of the CitiStat approach that by itself is key to success but rather, it is the combination of elements linking review and enhanced use of data with the accountability process by the Panel that is key.
Leadership
- The high level involvement of Chief Executives and senior non-Executives focuses the agenda on improvement, scrutiny and accountability and gives a clear sense of organisational and partnership priorities. The challenge process is an important driver, provides opportunities for praise in recognition of achievements and gives service managers confidence to implement required changes. It influences behaviours by highlighting respective responsibilities as managers for the implementation of policies.
Data Quality and Analysis
- Enhanced data quality and analysis are key to the process and set it apart from existing performance management models. CitiStat provides a focus on actionable data, rather than available data. It facilitates a review of existing performance indicators and data sources to enhance the accuracy, validity, quality and utility of data. The use of real-time data brings a new immediacy to the process and provides quicker feedback on actions. The analytical approach supports the more effective process of accountability through the Panel. The process has highlighted this tension between data designed for performance reporting rather than for driving forward continuous performance improvement.
Ownership and Actions
- Review of the data used for performance management enhances ownership of the indicators. CitiStat strengthens the links between the data, performance and agreed actions and provides a challenging, but strong and broadly welcome focus on taking action. This has been empowering for some managers.
- CitiStat creates a dialogue that enhances mutual understanding and communication between participants, enhances corporate memory and can be a tool to promote closer working within and between departments, agencies and partnership structures.
Dual Approach: Hot Spots and Whole System Working
- CitiStat is applicable both to the identification, analysis and action around hot spots and also to a whole system approach. These are not mutually exclusive approaches. There is scope for using the leverage of CitiStat to develop a long term and wider perspective, rather than simply focusing on 'quick fixes'. CitiStat can be a tool to promote closer working within and between departments, agencies and partnership structures.
Cultural and Organisational Change
- CitiStat can be a driver for organisational and cultural change. It challenges established cultures and ways of working in the public sector.
Key lessons
There is no single model of CitiStat that has been piloted in Scotland. The four CitiStat case studies have been based on distinctive structures and have developed in different ways. The Pilot has generated some useful lessons for any future wider implementation of CitiStat in Scotland. There are a number of positive signs of change that illustrate the potential value of the model.
Whilst the adage that " you can't manage what you can't measure" has been used a number of times within the CitiStat Pilot, a further question 'what do we measure and why?' has supported some of the most significant developments in terms of using the process to develop actionable data, rather than available data and to develop indicators that are meaningful for service managers rather than simply to meet performance reporting requirements.
There is no conclusive evidence that any one CitiStat model is better than any other. The approach to the Pilot illustrated the value of allowing CitiStat to develop according to the spirit of the Baltimore model, reflecting different perspectives and local contextual possibilities, rather than through strict adherence to formal structures and procedures. This might be termed 'reinvention' rather than 'replication' of good practice; it allows for local customisation of quality improvement approaches and facilitates the integration and use of evidence in practice. It is thus an important principle that any future developments should allow the model to be adapted to suit the context.
This has been a relatively short-term Pilot in which it was not expected that there would be substantial tangible improvements in service delivery or identifiable savings which could clearly be attributed to the Pilot. However, there were some early, limited indications of improved performance, including improvements in sickness absence in at least two of the case studies, which participants have ascribed to the CitiStat process. It has not been possible to undertake a full assessment of the value for money of the approach. This is due to the short duration of the Pilot, where the costs of establishing the process are concentrated in the early months and lack of full cost data. An action research approach has been used to explore the experience of the case studies and develop an understanding of the opportunities and constraints that exist within different public sector contexts in the implementation of CitiStat. The evaluation is able to identify what elements of CitiStat worked well in specific contexts and which elements the case studies feel are of sufficient value that they and others might continue to include them in the model.
Key issues for the future of CitiStat and the implications for wider implementation
The report identifies key issues for the future of CitiStat and the implications for any wider implementation within the Scottish public sector. These include discussion of the specific elements of the CitiStat approach that the case study organisations would wish to retain; whether enough is known to further implement CitiStat across Scotland and the sustainability of the process in the longer term.
Any wider implementation of CitiStat will need to recognise the distinctive governance and accountability structures within different parts of the public sector (including health services, local government, government agencies and NDPBs) and ensure than CitiStat dovetails with existing requirements, rather than adding to reporting burdens.
Most features of the Baltimore model should be retained, although without the confrontational element. The model should not be imposed but implemented flexibly to allow the reinvention of the model in different contexts. Whilst the spirit of the Baltimore model may be applicable in Scotland a key distinctive contextual factor in Baltimore is that there are no well developed audit processes or external scrutiny, as exist in Scotland. In Scotland this raises issues for any further implementation of CitiStat about where pressure of performance reporting requirements might be reduced and how resources devoted to performance management and reporting might be focused more strategically. Any further implementation of the CitiStat approach should ensure good communication about the primary purpose of CitiStat as a local performance management tool. CitiStat needs a dedicated resource if it is to be sustained.
Other key implications include ensuring sufficient resources are available to support the analysis and briefing process; making improvements to information systems to support efficient data provision and analysis; developing the scope of the model to focus both on hot spots and whole system working; developing an outcome focus; embedding on-going evaluation into the CitiStat process and continuing the more reflexive and learning driven approach to policy implementation demonstrated by the CitiStat Pilot.
« Previous | Contents | Next »