« Previous | Contents | Next »
Listen
Annex 5 Details of the Methodology
The Scottish Executive commissioned Research for Real to evaluate the CitiStat Pilot in Scotland between June 2005 and April 2006. The team comprises Dr Cathy Sharp (Edinburgh; Ayrshire and Arran), Jocelyn Jones (Orkney; Tayside), Alison M. Smith (Aberdeen). Professor Susan Weil from SOLAR at the University of the West of England assisted the team and co-facilitated the CitiStat Exchange Inquiry event.
Action research evaluation
The research team used an action research approach to the evaluation 3. The emphasis in action research is on collaboration between all those involved in the inquiry, so that the knowledge developed in the inquiry process is directly relevant to the issues being studied; this evaluation aimed to actively engage the case study teams in the evaluation process generating useful, practical findings as it went along. Emerging findings were fed back into the on-going CitiStat process in each area. The team worked slightly differently across the different case study areas; each evaluator adopted their own approach, and in each case the participants in the local area decided their own indicators of success to best reflect the way the Pilot was developing in their area and local priorities.
All the case studies held evaluation sessions after CitiStat sessions where participants were encouraged to reflect on the sessions and performance to date. Action research draws on many forms of evidence; this evaluation used a number of different approaches to data collection and analysis including;
- visual inquiry methods
- interviews with key Panel members
- questionnaires to Podium members and observers (Edinburgh)
- maintaining personal diaries (Aberdeen)
- observations of the independent evaluators and co-facilitators.
CitiStat Exchange Inquiry Event January 2006
The CitiStat Exchange Inquiry event was held on 27 January 2006 involving all the CitiStat case studies as a way of sharing experience across the Pilot and deciding on the key lessons, both locally and nationally.
The purpose of the event was to:
- Facilitate sharing of experience of each of the case studies.
- Draw out the strategic and systemic lessons of the different Pilot areas.
- Identify any emerging patterns across the Pilot areas and consider the key implications for future learning locally, nationally and across sectors.
- Explore the potential to sustain the CitiStat process and generate momentum to do this where there is interest.
- Provide further evidence of the impact of CitiStat for the final evaluation report.
Around 40 delegates attended the event from all four case studies and the Scottish Executive. The event was Chaired by the Scottish Executive PIU and opened by the Head of Analytical Services Group. The experience to date of each case study was presented to the event through the use of different presentational forms: story boards, graffiti boards, posters and a rolling video presentation on a laptop. The event commenced with an opportunity for delegates to view the different displays and talk to others about their experience of CitiStat. There were then facilitated small table discussions about what people had noticed and what seemed to be most important about what they had seen and heard. Delegates then identified a series of key questions for fuller discussion in the afternoon.
The afternoon sessions was structured to allow delegates to participate in a series of table discussions about the key questions identified in the morning.
These were:
- Do we know enough to roll out CitiStat?
- Which elements of CitiStat do we wish to keep?
- How do we make it sustainable?
- What is the potential for using CitiStat as a driver for organisational change?
- How can CitiStat work across agencies, services and departments to adopt a more 'whole system' approach?
The event closed with a plenary session in which a delegate from each of the case studies gave a short response to the day. Susan Weil provided some reflections on the day and closing comments.
« Previous | Contents | Next »