Inspecting the Inspectorates

The Commission for Health Improvement (CHI) has been reviewing NHS
organisations for three years, but does it meet the expectations of the public, the Government and other organisations? A new report Inspecting for improvement published by the Prime Minister’s Office for Public Services Reform (OPSR) asks the question what makes an effective public sector inspectorate? The report has attempted to identify what is meant by effective inspection, outline the arrangements by which effective inspection can be achieved and state its expectations of inspectors. The report defines inspection as follows: Inspection of public services is a review, conducted by external bodies that should:

• be independent of the service providers
• provide assurance to ministers and the public about the safe and proper
  delivery of those services
• contribute to improvement of those services
• report in public
• deliver value for money

The report lists a number of principles which should be followed by inspectorates. Below we have tried to outline where CHI is against each of the criteria. We are continuing to review the use of these criteria until 31 March 2003. It is then for CHAI to take forward.

1. The purpose of improvement

The report states that there should be an explicit concern on the part of inspectors to contribute to the improvement of the service being inspected. In framing recommendations, an inspector should recognise good performance and address any failure appropriately
Improvement has always been a stated part of CHI’s core principles and ambitions in its work. It can be demonstrated that CHI meets this to some degree through the anecdotal evidence that our work has had an effect and made an impact. CHI has worked with the strategic health authorities following up action plans and assessing progress that has been made to inform the star ratings in many trusts. We have also commissioned a small evaluation of our impact through the University of Manchester.

2. Focus on outcomes not on processes

Which means considering outcomes for the end users of the services rather than concentrating solely on internal management arrangements CHI meets this criteria. We believe that outcomes are driven by good processes, Clinical governance reviews are based on an assessment of the systems and processes that must support high quality patient care. Our publicly accessible reports reflect this.

3. Take a user perspective

Inspection should be delivered with a clear focus on the experience of those for whom the service is provided, as well as on internal management arrangements CHI has developed a robust process enabling service users to contribute their views and experiences to investigations and reviews of individual trusts – with such evidence seen as an important element of our findings. Every review or investigation team has a lay member on it. In mental health reviews CHI recruits mental health service users through MIND, to be an integral part of our review teams. They are fully trained and use their experience in mental health to interview current service users. For each review CHI also targets patient groups in each area inviting people by letter to take part in reviews, either in face to face interviews with a review manager, online using the website, by letter or over the telephone. Our reports are produced in plain English. One recently published investigation report, looking into learning disability services, was published in two versions, one being an accessible version for people with learning disabilities. We worked with Mencap to produce this report. We also commit significant resources to the production of translations and other formats such as Braille or audio tape as requested. Work is ongoing to improve the stakeholder involvement even more.

4. Be proportionate to risk

Over time inspectors should modify the extent of future inspection according to the quality of performance by the service provider. For example, good performers should be allowed to earn less inspection or other freedoms. CHI was established to undertake a programme of clinical governance reviews and local reviews of national service framework service delivery. CHI undertook to review all NHS organisations in England and Wales. However investigations are
commissioned on the basis of risk criteria.

5. Encourage rigorous self assessment by managers

Inspectors should encourage rigorous self assessment by managers. Inspectors should challenge the outcomes of managers’ self assessments, take them into account in the inspection process and provide a comparative benchmark. CHI has produced a number of self assessment tools on their website (www.chi.nhs.uk) which trusts can download and use when they want, outside the clinical governance reviews. However the self assessment tools are used as part of the clinical governance review in mental health reviews and work is underway on similar tools for use in primary care trusts’ clinical governance reviews. The tool is available on our website for trusts to use as a component of their own improvement endeavours. Following the secretary of state for health’s request for CHI to conduct an audit of child protection arrangements in the NHS, linked to the publication of the Laming report into the death of Victoria Climbie, we have developed a national audit of child protection arrangements that is based on self assessment. (www.chi.nhs.uk/eng/child/index.shtml)

6. Use impartial evidence

Inspectors should use impartial evidence. Evidence, whether quantative or qualitative should be validated and credible CHI uses a robust analytical process for reviewing a range of evidence and information. All evidence is recorded on specialist software. Reports are subject to quality assurance which tracks all the reported statements back to source.

7. Disclose the criteria for judgement

Inspectors should disclose the criteria they use to form judgement CHI is open about the processes and methods it uses to carry out the clinical governance reviews. We publish information about the scores used to assess the various components of clinical governance on our website: (www.chi.nhs.uk/eng/inspections.shtml). There is a detailed explanation of each of the scores and what they mean , as well as the review issues that are looked at. Similarly with performance ratings, CHI has shared the indicators on which organisations ’ performance is judged.

8. Be open

Inspectors should be open about their processes, willing to take any complaints seriously, and able to demonstrate a robust quality assurance process. CHI is open about the processes and methods it uses to carry out clinical governance reviews. There is a clinical governance review manual for each sector, which is published on the CHI website, explaining in detail the process used and the methodology. CHI takes complaints very seriously and has a formal complaint process in place with dedicated personnel dealing with complaints and enquiries.

Throughout CHI’s work and clinical governance reviews there is a robust quality assurance process, which has recently been improved with the appointment of quality managers, to read all reports and ensure quality assurance measures are undertaken. Each department within the organisation has their own quality assurance processes to ensure the consistency, robustness and quality of our findings. CHI also uses a readers panel, who have not been involved in a particular review, to read a report where there is any discussion or disagreement around the scores.

9. Have regard to value for money

Inspectors should have regard to value for money, their own included CHI redesigned its clinical governance review process last year to make it more efficient. We have now got quality managers in place to ensure that processes are efficient and effective and we have an impressive track record of working with other organisations including safe guards for children, and working with Social Services Inspectorate (SSI) on child protection work. The joint working with the Audit Commission and SSI on national service frameworks and the planned collaboration with SSI on joint mental health and social care reviews also reduces the burden on inspectorates and organisations being reviewed.

10 Continually learn from experience

Inspectors should continually learn from experience in order to become increasingly effective. This can be done by assessing their own impact the service provider’s ability to improve and by sharing best practice with other inspectors CHI has welcomed the NHS Confederation surveys about CHI’s work and we have worked alongside other organisations looking into CHI’s processes. We have also commissioned our own evaluation work. One of CHI’s principles is to apply the same standards of continuous improvement to ourselves that we expect of others. CHI consults on new methods it is developing and changes processes as a result of learning , for example in developing the methodology for inspecting PCTs CHI changed its process resulting in improvements, following consultation and discussions with the pilot review PCTs. The report by OPSR has been adapted into The Governement’s Policy on Inspection of Public Services. This is a policy document to support departments working with inspectorates and service providers to ensure that inspectorates are making a difference, an improvement and are meeting expectations. The OPSR report can be found on their website by clicking on this link: (www.pm.gov.uk)