Final Report > Chapter 27: Care of an Appropriate Standard > Monitoring of performance > Information and clinical audit

Information and clinical audit

46 Here we refer to the need to be able to collect information about performance, to assess it, to change and improve it where necessary, and, just as importantly, to revisit the standards with a view to changing or developing them. Information is the basic building block of any system of standards and quality. In the past, there have been great difficulties in collecting information, based not only on the technical problems associated with establishing proper systems for doing so, but also on differing approaches as to what should be recorded, and then how it should be classified and grouped for analysis.

47 It is a truism that central to any process of reporting and recording information is agreement on what is being reported. Regrettably, lack of agreement, as to how, for example, to categorise a particular procedure, or what period of time should be used to measure the outcome of a procedure, has bedevilled initiatives in the fields of audit and performance assessment. We heard from Professor de Leval, Professor of Cardiothoracic Surgery, Great Ormond Street Hospital, in the course of a Phase Two Seminar [23] that, even in the relatively small and cohesive world of paediatric cardiac surgery, three ways of categorising procedures exist at an international level, and it is extremely difficult for those involved to agree on which one to adopt. Indeed, Professor de Leval went so far as to urge that, in the absence of agreement, a solution should be imposed. Otherwise, he argued, progress would be impossible and the loser would be the patient. We agree that standardised methods of reporting and classification of information are crucial, both in paediatric cardiac surgery, and generally. We are persuaded that, in the case of paediatric cardiac surgery, with the active engagement of the Royal Colleges, the DoH and other interested bodies, agreement can be reached. But, if it cannot, the issue is sufficiently important to warrant the imposition of standardised categories by the NHS Information Authority.

48 We have seen that there have been sections of the medical profession in the past who have been suspicious of data collection and audit. It is important to understand the basis for this suspicion. Fundamentally, it arises from an anxiety that data would be either inaccurate, or that, when analysed, complexities such as case mix will not be adequately explained or understood. There are concerns that data will be interpreted and used in an uncritical way, to support criticism or invidious comparison. We note and welcome the point made to the Inquiry by the Royal College of Physicians on this matter: `physicians are not frightened or worried about having data to judge ... their performance, as long as those data are seen to be an accurate and fair representation of best care. Most in fact welcome the opportunity to have reliable data to monitor and improve the service they offer. The chance to do this jointly with management could be a huge opportunity for the NHS to make major improvements that would benefit patients.' [24]

49 We believe it important for the future that these anxieties and aspirations are acknowledged. In practical terms this should not mean that information about performance should be hidden. What it does mean is that serious attention must be paid to the development of robust methodologies which can be applied to data so as to give as fair and accurate a picture as is possible. Variables such as case mix and where possible, in the case of surgery, operative risk must be allowed for, so that, wherever feasible, it is possible to compare like with like. It also means that data must be properly analysed with appropriate expertise. We referred in an earlier chapter (Chapter 25 - The Competence of Healthcare Professionals) to the need for a higher priority to be given to educating healthcare professionals about clinical audit and the use and interpretation of data. In addition to this, we consider that clinical teams need to be able to have access to support and guidance in the process of analysis, and that such support and guidance should be available either in, or through, a central clinical audit office which should exist within each trust.

50 Clearly, for progress to be made in this vital area, some rapprochement is essential between the various healthcare professionals: the managers, the doctors, the nurses and others. A crucial first step is to make the process easier for the clinicians. They complain with justification that they do not have the necessary time or resources to collect and classify information properly, and that they work with multiple and incompatible clinical IT systems. At the same time, because they are not involved, clinicians are sceptical of the value of information collected and coded by non-clinicians, principally the information which is collected in hospitals' Patient Administration Systems, and which, in turn forms the basis of the national Hospital Episode Statistics (HES) and numerous national clinical indicators.

51 We have seen in Bristol in all too graphic detail, in what was in reality a small PCS service, how the surgeons, cardiologists and anaesthetists compiled separate sets of data, for similar purposes but without reference to each other. Looking back this now seems tragic. We have seen how the surgeons in Bristol and elsewhere had confidence in general in their `own' data recorded in their personal logs or through their departmental systems, but distrusted the data in hospital administrative systems, and in the UK Cardiac Surgical Register. And we have seen that the tendency was to criticise or explain away the statistics, rather than to ask questions or do something about what the data may imply. We have also seen that data existed at a national level, in the form of the HES database. We have discovered that (at least from 1991), when cross-matched with other sources of data, it is reliable. But, we have discovered that at the time covered by our Terms of Reference the use of HES as a means of learning about mortality rates, for example, was not properly recognised.

52 For the future, we are persuaded of the conclusion reached by our Experts: `... the current "dual" data collection in separate administrative and clinical systems is wasteful and anachronistic.' We also agree with their suggestion as to the way forward: `A vital aim should be two-way linkage between administrative and clinical systems.' [25] We do not advocate abandonment of current systems, far from it. What is needed is much greater attention to improving and streamlining the processes of data collection, coding and validation and to ensuring that respective systems can communicate with each other. The aim must be to collect data as a by-product of clinical practice and to reduce the multiplicity of methods and systems for collecting data. Collecting similar data on different occasions, in slightly different ways, for different purposes, is the discredited way of the past. The contribution that audit can make to improving the quality of care is simply so important that it must be given the priority it needs. It must be underpinned by systematic and reliable systems for collecting and analysing information which have the confidence of healthcare professionals, of those who manage the healthcare system, and the public. It must also be properly resourced, both in terms of funding and of dedicated support staff.

53 Second, as part of the culture of openness which we referred to earlier, and for the benefit of patient and professional alike, information and the results of audit must be shared within the hospital. The corollary is, of course, that, as we have said, healthcare professionals must feel that it is `safe' to share such information. They must be able to work in an environment of mutual understanding and support, in which the overriding purpose is to put patients first. Thus professionals must be entitled to add explanations and caveats to data, so as better to inform those who might study the results made available. Each trust should have, as we have said earlier, a central clinical audit office, which, apart from supporting the process of audit within the trust by providing advice and expertise, can ensure that audit is undertaken and that the results are brought together for the trust as a whole. We note that this is already taking place in most trusts as part of their arrangements for clinical governance. We would add one final point. Healthcare professionals are now coming to see audit as central to their role of caring for patients. We believe that in a patient-centred healthcare service, this recognition should be reinforced by making participation in clinical audit an explicit element in a healthcare professional's contract with the employer.

54 At the national level, we are convinced that it would be of benefit to the quality of healthcare if the monitoring of clinical performance were brought together and co-ordinated by one body. This would include the co-ordination of national audits and the validation and presentation to the public of information about performance in relation to national standards. It would also include filling the gap which we referred to earlier, the absence currently of a mechanism for surveillance, to review trends in performance over time and between trusts. We propose that all of these activities should be brought together under an independent Office for Monitoring Healthcare Performance which should be part of CHI. Its role could include:

• taking the lead in validating and publishing data, collected from within the NHS, on performance in relation to national clinical indicators and nationally agreed standards (this is already undertaken by CHI and the Audit Commission);
• guiding and funding the audit activities of the Royal Colleges and of the various medical organisations (such as, for example, the Society of Cardiothoracic Surgeons), so as to ensure that such activities are part of a wider strategy for auditing clinical performance;
• incorporating the audit functions currently performed by NICE (the funding of national audits by Royal Colleges and the Confidential Enquiries), and the multi-centre specialty audits necessary to monitor the implementation of standards;
• providing guidance on good practice in the analysis of data on performance, at both national and local levels, in particular, on the application of statistical techniques and methods; and
• providing a mechanism for the surveillance of data about performance (clinical indicators, the results of national audits and other relevant information) with a view to identifying patterns of poor (and good) performance.

The Office for Monitoring Healthcare Performance should be part of a reconstituted CHI, which itself must be fully independent of government and the DoH. It would play a key role in supplying rigorous and robust information about the performance of each trust. This in turn could be passed back to individual trusts and would be used in the process of validating trusts.

Footnotes

[23] Summary Report of Seminar 6. See Annex B

[24] Seminar 6. Royal College of Physicians. Position Paper

[25] Spiegelhalter D, et al. `Monitoring Clinical Performance: A statistical perspective', November 2000. See Annex B 9c