Annex A > Chapter 19 - Statistics Relating to the Clinical Performance of Paediatric Cardiac Surgical Services in Bristol Compared with Other Specialist Centres during the Period 1984 to 1995 > Comments of the clinicians at Bristol (and others) on the evidence of the Experts on statistics received by the Inquiry, and the Experts' responses > Differences in the data presented in the evidence of the Inquiry's Experts and the data submitted by the Bristol surgeons
<< previous | next >>
Differences in the data presented in the evidence of the Inquiry's Experts and the data submitted by the Bristol surgeons
`Mr Wisheart has drawn attention to the problems with the raw data and I would endorse those views. The mortality figures for the neonatal arterial switch programme are clearly wrong and I believe that those quoted for ASD and VSD are also incorrect.
`My own surgical log, which I believe to be the most accurate monitor of my surgical performance, demonstrates that between 1990 and 1995 I operated upon 61 children with ASD's and there were no deaths. Indeed for my entire consultant experience in this group I have only 1 recorded death out of 95 operations.
`Likewise my figures for VSD's are 2 deaths out of a total of 72 operations between 1990 and 1995 (6 deaths out of 117 for my whole experience).' [245]
`The conclusions [246] drawn by the statistical team on behalf of the Inquiry are not accepted. It is believed that there are serious flaws with some underlying data. In particular, the results quoted for ASD's and arterial switches are wrong.' [247]
173 Mr Wisheart questioned the accuracy of figures [248] presented in a report to the Inquiry by one of the Inquiry's Experts in statistics. In particular, he raised concerns (by reference to his own and Mr Dhasmana's data) about the number of operations and the number of deaths which the Inquiry's Expert had identified in relation to particular categories of procedure.
174 The report showed 5 deaths in 90 operations for Atrial Septal Defects in 1991-1995. Mr Wisheart stated:
`Mr Dhasmana and I believe there were no deaths out of 102 operations.' [249]
175 The report showed 5 deaths in 50 operations for aortic or pulmonary valve surgery. Mr Wisheart stated:
`Mr Dhasmana and I believe that there was one death out of 35 operations.' [250]
176 The report showed 3 deaths in 23 operations for mitral valve surgery. Mr Wisheart stated:
`Mr Dhasmana and I believe that there was one death out of eleven operations.' [251]
177 In relation to the estimated numbers of excess deaths based on HES data, as shown in the report, [252] Mr Wisheart submitted:
`... [the estimate] is substantially wrong and is likely to be a significant over-estimate of the number of excess deaths ... .' [253]
178 Dr Spiegelhalter, Professor Evans, Dr Aylin and Professor Murray responded:
`The point at issue is the classification of operations. There is no evidence that deaths have been recorded when they have not occurred in more than a very few instances overall. The problem is that the classification of operations is difficult. With random misclassification of type of operation, but accurate determination of death, then [there] will be a tendency for mortality rates in the different groups to be more similar to one another than would be the case if no misclassification occurred. In particular groups there may be a higher rate, but in other groups there will be a lower rate than there should be. Focusing only on the groups with a higher rate is biased. It is for this reason that examination of all open operations was also done in the statistical analysis. The other issue is that coders in different centres, who are each familiar with the OPCS4 system, will tend to code operations in a way that reflects that coding system, rather than clinicians' views. The key comparisons are made between centres, and no doubt, individual clinicians in those other centres are also likely to have different ways of classifying their operations. Random misclassification is likely to make the different groups more similar across centres also.
`... There is very little disagreement between the sources of data in regard to individual children as to whether they died or not. There is disagreement between Mr Wisheart's grouping by diagnosis, and the other sources that are grouped by operative procedure. While it is possible that some groups seem to show a higher rate in the statistical reports provided to the Inquiry than in Mr Wisheart's grouping of the data, there will be other groups where Mr Wisheart's data would seem to have a higher mortality rate than the statistical reports. He has not drawn attention to these, since his own comments apply only to selected groups.' [254]
179 Dr Spiegelhalter et al. further observed:
`It is important to emphasise that the entire analysis of paediatric cardiac surgery at UBHT has been based on operative procedures rather than on diagnosis. This was made very clear in our reports. Two of the major reasons for choosing to use operation were - a) the UKCSR recorded data by numbers of procedures rather than numbers of diagnoses, and b) when comparing different centres, it is likely that agreement about procedures may be greater than agreement on diagnosis. The Submission [by Mr Wisheart] presents its analyses based on diagnosis rather than on operation, and hence considerable discrepancies must be expected between the analysis of the Inquiry's Data (including that of the Surgeons' Logs) and the analysis in the Submission of the Surgeons' Logs.
`... Further analysis based on linkage of HES records with national death certification records has been carried out by Professor Murray ... This shows that in open operations HES identifies around 95% of 30-day deaths (in spite of HES only aiming to capture in-hospital deaths). In conclusion, we do not find statistical evidence to support the statement "that the estimate of excess deaths based on HES data is substantially wrong".' [255]
180 Referring to the apparent discrepancies between HES data and hospitals' departmental records, Dr Spiegelhalter et al. in their Overview Report observed:
`Stark (2000a, WIT 0567 0004) reports substantially lower counts of activity (sum of operations identified as "open" or "closed") measured by HES and reported in Aylin et al (1999) and Spiegelhalter (1999), compared to the numbers of operations recorded in contemporary departmental records. Some undercount must be expected due to the Inquiry's use of admissions [to hospital] as a measure of activity, rather than operations as used in the departmental records. There will be additional contributions due to miscoding of records in HES, and in particular from admissions excluded from the open/closed groups (see Section 2.4). [256] It is difficult to interpret such discrepancies, as there is unknown variability between departmental record systems in, say, what constitutes an "operation". What is important for the Inquiry's analysis is that the same coding and exclusions (on the basis of OPCS4 codes) have been applied to all centres in a consistent manner. As noted at Section 2.5 above, [257] random errors in coding will tend to reduce differences between groups and hence between centres.' [258]
181 Dr Spiegelhalter et al. reached the conclusion that:
`Although we have had some months to reflect on the issues and carry out further examination of the available data, we see no statistical justification to revise to any substantial extent the analyses and opinions stated in written and oral evidence to the Inquiry.' [259]
<< previous | next >> | back to top
Footnotes
[245] WIT 0084 0149 - 0150 Mr Dhasmana
[246] As set out in the initial statistical reports to the Inquiry, published in November 1999
[247] SUB 0010 0019 Mr Dhasmana; see Chapter 3 for an explanation of clinical terms
[248] Mr Wisheart refers in his submission to the table presented at INQ 0015 0004; `An initial synthesis of statistical sources concerning the nature and outcomes of paediatric cardiac surgical services at Bristol relative to other specialist centres from 1984 to 1995', October 1999, Dr Spiegelhalter. The number of operations and the number of deaths quoted by Mr Wisheart appear to be extracted from the table presented at INQ 0015 0048
[249] SUB 0009 0025 Mr Wisheart
[250] SUB 0009 0025 Mr Wisheart
[251] SUB 0009 0025 Mr Wisheart
[252] The estimated numbers of excess deaths are reported in the table at INQ 0015 0004
[253] SUB 0009 0025 Mr Wisheart
[254] INQ 0034 0015; `A Response to Submissions on behalf of Mr JD Wisheart, Appendix 2, The Inquiry's Statistical Analysis', May 2000, Dr Spiegelhalter et al. (emphasis added)
[255] INQ 0034 0008; `A Response to Submissions on behalf of Mr JD Wisheart, Appendix 2, The Inquiry's Statistical Analysis', May 2000, Dr Spiegelhalter et al.
[256] See INQ 0045 0015; `Overview of statistical evidence', September 2000
[257] See INQ 0045 0015; `Overview of statistical evidence', September 2000
[258] INQ 0045 0018; `Overview of statistical evidence', September 2000
[259] INQ 0034 0002 - 0003 ; `A Response to Submissions on behalf of Mr JD Wisheart, Appendix 2, The Inquiry's Statistical Analysis', May 2000, Dr Spiegelhalter et al.