|
|
|
|
 Home
Page
Search
Final
Report
Interim
Report
Evidence
- Transcripts
- Witness Statements
- Submissions
- CCNR Report and associated
documents
- Statistical Reports and associated
documents
- Background Papers
- Expert Papers
Inquiry
Seminars
About
the Inquiry
Help
|
 |
Daily transcripts
Hearing summary
12 July 1999
Inquiry hearings today heard evidence from Mr Hugh Ross, current Chief
Executive, United Bristol Healthcare NHS Trust (UBHT) and Mr Ian Barrington, General
Manager, Directorate of Children’s Services, UBHT. Issues under discussion today were
post-mortems and inquests. Mr Ross began by saying that procedures relating to post
mortems (PMs) at UBHT during the period of the Inquiry were consistent with elsewhere in
the NHS. He went on to describe the difference between a hospital PM and a Coroner’s
PM and explained the responsibility of clinicians to obtain informed consent. He also
discussed the issue of retention of tissue following PM and the keeping of records in
relation to this. Mr Ross then described UBHT’s reaction to the disclosure of
information to parents about their child’s PM and the co-operation of the Bristol
Heart Children Action Group in this matter.
Mr Ian Barrington then went on to describe his responsibility in
relation to PM, saying that he would have only been involved if a concern had been brought
to his attention. He then talked about his role in dealing with requests from parents for
information about their child’s PM. He described the setting up of a help line and
subsequent investigations assisted by Professor Berry (Consultant Paediatric Pathologist)
to deal sympathetically and swiftly, but as accurately as possible, with the queries from
parents. He concluded by discussing the information surrounding PM that is now discussed
with parents following bereavement.
|
FULL TRANSCRIPT
1 Day 38, 13th July, 1999
2 (9.45 am)
3 THE CHAIRMAN: Good morning, everyone. Good morning,
4 Mr Langstaff. Today we have a somewhat different
5 configuration of the Inquiry hearing chamber. This is
6 in conformity with our desire to use whatever method we
7 can so as to enable us to understand the matters into
8 which we are enquiring.
9 Mr Langstaff, you have the floor.
10 MR LANGSTAFF: INTRODUCTION RE STATISTICS
11 MR LANGSTAFF: Thank you, sir. Sir, some time ago, when
12 I stood up to open this case, as part of what I had to
13 say, I showed a slide on the screen. That slide has now
14 been called up again. It shows, in simple numbered
15 form, the stages by which this Inquiry proposes to lay
16 the evidence before you, the Panel, so far as statistics
17 can assist.
18 What we are at is essentially Stage 1, the
19 preliminary critical overview of sources of data.
20 Statistics are, of course, essentially figures and
21 conclusions that can be drawn from the figures as
22 figures. I would not wish anything to be taken from the
23 next two days which would in any way suggest that
24 figures can tell the whole picture. They can be
25 helpful, but statistics in themselves can never be the
0001
1 whole picture. Indeed, they can be dangerously
2 seductive.
3 The conclusion which you have to draw at the end
4 of this Inquiry as to the adequacy of care is
5 a summation of many different strands of evidence,
6 principal among them, the stories which you have heard
7 from individual parents, each one of which tells its own
8 tale; the conclusions that you are able to draw from the
9 evidence of the way in which matters were structured at
10 Bristol and elsewhere in the country; the conclusions
11 that you may wish to draw from seeing whether or not
12 national policies which are thought now to have force
13 and importance behind them were or were not adopted here
14 locally in the years 1983 to 1995; the conclusions which
15 we may reach from hearing individual people who were, as
16 it were, main players in what happened in Bristol; and,
17 as part of picture, but only as part of the picture, the
18 conclusions one gets from looking at figures as a whole.
19 As I said when I opened this Inquiry to you, it
20 would be very wrong to forget that at the heart of the
21 figures are the cases of individual children, some of
22 whom will have been treated and will thankfully have
23 survived; some of whom will have been treated and sadly
24 will have either suffered subsequently in consequence of
25 the treatment or of the condition which led to the
0002
1 treatment, and others, tragically, because we have
2 always said it was a tragedy, will have lost their
3 lives.
4 The GMC, when it examined the cases against the
5 two surgeons and Dr Roylance, looked at and examined in
6 detail 29 cases. We, for our part, have obtained the
7 clinical records of every child that has been identified
8 to us as having had cardiac surgery, closed or open,
9 from 1983 until 1995, and will, in a way which I shall
10 describe, attempt to draw such conclusions as we can
11 from them.
12 In that sense, every single child, survivor or
13 not, will have his or her case painted as part of the
14 picture which you will have to determine your findings
15 upon.
16 Today and tomorrow we will hear from a number of
17 experts who have different roles to perform in respect
18 of the various data sources which will deal with the
19 statistical side. That is part of the picture. First,
20 after the few words that I have to say by way of
21 introduction, I shall ask Dr David Spiegelhalter to come
22 and tell us about the way in which, as an expert
23 statistician, he sees statistics helping the Inquiry as
24 best they can towards reaching a conclusion in respect
25 of the terms of reference.
0003
1 Once he has identified essentially what the
2 process is, we will then concentrate on this part of the
3 process, the preliminary critical overview and will hear
4 first from Mr Richard Willmer, the chief statistician at
5 the Department of Health.
6 When he gives evidence, I hope that what will
7 develop will be a symposium or seminar, if you like, in
8 which, far from asking all the questions myself, Dr Paul
9 Aylin, who sits nearest the screen to my right, of
10 Imperial College, will intervene and develop a dialogue
11 with Richard Willmer in which three things essentially
12 will be explored, three essential messages for the
13 Inquiry. That is, so far as the data set with which
14 Mr Willmer is concerned -- I should say data sets in the
15 plural and I will explain why in a moment -- whether
16 they have weaknesses, for our purposes; secondly,
17 whether they have strengths for our purposes; and
18 thirdly, what use the Inquiry can potentially make of
19 the information which the data set can tell us.
20 After we have examined the data sets to which he
21 can speak, the national data sets, we will hear from
22 Mr Bruce Keogh, who will tell us about the
23 Cardiothoracic Register, a register which has been in
24 existence since 1977. Essentially, I anticipate that it
25 should be possible to develop a dialogue in the same way
0004
1 principally between him and Dr Spiegelhalter, who sits
2 a little bit further away from the screen to my right,
3 examining exactly the same things: the strengths, the
4 weaknesses and the utility of that data source for our
5 purposes.
6 Tomorrow we expect to narrow down the field from
7 the national to the local, and look at the major data
8 set available locally.
9 So far as the available data is concerned, if
10 I just run through what is available, and their apparent
11 utility to the Inquiry, the first of those data sets
12 that you will hear of from Mr Willmer is that contained
13 in the hospital episode statistics system, or HES for
14 simplicity. This was set up by the Department of Health
15 to collect information about activity in all National
16 Health Service hospitals in England. Immediately,
17 anyone listening with a critical ear will have detected
18 one gap: it deals with NHS hospitals and not others.
19 Anyone listening might already have picked up that it
20 deals with inpatient activity and it was not essentially
21 designed to tell surgeons or hospitals, putting it
22 crudely, how well or how badly they were doing by
23 comparison with others; that was not its function.
24 One of the matters which we will wish to explore
25 is whether it can, despite having been designed for
0005
1 other purposes, nonetheless be used for that function in
2 order to retrospectively inform us on the Inquiry. The
3 data should, in theory, be complete, but what it
4 measures is not operations; it is not directly
5 morbidity; it is what are called "finished consultant
6 episodes" and a finished consultant episode, as we have
7 already heard in evidence at an earlier stage in this
8 Inquiry, is an occasion when a patient comes under the
9 care of a particular named consultant. For as long as
10 that patient is an inpatient, under the care of that
11 particular consultant, that is one unfinished consultant
12 episode. When he is discharged from that care, it
13 becomes a finished consultant episode.
14 What it means, because finished consultant
15 episodes are picked up each as a unit, is that if
16 a particular patient is in the course of his inpatient
17 stay under the care of more than one consultant, he
18 moves from one department to another to another, let us
19 suppose -- it sometimes happens as we know -- then there
20 will be three finished consultant episodes although
21 there will be only one inpatient stay -- one course of
22 treatment, if you like. Looked at from the patient's
23 point of view it is one hospital stay; looked at from
24 the national statistics, it is three.
25 Similarly, if the surgery, let us suppose, is
0006
1 broken by a period of discharge from hospital followed
2 by a return of that patient, maybe within months,
3 perhaps, because the surgery needs to be revised, that
4 gets counted as a second and separate consultant
5 episode.
6 So immediately, one can begin to see something
7 of the advantages, which is in the completeness of the
8 data, and something of the disadvantages, which is that
9 it was designed for purposes which are not ours.
10 I do not want to say too much more about those
11 statistics because it is best dealt with, I think, by
12 explanation from Mr Willmer and in particular because
13 I am conscious here that he and our experts know far
14 more about HES than I could ever hope to.
15 The second source is the Cardiothoracic Register,
16 and again, a brief description will help. This was set
17 up by, it happens, Sir Terence English who gave evidence
18 to us, in 1977 following a pilot project in 1976, and it
19 aimed to provide information to cardiothoracic surgeons
20 on the volume, the type and the distribution of work
21 performed, to promote the rational use and future
22 planning of cardiothoracic services and to improve the
23 overall quality of the cardiac services delivered by
24 enabling individual surgeons to compare themselves
25 against national data.
0007
1 The way in which the data was collected was for
2 each unit to be asked, invited -- cajoled perhaps --
3 into supplying data to the Society centrally in the
4 country. There was no compulsion about it. You will
5 hear, when Mr Keogh gives his evidence, that in fact
6 some units did not provide data. It might appear, at
7 least from one of the charts which he has helpfully
8 produced for us, that some units consistently did not
9 provide data. If they had data to give and did not
10 consistently provide it, one has to ask whether there
11 may be some reason for that, whether it was shame at
12 poor results, for instance; whether it was simply an
13 unwillingness to be co-operative for some reason of
14 principle; whether it was chronic indolence; whether it
15 was some other reason. It is very difficult to
16 speculate. What one can say in respect of the figures
17 is that to the extent that individual units did not
18 provide figures to the central register, the central
19 figures give one less than the full story. The extent
20 to which that is less than the full story, that may
21 affect the use we in the Inquiry can make of those
22 figures, will be a matter for your judgment, having
23 heard what has to be said about them.
24 The register was kept carefully from 1977
25 throughout the period with which we are interested. In
0008
1 that respect, it differs from the HES data to which
2 I have already referred, because the HES data began as
3 a process in 1987/88; it evolved as a data collection
4 exercise becoming, we think, as a preliminary
5 observation, at its most reliable for various reasons in
6 1991/92. So whereas one may use the register for all
7 years from 1983 to 1995, one can only use HES, really,
8 probably, from 1991/92 onwards.
9 One of the problems with the register is that in
10 order to get it off the ground -- this is my probably
11 imperfect lay understanding -- the Society had to
12 promise the reporting surgeons that the data they
13 supplied would be confidential; it would be anonymised
14 so that the individual unit or surgeon would know what
15 his results or the unit's results were, but that would
16 be information kept to the unit or the surgeon. The
17 only point of comparison would be with the whole of the
18 rest of the country generally, such as was reported to
19 the register.
20 Considerable lengths were taken to secure that
21 anonymity, so that there was no fear among surgeons that
22 the data would be used for any purpose to which they did
23 not consent.
24 To an extent, that means that one cannot look at
25 the register and compare Bristol with, let us say,
0009
1 Southampton or Great Ormond Street or Birmingham,
2 because the data is not constructed to enable us to do
3 that.
4 So much for those two data sources.
5 The other four main sources which we believe we
6 have identified are these: there was a computerised
7 information system kept in the South West by consultant
8 cardiologists at the Bristol Royal Infirmary and the
9 Bristol Children's Hospital throughout the period of the
10 Inquiry. The evidence that we have so far suggests that
11 this system was designed to allow the cardiologists
12 ready access to essential summary information on
13 patients when clinical records were not available for
14 audit purposes and to facilitate the presentation of
15 clinical data at meetings.
16 The hardware and the software were provided by the
17 Bristol and South West Children's Heart Circle and the
18 current version is held on a personal computer using
19 Borland Paradox software. Most of the data were entered
20 by a secretary working for the cardiologists.
21 We have an electronic copy of that register and
22 plans are in hand to commission an independent review
23 with statistical analysis of that which it shows.
24 A full assessment of the data will necessarily involve
25 cross-comparisons with other relevant data sources so
0010
1 far as is feasible.
2 Next, we have the Trust Patient Administration
3 System. It is a computer system for the purpose of
4 tracking hospital activity, and it forms the basis for
5 the returns to the Department of Health under the HES
6 system.
7 We do not have data before 1st January 1988. The
8 period which it covers for our purposes is 1st January
9 1988 until 31st December 1995, and it again needs to be
10 emphasised that this system was not designed for audit
11 purposes; it was an information system to help with the
12 tracking of patients, so that the hospital knew who was
13 where, if you like, it had the ability to print off
14 labels and addresses to keep in touch with patients and
15 so on: essentially administrative purposes, also
16 planning purposes and when the purchaser/provider split
17 began, for contracting purposes. It was not used for
18 clinical purposes and therefore the usefulness of the
19 data may be limited when one looks to see whether it can
20 tell us anything about adequacy or quality of care.
21 The data was generated by clinical coders using
22 standard classification systems.
23 The quality of the data appears to be good. We
24 think that probably it will mainly serve as a useful
25 descriptor of what happened in respect of children
0011
1 receiving heart surgery from 1988 onwards, and to
2 cross-validate other information. One of the great
3 advantages of having a number of different sources of
4 data is that if they all end up saying pretty much the
5 same thing, one can draw strength from that.
6 Individually there may be weaknesses which would not
7 allow anyone with any integrity to place reliance upon
8 that individual source, but if the fact is that one
9 source cross-refers to another, or one source produces
10 the same result as another, one can draw strength from
11 that and the two together may be worth much more than
12 the sum of the individuals.
13 The next major data source is the records
14 themselves, the clinical records. These, the Inquiry
15 has in great number. They have all been copied, kept
16 and stored at the Inquiry offices, and each set has been
17 coded by clinical coders, an experienced team, using
18 codes to demonstrate what the diagnosis and what the
19 procedure in each case was.
20 A fuller description of the way in which the
21 Inquiry has dealt with the clinical case records, and
22 indeed, a fuller description of each of the data sources
23 to which I have briefly referred in this introduction,
24 is contained in a paper which the Inquiry published last
25 week entitled "A Preliminary Overview of the Data
0012
1 Sources."
2 Perhaps it is not necessary for me therefore to
3 deal in any particular detail at this stage with the
4 clinical cases, save to say this: that one of the great
5 advantages is that we can be confident that, unlike any
6 other researcher of whom we are aware who has looked at
7 Bristol over the years 1983 to 1995, we have as full
8 a picture as one possibly can by going back to the
9 original source data, as it were, the clinical records.
10 We have to accept that clinical records can of course
11 have pages missing; they can, of course, be incomplete.
12 But in general, they are kept for the purposes of care
13 of patients, in general they are of good quality, and
14 what is true in general throughout the Health Service we
15 have found, I am told, to be true in respect of these
16 records in Bristol.
17 Of course, one of the advantages of having the
18 available records is that one can look at those
19 diagnoses which have led to successful surgery from 1984
20 to 1995, those which have led to unsuccessful surgery,
21 and unsuccessful not only in terms of mortality at some
22 later date, but also in terms of morbidity. So the
23 whole range of outcomes it is possible to study from
24 looking at the clinical case records.
25 One of the hopes and expectations in the next
0013
1 couple of days is that those who are listening and have
2 an interest in the Inquiry's procedures and its findings
3 may be able to make to us critical, constructive
4 comments as to what may best be found from a combination
5 of the data sources I have mentioned thus far.
6 One cannot forget, as the last of the major
7 sources, the surgeons' logs. We have taken and we have
8 or will publish the statements of the doctors describing
9 the logs of the surgeons who were concerned in the
10 operations. Those logs, generally speaking, identify
11 the procedures which those surgeons carried out. They
12 are, of course, a useful means of cross-checking against
13 other data to see how complete and accurate that other
14 data source is.
15 Having spent some time dealing with the six main
16 sources as they appear to us, I do not want to take too
17 much time dealing with what might be thought to be
18 lesser sources, but lest it be thought that this Inquiry
19 is leaving any stone unturned, if I can simply run
20 through and identify thus far the 10 principal other
21 data sources, and invite anyone who is listening who
22 knows of any further data which may help us, at any
23 stage during our terms of reference, 1984 to 1995, to
24 let us know.
25 We have:
0014
1 (1) a card index system which the cardiologists
2 kept for part of the period;
3 (2) the logs which were kept in the operating
4 theatres;
5 (3) records which were maintained by clinical
6 perfusionists;
7 (4) logs which were kept by anaesthetists;
8 (5) the cardiac surgeons in Bristol themselves had
9 a system, a patient analysis and tracing system, PATS,
10 kept on a METASA system;
11 (6) the NHS regional data archives;
12 (7) something we will explore this morning with
13 the help of Mr Willmer: the hospital inpatient data
14 system, HIPE;
15 (8) the Confidential Enquiry into Stillbirths and
16 Deaths in Infancy, a national source but only, we think,
17 of partial relevance to us, sometimes known by its
18 acronym CESDI;
19 (9) the National Confidential Enquiry into
20 Peri-operative Deaths, NCEPOD;
21 (10) the national reporting system for congenital
22 abnormalities, which has an obvious relevance to the
23 conditions which led to operation in Bristol.
24 Those are the 10 principal lesser sources. I do
25 not wish to imply from that list that is all that we
0015
1 have actually covered; there is more, but nonetheless,
2 it will be unwise of anyone listening to assume that we
3 know of the data source that he or she may know of and
4 so if there is any further information which we have
5 overlooked or which no-one has told us about relating to
6 the years 1984 to 1995, to paediatric cardiac care, then
7 please tell us about it.
8 Sir, the purpose as I have indicated in looking at
9 each of the data sources is to identify so far as this
10 part of the picture is concerned what are the what are
11 the strengths, what are the weaknesses, what is the
12 usefulness for us. This is part of the overall picture
13 which the statistical investigation should help to show
14 us, as indeed indicated on the screen in the four
15 points, and it is perhaps helpful if then, at this
16 stage, I ask Dr David Spiegelhalter to come forward and
17 to explain from his perspective the content of each
18 these stages and the overall strategy by which figures
19 can assist, even though they may not control our
20 findings.
21 Dr Spiegelhalter, you would wish, I think, to
22 affirm?
23 DR DAVID SPIEGELHALTER (AFFIRMED):
24 Examined by MR LANGSTAFF:
25 Q. Dr Spiegelhalter, can you give us your full name and
0016
1 your qualifications, please?
2 A. My name is David John Spiegelhalter. I have an MA in
3 mathematics from Oxford University, an MSc and PhD in
4 mathematical statistics from London University. I am
5 a Chartered Statistician with the Royal Statistical
6 Society and I am currently senior scientist working for
7 the Medical Research Council at the Medical Research
8 Council Biostatistics Unit in Cambridge.
9 Q. You have had some involvement, I think, with the subject
10 matter of the Inquiry before, having given evidence at
11 the General Medical Council?
12 A. Yes.
13 Q. If we look at the screen, it sets out in very bare
14 bones, does it, the processes as you would see them by
15 which the statistics can inform the process of the
16 Inquiry?
17 A. Yes. Could I make a few preliminary points? I would
18 first like to say, as a professional statistician, that
19 one must really acknowledge this is a very tricky area
20 and for a number of reasons.
21 I think the first is that we are looking backwards
22 into a time when the whole process of collecting data in
23 a rigorous way was rudimentary, to say the least, not
24 least because of the lack of easy computer facilities,
25 but also just because of the culture, which has now
0017
1 changed quite dramatically.
2 So as you have made clear, there are a variety of
3 data sources, but none of them are what we would
4 consider as really perfect, and of course if we were
5 doing this prospectively, it would be a very different
6 matter.
7 The other crucial issue is that while, as you have
8 said, there can be very detailed analysis of what went
9 on in Bristol by going back through the case records,
10 that is not the only issue. One of the crucial things
11 is comparison with what, perhaps, might be considered as
12 acceptable practice or standard practice in levels of
13 performance in the country at the time, and while I do
14 not think the Inquiry has quite the resources to go
15 through every record in the country, it means that there
16 is going to be a problem which has to be faced of making
17 comparisons with other centres. That obviously has to
18 be faced by looking at the available data, as you have
19 described it.
20 The final point I would like to make, which
21 I suppose is slightly more technical, is what in the
22 trade is known as "multiple comparisons". As one looks
23 at finer and finer level in terms of, say, disease
24 categories or maybe over a short period of time or maybe
25 just looking at what one surgeon is doing, the numbers
0018
1 are going to get smaller and smaller and that means from
2 a statistical point of view, the play of chance, as bad
3 luck, is that starts having more and more of a role.
4 From a statistical point of view one would like big
5 numbers so everything is stable so one can attribute
6 differences to some underlying cause rather than just
7 the play of chance. So there is always a danger as one
8 looks more and more into these data sources and at finer
9 and finer detail that apparent differences are going to
10 come up which it is very difficult to say whether these
11 are real or not.
12 So inevitably, there has to be some sort of
13 grouping by diagnostic category or over time in order to
14 try to smooth out some of these potential I suppose, you
15 know, random or chance occurrences. I think we will
16 come on to that, but any grouping like that obviously
17 should be done in an unbiased way and should be done
18 without looking at the data first.
19 Q. You used the word "unbiased" there in the statistical
20 sense?
21 A. I could mean it in a somewhat non-technical sense as
22 well. One should do that sort of grouping before
23 looking at the results so one is not just picking out
24 the worst things one can find over a long period.
25 Q. So one is beginning, as it were, with the groupings, the
0019
1 criteria, and then looking at the data in the light of
2 those groupings and criteria?
3 A. That is the idea.
4 Q. The purpose, then, of this stage of the investigation is
5 to establish what, essentially? What would you hope
6 that the Inquiry will be able to take away from the next
7 day and a half?
8 A. We are starting on these four stages, really, and
9 bearing in mind the points I made about the difficulty
10 of drawing firm conclusions from the data available, the
11 first stage is really to identify the sources and to get
12 some idea of what their quality is and what conclusions
13 might be able to be drawn from them. As I said, it is
14 best to do that before we actually see what that data is
15 saying, so that we can make a fair judgment as to what
16 might be drawn without knowing what actually is found
17 inside those data sets.
18 The kind of information that would be very good to
19 know is who completed the data, who was responsible for
20 submitting this data, was it different people for each
21 of the different data sources? Because in a sense, as
22 you said, it would be very nice if these data sources
23 corroborated each other, we would feel happier about the
24 evidence, but only in as much as that evidence perhaps
25 came from different sources. If they are all coming
0020
1 from the same source, then that corroboration might be
2 somewhat illusory.
3 I think it would be very useful to know about what
4 the judgment is about the quality of the data sources
5 and that quality can be judged in terms of the coverage,
6 you know, are all the centres looking across the country
7 now contributing their data, and in terms of the
8 completeness of the records being given and in terms of
9 the accuracy whether we can actually believe what we can
10 see in front of us.
11 Q. Just distinguishing those three, because I think those
12 are words we will probably come across a number of
13 times, "coverage" you use to mean whether it covers
14 every part of the country: geographical, as it were?
15 A. Yes whether some centres are just systematically not
16 providing data.
17 Q. "Completeness" is giving the questions one is asked on
18 the form which is filled in and sent back to the data
19 centre: have all the questions been answered?
20 A. Yes, completeness is to do with missing data on some of
21 the aspects of the cases being submitted.
22 Q. So for instance if one had a data set in which the date
23 of birth was quite often missed off, or the date of the
24 operation was quite often missed out, that data source
25 to that extent would be incomplete, even though the
0021
1 coverage might be 100 per cent across the country?
2 A. Yes, and even more important would be data sources in
3 which the outcome was not recorded accurately.
4 Q. "Accuracy" is the last?
5 A. Accuracy is saying, given the data provided, can we
6 actually believe it, that it is appropriate? Are all
7 the cases being included from that centre; are all the
8 deaths being recorded appropriately; are the details
9 correct?
10 Q. I stopped you in mid-flow.
11 A. I think that, I suppose, covers the main aspects of what
12 we are trying to establish at this stage. In the case
13 of Bristol, as we will come on to later, by going deeper
14 into the clinical records as has been done, we will
15 actually be able to get a very good idea of these
16 matters, because we will be able to cross-check the
17 detailed data from the clinical records with what has
18 been reported to the various sources that you have
19 described.
20 One of the problems, of course, is how much can we
21 trust the data coming from other centres with which we
22 would like to make comparison. That is obviously
23 something that will require somewhat more of a judgment,
24 although there may be some other sources we could use.
25 We will come on to that later.
0022
1 Q. Can I just explore with you for a moment the need of the
2 data sources, if there is a need, for the data sources
3 to speak as it were a common language, so when each of
4 them talks about death, for instance, they mean the same
5 thing. It might be quite surprising to the listener to
6 think "death" might mean something different in
7 different data sources, but one might define it as
8 a death within 30 days of operation, one might record
9 death as "ever died" and receive that information from
10 the local register of deaths, for instance, and so on?
11 A. And another might record it as deaths in hospital, which
12 is a common one.
13 Q. So for the purposes of cross-reference, corroboration
14 between the data sources, would one need to know they
15 were talking a common language or could be made to talk
16 a common language?
17 A. That is obviously very crucial, to try to do that.
18 I think equally crucial, something that really has to be
19 tackled is to make sure in terms of the coding, the
20 diagnostic categories that the groupings are as similar
21 as possible. That has to be faced, that the different
22 sources of information use different ways of describing
23 patients, different ways of categorising them, and we
24 would really like to be able to project all these
25 sources on to a common framework, and I think I have to
0023
1 admit that framework might not be, you know, the latest
2 way if one was setting up a database from scratch we
3 would use, but one that would allow at least common
4 comparisons between the data sources.
5 Q. So if one has a particular operation -- whatever it may
6 be, it would be wrong to suggest one -- one would need
7 to know that in Bristol in the various data sources it
8 was described in the same way, or one could derive the
9 same description for the purposes of comparison from
10 each of the data sources in order to corroborate one
11 against the other. One would need to know that the same
12 description was applied by others elsewhere if one was
13 going to compare Bristol with elsewhere?
14 A. Ideally one would, but it will be difficult both because
15 of the types of data that have been collected and
16 because of this area which is intrinsically extremely
17 complex, as has been pointed out. The fact that there
18 may be multiple diagnoses in the child and the fact that
19 there may be multiple operations will mean that this
20 will be a difficult matter.
21 Q. I shall ask you to return to this theme tomorrow, but
22 given that this is Stage 1 and you have briefly
23 described some of the problems that are faced and what
24 we might hope to go away with, what do you see the next
25 three stages as involving, in brief?
0024
1 A. The next phase, which is labelled here as "Exploration",
2 is to look at the available sources of data and conduct
3 preliminary analyses of those, as independently as
4 possible, except that the only sort of common ground
5 between those analyses should be that they should be
6 trying to use as common a coding categorisation as is
7 feasible. You have indicated already that such analyses
8 have taken place or will be taking place for the main
9 sources such as the HES data, the cardiac register and
10 the data and the medical records themselves on the
11 Bristol patient.
12 That will enable one, one would hope, after that,
13 to get an idea -- first of all within Bristol itself, we
14 should be able to get a very good idea of what was going
15 on and what the outcomes were, in the descriptive sense,
16 for Bristol alone. That, when cross-checked against the
17 various other sources such as the logs you have
18 suggested, should enable a very accurate picture of what
19 went on in Bristol to be known. That also can be used
20 as a frame, to enable sampling for more detailed
21 examination of particular cases.
22 In terms of the data available for comparative
23 purposes for other centres, using the main sources will
24 be the HES and the Cardiac Register, then analysis of
25 those, keeping always in mind the limitations we are to
0025
1 hear about today, should reveal first of all what was
2 the variability being reported between centres even
3 without knowing the identity of any centres. The
4 background variability is extremely important to know:
5 what were the standards being reported for various
6 procedures at that time, and what was the reported
7 variability between centres? That requires some fairly
8 careful analysis. Then, by identifying Bristol and
9 other centres, one should be able to see whether, on the
10 face of it, there are some categories, some periods of
11 time when Bristol or other centres appeared to be pretty
12 systematically differing -- having differing outcomes to
13 other centres, even allowing for the sort of variability
14 that is inevitable, that there will be between centres.
15 That really just provides -- whether there are
16 variations, whether Bristol figured prominently in those
17 variations with regard to particular categories, and so
18 to provide whether there is any prima facie evidence for
19 further, more detailed examination of the data.
20 Q. So the further, more detailed examination of the data
21 is, is it, at the confirmational stage?
22 A. Yes. You can say the exploration, stage, really, is
23 seeking to identify whether Bristol is an outlier at
24 some time for some procedures without in any way
25 attributing any reason for it and keeping always in mind
0026
1 the quality of the data.
2 Q. So simply saying, it appears that Bristol is different?
3 A. Yes.
4 Q. That is the most one could conclude at that stage?
5 A. Yes. And of course, one has to keep in mind that, you
6 know, an analysis like that which will inevitably
7 involve some grouping, and bearing in mind the
8 limitations of the data and the lack of any risk
9 adjustment, severity adjustment in the data,
10 differences, you know, small differences in particular
11 areas, could be obscured by such approach, and so one
12 needs to look really quite carefully at that data.
13 Q. One could equally, looking ahead, come to the conclusion
14 that Bristol could be shown to be no different on
15 available data?
16 A. Certainly.
17 Q. In which case, one could not ask any further, could one?
18 A. Unless there was other evidence that existed that
19 perhaps the grouping had been too broad in certain
20 categories, so one could cautiously look at perhaps
21 finer categorisation, but with great caution, because as
22 I said, if you really dig deep enough and finely enough,
23 you will always find some variability. There are
24 statistics -- that is really what this subject about,
25 trying to sort out what can be explained about chance
0027
1 alone, luck, from what is systematically different.
2 Q. So if it appears after the exploratory stage that there
3 is a systematic difference as opposed to a chance
4 difference between Bristol and other centres, then one
5 goes on to a third stage of confirmation?
6 A. Yes. One of the things that is easiest to confirm is
7 whether Bristol's performance and true performance was
8 as reported in the national databases that are
9 available, because we only get a very clear idea of that
10 from the clinical records, so certainly Bristol's true
11 performance can be calibrated against what has come out
12 in the exploratory phase, which I regard as a sort of
13 screening phase.
14 The problem comes in making a judgment about
15 whether the data reported from the other centres is
16 accurate enough to be confident that there really are
17 the differences that showed up on the preliminary
18 screening. That will rely, I would imagine, to some
19 extent on judgments that we are going to hear about
20 today on the accuracy of the data being recorded, and if
21 some particular findings have been established at the
22 screening stage, quite strong findings, there might be
23 an argument that perhaps the fuller sources of
24 information from other centres might be useful as well
25 to confirm their results.
0028
1 Q. Following confirmation, one would then be in a position
2 to say either what appeared at the end of the
3 exploratory phase still appears to hold good; one would
4 then be looking for an explanation as to why it should,
5 because there would then be an identified systematic
6 difference?
7 A. Yes. The confirmatory phase still does not give any
8 explanation or reason for variability, it is saying we
9 can feel reasonably confident that there is something,
10 some cause for Bristol to be an outlier or to be
11 systematically different in some respects, because there
12 is a limit to how much that is a statistical issue.
13 Obviously by looking more carefully at the Bristol
14 notes, one might be able to identify reasons for this
15 variability: by that time, if these things have come
16 about, the discussion will be much more focused and so
17 I would imagine that would be the time that expertise
18 from the substantive areas, the clinical expertise,
19 would come in, and using perhaps the sampling that has
20 gone on in the case notes, to identify reasons for the
21 confirmed performance.
22 Q. Can I take a crude example and see whether I have
23 understood what you are saying.
24 Let us suppose it were shown at the end of stage 3
25 that Bristol were different from other centres; let us
0029
1 suppose the surgery was less successful in terms of
2 death following surgery in Bristol than elsewhere.
3 It might be said, "Well, the cases that were dealt
4 with at Bristol were actually much more difficult than
5 elsewhere", and if that were indeed the case, it would
6 be a perfectly rational and sensible explanation of the
7 difference.
8 One would then want to explore that difference and
9 one would do that by means of other expertise than the
10 statisticians?
11 A. Yes. I mean, if one had wonderful data available on all
12 centres, then there are very established statistical
13 methods of doing that risk adjustment which can be
14 carried out by adult cardiac surgery by the more uniform
15 conditions and the larger numbers involved.
16 In the area, there is only rather rudimentary
17 development of risk adjustment procedures, so those will
18 be carried out with more caution.
19 So in any identified area, though, one could
20 certainly identify the factors in Bristol from the case
21 notes, and then one perhaps could use other information
22 from other centres to have an idea of whether there was
23 systematic difference in the types of patient being
24 treated in Bristol compared with other centres. But
25 that would have to be a very focused activity in an area
0030
1 previously identified.
2 Q. In the example that I gave you, you would be focusing
3 upon the question whether or not the case mix was
4 different in Bristol than it was elsewhere generally,
5 for instance?
6 A. Yes, in that particular area, yes.
7 Q. So what it means is that even if one gets to stage 4,
8 one cannot simply say, well, the statistics show that
9 Bristol was -- I use the word to sum up -- "bad", it is
10 a very poor word sometimes to use, but one cannot draw
11 that conclusion; one has to look for the explanation and
12 evaluate the explanation there might be?
13 A. Statistics can never show a causal factor in this sense,
14 and I think even at the end of this stage one would say
15 that this variability -- if it is a negative result --
16 it cannot be explained by variations in case mix or by
17 chance alone, then it is left up to --
18 Q. There has to be some other explanation?
19 A. And other expertise to identify it.
20 Q. Whatever it may be?
21 A. Yes.
22 MR LANGSTAFF: Thank you, Dr Spiegelhalter. Sir, I am
23 conscious that it is time we might normally have a short
24 break, and perhaps it would be convenient to have one
25 now before we hear from Mr Willmer?
0031
1 THE CHAIRMAN: Yes, Mr Langstaff. Shall we say 15 minutes
2 and reconvene, therefore, at 11.00? Thank you.
3 (10.43 am)
4 (A short break)
5 (11.07 am)
6 MR LANGSTAFF: Mr Richard Willmer, please. Mr Willmer, if
7 you would like to stand to take the affirmation?
8 MR RICHARD WILLMER (Affirmed):
9 Examined by MR LANGSTAFF:
10 Q. Mr Willmer, we have your statement which I hope will
11 come up on the screen to your right, WIT 189/1: is that
12 a statement which you have provided for the purposes of
13 this Inquiry?
14 A. It is.
15 Q. And together with appendices, it runs for 132 pages.
16 You have, I think, signed it for us at -- I do not know
17 that you have signed it for us. In any event, is it
18 true and accurate?
19 A. It is, yes.
20 Q. You signed on the bottom of the front page, in fact,
21 I think?
22 A. Yes, on the front page.
23 Q. In that statement, do you deal with two sets of national
24 statistics of interest to us: HIPE and HES?
25 A. Yes. The questions that were put to us related both to
0032
1 HIPE and HES. As has been mentioned previously, HIPE
2 was the predecessor of HES, and our knowledge of HIPE is
3 very much less than it was of HES but we have tried to
4 answer the questions in relation to HIPE as best we
5 can. If the Inquiry requires further details, we can
6 explore it and gather further information if that is
7 required. The majority of the information relates to
8 HES.
9 Q. Can I deal with the two chronologically and deal first
10 of all with the Hospital Inpatient Enquiry, HIPE. First
11 of all, to get a broad idea of when that began, that
12 began, I think, in 1949, did it?
13 A. I think very close after the NHS actually came into
14 being.
15 Q. So it was, if you like, the first national tool which
16 the NHS developed?
17 A. Yes.
18 Q. A sea change came about in the way in which it was
19 compiled in the 1960s, in which a sample was drawn from
20 regional data sets?
21 A. That was the basis of the system from about the 1960s
22 through to the mid-1980s, as I understand it.
23 Q. It stopped when -- in 1985, did it?
24 A. Yes.
25 Q. So, so far as we are concerned, we only have the 1984 to
0033
1 1985 period which is of any relevance to us, directly.
2 Did it attempt to cover the entire activity in NHS
3 hospitals?
4 A. It was designed as a 10 per cent sample of activity
5 right across the NHS and it was actually put into three
6 areas, so it actually had a separate system for
7 psychiatric data and for maternity data.
8 Q. So, so far as paediatric cardiac surgery is concerned,
9 that would be in one of the three data sets?
10 A. It would be.
11 Q. And the data was collected from the regions?
12 A. Yes, from the HAA systems that were in the region at the
13 time.
14 Q. You have used an acronym there.
15 A. HAA? The hospital activity analysis brought together by
16 the regions at that time.
17 Q. In order to get the national picture, one would have to
18 depend upon the regional picture?
19 A. If I wanted a 100 per cent sample, one would presumably
20 have to go back to the regions if they still had the
21 information.
22 Q. And is it likely -- I do not expect you have carried out
23 any particular Inquiry -- that they do hold data for
24 a time as long ago as 1984 to 1995?
25 A. I would have to explore that, but off the top of my
0034
1 head, it is unlikely they would have that information.
2 Q. So one does have a 10 per cent sample drawn from the
3 regions?
4 A. Yes.
5 Q. Could you pull the microphone a little bit closer? It
6 is pretty good at picking up your voice, but your voice
7 is naturally quiet.
8 So in terms of what it can tell us, I identified
9 three main areas we wanted to focus on: the strengths,
10 the weaknesses, and any useful information that we can
11 gain from a data set.
12 What would you say were the principal weaknesses
13 for our purposes -- you have heard me describe them
14 briefly this morning -- of using the HIPE data set for
15 1984 to 1985?
16 A. Probably the main weakness is the fact that there were
17 very few analyses produced at the time down to the level
18 of detail that would probably be of interest to this
19 Inquiry. I think a lot of the information was produced
20 at regional level and probably down to the specialty and
21 some operative procedure codes and diagnosis codes, but
22 probably not to the level this Inquiry would be
23 interested in.
24 Q. So it would not enable one to focus on particular
25 operation codes in particular hospitals?
0035
1 A. As I understand it, no.
2 Q. Dealing with what one might achieve from it, would one
3 get a broad picture of level of activity across the
4 country in particular areas?
5 A. Yes.
6 Q. So far as one would be able to know, would one get
7 a broad picture of activity generally in cardiac
8 surgery, leaving aside for the moment whether it is
9 adult or paediatric?
10 A. I think we would have to look at the tables that were
11 produced at the time, and let you know exactly what
12 level of detail is available on those tables.
13 Q. Can I put HIPE on one side, given what you have said
14 about it, save to ask you this: at page 11 of your
15 statement, the bottom of the page, you deal with the
16 HIPE data and you use a quotation there which is why it
17 is in italics. The third line:
18 "It is pointed out by the author that nevertheless
19 the initial inclusion of some erroneous data is
20 inevitable."
21 If we go down to the next paragraph, we can see
22 some examples of "totally unacceptable information".
23 So far as you know, are those dreamt up or are
24 those actual examples of unacceptable information that
25 was in fact found in HIPE?
0036
1 A. I do not have personal knowledge of whether they are
2 actual or fictitious examples, but they look like
3 examples that probably occur.
4 Q. Non-existent hospitals?
5 A. Yes.
6 Q. So that happens?
7 A. Yes. I mean, you may get an incorrect hospital code or
8 something like that. I think that would be what they
9 mean.
10 Q. And males with female diseases is an obvious one?
11 A. Yes.
12 Q. Speaking generally -- this is probably true of all data
13 collection, but what probably gives rise to errors in
14 accuracy such as that?
15 A. It may be coding at source; it may be problems in the
16 transmission of the data from the hospital through
17 whatever route it goes nationally. When we receive
18 information nationally we ask for it to come in in
19 a common format with common codes and sometimes the
20 translation to local codes from national ones can create
21 problems.
22 Q. So in the days of HIPE, at any rate, just focusing on
23 old times before we come up to date, there would have to
24 be a human being who would actually translate from some
25 record on to a summary which constituted the HIPE
0037
1 record, presumably?
2 A. It would either be a person or maybe there is a computer
3 system that translates codes from the local system on to
4 something that is required for the national purposes and
5 transmitted. I would think in the days of HIPE, a lot
6 of the work was done manually.
7 Q. So there would be human input first of all, coding at
8 local source; secondly translating from local to
9 national?
10 A. Yes.
11 Q. Are you able to say anything as to the incentive that
12 there may have been for any particular individual doing
13 either the coding or the translation to get it right?
14 A. It is difficult to say at the time, because I think it
15 depends on the ethos at the time. I mean, we think
16 there is a great incentive if you can pass the
17 information back to the Health Service for their own
18 purposes so that they can see the importance of getting
19 the coding right, which is a philosophy we are following
20 at the moment.
21 Q. Taking it up to today: today it is all, presumably,
22 computerised, except at the point of the creation of the
23 computer record, is it?
24 A. Yes. Currently now information is put on to the Patient
25 Administration Systems locally in hospitals. It then
0038
1 passes through --
2 Q. Can I ask you to stop there: an individual, a human
3 being, puts the information from some source on to the
4 database?
5 A. Yes.
6 Q. Are you able to say what training, what pay, what level
7 of qualification that individual might have and how
8 prestigious a job it is?
9 A. I do not know a lot about the work that is done by
10 medical records clerks and clinical coding clerks within
11 the NHS. I do not know a lot about their work. We do
12 work with them to stress the importance of getting
13 coding accurate by showing what uses we make of the
14 information. I do know that when a change was made to
15 a new coding of international classification of diseases
16 called ICD 10 there was extensive training of coders in
17 order that they understood what the new codings were and
18 the importance of getting them right, so the training is
19 undertaken for people, but it would probably be better
20 to talk to the people who, as it were, trained those
21 people and get that information.
22 Q. Today, is there any feedback from the centre to the
23 individual? Let me tell you the point I have in mind.
24 If one has an individual who is, let us say, doing an
25 examination, there is a feedback in the sense of the
0039
1 results, so the individual might try harder to get it
2 right because he gets a better result if he does.
3 If there is no feedback and what matters is that
4 the piece of work is completed rather than that it is
5 completed and completed well, then it may not in fact be
6 completed as well, or as carefully.
7 So the difference between those two examples is
8 the feedback in the example I posed, the examiner's
9 result; in other situations, it may be a reward for
10 accuracy by promotion or pay or whatever.
11 Is there any sort of feedback that you know of to
12 those individuals who, at the moment, put the initial
13 codes into the system?
14 A. I do not know what sort of feedback there is. I know
15 some hospitals have introduced data accreditation, data
16 audit processes to check the accuracy of the information
17 held on their computer records, and as part of the
18 information for health strategy which was published last
19 year, the aim is to make that mandatory across the
20 Health Service in the acute sector. I think it is for
21 next year, 2000/2001, so there will be a greater
22 incentive in the future to get it right than in the
23 past.
24 Q. So we can say as at today what is being introduced is
25 a system which, if you like, may not bring rewards, but
0040
1 it will bring shame if you get it wrong? I am speaking,
2 again, generally. That is the effect of it, is it: to
3 make people realise that someone is checking on the
4 quality of their work?
5 A. I think it is checking on data accuracy, and I suppose
6 by the results that come out, it would be looking at the
7 quality of the information that goes into the computer,
8 but that may not be, as it were, reflecting on the
9 coders themselves; it may reflect on the information
10 they have in order to do that coding work. So there may
11 be a number of individuals involved locally in that
12 process.
13 Q. But the picture I have given you is just one person
14 coding. You are pointing out that it could be a number
15 of people before the information actually gets to the
16 person who makes the record?
17 A. Yes.
18 Q. So there is more than one opportunity for what one might
19 call "human error" to creep in?
20 A. Yes.
21 THE CHAIRMAN: May I interject and ask a question, just for
22 my own understanding? Given that coding involves to
23 a degree a judgment, how does the concept of accuracy
24 bear on that? Does that question make sense? The
25 ultimate decision as to what code to ascribe to a degree
0041
1 involves some judgment. Can one therefore use the
2 notion of accuracy there, or is it merely a question of
3 whether other people agree with the judgment made?
4 A. This is an area I do not have a lot of personal
5 knowledge about, but I think it is an area that this
6 Inquiry will wish to look at: the degree to which the
7 particular codings that are of interest to the Inquiry
8 are actually a matter of subjective or objective methods
9 when it is actually coded on the computer system.
10 MR LANGSTAFF: Where we began this discussion was
11 looking at males who were recorded as having female
12 diseases, which is unlikely ever to be a matter of
13 judgment?
14 A. Sometimes you find problems arise with the quality of
15 the data because you look at more than one field, like
16 the operation was a hysterectomy and the person was male
17 and you know that is wrong, but it is not until you
18 bring those two pieces of information together that you
19 find out that there is something that is incorrect
20 there.
21 As I said, whether that was coding at the hospital
22 or in something that was translating the information or
23 transmitting it to the centre, it is something you need
24 to explore once you find there is a problem.
25 Q. The author of the piece that we have on the screen
0042
1 plainly was aware that there could be unacceptable
2 information in HIPE. That must either have been because
3 of a process of checking which was part of the system,
4 or by review after the event by the commentator.
5 Do you know which it was?
6 A. No, I do not.
7 Q. Are you able to say at this stage, in 1999, whether
8 there were any, if I can loosely describe them as
9 "quality checks" to cover both questions of accuracy
10 and the stability of judgment?
11 A. I think the information we have are the quotes that we
12 found from the documents available at the time. That is
13 basically the information we have.
14 MR LANGSTAFF: I do not know, Dr Aylin, if you want to come
15 in and say anything?
16 DR AYLIN: There is this issue of accuracy and perhaps
17 a better word, or another way of thinking about it is
18 perhaps consistency. Certainly, there have been studies
19 looking at accuracy of diagnostic coding and procedural
20 coding, comparing a particular hospital, the coders
21 residing in a particular hospital with external coders
22 and comparing their agreement rates. The agreement
23 rates, I am thinking of a study by Dixon et al in 1988,
24 looked at coding of procedure codes and they showed
25 between about a 70 and 83 per cent agreement on
0043
1 procedure codes. I have this summarised on INQ 2/14.
2 MR LANGSTAFF: Can we have on the screen, please,
3 a summary chart and turn it round, please.
4 DR AYLIN: This was a study that compared local coders at
5 two large hospitals and they had a set of records that
6 they were both asked to code. The level of agreement
7 between the local coders and the external coders varied
8 by 70 per cent in one hospital and 83 per cent in the
9 second hospital at the 3 digit coding level, which
10 I guess you could -- I mean, it depends how positive you
11 want to look at that. One might say that is actually
12 quite good, but it is obviously not perfect. There were
13 similar differences in diagnostic codes.
14 MR LANGSTAFF: Could I just pause there? At the bottom of
15 that sheet there is the question posed, "Perhaps
16 a higher level of agreement in later years?"
17 Dixon's report which you are citing is 1998. The
18 questions we began with were in relation to this now
19 very old data set of HIPE.
20 Can I ask Mr Willmer: is it the general feeling
21 from where you sit that accuracy has got better
22 over time, or not?
23 A. I think certainly we can say that completeness has
24 improved of many of the fields that we use, and the
25 general quality of HES data seems to be improving, but
0044
1 we do not have, as it were, a rigorous study such as the
2 one quoted to show that the accuracy of these particular
3 fields has increased from A to B, or anything like that
4 to go by. We just have a general impression that the
5 completeness and usefulness of the data is improving
6 over time and it is an evolving process.
7 Q. If it is improving over time, if one goes back from 1999
8 to the last year of HIPE in 1985, one would expect the
9 level of accuracy of those who were coding at the time,
10 other things being equal, to be very much less reliable
11 than data produced today?
12 A. It is difficult to judge, I think, at this point in
13 time.
14 Q. But a big question mark one would have to put over it,
15 would one?
16 A. Yes.
17 Q. Can you help, again really for general public
18 consumption, as to the difference between the 3 and
19 4 digit level of the OPCS codes? These are procedure
20 codes, are they?
21 DR AYLIN: Yes.
22 MR LANGSTAFF: Each code has 4 characters assigned to it,
23 4 digits?
24 A. Yes, it can be up to 4 digits.
25 Q. Is it a question of, as it were, refining down so one
0045
1 begins with the first digit is very general; the second
2 digit is bringing from the general towards the
3 particular; the third digit more so; the fourth digit is
4 absolutely specific? I am using layman's terms, but do
5 I have the general picture right?
6 A. I think the 4-digit tends to be more specific, but I do
7 not think it is a completely hierarchical system in that
8 sense.
9 Q. How much does the fourth digit actually matter for
10 comparative purposes?
11 A. It might depend on the areas you are looking at.
12 DR AYLIN: Yes. I think it will not go down to individual
13 named procedures at a 4-digit level. It is still not as
14 specific. For instance, if you look at the K5 and K6
15 codes which are operation procedure codes for
16 transposition of the great vessels --
17 MR LANGSTAFF: That is only two digits --
18 DR AYLIN: K05 and K06, and then you have a fourth digit at
19 the end of each one, so K052, K053. So although these
20 can be grouped K05 whatever and K056 into operations on
21 transposition of the great vessels, and they will go
22 down to a little bit more detail if you look at the
23 4-digit level, you will not get named procedures like
24 the switch operation or anything like that. They are
25 more generic terms of operations rather than named
0046
1 operations.
2 MR LANGSTAFF: So what is one looking at in terms of the
3 3-digit level in terms of what is going to be of use to
4 us?
5 DR AYLIN: I think to give an example, the KO5 and KO6
6 codes, those are the 3 digit levels. You can look at
7 this group of operations on a transposition of the great
8 vessels as a fairly broad group, but it is difficult to
9 relate the OPCS 4 procedure codings to other coding
10 systems that might be in existence because they might go
11 to a finer level or use a different hierarchy or
12 whatever.
13 MR LANGSTAFF: It is plain what is shown on the screen there
14 is a comparison as to judgment. It is variability or
15 a measure of variability of judgment of different people
16 who do the work. I was going to say "experts", but
17 people who do the work is perhaps a better description
18 from what information we have.
19 DR AYLIN: Yes.
20 MR LANGSTAFF: If there is a measure of agreement which is
21 either better than two out of three or better than four
22 out of five cases, is that -- again, I translate it into
23 terms which might not be entirely appropriate, evaluated
24 terms: is it good or is it bad?
25 A. It partly depends on the uses to which you want to put
0047
1 the data. I think a point which is worth making about
2 any data source, but perhaps particularly about HES and
3 probably HIPE, is that you want to make it fit for
4 a purpose. Where you have a wide range of purposes, it
5 may be good for some but not other purposes.
6 Q. So if one were looking to compare, as it were, mortality
7 rates from a particular surgical procedure, conducted in
8 two separate hospitals, to what extent does it put
9 a question mark over your conclusions to know that if
10 you looked at one of those hospitals, you would, by
11 using two different people looking at the same data,
12 produce something which could be as much as 1 in 3 of
13 a difference? You get, at the 3 digit level,
14 a variability of some 30 per cent?
15 DR AYLIN: May I say there, I think some fields are
16 better coded than others and better mortality is
17 probably well coded, but if you want to relate it back
18 to the operations that they had, then you would have
19 this problem. Comparing one hospital with another, you
20 would have to make a decision as to whether you were
21 comparing actual differences from mortality rates in
22 operations or differences in coding procedures.
23 MR LANGSTAFF: You have introduced another very important
24 point by pointing out that mortality is a function of
25 someone actually recording it accurately, and they may
0048
1 be wrong on that?
2 DR AYLIN: Yes.
3 MR LANGSTAFF: So one may begin with, I do not know, 95,
4 99, 98 per cent accuracy in getting it right that
5 someone has died, but if you look, of that 98 per cent
6 there may be a variability of 70 per cent or 83 per cent
7 between two different coders looking at the same data.
8 Is that the position?
9 DR AYLIN: Yes.
10 MR LANGSTAFF: Which makes the overall position rather
11 less reliable?
12 A. I think it goes back to the point the previous people
13 mentioned: that once you have produced the information,
14 it raises questions rather than provides answers, and
15 one of those questions is about the quality of debate
16 and the quality of the coding behind the information,
17 and that is something that needs to be explored before
18 you start drawing conclusions from it.
19 Q. So let me move on from that. We will probably come back
20 to that and related issues in a moment or two, but
21 I have been asking you principally about HIPE and we got
22 into this discussion because we were looking at the
23 origins of the data and how it came into the central
24 database.
25 If one were to concentrate on HES, and again, can
0049
1 I deal with it in stages, I think from the history of
2 the database as it is understood.
3 After HIPE finished, was there any database for
4 a couple of years which was in regular use nationally?
5 A. There was no detailed database like HIPE or HES for that
6 period. The department did receive some sort of broad
7 aggregate returns which gave total numbers of finished
8 consultant episodes that you have mentioned by
9 specialty, but nothing down to specific operative
10 procedures.
11 Q. That, I think you explain in your statement to us, was
12 really because the system was changing and it was more
13 difficult to change than had originally been
14 anticipated?
15 A. The changes came about as a result of quite
16 a wide-ranging review of the Health Service's
17 information in the early 1980s by Mrs Korner, and
18 I think the changes were found to be quite substantial
19 for systems. I think once, as it were, there started to
20 be delays in the system, there was in some ways a sort
21 of downward spiral in the perceived importance of
22 information, because if you cannot get it in and turn it
23 round and make it timely, people do not see why it is
24 important to get it in. So you actually have a downward
25 spiral in terms of the use and presentation of
0050
1 information and that is probably what happened in the
2 late 1980s and early 1990s. What we tried to do was
3 actually turn that round and have an upward spiral so we
4 can get the data back out to the Health Service and
5 study use quickly, and hence people see it as important
6 and hence they have a greater incentive to get it right.
7 Q. We mentioned a moment ago the OPCS 4 codes. OPCS 4, the
8 "4" suggests there may have been a 3, a 2 and a 1
9 before that. Am I right?
10 A. There was certainly a 3. I assume there was a 2 and
11 a 1, but my memory does not go back that far.
12 Q. The OPCS codes are procedures as opposed to diagnoses?
13 A. Yes.
14 Q. Diagnoses had their own coding system, did they?
15 A. Yes, the ICD, International Classification of Diseases.
16 Q. Currently it is ICD 10?
17 A. Yes.
18 Q. And it was throughout the rest of the period with which
19 we are concerned ICD 9?
20 A. Yes, for the period you are interested in.
21 Q. Although ICD 10 was first promulgated in the early
22 1990s, it was first used for HES purposes in the 1994 to
23 1995 years?
24 A. I think it was 1995 it came in as the first year.
25 Q. So prior to 1995, ICD 9 was diagnosis code?
0051
1 A. Yes.
2 Q. So far as the procedure code, what was actually done in
3 response to the particular diagnosis, one would look for
4 OPCS 4, obviously given Dr Aylin's slide, some of the
5 time, and OPCS 3 for some of the time?
6 A. I think OPCS 3 was probably connected with the latter
7 half of HIPE, so I think you would probably only be
8 interested in OPCS 3 if you were looking at the HIPE
9 data.
10 Q. Did the changeover happen in 1988 to 1989?
11 A. It was in the late 1980s that the OPCS 4 was
12 introduced. As I have mentioned in my statement, there
13 were a number of revisions to that which added
14 complications, I think, for the people coding.
15 Q. What you have said, I think is that really, when HES
16 first developed and began, it started off using OPCS 3,
17 OPCS 4 began and there was what you described as an
18 unstable mix of the two, partly because 4 came in during
19 the course of a year when data was collected, so one of
20 the problems with data for that year, the late 1980s,
21 was the fact that you had two different sets of criteria
22 for establishing what procedures you had used?
23 A. Yes, and I think in some areas OPCS 4 was subject to
24 further revision during the year and unfortunately some
25 of the same codes were used for old procedures and new
0052
1 procedures, which again added complications.
2 For the late 1980s, the information is not very
3 good for operative procedures.
4 Q. So if we go back to the person putting the data in who
5 would have been trained on OPCS 3, they would think in
6 those terms, might not be familiar with OPCS 4 and might
7 therefore tend to make errors more easily?
8 A. I would not like to say what was going through the minds
9 of coders about 10 or 12 years ago, but certainly, if
10 a coding system changes, that can obviously create
11 problems for those feeding the information into systems.
12 Q. Is there evidence that when a coding system changes,
13 there are problems with data quality for a while?
14 DR AYLIN: I think, just thinking about the way that this
15 change happened, HIPE was based on hospital activity
16 analysis data which was provided by each hospital at
17 100 per cent level to the regions, and then the 10 per
18 cent sample was given to HIPE.
19 After HIPE finished in 1985, the hospital activity
20 analysis still continued into 1986 and 1987 and I think
21 it was adapted to provide information for HES. So the
22 system was still there, it was a gradual transformation
23 at different times in different areas, so I think it is
24 quite likely that there would have been problems
25 implementing new codes, and I think this would be
0053
1 a common problem in introducing new code systems, and
2 new computer systems as well, that different hospitals
3 would be using different systems of coding.
4 What also happened, an initial coding system,
5 OPCS 4 was introduced I think in April 1987, but there
6 are a number of revisions of that coding system which
7 went up to 1990, I think was the last consolidated
8 revision of OPCS 4. There were different versions of
9 OPCS 4 being used in different hospitals around the
10 country at that time. So it does not make it a very
11 reliable source of information.
12 MR LANGSTAFF: Some of what you are saying, I think, is
13 speculation: "people would have behaved in this way
14 because it is reasonable to think that is what would
15 have happened". So really what you are telling us is
16 that you do not know of any definitive research which
17 shows that they did, but it is a reasonable supposition.
18 DR AYLIN: I think it is a reasonable supposition. I think
19 it would be relatively easy to look back at the very
20 early HES data sets and see whether there was this
21 mixture of OPCS 3 and OPCS 4 codes. I do not know
22 whether that would be possible. I assume that is what
23 this statement is based on.
24 A. As much as what may be confusion for the individual
25 coder, obviously if individual hospitals were moving at
0054
1 different paces in their implementation, they may each
2 think they were doing the right thing, but the right
3 thing may be different in different places.
4 Q. If one is looking at the 1987/88/89/90 period, there
5 would be, if you use the expression in the current
6 context, a very big "health warning" over it?
7 A. Yes.
8 DR AYLIN: Yes.
9 MR LANGSTAFF: The disadvantages of HES as a system, then,
10 from 1987 to 1990, let us say, are they then such,
11 because of this confusion over the coding -- I say
12 "confusion", you know what I mean, describing the
13 process you have been talking about -- that really we
14 should not place reliance upon it for the purposes of
15 this Inquiry. Does it go that far?
16 DR AYLIN: I think there are not only the questions about
17 the coding, and as you say there is a big health warning
18 there, but also the question of coverage. If you look
19 at the national coverage, comparing your HES records
20 with the KP70 returns -- I do not know whether you want
21 to get into this now? The KP70 returns were based on
22 a paper system. I think essentially in the early days
23 certainly they were based on head counts by the night
24 nurse going through the hospital in the middle of the
25 night, and these are compared to the electronic records,
0055
1 the HES records.
2 If you look back at 1987/88, you have a coverage
3 of about 75 per cent, a national coverage, so if you
4 compare your paper returns with your electronic returns,
5 you are only catching three quarters of them on HES.
6 MR LANGSTAFF: Do you have a slide to show us this?
7 MR WILLMER: It is in report document 7. I do not know
8 whether that can easily be searched for, but it is near
9 the back of the statement.
10 THE CHAIRMAN: Mr Langstaff, while we are looking for
11 that, can I, just as a matter of housekeeping, make sure
12 that the stenographers are able happy to deal with the
13 innovation of today's hearing, voices coming thick and
14 fast from various quarters? I need to clarify that,
15 because others in other places would wish to read our
16 evidence, and I want to make sure that you are
17 comfortable in the process. I receive a nod and I am
18 grateful.
19 MR LANGSTAFF: If we could have WIT 189/110 on the
20 screen.
21 DR AYLIN: I am sorry, I was talking about diagnostic
22 coverage there, quoting those figures of 75 per cent.
23 MR LANGSTAFF: I thought you had been talking about KP70.
24 DR AYLIN: For KP70 it is actually 1987/88 coverage.
25 MR LANGSTAFF: Let us concentrate, since we have it on
0056
1 the screen, on what this tells us and shows us. You
2 have introduced the idea of KP70 for a moment. KP70 has
3 been called a "gold standard", has it not?
4 A. Yes.
5 Q. Does the "KP" stand for anything?
6 A. I think the K stands for Mrs Korner. I cannot remember
7 what the P stands for now. There is a whole series of
8 returns introduced as part of the Korner review and they
9 would have an initial K.
10 Q. So perhaps there is no magic in the initials but that is
11 what we have to work with. KP70 was actually a physical
12 head count, was it?
13 A. I am not sure whether it was a head count, but it
14 certainly was an aggregate set of information to tell us
15 how many finished consultant episodes there were for
16 each -- at that time it would have been by District
17 Health Authority, by specialty, which was seen as
18 a baseline count, if you like, some sort of "gold
19 standard" against which we could compare the number of
20 records that were coming in on HES.
21 Q. One could only do the comparison if one knew that the
22 KP70 figures were actually separately compiled from the
23 HES figures?
24 A. Yes.
25 Q. Does one know that?
0057
1 A. One does not know exactly how people did them, but
2 sometimes people would run off the information from
3 similar systems to HES, but then could do separate audit
4 checks. Because it is an aggregate number, you could
5 look at it and say "I had 3,000 finished consultant
6 episodes in the last year; yes, that looks about
7 right". If it is only 1,500, you know you have only
8 50 per cent of your records there. With something like
9 HES you cannot easily check because you are sending
10 a huge amount of information on individual records by
11 computer.
12 So people may or may not have run off the
13 information from a similar system to HES, but the idea
14 of the aggregate return as a standard is that that is
15 something you can look at and eyeball and say "It looks
16 about right", or does not.
17 Q. So you can pick up the obvious error?
18 A. People should hopefully pick up the obvious errors on
19 it.
20 Q. Is there a definition somewhere of the KP70, because --
21 A. Yes.
22 Q. There is a distinction in the way you have approached
23 it, talking about it showing finished consultant
24 episodes. Dr Aylin has described it as a "head count"?
25 DR AYLIN: I was trying to make a point that it is a -- in
0058
1 the early days, I assumed it was a manual count, the
2 computer systems were not involved in counting it, but
3 I think nowadays it is done on computer systems in
4 a number of hospitals.
5 MR LANGSTAFF: Can we take it in stages? In the older
6 days, do you actually know how it was done?
7 DR AYLIN: The description I have heard of it is that it
8 is much more of a manual system than an electronic
9 system. That is where you can make your comparison
10 between the HES data and the KP.
11 MR LANGSTAFF: Can you help us, because there is a slight
12 difference of information you both have about it, as to
13 what the manual system consisted of?
14 A. I think the system for actually producing KP70 was
15 different in different places. I think Mr Aylin is
16 right, there were manual systems in many places to do
17 it. What actually happens is that in terms of national
18 standards, we asked the NHS for information, but the
19 actual method of compiling it may not necessarily be set
20 out for people, so it is up to people locally to decide
21 the best way to actually produce that information.
22 Q. So we have got to the stage that KP70 constitutes some
23 form of independent check, albeit broadbrush check --
24 A. Yes.
25 Q. -- I think you are agreeing with that, of the general
0059
1 accuracy of the picture, completeness of coverage --
2 A. Yes.
3 Q. -- of the HES data?
4 A. Yes.
5 Q. Presumably, if one looks at the figures that you quote
6 here on page 110, a percentage like 88 per cent, even
7 though one is dealing with large numbers, is actually
8 quite specific. If we look to the independent count of
9 finished episodes, KP70, we have 8,052,319, so someone
10 has been able to go to the last one and say "This is not
11 broadbrush, we have a figure we can put on KP70 here".
12 Is that an actual accuracy, or is it a spurious
13 accuracy? Can you comment?
14 A. I mean, I think probably going down through the last
15 consultant episode, perhaps either on a HES or KP70
16 would introduce spurious accuracy. I think the key
17 thing from the Inquiry's point of view is to perhaps
18 note the 88 per cent as something which actually could
19 have been a lot worse, but at least we did have 88 per
20 cent of the records in, but was an indication to us that
21 we needed to get the complete numbers of records in as
22 far as we could, and the figures for later years
23 actually show that the overall coverage of HES compared
24 with KP70 has been pretty good.
25 Q. So again, looking down the lists that we have on the
0060
1 screen, there are two percentage columns -- one
2 percentage coverage, one percentage diagnostic
3 coverage. Looking at them separately: percentage
4 coverage obviously improves dramatically in the sense
5 that KP70 and HES begin to correspond much more exactly
6 up until -- between 1987 and 1990, when it gets up to
7 98 per cent, between there or thereabouts, certainly
8 within a 5 per cent margin of error of 100 per cent ever
9 since?
10 A. Yes. I mean, there may be variations between regions,
11 as I think Mr Aylin brought out in one or two of his
12 slides, but in general, the overall coverage of HES from
13 about 1989/90 onwards has been good.
14 Q. Talking about regions, if we go to 189/113 and just turn
15 it sideways, in 1991 to 1992 we can pick up South
16 Western Regional Health Authority there, with its
17 630,389, and there the correspondence is pretty good,
18 2 per cent margin of error. I say "pretty good"; is it
19 pretty good? Would one expect that figure to be 100 per
20 cent, or is 102 per cent actually pretty good?
21 A. In theory, I suppose, it should not be above 100 per
22 cent, but it highlights a problem we have on HES
23 sometimes that actually we receive duplicate records, so
24 sometimes regions have sent us the same records twice,
25 but because we cannot separately identify the records,
0061
1 you get two lots of them.
2 So this sort of analysis is as good for us at
3 spotting the duplicate record as it is for spotting
4 under-reporting.
5 Q. The change in completeness appears to come about at
6 about the same time as the purchaser/provider split. Is
7 this possibly because of any impact of contracting in
8 the Health Service upon the need to produce better
9 quality data so that the numbers contracted for could be
10 followed through with the purchasers?
11 A. Certainly, the overall coverage is at about what one
12 might term an acceptable level, 98 per cent or around
13 that mark, from 1989 to 1990 onwards, which is prior to
14 contracting.
15 If one goes back to the previous table, 110, the
16 percentage of diagnostic coverage did seem to do
17 a further step improvement, that is the far right-hand
18 column, where we notice an improvement from 1987/88 from
19 75 per cent up to 88 per cent in 1990/91, but from
20 1991/92 onwards when contracting was introduced it went
21 up to 95 per cent, so that may have been a step change
22 as a result of the contracting being introduced.
23 Q. The figure appears to be derived from the column
24 immediately before it, the number of HES records without
25 a usable primary diagnosis, and the informed speculation
0062
1 would be, would it, that you need a usable primary
2 diagnosis if that is what people have contracted for?
3 A. Yes.
4 Q. If there is no contract involved, then the need is not
5 as great, so one might reasonably hypothesise that the
6 change to purchaser/provider meant that people took
7 greater care over the records.
8 Is that a reasonable supposition?
9 DR AYLIN: I think so, yes.
10 A. Presumably if somebody goes into hospital, there is some
11 diagnosis associated with why they have gone into
12 hospital. So they should all have a primary diagnosis,
13 if not a hospital procedure.
14 MR LANGSTAFF: The incentive there is for people putting
15 information into the system to get it right?
16 A. Yes.
17 Q. That is what the purchaser/provider split made
18 a difference to, is the suggestion.
19 A. It is a possibility, yes.
20 Q. Going back for a moment, if I can, to 113 and turning it
21 sideways, we picked up that the South West Regional
22 Health Authority had a 102 per cent correspondence. Its
23 HES was showing rather more patients than its KP70 count
24 in 1991/92. If we look at 1992/93, it is 100 per cent.
25 If we turn over the page, we will find it is 100 per
0063
1 cent again in both the KP70, the HES and the diagnostic
2 coverage and the following year, 101 and 99
3 respectively. Just completing the picture so far as you
4 are able to show us, page 115, 101 and 98, that is the
5 regional office, now?
6 A. Yes.
7 Q. Where the difference may be because the numbers are
8 larger and obviously there are less regional offices
9 than there were Regional Health Authorities?
10 A. Yes. The figure of 1995/96 will include probably what
11 was most of Wessex.
12 Q. Yes, and Wessex's results are slightly more variable
13 throughout the period. What those figures would suggest
14 is that there appears to be, in the South Western
15 region, very great correspondence, viewed at over
16 650,000 admissions, between the numbers counted by HES
17 and the numbers counted independently by KP70, and an
18 appearance that of the records coming through, there is
19 a diagnosis recorded for each and every episode?
20 A. Yes.
21 Q. That would tend to give one a bit of confidence, would
22 it, at least as a starting point?
23 A. In overall terms, yes, it would give some confidence.
24 MR LANGSTAFF: You are agreeing with that?
25 DR AYLIN: I am agreeing with that.
0064
1 MR LANGSTAFF: If one looks at the problems which you
2 identify in your statement to us, and the difficulties
3 with the HES data, you, I think, have pointed out
4 a number of things. You have indicated that there may
5 be regional variations and that would be important if
6 one was trying to compare one region with another,
7 presumably?
8 A. Yes, and a variation at regional level, or a figure that
9 varies markedly from 100 per cent at regional level may
10 mean at Trust level there are even greater variations,
11 because you could have some Trusts that have sent in
12 very little information at all, whilst a lot of them
13 have achieved 100 per cent. So variations below
14 regional level may be even greater, so it tells you
15 something about that as well.
16 Q. So that if, for instance, one has 100 per cent
17 correspondence, it would argue that most Trusts have
18 sent in full data; if it is 98 per cent, the two per
19 cent may actually be a very bad Trust; it may be
20 responsible for virtually all of that?
21 A. Yes.
22 Q. That is the point. Do we have any evidence that
23 that inequality of data actually happened?
24 A. Certainly when we did analyses in the early 1990s,
25 because we were trying to decide how to make sense of
0065
1 what was coming in. We did find greater variability at
2 the Trust level than we did at regional level, which led
3 us to believe that it was safer to use the information
4 at regional level rather than down at individual Trust
5 level for a lot of purposes at that time.
6 Q. You make the point that the provision of data was not
7 compulsory?
8 A. There was no legal obligation to provide it; it was part
9 of what has been termed over the years a "mandatory"
10 set of information which all parts of the NHS should
11 submit. There was no, as it were, legal statutory
12 requirement.
13 Q. Does that actually matter if we see that, taken on the
14 whole, there is something like 98, 100 per cent
15 correspondence? It is not really a problem for us, is
16 it, if by and large it seems that everyone has actually
17 provided the data?
18 A. That shows us that everyone has provided the number of
19 records we would expect, and the vast majority have
20 added diagnosis in. It does not tell us anything about
21 the other 40 or so data items that we collect on HES and
22 the quality of those. That is in many areas more
23 difficult to assess until you start to use the HES
24 data. It is getting back to the male hysterectomies.
25 You do not pick those up until you start to run analyses
0066
1 of the system.
2 Q. In terms of usability for our purposes, moving from that
3 particular defect to utility, can you help at all with
4 what we need out of the data and whether it will provide
5 it to a sufficient degree, it not being compulsory to
6 supply all the data?
7 DR AYLIN: Given the problems with data quality, there are
8 specific problems involved in actually using HES
9 information for the kind of information that we want to
10 find out, and I have summarised in INQ 2/15 some of the
11 problems in actually analysing HES.
12 The first point that we brought up was this
13 problem of episodes versus spells. The next page, 16,
14 has a summary of episodes and spells, and perhaps
15 illustrates some of the problems, although it does not
16 come across very well. You can have a number of
17 episodes in a spell. If you have an operation in
18 a first episode within a stay or a spell in hospital and
19 you want to find out what happened to that person,
20 whether they died, whether they were sent home or
21 whether they were transferred to another hospital, you
22 have to be able to link these episodes together in order
23 to be able to get an outcome for the end of the hospital
24 spell.
25 There are problems with linking these --
0067
1 Q. Can I stop you there? What you are saying is --
2 underneath the black bar at the bottom, incidentally, on
3 the screen, if you have a photocopy, you would detect
4 the words "episode 1, episode 2" in the middle and
5 "episode 3" to the right, so that people watching can
6 follow it a bit more easily.
7 Are you saying that, taking episode 1, that is
8 a finished consultant episode because somebody comes in
9 under the care of a surgeon, let us say it is
10 orthopaedics, and because it is quite obvious in the
11 hospital he has a cardiac problem he is transferred to
12 a cardiologist?
13 A. Yes.
14 Q. So that is episode 2. If the result of the
15 cardiologist's intervention is that he goes for surgery
16 in a third discipline and then dies, is it the position
17 that each of the first two finished consultant episodes
18 will not record death as an outcome?
19 A. They will not have it directly on the record, yes,
20 because we will get separate records in for each of
21 those three episodes that are shown there.
22 What we have started doing now is what we call
23 "fuzzy matching the records": we cannot identify the
24 person, but identify whether it is the same person in
25 episode 1, 2 and 3, and hence, bring the records
0068
1 together. That forms the basis of some recent
2 publications, clinical indicators which came out
3 a couple of weeks ago.
4 The basic building bricks of HES as they came in
5 and were available, particularly in the early 1990s, as
6 Mr Aylin described, they were the finished consultant
7 episodes, the separate building bricks shown there of
8 episode 1, episode 2 and episode 3. You would not
9 necessarily know, if someone died as a result, in
10 episode 3, that they were linked with episode 1. It
11 would be a separate record that came in, and we would
12 not be able to identify that they were the same person.
13 Q. You quote a percentage in your evidence to us of the
14 number of people who have more than one finished
15 consultant episode in a spell?
16 A. Yes.
17 Q. I think it is 5 per cent?
18 A. Yes.
19 Q. How do you know that it is 5 per cent if, looking
20 back at the data, there is no way of saying it is the
21 same patient in each of these episodes?
22 A. We do get a field of information on the record, actually
23 called "episode order" and it tells us whether it is the
24 first in a spell, the second in a spell, the third or
25 a subsequent one. So we can count up all the episodes
0069
1 and see how many first episodes we have and see how many
2 people would have subsequently had second episodes.
3 Q. Looking back historically at the data, if it is 5 per
4 cent -- I do not know whether you want to ask any
5 questions of Mr Willmer about the percentage here?
6 DR AYLIN: I think that 5 per cent would refer to all
7 episodes, but I think there would be differences between
8 different specialties and probably between different age
9 groups and things like that, and perhaps within
10 different hospitals and different specialist units, so
11 that would be an overall figure. But you would expect
12 perhaps some specialties, if you go into a particular
13 specialty, you are more likely to be transferred to
14 another specialty or another consultant than other
15 specialties, so I think there would be differences.
16 A. I think that is absolutely right. If you look at basic
17 elective admissions for, say it was a hip operation or
18 something like that, the ratio is quite low. If you
19 start to look at emergency admissions, if you start to
20 look at the very elderly, then the rates start to rise
21 quite high, so there are quite wide variations.
22 It is one of the things you have to bear in mind
23 when you are using HES as to whether it is likely to be
24 a significant problem or not.
25 MR LANGSTAFF: So can either of you say, or would you
0070
1 like to come to a view to help us what the position is
2 likely to be when you are looking at the specialty of
3 cardiac surgery, paediatric cardiac surgery, which in
4 essence is unlikely to be anything other than the
5 rectification of a congenital heart problem, one might
6 have hypothesised that it would be admitted as such?
7 A. I do not know what the figures are, but we could
8 probably run something to find out what it is.
9 DR AYLIN: I do not know the figures offhand.
10 MR LANGSTAFF: But between you, you think that can be done?
11 A. We can run the HES data by specialty and look at the
12 ratio by specialty.
13 Q. Because if one were to show that the 5 per cent overall
14 figure, although it may be composed of a greater
15 percentage in some specialties than others, was a much
16 lesser percentage in paediatric cardiac surgery, then it
17 would not be a problem for us, whatever the problem may
18 be elsewhere?
19 A. Yes.
20 DR AYLIN: Yes.
21 MR LANGSTAFF: You were taking us, Dr Aylin, through the
22 previous page.
23 DR AYLIN: Yes, the list of problems in analysing HES
24 data. The second point, that there is no actual patient
25 identifier on the records, there is no patient
0071
1 identification number, is a problem in both looking at
2 joining these episodes to spells, and also not only
3 counting the number of spells and their outcomes that
4 are going through a particular unit, but counting the
5 number of individuals going through a particular unit.
6 It is very difficult to count individuals, the number of
7 children or people that go through a particular unit,
8 without this identifier.
9 So that is another issue which you have to take
10 into consideration when looking at the data.
11 A. I agree with that. It was actually a specific
12 recommendation of the review in the 1980s that the
13 Department of Health should not have identifiers, purely
14 for reasons of confidentiality, and it was not seen as
15 important at the time, but in order to do the type of
16 analysis that I think the Inquiry is interested in and
17 Mr Aylin has mentioned, you want to be able to
18 differentiate between individuals even if you do not
19 want to identify them. You want to see what happens to
20 the same individual over a period of time. And actually
21 from 1997, I think it is, we are getting a new NHS
22 number, so in future analyses of HES we will be able to
23 do that, subject to maintaining confidentiality, which
24 is a very firm thing we have to do all the time,
25 clearly. But for the period of the Inquiry, we would
0072
1 have to rely on the "fuzzy" matching process which
2 I have mentioned previously to identify or hopefully
3 identify the same people.
4 Q. Just exploring how great a problem this is for us in
5 terms of the utility of it, we dealt with the question
6 of episodes and spells, which is one spell consisting of
7 1, 2, 3, 4, 5, 6, however many, different episodes under
8 different consultants.
9 What the patient identifier would resolve is more
10 than one spell, is it?
11 DR AYLIN: If a patient was admitted into hospital, had
12 an operation, was sent home and then came back some time
13 later and died, if we had a patient identifier, we would
14 be able to pick up that outcome, but it is very
15 difficult at the moment without a patient identifier.
16 MR LANGSTAFF: So, to put a sort of character to it, if
17 one had a patient who came in for surgery, the surgery
18 was not particularly successful but the patient was able
19 to go home -- we had such an example earlier in one of
20 the parents who gave evidence before us -- but
21 subsequently then required further surgery which may or
22 may not have been a complication of the first surgery,
23 as well as of the condition, one would not pick it up
24 from this data.
25 DR AYLIN: It would be difficult without a patient
0073
1 identifier. You could use data, birth, sex and
2 postcode, but it might be that that person has moved
3 between the two admissions or there might be error on
4 inputting the postcode or the date of birth, so it would
5 be more difficult to pick up different spells within the
6 same children.
7 Q. But if one had a view from, let us suppose, clinical
8 experts that in general, with cardiac surgery in
9 infants, a patient was unlikely to come back within
10 a matter of years, except for those operations where,
11 necessarily, there would have to be further operations
12 conducted -- one thinks of the Sennings operation as one
13 we have heard and will hear of -- if the general view
14 was that it was an operation which was unlikely
15 therefore to bring someone back because, in crude terms,
16 the outcome was likely to be survival for a long time
17 or, sadly, death, then if that were the general
18 evidence, this would not be a particular problem for us?
19 DR AYLIN: No. You would have to identify what you
20 wanted to look at. If you were just interested in
21 deaths within a spell, then that would be easier to look
22 at than if you wanted to look at people coming back and
23 being readmitted.
24 Q. Again, if one were making a comparison using the
25 national database to make a comparison between, say,
0074
1 Bristol and other hospitals or another hospital, then if
2 in general terms within the same specialty surgery was
3 conducted in the same way, on that assumption, this
4 would not be a problem: you would still be able to make
5 the comparison across crude rates of mortality?
6 DR AYLIN: Yes.
7 A. If you compiled the information in the same way, and you
8 could make an assumption about the way the care is
9 provided, then that would not be so much of a problem.
10 Q. But the conclusion would be falsified if the assumption
11 is wrong, obviously?
12 A. Yes.
13 Q. One further matter before we have a break for lunch. It
14 is the next item that you have on your slide. What is
15 the point there?
16 DR AYLIN: We have already talked a little bit about the
17 limits of the OPCS 4 coding system and the inaccuracies
18 or the differences in the way that they are coded and
19 disagreement between different hospitals, and if we
20 wanted to look at comparing one unit with the rest of
21 the country or with other specialist centres, we would
22 have to think about this. Certainly we would not want
23 to be using data before 1991, say, because of these
24 problems with the procedure codes.
25 There is one point that was not brought up in the
0075
1 coding process, in that different hospitals, the coders
2 actually use different sources of information in
3 different hospitals to code the data. Some hospitals
4 use the clinical records to enter the data into the
5 Patient Administration Systems and some hospitals use
6 the discharge summaries. From my experience working in
7 a hospital in the early days of the code system, the
8 coders used discharge summaries, but certainly at
9 St Mary's where I am working at the moment, the coders
10 look at the individual clinical records and type the
11 information from there.
12 So there may be differences between variability
13 in coding because of that reason as well.
14 MR LANGSTAFF: Does that accord with your own
15 experience?
16 A. Yes. I think Mr Aylin has more firsthand experience
17 of this than I have. Certainly we are aware that there
18 are different practices and different ways of doing
19 things around the country which could give rise to
20 differences in the codes.
21 Q. What one is describing there is not so much a different
22 practice as a different primary source. I suppose the
23 discharge summary involves, again, somebody having not
24 only looked at the clinical records but also having had
25 experience clinically of the patient, saying in essence
0076
1 what has happened and that being translated into codes,
2 as opposed to someone actually trawling through the
3 records and doing it.
4 Would you hazard a view which was actually likely
5 to produce the more accurate result?
6 DR AYLIN: No!
7 MR LANGSTAFF: On that note, it might be an appropriate time
8 to break for lunch?
9 THE CHAIRMAN: If I may say, we are very grateful for what
10 we have heard this morning. We will adjourn now until
11 1 o'clock, when we will continue to hear this evidence.
12 Thank you.
13 (12.20 pm)
14 (Adjourned until 1.00 pm)
15 (1.07 pm)
16 MR LANGSTAFF: Dr Aylin, you have on the side, which you
17 have prepared, the last two items we have not yet dealt
18 with: potential weaknesses in HES, the first of them
19 data quality and the second a comparison with other
20 centres --
21 THE CHAIRMAN: Mr Langstaff, I think there were three.
22 MR LANGSTAFF: The definition of outcome mortality? We had
23 begun to touch on that.
24 DR AYLIN: We had covered it a little bit.
25 MR LANGSTAFF: Let us go on and talk about that. What needs
0077
1 to be said, or asked, about outcome mortality?
2 DR AYLIN: We just need to be aware when looking at the
3 HES data that one only looks at deaths in hospital, one
4 cannot look at deaths after people have gone home. It
5 is also very difficult if people have transferred to
6 another hospital. As long as we acknowledge that we can
7 just look at deaths occurring within a particular spell
8 within a hospital, then I think HES data is useful for
9 that.
10 A. I would agree with that. It is only picking up the
11 events actually happening in hospital. We are looking
12 at ways eventually to bring in the Office for National
13 Statistics, more general mortality information, to see
14 whether we can pick up eventually deaths outside
15 hospital.
16 We are also doing linkage work so that where
17 a patient is transferred from one hospital to another
18 and then subsequently dies, that would also be picked up
19 as part of the same hospital spell, as it were.
20 Q. You would be able to do that nowadays because of the NHS
21 number, presumably?
22 A. With the NHS number coming in, we will be able to do
23 linkage, and we need to look at the best way to do
24 linkage with other information, yes.
25 Q. Looking at it for our purposes, providing we understand
0078
1 that it is the hospital and therefore we cannot pick up,
2 let us suppose, tertiary referrals following surgery to
3 somewhere else where a child dies, if one eliminates
4 cases such as that -- that may not be the only
5 example -- then we have something which is usable?
6 DR AYLIN: Yes, I think that is fair.
7 THE CHAIRMAN: Mr Langstaff, Mrs Maclean has a question.
8 MRS MACLEAN: If I may put a question to Dr Aylin
9 before we go on, just to complete this section: you are
10 describing the difficulty of or at least the limitations
11 of HES relating to mortality data.
12 In the case of a patient who has more than one
13 episode in the same hospital, would the mortality data
14 be linked to the last episode?
15 DR AYLIN: Yes. Each finished consultant episode has
16 a field, or actually two fields, which give a method of
17 discharge. If that field is blank or if it has a dash
18 in it, that means that there is another episode in that
19 spell. It is only the final episode in a spell or in
20 a length of stay in hospital which has whether they were
21 discharged home, whether they were transferred to
22 another hospital or whether they died.
23 Q. The reason I asked is, in the case of the management of
24 intensive care units where a patient might be under the
25 clinical care of an intensivist rather than a surgeon.
0079
1 In that instance it would be the intensivist episode
2 which would be linked to the mortality.
3 A. Yes. That is one of the reasons why we would want to
4 try and match as many episodes to spells as we can, or
5 link episodes using date of birth, sex and postcode.
6 We have already talked about some of the problems with
7 that.
8 MRS MACLEAN: Thank you very much. It is very helpful.
9 THE CHAIRMAN: Mr Langstaff, again, Professor Jarman
10 has a question.
11 PROFESSOR JARMAN: While we are on the subject, if
12 a Trust had a practice whereby they tended to discharge
13 patients to another Trust for nursing care, et cetera,
14 and that was more likely to happen than average, you
15 would not pick up that death if the person were to die
16 outside the Trust, would you?
17 A. If they subsequently died in another Trust and it was
18 part of the same, as it were, spell in hospital, even
19 though it was not the same hospital, we are now picking
20 that up in some of our linkage work.
21 Q. In the time we are talking about, you were not able to?
22 A. We have not gone back and done that linkage work for the
23 earlier period of time.
24 Q. So in those types of hospitals, death rates would
25 perhaps artificially appear to be lower?
0080
1 A. Yes.
2 MR LANGSTAFF: We have not looked at the definition of
3 outcome. How easy is it to use HES, for instance, to
4 take the case of paediatric cardiac surgery which has
5 the result that, or put another way, leaves the child
6 with brain damage?
7 A. In a sense, HES does not really cover anything on
8 outcomes, apart from the deaths information. It was
9 designed as a sort of national tool to look at for
10 planning purposes and that type of thing.
11 DR AYLIN: There are a number of fields allocated for
12 diagnosis, at least four fields allocated for diagnosis,
13 but for each episode, if you have an operation in that
14 episode, you do not know whether the diagnosis given on
15 that episode happened after the procedure or before the
16 procedure.
17 I think perhaps multiple episodes within a spell,
18 one might be able to look at the episodes after an
19 operation and look at any diagnosis on those episodes,
20 but as we have already said, it seems that -- certainly,
21 generally speaking, it is only 5 per cent of spells
22 which have multiple episodes. So that would only be
23 able to contribute to that small percentage.
24 MR LANGSTAFF: There may be some measures which are,
25 as it were, surrogates for outcome: for instance, the
0081
1 length of time that a patient spends as an
2 inpatient: would that be measured? Can one take that
3 out of HES?
4 DR AYLIN: One could measure the length of stay; one
5 could measure the total length of a spell. One would
6 have to ask, what would that mean, what would that tell
7 you?
8 MR LANGSTAFF: There are obviously issues of interpretation.
9 Is it, for instance, that this hospital simply wants to
10 get its beds clear to bring in other patients in order
11 to secure patient care in the neighbourhood, or does it
12 have a policy of keeping patients in? One only has to
13 think of maternity hospitals, where nowadays some
14 mothers are out within 24 hours; at the time of my own
15 family, it was 7 days. Obviously policies change.
16 Nonetheless, given difficulties of interpretation, there
17 is some measure there that might be at least flagged up
18 and one could begin to interpret, is there?
19 DR AYLIN: Yes. One might be able to look at the time
20 from procedure to discharge, or death.
21 MR LANGSTAFF: So, data quality.
22 DR AYLIN: That was a general comment. I think we spent
23 most of the morning talking about the problems with data
24 quality and why that would give difficulties in
25 interpreting the data. Certainly going back before
0082
1 1991, I think there are at least serious "health
2 warnings", to use your words, on that.
3 From 1991, I think the quality is such that we
4 could certainly look at the data and see, maybe look at
5 some of these quality issues and use the data for the
6 sort of purposes that the Inquiry wants.
7 We did have the distinction of coverage,
8 completion and accuracy that David used as the
9 parameters of data quality. I think it is possible --
10 and Richard Willmer indeed does it -- to look at
11 coverage by comparing HES data with KP70 returns. We
12 have covered that a little bit. It is also possible to
13 look at completeness of fields, how many of the fields
14 actually have a valid entry. That can be looked at, and
15 indeed we have looked at that and seen that, certainly
16 for diagnosis, it has improved over the years.
17 But the problem of accuracy, how far do the
18 computer records actually reflect the truth, is a much
19 more difficult aspect to look at and may be impenetrable
20 in some respects.
21 I think, looking at the first two parameters,
22 things obviously seemed to be getting better, and it
23 looks as if we probably can use HES data for an analysis
24 of the things we want to look at.
25 A. I would support that. A first look at the tables you
0083
1 were showing us this morning indicate that. It may be
2 that when you get into a specialised area, you find
3 other issues about the quality. It is a bit like
4 getting back to the male hysterectomies I mentioned this
5 morning. It is not until you use the data that you
6 discover the point. There is data there that is useful
7 to the Inquiry.
8 MR LANGSTAFF: The starting point is that we look as
9 though we have useful material which will tell us on
10 a national basis what the position was, all due regard
11 being made to the points we have been discussing in
12 terms of coverage and completeness. There remains the
13 query over accuracy.
14 Can I just explore that a little bit more? Both
15 of you obviously think that despite that, or despite any
16 queries one might have, the data is usable.
17 Accuracy will depend upon obviously the accuracy
18 of input and the accuracy of input to an extent that it
19 is a matter of judgment, the interpretation of the
20 source material by the coder.
21 DR AYLIN: Yes.
22 MR LANGSTAFF: So one is back to the coder and how devoted,
23 how interested, how motivated, he or she is.
24 DR AYLIN: The coder also depends on the accuracy of the
25 clinician either in writing the discharge or in the
0084
1 notes that they make on the clinical records.
2 Q. So essentially, human factors?
3 DR AYLIN: Yes.
4 MR LANGSTAFF: But to an extent, is it true to say that
5 human factors are present in any data source from which
6 one would seek to derive information?
7 A. Yes.
8 Q. And if one were to say, human resources are a well-known
9 fallible source, one could not rule out information on
10 that basis, otherwise most information that we use for
11 epidemiological purposes we would never use at all,
12 would we?
13 A. I do not think it is a case for ruling it out. It is
14 a case of being aware of it when you start to analyse
15 the data.
16 Q. Are you answering as you do because you are both
17 involved in statistics and therefore have an interest in
18 this?
19 DR AYLIN: I think that is a common sense view that most
20 people would understand: there are limitations to any
21 data sources that you look at, but there is also some
22 utility in data sources and you can use them as long as
23 you acknowledge some of the limitations within it.
24 Q. I think this is coming to the last point, the comparison
25 of other centres. Subject to its limitations, what can
0085
1 HES from 1991 onwards tell us? We have pretty well
2 ruled out pre-1991 for the reasons we have spoken about.
3 A. Yes. The United Bristol Healthcare Trust was one of the
4 first-wave Trusts, I think, which came in in 1991, so we
5 had information back to 1991/92, and as other Trusts
6 came on stream, we had the information for those
7 separately as well. So subject to the limitations we
8 have discussed during the course of today, one can
9 produce some analyses from HES and probably use HES as
10 well as an audit tool with other data sources.
11 MR LANGSTAFF: So we can measure obviously straight
12 mortality, survival. We can get some idea, very
13 vaguely, of quality of outcome by length of time,
14 subject to interpretation?
15 A. That seems a suitable thing to look at, yes.
16 Q. We can get postcodes, can we not?
17 DR AYLIN: Yes.
18 MR LANGSTAFF: Which would give perhaps some idea of
19 referral patterns, would it?
20 DR AYLIN: That would be possible, yes. Not so much
21 referral patterns, because up until the time that we are
22 looking at, outpatient records were not included on the
23 HES data; is that right, Richard?
24 A. That is right. Outpatients are not part of the record.
25 DR AYLIN: So one would only be able to look at people who
0086
1 have been admitted to the hospitals.
2 To that extent, one could look at where patients
3 were coming from and whether patients within the
4 catchment area of Bristol were perhaps going elsewhere
5 as well.
6 MR LANGSTAFF: So we are unlikely in the context of
7 paediatric cardiac surgery to be concerned with the
8 outpatient attendances rather than the inpatient ones?
9 We may establish a referral pattern for those people who
10 were inpatient which may tell us or leave some gaps
11 which need to be explained. We might be able to find
12 out if we looked at Great Ormond Street records that
13 there were a substantial number of people with a Bath
14 postcode, for example?
15 A. Yes.
16 Q. That might raise questions. It would not tell us
17 anything, save that that was what the position was?
18 DR AYLIN: Yes.
19 A. Yes.
20 MR LANGSTAFF: What else, if anything, would one be able
21 to determine from the data? How valid a comparison
22 might one make between Bristol on the one hand and other
23 centres? You were talking about using the HES data to
24 audit or cross-refer other data.
25 A. Yes. You check at aggregate level whether the numbers
0087
1 of cases are the same held on other systems. It may
2 tell you something about the quality of HES or the
3 quality of the data on other systems and compare broad
4 numbers. If you have different sources, you can always
5 see whether they match or areas where they should do.
6 Q. So if the cardiothoracic register, for instance, was
7 broadly comparable to the results in the HES analysis,
8 you might have confidence in both?
9 A. Yes.
10 Q. What happens if it is not the case?
11 A. It is like everything else, it raises questions and you
12 start to explore which one you think may be right or
13 wrong.
14 Q. Or neither?
15 A. Or neither.
16 Q. We are going to hear about the register in a moment.
17 I am sorry to flag that up!
18 Is there any other point which you would wish to
19 make or discuss as to the historical position in respect
20 of HES, from 1984, HIPE, and HES from 1984 to 1995.
21 DR AYLIN: To sum up, I think the HIPE data, because of
22 the 10 per cent sampling and the published tables --
23 I have looked at the published tables and they do not
24 give us any of the information that we want. They will
25 give you region of treatment, but then, looking at the
0088
1 tables, you cannot disaggregate those into age and
2 specialty and things like that.
3 So far as HIPE is concerned, I think it is
4 unlikely that will shed much light on the Inquiry.
5 For the early HES data years up until 1991,
6 I think there are serious problems with those years.
7 I think if we wanted to analyse that, we would have to
8 analyse with a great deal of caution. I would suggest
9 that we would not want to analyse it. From 1991
10 onwards, I think we have some useful information which
11 has a fair degree of completeness and coverage. There
12 is a question mark over the accuracy, but I still think
13 we can go forward on that and take a look at it.
14 A. I would support that, yes.
15 MR LANGSTAFF: I have dealt with the historical picture.
16 You, of course, are responsible for HES now and have
17 learned, I think, judging by what you say in your
18 statement, some of the lessons of history. We are
19 operating of course in 1999, now.
20 Is there anything you would like to say generally
21 to us, and indeed to the wider public, about the quality
22 of the data now?
23 A. I think the point made today has been very valid about
24 the period in question and in general, we see HES and
25 its improvements as a sort of evolving process. If you
0089
1 look back ten years, we were not using HES at all. If
2 you look back five years, we were using it very broadly
3 for national planning purposes. If we look at it now,
4 there is the recently published clinical indicators
5 which are starting to help people ask the questions that
6 I think this Inquiry is interested in about rates
7 between different hospitals and why they differ.
8 That in itself is stimulating a lot of interest
9 in data quality, so we see, I think, a great interest in
10 data quality through using data and, getting back to
11 a point we were discussing five or ten minutes ago, the
12 reasons for not using the data, I think you can always
13 find reasons for not using data, but if you start to use
14 it, you start to improve the quality of it and you get
15 into the upper spiral I was mentioning this morning,
16 that people use it more to get it right rather than the
17 downward spiral and saying "This is not 100 per cent
18 perfect"; you spiral down.
19 There is a lot more interest in HES data quality
20 and getting it right. We are producing for all Trusts
21 around the country a quite sophisticated data quality
22 indicator to show how some of the key fields are now
23 performing so they have a chance to look at that and see
24 how to improve it.
25 With regard to accuracy, there is a large
0090
1 programme as a result of the information for health
2 strategy which came out last year, for improving
3 information generally and in the long-term, there is
4 a look at electronic patient records and electronic
5 health records which will probably help in the future,
6 to look into these areas.
7 But also, data accreditation at Trusts, which as
8 I understand it is going to become mandatory from next
9 year, that actually systems must be in place within the
10 Trusts to ensure the accuracy of the data that is being
11 coded there. I think that will be another sort of
12 positive step to help things for the future.
13 Finally, the other work we are doing, I mentioned
14 the inclusion of the NHS number, we are hoping by doing
15 that to be able to do more linkage of the type we have
16 been discussing today so we can answer the question not
17 just that somebody went into hospital, but what happened
18 next, what happened afterwards. That is the key, often,
19 to looking at outcomes of treatment. I think it was
20 just a message that there is a lot happening at the
21 moment which hopefully will prepare the ground for
22 better information in the future.
23 Q. It has been suggested in some circles, certainly in some
24 press circles, that the recent initiatives in publishing
25 data produced as it were a league table: one would
0091
1 describe one hospital as best and the other as worst.
2 There is an obvious parallel between those who might see
3 data showing Bristol, if it does, as an outlier, as
4 indicating that Bristol is either very much better or
5 very much worse than other hospitals.
6 Can data actually show anything as to the badness
7 or goodness of treatment without interpretation?
8 A. I think the Secretary of State actually said that the
9 data published recently should not be treated as league
10 tables, they should actually be treated as a set of
11 information which needs to be explored, as our first
12 speaker said this morning, to see why there are
13 differences rather than saying a place is good or
14 a place is bad.
15 DR AYLIN: I think in looking at differences between
16 hospitals, you need to remember that there may be many
17 explanations for these differences. You may be looking
18 at differences in severity of disease and case mix of
19 the unit or the hospital working on. You may be looking
20 at variations in coding practices and coding quality.
21 This whole issue of coverage, completeness and accuracy
22 may cause differences between centres. You may be
23 looking at chance variations, random fluctuations from
24 year to year, as David was saying, bad luck some years,
25 and on top of that, you may actually be looking at what
0092
1 you actually want to look at and I suppose the idea of
2 looking at this data is trying to look at actual quality
3 of care, but you have to take all those other things
4 into consideration when interpreting the analysis.
5 MR LANGSTAFF: But the first step is whether one can do
6 the analysis with a certain degree of confidence of
7 getting something which is worthwhile, and your joint
8 view is, yes, we can in this case.
9 DR AYLIN: Yes.
10 A. Yes.
11 MR LANGSTAFF: Sir, those are the questions I want to ask of
12 Mr Willmer.
13 THE CHAIRMAN: We do not have any questions. Mr Pirani?
14 MR PIRANI: I have no questions, thank you, sir.
15 THE CHAIRMAN: Mr Willmer, thank you very much indeed.
16 Please come and, as it were, join us in the hearing
17 chamber elsewhere, and we will move on, shall we, to our
18 next witness, Mr Langstaff?
19 MR LANGSTAFF: Yes. Mr Keogh. Mr Keogh, if you would not
20 mind standing to take the oath?
21 MR BRUCE KEOGH (SWORN):
22 Examined by MR LANGSTAFF:
23 Q. Mr Keogh, what is your full name?
24 A. Bruce Edward Keogh.
25 Q. You are a consultant cardiothoracic surgeon?
0093
1 A. Correct.
2 Q. In that capacity, you are, are you, the Secretary of the
3 Society of Cardiothoracic Surgeons of Great Britain and
4 Ireland?
5 A. That is correct.
6 Q. In that capacity, you have prepared for us a statement
7 which we find, WIT 163, beginning at page 1, and you
8 give us a number of various appendices to that: the
9 statement and the appendices together constitute 39
10 pages. Let us just go to the very last of those,
11 WIT 163/39.
12 Is that statement true and accurate?
13 A. It is.
14 Q. You appreciate -- having sat in on this morning's
15 discussion -- that the purpose is to look at the
16 strengths and weaknesses of the data sources, to find
17 out something about them so that those who follow these
18 proceedings on the Internet will understand, and know
19 where we are, and to identify what use, if any, this
20 Inquiry can make of the data which your Society has
21 kept.
22 Again, a bit of background. You tell us that the
23 register began in 1977?
24 A. Yes.
25 Q. If one were to look at some of the features of it, it
0094
1 sought returns from a number of different hospitals in
2 both cardiac and thoracic surgery?
3 A. There are two registers. Perhaps it would be helpful,
4 Mr Langstaff, if I just gave you a very brief overview
5 of the nature of data collection for the Society.
6 In 1977, under the guidance of Sir Terence
7 English, the Society established the Cardiac Surgical
8 Register, which strives to collect activity and
9 mortality data on a number of procedures, both adult and
10 paediatric, from all NHS cardiac units within the United
11 Kingdom.
12 At about the same time, a Thoracic Surgical
13 Register was also established with the aim of collecting
14 information on thoracic surgical procedures from
15 thoracic units within the United Kingdom. At that
16 point, there were two sources of information.
17 Then, in 1986, a Heart Valve Registry was
18 established by Professor Ken Taylor and that had
19 a slightly different remit in that it collected
20 information on heart valve implantations throughout NHS
21 units in the United Kingdom. This data was collected
22 from a slightly different source. It has been a useful
23 source of information, because it has enabled us to
24 cross-link and to see whether both registers are coming
25 up with approximately the same numbers.
0095
1 Then, in 1994, the Society recognised some of the
2 weaknesses of the Cardiac Surgical Register, in
3 particular, its inability to correct for case mix and
4 therefore its very limited value in any form of risk
5 stratification.
6 To that end, we strive to develop a more
7 comprehensive patient orientated database, with the aim
8 of collecting about 150 data points from each individual
9 patient undergoing adult cardiac surgery in the United
10 Kingdom.
11 We have come some way in the last five years, but
12 we have still not achieved 100 per cent collection of
13 that data throughout the UK. So that remains a parallel
14 initiative, parallel to the existing cardiac and
15 thoracic surgical registers and the United Kingdom heart
16 valve register.
17 Q. The register is voluntary, is it?
18 A. Indeed.
19 Q. So is there any funding, incentive, for people to
20 co-operate with the register?
21 A. There is absolutely no funding incentive at all. The
22 register, from its very inception, has been funded out
23 of subscription from our members. There is no other
24 source of income.
25 Q. Does it mean that those who contribute to the register
0096
1 have this advantage, possibly: that they are actually
2 interested in providing the data as accurate data, as
3 opposed to being paid for a clerical job, part of which
4 includes the collation and transmission of data?
5 A. Absolutely correct. The incentive for the average
6 cardiac surgeon is that he would like to see how the
7 specialty is evolving. He is interested in receiving
8 aggregated data and to that end the only way he will get
9 aggregated data is if he submits his own data.
10 Q. Is that strictly right? Can he not simply go along to
11 the next annual meeting of the Society and pick up
12 everybody else's aggregated data and secretly compare
13 his own?
14 A. Indeed, he could.
15 Q. So why does he not?
16 A. I would like to think because he is a person of
17 integrity, but assuming he were not, he would be
18 encouraged to submit his data by peer group pressure and
19 also from the Secretary of the Society, who sends out
20 numerous reminders asking for that data.
21 Q. That is you?
22 A. It is.
23 Q. Sending out numerous reminders?
24 A. Yes.
25 Q. So some people are reluctant to send data?
0097
1 A. They are not reluctant, they are busy people and have
2 other things to do, and maybe this is not always at the
3 top of the agenda.
4 Having said that, we get very good returns for the
5 data. Perhaps I could give you an example of the
6 completeness. For example, between 1985 and 1989,
7 a five-year period, there were only three units out of
8 approximately 35 who failed to return their data. Of
9 those three units in fact two units were the same. That
10 represents less than 2 per cent of contributions
11 missing. So we believe that the data is relatively
12 complete.
13 Q. In terms of those units, they were cardiac units?
14 A. Yes.
15 Q. Were any of those units, units performing paediatric
16 cardiac surgery, so far as you know?
17 A. Yes.
18 Q. And there is no problem, I think, in identifying them
19 because they were named and shamed at the time?
20 A. Yes, indeed. This goes back in some sense to your
21 previous question. There is an incentive to submit the
22 data, in that if you do not, you are named and shamed
23 and published and publicly mentioned at our annual
24 business meeting. That tends to encourage submission in
25 the following year.
0098
1 Q. Can you tell us which units deserved public humiliation
2 in the three units you mentioned?
3 A. There have been a variety of units over the years. Of
4 the three that I mention, only one springs to mind and
5 that was the Harefield Hospital.
6 Q. You produce a set of figures, if we go to 163/8, at
7 Appendix B you have given us a review of incomplete
8 returns. If one looks overleaf to 163/9, we can see --
9 your statement will go out on the Internet, so anyone
10 wishing to look at this can see -- that a number of the
11 units, the same numerical digit, appears time and time
12 again between 1984 and 1992 as having supplied
13 incomplete returns.
14 The status is described as "missing as stated".
15 How does this fit in with the answer you have
16 given me a moment ago that there were only three units,
17 so far as cardiac surgery was concerned, in the central
18 years of our Inquiry, which did not supply data?
19 A. Firstly, I would like to say that if you look in the
20 sentence just above this table, I say:
21 "Please find below a table of the missing data as
22 requested."
23 I think that should read "forms" because some of
24 the these are simply forms that are missing, the data
25 itself may not be missing.
0099
1 There are several reasons why this rather large
2 table gives the impression of a lot of missing data.
3 Firstly, some of the unit numbers which are described
4 represent units which perform pure thoracic surgery, so
5 they will go down as "no cardiac data". Some units have
6 ceased to practice cardiac surgery. Some units have
7 merged: for example, if you look at that table, units 2
8 and 1 have merged. New units evolve and attract new
9 codes, and to perhaps put that in perspective, in
10 London, for example, Charing Cross Hospital and the
11 Westminster Hospital no longer practice cardiac
12 surgery. The Brooke Hospital moved to Guy's and the
13 Guy's unit has now moved to St Thomas's. The National
14 Heart Hospital has closed and now moved to the
15 Brompton. The Glasgow Children's Hospital in Scotland
16 has moved to the Glasgow Royal. Killingbeck Hospital
17 has now closed and moved to the Leeds General Infirmary,
18 and more recently the London Hospital has moved to
19 Barts.
20 If one sees it in that perspective, those code
21 numbers are not reallocated, they remain in the system
22 and data will not be coming in from those units.
23 Q. So one should not look at the table and say "What a lot
24 of missing data", because in fact there is very little
25 missing data, is what you are telling us?
0100
1 A. There is very little missing data.
2 Q. Can we look at the advantages and disadvantages from our
3 point of view. One of the advantages would be the
4 completeness of coverage which, from your answers you
5 have just given me, would appear to be the case?
6 A. I believe so.
7 Q. What would you say were the weaknesses in drawing
8 conclusions for the purposes of comparative performance
9 between one centre and another? Perhaps
10 Dr Spiegelhalter would like to come in on this and
11 respond to what you have to say.
12 A. The register has many strengths but it also has some
13 weaknesses. I think one of the main weaknesses is that
14 there is voluntary contribution, and implicit within
15 voluntary contribution is the feeling that there is some
16 degree of tolerance or flexibility in the way that data
17 is treated.
18 Secondly, we have no validation of the data. We
19 would very much like to have validation of the data so
20 that we can be sure not only that the data we were
21 getting is accurate, but also that it is complete and
22 that the two issues are different. That, of course,
23 costs money and we are a Society with only a few hundred
24 members and to finance that sort of exercise is very
25 difficult, although we are exploring validation of data
0101
1 at the moment: I have a meeting with the Nuffield Trust
2 tomorrow and the Rand Organisation from the United
3 States, and we are meeting with the California Coronary
4 Artery Bypass Reporting Group to look at ways of
5 validating this. In fact we have acquired some external
6 funding via the Rand Organisation and the Nuffield Trust
7 to look at the validity and accuracy of data for
8 coronary surgery in selected units throughout the United
9 Kingdom over the next year.
10 So we are moving in that direction, towards
11 validating our data. I think those are the main
12 weaknesses.
13 Q. The question of validation obviously throws doubts on
14 the accuracy that one sees the figures as having. We
15 drew a distinction in the last evidence between
16 coverage, completeness and accuracy.
17 In terms of accuracy, how likely is it, as you see
18 it, from your knowledge of those in the trade, as it
19 were, that the data would be inaccurate?
20 A. I think broadly speaking the number of operations
21 performed is pretty accurate because that is easy to
22 validate internally within a unit, no matter who
23 collects that data.
24 If I would perhaps walk you through how this data
25 might be collected, in the old days, before the advent
0102
1 of local computer systems, most of the information on
2 activity was collected from a combination of examining
3 the theatre logbooks and the intensive care unit
4 logbooks.
5 Nowadays the activity data is collected from
6 specific dedicated computer systems within cardiac units
7 in the UK which are independent stand-alone systems, as
8 a rule.
9 The difficulty, really, is the collection of
10 mortality data. Firstly, I should say how we define
11 mortality data. Historically, it has been defined as
12 death within 30 days of surgery, but we have modified
13 that definition to death on the same admission as
14 surgery within the base hospital in which the surgery
15 was carried out, for the reasons that were outlined when
16 the HES data was being discussed.
17 Clearly, that is easily validated.
18 I think our major weakness is that there is
19 under-reporting of deaths. To give an example of how
20 that might happen, it is quite possible for somebody to
21 come in for an emergency operation at 5 o'clock on
22 Sunday evening, be admitted, taken to an operating
23 theatre, sadly dies, and be taken to the mortuary so
24 that by Monday morning there is very little trace left
25 of the admission or the operation. The outcome for that
0103
1 particular individual, if the data is looked at
2 retrospectively later, may be difficult to track.
3 So I think over the years there has been
4 under-reporting of deaths.
5 Q. Just pausing there, will there not be a surgeon's log
6 and theatre log for that emergency admission?
7 A. There will certainly be a theatre log. I remain unclear
8 as to what you mean by a "surgeon's log"?
9 Q. The surgeon's log is a log the surgeons may keep of
10 operations they do. I do not know if that is a regular
11 practice amongst all surgeons. Certainly we know that
12 the two surgeons who did most of the operations on
13 paediatric cases here in Bristol, at the time we are
14 concerned with, both kept logs of their own.
15 Is that a common feature?
16 A. I think it varies from unit to unit. I think those
17 units that have advance data collection systems simply
18 have a comprehensive database and if the surgeon wants
19 to know what he has done now, he does a search for
20 operations against his name. So he would not keep
21 a specific separate log.
22 Q. The example you were giving -- I stopped you in
23 mid-flow -- saying is it really the case that the death
24 would go missing because it is really there in the
25 theatre log, at any rate.
0104
1 A. That is true provided somebody has written the outcome
2 in the theatre book, which is not always the case, or if
3 a patient goes briefly to the intensive care unit, the
4 death may not be easily tracked by a member of staff
5 going back six months later.
6 Q. So the patient is tracked, the operation is tracked,
7 but not the outcome?
8 A. Indeed.
9 Q. And is what you are describing something which varies
10 as between one unit and another?
11 A. I think it is. I think, with the passage of time, the
12 ability to track deaths and to have accurate mortality
13 data has increased exponentially, so that most units now
14 have very good and accurate data of their own
15 performance.
16 MR LANGSTAFF: Dr Spiegelhalter, how does one approach the
17 problem that there may be over identifying mortality for
18 the purposes of a register such as this?
19 A. With some difficulty. We will have information from
20 some other sources: we will know exactly what happened
21 in Bristol, but the problem is that different centres
22 have different policies. If some have consistently
23 under-reported, then it is going to be very difficult to
24 identify what they did compared with other centres who
25 might produce very accurate figures. If there was
0105
1 consistent under-reporting, the data would be biased but
2 one would still perhaps get a reasonable idea of the
3 underlying variability between centres.
4 DR SPIEGELHALTER: A question I would quite like to ask, if
5 I could: whether there is any feeling in the trade
6 whether some centres were perhaps more reliable or
7 unreliable than others, without any names being
8 mentioned. It would be good to get the impression of
9 what was felt at the time, over this period.
10 A. I have never heard any allusion to the fact that one
11 unit is worse than another at reporting their
12 mortality. I think where there is an error it is small
13 and it is constant.
14 DR SPIEGELHALTER: Can I ask some others?
15 MR LANGSTAFF: Please do.
16 DR SPIEGELHALTER: As a point of clarification, on the
17 definition of deaths, what was it over the period that
18 we are considering? Late 1980s to mid-1990s.
19 A. It was 30-day mortality. I think there is an issue
20 there, because I think some units assiduously tried to
21 track patients after 30 days and I think others found it
22 more difficult.
23 DR SPIEGELHALTER: One of the other questions is whether,
24 when this data was received and sent off for
25 aggregation, did anyone look at the variability between
0106
1 the centres? Did anyone, even anonymously, look at what
2 was reported?
3 A. No, I do not think so, David. In order to get the
4 Cardiac Surgical Register established, some very
5 stringent reassurances were made relating to anonymity
6 of the data. You have to bear in mind that in 1977, to
7 start collecting mortality data on a surgical procedure
8 was way ahead of its time. That was 1977. We have not
9 seen it in any other surgical specialty, if you like,
10 until the recent release of outcome data in the last few
11 weeks. Do not under-estimate the diplomacy that was
12 required to attract this data. I think it is a great
13 tribute to Sir Terence English that he was able to
14 establish it.
15 As part of that, reassurances had to be given that
16 units would not be identified. There are advantages and
17 disadvantages to that. Clearly, the main advantages, if
18 you are not going to be identified, you have no interest
19 in "gaming" the system or falsifying results; you are
20 really simply interested in submitting the results as
21 they are.
22 The pro forma for the Cardiac Surgical Register
23 was filled in by a nominated individual in each unit.
24 That was then passed on to the Secretary of the Society,
25 who tore off the cover page which contained the name of
0107
1 the unit and allocated a number to that form and sent
2 the form on to a third part, who simply aggregated the
3 data.
4 Given the nature of the way that the forms were
5 returned, some coming in in advance of the deadline,
6 some coming in a few weeks after the deadline, the
7 arrival of these forms was really spread over several
8 weeks, or even months.
9 So given that the Secretary of the Society
10 received the forms, did not actually keep a written
11 record of the results, it would be quite difficult for
12 him to notice variations. On occasions, when there were
13 obviously strange results, the forms were returned to be
14 checked.
15 DR SPIEGELHALTER: When the results were presented annually
16 at the Society meetings, what was the professional view
17 of the register and its value, and did people actually
18 then go back and compare their unit's return with the
19 national aggregated totals? Was that the feeling that
20 that is what people did --
21 MR LANGSTAFF: Can I ask you just to move the microphone
22 a little bit away from you? You are doing what a pop
23 star might do and cuddling up close to it. You do not
24 have to, with that microphone.
25 A. I am sorry. Where were we?
0108
1 DR SPIEGELHALTER: What did people think about it? Did they
2 compare their results with?
3 A. I think they did. Perhaps I can answer that question in
4 two parts. Firstly, people found the aggregated
5 information interesting, in particular because it showed
6 a steady annual growth rate in the number of coronary
7 artery bypass procedures conducted throughout the United
8 Kingdom and that had implications for resources and the
9 development of the specialty.
10 Furthermore, it showed a steadily declining
11 operative mortality for some procedures which tended on
12 the whole over the years to level out and people found
13 that interesting. But there was a desire to be cautious
14 in the way that the data was interpreted, in the
15 knowledge that there was no validation in particular of
16 the mortality and in the knowledge that some procedures
17 were conducted in such small numbers that the mortality
18 information might not be that easy to interpret.
19 What I am trying to paint there is a picture of
20 people being very interested in the global data.
21 With respect to whether they went back and
22 compared their own results, that was an intention that
23 people would look at the cardiac surgical register,
24 would see how their own performance compared in terms of
25 mortality with the country at large.
0109
1 One particular difficulty with that was that
2 processing and aggregating the data took some time.
3 Quite often the register was being circulated almost
4 a year later, so in a sense it was almost out of date at
5 the time that it came and it would arrive on people's
6 desks, they would look at it and say "That is fine, that
7 was last year's data" and they were already comparing
8 next year's.
9 To get around that, what we have done over the
10 last couple of years is that we have actually sent back
11 a copy of the aggregated data along with the individual
12 unit's data in a separate column, which brings the
13 individual's attention to their own unit's performance
14 in relation to the rest of the country. Simply by
15 having the data in two columns, it draws the eye in that
16 direction.
17 DR SPIEGELHALTER: Do you want me to carry on?
18 MR LANGSTAFF: Yes, please.
19 DR SPIEGELHALTER: I realise you are not a paediatric
20 surgeon so I am not sure about answering this.
21 I suppose -- I am certainly not an expert in this area,
22 but I know what problems can come about in comparing
23 this, or the codes to use and the particular
24 classification that is used in reporting paediatric
25 results, that people are going to want to compare those
0110
1 with the classifications used in HES and with the coding
2 that is currently being done in the Bristol records.
3 The coding scheme stayed constant over that
4 period, I believe. You have given some information --
5 A. Broadly speaking, I think that is true. There have been
6 some minor changes to the register which I have
7 documented, but it is fair to say that on the whole it
8 has remained the same.
9 Q. Do you have any opinion about the reliability of how
10 that has been coded and the possibility of mapping it on
11 to other coding systems?
12 A. I do not feel qualified to answer that question, but
13 perhaps I could make one point. Whereas the adult
14 section of the Cardiac Surgical Register is very
15 procedure orientated, the paediatric section tends to be
16 more diagnostically orientated, so people fill in the
17 operations that they have done according to a diagnosis
18 which then makes it more difficult to tell what
19 operation they have done to treat that particular
20 diagnosis. Some people feel that is a weakness of the
21 register; others feel that it is a strength of the
22 paediatric register that it is diagnostically
23 orientated, and that debate continues.
24 Q. So, for example, for transposition of the great
25 arteries, it is going to be impossible to identify the
0111
1 switch operation specifically, for example?
2 A. That is exactly what I was alluding to.
3 MR LANGSTAFF: If we look at WIT 163/4, we have there
4 the changes. You flicked through your statement to see
5 where it was. This is the page. If we just scroll down
6 the page a bit so we have the table in the middle, there
7 are very few changes, I think until 1989, which might
8 affect any congenital case. I am not sure, perhaps you
9 can tell us: does "congenital valve" at the bottom
10 relate to paediatric cardiac surgery?
11 A. Yes, it does.
12 Q. So one sees "RVOT", for instance, added. Obviously
13 again it is a diagnosis rather than an operation. But
14 one question just following on from that which
15 Dr Spiegelhalter was asking: the origin of the
16 information was actually from the surgeons themselves?
17 A. Yes.
18 Q. Even though they may not have physically compiled the
19 return, they were the people supplying the information,
20 one way or another?
21 A. Yes.
22 Q. So there are not very many cardiac surgeons in the
23 country, are there?
24 A. No: just over 170.
25 Q. Would most of them tend to identify, looking at the
0112
1 paediatric cases, the same "condition", if I can call it
2 that, by using the same label?
3 A. Not always. I will cover my next comment with the
4 caveat that I am not a paediatric cardiac surgeon, but
5 congenital heart disease is a spectrum; it is not always
6 easy to assign a given congenital defect into
7 a pigeonhole, it may bridge two pigeonholes or there may
8 be more than one diagnosis, more than one defect of the
9 heart. Then a surgeon has to choose which of those is
10 the primary defect or the most important defect.
11 A combination of these two factors can lead to
12 considerable confusion in the classification of the
13 congenital heart defect.
14 Q. What Dr Spiegelhalter was asking, he was talking about
15 the coding. In essence, as I understand the register,
16 tell me if I am wrong, it is not so much a code as
17 a description of the condition?
18 A. You are absolutely correct.
19 Q. So one would have to relate the description of the
20 condition, whatever it may imply in terms of operative
21 procedure, into the codes we were talking about this
22 morning, the OPCS procedure codes, in order to have
23 a comparability of one data source with the other?
24 A. Absolutely right, and they do not easily marry.
25 DR SPIEGELHALTER: Ultimately the operations are
0113
1 broken down into whether they are palliative or
2 defensive operations?
3 A. Indeed.
4 DR SPIEGELHALTER: Which again might not make it
5 easy to make a direct comparison with the coding in
6 other databases.
7 MR LANGSTAFF: In general terms, would a palliative
8 operation be a closed heart operation?
9 A. In general terms, yes.
10 Q. So in general, the distinction between closed and open
11 would be between a palliative operation on the one hand
12 and one performed with bypass on the other?
13 A. I think that would a very general --
14 Q. That is too general?
15 A. I think a little too general.
16 Q. Qualify it for us.
17 A. If one were to insert a shunt or additional tube between
18 two vessels in the heart without coronary bypass, then
19 that is a palliative procedure, but there are some
20 palliative procedures which attempt to correct a heart
21 defect but do not do so completely.
22 I think, although I say your description is
23 general, there is some veracity in it.
24 Q. So sufficiently close to the mark for the purposes of
25 at least approaching the data?
0114
1 A. Yes.
2 Q. So far as the accumulation of data centrally is
3 concerned, there would be a limited number of operations
4 at the paediatric level. We know there were not that
5 many cases per year because we have seen what has been
6 said about it at the Department of Health level.
7 So if a particular condition was not recorded as
8 being under one head, it would of course be recorded
9 under another?
10 A. Yes.
11 Q. Would one be able to detect from looking at the returns
12 what the approach of the clinician responsible for the
13 recording was in terms of whether he put a particular
14 condition under a particular head or not, or is that
15 just something one has to recognise as a variable?
16 A. I think you have to recognise that as a variable. I do
17 not think it would be easy to dissect that out.
18 Q. To what extent does that give us problems in either
19 using this data or in using it for comparative
20 purposes?
21 DR SPIEGELHALTER: I think the way in which the
22 centres have differed in the way they have handled the
23 data is going to create problems with comparisons, and
24 again, if there is a sort of systematic bias across all
25 centres, it is not so essential if one is only trying to
0115
1 use this as a screening tool to identify potential
2 outliers.
3 So that is where it is quite important to try to
4 understand about how different centres have approached
5 the register and, for example, one of the things that it
6 would be interesting to know, although it is quite
7 difficult to answer, is whether the different centres'
8 approaches to perhaps the effort used in making the
9 returns, the seniority and the enthusiasm of the people
10 used in making returns, whether there is a feeling that
11 that varied between centres or whether reasonable effort
12 was made by everybody.
13 A. I think it has varied between centres. I think at the
14 one extreme consultants themselves have taken a very
15 vigorous interest and filled in the forms themselves,
16 and checked the data. At the other end of the spectrum,
17 this work has been delegated, sometimes to disinterested
18 members of junior surgical staff.
19 Having made that statement, I can assure you that
20 over the last few years the shift has been towards the
21 consultant end of the spectrum. I think consultants in
22 our specialty are now recognising with greater clarity
23 the importance of accurate data collection, not only for
24 their own personal use, but also for the specialty as
25 a whole. So they are taking, I think, much greater care
0116
1 over the quality of the data that is submitted to the
2 register.
3 THE CHAIRMAN: May I interject just a moment, just for
4 my own clarification? Mr Keogh, you said that this task
5 was delegated sometimes to "disinterested" members of
6 the junior surgical staff. Did you mean "uninterested"?
7 A. I meant uninterested.
8 THE CHAIRMAN: Thank you.
9 DR SPIEGELHALTER: Over the period that we are talking
10 about, would there be some marker, some way of knowing
11 from the return who was responsible for its completion
12 and therefore --
13 A. No.
14 DR SPIEGELHALTER: There is no indication at all in
15 the return as to ...
16 A. None whatsoever.
17 MR LANGSTAFF: Can I put a human problem to you? The
18 purpose of the data, part of it, you told us, was for
19 surgeons to go along to the annual presentation or to
20 get the data and say, "Look, I know what I have done;
21 this is how it compares to what they have done". In
22 order for him to do that, he would have to have some
23 feeling that he could rely upon the information as to
24 what they had done, would he not?
25 A. Yes.
0117
1 Q. If he knew what he had done was to say to his junior,
2 Bloggins, who had no interest at all, "Just shove down
3 any data you like" -- I am being deliberately
4 provocative here -- he would have no faith in the
5 information he was using because he would know from his
6 own experience it was worthless.
7 The conclusion of this human scenario is to ask
8 you whether it might perhaps have been the case that if
9 people did contribute to the register, they thought it
10 worthwhile and were probably, by and large, going to
11 treat it in a manner which more or less accurately
12 recorded what they were doing?
13 A. I think that is right.
14 MR LANGSTAFF: I do not know whether you want to come in
15 on that and whether my human scenario has any validity.
16 DR SPIEGELHALTER: It would have validity if perhaps
17 the person did not care about how that performance
18 compared with the rest of the country.
19 Q. But here one has a fairly small specialty interested in
20 how it is doing and how individuals within it are doing
21 by comparison with others, so they have to have a degree
22 of faith, one in the other, do they not?
23 A. Yes, they do.
24 Q. Because it is a small specialty, would I be right in
25 thinking that there is a degree of word-of-mouth
0118
1 reputation that goes around as to what is happening
2 elsewhere in other centres doing this particular work?
3 A. That is absolutely correct, yes.
4 Q. So it is only going to be fuelled in part by figures
5 such as this; it will also be fuelled by contact at
6 conferences and so on?
7 A. Yes.
8 Q. Does it follow from the need of one individual to
9 place himself against others that there was any need for
10 some degree of self-audit, checking that the surgeons
11 themselves had got it right in their contributions to
12 the register?
13 A. I am sorry, do I understand your question correctly that
14 you are alluding to some form of validation of the
15 data?
16 Q. Local validation.
17 A. Yes, that varied from unit to unit. Some units take
18 a more vigorous attitude towards validation of the data
19 than others, but on the whole, I think every unit
20 believes that the data they submit is accurate.
21 Q. That belief, you have told us, may be based on a variety
22 of circumstances. I am asking you to think back here,
23 really, to 1984 at the start. It may be that things
24 have changed between then and 1995.
25 The belief would have to be based on something,
0119
1 would it not? What would that "something" be: faith in
2 one's colleagues, something more than faith in one's
3 colleagues, or again -- I appreciate it may vary from
4 unit to unit, but can you give us some feel for what the
5 general position was?
6 A. I cannot actually take it back to 1984. As I understand
7 it, there was a desire for this initiative to work and
8 it was a very strong desire. There was a recognition
9 that this was the first attempt by any specialty in the
10 United Kingdom to look seriously at activity and
11 outcome, and there was a great pride in that, and there
12 still is.
13 I think the driving force was really the pride in
14 one's specialty. A small specialty which was growing,
15 which was introspective to some extent, looked at its
16 own results and that distinguished this growing
17 specialty from pretty well any other specialty in the
18 UK. So that was the driving force, as I understand it.
19 Q. That driving force would tend, if anything, experience
20 might suggest, to ensure accuracy rather than the
21 opposite?
22 A. Yes.
23 Q. Would it also tend to produce a feeling centrally that
24 individual surgeons were not necessarily satisfied with
25 the data because it could be improved?
0120
1 A. I think there is always that feeling, that the data can
2 be improved, the methods of data collection and
3 validation can be improved and the groups used to
4 collect the data can be improved. We have probably got
5 that right in adult cardiac surgery, but the debate
6 continues in paediatric cardiac surgery because of the
7 difficulties I have alluded to. So there is a continual
8 striving to include the data. And along those lines,
9 one of the other weaknesses of the data has been the
10 inability to correct for case mix, and you will see that
11 we have produced another document this year which
12 I believe you have, which is a 70 page document, which
13 examines the cardiac surgical register, alludes to its
14 shortcomings and indicates how we have attempted to
15 correct it by collecting very comprehensive
16 patient-orientated data from approximately 70 per cent
17 of adult cardiac surgical units in the United Kingdom.
18 This really is the beginning of Stage 1. We
19 envisage that over the course of the next few years we
20 will have extremely comprehensive data which will enable
21 risk stratification of patients in a reliable fashion.
22 We already have a risk model which we think is better
23 than anywhere else in the world. It will allow us to
24 look at aspects of resource consumption within the
25 specialty and it will facilitate individual,
0121
1 institutional and international comparisons of
2 performance.
3 MR LANGSTAFF: On that upbeat note, shall we take
4 a break? It will be the last break before our final
5 session of the afternoon, which I think, sir, you might
6 wish to begin at, say, 2.30?
7 THE CHAIRMAN: Yes, thank you, Mr Langstaff. 2.30 it is.
8 (2.18 pm)
9 (A short break)
10 (2.35 pm)
11 MR LANGSTAFF: Sir, just before we restart the final
12 session: Dr Spiegelhalter, during the last session you
13 said when you were talking about the data codes,
14 I think, that you were not an expert in this area.
15 What was "this area" in which you were not an
16 expert?
17 DR SPIEGELHALTER: By that I meant in coding
18 classifications for this area.
19 MR LANGSTAFF: That is what I thought you meant.
20 I am asking you simply so that no-one gets a misleading
21 impression from reading the transcript.
22 We were looking at the question of the quality of
23 the data and what one might use it for. We had I think
24 established that there might be variability as between
25 centres, but that in general, those who contributed the
0122
1 data had an interest in providing the data and getting
2 feedback from it. Was the feedback to be used for
3 a benchmark purpose in part?
4 A. I think in part, although I would not want to overstate
5 the value of the data for that.
6 Q. We have on the screen, really, something which
7 demonstrates that there was, so far as paediatric
8 cardiac surgery is concerned, very little difference in
9 content throughout the period -- a little change in
10 1989. I describe it as "a little change"; is that
11 appropriate?
12 A. I think it is a very little change.
13 Q. So in essence, one can compare 1984 with 1985, 1986
14 and 1994/95?
15 A. Yes.
16 Q. The question of adjustment for case mix we were
17 beginning to talk about, and you say there is no
18 adjustment for case mix, or was not then; that you can
19 do it with adults but you cannot do it with children?
20 A. Yes.
21 Q. Can I bring you into this, Dr Spiegelhalter, whether
22 you want to ask any questions arising and around that of
23 Mr Keogh? Does it give us a particular problem?
24 DR SPIEGELHALTER: Yes, it does, because in a sense, any
25 comparison between institutional performance should take
0123
1 into account the type of patients that they are looking
2 after. As Mr Keogh has said, in adult cardiac surgery
3 there are statistical algorithms being developed to
4 provide that sort of risk stratification. So one
5 attempts to compare like with like when you are
6 comparing institutions.
7 In paediatric cardiac surgery, there has been some
8 attempt at that in the literature. There are some
9 published risk adjustment procedures, for example for
10 switch operations, but they are not really validated
11 outside their centre of development, and there is such
12 a wide range of paediatric cardiac surgery, it covers
13 such a wide range and generally in fairly low numbers,
14 that it is difficult to think that a risk stratification
15 procedure of the type that one sees in adult cardiac
16 surgery is feasible at the present time, even with good
17 data available. It would be very difficult.
18 MR LANGSTAFF: So we have to accept there is a problem
19 with the data. Does that mean to say we cannot use it?
20 DR SPIEGELHALTER: Not at all. Broadly by classifying
21 operations into types and in classifying them by
22 severity, you are doing to some extent a risk
23 stratification already and within operative categories,
24 one could provide some basic risk stratification into
25 perhaps high risk and low risk cases. That can
0124
1 certainly be done in the Bristol data. The problem is
2 that that will not be feasible to be done
3 retrospectively on data from other centres, so
4 essentially, as we discussed earlier, one is left with
5 this exploratory screening procedure which is really
6 using the data as is to throw up possible hypotheses,
7 possible outlying centres to be examined more closely.
8 MR LANGSTAFF: You refer in your statement to the danger
9 of overinterpreting the data, which does not mean to say
10 one cannot interpret it, but warns against going too
11 far.
12 What in particular did you have in mind when you
13 were talking about the dangers of overinterpretation?
14 A. I think my main concern is in performing comparative
15 analyses on procedures which are performed in small
16 numbers, where the denominator is therefore very small
17 and the confidence limits of the outcome may be very
18 large. That would refer in particular to some of the
19 area of paediatric procedures.
20 Q. How far does one get over that by aggregating one year
21 with the other?
22 DR SPIEGELHALTER: It is a natural first thing to do,
23 although there must be a limit as to how much you do
24 that because particularly in paediatric surgery the
25 success rates have been changing over time, so era is an
0125
1 important factor.
2 MR LANGSTAFF: And procedures change?
3 DR SPIEGELHALTER: And procedures change as well, but we do
4 not know on this register, because it is diagnosis
5 based, what procedures were done specifically for these
6 conditions. One will aggregate over years; one will
7 not -- it would not be reasonable to look at individual
8 years' data. Completely off the top of my head, I think
9 one could look at two five-year bands over this period
10 and compare between those.
11 Even doing that, in some of the procedures the
12 denominator would still not be large. So that exactly
13 points to the sort of caution that would be necessary in
14 a statistical analysis. Even assuming the data were
15 completely perfect, it would be quite inappropriate to
16 label a centre as being an outlier just because they
17 happen to rank bottom out of the perhaps nine centres
18 carrying out these operations.
19 MR LANGSTAFF: To see if I have understood the point
20 properly, in lay terms, if one let us suppose had
21 a death rate of 2 per cent, so there were 2 in every 100
22 operations, it may be easy enough to see if you have
23 a series with 100 operations, but if in one centre you
24 have 2 and one of those 2 operations is unfortunate
25 enough to be one of the 2 per cent, then it would show
0126
1 on the figures as a 50 per cent risk, yet actually, if
2 one did the next 99 operations, it would be no worse and
3 no better than anywhere else; it is purely a function of
4 numbers?
5 DR SPIEGELHALTER: Exactly. There is quite a lot of
6 techniques for trying to get around that, which I will
7 not go into at the moment.
8 Q. But techniques that may be useful to us?
9 A. I think so, yes.
10 Q. So the numbers are not so small as to deprive the
11 analyst of any useful function?
12 A. Aggregated over a suitable number of years, there will
13 be sufficient cases.
14 Q. How did the surgeons themselves utilise the data for
15 seeing whether their performance was or was not in
16 step?
17 A. I think with the more common procedures, they simply
18 look at a year's data. If someone is performing
19 a procedure infrequently, then as a rule, other surgeons
20 will be performing it infrequently, and if you want to
21 see how you are performing, you have to accept that you
22 need to aggregate data over a number of years to see if
23 you get a meaningful number of patients, both your own
24 data and the national data, to get a feel for where you
25 lie in it.
0127
1 Q. So you have to approach it as a surgeon having an
2 interest in exploring whether you are doing your best
3 reasonably for patients, but accepting it may take time
4 to tell?
5 A. Absolutely right.
6 Q. In the less common procedures?
7 A. Yes.
8 Q. Thus far, I think we have dealt with a number of the
9 problems of the register in terms of the way it can help
10 us. What about the advantages? We have, have we,
11 a tool actually used for the purpose of benchmarking by
12 surgeons; we have input from those who are interested in
13 using it in part for that purpose? Honest attempts and
14 concern to get it right, even though there may have been
15 variability in approach, they are are all strengths, are
16 they?
17 A. I think they are strengths. You will see that I have
18 indicated in my statement on page 1 of Appendix A that
19 the register had three basic uses: firstly, it provided
20 members of the Society with a feel for the growth of
21 a specialty, and that remains true. Secondly, it
22 provided aggregated activity and mortality data which
23 gives a feel for whether you are performing a similar
24 spectrum for procedures as other units in the country.
25 It also facilitates analysis of surgical intervention
0128
1 rate by category for the population, because over the
2 years we have also had occasional reports on the number
3 of procedures performed per million of population.
4 The benchmarking activity is something which has
5 really come to the fore over recent years and I do not
6 think was a primary design for the register, but it is
7 of growing importance. We have expanded the register
8 recently to take this into account in that we now look
9 very specifically at activity and mortality data for
10 certain marker operations, for isolated first-time
11 coronary surgery in adult cardiac surgery, and then for
12 a variety of paediatric procedures.
13 Q. One of the features of the register is the anonymity
14 which is provided to units under the agreement which, as
15 you have described, made it impossible with the
16 diplomacy which led to its origin. That has not
17 prevented the individual surgeon or unit knowing what
18 their own data were for the purposes of seeing how it
19 compared with the national trend or the national data.
20 Would one suspect that for a number of years from
21 1984 to 1995, in most units performing the work, there
22 are still likely to be records of their returns to the
23 register?
24 A. In all honesty, I suspect not. Some will clearly have
25 kept records, but those records will probably be in the
0129
1 filing cabinet of the person responsible for submitting
2 the records.
3 Q. So if one were able to identify in respect of a given
4 unit the person responsible, one might be able to find
5 out whether he or she had kept the returns?
6 A. Yes.
7 Q. Am I right in thinking that the returns are not kept
8 centrally?
9 A. No, they are not. What has happened in the past is
10 that the forms have been completed, they have been
11 passed on to the third party that aggregates the data.
12 It transpired when the Inquiry asked for the original
13 data --
14 Q. Is that Mr Bailey?
15 A. Yes, it transpired he had kept all the returns and that
16 is how we were able to help the Inquiry by passing those
17 on. But those are the only records which we have.
18 DR SPIEGELHALTER: I would just like to make the point that
19 I see one of the great strengths of this data is that in
20 comparison, say, with the HES data for which there is
21 rather a tenuous link between the surgeon and the data
22 that is actually going to be analysed, which goes
23 through the record, through a coder into an
24 administrative system and is then derived from
25 a complicated analysis in order to get actual figures;
0130
1 whereas in a sense this data is owned by the surgeons
2 themselves and there is an absolute direct accountable
3 link in the data entered by them in the return and that
4 which is going to be analysed. It is a complete
5 short-cut as opposed to the extremely circuitous route
6 the HES data has taken. These now, it looks as if they
7 have both been analysed and compared, which will be an
8 interesting exercise.
9 Given, as you have said, that it was certainly
10 never the intention that such an analysis would take
11 place for this data, are you interested in what perhaps
12 the people who provided the data over that period will
13 feel, not about the fact that this is not what they
14 intended but about what they might feel about the
15 results of such an analysis? It might be useful to
16 discuss that now rather than after the analysis to see
17 what you think: what confidence do you think your
18 colleagues might have?
19 A. I am sorry, Dr Spiegelhalter, what sort of analysis?
20 DR SPIEGELHALTER: I am thinking of analysis in which,
21 for example, disease categories were pooled over five
22 years and the variability between centres in terms of
23 their mortality rates was explicitly calculated. Not to
24 do with anonymity, this is not an issue of breaking
25 anonymity, it is an issue of whether they actually
0131
1 believe the results that came out in such an analysis.
2 A. I think the majority of people would believe the results
3 that came out of such an analysis, but I think it would
4 have to be seen within the context of how individual
5 surgeons perceived the validity of the data within the
6 register as a whole. For those that might perceive it
7 as being a little shaky, then clearly they are going to
8 place little credence on the results of any analysis.
9 But those who believe that on the whole the
10 register data is solid, I believe that they will have
11 faith in that analysis and they will be interested in
12 the results.
13 DR SPIEGELHALTER: There was never any call to carry
14 out such an analysis? Nobody had suggested doing it
15 before?
16 A. With respect to HES data, or just --
17 DR SPIEGELHALTER: No, with respect to the Cardiac
18 Register.
19 A. No, as part of the anonymity, that would have been
20 deemed to be meddlesome. You see, there is still some
21 concern within our specialty that this sort of data can
22 be misinterpreted, often in a well-meaning way, but in
23 a way that can cause local difficulties. Surgeons do
24 get suspended within our specialty for a variety of
25 reasons which are frequently unfounded and this creates
0132
1 at times a very unhappy environment. People are very
2 keen to avoid any unnecessary difficulties of this
3 nature and I think there are those that would be fearful
4 of this sort of analysis in the past. I think times are
5 changing. I think we recognise now that
6 inter-institutional comparisons, and indeed,
7 inter-surgeon comparisons, are becoming the norm. That
8 is one of the great stimuluses for ensuring that the
9 data is accurate, clean and valid, but I think a clarity
10 of understanding of that is still growing and
11 over-analysis of old data may not be received very
12 well.
13 There are already some concerns that data which
14 was given in good faith in an anonymous fashion has been
15 passed on to this Inquiry and there are already units
16 that have indicated that because that data was not
17 validated and then, therefore, may give rise to
18 erroneous results following analysis that they no longer
19 feel keen to submit data to the Cardiac Surgical
20 Register. So mishandling this data could threaten the
21 very existence of something which our specialty has been
22 proud of for the last 30 years. This is a unique
23 register; it does not exist anywhere else in the world.
24 It has been done by the goodwill of surgeons in this
25 country, and I would be very fearful that any
0133
1 over-analysis of retrospective data which was handed in
2 in good faith could compromise the future of what is,
3 quite frankly, the most reliable source of cardiac
4 surgical information in the United Kingdom.
5 MR LANGSTAFF: I am just going to pick up on those
6 last few words, "the most reliable source" in the United
7 Kingdom of information on cardiac surgery, is the way
8 you put it?
9 A. Yes.
10 Q. So plainly your view is one of the majority which you
11 are describing as to the accuracy, the reliability one
12 can place on the register?
13 A. Yes.
14 Q. How long have you personally been involved with the
15 register and its administration?
16 A. Since 1994.
17 Q. And throughout that time, have you had cause yourself to
18 doubt your conviction, your own conviction, as to the
19 general reliability of the register, or not?
20 A. I think it is incumbent upon me to doubt the
21 reliability, otherwise I would not be doing my job
22 properly. I have less reason to doubt the activity
23 data, but I do sometimes feel that operative mortalities
24 that are reported may be a bit low.
25 We have looked at operative mortality for valve
0134
1 surgery compared between the cardiac surgical register
2 in the United Kingdom heart valve registry, and broadly
3 speaking the mortalities are very much the same and the
4 activity data is very similar. Given the source of
5 information for both of these two registers is
6 different, I find that very encouraging.
7 I think most surgeons in the country also believe
8 that the activity data is good.
9 With the advent of specialised medical databases
10 which allow us to collect a large number of data points
11 on patients undergoing cardiac surgery, it has become
12 clear, again, using these more advanced systems, that
13 the sort of numbers that we are acquiring indicate that
14 the register data is more accurate, perhaps, than I had
15 thought. That is outlined in the document I alluded to
16 before the break.
17 We feel that the current NHS information
18 technology is not sophisticated enough to deal with the
19 sort of analyses that we require and that is underlined
20 by the fact that of the 35 adult cardiac surgical units
21 in the country, 33 have bespoke databases for running
22 their own standalone systems that are not part of the,
23 if you like, corporate IT structure, and the remaining
24 two units are in the process of purchasing their own
25 systems.
0135
1 The reason for that is that in order for us to
2 fulfil the requirements and the expectations of our own
3 patients, of ourselves, of the media and also of the new
4 political agenda that is outlined in a first-class
5 service, we have to be able to look at our performance
6 with respect to case mix and risk stratification. That
7 has been clearly outlined by the Ministry of Health.
8 To do that, we cannot use the existing NHS IT
9 systems, we have to use something different. Units are
10 heading in that direction now and I would anticipate
11 that within 18 months, approaching 90 per cent of adult
12 cardiac surgical units in the country will be collecting
13 very accurate data.
14 Collecting data is not just an issue of entering
15 a few data points each time you do an operation; the
16 data has to be collated and this has implications for
17 personnel within the units. Most units now employ
18 professionals who collect and help enter the data and
19 chase up those very patients who might go missing that
20 we mentioned before the break so I believe that not only
21 is the quantity of data improving dramatically, but the
22 quality of data is improving out of all recognition,
23 really. I would imagine that we will be able to abolish
24 the cardiac surgical register within five years and have
25 it replaced by a national database of certainly adult
0136
1 cardiac surgical patients undergoing cardiac surgery and
2 that will be a very comprehensive database.
3 Q. Looking to the future for a moment, would you be looking
4 for that to be, as it were, a mandatory system financed
5 by the NHS or publicly in some way, as opposed to the
6 voluntary system which has depended for its success upon
7 the co-operation and goodwill of cardiac professionals?
8 A. I would. I would rather those questions had been in
9 a different order in that I think the key issue is the
10 funding. This costs money both at a local level and at
11 a central level, for the merging of the data and the
12 production of a report. It is perhaps not as expensive
13 as people imagine and our current report was done for
14 about œ40,000. That represents 70 per cent of units
15 within the United Kingdom.
16 I do not think the cost would go up enormously to
17 make that 100 per cent, but we do need funding and
18 I think some of the Trusts need to recognise their
19 responsibility to funding the sort of data collection,
20 given that the current NHS IT is not sophisticated
21 enough to do it, the current corporate systems.
22 With respect to making it mandatory, that is
23 a slightly more tricky issue because I think there is
24 still a pride in our specialty in the way that we are
25 trying to move forward, and I believe that most units
0137
1 would submit their data anyway.
2 The main advantage of having it mandatory is that
3 one could accelerate that process and it would be easier
4 to chase up data that came in late.
5 The converse side of making it mandatory is that
6 it raises all sorts of issues about who owns and who
7 polices the data. I would think in the first instance
8 it would be better to ensure that each unit and the
9 Society were adequately resourced to enable us to
10 collect data effectively and efficiently and accurately,
11 rather than adopting necessarily an approach which might
12 be seen as high-handed. Although, at the end of day,
13 I think mandatory collection is probably going to be the
14 answer, but it would be nice to avoid that if at all
15 possible.
16 Q. In exploring that, I was exploring something of the
17 future, which is a concern, certainly for this Inquiry,
18 particularly in part 2, or may be a particular concern
19 in part 2, which, as you know, looks at recommendations
20 for the future.
21 Returning to the past, the three questions I posed
22 this morning in respect of HES were the advantages, the
23 disadvantages and what use the Inquiry might make of the
24 database.
25 We have dealt, I think, unless you wish to comment
0138
1 further -- and please do -- with the advantages and
2 disadvantages that there may be in using the
3 Cardiothoracic Register, and we have more than touched
4 on the use that the Inquiry might make of it.
5 Dr Spiegelhalter, is there more you would wish to
6 say on the last of those three, or for that matter, the
7 first two, but particularly the last three?
8 DR SPIEGELHALTER: No, I do not think so. I have already
9 raised the issue that if an analysis goes ahead with all
10 the caveats and with all the concerns, I suppose I would
11 like to feel that it would be taken with some moderate
12 degree of seriousness by the people that submitted the
13 data.
14 MR LANGSTAFF: I think your answer was, "In general, yes,
15 although there are some who have their reserve"?
16 A. Yes, I think it would be taken very seriously.
17 Q. From what you have heard in the course of the
18 discussion, does it seem to you that the Inquiry are
19 entitled to make use of the findings of the
20 Cardiothoracic Register, bearing in mind its
21 shortcomings, but also its advantages, in analysing the
22 position between 1984 and 1995 so far as Bristol might
23 be seen to compare with the UK as a whole?
24 A. I certainly think that using it to see how Bristol has
25 compared with the UK as a whole is reasonable. There
0139
1 are still people within our specialty who would argue
2 that that data was given in a confidential fashion and
3 that that confidentiality should be respected. I think
4 the majority view is that this is a very important
5 Inquiry; its findings are likely to have a profound
6 influence on the future of our specialty and that those
7 influences will almost certainly be beneficial to all
8 concerned, to patients, to their families, and to
9 surgeons who work within the NHS.
10 So we recognise the importance of allowing use of
11 that data.
12 Q. If one is to take the data, can I ask you both the same
13 questions as I asked this morning in respect of HES: if
14 one were to find that the results as demonstrated by the
15 register were broadly comparable to that which one would
16 derive from an analysis of HES, providing one could get
17 them to speak the same language, as my description this
18 morning, that could only tend to strengthen both as
19 being reliable and useful data sources?
20 DR SPIEGELHALTER: I think given the two very different
21 routes by which the information has been gathered, that
22 if they corroborated, that would tend to reinforce them.
23 A. Yes, I think that sort of analysis would be comparable.
24 MR LANGSTAFF: But if they did not?
25 A. If they did not, the surgeons would believe the
0140
1 register, without any shadow of a doubt.
2 Q. You have sat here listening to the discussion on HES
3 this morning. In a couple of sentences: why?
4 A. It is difficult to compact it into a couple of
5 sentences --
6 Q. Well, a couple of paragraphs.
7 A. In essence, HES data is complicated by the fact that it
8 is primarily focused on finished consultant episodes and
9 even the logic required to bring those into patient
10 spells is not perfect. We know that the number of
11 operations or procedures recorded, certainly on HES data
12 which I have had anything to do with, indicate quite
13 a significant difference from what surgeons think they
14 have done. For example, I have looked at the coding of
15 my own work where there is a computerised operation note
16 which makes coding of the procedure really very simple,
17 and there is still about a 10 per cent error in terms of
18 activity.
19 MR LANGSTAFF: Sir, those are the questions which I have to
20 ask, unless either of you would wish to add anything?
21 A. I am sorry, I am referring now to adult cardiac
22 surgery. That difference, I think, has the potential of
23 being magnified enormously with paediatric cardiac
24 surgery, where the paediatric cardiac surgeons have
25 enough difficulty in agreeing on a diagnosis, let alone
0141
1 asking somebody in a coding room to code patients with
2 a hierarchy of different complex congenital disorders
3 and to get that right.
4 So I think that is why surgically collected data
5 is more likely to be trusted by the surgical community.
6 MR LANGSTAFF: Every witness who has come has been given the
7 opportunity of saying something, if they wish, at the
8 end, in case they had not been asked questions which
9 they would have wished to answer.
10 Is there anything you would wish to add yourself
11 to what you have already said today?
12 A. I would simply like to say, on behalf of the 200 or so
13 cardiac surgeons in this country, that we recognise the
14 pain and suffering that the families involved in the
15 Bristol Inquiry have been through, and whatever the
16 outcome of the Inquiry is, we hope that it is
17 satisfactory and helps to alleviate that pain.
18 MR LANGSTAFF: There may be some questions from the
19 Panel.
20 THE CHAIRMAN: Mr Keogh, I think Professor Jarman
21 has a couple of questions.
22 Examined by THE PANEL:
23 PROFESSOR JARMAN: One general, one specifically. Would you
24 have a record of patients with Down's syndrome?
25 A. No.
0142
1 Q. Generally, you pointed out that you are 20 years ahead
2 of any other specialty, even surgical specialties. How
3 have you managed to achieve that?
4 A. I think through the foresight of Sir Terence English.
5 That is the first thing.
6 The second thing is, we are a small specialty and
7 the number of operations which we perform, certainly in
8 the adult sector, is relatively compact, which makes it
9 a lot easier. It is more difficult in the paediatric
10 field, where diagnoses and procedures are more varied.
11 PROFESSOR JARMAN: Thank you.
12 THE CHAIRMAN: Mr Keogh, for my part, on behalf of the
13 Panel, we heard what you said at the end and I am sure
14 it will be appreciated elsewhere. Thank you for that.
15 I would like, if I may, to thank both of our
16 witnesses -- I confess, I may have been remiss in not
17 thanking our previous witness, Mr Willmer, who is no
18 longer with us. I apologise, because I would like to
19 thank both him and you, Mr Keogh, for coming to help us
20 today. We have been greatly assisted in what is clearly
21 a complex area.
22 I think our thanks are also on behalf of the Panel
23 due to our two experts, Dr Aylin who has had to leave
24 and Dr Spiegelhalter whom we will see again tomorrow.
25 They have greatly helped us in exploring the evidence.
0143
1 This is a vital task which you do for the Panel and if
2 I may say so, you have discharged it most diligently
3 today and we are grateful to you.
4 Mr Keogh, you may, if you wish, now stand down and
5 join the rest of the world, if I may put it like that.
6 Mr Langstaff, what do you have to tell me?
7 (The witness withdrew)
8 MR LANGSTAFF RE TIMETABLE:
9 MR LANGSTAFF: Sir, tomorrow we explore the delights of the
10 Patient Administration System -- that is the local
11 system. We do so in the same way very much as we have
12 done today and we will hear from Mr Andrew Hooper, the
13 Health Records Manager of the Bristol Royal Infirmary.
14 I and you will be assisted both by Dr Spiegelhalter and
15 by Anne Harding, who is our expert in the Patient
16 Administration Systems generally.
17 It needs to be said as well, and I will probably
18 come back to this tomorrow, that we have in addition
19 taken and we have, or will, put on the Internet a number
20 of statements in respect of individual data sources.
21 I mentioned, for instance, the surgeons' logs and we
22 have witness statements from both Mr Wisheart and
23 Mr Dhasmana in respect of the provenance of those
24 sources.
25 We do not propose to call them to give oral
0144
1 evidence because their evidence is quite plain on
2 paper. As I have said more than once from here, this
3 Inquiry is not an oral inquiry, it is an inquiry which
4 takes evidence both on paper and orally, and I have
5 said, more than once I think, that no conclusion should
6 be drawn as to the importance of the evidence or
7 unimportance of it because it is, as it were, relegated
8 to paper rather than being displayed orally.
9 We have also taken a statement from Mr John Gray,
10 the legal services manager of the Trust, and similarly,
11 that is a written statement giving evidence to us of the
12 sources of a number of the data upon which we will seek
13 to place some proper reliance, such as we can.
14 I am reminded by Mr Maclean, I should not ignore
15 Mr Hutter's statements, of which we have two, in respect
16 of this part. It should not be thought that I ignore
17 anyone in referring to the written data sources. I am
18 grateful to Mr Maclean for reminding me and apologise to
19 Mr Hutter for any unintended slight.
20 Tomorrow we begin at 10 o'clock, and not at 9.30
21 as we would normally. We go through to 1 o'clock and we
22 hope tomorrow, before we finish, to have the advantage
23 of hearing from Dr Spiegelhalter, who has sat throughout
24 these sessions, as to the way forward, anticipating,
25 having achieved we hope the preliminary critical
0145
1 overview that we anticipated for Stage 1, achieving
2 stages 2, 3 and 4 of the process which I put on the
3 screen at the start of today.
4 Thank you, sir.
5 THE CHAIRMAN: Mr Langstaff, thank you. I just
6 interject that the Panel is particularly pleased that
7 today's process, which is somewhat unusual in public
8 inquiries, seems to have worked to the advantage of
9 everybody in being able to follow and have hard and
10 complex material analysed for us en route, and we intend
11 to use our experts in this way as time goes on for the
12 benefit of all, as well as for the benefit of us in
13 particular.
14 So I repeat my thanks to them and to everyone else
15 for today. We adjourn now until 10. Thank you,
16 Mr Langstaff.
17 (3.20 pm)
18 (Adjourned until 10.00 am on Wednesday, 14th July, 1999)
19
20
21
22
23
24
25
0146
1 I N D E X
2
3
4 MR LANGSTAFF: INTRODUCTION RE STATISTICS ........... 1
5 DR DAVID SPIEGELHALTER (Affirmed)
6 Examined by MR LANGSTAFF ...................... 16
7 MR RICHARD WILLMER (Affirmed)
8 Examined by MR LANGSTAFF ...................... 32
9 MR BRUCE KEOGH (Sworn)
10 Examined by MR LANGSTAFF ...................... 93
11 Examined by THE PANEL ......................... 142
12 MR LANGSTAFF RE TIMETABLE .......................... 144
13
Return to top of page
Published by the Bristol Royal Infirmary Inquiry, July 2001
© Crown Copyright 2001