Decision-Support Systems designed for Critical Care

M. Frize, H.C.E. Trigg, F.G. Solven, M. Stevenson. B.G. Nickerson

University of New Brunswick, Fredericton, N.B. , Canada E3B 5A3

ABSTRACT

A case-based reasoner tool has been developed, allowing users to compare the ten-closest matching cases to the newest patient admission, from a database of intensive care medical records. A back-propagation, feed-forward artificial neural network has been trained and tested to estimate patient outcomes: duration of artificial ventilation, the length of stay , and mortality.

INTRODUCTION

Several authors have developed, and others have described, various scoring systems to assess patient outcomes in critical care medicine. Despite the ongoing research on the prediction of mortality, much less has been done for estimating length of stay, and duration of artificial ventilation. Hyzg ¹ and LeGall et al.² are of the opinion that many clinicians remain sceptical about using scoring models in their actual patient care. Castella ³ and Fery-Lemonier ⁴, for their part, have provided a comparison between various scoring systems and this is helpful to guide efforts in this area of research.

Recently, case-based reasoners (CBR) and knowledge-based systems have been increasingly accepted as clinical decision-support systems. Johnston et al ⁵, in a broad review of the literature found that, of 793 citations in the area of clinical decision-support systems (CDSS), only 28 controlled trials met the predefined criteria for proper study design and assessment. They also concluded that these systems can improve clinician performance and patient outcomes in clinical settings such as: computer-assisted dosing, preventive care reminders, and computer-aided quality assurance for active medical care. These authors also pointed out the need for additional well-designed studies to assess the effects and the cost-effectiveness of decision-support systems, especially for those attempting to affect patient outcomes.

Because of their non-linear modelling capabilities, artificial neural networks (ANNs) have been extensively applied to non-linear statistical modelling problems and can be used to perform very complex recognition tasks. As a result, they are a natural choice for modelling complex medical problems when large databases of relevant medical information are available. Many have been applied to a particular pathology, such as Baxt ^{6, 7} who used ANNs as an aid to diagnose acute coronary occlusion and later for myocardial infarction; and Kuntz ⁸, to estimate mortality and length of stay (LOS) for patients with closed-head injuries. Tu and Guerriere ⁹ reported estimates of LOS and mortality. Buskard et al. ¹⁰ added studies of estimated duration of artificial ventilation to those of mortality and LOS. In view of the limitations of severity of disease classification systems, much could be gained in investigating an integrated, rather than compartmentalised, approach to critical care (and other medical environments). Most scoring systems are more useful in estimating outcomes for a group of patients, than for a single one. Thus, new approaches should attempt to make estimates on a patient by patient basis. This has been a main focus of the work reported here.

DESIGN OBJECTIVES

A multi-disciplinary group (University of New Brunswick) brings together electrical engineers, computer scientists, graduate students, and physicians from several health care facilities across Canada. The focus of the work is to use artificial intelligence (AI) to integrate a number of promising decision-aid systems (http://www.unb.ca/web/mirg/). Since its inception, the group's approach has been to view AI as a set of tools to simulate (and support) common clinical thinking, rather than replace it. As a result, each of the techniques reported here were chosen because of a particular utility in modelling a particular aspect of the clinical approach. For example, a clinician's approach may be to think: "I have seen patients like that and this is what happened to them". The corresponding AI approach is:"The case-based reasoner selects a number of "closest-matching cases" and displays selected aspects of their clinical course". Or, a clinician's approach may be to think: "And for this particular patient, this is what I think will happen". The AI approach: "A trained artificial neural network provides an estimate of selected clinical outcomes".

SYSTEM DESCRIPTION

a) The Case Based Reasoner (CBR) A medical database of over 2000 intensive care patients, containing 98 fields of clinical and administrative information on patients admitted to the ICU at the Dr. E.Chalmers Hospital (DECH, since 1988) was transferred from its DBASE format to a case-based reasoner prototype. The case-based reasoner uses a modified Rete match algorithm to develop a match score based on string, word, character, or number matching (ART-IM software). The Rete match algorithm is faster than conventional matching algorithms. Each of the matching methods and the weights associated with them were selected in direct consultation with physicians to determine the preset (default) values. In addition to this default mode, the system allows the physician to fine-tune any of the matching field weights to a patient's specific characteristics. This flexibility was felt to be important, since the same procedures performed on individual patients may have varying degrees of significance. As more information is known on the index patient case and entered into the program, the matched cases also change. When a patient is discharged, the file is automatically added to the database, allowing the accuracy and quality of the data to improve with time and use of the system.

The IDEAS for ICUs (Intelligent Decision Aid System for Intensive Care Units) system grew out of this early model by incorporating a graphical-user interface. The presented (new) patient admission information is entered into the CBR, along with any changes of match weights (if desired), and the system can then generate a screen of ten closest-matching cases to the newest admission case. It is the blending of the rule-based ability along with the knowledge of medical experts on the team, that provided the foundation for the development of this decision-aid system. IDEAS for ICUs issues warnings, especially in cases where a high match score is made with a patient who died.^{11, 12.}

The original DBASE IV patient database had been previously used to provide statistics on the ICU performance. Unfortunately, it is limited to finding exact matches through its 'Structured Query Language' (SQL), which is far from being as useful as 'near' matches provided by CBR. In addition to this additional power in the matching capability, the system can be expanded by using the expert shell's rule-based abilities. The Graphical user interface (GUI) was written in VISUAL BASIC (Version 3.0), allowing for a natural integration of the patient database and the case-base, thus providing combined storage and information retrieval. The GUI presents a choice of several windows to enter or to display the various types of information. The system is user-friendly and allows to flow from window to window and back to any section desired. The various parts interact with each other, as well as with other text and binary files, to form a dynamic software tool.

b) The Artificial Neural Network (ANN) To estimate medical outcomes for individual patients in the ICU, a feedforward ANN was trained using the backpropagation algorithm, because of its relative ease of implementation and past success on various classification tasks. In the initial experiments, networks which estimated 'length of stay' (LOS), 'mortality', and 'duration of artificial ventilation' were trained and tested on a subset of the DECH ICU database (1322 patients). Two-thirds of these patients records were used for training the network, while the remaining third were used to test the ANN's performance. After experimenting with various network architectures, an ANN was considered as acceptable if it provided better results than a Constant Predictor (CP), which is a simple statistical benchmark that classifies all patterns as belonging to the class with the highest training set a priori probability. Although the initial results were promising, the networks exhibited a behaviour characteristic of memorisation (over-fitting), that is, after a few hundred epochs, the classification rate and error curves for the training and test sets began to diverge. Since the generalisation ability of an ANN depends upon a balance between network complexity and the information contained in the training examples¹³, the over-fitting was most likely caused by the fact that the number of parameters (there were 41) required too many weights for the relatively small number of example patterns (868). Rather than arbitrarily eliminating potentially useful information, to reduce the network size (and complexity), or waiting until a significant amount of additional data were collected, it was decided that implementing weight-elimination ¹⁴ might be a better approach.

A new set of experiments were performed on a slightly larger database (1491 patients records), using 51 input variables, to estimate duration of artificial ventilation. Among the 51 inputs provided to the network were several new variable fields describing admission diagnosis and patient admission source. Weight-elimination was turned on and turned off, and the results were compared. ¹⁵

RESULTS

a) The case-based reasoner The IDEAS for ICUs prototype was placed for a short period in a clinical setting (three weeks). Several constructive comments were collected and this led to a number of revisions of the prototype's software. The new version (2.2) includes many new features and a faster matching engine is currently being incorporated. The new software has been customised for two hospitals where a more substantial clinical evaluation will be conducted this year. The clinical setting will allow to assess the system's usefulness and performance. Some of the qualitative points to be studied are: Is the system helpful as an instant 'memory' of past similar cases? Does the information change the diagnosis that would have been made without the tool? Does the information help the physician to explain the prognosis to the family? To the nurses? Does the information help in the choice of treatment and management of the patient? This clinical assessment is essential before claims can be made on the system's relevance and usefulness. The CBR has generated substantial interest in the medical community in Canada and other sites are currently thinking of participating in the study .

b) The Artificial Neural Network The results of our most recent experiments with the ANN showed that the weight-elimination technique improves both the generalization and the overall performance of a fully-optimized network trained to estimate the outcome 'postoperative VENT8 ^15. VENT8 is a binary output variable which was assigned a value of -1 when a patient's actual duration of ventilation was less than, or equal to 8 hours, and a value of 1 when a patient's actual duration of artificial ventilation was greater than 8 hours. When weight-elimination was used, the weights associated with parameters that have little significance in determining the network's output were driven to zero, simplifying the network's structure and resulting in improved network generalization and performance. Thus weight-elimination provided the network with its own means of screening out unimportant variables and eliminates the need for making preconceived judgements as to what medical parameters are most instrumental in determining a particular patient outcome.

By combining weight-elimination with a second technique, that is, to present 'high' and 'low' values of continuous and integer-valued medical parameters to a pair of input nodes, rather than presenting these just to a single node, a single-layered ANN was constructed from which interesting information could be extracted: i.e. the variables which the ANN considered to be most important in estimating postoperative VENT8 could be extracted ¹⁵. The 'high/low' nodes data presentation technique facilitates the independent interpretation of high and low values of each input parameter by the ANN model, and hence the weights selected by such a network should be more representative of the true significance of each input parameter in determining an outcome such as VENT8, than those identified by a network which does not employ 'high/low' nodes techniques.

For this experiment, the five largest weights extracted at the point of maximum test set classification performance were obtained for the following input variables: lower-than-normal respiratory rate, higher-than-normal arterial pH and higher-than-normal fraction of inspired oxygen all favoured a patient having a duration of ventilation longer than 8 hours; whereas, a higher-than-normal of Glasgow Coma Score, and having had a carotid endarectomy, favoured a patient having a duration of ventilation less than or equal to 8 hours ¹⁵.

DISCUSSION AND FUTURE WORK

Building on the success of the initial phases of this work, the following research aspects are in progress: Generalising the case-based reasoner (IDEAS for ICUs) approach for use in other medical environment: This is currently being applied to neonatal ICU and rheumatoid arthritis populations. An epidemiologist (whose expertise is in the ICU environment and clinical trials) is helping to develop the clinical evaluation plan and questionnaire.

For the ANN work, the current experience will be enlarged in several ways: the experiments will be repeated using more output classes (possibly three or four) for the output "duration of ventilation" with the adult ICU database. The next step will be to extend this entire approach to study outcomes using a neonatal database to see if the network performs equally well with this different type of data.

An interface for on-line data acquisition from monitors and ventilators is under development ¹⁶. The interface was designed to communicate information aquired from the patient bedside, over modem facilities, stored and eventually to be used by both the case-based reasoner and the artificial neural network. The data acquisition itself, although not a major technical problem, creates a much larger challenge, that is, of modifying the case-based reasoner and neural network to accommodate and use effectively the temporal information. Addressing this challenge will be part of the future work planned by the team.

CONCLUSION

The results of the group's research efforts to develop a decision-support system in critical care provide an enhancement of the conventional medical model used by physicians for patient management in an Intensive Care environment. To date, the research done by our group compares previous patient experience held in a large clinical database, with new patient admissions, and generates outputs which maybe used to aid physician decision making. Our system positions itself between the medical model and usual testing systems and represents a new class of "intelligent monitoring instrumentation" for critical care. In the future, this approach may be applicable to other patient care environments, such as neonatal care, cardiac units, neuro-intensive care and the operating room.

The authors would like to acknowledge the support of the Medical Research Council of Canada and NSERC for its PGS-A Scholarship.

REFERENCES

1. Hyzg RC, ICU scoring and clinical decision making. [Edit.] Chest. 1995; 107(2): 1482-1483.

2. LeGall J et .al. Customized probability models for early severe sepsis in adult intensive care patients. JAMA. 1995; 273(8): 644-650.

3. Castella X et .al. A comparison of severity of illness scoring systems for intensive care unit patients: results of a multicenter, multinational study. Crit. Care Med. 1995; 23(8): 1327-1335.

4. Fery-Lemonnier E et .al. Evaluation of severity scoring systems in ICUs - translation, conversion and definition ambiguities as a source of inter-observer variability in APACHE II, SAPS and OSF. Int. Care Med. 1995; 21: 356-360.

5. Johnston ME., Langton KB, Haynes RB, Mathieu, A. Effects of computer-based clinical decision-support systems on clinician performance and patient outcome. Annals of Internal Medicine. 1994; 120: 135-142.

6. Baxt WG. Use of an Artificial Neural Network for Data Analysis in Clinical Decision-Making! The Diagnosis of Acute Coronary Occlusion. Neural Computation. 1990; 2: 480-489.

7. Baxt WG. Use of an Artificial Neural Network for the Diagnosis of Myocardial Infarction. Annals of Internal Medicine. 1991; 115: 843-848.

8. Kuntz GR, Hammond J. Predicting Outcomes after Closed-Head Injury (CHI) using a Cascade- Correlation Artificial Neural Network. Crit. Care Med. 1994; 22(1): A25.

9. Tu JV. and Guerriere MRJ. Use of a Neural Network as a Predictive Instrument for Length of Stay in the Intensive Care Unit Following Cardiac Surgery.' Computers and Biomed. Research. 1993; 26: 220-229.

10. Buskard T, Stevenson M, Frize M, Solven FG.. Estimation of Ventilation, Length of Stay, and Mortality Using Artificial Neural Networks. Proc. of Canadian Conference on Electrical and Comp. Eng. (IEEE). 1994, Halifax: 726-729.

11. Taylor KB, Nickerson BG, Frize M, Solven FG, Dunfield V. Use of Case-Based Reasoning to Assis Patient Management in an Intensive Care Unit. Proc. of Joint Conf. (COMP and CMBES), Ottawa. 1993:.248-249.

12. Frize M, Solven FG, Stevenson M, Nickerson BG, Buskard T, Taylor K. Computer-Assisted Decision- Support Systems for Patient Management in an Intensive Care Unit. Proc. of Medinfo95, Vancouver. 1995: 1009-1012.

13. Krogh A, Hertz JA. A Simple Weight Decay can Improve Generalisation. In RP. Lippman, J Moody, and DS Touretzky, ed. Advances in Neural Information Processing Systems. Morgan Kaufman: San Mateo. 1992; 4: 950-957.

14. Weigend AS, Rumelhart DE, Huberman BA. Generalisation by Weight-Elimination with Application to Forecasting. In Lippman, RP, Neural Network Classifiers Estimate Bayesian a posteriori Probabilities. Neural Computation. 1991; 3(4): 461-483.

15. Trigg HCE. An Investigation of Methods to Enhance the Performance of Artificial Neural Networks Used to estimate ICU Outcomes. MScE Thesis, U. of New Brunswick, January 1997.

16. Barton M, Frize M. Patient Data Acquisition and Telemedicine Access to a Decision-Support System. Medicine 2001 Conf. Montreal. 1996:3.