Medical education is associated with significant costs
[1]. These costs have led to a growing interest in how to deliver high quality or high quantity education on a limited budget. This in turn has led to an interest in how best to measure quantity, quality and cost and how to track these variables over time. The ultimate aim of the interest in cost and value in medical education is to improve quality or quantity over time and to reduce costs. This may occur by a number of methods – which might include e-learning or simple simulations
[2–
3]. In order for improvements in cost and value to occur, there needs to be a usable and easily understood methodology to demonstrate improvement. There are good reasons to believe that statistical process control is one such methodology.
Statistical process control is a discipline within statistics that enables analysis of data over time and graphical representation of that data
[4]. Statistical process control is a methodology that can be used with minimal statistical training and that can be understood by a range of different types of health and education professionals. Measurements of data can change over time and can change for a variety of reasons. These changes might be due to inherent variation or to defects in the technique of measurement or to deliberate attempts to improve the data or accidental disimprovements in the data. For example, changes in student examination marks from year to year might be due to more intelligent students (inherent variation) or problems with the exam (defects in the technique of measurement) or an investment in curriculum delivery that resulted in better educated students (a deliberate attempt to improve the data). There are a variety of means to demonstrate change – the advantage of statistical process control is that it enables the detection of change in real time – while there is still time to do something about it – and it can be easily understood by non-experts. At the same time, statistical process control is a rigorous methodology – it is not a means of getting a "rough" view of data.
In statistical process control, changes over time can be graphically represented by a statistical control chart or a control chart
[1]. According to statistical process control nomenclature, natural changes in data over time are known as common cause variation. Depending on the nature of the data, natural changes over time may follow a normal distribution or perhaps another form of distribution (for example a binomial distribution). If, however, data changes as result of something going wrong or as a result of an intervention to cause improvement, this is known as special cause variation. The control chart enables the measurement of data over time and also enables all stakeholders to see whether the data are within the upper control limit and/or the lower control limit. If the data are within these limits, then it is assumed that changes in data are a result of common cause variation. If, however, the measurements fall above or below the upper or the lower control limit, then it is assumed that changes in data are a result of special cause variation. Most experts recommend that the upper and lower control limits should be placed at ±3 standard deviations from the mean. Because 99.73% of the values lie within three standard deviations of the mean, there is only a 0.27% chance that a single measurement will lie outside of the upper and lower control limits. However, control charts consist of multiple measurements and so there is a higher chance in the typical control chart that a measurement will fall outside of these two limits. The method used to calculate the standard deviation depends on the distribution of the data being measured.
Measurements outside the control limits (set at 3 standard deviations) are thus likely to be a result of special cause variation. However, these are not the only sign of special cause variation. Others include two of three consecutive measurements more than 2 standard deviations from the mean; four of five consecutive measurements more than one standard deviation from the mean; eight consecutive measurements above or below the mean; six consecutive measurements that are increasing or decreasing
[6]. Control charts are essentially time series and enable the early detection of trends in data. So it is best to look at them continually and to share them widely with the team on a continual basis. It is worth considering creating a large poster in the medical education institution so that everyone will be able to see what progress is being made.
In the following example, a hospital has an incidence of pressure ulcers that is above the national average. The management at the hospital institute a multimodal educational and quality improvement initiative to reduce the incidence of pressure ulcers. First of all, a series of lectures and e-learning resources are delivered to physicians, nurses and interprofessional team members at the hospital
[7]. This is expensive- but it makes little difference to the incidence of pressure ulcers. Next, a series of interdisciplinary education workshops involving physicians, nurses and allied healthcare professionals are set up. These are even more expensive: however, they do have an effect. The fall in incidence is a planned special cause variation. Finally, a new checklist is introduced to ensure that all patients on all wards have a thorough risk assessment on admission. The new checklist further reduces the incidence. It turns out to be the lowest cost method used. The control chart is shown in
Fig. 1.
Statistic process control is one method of looking at data over time. It can be used to evaluate the effectiveness of interventions. When these interventions are costed, then conclusions can be drawn on the cost and value of the interventions. There is no single best way to analyze data – the tools chosen will depend of the particular context. For example cost-effective analysis is a means of comparing the costs and effectiveness of two or more interventions. In contrast statistical process control is more of a real world method and can be used to portray the effects of single interventions over time.
PDF
(86KB)
