ABSTRACT
Health outcomes evaluation seeks to compare a new treatment or novel programme with the current standard of care, or to identify variation of outcomes across different healthcare providers. In the real world, it is not always possible to conduct randomised controlled trials to address the issue of comparator groups being different with respect to baseline risk factors for the outcomes. Therefore, risk adjustment is required to address patient factors that may lead to biases in estimates of treatment effects. It is essential when conducting outcomes evaluation of more than trivial significance. Risk adjustment begins by asking 4 questions: what outcome, what time frame, what population, and what purpose. Next, design issues are considered. This involves choosing the data source, planning data collection, defining the sample required, and selecting the variables carefully. Finally, analytical issues are considered. Regression modelling is central to every analytic strategy. Other methods that may augment regression include restriction, stratification, propensity scores, instrumental variables, and difference-in-differences. The construction of risk adjustment models is an iterative process requiring both art and science. Derived models should be validated. Limitations of risk adjustment include reliance on data availability and quality, imperfect method, ineffectiveness when comparators are very different, and sensitivity to different methods used. Thoughtful application of risk adjustment can improve the validity of comparisons between different treatments, programmes and providers. The extent of risk adjustment should be guided by its purpose. Finally, its methodology should be made explicit, so that informed readers can judge the robustness of results obtained.
Consider these hypothetical situations. Patients with heart failure managed within a disease management programme are found to have reduced hospitalisations and improved quality of life scores, compared with patients managed outside this programme. Can we conclude that this programme is an effective intervention, and should therefore be expanded to include all patients with heart failure? The length of stay for adults admitted for pneumonia is shorter (5 days) in hospital A, than in hospital B (7 days). Can we confidently say that hospital A is more efficient than hospital B, for the management of pneumonia? With increased availability of outcomes data, questions like these confront physicians, hospitals and the healthcare system in their quest to understand and improve quality of care. The astute are aware that such conclusions may be overly simplistic and even misleading, unless appropriate measures were taken to ensure that treatment groups were similar with respect to patient factors that could influence outcomes. Without doing so, we cannot be confident that the results are valid.
This article is available only as a PDF. Please click on “Download PDF” to view the full article.
