Introduction: Inappropriate attendances (IAs) to emergency departments (ED) create an unnecessary strain on healthcare systems. With decreased ED attendance during the COVID-19 pandemic, this study postulates that there are less IAs compared to before the pandemic and identifies factors associated with IAs.Methods: We performed a retrospective review of 29,267 patient presentations to a healthcare cluster in Singapore from 7 April 2020 to 1 June 2020, and 36,370 patients within a corresponding period in 2019. This time frame coincided with local COVID-19 lockdown measures. IAs were defined as patient presentations with no investigations required, with patients eventually discharged from the ED. IAs in the 2020 period during the pandemic were compared with 2019. Multivariable logistic regression was performed to identify factors associated with IAs. Results: There was a decrease in daily IAs in 2020 compared to 2019 (9.91±3.06 versus 24.96±5.92, P<0.001). IAs were more likely with self-referrals (adjusted odds ratio [aOR] 1.58, 95% confidence interval [CI] 1.50–1.66) and walk-ins (aOR 4.96, 95% CI 4.59–5.36), and those diagnosed with non-specific headache (aOR 2.08, 95% CI 1.85–2.34), or non-specific low back pain (aOR 1.28, 95% CI 1.15–1.42). IAs were less likely in 2020 compared to 2019 (aOR 0.67, 95% CI 0.65–0.71) and older patients (aOR 0.79 each 10 years, 95% CI 0.78–0.80). Conclusion: ED IAs decreased during COVID-19. The pandemic has provided a unique opportunity to examine factors associated with IAs.
Since the first case of coronavirus disease 2019 (COVID-19) was reported in China in late December 2019, the pandemic has spread throughout the world, with the World Health Organization (WHO) reporting over 206 million cases and over 4 million deaths globally as of 15 August 2021.1 To contain the spread of the disease, various measures such as lockdowns with movement restrictions,2 socioeconomic curbs with physical distancing, and prioritisation of healthcare resources were implemented in various countries.2-4
These measures have been suggested as key factors leading to a decrease in emergency department (ED) attendances for non-COVID-19 conditions.5-7 While there have been concerns of treatment delay and harm for patients with serious time-sensitive conditions such as acute coronary syndromes and cerebrovascular accidents,8,9 a positive externality of the pandemic in concomitantly redirecting minor conditions away from the ED has also been suggested.5,10
Such inappropriate attendances contribute significantly to over-crowding in EDs.11 Efforts to address overcrowding have focused on reducing inappropriate attendances by redirecting these patients towards alternative settings.12 However, redirecting patients away from the ED has raised ethical and safety concerns due to the difficulty in selecting suitable patients for redirection at their initial presentation, and the uncertainty in recommending alternate sources of care.13 Interventions to decrease inappropriate attendances have had unequal effects across different settings and environments.12,14
To date, there has been scant information on ED utilisation patterns regarding inappropriate attendances during the COVID-19 pandemic. The aim of this study was to compare and analyse the inappropriate attendances to the ED before and during the COVID-19 pandemic, and evaluate the visit, patient and disease factors associated with inappropriate attendances.
The methods of this study were reviewed and qualified for exemption by the National Healthcare Group Domain Specific Review Board.
We conducted a retrospective analysis of data extracted from the electronic medical records of patients aged 21 years and above who presented to 2 EDs and an Urgent Care Centre (UCC) of the National University Health System, a regional health cluster in Singapore. The UCC had been operational since June 2018 and provided 24-hour staffing by emergency physicians since 2019. It serves a hospital with a full complement of specialty inpatient and outpatient services except for interventional radiology and interventional cardiology. These services are largely unchanged across both study periods. The total annual census of the 3 departments is approximately 220,000. At the time of study, the regional health cluster did not have clinical nor operational definitions for inappropriate attendance. There were no protocols for diverting ED attendances to primary care.
Data were collected for patient presentations from 7 April 2020 to 1 June 2020. This period was chosen as it coincided with the timeframe of the “circuit breaker” in Singapore, during which an enhanced set of social distancing measures were instituted by the government in response to increasing local transmission of COVID-19.15 These measures included closure of non-essential services and workplace premises, closure of recreational venues and places of worship, home-based learning for students, and restriction of social contact to members living within the same household. The EDs and UCC remained fully operational and minimised compromise or disruption to usual patient-care.
Data for presentations to the EDs and UCC from 7 April 2019 to 1 June 2019 (the corresponding period in the preceding year) were collected for comparison. Anonymised data were extracted from the hospital records by administrative staff who had no participation in the study. Ethics approval was obtained from the relevant institutional review board for exemption (reference number 2020/00670).
This data included demographics such as age, sex and ethnicity; existing comorbidities if they were components of the Charlson Comorbidity Index;16 ED presentation details such as the triage patient acuity category (PAC) of 1 to 3 (PAC 1: critically ill patients requiring immediate attention, PAC 2: major illness requiring early attention, and PAC 3: ambulatory patients with minor conditions);17 usage of laboratory or radiological investigations; disposition; as well as diagnosis for the visit through Systematized Nomenclature of Medicine – Clinical Terms (SNOMED-CT), International Classification of Diseases (ICD), 9th Revision, Clinical Modification (ICD-9-CM) and ICD-10 codes. The ICD-9-CM and ICD-10 codes were retrieved from the WHO’s disease classification site,18 with complications from common ailments removed from the extraction process.
The primary outcome was daily incidence of inappropriate attendances. While there is no consensus on a standard definition,13 an ED visit is generally deemed inappropriate if the patient’s illness may be adequately managed in a primary care setting; does not require further diagnostic tests, procedures, or medications; and does not warrant an admission.19,20 We defined inappropriate attendances as presentations with no laboratory or radiological investigations performed, with the patient being discharged from the ED or UCC.13,21 This definition was chosen as such patients may be potentially managed by primary healthcare providers who have capabilities for outpatient treatment but do not have same-day laboratory or radiological results returned, or extended monitoring and treatment facilities.
Secondary outcomes included visit, patient and disease factors associated with inappropriate attendance. Multivariable logistic regression was conducted to explore these factors. Year of visit (2019 versus 2020), after-hours visits (defined as 5pm to the following day 8am regardless of which day of the week), patient demographics, past medical history and most commonly occurring primary diagnosis of the current presentation were entered into the regression model.
To account for government pandemic-response policies prescribing investigations for patients with any respiratory symptoms, which may have influenced health-seeking behaviour, we adjusted our analysis for the anticipated large numbers of attendances for upper respiratory tract infection and related diagnoses.
Statistical analyses were carried out using SPSS Statistics software version 26.0 (IBM Corp, Armonk, US). Descriptive statistics for numerical variables are presented as mean ± standard deviation (SD) when normality assumption was satisfied, otherwise the geometric mean for natural log-transformed variables is presented. Categorical variables are presented as incidence with percentage. Comparisons between categorical variables were carried out with chi-square test (dichotomous) or multivariable logistic regression (non-dichotomous: ethnicity). Continuous variables were compared using the Student’s t-test. Logistic regression with backward elimination was used to explore the variables associated with inappropriate attendances. The area under the receiver operating characteristic curve of the logistic regression model was also computed.
A total of 29,267 attendances (522.6±65.94 daily) were registered at the EDs and UCC from 7 April to 1 June 2020 compared to 36,370 (649.4±72.17 daily) during the corresponding period in 2019 (P<0.001). Despite the decrease in total ED attendances, the number of daily attendances for upper respiratory tract infections (URTI) (72.25±40.02 versus 34.30±10.43, P<0.001) and lower respiratory tract infections (28.16±8.55 versus 25.02±5.20, P=0.021) were higher. To adjust for attendances for minor symptoms influenced by pandemic-response policies, we excluded presentations with a diagnosis of URTI from subsequent analyses during the 2 periods (N=20,110 and 33,832, respectively). The daily ED attendance without URTI in 2020 (365.64±81.03) was significantly lower than in 2019 (615.13±70.75).
The demographic characteristics of these attendances are detailed in Table 1. During the circuit breaker from 7 April to 1 June 2020, patients’ Charlson Comorbidity Index was lower compared to the corresponding period in 2019 (1.60±2.07 versus 1.77±2.37, P<0.001), PAC 3 attendances per day decreased (187.46±41.84 versus 304.71±40.04, P<0.001), and there were fewer patients who walked-in and were self-referred (defined as a patient who did not seek primary healthcare attention prior to the ED visit) (12,654 versus 20,682, P<0.001). These attendance characteristics across both EDs and UCC for the 2019 and 2020 study periods are compared in Fig. 1, presenting the main differences of the UCC seeing a lower percentage of PAC 1 patients.
The number of daily inappropriate attendances was lower in 2020 than in 2019 (9.91±3.06 versus 24.96±5.92, P<0.001). More investigations were performed (82.62% versus 73.61%, P<0.001) and a larger proportion of patients were admitted or transferred (40.29% versus 35.09%, P<0.001) in 2020 as compared to 2019. We were only able to match the first visits and re-attended visits for the UCC and one of the EDs. Out of 8,346 first visits that were inappropriate attendances, 315 (3.8%) patients re-attended the ED or UCC within 72 hours (3.7% versus 3.9%, P=0.685). There were 356 ED re-attendances from those 315 patients re-attended ED in 72 hours. Of those 356 ED re-attendances, 159 (44.7%) required at least one investigation and 79 (22.2%) were admitted but both percentages were not different between years 2020 and 2019 (P=0.363 and P=0.443, respectively). None of them died in ED during the re-attendance.
Patient factors associated with inappropriate attendances in both periods are detailed in Table 2. Multivariable logistic regression analyses revealed that self-referred (adjusted odds ratio [aOR] 1.58, 95% confidence interval [CI] 1.50–1.66) and walk-in (aOR 4.96, 95% CI 4.59–5.36) presentations were associated with inappropriate attendance, whereas age (aOR 0.79 for each 10 years, 95% CI 0.78–0.80) was less likely to have inappropriate attendance. Past medical history of previous myocardial infarction, chronic heart failure, cerebrovascular accident, uncomplicated diabetes mellitus, chronic kidney disease, and malignancy with or without metastases were all negatively associated with inappropriate attendances. We identified the most frequently occurring diagnoses out of which non-specific headache (aOR 2.08, 95% CI 1.85–2.34), and non-specific low back pain (aOR 1.28, 95% CI 1.15–1.42) were associated with inappropriate attendances. There was a lower relative adjusted odds between the year 2020 and inappropriate attendances (aOR 0.67, 95% CI 0.65–0.71), suggesting that ED attendances in 2019 were more likely to be inappropriate compared to 2020. The area under curve of this logistic regression model was 0.769 (95% CI 0.765–0.773, P<0.001).
Subanalysis of UCC visits (n=5,937) was conducted. Walk-in (aOR 2.472, 95% CI 1.921–3.183) and diagnosis of headache (aOR 2.130, 95% CI 1.492–3.041) retained their positive association with inappropriate attendance. Past medical history of any malignancy was also associated with attendance without any investigations (aOR 7.692, 95% CI 3.012–19.641), which was due to the protocol of minimising investigations under the special direct admission to the palliative ward in Alexandra Hospital via UCC. Otherwise, the patient’s age (aOR 0.738 each 10 years, 95% CI 0.716–0.760), past medical history of chronic heart failure and uncomplicated diabetes mellitus, ED diagnoses of urinary tract infection and trauma were negatively associated with inappropriate attendance.
Subanalysis of visits from 7 April to 1 June 2019 was conducted. All variables identified to be associated with inappropriate attendance in both time periods remained.
Overall, our data showed a marked decrease in daily ED attendances in 2020 as compared to the same period in 2019, consistent with reports from many countries affected by COVID-19.5-7,22,23 The decrease especially in attendances that are triaged as PAC 3, self-referred or made by patients walking-in to the ED themselves suggest a change in health-seeking behaviour for minor ailments during the pandemic. However, its effect on inappropriate ED attendances has not been directly studied. Given that identifying inappropriate ED attendances and right-siting them to alternate care facilities has been controversial due to safety risks and differences in health-seeking behaviour and healthcare systems between communities,13,14,24 the COVID-19 pandemic provides a unique opportunity to examine this topic.
Our study is unique as we quantified inappropriate ED attendances, which we defined as ED attendances that did not require any investigations, admission or transfer. These attendances decreased by 40%, from 24.96±5.92 cases daily in 2019 to 9.91±3.06 cases daily in 2020. The decrease in inappropriate ED attendances is multifactorial. One postulated reason for this is the circuit breaker measures that discouraged unnecessary travel from home. The public may have also been concerned about getting infected by visiting the ED during the pandemic, and hence chose to wait, self-medicate, visit non-ED primary healthcare providers such as a general practitioner, or resort to telemedicine as an alternative. In addition, due to the widespread adoption of home-based work and study arrangements, there was also a reduced need for medical certificates to explain absences, which may have reduced the need for ED attendances for this purpose. Further studies on attendances to non-ED primary healthcare providers during COVID-19 may provide more insight into this hypothesis.
Examining factors associated with inappropriate attendances showed certain visit patterns or diseases that were more or less likely to require ED care. Patients who self-referred or walked-in were associated with inappropriate attendance, suggesting that the ED should not have been the first point of healthcare contact. Additionally, presentations for non-specific headaches or low back pains were associated with inappropriate attendance. This reflects a subset of diseases that may be treated in the non-ED primary healthcare setting but are still presenting to the ED.25 Education of the public remains crucial and can be made more specific by elaborating symptoms of headache or back pain, and a younger age range for consideration of attendance to primary healthcare providers instead, which can come in the form of circulars and posters put up at the ED.26 Healthcare fee structures can be directed as well, for example subsidising primary care visits for headaches or back pain through job benefits that tend to target younger individuals, and ED visits through retirement benefits for older patients.
These strategies to redirect this group of patients is further supported by the fact that none of these patients discharged from the ED required inpatient admission even if they re-attended the ED. Although other adverse outcomes were not studied, our study suggests that this identified group of patients suitable for primary healthcare remain so after discharge from the ED.
Conversely, patients who were older, or who have serious pre-existing conditions such as myocardial infarctions, heart failure, cerebrovascular accidents, connective tissue diseases, diabetes mellitus, chronic kidney disease and malignancies were shown to be negatively associated with inappropriate attendance. This reflects the increased complexity of care, need for investigation and admission for such patients. Time-sensitive critical illnesses like acute myocardial infarction and cerebrovascular accident were also unsurprisingly shown to be strongly associated with appropriate attendances. Public education to identify them and present immediately to the ED should be enhanced,27 especially during the pandemic given concerns of treatment delay.9,28
Our definition of inappropriate attendance was also limited in assessing certain conditions, such as psychiatric conditions, due to the nature of these presentations that often do not require laboratory or radiological investigations. The necessity of ED attendances for this group of patients would have been better reflected by the utilisation of psychiatry consults, admissions with a primary psychiatric diagnosis, and available alternatives in primary care, which were not captured in our study.
Unexpectedly, while pre-existing medical conditions were found to be associated with appropriate attendance, our patients’ average Charlson Comorbidity Index (CCI) had decreased from 2.59±2.51 in 2019 to 1.41±2.00 in 2020. The trend remained even after adjusting for URTI that we postulated would contribute to a lower mean CCI. Possible reasons are that patients with multiple comorbidities opted to seek medical attention in non- ED primary healthcare providers for fear of contracting COVID-19 in hospitals, given the association of COVID-19 disease severity with comorbidity.29,30 This raises concerns of delay or avoidance of necessary medical care in a group of patients whose care needs are more complex and would require ED attendance. Another postulation is the increased availability of help from family members with work-from-home arrangements, thus affording avoidance of inpatient care for these patients with multiple comorbidities.31 Once again, further studies are needed to evaluate the health-seeking behaviour and home resources of these patients to guide appropriate siting of their care to or away from the ED.
The strengths of our study include the large patient population, the multicentre nature across EDs and a UCC, and a unique study exposure brought about by COVID-19. Most research on inappropriate ED attendance examines exposures such as triage modifications, cost revisions, empowerment of primary care and education;12,32 while a far-reaching exposure that affects everyone like COVID-19 has not been studied.
Our study has several limitations. First, the definition of inappropriate ED attendance is controversial. A systematic review found no specific universal definition of inappropriate attendance.13 They span varying combinations of explicit criteria—such as mode of arrival to ED, presenting complaint, consultation in ED—and implicit criteria based on patient’s self-categorisation and expert opinion. We defined inappropriate ED attendance as attendances that do not require any investigation, admission or transfer, after in-depth consideration of the context of the local healthcare system. Of note, some studies considered receipt of treatment in the ED as grounds for appropriate attendance.13 We did not define appropriateness of ED attendance by receipt of ED treatment considering that many similar options for treatment are also provided in other non-ED primary health settings. This variability between different health systems thus limits the generalisability of our findings.
Second, the retrospective nature of our data is associated with its known issues such as missing information and misclassification biases. The use of multivariable logistic regression to identify factors associated with inappropriate attendance can only suggest a subset of patients to focus on for right-siting to an alternative primary healthcare provider. Also, the unavailability of triage history, and the use of SNOMED-CT, ICD-9-CM and ICD-10 codes in ED diagnoses that introduce heterogeneity to reason for attendance, limits the applicability of this study to predict inappropriate attendances at or before triage.
Third, although our study is multicentre, Singapore’s healthcare setting may be different from that of another country. Prospective experimental or quasi-experimental studies across different countries on reducing inappropriate attendances by redirecting this identified subgroup of patients away from the ED are needed, to validate our findings and investigate outcomes including morbidity and mortality from delayed presentations.
Fourth, the observational nature of this study is vulnerable to confounding by decreased disease incidence due to the pandemic. For example, ED inappropriate attendance could have decreased due to circuit breaker movement restrictions through a decrease in manual work and hence presentations for low back pain. However, the factors this study identified to be associated with inappropriate attendance are likely to occur independent of the presence of a pandemic. An interventional study design is needed to control for this confounder.
Lastly, our study is vulnerable to biasing patients with significant past medical history towards investigations in the ED, and hence being labelled as appropriate ED attendance regardless of other details within the ED attendance in question. Past medical history as an identified factor negatively associated with inappropriate attendance is thus potentially self-fulfilling and is difficult to isolate from the inherent bias of ED practitioners.
Our study demonstrated that the COVID-19 pandemic reduced inappropriate ED attendances within a healthcare cluster in Singapore. This observation reflects a change in patients’ health-seeking behaviour during the pandemic and may provide insight to right-site patients away from the ED. Public education to visit non-ED primary healthcare providers can be targeted at young patients with no serious pre-existing diseases who present for conditions like headache or low back pain.
We thank Dr Er Luen Lim and Ms Nurul Atiqah Mohd Taha for assisting in anonymised data retrieval.
- World Health Organization. Coronavirus Disease (COVID-19) Situation Reports. Available at: https://www.who.int/emergencies/ diseases/novel-coronavirus-2019/situation-reports. Accessed on 17 August 2021.
- Bruinen de Bruin Y, Lequarre AS, McCourt J, et al. Initial impacts of global risk mitigation measures taken during the combatting of the COVID-19 pandemic. Saf Sci 2020;128:104773.
- Oh J, Lee JK, Schwarz D, et al. National Response to COVID-19 in the Republic of Korea and Lessons Learned for Other Countries. Heal Syst Reform 2020;6:e1753464.
- Chen J, Yap J, Hsu L, et al. COVID-19 and Singapore: From Early Response to Circuit Breaker. Ann Acad Med Singap 2020;49:561-72.
- Hartnett KP, Kite-Powell A, DeVies J, et al. Impact of the COVID-19 Pandemic on Emergency Department Visits — United States, January 1, 2019–May 30, 2020. MMWR Morb Mortal Wkly Rep 2020;69:699-704.
- Barten DG, Latten GHP, van Osch FHM. Reduced Emergency Department Utilization During the Early Phase of the COVID-19 Pandemic: Viral Fear or Lockdown Effect? Disaster Med Public Health Prep 2020:1-4.
- Jeffery MM, D’Onofrio G, Paek H, et al. Trends in Emergency Department Visits and Hospital Admissions in Health Care Systems in 5 States in the First Months of the COVID-19 Pandemic in the US. JAMA Intern Med 2020;06519:1328-33.
- Metzler B, Siostrzonek P, Binder RK, et al. Decline of acute coronary syndrome admissions in Austria since the outbreak of COVID-19: the pandemic response causes cardiac collateral damage. Eur Heart J 2020;41:1852-3.
- Zhao J, Li H, Kung D, et al. Impact of the COVID-19 Epidemic on Stroke Care and Potential Solutions. Stroke 2020;51:1996-2001.
- Vollmer MAC, Radhakrishnan S, Kont MD, et al. The impact of the COVID-19 pandemic on patterns of attendance at emergency departments in two large London hospitals: an observational study BMC Health Serv Res 2021;21:1008.
- McHale P, Wood S, Hughes K, et al. Who uses emergency departments inappropriately and when – a national cross-sectional study using a monitoring data system. BMC Med 2013;11:258.
- Van den Heede K, Van de Voorde C. Interventions to reduce emergency department utilisation: A review of reviews. Health Policy 2016;120:1337-49.
- Durand AC, Gentile S, Devictor B, et al. ED patients: How nonurgent are they? Systematic review of the emergency medicine literature. Am J Emerg Med 2011;29:333-45.
- Lowe RA, Bindman AB, Ulrich SK, et al. Refusing care to emergency department of patients: evaluation of published triage guidelines. Ann Emerg Med 1994;23:286-93.
- Ministry of Health Singapore. Circuit breaker to minimise further spread of COVID-19, 3 April 2020. Available at: https://www.moh. gov.sg/news-highlights/details/circuit-breaker-to-minimise-further-spread-of-covid-19. Accessed on 1 December 2020.
- Quan H, Li B, Couris CM, et al. Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol 2011;173:676-82.
- Sun Y, Heng BH, Tay SY, et al. Predicting hospital admissions at emergency department triage using routine administrative data. Acad Emerg Med 2011;18:844-50.
- World Health Organization. ICD-10 Version: 2019. Available at: https://icd.who.int/browse10/2019/en. Accessed on 22 October 2020.
- Bezzina AJ, Smith PB, Cromwell D, et al. Primary care patients in the emergency department: who are they? A review of the definition of the “primary care patient” in the emergency department. Emerg Med Australas 2005;17:472-9.
- Oh HC, Chow WL, Gao Y, et al. Factors associated with inappropriate attendances at the emergency department of a tertiary hospital in Singapore. Singapore Med J 2020;61:75-80.
- David M, Schwartau I, Anand Pant H, et al. Emergency outpatient services in the city of Berlin: Factors for appropriate use and predictors for hospital admission. Eur J Emerg Med Off J Eur Soc Emerg Med 2006;13:352-7.
- Tan R, Ganapathy S, Tyebally A, et al. Paediatric emergency department attendances during COVID-19 and SARS in Singapore. Ann Acad Med Singap 2021;50:126-34.
- Ang H, Omar E, Pek JH. Decrease in emergency department attendances during COVID-19 especially in school-going children. Ann Acad Med Singap 2021;50:184-7.
- Driscoll PA, Vincent CA, Wilkinson M. The use of the accident and emergency department. Arch Emerg Med 1987;4:77-82.
- Dale J, Green J, Reid F, et al. Primary care in the accident and emergency department: I. Prospective identification of patients. BMJ 1995;311:423-6.
- Ministry of Health Singapore. The Experience of the GPFirst Programme, 14 October 2020. Available at: https://www.moh.gov. sg/news-highlights/details/the-experience-of-the-gpfirst-programme. Accessed on 2 January 2021.
- Singapore Civil Defence Force. Civil Defence Emergency Handbook: 8th edition, 2016. Availabe at: https://www.scdf.gov.sg/ docs/default-source/scdf-library/publications/publications/5372-scdf-emergency-handbook-(eng).pdf. Accessed on 12 October 2020.
- De Rosa S, Spaccarotella C, Basso C, et al. Reduction of hospitalizations for myocardial infarction in Italy in the COVID-19 era. Eur Heart J 2020;41:2083-8.
- Czeisler MÉ, Marynak K, Clarke KEN, et al. Delay or Avoidance of Medical Care Because of COVID-19–Related Concerns — United States, June 2020. MMWR Morb Mortal Wkly Rep 2020; 69:1250-7.
- Centers for Disease Control and Prevention. Coronavirus Disease 2019 (COVID-19): Who Is at Increased Risk for Severe Illness? – People of Any Age with Underlying Medical Conditions. Available at: http://www.cdc.gov/coronavirus/2019-ncov/need-extra-precautions/people-with-medical-conditions.html. Accessed on 30 November 2020.
- Government of Singapore. COVID-19 circuit breaker: Closure of workplace premises, 3 April 2020. Available at: https://www.gov. sg/article/covid-19-circuit-breaker-closure-of-workplace-premises. Accessed on 8 June 2021.
- Raven MC, Kushel M, Ko MJ, et al. The Effectiveness of Emergency Department Visit Reduction Programs: A Systematic Review. Ann Emerg Med 2016;68:467-83.e15.