The novel 2019 coronavirus (COVID-19) pandemic disrupted medical services in numerous ways, including placing unprecedented strain on healthcare resources. Publicly reported hospital mortality rates increased overall,(1) in part due to the deadliness of early strains of COVID-19 in a naïve population, lack of clinical experience or guidelines for treating a novel illness, and potentially because of strained system capacity during pandemic peaks.
Mortality outcomes from intensive care units are traditionally risk-adjusted by considering age, admitting diagnosis, co-morbidity burden, and acute physiology at presentation to adjust for case-mix differences. Several risk-adjustment tools are available, with APACHE (Acute Physiology and Chronic Health Evaluation) being widely used in adult intensive care units.(2)
Benchmarking tools are developed using large (generally more than 100,00 patients) nationally representative databases to assess the relationship between outcomes (mortality, morbidity, length of stay) and presenting condition of the patient (age, diagnosis, co-morbidities, and degree of physiologic stress).
Critical to this process is assigning a diagnosis-specific coefficient when estimating mortality.(2) For example, patients with diabetic ketoacidosis (DKA) typically have massive metabolic and physiologic derangement at presentation, which should generate a high mortality risk based on unstable physiology. This altered physiology must be interpreted in light of the admitting diagnosis, however, and patients with DKA ordinarily have a high survival rate when properly managed. In contrast, patients with heart failure may demonstrate less physiologic derangement on presentation, but carry a much worse prognosis, given their diminished physiologic reserve.
The prognostic impact of altered physiology, age, and co-morbidities must thus be considered specifically for an admitting diagnosis. The coefficient to adjust expected mortality by admitting diagnosis is determined during model development based on previously collected data — generally thousands of patients over several years.
Unfortunately, the proper coefficient for such predictions cannot be known early in an evolving pandemic, so one approach for benchmarking outcomes would be to choose the closest match for diagnosis. For COVID-19, that would logically be the APACHE diagnostic category “viral pneumonia.”
A study of 43 hospitals contributing data to APACHE between 2014 and 2020 demonstrated similar severity of illness on presentation with COVID or viral pneumonia, but one and a half times the mortality, longer hospital and ICU length of stay, and more days on the ventilator in risk-adjusted COVID patients versus those with non-COVID viral pneumonia.(3)
In this journal, we previously reported the potential impact of COVID-19 on benchmarking ICU and hospital performance during the pandemic.(4) Even after controlling for severity of illness on presentation using APACHE IVb methodology, risk-adjusted mortality, length of stay, and ventilator days were significantly higher than expected in the early COVID-19 population than what would be predicted for non-COVID viral pneumonia.
Additionally, even non-COVID, non-pneumonia patients demonstrated higher standardized mortality rates during each pandemic peak between March 2020 and December 2021. Benchmarking ICU and hospital outcomes during the early pandemic was thus problematic.
We now extend our analysis through calendar year 2022, a time when the predominant circulating strain of COVID shifted from Delta to Omicron, and observe that APACHE-IVb clinical outcomes for COVID-19 patients are returning closer to those predicted for non-COVID viral pneumonia patients.
METHODS
We analyzed anonymized data from 31 hospitals providing APACHE IVb data continuously between Jan. 1, 2016, and Dec. 31, 2022. IRB approval was previously deemed unnecessary for patient protection because data could not be tied to individual patients.
We used four years of pre-pandemic data to establish a baseline risk-adjusted mortality rate for viral pneumonia, along with length of stay (LOS) and mechanical ventilation rates. Analysis was limited to complete records for non-surgical, non-trauma patients age 45 or older to reduce the likelihood of including patients where COVID was an incidental diagnosis.
Assessed outcomes included death within the ICU or during the hospital stay, ICU and hospital length of stay, the percentage of patients receiving active intensive therapy (as opposed to simply ICU-level monitoring), and days on mechanical ventilation.
Predicted and observed rates of each outcome were calculated for the patients and then reported in aggregate standardized rates, including standardized mortality ratios (SMRs) and associated LOS, active therapy, and ventilation day ratios. Data were tabulated in Excel, using chi-square or t-tests as appropriate with p < 0.05 considered significant.
The APACHE methodology is based on logistic and linear regression of important outcomes (mortality, length of stay) against putative risk factors.(5) In clinical practice, patient characteristics, collected locally, are used to predict outcomes, which are then compared to actual local outcomes in aggregate form. Standardized outcomes, such as SMR, are calculated by dividing observed aggregate outcome by predicted aggregate outcome, yielding a ratio where 1.0 indicates “as expected,” with higher and lower values indicating outcomes worse or better performance.
RESULTS
Thirty-one hospitals (three large teaching, 13 small teaching, 15 community) provided data on 293,127 hospital admissions with ICU encounters, of which 178,604 (Group 1) were before April 2020 and 114,523 (Group 2) were from the period April 2020 to December 2022. Table 1 summarizes the demographic findings. The groups were clinically similar in age (68.5 years), gender (47% female), and racial distribution (17% Black, 74% White, 1.6% Native American, 1.2% Asian/Pacific Islander, 6.2% other or unspecified).
The Average Acute Physiology Score was 42.9 with predicted hospital mortality of 12.8% in Group 1, and 44.9 in Group 2 with predicted mortality of 15.0%. Actual mortality was 11.27% in Group 1, yielding an observed/expected SMR of 0.88. Observed mortality in Group 2 was 16.03%, yielding a SMR of 1.07. Figure 1 demonstrates SMR over time for all patients, viral pneumonia patients, and beginning in 2020, separately identified COVID-19 patients.
Table 2 catalogues the top 20 diagnoses in each time period. Notably, viral pneumonia was the 36th most-common diagnosis from 2016 to 2020, accounting for just 0.76% of ICU admissions. During the pandemic, viral pneumonia became the most-common admitting diagnosis, representing 8.18% of all ICU admissions.
There were notable increases during the pandemic of patient admissions for cardiac arrest and sepsis of unknown origin, coupled with decreases in admissions for heart failure, cardiac rhythm disturbances, COPD, and bacterial pneumonia.
Viral pneumonia, including COVID-19, became the top admitting diagnosis during the pandemic; previously it was ranked No. 36. In this sub-population, the raw hospital mortality was 25.5% between April 2020 and December 2022; APACHE-predicted mortality for viral pneumonia was 13.29%, yielding an overall SMR of 1.85. However, this excess mortality was concentrated in 2020–2021; it began to decrease in the first quarter of 2022 and thereafter was not statistically different than expected. Figure 1 demonstrates SMR over time for the non-surgical, non-trauma ICU population aged 45–100, and the impact of successive waves of COVID-19.
Examining all ICU patients (not just viral pneumonia or COVID), SMR in these 31 hospitals was 0.92 in 2016, declined to 0.84 in 2019, rose to 1.00 and 1.19 respectively in 2020–2021 (p < 0.01), and returned to 0.95 in 2022. This SMR increase was seen across all age groups but was especially pronounced in ages 45–64 (SMR = 0.87 to 1.13) versus 0.88 to 1.02 in those older than 75. Unadjusted ICU and hospital LOS peaked at 5.13 and 11.1 days in late 2021, reverting to baseline (3.99/9.56) in 2022.
ICU Utilization
Participating hospitals do not dynamically adjust reported bed count by actual staffing levels, so bed utilization cannot be calculated on a monthly or quarterly basis. However, a proxy for ICU utilization is the percentage of patients receiving active therapy versus simply monitoring for patients at high or low risk of deterioration.
Active therapy refers to patients receiving one or more ICU-specific therapies such as mechanical ventilation, ECMO, CRRT, or vasoactive agents. Monitored patients are divided into those with more than 10% risk of requiring active therapy (high-risk monitor, HRM) versus those unlikely to require intervention (low-risk monitor, LRM).
We hypothesized that resource limitations during the pandemic might increase the proportion of patients on active therapy, but the observed effect was modest: 58% pre-COVID and 62% during the pandemic. However, a more pronounced effect on outcome was seen in ventilator days, which were 4.1 pre-COVID, peaked at 6.4 days in the third quarter of 2021 and returned to 4.4 in 2022.
Figure 2 shows the normalized ratios for ICU length of stay and ventilator days. All metrics were stable pre-pandemic, and then increased, with peaks tracking successive COVID-19 waves. While ventilator days normalized in 2022, ICU length of stay and percentage of patients receiving active therapy remain above baseline levels.
DISCUSSION
The COVID-19 pandemic created substantial and unprecedented demand for intensive care and was associated with increased mortality rates compared to baseline. In this cohort of 31 hospitals consistently contributing APACHE-adjusted outcome data during the period 2016 to 2022, we observed increased “raw” (unadjusted) and severity-adjusted mortality, length of stay and days of mechanical ventilation. While much of the increase can be attributed to high mortality with COVID-19, impacts were seen on non-COVID patients as well, possibly because of capacity strain.
Since hospitals have wide variations in case mix, including diagnoses encountered, proportion of medical versus elective surgical patients, average patient age, and the burden of chronic illnesses, interpreting outcomes requires adjustment for these factors.
APACHE is an acronym for Acute Physiology, Age and Chronic Health Evaluation and has been used since 1985 to stratify outcome by the presenting status of critically ill patients. The APACHE methodology uses clinical data to generate an Acute Physiology Score (APS) based on the patient’s deviation from normal physiology for a critically ill patient. Alterations in level of consciousness (Glasgow Coma Score), heart rate, respiratory rate, blood pressure, temperature, urinary output, and abnormal laboratory values at presentation contribute to the APS.
A mortality prediction, however, requires both APS and consideration of the patient’s age, comorbidities, and presenting diagnosis. While it might be assumed that the diagnostic coefficient for “viral pneumonia” would adequately encompass COVID-19, early experience suggested that this novel pathogen carried a higher mortality penalty.(3)
In normal times, viral pneumonia accounts for fewer than 1% of ICU admissions; during the height of the pandemic, many intensive care units were filled with COVID patients, and over the three-year period 2020 to 2023, viral pneumonia jumped to first place with a prevalence of over 8% in these 31 hospitals with varying bed size and teaching status.
Many systems saw a shift in referral patterns as overwhelmed community hospitals transferred COVID-19 patients to tertiary care facilities. Tertiary-care referral hospitals were thus likely to encounter higher overall mortality as the result of accepting the “sickest” patients in transfer for interventions such as extracorporeal membrane oxygenation (ECMO) and advanced ventilation modes for acute respiratory distress syndrome (ARDS).
In this cohort of more than 290,000 patients from 31 hospitals across the United States, mortality in COVID-19 patients was initially very high (up to 43%), but has recently stabilized at 15-19% coincident with predominance of the Omicron strain prevalent during 2022. Our data are concordant with prior reports that mortality is lower in 2022 than earlier years.(6) Increased confidence in caring for COVID-19 patients outside the ICU setting may be contributing to the relatively high mortality rate in those still requiring ICU admission.
The existing coefficient for “viral pneumonia” apparently did not calibrate well to risk-adjust mortality predictions for the early (original, Alpha, Delta) strains of COVID-19, resulting in SMRs well above expected. However, once Omicron became the dominant strain in early 2022, SMR, length of stay, and ventilator day metrics more closely aligned with those expected from traditional viral pneumonia.
Others have commented on the apparent reduced lethality of the Omicron strain compared to earlier variants.(7) Xie and colleagues found in a Veterans Administration population that the 30-day mortality rate for hospitalized patients (whether or not in ICU) was 5.97% with COVID-19 versus 3.16% with influenza, a hazard ratio of 1.61. However, the COVID-19 associated mortality rate decreased from 17–21% in 2020 to about 6%, while influenza mortality remained at 3.7–3.8% throughout. Xie also noted increased death rates among unvaccinated individuals compared with those vaccinated or boosted.(8)
Our data do not offer guidance as to whether the recent reduction in COVID-associated SMR is because of reduced lethality of newer viral strains, growing population immunity due to vaccination or prior infection, or improvements in medical therapy for COVID. It is possible that all three contribute to improved recent results.
Although SMR appears to be returning to pre-pandemic values, LOS remains higher than historic norms. Ventilator days have normalized, but average ICU LOS remains 0.8 days longer, while hospital LOS has increased by 1.4 days. This suggests that challenges to prompt hospital discharge are contributory. While we do not have definitive data, others have reported that referral volumes to post-acute facilities have increased post-pandemic, while at the same time many facilities have closed or limited census because of ongoing staffing shortages.(9)
The proportion of patients receiving active therapy (versus those receiving monitoring for risk events) increased modestly during the pandemic, likely reflecting bed capacity issues that prioritized admission for patients requiring active treatment. The average number of days on mechanical ventilation increased by about 38% during the first two years of the pandemic. More recently, however, this severity-adjusted metric is also returning toward pre-pandemic baseline values.
This analysis was limited to hospitals consistently contributing APACHE data during the seven-year span from 2016 to 2022. This “same-store” approach offers insight into changes in ICU admitting diagnoses since 2020.
Not surprisingly, the pandemic was associated with more admissions for cardiac arrest and sepsis of unknown origin. There were decreased admissions for heart failure, COPD, and bacterial pneumonia diagnoses. We were surprised to see fewer drug overdose admissions (0.36% pre-pandemic vs. 0.26% during the pandemic), since evidence suggests substance abuse accelerated during the pandemic.(10)
We previously raised cautions about benchmarking ICU outcomes(4) based on 2020–2021 data. With more recent (2022) data coinciding with Omicron becoming the latest variant, APACHE IVb can again confidently be used to adjust mortality, LOS, and ventilation outcomes. However, COVID-19 continues to evolve, and there is no guarantee that future mutations will again change the lethality of the virus. We plan ongoing surveillance and assessment as COVID-19 viruses continue to evolve.
Review and presentation of outcomes data is an important task for the physician executive, and a sudden increase in severity-adjusted mortality requires investigation and intervention. In the case of COVID-19, a novel virus in a vulnerable population appears to be the root cause, and the early SMR penalty has been mitigated partly by advancements in medical care (anti-viral agents, vaccination, ventilation techniques), provider experience, and possibly lower lethality with emerging viral variants.
For an individual hospital, statistical differences in SMR may be difficult to demonstrate on a quarterly or even annual basis, but the collective data from APACHE participants was helpful in early identification of this problem, and also its resolution.
An unexplained increase in mortality is worrisome for patients and providers, but also may have knock-on effects with pay for performance, “magnet hospital” designation, or resource allocation. Identification and remediation of contributory factors for apparently worse outcome is an important task for the chief medical officer, vice president of medical affairs, as well as other hospital leaders.
Limitations of this study include that data are collected from just 31 hospitals who voluntarily contribute data via APACHE. Although these hospitals are geographically dispersed and a mix of large and small facilities, including some teaching hospitals, they may not be representative of the United States as a whole.
CONCLUSION
APACHE risk-adjusted outcomes have normalized since Omicron became the dominant variant in the United States. Data are insufficient to determine if this improvement occurred because of better ICU care, lower lethality of newer variants, higher population vaccination or acquired immunity, or other causes.
Our observation of recent reductions in severity-adjusted mortality are concordant with other studies suggesting Omicron is associated with less-severe outcomes than prior COVID-19 strains. Physician leaders can be reassured that the COVID-related impact on APACHE-benchmarked ICU outcomes largely resolved during calendar year 2022, but ongoing attention is needed as the virus continues to evolve.
REFERENCES
Lau VI, Dhanoa S, Cheema H, Lewis K, Geeraert P, et al. Non-COVID Outcomes Associated with the Coronavirus Disease-2019 (COVID-19) Pandemic Effects Study (COPES): A Systematic Review and Meta-Analysis. PLoS One. 2022;17(6):e0269871. https://doi.org/10.1371/journal.pone.0269871 . PMID: 35749400; PMCID: PMC9231780
Kramer A, Higgins TL, Zimmerman J. Comparing Observed and Predicted Mortality Among Intensive Care Units Using Different Prognostic Systems: Why Do Performance Assessments Differ? Crit Care Med. 2015;43:261–269. https://doi.org/10.1097/CCM.0000000000000694
Higgins TL, Stark MM, Henson KN, Freeseman-Freeman L. Coronavirus Disease 2019 ICU Patients Have Higher-Than-Expected Acute Physiology and Chronic Health Evaluation-Adjusted Mortality and Length of Stay Than Viral Pneumonia ICU Patients. Crit Care Med. 2021;49(7):e701–e706. https://doi.org/10.1097/CCM.0000000000005012 . PMID: 33861555.
Higgins TL, Ringle E, Henson KN: The Impact of COVID-19 on Benchmarking ICU and Hospital Performance. Physician Leadership Journal. 2022;9(6):28–34. https://doi.org/10.55834/plj.6582574723
Nathanson BH, Higgins TL. An Introduction to Statistical Methods Used in Binary Outcome Modeling. Semin Cardiothor Vasc Anes. 2008;12(3):153–166. https://doi.org/10.1177/1089253208323415
Adjei S, Hong K, Molinari NM, et al. Mortality Risk Among Patients Hospitalized Primarily for COVID-19 During the Omicron and Delta Variant Pandemic Periods — United States, April 2020–June 2022. MMWR Morb Mortal Wkly Rep. 2022;71:1182–1189. https://doi.org/10.15585/mmwr.mm7137a4
Lorenzo-Redondo R, Ozer EA, Hultquist JF. COVID-19: Is Omicron Less Lethal Than Delta? BMJ. 2022:378:o1806. https://doi.org/10.1136/bmj.o1806
Xie Y, Choi T, Al-Aly Z. Risk of Death in Patients Hospitalized for COVID-19 vs. Seasonal Influenza in Fall-Winter 2022–2023. JAMA. 2023;329:1697–1699. https://doi.org/10.1001/jama.2023.5348
WellSky. The WellSky 2023 Evolution of Care Report Reveals Critical Changes To Care Delivery in the Past Year. News Release. WellSky. July 25, 2023. https://wellsky.com/the-wellsky-2023-evolution-of-care-report-reveals-critical-changes-to-care-delivery-in-the-past-year/
Roberts A, Rogers J, Mason R, et al. Alcohol and Other Substance Use During The COVID-19 Pandemic: A Systematic Review. Drug and Alcohol Dependence. 2021;229: 109150. https://doi.org/10.1016/j.drugalcdep.2021.109150 .