Inherent Biases in Patient-Reported Satisfaction With Care in the Outpatient Setting

The Medicare Access and CHIP Reauthorization Act of 2015 (MACRA) established new ways to pay physicians caring for Medicare beneficiaries based on the quality and effectiveness of the care they provide.(1) In the past, physicians were paid on an adjustable fee-for-service model, which allowed fees to be adjusted based on factors such as geographic area and malpractice insurance cost.(2) MACRA modified this by attempting to tie physician performance to payment through the introduction of the Merit-Based Incentive Payments System (MIPS).(3) MIPS itself is broken down into four components: (1) quality; (2) advancing care information; (3) clinical practice improvement activities; and (4) resource use.(4) As MIPS is phased in, it eventually will be able to adjust a physician’s reimbursement by a significant amount, adding a maximum of 27% if a physician is an exceptional performer, or decreasing payment by 9% if they do not meet certain standards.(5)

Out of all components of MIPS, quality is arguably the most important, accounting for 50% of a physician’s MIPS score.(6) A factor that can tie into the quality score is patient-reported satisfaction (PRS), as measured by the Press-Ganey Consumer Assessment of Healthcare Providers and Systems survey.(7) This makes identifying and correcting for any confounding or transforming factors that impact PRS metrics of paramount importance, so that physicians can avoid being penalized for patients who will intrinsically score lower in satisfaction measures. In this study, we compared PRS in an outpatient setting to determine whether scores are influenced by various demographic and socioeconomic characteristics when patients are seeing the same set of physicians for similar diagnosed conditions.

Materials and Methods

After institutional review board approval, we prospectively collected anonymous, patient-reported questionnaires dealing with PRS with medical care. All questionnaires were administered by our front desk staff after patient check-in and completed prior to patient check-out at our institution (Texas Tech University Health Sciences Center) between July 1, 2017, and March 1, 2018, in an outpatient urology clinic setting.

Questionnaires were distributed to and voluntarily returned by consecutive adult male and female follow-up patients who saw a provider in our outpatient urology clinic during the stated time period. We did not offer any financial or other incentive for completion of our study instrument. New patients, patients younger than 18 years of age, patients who presented for nursing visits only, freedom-impaired patients (i.e., prison inmates), or any other patient who could not complete the survey due to a disability (e.g., blindness, neurological disorder that inhibited manual dexterity) were excluded. We did not include or score partially completed questionnaires.

Our study instrument was the standardized, externally validated, PRS screening tool called the Patient Satisfaction Questionnaire-18 (PSQ-18) by RAND Health (www.rand.org/health/surveys_tools/psq.html ).(8) To maximize clinic work flow, we used the short form of the questionnaire, which took on average five to eight minutes to complete. A copy of the PSQ-18 is provided in Figure 1. Data from patients who voluntarily completed the questionnaire were collected and recorded, and the PSQ-18 scoring guidelines were used to measure patient-reported outcomes across several measurement indicators, including:

• General satisfaction;

• Technical quality;

• Interpersonal manner;

• Communication;

• Financial aspects;

• Time spent with the doctor; and

• Accessibility and convenience.

Figure 1. Patient Satisfaction Questionnaire-18 (PSQ-18).

Scoring guidelines for the PSQ-18 are provided in Figure 2.

Figure 2. Scoring for the Patient Satisfaction Questionnaire-18 (PSQ-18) (continues).

Figure 2 (continued). Scoring for the Patient Satisfaction Questionnaire-18 (PSQ-18).

Several “categorization” questions about demographics and socioeconomic status were included at the beginning of our study instrument. These included multiple-choice questions about the patient’s age, gender, race, education level, mean income level, medical insurance coverage, time since last follow-up, and travel distance to our facility. Answers were grouped into ranges based on predetermined cut-off thresholds. The categorization questions included in our study instrument prior to the PSQ-18 are provided in Figure 3.

Figure 3. Study instrument categorization questions (continues).

Figure 3 (continued). Study instrument categorization questions

Responses were scored and presented as continuous variables with means and standard deviations (SD). Mean scores were reported across the seven measurement indicators for the PSQ-18 as noted above. We used the one-way analysis of variance (ANOVA) test to compare mean scores across our several categorization variables. Statistical analysis was performed with the Statistical Package for the Social Sciences (SPSS) software package (IBM Corporation, Armonk, New York). All tests were two-sided, with p <.05 considered to be statistically significant.

Results

We collected 306 anonymous, voluntary, completed PRS questionnaires during the study period. The overall complete response rate to our study instrument from qualified patients was 10%. Most of the qualified follow-up patients (85%) did not want to voluntarily participate in our study, and 5% returned an incomplete response to our questionnaire with regard to either the categorization questions or the PSQ-18.

Most of the participants in our study population fell into the following categories:

• 55 to 74 years of age (52.9%);

• Male (72.2%);

• Non-Hispanic white (64.7%);

• Some college or a two-year degree (33.3%);

• Mean income level between $50,000 and $100,000 (26.5%);

• Private/commercial medical insurance coverage (38.9%);

• Were being seen at 3 to 6 months follow-up (25.5%); and

• Had a travel distance of less than 10 miles to our facility (46.7%).

Mean scores (and SDs) from the PSQ-18 were as follows:

• General satisfaction: 4.08 (0.84);

• Technical quality: 4.06 (0.67);

• Interpersonal manner: 4.15 (0.71);

• Communication: 4.17 (0.71);

• Financial aspects: 3.78 (0.93);

• Time spent with doctor: 3.90 (0.81); and

• Accessibility and convenience: 3.67 (0.80).

Patients aged 35 to 54 years reported lower mean scores in general satisfaction compared with patients in other age groups. This trend also was seen in other measurement indicators across the PSQ-18, with patients aged 35 to 54 years reporting lower mean scores in technical quality, financial aspects, time spent with the doctor, and accessibility and convenience compared with other age groups (Table 1).

African-American and Hispanic/Latino patients reported lower mean scores in financial aspects compared with non-Hispanic white, Asian, and other racial groups (Table 2). Patients with mean income levels of $50,000 to $100,000 reported higher mean scores in technical quality compared with patients with mean income levels of less than $10,000 (Table 3).

Self-pay patients and those with commercial insurance reported lower mean scores in financial satisfaction and accessibility and convenience compared with patients covered by Medicaid, Medicare, or Tricare (Table 4).

Patients whose last visit was more than 12 months ago reported higher mean scores in general satisfaction compared with patients seen within 1 month or 1 to 3 months since their last visit (Table 5). PRS was not significantly different across gender (Table 6), education level (Table 7), or travel distance (Table 8).

Discussion

In this study, we demonstrated that PRS in an outpatient setting is highly influenced by patient demographics and socioeconomic status. In our cohort of patients retuning for a visit with a provider in the urology clinic, satisfaction with financial aspects or the accessibility and convenience of medical care was significantly lower in self-pay patients or those with commercial insurance compared with patients covered by some form of government assistance (i.e., Medicaid, Medicare, or Tricare). These findings seem intuitive, given that self-pay patients or those with commercial insurance have larger out-of-pocket costs through copays or upfront cash payments compared with patients who have Medicaid, Medicare, and Tricare. Expectations in terms of speed and efficiency of medical care may be higher given these larger out-of-pocket expenses.

Additionally, African-American and Hispanic/Latino patients reported lower mean scores in financial aspects of care compared to other racial groups. This could be explained by the fact that African American and Hispanic/Latino populations are in historically lower socioeconomic groups and represent a higher percentage of indigent care. This also confirms our hypothesis that PRS measurements are inherently biased based on demographic and socioeconomic characteristics. The CMS should consider adjustments based on patient demographics, socioeconomic status, and payer mix when determining physician reimbursements through MIPS.

The existing literature seems to support a connection between insurance status and PRS. Armstrong et al.(9) examined how insurance status affected the Press-Ganey scores of 1597 patients in inpatient surgical settings, and they found that Medicaid patients were more likely to be in the top half of satisfaction scores regarding nursing care (p = .01) and personal factors (p = .05) when compared with patients with commercial insurance.

Huynh et al.(10) also studied the satisfaction of 380 patients who were visiting a surgical clinic for an initial consultation. They reported that patients with some form of insurance coverage (e.g., Medicare, Medicaid, or PPO/HMO) were more satisfied with their experience compared with patients with no insurance. The authors speculated that these lower satisfaction scores may have reflected the concern of self-pay patients over their lack of access to medical care or their inability to pay the full cost for their surgeries.

Bible et al.(11) found in their study of 200 patients in an outpatient spine clinic that patients with Medicare were the most satisfied with their provider (p = .005) and their quality of care (p = .028). The authors theorized that this phenomenon may be in part due to older patients having higher overall satisfaction with life, and since the majority of elderly use Medicare for healthcare coverage, this relationship would carry over to PRS.

Xesfingi and Vozikis,(12) using data from 31 countries, reported that increased public healthcare spending had a dramatic impact on improved PRS. They concluded that increased public healthcare spending translated into an improved perception of free healthcare services, which raised patient satisfaction. Additionally, they reported that increased private healthcare spending correlated negatively with patient satisfaction. These findings correlate similarly with our study results, because patients whose medical expenses were covered by publicly subsidized federal- or state-run programs were more satisfied with certain aspects of their care than those who had private/commercial insurance with larger out-of-pocket costs.

Finally, Chino et al.(13) found that in their survey of 168 patients in an oncology setting, patients were more dissatisfied with the technical quality of their healthcare when they had a higher financial burden. Our results corroborate these findings, with higher-income patients reporting increased satisfaction with the technical quality of their care compared with lower-income patients.

Several studies, on the other hand, have reported no relation between PRS and demographic and socioeconomic factors. Boudreaux et al. surveyed 437 patients who had recently visited their institution’s emergency department and found that insurance status had no significant relationship with the patient’s satisfaction. They did comment, however, that the patient’s likelihood of recommending the emergency department correlated with the patient’s insurance status (p = .018).(14) Wongus et al. also surveyed 1065 patients at a family medicine clinic and found that PRS was not significantly related to medical insurance status.(15)

There were several limitations to our study. We had a poor overall response rate to the questionnaire from qualified patients during the study period, given its voluntary nature. This may have led to a “response bias” where the distribution of demographic and socioeconomic characteristics of patients that responded to the questionnaire were inherently different compared with our overall clinic population. For example, the rate of Medicaid coverage and private/commercial insurance in our study population was 9.2% and 38.9%, respectively. Based on fiscal year 2017 data, however, Medicaid made up 20.1% of our payer mix in the outpatient urology clinic, whereas private/commercial insurance consisted of 27.0%. Because we did not prospectively collect demographic and socioeconomic data on patients who did not respond to our survey instrument, we were unable to compare the characteristics of included versus excluded populations.

The survey-type nature of our study also carries the possibility of “recall bias.” Our results showed that follow-up patients who had their last provider visit more than 12 months ago reported higher mean general satisfaction scores compared with patients seen within 3 months. This may have been influenced by the length of the recall period, with longer recall periods associated with better outcomes due to memory-associated biases. In fact, many healthcare questionnaire studies have shown that recall error increases with the length of the recall period.(16)

Our study population may not reflect that of other institutions or other parts of the country. For example, our state—Texas—did not expand Medicaid and has the largest rate of uninsured residents in the United States (approximately 16.6%).(17) Additionally, as a public medical school that receives state and federal funding, we deal with a disproportionately larger percentage of patients with government-sponsored healthcare compared with clinics in a private practice setting. We also provide a larger amount of indigent care compared with our peers in other clinical settings or higher-income areas.

Finally, because we did not have any research personnel associated with our study who were supervising and facilitating completion of the questionnaires by willing participants and were available to answer any questions (the survey instruments were administered by our front desk staff), the accuracy of the responses could come into question. Despite these limitations, the results of our study are still relevant in showing biases in PRS that potentially could influence reimbursements and penalize physicians.

Conclusion

PRS can vary inherently based on demographic and socioeconomic factors, including age, race, income level, and insurance status. Efforts to further understand and minimize these influences should be considered to avoid penalizing physicians who care for indigent, underserved groups. This could help prevent further disparities in medical care across our health system.

Acknowledgement: The authors thank the Texas Tech University Health Sciences Center Clinical Research Institute for their assistance with this research.

References

1. Mullins A. Medicare Payment Reform: making Sense of MACRA. Fam Pract Manag. 2016;23(2):12-15.

2. Olds JW. Medicare fee for service: a model of managed care? JAMA. 1999;282(11):1037.

3. Miller P, Mosley K. Physician reimbursement: from fee-for-service to MACRA, MIPS and APMs. J Med Pract Manage. 2016;31(5):266-269.

4. CMS. The Merit-Based Incentive Payment System: MIPS scoring methodology overview. Center for Medicare & Medicaid Services; 2015. www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Value-Based-Programs/MACRA-MIPS-and-APMs/MIPS-Scoring-Methodology-slide-deck.pdf.

5. Centers for Medicare and Medicaid Services HHS. Medicare Program; Merit-Based Incentive Payment System (MIPS) and Alternative Payment Model (APM) Incentive Under the Physician Fee Schedule, and Criteria for Physician-Focused Payment Models. Final rule with comment period. Fed Regist. 2016;81(214):77008-77831. www.federalregister.gov/documents/2016/11/04/2016-25240/medicare-program-merit-based-incentive-payment-system-mips-and-alternative-payment-model-apm .

6. Hirsch JA, Rosenkrantz AB, Ansari SA, Manchikanti L, Nicola GN. MACRA 2.0: are you ready for MIPS? J Neurointerv Surg. 2017;9:714-716.

7. CMS. CAHPS for MIPS Survey. 2018. www.cms.gov/Research-Statistics-Data-and-Systems/Research/CAHPS/mips.html . Accessed April 4, 2018.

8. Thayaparan AJ, Mahdi E. The Patient Satisfaction Questionnaire Short Form (PSQ-18) as an adaptable, reliable, and validated tool for use in various settings. Med Educ Online. 2013;18(1):21747.

9. Armstrong JG, Weigel PA, Cromwell JW, Byrn JC. Postoperative complications and patient satisfaction: does payer status have an impact? Am J Surg. 2016;211:1099-1105.e1091.

10. Huynh HP, Legg AM, Ghane A, Tabuenca A, Sweeny K. Who is satisfied with general surgery clinic visits? J Surg Res. 2014;192(2):339-347.

11. Bible JE, Kay HF, Shau DN, O’Neill KR, Segebarth PB, Devin CJ. What patient characteristics could potentially affect patient satisfaction scores during spine clinic? Spine (Phila Pa 1976). 2015;40(13):1039-1044.

12. Xesfingi S, Vozikis A. Patient satisfaction with the healthcare system: assessing the impact of socio-economic and healthcare provision factors. BMC Health Serv Res. 2016;16:94.

13. Chino F, Peppercorn J, Taylor DH, Jr., et al. Self-reported financial burden and satisfaction with care among patients with cancer. Oncologist. 2014;19(4):414-420.

14. Boudreaux ED, Ary RD, Mandry CV, McCabe B. Determinants of patient satisfaction in a large, municipal ED: The role of demographic variables, visit characteristics, and patient perceptions. Am J Emerg Med. 2000;18(4):394-400.

15. Wongus R, Schluterman NH, Feinstein S, McGirt N, Greenberg DR, Schwartz DB. Patient satisfaction reported by in-visit and after-visit surveys. Patient Experience Journal. 2015;2(1):68-74.

16. Kjellsson G, Clarke P, Gerdtham UG. Forgetting to remember or remembering to forget: a study of the recall period length in health care survey questions. J Health Econ. 2014;35:34-46.

17. Sered S. Uninsured in Texas, then and now. Health Aff (Millwood). 2016;35(9):1734-1737.