Perceived Operating Room Management Strengths and Weaknesses: A Novel Coding System of Survey Responses

Operating rooms (ORs) in the United States are a vital part of hospitals, in terms of both costs and revenue streams. A typical OR has an annual operating budget of approximately $21 million and accounts for up to 60% of hospital revenue and 40% of hospital expenses.(1-3) Previous literature has established that the optimization of certain perioperative factors may lead to profound savings. The drive for this optimization has led perioperative administrators to seek additional management education and training after they finish their medical education.(4)

Although there is a perception that hospital administrators may want perioperative leaders with dual medical and management training, no literature currently is available that examines the value of formal business administration training, the strengths and weaknesses of OR managers, or the various governance structures. In an attempt to fill this gap, we developed a survey tool to assess the strengths and weaknesses of OR management. The first outcome of the survey is a summative content analysis examining how often and in what ways “education” is mentioned as either a strength or a weakness of the governing bodies. Second, we identified common trends among free text responses and developed a broadly applicable coding scheme that can be used by administrators to assess the shortcomings of their governance structures, and, perhaps more importantly, accurately assess the value proposition for individuals with advanced training.

Methods

To achieve these goals, we adapted the questions from a survey sent to the Vermont Society of Anesthesiologists. The requirement for written informed consent was waived for this study. We validated the answers to the questions with a telephone follow-up to the OR managers or medical directors at the 10 hospitals that performed surgeries. The survey consists of 25 questions covering the following topics:

• OR demographics and staffing (i.e., district, type of hospital, case volumes and specialty categories, number of anesthetizing locations, composition of anesthesia healthcare teams);

• OR management (i.e., first case start and turnover times, utilization rates, staffing ratios, overtime, contribution margin, growth in caseload, and block allocations); and

• OR governance structures.

Embedded in this survey were two open-ended questions, which are the only questions used in the analysis presented here. The remaining 23 questions provided context for other studies, described elsewhere. The two questions used in this analysis were:

1. In your experience, what have been the greatest weaknesses in individuals who have been in charge of running operating rooms?

2. What are the prime attributes you look for in someone to run an operating suite?

To assess our primary outcome, we performed a quantitative content analysis on the importance of management training. We originally evaluated the free text responses by looking for how often “training” was mentioned. To ensure that we captured all related responses, we also included the synonyms knowledge, understanding, education, course, class, school, and teaching in the analysis.

To perform the coding analysis, we used survey data that were originally collected as part of a survey tool sent to the New York State Society of Anesthesiologists (NYSSA) e-mail database. Two coders (KK, SV) independently performed the initial coding on the NYSSA data set using the conventional content analysis described elsewhere.(5) An expert in the field (DL) validated the original coding analysis by both coders. Coding schemes from the two coders were then compared. Any differences were discussed and rectified to establish a common scheme. We additionally surveyed the Texas Society of Anesthesiologists (TSA). Following previous work in qualitative analysis, the coding scheme was applied to the TSA data by both KK and SV separately, to ensure agreement and validity of the tool.(6) Any disagreement between codes or the need for additional codes was discussed until consensus was reached. In this process the two coders agreed on the generalized coding scheme; however, in some instances there was disagreement on the appropriate coding label. For example, one coder frequently referred to “Understanding the environment” while the other researcher coded these responses as “Lack of insight.” In these instances, the researchers went back to the original data to see which statement would best capture the thoughts and sentiments of the respondents. Other than these differences, the most common disagreement between researchers was the over- or undercoding of a response, where one researcher would have more codes for a single response than the other. Most frequently this was a simple error by the coder and easily rectified. Rarely did the two researchers disagree on how comments should be coded.

Results

We received 49 responses (1.5%) from the approximately 3500 e-mails sent to the NYSSA membership database and 60 responses (approximately 2%) from the 2900 e-mails sent to the TSA membership database. For the quantitative content analysis we found training was mentioned only one time among all 109 responses, whereas the synonyms knowledge and understanding were each mentioned four times, and none of the other synonyms appeared in the data.

Figure 1 depicts the final coding scheme, as developed by the two coders. In short, four strengths—experience, leadership, team player, and efficient—were identified initially, as were five weaknesses—poor leadership, inexperience, not a team player, inefficient, and organizational structure. In total, these nine descriptors served as individual clusters, each containing unique yet related codes. For example, Team Player is a strength cluster, containing the individual codes Fair & Thoughtful, Consensus Builder, and Flexibility. When this coding scheme was applied to the TSA dataset, analysis demonstrated that interrater agreement was significantly better than chance, with Cohen’s kappa coefficient of 0.87 for strengths and 0.88 for weaknesses.

Figure 1. Final coding scheme.

Table 1 describes the results of our coding system, as applied to both the TSA and NYSSA data sets. Some of the differences are highlighted, as follows: within the Leadership cluster data, results were split into two groups. Firm, Level-headed, and Good Communication Skills were the three largest contributors, with 13, 10, and 9 responses, respectively. The remaining two codes (Common Sense and Manage Personalities) had only 3 and 2 responses, respectively. This is in sharp contrast to the Efficiency cluster, where the codes were perfectly split between Organized and Multi-tasker. Results for the Team Player cluster focused on being Fair & Thoughtful, with over 50% of the responses being of this code. Flexibility and Consensus Builder completed this cluster, with 6 and 7 responses, respectively. When we stratify the strength data between two data sets, we see the same general trends, with a few exceptions. The strength Flexibility is one such exception: it was not mentioned in the NYSSA data, but it was mentioned six times in the TSA survey data. Conversely, the weakness Inflexibility was only mentioned as a weakness once among NYSSA respondents, whereas it was mentioned six times in the TSA data.

Discussion

These results from our quantitative content analysis build upon previously published literature in OR management sciences. These publications have established that people who lack education in the management sciences think they make optimal decisions, when, in fact, their decisions are worse than decisions made randomly by chance.(7,8) The results presented here show that when individuals are queried on the desirable attributes of OR managers, they fail to recognize education as an important strength. This preliminary finding suggests that many of those who work in ORs or are directly involved in perioperative processes do not value formal training in their supervisors. This result is disconcerting, because unpublished data from our survey suggest otherwise: 58% of our respondents answered “Yes” when asked “Do you think [the greatest weakness] could be improved/altered with education or training?” This apparent inconsistency in responses draws into question the perceived value of additional training for OR managers.

Due to the small sample, it would be inappropriate to assume that the trends identified here apply at large. Regardless, this study establishes a coding system that individuals can use to assess strengths and weaknesses of their OR managers. Validation of such a coding system opens the door for further study of OR management systems, and will allow for optimization and increased efficiency. We believe that expanded studies in different states or in larger organizations, such as the national anesthesiology, surgery, and nursing societies, will continue to highlight state-to-state variability in management structures. It is our hope that, with these data, we will be able to standardize OR governance structures in a way that maximizes strengths and mitigates weaknesses of the individuals charged with managing an expensive, costly service line for a hospital.

References

1. Saver C. OR managers’ salaries increase, but so do their responsibilities. OR Manager. 2008;24(10): 9-14.

2. Randa K, Heiser R, Gill R. Strategic investments in the operating room (OR): information technology (IT) to generate rapid ROI and long-term competitive advantage. http://dev.healthleadersmedia.com/content/242938.pdf

3. Weiss A, Elixhauser A, Andrews RM. Characteristics of operating room procedures in U.S. Hospitals, 2011. Agency for Healthcare Research and Quality. Statistical Brief #170. February, 2014.

4. Desai AM, Trillo RA, Macario A. Should I get a Master of Business Administration? The anesthesiologist with education training: training options and professional opportunities. Curr Opin Anaesthesiol. 2009;22:191-198.

5. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277-1288.

6. Attanasio L, Kozhimannil KB, Jou J, Mcpherson ME, Camann W. Women’s experiences with neuraxial labor analgesia in the Listening to Mothers II Survey: a content analysis of open-ended responses. Anesth Analg. 2015;121:974-980.

7. Prahl A, Dexter F, Braun MT, Van Swol L. Review of experimental studies in social psychology of small groups when an optimal choice exists and application to operating room management decision-making. Anesth Analg. 2013;117:1221-1229.

8. Wang J, Dexter F, Yang K. A behavioral study of daily mean turnover times and first case of the day start tardiness. Anesth Analg. 2013;116:1333-1341.