The Laryngoscope

© 2018 The American Laryngological,

Rhinological and Otological Society, Inc.

Appropriateness Criteria for Surgery in the Management of Adult

Recurrent Acute Rhinosinusitis

Luke Rudmik, MD, MSc, FRCSC; Daniel M. Beswick, MD; Jeremiah A. Alt, MD, PhD;
Neil Bhattacharyya, MD; Alexander C. Chester, MD; Stacey T. Gray, MD; David M. Poetker, MD, MA;
Michael G. Stewart, MD, MPH and Timothy L. Smith, MD, MPH

Objectives/Hypothesis: Endoscopic sinus surgery (ESS) is frequently performed for recurrent acute rhinosinusitis
(RARS). Appropriate indications for surgery among patients with RARS have not yet been rigorously determined. The objective
of this study was to define appropriateness criteria for ESS in the management of adult RARS.
Study Design: Application of RAND-UCLA appropriateness methodology.
Methods: A panel of nine multidisciplinary experts in RARS was formed to evaluate RARS scenarios generated from cur-
rent evidence. The panel completed two rounds of a modified Delphi-ranking process and a teleconference.
Results: A total of 32 clinical scenarios were ranked in each round. For adult patients with RARS, ESS can appropriately
be offered as a treatment option when patients experience ≥ four annual episodes, and there is confirmation of at least one
episode via computed tomography or nasal endoscopy, and the patient and clinician jointly participate in shared decision mak-
ing, and the patient has either failed a trial of topical nasal steroids or experienced RARS-related productivity loss.
Conclusions: This study has defined appropriateness criteria for ESS as a management option for adult patients with
RARS. These criteria are intended to represent a minimum threshold for which ESS should be considered in the treatment of
RARS and do not suggest that all patients who meet these criteria should undergo surgery. These criteria may serve as a base-
line set of indications for ESS in patients with RARS.
Key Words: Recurrent acute rhinosinusitis, endoscopic sinus surgery, appropriateness criteria.
Level of Evidence: NA.
Laryngoscope, n/a:1–8, 2018

INTRODUCTION entity is at particular risk of overdiagnosis and overtreat-

Our current healthcare climate has constrained
resources despite rising expenditures per capita. It is crit-
ical to optimize both the quality of and the value provided
by surgical inventions.1 One important strategy to
increase value of care is through improving patient selec-
tion for surgical procedures by defining appropriateness
criteria for surgical candidacy.2,3
Recurrent acute rhinosinusitis (RARS) in the adult
patient is a subtype of rhinosinusitis that can be challenging
to diagnose given its intermittent nature, its confusion with
viral illnesses, and the patient’s rapidly changing symptom-
atic and clinical testing profile.4 Furthermore, this disease
ment, subjecting patients to a variety of complications and
unintended consequences.
The current clinical practice guidelines for RARS
state that a patient must experience four or more epi-
sodes of acute bacterial rhinosinusitis (ABRS) per year,
with asymptomatic periods between episodes.5 Once the
somewhat challenging diagnosis of RARS is confirmed,
treatment decision making can be equally challenging
and often depends on several factors, such as the fre-
quency and severity of ABRS episodes and the associated
degree of quality of life (QoL) and productivity impair-
ment.6 Current evidence suggests that in selected cases,

From the Department of Surgery, Division of Otolaryngology–Head and Neck Surgery, University of Calgary (L.R.), Calgary, Alberta, Canada; The
Department of Otolaryngology–Head and Neck Surgery, Oregon Health and Science University (D.M.B., T.L.S.), Portland, Oregon; Sinus and Skull Base
Surgery Program, Division of Otolaryngology–Head and Neck Surgery, Department of Surgery, University of Utah (J.A.A.), Salt Lake City, Utah;
Department of Medicine, Georgetown University Hospital (A.C.C.), Washington, DC; Department of Otology and Laryngology (N.B.); Department of
Otolaryngology (S.T.G.), Harvard Medical School; Department of Otolaryngology, Massachusetts Eye and Ear Infirmary (S.T.G.), Boston, Massachusetts;
Department of Otolaryngology and Communication Sciences, Medical College of Wisconsin (D.M.P.), Milwaukee, Wisconsin; Department of
Otorhinolaryngology, Weill Medical College of Cornell University (M.G.S.), New York City, New York, U.S.A
Editor’s Note: This Manuscript was accepted for publication on June 20, 2018.
Jeremiah Alt is supported by grants from the University of Utah Program in Personalized Health and the National Center for Advancing Transla-
tional Sciences (NCATS) of the National Institute of Health (NIH) (KL2TR001065) and from the National Institute of Allergy and Infectious Diseases (1R43
AI126987). Timothy Smith and Jeremiah Alt are supported by a grant from the National Institute on Deafness and Other Communication Disorders
(NIDCD) (R01 DC005805). The NIDCD did not contribute to the design or conduct of this study.
Luke Rudmik is a member of the scientific advisory board for BioInspire Technologies Inc. Jeremiah A. Alt is a consultant for Medtronic, Inc.; Glyco-
Mira Therapeutics, Inc.; and Spirox Corp. None of these are affiliated with this research. Neil Bhattacharyya is a consultant for Intersect ENT, Inc., and
Sanofi Ventures David Poetker is a member of the speaker bureau for Intersect ENT, Inc. The authors have no other funding, financial relationships, or con-
flicts of interest to disclose.
Send correspondence to Timothy L. Smith, MD, MPH, 3303 SW Bond Ave, Oregon Sinus Center, Portland, OR 97239. E-mail: smithtim@ohsu.edu

DOI: 10.1002/lary.27438

Laryngoscope Rudmik et al.: Appropriateness Criteria: ESS for RARS

1
endoscopic sinus surgery (ESS) may reduce the frequency
of episodes, improve QoL, and reduce indirect costs
(i.e., improve productivity).7–9
The primary objective of this study is to define appro-
priateness criteria for ESS in the management of Adult
RARS. Utilizing current evidence and a consensus of expert
opinion, the primary outcome of this investigation will be a
set of clinical criteria that will define under what circum-
stances it is appropriate to consider ESS for adult RARS.
MATERIALS AND METHODS
Overview
This study employed the RAND/UCLA appropriateness
methodology using the multi-step process presented in Figure 1.3
The intended patient population is adults with RARS, as noted
in Table I based on clinical practice guideline definitions.5 ESS
was defined as a surgical procedure that opens the paranasal
sinuses using endoscopes for visualization and endoscopic instru-
ments for tissue manipulation through the patient’s nostrils. The
procedure involves the opening of anatomical blockages, includ-
ing removal and/or dilation of tissue adjacent to the paranasal
sinuses, in order to open the sinus cavities and improve ventila-
tion, mucus egress, and delivery of topical medications.10
A multi-disciplinary expert panel of 10 members was
assembled consisting of representatives from the United States
and Canada (Table II). Panel nomination was performed by the
project leads (L.R. and T.L.S.) and focused on the following criteria:
Fig. 1. Methodology used to determine appropriateness criteria.
Laryngoscope
2
TABLE I.
Definition of the Recurrent Acute Rhinosinusitis Patient Population
for this Study.
Adult Age ≥ 18 Years Old
Guideline-based 1. Four episodes per year of ABRS, with distinct
diagnosis of RARS5 symptom-free intervals between each episode
2. Each episode of ABRS must meet one of the
following criteria:
• Symptoms or signs of acute rhinosinusitis that
fail to improve within 10 days or more beyond
the onset of symptoms of an upper respiratory
infection
• Symptoms or signs of acute rhinosinusitis
worsen within 10 days after an initial period of
clinical improvement (i.e., double-worsening)
ABRS = acute bacterial rhinosinusitis; RARS = recurrent acute
rhinosinusitis.
1) leader in the management of RARS, 2) knowledgeable in
evidence-based medicine, 3) geographic diversity, 4) diversity of
research interests, and 5) lack of conflicts of interest that would
potentially influence the study outcomes.
It is recommended that the RAND/UCLA appropriateness
methodology be performed for interventions that fulfill the cri-
teria listed in Table III. Based on the evidence,11–23 the expert
panel decided that ESS for RARS fulfills all five criteria, and that
it was justified to proceed with this study.
Variable Selection and Scenario Development
The selection of clinical variables was based on the factors
that physicians take into account when deciding whether an adult
patient with RARS is an appropriate candidate for ESS. Based on
the literature review, the important clinical variables used for the
development of clinical scenarios are outlined in Table IV.
Ranking and Classifying Appropriateness
The development of appropriateness criteria involved two
rounds of ranking and a teleconference between the first and second
rounds. During both rounds of ranking, each scenario was scored
from 1 to 9, where 1 indicates that ESS is “highly inappropriate” and
9 indicates that ESS is “highly appropriate.” During the ranking pro-
cess, each panelist ranks each scenario for level of appropriateness
using their best clinical judgment based on the evidence and consid-
ering the average adult patient with RARS as defined in Table I.
Round 1 ranking was performed independently by panelists without
panel discussion. The teleconference occurred after all round 1 rank-
ings were independently submitted. The purpose of the conference
call was to review the results from the round 1 ranking and focus
the discussion on disagreements among the rankings. A “disagree-
ment” for a specific scenario was defined when at least one panelist
ranked a score of 1 to 3 and another panelist ranked a score of 7 to
9, regardless of the mean score. In contrast to practice guideline
development, the conference discussion was not intended to achieve
consensus for each scenario but rather to improve the understand-
ing of various panelist perspectives to help inform round 2 rankings.
Round 2 ranking was performed after the conference call.
The classification of appropriateness was based on both the
mean panel ranking score for each clinical scenario after round
2 and whether a “disagreement” occurred. Appropriate indica-
tions achieved a mean score of 7 to 9 without any disagreements.
Uncertain indications achieved a mean score of 4 to 6 OR any
indication with a “disagreement.” Inappropriate indications
achieved a mean score from 1 and 3 without any disagreements.
Rudmik et al.: Appropriateness Criteria: ESS for RARS
 TABLE II.
Expert Panel Members
Name Clinical Perspective on Panel Country Academic Affiliation Potential COI

Luke Rudmik Rhinology Canada University of Calgary Scientific advisory board for BioInspire Technologies Inc.
Daniel M. Beswick Rhinology United States Oregon Health and None
Science University
Jeremiah A. Alt Rhinology United States University of Utah Consultant for Medtronic, Inc.; Spirox Corp.;
AxioSonic, LLC; GlycoMira Therapeutics, Inc.
David Poetker Rhinology United States Medical College of Wisconsin Speakers bureau for Intersect ENT, Inc.
Neil Bhattacharyya Oto–HNS United States Harvard Medical School Consultant for Intersect ENT, Inc.;
and Sanofi Ventures
Alexander C. Chester Internal medicine/ United States Georgetown University None
primary Care Medical Center
Stacey T. Gray Rhinology United States Harvard Medical School None
Michael G. Stewart Oto–HNS United States Weill Cornell Medical College None
Timothy L. Smith Rhinology United States Oregon Health None
and Science University

COI = conflicts of interest; Oto–HNS = otolaryngology–head and neck surgery.

RESULTS shared decision making between patient and clinician to

The expert panel ranked a total of 32 clinical scenar-
ios produced from the combination of five previously
defined dichotomous outcome variables (Table IV). After
the second round of ranking, there were six clinical sce-
narios that received a mean score of > 7 without any dis-
agreements and were therefore considered “appropriate”
to perform ESS for RARS. There were two scenarios with
a “disagreement” whereby at least one panelist ranked as
inappropriate (score 1–3) and at least one panelist ranked
as appropriate (score 7–9).
Appropriate Scenarios to Perform ESS for RARS
For adult patients with RARS, six scenarios were
considered appropriate to perform ESS for RARS
(Table VI). All scenarios required the following two vari-
ables: 1) at least one episode of ABRS confirmed with an
objective measure such as CT or nasal endoscopy; and 2)
review the risks and benefits of ESS, along with the
expected outcomes and treatment alternatives prior to
ESS. Four of the six appropriate scenarios required a
trial of topical intranasal steroid therapy prior to consid-
ering ESS, and four of the six required significant reduc-
tions in daily productivity from RARS. Scenarios with
both objectively confirmed disease and shared decision
making, as well as either a trial of nasal steroids or sig-
nificant productivity loss (or both), were identified as
appropriate.
In summary, the outcomes from this study support
that it is appropriate to perform ESS for RARS when the
following clinical criteria are achieved: 1) confirmation of
at least one episode of ABRS using an objective measure
such as nasal endoscopy or CT imaging, 2) shared deci-
sion making between patient and clinician regarding sur-
gical treatment, and 3) either a failed trial of topical
intranasal steroid therapy or the presence of significant
reductions in daily productivity due to RARS.

TABLE III.
Criteria to Perform a RAND/UCLA Appropriateness Study for RARS
Criteria ESS for RARS Fulfills Supporting Evidence

Procedure is frequently utilized.
Procedure is associated with substantial
potential morbidity or mortality.
Procedure consumes
significant resources.
Procedure has wide geographic
variations in rates of use.
Procedure is controversial.
Yes ESS is in the top 10 most common ambulatory surgeries in the U.S.16 A significant
percentage of these cases are performed for RARS.
Yes Risk of orbital or intracranial complication of ESS for CRS is near 0.25%.19 The risk of
serious complication of ESS for RARS is probably lower because the dissection is
typically limited to the anterior ethmoid and maxillary sinuses. 5% risk of re-visit after ESS
(67% ED visit; 19% surgery center; 14% inpatient admission)14
Yes Estimated direct healthcare cost per case of ESS is between $8,200 to $10,000 in the
U.S.22; $3,500 to $5,000 in Canada.12 Estimated overall annual direct cost of RARS to the
U.S. healthcare system is in excess of $120 million for medical costs, plus additional
surgical costs.
Yes There are large geographic variations of ESS utilization for CRS in the U.S.,21,23 Canada,20
and the U.K.11 Given that there is similar uncertainty surrounding the ESS indications for
RARS, it is assumed that there will be similar geographic variations for RARS-related
ESS.
Yes Current grade C evidence supports ESS as an option for RARS in properly selected cases.24
No criteria to define “properly selected cases”

CRS = chronic rhinosinusitis; ED = emergency department; ESS = endoscopic sinus surgery; RARS = recurrent acute rhinosinusitis; U.K. = United King-
dom; U.S. = United States.

Laryngoscope Rudmik et al.: Appropriateness Criteria: ESS for RARS

3
 TABLE IV.
List of Criteria Variables Used to Develop the Clinical Scenarios
Clinical Variable Variable Outcome Evidence and Rationale

At least one ABRS episode documented
with either CT or nasal endoscopy
(Table V)
Topical intranasal corticosteroid therapy
used between ABRS episodes
(Table V)
Number of ABRS episodes per year
Significantly reduced productivity
associated with RARS*
Shared decision making including discussion
with patient regarding risks versus expected
benefit from surgery and treatment
alternatives
1. No Diagnosing RARS is fundamental to the appropriate management of RARS.
2. Yes 96.5% of patient self-reported acute sinusitis episodes lack CT findings that confirm ABRS
(the final diagnosis of RARS patient self-reported ABRS episodes was: rhinitis = 47%;
migraine = 37%; and facial pain disorder = 12.5%).4
Due to the high risk of RARS misdiagnosis when based purely on patient self-reported
episodes, at least one episode should be confirmed with either nasal endoscopy or CT
scan findings prior to considering ESS as a treatment option.
1. No Evidence supports that use of daily INS therapy during and between episodes of ABRS may
reduce frequency of ABRS episodes and improve overall QoL.25,26
2. Yes
Given the low risk for adverse events and potential clinical benefit, a trial of INS therapy is
an appropriate intervention prior to considering surgery.24
1. ≥ 4 Economic evaluations demonstrate that the benefits from surgery outweigh the risk and cost
of surgery when the patient has 5 or more ABRS episodes per year.27,28
2. ≥ 5
However, it may be inappropriate to require a defined threshold frequency of ABRS in order
to consider ESS because the frequency of episodes should always be put in a clinical
context (severity, length, impact on productivity and QoL).
For example, a patient with the minimum of 4 ABRS episodes per year but with severe
impact on work productivity and QoL may be more appropriate for ESS compared to a
patient with 6 ABRS episodes per year with minor impact on work productivity and QoL.
1. Yes Patients with ABRS have varying degrees of reduced productivity. Impairment in
productivity is a patient-centered variable that is often reported to be a driving factor to
2. No
pursue surgery for RARS.
Because ESS has been shown to improve RARS-related daily productivity outcomes, using
the degree of reduced work productivity as one of the indications for ESS may help select
those RARS patients who would most benefit from surgery.
However, similar to QoL impairment, there is no evidence to support what threshold of
reduced daily productivity should be present in order to offer ESS as a treatment option.
1. No Potential risks and harm from ESS for RARS:
2. Yes 1. Approximately 0.25% risk for serious complication to the brain and eye19
2. 5% risk of unanticipated re-visit after ESS (67% ED visit; 19% surgery center; 14%
inpatient admission)14
3. Risk of no clinical benefit in terms of reduced ABRS episode frequency or severity or
improved QoL
4. Risk of revision ESS related to scarring or incomplete surgery
Potential benefits of ESS for RARS include:
1. Improved QoL8,29
2. Reduced annual frequency of ABRS episodes (mean of 50% reduction in ABRS
episodes). This may reduce number of antibiotic courses per year.26,29
3. Improved work productivity9,26

ABRS = acute bacterial rhinosinusitis; CT = computed tomography; ESS = endoscopic sinus surgery; INS = intranasal steroid; QoL = quality of life;
RARS = recurrent acute rhinosinusitis.
*In the context of reduced productivity related to RARS, the term significant is defined as a reduction in daily functioning that prevents the ability of the
patient to perform routine tasks such as paid employment (i.e., missed work) or other nonpaid tasks such as volunteerism and household work.

Scenarios With a Disagreement to Perform ESS
for RARS
Two scenarios contained a disagreement among the
panelist rankings (Table VII). Both disagreements
occurred in scenarios in which patients lacked two cri-
teria: a trial of a topical intranasal steroid therapy and
the presence of daily productivity loss related to RARS.
These disagreements support the notion that the appro-
priateness to perform ESS for RARS is uncertain in
patients who do not fulfill at least one of these criteria.
Inappropriate Scenarios to Perform ESS
for RARS
The majority of clinical scenario combinations (24 of
32) evaluated in this study were considered inappropriate
to perform ESS for RARS (i.e., mean score 1–3). Overall,
the panelists agreed that it was inappropriate to perform
ESS for RARS when a patient lacks at least one ABRS
episode confirmed with an objective measure such as
nasal endoscopy or CT imaging, and when there is an
absence of the shared decision-making process that
focuses on patient preference for treatment after patients
are informed of treatment alternatives, risks, and
benefits—along with the anticipated outcomes.
DISCUSSION
Summary
Prudent selection of patients with RARS for ESS is
essential to optimize outcomes. In this study, established
methodology was employed to define appropriateness cri-
teria for ESS for patients with RARS (Table VIII). These
appropriateness criteria are intended to represent the
minimum threshold for which ESS should be considered

Laryngoscope Rudmik et al.: Appropriateness Criteria: ESS for RARS

4
 TABLE V. TABLE VII.
Definitions for RARS Clinical Variables Summary of Scenarios With a Disagreement About the
Appropriateness of Performing ESS for RARS
CT findings Sinus mucosal thickening with air–fluid level30
of ABRS Clinical Scenarios
Bubbles within sinus30
Obstruction of the osteomeatal complex 30 No. 1 No. 2

Nasal endoscopy Discolored discharge and purulent secretions within

Confirmed 1 episode Yes Yes
findings of ABRS the nasal cavity24,31 of ABRS CT/endoscopy
CT findings that Concha bullosa 32
Shared decision making Yes Yes
predispose to RARS
Prominent agger nasi cell32
Trial topical steroids No No
Infraorbital cell32,33
Productivity loss present No No
Paradoxical middle turbinate34
No. of ABRS episodes per year 4 5+
Increased radiodensity of the osteomeatal
Mean scenario score 6.7 7.1
complex35
Panelist score breakdown 6 panelists 8 panelists scored
Septal deviation causing obstruction of scored between between 7 and
osteomeatal complex obstruction32 7 and 9 2 panelists 9 1 panelist
Medical therapy Daily topical intranasal corticosteroid ± topical high score of 6 1 panelist score of 3
used between volume saline irrigation25 score of 3
ABRS episodes
ABRS = acute bacterial rhinosinusitis; CT = computed tomography;
ABRS = acute bacterial rhinosinusitis; CT = computed tomography; ESS = endoscopic sinus surgery; RARS = recurrent acute rhinosinusitis.
RARS = recurrent acute rhinosinusitis.

each episode associated with a filled antibiotic prescrip-

as an option in the management of RARS; these criteria tion and excluding patients who underwent sinonasal
do not suggest that all patients who meet this threshold surgery, Bhattacharyya et al. reported the prevalence of
should undergo surgery. These appropriateness criteria RARS to be 0.035% in the United States.37 It seems likely
were developed with the intent of improving the quality that the true prevalence of RARS may be higher than this
of care that is delivered for RARS. study has identified because the methodology did not cap-
ture patients who 1) were treated with watchful waiting
for an ABRS episode(s), a viable treatment option in clini-
Challenges With RARS Management and cal guidelines5; 2) did not fill their prescriptions; or 3)
Research were treated surgically.
Management and research of RARS are associated with Numerous indicators point to the fact that ESS is
numerous challenges. Guideline definitions for the diagnosis being performed frequently for RARS. However, the exact
of RARS were only recently formalized and published.5 Even surgical burden of RARS has not been specifically
with published diagnostic criteria, RARS can be challenging reported to date. Estimates of surgical case volume often
to diagnose clinically given that there are several other con- rely on diagnostic codes, and only in the 10th revision of
ditions that mimic the symptoms of ABRS.4 Relatively few the International Classification of Diseases set, which
studies delineating outcomes of RARS have been reported to was widely implemented in 2015, does RARS have a dis-
date. RARS has been identified as one of the two leading dis- crete diagnostic code that could be used to determine
ease categories in need of quality improvement according to prevalence. Identifying the surgical burden of RARS is
the American Rhinologic Society.36 the subject of ongoing research.
Establishing an accurate prevalence of RARS is chal- The impact of RARS on patients has been described
lenging. Using a definition for RARS of at least four epi- by several studies, although much remains to be eluci-
sodes of acute rhinosinusitis in a 12-month period, with dated. Patients with RARS have impaired QoL, as well as
associated productivity loss and impaired health utility,
similar to patients with chronic rhinosinusitis (CRS)
TABLE VI.
without nasal polyposis.8,9,29 However the specific QoL
Summary Clinical Scenarios Ranked as Appropriate to Perform
ESS for RARS deficit experienced by patients with RARS during acute
episodes as compared to between acute episodes has not
Clinical Scenarios
been determined. In addition, medical management for
no. 1 no. 2 no. 3 no. 4 no. 5 no. 6 RARS has been shown to improve QoL in patients with
RARS7; however, an optimal medical regimen to mini-
Confirmed 1 episode of Yes Yes Yes Yes Yes Yes
ABRS CT/Endoscopy mize recurrent ABRS episodes remains elusive. Topical
Shared decision making Yes Yes Yes Yes Yes Yes nasal corticosteroids appear to lead to earlier symptom-
Trial topical steroids Yes Yes Yes Yes No No
atic improvement for patients with RARS, although a
definitive benefit has not been demonstrated.25
Productivity loss present No No Yes Yes Yes Yes
Surgical outcomes for RARS have been explored.
No. of ABRS episodes per year 4 5+ 4 5+ 4 5+
Patients with RARS who undergo ESS report improved
Mean Scenario Score 7.9 8.3 8.8 9.0 7.8 8.0
symptoms and QoL and decreased missed work days, with
ABRS = acute bacterial rhinosinusitis; CT = computed tomography; mixed results on decreases in antibiotic and sinus medica-
ESS = endoscopic sinus surgery; RARS = recurrent acute rhinosinusitis. tion utilization postoperatively.26,29 Costa et al. identified

Laryngoscope Rudmik et al.: Appropriateness Criteria: ESS for RARS

5
 TABLE VIII.
Appropriateness Criteria for Endoscopic Sinus Surgery for Adult Patients With RARS
No. ABRS Confirmation of at Least 1 ABRS
Episodes/Year Episode via CT or Endoscopy
Appropriate 4 or more Present
Uncertain 4 or more Present
ABRS = acute bacterial rhinosinusitis; CT = computed tomography; RARS = recurrent acute rhinosinusitis.
that surgical treatment for RARS resulted in greater mean
QoL improvement than medical treatment.7 Economic
analyses suggest that five-to-six annual episodes of ABRS
is the breakeven point to justify surgical treatment,
accounting for lost income, costs, QoL, and lost productiv-
ity in the postoperative period.27,28 Michalowski and
Kacker recently reviewed five studies on RARS and con-
cluded that surgery for RARS is indicated when certain
criteria were met.6
This study sought to apply well-established method-
ology to the existing RARS literature to determine appro-
priateness for surgical candidacy in this population.
Appropriateness criteria utilizing the same methodology
have been developed for the management of other sinona-
sal diseases. The American College of Radiology pub-
lishes regularly updated appropriateness criteria on
obtaining imaging tests for RARS, CRS, and invasive fun-
gal sinusitis.38 A multi-specialty, multi-national group
developed appropriateness criteria for ESS as a manage-
ment option for uncomplicated CRS.39,40 Validation of the
CRS appropriateness criteria demonstrated that these
criteria predict outcomes for patients undergoing ESS for
this disease in certain practice settings.41
Rationale for Appropriateness Criteria
The panel felt that it was essential for practitioners
to confirm objective evidence of inflammation by CT or
nasal endoscopy prior to performing ESS for RARS. Objec-
tive confirmation of at least one episode was felt to be nec-
essary due to other conditions that patients may perceive
as ABRS, such as rhinitis, migraine, and facial pain, and
the large discrepancy between patient-reported episodes
of sinusitis and radiographic evidence of disease.4 The
spectrum of diagnosis confirmation considered by the
panel ranged from accepting symptomatic reports from
patients (i.e., no objective confirmation of ABRS) to objec-
tively diagnosing at least four annual separate episodes of
ABRS. The panel felt that objective confirmation of at
least one episode of ABRS was both required and suffi-
cient to perform ESS. Requiring objective confirmation of
multiple episodes would be resource-intensive (e.g., travel
costs, appointments, multiple endoscopies, and/or CT
scans), and would delay consideration of other treatment
options while potentially exposing patients to avoidable
medications and their associated costs and side effects.
Given the challenges in diagnosing and managing
RARS and the limited evidence surrounding treatment
outcomes, the panel considered shared decision making
Laryngoscope
6
Shared Patient and
Decision Making Treatment Factors
Present Failed trial of nasal steroids and/or
presence of productivity loss related to RARS
Present Absence of both nasal steroid trial and
productivity loss related to RARS
essential prior to performing ESS for RARS. In this
study, the definition of shared decision making was mutu-
ally agreed upon by the panelists to be a process in which
clinicians and patients work together to make decisions
and select management, based on clinical evidence that
balances risks and expected outcomes with patient prefer-
ences and values.42 For RARS, this includes discussion of
the treatment alternatives, risks and benefits, and antici-
pated outcomes—and is ultimately driven by patient pref-
erence after a thorough discussion.
For scenarios in which both the objectively confirmed
evidence of RARS and shared decision making were pre-
sent, the panel determined that four or more annual epi-
sodes of ABRS were sufficient grounds upon which to
perform ESS. Although economic analyses have sug-
gested that greater than or equal to five episodes per year
can be used as a threshold to offer ESS to patients with
RARS,27,28 the panel felt there are notable variations in
the severity of episodes for different patients and that
surgical treatment was a reasonable option for patients
with at least four annual episodes.
The inclusion of either a failed trial of intranasal ste-
roids and/or presence of significantly reduced productivity
due to RARS was considered essential by the panel to per-
forming ESS. A systematic review of topical corticosteroid
use in RARS determined that there is limited benefit for
this medication, although patients did report symptom
improvement earlier when using intranasal corticosteroids
compared to placebo.25 In the three trials included in this
systematic review, RARS was not defined according to the
American Academy of Otolaryngology–Head and Neck
Surgery Clinical Practice Guideline, and each study
evaluated intranasal corticosteroids as an adjunct to antibi-
otic therapy.43–45 The finding of expedited symptomatic
improvement with intranasal corticosteroid use was based
on the results of a trial with moderate directness of evidence
and low risk of bias that demonstrated patients who
received intranasal corticosteroids + oral cefuroxime +
intranasal oxymetazoline compared to placebo + oral cefur-
oxime + intranasal oxymetazoline experienced improved
symptoms at median 6 versus 9 days (P = 0.01) and may
have experienced fewer ABRS recurrences (P = 0.06).43
Although existing evidence confirming efficacy of intranasal
steroids for RARS is lacking, the panel felt that it does not
necessarily imply a lack of effectiveness for this therapy.
Trials that specifically evaluate the utility of topical medica-
tions in RARS are needed.
Quantifying and defining levels of productivity loss in
RARS is admittedly challenging. The panel acknowledged
Rudmik et al.: Appropriateness Criteria: ESS for RARS
that measuring and documenting RARS-related productiv-
ity loss is an area under investigation, and this criterion
must rely on discussions between the clinician and patient.
This criterion is thus left intentionally vague. Factoring in
the relatively limited evidence for productivity loss and
intranasal corticosteroids, the panel felt that at least one
of these two criteria was necessary to perform ESS for
patients with RARS. For scenarios without at least one of
these two criteria, there was disagreement among the pan-
elists about the appropriateness of performing ESS, lead-
ing to a categorization of “uncertain” for these patients.
Caveats
The methodology used in this study categorizes
appropriateness based on the typical patient with RARS.
Three caveats arose that were worth further discussion.
First, important comorbidities such as immunosuppres-
sion, severe atopy, and ciliary dysfunction—among
others—could impact the complex decision for appropri-
ate treatment. Little has been published on the role of
immune dysfunction in RARS, and patients with inflam-
matory sinus manifestations due to immune compromise
may have CRS as opposed to RARS. If immune dysfunc-
tion is suspected in a patient with RARS, it is sensible to
evaluate this as part of the management of this disease,
as has been suggested for CRS.24 Second, the role of bac-
terial culture in management of RARS has not yet been
thoroughly established. International consensus recom-
mendations suggest that at times it is useful to obtain
microbiologic confirmation of bacterial infection during an
acute episode of RARS.24 However, in the management of
RARS, there was insufficient published evidence on the
value of obtaining cultures to include this as potential cri-
terion in the current appropriateness criteria. Third, the
panel’s collective experience suggests that some patients
suffer substantial productivity loss and/or reduced QOL
with acute episodes. In this severely impacted patient
population, the panel felt that it was reasonable to con-
sider ESS if patients had fewer than four episodes
per year.
Limitations and Future Directions
Several limitations need to be considered when
interpreting these results. No randomized controlled tri-
als have been performed on surgical outcomes for RARS;
therefore, the appropriateness criteria developed in this
study are based partly on the experts’ opinion and inter-
pretation of the current evidence. To minimize bias, the
panel was composed of clinicians in two countries and
three specialties (internal medicine, general otolaryngol-
ogy, and rhinology), along with various practice settings
(Table II). Prior work in surgical fields has demonstrated
that mixed specialty panels produce less aggressive rec-
ommendations for surgical intervention.46
Although diversity of panelists was a factor in panel
selection, four rhinologists included in this project under-
went fellowship training with the senior author. This
raises the possibility of selection bias in panel composi-
tion; in addition, this aspect of training may predispose to
Laryngoscope
academic and judgment bias in scenario interpretation.
However, the potential effect of this similar training was
felt to be insignificant given the time since training and
other factors. For example, among the scenario interpre-
tations in the round 1 rankings, individuals from this
subgroup were at times outliers in their appropriateness
assessments. In the final ranking, a single disagreement
by any panelist would prevent consensus for a scenario.
These appropriateness criteria were developed with
an “average” patient in mind using theoretical scenarios.
To account for individual patient preferences and expec-
tations, shared decision making was incorporated into the
ranking scenarios. Despite this, the typical patient
described by the scenarios in the appropriateness criteria
do not represent the entire population of patients with
RARS. Appropriateness criteria must be applied to the
individual patient with great care. Finally, surgical
extent and surgical technique are not addressed in this
study.
These appropriateness criteria represent an attempt
to define a minimum threshold for which ESS could be
offered as an option in the management of RARS. These
criteria should be validated in multiple practice settings
to determine if the patient selection guided by these cri-
teria results in improved outcomes. As more research on
RARS is conducted, these criteria will need to be updated
to reflect the current state of the evidence.
CONCLUSION
Appropriateness criteria to offer ESS as an option in
the management of RARS were developed according to
established methodology. These criteria are intended to
represent a minimum threshold for which ESS should be
considered in the treatment of RARS and do not suggest
that all patients who meet these criteria should undergo
surgery. These criteria are based on an appropriate diag-
nosis of RARS with at least four annual episodes of
ABRS, objective confirmation of at least one acute episode
by CT or nasal endoscopy, the presence of shared decision
making between clinician and patient, and either a failed
trial of intranasal corticosteroids or presence of signifi-
cant productivity loss associated with RARS. These cri-
teria may serve as a baseline set of indications for ESS
among patients with RARS and may improve the quality
of care that is delivered for this disease.
BIBLIOGRAPHY
1. Cutler DM. Rising medical costs mean more rough times ahead. JAMA
2017;318:508–509.
2. Reames BN, Shubeck SP, Birkmeyer JD. Strategies for reducing regional
variation in the use of surgery: a systematic review. Ann Surg 2014;259:
616–627.
3. Brook RH, Chassin MR, Fink A, Solomon DH, Kosecoff J, Park RE. A
method for the detailed assessment of the appropriateness of medical
technologies. Int J Technol Assess Health Care 1986;2:53–63.
4. Barham HP, Zhang AS, Christensen JM, Sacks R, Harvey RJ. Acute radiol-
ogy rarely confirms sinus disease in suspected recurrent acute rhinosinu-
sitis. Int Forum Allergy Rhinol 2017;7:726–733.
5. Rosenfeld RM, Piccirillo JF, Chandrasekhar SS, et al. Clinical practice
guideline (update): adult sinusitis. Otolaryngol Head Neck Surg 2015;152:
S1–S39.
6. Michalowski A, Kacker A. Is sinus surgery indicated for recurrent acute rhi-
nosinusitis? Laryngoscope 2017;127:1255–1256.
Rudmik et al.: Appropriateness Criteria: ESS for RARS
7
 7. Costa ML, Psaltis AJ, Nayak JV, Hwang PH. Medical therapy vs surgery
for recurrent acute rhinosinusitis. Int Forum Allergy Rhinol 2015;5:
667–673.
8. Steele TO, Mace JC, Dedhia R, Rudmik L, Smith TL, Alt JA. Health utility
values for patients with recurrent acute rhinosinusitis undergoing endo-
scopic sinus surgery: a nested case control study. Int Forum Allergy Rhi-
nol 2016;6:1182–1187.
9. Steele TO, Detwiller KY, Mace JC, Strong EB, Smith TL, Alt JA. Productiv-
ity outcomes following endoscopic sinus surgery for recurrent acute rhino-
sinusitis. Laryngoscope 2016;126:1046–1053.
10. Bublik M, Herman B, Younis R. Functional endoscopic sinus surgery. Oper
Tech Otolayngol Head Neck Surg 2009;20:167–171.
11. UK Commissioning Guide: Rhinosinusitis. Available at: https://www.rcseng.
ac.uk/healthcare-bodies/docs/published-guides/rhinosinusitus. Accessed
Dec. 24, 2015.
12. Au J, Rudmik L. Cost of outpatient endoscopic sinus surgery from the per-
spective of the Canadian government: a time-driven activity-based costing
approach. Int Forum Allergy Rhinol 2013;3:748–754.
13. Bhattacharyya N. Ambulatory sinus and nasal surgery in the United
States: demographics and perioperative outcomes. Laryngoscope 2010;
120:635–638.
14. Bhattacharyya N. Unplanned revisits and readmissions after ambulatory
sinonasal surgery. Laryngoscope 2014;124:1983–1987.
15. Caulley L, Thavorn K, Rudmik L, Cameron C, Kilty SJ. Direct costs of adult
chronic rhinosinusitis by using 4 methods of estimation: results of the US
Medical Expenditure Panel Survey. J Allergy Clin Immunol 2015;136:
1517–1522.
16. Cullen KA, Hall MJ, Golosinskiy A. Ambulatory surgery in the United
States, 2006. Natl Health Stat Report 2009:1–25.
17. Dautremont JF, Rudmik L. When are we operating for chronic rhinosinusi-
tis? A systematic review of maximal medical therapy protocols prior to
endoscopic sinus surgery. Int Forum Allergy Rhinol 2015;5:1095–103.
18. Lund VJ. Maximal medical therapy for chronic rhinosinusitis. Otolaryngol
Clin North Am 2005;38:1301–1310, x.
19. Ramakrishnan VR, Kingdom TT, Nayak JV, Hwang PH, Orlandi RR.
Nationwide incidence of major complications in endoscopic sinus surgery.
Int Forum Allergy Rhinol 2012;2:34–39.
20. Rudmik L, Bird C, Dean S, Dort JC, Schorn R, Kukec E. Geographic var-
iation of endoscopic sinus surgery in Canada: an Alberta-based small
area variation analysis. Otolaryngol Head Neck Surg 2015;153:
865–874.
21. Rudmik L, Holy CE, Smith TL. Geographic variation of endoscopic sinus
surgery in the United States. Laryngoscope 2015;125:1772–1778.
22. Smith KA, Orlandi RR, Rudmik L. Cost of adult chronic rhinosinusitis: a
systematic review. Laryngoscope 2015;125:1547–1556.
23. Venkatraman G, Likosky DS, Morrison D, Zhou W, Finlayson SR,
Goodman DC. Small area variation in endoscopic sinus surgery rates
among the Medicare population. Arch Otolaryngol Head Neck Surg 2011;
137:253–257.
24. Orlandi RR, Kingdom TT, Hwang PH, et al. International Consensus state-
ment on allergy and rhinology: rhinosinusitis. Int Forum Allergy Rhinol
2016;6(suppl 1):S22–209.
25. van Loon JW, van Harn RP, Venekamp RP, Kaper NM, Sachs AP, van der
Heijden GJ. Limited evidence for effects of intranasal corticosteroids on
symptom relief for recurrent acute rhinosinusitis. Otolaryngol Head Neck
Surg 2013;149:668–673.
26. Bhattacharyya N. Surgical treatment of chronic recurrent rhinosinusitis: a
preliminary report. Laryngoscope 2006;116:1805–1808.
27. Leung R, Almassian S, Kern R, Conley D, Tan B, Chandra R. Patient level
decision making in recurrent acute rhinosinusitis: a cost-benefit threshold
for surgery. Laryngoscope 2013;123:11–16.
Laryngoscope
8
28. Leung R, Kern RC, Conley DB, Tan BK, Chandra RK. Establishing a thresh-
old for surgery in recurrent acute rhinosinusitis: a productivity-based anal-
ysis. Otolaryngol Head Neck Surg 2012;146:829–833.
29. Poetker DM, Litvack JR, Mace JC, Smith TL. Recurrent acute rhinosinusi-
tis: presentation and outcomes of sinus surgery. Am J Rhinol 2008;22:
329–333.
30. Ebell MH, McKay B, Guilbault R, Ermias Y. Diagnosis of acute rhinosinusi-
tis in primary care: a systematic review of test accuracy. Br J Gen Pract
2016;66:e612–e632.
31. Fokkens WJ, Lund VJ, Mullol J, et al. EPOS 2012: European position paper
on rhinosinusitis and nasal polyps 2012. A summary for otorhinolaryngol-
ogists. Rhinology 2012; 50:1–12.
32. Loftus PA, Lin J, Tabaee A. Anatomic variants of the paranasal sinuses in
patients with recurrent acute rhinosinusitis. Int Forum Allergy Rhinol
2016;6:328–333.
33. Alkire BC, Bhattacharyya N. An assessment of sinonasal anatomic variants
potentially associated with recurrent acute rhinosinusitis. Laryngoscope
2010;120:631–634.
34. Sadr SM, Ahmadinejad M, Saedi B, Razaghian F, Rafiee M. Anatomical var-
iations in sinus imaging in sinusitis: a case control study. B-ENT 2012;8:
185–189.
35. Johnson SM, Honeybrook AL, Ramprasad VH, Abi Hachem R, Jang DW.
Radiodensity of the ostiomeatal complex in recurrent acute rhinosinusitis.
Otolaryngol Head Neck Surg 2017;157:887–890.
36. Rudmik L, Mattos JL, Stokken JK, et al. Rhinology-specific priority setting
for quality improvement: a modified Delphi study from the Quality
Improvement Committee of the American Rhinologic Society. Int Forum
Allergy Rhinol 2017;7:937–944.
37. Bhattacharyya N, Grebner J, Martinson NG. Recurrent acute rhinosinusi-
tis: epidemiology and health care cost burden. Otolaryngol Head Neck
Surg 2012;146:307–312.
38. Expert Panel on Neurologic Imaging; Kirsch CFE, Bykowski J, et al. ACR
appropriateness criteria sinonasal disease. J Am Coll Radiol 2017;14:
S550–S559.
39. Rudmik L, Soler ZM, Hopkins C, et al. Defining appropriateness criteria for
endoscopic sinus surgery during management of uncomplicated adult
chronic rhinosinusitis: a RAND/UCLA appropriateness study. Int Forum
Allergy Rhinol 2016;6:557–567.
40. Rudmik L, Soler ZM, Hopkins C, et al. Defining appropriateness criteria for
endoscopic sinus surgery during management of uncomplicated adult
chronic rhinosinusitis: a RAND/UCLA appropriateness study. Rhinology
2016; 54:117–128.
41. Beswick DM, Mace J, Soler ZM, et al. Appropriateness criteria predict out-
comes for sinus surgery and may aid in future patient selection. Laryngo-
scope 2018. doi: 10.1002/lary.27227.
42. National Learning Consortium. Fact Sheet: Shared Decision Making. Avail-
able at: https://www.healthit.gov/sites/default/files/nlc_shared_decision_
making_fact_sheet.pdf. Accessed April 11, 2018.
43. Dolor RJ, Witsell DL, Hellkamp AS, et al. Comparison of cefuroxime with
or without intranasal fluticasone for the treatment of rhinosinusitis.
The CAFFS Trial: a randomized controlled trial. JAMA 2001;286:
3097–3105.
44. Meltzer EO, Charous BL, Busse WW, Zinreich SJ, Lorber RR, Danzig MR.
Added relief in the treatment of acute recurrent sinusitis with adjunctive
mometasone furoate nasal spray. The Nasonex Sinusitis Group. J Allergy
Clin Immunol 2000;106:630–637.
45. Qvarnberg Y, Kantola O, Salo J, Toivanen M, Valtonen H, Vuori E. Influ-
ence of topical steroid treatment on maxillary sinusitis. Rhinology 1992;
30:103–112.
46. Fraser GM, Pilpel D, Kosecoff J, Brook RH. Effect of panel composition on
appropriateness ratings. Int J Qual Health Care 1994;6:251–255.
Rudmik et al.: Appropriateness Criteria: ESS for RARS