Periodontology 2000, Vol. 37, 2005, 1228 Printed in Denmark.

All rights reserved

Copyright Blackwell Munksgaard 2005

PERIODONTOLOGY 2000

Evidence-based periodontology, systematic reviews and research quality

I A N N E E D L E M A N , D A V I D R. M O L E S & H E L E N W O R T H I N G T O N

Periodontology has a rich background of research and scholarship. A simple MEDLINE search of Periodontal Diseases OR Periodontitis alone from 1966 to 2003 brings up more than 45,000 hits. Therefore, efcient use of this wealth of research data needs to be a part of periodontal practice. Evidence-based periodontology aims to facilitate such an approach, accelerating the introduction of the best research into patient care. This chapter will review the concepts of evidencebased periodontology, introduce the systematic review as a research tool and examine how evidencebased periodontology can both inform on and benet healthcare in periodontology. Finally, we will examine the strengths and limitations of different research designs and their appraisal. We hope that the information in this chapter will provide a basic understanding of the concepts that will be relevant to reading and enjoying the other chapters in this volume of Periodontology 2000.

The highest quality evidence will be used if it exists, but if it does not, lower levels of evidence will be considered. Lower levels of evidence usually means research designs more prone to bias and therefore with less reliable data. However, the nature, strengths and weaknesses of the evidence will be made clear to the reader. In addition, wherever possible, the data presentation supplies more clinically relevant information, including the probability of achieving a certain effect such as a benet, and considering possible adverse effects.

What evidence-based periodontology is not

Evidence-based periodontology is not simply systematic reviews of randomized controlled trials, although this can be an important aspect. Evidencebased periodontology is an approach to patient-care and nothing more. The expectations that are sometimes laid on it can be inappropriate. It cannot provide answers if research data do not exist (other than using expert opinion) and it cannot substitute for highly developed clinical skills. Therefore, it can never be cookbook healthcare or use statistics in isolation to drive clinical care. Instead it is the comprehensive integration of appropriate research evidence, patient preference and clinical expertise (Fig. 1). This can be illustrated with data from a recent systematic review on periodontal plastic surgery for root surface coverage in localized Miller Class I and II defects (25). The data from the systematic review demonstrated that connective tissue grafts were signicantly better than guided tissue regeneration in

What is evidence-based periodontology?

Evidence-based periodontology is the application of evidence-based health care to periodontology. A useful denition of evidence-based health care has been proposed by Muir Gray: An approach to decision making in which the clinician uses the best evidence available, in consultation with the patient, to decide upon the option which suits that patient best (8). Therefore, evidence-based periodontology is a tool to support decision making and integrating the best evidence available with clinical practice.

12

Evidence-based periodontology

Clinicians skills

Patient preferences and views

Evidence-based periodontology

and this might inform on the decision. However, surprisingly, no data are available on patient views on aesthetics comparing the two procedures. Therefore, this evidence-based approach to management of recession has produced the best available evidence, shown how precise this estimate actually is, and highlighted the limitations of the evidence, in this case the lack of data on some outcomes that are relevant to the decision making process.

Clinical relevance
Best evidence available

Fig. 1. How evidence-based periodontology ts into healthcare. Reproduced with permission from Clarkson, J, Harrison, JE, Ismail, AI, Needleman, IG, Worthington, H, eds. Evidence Based Dentistry for Effective Practice. London: Martin Dunitz, 2003 (20).

reducing recession (mean difference 0.43 mm, 95%CI [0.62,0.23], chi square for heterogeneity 7.8 (df 5) P 0.17). This indicates that the pooled difference between six studies included in the review is 0.43 mm, with a 95% condence interval from 0.62 to 0.23. The chi-square test indicates that there is no evidence of any heterogeneity between the studies (they could theoretically all be measuring the same difference). So, does this mean that only connective tissue grafts should be used in the treatment of localized recession defects? Clearly, this would not be appropriate. The data show that both GTR and connective tissue grafts can work. For the selected outcome, which was recession reduction, connective tissue grafts produce 0.43 mm greater effect; the result is both reasonably precise (judged by the condence interval) and the studies from which the data were taken were similar (no evidence of heterogeneity). However, recession reduction might not be the only outcome of interest. The two surgical procedures are very different. One requires the harvesting of a soft tissue graft from the palate and the other does not. There are no data available examining patient preferences, but it is likely that some individuals will prefer a procedure that does not involve two surgical sites, even if it does not reduce recession to the same extent. It is also possible that aesthetics are different following the two procedures

One of the barriers to the application of research ndings in clinical practice is the way that results are often presented. Typically, a mean value will be published, based on a statistical analysis comparing experimental groups. Such a value in conjunction with its associated 95% condence interval is useful to determine whether there is a statistically signicant difference between groups and will often be a requirement of a study designed for regulatory approval. However, this type of analysis is not designed to provide information about the probability of achieving a certain outcome were the reader to apply it in practice. Such an outcome could include achieving a health benet or preventing further disease. For instance, in a meta-analysis from a systematic review on guided tissue regeneration (GTR) for periodontal infrabony defects, the additional benet of using GTR over access ap surgery was a 1.1 mm gain in clinical attachment (21, 22). This should, however, not be interpreted as the additional benet to be expected every time that GTR is used instead of access ap surgery. One approach to analysing and presenting data in a more clinically useful format is to calculate the number needed to treat (NNT). This is the number of patients that would need to be treated to achieve a stated benet (NNTb) or to avoid a stated harm (NNTh). It is derived from a dichotomous outcome such as the proportion of sites achieving at least 2 mm gain in attachment. For the GTR meta-analysis, and using this benet, the NNTb is eight. In other words, for every eight patients treated with GTR, you can expect one to have at least 2 mm more gain in clinical attachment than if you had used an access ap (95% condence interval [4,33]). For detailed guidance regarding the use and calculation of the NNT the reader is recommended to the electronic journal Bandolier: http://www.jr2.ox.ac.uk/ bandolier/booth/painpag/NNTstuff/numeric.htm.

13

Needleman et al.

Evidence-based periodontology vs. traditional periodontology

High quality research and the use of evidence are fundamental to both evidence-based periodontology and traditional periodontology. The differences between these approaches emanate from how research informs clinical practice. Evidence-based periodontology uses a more transparent approach to acknowledge both the strengths and the limitations of the evidence. An appreciation of the level of uncertainty or imprecision of the data is essential in order to offer choices to the patient regarding treatment options. Evidence-based periodontology also attempts to gather all available data and to minimize bias in summarizing the data. These aspects are key to decision making and are highlighted in Table 1. Furthermore, evidence-based periodontology acknowledges explicitly the type or level of research on which conclusions are drawn. The research hierarchy is discussed in more detail later in this chapter. However, one aspect that inuences the reliability of the data is the control of bias. Bias is a collective term for factors that systematically distort the results of research away from the truth. Different research designs offer different possibilities for the control of

bias and therefore vary in their reliability and will be discussed further below.

The components of evidence-based periodontology

An overview of the components is given in Fig. 2. Evidence-based periodontology starts with the recognition of a knowledge gap. From the knowledge gap comes a focussed question that leads on to a search for relevant information. Once the relevant information is located, the validity of the research needs to be considered in two broad areas. Firstly, is the science good (internal validity)? Internal validity focuses on the methodology of research. Secondly, can the ndings be generalized outside of the study (external validity)? External validity might be affected by the way treatment was performed. For instance, if the time spent on treatment was extensive it might not be practical to provide this therapy outside of a research study. Another example could relate to the use of many specic inclusion criteria in a trial which could make it difcult to generalize the ndings to a wider group of patients. The question the reader should ask is whether their types of patients are so different from the study that it is

Table 1. Comparison of evidence-based periodontology vs. traditional periodontology

Evidence-based periodontology Traditional periodontology

Recognize clinical knowledge gap

Develop into a focussed question

Similarities High value of clinical skills and experience Fundamental importance of integrating evidence with patient values Differences Uses best evidence available Systematic appraisal of quality of evidence More objective, more transparent and less biased process Greater acceptance of levels of uncertainty Unclear basis of evidence Unclear or absent appraisal of quality of evidence More subjective, more opaque and more biased process Greater tendency to black and white conclusions

Search for evidence

Reject if invalid or poor

Evaluate the evidence

Integrate into practice

Evaluate the effects

Fig. 2. The steps of evidence-based periodontology. Reproduced with permission from Clarkson, J, Harrison, JE, Ismail, AI, Needleman, IG, Worthington, H, eds. Evidence Based Dentistry for Effective Practice. London: Martin Dunitz, 2003 (20).

Reproduced with permission from Clarkson, J, Harrison, JE, Ismail, AI, Needleman, IG, Worthington, H, eds. Evidence Based Dentistry for Effective Practice. London: Martin Dunitz, 2003 (20).

14

Evidence-based periodontology

reasonable to expect differences in outcomes. After locating and appraising the research, the results then need to be applied clinically, or at least included in a range of options. Finally, the results in clinical practice need to be evaluated to reveal whether the adopted technique achieved the expected outcome. The example of gingival recession mentioned earlier can be used to illustrate this approach. The uncertainty might relate to whether to change from using connective tissue grafts for recession defects to guided tissue regeneration and can be translated into a focussed question. Here the patient or problem group could be rened more closely to localized recession defects and perhaps Miller Class I or II, as we might reasonably expect these lesions to respond differently from more advanced lesions. The intervention is guided tissue regeneration and the comparison, connective tissue grafts. The outcomes would include change in recession or possibly the chance of achieving complete root coverage. Since the procedure is primarily for aesthetics, a patient-centred assessment of aesthetics should be an outcome. As always, there must be a consideration of adverse effects and these might include pain, postoperative infection, and severe bleeding postoperatively. Reassembling this structure into a focussed question would lead to In patients with localized Miller Class I or II recession defects, what is the effect of guided tissue regeneration vs. connective tissue grafts on change in recession, chance of complete defect coverage and aesthetics and what are the adverse effects? For this particular research question, the randomized controlled trial is best able to address the change in recession outcome. For the other outcomes, other research designs might have been used, such as observational studies. Preferably, we would like to nd a systematic review that will have completed the searching and study appraisal for us. The search quickly identies a systematic review (25). The review has a research question that is appropriate to our question and demonstrates a statistically superior effect of connective tissue grafts compared with guided tissue regeneration. The review also acknowledges certain limitations. In terms of the validity of the meta-analysis, the reviewers urge caution as publication bias could be affecting the overall result, but this could not be tested due to the low number of studies. Publication bias is discussed later in this chapter. Another limitation was that there were no data on aesthetics or adverse effects.

Therefore, having reviewed the data, it is clear that there is good evidence to indicate that connective tissue grafts have a greater effect on change in recession than guided tissue regeneration, although there are several limitations to this evidence. Clinical recommendation is tempered by the lack of data on aesthetics and adverse effects and the possible exaggeration of benet through publication bias. This information can then inform on the case presentation to the patient and a choice of options discussed and agreed. The outcome of treatment can then be evaluated to see whether the desired endpoint was achieved and this helps to rene the case presentation discussion in future.

Systematic reviews
One important element of evidence-based periodontology is the systematic review. Systematic reviews are a research design termed research synthesis. That is, they use research methodology to pool data from multiple studies that address a particular hypothesis. A systematic review can be dened as a review of a clearly formulated question that attempts to minimize bias using systematic and explicit methods to identify, select, critically appraise and summarize relevant research. The description of systematic reviews as providing the highest level of evidence is widespread but also raises expectations that may or may not be fullled. A realistic understanding of what a systematic review can provide is important for the appropriate use of this type of evidence (Table 2). More detailed information on systematic reviews exist (5, 19), and guides to conducting them are freely available (1, 13). As with all research, a systematic review starts from an hypothesis. This is derived from a focussed question which is set to answer a particular area of uncertainty. For instance, for the systematic review on smoking and periodontal therapy in the chapter by Labriola et al. in this volume, the focussed question was: In patients with chronic periodontitis, what is the effect of smoking or smoking cessation on the response to nonsurgical periodontal therapy in terms of clinical and patient-centred outcomes?(14). The question has set the types of patients (individuals with chronic periodontitis undergoing nonsurgical therapy), type of exposure (cigarette smoking) and types of outcomes (clinical and patient-centred) to be investigated, each aspect being dened in more detail within the protocol. As this is a prognostic research question, where exposure (smoking) cannot be

15

Needleman et al.

Table 2. The potential of systematic reviews

What a high quality systematic review can do Find and summarize all available studies. A comprehensive search will identify all relevant studies up to the point of the date of the completion of the search. This should give the reader greater condence that bias in selecting studies has been minimized. Provide an objective assessment of the quality or research and in particular the degree of protection from bias within the original studies. Components of methodological quality that have evidence of affecting bias can be evaluated. It may be much harder to evaluate the impact of other quality issues if there is no consensus on how to measure them. An example could be the quality of the treatment provided. Estimate research effects across multiple studies with meta-analysis. Meta-analysis is valid only if studies are similar in their research question and design. Meta-analysis can estimate uncertainty and precision of the effect. Meta-analysis may generate hypotheses for differential effects across subgroups of the population tested. If the effect is consistent across multiple studies (with small differences in design), then it may more readily possible to generalise the results to clinical practice than the results from a single study. Overcome limitations of underpowered studies in detecting a true difference if such a true difference really exists. What a high quality systematic review cannot do It cannot be used in isolation to dictate clinical practice. It is a synthesis of available research and must be used in context with clinical judgement and patient preference. Produce strong conclusions if the research base is weak in quality. The value of the review will then be to present a comprehensive objective summary of the strength of the data and to identify the design of research to answer important gaps. Overcome limitations of narrowly designed clinical research. If the clinical studies only investigate the effect of an intervention in highly selected individuals, the conclusions cannot be generalised outside of these conditions. However, the objective communication of any limitations in the research base will help to set the degree of uncertainty and indicate the priorities for future research. Exclude relevant studies. Although the majority of hits from the search will be excluded, this is due to the deliberate strategy of achieving high sensitivity (likelihood of nding all relevant studies) but low precision (likelihood of only nding relevant studies). Therefore, it is common to nd that more than 90% of the search records are totally irrelevant to the question and must be excluded. The alternative approach of aiming for high precision also carries a high risk of missing relevant studies, although it will appear as if few studies are being excluded. Be a miracle research design. All research has strengths and limitations/weaknesses. Systematic reviews are no different from other research designs in this respect.

randomized, the cohort study is the research design of choice to incorporate into this investigation. These components help in the design of the search strategy that aims to be comprehensive. Usually, searching of multiple electronic databases is carried out together with searching other sources. The most commonly searched databases include MEDLINE (strong on English-language studies), EMBASE (strong on other European languages), and CENTRAL (the Cochrane Collaboration register of trials records). Searching only electronic databases can miss important data, as records on the database may not be appropriately coded. To supplement the electronic search, other approaches are used. Typically, this will include checking for publications in the bibliographies of retrieved studies and review articles, hand-searching of journals for missed

reports, and contacting researchers, industry and journals for unpublished data. The search strategy aims for high sensitivity, i.e. the greatest chance of nding all relevant studies. The downside of this approach is low precision, i.e. in addition to the relevant studies, the search will identify many irrelevant hits (probably more than 90% of hits from the search will not be relevant). For example, in a systematic review on systemic antimicrobials, the search identied 1,300 hits. Screening of the title and abstracts (if available) indicated that 158 papers might be relevant. Once the full text of the studies had been reviewed, 25 trials were judged relevant and could be included (9). At rst sight, rejecting 1,275/1,300 studies would appear to be wasting potentially useful data. However, the deliberately inclusive search identies a large number of

16

Evidence-based periodontology

irrelevant papers, including veterinary medicine, review papers, duplicate reporting of research and laboratory studies. The systematic review screens the search ndings against prestated criteria. These criteria aim to exclude studies irrelevant to answering the question, but do not attempt to exclude on the basis of the quality of the study. Instead, the quality of relevant studies is critically appraised using objective criteria that could inuence the study outcome. The dimension of quality can be incorporated into a systematic review in a number of ways. If the studies are similar enough to be combined in a metaanalysis, the impact of quality on the overall result can be estimated (through sensitivity analyses or meta-regression). If meta-analysis is not possible, the quality of studies can be summarized in narrative tables, in particular for those elements designed to protect against bias. Whilst this may not be as powerful as the use of meta-analysis, it will highlight limitations to be placed upon the conclusions. Following pooling of the data with meta-analysis or qualitative methods, the conclusions from the investigation can be drawn and related to the data derived from the review. The systematic review will not be appropriate for some questions. For instance, to address the question, Which indices have been used to measure gingivitis? a descriptive survey will be more appropriate. However, systematic methods should be adopted for some aspects, in particular to ensure that the search is both comprehensive and contemporary. This might form an important initial stage to answering a question such as Which gingivitis indices have been validated? This is a research question answerable by a systematic review.

The development of evidencebased periodontology

Evidence-based periodontology is built upon developments in clinical research design throughout the 18th, 19th and 20th centuries (15, 20, 23, 28). Evidence-based medicine has only been known for just over a decade and the term was coined by the clinical epidemiology group at McMaster University in Canada (4). The inuence of the McMaster group spread far. One of the earliest to take up the challenge in periodontology (in fact in oral health research overall) was Alexia Antczak Bouckoms in Boston, USA. Antczak Bouckoms and colleagues challenged the methods and quality of periodontal clinical research

in the mid 1980s (3) and set up an Oral Health Group as part of the Cochrane Collaboration in 1994. The editorial base of the Oral Health group subsequently moved to Manchester University in 1997 with Bill Shaw and Helen Worthington as co-ordinating editors (http://www.cochrane-oral.man.ac.uk/). The rst Cochrane systematic review in periodontology was published in 2001 and researched the effect of guided tissue regeneration for infrabony defects (21). Many individuals have been active in the critical analysis of the periodontal literature. These include Jan Egelberg, Loma Linda University, Noel Claffey, Trinity College Dublin, and Gary Greenstein, University of Medicine and Dentistry of New Jersey. There have been many notable events in evidencebased periodontology. The 1996 World Workshop in Periodontology held by the American Academy of Periodontology included elements of evidence-based healthcare, supported by Michael Newman at UCLA (2). The 2002 European Workshop on Periodontology became the rst international workshop to use rigorous systematic reviews to inform the consensus. The workshop was organized by the European Academy of Periodontology for the European Federation of Periodontology, under the chairmanship of Professor Klaus Lang. Sixteen focussed and rigorous systematic reviews formed the basis of intense consensus discussions. A similar approach was used subsequently by the American Academy of Periodontology for the Contemporary Science Workshop in 2003. Many other groups are now using similar methods in healthcare and research. Most recently, the International Center for Evidence-Based Oral Health was launched in 2003 (http://www.eastman.ucl.ac.uk/ iceboh) to produce high quality evidence-based research with an emphasis on, but not limited to, periodontology and implants and to provide generic training in systematic reviews and research methods.

Study designs and critical appraisal

Different study designs
Different clinical research questions require evaluation through different study designs. A study to determine the effectiveness of surgical therapy compared with nonsurgical debridement deals with the effectiveness of a treatment option and would be best answered by a randomized controlled trial (RCT) or, ideally, a systematic review of RCTs. However, it must be noted that although RCTs and systematic reviews of RCTs may well be the gold standard upon which

17

Needleman et al.

to base decisions on the effectiveness of interventions, they are not necessarily appropriate, or ethical, to answer all questions. An RCT would obviously not be helpful in answering the question posed on the epidemiological evidence of plaque in the etiology of periodontitis. For such questions regarding prognosis or etiology, cohort studies would be more appropriate. Table 3 illustrates the types of study designs most suitable for different types of research questions arising in periodontology. The most appropriate source of information will depend upon the type of study design being sought.

Critical appraisal: Why, what and how?

Why critically appraise? Evidence-based periodontology, as its name implies, is periodontology that is based on evidence, but not just any so-called evidence. Richards wrote a toolbox article for the journal Evidence-based Dentistry entitled Not all evidence is created equal (24). We have already seen in this chapter that the quality of evidence may vary according to study design and that this has led to the concept that there can be a hierarchy of evidence. One hierarchy is illustrated in Table 4 and is specic to studies on therapy, prevention, etiology, and harm. Other suggested levels for different types of research question can be found at the Center for Evidence-Based Medicine: (http:// www.cebm.net/levels_of_evidence.asp#levels). The publication of research in a high-ranking journal may not be an absolute guarantee of quality. Within the medical literature there are methodological studies which have empirically shown that quality is not merely a hypothetical concept but also affects study outcomes. As examples of this, the reviews of Schulz et al. (26), Moher et al. (17) and Juni et al. (12) showed that in studies in which there was inadequate concealment of treatment allocation, the treatment effects were exaggerated by about 40% compared to trials of higher quality. Quality assessment of trials in periodontology and implantology Two recent studies have investigated the methodological quality of RCTs in periodontology and implantology as assessed by their publications. Both studies targeted RCTs for investigation due to the importance of the RCT in providing evidence for the effect of interventions and also because of the empiric data indicating the effect of key domains of methodology on bias.

Montenegro et al. (18) conducted a systematic review of the quality of RCTs of periodontal therapy, published in Journal of Periodontology, Journal of Clinical Periodontology or Journal of Periodontal research over a 3-year period from 1996 to 1998. From the electronic search, 283 papers were possibly relevant and 177 studies met the inclusion criteria of being an RCT, performed on humans and for which a full text article was available. Screening and data abstraction were performed independently and in duplicate to minimise error and bias. The evaluation was not performed blind to author afliation identity of the RCTs as the evidence suggests that this has a minimal impact on outcome (16). In view of the empirical data described above, the quality components chosen were those demonstrated to be important for protection from bias: adequacy of method of generation of the random sequence, adequacy of method of concealing the allocation sequence from the patient recruitment, examiner blinding (where it was judged possible to achieve), and handling of losses and withdrawals. The results indicated that 29/177 (17%) of RCTs employed a clearly adequate method of generating the random number sequence, and that 12/177 (7%) of studies described adequate allocation concealment (Fig. 3). Furthermore, where examiner blinding was possible, 97/177 (55%) of studies reported an adequate method. Clear accounting for study subjects was present in 100/177 (55%) of reports. Since the study was conducted on trial reports, it is not clear how much of the inadequacy was due to incomplete reporting rather than inadequate study methods. If the data do reect study conduct, then bias and exaggeration of the effect of the test interventions could be a problem with some trials in periodontology. Similar results were found when investigating RCTs of oral implants (6). This study searched for RCTs up to the end of 1999 in multiple databases. Seventy-four publications were located and 43 RCTs were quality assessed as many studies were presented in multiple publications. Although the methods and criteria were a little different for this study compared with the quality appraisal of periodontal studies, the results are broadly comparable. A clearly adequate method of randomization/concealment of allocation was present in 1/43 (2%) papers. Blinding was described in 12/43 (28%) studies and the reasons for withdrawals and losses to follow-up were specied in 33/43 (77%) of reports.

18

Table 3. Study designs and the types of questions they address

Used for (examples given in italics) Evaluating the effectiveness of an intervention Randomized controlled trial comparing the effectiveness of surgical therapy and nonsurgical debridement.

Denition of study design

Experimental studies Randomized-controlled trial: parallel group design a group of participants (or other unit of analysis, e.g. teeth) is randomized into different treatment groups. These groups are followed up for the outcomes of interest

Randomized-controlled trial: split-mouth design each patient is his/her own control. A pair of similar teeth, or groups of teeth (quadrants), may be selected and randomly allocated to different treatment groups. Controlled trial comparing two methods of treating periodontal intrabony defects using pairs of sites where the LHS is always group A and the RHS group B.

Non-randomized controlled trial allocation of participants under the control of the investigator, but the method falls short of genuine randomization.

Observational studies Cohort: a longitudinal study, identifying groups of participants according to their exposure/intervention status. Groups are followed forward in time to measure the development of different outcomes.

Measuring the incidence of a disease; looking at the causes of disease; determining prognosis. Cohort study looking at the progress of periodontitis over time and relating this to external factors such as smoking or plaque. Identifying potential risk factors for a disease; looking at the possible causes of disease. Case-control study looking at the prevalence of periodontitis and relating this to factors such as genetic markers. Evidence-based periodontology Measuring the prevalence of a disease or risk factor in a dened population at a specic time. A cross-sectional study to determine the current periodontal treatment needs in a specic population.

Case-Control: a study that identies groups of participants according to their disease/outcome status. Groups are investigated/ questioned to determine their exposure status

Cross-sectional: a study (survey) undertaken on a dened population at a single point in time (snap-shot). Subjects are observed on just one occasion and are not followed up.

19

Needleman et al.

Table 4. Center for Evidence-Based Medicine hierarchy of evidence for studies on therapy, prevention, etiology or harm (http://www.cebm.net/levels_of_ evidence.asp#levels)
Level Ia 1b 2a 2b 2c 3a 3b 4 5 Type of evidence Systematic review (with homogeneity*) of randomized controlled trials (RCT). Individual RCT (with narrow condence interval, see notes below). Systematic review (with homogeneity*) of cohort studies. Individual cohort study (including low quality RCT; e.g. < 80% follow-up). Outcomes research; Ecological studies. Systematic review (with homogeneity*) of case-control studies. Individual case-control study. Case-series (and poor quality cohort and case-control studies) Expert opinion without explicit critical appraisal, or based on physiology, bench research or rst principles.

60%

50%

40%

30%

20%

10%

0% Randomization method Allocation concealment Examiner blinding Accounting for all subjects

Fig. 3. Quality of reporting of randomized controlled trials in periodontology (18). Percentage of studies with adequate method.

Users can add a minus-sign to denote the level of that fails to provide a conclusive answer because of: EITHER a single result with a wide Condence Interval (such that, for example, an absolute risk reduction in an RCT is not statistically signicant but whose condence intervals fail to exclude clinically important benet or harm); OR a Systematic Review with troublesome (and statistically signicant) heterogeneity. Such evidence is inconclusive. *A systematic review that is free of worrisome variations (heterogeneity) in the directions and degrees of results between individual studies. Not all systematic reviews with statistically signicant heterogeneity need be worrisome, and not all worrisome heterogeneity need be statistically signicant. As noted above, studies displaying worrisome heterogeneity should be tagged with a at the end of their designated level. Poor quality cohort study: one that failed to clearly dene comparison groups and/or failed to measure exposures and outcomes in the same (preferably blinded), objective way in both exposed and nonexposed individuals and/or failed to identify or appropriately control known confounders and/or failed to carry out a sufciently long and complete follow-up of patients. Poor quality case-control study: one that failed to clearly dene comparison groups and/or failed to measure exposures and outcomes in the same (preferably blinded), objective way in both cases and controls and/or failed to identify or appropriately control known confounders.

Improving the quality of reporting of clinical research in periodontology

The adequacy of reporting of clinical research is crucial if the reader is to evaluate the quality and

possible impact of studies. The importance of several of the quality issues that we have described has not been thoroughly appreciated until relatively recently. Therefore, it is unfair to judge the past from the standpoint of current knowledge. In addition, the pressure on page numbers in paperbased journals can restrict detail. Hopefully this aspect will be alleviated by initiatives in electronic publication. Guidelines are available to help the publication of clinical research. These guidelines are well accepted by high impact biomedical journals and offer guidance not only to authors but also to editors and reviewers. These guidelines include CONSORT (Consolidated Standards of Reporting Trials) for reporting randomized controlled trials and STARD (Standards for Reporting of Diagnostic Accuracy) for reporting studies on diagnostic tests (http://consortstatement.org/). In addition, three guidelines for reporting systematic reviews are available: QUOROM (Quality of Reporting of Meta-analyses) (http:// consort-statement.org/), MOOSE (Meta-analysis Of Observational Studies in Epidemiology) (27), and QUADAS (Quality Assessment of studies of Diagnostic Accuracy included in Systematic reviews) (29). For clarication, it should be remembered that systematic reviews are termed meta-analyses by some in North America, whereas the term meta-analysis is usually reserved only for the statistical combining of data which may or may not be part of a systematic review. The format of these guidelines is similar. Each presents a checklist of items for incorporation into the research report. The selection of items is evidence-based as far as possible and otherwise derived

20

Evidence-based periodontology

(a)
PAPER SECTION And topic TITLE & ABSTRACT INTRODUCTION Background METHODS Participants Interventions Item Description Reported on Page #

2 3 4

How participants were allocated to interventions (e.g., "random allocation", "randomized", or "randomly assigned"). Scientific background and explanation of rationale. Eligibility criteria for participants and the settings and locations where the data were collected. Precise details of the interventions intended for each group and how and when they were actually administered. Specific objectives and hypotheses. Clearly defined primary and secondary outcome measures and, when applicable, any methods used to enhance the quality of measurements (e.g., multiple observations, training of assessors). How sample size was determined and, when applicable, explanation of any interim analyses and stopping rules. Method used to generate the random allocation sequence, including details of any restrictions (e.g., blocking, stratification). Method used to implement the random allocation sequence (e.g., numbered containers or central telephone), clarifying whether the sequence was concealed until interventions were assigned. Who generated the allocation sequence, who enrolled participants, and who assigned participants to their groups. Whether or not participants, those administering the interventions, and those assessing the outcomes were blinded to group assignment. When relevant, how the success of blinding was evaluated. Statistical methods used to compare groups for primary outcome(s). Methods for additional analyses, such as subgroup analyses and adjusted analyses. Flow of participants through each stage (a diagram is strongly recommended). Specifically, for each group report the numbers of participants randomly assigned, receiving intended treatment, completing the study protocol, and analyzed for the primary outcome. Describe protocol deviations from study as planned, together with reasons. Dates defining the periods of recruitment and follow-up. Baseline demographic and clinical characteristics of each group. Number of participants (denominator) in each group included in each analysis and whether the analysis was by intention-to-treat. State the results in absolute numbers when feasible (e.g., 10/20, not 50%). For each primary and secondary outcome, a summary of results for each group, and the estimated effect size and its precision (e.g. 95% confidence interval). Address multiplicity by reporting any other analyses performed, including subgroup analyses and adjusted analyses, indicating those prespecified and those exploratory. All important adverse events or side effects in each intervention group. Interpretation of the results, taking into account study hypotheses, sources of potential bias or imprecision and the dangers associated with multiplicity of analyses and outcomes. Generalisability (external validity) of the trial findings. General interpretation of the results in the context of current evidence.

Objectives Outcomes

5 6

Sample size

Randomization -sequence generation Randomization -allocation concealment Randomization -Implementation Blinding (masking)

10

11

Statistical methods

12

RESULTS Participant flow

13

Recruitment Baseline data Numbers analyzed

14 15 16

Outcomes and estimation

17

Ancillary analyses

18

Adverse events DISCUSSION Interpretation

19 20

Generalisability Overall evidence

21 22

Fig. 4. a) CONSORT Checklist of items to include when reporting a randomized trial. b) CONSORT Flow chart. Available from: http://www.consort-statement.org/.

21

Needleman et al.

(b)
Enrollment

Assessed for eligibility (n= ... )

Excluded (n = ... )
Not meeting inclusion criteria (n = ... ) Refused to participate Other reasons (n = ... ) Randomized (n = ... ) Allocated to intervention (n = ... ) Received allocated intervention (n = ... ) Did not receive allocated intervention (give reasons) (n = ... ) Allocated to intervention (n = ... ) Received allocated intervention (n = ... ) Did not receive allocated intervention (give reasons) (n = ... )

(n = ... )

Allocation

though deliberate deception is always a possibility, the majority of problems that arise are in fact unintentional. Most methodological errors may be classied as being the result of bias, confounding, or chance. Therefore, for the purpose of this chapter, quality will be discussed in relation to these methodological issues. Other aspects of study conduct may well be critical to the validity of a study but will not be considered in this chapter as they will be specic for a particular study. Such factors could include how well treatment or supportive maintenance was provided.

Bias
Bias is a systematic error. It leads to results which are consistently wrong in one or another direction. Bias leads to an incorrect estimate of the effect of a risk factor or exposure (e.g. smoking) on the development of a disease or outcome of interest (e.g. response to periodontal therapy). The observed effect will be either above or below the true value. Many types of bias have been identied, however, the main types relate to: how subjects were selected for inclusion in a study (selection bias); provision of care (performance bias); assessment of outcomes (detection/measurement bias); occurrence and handling of patient attrition (attrition bias). Selection bias occurs when there is a systematic difference between the characteristics of the subjects selected for a study and the characteristics of those who were not. For instance, selection bias will often occur with volunteers (self-selection bias). People who volunteer to participate in a study tend to be different from the general population. Similarly, it is important to consider whether people might have selectively withdrawn from the study before its completion (attrition bias). They may have withdrawn at random, or because of some factor related to the study, e.g. the treatment they were receiving was ineffective or uncomfortable in comparison with the alternative treatment. It is necessary to decide whether the results of the investigation were likely to have been compromised if one group of subjects had, on average, a shorter follow-up as a result of more people dropping-out. The avoidance of selection bias is a major concern in the design of case-control studies. In this type of study it is essential to ensure that controls are representative of the population from which the cases originated. Suppose a group of researchers is con-

Follow up

Lost to follow up (n = ... ) (give reasons) Discontinued intervention (n = ... ) (give reasons)

Lost to follow up (n = ... ) (give reasons) Discontinued intervention (n = ... ) (give reasons) Analysed (n = ... ) Excluded from analysis (give reasons) (n = ... )

Analysis

Analysed (n = ... ) Excluded from analysis (give reasons) (n = ... )

Fig. 4. Continued.

by a Delphi approach to consensus. In addition to the checklist, a chart is used to illustrate the ow of patients through the study. The checklist and chart for CONSORT are illustrated in Fig. 4a, b. The checklist should accompany the manuscript in its journal submission but not be part of the nal paper. The intention with the chart, however, is that it should be published as part of the paper. Whilst the checklist has numerous items, each can be concisely addressed and is unlikely to be the main cause for excessive length of a publication. At the time of writing, oral health journals that have adopted CONSORT as editorial policy and their dates of adoption are: British Dental Journal (1999), Journal of Orthodontics (2000), International Journal of Endodontics (2003) and Journal of Dental Research (2004). The British Dental Journal is the only one that has adopted QUOROM (2002).

What should be appraised?

Given that some evidence is better than other evidence, it seems reasonable to place greater emphasis on good than on poor quality evidence when making clinical decisions. The problem arises as to how exactly we decide what constitutes good quality evidence. This process is critical appraisal. The validity of published evidence is potentially affected by the quality of every stage of the experimental process from aims and objectives, through design, execution, analysis, interpretation, and nally publication. Al-

22

Evidence-based periodontology

ducting a case-control study to assess the effect of cigarette smoking on the development of aggressive periodontitis. In our hypothetical example, cases are patients referred to a dental hospital with aggressive periodontitis and controls are non-dental patients admitted to a nearby hospital with chronic bronchitis. A standard questionnaire is administered to both cases and controls that includes questions on lifetime smoking habits. The researchers may nd no evidence from this study of an association between cigarette smoking and aggressive periodontitis. Can we accept this conclusion? The problem with this study is that the choice of controls is biased, as the prevalence of smoking among patients admitted with chronic bronchitis is likely to be much higher than among the general population resident in the catchment area of the hospitals from which the cases and controls originated. Consequently, the strength of the association between smoking and aggressive periodontitis will most likely be under-estimated in this study. Randomized controlled trials are less likely to be affected by selection bias if the randomization is properly conducted. Randomization is a two-stage process. The rst stage is the generation of a true random sequence. Typically, this is achieved through computer software or a random number table. Whilst a tossed coin is theoretically acceptable, we suggest using a method that can be audited later on for the purposes of quality assessment, such as a computer generated list. The second stage of randomization is less well understood or carried out. Once the random sequence is generated, it must be concealed from those selecting patients for a study until the individual has been recruited into the trial. If not, despite the sequence being random, the researcher will be aware of whether the patient will be entered into test or control group. This knowledge provides the opportunity for selection bias, whether intentional or not. This second stage of randomization is termed allocation concealment (see Fig. 5 for an outline of this process). The key question to ask is, Was the recruitment of patients into a trial entirely unpredictable with respect to test or control group? Performance bias occurs when different study groups do not receive therapy in the same fashion or to the same standard. This may occur if the people providing the therapy are aware of which groups the participants have been allocated to. Depending on the nature of the investigation, it may be either a relatively simple or a difcult task to ensure that

Step 1 Generate a true random sequence - computer generated is best - tossed coin is acceptable Step 2 Allocation concealment - conceal the sequence for study recruitment - sequentially numbered truly opaque envelopes/drug containers, centrally kept randomization accessed by telephone, e.g. pharmacy

Fig. 5. The two stages of randomization.

therapists remain masked (blinded) to the treatment allocation. The use of placebo, where appropriate, greatly facilitates masking; placebo controlled trials are usually easy to organize in such a way as to leave the therapist masked to the treatment allocation. However, if the interventions to be compared are quite dissimilar in their delivery (e.g. surgical vs. nonsurgical therapy), then masking becomes considerably more challenging. Under these circumstances the best available option might be to ensure that the therapist remains masked until the last possible moment to ensure that all therapy prior to that point has been undertaken as even-handedly as possible. So, for example, in a split-mouth study comparing scaling and root planning vs. scaling and root planning plus an adjunctive locally delivered antimicrobial, it might be possible to complete the mechanical therapy at all appropriate sites prior to the therapist nding out which particular sites are to receive the adjunctive therapy. However, the risk of carry-over effects of the local antibiotic affecting the scaling and root planning-only sites should not be ignored. Measurement (information) bias occurs when the measurements of exposure and/or outcome are not valid (i.e. they do not measure correctly what they are supposed to measure). Errors in measurement may be introduced by the observer (observer bias), by the study individual (responder bias), or by the instruments (instrument bias) used to make the measurements (e.g. a badly designed questionnaire). As a result of measurement errors, study participants will be misclassied in relation to their exposure and/or outcome status. This misclassication has particularly serious implications if the errors in exposure measurement are related to the participants outcome status. Ideally the person undertaking the examination should be blinded

23

Needleman et al.

(masked). This is more important for measures in which there is the potential for subjectivity (e.g. pocket depth, colour change) than for objective measures (e.g. tooth loss). Bias is a consequence of defects in the design or execution of a study. Bias cannot be controlled during the statistical analysis of the data and cannot be eliminated by increasing the size of the study. Publication bias Publication bias refers to the greater likelihood of publication of studies with positive results than those with neutral or negative results (5). The risk with this type of bias is that interventions appear to perform better than they will in clinical practice. For instance, publication bias might mean that although several studies were published and the data available to be included in a meta- analysis, a larger number of studies were actually conducted but not published. Of these missing studies, some may show no difference between the intervention group and the control group, or even the control group performing better. If these additional studies had been published, the results of the meta-analysis could have been different. Therefore, the sample of published studies is a biased sample and does not represent the complete population of all research on this question. Graphic and formal statistical tests are available to investigate publication bias but need approximately 10 or more studies to have adequate power. Figure 6 illustrates this situation using hypothetical data.

Confounding
Confounding is a term that describes the situation where an estimate of the association between an exposure and the disease is mixed up with the real effect of another exposure on the same disease, the two exposures being correlated. It is a difcult concept that may be illustrated with the help of the following example. Suppose we nd that coffee drinkers have a poorer response to periodontal therapy than noncoffee drinkers. Does it mean that coffee drinking affects the response to therapy? The problem here is that there is an alternative explanation. Smoking may be an independent risk factor for poor treatment response and it is possible that people who drink coffee are more likely to smoke than those who do not. Perhaps the observed association is actually due to smoking habits, not coffee drinking (Fig. 7). Age and sex are the most common confounding variables in health-related studies because these two variables are not only associated with most exposures

we are interested in, such as diet, smoking habits, health beliefs, etc., but are also independent risk factors for many diseases. Confounding can be dealt with at the design stage of an investigation by: Randomization By randomly allocating subjects to study groups it is hoped that confounders are distributed equally between the groups. This is usually the most effective way of minimizing the problem of confounding. If randomization is done properly, it has the advantage that it controls for both known and unknown confounders provided the sample size is sufciently large. Restriction This limits participation in a study to specic groups which are similar to each other with respect to the confounder (e.g. if smoking is likely to be a confounder then only nonsmokers will be included in the study). Matching This selects comparison groups with similar backgrounds (e.g. nonsmokers are matched with other nonsmokers, while smokers are matched with other smokers). Confounding can also be controlled for in the analysis by: Stratication Here the strength of the association is measured separately in each well-dened subgroup (e.g. in the smokers and the nonsmokers separately). The results are then pooled together using basic statistical techniques to obtain an overall summary measure of the association adjusted or controlled for the effects of the confounder. Statistical modelling These are more sophisticated mathematical techniques that can simultaneously take into consideration the effects of several possible confounders that have been recorded by the investigators. It is only possible to control for confounders in the analysis if data on them were collected during the study. Obviously, the extent to which confounding can be controlled for will depend on the accuracy of these data. However, in some situations it may be virtually impossible to gain complete and accurate information on confounders. Some confounders may be so difcult to assess that even attempting to adjust for them in a statistical model will not completely control for their effect. For example, Hujoel et al. have argued that the confounding effect of smoking is virtually impossible to measure with sufcient precision in studies that attempt to look at the association between periodontal diseases and systemic health and that such studies may only provide valid results if they are restricted to nonsmokers (10).

24

Evidence-based periodontology
(a)
study 1 study 2 study 3 study 4 study 5 study 6 study 7 study 8 study 9 study 10 study 11 study 12 study 13 study 14 study 15 Combined 2
Favours control Favours intervention

(b)
50 45 40 35
1/standard error

30 25 20 15 10 5 0

1 mean

.4

.2

0 mean difference

.2

.4

(c)
study 1 study 2 study 3 study 1 study 2 study 4 study 3 study 5 study 6 study 4 study 7 study 8 study 9 study 5 study 10 study 11 study 12 study 13 study 14 study 15 Combined

(d)
50 45 40 35
1/standard error

30 25 20 15 10 5 0

2
Favours control

0 mean
Favours intervention

.4

.2

0 mean difference

.2

.4

Fig. 6. Illustration of publication bias. a) Forest plot showing the results of a meta-analysis of 15 studies. A very small improvement is indicated in favour of the test intervention since the diamond shape (representing the 95% condence interval for the pooled result) does not cross the zero (no-effect) line. b) Funnel plot for these studies. In the absence of publication bias, it is anticipated that the plot would form a funnel shape. As no funnel

shape was produced, this indicates the possibility of missing studies. These studies would be expected to produce data points that lie somewhere within the shaded area. c) Another Forest plot, including the data from these extra, previously missing studies. The 95% condence interval for the pooled result now crosses the line of no effect, indicating no evidence that the intervention is any more effective than the control.

spurious association
Coffee drinking Poor treatment response

association Smoking habits (confounder)

risk factor

Fig. 7. An example of confounding.

Chance
Chance (sampling error) plays a role in most studies of humans since it is rarely if ever possible to include an entire population in an investigation. We therefore attempt to infer information about the population on the basis of information obtained from representative

samples drawn from that population. The extent to which the sample results reect the likely result in the population is assessed by performing statistical signicance tests and, more importantly, by calculating condence intervals. A proper discussion of these methods is beyond the scope of this chapter, but in general, studies with small sample sizes will be more prone to sampling error and will provide less robust estimates than studies with larger samples.

Interpretation
It is worth noting that authors may also fail to interpret their experimental results correctly. So,

25

Needleman et al.

even if the study has been well conducted and appropriately analyzed, there is still the potential to draw incorrect conclusions from the results.

How to critically appraise?

When appraising quality it is necessary to consider those factors that may affect the outcome of a study. These will inevitably vary according to both the topic of the original research and the study designs employed, so it is not possible to devise a single system that will be appropriate for every occasion. As a general rule, the aforementioned domains of bias, confounding and chance will all have to be appraised.

Some reviewers have attempted to devise composite scales that give scores for the various quality domains (11). These scores are then summed to an overall summary measure for the study as a whole. There are problems with this approach. Many quality items may not be based on empirical evidence and the scores attached to each item will inevitably be subjective. It is also doubtful whether a single summary score is likely to provide an adequate overall assessment of the quality of a particular study. When different composite scales are applied to the same studies, differing scores and rankings may occur. For these reasons, composite scales have largely gone out of favour. An alternative approach is to appraise each quality component separately (12).

Table 5. Quality assessment checklist for randomized controlled trials in periodontology used by Montenegro et al. (18)
Item Randomization Classication Adequate Denition If generated by random number table (computer generated or not); tossed coin; and shufed cards. Study refers to randomization but either does not adequately explain the method or no method was reported. Methods include alternate assignment, hospital number, and odd/even birth date. Methods included central randomization (e.g. by telephone to a pharmacy or trial ofce), pharmacy sequentially numbered/ coded containers, and sequentially numbered opaque envelopes. If the study referred to allocation concealment but either did not adequately explain the method or no method was reported. Involved methods where randomization could not be concealed, such as alternate assignment, hospital number, and odd/even birth date.

Unclear

Inadequate Allocation concealment Adequate

Unclear

Inadequate

Blinding of patient, caregiver, and examiner were considered separately

Recorded as adequate, inadequate, unclear, or for examiner blinding, not applicable if the study design precluded the possibility of blinding. Were all patients who entered the trial properly accounted for at the end? Where dropouts occurred, the use of analyses to allow for losses (such as intention to treat) was noted.

Withdrawals and drop outs

26

Evidence-based periodontology

For rigorous systematic reviews, independent reviewers usually undertake quality appraisal in duplicate and checklists are frequently employed for this purpose. Two such checklists that have been used previously are reproduced here as examples (Tables 5 and 6) (7, 18). These checklists are based on a combination of factors that have been shown empirically to affect quality (such as allocation

Table 6. Quality assessment checklist for systematic reviews in dentistry used by Glenny et al. (7)
Question (possible categories) A. B. C. Did review address a focused question? (yes, no, cant tell) Did authors look for appropriate papers? (yes, no, cant tell) Do you think authors attempted to identify all relevant studies? (yes, no, cant tell) Search for published and unpublished literature (yes, no, cant tell) Were all languages considered? (yes, no, cant tell) Was any hand searching carried out? (yes, no, cant tell) Was it stated that the inclusion criteria were carried out by at least two reviewers? (yes, no, cant tell) Did reviewers attempt to assess the quality of the included studies? (yes, no) If so did they include this in the analysis? (yes, no, cant tell, not applicable) Was it stated that the quality assessment was carried out by at least two reviewers? (yes, no, not applicable) Are the results given in a narrative or pooled statistical analysis? (narrative, pooled, not applicable) If the results have been combined was it reasonable to do so? (yes, no, cant tell, not applicable) Are the results clearly displayed? (yes, no, not applicable) Was an assessment of heterogeneity made and reasons for variation discussed? (yes, no, not applicable) Were results of review interpreted appropriately? (yes, no, cant tell, not applicable)

D. E. F. G.

concealment) and also topic specic factors deemed important by the reviewers. Other checklists cover a broad range of types of research and can be found on the excellent Critical Appraisal Skills Programme website (http://www.phru.nhs.uk/casp/ appraisa.htm). The use of checklists with objective criteria helps to safeguard the quality of the quality appraisal process itself. The process of devising the checklist helps to ensure that all relevant quality issues are included in the assessment. Written, piloted checklists reduce, but can never completely eliminate individual subjectivity in decisions. Having a written list means that it is more likely that the quality assessors will be both consistent and repeatable. The results of the quality appraisal are used to assess the value of the evidence and to aid clinicians and reviewers in their efforts to place the evidence into context. This might be a part of the formal process of undertaking a systematic review or the informal act of reading and assessing recently published literature as part of everyday periodontal practice.

Conclusions
The principles of evidence-based healthcare provide structure and guidance to facilitate the highest levels of patient care. There are numerous components to evidence-based periodontology including the production of best available evidence, the critical appraisal and interpretation of the evidence, the communication and discussion of the evidence to individuals seeking care and the integration of the evidence with clinical skills and patient values. This volume of Periodontology 2000 is mainly concerned with the rst component, i.e. the generation of best evidence and, alone, is not enough to practise evidence-based healthcare. However, an understanding of the principles should help to underpin the latter aspects. Evidence-based healthcare is not an easier approach to patient management, but should provide both clinicians and patients with greater condence and trust in their mutual relationship.

H.

I. J.

K.

L.

M. N.

References
1. Alderson P, Green S, Higgins JPT, eds. Cochrane Reviewers Handbook 4.2.2 [updated. ]. In: The Cochrane Library, Issue 1. Chichester: John Wiley & Sons, 2004.

O.

27

Needleman et al.
2. American Academy of Periodontology. Proceedings of the 1996 World Workshop in Periodontics. Lansdowne, Virginia, July 13-17, 1996. Ann Periodontol 1996: 1: 1947. 3. Antczak-Bouckoms A, Tang J, Chalmers TC. Quality assessment of randomized controlled trials in dental research. J Periodontal Res 1986: 21: 305314. 4. EBM Working Group. Evidence-based medicine. JAMA 1992: 268: 24202425. 5. Egger M, Davey-Smith G, Altman DG. Systematic Reviews in Health Care, 2nd edn, London: BMJ Books, 2001. 6. Esposito M, Coulthard P, Worthington H, Jokstad A. Quality assessment of randomized controlled trials of oral implants. Int J Oral Maxillofac Implants 2001: 16: 783792. 7. Glenny AM, Worthington H, Esposito M, Coulthard P. The assessment of systematic reviews in dentistry. Eur J Oral Sci 2003: 111: 8592. 8. Gray JAM. Evidence-based Healthcare. Edinburgh: Churchill Livingstone, 1997. 9. Herrera D, Sanz M, Jepsen SJ, Needleman IG, Roldan S. A systematic review on the effect of systemic antimicrobials as an adjunct to scaling and root planning in periodontitis patients. J Clin Periodontol 2002: 29 (Suppl 3): 136159. 10. Hujoel PP, Drangsholt M, DeRouen TA. Periodontitis systemic disease associations in the presence of smoking causal or coincidental? Periodontol 2000 2002: 30: 5160. 11. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996: 17: 112. 12. Juni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials. Br Med J 2001: 323: 4246. 13. Khan KS, ter Riet G, Glanville J, Sowden AJ, Kleijnen J. Undertaking systematic reviews of research on effectiveness. Http://www.York.Ac.Uk/Institute/crd/report4.htm [2nd], 2001. 14. Labriola A, Needleman I, Moles DR. Systematic review of the effect of smoking on nonsurgical periodontal therapy. Periodontol 2000 2005: 37: 124137. 15. Mathews JR. Quantication and the Quest for Medical Certainty. Princeton: Princeton University Press, 1995. 16. Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A, et al. Assessing the quality of reports of randomized trials: implications for the conduct of meta-analyses. Health Technol Assess 1999: 3: 198. 17. Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, et al. Does quality of reports of randomized trials affect estimates of intervention efcacy reported in meta-analyses? Lancet 1998: 352: 609613. 18. Montenegro R, Needleman I, Moles D, Tonetti M. Quality of RCTs in periodontology a systematic review. J Dent Res 2002: 81: 866870. 19. Needleman IG. A guide to systematic reviews. J Clin Periodontol 2002: 29 (Suppl. 3): 69. 20. Needleman IG. Introduction to evidence based dentistry. In: Clarkson, J, Harrison, JE, Ismail, AI, Needleman, IG, Worthington, H, eds. Evidence Based Dentistry for Effective Practice. London: Martin Dunitz, 2003: 117. 21. Needleman IG, Giedrys-Leeper E, Tucker RJ, Worthington HV. Guided tissue regeneration for periodontal infra-bony defects (Cochrane Review). The Cochrane Library. Oxford: Update Software. http://www.update-software.com/clibhome/ clib.htm, 2001. 22. Needleman IG, Giedrys-Leeper E, Tucker RJ, Worthington HV. Guided tissue regeneration for infrabony defects A systematic review. J Periodontal Res 2002: 37: 380 388. 23. Rangachari PK. Evidence-based medicine: old French wine with a new Canadian label? J R Soc Med 1997: 90: 280284. 24. Richards D. Not all evidence is created equal so what is good evidence? Evid Based Dent 2003: 4: 1718. 25. Roccuzzo M, Bunio M, Needleman I, Sanz M. Periodontal plastic surgery for treatment of localized gingival recessions A systematic review. J Clin Periodontol 2002: 29 (Suppl 3): 178194. 26. Schultz KF, Chalmers I, Hayes RJ, Altman D. Empirical evidence of bias: Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995: 273: 408412. 27. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000: 283: 20082012. 28. Swales J. The troublesome search for evidence: three cultures in need of integration. J R Soc Med 2000: 93: 402 407. 29. Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 2003: 3: 25.

28