Sensitivity, Specificity, Predictive Value and Efficiency

The sensitivity of a test

A : The number of true negatives detected by the test divided by the number of all true negatives in the population tested. B : The number of true positives detected by the test divided by the number of all true positives in the population tested. C : The number of true positives detected by the test divided by the total number of true positives in the population tested. D : The number of true negatives detected by the test divided by the total number of true negatives in the population tested. E : The number of true negatives detected by the test divided by the total number of true positives in the population tested.

If screening for breast cancer by mammography has a sensitivity of 75% and a specificity of 95% in a certain population, what is the positive predictive value?

A. B. C. D. E.

1/16 5/24 19/24 15/16 There is not enough information to calculate the positive predictive value

A new screening test is to be applied in a population. The test has a sensitivity of 99% and a specificity of 90%.

If the prevalence of the disease is 10 per 1000 population, on average what proportion of those that test positive will truly have the disease? A. B. C. D. E. 1/11 1/10 1/2 9/10 99/109

Sensitivity of the test

You are drawing up a trial of a new screening test. If the threshold of the screening test is increased, which of the following would increase? A. A. A. A. A. Sensitivity Specificity Prevalence Negative predictive value Positive predictive value

Ultrasound scanning at 20 weeks was found to correctly detect 35 of the 50 babies ultimately diagnosed as having a congenital heart defect. However, five of the 500 babies born with no congenital defect were also recorded as problematic when screened.

Which of the following is a correct statement of the properties of ultrasound scanning as a screening test for congenital heart defects? A. B. C. D. E. Positive predictive value = 70%; negative predictive value = 99% Positive predictive value = 70%; negative predictive value = 1% Sensitivity = 70%; specificity = 99% Sensitivity = 70%; specificity = 1% No test properties can be estimated as these are all affected by the prevalence of congenital heart defects in the study population - this figure is not given

The specificity of a test is defined as follows:

A. B. C. D. E.

A : The number of true negatives detected by the test divided by the number of all true negatives in the population tested. B : The number of true positives detected by the test divided by the number of all true positives in the population tested. C : The number of true positives detected by the test divided by the total number of true positives in the population tested. D : The number of true negatives detected by the test divided by the total number of true negatives in the population tested. E : The number of true negatives detected by the test divided by the total number of true positives in the population tested.

A new test is developed for the diagnosis of hepatitis C. Out of 10,000 tests 100 were positive using the current gold standard method for diagnosing hepatitis C, whereas using the new method 150 tests were positive.

What is the positive predictive value of the new test? A. 25% B. 30% C. 66% D. 150% E. 50%

Distinguish Between Trial Types

Superiority trials

Equivalence trials neither any better or any worse Non-inferiority not much worse than the active comparator

Which of the following statements best describes a type-2 error?

A. B. C. D. E.

Risk of a false-positive result Alpha error Risk of detecting a treatment difference when there is none Risk of not detecting a significant difference when there is one None of the above

An unpaired t-test is used to compare the mean height in two populations. A random sample is taken from each population and the difference in sample means is 10 cm, with a 95% confidence interval from 2 cm to 22 cm. If the true means in the two populations are 150 cm and 165 cm, which one of the following statements is correct?

A. B.
C. D. E.

The test shows we can be 95% sure there is a difference There is no difference in height between the two populations
A type-1 error has occurred A type-2 error has occurred A paired t-test should have been used

A type 1 statistical error in a clinical trial means that:

A : patients were not allocated into groups with an appropriate randomisation method B : the null hypothesis is falsely accepted C : the null hypothesis is falsely rejected D : the statistical analysis was incomplete or incorrect E : the statement of the hypothesis to be tested was incomplete or flawed.

A manuscript is submitted to a medical journal regarding a randomised trial in which a new treatment for Clostridium difficile diarrhea is compared with an established treatment. A reviewer states that they are concerned that there might be type 2 statistical error. What does this mean?

A : That the method of statistical analysis used is inappropriate B : That the study has shown a difference between the treatments that is statistically significant but which is unlikely to be clinically significant C : That the study claims to find a difference that does not really exist, i.e.. the result is a statistical fluke D : That the data is skewed (not normally distributed) and analysis should have used non-parametric rather than parametric statistical techniques E : That the study claims that there is no difference between the treatments, when in reality the trial was just too small to detect a difference.

Power and Sample size

The power of a statistical test is the probability that the test will reject the null hypothesis when the null hypothesis is actually false (i.e. the probability of not committing a Type II error, or making a false negative decision). The power is in general a function of the possible distributions, often determined by a parameter, under the alternative hypothesis. As the power increases, the chances of a Type II error occurring decrease. The probability of a Type II error occurring is referred to as the false negative rate (). Therefore power is equal to 1 , which is also known as the sensitivity.

In the design of a randomised controlled trial, to what does the power of the study refer?

A.

B.
C. D. E.

The size of treatment difference that the trial can be expected to detect The probability of rejecting the null hypothesis that the treatments have the same effect The chance that a clinically significant difference will be observed The probability of a type-2 error The probability of a statistically significant treatment effect if the true treatment difference is at a prespecified level

You are asked to design a trial for a new anti-hypertensive agent after taking part in the phase 2 studies. You suspect that it has a power to achieve a relative risk reduction of 18% in CV events. You are examining methodology and statistical tests to determine effectiveness of this agent. What is the power of a statistical test?

A. B.

The probability that it will correctly lead to rejection of a false null hypothesis The probability that it will falsely lead to rejection of a true null hypothesis

C.
D. E.

The probability that it will falsely lead to rejection of a false null hypothesis
The probability that it will correctly lead to rejection of a true null hypothesis The sample size needed to detect a significant difference

Sample size determination

Sample size determination is the act of choosing the number of observations to include in a statistical sample. The three most important are: - How accurate you wish to be - How confident you are in the results - What budget you have available

Sample Size Calculations (2005)

20 % reduction in the risk of disease-freesurvival event (hazard ratio 0.80) Two-sided alpha level 0.05 80% power Needed 647 events

Sample Size Calculations (2009, post hoc)

Risk of primary-end point event at least 26.7% reduction 80% chance that the 99% confidence interval would exclude HR of 1

Expressing risk Used for nominal data Use a 2x2 tablehelpful for organizing data in study Expressed as odds ratio, relative risk or hazard ratio

OR vs RR

Assessing Risk

A case-control study investigating the effect of keeping pet birds on the risk of lung cancer enrolled 200 patients presenting with lung cancer and 210 controls who were free of lung cancer. Of the 200 cases, 80 reported a history of at least 6 months exposure to a pet bird; of the 210 controls, 70 reported such an exposure. What was the odds ratio for the development of lung cancer associated with keeping pet birds in this study? A. B. C. D. E. 1 8/7 52/45 6/5 4/3

Which of the following statements is most appropriate regarding the relative risk of a disease?

A. B. C. D. E.

Always lies between zero and one Is always positive Measures the increased (or decreased) risk of the factor when the individual has the disease Measures the risk of the disease in the population Takes the value zero when the risk is equally likely in those exposed and unexposed to the factor of interest

In a cohort study of 1000 non-smokers and 1000 smokers, 450 of the smokers and 50 of the non-smokers developed COPD. What is the relative risk of smoking for COPD?

A. B. C. D. E.

9 1/9 8 1/8 7

In a study of the recurrence of colorectal adenomas, 50 patients were randomised to receive daily aspirin and 50 to receive placebo. After 1 year, at least one adenoma was observed in 15 patients receiving aspirin and 20 patients receiving placebo. What was the relative risk reduction associated with aspirin?

A. B. C. D. E.

10% 25% 30% 40% 75%

In a randomised controlled trial of a new treatment for preventing recurrence of stroke, 1000 patients are randomised to the new treatment and 1000 to standard therapy. A total of 66 patients receiving the new treatment suffered recurrent stroke, compared to 110 in the control arm. What was the relative risk reduction?

A. B. C. D. E.

4.4% 6.6% 11% 40% 60%

In a trial of a new drug compared to standard therapy, the risk of complications in the control arm is 2.4% and the risk of complications in the treatment arm is 1.8%. What is the absolute risk reduction associated with the new treatment? A. B. C. D. E. 0.6% 1.8% 2.4% 25% 33%

In a trial of a new cancer drug, the mortality in the control arm was 12.5% and the relative risk for patients receiving the new treatment was 0.8. How many patients must be treated with the new drug to save one life?

A. B. C. D. E.

8 10 25 40 80

A study is planned into the effects on risk of myocardial infarction of a new lipid modifying agent, and the steering committee states that they plan to include number needed to treat (NNT) as part of the primary manuscript. How is the number needed to treat over the duration of the study calculated?

A. B. C. D. E.

Ratio of absolute to relative risk reduction Ratio of relative to absolute risk reduction Reciprocal of relative risk reduction Reciprocal of absolute risk reduction Number of months of study divided by the absolute risk reduction

A 73-year-old man is admitted to the hospital after suffering a left carotid territory stroke. His blood pressure is 155/90 mmHg and he is in sinus rhythm with a pulse of 65 bpm. On carotid duplex it appears there is a stenosis which is amenable to surgical therapy. He asks you about the risks of surgery versus intensive medical therapy. On reviewing the trials you note that the incidence of stroke in the surgical intervention group was 8%, versus 18% in those who had medical intervention only. How would you calculate the number needed to treat over 2 years with surgery to prevent 1 stroke?

A. B. C. D. E.

100/(18-8) 18/8 100/(18/8) 18-8 100-(18/8)