0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Special Communication |

Challenges in Applying the Results of Clinical Trials to Clinical Practice

Marco A. Zarbin, MD, PhD1
[+] Author Affiliations
1Institute of Ophthalmology and Visual Science, Rutgers New Jersey Medical School, Rutgers–The State University of New Jersey, Newark
JAMA Ophthalmol. 2016;134(8):928-933. doi:10.1001/jamaophthalmol.2016.1266.
Text Size: A A A
Published online

The relevance of clinical trial results to clinical practice hinges on 2 critical questions: Will the results be replicated in one’s practice, and Are the results clinically important? The answers to the following 5 questions may help one determine how relevant a study result is to clinical practice. First, have steps been taken to minimize bias (eg, masking, randomization)? Second, is the result likely due to the treatment (vs confounding factors)? Third, is the result unlikely to be due to chance? Fourth, is the study population representative of your patients? Fifth, is the totality of the evidence consistent across studies? To determine if a study result is likely to be clinically important, consider a 3-step approach. In step 1, decide, a priori, what a clinically meaningful difference between 2 treatments would be to define regions of beneficial, harmful, and trivial outcomes. In step 2, determine whether the CIs around the average outcome include the range of beneficial outcomes and lie outside the range of harmful outcomes. In step 3, determine the proportion of patients achieving a clinically meaningful benefit. If the CIs mostly include the range of beneficial outcomes and lie outside the range of clinically harmful outcomes and if a substantial proportion of patients achieve a clinically meaningful benefit, then the intervention is probably clinically important. Application of clinical trial results to clinical practice requires critical analysis of the extant literature as well as good clinical judgment.

Figures in this Article

Sign in

Purchase Options

• Buy this article
• Subscribe to the journal
• Rent this article ?

Figures

Place holder to copy figure label and caption
Figure 1.
A 3-Step Approach to Assessing Clinical Importance

A, In this example, 4 Early Treatment Diabetic Retinopathy Study (ETDRS) letters would be a clinically meaningful difference. B, If the CI lies entirely within the beneficial range of outcomes or even extends into the trivial range, the result may be both clinically important and statistically significant. If the CI lies mostly within the beneficial range but extends into the trivial range, even crossing the value of 0 (no difference between the 2 treatments), then the result may be clinically important, but it is not statistically significant if it crosses 0. If the CI lies mostly in the trivial range, crosses the value 0, and extends into the beneficial range only slightly, the treatment is not likely to be clinically important. It is not statistically significant. If the CI lies entirely within the trivial range but does not include the value 0, then the result is statistically significant but may not be clinically important. Therefore, some clinically meaningful results might not be statistically significant, and some statistically significant results might not be clinically meaningful. C, Shown is the distribution overlap of 2 populations associated with different Cohen d coefficients. Cohen d is a quantity that indicates how large the treatment difference between 2 groups is relative to the standard deviation of the samples (eAppendix, E in the Supplement). Cohen d of 0.2 indicates a small treatment effect, 0.5 a moderate treatment effect, and 0.8 a large treatment effect.26,27 When Cohen d is 0.2 and the data are normally distributed, the outcome of 58% of the treatment group is greater than the mean outcome of the control group, and the outcomes of 92% of the treatment and control groups overlap. To have 1 more favorable outcome in the treatment group vs the control group (the number needed to treat [NNT]), one needs to treat approximately 17 patients. When Cohen d is 0.8, the outcome of 79% of the treatment group is greater than the mean outcome of the control group, but the outcomes of 69% of the 2 groups overlap, and the NNT is approximately 4. A Cohen effect size of 0.8 is generally accepted as a clinically meaningful outcome. To have less than 50% of the treatment and control group outcomes overlapping, Cohen d must be at least 1.4, in which case 92% of the treatment group has an outcome greater than the mean outcome of the control group, and the NNT is approximately 2. A web-based calculator for these estimations and graphs is available at http://rpsychologist.com/d3/cohend (created by Kristoffer Magnusson). Parts A and B are adapted from the study by Batterham and Hopkins.24 C is generated from http://rpsychologist.com/d3/cohend under the terms of a Creative Commons license (http://creativecommons.org/licenses/by/4.0).

aStatistical significance is misleading.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.
Assessing Clinical Importance in the Comparison of Age-Related Macular Degeneration Treatments Trials23 (CATT)

Shown is an analysis of CATT data regarding monthly vs as-needed injection posology for choroidal new vessels in patients with age-related macular degeneration. Depending on the definition of a minimally beneficial outcome, the CI can indicate that the result falls mostly within the trivial range of outcomes as shown here (where the minimum beneficial outcome is posited to be a gain of 4 Early Treatment Diabetic Retinopathy Study [ETDRS] letters) or mostly within the beneficial range if the minimum beneficial outcome is a gain of 2 ETDRS letters (not shown). Adapted from the study by Batterham and Hopkins.24

Graphic Jump Location

Tables

References

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

1,916 Views
0 Citations
×

Sign in

Purchase Options

• Buy this article
• Subscribe to the journal
• Rent this article ?

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles
Jobs
brightcove.createExperiences();