We're unable to sign you in at this time. Please try again in a few minutes.
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Original Investigation |

Expert Diagnosis of Plus Disease in Retinopathy of Prematurity From Computer-Based Image Analysis

J. Peter Campbell, MD, MPH1; Esra Ataer-Cansizoglu, PhD2; Veronica Bolon-Canedo, PhD3; Alican Bozkurt, MSc2; Deniz Erdogmus, PhD2; Jayashree Kalpathy-Cramer, PhD4; Samir N. Patel, BS5; James D. Reynolds, MD6; Jason Horowitz, MD7; Kelly Hutcheson, MD8; Michael Shapiro, MD9; Michael X. Repka, MD, MBA10; Phillip Ferrone, MD11; Kimberly Drenser, MD12; Maria Ana Martinez-Castellanos, MD13; Susan Ostmo, MS1; Karyn Jonas, RN14; R.V. Paul Chan, MD14; Michael F. Chiang, MD1,15 ; for the Imaging and Informatics in ROP (i-ROP) Research Consortium
[+] Author Affiliations
1Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland
2Cognitive Systems Laboratory, Northeastern University, Boston, Massachusetts
3Department of Computer Science, Universidade da Coruña, A Coruña, Spain
4Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown
5Department of Ophthalmology, Weill Cornell Medical College, New York, New York
6Department of Ophthalmology, Ross Eye Institute, State University of New York at Buffalo
7Department of Ophthalmology, Columbia University, New York, New York
8Department of Ophthalmology, Sidra Medical and Research Center, Doha, Qatar
9Retina Consultants, Chicago, Illinois
10Wilmer Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland
11Long Island Vitreoretinal Consultants, Great Neck, New York
12Associated Retinal Consultants, Oakland University, Royal Oak, Michigan
13Asociacion para Evitar la Ceguera en Mexico, Mexico City, Mexico
14Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago
15Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland
JAMA Ophthalmol. 2016;134(6):651-657. doi:10.1001/jamaophthalmol.2016.0611.
Text Size: A A A
Published online

Importance  Published definitions of plus disease in retinopathy of prematurity (ROP) reference arterial tortuosity and venous dilation within the posterior pole based on a standard published photograph. One possible explanation for limited interexpert reliability for a diagnosis of plus disease is that experts deviate from the published definitions.

Objective  To identify vascular features used by experts for diagnosis of plus disease through quantitative image analysis.

Design, Setting, and Participants  A computer-based image analysis system (Imaging and Informatics in ROP [i-ROP]) was developed using a set of 77 digital fundus images, and the system was designed to classify images compared with a reference standard diagnosis (RSD). System performance was analyzed as a function of the field of view (circular crops with a radius of 1-6 disc diameters) and vessel subtype (arteries only, veins only, or all vessels). Routine ROP screening was conducted from June 29, 2011, to October 14, 2014, in neonatal intensive care units at 8 academic institutions, with a subset of 73 images independently classified by 11 ROP experts for validation. The RSD was compared with the majority diagnosis of experts.

Main Outcomes and Measures  The primary outcome measure was the percentage of accuracy of the i-ROP system classification of plus disease, with the RSD as a function of the field of view and vessel type. Secondary outcome measures included the accuracy of the 11 experts compared with the RSD.

Results  Accuracy of plus disease diagnosis by the i-ROP computer-based system was highest (95%; 95% CI, 94%-95%) when it incorporated vascular tortuosity from both arteries and veins and with the widest field of view (6–disc diameter radius). Accuracy was 90% or less when using only arterial tortuosity and 85% or less using a 2– to 3–disc diameter view similar to the standard published photograph. Diagnostic accuracy of the i-ROP system (95%) was comparable to that of 11 expert physicians (mean 87%, range 79%-99%).

Conclusions and Relevance  Experts in ROP appear to consider findings from beyond the posterior retina when diagnosing plus disease and consider tortuosity of both arteries and veins, in contrast with published definitions. It is feasible for a computer-based image analysis system to perform comparably with ROP experts, using manually segmented images.

Figures in this Article


Place holder to copy figure label and caption
Figure 1.
Computer-Based Image Analysis

A, Standard published photograph of plus disease. B, A representative fundus image, where the white dashed line demarcates the approximate field of view of the standard published photograph. C, Results of manual segmentation of retinal vessels. D, Circular crops with the radius of 1 to 6 disc diameters for subsequent analysis.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.
Examples of Study Images With Reference Standard Diagnosis of Plus Disease

A, Standard photograph. B, Representative wide-angle example of plus disease. C, The same retinal image as in part B, cropped to the field of view of the standard photograph. D, Study image with plus disease, with venous tortuosity out of proportion to arterial tortuosity.

Graphic Jump Location




Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.


Some tools below are only available to our subscribers or users with an online account.

1 Citations

Sign in

Purchase Options

• Buy this article
• Subscribe to the journal
• Rent this article ?

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections