0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Clinical Sciences |

Interexpert Agreement in the Identification of Macular Location in Infants at Risk for Retinopathy of Prematurity FREE

Michael F. Chiang, MD; Preeti J. Thyparampil, MD; Daniel Rabinowitz, PhD
[+] Author Affiliations

Author Affiliations: Departments of Ophthalmology (Drs Chiang and Thyparampil) and Biomedical Informatics (Dr Chiang), College of Physicians and Surgeons, and Department of Statistics (Dr Rabinowitz), Columbia University, New York, New York.


Arch Ophthalmol. 2010;128(9):1153-1159. doi:10.1001/archophthalmol.2010.199.
Text Size: A A A
Published online

Objective  To characterize variability in the identification of the macular center among retinopathy of prematurity (ROP) experts.

Methods  A printed set of 25 wide-angle retinal images was compiled from infants at risk for ROP using a commercially available camera. Ten recognized ROP experts were asked to mark the macular center on each image. For each image, we measured the distance from the optic disc center to the marked macular center. Distances were standardized by normalizing the horizontal optic disc diameter in each image to 0.93 mm. In images with visible peripheral disease, interexpert agreement on the presence of zone I disease was also determined.

Results  For the image with the least variability among experts, mean (SD) distance from the optic disc to the macular center was 3.69 (0.21) mm (range, 3.13-3.81 mm). For the image with the greatest variability among experts, distance from the optic disc to the macular center was 4.32 (1.19) mm (range, 3.21-7.19 mm). In 7 of 21 images (33%) with visible peripheral disease, there would have been disagreement among experts in the diagnosis of zone I disease based on identification of the macular center. Among the 10 experts, in 17 of 25 images (68%), 1 expert identified the distance between the optic disc and macular center to be greater than 1 SD from the mean.

Conclusion  Significant variability exists among experts in identification of the macular center from wide-angle images, which raises concerns about the reliability of zone I ROP diagnosis.

Figures in this Article

Retinopathy of prematurity (ROP) is a vasoproliferative disease affecting low-birth-weight infants and is a leading cause of childhood blindness worldwide. Major advances in diagnosis and treatment have occurred. For example, the International Classification of ROP established a standard system for describing examination findings, thereby creating a foundation for clinical care and research.1,2 The Cryotherapy for ROP3 (CRYO-ROP) and Early Treatment for ROP4 (ETROP) trials identified criteria for which treatment with cryotherapy or laser photocoagulation was shown to improve visual outcomes.

These studies have shown that identification of zone I ROP is critical for decisions regarding treatment and prognosis. Zone I of the retina is defined as a circle, the radius of which extends from the optic disc center to twice the distance from the optic disc center to the macular center. In the CRYO-ROP study, poor structural outcomes occurred in 78% of zone I eyes treated for threshold disease, whereas poor structural outcomes occurred in only 26% of zone II eyes that were treated.5 The ETROP trial subsequently showed improved outcomes in zone I eyes randomized to early treatment.4 On the basis of these studies, treatment is recommended for type 1 ROP, which is defined as zone I ROP of any stage in the presence of plus disease; zone I, stage 3 disease whether or not plus disease is present; and zone II ROP of stage 2 or 3 in the presence of plus disease. Careful follow-up is recommended for type 2 ROP, which is defined as zone I ROP of stage 1 or 2 without plus disease, or zone II, stage 3 disease without plus disease. Therefore, zone I disease must be detected accurately and reproducibly.

However, zone I diagnosis may be subjective because it requires identification of the macular center with delineation of the appropriate circular radius. Although clinical examination is considered the criterion standard for ROP care, to our knowledge agreement among multiple examiners in recognizing zone I disease has not been systematically studied. This study was designed to characterize variability in the identification of the macular center among 10 recognized experts using a set of wide-angle retinal images from infants at risk for ROP.

This study was approved by the institutional review board at Columbia University. Written informed consent was obtained from all expert participants, and waiver of consent was obtained for the use of deidentified retinal images.

STUDY PARTICIPANTS

A set of 25 wide-angle retinal images was obtained from premature infants during routine ROP care using a commercially available camera (RetCam-II; Clarity Medical Systems, Pleasanton, California). Images were taken of infants between 32 and 38 weeks' postmenstrual age (PMA) and were not annotated with any descriptive information. No images were repeated.

Eligible experts for this study were defined as practicing pediatric ophthalmologists or retina specialists who met at least 1 of the following criteria: having been a principal investigator or certified investigator for the CRYO-ROP or ETROP study, or having published at least 2 peer-reviewed ROP articles. Experts known to the authors as colleagues or through national conferences were invited to participate.

IMAGE INTERPRETATION

Sets of retinal images, 1 for each expert, were printed on high-quality photographic paper (UPC-741; Sony, Tokyo, Japan) at a size of 17.8 cm in width by 12.4 cm in height. Images were presented in a quiet setting, sequentially, and in the same order to each expert by one of us (M.F.C. or P.J.T.). Experts were asked to mark the macular center with an X. To allow for the modification of responses when thought to be warranted by experts, the marks were drawn on transparent adhesive tape, which was placed on the printed images.

Experts were asked whether their institution had a RetCam and about their experience in interpreting these images (experienced, somewhat experienced, or not experienced). Finally, experts were asked to rate each image based on (1) adequacy of photographic quality (adequate, possibly adequate, or inadequate) and (2) confidence with marking (confident, somewhat confident, or not confident).

To simulate a real-world situation, participants were not provided with any references for identification of the macular center or for definition of zone I, although it was assumed that they would be familiar with these standards.

DATA ANALYSIS

For each expert response to each image, we measured the linear distance from the optic disc center to the marked macular center. For images in which peripheral ROP disease was visible, the closest linear distance from the optic disc center to the peripheral disease was measured. To account for differences in magnification among images, all distances were standardized by multiplying by 0.93/(measured optic disc width in millimeters), based on published data showing the mean optic disc width in premature infants to be 0.93 mm.6 In images that did not display the entire optic disc, the location and size of the optic disc were extrapolated using other images taken from the same eyes on the same day. Images were identified in which there would have been disagreement among some experts regarding the presence of zone I disease according to macular center markings and the location of visible peripheral ROP disease.

A 2-way mixed-effects analysis of variance (ANOVA) was performed to characterize the variability in the distance from the optic disc center to the experts' markings. Variability was composed of image-specific, rater-specific, and residual variance components. The ANOVA results were used to estimate the probability that an expert would diagnose zone I ROP as a function of peripheral disease location with the assumptions that rater-specific and residual components of variance were normally distributed, and that there was no systematic bias in responses among the population of raters.

Statistical analysis was performed using spreadsheet (Excel 2007; Microsoft Corp, Redmond, Washington) and statistical software (SPSS 14.0; SPSS Inc, Chicago, Illinois, and SAS; SAS Inc, Cary, North Carolina). Statistical significance was defined as P < .05 by a 2-tailed test.

CHARACTERISTICS OF EXPERT PARTICIPANTS AND RETINAL IMAGES

On the basis of the study definition of expertise, 11 individuals were invited, of whom 10 (91%) consented to participate. Among these 10 experts, 5 (50%) had served as principal investigators or study investigators in the CRYO-ROP or ETROP trials, and all (100%) had coauthored at least 2 peer-reviewed ROP articles. All experts worked at institutions with a RetCam, and all reported that they were experienced in RetCam image review. Four experts (40%) were pediatric ophthalmologists and 6 (60%) were retina specialists.

The Table summarizes characteristics of the 25 wide-angle retinal images. Five images (20%) were taken at 32 to 33 weeks' PMA, 3 (12%) were taken at 34 to 35 weeks' PMA, and 17 (68%) were taken at 36 to 38 weeks' PMA. Twenty-one images (84%) had visible peripheral disease; in all of these images, the nearest distance to visible peripheral disease was greater than twice the mean of experts' marked distances to the macular center.

Table Graphic Jump LocationTable. Characteristics of 25 Retinal Images Evaluated by 10 ROP Expertsa
IMAGE QUALITY AND EXPERT CONFIDENCE RATINGS

Each expert independently reviewed all 25 images, for a total of 250 macular center, image quality, and confidence-rating responses. Image quality was scored as “adequate” for identification of the macular center in 161 of 250 responses (64%), “possibly adequate” in 68 responses (27%), and “inadequate” in 21 responses (8%). Confidence in identifying the macular center was scored as “confident” in 109 of 250 responses (44%), “somewhat confident” in 103 responses (41%), and “not confident” in 38 responses (15%).

IDENTIFICATION OF MACULAR CENTER

Figure 1 shows examples of expert responses, indicating different patterns of discrepancy. Distances from the optic disc center to the marked macular center are displayed graphically in Figure 2. The mean (SD) distance from the optic disc to the macular center was 3.69 (0.21) mm (range, 3.13-3.81 mm) for the image with the best interexpert agreement (image 17 in the Table and Figure 2) and 4.32 (1.19) mm (range, 3.21-7.19 mm) for the image with the worst agreement (image 22 in the Table and Figure 2). When images in which experts were “not confident with marking” were excluded, the subject-specific and residual variance component estimates were smaller. However, there was an insufficient sample size to conclude that restricting attention to images in which experts were confident would result in decreased variability of macular markings.

Place holder to copy figure label and caption
Figure 1.

Representative responses by 10 experts who identified the macular center in 25 posterior pole images. A, Image 17 in the Table and Figure 2, taken at 34 weeks' postmenstrual age (PMA), shows good agreement. B, Image 19 in the Table and Figure 2, taken at 33 weeks' PMA, shows a bimodal-type distribution. C, Image 25 in the Table and Figure 2, taken at 36 weeks' PMA, and D, Image 4 in the Table and Figure 2, taken at 38 weeks' PMA, show distributions along a continuum, as well as disagreement among some experts on presence of zone I disease based on the macular center.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.

Identification of the macular center in 25 posterior pole images by 10 experts, displayed by image. Distances between the macular center and optic disc were measured for each response. Boxes represent 25% to 75% interquartile ranges, horizontal lines represent median values, and circles represent outliers. Dagger indicates that in these images, we would expect interexpert disagreement on presence of zone 1 disease based on the marked macular center.

Graphic Jump Location

Figure 3 displays variations in distances from the optic disc center to the marked macular center among the 10 study experts, scaled as number of standard deviations from the mean distance for each image. For the expert with the least mean deviation from zero (expert 3 in Figure 3), the mean deviation was 0.53 SDs from the mean for each image. For the expert with the highest deviation (expert 2 in Figure 3), the mean deviation was 1.45 SDs from the mean for each image. Linear regression analysis found no statistically significant relationship between PMA and variation in marked distances to the macular center (ie, number of standard deviations from the mean) (P = .21).

Place holder to copy figure label and caption
Figure 3.

Identification of the macular center in 25 posterior pole images by 10 experts, displayed by expert. Marked distances for all images (y-axis), scaled as number of standard deviations from the mean distance for each image, are displayed for each expert (x-axis). Boxes represent 25% to 75% interquartile ranges, horizontal lines represent median values, and circles represent outliers.

Graphic Jump Location

The 2-way mixed-effects ANOVA found the ratio of rater-specific variability to total variability around the expected distances to be 0.40 (P = .02). Therefore, significant evidence indicates that different raters have different propensities, with some tending to systematically provide higher measurements than expected and others tending to provide lower ones. Because of the possibility of nonnormality, the analysis was repeated using a nonparametric rank-based ANOVA with no qualitative changes in result.

DIAGNOSIS OF ZONE I DISEASE

In 7 of 21 images with visible peripheral disease (33%), there would have been disagreement among some experts in the diagnosis of zone I disease based on the identification of the macular center and the location of visible peripheral disease. Among the 10 experts, in 17 of 25 images (68%), 1 expert identified the distance between the optic disc and macular center to be greater than 1 SD from the mean (Figure 3).

Using a 2-way mixed-effects ANOVA, the estimates (standard errors) of variances were 0.15 (P = .07) for expert-specific variability and 0.22 (P = .02) for residual variability. With the assumptions and variance estimates from the ANOVA model, the probability of diagnosing zone I disease depends on the location of peripheral disease according to the following function:

 Image not available.

where Φ(t) is the probability that a standard normal variable is less than t, and x is the [(distance to peripheral disease) − (2× distance to the true macular center)].

This estimate was compared with the data by treating the mean of the experts' measurements as the true distance and by determining the proportion of raters who would have diagnosed zone I disease as a function of the peripheral disease location (Figure 4). If all experts would have diagnosed zone I disease when (distance to peripheral disease) − (2×distance to the true macular center) less than 0, then this curve would show that y-axis values equal 1.0 for all x-axis values less than or equal to 0. If all experts would have diagnosed zone I disease only when (distance to peripheral disease) − (2×distance to the true macular center) less than 0, then this graph would show that y-axis values equal 0 for all x-axis values greater than or equal to greater than or equal to 0. Rather, the curve suggests that there is imperfection in diagnosis of zone I ROP as the location of peripheral disease approaches twice the distance to the macular center.

Place holder to copy figure label and caption
Figure 4.

Probability of diagnosing zone I retinopathy of prematurity (ROP) as a function of closest peripheral disease location, based on analysis of variance (ANOVA) model. The x-axis shows the closest peripheral disease minus twice the true distance to the macular center (defined as the mean of all experts' measurements). The y-axis shows the probability of diagnosing zone I disease. Solid curve represents the ANOVA model, whereas dots show the actual study expert responses.

Graphic Jump Location

The key findings from this study are as follows: (1) There is significant variability among experts in the identification of the macular center from wide-angle images. (2) Interexpert agreement in the diagnosis of zone I ROP is likely to be imperfect. The CRYO-ROP and ETROP trials demonstrated that zone I disease is a critical feature of clinically significant ROP and that there are important benefits to early treatment of these eyes.3,4 Interestingly, there was a greater frequency of zone I disease in the ETROP study compared with the CRYO-ROP study, even when infants were controlled for birth weight and gestational age. At the same time, the incidence of favorable structural outcome in eyes treated for zone I disease was better in the ETROP cohort than in the CRYO-ROP cohort.7 One possible explanation for these differences may be that unidentified changes in the care of premature infants are causing an increased incidence of zone I disease.7 Alternatively, the differences might result from discrepancies in ascertainment, so that more zone I eyes in the CRYO-ROP study may have been underdiagnosed or more zone II eyes in the ETROP trial may have been overdiagnosed. We note that accurate and reproducible disease classification is essential for clinical research studies.

Retinal zones are defined using landmarks such as the optic disc, macular center, and ora serrata.1,2 Although this system assigns discrete values to each zone, we find interexpert variability in perceived location of the macular center (Table). Some images displayed a bimodal-type response distribution (Figure 1B), suggesting that 2 specific points had features that experts associated with the macular center. Other images displayed a continuous response distribution (Figure 1C and D), suggesting lack of consensus regarding the macular center. Overall, 7 of 21 images with visible peripheral disease (33%) showed discrepancies in zone I diagnosis according to the location of marked macular centers. A variance model showed that the experts would have diagnosed zone I disease based on macular marking with a probability that varied according to the location of peripheral disease (Figure 4). For example, for an image in which the distance to peripheral disease was 1 mm greater than twice the distance to the true macular center (defined as the mean of marked distances), approximately 12% of experts would be expected to overdiagnose this as zone I ROP. For an image in which the distance to peripheral disease was 1 mm less than twice the distance to the true macular center, approximately 12% of experts would be expected to underdiagnose this as not zone I ROP. These findings suggest that there is likely to be imperfect agreement among experts for zone I diagnosis.

Our findings are consistent with previous studies examining interexpert agreement in ROP and other medical conditions. Plus disease diagnosis using retinal photographs was variable among pediatric ophthalmologists and retinal specialists.8,9 In the CRYO-ROP study, 12% of eyes diagnosed as having threshold disease after ophthalmoscopy by a certified examiner were diagnosed at a less-than-threshold level during confirmatory ophthalmoscopy by a different certified examiner.10,11 It is conceivable that a study design in which experts were masked to the findings of other experts would have shown even higher levels of disagreement. In an article examining the use of fluorescein angiography for the evaluation of photodynamic therapy eligibility in patients with neovascular age-related macular degeneration, the κ statistic for interphysician reliability was 0.37 to 0.40.12 In the domain of emergency medicine, the κ statistic for reliability among physicians, residents, and surgeons for abdominal examination findings in adolescent patients was −0.04 to 0.38.13 For these reasons, we believe that training and education in ROP are crucial to ensure the highest level of patient care and to allow findings from major studies such as the CRYO-ROP and ETROP trials to be applied properly and consistently.3,4

The macular center may be difficult to identify precisely because foveal morphologic characteristics evolve significantly between 22 and 36 weeks' PMA and may not reach full maturity until the child is 15 to 45 months of age.14 One study identified a sequence of visible changes associated with macular development in the premature infant, ranging from early pigmentation of the macular area at 34 weeks to subsequent development of the macular annular ring reflex and foveolar reflex.15 Lack of consensus among experts on these features related to macular architecture may have contributed to variability in our study. Recognition of these features might increase consistency and decrease ambiguity in clinical diagnosis. For example, darker eyes may have pigmentary areas that are distinct from the actual macula (Figure 1D), and the retinal vascular arcades might be expected to branch lateral to the fovea (Figure 1B). Alternative methods for defining zone I disease, based on retinal morphologic features or differences between underlying vasculogenic and angiogenic processes, might be developed in the future.16

Clinically, it would be most useful to know the agreement of zone I diagnosis among multiple experts performing serial indirect ophthalmoscopy on the same infant. However, that type of study may be impractical because of safety concerns.17 To simulate a real-world situation in this study, we used RetCam images obtained from infants routinely examined for ROP during a 1-year period. This wide-angle contact camera is the most well-known instrument for pediatric retinal imaging, with a field of view of more than 100°.1828 In contrast, indirect ophthalmoscopy provides a 40° to 50° field of view, with better visualization of 3-dimensional retinal contours and potential for using this field to estimate the boundary of zone I.1,2 This difference in perspective could affect the interpretation of study findings, particularly if some experts had less experience in interpreting RetCam images. It is also conceivable that pressure on the eye from the camera could alter the appearance of retinal vessels.29 However, all study experts reported a high level of experience interpreting RetCam images, and there were no statistically significant differences in mean marked distances to the macular center after excluding images in which experts were not confident. In fact, wide-angle imaging may provide important advantages over ophthalmoscopy with regard to the visualization of macular reflexes and vascular architecture. Furthermore, all experts were asked to review the exact same images, which might have decreased the variability compared with serial ophthalmoscopy because examination quality may vary based on infant stability or cooperation.30

Several other limitations should be noted: (1) Images were taken of infants ranging from 32 to 38 weeks' PMA. Because of progressive macular development during this period,14,15 it might be expected that interexpert agreement would be poorest in the youngest infants. Linear regression analysis found no statistically significant relationship between PMA and variation in marked distances to the macular center, although studies with larger sample sizes may be useful. (2) Images were not annotated with any clinical data. Although it is not clear that this would have affected diagnostic agreement, it may have biased the results against accurate photographic interpretation. (3) To account for potential differences in image magnification, all horizontal optic disc widths were standardized to 0.93 mm, based on previous research in postmortem samples from premature infants.6 The mean disc-to-macula distance among all images in this study was 3.70 mm, whereas a different investigation calculated a mean distance of 4.4 mm using analysis of retinal photographs.31 This discrepancy may result from shrinkage during tissue preservation methods.6 Although there may be uncertainty about the exact disc-to-macula distances depending on the technique used, this would not change our key study findings. (4) There were no study images in which most experts would have diagnosed zone I disease. Because the findings still demonstrate significant variability among experts, it is not clear that this would have affected the study findings. From a practical standpoint, infants with posterior ROP tend to be younger and therefore more difficult to photograph clearly. (5) Experts were invited according to academic criteria, but this might not reflect real-world clinical expertise. However, it could be argued that academic ROP experts may have greater familiarity with the rigorous classification of retinal findings than the overall population of ophthalmologists who treat patients with ROP. Therefore, we hypothesize that a study involving a wider range of ophthalmologists might show even greater variability in the identification of the macular center.

In conclusion, this study shows that interexpert agreement in the identification of the macular center, and in the diagnosis of zone I ROP, is imperfect. This may have important implications for clinical care and for continued refinement of the international classification system.

Correspondence: Michael F. Chiang, MD, Department of Ophthalmology, College of Physicians and Surgeons, Columbia University, 635 W 165th St, Box 92, New York, NY 10032 (chiang@dbmi.columbia.edu).

Submitted for Publication: December 7, 2009; final revision received February 17, 2010; accepted February 22, 2010.

Financial Disclosure: Dr Chiang is an unpaid member of the Scientific Advisory Board for Clarity Medical Systems (Pleasanton, California). He is supported by a Career Development Award from Research to Prevent Blindness (New York, New York) and by grant EY13972 from the National Institutes of Health (Bethesda, Maryland).

Additional Contributions: We thank the 10 expert participants for their assistance, and Anna Ells, MD, and P. Lloyd Hildebrand, MD, for their insightful suggestions.

Committee for the Classification of Retinopathy of Prematurity, An international classification of retinopathy of prematurity. Arch Ophthalmol 1984;102 (8) 1130- 1134
PubMed
International Committee for the Classification of Retinopathy of Prematurity, The International Classification of Retinopathy of Prematurity revisited. Arch Ophthalmol 2005;123 (7) 991- 999
PubMed
Cryotherapy for Retinopathy of Prematurity Cooperative Group, Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results. Arch Ophthalmol 1988;106 (4) 471- 479
PubMed
Early Treatment for Retinopathy of Prematurity Cooperative Group, Revised indications for the treatment of retinopathy of prematurity: results of the Early Treatment for Retinopathy of Prematurity randomized trial. Arch Ophthalmol 2003;121 (12) 1684- 1694
PubMed
Cryotherapy for Retinopathy of Prematurity Cooperative Group, Multicenter trial of cryotherapy for retinopathy of prematurity: three-month outcome. Arch Ophthalmol 1990;108 (2) 195- 204
PubMed
Rimmer  SKeating  CChou  T  et al.  Growth of the human optic disc and nerve during gestation, childhood, and early adulthood. Am J Ophthalmol 1993;116 (6) 748- 753
PubMed
Good  WVEarly Treatment for Retinopathy of Prematurity Cooperative Group, Final results of the Early Treatment for Retinopathy of Prematurity (ETROP) randomized trial. Trans Am Ophthalmol Soc 2004;102233- 250
PubMed
Chiang  MFJiang  LGelman  RDu  YEFlynn  JT Interexpert agreement of plus disease diagnosis in retinopathy of prematurity. Arch Ophthalmol 2007;125 (7) 875- 880
PubMed
Wallace  DKQuinn  GEFreedman  SFChiang  MF Agreement among pediatric ophthalmologists in diagnosing plus and pre-plus disease in retinopathy of prematurity. J AAPOS 2008;12 (4) 352- 356
PubMed
Reynolds  JDDobson  VQuinn  GE  et al. CRYO-ROP and LIGHT-ROP Cooperative Study Groups, Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies. Arch Ophthalmol 2002;120 (11) 1470- 1476
PubMed
Phelps  DL It's plus disease, isn't it? Arch Ophthalmol 2007;125 (7) 963- 964
PubMed
Holz  FGJorzik  ISchutt  FFlach  UUnnebrink  K Agreement among ophthalmologists in evaluating fluorescein angiograms in patients with neovascular age-related macular degeneration for photodynamic therapy eligibility (FLAP-study). Ophthalmology 2003;110 (2) 400- 405
PubMed
Yen  KKarpas  APinkerton  HJGorelick  MH Interexaminer reliability in physical examination of pediatric patients with abdominal pain. Arch Pediatr Adolesc Med 2005;159 (4) 373- 376
PubMed
Hendrickson  AEYuodelis  C The morphological development of the human fovea. Ophthalmology 1984;91 (6) 603- 612
PubMed
Isenberg  SJ Macular development in the premature infant. Am J Ophthalmol 1986;101 (1) 74- 80
PubMed
Flynn  JTChan-Ling  T Retinopathy of prematurity: two distinct mechanisms that underlie zone 1 and zone 2 disease. Am J Ophthalmol 2006;142 (1) 46- 59
PubMed
Laws  DEMorton  CWeindling  MClark  D Systemic effects of screening for retinopathy of prematurity. Br J Ophthalmol 1996;80 (5) 425- 428
PubMed
Roth  DBMorales  DFeuer  WJHess  DJohnson  RAFlynn  JT Screening for retinopathy of prematurity employing the RetCam 120: sensitivity and specificity. Arch Ophthalmol 2001;119 (2) 268- 272
PubMed
Yen  KGHess  DBurke  BJohnson  RAFeuer  WJFlynn  JT Telephotoscreening to detect retinopathy of prematurity: preliminary study of the optimum time to employ digital fundus camera imaging to detect ROP. J AAPOS 2002;6 (2) 64- 70
PubMed
Ells  ALHolmes  JMAstle  WF  et al.  Telemedicine approach to screening for severe retinopathy of prematurity: a pilot study. Ophthalmology 2003;110 (11) 2113- 2117
PubMed
Chiang  MFKeenan  JDStarren  J  et al.  Accuracy and reliability of remote retinopathy of prematurity diagnosis. Arch Ophthalmol 2006;124 (3) 322- 327
PubMed
Chiang  MFWang  LBusuioc  M  et al.  Telemedical retinopathy of prematurity diagnosis: accuracy, reliability, and image quality. Arch Ophthalmol 2007;125 (11) 1531- 1538
PubMed
Chiang  MFStarren  JDu  YE  et al.  Remote image based retinopathy of prematurity diagnosis: a receiver operating characteristic analysis of accuracy. Br J Ophthalmol 2006;90 (10) 1292- 1296
PubMed
Gelman  RJiang  LDu  YEMartinez-Perez  MEFlynn  JTChiang  MF Plus disease in retinopathy of prematurity: pilot study of computer-based and expert diagnosis. J AAPOS 2007;11 (6) 532- 540
PubMed
Koreen  SGelman  RMartinez-Perez  ME  et al.  Evaluation of a computer-based system for plus disease diagnosis in retinopathy of prematurity. Ophthalmology 2007;114 (12) e59- e67
PubMed10.1016/j.ophtha.2007.10.006
Shah  PKNarendran  VSaravanan  VRRaghuram  AChattopadhyay  AKashyap  M Screening for retinopathy of prematurity: a comparison between binocular indirect ophthalmoscopy and RetCam 120. Indian J Ophthalmol 2006;54 (1) 35- 38
PubMed
Wu  CPetersen  RAVanderVeen  DK RetCam imaging for retinopathy of prematurity screening. J AAPOS 2006;10 (2) 107- 111
PubMed
Schwartz  SDHarrison  SAFerrone  PJTrese  MT Telemedical evaluation and management of retinopathy of prematurity using a fiberoptic digital fundus camera. Ophthalmology 2000;107 (1) 25- 28
PubMed
Koreen  SLopez  RJokl  DHFlynn  JTChiang  MF Variation in appearance of severe zone 1 retinopathy of prematurity during wide-angle contact photography. Arch Ophthalmol 2008;126 (5) 736- 737
PubMed
Scott  KEKim  DYWang  L  et al.  Telemedical diagnosis of retinopathy of prematurity: intraphysician agreement between ophthalmoscopic examination and image-based interpretation [published online ahead of print May 23, 2008]. Ophthalmology 2008;115 (7) 1222- 1228.e3
PubMed10.1016/j.ophtha.2007.09.006
De Silva  DJCocker  KDLau  GClay  STFielder  ARMoseley  MJ Optic disk size and optic disk-to-fovea distance in preterm and full-term infants. Invest Ophthalmol Vis Sci 2006;47 (11) 4683- 4686
PubMed

Figures

Place holder to copy figure label and caption
Figure 1.

Representative responses by 10 experts who identified the macular center in 25 posterior pole images. A, Image 17 in the Table and Figure 2, taken at 34 weeks' postmenstrual age (PMA), shows good agreement. B, Image 19 in the Table and Figure 2, taken at 33 weeks' PMA, shows a bimodal-type distribution. C, Image 25 in the Table and Figure 2, taken at 36 weeks' PMA, and D, Image 4 in the Table and Figure 2, taken at 38 weeks' PMA, show distributions along a continuum, as well as disagreement among some experts on presence of zone I disease based on the macular center.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 2.

Identification of the macular center in 25 posterior pole images by 10 experts, displayed by image. Distances between the macular center and optic disc were measured for each response. Boxes represent 25% to 75% interquartile ranges, horizontal lines represent median values, and circles represent outliers. Dagger indicates that in these images, we would expect interexpert disagreement on presence of zone 1 disease based on the marked macular center.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 3.

Identification of the macular center in 25 posterior pole images by 10 experts, displayed by expert. Marked distances for all images (y-axis), scaled as number of standard deviations from the mean distance for each image, are displayed for each expert (x-axis). Boxes represent 25% to 75% interquartile ranges, horizontal lines represent median values, and circles represent outliers.

Graphic Jump Location
Place holder to copy figure label and caption
Figure 4.

Probability of diagnosing zone I retinopathy of prematurity (ROP) as a function of closest peripheral disease location, based on analysis of variance (ANOVA) model. The x-axis shows the closest peripheral disease minus twice the true distance to the macular center (defined as the mean of all experts' measurements). The y-axis shows the probability of diagnosing zone I disease. Solid curve represents the ANOVA model, whereas dots show the actual study expert responses.

Graphic Jump Location

Tables

Table Graphic Jump LocationTable. Characteristics of 25 Retinal Images Evaluated by 10 ROP Expertsa

References

Committee for the Classification of Retinopathy of Prematurity, An international classification of retinopathy of prematurity. Arch Ophthalmol 1984;102 (8) 1130- 1134
PubMed
International Committee for the Classification of Retinopathy of Prematurity, The International Classification of Retinopathy of Prematurity revisited. Arch Ophthalmol 2005;123 (7) 991- 999
PubMed
Cryotherapy for Retinopathy of Prematurity Cooperative Group, Multicenter trial of cryotherapy for retinopathy of prematurity: preliminary results. Arch Ophthalmol 1988;106 (4) 471- 479
PubMed
Early Treatment for Retinopathy of Prematurity Cooperative Group, Revised indications for the treatment of retinopathy of prematurity: results of the Early Treatment for Retinopathy of Prematurity randomized trial. Arch Ophthalmol 2003;121 (12) 1684- 1694
PubMed
Cryotherapy for Retinopathy of Prematurity Cooperative Group, Multicenter trial of cryotherapy for retinopathy of prematurity: three-month outcome. Arch Ophthalmol 1990;108 (2) 195- 204
PubMed
Rimmer  SKeating  CChou  T  et al.  Growth of the human optic disc and nerve during gestation, childhood, and early adulthood. Am J Ophthalmol 1993;116 (6) 748- 753
PubMed
Good  WVEarly Treatment for Retinopathy of Prematurity Cooperative Group, Final results of the Early Treatment for Retinopathy of Prematurity (ETROP) randomized trial. Trans Am Ophthalmol Soc 2004;102233- 250
PubMed
Chiang  MFJiang  LGelman  RDu  YEFlynn  JT Interexpert agreement of plus disease diagnosis in retinopathy of prematurity. Arch Ophthalmol 2007;125 (7) 875- 880
PubMed
Wallace  DKQuinn  GEFreedman  SFChiang  MF Agreement among pediatric ophthalmologists in diagnosing plus and pre-plus disease in retinopathy of prematurity. J AAPOS 2008;12 (4) 352- 356
PubMed
Reynolds  JDDobson  VQuinn  GE  et al. CRYO-ROP and LIGHT-ROP Cooperative Study Groups, Evidence-based screening criteria for retinopathy of prematurity: natural history data from the CRYO-ROP and LIGHT-ROP studies. Arch Ophthalmol 2002;120 (11) 1470- 1476
PubMed
Phelps  DL It's plus disease, isn't it? Arch Ophthalmol 2007;125 (7) 963- 964
PubMed
Holz  FGJorzik  ISchutt  FFlach  UUnnebrink  K Agreement among ophthalmologists in evaluating fluorescein angiograms in patients with neovascular age-related macular degeneration for photodynamic therapy eligibility (FLAP-study). Ophthalmology 2003;110 (2) 400- 405
PubMed
Yen  KKarpas  APinkerton  HJGorelick  MH Interexaminer reliability in physical examination of pediatric patients with abdominal pain. Arch Pediatr Adolesc Med 2005;159 (4) 373- 376
PubMed
Hendrickson  AEYuodelis  C The morphological development of the human fovea. Ophthalmology 1984;91 (6) 603- 612
PubMed
Isenberg  SJ Macular development in the premature infant. Am J Ophthalmol 1986;101 (1) 74- 80
PubMed
Flynn  JTChan-Ling  T Retinopathy of prematurity: two distinct mechanisms that underlie zone 1 and zone 2 disease. Am J Ophthalmol 2006;142 (1) 46- 59
PubMed
Laws  DEMorton  CWeindling  MClark  D Systemic effects of screening for retinopathy of prematurity. Br J Ophthalmol 1996;80 (5) 425- 428
PubMed
Roth  DBMorales  DFeuer  WJHess  DJohnson  RAFlynn  JT Screening for retinopathy of prematurity employing the RetCam 120: sensitivity and specificity. Arch Ophthalmol 2001;119 (2) 268- 272
PubMed
Yen  KGHess  DBurke  BJohnson  RAFeuer  WJFlynn  JT Telephotoscreening to detect retinopathy of prematurity: preliminary study of the optimum time to employ digital fundus camera imaging to detect ROP. J AAPOS 2002;6 (2) 64- 70
PubMed
Ells  ALHolmes  JMAstle  WF  et al.  Telemedicine approach to screening for severe retinopathy of prematurity: a pilot study. Ophthalmology 2003;110 (11) 2113- 2117
PubMed
Chiang  MFKeenan  JDStarren  J  et al.  Accuracy and reliability of remote retinopathy of prematurity diagnosis. Arch Ophthalmol 2006;124 (3) 322- 327
PubMed
Chiang  MFWang  LBusuioc  M  et al.  Telemedical retinopathy of prematurity diagnosis: accuracy, reliability, and image quality. Arch Ophthalmol 2007;125 (11) 1531- 1538
PubMed
Chiang  MFStarren  JDu  YE  et al.  Remote image based retinopathy of prematurity diagnosis: a receiver operating characteristic analysis of accuracy. Br J Ophthalmol 2006;90 (10) 1292- 1296
PubMed
Gelman  RJiang  LDu  YEMartinez-Perez  MEFlynn  JTChiang  MF Plus disease in retinopathy of prematurity: pilot study of computer-based and expert diagnosis. J AAPOS 2007;11 (6) 532- 540
PubMed
Koreen  SGelman  RMartinez-Perez  ME  et al.  Evaluation of a computer-based system for plus disease diagnosis in retinopathy of prematurity. Ophthalmology 2007;114 (12) e59- e67
PubMed10.1016/j.ophtha.2007.10.006
Shah  PKNarendran  VSaravanan  VRRaghuram  AChattopadhyay  AKashyap  M Screening for retinopathy of prematurity: a comparison between binocular indirect ophthalmoscopy and RetCam 120. Indian J Ophthalmol 2006;54 (1) 35- 38
PubMed
Wu  CPetersen  RAVanderVeen  DK RetCam imaging for retinopathy of prematurity screening. J AAPOS 2006;10 (2) 107- 111
PubMed
Schwartz  SDHarrison  SAFerrone  PJTrese  MT Telemedical evaluation and management of retinopathy of prematurity using a fiberoptic digital fundus camera. Ophthalmology 2000;107 (1) 25- 28
PubMed
Koreen  SLopez  RJokl  DHFlynn  JTChiang  MF Variation in appearance of severe zone 1 retinopathy of prematurity during wide-angle contact photography. Arch Ophthalmol 2008;126 (5) 736- 737
PubMed
Scott  KEKim  DYWang  L  et al.  Telemedical diagnosis of retinopathy of prematurity: intraphysician agreement between ophthalmoscopic examination and image-based interpretation [published online ahead of print May 23, 2008]. Ophthalmology 2008;115 (7) 1222- 1228.e3
PubMed10.1016/j.ophtha.2007.09.006
De Silva  DJCocker  KDLau  GClay  STFielder  ARMoseley  MJ Optic disk size and optic disk-to-fovea distance in preterm and full-term infants. Invest Ophthalmol Vis Sci 2006;47 (11) 4683- 4686
PubMed

Correspondence

CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Please click the checkbox indicating that you have read the full article in order to submit your answers.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

662 Views
14 Citations
×

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
Jobs