Repeatability and Reproducibility of Fast Macular Thickness Mapping With Stratus Optical Coherence Tomography FREE

Optical coherence tomography (OCT) is a noninvasive, noncontact imaging instrument capable of generating high-resolution optical cross sections of the retina. Many retinal disorders have been extensively described and their pathogenesis studied by using OCT.^1- 2 Its particular ability to detect with high accuracy the inner and outer retinal boundaries from the acquired scans and to measure their distance (ie, retinal thickness) has made OCT increasingly popular in the quantitative assessment of macular edema.^3- 4 Traditional methods for assessing retinal thickening, ie, slitlamp biomicroscopy and stereoscopic photography, are qualitative and subjective and may not detect subtle macular thickening. Therefore, methods providing quantitative measurements of retinal thickening are needed.

Retinal thickness measurements can be automatically or manually obtained from a single linear cross-sectional image through a selected location or automatically generated by a mapping protocol and displayed as 2-dimensional color-coded and numeric maps.⁵ This protocol facilitates mapping of the retinal thickness across a 6-mm-diameter disc, centered on the patient’s fixation point, and it allows for the exact location and quantification of areas of macular thickening.

A number of studies have demonstrated good reproducibility of OCT measurements of single scans and retinal mapping made with the Humphrey 2000 OCT system (Humphrey Instruments, Inc, Dublin, Calif) using its A5 mapping software in healthy eyes and in eyes with diabetic macular edema, suggesting that OCT can reliably monitor retinal thickness changes over time.^6- 9 In a study by Massin et al,⁹ reproducibilities of ±5% and ±6% were found for healthy subjects and patients with diabetic macular edema, respectively. However, since the commercialization of the new Stratus OCT (Carl Zeiss Meditec, Dublin), a new macular mapping algorithm has become available. The main innovation of this algorithm is that the 6 radial optical cross sections previously acquired with 6 different linear scans at approximately 1 second per single scan are now obtained with 1 composite scan in 1.92 seconds. This feature has the advantage of significantly decreasing the total acquisition time and improving the tolerability by the patient. Therefore, although the longer acquisition time of this scan may increase the dependency on patient fixation, the advantage of performing just 1 alignment instead of making separate alignments for each scan may reduce variability of measurements due to eye motion. This new algorithm has been recently demonstrated to be reproducible for measuring macular thickness in healthy subjects.¹⁰ However, to our knowledge, no reproducibility study with this software in diabetic patients with clinically significant macular edema (CSME) has been published. The goal of this study was to determine the repeatability and reproducibility of retinal thickness measurements in healthy subjects and in diabetic patients with CSME using the Stratus OCT fast macular mapping protocol.

During a 3-month period, we consecutively enrolled 10 healthy volunteers (4 men and 6 women) ranging in age from 18 to 73 years (mean age, 40 years) and 15 diabetic patients with CSME (11 men and 4 women) ranging in age from 51 to 80 years (mean age, 64 years). Inclusion criteria for healthy subjects included a best-corrected visual acuity of 20/25 or better and a normal-appearing macula on contact lens biomicroscopy. Eligibility criteria for the diabetic patients included a diagnosis of diabetic retinopathy with retinal thickening involving the center of the macula, ie, CSME as defined by the Early Treatment Diabetic Retinopathy Study (ETDRS),¹¹ and sufficient media clarity to visualize the fundus using contact lens biomicroscopy. All eligible diabetic patients with CSME who were examined during a 3-month period by a single retina specialist (A.P.) participated in the study. All subjects (healthy and diabetic) underwent visual acuity testing with refraction and a complete slitlamp examination. The OCT examination was performed before and after pupil dilation with 1 drop of 2.5% phenylephrine hydrochloride and 0.5% tropicamide on 1 randomly selected eye in all healthy subjects and in the diabetic patients with CSME in both eyes. The study was conducted according to the tenets of the Declaration of Helsinki, and all subjects gave informed consent after the intent of the study had been explained. Visual acuity in the healthy group ranged from 20/20 to 20/25 (median 20/20); in the diabetic group, from 20/20 to 20/100 (median, 20/50). All diabetic patients with CSME had type 2 diabetes mellitus; 10 had nonproliferative diabetic retinopathy and 5 had proliferative diabetic retinopathy that was treated with panretinal photocoagulation. Eight patients had received focal/grid laser photocoagulation for macular edema.

The scanner used in this study was the commercially available Stratus OCT with the A1.1 version software. The principles of OCT, based on low-coherence interferometry, have been reported in detail elsewhere.¹² The main difference between the Stratus OCT system and the 2000 OCT system is that the transverse resolution of the optical cross sections, which previously depended on the fixed number of 100 axial measurements (A-scans) for the scan line (B-scan), can now be increased progressively by performing 128, 256, or 512 A-scans per B-scan. The longitudinal resolution is also slightly higher and approximately 10 μm. Retinal thickness is measured automatically as the distance between the vitreoretinal interface and the anterior boundary of the retinal pigment epithelium. As it was in the previous version, the location of these boundaries is determined by a thresholding algorithm that searches for the change in reflectivity present at each of these interfaces. In addition to the mapping protocol introduced by Hee and coworkers⁴ that uses the radial-lines scan pattern, the new system features a program allowing for a quicker and easier image acquisition. This program is the fast macular thickness mapping protocol, which compresses the 6 radial lines of the 2000 OCT macular thickness mapping protocol into 1 scan. In fact, this protocol acquires six 6-mm radial lines consisting of 128 A-scans per line in 1.92 seconds of scanning. Thus, retinal thickness is measured at a total of 768 points along these 6 intersecting lines. The characteristics of the 2 retinal thickness mapping protocols, the older 2000 OCT and the newer Stratus OCT, and their respective transverse and axial resolutions are briefly summarized in Table 1.

Subjects underwent 2 OCT scanning sessions, 1 before and 1 after pupil dilation. Three scans were performed by selecting the fast macular thickness map protocol for each session by 2 experienced examiners (A.P. and M.D.B.) with intervals of approximately 5 minutes between scans. Two scans were performed by the same examiner (A.P.) to determine intraobserver reproducibility. A third scan was performed by the second examiner (M.D.B.) with CSME and the results were compared with those of the first scan to determine interobserver reproducibility. An internal fixation light was used during scanning. The scans are displayed continuously by the system at a rate of 6 tomograms per 1.92 seconds. Acceptable scans were selected as soon as they appeared. Images were judged to be of adequate quality and correctly positioned on the basis of the following acceptance criteria: good demarcation of the vitreoretinal and chorioretinal interface; absence of artifacts owing to motion and pupillary shadowing; and presence of clearly identifiable OCT landmarks in the center of each scan, such as the foveal pit in healthy subjects or large protruding cystoid spaces with minimal internal reflectivity in diabetic patients with CSME. Between the examiners, the instrument alignment and controls were randomly changed, so all alignment and focusing had to be restarted. To assess intervisit reproducibility, an additional scan was performed approximately 1 week later in healthy subjects. In this case, the examination was performed only in undilated eyes because our initial data did not show a significantly negative effect of pupil size on reproducibility. Intervisit reproducibility was not assessed in diabetic patients with CSME because observed differences may be dependent on variations of macular edema. Each scan was analyzed by selecting the retinal thickness and volume quantitative analysis protocol, which displays retinal thickness and volume measurements (in micrometers and square millimeters, respectively) as color-coded and numeric retinal thickness maps. In the numeric map, the measurements are averaged across each of the following 9 ETDRS areas: 1 central area within an inner 1-mm-diameter circle (A1); 4 inner areas (superior [A2], temporal [A3], inferior [A4], and nasal [A5]) between the inner and a middle 3-mm-diameter circle; and 4 outer areas (superior [A6], temporal [A7], inferior [A8], and nasal [A9]) between the middle and an outer 6-mm-diameter circle (Figure 1). Only the retinal thickness measurements were recorded. The density of measured points is higher in the central area and progressively decreases in the inner and outer rings (Table 1). The central foveolar thickness was calculated as the average of the 6 measurements obtained where the entire 6 radial scans intercept.

The mean and standard deviation of the retinal thickness measurements averaged across each of the 9 ETDRS areas (A1-A9) and the foveola for healthy subjects and diabetic patients with CSME were first calculated for each observer and visit. To quantify the reproducibility of repeated measurements performed by the same observer, by different observers, and at different visits, we calculated the repeatability, reproducibility, and intraclass correlation coefficients (ICCs). As proposed by the British Standards Institution and recommended by Bland and Altman,¹³ the coefficient of repeatability was defined as 2 SDs of the differences between pairs of measurements in the same subjects obtained during the same visit by the same observer divided by the average of the means of each pair of readings. The coefficient of reproducibility was defined as 2 SDs of the difference between measurements obtained from the repetition of the test under different conditions (change of observer or visit) divided by the average response.¹³ The Wilcoxon matched-pairs test (5% significance level) was used to determine whether there was any statistically significant difference between measurements obtained by different observers or during different visits. Repeated retinal thickness measurements were also compared by calculating the ICC, a measure of exact agreement commonly used to evaluate reliability. We calculated the ICC on the basis of the analysis of variance for mixed models corresponding to each condition as proposed by Bartko and Carpenter.¹⁴ Values close to 1 indicate that the reproducibility of the method is high. We calculated the ICC using the software package SPSS version 11.5.1 (SPSS Inc, Chicago, Ill). Moreover, intrasession coefficients of variation for healthy subjects and diabetic patients with CSME were computed by dividing each standard deviation of repeated measurements by its corresponding session mean and then averaging the result for all subjects.

For all healthy subjects and diabetic patients with CSME, the average thickness automatically calculated by the software for each ETDRS region and the foveola during each session is given in Table 2 and Table 3. The means and standard deviations of the differences between measurements obtained under different conditions in dilated and undilated eyes for healthy subjects and diabetic patients with CSME are also shown in Tables 2 and 3. The coefficients of repeatability and reproducibility were computed from the standard deviations of the differences between measurements made at each session. The results obtained are shown in Table 4. The Wilcoxon paired-measurements test (5% significance level) showed that there were no statistically significant differences between measurements obtained by different observers or during different visits in healthy subjects and diabetic patients with CSME. In the undilated eyes of healthy subjects, the coefficient of repeatability was less than 6% for all macular areas except 2 (ie, in A6 and A7), and at worst it was 7%; in dilated eyes, it was less than 5% in all macular areas except the central foveola and A7, and at worst it was 7% (ie, in A7). In the undilated eyes, the coefficient of reproducibility was less than 6% in all regions but 4 (ie, in A6, A7, A8, and the central foveola), and at worst it was 9% (ie, in A7); in dilated eyes, it was less than 6% for all areas except 2 (ie, in A6 and the central foveola). In diabetic patients with CSME, the coefficient of repeatability was less than 9% for all macular areas in undilated eyes and less than 7% for all areas in dilated eyes. For dilated eyes, the coefficient of reproducibility was less than 8% for all regions except 2 (ie, A3 and A7), and at worst it was 10% (ie, in A7); for dilated eyes, it was less than 10% for all macular areas except 2 (ie, A1 and central foveolar), and at worst it was 11% (ie, in central foveolar). The ICCs were greater than or equal to 0.80 in all macular regions except 1 (ie, A7) in healthy subjects and always greater than 0.98 in diabetic patients with CSME (Table 5). Moreover, intrasession coefficients of variation ranged from 0.6% to 2.1% in healthy subjects and and from 1.2% to 2.2% in diabetic patients with CSME (Table 6).

In this study, macular thickness measurements obtained with the Stratus OCT system using the fast macular mapping protocol were shown to be repeatable and reproducible. The measurements made for the healthy subjects showed that coefficients of repeatability ranged from 2% to 7% in dilated and undilated eyes, and coefficients of reproducibility ranged from 2% to 9% in undilated eyes and from 2% to 7% in dilated eyes. In the diabetic patients with CSME, the coefficients of repeatability ranged from 3% to 8%, and coefficients of reproducibility ranged from 4% to 11% in dilated and undilated eyes. The ICCs were particularly high for both groups, confirming a good reliability of the measurements. Intrasession coefficients of variation of less than 2% and 3% in healthy subjects and diabetic patients with CSME, respectively, demonstrated a high level of intrasession reproducibility. These data are slightly lower than the reproducibility values reported by Massin et al,⁹ probably owing to differences in the mapping protocol, but nevertheless show an adequate degree of repeatability and reproducibility, particularly for the central 1-mm-diameter disc. Similarly, we found the diabetic patients with CSME to be more variable than the healthy subjects. However, age differences between groups preclude us from being able to attribute this difference solely to the different diagnosis. A recent publication by Paunescu et al¹⁰ regarding the reproducibility of fast macular thickness mapping in healthy subjects found ICCs of 51% to 92%. The ICCs of 69% to 99% achieved in the present study in all macular regions in dilated and undilated eyes are comparable. In addition, Paunescu et al¹⁰ found lower mean macular thickness values for the central (ie, 205.9 ± 21.0 μm) and all other ETDRS subfields. This discrepancy may most likely result from the use of a different and more recent software version in their study, that is, A2 instead A1.1, which may be more accurate in detecting and profiling the retinal boundaries.

Repeatability and reproducibility of measurements are strictly dependent on the following issues: how easily the optical cross sections can be consistently placed over the same points during each scan, how great the variation in retinal thickness along neighboring points is, and how many points are measured for each region. With the macula fast mapping protocol, each single scan combining the 6 radial tomograms has a longer acquisition time of 1.92 seconds than each tomogram acquired separately as in the previous protocol. This may result in an increased difficulty in positioning the scan over the same location owing to eye motion. The impact of unstable fixation could therefore be even greater with this protocol. On the other hand, the possibility of acquiring all of the tomograms with a single alignment may reduce scan positioning errors occurring with separate alignments. Inaccuracies in scan positioning may also result in a higher variability of measurements in locations where the contour of the retina varies, such as the foveal depression or the edge of areas of focal thickening in case of edema. Moreover, as with the previous protocol, the number of measured points significantly decreases from the center to the periphery, where the tomograms are more spaced, and this may result in a greater variability of the measurements for the outer macular regions.

As expected, coefficients of repeatability and reproducibility were lower for the central foveolar point, where large variations in retinal thickness occur owing to the shape of the foveal depression and a small number of points are sampled. In the diabetic patients with CSME, most of the variability encountered in this region was primarily determined by those cases in which significant variations in foveal contour due to retinal thickening were present (Figure 2). In such cases, minimal changes in scan positioning at the point of intersection resulted in significantly different retinal thickness measurements. However, similar to the findings of Massin et al,⁹ our results showed that the repeatability and reproducibility in the 1-mm-diameter central macular area remained good in healthy subjects and diabetic patients with CSME and impaired vision (ie, the coefficients of repeatability were 4% and 6%, respectively, and the coefficients of reproducibility were 6% and 9%, respectively). In particular, lower levels of visual acuity (ie, worse than 20/50) in diabetic patients with CSME did not seem to reduce the degree of repeatability and reproducibility. This may be a result of the larger number of measured points in this region, which compensates for errors due to changes in scan placement. Conversely, some of the variability found in the outer macular areas (ie, A6, A7, and A8) in healthy subjects and diabetic patients with CSME may be attributed to the relatively smaller numbers of sampled points.

The reproducibility of retinal thickness measurements in undilated eyes has been previously tested only for single linear scans but not for retinal maps generated by multiple radial scans.⁷ The poorer visualization of the fundus may significantly limit the accuracy in the positioning of the scan and therefore increase variability of measurements. However, the ability to perform OCT examinations in undilated eyes may extend its use to those cases in which dilation is difficult. In our study, the high reproducibility found for undilated eyes of healthy subjects and diabetic patients with CSME suggests that pupil dilation may not always be necessary to obtain reliable measurements.

As with previous findings,⁹ we did not find the change of examiner to significantly affect the reproducibility of measurements in healthy eyes and in those with diabetic retinopathy and CSME. However, reliable retinal thickness values can only be obtained from scans of adequate quality. In our study, both examiners were experienced and accepted only scans fulfilling defined acceptance criteria such as an adequate identification of the 2 retinal boundaries and the visualization of OCT landmarks such as the deepest part of the foveal pit or areas of a local minimum intraretinal reflectivity as the position of the central fovea in healthy subjects and diabetic patients with CSME, respectively. However, a new software that allows for a user-assisted registration of scan lines with the fundus image and compensates for small inaccuracies in scan positioning due to unstable fixation would enable more precise repeat scanning.

Another potential source of variability could be attributed to epiretinal or intraretinal features, such as vitreoretinal membranes or hard exudates, that provoke artifacts by impairing a correct detection of retinal boundaries by the software. However, even if most of our diabetic patients with CSME had hard exudates, this did not seem to be a major source of errors. Nevertheless, we agree with Massin et al⁹ that the addition of a software that could allow for manual correction of the boundaries in case of obvious artifacts would be very useful.

The fast macular mapping protocol of the Stratus OCT system offers a way to obtain retina thickness maps of normal and thickened retinas in both dilated and undilated eyes more quickly and easily than before, without compromising a high degree of repeatability and reproducibility. These findings are particularly useful because they indicate that any retinal thickness changes of greater than 6% and 10% in the 1-mm-diameter central macular area in healthy subjects and diabetic patients with CSME, respectively, are likely to be caused by changes in retinal thickness rather than by inconsistencies in the measurements given by the OCT system. This finding implies that this new mapping protocol is reliable for monitoring patients with macular edema and assessing treatment efficacy.

Correspondence: Francesco Bandello, MD, Department of Ophthalmology, University of Udine, Piazzale S Maria della Misericordia, 33100 Udine, Italy (francesco.bandello@uniud.it).

Submitted for Publication: March 8, 2004; final revision received October 24, 2004; accepted December 7, 2004.

Financial Disclosure: None.

Previous Presentations: This study was presented in part at the inaugural meeting of the International Society for Imaging in the Eye; May 2, 2003; Fort Lauderdale, Fla; and at the Retina Society Meeting; September 19, 2003; Chicago, Ill.