Development of the 25-list-item National Eye Institute Visual Function Questionnaire FREE

HEALTH-RELATED quality of life (HRQOL) measures functioning and well-being in physical, mental, and social health realms of life and reflects the influence of a broad range of health conditions simultaneously. Because of the response burden and impact on participation rates, it is imperative that HRQOL measures be as short as feasible. Recognition of the important role that survey length plays in both data quality and costs has led to the creation of short-form versions of health surveys such as the 18-list-item version of the Patient Satisfaction Questionnaire,¹ the 5-list-item version of the Mental Health Inventory,² and the 36-list-item Short-Form Health Survey³ and 12-list-item Short-Form Health Survey.⁴ There is even greater need for short-form versions of vision-targeted surveys such as the National Eye Institute Visual Function Questionnaire (NEI VFQ),⁵ because to comprehensively evaluate HRQOL requires the use of vision-targeted measures in combination with generic measures.

A number of reliable and valid short questionnaires assess difficulty with activities that require vision or assess symptoms from eye diseases and their treatments.^{6 - 11} These vary in length from 14 to 31 questions. However, most of these surveys only capture one dimension of vision-targeted HRQOL. The 51-list-item field test version of the NEI VFQ^{5 ,12} was designed to capture the influence of vision on multiple dimensions of HRQOL, such as emotional well-being and social functioning. Early feedback from users indicated that a shorter version was needed for both research and clinical settings.

The goal of this study was to develop a short-form version of the NEI VFQ that preserves the multidimensional content, reliability, and validity of the full-length survey and also could be completed in approximately 5 minutes. This report describes the results of analyses designed to identify the best questions for inclusion in the 25-list-item version of the NEI VFQ (NEI VFQ-25) and examines the reliability and validity of the short-form version.

Two separate samples of visually impaired persons who completed the 51-list-item NEI VFQ were pooled for analyses. The first sample consisted of 262 persons from 5 academic centers who participated in the 1994 pilot test of the NEI VFQ (pilot study), and the second sample consisted of 597 persons from the 1996 NEI VFQ Psychometric Field Test (field test study). These 2 data sets were combined to provide a broader spectrum of disease severity than either data set represents alone.

A description of the sample-specific enrollment criteria is provided elsewhere.^{5 ,12} Briefly, in both samples eligible participants had to be at least 21 years old, had to be English speaking, and had to pass a cognitive test based on an abbreviated version of the Folstein Mini-Mental State Examination,¹³ where participants were only asked to complete the whole measure if they made an error on the initial orientation, short-term memory, or attention questions. Both the pilot study and field test protocols were approved by all institutional review boards, and all participants provided written informed consent.

Pilot study participants had 1 or more of the following eye conditions: age-related cataracts, age-related macular degeneration (ARMD), diabetic retinopathy, primary open-angle glaucoma, or cytomegalovirus retinitis. To be eligible for the field test, participants had 1 ocular condition only, whereas pilot study participants with multiple conditions were eligible. For these reasons, all tests of validity between the NEI VFQ-25 and clinical variables were performed solely with field test study data. The field test also enrolled persons with low vision from any cause and a reference group with no underlying eye disease. Condition-specific eligibility is described elsewhere.^{5 ,12}

Each participant completed a 16-list-item medical comorbidity checklist adapted from the Medical Outcomes Study¹⁴ and reported demographic characteristics. In the pilot study, currently corrected Snellen acuity and all diagnoses of eye diseases were abstracted from the medical records. As part of a research protocol,¹⁵ field test study participants completed a dilated examination that included an assessment of current eye diseases and previous ophthalmic surgical procedures. These participants were tested using binocular and monocular Early Treatment Diabetic Retinopathy Study (ETDRS) visual acuity while wearing their current, or "walking about," correction.¹⁶ Patients with visual acuity so poor that they could not read any of the largest letters at 4 m were tested at 1 m and for light perception. The presence of cataracts was graded during a slitlamp examination using the Age-Related Eye Diseases Study reference standards.¹⁷ Each participant had a complete fundus examination to rule out significant additional ocular pathologic conditions and to grade the severity of diabetic retinopathy, ARMD, primary open-angle glaucoma, and cytomegalovirus retinitis. Field test study ophthalmologic examinations were performed prospectively by trained examiners who followed the protocol in the NEI VFQ manual of procedures.¹⁵

The 51-list-item NEI VFQ is a vision-targeted survey that assesses the influence of visual impairment on HRQOL. The content of the NEI VFQ was derived from multicondition focus groups.¹² The 51-list-item NEI VFQ takes 15 minutes on average to administer as an interview and includes the following: multi-list-item scales to rate overall health (2 list-items), overall vision (2 list-items), difficulty with near vision (7 list-items), difficulty with distance vision (7 list-items), limitations in social functioning due to vision (4 list-items), role limitations due to vision (5 list-items), dependency on others due to vision(5 list-items), mental health symptoms due to vision (8 list-items), future expectations for vision (3 list-items), driving difficulties (4 list-items), and pain and discomfort around the eyes (2 list-items) and single list-items to assess peripheral and color vision. Each subscale is scored so that 0 represents the lowest and 100 the best possible score. A full description of the 51-list-item field test NEI VFQ has been published.⁵ All field test participants completed the 51-list-item NEI VFQ as part of a larger interview that contained multiple surveys. Pilot study participants completed the 96-list-item NEI VFQ, which included the 51 list-items that were eventually retained in the field test version. Only the 51 field test list-items were candidates for the list-item reduction analysis described herein.

The following qualitative criteria were used to identify candidate list-itemsfor the NEI VFQ-25:

The distributions of chronic ophthalmologic and medical conditions and demographic characteristics are displayed by sample. The NEI VFQ subscales are not strictly ordinal or equal interval measures, but because they approximate interval-level measures, parametric statistics are computed. Parametric statistics are robust to minor deviations from assumptions and more powerful than nonparametric statistical methods.¹⁸ The list-item-subscale correlations are calculated, and the proportion of persons with NEI VFQ list-item-level scores at the ceilings and floors for each question is also presented for the combined sample.

To create a derivation sample to construct linear regression models and a validation sample to assess the accuracy of the list-item selection results, the combined pilot study (n = 262) and field test samples (n = 597) were randomly split into halves. In each linear regression model, the dependent variable is a multi-list-item subscale from the 51-list-item NEI VFQ, and the independent variables are the individual list-items that constitute each subscale. A stepwise regression procedure is used to maximize the proportion of variance in the subscale score explained by a smaller set of list-items from that specific subscale.¹⁹ For example, if 2 list-items, such as reading ordinary print in newspapers and reading the fine print on medication bottles, were both part of the same near vision subscale, the list-item with the higher list-item-subscale correlation in the linear regression model is selected for inclusion in the shorter version. This procedure is designed to select the fewest list-items that account for the greatest proportion of variance in the long-form subscale. Rather than applying an absolute cutoff value of the inclusion, list-items were selected that maximized the proportion of variance explained. To determine whether the results are similar when the field test and the pilot study samples are analyzed separately, sample-specific models were constructed. These models identified 24 of the same 25 list-items reported for the combined sample and are not described further in this report.

To address potential biases in list-item selection introduced by missing data, 2 sets of linear regression models were constructed for each of the NEI VFQ subscales. The first set did not substitute values for list-item-level missing data, and the second set assigned the mean list-item-level score from participants who answered the list-item to list-items that were missed. Final list-item selection is based on consensus of authors after comparing the proportion of variance explained and list-items selected by both sets of models. All statistical tests are considered to be significant at P≤.05 (2-tailed), and exact P values are reported.

To estimate the internal consistency of the NEI VFQ-25, we calculated the Cronbach coefficient α²⁰ for each of the multi-list-item scales. Because coefficient α increases with the number of scale list-items and the size of their covariances, it is important to compare the values from the short-form subscales to results obtained from the original long-form version.

To estimate the validity of the NEI VFQ-25 scores across eye conditions, a series of comparisons were conducted that follow the same a priori between-condition comparisons reported for the 51-list-item NEI VFQ.⁵ The first set of comparisons assesses the magnitude of correlation between each 51-list-item subscale score, representing a "gold standard" measure of each construct and the 25-list-item subscale score that represents the same construct.

The second comparisons assess whether participants with poorer vision have statistically significantly lower scores on the NEI VFQ-25 than those with better vision. For example, participants in the reference group should have the best average scores, those in the low-vision group should have the poorest scores, and depending on the severity of the underlying condition, the remainder of the groups should have scores between these 2 extreme groups. These statistical comparisons of mean scores include the following: (1) scores from persons with low vision vs the reference group; (2) scores from persons with visually significant cataract vs the reference group; and (3) comparisons of selected subscales, such as the near vision scale for persons with ARMD vs the reference group, and peripheral vision scores for persons with glaucoma vs the reference group. In parallel with previously reported analyses for the 51-list-item NEI VFQ, linear regression models were used to adjust for between-group differences in age, sex, race, and a summary count of medical comorbidities, since these characteristics may independently influence HRQOL. Adjusted means were compared for the different subgroups.

The third set of comparisons assesses the magnitude of correlations between responses on the NEI VFQ-25 and performance-based measures that are part of the ophthalmologic examination. For example, patients with poorer ETDRS visual acuity across all conditions should have lower scores on the NEI VFQ-25. These correlations should be greatest for the subscales that capture difficulty with activities that require central visual acuity, such as the near vision scale and the driving scale, and lowest for subscales that capture other aspects of vision-targeted HRQOL, such as ocular pain. To test the significance of these associations, Pearson correlation coefficients were calculated between NEI VFQ-25 scores and visual acuity for all field test study participants and between visual field scores in the better and worse eyes for persons from the field test sample with glaucoma.

Across the 2 samples, 859 persons contributed data for the list-item reduction analyses. By design, the field test study included fewer whites (63% vs 81%) and oversampled African Americans (29% vs 11%; χ² across race categories, P = .01) compared with the pilot study. Women were recruited for both studies in similar proportions (pilot study, 54%; field test, 59%; χ², P = .15) and similar proportions were working (pilot, 40%; field test, 36%; χ², P = .25). Participants in the field test study were older than those in the pilot study (64 vs 61 years; t test, P = .02). Visual acuity between the 2 samples was similar when the normal reference group was included in the field test sample (P = .46) (Table 1), but was significantly poorer for the field test study when only persons with ocular disease were considered (P = .01).

Except for the general vision subscale, each of the remaining 8 mean subscale scores for the NEI VFQ-25 were significantly lower than scores for the same 51-list-item subscales. This was primarily due to the deletion of questions with ceiling effects.

The percentage of list-item-level responses that were missing or at the extremes of the distributions for the combined sample is shown in Table 2. These data helped identify which list-items to delete when constructing the NEI VFQ-25. For example, 21% of persons surveyed do not participate in games or card playing, and among those who do, 47% reported no difficulty with this activity. Therefore, by 2 criteria (high missing rate and skewed distribution), this list-item was not retained in the short-form version of the survey.

A number of qualitative list-item reduction decisions were made before constructing the linear regression models. For example, based on reliability and content validity considerations, both ocular pain questions were retained in the NEI VFQ-25. Also, the 3 list-items that constitute the expectations for future vision subscale were not retained in the NEI VFQ-25, because responses to formal cognitive debriefing questions indicated that participants found these list-itemsdifficult to answer, as evidenced by the low internal consistency of this subscale (coefficient α = .66). The cognitive debriefing questions were asked of field test participants after the administration of the 51-list-item NEI VFQ. These questions are described in the NEI VFQ manual of operations.¹⁵

Regression results (Table 3)show that for all of the multi-list-item subscales examined, more than 85% of the variance in the long-form scores is explained by the list-items retained in the short-form versions, with most exceeding 90%. A comparison of results for models that did not substitute mean values for missing data vs those that did (not shown) indicates that the list-items selected and the proportion of variance explained were similar for both methods. Both random halves of the data set identified the same subset of list-items for retention in the short-form version of the survey. Residuals from these models were found to be normally distributed. Application of the qualitative list-item reduction criteria described herein and the results of the regression analyses led to the identification of 25 list-itemsfor inclusion in the short-form version. NEI VFQ 51-list-item subscale scores can be estimated from NEI VFQ-25 scores with the β coefficients from the regression equations displayed in Table 3.

In summary, decisions regarding 16 of the 25 list-items dropped were based on the results of the linear regression models. The reasons for dropping the remaining 9 list-items can be found in Table 2.

The list-items identified for inclusion in the NEI VFQ-25 are shown in boldface type in Table 2. The reduction in subscale list-items is as follows: the general health and general vision subscales were shortened to single-list-item scales; the near vision, distance vision, and driving subscales were decreased to 3 list-items each; the social functioning subscales was halved to 2 list-items; the mental health subscale was condensed from 8 to 4 list-items; the dependency subscale was decreased from 5 to 3 list-items; and the role functioning subscale was shortened from 5 to 2 list-items. The remaining ocular pain (2 list-items), color vision (1 list-item), and peripheral vision (1 list-item) subscales are unchanged. A prepublication version of the NEI VFQ-25 has a 2-list-item, rather than the current 3-list-item, driving scale. We recommend that the 3-list-item driving scale described herein be used to increase the internal consistency of this subscale. Mean scale scores for the NEI VFQ-25 are displayed by ocular condition for the field test sample in Table 4. The NEI VFQ-25 subscale scores are an average of the list-items in the subscale transformed to a 0 to 100 scale, where 100 represents the best possible score on the measure and 0 represents the worst. The composite NEI VFQ-25 score is an unweighted average of the responses to all list-items except for the general health rating question. The general health question is treated as a stand-alonelist-item, because it is a robust marker of overall health status in many population-based studies and provides a comparative benchmark for groups of persons who complete the NEI VFQ-25. A copy of the NEI VFQ-25 and the scoring algorithm can be obtained from the NEI or the RAND Health Web site (http://www.rand.org/health_surveys/vfq25/).

Internal consistency estimates for the NEI VFQ-25 subscales ranged from 0.71 to 0.85. Among persons with eye diseases, all of the 8 multi-list-item subscales had internal consistency estimates greater than or equal to 0.70, indicating that the measure has acceptable reliability for group-level comparisons.²¹

Correlations between the NEI VFQ-25 versions of each subscale and their respective long-form version were greater than 0.90 (Table 3). For the 9 subscales that were shortened for the NEI VFQ-25, the adjusted mean scores for persons in the reference group were significantly higher compared with those with either low vision or visually significant cataract (Figure 1). Adjusted mean(SEM) scores for the NEI VFQ-25 near and distance vision scores for participants in the reference group vs those with ARMD were 90 (2) and 57 (2) and 91 (2) and 58 (2), respectively. Finally, adjusted mean (SEM) scores for the NEI VFQ-25 peripheral vision question for reference group members vs those with glaucoma were 97 (2) and 76 (2). All of these differences in mean scores were statistically significant. These selected comparisons provide evidence of between-group validity for the NEI VFQ-25.

Correlations between responses on the NEI VFQ-25 and ETDRS visual acuity were in the range of 0.65 to 0.70 for subscales that reflected degree of difficulty with visual activities related to general vision, near vision, and distance vision (Table 5). The remaining subscales showed lower correlations ranging from 0.39 to 0.69, with the exception of the ocular pain subscale, which showed the lowest correlations (between 0.06 and 0.11). Correlations between each of the subscales and visual acuity in the better and worse eyes were similar in magnitude. Among those with glaucoma, the Advanced Glaucoma Intervention Study²² visual field loss scores had moderate statistically significant correlations with the NEI VFQ-25 composite score, general vision, distance vision, near vision, peripheral vision, social functioning, dependency, and mental health subscales (Table 5). Additional evidence for the validity of the NEI VFQ-25 is shown in Table 4 with comparisons of condition-specific mean scores using the field test sample.

Power calculations based on observed between-eye condition differences in NEI VFQ-25 scores from the cross-sectional data collected in the field test study are presented in Table 6, Table 7, and Table 8 to assist with planning future research studies with sample size estimation.²³ Table 6 and Table 7should be used for designing randomized, experimental studies to maximize drawing strong causal inferences. Table 8 (self-selected groups) should be used to estimate statistical power for situations when it is not possible to randomize people to groups. To avoid a type I error due to multiple testing across the subscales, we advise researchers to develop hypotheses for which dimensions of vision are most likely to be affected by a given intervention and to conduct preplanned tests of associations for those specific subscales.

In this report we present the methods used to create the NEI VFQ-25 and provide evidence supporting its reliability and validity across multiple chronic eye conditions. Considerations in selecting list-items for the NEI VFQ-25 included missing data rates, the distribution of the list-item-level responses, content validity considerations, and linear regressions that identified list-itemsthat accounted for the maximum variability in the long-form subscale scores.

Short-form versions of any survey trade details about content and measurement precision for increased efficiency. To minimize the loss of content, we used methods that were likely to eliminate redundant or incomplete information rather than discard questions that provided unique information about vision-targeted HRQOL. The reported tests of reliability and validity indicate that although the NEI VFQ-25 is half as long as the 51-list-item version of the NEI VFQ, its psychometric properties are similar. In addition, the large proportion of variance in the long-form subscales explained by the smaller set of list-itemsindicates that much of the original content is retained in the shorter measure. Results of the combined sample, random split-half analysis of the data set provide evidence for the robustness of the selected list-items over a wide range of patients with 1 or more chronic eye conditions.

The evidence for validity of the NEI VFQ-25 includes comparisons of subscale-specific scores from the short- and long-form versions of the measure, between-group differences in NEI VFQ-25 scores for persons with different eye diseases of varying severity, and the moderate-to-high correlations between the NEI VFQ-25 subscales that have the most to do with central vision and measured visual acuity. Specifically, this article demonstrates that the NEI VFQ-25 is sensitive to the influence of age-related cataract, macular degeneration, glaucomatous field loss, and low vision from any cause. The correlations with clinical markers of disease severity provide evidence of clinical validity for the measure.

There are many surveys in the literature that assess difficulties with visual activities and/or condition-specific symptoms for persons with a wide variety of chronic eye diseases.^{6 - 8 ,10 - 11 ,24} Although one may trade off sensitivity in a narrow domain uniquely affected by a specific condition, there are a number of reasons why one might select a measure such as the NEI VFQ-25, which is specific for persons with vision problems but not designed for one specific condition. These reasons include the empiric patient-driven basis for the content of the NEI VFQ-25⁵ ; the value added to any disease-specific study of being able to compare the relative burden of one condition with another on the same scale; the multidimensional nature of the NEI VFQ-25 subscales, which are designed to capture the impact of visual problems on physical functioning, emotional well-being, and social functioning; and finally the rigorous multicondition evaluation of the reliability and validity of the NEI VFQ-25.

When interpreting this report, it is important to consider the following limitations. First, although persons across a large number of conditions and geographic regions were recruited for this study, to minimize the possibility of enrolling persons with false-positive diagnoses, the condition-specific enrollment criteria selected persons with moderate-to-severe disease. For this reason, we do not know whether the NEI VFQ-25 will be sensitive to the visual disability that is associated with earlier and milder forms of these or other ocular conditions. In addition, the NEI VFQ-25 estimates reported herein were derived from persons who completed the 51-list-item or the 96-list-item version of the measure. Although we suspect that the performance of the NEI VFQ-25 will be similar when these 25 list-items are administered alone, studies are currently under way to determine whether this is actually the case. It is important to also note that the estimates of statistical power reported in Table 6, Table 7, and Table 8are based on cross-sectional rather than longitudinal data. Further investigations are needed to establish the responsiveness of the NEI VFQ-25 in longitudinal studies.

Incorporating vision-targeted HRQOL measures into clinical studies will help form a more comprehensive understanding of treatment outcomes, not only in terms of benefits but also in terms of possible adverse effects. The need for a brief multidimensional vision-targeted HRQOL measure is illustrated by the fact that widespread use of the NEI VFQ-25 has preceded this report. To our knowledge, the NEI VFQ-25 has been translated into 8 languages and is currently being used in at least 7 federally funded research studies that are examining a range of ocular conditions. The evidence presented in this report illustrates the potentially useful information one might expect to gain with a relatively brief survey. Researchers are encouraged to use the NEI VFQ-25 to examine the influence various eye diseases and interventions have on a patient's day-to-day functioning and well-being.

Accepted for publication November 6, 2000.

This work was supported by the NEI, Bethesda, Md (contract C950424), and the Research Division of Merck Pharmaceuticals, Whitehouse Station, NJ. Dr Mangione is a recipient of a Clinical Investigator Award (1K08-AG00605) from the National Institute on Aging, Bethesda, Md, and a Generalist Faculty Scholar's Award from the Robert Wood Johnson Foundation, Princeton, NJ (029250). Dr Lee is a recipient of a Lew Wasserman Merit Award from Research to Prevent Blindness, New York, NY.

We would like to acknowledge and thank Leon Ellwein, PhD, Frederick Ferris III, MD, Argye Hillis, PhD, and Donald Patrick, PhD, for their methodological guidance and participation on the Scientific Advisory Panel for the NEI VFQ Psychometric Field Test.

NEI VFQ Field Test Investigators are as follows: Jonathan C. Javitt, MD, Department of Ophthalmology, The Johns Hopkins University, Baltimore, Md; Fang Wang, MD, PhD, Pfizer, Inc, Cambridge, Mass; Julian Nussbaum, MD, Rhett Schiffman, MD, Henry Ford Health Systems, Detroit, Mich; Emily Chew, MD, NEI, National Institutes of Health, Bethesda, Md; Janet DeBerry Steinberg, OD, Scheie Eye Institute, University of Pennsylvania, Philadelphia; Cynthia Owsley, PhD, Department of Ophthalmology, University of Alabama at Birmingham; Nancy K. Janz, PhD, School of Public Health, University of Michigan, Ann Arbor; Ron Klein, MD, Department of Ophthalmology, University of Wisconsin, Madison; Amy Chomsky, MD, Denis O'Day, MD, Department of Ophthalmology, Vanderbilt School of Medicine, Nashville, Tenn; Mae Gordon, PhD, Anthony Lubniewski, MD, Department of Ophthalmology, Washington University Medical School, St Louis, Mo.

Corresponding author: Carol M. Mangione, MD, MSPH, Division of General Internal Medicine and Health Services Research, Department of Medicine, UCLA, 911 Broxton Plaza, Box 951736, Los Angeles, CA 90095-1736.