Author Affiliations: Department of Ophthalmology, Royal Surrey County Hospital, Surrey (Drs Saleh and Voyazis), and Department of Surgical Oncology and Technology, Imperial College, London (Drs Hance, Ratnasothy, and Darzi), England.
This article was corrected | View correction
To evaluate motion tracking as an aid to a more objective assessment of ophthalmic microsurgical skill.
In a cohort study, 3 groups of differing levels of surgical experience were assessed. The groups included novice surgeons (n = 10) with fewer than 5 previously performed corneal sutures, trainee surgeons (n = 10) with 5 to 100 previously performed corneal sutures, and expert surgeons (n = 10) with more than 100 previously performed corneal sutures. The Imperial College Surgical Assessment Device was used for the objective assessment of surgical dexterity during corneal suturing. Each of the subjects used a 10-0 nylon suture in a 3-1-1 pattern on an artificial eye (Royal College of Ophthalmologists, London, England). The Imperial College Surgical Assessment Device measures 3-dimensional spatial vectors via electromagnetic sensors attached to the surgeon's fingers. The number of movements, path length for the respective movements, and time taken to complete the given task were recorded.
Highly statistically significant differences were found between the 3 grades of surgeon experience for time taken (P<.001), number of hand movements (P<.001), and path length of the hand movements (P = .002) to complete the given task.
Motion analysis measured by this technology may be useful in the formal surgical training of residents and as an objective quantitative measure of dexterity.
The style of medical training has changed in recent years, with much more emphasis on standardized and objective assessment of clinical, academic, and surgical knowledge. Greater peer and public scrutiny of surgical performance has also highlighted the need for more objective ways of measuring technical skill.1,2 In ophthalmology, this is often assessed in the operating theater environment with the supervising surgeon directly observing (and commenting) during a procedure. Often, feedback is also offered while watching a recording of the operation. This technique can be highly subjective, has large interobserver variations, is not readily reproducible, and is difficult to interpret.3 Even studies using experts in their fields to assess procedures by medical residents can prove to be unreliable with great interobserver variability when using nonstandardized criteria.4 The Objective Assessment of Skills in Intraocular Surgery and the Global Rating Assessment of Skills in Intraocular Surgery are ophthalmology-specific tools that have emerged as objective measures to begin addressing these problems.5,6
Within the context of this subjective appraisal, a technically good surgeon is one perceived as both being quick and displaying greater economy and precision of movement.7 Certain components of this surgical skill can be determined by motion analysis using instruments such as the Imperial College Surgical Assessment Device (ICSAD). This tool allows for meticulous interrogation of performance, including a detailed analysis of movement behavior.8,9 Motion tracking has been shown to be a valid measure of dexterity in laparoscopic, open, and other surgical simulations.10,11
Tracking often occurs through markers located on body articulations to garner movement information from a particular limb, and similar technology is used in fields such as gait analysis.12 By using these techniques, the feasibility and validity of detailed hand movement measurement in assessing surgical skills have been demonstrated.10,13 Motion analysis could thus add a highly useful dimension to the Objective Assessment of Skills in Intraocular Surgery and Global Rating Assessment of Skills in Intraocular Surgery tools.
We describe the ophthalmic adaptation of the ICSAD for microsurgical motion tracking analysis.
The ICSAD performs quantitative motion analysis. This system is a combination of a commercially available electromagnetic tracking system (Isotrak II; Polhemus Inc, Colchester, Vt) and a custom-made computer software program. The latter has been developed within the Imperial College Surgical Division, London, England, in conjunction with the Department of Computing.
A single 10-mm electromagnetic tracker was attached to the middle interphalangeal joint of the index finger of each hand. Inside each of the trackers are 3 electromagnetic coils that collect data from the x, y or z plane as they move through and interrupt the magnetic field generated locally by the device. Surgical latex gloves are worn to help secure the trackers, ensuring no unwanted slippage, and to mimic real-life conditions.
The system collects the x, y, and z Cartesian coordinate information from each tracker at a resolution of 1 mm and a frequency of 20 Hz. The raw 3-dimensional positional data collated from the tracking system are transferred to the computer and extrapolated in scores of dexterity by the custom software. These are the number of movements made by each index finger (an individual movement being defined as a change in velocity), the path length of these respective movements (defined as the distance between the 2 points in 3-dimensional space where the change in velocity occurred), and the time taken to complete the procedure. The software program includes a Gaussian filter that eliminates background noise, ensuring that only the purposeful actions are recorded.
Previous work14 using this apparatus suggested that there was no additional benefit to analyzing the movements or path length of individual hands, so analysis was confined to the sum of both hands. Therefore, the combined results for both hands were used in the final analysis.
The selected task was the insertion of a single 10-0 nylon suture (Ethicon, Inc, Somerville, NJ) into the wound of an artificial cornea (Royal College of Ophthalmologists, London). A 3-1-1 suture was undertaken and the knot cut. An independent assessor ensured correct completion of the task. All of the subjects were asked to undertake the task using the same instruments (Castroviejo Needleholders [Ref 305]; Altomed, Tyne and Wear, England), microscope (US-1; Storz Urban, San Dimas, Calif), and setup to allow for consistency. The needle holders were given to the subjects with the needle premounted.
Subjects were divided into 3 cohorts (n = 10 in each group). The first group (novice) had performed fewer than 5 corneal sutures prior to the study, the second (trainee) had performed between 5 and 100 corneal sutures, and the third (expert) had performed more than 100 corneal sutures.
The Kruskal-Wallis test was used to compare groups and generate median scores along with interquartile ranges. A nonparametric test was selected to analyze the data owing to the relatively small sample sizes.
Summaries of path length, movement, and time are shown in the Table. The results demonstrate significant differences in the number of finger movements, path length, and time taken to complete the corneal suture, with the more experienced surgeons demonstrating greater economy to complete the task. The results also show that as experience increased, there was a reduction in the variability between the individuals within the cohort.
A graphical representation of the results is expressed in Figures 1, 2, and 3. Highly statistically significant differences were found between the 3 grades of surgeon experience. With a greater degree of experience, there was a shorter time taken (Kruskal-Wallis, P<.001), fewer hand movements were made (Kruskal-Wallis, P<.001), and the path length of the hand movements was shorter (Kruskal-Wallis, P = .002) in completing the given task.
Box-and-whisker plot comparing the time taken to place a single suture in an artificial eye by the 3 different cohorts of surgeons (P<.001).
Box-and-whisker plot comparing the number of finger movements required to place a single suture in an artificial eye by the 3 different cohorts of surgeons (P<.001).
Box-and-whisker plot comparing the total path length of both index fingers required to place a single suture in an artificial eye by the 3 different cohorts of surgeons (P = .002).
This study aimed to gauge the feasibility of using the ICSAD with the task of corneal suturing. The results show that motion analysis through this device was able to stratify performance according to surgical experience. The more experienced the surgeon was, the less time, movement, and distance were required to complete the given task. As experience increased, there was a reduction in the variability between the individuals within the cohort. This suggests that as experience is gained, surgical skill may converge to a narrower spread of efficiency toward which all trainees should be aspiring. The results also show that the ICSAD has construct validity, the ability to differentiate between different levels of skill, when applied to corneal suturing.
Factors in surgical proficiency such as core knowledge, decision-making ability, and communication skills are already formally appraised in written, clinical, and oral examinations. However, although poor clinical outcomes can be the result of inadequate technical skill, there have been few formal attempts to evaluate them.14- 16 The move toward competency-based curricula requires the appropriate tools for assessment. Motion analysis may therefore help with the most elusive assessment criterion thus far: an objective standardized score for appraising technical surgical skill, which is an integral part of overall surgical competence.17,18
Concern about a decline in the standard of surgical training has been raised in both Europe (with the reduction of working hours through the European working time directive) and the United States (where the Accreditation Council of Graduate Medical Education has addressed both residents' hours work and assessment of competencies).19 The introduction of skills courses and wet laboratories into surgical training has been suggested as a substitute to live exposure at the early stages.15,20 Motion tracking may prove useful in these environments as well, helping to provide a numerical rating as a benchmark for each individual. The trainee can then aim to improve his or her score through enhancement of surgical skill. It may eventually be useful to break down individual parts of various operations and to have normative pools of motion data for surgeons of varying experience. A trainee having difficulty with a certain portion of a procedure could then compare his or her motion data and get feedback on the efficiency of movement to help improve the skill.
This study was undertaken in a wet laboratory environment using artificial eyes. The task of 3-1-1 corneal suturing is sometimes used at the end of cataract surgery and in corneal graft surgery, but by no means is it universally used. Thus, the task selection has inherent limitations. However, it has been demonstrated that the use of motion tracking can be applied to generic ophthalmic surgical tasks and can discriminate quite clearly based on experience. Further studies will be required, but the ICSAD may prove useful in the assessment of other aspects of corneal and more general ocular surgery. In principle, this technique could be expanded for use in live surgery as well.
This tool addresses a specific part of technical competence. Although it has been shown to be very effective in stratifying skill, it can still only act as an adjunct to the current systems of assessment. The results shown in the box plots are interesting because it would be expected that surgical outcome would follow a similar trend, with more experienced surgeons frequently attaining better outcomes than their trainees. It is, however, a limitation of this study that surgical experience was the primary benchmark for surgical skill. Further research could attempt to quantify the correlation and more clearly define these relationships.
The techniques described in this study have been validated in other surgical disciplines but have not been previously applied to ophthalmic microsurgery. The effectiveness with which the surgical experience was stratified is encouraging and will hopefully spur further investigation into this field, perhaps allowing for the development of a more objective surgical skill assessment for trainees.
Correspondence: George M. Saleh, MRCS(Ed), Department of Ophthalmology, Royal Surrey County Hospital, Egerton Road, Guildford GU2 7XX, Surrey, England (firstname.lastname@example.org).
Submitted for Publication: March 24, 2006; final revision received June 7, 2006; accepted June 14, 2006.
Financial Disclosure: None reported.
Thank you for submitting a comment on this article. It will be reviewed by JAMA Ophthalmology editors. You will be notified when your comment has been published. Comments should not exceed 500 words of text and 10 references.
Do not submit personal medical questions or information that could identify a specific patient, questions about a particular case, or general inquiries to an author. Only content that has not been published, posted, or submitted elsewhere should be submitted. By submitting this Comment, you and any coauthors transfer copyright to the journal if your Comment is posted.
* = Required Field
Disclosure of Any Conflicts of Interest*
Indicate all relevant conflicts of interest of each author below, including all relevant financial interests, activities, and relationships within the past 3 years including, but not limited to, employment, affiliation, grants or funding, consultancies, honoraria or payment, speakers’ bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued. If all authors have none, check "No potential conflicts or relevant financial interests" in the box below. Please also indicate any funding received in support of this work. The information will be posted with your response.
Some tools below are only available to our subscribers or users with an online account.
Download citation file:
Web of Science® Times Cited: 23
Customize your page view by dragging & repositioning the boxes below.
Enter your username and email address. We'll send you a link to reset your password.
Enter your username and email address. We'll send instructions on how to reset your password to the email address we have on record.
Athens and Shibboleth are access management services that provide single sign-on to protected resources. They replace the multiple user names and passwords necessary to access subscription-based content with a single user name and password that can be entered once per session. It operates independently of a user's location or IP address. If your institution uses Athens or Shibboleth authentication, please contact your site administrator to receive your user name and password.