Introduction

Whether in sophisticated first-world eye clinics or remote outreach clinics, the sensory aspects of the visual system must be reliably measured.¹ Visual acuity is the most important aspect of screening, diagnosing and monitoring eye diseases² whether in adults or children.^3,4 Stereopsis measures the degree of central binocular function.

Visual acuity requires a presentation format, a selection of optotypes, a paradigm for ordered presentation of the optotypes, a method to ensure monocular testing, a means by which the patient communicates their perception of the optotype. The presentation of optotypes usually has phases including orientation, refinement (approach) and then threshold determination.

Two distance visual acuity protocols have been developed and validated by the Pediatric Eye Disease Investigator Group (PEDIG). In amblyopic patients mean age 9 years, the Electronic Visual Acuity (EVA) surround HOTV protocol predicted 0.68 logMAR finer acuity in amblyopic eyes and 0.25 logMAR finer in sound eyes than the electronic Early Treatment of Diabetic Retinopathy (e-ETDRS) protocol which utilizes the ten Sloan optotypes.⁵

Description of Device

The VisionKeys system (version 3.6.0–127.0.6533.103, EyeMobil, South Bend, Indiana, USA) utilizes an Android operating system, 39 cm touch tablet with a custom multi-key remote designed using symbols instead of English words so as to reduce dependence on any one language. The device runs on A/C power or battery. A control function allows users to adapt to a range of appropriate distances from the patient from 1.5 meters to 7.6 meters. Sound operations can be customized, but version does not have a simple function to adjust screen luminance. The screen can be reversed to operate in a shorter room with a mirror. Several optotypes can be selected including HOTV, Sloan, Tumbling E and Allen pictures and the optotypes can be crowded with an arrow pointer, or with crowding bars. The optotypes are presented in classic Western/US sizes instead of defined logMAR. The device does not yet come with automated paradigm for PEDIG EVA presentation or for eETDRS.

Distance stereo is an option selected from a menu and driven by the same remote when the patient wears a supplied pair of red-blue anaglyph goggles which fit over most prescription spectacles. The stereo is overlapping ring format with fixed disparity between the red and blue visible options. With rapid selection, it is possible to observe the lateral movement of larger, distance stereo discs, thus introducing the possibility a monocular patient could correctly guess stereo on the basis of monocular, dynamic clues.⁶

For either acuity or stereo, the patient must verbally communicate their choice to a screener; there is not currently a self-selection option except for the PDI Check game (PDI Check, Anchorage Alaska, USA) presented on a Nintendo 3DS (Redmond, Washington USA). The name PDICheck is a short form of PeDiatrIc eye.

This study is intended to provide practical validation for the visual acuity and the stereopsis functions of the portable, missions designed VisionKeys system in a group of children and young adults.

Methods

Institutional Review Board (IRB)

The Alaska Blind Child Discovery is a cooperative charitable research project which helps validate objective and sensory vision screening and testing devices. This comparison of screening tests complies with the Declaration of Helsinki and has IRB approval from Providence Alaska Medical Center. Parents and legal guardians gave written informed consent and children over age 7 years gave signed assent. De-identified data is available at https://www.abcd-vision.org/references/VK%20MS%20PDI%20de-ID.pdf.

Sample Size

Given a standard deviation in logMAR acuity of 0.33 to detect a difference in means or 0.1 logMAR, with a power or 0.8 and an alpha of 0.05, a sample size of 142 is needed.⁷

Patient Selection

To avoid validation across only a normal range of visual acuity and to include pediatric as well as young adult patients, subjects were enrolled from a subspecialty pediatric ophthalmology and adult strabismus practice with various types of vision disorders representing a range of visual acuity from superb^8,9 to no light perception (NLP).

The luminance of the screens was checked with Dr. Meter LX1330B (Newark, California, USA) directly placed on the surface finding 606 Lux for the Vision Keys compared to 131 Lux for M&S. The M&S systems (M&S Technologies Niles, Illinois, USA) have controls to adjust for luminance, while the current version of VisionKeys does not. Most patients preferred the brighter screen of VisionKeys to the dimmer M&S screen that had been set to conform with PEDIG EVA and eETDRS protocols. The M&S was set at 3.67 meters and the VisionKeys was placed in alignment and identical room luminance at 3.0 meters.

Optotype Paradigm

First to clarify a definition, rather than describing “better” or “worse” vision, “fine” and “coarse” will describe smaller or larger optotypes on the acuity device screen. Finer implies a level of visual acuity achieved by recognizing smaller optotypes on the screening representing a logMAR of lesser value. Coarser represents larger screen optotypes and a logMAR of greater value. Comparing two visual acuity systems, if the same group of subjects scores a median logMAR of 0.0 on test “A” and a median logMAR on test “B”, we define test “A” as representing “finer” acuity, while test “B” would represent “coarser” acuity. Neither test would necessarily be “better” or “worse” than the other.

Orientation Phase

For our screener, manually selected vision acuity paradigm, using the remote control for either M&S Clinical Trails or the VisionKeys system, the initial “orientation phase” is to help the subject to understand the quantity and variety of the optotypes. This is best done by presenting a matching card to the subject before placing an adhesive patch over either eye and then showing a single 20/200 (logMAR 1.0) optotype on the screen. The subject then says, or points to the identically shaped optotype on the matching card until they consistently can match some or all of them. Incomplete understanding or confusion about some of the options for optotypes does not disqualify the subject, especially if a young or developmentally delayed child, from visual acuity testing, but it reduces the precision and reliability of the eventual outcome. If the optotypes like HOTV are presented on the screen with surround bars, then the matching card should have the identical surround bars representing each optotype. If the logMAR 1.0 is too fine for this subject, then present the largest possible optotype for that screen, which for our study was logMAR 1.7 for M&S and 1.6 for the 39 cm diagonal VisionKeys screen. If the subject can correctly identify, either by saying or by pointing to the matching card, then proceed to the “Approach Phase.” If the subject cannot identify the coarsest optotype size for the distance screen, then there are four options. First is to allow the subject to step down from the exam chair and walk closer to the screen until they can correctly identify in which case the coarseness of the level of acuity is geometrically estimated. The second option is to present finger counting or hand motion from screen distance or at arm’s length from the subject until they can correctly identify number of fingers or direction of hand movement for which logMAR estimates have previously been defined for children.¹⁰ The third option is if the subject seems to be demonstrating inconsistent responses, as is typical for factitious or hysterical denial of vision (non-organic visual loss, conversion disorder, malingering).¹¹ If the patient is consistently saying the opposite, or preceding optotype, then an attempt at the “Approach Phase” might be possible. The fourth option is for a patient who is either too young or developmentally unable to talk or point to matching optotypes. Such subject might have a communicator device, or could potentially perform eye-tracking¹² to determine the initial size and even a threshold fine level of optotype.

Approach Phase

After a consistent “pass” at the initial phase of the visual acuity paradigm, then presenting sequential, finer presentation of single, random-ordered optotypes can be done balancing precision and quickness of testing. During this “Approach Phase” of the visual acuity paradigm, a new optotype one logMAR line finer is presented to determine “pass” or “fail” at that level. If a “pass” then the next finer optotype is presented. If the subject fails to correctly identify the next optotype, or says “too small”, then another optotype at the same fineness level is presented. It is preferred that the subject be encouraged to “guess” even if they say “too small.” Jumping in fineness from logMAR 0.4 to 0.3 is “one line” more fine, whereas jumping from logMAR 0.4 to 0.1 is considered “one octave” or “three lines” more fine of the optotype presentation.

Threshold Phase

The threshold for a visual acuity test is one for which the subject has a very small chance of accidentally guessing to achieve a Threshold Phase “pass.” Computer presented visual acuity tests have advantages over pre-printed wall charts since the subject is unable to memorize the chart or repeat the answers the subjects before them have given. The four alternative HOTV options can be presented in a true random order with one possibility being “H”, “H”, and next “H.” Instead, both the M&S system and the VisionKeys systems are designed so their computer-delivered order of single optotype presentation is “non-repeat, pseudo-random” order where only “O”, “T”, or “V” would follow the “H” optotype at the same fineness level or at a different fineness level. To determine “Threshold” on either M&S or VisionKeys, the subject must correctly identify 3 of 4 presentations at that level. It is possible to miss 1 of the 4 and still “pass.”

Each patient had visual acuity each eye tested by both M&S system and also the VisionKeys system. The screener made the selection of surround HOTV optotypes for each and followed the identical paradigm however with next optotype non-repeat randomized. The level of visual acuity was shown as the 20-foot fraction at the side of each presentation screen for each device. Our M&S system adhered to strict logMAR from coarse of logMAR 1.7 to a floor fine level at logMAR −0.4. The VisionKeys system at 3 meters followed non-research, typical US format starting at 20/600 and proceeding finer through 20/70, 20/60, 20/50, 20/40, 20/30, 20/25, 20/20, 20/15 and finest 20/10. For analysis, the non-exact logMAR (ie, 20/70, 20/60, 20/30) were analyzed by their non-integer logMAR equivalent. The order of testing between M&S system and VisionKeys was randomized. The right eye was tested before the left eye for all patients. Most patients were tested in their current optical correction, but some were tested through the best refined refraction in a phoropter. The testing distance was kept constant in an exam chair with deliberate restraint of any patient who attempted to lean forward for testing.

Monocular Visual Acuity

No Peeking adhesive eye patches (Ad Tape and Label, Menomenee Falls, Wisconsin, USA) were applied to ensure monocular testing.¹³

Stereo

The VisionKeys system offers distance stereo testing by horizontally displacing two uniquely colored rings visible with the accompanying red-blue anaglyph goggles, which fit over the patient’s customary spectacles. For Coarse levels of stereo, the screener who is not wearing anaglyph goggles is able to see a “jump” in the one-out-of-four options, which has displaced rings, but for finer levels of stereo, it may not be possible for even a good-vision screener to be confident which of the four options has the randomly-determined “visual elevation.” Therefore, VisionKeys has provided the option for four keys on the remote to be able to select the corresponding option (i.e upper left or lower right) from which the patient can attempt to identify the appearance of elevated (coming toward you) ring. The level of stereopsis is shown to the side of the VisionKeys screen. The paradigm was to start at the coarsest level of stereo and proceed sequentially until the patient could not perceive the correct “elevated” rings with the finest level of stereo correctly identified once recorded as the stereopsis. This was then converted into log arc seconds of stereopsis for analysis.

The Titmus stereo rings were performed by manufacturer’s guidelines with Polarized goggles over current optical correction viewed at 40 cm starting at the coarsest (800 arc seconds) and proceeding sequentially until the finest level the patient could correctly identify at either the floor level for the test (40 arc seconds) or the next coarsest level to the level unable to be perceived. The patient’s threshold level was then converted to log arc seconds for analysis.

The PDI Check stereo is a dynamic, 4-way forced choice near vision testing tool on the autostereoscopic screen of a Nintendo 3DS game console.¹⁴ In advance of determining a score, the patient undergoes an orientation with coarse presentations of 3-cone color circles, Landolt C visual acuity either “C” or “U” configuration, and also coarse 1070 arc seconds horizontally displaced rings. Once the subject perceives that one of four presentations represents the non-repeat randomized order, they tap the touch screen, which record their lag-time from perception to tap, and then allows the program to non-repeat randomly present the next task. Following orientation for near acuity, color and stereo, the user can select to test for threshold fine levels of right-and-left autostereoscopically (no anaglyph or Polarized goggles) determined visual acuity, stereopsis, and 3-cone right and left eye color vision. Instead of truncated levels of stereo, the PDI Check game yields an estimate of near stereo in continuous arc seconds without a floor.⁹ We used PDI Check version 2.1.7 which has been validated against EyeSwift¹² and other vision screening tools.¹⁵ The direct arc second score from PDI Check was converted to log₁₀ arc seconds for analysis.

Figure 1 shows the VisionKeys system compared to the M&S distance acuity screen, the near Titmus test and the PDI Check game on a Nintendo 3DS console. Utilization of each device is shown on the video abstract that accompanies this manuscript.

Figure 1 The VisionKeys portable vision testing device compared to the M&S distance screen, the Titmus Polarized stereo test and the PDI Check near vision game on a Nintendo 3DS console (right foreground) with NoPeeking patches in the lower left.

Order of Testing

The distance VisionKeys with anaglyph spectacles was tested before Titmus circles with Polarized goggles and then the PDI Check on a Nintendo 3DS autostereoscopic screen.

The Shapiro Wilk test was used to determine normality of data distributions. LogMAR acuity from a highest resolution of 20/8 (logMAR −0.4) spanned to a maximum arbitrarily set to 2.6 for no light perception (NLP) visual acuity.¹⁰ There is debate about whether or not a digital value should be assigned to alpha-designated vision poorer than hand motions (HM) such as light perception or no light perception (NLP),¹⁶ therefore, we also report our results with all patient’s with a blind eye excluded.

Results

From May 2024 through July 2024, 145 patients from a pediatric-adult strabismus practice participated in comparing distance VisionKeys to other conventional distance and near vision tests. Of these patients, 74 were male, and the racial/ethnic distribution was Asian 7, Black 14 Caucasian 80, Hispanic 16, Alaska Native 17 and Pacific Islander 11. Ages ranged from 4 to 38 years with mean 12±6 years with 20 patients less than 7 years old.

The primary vision-influencing diagnostic category for each patient was refractive error in 44, amblyopia in 32, strabismus in 25, neurologic in 11, optic nerve in 9, iritis in 7 cornea in 5, retinopathy of prematurity in 5, eyelid in 3, lens in 2 and denial of vision in 2.

Descriptive statistics including mean ± standard deviation, median, interquartile ranges and the Shapiro Wilk z score indicating lack of normally distributed data for logMAR visual acuity comparing distance M&S versus VisionKeys are shown in Table 1. Table 2 gives Bland Altman results showing significant intraclass correlation (ICC) between M&S and VisionKeys distance visual acuity. The Bland Altman plot of right eye plus left eye logMAR visual acuity comparing M&S with VisionKeys is shown in Figure 2.

Table 1 LogMAR Visual Acuity Comparing VisionKeys and M&S Systems. For the Portable VisionKeys and the Conventional M&S Clinical Trials Systems, Patched Monocular Values (Means ± Standard Deviation and Median) for Right Eye, Left Eye, Inter Eye Difference (Right – Left), and Combined Right Plus Left. IQR1 and IQR3 are First and Third Inter Quartile Ranges While Min is the Minimum logMAR Acuity While Max is the Maximum. Shapiro Wilk Demonstrated That These Data Were Not Normally Distributed

Table 2 Bland Altman Statistics Comparing VisionKeys and M&S Systems. The Metrics for Bland Altman Analysis Showing the Mean Difference Between VisionKeys and M&S Systems, the Standard Deviation of the Difference, the Intraclass Coefficient and the Sample Size for Right Eye, Left Eye, Difference Between the Two Eyes and Analyzing Right and Left Eye Visual Acuities (Right + Left) and Recalculated with the Two Cases with Left Eye No Light Perception (NLP) Excluded

Figure 2 Bland Altman comparing logMAR distance visual acuity evaluated with M&S and VisionKeys right eye and left eye (n=290).

The medians of logMAR visual acuity right plus left eyes (M&S 0.10, VisionKeys 0.00) were significantly different (Mann–Whitney z=2.6, p=0.01, n=290). The 125 older patients had median finer visual acuity than the 20 patients under age 7 years for both M&S (logMAR 0.10 versus 0.20, Mann Whitney z=3.2, p=0.001) and for VisionKeys (logMAR 0.0 versus 0.10, Mann Whitney z=3.3, p=0.001) as shown in Figure 3. The Bland Altman for inter-eye difference logMAR distance visual acuity is shown in Figure 4.

Figure 3 The impact of age on the distribution of logMAR distance visual acuities for right and left eyes tested with the M&S and the VisionKeys systems.

Figure 4 A Bland Altman plot of the intereye difference in distance logMAR visual acuity comparing M&S and VisionKeys systems.

Values for log arc stereo for VisionKeys, near Titmus circles and near PDI Check are given in Table 3 and metrics of Bland Altman analysis for Stereo are shown in Table 4. The medians of the log Stereo (Titmus 1.78, PDI Check 1.89 and VisionKeys 2.00) were significantly different (Kruskal–Wallis X²(2)=9.8, p=0.007). The Bland Altman 2-way comparisons of log arc seconds stereo measured by distance VisionKeys, near Titmus circles and near PDI Check autostereoscopic games are shown in Figure 5.

Table 3 Statistics for Stereo Comparing VisionKeys (Distance) with Near Titmus Circles and PDI Check. The Shapiro Wilk Indicated That Logarithm of Arc Seconds Stereo Data Were Not Normally Distributed, Therefore in Addition to Mean ± Standard Deviation, the Median and First and Third Interquartile Ranges (IQR1, IQR3) are Given Including the Range from the Minimum (Min) to the Maximum (Max)

Table 4 Bland Altman Statistics for Stereo in Logarithm Arc Seconds (Log Arc Sec): for Vision Keys (Distance) Compared to Titmus Circles (Near) and PDI Check (Near). To Describe the Significant, Positive Correlation Between These Device’s Measurement of Logarithm of Arc Seconds of Stereo Showing the Mean Difference, the Standard Deviation of the Difference and the Inter Class Correlation (ICC) with the Sample Size (n)

Figure 5 Three Bland Altman plots of log arc seconds stereo with 2-way comparisons between distance VisionKeys, near Titmus circles and near autostereoscopic PDI Check.

The order of eye testing did not appear to matter since the right and left visual acuities were not significantly different (M&S; Mann Whitney z=1.48, p=0.1 and VisionKeys; Mann Whitney z=1.52, p=0.13).

Discussion

Utilizing state-or-the art M&S clinical trials visual system and surround HOTV optotypes, and Bland Altman analysis, the new VisionKeys portable distance vision system showed excellent similarity. On the other hand, the median performance by ethnically diverse children and young adults with a variety of visual disorders scored finer logMAR visual acuity for VisionKeys compared to M&S by about 1 logMAR line. Patients preferred the brighter screen on the VisionKeys, which was not adjustable in the current version of the device to afford the option of luminance comparable with the standardized M&S system. The M&S system presents optotypes strictly matching logMAR from −0.4 to 1.7. In contrast, the VisionKeys follows conventional US format including 20/30, 20/60 and 20/70 which do not exactly align with logMAR visual acuity increments. The multi-language, internationally-designed remote and the touch screen worked well in a luminance, temperature and humidity controlled ophthalmology office. Further validation experience in remote clinics is warranted.

The VisionKeys provided valid estimates of stereo at a distance using anaglyph goggles and discrete, instant horizontal shifted figures rather than random dot format. By Bland Altman, the distance stereo on VisionKeys matched well near stereo by traditional Titmus circles and also the dynamic, autostereoscopic forced choice PDI Check system. It was possible for patients with good vision to notice the shift in horizontal displacement when a coarse level was presented. As such, a monocular, or non-stereo patient might falsely pass a stereo level when they actually only monocularly perceived a ring shift instead of true depth. Providing a short gap between showing any of the ring images might remedy that minor deficiency.

Other portable devices have been utilized to measure visual acuity and/or stereopsis. In absence of a computer screen, a flip card can present single, “randomized” optotypes with or without surround bars calibrated for a specific distance capable of yielding very valid (ICC 0.92, standard deviation of the difference 0.10 re-analyzed) results.¹⁷ The EyeChart Pro on an iPad did not perform as well as a light box unless visual acuity was better than logMAR 1.0.¹⁸ Head-mounted display Snellen did not perform as well as a wall chart even in the 88% capable of performing the portable test.¹⁹ The sensitivity of smart phone visual acuity using the Peek Acuity Test was similar to a back-lit wall chart for Kenyan children.²⁰ Several portable acuity charts and Frisby and Titmus stereo compared in children in India, with preferred optotype being HOTV.²¹ The EVA test and our paradigm deliver levels of visual acuity roughly 1 logMAR apart, while the eETDRS protocol can give reliable acuity increments to 1/5 of a logMAR line. When various visual acuity tests are directly compared to define “Optimum”, “Excellent”, “Good”, “Fair”, and “Passing” similarities of ICC and Bland Altman distribution, test-reTest of eETDRS in cooperative patients can yield excellent Bland Altman scores with a standard deviation of the difference as low as 0.06 logMAR.⁸ Encouragingly, in this validation study, our compared VisionKeys versus M&S yielded “optimum” levels of ICC and standard deviation of the difference between the two tests.

Conclusion

The simplicity, the size, the excellent remote and programming and now the validation make the VisionKeys a promising addition to the equipment armamentarium for remote eye clinics, and it works well in first-world, pediatric ophthalmology clinics as well.