Introduction

Glaucoma is a leading cause of irreversible blindness all over the world. Sixty percent of global glaucoma cases are found in Asia, where the disease is the major cause of legal blindness. The global increase of glaucoma over the next 15 years will be largely in Asia, where by 2040 there will be 18.8 million people with primary open-angle glaucoma (POAG) and 9 million with primary angle closure glaucoma (PACG).¹ Early diagnosis, timely intervention and lifelong management will ease the rising burden of glaucoma not only in Asia but throughout the world.

Visual field assessment is key to detect and monitor glaucoma. Conventional perimetry techniques require expensive equipment with well-trained staff in specialized settings.^2,3 Desktop computers and laptops are common devices. Any online application only requires a basic computer, keyboard, mouse and internet connection. Therefore, digitalizing visual field testing paves the way for teleglaucoma care and enhances accessibility to glaucoma screening and management services, especially in remote Asian territories with lower levels of economic development.^4–7 Several alternative visual field techniques have been introduced such as tablet-based perimetries^8,9 and virual reality perimetries)^10–14 to complement traditional standard automated perimetry (SAP) by providing portable perimetry. These technologies have been demonstrated to have varying degrees of agreement with SAP and test–retest stability.

Several studies on online perimetry have been conducted in Caucasian populations.^4–7 However, data from an Asian population regarding the correlation and agreement between the new approach and standard automated perimetry (SAP) are lacking. A web-based solution like online circular contrast perimetry (OCCP) that targets a wide range of populations globally will be strengthened by a comprehensive database covering different ethnic groups with a variety of genetic, cultural and linguistic backgrounds, allowing appropriate fine tuning of test parameters to different population groups. Furthermore, the Australian-made software should be piloted in different internet infrastructures to ensure it is robust and compatible with any network globally. Hence, we performed the current study to compare OCCP and SAP in a Vietnamese clinical setting.

Methods

Our research strictly followed the tenets of the declaration of Helsinki and has been ethically approved by Hanoi Medical University (IRB-VN01001). All participants signed the consent form and were informed about the study purpose.

Participants

The current study was conducted at Hanoi Medical University in 2023 on stable glaucoma patients and healthy volunteers with eligible participants consecutively enrolled. Inclusion criteria included best corrected visual acuity (BCVA) ≥20/60; controlled stable PACG or POAG; no other ocular disease except for glaucoma (including maculopathy, other optic neuropathies, cataract grade II or above¹⁵), reliable visual field indices on SAP (with false positive (FP) rate < 15%, false negative (FN) rate < 33% and fixation loss (FL) rate < 20%), and previous experience in using computers and internet-based web browsers. Exclusion criteria included secondary or developmental glaucoma, spherical refraction error ≥±5 diopter, non-glaucomatous optic disc anomalies, central nervous system disease, medications that might alter the visual field (chloroquine, vigabatrin, and pilocarpine), and previous vitreoretinal surgery.

If visual acuity met inclusion criteria, both eyes from each patient were included in the study. Glaucoma diagnosis and classification was based on pre-established criteria outlined by the American Academy of Ophthalmology and Hodapp-Parrish-Anderson system.^16–18 The control group included healthy volunteers with normal intraocular pressure (IOP 10–21 mmHg, Goldmann Applanation Tonometry), optic nerve head, retinal nerve fiber layer (RNFL) and no other ocular pathology.

Clinical Examination

Participants underwent a comprehensive assessment with structural testing at baseline and visual field testing at baseline, after 3 months and 6 months (Figure 1). In clinic, patients underwent visual field testing using SAP (Carl Zeiss Meditec Inc, Dublin, California, USA) 24–2 test SITA Standard (Swedish Interactive Threshold Algorithm Standard) and 24 degree OCCP (Eyeonic Pty Ltd, Australia). Visual field reports were considered reliable when FP < 15%, FN < 33% and FL < 20%. Glaucoma visual field damage was defined as per the Hoddapp–Parrish–Anderson criteria.^16,19 Optical Coherence Tomography (Cirrus, Zeiss) scans of macula and peripapillary RNFL were obtained within 2 months of the baseline visit. Reports with signal strength <7/10 were excluded.

Figure 1 Study protocol.

Online Circular Contrast Perimetry

OCCP technology has been described in several recent papers.^4–7,20 Essentially, OCCP works on any computer or tablet by providing perimetry through a web browser. Participants are shown flickering circular targets with alternating rings of light and dark; these are spaced six degrees apart and have a diameter of 3.5 degrees of viewing angle (Figure 2A). These targets share similarities with Pulsar Perimetry (Haag-Streit International); however, they maintain consistent contrast across their spatial extent, with a slight contrast reduction at the peripheral edges to minimize light scatter and unintended stimulation of ganglion cells. In addition, OCCP targets enable a more detailed spatial distribution of visual field loss compared to those used in Pulsar perimetry, being comparatively smaller in size (3.5 vs 5 degrees).^21,22 52 loci over 24 degrees of peripheral vision are evaluated by OCCP (Figure 2B). For a total of 360 milliseconds, each target flickers every 60 milliseconds during three on/off cycles. Targets contain temporal counter phase flickering at 9 hz and sinusoidal contrast with spatial frequency of 0.55 cycles/degree, similar to frequency doubling perimetry (FDP, Welch Allyn, Skaneateles, N.Y., and Carl Zeiss Meditec, Dublin, Calif).^23,24 In order to avoid temporal transients and saccades, the contrast is linearly ramped up and down over 30 milliseconds at the beginning and end of target presentations (Figure 2C).^25,26 Unlike typical FDP, where target bands are adjusted around a mean of background luminance, OCCP’s light rings are adjusted to match the color of the background screen (light grey), and the dark rings’ intensity is adjusted to achieve the desired contrast. This is comparable to utilizing a luminance pedestal flicker for stimulus decrements, and this approach is used to reduce the number of greyscale colors in the target and backdrop design for maintaining consistency of display settings with gamma correction.²⁷

Figure 2 Online circular contrast perimetry features – figure adapted from Meyerov et al5,39 (A) Flickering test target. (B) Map of 24–2 perimetry loci testing for the inferior hemifield. The fixation object then shifts to the bottom of the screen to assess the superior hemifield. In order to compensate for the eccentric view from a flat screen as opposed to a Ganzfeld bowl, the size of test targets displayed on the screen has a magnification factor that grows with eccentricity. (C) Target presentation sequence: contrast is graded at the beginning and end of target presentation, and targets appear for three 360-millisecond counterphase flicker cycles. Figure adapted from Alawa et al43 (D) Fixation target: spinning golden star. (E) Blind spot localization optimizes the distance at which the user can see. (B and E) The diagrammatic representation of potential test target locations is provided by the dark and grey homogeneous circles.

The output of luminance varies from 100% relative brightness percentage for pure white (255, 255, 255) to 0% for black (0, 0, 0). Using the Web Content Accessibility Guidelines as a guide, relative brightness was computed for every 256-greyscale level.²⁸ By measuring the target rings’ relative luminance in the light and dark bands, the Michaelson formula is used to determine contrast.²⁹

Contrast = (RL1 – RL2)/(RL1 + RL2)

Where RL2 is the dark band minimum and RL1 is the light band maximum relative brightness. Screen brightness was increased to 90%, corresponding to 224 candela per square meter (cd/m2). After that, contrast was transformed to relative decibels using a similar approach to what is used for FDP.²³

Relative decibel (rdB) = - 20log(contrast)

Target intensity has a dynamic range of 0 to 36 dB, which is comparable to the range used in HFA and other perimetry devices to support human threshold estimates.⁸ The OCCP testing process is based on an a priori probability density functions with a 4/2 decibel (dB) staircase, similar to SITA.

Participants are asked to focus on a golden star that is continuously spinning (4 degrees of visual angle), and to click the mouse whenever a target appeared in their peripheral vision (Figure 2D). As demonstrated by our earlier research, using a dynamic fixation target may aid in enhancing focus and lowering fixation losses.⁴ Clicks outside of the permitted response window are noted as FP responses. Similar to SAP, FN responses are noted when subjects do not react to stimuli at greater contrast than recorded in regions of previously determined threshold sensitivity.³⁰ Responses are time-adapted based on the user’s prior reaction time to account for inter-user variability; random delays are also incorporated to avoid rhythmic responses.³¹

Three mechanisms keep the viewing distance correct. First, in order to guarantee viewing angle uniformity, the app instructs users on the proper viewing distance to utilize for the test. This distance is determined trigonometrically based on the size of the computer monitor. Second, small patches on a 4 × 10-degree grid covering the suggested blind spot—which was assessed to be 15 degrees temporal and 0.5 degrees inferior to fixation—are tested to map the user’s blind spot (Figure 2E). Third, a webcam is used to track the user’s head movement. This feeds data to the app’s machine learning-powered facial detection (as opposed to recognition) software, which has a one-second refresh rate. There was no usage of specific head or neck supports.

When the test found deviations in face position monitoring of more than 15% in four planes, it was stopped, and the user was given instructions to correct their position. The spoken instructions in Vietnamese from OCCP were recorded in advance, but other language options are integrated into the application (Figure 3).

Figure 3 (A) Example of Vietnamese pre-test instructions, translated to: 1. Please sit 50cm from the screen. 2. Close your right eye. 3. Press space bar or click the mouse button when the target appears. 4. Press space or click the mouse button to begin. To pause press “return”, to abort press “delete”. (B) Testing condition of OCCP in Vietnam.

Dark, large flickering targets on a light grey background provides increased resistance to changes in background lighting compared to other forms of perimetry.³² The properties of the test including spatial frequency, target size, flicker rate and background colour have all been carefully chosen to maximize consistency despite potential differences in testing environments and computer screen output displays.³³ Such OCCP design features have been carefully included to account for variations in screen output and background brightness. In addition, the application contains internal calibration mechanisms based on early responses. However, to maximise consistency for this study, all computers had brightness increased to at least 90%. The OCCP application detects different screen sizes and automatically resizes. All these features aim to ensure calibration consistency.

Testing Conditions

The ambient lighting, temperature, and background noise levels were all standardized during the testing, which took place in a controlled clinical setting. SAP was carried out in a specialized, silent, and dimly lit space. Regular fully-functioning laptops in quiet, undisturbed clinical rooms were used for OCCP testing at one location. The backdrop lighting was turned down so that the computer monitor served as the primary source of illumination. Every computer featured its own webcam, volume control, mouse, and internet connectivity.

To guarantee the uniformity of research protocols, all optometrists had received additional training for delivering OCCP and possessed a great deal of experience in operating perimetry. Before the test started, participants were situated at the appropriate viewing distance of 50 cm for this size screen. One eye was patched. One of the trained optometrists supervised the test and made adjustments to head position and height. The participant was given clear and easy-to-understand instruction before starting a trial evaluation. If the participant was unable to complete the first test, they rested and were encouraged to repeat the visual field assessment.

Research Outcomes

Primary outcome measures included global perimetric indices such as mean deviation (MD), pattern standard deviation (PSD) and Visual Field Index (VFI)/Visual Index (VI). Calculation of outcome measures was dependent on data obtained from an established normative dataset.⁷

Secondary outcome measures included age, gender, diagnosis, disease severity, visual acuity, IOP, refraction, RNFL, central cornea thickness (CCT), and perimetric test duration.

Statistical Analysis

Descriptive statistics included frequency and percentage for qualitative measures, and mean and standard deviation for quantitative measures. Statistical comparative tests included t test, intraclass correlation coefficient (ICC), Pearson correlation, linear regression, Bland–Altman plot and receiver operating curve (ROC) analysis. The level of statistical significance applied was p <0.05. SPSS 23 and R were employed for data analysis. ICCs were defined as excellent (≥0.90); good (0.75 to 0.9); moderate (0.5 to 0.75); or poor (<0.5).³⁴ Pearson’s values were defined as strong (>0.75); moderate (0.45 to 0.75); weak (0.25 to 0.45) or very weak (<0.25).³⁵ As the data involved two eyes from most cases leading to potential clustering effects, generalized estimating equations were used to adjust for intra-subject correlation, and the Hanley and McNeil test was applied to compare ROC curves. Sample size was calculated based on expected AUC results using perimetry in a mixed glaucoma/normal cohort. The AUC ranges from 0.7 to 0.89;^4,5 to be conservative, we used the lower end of the range to ensure appropriate sample size, taking into account the potential intraclass correlation of two eyes from each individual. Using MedCalc software³⁶ with an alpha of 0.05 and type 2 error 0.2, a sample size of 64 was obtained. To account for subject dropout, we aimed to recruit slightly greater numbers, which made up 87 patients.

Results

One hundred sixty-eight eyes of 87 patients were recruited, of whom 58 were in the glaucoma cohort and 29 were controls. Their characteristics are outlined in Table 1. Eighty-one patients with 2 eyes and 6 patients with 1 eye were included. Sixty-nine patients (79.3%) attended the 3-month follow-up visit, and 63 patients (72.4%) attended the 6-month visit.

Table 1 Research Population Characteristics

Generally, OCCP generated similar visual field findings to SAP across different stages of glaucoma (Figure 4).

Figure 4 Visual field pattern and total deviation plots for individuals with mild, moderate and severe glaucoma; comparing OCCP and SAP.

There was no significant difference between test duration, FL, PSD, VFI/VI of SAP and OCCP (Table 2). FP was higher in the OCCP group (p < 0.001) and FN was higher in the SAP group (p < 0.001). MD was lower in the SAP group. (p < 0.001).

Table 2 Test Duration, Reliability Indexes and Global Visual Field Indexes of SAP and OCCP

At baseline, OCCP and SAP MD and VI/VFI were strongly correlated, while OCCP and SAP PSD were moderately correlated (Table 3). OCCP parameters were positively strongly correlated with superior and inferior RNFL thickness on OCT. PSD negatively moderately correlated with superior and inferior RNFL thickness on OCT.

Table 3 Pearson Correlation Between SAP, OCT and OCCP at Baseline

Linear regression analysis demonstrated a significant correlation between OCCP and SAP MD (R² = 0.804, p < 0.001); and VFI (R² = 0.892, p < 0.001) (Figure 5).

Figure 5 Linear regression between OCCP and SAP MD, VFI.

At baseline, Bland–Altman plots showed significant agreement between OCCP and SAP MD (Bias 1.11; Lower LoA −6.73; Upper LoA 8.95), and VI/VFI (Bias 1.57; Lower LoA −16.63; Upper LoA 19.76, Figure 6A and B). Across 6 months, Bland–Altman plots demonstrated significant agreement between baseline and 6-month MD (Bias 0.38; Lower LoA −6.18; Upper LoA 6.94), VFI (Bias −0.90; Lower LoA −17.43; Upper LoA 15.63) of OCCP and VFI (Figure 6C–F).

Figure 6 Bland–Altman plots of OCCP and SAP MD, VI/VFI. (A) OCCP v SAP MD at baseline visit. (B) OCCP v SAP VI/VFI at baseline visit. (C) OCCP MD at 6 months v baseline. (D) SAP MD at 6 months v baseline. (E) OCCP VI at 6 months v baseline. (F) SAP VFI at 6 months v baseline. The continuous horizontal line represents the mean difference (bias) between tests, dashed and dotted horizontal lines represent the 95% limits of agreement (Bias ± 1.96SD). Black-coloured circles represent controls and white-coloured circles represent glaucomatous eyes.

OCCP and SAP area under the curve (AUC) are presented in Table 4. Hanley–McNeil analysis showed no significant difference between AUC of OCCP and SAP MD, VI/VFI at most points of time except for VI/VFI at 6 months, at which time SAP VFI was significantly higher than OCCP VI (p = 0.036).

Table 4 Comparison Between AUC of OCCP and SAP MD, VI/VFI at Baseline, 3 Months and 6 Months

Consistency over time was assessed with ICC and Pearson correlation over the 6-month period (Table 5). ICCs over time were good or excellent, and Pearson’s values strong, for MD and VI/VFI for both forms of perimetry, and stronger for SAP than OCCP.

Table 5 Intraclass Correlation Coefficient and Pearson Correlation Between OCCP and SAP Over 6 Months

Discussion

The current study explores the feasibility, usability and diagnostic accuracy of OCCP compared to SAP, with moderate to strong agreement to SAP global indices at baseline and over time. While test–retest stability over time was very strong for both tests, it was stronger for SAP. This data is the first such data for OCCP in an emerging Asian economy and is consistent with previous studies performed in Australia.^4–7,20

In the current study, it was found out that OCCP and SAP MD and VI/VFI had strong correlation while PSD had moderate correlation. This was consistent with a study of Meyerov et al (MD 0.90, p < 0.0001; PSD 0.87, p < 0.0001).⁵ Comparing SAP with Melbourne Rapid Fields (MRF, Glance Vision, Australia), a tablet-based perimetry, Kong et al pointed out that MD and PSD were significantly correlated (MD 0.9, PSD 0.7, Pearson Correlation) between MRF and SAP.⁸ In our study, OCCP parameters were significantly correlated with superior and inferior OCT RNFL thickness, indicating a structure-function correlation. These findings support previous OCCP papers. Using the Hanley–McNeil method to compare AUC for OCCP MD and OCT RNFL inferior thickness, no significant difference was found between the two diagnostic modalities in terms of ability to discriminate between glaucomatous and normal eyes.^4,5

Linear regression analysis demonstrated a significant correlation between OCCP and SAP MD (R² = 0.804, p < 0.001); and VFI (R² = 0.892, p < 0.001). Comparing OCCP and SAP, Meyerov et al showed that OCCP and SAP MD was significantly correlated with R² = 0.8, p < 0.0001.⁵ Kong et al found a similar result for MRF and SAP (R² = 0.8).⁸ At baseline, Bland–Altman plots showed significant agreement between OCCP and SAP MD (Bias 1.11; Lower LoA −6.73; Upper LoA 8.95), VFI (Bias 1.57; Lower LoA −16.63; Upper LoA 19.76). This finding was lower than the earlier OCCP papers of Meyerov et al⁴ (MD 4.3 dB) and Skalicky et al⁷ (MD 4.02 dB), but close to the more recent OCCP papers of Meyerov et al⁵ (MD 0.51 dB) and Meyerov et al²⁰ (MD 0.38 dB). The result reflected the evolution of OCCP technology, refining the agreement between the online and conventional perimetry. Hanley–McNeil analysis showed no significant difference between AUC comparing MD and VFI/VI for OCCP and SAP at baseline. The finding was consistent with studies of Meyerov.^4,5

OCCP targets share some common attributes with Pulsar Perimetry (Haag-Streit International), which has been shown to be useful for detecting early glaucoma.²² In another study, Zeppieri found that Pulsar T30W was a rapid and easy perimetric technology, which demonstrated higher sensitivity than SAP in early diagnosis of glaucomatous visual field damage.²¹ The finding was later on confirmed in a study of Hirasawa, which revealed that Pulsar perimetry was able to diagnose early glaucoma (pre-perimetric on SAP) with similar capacity to Spectral Domain OCT.³⁷

OCCP demonstrated strong repeatability over 6 months. Bland–Altman plots demonstrated significant agreement between baseline and 6-month MD (Bias 0.38; Lower LoA −6.18; Upper LoA 6.94), and VI (Bias −0.90; Lower LoA −17.43; Upper LoA 15.63). The bias was comparable to that after 18 weeks in a study of Meyerov²⁰ (MD bias 0.87 dB). Hanley–McNeil analysis showed no significant difference between AUC of OCCP and HVF MD, VFI at most points of time except for VI at 6 months.

Pearson and ICC analysis for each perimetry method demonstrated a very strong correlation between baseline and 6-months for MD and VI/VFI, ranging from 0.85 to 0.98. This was comparable to a study of Meyerov that showed OCCP MD ICC at 18 weeks was 0.87 (95% CI: 0.49–0.99).²⁰ According to another study of Prea on MRF, MD ICC at 6 months was 0.87 (95% CI: 0.77–0.92).³⁸

In some ways, the repeatability of OCCP was not as strong as SAP, with lower correlation metrics between baseline and 6 months for OCCP than SAP. Despite the instruction provided for OCCP, participants were likely more familiar with SAP than OCCP, which might lead to potential bias and explain the difference between two modalities. However, the strength of correlation for OCCP at baseline versus 6 months was still high. This is in keeping with previous studies.²⁰ This needs to be considered in light of the other advantages of OCCP over SAP, namely reduced cost per test, ease of use and access, and the potential for frequent home monitoring. With further software refinement and the use of database optimisation, it is hopeful that the accuracy of OCCP will improve over time.

We found no difference between OCCP and SAP for test duration, FL, PSD and VFI/VI. This is similar to previous OCCP papers; for Meyerov et al, the test durations of two methods were OCCP 5.29 ± 1.24 (sec) and SAP 6.00 ± 1.05 (sec).²⁰ It was also similar to a study of Kong et al which showed the test duration of MRF and SAP were 5.7 ± 0.1 (sec) and 6.3 ± 0.1 (sec), respectively.⁸ However, MD was lower in SAP group, FP was higher in OCCP group, and FN was higher in SAP group. FP is influenced by different perimetric paramaters such as target presentation time and inter-trial interval, while FN is influenced by the brightness difference between the test target and estimated threshold, and proximity to a scotoma. It is challenging to interpret the differences in FP, FN rate and MD between these two perimetric methods, given the fundamental differences in testing parameters. Overall, test design, participant familiarity or device sensitivity might contribute to these differences.

Both OCCP and SAP had a mean of around −2 dB in the control groups. This is different from previous OCCP studies, which found a mean of 0 dB in control groups.^4,5,39 This may reflect some overall unfamiliarity with perimetry among healthy Asians, and also some normative data issues when applying normative datasets generated in primarily Caucasian populations to Asian ones – hence the importance of validation of a normative Asian cohort.⁴⁰

This study focused on 24–2 perimetry; however, OCCP also offers 10–2, 30–2 perimetry as well as licensing assessment. These extra modalities are important for comprehensive glaucoma services. A study of Chen establishing the normative database for central 10-degree of OCCP demonstrated that 10-degree OCCP had similar perimetric results compared to SAP in a normal cohort.⁶

Visual field assessment is critical for glaucoma detection and monitoring, especially in Asia, the continent with such a large and growing proportion of glaucoma cases globally.¹ Developing economies with limited resources in this region encounter critical issues such as funding restraints, lack of well-trained human resources and infrastructure. Even in developed countries like USA, less than 75% of glaucoma patients have their visual field assessed annually.⁴¹ It is recommended that within the first 2 years, newly-diagnosed glaucoma patients should complete at least one test every 4 months, and then at least 1–2 tests should be performed regularly depending on the disease progression and severity.⁴² Inevitably, a new gamechanger, which could be cheaper, more popular and easier to be accessed will tackle the mission of providing regular visual field tests in developing Asian economies.

A study of 95 glaucoma patients and controls surveyed users’ experience of OCCP compared to SAP. Participants demonstrated a strong preference for the web-based test than the conventional one (p < 0.0001).⁴ OCCP will improve the user experience of perimetry, enhance the accessibility to visual field service, allow earlier detection and prompt treatment of glaucoma progression and pave the way for teleglaucoma. As an online application, using pre-recorded voice prompts and written text clips as demonstrated in Figure 3, it can be easily translated to multiple languages, improving the cultural use experience of perimetry and accordingly the co-operation of individual patients and accuracy of results.

The current study has some limitations. Drop-out is an issue of any longitudinal study. However, analysis of non-dropout versus dropout participants at three- and six-month follow-up demonstrated no difference, indicating that the influence of missing data was unremarkable. (Supplement) Selection of participants from a single practice is a potential weakness, but this is compensated for several studies of OCCP from across the globe. A significant difference in age between glaucoma and control group was noted – this is due to the fact that glaucoma damage accumulates with age, however, it can cause some confounding of comparative group results. The control group was not matched with glaucoma group in terms of age, sex and relevant factors, which might impact the reliability of abnormality detection results. Using data from both eyes from each patient can result in inadvertent correlation and confounding. A comparison of baseline features of right eye and left eye was conducted (Supplement), and found no difference was found between both eyes. Therefore, the intra-subject correlation is less likely to impact our findings. Due to the large volume of data, we were unable to analyse individual test points, which would be intersting in future studies. Despite the challenges of providing comprehensive glaucoma care and additionally recruiting for studies in a developing-world setting, we were able to recruit a sufficient study population within an acceptable timeframe to generate the very first database for Asian people to use online perimetry.

In conclusion, the performance of OCCP seemed to be similar to the performance of SAP. Over 6 months, OCCP demonstrated excellent repeatability. OCCP holds the potential to provide a complimentary role to SAP in glaucoma screening and monitoring in clinic as well as at home.