Introduction

Obstructive sleep apnea (OSA) has been widely recognized as a prevalent disorder and its prevalence has been increasing worldwide over the past two decades,¹ probably in association with an aging population and increasing obesity. However, many patients may remain undiagnosed. Although in-laboratory polysomnography is regarded as the gold standard of OSA diagnosis, it is costly, labor-intensive, has limited accessibility, and often has long waiting times.^2–4 To resolve these issues, home sleep apnea tests (HSATs) with a polygraphy device has been utilized in the clinical setting and could facilitate the diagnosis of OSA.^5–7 Since the coronavirus disease 2019 (COVID-19) pandemic, HSATs with polygraphy devices, are increasingly important for the evaluation of OSA.^8–10 Alice NightOne (Philips Respironics, Murrysville, PA) is a type 3 polygraphy device that equips a pulse oximeter, a respiratory effort belt for respiratory inductance plethysmography, nasal pressure sensor, and position sensor and has been used for HSAT in clinical settings.^11–13 Generally, the data recorded from this device are scored manually by sleep medicine specialists, and thus requires time to obtain results. This may be burdensome for both patients and sleep specialists.

The recorded HSAT data with this polygraphy device can be scored using Sleepware G3 (Philips Respironics), a comprehensive software/workplace package that has mainly been developed for use with fully equipped polysomnography systems. This software/workplace is equipped with a default automatic scoring function that is generally used to help with manual scoring;^12,13 however, adding an advanced application with an integrated computer-assisted Somnolyzer scoring system (Philips Respironics) can yield more reliable scoring results. Somnolyzer was developed as an automatic scoring application for full polysomnography,^14–16 but it can also be utilized for the automatic scoring of data from this polygraphy device, thus further alleviating the burden of manual scoring for patients and sleep specialists. However, the validity of this Somnolyzer-based automatic scoring with this polygraphy device has not yet been specifically investigated.

Thus, in this study, we aimed to compare the respiratory event index (REI), a measure of OSA severity, obtained by automatic scoring versus manual scoring when using a polygraphy device rather than when such automatic scoring is performed with data obtained by polysomnography.

Materials and Methods

Patients

Data from patients with suspected OSA who underwent overnight HSAT with this polygraphy device at our sleep outpatient clinic from January 2017 to April 2018 were assessed. The following patients’ data were excluded from the study: (1) age, <20 years; (2) history of heart failure; (3) history of atrial fibrillation; (4) history of stroke with neurological deficit; (5) undergoing dialysis; (6) history of chronic pulmonary disease; (7) use of oxygen inhalation at the time of HSAT; (8) undergoing treatments for OSA at the time of HSAT; (8) total HSAT recording time <180 min; and (9) poor signals from the HSAT device and/or inappropriate installation of HSAT sensors. The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of Juntendo University Hospital (#17-311) with a waiver of the requirement to obtain informed consent from the patients using opt-out methods, as this research used anonymized data from human participants taken during routine clinical management.

HSAT Device

This type 3 polygraphy device includes a pulse oximeter, a respiratory effort belt for respiratory inductance plethysmography, nasal pressure sensor, and position sensor. This device measures oxyhemoglobin saturation (SpO₂) with a sampling frequency of 75 Hz with the averaging time of 4 beats (dependent on pulse rate), airflow, respiratory effort, snoring, posture, and pulse rate.

Scoring of Polygraphy

All recorded data from eligible patients were retrospectively analyzed using the following three methods: 1) manual scoring by an experienced sleep specialist blinded to the automatic scoring results, 2) automatic scoring using the default function of Sleepware G3, and 3) automatic scoring using the Somnolyzer system. Apneas and hypopneas were scored based on these three methods. The REI, defined as (total number of apneas and hypopneas) / (total recording hours), and other indices were calculated. The REI values obtained by manual scoring, automatic scoring with the default function, and scoring with Somnolyzer are expressed as mREI, aREI, and sREI, respectively.

Manual scoring was performed according to the American Academy of Sleep Medicine (AASM) scoring manual version 2.4.¹⁷ In brief, apneas were scored if the maximum signal fluctuation on the nasal pressure sensor was ≤90% of the baseline before the event and if the duration was 10s or more. If the respiratory effort was sustained or increased during the entire period when the airflow was stopped, it was judged as obstructive. If respiratory effort is lost during the entire period when the airflow stops, it is considered central. If respiratory effort was lost during the early part of the event and then resumed, it was judged as mixed. Hypopnea was scored if the maximum signal fluctuation on the nasal pressure sensor was ≤30% of the baseline before the event, it lasted 10s or longer, and was accompanied by a decrease in oxygen saturation of ≥3%.

Automatic scoring using the Sleepware G3 default function also provides the number and type (obstructive, central, or mixed) of apneas, number of hypopneas, snoring, and desaturation. The scoring criteria were the same as the aforementioned AASM scoring criteria; however, all measurements of airflow, respiratory effort, and desaturation were performed automatically by an algorithm.

Somnolyzer is an application with an integrated computer-assisted scoring system that can optionally be added to Sleepware G3. It consists of an advanced automatic scoring algorithm developed in collaboration with over 40 experts in sleep medicine and biosignal analysis from 12 universities.^14,15 The Somnolyzer system also provides the number and type (obstructive, central, or mixed) of apneas, number of hypopneas, snoring, and desaturation.¹⁶ Although this automatic scoring system was mainly developed for the automatic scoring of full polysomnography data, no specific comparison data against manual scoring regarding respiratory events by polygraphy, especially by this polygraphy device, have been reported. With the Somnolyzer system, each event is detected using multiple analysis methods with an automatic management program against noise signals, enabling scoring similar to that achieved by experienced technologists.

Statistical Analysis

Because REIs are fundamentally similar parameters, we assumed that there would be a strong correlation between mREI and sREI. We set the correlation coefficient between them as 0.70 and found that we needed the data of at least 13 patients to be able to reject the null hypothesis that this correlation coefficient equals zero with a power of 0.8 when the Type I error probability associated with this test of this null hypothesis is 0.05. Meanwhile, we assumed that the correlation between mREI and aREI would be weaker than that between mREI and sREI. Thus, we set the correlation coefficient between them as 0.60 and found that we needed the data of at least 19 patients to be able to reject the null hypothesis that this correlation coefficient equals zero with probability (power) 0.8 when the Type I error probability associated with this test of this null hypothesis is 0.05. Considering the results of those calculations, we set a sample size of 20 in the present study.

Continuous variables were expressed as mean ± standard deviation or median and interquartile range. Categorical data were tabulated as frequencies and ratios (%). The data by manual scoring and default automatic scoring function or by the Somnolyzer system were compared using the t-test for normally distributed data, or the Wilcoxon rank-sum test for non-normally distributed data.

The correlations and agreements between the mREI and aREI as well as those between the mREI and sREI were assessed using regression analysis and Bland–Altman plots.¹⁸ In the Bland–Altman plots, the differences between the mREI and aREI or sREI were calculated using [mREI]-[aREI] or [mREI]-[sREI]. The correlations and agreements between oxygenation parameters (ie, 3% oxygen desaturation index [ODI], lowest SpO₂, time spent at SpO₂ <90%) by manual and automated scoring, and between manual and Somnolyzer scoring were also analyzed.

P-values of less than 0.05 were considered statistically significant. All statistical analyses were performed using SPSS 30.0 (IBM, Armonk, NY).

Results

The data of 20 eligible patients were analyzed. A study flowchart is shown in Figure 1. The patient characteristics are shown in Table 1. Five patients had hypertension (all treated with antihypertensive drugs), and one had diabetes mellitus (treated with oral hypoglycemic drugs). Four patients had a history of cardiovascular disease, including two with treated angina pectoris, one with an aortic aneurysm, and one with mild aortic valve stenosis.

Table 1 Characteristics of Patients (N=20)

Figure 1 Study flowchart. Abbreviations: HSAT, home sleep apnea test; OSA, obstructive sleep apnea.

The HSAT findings for each score are summarized in Table 2. No significant differences were observed between mREI and aREI, or between mREI and sREI: 14.7±13.3 events/h, 13.7±11.8 events/h, and 14.3±13.4 events/h, respectively. However, when the overall respiratory events were separated into apneas and hypopneas, we noted significant differences in the apnea and hypopnea indices between manual and automatic, and manual and Somnolyzer scoring.

Table 2 HSAT Findings by Each Scoring Method

If the apnea index was further classified according to the obstructive apnea index, both the automatic and Somnolyzer scoring overestimated it slightly, and a significant difference were observed in the obstructive apnea index between the manual and Somnolyzer scoring.

In terms of oxygenation, the 3% ODI was underestimated in both the automatic and Somnolyzer scoring. Median values and interquartile ranges of time spent at SpO₂<90% in automatic and Somnolyzer scoring were similar; however, a statistically significant difference was found between manual and automatic scoring, but not between manual and Somnolyzer scoring.

A strong correlation was found between the aREI and mREI (coefficient = 0.976; P<0.01, Figure 2a). Their mean was 1.0 and the limit of agreement was −5.3 to 7.3 (Figure 2b). The correlation between the sREI and mREI was more prominent (coefficient, 0.996; P<0.001, Figure 3a), with a mean difference of 0.1 and a limit of agreement of −2.1 to 2.9 (Figure 3b). Among oxygenation parameters, a strong correlation was found between the 3% ODI by manual scoring and by automatic scoring (coefficient = 0.977; P<0.01) with a mean difference of 1.1 and a limit of agreement of −1.8 to 3.9, and more prominently between manual scoring and scoring with Somnolyzer (coefficient = 0.994; P<0.01) with a mean difference of 0.5 and a limit of agreement of −1.6 to 2.6. A strong correlation was also found between the lowest SpO₂ by manual scoring and by automatic scoring (coefficient = 0.966; P<0.01) with a mean difference of 0.4 and a limit of agreement of −3.8 to 4.6, and more prominently between manual scoring and scoring with Somnolyzer (coefficient = 0.998; P<0.01) with a mean difference of −0.1 and a limit of agreement of −1.0 to 0.8. Similarly, a strong correlation was also found between the time spent at SpO₂<90% by manual scoring and by automatic scoring (coefficient = 0.989; P<0.01), with a mean difference of 0.06 and a limit of agreement of −0.22 to 0.33, and more prominently between manual scoring and scoring with Somnolyzer (coefficient = 0.999; P<0.01), with a mean difference of −0.01 and a limit of agreement of −0.07 to 0.05.

Figure 2 Scatter plot and Bland–Altman plot regarding mREI and aREI. (a) Scatter plot: A significantly strong correlation between mREI and aREI (r=0.976, P<0.001) was found. (b) Bland–Altman plot: The Y-axis indicates the difference between mREI and aREI ([mREI]-[aREI]) was 1.0 (solid line), and the limit of agreement was −5.3 to 7.3 (dashed lines). The X-axis indicates the mean difference. Abbreviations: mREI, manually scored respiratory event index; aREI, default automatically scored respiratory event index.

Figure 3 Scatter plot and Bland–Altman plot regarding mREI and sREI. (a) Scatter plot: A significantly strong correlation between mREI and sREI (r=0.996, P<0.001) was observed. (b) Bland–Altman plot: The Y-axis indicates the difference between mREI and sREI ([mREI]-[sREI]) was 0.4 (solid line), and the limit of agreement was −2.1 to 2.9 (dashed lines). The X-axis indicates the mean difference. Abbreviations: mREI, manually scored respiratory event index; sREI, Somnolyzer-scored respiratory event index.

Discussion

The findings of the present study provide several novel insights into the automatic analysis of type 3 HSAT devices. Although some differences in HSAT findings across the scoring methodologies were observed, especially subclassified respiratory events and oxygenation parameters between manual and automatic scoring or between manual and Somnolyzer-based scoring, the overall correlations between mREI and aREI and between mREI and sREI were strong. Bland–Altman plots indicated that the agreements of aREI and sREI against mREI were also good. The same was true in oxygenation parameters including 3% ODI, lowest SO₂, and time spent at SpO₂ <90%. Furthermore, the correlation and agreement of the sREI with the mREI were generally better than those with aREI. Similarly, correlations and agreements of oxygenation parameters by Somnolyzer with those by manual scoring were generally better than those by automatic scoring. These suggest that automatic scoring with this polygraphy device based either on the default function of Sleepware G3 or Somnolyzer were reliable, and that automatic scoring by Somnolyzer was slightly better than that by the default automatic function.

Since the COVID-19 pandemic, HSAT using type 3 polygraphy devices with manual scoring by sleep medicine specialists has gained attention in research.^8,9 In an actual clinical setting, when patients undergo HSAT with this polygraphy device and return the device to the clinic, patients must wait to obtain manually scored results. While there is a growing demand for HSAT with devices including this polygraphy device, such wait times may be burdensome for patients, who need to come to the clinic twice: first to return the device, and then to receive an explanation of the results in separate clinic visits. It also creates a burden for sleep specialists, who need to swiftly perform manual scoring. If automatic scoring can alleviate these issues, it would reduce the burdens on patients and sleep specialists. There are several HSAT devices with automatic scoring in the clinical setting, and REIs have been compared between automatic scoring and other modalities.^5,18–20 However, there are no data specifically comparing the REI derived from HSAT recording by automatic scoring with manual scoring. The strength of the present study is that we investigated the reliability of such automatic scoring for REIs recorded with this widely used HSAT device either by the default function of Sleepware G3 or by Somnolyzer, both of which were not previously and specifically compared with manual scoring. Therefore, the findings of the present study are worth reporting considering the growing demand for HSAT.

Although the overall REI was identical across manual and automatic scoring, automatic scoring tended to overestimate apneas and underestimate hypopneas. This may be due to differences in the assessments of flow signal amplitude and visual assessments in manual scoring versus the exact measurements in automatic scoring. From this perspective, automatic scoring may be more accurate than manual scoring. Nevertheless, in an HSAT, the overall REI is more important than the separate apnea and hypopnea indices, since the overall REI is generally used to determine therapeutic strategy and assess the criteria to initiate therapy.^3,4 For the obstructive apnea index, automatic scoring slightly overestimated compared to manual scoring. This may also reflect differences in the assessments of respiratory effort signal amplitude—namely, visual or algorithmic interpretation. For oxygenation parameters, automatic scoring underestimated 3% ODI in both the default automatic and Somnolyzer results. This discrepancy may stem from differences in how SpO₂ changes are assessed (visual assessments in manual scoring versus exact measurements in automatic scoring). Considering that strong correlations and acceptable agreements between manual and both automatic scoring were found in all oxygenation parameters, the effects of such differences on scoring of respiratory events may be minimal. The correlation between the sREI and mREI was better than that between the aREI and mREI. In addition, agreement between manual scoring and scoring by Somnolyzer was generally better than with the default automatic scoring. Therefore, although the default function of Sleepware G3 automatic scoring function demonstrated acceptable correlation with manual scoring, Somnolyzer appears to be more reliable.

This study had some limitations. First, although the sample size was determined by a specific calculation, the collected data analyzed in the present study was small in scale. Analyzing the correlations between the same fundamental parameters does not require many samples in general; thus, this sample size was an appropriate in terms of statistical analysis. However, a sample size of 20 may not be enough when generalizing our study results to the wide spectrum of OSA severity and patients with various backgrounds. Therefore, further research with a larger sample size is warranted. Second, patients who were likely to have experienced cardiovascular events were excluded. The number of central or mixed apnea indices was almost zero. However, because the polygraphy used in the present study is a type 3 device that can distinguish types of apnea events, further research on the detection of central apneas is needed to assess such issues. However, HSAT cannot be performed in patients who are likely to experience central respiratory events.^3,4

Lastly, our findings were derived from one particular polygraphy device for the HSAT and are not applicable to other polygraphy devices. Thus, our findings should be interpreted with caution.

Conclusions

Both the aREI and sREI showed high degrees of validity compared with mREI, with sREI showing better correlation and agreement. In an analysis of the data obtained by this polygraphy device using Sleepware G3, both the aREI and sREI were shown to be reliable in the selected patient population. However, the generalizability of our study results to patients with various degrees of OSA severity and of various backgrounds remains to be elucidated. Therefore, further investigation is needed to determine the factors that cause large differences between REIs.