Non-contact detection of oxygen saturation based on visible light imaging device using ambient light

Transcription

1 Non-contact detection of oxygen saturation based on visible light imaging device using ambient light Lingqin Kong, Yuejin Zhao, Liquan Dong, * Yiyun Jian, Xiaoli Jin, Bing Li, Yun Feng, Ming Liu, Xiaohua Liu and Hong Wu Beijing Key Lab for Precision Optoelectronic Measurement Instrument and Technology, School of Optoelectronics, Beijing Institute of Technology, Beijing, , China * kylind@bit.edu.cn Abstract: A method that remotely measures blood oxygen saturation through two cameras under regular lighting is proposed and experimentally demonstrated. Two narrow-band filters with their visible wavelength of 660nm and 520nm are mounted to two cameras respectively, which are then used to capture two photoplethysmographic (PPG) from the subject simultaneously. The data gathered from this system, including both blood oxygen saturation and heart rate, is compared to the output of a traditional figure blood volume pulse (BVP) senor that was employed on the subject at the same time. Result of the comparison showed that the data from the new, non-contact system is consistent and comparable with the BVP senor. Compared to other camera-based measuring method, which requires additional close-up lighting, this new method is achievable under regular lighting condition, therefore more stable and easier to implement. This is the first demonstration of an accurate video-based method for non-contact oxygen saturation measurements by using ambient light with their respective visible wavelength of 660nm and 520nm which is free from interference of the light in other bands Optical Society of America OCIS codes: ( ) Medical optics and biotechnology; ( ) Passive remote sensing; ( ) Optical devices. References and links 1. X. F. Teng and Y. T. Zhang, The effect of contacting force on photoplethysmographic signals, Physiol. Meas. 25(5), (2004). 2. C. Takano and Y. Ohta, Heart rate measurement based on a time-lapse image, Med. Eng. Phys. 29(8), (2007). 3. M. Z. Poh, D. J. McDuff, and R. W. Picard, Non-contact, automated cardiac pulse measurements using video imaging and blind source separation, Opt. Express 18(10), (2010). 4. W. Verkruysse, L. O. Svaasand, and J. S. Nelson, Remote plethysmographic imaging using ambient light, Opt. Express 16(26), (2008). 5. F. P. Wieringa, F. Mastik, and A. F. W. van der Steen, Contactless multiple wavelength photoplethysmographic imaging: a first step toward SpO(2) camera technology, Ann. Biomed. Eng. 33(8), (2005). 6. M. Z. Poh, D. J. McDuff, and R. W. Picard, Advancements in noncontact, multiparameter physiological measurements using a webcam, IEEE Trans. Biomed. Eng. 58(1), 7 11 (2011). 7. C. G. Scully, J. Lee, J. Meyer, A. M. Gorbach, D. Granquist-Fraser, Y. Mendelson, and K. H. Chon, Physiological parameter monitoring from optical recordings with a mobile phone, IEEE Trans. Biomed. Eng. 59(2), (2012). 8. J. A. Pollard, Cardiac arrhythmias and pulse variability: a plethysmographic study, Anaesthesia 25(1), (1970). 9. H. D. Hummler, A. Engelmann, F. Pohlandt, J. Högel, and A. R. Franz, Accuracy of pulse oximetry readings in an animal model of low perfusion caused by emerging pneumonia and sepsis, Intensive Care Med. 30(4), (2004). 10. N. S. Trivedi, A. F. Ghouri, N. K. Shah, E. Lai, and S. J. Barker, Effects of motion, ambient light, and hypoperfusion on pulse oximeter function, J. Clin. Anesth. 9(3), (1997). (C) 2013 OSA 29 July 2013 Vol. 21, No. 15 DOI: /OE OPTICS EXPRESS 17464

2 11. K. Humphreys, T. Ward, and C. Markham, Noncontact simultaneous dual wavelength photoplethysmography: a further step toward noncontact pulse oximetry, Rev. Sci. Instrum. 78(4), (2007). 12. K. Humphreys, T. Ward, and C. Markham, A CMOS camera-based pulse oximetry imaging system, Conf. Proc. IEEE Eng. Med. Biol. Soc. 4, (2005). 13. J. Zheng, S. Hu, A. S. Echiadis, V. A. Peris, P. Shi, and V. Chouliaras, A remote approach to measure blood perfusion from the human face, Proc. SPIE 7169, (2009). 14. W. Verkruysse, L. O. Svaasand, and J. S. Nelson, Remote plethysmographic imaging using ambient light, Opt. Express 16(26), (2008). 15. H. S. Zhu, Y. J. Zhao, and L. Q. Dong, Non-contact detection of cardiac rate based on visible light imaging device, Proc. SPIE 8498, , (2012). 16. J. M. Bland and D. G. Altman, Statistical methods for assessing agreement between two methods of clinical measurement, Lancet 327(8476), (1986). 17. Y. Sun, C. Papin, V. Azorin-Peris, R. Kalawsky, S. Greenwald, and S. Hu, Use of ambient light in remote photoplethysmographic systems: comparison between a high-performance camera and a low-cost webcam, J. Biomed. Opt. 17(3), (2012). 18. Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise, J. Biomed. Opt. 16(7), (2011). 1. Introduction Oxygen saturation, an important physiological parameter, has been identified as a risk factor for the chronic diseases of circulatory system and respiratory system. Regular and non-invasive assessments of oxygen saturation are of great importance in surveillance for cardiovascular catastrophes and treatment therapies of chronic diseases. The conventional contact pulse oximeter has been widely used in routine and critical clinical applications. However, it cannot be used when mechanical isolation is required, such as the case for burned patients in an emergency room and the patients with shaky hands and feet due to various reasons. On the other hand, it has been demonstrated that the spring loaded clips in conventional finger oximeter can affect the waveform of photoplethysmography (PPG) signals due to the contact force between the sensor and the measurement parts [1]. While non-contact measuring method can solve the problems mentioned above well. In recent years, the imaging photoplethysmography (ippg) has become a more attractive method for non-contact detection of the physiological parameters (e.g. heart rate, respiratory rate, oxygen saturation, etc.) [2 7]. C. Takano and Y. Ohta first reported that heart and respiratory rate can be extracted based on the imaging device [2]. To our knowledge, the prerequisite for measurement of oxygen saturation (SPO 2 ) is to obtain the cardiovascular pulse wave signal at two different wavelengths. Due to the historical emphasis on pulse oximetry and the wave penetrability needed to be relatively same deep (e.g. 1mm) to veins and arteries, the wavelengths are usually selected as red(660nm) and/or infra-red (IR:940nm) [8 10]. To achieve the noncontact detection of oxygen saturation by using the ippg technology, an LED array with two different wavelengths and a camera are usually employed as the illumination source and detector, respectively. In 2007, Kenneth Humphreys et al. for the first time realized the extraction of SPO 2. In their experiment, a dual wavelength light emitting diodes array of 760 and 880 nm is used as the light source and a CMOS is used as the detector [11]. However, the active measurement [11, 12], which requires light source being especially close to skin or even contacting to skin, fails to solve the problem of non-contact measurement thoroughly; meanwhile, the results of which are undoubtedly affected by the surrounding ambient light due to its fixed mode of data acquisition. While passive measurement (i.e. by using ambient light) can break through the limitation of the distance between light source and the tested skin. In fact, the PPG signal can be acquired using cameras with normal ambient light as the illumination source [2 4]. So far, for ambient light, studies have mainly focused on the extraction of heart and respiratory rate [13, 14]. As for how to realize the anti-interference measurement of oxygen saturation by using ambient light has not been well understood. In this paper, we propose and demonstrate a non-contact method for blood saturation measurement based on double CCD each with a narrow band pass filter using regular ambient (C) 2013 OSA 29 July 2013 Vol. 21, No. 15 DOI: /OE OPTICS EXPRESS 17465

3 light. The principal advantages of this method are listed as follows. 1) The cardiovascular pulse wave is extracted directly by using ambient light without additional light sources. Therefore, the detection does not limited by the distance between the detector and the skin being tested. 2) The adoption of the band pass filter insulates and blocks the interference of ambient light with other bands except for those lights of needed bands. 2. Methods and instruments 2.1 Band analysis and selection Blood saturation is used to indicate the intensity of oxygen in blood, which is defined as SPO = HbO % HbO + Hb (1) 2 HbO 2 is oxygenated hemoglobin, and Hb is deoxygenated hemoglobin. According to the Beer-Lambert law and the theory of light reflection, conventional pulse oximeter usually select two different wavelengths (typically 660nm and 940nm) as the detecting source to measure the maximum transmissive/reflective intensity (I λ1 DC and I λ2 DC ) of two beams of light, and the maximum variables of transmission intensity caused by pulsation. The value of blood saturation can be expressed as λ1 λ1 AC / DC SPO = I I 2 A B A R B λ2 λ2 I / I + = + (2) AC DC where, A and B are empirical coefficients determined by calibration. The wavelength selection of dual-wavelength oximeter should follow two principles. First, the absorption coefficients of HbO 2 and Hb at one wavelength should differ greatly. The second is approximately equal absorption coefficient in terms of HbO 2 and Hb at the other wavelength. The conventional oximeter generally adopts the wavelength of 660nm and 940nm. However, in addition to the two factors mentioned above, the selection of wavelength of video-based non-contact oximeter should also take the response curve of the imaging device into account. For the ippg adopted in this paper is reflection-based PPG signal measurement, the PPG signal measured in this way is weaker than that of the conventional transmission method, and the response of common imaging device to infrared light (IR) is weaker; therefore, no PPG signal can be extracted at IR with ippg under no active lighting condition. So we need to select other wavelengths and other ways of signal capturing in order to realize the measurement of blood saturation by using ambient light. Fig. 1. The absorption spectrum of HbO 2 and Hb. As shown in Fig. 1, except the wavelength of 805nm and 940nm, we can see that several other wavelengths can make up dual wavelength with 660nm, i.e, 440nm and 520nm. Although (C) 2013 OSA 29 July 2013 Vol. 21, No. 15 DOI: /OE OPTICS EXPRESS 17466

4 the absorption coefficients of HbO 2 and Hb are equivalent at the wavelength of 440nm, both changes so rapidly that the wavelength is not suitable for measurement of blood saturation. On the contrary, at the wavelength of 520nm, not only the absorption coefficients of HbO 2 and Hb are equivalent, but also the curves of them change more moderately, making it quite suitable for blood saturation measurement. To sum up, taken the absorption spectrum of hemoglobin and the response curve of CCD into consideration, the wavelengths of 660nm and 520nm are chosen for measuring blood saturation in this paper. 2.2 Experimental setup and study description Fig. 2. Schematic of the monitoring of oxygen saturation with visible light imaging device. Our experimental scheme is shown in Fig. 2. All measurements were taken in a temperature controlled (26 ± 1 C) room with stable ambient light, avoiding interference of light intensity mutation on the results of experiment (except the Section 5). Two monochrome CCD (WAT-535EX2, Watec, Japan, Minimum illumination: 0.003lx), each mounted with a different narrow band pass filter, are used as the video-capturing unit, and a personal computer (PC) is employed to record the videos for analysis. All videos including human face were recorded at 25 frames per second (fps) with pixel resolution of and saved as AVI format in the PC. To ensure the accuracy of the results measured by the system, the band width of the filters should be as narrow as possible. In the paper, the full width at half maximum (FWHM) of the filters is 10nm, i.e. 660 ± 5nm (F OD4, OTF, China) and 520 ± 5nm (F OD4, OTF, China). The filter is mounted in front of the chip of CCD so as to ensure that no other light than the required light can enter into CCD. The use of double CCD mounted with narrow band pass filter can effectively resist the disturbance of ambient light in other bands. For all experiments, videos containing human face are captured by double CCD simultaneously and subsequently transmitted to PC via video capture card for storage and analysis. Meanwhile, a finger BVP sensor (Qinhuangdao Contec Medical Systema Co., Ltd) is used to measure relevant signals of the subject. In experiments, 30 volunteers with an age range from 18 to 58 are enrolled. The subjects are required to sit motionlessly at the position of 1.5m away from CCD and the distance to each CCD should be equal. The subjects are required to keep their head and finger as still as possible. During the experiment, the subjects are asked to keep two kinds of different respiratory status respectively. The one is that subjects are in the state of spontaneous breathing during the shooting of experimental videos. The other is that the subjects are holding their breath when the videos are captured, to verify the linear consistency between SPO 2 and R (shown in Eq. (2)). Unless otherwise stated, the subjects are in the state of spontaneous breathing. 3. Analysis of the gathered data and parameter extraction The extraction of respiration rate and heart rate: videos including human face are saved as AVI format by the CCD and then transferred to a PC. In the paper, we select the region below eyes as the region of interest (ROI) for studying the changes of blood perfusion in skin, as shown in Fig. (C) 2013 OSA 29 July 2013 Vol. 21, No. 15 DOI: /OE OPTICS EXPRESS 17467

5 3(a). Used Matlab, the average of all pixel values (8bit, 0-255) in the ROI are extracted from each movie frame providing a set of average pixel values (APV, t), and t is time corresponding to the frame rate. Then the PPG signals curve is obtained, as shown in Fig. 3(b). Low-frequency respiration signal (red line) and high-frequency pulse wave signal can be seen clearly. The Fast Fourier Transform (FFT) and the frequency spectrum superimposed algorithm [15] process the respiration rate and heart rate signal in frequency domain. The Fourier spectra of the PPG signals at 520nm in Fig. 3(b) is plotted in Fig. 3(c), from which the respiration rate and heart rate can been clearly extracted (respiratory rate = 15beat/min, cardiac/heart rate = 64beat/min). Figure 3(d) shows the pulse wave signal after bandpass (B1-B2) filtering. ide Av eq os n ue ce m Ti e Fig. 3. The process of extraction of PPG signals from a video sequence. (a) A video sequence is illustrated along the time axis and the ROI marked with solid line box. (b) Evolution of the spatially-averaged pixels value during the recorded video. (c) The Fourier spectra corresponding to b. (d) The PPG signals after band pass filtering(b1-b2, B1 = 0.7Hz, B2 = 3Hz). The extraction of blood saturation: SPO2 values are calculated by comparing the 660nm and 520nm bands according to Eq. (2), where 660 and 520 are representative of the λ1 and λ2 respectively. The peak to peak value obtained from PPG after de-noising and band pass (0.7Hz-3Hz) filtering is used as the AC signal, and the DC components are computed as the average value of the PPG signals at corresponding periods of time is computed as the DC component. The peak to peak value extracted from the PPG signal in each cycle also varies when the blood saturation remains unchanged, as shown in Fig. 3(d). Therefore we can t simply adopt the peak to peak value of each cardiac cycle as IAC, and a 10s moving average window is then applied to the AC and DC components. Theoretically, R should be linear or nearly linear with SPO2. According to the analysis mentioned in Section 2.1, we know that the wave band of 520nm and 660nm meet the theoretical requirements of oximeter. However, the linearity of R and SPO2 would be affected by the difference between the penetration of 520nm and 660nm. Therefore, it is necessary to verify the linearity of R and SPO2 when the two wavelengths are selected. # $15.00 USD (C) 2013 OSA Received 29 Apr 2013; revised 15 Jun 2013; accepted 1 Jul 2013; published 15 Jul July 2013 Vol. 21, No. 15 DOI: /OE OPTICS EXPRESS 17468

6 In the experiment, contrast detection is conducted when subjects hold their breath until they feel discomfortable. The results are shown in Fig. 4. The shaded part in Fig. 4 shows that the R value measured by the noncontact devices has a good linear relation with SPO 2 detected by finger BVP sensor. However, the result measured with noncontact device is higher than that of the BVP sensor after 28 seconds. It is because that the subject has begun to breathe after this time point, while the blood saturation is still on the fall due to the recovery of blood saturation in human face is faster than that of hands. Fig. 4. Comparison of the R measured by noncontact device proposed and the SPO 2 measured by contact device versus time. (i) The linear fitting curve of SPO2 and R (the valid data in shadow section). The extraction of the empirical coefficients A and B: After several groups of breath holding comparison test of multiple volunteer subjects in the same environment temperature and light conditions, 30 sets of data obtained from 30 different volunteers are used to calibrate the empirical coefficients A and B. The valid points, as shown in the shadow section in Fig. 4, are obtained in each set of data. Then least square method is adopted to linearly fit these data to obtain the values for the empirical coefficients A and B. Thus, substituting A and B in Eq. (2) leads to SPO2 = R (3) 4. Experimental results Comparative tests are performed between the conventional finger BVP sensor and the device proposed when subjects are in state of spontaneous breathing. After the comparison test with 30 volunteers, the measuring results of both devices are shown in Fig. 5. It shows that the noncontact device is comparable to contact device as a monitor of heart rate (Fig. 5(a)), and the difference between the results measured by both devices is within 3beat/min. Both results of oxygen saturation are also in good agreement with each other (Fig. 5(b)). Fig. 5. Scatter plot showing noncontact device s measurement of heart rate and oxygen saturation versus the contact device s measurement. (a) heart rate. (b) oxygen saturation. (C) 2013 OSA 29 July 2013 Vol. 21, No. 15 DOI: /OE OPTICS EXPRESS 17469

7 As can be seen from Fig. 5(a), the heart rate obtained from the noncontact device proposed is generally in agreement with that of the finger BVP sensor. We have analyzed the results by Bland-Altman method [16] in order to quantitatively describe the consistency of both devices. Figure 6(a) shows the Bland-Altman plot based on the results shown in Fig. 5(a). Assuming that the mean difference between the devices is normally distributed, 95% confidence interval can be calculated at d ± 1.96 Sd (d = beat/min is the mean of the difference between measurements by both devices, and Sd is the standard deviation between measurements of both devices). The difference between measurements by both devices is within the limitation of confidence interval. It indicates that the results measured by the non-contact device are in good agreement with that by the finger BVP sensor, and they can be used interchangeably. The consistency of the SPO 2 measured by both devices by Bland-Altman method is also assessed. Figure 6(b) shows a Bland-Altman plot. One among the 30 groups of data lies beyond the limitation of d Sd and d Sd. It indicates that the accuracy of measurement by the non-contact device proposed can t reach the level of clinical application. Fig. 6. Bland-Altman plot of the mean of the measurements by both devices for each subject versus the difference between the measurements by both devices for each subject. (a) heart rate, (b) SPO 2 (The number next to the data point is the sample number at that location). 5. The influence of ambient temperature and light intensity This method utilizes the ambient light as the light source, to which the PPG signal is sensitive. So the ambient light intensity has to be considered for its influences on the test results. Y. Sun et al demonstrated that the measurement of heart rate is almost blind to the ambient light change [17]. The experiment detecting the influences of ambient light change on blood oxygen saturation is conducted in darkroom with the same setups as those shown in Fig. 2. The differences are light-averaging sheet and intensity adjusting white LED array added to use as the illumination source to simulate the ambient lighting conditions under various climate. The relative intensity of the ambient light is defined as the average gray value of the human face pixel collected by CCD with 520nm filter. Table 1. Descriptive Statistics for the effect of the ambient light intensity on SPO 2 Statistic Relative intensity of the ambient light (average of all pixel values (520nm)) 130 ± ± 5 70 ± 5 Mean bias (%) Sd of bias (%) Upper limit (%) Lower limit (%) Table 1 summarizes the descriptive statistics for critical evaluation of the proposed method compared to a finger BVP sensor at different ambient light intensity. It shows that the relatively stable ambient light intensity has no obvious influences on SPO 2 (the average gray value (C) 2013 OSA 29 July 2013 Vol. 21, No. 15 DOI: /OE OPTICS EXPRESS 17470

8 fluctuation is within ± 5). Temperature is also the factor that should be considered. The test is conducted under conditions with controllable temperature and stable ambient light while the temperatures are controlled as 16 ± 1 C, 20 ± 1 C, 24 ± 1 C, 28 ± 1 C, 32 ± 1 C respectively. The blood oxygen saturation of the volunteers is measured respectively after they stayed in the conditions with different temperature for 45 minutes. It turns out that the temperature almost has no influences on the test results. 6. Discussions The study considers the remote accessing of physiological information (i.e., heart rate, SPO 2 ) by using ambient light. The performance of the system proposed shows good agreement with the result measured by a commercial pulse oximeter sensor. In the system, two CCD, being equidistant from the tested face, are used to capture the same section on the face. So debugging and adjustment is necessary. The adoption of the narrow band-filter allows the device be free from any interference produced by the ambient light of other band. Also, the FWHM of the narrow filters adopted in this paper is 10nm, increasing the tolerance of the system to light source. When the grey value of image captured by CCD is below 50, an approximately stable artificial light source, whose spectral range is wide enough to cover the needed wavelengths of 520nm and 660nm, is needed. In our experiments, a lamp with light diffusion plate is used for supplementing light under inadequate lighting condition. Another limitation is the elimination of motion artifact. The physiological changes in blood volume due to motion are not well understood and could also not meet the linear relationship [3]. Although the pioneering researchers have proposed many good solutions for the eliminating of the motion artifact [3, 18], the work to eliminate motion artifact for the remote measurements of SPO 2 is relatively short and a further work should be done to remove the motion artifacts from ippg signals to enable the clinical application. 7. Conclusions The emerging field of imaging PPG technologies offers some nascent opportunities in effective and comprehensive interpretation of the physiological parameters (e.g., heart/respiratory rate, tissue blood perfusion, and oxygen saturation), indicating a promising alternative to conventional contact PPG. We have described, implemented and evaluated a novel method for obtaining the oxygen saturation and heart rate from video recordings of the human face by using two CCD mounted with narrow band pass filter. Moreover, contrast test is performed with finger BVP sensor. Results from the experiment on 30 subjects indicate that both the contact device and non-contact device are in perfect agreement with each other. Compared with the method of using one camera added with additional lighting, this method breaks through the limitation of distance between the measured subject and the light source. It is of great significance for the clinical application of non-contact detection of oxygen saturation and heart rate. Acknowledgments This work was supported by the State Key Program of National Natural Science Foundation of China (No ), the National Natural Science Foundation of China (No ) and the Doctoral Program Foundation of Higher Education of China (No ). (C) 2013 OSA 29 July 2013 Vol. 21, No. 15 DOI: /OE OPTICS EXPRESS 17471