Research Summary: Assistive Lifeguard Technology

RESEARCH

This meta-analysis of 26 original studies on assistive lifeguard technology (drowning detection systems) in swimming pools and beaches between 2002 and 2022. Articles are presented in order of recency. 

Alqahtani, A., Alsubai, S., Sha, M., Peter, V., Almadhor, A. and Abbas, S. 2022. Falling and drowning detection framework using smartphone sensors. Computational Intelligence and Neuroscience, pp.1-12. 

Alquahtani et al. developed a machine learning algorithm for use by a waterproof smartphone in online and offline modes to sense when an individual is falling or drowning. 5 test subjects were used to simulate falling and drowning behaviours. 11,091 samples of drowning activity were collected, 11,700 of routine activity, and 6,547 samples of falling activity. Three logic models (Logistic Model Trees, Bayes Net, and Logistic Regression) were used to recognize the drowning, routine and fall activities. 1 in 5 samples was used for testing, and 4 in 5 samples were used for training the model. 

The model works by using data from the accelerometer, gyroscope, magnetometer and GPS. This enabled 12 indicators to be identified. The data collection for each of the 5 test subjects was around 13-18 minutes of video footage for each participant.  The LMT model produced a drowning false detection rate of less than 1%, the BN model less than 5%, and the LR model less than 1%.  26% of falling examples were wrongly recognized as drowning examples. 16% of drowning examples were wrongly recognized as falling activities. 

Laxman, P. and Jain, A. 2021. Drown alerting, preventing and autonomous rescue system using Ardiuno, tactile switches (weight sensors) and artificial intelligence. International Journal of Advanced Trends in Computer Science and Engineering, 10(2), pp.1-5. 

A tactile sensor network was installed on the bottom of a swimming pool. After 50 seconds, if the tactile sensors remain depressed, then an alarm activates a mechanical mechanism that raises the pool floor to surface level and drains, allowing the pool water to drain from the pool. The result is that the swimmer remains on the pool floor, which is now surface level and is no longer submerged under the water. A single button can reverse the process, lowering the pool floor and allowing the water to return to the pool tank. 

Wang, F., Ai, Y. and Zhang, W. 2021. Detection of early dangerous state in deep water of indoor swimming pool based on surveillance video. Signal, Image, and Video Processing, pp.1-9. 

A system was developed to overcome the challenges drowning detection systems face in distinguishing swimmers who are treading water from those in danger. Surveillance videos around 16 seconds long involving the four primary strokes were captured from the southwest corner of a swimming pool in Beijing. The swim lanes were 2.4 metres deep and 2.5 metres wide. 5033 images were captured. 1511 images were used for training the system, and a further 500 images were reserved for testing. The system was trained 20,000 times to develop the head detection capability. The system was trained to distinguish the head from the body to overcome the problem that the long and short axis of the outer ellipse of a human body does not work when the optical axis of the camera is not perpendicular to the swimming direction of the swimmer. The system was able to overcome this challenge. 

Jose, A. and Udupa, G. 2021. Gantry robot system for preventing drowning accidents in swimming pools. Materials Today: Proceedings, 46, pp.4975-4981. 

The proposed system consists of three parts: an overhead camera, a gantry robot and an LED light display and alarm unit. The overhead camera was equipped with a drowning detection algorithm. When the system detected a person in difficulty, the alarm would sound, and the gantry robot would move to drop a buoyancy aid in the location of the distressed pool user. The size of the pool was 20 metres by 15 metres. A simulation was performed to demonstrate the use of the system. 

Hughes, G., Camomilla, V., Vanwanseele, B., Harrison, A., Fong, D. and Bradshaw, E. 2021. Novel technology in sports biomechanics. Some words of caution. Sports Biomechanics, pp.1-9.

Whilst not writing about drowning detection systems per se, the authors suggest questions that any drowning detection system manufacturer should be able to answer about their novel technology: 

1. Does the proposed system provide practical value when obtained and used in the real world? 
2. Is the system as simple as required for it to be used as a decision-making tool? 
3. Could you, as the user, explain how it works (in general) to anyone who is interested? 
4. Are you sure the system is valid and reliable, and are you aware of its uncertainty? 
5. Do you use it appropriately, with the right level of scepticism? 

Du, J. 2021. Characteristics and function analysis of swimming life saving system based on machine vision technology. Journal of Physics Conference Series, 1881, 042079, pp.1-7. 

The paper involves a general discussion of computer vision usage in swimming pool safety. 

Alotaibi, A. 2020. Automated and intelligent system for monitoring swimming pool safety based on the IoT and transfer learning. Electronics, 9, 2082, pp.1-13. 

Iotaibi designed a system which used a prototype swimming pool and a dataset of 300 images. The 2D images were classified into three types: human, animal and object. 66% of the samples were used for training purposes, and 33% for test purposes. To explore the efficiency of the proposed system, different deep learning modules were evaluated using the dataset. The deep convolution neural network (DCNN) achieved an accuracy of 95%. Residual network 50 (ResNet50) achieved an accuracy of 97%. The proposed system achieved 99% accuracy, including 100% accuracy in the human and animal classes in prototype laboratory conditions. 

Akkayaynak, D. and Treibitz, T. 2019. Sea-thru. A method for removing water from underwater images. IEEE Xplore Computer Vision, pp.1682-1691.

A study of five RGBD datasets of saltwater environments at depths ranging from 4-10 metres below sea level was assessed using a novel algorithm to improve image quality. The enhanced image quality produced provides significantly better visibility in challenging visual scenes, which could significantly enhance (or remove entirely) issues with poor turbidity, which negatively impacts visual surveillance systems in swimming pools.

Present drowning detection systems in swimming pools, particularly overhead cameras, require very high water quality levels to be maintained by the pool operator to remain effective. This algorithm could reduce the burden on operators and improve detection rates in swimming pools, especially during periods with high bather loads. 

Laxman, P. and Jain, A. 2019. A review paper on design and performance evaluation of drowning death prevention system with various technologies. Journal of Emerging Technologies and Innovative Research. 6(1), pp.1258-1264. 

Laxman and Jain present a literature review of drowning detection system types. The following types were reported: 

• Wearable swimming goggles with a hydrophone (Roy, 2018) 
• Wearable inflatable wristband with an accelerometer
• Video image frames
• Underwater ultrasonic sensors
• Water pressure sensor, GPS, and an accelerometer
• Sonar and thermal imaging
• Headband with a pulse-oximeter sensor (Ramdhan and Ali, 2018)
• Wearable piezoelectric and humidity sensor (Kulkarni, 2016)
• Wearable motion sensor (Nishida, 2007)
• Image processing protocol (Fei and Xueli, 2009)
• Thermal imaging and image enlargement (Soren Bonderup, 2016)
• Autonomous underwater robot with rescue capability
• Image processing protocol

John, S., Godswill, U., Osemwegie, O., Onyiagha, G., Noma-Osaghae, E. and Okokpukie, K. 2019. Design of a drowning rescue alert system. International Journal of Mechanical Engineering and Technology, 10(1), pp.1987-1995. 

A wearable drowning rescue alert system shaped like a watch and worn on the wrist was developed. The system was based on an oximeter to measure heart rate. A visual monitor was placed by the lifeguard position displaying a map of the pool and the location of any alert. No results were published in relation to this system. 

Lopez-Fuentes, L., Weijer, J., Gonzalez-Hidalgo, M., Skinnemoen, H. and Bagdanov, A. 2018. Review on computer vision techniques in emergency situations. Multimed Tools Applications, 77, pp.17069-17107. 

A general review of computer vision techniques and uses in emergency situations, including swimming pools. The literature review covers each stage of the emergency sequence of events in chronological order. 

Akkaynak, D. and Treibitz, T. 2018. A revised underwater image formation model. IEEE Xplore Computer Vision Foundation. pp.6723-6732.

Akkaynal and Treibitz provide an explanation as to why drowning detection systems reliant, or partially reliant, on the colour correction of underwater images frequently find the results of their models unstable. A revised equation for the colour correction of underwater images is proposed to account for previously unknown effects. The system is tested in salt water and images produced by the revised equation show a noticeable improvement in increasing the quality of underwater images. The article does not explore the utility of the revised equation in the context of poor turbidity. 

Prakash, D. 2018. Near-drowning early prediction technique using novel equations (NEPTUNE) for swimming pools. 5th International Conference on Computer Science, Engineering and Information Technology. pp.1-14. 

A system was developed to detect a drowning swimmer in pre-recorded video footage. Two videos were broken down into 125 images. The system analyzed the images in laboratory conditions, and the time to detection was recorded. The system required between 1-5 seconds to detect a drowning. False alarms were not reported. 

Ramdhan, M., Ali, M., Eberechukwu, P., Effiyana, N., Ali, S. and Kamaludin, M. 2018. An early drowning detection system for the internet of things (IoT) applications. Telkomnika. 16(4), pp.1870-1876. 

Ramdhan et al. designed a wearable drowning detection device (a headband) which monitors abnormal heart rhythms using a pulse sensor and submersion duration using a radio frequency signal. If the radio frequency signal is lost for more than 30 seconds, the alarm is activated. If an abnormal heart rhythm is detected, the alarm is activated.  The system's accuracy was tested in laboratory conditions and measured at 94%. The number of false detections was not reported. 

Lei, Y., Chen, M., Sun, T., Li, W., Gou, W. and Qin, Y. 2018. Application of BeiDou navigation satellite system in anti-drowning system. IOP conference series on materials science and engineering. 428, 012009, pp.1-9. 

Lei et al. designed a system which uses a humidity sensor, GPS, and an oximeter (pulse sensor). The humidity detector can determine whether the person is submerged. The GPS can determine whether a person is in the water. The pulse sensor can detect whether there is an abnormal heart rhythm. The system was tested in laboratory conditions. The accuracy of the system in detecting drowning casualties was not tested. 

Kanchana, A., Kavya, G., Kavitha, C., Soumyashree, V., and Salila, H. 2017. International Research Journal of Engineering and Technology. 4(6), pp.2938-2941. 

Kanchana et al. propose a system based on LDR, lasers, and a water pressure sensor. Firstly, each pool user wears a wristband. The pressure sensors in the wristband are monitored. If the threshold is reached, the alarm sounds, and the acrylic floor automatically raises the pool floor to ground level, saving the swimmer's life. Test results for the proposed system were not included. 

Bernardina, G., Cerveri, P., Barros, R., Marins, J. and Silvatti, A. 2016. Action sports cameras as an instrument to perform a 3D underwater motion analysis. PLOS One. 11(8): e0160490. 

Bernardina et al. explore the image quality of two action sports cameras with underwater housing in an empty swimming pool. Underwater 3D analysis using traditional systems provided a reconstruction error higher than 5mm. Underwater calibration using industrial cameras and a non-linear camera method improved accuracy up to 1mm across a water volume of around 7m3. This was comparable to the 2mm accuracy at a 10-metre distance reported by the two action sports cameras. 

Salehi, N., Keyvanara, M., and Monadjemmi, S. 2016. An automatic video-based drowning detection system for swimming pools using active contours. I.J. Image, graphics and signal processing. 8, pp.1-8. 

Salehi et al. test a drowning detection system using real-time video footage from a swimming pool. 3 video sequences were used, resulting in 6 correct detections and one false detection. The average time to make the detection ranged from 12.8-20.1 seconds. 

Chan, K. 2013. Detection of swimmer using dense optical flow motion map and intensity information. Machine Vision and Applications, 24, pp.75-101. 

A system was developed called motion and intensity information in background subtraction (MIBS). This system relied on video surveillance of an aquatic scene to identify single pool users without using a polarizer or specular colour removal. One of the system's limitations is that it did not consider detection where multiple pool users are present. MIBS is more computationally demanding than intensity-based methods. The utilization of intensity and motion gave rise to better results. The system was designed to detect the stroke the swimmer was using rather than a drowning casualty. 

Kharrat, M. 2013. Early drowning context awareness using wearable sensors. Computer Science, pp.1-86. 

A wearable pressure sensor worn on the pool user's head and an accelerometer were used to determine the time spent underwater and detect the absence of movement to detect drowning incidents (called the 'Hakim Drowning Prevention System'). An airbag worn around the waist was also triggered if a drowning pattern is detected. A cloud-based system of surveillance was implemented. The study was carried out at the University of Tokyo's main swimming pool, with a water depth of 1.5 metres. 

The dataset comprised 160 vectors, each composed of 200 samples. 70% of the data (112 samples) was used for training the system, whilst 15% (24 samples) was used for the validation and testing phases. The system achieved a 100% positive detection rate in the laboratory but 93-94% in the swimming pool environment. 

Wong, W., Hui, J., Loo, C. and Lim, W. 2013. Thermal imaging based off-time swimming pool surveillance system. International Journal of Innovative Computing, Information and Control, 9(3), pp.1293-1320. 

The paper begins with a critical discussion of the capability of available surveillance detection system technologies. A thermal imaging camera was used to track pool user movement in and around the swimming pool under test conditions. The system attained 88.63% accuracy for detecting persons in the swimming pool using head detection only. This was improved to 95.58% when persons were in the pool surround. The system was not trained to detect drowning casualties, but others have used similar mechanisms in thermal imaging drowning detection system devices. 

Dobashi, H., Tajima, T., Abe, T., Nambo, H. and Kimura, H. 2012. Improvement of abnormality detection systems for bathers using ultrasonic sensors. Electrical Engineering in Japan. 179(3), pp.6-13. 

A system of ultrasonic displacement sensors was used to provide surveillance of a small circular pool akin to a jacuzzi. The ultrasonic sensors were aimed at the head and neck of the pool user.  The system was programmed to understand and trigger an alarm if any of the abnormal conditions signify the pool user may be in danger. An experiment was conducted to test the system. The system's accuracy was reported as between 99-100% for pool users in normal conditions and 70-81% for those in abnormal conditions. 

Eng, H. 2011. Chapter 13 Swimmer behavior analysis and early drowning detection at pool. Published in Li, H. and Li, L. (eds). 2011. Advanced Topics on Biometrics (1st edition, World Scientific Publishing, Singapore). 

Eng proposes a drowning early warning system (DEWS) which develops the work of  Menoud (1999), Menire (2000), Guichard et al. (2001) and Lavest et al. (2002). Their system of underwater cameras could only detect drowning when the casualty remains motionless at the bottom of the pool. Eng proposes using overhead cameras to overcome the challenges that underwater camera systems face, such as occlusions caused by swimmers standing in shallow water. The DEWS relies on six indicators to detect drowning and is based on Pia's 1974 description of distressed and early drowning casualties: 

1. Movement range - drowning casualties show reduced spatial displacement ranging over some time when compared with swimmers. 
2. Speed product - drowning casualties display erratic or fast limb movements, which can be measured using the magnitude of the velocity vector. 
3. Posture variation - drowning casualties show repetitive arm and leg movements corresponding to a continual change in a swimmer's posture. The angle between a computer-generated best-fit ellipse and the horizontal axis provides the orientation of the swimmer. The range between the minimum and maximum values of the orientation angle over a given period can indicate the erratic movements of a distressed pool user. 
4. Activity variation - drowning casualties display erratic arm and leg movements which result in a change to the ratio between the best-fit ellipse and the size of the foreground silhouette. The range between the maximum and minimum values of those ratios over a short period can provide an indication of a distressed swimmer. 
5. Size variation - drowning swimmers, typically display a higher range between the maximum and minimum size of the foreground silhouette over a short period. This is because a greater amount of water disturbance is generated by a drowning pool user than by a normal pool user. 
6. Submersion index - sinking swimmers, typically demonstrate higher colour saturation than those on the surface. The difference between the colour saturation of the swimmer when they are first tracked to their colour saturation now sinking underwater is compared. 

Eight overhead cameras were installed above a swimmer pool. It is not reported whether this was an indoor or outdoor pool. A recording was taken between 08:00-21:00, which included a general swimming session and a swimming lesson. No polarizer was used. Three tests were carried out:

1. Results from a pre-recorded video (08:00-21:00) of a single pool were analyzed. The DEWS produced results of 99% accuracy during the daytime and 95% accuracy at night.
2. Results from 20 sequences of recorded video footage showing swimming, treading water and pool users in distress were analyzed ten times each by the DEWS. The system's accuracy in these tests was between 81-91%.

Amin, I., Al-Habaibeh, A., Junejo, F., Taylor, A., and Parkin, R. 2008. Automated people-counting by using low-resolution infrared and visual cameras. Measurement. pp.1-15.

The experiment was conducted using low-cost visual imaging devices and a thermal imager. A scenario was devised that involved six experiments. Each experiment contained around 150 visual and infrared sample images. Each experiment took between 3-5 minutes. Results recorded report system accuracy at between 95-100% across the 200 samples. 

Eng, H., Toh, K., Kam, A., Wang, J., and Yau, W. 2003. An automatic drowning detection surveillance system for challenging outdoor pool environments. Ninth IEEE International Conference on Computer Vision. pp.1-8. 

Eng et al. propose a block-based background model for pool user detection. This is tested using video footage from an outdoor swimming pool. Detection is based on five indicators: 

1. Speed - a pool user's speed is defined as the difference in their average centroid position computed over one second. 
2. Posture - a pool user's dominant position over three seconds based on a best-fit ellipse. 
3. Submersion index - a pool user's colour saturation is compared to their average saturation to detect their position below the water's surface. 
4. Activity index - the average area of a pool user in pixels, is compared with the average area of a best-fit ellipse over the same duration. 
5. Splash index - the number of splash pixels within a defined area around the swimmer. 

A real-time aquatic surveillance system, using overhead cameras, was set up at a public swimming pool over six months. Results showed a detection accuracy rate of 97%. 

Lu, W. and Tan, Y. 2002. A camera-based system for early detection of drowning incidents. IEEE ICIP. pp.445-448. 

Lu and Tan present a novel camera-based system designed to detect potential drowning incidents. The system makes use of overhead cameras. The system can assess moving speed, size variation, and body posture. The system has three underlying rules: 

1. A drowning swimmer moves slowly or remains active within a small neighbourhood
2. A drowning swimmer has a nearly vertical body posture in the water
3. A drowning swimmer exhibits irregular or fast limb movements

The system was tested using video footage of simulated drowning incidents, primarily 15 videos from the American Red Cross video "On Drowning". 71 tests were conducted. All eight simulated drowning incidents were detected. 2 false alarms occurred when the swimmer was treading water. 

Citation. Jacklin, D. 2022. Research summary: Assistive Lifeguard Technology. Water Incident Research Hub, 29 June; last updated 8 January 2023.