Development of real time systems to detect and track on duty injured firefighters using advanced signal processing techniques

Defining Problem

In 2015, a total of 68,085 on-duty US firefighters were injured, while 64 lost their lives, according to references [22, 45] Notably, fire ground injuries constituted the largest share of these incidents, with approximately 29,130 occurrences, representing around 43% of all reported firefighter injuries.

Figure 0.1 US firefighter injuries by type of duty during 2015 [45]

Vietnam experiences thousands of fires each year, with 2,357 incidents reported in 2014 and 2,792 in 2015 Firefighters face numerous risks during their duties due to inadequate protective systems This study aims to propose an effective and precise support system designed to safeguard firefighters' lives during fire and rescue operations.

Various injury detection systems, including fall detection systems and personal alert safety systems (PASS), have been created However, these systems have limitations when applied to on-duty firefighters, as they are primarily designed for the elderly or individuals with slower movement The PASS system, developed by Homeland Security, exemplifies this issue.

Responding to/or returing from incidents

A security system designed to detect inactivity over a specific timeframe activates a 95-decibel alarm when movement is absent However, in large-scale fire scenarios, various sounds, including human voices and the operation of fire detection and suppression systems, can render this audible alarm ineffective for on-duty firefighters To address this issue, U.S firefighters utilize the PASS system, yet tragic incidents, such as the Worcester cold storage and warehouse fire, highlight that some firefighters have still lost their lives despite these safety measures.

A tragic fire incident at the Charleston Sofa Super Store in Charleston, South Carolina, resulted in the deaths of nine firefighters, while another fire at 266 Franklin Street in Worcester, Massachusetts, led to the loss of six firefighters In response to these tragedies, this study proposes the integration of injury detection and indoor positioning algorithms to effectively locate on-duty firefighters and monitor their physical performance, aiming to enhance their safety and save lives.

The purpose of thesis

The thesis aims to research and develop a method to detect and track on-duty injured firefighters based on solving the five tasks as below:

+ Choosing the sensors and designing the proposed system, calibrating sensors, sensors fusion and map processing

+ Classifying activities and detecting the injured firefighter

+ Tracking and locating the indoor position of firefighter

To address the identified challenges, this study examines the causes of injuries among firefighters and evaluates the most effective methods for injury detection, fall detection, and indoor position localization We have selected the most appropriate sensors for integration into the proposed system Following the calibration of each sensor type, we analyzed the data recorded from these sensors.

The system integrates data from 20 sensors to identify firefighters' statuses, locate injured individuals, and assess their movement patterns, including turning direction, timing, step count, and step length Additionally, the building's map is processed to remove anomalies and establish a consistent scale This fused data, combined with the refined map information, allows for accurate tracking of firefighters' indoor positions.

Collecting and reviewing international publications on indoor positioning and injury detection systems, along with studies from Vietnam, is crucial for understanding current research trends This evaluation highlights the strengths and weaknesses of these methods, guiding the development of research directions specifically for on-duty firefighters in fire scenarios This process is vital for selecting appropriate sensors, designing the hardware system, proposing effective processing techniques, and analyzing data effectively.

- Choosing sensors and designning the proposed system, calibrating sensors, sensors fusion and map processing

The study utilizes a range of calibrated sensors, including a 3-DoF accelerometer, 3-DoF gyroscope, 3-DoF magnetometer, BMP180 pressure sensor, and MQ7 sensor, to eliminate random and systematic errors These sensors are connected to a microcontroller that records and processes data using an embedded algorithm The processed data is transmitted from within the building to an external workstation via a transceiver operating at an appropriate frequency This workstation displays the information on a building map, enabling real-time monitoring In the event of falls or crashes, the system sends incident details and the firefighter's location to the workstation, facilitating prompt and effective responses to support injured personnel.

- Activity classification and firefighter injury detection

The thesis introduces algorithms that utilize signal processing techniques for activity classification and injury detection in firefighters It highlights the benefits and limitations of these methods in monitoring and categorizing the states of on-duty firefighters, as well as in identifying instances of falling or fainting Once proposed, the algorithms will be tested on both public and private datasets to assess their performance effectively.

- Tracking and locating the indoor position of the firefighter

This thesis centers on developing algorithms to accurately assess the number and length of firefighters' steps, their turning times and directions, and altitude, utilizing multisensor data fusion The outcomes from these algorithms will be integrated with mapping information to effectively track and pinpoint the indoor locations of firefighters.

Based on the results obtained from the proposed algorithm, we will optimize the proposed algorithm to meet stable working and actual conditions.

Objectives and Scope of the Thesis

The thesis focuses on developing algorithms to detect and track injured firefighters in real fire conditions, emphasizing fall detection, physical performance loss, step counting, and turning estimations These algorithms aim to function without reliance on pre-installed systems, which may be compromised by heat or flames By utilizing a Decision Tree model rather than more complex machine learning models, the proposed solution minimizes computational demands, conserving energy and enabling real-time operation.

Research Methodology

This study successfully integrates theoretical research, simulation, experimental testing, and expert consultation to develop a multi-task system for on-duty firefighters It begins with an exploration of machine learning and signal processing techniques, followed by an analysis of fire conditions within buildings and the psychological states of firefighters The resulting system incorporates algorithms for injury detection and indoor position tracking, specifically designed for real fire scenarios.

Scientific significance and Contributions of the Thesis

Recent research on classifying firefighter activities, detecting injuries, and tracking their locations in indoor fire scenarios has produced significant findings that have practical applications aimed at enhancing the safety and protection of on-duty firefighters.

In addition, these findings help shorten the time for indoor victim search which contributes to ensure national security, social order and safety in the new situation

The thesis has made two novel contributions, as follows:

The integration of data from a three-axis accelerometer, gyroscope, magnetometer, barometer, and MQ7 sensor aims to enhance firefighter safety by detecting falls and declines in physical performance, monitoring high carbon monoxide levels to improve the efficiency of self-contained breathing apparatuses (SCBA), and providing metrics such as step count, step length estimation, vertical position, turning time, and turning direction.

A new, highly accurate step counting method has been introduced, distinguished by four key features: minimal peak distance, minimal peak prominence, dynamic thresholding, and vibration elimination This innovative approach enhances the precision of step detection, as highlighted in Publications No 3 and 5.

Thesis structure

Apart from introduction and conclusion, the thesis is divided into 5 chapters as follows:

Chapter 1 provides an overview of key literature on injury detection and indoor positioning systems, highlighting significant research findings It also addresses the challenges faced in these fields, offering insights into the complexities of advancing these technologies.

Chapter 2 focuses on the system description, sensor calibration, and map processing techniques It details the selection of sensors integrated into the proposed system, highlighting the measurement of errors associated with each sensor and the implementation of appropriate calibration methods to rectify anomalies in recorded signals Additionally, the chapter outlines the map processing algorithm, which employs dilation and erosion operations for effective map simplification, and presents the map scale utilized in the system.

Chapter 3 focuses on the development of a real-time, high-accuracy method for detecting injured firefighters using a novel data fusion algorithm This method leverages data from various sensors, including a 3-DOF accelerometer, gyroscope, magnetometer, barometer, and MQ7 sensor By integrating fall detection and loss of physical performance algorithms, the system effectively determines if a firefighter has experienced a fall or is suffering from diminished physical capabilities Additionally, the method incorporates a CO detection module to monitor carbon monoxide levels on the fire ground, issuing alerts for firefighters to don self-contained breathing apparatus (SCBA) when concentrations are high, while signaling safety under low concentration conditions to conserve fresh air for critical situations.

Chapter 4 focuses on developing a method for tracking on-duty injured firefighters, emphasizing step counting, step length estimation, turning time, and turning direction estimation The method achieves highly accurate step counting by incorporating four key features: minimal peak distance, minimal peak prominence, dynamic thresholding, and vibration elimination, with optimal thresholds tailored for different movement states.

24 features are combined with periodicity and similarity features to solve false walking problem

The adaptive step length estimation method utilizes 3-DoF acceleration and a height-based K parameter, significantly enhancing performance and flexibility without the need for predefined environment settings or fixed axes.

The estimation of turning time and direction utilizes a magnetic sensor to capture the Earth's magnetic field across the Mx, My, and Mz axes Positioned in trouser pockets, the hardware aligns the My-axis parallel to Earth's gravity, resulting in minor fluctuations in the My-axis readings, while more significant variations occur in the Mx and Mz axes By analyzing these changes in magnetic values from the Mx and Mz axes, the user's directional movements can be accurately estimated.

Chapter 5: Indoor Firefighter Positioning and Tracking Using Multi-Sensor Data Fusion and Map Matching Algorithm

This chapter outlines key metrics such as step count, step length, turning time and direction, altitude, and injury detection, all of which are visually represented on the map Furthermore, Chapter 5 details the testing and evaluation of the proposed method, demonstrating its performance across a range of scenarios, showcasing high stability and accuracy from simple to complex tasks.

OVERVIEW OF THE RESEARCH

Literature review

Numerous studies have focused on developing safety systems and devices to aid firefighters, such as the Personal Alert Safety System (PASS) created by the Building and Fire Research Laboratory at the National Institute of Standards and Technology This device emits an audible alarm if the wearer remains motionless for a set period, alerting nearby firefighters However, the effectiveness of the PASS device can be compromised in real fire situations due to excessive background noise from engines, sirens, emergency equipment, and fire protection systems, making it less suitable for use in such environments.

The study methods on indoor positioning using pre-installed systems as in [18,

Recent studies have highlighted the growing popularity of combining portable devices with pre-installed systems like transmitters and access points for accurate position tracking However, these pre-installed systems are vulnerable to damage from flames or heat during a fire, limiting their effectiveness Additionally, this approach lacks mobility, as explosions or fires can happen unexpectedly, making it suitable only for predetermined areas that have been specifically set up.

Recent studies have explored indoor positioning techniques that do not require pre-installed systems, as detailed in various publications These methods involve users carrying portable devices equipped with sensors to collect and process data for accurate location tracking.

A research approach suitable for indoor positioning in emergency situations—such as security, defense, and fire incidents—does not necessitate pre-installed systems Portable devices can be safeguarded against high temperatures by being worn under firefighters' clothing, maintaining system integrity in fire conditions However, a significant limitation of these studies is the low accuracy in indoor positioning, primarily due to drift and cumulative errors associated with the sensors.

Related Studies on Injured Detection

Most published studies on injury detection primarily focus on fall detection, utilizing various technologies such as cameras, location sensors, smartphones, accelerometers, and wearables like wristbands and smartwatches These studies often develop fall detection algorithms based on the collected data, predominantly targeting elderly individuals engaged in less physically demanding activities However, the reliance on cameras can be problematic in fire conditions due to visibility issues, making these methods inadequate for protecting on-duty firefighters who face challenging and hazardous environments.

A waist-mounted system integrating four sensor types—3-DOF accelerometer, 3-DOF gyroscope, 3-DOF magnetometer, and barometer—was proposed for detecting fall events in the elderly The study introduced four key features: Impact, Aftermath, Posture, and Altitude variation, highlighting the novel use of barometric data to differentiate between normal and fall states based on altitude changes However, the method's accuracy may be compromised due to variations in wearing positions, volunteer height, and sensor types.

The proposed method for measuring barometer and environmental noises is primarily designed for elderly individuals, which may limit its effectiveness when applied to other groups, particularly firefighters.

The Proetex project aims to enhance safety by integrating multiple sensors, including a 3-DOF accelerometer, motion sensor, and various health monitors like breathing, heart rate, temperature, and gas sensors These sensors transmit data to a workstation via Wi-Fi, enabling improved data fusion for more accurate monitoring However, the project's fall detection algorithm is relatively basic, which may affect system performance during common firefighter activities such as crawling or sprawling.

A recent publication proposed a wearable system designed to monitor the heartbeat and respiration cycles of firefighters, focusing on their health during training drills However, the study did not address the detection of fall events or injuries, making the system unsuitable for monitoring injuries sustained by firefighters while on duty.

Various studies have explored the use of Microsoft's Kinect depth cameras for fall detection via image processing However, these methods face significant challenges in real-world applications due to their reliance on camera resolution, the distance between the camera and objects, and user privacy concerns Additionally, in the event of a fire within a structure, the resulting obscured environment makes the effectiveness of cameras in such conditions highly questionable.

The study [53] introduces the use of location sensors equipped with accelerometers for activity recognition aimed at fall detection After preprocessing, the sensor data is utilized to identify falls, with the results further refined by analyzing changes in acceleration However, a significant limitation of this research is the requirement for sensors to be placed at multiple locations on the human body, including the chest and waist.

28 ankles may create inconvenience during movement, especially for on-duty firefighters

Recent studies have demonstrated the widespread use of low-cost accelerometers for fall detection, employing posture recognition and fall detection algorithms to analyze data from accelerometers and gyroscopes placed on the waist, chest, and thigh However, these studies have faced challenges regarding precision Additionally, one study suggested using a minimum of three accelerometers positioned at different body locations, which may lead to inconvenience and discomfort for users.

Recent studies have highlighted the use of smartphone built-in sensors for fall detection, but the performance of algorithms can vary significantly due to differences in 3-DOF accelerometers and gyroscopes across devices Additionally, incoming and outgoing calls may interfere with fall detection accuracy Low-cost smartphones often lack essential sensors like barometers and heart rate monitors, which are crucial for injury detection and indoor positioning Meanwhile, wristbands and smartwatches have gained popularity for fall detection, as they are comfortable to wear However, a significant challenge remains in distinguishing between daily activities and fall events due to frequent hand movements.

Despite numerous studies aiming to develop effective fall detection methods, most have primarily targeted the elderly and patients This study highlights a significant gap in research, as it focuses on on-duty firefighters who engage in diverse activities under challenging fire conditions.

Furthermore, the use of camera to detect a fall in fire conditions is not applicable because of invisible environment Therefore, the above studies are insufficient in protecting on-duty firefighters.

Related Studies on Indoor Positioning

Indoor positioning systems are gaining significant attention from researchers due to their diverse applications, especially since GPS is often unreliable for indoor location tracking Various studies have proposed solutions to this challenge, primarily focusing on systems that rely on pre-installed infrastructures such as access points, transmitters, and wireless networks While these systems can achieve high accuracy, they come with substantial costs and stringent setup requirements, making them suitable only for pre-installed buildings In contrast, other research has explored the use of built-in sensors and algorithms to determine indoor positions without the need for pre-installed infrastructure, allowing for application in unknown environments; however, the accuracy of these methods remains a concern.

This study focuses on determining the indoor positioning of firefighters operating in unfamiliar environments during fire conditions An optimal solution is an indoor positioning system that does not rely on pre-installed infrastructure The research proposes algorithms for accurate step counting, estimating step length, turning time, turning direction, and vertical position Integrating various information sources is crucial for improving the accuracy of an indoor positioning system utilizing Inertial Measurement Units (IMUs).

The challenges in study on injury detection and indoor positioning

Based on the above literature review as well as the limitations of these publications, I can ensure that huge challenges related to injury detection are

Firefighters face a variety of injuries during their tasks, and accurately estimating these injuries is crucial Additionally, developing an algorithm to detect injured firefighters with high precision presents significant challenges Furthermore, utilizing indoor positioning based on Inertial Measurement Units (IMU) can effectively locate firefighters within a fire scene without the need for pre-installed systems; however, this method may suffer from low accuracy due to sensor drift and cumulative errors.

Summary

This chapter examines relevant studies on injury detection and indoor positioning methods, highlighting their limitations It also outlines the research goals and the challenges faced throughout the study.

SYSTEM DESCRIPTION, SENSOR ERRORS

System Description

The proposed thesis introduces a multi-sensor system that integrates an IMU, barometer, and MQ7 sensor, connected to a microcontroller unit via I2C (Inter-integrated Circuit) interface This system is divided into two components: a portable transmitter carried by volunteers and a stationary receiver positioned at a base station outside the monitored building Volunteers can conveniently keep the transmitter in their trousers pocket to collect data, which is then wirelessly transmitted to the base station using the nRF2401 module Detailed system descriptions are illustrated in Figure 2.1.

Figure 2.1 The block diagram of the proposed system

The Inertial Measurement Unit (IMU) collected data on acceleration, angular velocity, and magnetic fields along the Ax, Ay, and Az axes at a sampling frequency of 50 Hz This frequency was specifically chosen for the study, as it aligns with the rapid activities performed by firefighters.

The proposed system incorporates a barometer to measure pressure data, which is then utilized to estimate altitude This estimated altitude is combined with map information to accurately determine vertical position, allowing for the differentiation between loss of physical performance and other on-duty activities, such as using an elevator.

The MQ7 sensor is crucial for measuring carbon monoxide (CO) levels in fire environments, as CO is a prevalent and hazardous toxin during fires Given that the compressed air supply in a self-contained breathing apparatus (SCBA) is limited to approximately 30, 45, or 60 minutes, the integration of the MQ7 sensor with an embedded algorithm empowers firefighters to make informed decisions about when to utilize their compressed air supply effectively.

Utilizing second-order integration of accelerometer signals for indoor positioning can lead to significant cumulative errors To mitigate these inaccuracies, we propose a sensor fusion-based approach that develops an indoor positioning method by creating algorithms for step counting, step length estimation, turning time, turning direction, and altitude measurement The integration of these algorithmic results will enhance the detection of indoor positions, with detailed explanations provided in Chapters 3 and 4 of the thesis.

Sensors Errors Elimination

The systematic and random errors are two common errors in sensors Hence, in the thesis, the calibration process and the Kalman filter were appliedto eliminate these errors.

Accelerometer

Map Processing

The map undergoes a transformation from color to grayscale images, which are then converted into binary images using a proposed threshold of 0.5412 In this binary representation, pixel values are categorized as either 0 or 1 If a pixel in the grayscale image exceeds the specified threshold, it is represented as 1 in the binary image.

39 it is assigned to equal 1 in binary image and 0 when it is smaller than the proposed threshold value

The binary image contains only two discrete values: 0 or 1, thus being more convenient for image processing The following formula is used to convert to binary image:

Figure 2.7 illustrates a binary image of a floor utilized in map processing and simplification, featuring various objects such as walls, stairs, elevators, tables, chairs, beds, toilets, texts, televisions, windows, and doors However, certain objects within the image are either movable or insignificant for accurately tracking and localizing indoor positions.

Figure 2.7 The binary image of a floor used in experimental testing

The presence of movable furniture, such as tables, chairs, and beds, can complicate tracking and introduce computational challenges To address this issue, the proposed algorithm focuses on eliminating these non-essential objects from images, retaining only fixed structures like walls and stairs This approach employs two fundamental operations in mathematical morphology—Dilation and Erosion—to enhance image clarity and facilitate better object recognition.

Figure 2.8 The map simplification algorithm

The Dilation operation enhances binary image elements using a structuring element, transforming discrete elements into continuous ones This transformation improves the accuracy of indoor positioning algorithms by enabling automatic adjustments to indoor locations For instance, when estimated positions are inaccurately distributed on either side of a wall—an unlikely scenario—unwanted objects are automatically eliminated, refining the positioning process.

The dilation of A by the structuring element B is defined as the following [29]:

The erosion operation plays a crucial role in image processing by reducing the size of elements in a binary image using a structuring element, effectively eliminating noise and unwanted objects like tables and chairs However, this process also inadvertently shrinks important features such as walls and stairs To counteract this effect, the dilation operation should be applied after erosion to restore the main elements to their original size Additionally, the parameters for the erosion operation should be determined through experimental testing to achieve optimal results.

Figure 2.9 The Erosion and Dilation operations [17]

The erosion of A by the structuring element B is defined as the following [29]:

In binary images, the values 0 and 1 correspond to black and white colors This study introduces the use of two structuring elements, specifically (7 × 7) and (4 × 4) sizes The larger (7 × 7) structuring element is utilized to preserve significant structures like walls, while the smaller (4 × 4) element focuses on retaining essential details such as stairs and windows Detailed results of this approach are presented in the following sections.

Figure 2.10 The result of applying Erosion and Dilation with structuring element size of (7 × 7) for keeping wall

Figure 2.11 The result of applying Erosion and Dilation with structuring element size of (4 × 4) for keeping windows and stairs

With the combination of Figure 2.10 and Figure 2.11, the achieved map simplification result is as the following (see Figure 2.12):

Figure 2.12 The Map simplification achieved after using Erosion and Dilation operations

Map scale refers to the ratio between the size of a map and the actual ground distance, specifically relating image pixels to real-world dimensions in centimeters This article discusses an algorithm developed to calculate this ratio by analyzing the pixel dimensions of an image alongside the real size of buildings or constructions To determine the size of a floor, the flood fill algorithm is utilized, providing an effective method for accurate measurement.

The flood fill algorithm is applied to detect the size of a floor

Function void Floodfill(pixel_x, pixel_y)

If (InsideBoundary(pixel_x, pixel_y) && NotVisited(pixel_x, pixel_y)) Then

To implement the flood fill algorithm, the central pixel is designated as the initial coordinate (pixel_x, pixel_y) Subsequently, the algorithm is utilized to determine the size of the floor.

The 𝐼𝑛𝑠𝑖𝑑𝑒𝐵𝑜𝑢𝑛𝑑𝑎𝑟𝑦 function determines if a pixel is within a specified boundary A pixel value of 1 indicates that it is inside the boundary (white color), while any other value signifies that it lies outside the boundary.

The NotVisited function is used to confirm if a pixel has been considered or not The SetVisited function used to tick on the pixel that has been considered

Figure 2.13 The floor size detected after applying the flood fill algorithm

Figure 2.13 depicts the floor area obtained through the application of the flood fill algorithm The calculation of the ratio between the pixel count of white areas along the length and width and the actual floor size in centimeters is derived from this data.

For unclosed opening floor design as shown in Figure 2.14, it needs to apply the dilation operation for preprocessing before applying the flood fill algorithm

Figure 2.14 The unclosed floor structure

Figure 2.15 illustrates the outcome of an unclosed floor structure obtained through the application of the dilation operation and flood fill algorithm.

Summary

This chapter introduces a novel algorithm for calibrating multi-sensors, including a 3-DOF accelerometer, 3-DOF magnetometer, barometer, and MQ7 sensor The calibration process effectively eliminates abnormal signal components, enhancing data accuracy Additionally, advanced map processing techniques are utilized to filter out irrelevant objects while preserving essential structures like walls and stairs, and accurately estimating the map scale.

DEVELOPMENT OF A METHOD TO DETECT

Fall Detection Method

The proposed fall detection algorithm, illustrated in Figure 3.1, utilizes acceleration data recorded along the Ax, Ay, and Az axes To analyze this data, the root mean square (RMS) of the accelerations across the three axes is calculated using Equation (3.1).

A fall can be categorized into three distinct phases: the flight of fall, initial contact with the ground, and the resting state During the flight phase, the acceleration (Acc) value drops below the lower fall threshold (LFT), while in the initial contact phase, it rises above the upper fall threshold (UFT) In the final resting phase, the Acc value stabilizes around [0, 1, 0] g This study focuses on comparing Acc values against the LFT and UFT thresholds to identify falls A fall is confirmed if the Acc values exceed both thresholds and the duration from when the Acc surpasses the LFT until it exceeds the UFT, denoted as t FE, exceeds a specified time threshold.

47 Figure 3.1 The proposed fall detection algorithm

Figure 3.2 depicts a real fall event, highlighting three distinct phases and the threshold values of LFT, UFT, and tFE During the fall, the Acc value drops below the LFT threshold and fluctuates within the tFE range before surpassing the UFT threshold In the final phase, the body reaches a resting state, with recorded Acc values stabilizing around 9.81 m/s².

Figure 3.2 An example of a fall event and the UFT, LFT and tFE thresholds

After a fall, the body enters a resting state where the recorded acceleration values stabilize around 9.81 m/s² To improve user state recognition and enhance the effectiveness of the proposed method, this thesis introduces a post-fall recognition module, which encompasses both posture recognition and vertical velocity estimation components.

The posture recognition module estimates the state of the user after 2s after an event is confirmed by the fall detection module as a fall event Checking after 2s is to

49 confirm that any fluctuations after falling do not affect the accuracy of the proposed algorithm

The proposed algorithms implement a double consecutive check with a 0.5-second interval to prevent overlapping peak check times, as each step cycle is estimated to last approximately 1 second, as illustrated in Figure 3.3.

Figure 3.3 The time cycle of 6 walking steps

Posture recognition following a fall is crucial in the proposed fall detection algorithm, as it verifies the accuracy of the detected fall event This verification involves estimating the angle 𝜃 between the Ay axis and gravity, with the accelerometer positioned on the user's waist In a standing position, the Ay axis aligns with gravitational acceleration (see Figure 3.5a), but this alignment shifts during activities like walking or running, though these changes remain under 90 degrees In contrast, the angle approaches 90 degrees after a fall The posture is assessed using the scalar product of the reference gravity acceleration vector 𝐴⃗ 0 and the Ay vector at time (t).

The acceleration values are around 9.81 m/s 2 at the rest states such as standing, sitting and lying, and these values are different in other states such as running,

The estimation of vertical velocity can effectively differentiate between resting and active states, such as walking, crawling, or lying down When gravity is accounted for, the vertical velocity during rest approaches zero The following equation is utilized to estimate vertical velocity accurately.

𝑉 < 𝑣 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 (3.5) where, 𝑣 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 is the threshold to distinguish between the rest and the active states

In the case of the Formula (3.5) being satisfied, the algorithm confirms that the user is staying in the rest state and otherwise, it is confirmed as an active state

Figure 3.4 The different states of the user along three axes Ax, Ay and Az a) The relationship among Acc acceleration and UFT, LFT thresholds

The proposed method effectively detects fall events with high accuracy across both private and public datasets However, it faces limitations when applied to injured on-duty firefighters, as they engage in complex and strenuous activities that differ significantly from those performed by the general population or the elderly.

The thesis covers some new and special activities such as elevator using, crawling, crawling then falling Therefore, the thesis proposed to improve this fall

52 detection method as well as integrating barometer with embedded algorithm to enhance the accuracy and the performance of the proposed injury detection system for on-duty firefighters.

Injury Detection for On-Duty Firefighters

The injury detection algorithm designed for on-duty firefighters, illustrated in Figure 3.6, consists of two primary components: the fall detection module and the loss of physical performance module The fall detection module incorporates three key features: an upper threshold, post-fall assessment, and posture recognition Meanwhile, the loss of physical performance module introduces two essential features: an altitude threshold and a loss of physical performance threshold.

Figure 3.6 The injury detection algorithm for firefighters

The Proposed Fall Detection Algorithm for Firefighter

The proposed fall detection algorithm, designed specifically for on-duty firefighters, integrates three key features: Upper Threshold, Post Fall, and Posture Recognition This algorithm has been rigorously tested using experimental data collected from firefighters simulating typical activities encountered during fire response operations Detailed descriptions of these features are provided in the subsequent sections.

The body enters a "flight or fall" phase after losing contact with the ground, leading to a significant increase in acceleration due to gravitational forces upon initial contact with the ground or other objects Therefore, this thesis suggests utilizing the upper threshold to identify sudden changes in acceleration.

𝐴𝑐𝑐 (𝑗) − 𝑈 𝑡ℎ > 0 (3.6) where, 𝐴𝑐𝑐 (𝑗) is the acceleration value at the sample/time of j; 𝑈 𝑡ℎ is the upper threshold

After a fall, the body experiences fluctuations for several seconds before settling into a rest state, where the root mean square (RMS) of acceleration is approximately 1g This study introduces a post-fall feature that utilizes upper and lower thresholds to assess the volunteer's condition three seconds after the acceleration surpasses the upper threshold Experimental testing determined the upper threshold at 1.25g and the lower threshold at 0.75g If the RMS of acceleration remains between these threshold values for two seconds, the fall event is confirmed based on the post-fall feature.

&&L 𝑝𝑡 < (𝐴𝑐𝑐 𝑗+3∗𝐹𝑠 : 𝐴𝑐𝑐 𝑗+5∗𝐹𝑠 )) (3.7) where, U 𝑝𝑡 and L 𝑝𝑡 are the upper and lower thresholds to check the post-fall condition

Posture recognition is crucial for assessing the position of volunteers after a fall, as their roll, pitch, and yaw angles shift relative to a reference frame This study focuses on utilizing roll and pitch angles to determine firefighters' postures post-fall To improve the accuracy of the fall detection method, the theta (T) angle, influenced by the Ay axis and gravity, is integrated with the roll and pitch angles When standing, the theta (T) angle is approximately 0°, but it approaches 90° after a fall, indicating a significant change in posture Specific conditions are established to calculate the theta, roll, and pitch angles effectively.

- Condition 1: The 𝑇 angle estimation: The 𝑇 angle is estimated by the use of the Equation (3.8):

The value of the 𝑇 angle is essential for checking the postures of the volunteers before the fall event is confirmed or rejected

The Madgwick orientation filter is applied to eliminate noises in the inertial measurement unit (IMU) Then, the Euler angles are estimated by the following equations [83]:

𝑃 = −𝑠𝑖𝑛 −1 (2𝑞 2 𝑞 4 + 2𝑞 1 𝑞 3 ) (3.10) R= 𝐴𝑡𝑎𝑛2(2𝑞 3 𝑞 4 − 2𝑞 1 𝑞 2 , 2𝑞 1 2 + 2𝑞 4 2 − 1) (3.11) where, 𝑌, 𝑃, and R are the yaw, pitch, and roll angles which rotate around the Az, Ay, and Ax axes of the reference frame, respectively

- The combination of the condition 1 and condition 2:

The combination of theta, pitch, and roll angles significantly enhances the accuracy of the proposed fall detection method Additionally, implementing a double-check mechanism with a 0.5-second interval further improves the method's precision The outcomes derived from this combination of angles are detailed in Table 3.1.

Table 3.1 The decision based on the combination of the theta, pitch, and roll angles

Theta Angle (T) Pitch Angle (P) Roll Angle (R)

1st 2nd 1st 2nd 1st 2nd

One/some or all of these angles is/are smaller than their threshold

After 3 seconds of the RMS of the acceleration signal exceeding the upper threshold (𝑈 𝑡ℎ), post-fall and posture recognition features are employed to validate the fall event The fall event is confirmed only when all features are satisfied; if any feature is not met, the event is disregarded.

Upon confirmation of a fall event, an "Alert message" is dispatched to the commander via the integrated indoor positioning system, which effectively tracks and locates on-duty firefighters.

56 Figure 3.7 The proposed fall detection algorithm

The Proposed Loss of Physical Performance Detection Algorithm for Firefighter

The fall detection algorithm typically identifies falls by monitoring significant changes in acceleration that exceed a predetermined threshold However, on-duty firefighters face unique risks, including incidents like crawling, falling, or becoming trapped in tight spaces, which can complicate the effectiveness of these algorithms Consequently, relying solely on acceleration changes may not accurately capture the complexities of injuries encountered by firefighters in the line of duty, potentially undermining the reliability of the detection methods.

The proposed loss of physical performance algorithm addresses the limitations of previous fall detection systems by utilizing data fusion from pressure sensors and 3-DoF acceleration to effectively identify accident events among on-duty firefighters Key components of this algorithm include the Loss of Physical Performance Threshold and the Altitude Threshold, which will be elaborated on in the subsequent section.

The Loss of Physical Performance Threshold is crucial for identifying when firefighters are unable to move through narrow spaces, as illustrated in Figure 3.8 If a firefighter remains stationary for approximately 4 seconds due to a fall, being trapped, or other circumstances, an "Alert message" is triggered and sent to the external command station This allows the commander to make informed decisions to provide necessary support.

Figure 3.8 Firefighters move through the narrow paths or spaces

The proposed system, equipped with a barometer, is designed to be conveniently mounted in the front trouser pocket of volunteers By employing a simple Kalman filter, the system effectively removes anomalies and vibrations from the recorded barometric data, resulting in a clean and stable signal This altitude information is crucial for accurately predicting the state of the volunteers.

The thesis integrates altitude variations and acceleration values to predict user states, with the estimated states derived from the calculations outlined in Formulas (3.15) to (3.17).

The proposed algorithm predicts whether a user is in a "moving up" state based on acceleration values and altitude variation that meet the criteria of Formula (3.15) Additionally, Formula (3.16) and Formula (3.17) are utilized to assess the "moving down" state and the loss of physical performance state, respectively.

Figure 3.9 The proposed loss of physical performance detection algorithm

Figure 3.10 The high CO level alerting algorithm

The proposed system integrates a MQ7 sensor to record CO data, as illustrated in Figure 3.10 Following data collection, a preprocessing step is employed to eliminate any abnormal signals The estimated CO level, denoted as 𝐶𝑂_𝑚𝑒𝑎, is then compared with the threshold level, 𝐶𝑂_𝑡ℎ, which is determined through statistical analysis and the signs and symptoms of blood Carboxyhemoglobin (COHb) related to adverse health effects This comparison is crucial for deciding whether to trigger an alarm.

[56] are shown in Table 3.2 below:

Table 3.2 Carbon monoxide concentrations, COHb levels, and symptoms

COHb Level Signs and Symptoms

35 ppm 10% Slight headache in 2 to 3 h

200 ppm 20% Slight headache, fatigue within 2 to 3 h

400 ppm 25% Frontal headache within 1 to 2 h

800 ppm 30% Dizziness, nausea, and convulsions within 45 min; insensible within 2 h

1600 ppm 40% Headache, tachycardia, dizziness, and nausea within 20 min; death in less than 2 h

3200 ppm 50% Headache, dizziness, and nausea in 5 to 10 min; death within 30 min

6400 ppm 60% Headache and dizziness in 1 to 2 min; convulsions, respiratory arrest, and death in less than 20 min

12,800 ppm >70% Death in less than 3 min.

Result and Discussion

In the experimental setup, volunteers were randomly chosen from a pool of firefighters at the University of Fire Prevention and Fighting (UFPF) in Vietnam Table 3.3 provides detailed information regarding the height, gender, age, and weight of the selected volunteers.

Number of Volunteers The Trial Times Gender Age Height Weight

In an experiment involving five volunteers, data was collected over three trial sessions to differentiate between various activities, including falls, loss of physical performance, and movements such as running, walking, crawling, and standing.

62 jumping and jogging Figure 3.11 is about a volunteer who was crawling to record data and executing the experimenetal resting with the proposed device in his trouser pocket

Figure 3.11 The volunteer is carrying the support device in his trouser pocket in the crawling state viewed from a side (a) and from above (b)

Table 3.4 The parameters value that used in the proposed algorithms

The Symbol The Value The Meaning

𝑈 𝑡ℎ 1.8 g The threshold to detect the acceleration exceeds

U 𝑝𝑡 1.25 g The upper threshold to check post-fall condition

L 𝑝𝑡 0.75 g The lower threshold to check post-fall condition

𝑇 25° The theta threshold angle to check Posture

𝑃 30° The pitch threshold angle to check Posture

𝑅 30° The roll threshold angle to check Posture Recognition condition

𝛥 𝐴𝑙𝑡𝑖𝑡𝑢𝑑𝑒 0.5 m The altitude threshold to confirm loss of physical performance condition

𝐿 𝑢_𝑚𝑜𝑣 1.2 g The upper threshold to check loss of physical performance condition

𝐿 𝑙_𝑚𝑜𝑣 0.8 g The lower threshold to check loss of physical performance condition CO_th 35 ppm The threshold to check high CO concentration environment

Table 3.4 presents the parameter values utilized in our proposed algorithm, with the exception of the CO_th value These parameters were selected based on experimental testing conducted through the ROC (receiver operating characteristic) curve analysis.

Figure 3.12 illustrates the RMS of acceleration during a forward fall event, highlighting the variations in RMS magnitude, theta angle, and both pitch and roll angles that occur during such an incident.

In Figure 3.12a, the RMS signal magnitude abruptly shifted when the body made contact with the ground, indicating a transition from "flight of fall" to a resting state Following the fall event, which occurred around 3 seconds, the theta, pitch, and roll angles also experienced significant changes, as illustrated in Figures 3.12b and 3.12c Specifically, the theta angle increased from approximately 10° to nearly 85°, while the pitch and roll angles shifted from 0° to 17° and from 0° to 60°, respectively.

Figure 3.12 (a) The RMS of acceleration of a fall forward from standing, first impact on knees; (b) the theta angle; (c) the pitch and roll angles

The study aims to evaluate the post-fall condition after a delay of 3 seconds, during which the root mean square (RMS) of acceleration surpasses the upper threshold This 3-second interval is chosen to confirm that the body has stabilized following contact with the ground or other objects.

Loss of Physical Performance Detection

Firefighting and rescue operations in high-rise buildings often necessitate the use of elevators for efficient movement between floors However, the activation of the PASS device in these scenarios can lead to false alarms This article proposes an innovative algorithm that utilizes data fusion of acceleration and pressure readings to accurately differentiate between actual emergencies and the normal use of elevators, enhancing operational efficiency and safety during such critical situations.

Figure 3.13 illustrates the impact of physical performance loss due to being immobilized or involved in an accident compared to moving in an elevator Specifically, Figures 3.13a and 3.13b depict the root mean square (RMS) of acceleration and pressure changes, respectively, during a firefighter's transition from crawling to falling, highlighting minimal variations in the signals.

In the case of the elevator moving up as shown in Figure 3.14, the RMS of acceleration values have little change while the pressure values decrease dramatically

Figure 3.13 The loss of physical performance because of the accident (crawling then falling); (a) the RMS of accelerometer data; (b) the barometric data

(b) Figure 3.14 The loss of physical performance because of moving up in an elevator; (a) the RMS of accelerometer data; (b) the barometric data

The fusion of RMS acceleration and pressure data effectively identifies declines in physical performance among firefighters, enabling accurate predictions of their condition in critical situations.

The High CO Level Alerting Algorithm

Firefighters face serious health risks from toxic gases produced during fires, including aldehydes, fine particles, hydrogen sulfide (H2S), hydrogen cyanide (HCN), carbon dioxide (CO2), nitrogen dioxide (NO2), and nitrous oxide (N2O) Aldehydes are linked to various diseases, such as cancer, cardiovascular issues, respiratory allergies, cerebral ischemia, neurodegenerative disorders, and stroke Additionally, fine particulate matter is known to contribute to respiratory diseases, cardiovascular problems, and cancer.

The proposed system is a crucial component of an ongoing project aimed at supporting on-duty firefighters Future research will integrate additional sensors to detect aldehyde, fine particles, CO2, and HCN This study primarily focuses on developing an algorithm to monitor CO concentration and alert users to elevated CO levels.

Figure 3.15 illustrates the experimental measurement of CO levels in the fire condition and the clean air environments The CO values in clean environment are

67 around 7 ppm (parts per millions), and they significantly increaseto the range of 33 ppm to 45 ppm (see Figure 3.15b) in the fire environment

Figure 3.15 a) Testing and measuring the CO level in the fire; b) the measured

An experiment conducted at the University of Fire Prevention and Fighting in Vietnam indicates that a carbon monoxide (CO) concentration of approximately 35 ppm poses significant health risks to firefighters, as prolonged exposure of 6-8 hours can lead to symptoms such as headaches and dizziness Additionally, the U.S Occupational Safety and Health Administration (OSHA) has established guidelines that identify CO levels as hazardous in workplace environments.

35 ppm Based on the experimental testing, the signs/symptoms and the suggested of

According to the US OSHA CFR, the proposed carbon monoxide (CO) threshold for alerting individuals to hazardous environments is set at 3 ppm When CO levels fall below this threshold, the system advises firefighters to remove their breathing apparatus to conserve fresh air for more critical situations Conversely, if CO levels exceed this limit, it is essential to maintain protective equipment.

The Comparison

The thesis presents an innovative system that integrates five key sensors: a three-axis accelerometer, a three-axis gyroscope, a three-axis magnetometer, a barometer, and an MQ7 sensor Additionally, it introduces a sophisticated algorithm designed to effectively combine data from these sensors to detect falls and monitor declines in physical performance.

In the comparison, the thesis considered algorithms as follows:

 Algorithm 1: The proposed algorithm with the full features as proposed in Figure 3.6

 Algorithm 2: The reduced version of algorithm 1 (without checking theta, pitch, and roll angles at the second stage)

 Algorithm 3: The reduced version of algorithm 2 (without using condition 1)

 Algorithm 4: The reduced version of algorithm 2 (without using condition 2)

 Our previous fall detection algorithm (publication number 3)

 Paola Pierleoni et al algorithm [70]

The previous fall detection algorithm primarily utilized a three-axis accelerometer to identify fall events among the elderly In contrast, Paola Pierleoni et al proposed an enhanced fall detection system that integrates four types of sensors: a three-axis accelerometer, a three-axis gyroscope, a three-axis magnetometer, and a barometer, along with an embedded algorithm for improved accuracy.

The Comparison on the Experimental Data

Table 3.5 The features of the experimental datasets

Falls Forward fall, Backward fall, Lateral left fall, Lateral right fall

Engaging in various forms of movement, such as walking, running, and crawling on flat surfaces and stairs, is essential for physical fitness Activities like walking up and down stairs, running, and even crawling can enhance strength and endurance Additionally, incorporating jumping exercises and using elevators for vertical movement adds variety to your routine Embracing these diverse movements promotes overall health and mobility.

The results from Figure 3.16 to Figure 3.18 presented the forward fall from standing (see Figure 3.16) and crawling then falling (see Figure 3.17 and Figure 3.18)

It can be seen that, applying the fall detection algorithm by us in publication number

The methods proposed by 3 and Paola Pierleoni et al [70] can successfully detect fall events, as illustrated in Figure 3.16 However, they struggle to identify falls depicted in Figures 3.17 and 3.18 due to insufficient changes in acceleration that fail to surpass the established thresholds, specifically the UFT threshold from our publication and the Impact threshold from [70] Notably, the peak generated by crawling and falling in Figure 3.18 is significantly lower than those produced by leg movement Consequently, these techniques exhibit limitations in effectively detecting injuries among on-duty firefighters.

Figure 3.16 The RMS of acceleration of a fall forward from standing

Figure 3.17 The RMS of acceleration of crawling then falling as the scenario of

To evaluate the accuracy, specificity, and sensitivity of the proposed algorithm and the comparison methods, the thesis used the following equations:

The equation TP + TN + FP + FN (3.20) is used to evaluate the effectiveness of a fall detection system In this context, True Positives (TP) represent instances where a fall is correctly detected, while False Positives (FP) indicate normal activities mistakenly identified as falls True Negatives (TN) refer to fall-like events accurately recognized as normal activities, and False Negatives (FN) signify falls that occur but go undetected by the system.

Figure 3.18 The RMS of acceleration of crawling then falling as the scenario in

Table 3.6 presents a comparison of the testing performance of our newly proposed algorithms for fall detection and loss of physical performance detection alongside our previous fall detection algorithm and the algorithm developed by Paola Pierleoni et al., evaluated on our experimental datasets.

The Algorithms Comparison Sen Spec Acc

Our previous fall detection algorithm

Paola Pierleoni et al algorithm [70] 66.7% 100% 83.33%

The proposed algorithms demonstrate ultra-high accuracy in detecting both falls and loss of physical performance events, as indicated in Table 3.6 The integrated barometer within the system effectively differentiates between genuine loss of physical performance due to accidents, injuries, or being stuck, and false loss resulting from elevator movement Additionally, the loss of physical performance detection algorithm proves effective in identifying performance loss when volunteers are crawling, even if their accelerations do not exceed the established threshold.

The Comparison on Public Datasets

Table 3.7 The features of the public datasets [75], [67]

901 front-lying, from vertical falling forward to the floor

902 front-protecting-lying, from vertical falling forward to the floor with arm protection

903 front-knees, from vertical falling down on the knees

904 front-knees-lying, from vertical falling down on the knees and then lying on the floor

905 front-quick-recovery, from vertical falling on the floor and quick recovery

906 front-slow-recovery, from vertical falling on the floor and slow recovery

907 front-right, from vertical falling down on the floor, ending in right lateral position

908 front-left, from vertical falling down on the floor, ending in left lateral position

909 back-sitting, from vertical falling on the floor, ending in sitting

910 back-lying, from vertical falling on the floor, ending in lying

911 back-right, from vertical falling on the floor, ending in lying in right lateral position

912 back-left, from vertical falling on the floor, ending lying in left lateral position

913 right-sideway, from vertical falling on the floor, ending in lying

914 right-recovery, from vertical falling on the floor with subsequent recovery

915 left-sideway, from vertical falling on the floor, ending lying

916 left-recovery, from vertical falling on the floor with subsequent recovery

917 rolling out of bed, from lying, rolling out of bed and going on the floor

918 podium, from vertical standing on a podium going on the floor

919 syncope, from standing falling on the floor following a vertical trajectory

920 syncope-wall, from standing falling down slowly slipping on a wall OADs 801 walking-fw, walking forward

804 squatting-down, squatting, then standing up

806 bending-pick-up, bending to pick up an object on the floor

809 trip-over, bending while walking and then continuing walking

810 coughing-sneezing, coughing or sneezing

811 sit-chair from vertical, to sitting with a certain acceleration onto a chair (hard surface)

812 sit-sofa from vertical, to sitting with a certain acceleration onto a sofa (soft surface)

813 sit-air from vertical, to sitting in the air exploiting the muscles of legs

814 sit-bed from vertical, to sitting with a certain acceleration onto a bed (soft surface)

815 lying-bed, from vertical lying on the bed

816 rising-bed, from lying to sitting

Beside the comparison on the experimental datasets, the above algorithms are also compared on two public datasets [75], [67] The features of these datasets are

The proposed injury event detection algorithm for firefighters faces limitations due to the unsuitability of certain public datasets, particularly those featuring recorded data from females and specific sensor mounting positions, such as on the head, wrist, thigh, and chest, which are not ideal for active male firefighters This thesis focuses on analyzing data from male firefighters with waist-mounted sensors to evaluate the proposed algorithm against its reduced versions and fall detection algorithms from previous publications, including those by Paola Pierleoni et al Additionally, some fall and non-fall events involving males are excluded from this comparative analysis.

+ 811 sit-chair from vertical, to sitting with a certain acceleration onto a chair (hard surface)

+ 812 sit-sofa from vertical, to sitting with a certain acceleration onto a sofa (soft surface)

+ 814 sit-bed from vertical, to sitting with a certain acceleration onto a bed (soft surface)

+ 815 lying-bed, from vertical lying on the bed

+ 816 rising-bed, from lying to sitting

+ 903 front-knees, from vertical falling down on the knees

+ 909 back-sitting, from vertical falling on the floor, ending in sitting

The 917 rolling-out-bed system allows users to transition from lying down to walking on the floor seamlessly Additionally, the proposed algorithm will recognize self-recovery after a fall, categorizing these instances as non-fall events This feature aims to minimize unnecessary alert notifications sent externally.

+ 905 front-quick-recovery, from vertical falling on the floor and quick recovery + 914 right-recovery, from vertical falling on the floor with subsequent recovery

+ 916 left-recovery, from vertical falling on the floor with subsequent recovery

The pressure signal from the public dataset [75] is illustrated in Figure 3.19, showcasing both the raw pressure data and the estimated pressure data The data has been processed using a simple Kalman filter and a complementary filter, with a detailed view provided in the zoomed-in section.

Paola Pierleoni et al [70] introduced a fall detection algorithm that integrates pressure and acceleration data However, their algorithm demonstrated limitations when evaluated against public datasets [75], primarily due to noise in the pressure data (refer to Figure 3.19a) Consequently, the effectiveness of the fall detection algorithm utilizing barometric data was called into question.

76 by Paola Pierleoni is insufficiently stable Figure 3.19 is an example about a forward fall [75], and the pressure data does not clearly distinguish between before and after the fall has occurred

Applying Formula (3.21) as below to calculate the altitude change in Figure 3.19:

The analysis based on Formula (3.21) and public datasets reveals that the calculated altitude variation is significantly lower than the 0.52m threshold for fall detection proposed by Paola Pierleoni et al Specifically, the example calculated altitude variation is only 0.23m This discrepancy highlights the limitations of using barometric measurements for fall detection, as it heavily depends on the wearer's position and height, as illustrated in Figure 3.20.

Figure 3.20 The altitude variations of the fall events based on the different mounting positions [75]

The testing results presented in Table 3.8 demonstrate that the proposed Algorithm 1 outperformed others in specificity and accuracy It successfully detected all 762 non-fall events and 724 out of 755 fall events However, certain fall scenarios, such as the "920 syncope-wall" incident—where an individual slowly falls against a slipping wall from a standing position—were misclassified as normal activities in folders 107, 109, and 110.

Table 3.8 presents the testing performance of the proposed algorithms for fall detection and loss of physical performance detection, comparing them with our previous fall detection algorithm (publication number 3) and the algorithm developed by Paola Pierleoni et al., using public datasets.

The Algorithms Comparison Sen Spec Acc

Our previous fall detection algorithm

Paola Pierleoni et al algorithm [70] 36.95% 97.76% 67.5%

DEVELOPMENT OF A METHOD TO TRACK ON-

The Step Counting Method

Numerous studies have aimed to enhance step counting algorithms, including those referenced in [13, 23, 26, 37, 41, 44, 46, 74, 86, 89, 93, 98] Specifically, publications [6, 30, 37, 98] utilize built-in smartphone accelerometers and introduce features for effective step counting In [37], the authors established thresholds to identify maximum peaks and minimum valleys, employing the minimum distance feature to mitigate over-counting issues However, low-cost IMUs often capture noisy signals, rendering the method in Reference [37] insufficient for addressing both over-counting and false walking scenarios, where a user may appear to be moving while remaining still Reference [98] introduced three innovative features—periodicity, similarity, and continuity—to combat false walking; yet, this algorithm still encounters false negatives during intermittent motion, leading to the exclusion of genuine peaks.

The detection of 81 false peaks in noise signals highlights ongoing issues with under-counting and false peak detection in low-cost IMUs, where the actual number of steps may not be accurately represented Our proposed system addresses the challenge of data transfer during fire conditions, where traditional pre-installed transmitters may fail due to extreme heat Current methods utilizing smartphone accelerometers struggle with integrating additional sensors and wireless modules, as they cannot effectively transmit signals between the interior and exterior of a building without a pre-installed transmitter Additionally, the system's performance is inconsistent and heavily reliant on the type of accelerometer used.

The authors in Reference [98] suggest utilizing low-cost MEMS sensors in a navigation system, integrating peak detection and a band-pass filter They identify varying threshold values for different movement states, such as walking and running up and down stairs, using Support Vector Machine (SVM) for state classification, which demands extensive computational resources Additionally, this system faces significant cumulative errors during prolonged navigation.

Wristbands have emerged as a popular tool for step counting, offering convenience for users in health monitoring and indoor positioning systems Despite numerous studies aimed at improving the algorithms and devices used for this purpose, the performance remains inconsistent due to factors such as noise, arm swings, and unpredictable environments Evaluations of wristband step counting can be found in various publications, highlighting the ongoing challenges in achieving reliable results.

Dynamic thresholding for step counting in slow and intermittent walking conditions has been proposed, but it still faces challenges such as over-counting due to noise signals and misinterpretation of walking states.

A study evaluating the performance of seven popular wristband brands revealed inconsistencies in their effectiveness across different brands and activities The research proposed a dual-feature approach for step counting, utilizing Fast Fourier Transform (FFT) to identify walking states based on walking frequency This method calculates the number of steps by multiplying walking frequency with walking duration However, it is computationally intensive and struggles with accurately counting steps during fast walking, as well as eliminating false walking detections.

The study [23] employed Piecewise Aggregate Approximation (PAA) and Symbolic Aggregate Approximation (SAX) to estimate step counts during slow walking The primary focus of the research was to develop an algorithm that evaluates the accuracy and effectiveness of three commercial accelerometers in measuring steps during slow walking.

The step counting method introduced in this thesis is based on four key features: minimal peak distance, minimal peak prominence, dynamic thresholding, and vibration elimination These features are designed to adapt to the user's state, with thresholds increasing during slow walking and decreasing during fast walking, thereby preventing both over-counting and under-counting of steps.

The proposed method is divided into four phases as shown in Figure 4.1:

83 Figure 4.1 The flowchart depicting our step counting method

In the initial phase, data was captured and noise was reduced using a low-pass filter The subsequent phase involved classifying states based on features outlined in Reference [26] The third phase introduced four key features for effective peak detection: minimal peak distance, minimal peak prominence, dynamic thresholding, and vibration elimination Finally, the fourth phase established the criteria for verifying and counting the identified peaks.

If a peak satisfies all of the conditions which proposed in Phase 4, it is confirmed as a step The following is the details of each phase:

Phase 1: Data recording and signal processing

A 3-DOF accelerometer captures acceleration data along the Ax, Ay, and Az axes To minimize the impact of axis orientation, the Root Mean Square (RMS) of the acceleration across these axes is calculated.

In walking states, the step frequency typically remains below 3 Hz, necessitating the use of a low-pass filter with a cutoff frequency of 3 Hz to reduce noise in the recorded signal As illustrated in Figure 4.2, the raw signal exhibits multiple false peaks before filtering (Figure 4.2a) However, after applying the proposed low-pass filter, the signal becomes significantly smoother, effectively removing insignificant components (Figure 4.2b).

(b) Figure 4.2 The signal before and after applying the proposed low-pass filter: (a) Before applying the low-pass filter; and (b) After applying the low-pass filter

The mean acceleration magnitude is an effective metric for differentiating between slow, normal, and fast walking speeds, particularly in the context of complex and physically demanding activities like those performed by firefighters This simple yet powerful feature allows for clear distinctions in the recorded signals from various walking states, as illustrated in Figure 4.3.

Figure 4.3 illustrates the acceleration magnitude associated with three types of walking: fast, normal, and slow/intermittent The data indicates that fast walking exhibits a greater oscillation amplitude compared to normal walking, while normal walking has a higher amplitude than slow and intermittent walking Consequently, the established M1 and M2 thresholds effectively differentiate between fast, normal, and slow/intermittent walking styles.

The thesis introduces a step counting algorithm characterized by four key features: minimal peak distance, minimal peak prominence, dynamic thresholding, and vibration elimination However, the algorithm's accuracy is challenged by data recorded from a 3 DOF low-cost sensor, which suffers from excessive noise levels Additionally, volunteers may exhibit various movement states or engage in false walking activities without actual movement, further complicating the accuracy of the step counting method To address these challenges, the proposed method emphasizes the importance of periodicity, similarity, and continuity in step counting.

[33] can solve the problem of false walking, but it may be the cause of true peak elimination because of noises in the recorded data from 3-DOF low-cost sensor

This chapter introduces four key constraints—minimal peak distance, minimal peak prominence, dynamic thresholding, and vibration elimination—to address the false walking issue and improve the algorithm's reliability These constraints are integrated with periodicity and similarity features to enhance overall performance.

Step Length Estimation

Numerous studies have examined step length using data from 3-DOF accelerometers; however, these studies share common limitations regarding their reliability and accuracy.

[35] and related methods [37, 99] proposed the formula to estimate step length and distance measurement as follows:

Length = K × √𝐴 4 𝑚𝑎𝑥 − 𝐴 𝑚𝑖𝑛 (4.9) where Amax and Amin are maximum and minimum values of acceleration

The equation above is derived from a biomechanical-based method:

In the context of vertical displacement of the hip, the term 𝐵𝑜𝑢𝑛𝑐𝑒 refers to the measurement of this displacement, while 𝛼 represents the small angle formed between the horizontal line and the line connecting the lowest and highest Center of Gravity (CG) points in vertical measurement.

Figure 4.15 below relates the model with its inspiration of hip movement:

Figure 4.15 Model of the method based on hip movement by Weinberg in his

Since both 𝐵𝑜𝑢𝑛𝑐𝑒 and 𝛼 parameters are difficult to measure and prone to errors, Weinberg’s formula presents an approach inspired by the empirical relationship with

K as a constant for unit conversion in the formula above To improve the measurement, Gu-Min Jeong et al [37] made a change where K is not a constant

A linear regression polynomial function, which adapts effectively under specific conditions, utilizes vstep to represent the average velocities along the Ax, Ay, and Az axes in each step.

K = 0.68 – 0.37×vstep + 0.15×vstep 2 (4.11) v step = √v step_Ax 2 + v step_Ay 2 + v step_Az 2 (4.12) where vstep_Ax,vstep_Ay and vstep_Az were obtained by the double integration of 3-DoF acceleration

Different users exhibit varying step lengths due to factors such as biological conditions, walking patterns, and gender Restricting the coefficient to a single value can skew estimation outcomes To address this issue, we propose an adaptive approach that adjusts parameters during K training to optimize results This is achieved through regular state updates and leveraging the relationship between height and K training.

Table 4.9 Calculation of K adapted to change of height and state

Height Range State Calculation of K

The periodic state update can be performed using the previously described method in step detection to determine if the user is walking, running, or stationary within a 2-second interval Although there are no existing studies linking height to K, we collected data from three groups of participants categorized by height ranges, specifically targeting adults.

109 typical Asian figures With each group comes a set of parameters for K trained with walking and running by a simple linear regression model as we can see in Table 4.9

The process starts by inputting the height and confirming the state Once the state is validated, a step check is performed, and the appropriate K value is selected The maximum acceleration (Amax) and minimum acceleration (Amin) of the step are used to calculate the length based on the provided formula The total distance covered is the cumulative length of all steps executed during the movement period.

Figure 4.16 Adaptive Step Length Estimation Algorithm

Figure 4.16 presents in details the algorithm of our proposed adaptive step length estimation The process begins with the input of height, then records data along

After removing abnormal components from the recorded signal using a low-pass filter and a simple Kalman filter, the motion features will be utilized for state recognition Once the state is confirmed, a step check will be performed to select the appropriate value of K The step length is then calculated using the maximum acceleration (Amax) and minimum acceleration (Amin) of the step, following a specific formula.

The distance is sum of all length of steps taken during the moving period as the following formula:

D = ∑ 𝑛 𝑖=1 Step length(𝑖), (4.14) where, D is the traveled distance and n is the number of steps

Volunteers for the experimental testing were randomly selected from students at the University of Engineering and Technology – Vietnam National University Hanoi (UET-VNU) and the University of Fire Prevention and Fighting – Vietnam (UFPF), specifically those majoring in firefighting Detailed information about the volunteers is provided in Table 4.10.

Volunteers Gender Age Height Range Weight

Results from different test subjects can be found in the Table 4.11 below:

Table 4.11 Result of the proposed method on different subjects

The error between the reference length and the estimated length is calculated as the Equation (4.15)

The proposed adaptive step length method demonstrates impressive accuracy, achieving 97.11% Its correlation with height and state recognition enhances detection efficiency and reliability across varying conditions Testing was conducted with a reference distance of 81m, comparing Weinberg’s method, Gu-Min Jeong’s method, and the proposed approach, as illustrated in Figures 4.17 and 4.18.

Figure 4.17 Comparison among our proposed method, Weinberg and Gu-Min

Jeong’s methods on walking state

The results illustrated in Figure 4.17 indicate that the Gu-Min Jeong method outperforms the original Weinberg method, yet it still falls short compared to our proposed distance estimation method Notably, the mean represented by the red line in the box of our method is the closest to the reference length, highlighting its superior accuracy.

Figure 4.18 Comparison among our proposed method, Weinberg and Gu-Min

Jeong methods on running state

Weinberg’s and Gu-Min Jeong’s methods demonstrate limited adaptability in running state scenarios, as Weinberg’s approach relies on a constant value, while Jeong’s method uses a polynomial function with coefficients trained without accounting for state variations In contrast, our proposed method maintains a mean that closely aligns with the reference, showcasing its effectiveness.

Turning Time and Direction Estimation

Time estimation is crucial for indoor positioning systems, enabling the precise and real-time tracking of on-duty firefighters' locations The magnetic sensor within the Inertial Measurement Unit (IMU) captures the Earth's magnetic field components, Mx and My, to enhance positioning accuracy.

The hardware device was attached to the waist of volunteers, aligning the My-axis with Earth's gravity, as illustrated in Figure 4.10b This setup resulted in minimal changes in the recorded magnetic values along the My-axis, while significant variations were observed in the Mx and Mz axes By analyzing the magnetic data from Mx and Mz, researchers were able to estimate the turning times of users during movement.

The magnetometer operates at a sampling frequency of 50Hz, allowing the signal to be segmented into an array with m = FS/2 samples By calculating the maximum and minimum values of these samples, we can estimate signal changes based on their differences Significant changes in the signal indicate a shift in the user's direction.

𝑀𝑎𝑥𝑀𝑖𝑛_𝑋(𝑖) = 𝑀𝑎𝑥_𝑋(𝑖) − 𝑀𝑖𝑛_𝑋(𝑖); (4.17) 𝑀𝑎𝑥𝑀𝑖𝑛_𝑍(𝑖) = 𝑀𝑎𝑥_𝑍(𝑖) – 𝑀𝑖𝑛_𝑍(𝑖); (4.18) where 𝑀𝑎𝑥_𝑋(𝑖) and 𝑀𝑖𝑛_𝑋(𝑖) are the maximum and minimum values in an array of Mx axis; 𝑀𝑎𝑥_𝑍(𝑖) and 𝑀𝑖𝑛_𝑍(𝑖) are those of Mz axis

Figure 4.19 The Max-Min values of Mx and Mz axes

When the difference between the maximum and minimum values is minimal, it indicates that the user is moving in a straight line As illustrated in Figure 4.19, significant changes in the Mx axis values result in only slight variations in the Mz values, and the reverse is also true To effectively demonstrate the changes in magnetic signals during rotation, we recommend calculating the mean of MaxMin_X(i) and MaxMin_Z(i) to identify significant shifts in the magnetic signal, utilizing the following equation:

(4.19) where, M(i) is the average value of 𝑀𝑎𝑥𝑀𝑖𝑛_𝑥(𝑖) and 𝑀𝑎𝑥𝑀𝑖𝑛_𝑧(𝑖)

Figure 4.20 illustrates the combined signals in the Mx and Mz axes, highlighting a significant change in the magnetic signal that effectively addresses the limitations of using a single or independent axis This combination enables the detection of rotational changes during volunteer movements, utilizing a proposed threshold defined as thresh = max(M) – min(M).

Figure 4.20 The average values of 𝑀𝑎𝑥𝑀𝑖𝑛_𝑋(𝑖) and 𝑀𝑎𝑥𝑀𝑖𝑛_𝑍(𝑖)

In this section, a normalization algorithm is introduced to effectively present the turning time The average value is compared to a calculated threshold using Equation (4.20); when the average exceeds this threshold, it is assigned a value of 1, indicating a rotating state, while a value of 0 signifies non-rotation This normalization process results in a clear representation of the turning time, as illustrated in Figures 4.21 and 4.22.

The proposed indoor positioning system for on-duty firefighters integrates a turning estimation algorithm, which assesses both turning time and direction, along with step counting and step length algorithms, and utilizes map information to accurately determine their indoor location.

The turning directions depend on the change of signal in Mx and Mz The thresholds 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑_𝑥 and 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑_𝑧 are used to detect the change of signal in

Mx and Mz axes If the signal value is greater or smaller than the proposed threshold, it will be normalized to 1 or 0, respectively (see Figure 4.23)

The turning direction is estimated based on the standardization signal in both 𝑀𝑥_𝑛𝑜𝑟𝑚𝑎𝑙 and 𝑀𝑧_𝑛𝑜𝑟𝑚𝑎𝑙 When the standardization signal equals 1, it assigns high state (H) and low state (L) for other

Figure 4.24 The proposed algorithm to detect magnetic variability

Furthermore, when the standardization signal increases from 0 to 1, it is called signal increase (↑) and when it decreases from 1 to 0, it is called signal decrease (↓)

Signal variations in both 𝑀𝑥_𝑛𝑜𝑟𝑚𝑎𝑙 and 𝑀𝑧_𝑛𝑜𝑟𝑚𝑎𝑙 are utilized to determine turning directions The experimental results, detailed in Table 4.12, provide insights into the estimation of these turning directions.

Table 4.12 The turning directions estimation based on the fusion of 𝑀𝑥_𝑛𝑜𝑟𝑚𝑎𝑙 and 𝑀𝑧_𝑛𝑜𝑟𝑚𝑎𝑙

Signal Turn right Turn left

Figure 4.25.The turning directions estimation

Applying the result in Table 4.12 to detect the turning directions in Figure 4.23, we can be explained as the following:

+ The first turning (at the sample 275 in Figure 4.25): The value of “Normalized Mx” changed from 0 to 1 which corresponding to the state of increase (↑) while

“Normalized Mz” still remains at high state (H) Based on these informations combination and statistical results in Table 4.12, we can estimate the turning direction in this scenario is “Right turning”

+ The second turning (at the sample 450 in Figure 4.25): The value of

The "Normalized Mz" value decreased from 1 to 0, indicating a downward trend (↓), while "Normalized Mx" maintained a high state (H) According to the data presented in Table 4.12, we can infer that the direction of turning in the second scenario is "Right turning."

Similarly, it is simple to detect the direction of turning in the third time is “Right turning” (at the sample 675 in Figure 4.25)

Our proposed algorithm accurately estimated three instances of "Right turning" in this scenario, aligning perfectly with the actual turning directions observed during experimental testing.

The turning directions estimated will be saved in 𝑡𝑢𝑟𝑛 [] array with value 1 is related to right turning and 0 is left turning

Figure 4.26 below is our proposed algorithm to estimate the turning directions of the users:

120 Figure 4.26 The proposed algorithm for turning directions estimation

Vertical Position Estimation

To estimate the vertical position of volunteers within a building, we recommend utilizing signals from a barometer, which measures environmental pressure By analyzing the changes in pressure with altitude, we can calculate the altitude based on the measured pressure values using a specific equation.

) (4.21) where, 𝑝0 is the pressure at sea level (p0= 1013,25 hPa) and 𝑝 is the measured pressure by the barometer

The vertical positioning of volunteers is determined using estimated altitude and map data; however, environmental factors like temperature and moisture can compromise data accuracy To address this issue, the Kalman filter has been suggested to enhance the calibration of signals obtained from barometric readings In our prior publication, we demonstrated the effectiveness of utilizing pressure signals alongside map information to achieve highly accurate vertical position estimates, which is crucial for indoor positioning applications.

In the experiment of testing, the volunteers performed two scenarios, including up or down movements using an elevator

Scenario 1: Figure 4.27 presents result of an elevator moving down from the 7 th floor to the 2 nd one Applying the Equation (4.21) for the measured pressure data, we calculated the altitude change from around 21m to 5m

Figure 4.27 Altitude variation when the elevator moved down from floor 7 to floor

Figure 4.28 Altitude variation when elevator moved up from floor 3 to floor 4

Similarly, we can estimate the vertical position in scenario 2 when the elevator moves up from floor 3 to floor 4 with the altitude changing from around 8m to 11.5m as shown in Figure 4.28

The estimated altitude variation will be integrated with mapping data that encompasses building dimensions—length, width, and height—along with the height of each floor, enabling precise determination of firefighters' vertical positioning within the structure.

For two scenarios of testing shown in Figure 4.27 and Figure 4.28, we conducted at the University of Fire Prevention and Fighting, Hanoi, Vietnam The building

123 included 7 stories with the height of 4 m each When applying the Formulas (3.15) to (3.17), we can predict the state of moving as “elevator moving down” for the scenario in Figure 4.27

The altitude can be estimated using the formula \( \text{Height} = \text{Height measured} + 1 \times \text{Height of floor} \) (4.22) In this context, "Height measured" refers to the altitude determined from barometric pressure, while "Height position of wearing" indicates the vertical distance from the ground floor to the trouser pockets of firefighters Additionally, "Height floor" represents the height of each individual floor.

Applying the Equation (4.22), we can predict the real-time vertical position of firefighters The details of the predicted results of the above two scenarios are shown in Table 4.13:

Table 4.13 The estimated floor number

Scenarios Predicted References Floor estimated results

Changed from floor 7 th to floor 2 nd

Elevator moving down from floor 7 th to floor 2 nd

Changed from floor 3 rd to floor 4 th

Elevator moving up from floor 3 rd to floor

The achieved results in Table 4.13 can prove that the proposed method “correctly estimated” the floor number-based the vertical position

In addition to utilizing accelerometer and magnetometer data for direction estimation, we have also developed a method for vertical position estimation using a barometer, as detailed in our previous publication (publication number 6) This vertical estimation algorithm has been integrated with a turning algorithm to enhance step tracking accuracy.

124 counting and step length estimation algorithms to detect the indoor position of the user.

INDOOR FIREFIGHTER POSITIONING AND