Figure 1. Approach for Drowsiness Detection and Driver Warning
A large number of serious or fatal accidents are occurring due to excessive or inappropriate speed of the vehicle. Distraction and drowsiness of the driver has been important factors for a large number of major accidents. They reduce the decision making capability and perception level of the driver which negatively affect the ability of the driver to control the vehicle. Monitoring the driver’s behavior is one of the ways to prevent the fatal accidents and it is necessary to alert the driver when they are drowsy or in a distracted state. With the new developed systems, there is a possibility of self controlling the vehicle when the driver was drunk or reckless or fatigue so that major accidents may be reduced. There is also a possibility of tracking the location of accident, occurrence, through Wireless Access Technology (GSM, GPS) so that proper measures are taken at correct time.
In the developed world, road fatalities are of major concern. At present time, private vehicles are widely used daily by huge numbers of people. Regarding the increased use of private transport, the biggest problem is the rising number of fatalities. The consequence of accidents on the roads has been recognized as a serious problem. The main factors responsible for most of the road accidents are due to drivers affected by fatigue, being drunk or reckless driving [1].
For the improvement in Intelligent Transport Systems (ITS), Wireless Communications and Mobile Computing focused on road safety applications. Vehicle Adhoc Networks (VANET) safety applications are considered to be a vital step towards enhancing road safety and improving traffic efficiency. The most important ITS applications in the world have occurred in Europe, Japan and South America.
There is a dynamic behavior model for real time inferring four types of driving behavior (normal, drunk, reckless and fatigue). This dynamic behavior model can capture the static and temporal aspects related to the driver’s behavior, and thus it leads to accurate and robust detection of the behavior.
The four categories of driving behavior are as follows:
The driver behavior is said to be normal when he /she concentrates on the driving task. This can be characterized by controlling the speed of the vehicle, avoiding sudden acceleration, driving without alcohol intoxication.
The behavior is said to be drunk, when the driver is intoxicated by alcohol and is characterized by a set of observable actions like sudden acceleration and driving without controlling the speed.
It stated that a driver driving after a period of 17 hours with no sleep behaves exactly as a driver who has 0.05% intoxication of alcohol. Based on this argument, fatigue driving was defined as driving that exhibits the same characteristics as drunk driving, but there is no alcohol intoxication in the blood of the driver.
It is the behavior of the driver who drives at high speed, with high degree of acceleration and leaves other traffic participants at risk.
There is no alcohol intoxication and the driver's eyes are opened, but the following behaviors such as driving with sudden acceleration, not maintaining the proper lane position and not controlling the vehicle's speed are exhibited.
Several methods were proposed by different researchers to monitor the behavior of driver. Some have attempted to measure the driver's state or the vehicle's behavior to detect fatigued and drunk drivers. Meanwhile, other researchers have tried to monitor the driver, vehicle and the environment in order to detect the state of the driver. The main researches are summarized below.
A custom made webcam is placed on the dashboard of the driver seat to capture the image of the driver [2]. The webcam is connected via USB port, with resolution up to 800×600 VGA 30 frames per second. Several infrared LEDs are placed at the right side and left side near to the camera to capture image of the driver more clearly during the night time[7].
In order to prevent interpretation of incorrect information, the data packet which is received is first validated by its length, channel number, and identification number before the analysis take place. The video sensor is directly connected to the web camera using USB port, whereas the biomedical sensor transmits the data packets wirelessly to the smart phone through the transceiver connected by RS232 serial port.
To detect the drowsiness of the driver by visual features is quite easy. Facial movements usually such as blinking of eyes, yawning frequently and nodding or swinging head are key elements among visual features which are used for detecting drowsiness of the driver [3]. Most of the research work is focused on eye behaviors, in particular, to monitor the alertness of the driver. A most reliable and valid method for monitoring a driver's alertness level is PERCLOS (Percent Eye Closure).
PERCLOS is the percentage of total time that the driver's eyelid is closed 80% (or more) over the pupil. When PERCLOS exceeds a given threshold, the warning system generates a drowsiness warning immediately [6], so that the driver may get alert. However, one limitation of PERCLOS is that sometimes a driver who is trying to stay awake is able to fall asleep with his eyes open.
To calculate PERCLOS, we have to extract the region of the eye including the area of pupil. However, there are some limitations while extracting those visual features. One of them is the problem of proper lighting. Drowsiness should be monitored under real conditions, i.e., throughout daytime and night, and regardless of whether the driver is wearing glasses or sunglasses.
Usually, web camera or a simple CCD is used during the daytimes, and an IR camera is used at night. In order to detect the eye of the driver who is wearing sunglasses, it is necessary to find a proper wavelength of Near IR (NIR) illumination. One possible wavelength is 850nm.
In a real automotive environment, there is a possibility of generating sunlight on the outer surface of the eyeglasses. To avoid the reflection effect, a NIR illuminator with a narrow band pass filter is used that restricts the incoming wavelength of light to 850nm. This is because the LED illuminator with a high power is more powerful than the sunlight.
Another sign of driver drowsiness is yawning. It is detected by measuring both the rate and the certain amount of changes in the mouth contourness of the driver. Head pose and head motion estimation movements such as nodding are also important to monitor driver alertness. In addition, driver's facial wrinkles appearing on the mouth, brows are excellent physical signs in order to detect drowsiness of the driver. Monitoring driver drowsiness by visual features provides more benefits when compared to non-visual features.
The George Washington University Virginia Campus Driving Simulator Laboratory, which is a part of the University's Center for Intelligent Systems Research, conducts behavior of driving and vehicle control experiments for making the vehicle to travel safer.
Through their Drowsy Driver Detection System, researchers track normal and drowsy steering behaviors by using artificial neural networks and later they classify normal and drowsy steering behaviors that mimic brain function.
In 1998 Azim Eskandarian, Professor of Engineering and Applied Science, started this project under a contract for the Turner-Fairbank Highway Research Center of the Federal Highway Administration (FHWA).
In the initial phase, data which is gathered on sleepdeprived drivers by the FHWA was analyzed by using a driving simulator. They measured driver and vehicle variables like steering activity, speed, braking, lane control and acceleration. They also analyzed the eye movement of driver in subsequent phases.
The data gathered in the simulator has been processed by the artificial neural networks. They created a framework to detect or predict whether the driver is drowsy or not. In the second and third phase, Eskandarian, his team and students gathered another data set in the Passenger Car Driving Simulator Lab [4]. They tried to study people driving in the morning and in the evening. The same test was done by the Federal Motor Career Safety Administration of the Department of Transportation with truck drivers.
With these results of the tests, Eskandarian and his team were able to further validate the detection system. From these three sets of data, the system achieved a percent of 86 to 91 success rates in the detection of drowsy driving patterns. With the eye monitoring in the second and third phases, this detection accuracy increased by about 5 percent.
Figure 1. Approach for Drowsiness Detection and Driver Warning
Figure 1 shows how the Artificial Neural Networks(ANN) works. As shown in the figure, the physiological signals of the driver(EEG, heartbeat ,blood pressure), the physical reactions of the driver ( eye monitoring ,head posture) and vehicle behavior(steering angle, vehicle speed etc.,) are measured simultaneously .The outputs of the above cases are given as inputs to the Artificial Neural Network(ANN).This ANN weighted all the inputs and produces the final output. The resultant(output) is fed to the driver and it warns him if any one of the above case was violated.
Brain activity and heart rate are the nonvisual features that are useful in predicting the drowsiness of driver [5]. The driver drowsiness prediction based on these physiological signals allows warning a drowsy driver who was in drowsy condition.
Electrocardiogram (ECG), Electro-Encephalogram (EEG), Electromyogram (EMG) are used as physiological signals. Heart Rate (HR) can be extracted from the ECG signal. Since the heart rate varies significantly between alertness and drowsiness states, it can be used to detect drowsiness.
The beat-to-beat change in the heart rate which is measured by Heart Rate Variability (HRV) is also used to detect the driver's drowsiness[9]. The low frequency to high frequency ratio beats in the ECG signals progressively decreases as the driver goes from an alert to a drowsy state.
When compared to visible features, the reliability and accuracy in detecting driver's drowsiness based on physiological signals is high. But an important limitation of physiological signal measurement is its intrusive nature. One solution to solve this limitation is by using wireless technologies such as Bluetooth and Zigbee.
Another important factor causing impairment of driver attention is distraction [8]. Distraction of driver is defined as the driver not paying sufficient attention to the road due to the presence of obstacles. The trend in increasing the use of in-vehicle information systems also leads to driver distraction. Distraction is one of the main causes for most of the accidents.
The monitoring of the driver’s head pose and gaze direction is an important step in detecting driver distraction and inattention. To determine driver distraction or whether the driver is looking straight ahead, a forward warning system is used.
To recognize driver awareness and to estimate continuous head orientation and gaze direction, Kaminski et al. proposed a system. Distractions can be categorized as cognitive, i.e., “mind-off-road” and visual, i.e., “eye-offroad”. Liang [8] proposes a method to detect the interactions of cognitive & visual distractions.
It is necessary to extract head pose or gaze information to detect driver distraction. Estimation of Head pose provides the current focus of attention and driver's field of view. It is intrinsically linked to visual gaze direction. Head pose is used to estimate the gaze direction when the eyes are not visible. The combinations of both eye direction and head pose provide gaze information of the person.
It is a programme that works on a mobile phone and it contains orientation sensors and accelerometer. They are placed in the vehicle to detect drunken nature of the driver in real time [10]. The programme compares the current accelerations with typical drunk driving patterns When the programme indicates that the driver is influenced by alcohol, it generates the warning messages in order to alert the driver and a message is sent to inform police so that proper measures are taken immediately.
Another visual cue that can potentially capture one's level of fatigue is his/her facial expression [11]. In general, under different levels of vigilance, people tend to exhibit different facial expressions. The facial features around mouth and eyes represent the most important spatial patterns composing the facial expression [12]. The facial expression of a person in fatigue can usually be characterized by lagging facial muscles, less expression, and frequent yawning [13]. For the fatigue detection, the facial features around eyes and mouth include enough information to capture the limited expressions. Figure 2 shows the results of a typical sequence of a person in fatigue.
Figure 2. Tracked facial features
Driving behavior features include movement of steering wheel, lane keeping, movement of pedal and braking, etc. For detecting the level of driver’s drowsiness, the two most commonly used driving behavior measures are the movement of steering wheel and the standard deviation in lateral position. Movement of Steering Wheel (MOSW) is measured by using steering angle sensor mounted on the steering column [14]. When the driver is drowsy, the number of micro-corrections to the steering wheel is reduced.
The driver's drowsiness state is determined from small MOSWs between 0.5° and 5°. MOSWs are being adopted by car companies such as Renault and Nissan. They work in very limited situations due to their reliability only in particular environments.
Another sleepiness sensitive continuous performance measure is Standard Deviation of Lateral Position (SDLP).SDLP is dependent on external factors such as lighting, road markings and conditions of the climate.
Figure 3. Proposed Embedded System Design
This proposed system consists of, MEMS-Micro Electro Mechanical System, GSM-Global Subscriber Module, GPS-Global position System, alcohol detector, eye blink detector, gas leak detector and fire detector. Most of the researchers tried to monitor the behavior of the vehicle or the driver in isolation, while others have focused on monitoring a combination of the vehicle, the driver and the environment in order to detect the status of the driver so as to prevent road accidents.
There is still no comprehensive system which can effectively monitor a driver's behavior, the state of the vehicle and changes of the environment. Figure 3 shows the result of the Proposed Embedded System Design.
Alerting other vehicles on the road by sending warning messages in time to the relevant vehicles in the vicinity, includes some corrective actions to avoid accidents.
Different sensors (alcohol, gas, eye blink) are used to sense the alcoholic nature of driver, any leakage of gas, the eye-lid movement of the driver. The output of different sensors is given to the microcontroller. The output of microcontroller is displayed on LCD. If any gas leakage occurs or when the driver is intoxicated with alcohol or if the driver blinks his eyes, immediately the motor of the vehicle will stop.
If the accident occurs by vibration of the rotating machinery of vehicle, immediately the exact location of the accident is identified by an authority through Global Positioning System(GPS)and message is sent via Global System for Mobile communications(GSM).
Now-a-days accidents occur in all the places. Major accidents occur in highways because of the high speed of vehicle. The main objective of Global Positioning System (GPS) is to detect the accident if occurred and informs the respective authority.
The GPS tracker will track the accident from at least three orbiting GPS satellites. The GPS tracker will transmit that related information to a server (computer) , and it will display it on a web-based portal which is online with mapping software.
If any accident occurs in highway or at any other place, the information system of accident will get activated and message will be transmitted to the respective authority through Global System for Mobile (GSM). In case of any accident, the vibration of vibration sensor increases beyond the limit and information is sent to GSM module. The GSM can send message to the respective authority.
Detecting and monitoring the driver the behavior is vital to ensure road safety by alerting the driver and other vehicles on the road mainly in cases of abnormal driving behaviors. Behavior of driver is affected by many factors that are related to the vehicle, the driver and the environment .It is important to capture the static and the dynamic aspects of behavior and take into account, the contextual information that relates to behavior of the driver. Nowadays, different driving support systems assist the driver in reaching his destination safely. In many cases, these systems capture sensory data and present them to the driver. These systems actually process the data captured by the sensors. They act respectively and quantitatively relative to the processed data. The growth of sensor technology and network based information technology has expanded the reach of wire-less sensor networks into numerous areas such as remote control, health care, monitoring of wildlife habitat, detection of military explosive, intelligent home monitoring and environment observation and forecasting system.