Figure 1. Fundamental Structure of a Convolutional Neural Network
This paper outlines a state-of-the-art method for smoke and fire detection utilizing Convolutional Neural Networks (CNNs). The current smoke detectors installed in buildings pose a challenge for effective fire detection. The inefficiency of traditional methods in terms of speed and cost led to the exploration of using Artificial Intelligence (AI) to identify and alert from Closed Circuit Television (CCTV) footage. In this paper, an analytical overview of AI is conducted by using a selfcreated dataset of video frames containing flames and smoke. The data undergoes pre-processing before being used to train a CNN-based machine learning model. The goal of this review study is to understand the available literature in the field and propose a highly accurate, cost-effective, and simple system for fire detection in various scenarios.
In today's hectic environment, fire and smoke can present a serious threat. As a result of the rise in fire incidents, all public buildings and transportation vehicles are outfitted with fire protection systems. A lot of businesses also do simulated fire drills once a month to safeguard their workers from potential fire risks.
One of the key elements in maintaining the equilibrium of an ecosystem is forest cover. Forest fires may be quite destructive (Breejen et al., 1998). However, forest fires are typically discovered after they have spread widely. As a result, the environmental harm is worse than anticipated.
Environmental damage is caused by large-scale carbon dioxide (CO2) emissions from forest fires. Additionally, it will result in the extinction of all rare species worldwide. It also has an impact on the weather, which can result in severe issues like earthquakes, torrential rain, and flooding. A forest is an expansive region covered in trees, plenty of fallen leaves, etc. Once the fire gets going, these substances aid it (Fernandes et al., 2004). There are a variety of causes for fires, including summer heat, smoking, and fireworks. Once it starts, it will burn until it is entirely put out. If flames are discovered as soon as feasible, wildfire damage and fire-fighting expenses can be decreased. Therefore, in this case, fire detection is crucial.
According to a recent study (Muhammad et al., 2018), there are three types of automatic fire detection: airborne, ground, and ground detection. Black-and-white video cameras are used by ground systems to identify smoke from fires and compare it to ambient smoke. The great spatial resolution of this technology is a significant benefit for simplifying smoke detection. These technologies still struggle to detect fire in its earliest phases. Therefore, it is crucial to implement modern and cutting-edge fire detection systems as soon as possible.
The study has been done on a CNN convolutional neural network to analyze live video from a fire monitoring system in order to identify fires. This study contends that YOLO Version 2 (YOLOv2) CNN is among the top options for detecting fire and smoke both indoors and outdoors. An object identification model for deep learning is called You Only Look Once (YOLO). The second version, dubbed YOLOv2, has been improved to solve YOLO's drawbacks, which include low recognition rates compared to other region-oriented algorithms and inaccurate location and labelling of Region of Interest (ROI) in photos. They used a 128x128 3-pixel input picture. In order to map the characteristics to the input picture, a convolutional layer was utilized. The YOLOv2 object detection sub-network receives input from the retrieved features. The object location network has been made more reliable by adding a YOLOv2 translation layer (Saponara et al., 2021).
The research presented a vision-based fire detection system that could be fitted to an Unmanned Aerial Vehicle (UAV). The report highly advises CNNs to recognize smoke and fire from video frames taken as still photos.Datasets were gathered from a variety of online sources. The image was downsized to a 20x320 standard size. This article's main goal is to identify burn scars in pictures. The authors suggested two techniques for creating models using algorithms. The initial step was to completely reapply the burn classification. When the picture frames contained fire, the second step involved training a whole image classifier and using a fine-tuned patch classifier. The accuracy results of (Support Vector Machine) SVM-Pool 5 and CNN-Pool 5 are 95.6% and 97.3%, respectively, with a detection rate of 8.8%, showing that the CNN-Pool 5 network performs better than the SVM Pool 5 classifier (Zhang et al., 2016).
The author focused on how forest fires may do irreparable harm to the ecology. The Amazon jungle recently saw a fire that raged for more than 15 days. This had a severe impact on diversity and international affairs and caused enormous losses. Forest fire detection is aided by wireless sensor networks. As soon as unusual activities take place, alerts may be sent out. Due to erroneous positives, these networks may cause false alarms. Machine learning algorithms can be applied in these situations to stop such incidents. Fire detection systems based on satellites were formerly employed. However it take pictures of the Earth's surface every two days, so it might not be able to identify a turn. As a result, it cannot be deemed an efficient method. Image quality might also be impacted by the weather. The use of watchtowers was another technique for spotting fire and smoke. It was managed manually by looking out from the tower over the entire woodland region and noting if there was a fire there. The other makes use of a digital camera and an optical sensor which is not very efficient since hills and big trees might obscure the view (Pragati et al., 2020).
The quantity of smoke can be used to identify fire. When smoke from a fire reaches a certain level, it is deemed to be a fire situation, and this level is measured by a smoke sensor and compared to a threshold value. Using image processing, fires can be detected earlier. CCTV cameras can be equipped anywhere and footage from these cameras can be analyzed to track fires. When changes happen, fires may be swiftly located and put out. When the alarm went off, this system had a water extinguisher to put out the fire. A Raspberry Pi computer system that is coupled with a CCTV camera to capture footage of a particular area gives you access to ongoing video footage of a certain region. As soon as a fire is found, the video image is evaluated frame by frame, and the alarm is activated. Additionally, the alarm stops sounding once the fire has been totally put out. The application is operated using virtual network computing, and visual data is sent from the Raspberry Pi to the computer screen. The system consists of components for software, network, detection, alert, and suppression (Arul et al., 2021).
The hue of the camera's image is crucial for detecting fires. Given that fires might be challenging to spot, you might not be able to view the complete forest picture, depending on its size. Therefore, it is simpler to avoid blindness and identify precise degrees of fire when employing CNN technology. Utilize the support vector method while classifying images and with this method, photos are divided up based on the color of the flame and sent to the CNN network. More characteristics will be detected in order to predict the likelihood of a fire. By examining the hue of the flames in the photograph, fires can be located. The number of pixels plotted in the picture in accordance with the color of the fire may be used to locate the fire and assess its intensity. This makes locating and putting out fires simpler. Many different types of data are used to train and test the system. To find the fire, segment the image using an algorithm. This approach ought to be more dependable and efficient in locating fires. Precision ought to be significantly higher than with previous approaches (Wang et al., 2019).
In this study, a novel ensemble learning-based fire detection approach was put forth. 10581 pictures from a variety of public sources, including Bow Fire, FD-Dataset, Forestry Images, and Vis Fire, were used to create the dataset. To obtain more accuracy than a single object detector, the dataset is pre-processed and supplied to not one, but two different object detectors (YOLOv5 and Efficient Det), which are then combined in a parallel mode of operation. It makes use of the built-in item detection but ignores the big picture. To address this issue, a new classifier is introduced. Efficient Net examines the image as a whole to extract as much information as possible. A decision-making algorithm that considers the viewpoints of the three distinct object detectors produces the results. This increases the model's effectiveness and lowers the frequency of false positives. According to the report, average accuracy, average recall, false positives, and latency are traded off favorably (Kim & Lee, 2019).
The authors of this study suggested a system that resembles the human fire alarm system. It makes us of the region-based method Faster R-CNN to find suspicious places of interest. The LSTM long short-term memory receives the characteristics retrieved from the bounding box once a region of interest (ROI) has been marked, and it then classifies whether or not fire happens quickly. Faster R-CNN utilizes CNN's capabilities and adds a region proposal network to map the input image's features. The Region of Interest (ROI) pooling procedure is used to extract features, which are then categorized based on the location class value of the object. (Xu et al., 2021).
The study proposed a method for real-time wildfire detection using wireless sensor networks. Compared to previous approaches for detecting forest fires, this method was more accurate in detecting and forecasting fires. Sensor networks first capture data on temperature, humidity, smoke, and wind speed because these elements have an impact on forest fires. Clusters of sensor nodes are positioned deep inside the forest. Sensor nodes use Global Positioning System (GPS) to track their location. This is so that cluster heads may receive position information together with other data, such as temperature readings. The cluster header then calculates weather indicators using Convolutional Neural Network (CNN) techniques and delivers this information to the management node. Wind sensor nodes that were manually installed in the forest are used to calculate wind speed. When abnormal occurrences like high temperatures or smoke are observed, management nodes alert users. Manager nodes additionally offer data on the frequency of forest fires based on meteorological indications from different clusters. Thus, if a forest fire breaks out, the user may quickly determine its precise position. Early detection also prevented forest fires (Yu et al., 2005; Alkhatib, 2014).
Heat and smoke sensors make up the fire detection system that is currently in place. Traditional smoke sensor alarms and heat sensors have the significant flaw that only one module is capable of monitoring all fire-prone regions (Arrue et al., 2000). Being watchful at all times is the best approach to preventing fires. Even a corner-tocorner placement is insufficient to guarantee effective operation. The price rises when more smoke sensors are needed, and vice versa. Within seconds of a fire starting, the suggested system is capable of generating reliable, highly accurate alerts. By using a single piece of software to run a network of surveillance equipment, expenses may be reduced. Data scientists, including machinelearning scientists, do research in this field. The major difficulty is reducing false alarms in fire detection and quickly sending out alarms.
The existing fire and smoke detection systems can be broadly classified into two categories: traditional and deep learning-based systems. Traditional systems typically involve the use of heat and smoke sensors that activate an alarm in the presence of fire or smoke. These systems are limited in their ability to accurately detect fires, especially in complex and cluttered environments.
In recent years, deep learning-based systems have emerged as a promising solution to the limitations of traditional systems. These systems use Convolutional Neural Networks (CNNs) to analyze images and videos from CCTV footage to detect fires and smoke. The key advantage of these systems is their ability to learn and recognize patterns in data, allowing them to detect fires and smoke even in complex environments. Some of the existing deep learning-based fire and smoke detection systems use transfer learning, where a pre-trained network is fine-tuned for the specific task of fire and smoke detection. Other systems use end-to-end training, where the network is trained from scratch on a large dataset of fire and smoke images and videos (cheng et al., 2011).
The existing fire and smoke detection systems range from traditional methods relying on sensors to deep learningbased methods that analyze images and videos to detect fires and smoke. The deep learning-based systems offer improved accuracy and versatility compared to traditional methods, making them a promising solution for fire and smoke detection in various environments.
The proposed framework makes use of convolutional neural networks. A Convolutional Neural Network (CNN) takes inputs, pre-processes them, and aggregates them using proposal regions. A CNN region-based object detection algorithm then uses convolutional layers to classify these suggestions by Region of Interest (ROI) for fire and non-fire.
Using supervised learning, convolutional neural networks, a particular kind of artificial neural network, can evaluate data by simulating the activity of the human brain. CNN, which stands for "fully connected network," is short for "modified multilayer perceptron”. Modified Multilayer Perceptron (MLP) is a type of artificial neural network that can be used for image and video analysis. In the context of fire and smoke detection, modified MLP networks have been used as an alternative to CNNs for analyzing images and videos to detect the presence of fire and smoke. It has an input layer, an output layer, and several hidden layers in order to do it. Convolutional neural networks get their name from the convolutions in these hidden layers. It provides the potential for international object recognition (Angayarkkani & Radhakrishnan, 2009).
A modified MLP network consists of multiple layers of artificial neurons that are connected in a feed-forward manner. Each layer processes the input from the previous layer and passes the output to the next layer. The last layer of the network produces the final output, which, in the case of fire and smoke detection, is a binary classification indicating the presence or absence of fire and smoke.
Modified MLP networks have been used in fire and smoke detection by pre-processing the input image or video to extract features that are relevant for fire and smoke detection. These features are then passed as input to the MLP network, which uses them to classify the image or video as containing fire and smoke or not.
The network is completely coupled; therefore, over-fitting is conceivable. Convolutional Neural Networks (CNNs) use hierarchical patterns in the input to filter the data by complexity, from basic to complicated patterns divided into layers, to prevent this problem. The inputs are defined as a tensor of inputs, input channels, height, width, and number of inputs. The image turns into an abstract shape, which the layer then turns into a feature map. This layer-bylayer repetition mimics the activity of brain neurons. All outputs are filtered and aggregated into a single output at the output layer because this is a fully linked network. The size of the feature map is directly proportional to the number of filters.
Convolutional layers make up the architecture of a convolutional neural network. The ability of CNNs to create areas of interest within the original picture using image modification filters known as "convolution kernels" sets them apart from other object recognition techniques. The number of kernels and the number of feature maps produced are equal. The functional map's pixel colors correspond to activation sites. Points in the original picture with significant activation points are represented by white pixels in the feature map. Black pixels signify potent negative activation sites, whereas grey pixels signify weak negative activation points. The convolution kernel converts the original frame's reddish-orange pixels in the flame region to white. In a convolutional neural network, just a little fraction of the layer before is used as input for each neuron. By applying a function to the output of the layer before it, each neuron in the network generates an output. The weights of the input values are used to calculate these functions. Convolutional neural networks are distinct in that they function the same at all levels. The network's feature extractors are known as Alex Net Deep Convolutional Neural Networks (CNNs), a straight forward CNN application that enables straightforward object recognition in pictures. A CNN's straight-forward construction is shown in Figure 1.
Figure 1. Fundamental Structure of a Convolutional Neural Network
The fundamental structure of a convolutional neural network, which accepts data as input, is shown in Figure 1. It looks like a flame in this instance and the image is subsequently transformed into an abstract shape by a layer of networks, which also removes any traces of background noise and highlights the items that require recognition. Decision-making algorithms examine the layer outputs to make inferences. Layers provide areas of recommendations that are eventually integrated to construct a machine learning model with fully linked layers.
The physical arrangement of an Event-Based Fire and Smoke Detection and Classification (EFDC) node is an important aspect in the design of fire and smoke detection systems. The EFDC node typically consists of multiple components, including cameras, processing units, and communication interfaces. The cameras are typically positioned in strategic locations within a building to capture video footage of potential fire and smoke sources. The cameras can be analog or digital and may use infrared technology to capture images in low-light conditions. The processing units in the EFDC node are responsible for analyzing the video footage captured by the cameras and detecting the presence of fire and smoke. The processing units can be standalone devices or integrated into the cameras themselves. They may use algorithms such as Convolutional Neural Networks (CNNs) or modified Multi-Layer Perceptrons (MLPs) to detect fire and smoke in the video footage. The communication interfaces in the EFDC node allow for communication between the various components as well as communication with other nodes and a central control system. This enables the EFDC node to receive commands and send alerts to other systems in the event of a fire or smoke detection.
The physical arrangement of an EFDC node is a critical aspect in the design of fire and smoke detection systems. The node is typically made up of cameras, processing units, and communication interfaces that collaborate to detect and alert on the presence of fire and smoke in a building. The block diagram and the physical arrangement of the EFDC node of CCTV connected with a switch are shown in Figure 2.
Figure 2. Physical Arrangement of EFDC Node of CCTVs Connected with Switch
The Physical arrangement of the event-based fire and smoke detection and classification node of CCTVs in this system involves the placement of cameras at strategic locations. These cameras are connected to a switch, which facilitates communication between them and the rest of the system components. The system also includes an automatic warning system that activates in case of a fire detection, a router that manages the network communication, and an optical fiber cable for transmitting data. The fire and smoke detection is carried out using deep learning algorithms, which allow for realtime analysis and classification of the video footage captured by the cameras. The system, as a whole, provides a comprehensive solution for detecting and responding to fire incidents in real-time.
The system continuously monitors the video footage for signs of fire and smoke, and in the event of detection, the automatic warning system is activated, providing an immediate response to the incident. The optical fiber cable used in the system ensures fast and reliable transmission of data, allowing for real-time analysis and classification of the video footage. The combination of cameras, deep learning algorithms, and an automatic warning system makes this system a comprehensive solution for real-time fire and smoke detection.
The Event-based Fire and Smoke Detection and Classification Node of CCTVs in this system is designed to detect and classify fire and smoke incidents in real-time. The cameras are connected to a digital video recorder (DVR), which captures and records the video footage. The video footage is then analyzed by deep learning algorithms to detect and classify any fire and smoke incidents. The system also includes an automatic warning system, which is activated in the event of fire or smoke detection, providing an immediate response to the incident. The cameras are connected to the system through a CCTV coaxial cable, which transmits the video footage to the DVR for analysis. The system also includes a router for managing the network communication between the cameras and the DVR. The combination of cameras, DVR, deep learning algorithms, automatic warning system, and Closed Circuit Television (CCTV) coaxial cable makes this system a comprehensive solution for real-time fire and smoke detection and classification. Block diagram of the arrangement of EFDC node of CCTV with DVR is shown in Figure 3.
Figure 3. Block diagram of the Arrangement of EFDC Node of CCTV with DVR
A Certified Forensic Professional Engineer (CFPE) is a professional engineer who specializes in the investigation and analysis of fire and smoke incidents. The software used by the CFPE includes a flowchart that outlines the process for analyzing the data collected from the eventbased fire and smoke detection and classification system. The flowchart provides a clear and concise guideline for the CFPE to follow, ensuring that the correct output result is obtained. The software also includes remedial measures to avoid serious damage from fires and smoke, such as evacuation plans and fire suppression systems. The CFPE uses their expertise and knowledge to analyze the data collected by the system and provide recommendations for preventing similar incidents from occurring in the future. By combining the expertise of the CFPE with the capabilities of the software, this system provides a comprehensive solution for realtime fire and smoke detection and classification, as well as effective remedial measures to prevent serious damage.
The detection levels of fire and smoke using the CFPE software vary based on the deep learning algorithms used for analysis. Typically, the software is capable of detecting both the presence of fire and smoke as well as the severity of the incident. The quick smoke identification capabilities of the software allow for rapid analysis and classification of the video footage, enabling the CFPE to quickly assess the situation and respond appropriately. The CFPE software also includes the capability to alert the appropriate personnel in real-time, allowing for a prompt response to the incident. The combination of the CFPE's expertise and the capabilities of the software ensures that fire and smoke incidents are detected quickly and accurately, allowing for a rapid response and effective remediation measures to minimize damage. Figure 4 depicts the CFPE software work flow.
Figure 4. CFPE Software Work Flow
The work flow for the CFPE software's detection of fire and smoke typically involves the following steps: Video footage collection in the event-based fire and smoke detection and classification system captures video footage, which is then transmitted to the DVR or other recording device for analysis. Data analysis uses deep learning algorithms to analyze the video footage and detect any signs of fire and smoke. In the event of a fire or smoke detection, the software activates the automatic warning system, which alerts the appropriate personnel. The CFPE software classifies the incident based on the severity of the fire or smoke.
Response and Remediation for the CFPE software, is done using their expertise and the capabilities of the software, and provides recommendations for a rapid response to the incident, including evacuation plans and fire suppression systems to minimize damage. Incident investigation allows for a thorough investigation of the incident, including the analysis of data collected from the fire and smoke detection system and other sources.
Report generation in Certified Forensic Professional Engineer (CFPE) generates a comprehensive report of the incident, including recommendations for preventing similar incidents from occurring in the future. This work flow demonstrates the comprehensive approach of the CFPE software for the detection, classification, and response to fire and smoke incidents, ensuring a rapid and effective response to minimize damage.
The suggested technique is broken down into a number of steps in this paper. Data set acquisition, data preparation, feature extraction, model building, validation, and testing are some of the phases.
The data for recordings is in the form of still images taken from CCTV footage, however for training and testing purposes, bespoke movies should be utilized. The job of gathering these recordings by fire is laborious. Accordingly, frames with and without fire are saved. The dataset should then be divided into a test set and a training set. This must be done carefully since errors in the data that the neural network receives can distort the findings and result in a system that is inaccurate.
The next stage in creating top-notch machine learning models is data preparation. This is the stage when the data is cleaned, processed, or simply made accessible for usage. Noise and other undesired elements are removed from the frames during data preparation. Relevant data is necessary for algorithms. Otherwise, unpleasant outcomes could happen.
A neural network has to be familiar with the characteristics of flames as they are seen by the computer in order to detect them properly. To the human eye, fire characteristics are clearly distinguishable. Fire gives off a crimson hue. Depending on the fuel it burns, it takes on a different form and moves in different ways. In this piece, smoke and fire are detected by their shape, color, and movement. The characteristics are identified in the training set's various frames. These characteristics are extracted by a neural network utilizing a CNN feature extraction network driven by a unique algorithm. These video frames are divided into fire and non-fire scenarios once the features have been extracted. With the use of picture descriptors and bounding boxes, features are retrieved.
If accuracy is to be maintained and the system is to function properly, machine learning models must be validated. Another set of video frames, wholly unrelated to the data set used to create the model, is used for the validation procedure. According to test findings, the system has a 93% accuracy rate on the validation set.
Machine learning models must be validated if accuracy is to be maintained and whether the system is functioning properly. Another set of video frames, wholly unrelated to the data set used to create the model, are used for the validation procedure. According to test findings, the system has a 93% accuracy rate on the validation set.
It is difficult to say that the methods proposed thus far do not have any shortcomings. Our proposed method may also cause errors and consider electrical lamps as real fires. This mainly occurs at night because several objects suffer from blurring problem.
Fire detection can be difficult during a rainy night. To address this issue, researchers are currently experimenting with datasets containing images of fire at night in urban areas. In addition, a CNN-based model was proposed to solve blurry environments and generate sharp video frame sequences efficiently. This method is applied to illuminate blurred aspects in the future.
The algorithm is used in the cloud so that the computational power of the local server can be reduced to a great extent. The filtered and segregated video stream or images only will be sent to the cloud for processing and detecting the smoke and fire. This proposed methodology may speed up our model's ability to process multiple camera video inputs at once.
The expected outcomes for a system using fire and smoke detection with deep learning are Real-time detection and is capable of detecting and classifying fire and smoke incidents in real-time, enabling an immediate response to the incident. The deep learning algorithms used in the system provide quick and accurate identification of smoke, enabling a rapid response to the incident used for quick smoke identification. Effective responses from the automatic warning system, evacuation plans, and fire suppression systems included in the system ensure an effective response to the incident, minimizing damage and ensuring safety. The CFPE software allows for a comprehensive investigation of the incident, including the analysis of data from the fire and smoke detection system and other sources. The CFPE generates a report of the incident, including recommendations for preventing similar incidents from occurring in the future.
Overall, the expected outcomes of a system using fire and smoke detection with deep learning are improved safety, rapid and effective response to incidents, and a reduction in damage and losses. The review of such systems can provide valuable insights into the capabilities and limitations of the technology, as well as recommendations for its effective implementation.
In today's fast-paced world, fire and smoke pose a significant danger. The use of video images for machine learning-based fire detection is both novel and difficult. The monitoring system can be utilized to stop damage and loss from unintentional fire occurrences if it can be implemented in a large-scale, such as in major industries, residences, and woods. By fusing wireless sensors with Closed Circuit Television (CCTV), the suggested system may be upgraded to a more accurate and protective version. This algorithm demonstrates the ability to adjust to various situations.
The review of the systems using fire and smoke detection with deep learning highlights the effectiveness of the technology in real-time detection and classification of incidents, quick smoke identification, and the ability to provide effective response measures. The integration of the Certified Forensic Professional Engineer's (CFPE) expertise with the capabilities of the software ensures a comprehensive approach to the detection, classification, and response to fire and smoke incidents. The thorough investigation and reporting capabilities of the Certified Forensic Professional Engineer (CFPE) software also provide valuable insights into the incident, enabling recommendations for preventing similar incidents from occurring in the future. The use of fire and smoke detection with deep learning is an important tool for ensuring safety and reducing losses in the event of a fire or smoke incident.
The future scope for fire and smoke detection with deep learning is given by the continued development of deep learning algorithms, which will likely lead to improved accuracy in the detection and classification of fire and smoke incidents. The development of new sensors and technologies, such as thermal imaging cameras and smoke detectors with artificial intelligence capabilities, can provide additional data for deep learning algorithms to analyze, leading to more accurate and reliable fire and smoke detection. It also provides valuable data for predictive maintenance, enabling the identification of potential issues before they become critical and leading to more proactive maintenance and improved safety. The integration of fire and smoke detection with deep learning and emergency services, such as fire departments and ambulance services, can provide a more effective and rapid response to incidents, reducing damage and saving lives. The future scope of fire and smoke detection with deep learning is promising, with continued advancements in the technology likely to lead to improved safety, reduced damage and losses, and more effective responses to incidents.