Table 1. Analysis of Color Blindness (Nigam & Bhattacharya, 2013)
Video re-coloring and wavelength adjustment algorithm that have been performed for the anomalous Trichromats subtype Tritanomaly is presented in this paper. A method to handle the weaknesses faced by the anomalous Trichromacy was proposed and finally an interface is built that will bridge the gap for the color blind to view visual media without any hindrance. The RGB to LMS theory has been utilized for the required purpose. The detailed study about the RGB to LMS theory with respect to the required equations and pseudo code pertaining to the type Tritanomaly, i.e. Blue-Yellow weakness has been proposed to be done here using color Compensation Algorithm. Certain merits include Easy Detection, RGB to LMS Mapping, and Mapping of Frames to Easy Access of Videos.
Color seems to be the most pleasing and attractive characteristic for human beings and also most valuable in today's world of technology and digital media. As there are two faces to every coin, where we enjoy this luxury with great comfort at the same time, there are people who are deprived of this necessity. After luminance striking the human eye, the next most important entity that plays a vital role in human visual perception is color which is then categorized. Many individuals all over the globe suffer from distinct forms of color blindness. Some category of colors becomes completely unperceivable by the color blind (Machado et al., 2009).
Consequently, they may encounter numerous trifle and drastic accidents in their personal and professional lives. A system needs to be developed that helps to provide the people affected in their basic necessity so that they can eradicate the problem on a permanent basis and live a more fulfilling and happy lives (Machado et al., 2009).
Color is a visual perception property of the human eye to form different sensation of an object. Color are perceived by the photoreceptor cell found in the retina. The photoreceptor converts the color light into the signals which are passed to the brain. There are two types of photoreceptor cells (Ro & Yang, 2004), i.e. rods and cones. Rods works in low light conditions to help night vision, but cones work in daylight and are responsible for color discrimination (Yang et al., 2006).
Color Vision Deficiency (CVD) is the vision problem, where the person is unable to perceive colors such as red, green, and blue. There are three types of cone cells, namely s-cones, m-cones, and l-cones and each type has a different sensitivity to light wavelengths (Srividhya et al., 2011). S-cones perceives short wavelength color, mcones perceives medium wavelength color and the lcones perceives long wavelength color (Srividhya et al., 2011).
The majority of color vision deficient are caused by genetic disorder. The gene responsible for color blindness is carried on the x-chromosome, which causes more men to get affected to color blindness than women. Protanopia and deuteranopia are two of the most common forms of color blindness and third one being tritanopia caused by genetic disorder (Webexhibits, 1999). Diseases, drugs, and chemicals may cause color blindness. Color blindness occurs due to amentia caused due to shaken baby syndrome or by accident. Deficiency of ‘vitamin A’ may also be one of the causes of color blindness and the analysis of color blindness is shown in Table 1 (Nigam & Bhattacharya, 2013).
Table 1. Analysis of Color Blindness (Nigam & Bhattacharya, 2013)
There is no treatment to cure color vision deficiencies. However, people with color blindness may be able to use a special set of lenses to help them perceive colors more accurately (Webexhibits, 1999). These lenses are very expensive. These lenses can only be used outdoors under bright lighting conditions thus making it difficult to use (Kim et al., 2012).
Color blindness currently is a conundrum. It is difficult to read color-coded information like bar graphs or pie charts. It is more troubling for children who are not yet diagnosed with color blindness, since educational materials often includes color-coded information (Color Vision Testing, 2017). Children with color blindness may find it difficult to read a colored chalk on blackboard (Kim et al., 2012). Art classes which require selecting appropriate colors of paint or crayons, may be challenging (Oliveira, 2013).
Simple everyday tasks which includes cooking meat or selecting ripe produce can be a challenge for some housewives. Traffic signal lights may also cause complication to color blind people (You & Park, 2016). Since most lights are positioned horizontally, they can mentally learn to adapt, but if the position are changed then it may cause a big problem (You & Park, 2016).
In this paper, the authors have focused on how to re-color visual media involving images and videos for anomalous subdivision type Tritanomaly. They have advanced it with a simple approach by using an image coloring method with any previous theory applied on every frame of a video, as every video is divided into individual frames and considering each frame as an image (Colblindor, n.d.). The issue over here is that similar color appearance in distinct frames cannot be assured to be mapped and to be similar or twin new color because the temporal coherence between the different frames is not surfaced (Huang et al., 2011).
The purpose of visual correction is to improve the perspicacity viewing color information in images or videos with this type of deficiency. Basically, the core idea behind this algorithm is to re-map the colors of the original image for accommodating the trichromacy in order to provide a better color representation in the image and videos (Huang et al., 2011). However, in this paper, RGB to LMS conversion for the visual correction is used. The detail of the approach is discussed (Masra et al., 2018).
The objectives of the research work are as follows.
The system that has been developed is divided into two leading parts, namely the front end and back end of a system.
It is built keeping the end user in mind, wherein the color blind can log into the system, load the media he/she wishes to recolor, and accordingly get the output of the loaded image or video as shown in Figure 1. Note that the color compensation algorithm works only for the color blindness type Tritanomaly so the images and videos will be converted accordingly. The steps for the working of the interface are as follows:
Figure 1. Block Diagram of the System
The back end of the system describes the working of the color compensation algorithm for image and video recoloring. Here the steps or procedure are not visible to the color blind person. It involves step by step process to obtain the transformed recolored image/ video. The steps include as follows:
Step 1: Convert the video into individual frames
Since video file has a number of frames, it can vary from a hundred frames to even thousand frames. We can operate on the video all at once. Hence, we need to operate on single frame basis. Therefore, the video is converted into individual frames to process them.
Step 2: Extract the RGB color space of a frame
Each frame has various attributes associated with it, such as frame size, height, width, and the pixel values. By default, the frame has RGB color space. Hence, it is extracted to process it and modify the colors.
Step 3: Convert the RGB into LMS color space
Color blindness is associated with cones of our eyes. The cones such as long median or short are affected. Hence it is easier to work in LMS workspace. RGB is converted to LMS color space.
Step 4: Transform to colorblind LMS values
The authors have worked on people with Tritanomaly. They has s-cone absent. The matrix LMS2LMST converts the LMS matrix to the new LMS matrix which shows us how the person with Tritanopia sees the image.
Step 5: Transform the new LMS values back to RGB values
After the processing, the frame is converted from LMS space to RGB space.
NEWRGB = LMST * INVERSE (RGB2LMS)
Step 6: Calculate the error between original RGB and new RGB
Now we have the new RGB values, which is used to calculate the error. Error is calculated by subtracting the new RGB value from old RGB value.
Step 7: Modify the error for people with Tritanopia
Once the error is calculated, one knows what parts the affected people cannot see. These errors are modified so that people with Tritanomaly can easily distinguish colors.
Step 8: Add the error to original RGB
Now add the corrected error to the original image. This will keep the original image same and modify the colors, which people with Tritanomaly could not distinguish earlier.
Step 9: Merge the individual frames to finally make the new video
Once all the frames are processed individually, they are merged to get the output. The output is stored in avi file.
This whole process can be summarized as:
This program takes in media files and first makes the RGB to LMS conversion. Then assuming s-cone is missing, it deletes information related to the cone, and then reconverts it into the RGB space. This image roughly represents the normal image perceived by a Trichromat. Then, subtracting this from the original image, we find the information lost when the image is seen by a Trichromat. Then a transformation on the error function is made so as to map it to something that could be perceived by a Trichromat. Finally, new modified error function is added to the original image. So, the previously invisible details become visible.
Color correction is a lengthy and performance heavy task (Table 2). It requires a huge amount of time and computing for process. The time is directly proportional to the number of frames we process (Color blindness, n.d.). For higher number of frames, the time required is higher and for low number of frames, the time required is lower.
Table 2. Performance Table
It can be noted that the time per frame is similar whether we take a longer or shorter video.
This software was run on MATLAB r2015a on Intel i5 7200u processor with 2.71GHz speed and 8 GB RAM. We can predict a better performance result, if we run the same piece of code on better CPUs.
The original image and daltonized image of Balloons, Color spectrum, and Tibet are shown in Figures 2 to 4, respectively.
Figure 2. Comparative View of Image Processing using Balloons
Figure 3. Comparative View of Image Processing using Color Spectrum
Figure 4. Comparative View of Image Processing using Tibet
In terms of video processing, the video is divided into frames. Each frame is then processed on the same algorithm of the image processing for the respected type of color blindness. These recolor frames are then combined to form the video. The resulting video appears vivid to CVD (Daltonize, n.d.).
This system was tested by color blind people and the results were gathered. Each respondent was asked to give rating to the new recolored media file. Worst means the condition, if image deteriorated further. Some people saw no changes in the new and original media, whereas significant number of respondents found the system to be better. The people who responded much better were able to distinguish the colors, which they were unable to do so before. The survey was conducted online using social media and the recolor of frames are shown in Figure 5 and Table 3.
Figure 5. Recolor on Frames
Table 3. Survey Results
The details about Tritanomaly was studied and designed as a visual system for color vision deficient people. The system is designed keeping the front end with respect to the colour blind person. This system takes input in the form of image or video, processes from mapping RGB to LMS recoloring, and gives the extant output irrespective of the time of frames for shorter and longer videos used in the processes. This system can be implemented real time and used by various social medias and online websites such as YouTube and Facebook.