Vol.8/No.1 (2016) INTERNETWORKING INDONESIA JOURNAL 11 Daytime Road Marker Recogitio Usig Grayscale Histogram ad Pixel Values Zamai Md Sai, Hadhrami Abd Ghai, Rosli Besar, W.S. Loi, Member IEEE Abstract Rapid ecoomic ad idustrial growth has impelled the use of motor vehicle o the roads worldwide. Despite the advacemet i road systems that ca hadle the vehicle uptur, the icrease i traffic accidets has yet to be coteded. Road markers are very iformative to the drivers while drivig ad differet sets of markers are ormally used betwee the highways ad the ormal road. Sigificatly, recogizig the correct road marker is highly essetial due to the differet types of marker (broke lae, cotiuous lae, double laes) which ca alert the drivers o the coditio especially o the o-highway roads to war the drivers from ot to overtakig at the prohibited area. At day time, from morig util eveig, the illumiatio plays major roles where the light from the su could impact the images captured o the road. The proposed lae marker detectio method is a visio system usig ew algorithm applyig the gray level histogram average media i defiig the threshold value to couter the illumiatio issues ad the average media pixel cout algorithm for the road marker classificatio process classifyig the correct types of marker throughout the day. The algorithm had bee tested at 3 differet times amely i the morig, afteroo ad eveig. The best accuracy is i the afteroo at 98% due to excellet illumiatio coditio whereas the accuracy were lower i the morig ad eveig at 81.13% ad 94.84% respectively due to low light coditio, effectig the markers illumiatio o the road. Idex Terms Lae marker, Real-Time, graylevel histogram, pixel cout algorithm, illumiatio. T I. INTRODUCTION HE statistic o traffic accidets i Malaysia has spawed a cocerig icrease at a average of 9.7% yearly over the last three decades. It is believed that the icrease i road accidets is liked to the rapid growth of populatio, ecoomic developmet, idustrializatio, ad motorizatio developmet experieced by the atio [1]. Geerally, the accidets are caused by the combiatio of these three factors which are road user errors, road eviromet faults ad vehicle defects [2]. Some examples of road user errors which cotribute to the icrease i road accidets are deliberately overtakig o prohibited areas especially at double lies, cotiuous lies, ad astoishigly; the emergecy laes. There are also those who switch laes without givig prior sigals causig loger reactio time towards ad emergecy situatio. Aother cocerig cotributig factors of users errors are drivig whe feelig fatigue ad sleepy. Hece the developmet of the visio based for lae detectio ad lae chagig system could be explored ad implemeted to assist road users while drivig. The research related to the lae detectio ad its subdivisio, had bee active i these past two decades with cosiderable outputs made i the past few years [3-4]. More autoomous features are developed ad added to the Auto- Assist Drivig System (ADS) such as Lae Departure Warig (LDW) the most portios where the attetio had bee give. Adaptive Cruise Cotrol (ACC), Lae Keepig (LK), Lae Ceterig (LC), Lae Chage Assist (LCA) ad Tur Assist (TA). For LDW, the mai objective is to war the uitetioally lae departure evets, while LCA is to assist the driver for a safer process of lae chagig. I LCA studies, some examples of the works are, the research o the distace estimatio betwee cars for safety prior of chagig laes [5-6] ad the issues o the usee area by the driver called blid spot, which is doe by scaig the blid zoe area to make it safe prior to the lae chagig evet [7]. With these features, the driver will be alert ad iformed that it is safe to chage lae or overtakig at that particular time. However, while chagig laes are allowed by usig these features, still the fudametal of lae markers recogitio is essetial for driver to kow whether the area is safe for the actio. Commo road markers are as show as i Fig 1. Mauscript received July 1, 2015. This work was supported i part by the Uiversiti Tekikal Malaysia Melaka Grat PJP/2014/FKE(15D)/S01357 Road Lae Recogitio Usig Visio ad Artificial Itelligece for Auto Assist Driver Alert System. All authors except W.S. Loi are with the Faculty of Egieerig ad Techology, Multimedia Uiversity, Jala Ayer Keroh Lama, Bukit Beruag, 75450 Melaka, Malaysia; (e-mail: zamaisai@utem.edu.my, hadhrami.abdghai@mmu.edu.my, rosli@mmu.edu.my, loiws@utem.edu.my). M.S. Zamai, the correspodig author together with W.S. Loi are also with Fakulti Kejuruteraa Elektrik, Uiversiti Tekikal Malaysia Melaka, Duria Tuggal, 76100, Malaysia. Broke Marker Fig. 1. Road Markers ISSN: 1942-9703 / 2016 IIJ Double Marker Curretly, the recogitio of the lae marker types i
12 INTERNETWORKING INDONESIA JOURNAL ZAMANI & HADHRAMI determiig the road coditio for driver s alert, for LCA is ot yet resolved. This is whe the lae marker recogitio would be a useful parameter for the ADS i makig better ad safer decisio while chagig laes. For example, the double lae is strictly hazardous area ad the driver is ot allowed to overtake others to avoid accidet while broke lae is vice versa. As the chagig laes will require the vehicles to be i the movig state, this is why the real-time recogitio system is importat to recogize the patter sequece of the markers ad able to alert ad disallow driver from overtakig itetioally at the prohibited area. The proposed visio system i this paper is usig the ew algorithm i image processig techiques to detect ad recogize the marker i the o-urba road (broke ad double lae) which ca be used as oe of the features for lae chagig warig system decisio to avoid driver from overtakig at the prohibited area ad causig accidet. The graylevel histogram is used for the thresholdig limit to cater the illumiatio issue ad the calculatio o the total white pixels compariso used to recogize the types of the lae markers. A. Previous Work II. RELATED WORKS The lae detectio process locates the paited lae markigs o a road surface with may of them usig visio-based systems. Sice the lae detectio would eed to track the marker, the pre-process marker recogitio also may also apply the same process. This is achieved by usig oe or two cameras placed at the frot of the widshield. The video recorded i a digital format are the aalyzed to extract the features related to the lae for the recogitio process. Researchers would agree that, object trackig may be exiget due to ubridlig issues ecoutered while developig the algorithm. Various approaches have bee used to accomplish the task either usig a sigle mode camera or stereovisio camera for 3D setup [8-9]. Nevertheless, the type of markers detectio o the road had ot bee explored due to majority of the researchers was workig o the road lae detectio ad its applicatio. The latest camera sesor techology ca capture images either i color or black ad white eablig researchers to choose whether to process images i grey scale or color, or both whe selected for optimum pre-processig method. Devisig optimum pre-processig method is vital because failure to detect accurate result will lead to more hitches that may call for rectificatio durig post-processig leadig to further exhaustive procedure. The iput images or videos take are either i the form of straight laes, complex clothoid or splie models. The images will be processed ad the features such as textures, colors, edges, local gradiet, cotours, frequecy domai ad geometrical shapes [10-13] ca be used i represetig the laes. Edges are oe of the features frequetly used i lae detectio process where it idetifies the cotour of a object by lookig o the pixel brightess chages i sharp output ad has o discotiuities. The gradiet of the pixels plays major roles i recogizig the cotour lies of a object i a image. This is because the o the road surface, the pait ad cocrete create strog edges implyig the large gradiets betwee the road ad lae exists due to their differeces i their itesities. May researchers are usig the covetioal gradiet-based feature method to solve this problem [14]. I video or image processig, the regio of iterest (ROI) is very importat to be defied before the image is processed. This is to reduce the computatioal complexity because most iformatio is useless i a image color spaces may be selected i color processig to the desired model such as the RGB, HSL, HSV or others. Prior to establishig the image threshold, some researchers would ehace their iput video by filterig ad elimiatig oise iterferece. Oe of the most popular filterig methods is Gaussia Blur which is to improve the detectio of the edge ad producig more accurate output [15]. Other researchers have attempted other methods such as by covertig the biary image back to its grey level to eable the edge detectio process ad fially detectig the lae [16]. Issues related to the illumiatio, occlusio, shadow, road coditio ad etc., still a cotributig factor for the researchers to improve the accuracy ad the capability of the trackig ad recogitio system. While the lae recogitio system is importat, research should also look o the typical road sig marker as ad its fuctio to embed the feature i the ADS. III. PROPOSED METHOD The complete proposed method for the system is show as i Fig. 2. Fig. 2. Proposed Method The process started with the video image acquirig process ad goig through the image processig steps. I the thresholdig for the biary image, samples of data were used i decidig the value for the thresholdig process which is importat to cater the illumiatio issues cotributed by the sulight discussed i sectio III-C. The biary images from the image will the bee aalyzed o the level of pixel quatity i the classificatio of the road marker type as discussed i sectio III-D. The result from the pixel quatity calculatio will be used as the threshold to differetiate the type of the marker i the last output decisio.
Vol.8/No.1 (2016) INTERNETWORKING INDONESIA JOURNAL 13 A. Camera Setup ad Positioig Fig. 3. The positioig of the USB camera By usig the USB type camera with video resolutio of 1280720 ad 30fps (HD 720p) ad 5MP still images, the camera is located at the ceter of the car as i Fig. 3. The video of the road images are recorded for aalysis. Three differet mai time (the worst case sceario) are recorded amely i the morig, afteroo ad i the eveig to check the effect of the illumiatio issue o the road marker as show i Fig.4. The cotrast of the lae marker versus the road is higher i the afteroo due to the high itesity of direct sulight ad its agle. While i the morig ad eveig, the itesity of the sulight ad the shadow of the car might affect the images from the video. Fig. 4. Video Images for three differet time zoes (morig, afteroo, eveig) I this experimet, the shadows, obstacles ad the su locatio which is opposite to the camera causig glarig which affectig the video quality is igored. The speed of the car recordig the video is betwee 50-60 km/h. B. Image Processig ad Feature Extractio Process The video image will be filtered to get the Regio of Iterest (ROI). With the regio, less computig will be achieved by removig all the uecessary iformatio i the video images. I this video, oly the small part, the trapezium will filter the importat part of the images for the processig. This process is doe by maskig the video images usig the bitwise AND operatio [14] as i equatio (1) where dst destiatio, src source. I this process, a mask is created with a biary value 0 ad 1. The area which requires to be passed o will be set as 1 i the mask filter. By usig this method, oly a partial of area or the ROI ca be filtered ad used for the processig. dst( I) src1( I) src2( I) if mask ( I) 0 (1) The fuctio i equatio (1) calculates the pre-elemet bitwise cojuctio for two arrays whe src 1 ad src 2 have the same size. C. Thresholdig value ad Image Histogram The thresholdig [15] as i equatio (2) is the process to get the best value for differetiatig the white lae ad the road as backgroud where dst destiatio, src source, x x pixel coordiate ad y y pixel coordiate. I this segmetatio method, the separatio is based o the itesity betwee the lae ad the road. max Val if src( x, y) thresh dst x, y) src( x, y) otherwise ( (2) To calculate the threshold value, the oe-stage thresholdig approach is used, where a image is separated ito two classes of pixels: the foregroud pixels (the white lae) ad backgroud pixels (the road). I this process, the road images captured i the morig, afteroo ad eveig are aalyzed to get the best value for the thresholdig. The images are coverted to the grayscale images ad by usig the histogram method [15] to decide the threshold value to differetiate the lae ad the road. I the histogram, the rage to differetiate the lae ad the backgroud will be aalyzed. Each of the data set (morig, afteroo, eveig) will provide the rage media value from the histogram for the bi value 200 pixels which situated betwee the bi for the markers pixel ad the road pixel, as i equatio (3) where T Time Take (morig, afteroo, eveig) ad 2,3,4,. The average value from these three data will be calculated as i equatio (4) ad will be used as the fial threshold value. Med T BiValue T1 200 pixels mi BiValue T 200 pixels max (3) Ave Med Ti i Th 1 (4) The average threshold value is calculated from the data, ad later used i the thresholdig process. The images are the coverted to biary where the white lae detected is coverted to 1 ad the rest is 0 which resembles the road backgroud. D. Pixel calculatio ad decisio DoubleLae Med T MiPix BrokeLae 2 MaxPix 200 pixels max ISSN: 1942-9703 / 2016 IIJ
14 INTERNETWORKING INDONESIA JOURNAL ZAMANI & HADHRAMI Ave PixTh Med (5) Ti i 1 (6) I the histogram, the rage betwee the white lae ad the backgroud cotais pixel lower tha 200 uits are selected. IV. RESULTS A. Data collectio ad pre processig The color video images are recorded for differet time zoe amely i the morig, afteroo ad i the eveig. The bitwise AND method is used by a mask biary image to filter the video images to get the ROI. I the result, the square shape is used to filter the importat iformatio to reduce the processig time as show i Fig. 5. Fig. 7. Grayscale Histogram for three differet time zoes Fig. 5. The process to filter for the Regio of Iterest (ROI) Various patters for the double ad broke laes captured as i Fig. 6, were processed to extract the features ad detect the type of the markers. Double lae marker captured havig more or less o the same patters, while the broke marker cosists a sequece of differet patters. Based o the combiatio of differet output histograms as i Fig.7, the fial value for optimal thresholdig calculated must be betwee the rages of the three data lies. The media of the rage will be selected to be the threshold settig value. The calculatio of the threshold value is show i Table 1. TABLE I THRESHOLDING VALUE ANALYSIS BIN NUMBER TIME Morig Afteroo Eveig Rage Bis with < 182-236 218-235 216-247 200 pixels Media 209 226.5 231.5 Average Media (Threshold value) 222 (a) The selected threshold value is used ad the output would be a biary image as i Fig. 8. (b) Fig. 6. Patters captured (a) sequece of double lae (b) sequece of broke lae B. Thresholdig value The histogram is used to fid the best value for the thresholdig. Separately aalyzed accordig to the differet time zoe (morig, afteroo & eveig), the data from the afteroo video is givig a sigificat differece betwee the white laes ad the backgroud, whereas both i the morig ad i the eveig, the graylevel values got shifted ad gettig earer betwee both the backgroud ad the white lae as show i Fig. 5. This is due to the illumiatio issue based o the sulight agle directio ad the itesity of the light which affected the color of the lae marker causig it to be easier i differetiatig the laes ad the backgroud compared to lower light situatio such as i the morig ad i the eveig. Fig. 8. The output from the thresholdig process for broke ad double lae markers C. Pixel Total Decisio The results of the thresholdig process of the images as show as i Fig.9. For double lae, the white areas are more or less the same o each of the frame, whereas the broke laes had show that the totals of white pixels are chagig throughout the full sequece while the car is movig.
Vol.8/No.1 (2016) INTERNETWORKING INDONESIA JOURNAL 15 (a) (b) Fig. 9. The thresholdig output (a) sequece of double lae (b) sequece of broke lae Based o the three data sets from three differet time zoe (morig, afteroo ad eveig), the media of each rage betwee the total pixels of double ad broke lae is calculated. From the graphs as i Fig.10-12, the cout of pixels for both double ad broke ca be see separated due to the total white pixels lae are differet i quatity. The higher rage of pixel cout i the morig ad the eveig are due to the illumiatio factor causig the differetiatig process betwee the lae ad the backgroud gettig harder i the thresholdig process prior to the pixel cout process. The calculatio of the pixel value cout for the classificatio is show i Table 2 startig from gettig the media for three differet times ad fially the averagig process. Fig. 12. Total of white pixel (lae marker) i the eveig TABLE II PIXEL COUNT ANALYSIS FOR CLASSIFICATION THRESHOLD PIXEL TIME Morig Afteroo Eveig Rage for lowest pixel double lae ad highest pixel broke lae 1868-2624 2418-2575 1721-4081 Media 2246 2496.5 2901 Average Media (Total pixel value) 2547 D. Accuracy Testig The average thresholdig value for biary image coversio ad the average value of the total pixel for classificatio obtaied from the previous sectios are used i the image processig algorithm ad had bee tested i the video images. Each of the output frames will be label as either broke or double lae o top left corer after the process of recogitio completed as i Fig. 13. The proposed method is tested at 3 differet times amely morig, afteroo ad eveig ad the result as i Table 3. Fig. 10. Total of white pixel (lae marker) i the morig Fig. 13. The output of the video frames with label o top right Total Frames Tested TABLE III ACCURACY TESTING Total Frames with correct result Accuracy Morig 1023 830 81.13% Afteroo 1029 1027 99.8% Eveig 989 938 94.84% Fig. 11. Total of white pixel (lae marker) i the afteroo From Table 3, the highest accuracy of the proposed algorithm is i the afteroo, where the light from the su, is perpedicular to the road. The illumiatio effects, give o shadow ad cause the white lae distictly ca be detected ISSN: 1942-9703 / 2016 IIJ
16 INTERNETWORKING INDONESIA JOURNAL ZAMANI & HADHRAMI from the backgroud i the image processig method. Whereas, the accuracy i the morig ad eveig are lower due to yellow lightig from the su ad also its agle, creatig more shadows ad glarig o the road, affectig the histogram value causig the white lae a bit harder to be detected from the backgroud. V. CONCLUSION From the result, the type of lae marker recogitio ca be performed by usig the image processig methods which are the grayscale histogram ad the pixel cout value. The average thresholdig ad classificatio value calculated at differet time i a day is importat for the system to accurately detectig the marker throughout the day. The illumiatio from the su chagig from the morig to eveig will affects the accuracy of detectig the white laes. Shadows from the other objects such as cars, trees ad other, also has a otorious effect o the lae marker detectio as the white pixel values ad couts will differ. The best illumiatio time, amely i the afteroo, respose better with the selected value from the thresholdig aalysis due to the better differeces betwee the white lae ad the backgroud with the accuracy i the afteroo detectio at 99.8%. I the low light coditio ad with certai agle illumiatio, the backgroud appears to be early the same with the backgroud, causig the algorithm to have errors i the classificatio with the data i the morig ad eveig at 81.13% ad 94.84%. Future works is to fid the image illumiatio factor for the system to eable the algorithm i ehacig the thresholdig process for the low light coditio (morig ad eveig) which to improve further o the detectio accuracy. By detectig the right marker o the road, it ca alert the driver about the coditio of the road especially o the federal road ad also to remid the drivers ot to overtakig o the prohibited area. ACKNOWLEDGMENT The authors would like to ackowledge the fudig support received from Uiversiti Tekikal Malaysia Melaka (UTeM) from PJP/2014/FKE(15D)/S01357 Road Lae Recogitio Usig Visio ad Artificial Itelligece for Auto Assist Driver Alert System, throughout the full project completio. REFERENCES [1] M. N. Mustafa, Overview Of Curret Road Safety Situatio I Malaysia, Highway plaig Uit, Road Safety Sectio, Miistry of Works, 2005. [2] T.Ozgur, L Guvec, F.Cosku ad E.Kashgil, Visio Based Lae Keepig Assistace Cotrol Triggered by a Driver Iattetio Moitor, i Systems Ma ad Cyberetics (SMC), 2010 IEEE Iteratioal Coferece, 2010, pp. 289 297. [3] J. C. McCall ad M. M. Trivedi, Video-Based Lae Estimatio ad Trackig for Driver Assistace: Survey, System, ad Evaluatio, IEEE Tras. Itell. Trasp. Syst., vol. 7, o. 1, pp. 20 37, Mar. 2006. [4] A. Bar Hillel, R. Lerer, D. Levi, ad G. Raz, Recet progress i road ad lae detectio: a survey, Mach. Vis. Appl., pp. 1 19, 2012. [5] W. L. W. Liu, X. W. X. We, B. D. B. Dua, H. Y. H. Yua, ad N. W. N. Wag, Rear Vehicle Detectio ad Trackig for Lae Chage Assist, 2007 IEEE Itell. Veh. Symp., pp. 0 5, 2007. [6] B.-F. W. B.-F. Wu, W.-H. C. W.-H. Che, C.-W. C. C.-W. Chag, C.-J. C. C.-J. Che, ad M.-W. C. M.-W. Chug, A New Vehicle Detectio with Distace Estimatio for Lae Chage Warig Systems, 2007 IEEE Itell. Veh. Symp., pp. 698 703, 2007. [7] R. J. Kiefer ad J. M. Hakey, Lae chage behavior with a side blid zoe alert system, Accid. Aal. Prev., vol. 40, o. 2, pp. 683 690, 2008. [8] R. Daescu ad S. Nedevschi, Probabilistic Lae Trackig i Difficult Road Scearios Usig Stereovisio, IEEE Tras. Itell. Trasp. Syst., vol. 10, o. 2, pp. 272 282, Ju. 2009. [9] K. H. Lim, K. P. Seg, A. C. Le Ngo, ad L.-M. Ag, Real-Time Implemetatio of Visio-Based Lae Detectio ad Trackig, It. Cof. Itell. Huma-Machie Syst. Cyber., pp. 364 367, 2009. [10] S. Mita ad D. McAllester, Lae detectio ad trackig i challegig eviromets based o a weighted graph ad itegrated cues, 2010 IEEE/RSJ It. Cof. Itell. Robot. Syst., pp. 5543 5550, Oct. 2010. [11] A. Parajuli, M. Celek, ad H. B. Riley, Robust Lae Detectio i Shadows ad Low Illumiatio Coditios usig Local Gradiet Features, Ope J. Appl. Sci., vol. 3, o. March, pp. 68 74, 2013. [12] Y. Leg ad C. Che, Visio-Based Lae Departure Detectio System i Urba Traffic Scees, i 11th Iteratioal Coferece Cotrol Automatio Robotics & Visio (ICARCV), 2010, 2010, o. December, pp. 1875 1880. [13] X. Zhe ad L. Zhifeg, A robust lae detectio method i the differet scearios, 2012 IEEE It. Cof. Mechatroics Autom., pp. 1358 1363, Aug. 2012 [14] K. S. Hujae Yoo, Ukil Yag, Gradiet-Ehacig Coversio for Illumiatio-Robust Lae Detectio, IEEE Tras. Itell. Trasp. Syst., vol. 14, o. 3, pp. 1083 1094, 2013. [15] P. Hsiao, C. Yeh, S. Huag, ad L. Fu, A Portable Visio-Based Real- Time Lae Departure Warig System: Day ad Night, IEEE Tras. Veh. Techol., vol. 58, o. 4, pp. 2089 2094, May 2009. [16] A. F. Sharifuddi, A. S. Abdullah, ad H. Arof, Developmet of realtime lae detectio software for automated steerig system, 2009 It. Cof. Tech. Postgraduates, pp. 1 6, Dec. 2009. [17] Shapiro, Lida G. & Stockma, George C. "Computer Visio". Pretice Hall. ISBN 0-13-030796-3. 2002. [18] G. Bradski ad A.Kaehler, Learig OPENCV Computer Visio With the OPENCV library, Safari Books Olie. ISBN 978-0-596-51613-0 2008. [19] Joh C.Russ & J. Christia Russ. Itroductio to Image Processig ad Aalysis, CRC Press. ISBN 978-0-8493-7073-1 2008. Zamai Md Sai received his Bachelor ad Master i Electrical ad Electroics Egieerig from Uiversity Sais Malaysia i 2000 ad 2009. Curretly, he is pursuig his Ph.D. degree from MMU ad also a staff member at Uiversiti Tekikal Malaysia Melaka. Hadhrami Ab Ghai received his bachelor degree i electroics egieerig from Multimedia Uiversity Malaysia (MMU) i 2002. I 2004, he completed his masters degree i Telecommuicatio Egieerig at The Uiversity of Melboure. He the pursued his Ph.D. at Imperial College Lodo i the same study area ad completed his Ph.D. research i 2011. Curretly, he serves as oe of the academic ad research staff members at MMU. Rosli Besar is curretly associate professor at Uiversity of Multimedia, received the B.Eg (Hos) ad M.Sc degrees from the Uiversity of Sciece Malaysia (USM), Malaysia, i 1990 ad 1993, respectively ad the Ph.D. degree from the Multimedia Uiversity, Malaysia, i 2004. His curret iterests iclude Sigal ad Image Processig, Medical Imagig.