DOI: 10.1007/s11771 014 2105 2 Real time lae departure warig system based o pricipal compoet aalysis of grayscale distributio ad risk evaluatio model ZHANG Wei wei( 张伟伟 ), SONG Xiao li( 宋晓琳 ), ZHANG Gui xiag( 张桂香 ) State Key Laboratory of Advaced Desig ad Maufacturig for Vehicle Body (Hua Uiversity), Chagsha 410082, Chia Cetral South Uiversity Press ad Spriger Verlag Berli Heidelberg 2014 Abstract: A techology for uiteded lae departure warig was proposed. As crucial iformatio, lae boudaries were detected based o pricipal compoet aalysis of grayscale distributio i search bars of give umber ad the each search bar was tracked usig Kalma filter betwee frames. The lae detectio performace was evaluated ad demostrated i ways of receiver operatig characteristic, dice similarity coefficiet ad real time performace. For lae departure detectio, a lae departure risk evaluatio model based o lastig time ad frequecy was effectively executed o the ARM based platform. Experimetal results idicate that the algorithm geerates satisfactory lae detectio results uder differet traffic ad lightig coditios, ad the proposed warig mechaism seds effective warig sigals, avoidig most false warig. Key words: lae departure warig system; lae detectio; lae trackig; pricipal compoet aalysis; risk evaluatio model; ARM based real time system 1 Itroductio Traffic safety o highways has bee a area of iterest for may years, ad most occurreces of vehicle accidets result from egligece ad drowsiess of the driver. Cosiderig advisory ad warig systems are more easily implemeted tha fully autoomous cotrol, ad machie visio algorithms wi proved popular i target detectio ad trackig [1], it is expected that visio based drivig assistace system ca be used to effectively decrease the umber of accidets. Particularly, a visio based lae departure warig system (LDWS) ca alert drivers to the potetial dager of departure. Curretly, may works have bee doe to develop visio based system for lae detectio ad lae departure warig. However, most of them were implemeted o persoal computer based platforms other tha embedded platforms, ad preseted limitatios i situatios like shadows, varyig illumiatio coditios, curved road model ad algorithm implemetatio complexity. Before extractig lae feature poits, iverse perspective mappig (IPM) [2 4] were usually used to compesate the distortio by perspective, but required olie computatio to perform coordiate trasformatio (ad camera calibratio). Therefore, it is reasoable to cocetrate o developig lae detectio algorithm without applyig IPM i this research, sice computig resource is rare o embedded platforms. Two mai classes of lae feature poit detectio methods have bee proposed. Quite a few of methods commoly used i LDWS [5 6] were feature drive, which was based o the edge fidig. Such methods relied o the itesity discotiuities ad thus suffered from oise effects ad irrelevat feature structure. Ofte i practice, the strogest edges like shadows were ot the lae edges, so that the detected edges do ot ecessarily fit a straight lie or a smoothly varyig model. Hece, some excellet edge operators like Sobel operator [7] ad steerable filter [8 9] were proposed, ad WANG et al [10] used Cay feature operator after applyig the self clusterig algorithm, fuzzy C mea ad fuzzy rules ito automatic brightess compesatio. Alteratively, lae regio aalysis, which ca be modeled as a classificatio problem based o color space ad texture, was usually applied ito lae detectio. I case of color, RGB ad eve CIE space [11 12] were used for classificatio by the mea ad variace of road ad o road classes. However, the apparet color is ot cosistet due to several factors such as illumiatio ad viewig geometry, thus the color statistics of the road ad o road models eed be updated to adapt the chagig coditios. Besides color space, the texture based o the amplitude of gradiet operator at some Foudatio item: Project(51175159) supported by the Natioal Natural Sciece Foudatio of Chia; Project(2013WK3024) supported by the Sciece ad Techology Plaig Program of Hua Provice, Chia; Project(CX2013B146) supported by the Hua Provicial Iovatio Foudatio for Postgraduate, Chia Received date: 2012 11 13; Accepted date: 2013 03 04 Correspodig author: SONG Xiao li, Professor, PhD; Tel: +86 13517312418; E mail: jqysxl@hu.edu.c
1634 image area has bee used for regio separatio ad used i umarked road i most cases [13 14]. Whe it comes to lae detectio i marked road, the texture aalysis based o Gabor filter was combied with color to improve feature drive lae detectio [15]. However, this approach was of high computatio complexity ad the process speed was 3 frame/s due to the high dimesio of feature vectors. So, it is ecessary to use pricipal compoet aalysis (PCA) to reduce the dimesio ad the proportio of redudacies [16]. Accordigly, to detect the lae robustly, the advatages of feature drive ad lae regio aalysis were both cosidered. After iitial detectio with Hough trasform i the Gauss pyramid of low resolutio image, i some fixed rows lay the search bars, rather tha eedig statistical iformatio of the whole regio of iterest (ROI) for EDF as Ref. [17]. I each search bar, the lae feature poit was located based o PCA of grayscale distributio ad the fitted as parabola or straight lae model. Meawhile, the ceter positio of each search bar was tracked usig Kalma filter to reduce the sesitivity to oise poit whe fittig parabola ad straight lie. I the part of lae departure warig, much literature utilized the agle based lae departure warig algorithm. LEE [17] proposed a lae departure detectio method by symmetry axis ad local maxima of EDF. JUNG ad KELBER [6] ad WANG et al [10] used the same method based o agle relatios of the boudaries. The spatial warig mechaism i Ref. [5] set warig sigals if either d M or d S was larger tha (1/4)I w, which essetially was based o agle threshold of arcta (I w /(2I h )). However, the agle based warig mechaism would cause too much false egatives due to the obvious agle variatio i most situatios. As a result, JUNG ad KELBER [18] started to focus o the lateral offset feature i lae departure. Most LDWS set warig sigals whe vehicle states reach to the predetermied agle ad offset threshold, ad these methods were likely to lead to some false positives ad egatives ad brought some disturbaces ad crucial dagers. I fact, due to the exteral eviromet ad some subjective factors of drivers, vehicle may ievitably depart the curret lae i a short time, which is acceptable i ormal drivig. Therefore, cosiderig the driver s acceptability, road regio was divided ito four zoes with differet dager degrees i this research, ad a lae departure risk evaluatio model based o the lastig time i each zoe was proposed. However, whe it comes to some drowsy but coscious driver, vehicle teds to travel i a serpetie trajectory ad drive ito some dager zoe irregularly, i which situatio the model based o lastig time may fail accordig to the departure degree. So, the model based o frequecy of travellig ito each dager zoe was put forward as a complemet to form a dual warig mechaism. 2 System framework As a typical producer/cosumer model icludig image acquisitio ad processig, the system framework is exhibited i the way of state machie flow i Fig. 1, ad the lae departure moitor ad warig mechaism, actually, should be embedded i each sub state of image processig. The trasitio coditios of each sub state i image processig represet the process results i the previous frames. CAMIF represets the dedicated camera iterface i the embedded hardware platform. I the part of image acquisitio, peripheral camera was set to output VGA format image, ad the CAMIF was cofigured to execute the widow cut fuctio to get the ROI image directly. All the cofiguratios were fiished i the iitial phase ad show i shallow history pseudo state. So did the lae iitial detectio ad locatio for both boudaries i image processig. Fig. 1 Framework of LDWS
1635 3 Lae detectio based o PCA of grayscale distributio 3.1 Pricipal compoet aalysis of matrix Accordig to sigular value decompositio rule, ay real valued M N matrix A ca be writte as T m m p p p p A = U V = T σ 0 v 0 u L u O L (1) 0 p 1 σ 1 T p v p 1 where p=mi(m, ). The matrices U ad V are orthoormal, i.e., U T U=I ad V T V=I, ad so are their colum vectors. ui u j = vi v j = δ ij (2) The sigular values are all o egative ad ca be ordered i descedig order σ 0 σ1 L σ p 1 0 (3) Especially whe C is positive semi defiite, all eigevalues are o egative, ad C ca be costructed as the sum of a umber of outer products, i T T T T T i i ( ) C = a a = AA = U V V U = U U (4) So, Eq. (4) states λi = σ i where λ i is the eigevalues of C. From Eq. (1), aother variat ca be defied as AV = U Av = σ u (5) or j j j The equatio states that the matrix ca take ay basis vector v j ad map it i to a directio of u j with legth σ j, as show i Fig. 2. Fig. 2 Visualizatio of actio of matrix A Whe A is take as a symmetric ad positive semi defiite covariace matrix, after performig the eigevalue decompositio, which is kow as pricipal compoet aalysis, the pricipal orietatio ad magitudes of variatio of the data distributio aroud their mea ca be modeled whe Fig. 2 idicates how the pricipal compoet of the covariace matrices show the pricipal axis u j of the ucertaity ellipse correspodig to the data distributio ad how σ i = λ i show the stadard deviatios alog each axis. 2 3.2 PCA of grayscale distributio Edges usually exist at boudaries betwee regios with differet itesities, textures, or colors. The slope ad orietatio of grayscale distributio surface, which are described as gray value gradiet i mathematics, are useful for edge detectio. However, takig image derivatives highlights the high frequecies ad hece the oises are amplified, sice sigal to oise ratio is relatively lower i high frequecy sectio. So, a low pass filter was selected to smooth the ROI of image. Typically, the circularly symmetric Gauss filter is a proper oe which is idepedet of orietatio ad separable i recursive computatio. Let g X ad g Y be the first derivatives i x ad y, respectively, the the local mea value ad the covariace for x or y are X X (6) i= j= µ = w( i, j) g ( x + i, y + j ) ad σ = w( i, j)( g ( x + i, y + j ) µ ) XY X X i= j= ( g ( x + i, y + j ) µ ) (7) Y Y where w(i, j) are ormal Gaussia weights, ormalized so that w( i, j ) = 1. The covariace matrix of i= j= grayscale value gradiet is defied as σ xx σ xy C = (8) σ yx σ yy which is symmetric ad positive semi defiite. Takig PCA for matrix C, its eigevalues λ 1 ad λ 2 ca be obtaied, where λ 1 is ot smaller tha λ 2. Meawhile, it is here appropriate to take the orthoormal matrix U of C as vector [cosφ siφ] where φ resets the directio agel relative to x axis of the first pricipal compoet ad its value is 1 2 σ xy arcta, for σ xx σ yy 0 2 σ xx σ yy φ = π, for σ xx σ yy = 0 4 (9) At the areas icludig lae edge, the eigevalue alog the first pricipal axis is greater tha that alog the secod pricipal axis ad its directio agle is perpedicular to that of edge. O the cotrary, i a homogeeous regio, both eigevalues are very small. The experimetal results show that PCA of grayscale pixels reflects the gray value chages, ad this poit is explaied well i Fig. 3(a) as ellipses. I Fig. 3(a), a sub regio of a road image is show i 20 20 pixels, with the widow size for PCA beig
1636 eccetricity of a ellipse, c is the gray ormalizatio factor, p(i, j) presets the gray value of pixel. By itegratig trace(σ) ad gray value ito the defiitio i Eq. (10), it becomes a powerful meas to distiguish the regios with lae edge from the homogeeous oes. As for the gray ormalizatio factor c, T is a real value betwee 0 ad 255. The larger the T is, the more likely the pixel poit is to be a lae feature poit. So, the lae feature poit extractio algorithm (PCA EA) based o PCA of gray distributio is available with some threshold TG through OTSU [19]. 3.3 Lae trackig usig Kalma filter I order to extract lae edge iformatio efficietly, thirtee search bars, whose width is about 2 times of that of lae, were set i several fixed rows for both sides of curret laes, as show i Fig. 4. Due to camera perspective, the bar should be shorter at the top of ROI (far field), ad loger at the bottom(ear field), as the red bar shows. I iitial state, a Hough trasform was Fig. 3 Display of PCA of gray value chages as ellipses (a) ad trace value (b) 5 5 pixels. Complemetally, the trace of auto correlatio matrix for each pixel is described i Fig. 3(b), i which trace values reach their peaks at the lae edges, with the origi of coordiates locatig i the top left poit of Fig. 3(a). I reality, λ2 is merely small i compariso to λ1 due to oise, makig the ellipses alog lae edge be evolved ito some lies as show i blue rectagle i Fig. 3(a), while the others i the homogeeous regios arrow dow to some poits i orage rectagle because of λ1 ad λ2 beig small. More tha that, the gray average value o lae edges is much greater tha that of other pixels i a search bar, which ca ehace the detectio robustess. With the help of the elliptical visualizatio of covariace matrices, the behavior of grayscale distributio has bee aalyzed. The agle f oly poits out the directio of gray value chages, without idicatig the extet of gray value chages. I order to locate the lae feature poit exactly, the edge stregth of each pixel is defied as l l 1 T = (1-2 )trace( å ) p (i, j ) = 1 e 2 p (i, j ) c l1 c With e = l12 - l22 l1 ad it presets (10) the Fig. 4 Detectio result based o PCA of grayscale distributio ad Kalma trackig i day time (a), raiy day (b) ad ight time (c)
1637 implemeted i the Gauss pyramid of low resolutio image to reduce computig complexity, ad the the itersectio poit of the fixed row ad Hough lie was take as the system state vector for trackig. Suppose that the positio of lae feature poit ca be obtaied every costat time T (30 frame/s) ad laes move i lie with costat speed, so the liear discrete model of each lae feature poit ca be defied as X = A X + w Zk = H k X k + v k k + 1 k + 1, k k k where X = [ x x & ] k k k (11) ad it represets the positio ad velocity of lae feature poit alog X axis at time k; 1 T A k+1,k is the trasitio matrix ad A k + 1, k = 0 1 i the track algorithm; Z k is the measure output; H k is the observatio model that maps the true space to the observed space ad H k =[1 0]; w k ad v k are kow as process oise ad measuremet oise, all of which coform to Gauss distributio. I Fig. 4, the ceter of each detectio bar was tracked usig Kalma filter. The blue pixels are cadidate lae feature poits determied by PCA of grayscale distributio ad edge stregth. Cosiderig the gray average value of blue pixels was higher tha that of red pixels of about 17%, the ivalid cadidate feature poit ca be easily igored, thus leavig the valid feature poit of max value T aroud by red circles. The feature poits of each lae were subsequetly fitted as dashed/ solid parabola or straight lie accordig to the trackig result. As demostrated i Fig. 4, the whole detectio algorithm ca get satisfactory lae boudary uder various illumiatio coditios. Followig Ref. [2], the groud truth pixels about lae markigs ca be geerated efficietly through TS images. As the umber of pixels correspodig to lae markigs is small (about 1% 5%), so oly the left part of ROC makes sese to lae detectio algorithm, which is the reaso DSC (C DS ) is used for complemetary aalysis. DSC is defied as the harmoic mea of precisio rate (PR) ad recall rate (also amed TPR) at differet threshold T G. 2 1 1 N = + = C R R N N TP with R P DS P TP TP + FP C DS 2 N TP = 2 N + N + N TP FP FN (13) DSC curves imply the overlap betwee the groud truth pixels ad the extractio feature poits provided by differet algorithms, typically takig the 43rd percetile local threshold (PLT), media symmetrical local threshold (MLT SLT)[20] as compariso object for the PCA EA. The valuatio is performed alog a referece database [21] icludig su reflectio, o uiform shadows ad the result is show i Fig. 5. I Fig. 5, it ca be cocluded that the performace of MLT SLT is close to that of PCA EA from ROC 3.4 Lae detectio performace evaluatio The essece of lae detectio is classifyig the image pixels ito two classes: lae feature poits ad o lae markig poits, which make it be a classifier evaluatio problem. Classificatio accuracy is the usual idex to evaluate the classifier performace ad whe it comes to the biary classificatio problem, two classical evaluatio tools are used: receiver operatig characteristic (ROC) ad dice similarity coefficiet (DSC). ROC curves are obtaied by plottig TP rate (TPR) versus FP rate (FPR) at differet local gray or gradiet threshold T G, which are defied as R TP N N = = N N N N TP FP, R FP TP + FN FP + TN (12) where N TP correspods to the umber of true positive, N TN to true egative, N FP to false positive, FN to false egative. So, the larger the area uder the ROC curve, the better the detectio ad classificatio algorithm. Fig. 5 ROC (a) ad DSC (b) curves for lae feature poit extractio algorithm
1638 curves, ad the latter whose true positive rate reaches to 0.95 at most time is better tha the other two. Meawhile, the DSC curves show that PCA EA has the biggest peaks tha MLT SLT ad PLT at optimal threshold 45 aroud, ad the width of peak o PCA EA is larger tha MLT SLT, which meas the PCA EA is less sesitive o the selectio of threshold value ad has strog robustess. Figure 6 shows the digital waveform display of image acquisitio ad trasfer to processor, startig at blue colum bar 1 ad edig at blue colum 2. The digital sigals are show i white waveform. Table 1 describes the real time performace of the mai compoets of algorithm, idicatig the satisfactory requiremets of 30 frame/s. Fig. 7 Partitio of road regio without itetioal directio sigal, the biggest dager occurs. Some warig sigals should be set accordig to the dager degree. Fig. 6 Digital waveform of image acquisitio through CAMIF Table 1 Real time performace Algorithm Image acquisitio Lae feature poits locatio usig PCA Bar trackig usig Kalma filter UI update Total 4 Lae departure detectio algorithm Time/s 11.64 10 3 4.56 10 3 2.73 10 3 12.07 10 3 31 10 3 I geeral, the dagerous lae departure situatios result from the vehicle gettig close to or eve crossig the lae boudaries, but they ca be classified ito two differet forms, i differet lastig time ad i differet frequecies. Therefore, road was divided ito some regios of differet security levels firstly ad the the risk evaluatio models based o the lastig time ad frequecy were proposed to detect these two departure situatios respectively sice the relative lateral offset l 0 /W ca be got usig the liear part of the lae boudary model [18]. Suppose the width of sigle lae is b, the zoes Z 1, Z 2, Z 3, Z 4 were respectively, defied as safety zoe, trasitio zoe, alert zoe, dager zoe as show i Fig. 7. Safety zoe is of the highest security, whose width is b 2 b 1. Trasitio zoe with the width of b 1 is ext to the ier side of the lae lies ad has lower security, so vehicle ruig i this zoe with log lastig time ad ruig ito this zoe frequetly meas some potetial safety hazard. Vehicle beig i alter zoe of width b 2 meas pressig lie ad has bigger potetial safety hazard tha that of trasitio zoe. Oce the vehicle rus over the alter zoe ad ito dager zoe 4.1 Defiitio of risk evaluatio model The four zoes have their correspodig security level: Z 1 >Z 2 >Z 3 >Z 4, ad the risk evaluatio fuctio based o lastig time is expressed as: R t =f(z i, t) where i=1, 2, 3, 4, R t is the risk coefficiet, Z i is the divided zoe ad t is the correspodig lastig time i zoe Z i. The risk evaluatio model based o lastig time is defied i Fig. 8(a). Similarly, the risk evaluatio coefficiet based o frequecy R N ca be modeled as show i Fig. 8(b), ad N i (i=1, 2) is trasitio threshold for N. Fig. 8 Risk evaluatio model based o lastig time (a) ad frequecy (b) 4.2 Risk comprehesive evaluatio algorithm At a give piece of time, the timestamp ad frame stamp of vehicle drivig ito or out of zoe Z i (i=
1639 2, 3, 4) ca be got ad so is the lastig time for each zoe. It is worth otig that: 1) The actual lastig time for trasitio zoe was a combiatio of the time drivig ito trasitio zoe, alter zoe ad dager zoe; 2) The actual lastig time for alter zoe was a combiatio of the time drivig ito alter zoe ad dager zoe; 3) The lastig time for zoe Z i (i=2, 3, 4) was reset to zero whe vehicle just ruig i zoe Z 1. I the lae departure course, the risk evaluatio coefficiet based o lastig time for every zoe was computed oce vehicle touch ay zoe boudary, ad the the average of all the coefficiet values was take as the actual risk evaluatio coefficiet. Similarly, the risk evaluatio coefficiet based o frequecy ca be computed. So, the risk comprehesive evaluatio value ca be achieved ad updated from the bigger of the two coefficiets whe vehicle touch ay zoe boudary. Furthermore, some warig sigs should be issued oce the evaluatio value exceeds threshold R th (about 0.3) which is obtaied by empirical aalysis ad slightly chages with vehicle speed, ad the the lastig time ad frequecy of each zoe was reset to zero. Figure 9 shows a example of lae departure i which vehicle travel at speed of 65 80 km/h. Providig that b 1 =b/10, b 2 =b/7, the gree, blue, red lie correspodig to their couterparts i Fig. 7 are show i Fig. 9 where lae width is described as a stadard uit. The black curve alog 1800 m travelig distace represets vehicle s relative lateral offset l 0 /W, which was recorded at 30 frame/s i 92 s. It is clear that vehicle experieced several severe left ad right lae departure evets. I real world experimet parameters of lastig time ad frequecy were set as: t 0 =1 s, t 1 =10 s, t 2 =12 s, N 1 =16, N 2 =20. Table 2 shows a example of the departure lastig time for each zoe i the first 30 s time rage which is overlaid by gree trasparet layer i Fig. 9. I half a miute occur four left departure evets ad oe right departure evet. For example, i the 1st left departure evet, after ruig ito Z 4 with 1.71 s, R t reaches to 0.694, which is got from: [(0.76+0.80+1.71 t 0 )/(t 2 t 0 )+(0.80+1.71 t 0 )/ (t 1 t 0 )+1]/3. The frequecies i half a miute for vehicle ruig ito zoe Z 2, Z 3, Z 4 are 5, 4, 2 respectively, so just for the whole experimet R N reaches to 0.675, which is (5 2/N 1 +4 2/N 2 +1)/3. Actually warig sigs issued at the 1st, 5th case. Whe it comes to lae departure mechaism based o agle symmetry axis ad local maxima [17], the detectio mechaism issued warig sigals every time whe vehicle s relative lateral offset cross the blue lae boudaries i Fig. 9, which lead to some disturbace to the driver. Complemetally, after icreasig agle threshold slightly, the detectio mechaism teds to miss departure detectio whe the relative lateral offset cross ito Z 4, brigig much potetial risk without warig. 5 Experimets Fig. 9 vehicle relative lateral offset i some severe lae departure evets A hardware platform with a embedded warig system was implemeted. Sice ARM processors are characterized by their cost effective, low power capabilities, ad high performace, the core processor S3C6410 was selected, which was ARM11 based 32 bit RISC CPU of 667 MHz. The power cosumptio is about 1.5 W esured by a high efficiecy DC/DC coverter with wide 4 V to 60 V iput voltage rage. The CMOS sesor OV9650 was cofigurable to extract the VGA images i 30 frame/s through stadard SCCB iterface. Furthermore, to optimize memory utilizatio, the search bar was represeted i ru legth ecodig; some variables i courses like Gauss low pass filterig ca recursively be accumulated ad restored i memory buffers, istead of set a block of memory to read/write Table 2 Lastig time i lae departure experimet Lastig time(s)/r t Z 2 /R t Z 3 /R t Z 4 /R t Z 3 /R t Z 2 /R t 1st (left) 0.76/0 0.80/0.025 1.71/0.457 0.68/0.459 0.70/0.480 2d (left) 0.70/0 0.38/0.004 0/0.004 0/0.004 0.87/0.043 3rd (left) 0.65/0 0.84/0.022 0/0.022 0/0.022 0.66/0.052 4th (left) 0.12/0 0/0 0/0 0/0 0/0 5th (right) 0.21/0 0.24/0 1.92/0.418 0.35/0.441 0.21/0.448
1640 them i every step. With the help of DMA chael ad the widow cut fuctio i CAMIF of S3C6410, the whole processig performace for ROI of a image size of 640 480 is o less tha 25 frame/s. I highway experimets, three videos captured i daytime, as well as two videos captured i ight time, were radomly selected icludig some complex road types ad high dyamic rage scees, which ca be see from the sapshots i Fig. 10. The average detectio rate of lae detectio ca be more tha 99% uder differet traffic ad lightig coditios. Table 3 shows the statistics of the detectio rate over the five videos, ad the experimet results of lae detectio ad lae departure detectio are elaborated respectively, icludig the high dyamic brightess situatios i aother 16 videos. 5.1 Frame based daytime evaluatio I the first video referred i the ormal situatio, the lae markigs were well paied ad o u favorable pheomea occurred, so lae successful detectio rate ca reach up to 100%. The warig sigals were also correctly set. The warig mechaism was issued whe the risk comprehesive evaluatio value is more tha 0.31 cosiderig vehicle speed is about 80 km/h. The system issued warig sigal at the 502d frame ad stopped at the 563rd frame. Accordig to itermediate record, this warig is triggered through warig mechaism based o lastig time, because vehicle rus ito the dager zoe with 0.8 s. The secod video depicted a lae chagig situatio. Whe chagig laes, the curret lae lie to cross was vertical to image bottom ad the other two lae lies was almost symmetrical to the vertical lie, so by meas of this poit, the successful detectio ad trackig rate ca be achieved up to 99.88%. The system started warig at the 597th ad 1334th frame. The approachig situatio, which meas the host vehicle was approachig the frot vehicle ad the overtakes the frot vehicle, was demostrated i the third video. Ufortuately, lae boudaries were partly occluded by the vehicle i frot, thus resultig i usatisfactory detectio ad trackig results. This problem ca be solved by adjustig the parameters of lae chagig model. Fig. 10 Sapshots of videos i differet weather coditios: (a) Day time (b) Raiy time (c) Night time Table 3 Frame based successful detectio rate i differet situatios Time Frame Successful Failed Detectio rate/% Normal 1720 1720 0 100.00 Lae chage 1673 1671 2 99.88 Approachig 1620 1598 22 98.64 Total 5013 4989 24 99.52 No street light 1557 1518 39 97.50 Well lighted 1670 1657 13 99.22 Total 3227 3175 52 98.39 High dyamic brightess 2110 2086 24 98.86 10350 10263 100 99.15 Day Night Day & ight Video Total
1641 Table 4 Miss detectio rate ad false warig rate of frames i diverse situatios Groud truth Detectio result Straight lae Left curved lae Right curved lae Total No departure No departure 4503 frames (100%) 4257 frames (92.71%) 3745 frames (95.44%) 12505 frames (96.05%) Departure 0 frame 335 frames (7.29%) 179 frames (4.56%) 514 frames (3.95%) Departure No departure 0 frame 27 frames (1.26%) 165 frames (6.53%) 192 frames (3.02%) Departure 1709 frames (100%) 2102 frames (98.74%) 2359 frames (93.47) 6170 frames (96.98%) 5.2 Frame based ighttime evaluatio The 4th video shows o street light situatio i ighttime. I this case, the oly light source was from the host vehicle, ad the system was susceptible to the head light of vehicles i opposite directio. So, the successful detectio rate dropped to 98.64% sice the detectio of lae markigs was difficult i this case. However, the system ca still set warig sigals correctly at the 375th ad 962d frames. I the well lighted situatio, the system successfully detected ad tracked the lae with high efficiecy. After a chai of mild departure itesios i a short time, the system triggered the warig sigals correctly whe R N was more tha 0.33 cosiderig vehicle speed is about 70 km/h. 5.3 High dyamic brightess evaluatio I additio to evaluatig the detectio accuracy o some videos, some fragmets of high dyamic brightess situatios, like shadows, direct sulight, suddely pullig out of the tuel, were further validated i the other 16 videos. Owig to the strategy of lae trackig, the system obtaied a satisfactory detectio rate up to 98.86%. Whe it comes to direct sulight for a log time, the detectio performace degeerated more or less. This problem was ufamiliar ad ca be solved by automatic exposure cotrol, automatic gai cotrol ad icreasig the umber of fixed rows i ear field, sice image i ear field was much less likely to be affected by direct sulight. 5.4 Evet based warig evaluatio I order to validate the algorithm for detectig lae departure evets, videos i both day time ad ight time uder differet situatios were segmeted for departure evet. Furthermore, these evets were classified ito three categories: i straight laes, i left curved laes ad i right curved laes. To demostrate the performace of the proposed algorithm, the miss detectio rate ad the false warig rate were obtaied for all kid of video segmets. The detectio results are listed i Table 4. For departure evets from straight lae, the missdetectio rate ad the false warig rate are 0%. For departure evets from left curved lae, the missdetectio rate is 1.26% ad the false warig rate is 7.29%, which is higher tha that of right curved lae of 4.56%. This is due to the deviatio of the host vehicle from the ceter of lae, especially whe host vehicle with driver o the left side travel o the left curved laes it is accustomed to deviatig to the left side lae boudary. Noetheless, the whole mechaism still provide some miss detectios ad false alarms maily because of some road with sharp curved directio ad large lae fittig error o roads with severe oisy factors. However, the system possesses satisfactory performace sice the miss detectio rate ad false warig rate are o more tha 4% i etire situatios ad the whole performace ca be improved by more robust lae detectio method ad cosiderig curved model ito road regio partitio. 6 Coclusios 1) Aalysis results o auto geerated TS images idicate that the proposed lae detectio algorithm provides a accurate locatio ad fit to lae boudaries, showig better results tha other similar algorithms. 2) The dual mechaism warig algorithm based o lastig time ad frequecy is valid to detect uiteded lae departure uder various coditios, ad its missdetectio rate ad false warig rate are both acceptable. 3) The average processig speed is o less tha 25 frame/s o embedded platform with S3C6410 processor. Future work will adjust the warig trigger timig to get more comfortable drivig experiece ad some sophisticated strategies like matched filter would be itegrated ito the lae detectio approach to get more accurate lae fittig results. Refereces [1] JEFFREY D, ANTHONY G. Autoomous drivig A practical roadmap [C]// 2010 SAE Covergece, Detroit, Michiga, Uited States: SAE, 2010: 1 22. [2] AMOL B, MONSON H, MARK T. A ovel lae detectio system with efficiet groud truth geeratio [J]. IEEE Trasactios o Itelliget Trasportatio Systems, 2012, 13(1): 365 374. [3] STEPHAN S, SARATH K, ALEN A, GAMINI D. Robust lae detectio i urba eviromets [C]// Proceedigs of the 2007 IEEE/RSJ Iteratioal Coferece o Itelliget Robots ad Systems. Sa Diego, CA, USA: IEEE Press, 2007: 123 128. [4] MOHAMED A. Real time detectio of lae markers i urba streets
1642 [C]// IEEE Itelliget Vehicles Symposium. Eidhove, The Netherlads: IEEE Press, 2008: 7 12. [5] HSIAO P, YEH C, HUANG S, FU L. A portable visio based real time lae departure warig system: day ad ight [J]. IEEE Trasactios o Vehicular Techology, 2009, 58(4): 2089 2094. [6] JUNG C, KELBER C. Lae followig ad lae departure usig a liear parabolic model [J]. Image ad Visio Computig, 2005, 23: 1192 1202. [7] LIU Xi, XU Xi, DAI Bi. Visio based log distace lae perceptio ad frot vehicle locatio for full autoomous vehicles o high roads [J]. Joural of Cetral South Uiversity, 2012, 19: 1454 1465. [8] JOEL C, MOHAN M. Video based lae estimatio ad trackig for driver assistace: Survey, system, ad evaluatio [J]. IEEE Trasactios o Itelliget Trasportatio Systems, 2006, 7(1): 20 37. [9] WANG Y, NAIM D, ALIN A. A ovel system for robust lae detectio ad trackig [J]. Sigal Processig, 2012, 92: 319 334. [10] WANG J, LIN C, CHEN S. Applyig fuzzy method to visio based lae detectio ad departure warig system [J]. Expert Systems with Applicatios, 2010, 37: 113 126. [11] CHENG H, JENG B, TSENG P, FAN K. Lae detectio with movig vehicles i the traffic scees [J]. IEEE Trasactios o Itelliget Trasportatio Systems, 2006, 7(4): 571 582. [12] MA Chao, XIE Mei. A method for lae detectio based o color clusterig [C]// 2010 Third Iteratioal Coferece o Kowledge Discovery ad Data Miig. Phuket, Tailad: IEEE Press, 2010: 200 203. [13] BUI T, NOBUYAMA E. A local soft votig method for texture based vaishig poit detectio from ustructured road images [C]// SICE Aual Coferece. Akita, Japa: SICE, 2012: 396 401. [14] RASMUSSEN C, KORAH T. O vehicle ad aerial texture aalysis for visio based desert road followig [C]// Proceedigs of IEEE Computer Society Coferece o Computer Visio ad Patter Recogitio (CVPR). Sa Diego, CA, USA: IEEE Press, 2005: 1063 6919. [15] RODOLFO T, MIGUEL T. A compariso of gradiet versus color ad texture aalysis for lae detectio ad trackig [C]// The 6th Robotics Symposium i Lati America (LARS). Valparaiso, Chile: IEEE Press, 2006: 1 6. [16] MANISH H, LIU J, JOHN F. Image texture aalysis: Methods ad comparisos [J] Chemometrics ad Itelliget Laboratory Systems, 2004, 72: 57 71. [17] LEE J. A machie visio system for lae departure detectio [J]. Computer Visio ad Image Uderstadig, 2002, 86: 52 78. [18] JUNG C, KELBER C. A lae departure warig system usig lateral warig usig u calibrated camera [C]// The 8th Iteratioal IEEE Coferece o Itelliget Trasportatio Systems. Viea, Austria: IEEE Press, 2005: 348 353. [19] XU X, XU S, JIN L, SONG E. Characteristic aalysis of Otsu threshold ad its applicatios [J]. Patter Recogitio Letters, 2011, 32: 956 961. [20] POLLARD E, GRUYER D, TAREL J, IENG S, CORD A. Lae markig extractio with combiatio strategy ad comparative evaluatio o sythetic ad camera images [C]// The 14th Iteratioal IEEE Coferece o Itelliget Trasportatio Systems. Washigto, DC, USA: IEEE Press, 2011: 1741 1746. [21] VEIT T J, NICDLLE P. ROMA (ROadMAkigs). [EB/OL]. http:// www.lcpc.fr/fracais/produits/bases de doees de referece/article/ roma road markigs (Edited by DENG Lü xiag)