A People Counting Method Based on Head Detection and Tracking

Transcription

1 SMARTCOMP 2014 A People Counting Metod Based on Head Detection and Tracking Bin Li, Jian Zang, Zeng Zang, Yong Xu Bio-Computing Researc Center, Senzen Graduate Scool, Harbin Institute of Tecnology, Senzen, Cina olybiner@gmail.com, zpower007@163.com, darrenzz219@gmail.com, laterfall@itsz.edu.cn Abstract Tis paper proposes a novel people counting metod based on ead detection and tracking to evaluate te number of people wo move under an over-ead camera. Tere are four main parts in te proposed metod: foreground extraction, ead detection, ead tracking, and crossing-line judgment. Te proposed metod first utilizes an effective foreground extraction metod to obtain foreground regions of moving people, and some morpological operations are employed to optimize te foreground regions. Ten it exploits a LBP feature based Adaboost classifier for ead detection in te optimized foreground regions. After ead detection is performed, te candidate ead object is tracked by a local ead tracking metod based on Meansift algoritm. Based on ead tracking, te metod finally uses crossing-line judgment to determine weter te candidate ead object will be counted or not. Experiments sow tat our metod can obtain promising people counting accuracy about 96% and acceptable computation speed under different circumstances. Keywords people counting; ead detection; LBP; ead tracking I. INTRODUCTION People counting tecniques ave been applied in many public places wit entrances, suc as supermarkets, subways and bus stations. Te people flow data of tese scenes can supply useful information for public security, marketing decision and resource allocation. Wit te increasing requirements for automatic people counting systems based on digital image processing and computer vision, effective people counting metods become remarkable and meaningful. Many studies on people counting ave been done [1-5]. Caroenpong [1] introduced a ead detection metod by using partial ead contour, and tey used te eclipse model to fit te contour. Zao et al. [2] utilized te air-color and ead contour features to detect uman eads. It is known tat uman eads ave an approximate circular sape, so te contour template is commonly used in ead detection. But te contour template based metod is often influenced by oter objects similar to uman eads in scenes. Mukerjee et al. [3] proposed an effective passenger counting metod, wic first uses Houg [4] circle transform to detect uman eads, and ten performs ead tracking using te optical flow metod [5]. However, te tracking metod based on te optical flow requires a large amount of calculation. In tis paper, we propose a people counting metod based on ead detection and tracking to solve te mentioned problems. Te proposed metod uses an over-ead camera to acquire videos of walking people. As a result, te ead is a stable, visible and evident part of a moving uman, and te proposed metod can divide umans into individuals. We also propose a novel metod for ead detection using a LBP feature based Adaboost classifier, and a kind of local ead tracking metod based on Meansift algoritm. We also propose an effective foreground extraction metod for ead detection and a crossing-line judgment metod wic is integrated wit ead tracking to perform people counting. Te remainder of tis paper is structured as follows. In Section 2, te framework of our metod is introduced. In Section 3, we describe te details of our metod including foreground extraction, ead detection, ead tracking and crossing-line judgment. In Section 4, experimental results and analyses are presented. Finally, we draw our conclusions in Section 5. II. FRAMEWORK OF OUR METHOD Te overall framework of te proposed metod as been presented in Fig. 1. Fig. 1 Framework of our metod /14/$ IEEE

2 Tere are four main parts in our metod: foreground extraction, ead detection, ead tracking, and crossing-line judgment. Te metod starts wit frame image acquisition from input videos. And ten te foreground extraction is performed to obtain regions of moving objects in frame images. Here VIBE proposed by Olivier Barnic et al. [6][7] is used as te foreground extraction tecnique. And some morpological operations are also employed to optimize te foreground regions. After te foreground extraction is implemented, a trained LBP feature based Adaboost classifier is applied to detect uman eads in te extracted foreground regions. Next, a local tracking metod based on Meansift algoritm is used to track te detected uman eads. Finally, te tracking results combined wit positions of te uman eads are applied to count te eads wic are crossing a counting line. Te counting line is a straigt line drawn by user and sould be vertical or orizontal across te frame image. It will be introduced in details later in tis paper. It is clear tat te number of eads and people are te same. III. PROPOSED METHODS A. Foreground Extraction In order to obtain regions of moving objects in frame images, foreground extraction tecniques sould be performed. Many tecniques, suc as background subtraction, frame difference, and GMM ave been proposed for foreground extraction [8]. Te background subtraction metod is quite sensitive to ligt canges and sadows and frame difference metod often produces gosts and oles. Te GMM metod [8] is very time-consuming in real-time applications. In contrast, te VIBE algoritm [6][7] is efficient, anti-noise and it takes up less memory. Terefore, we adopt it to perform foreground extraction. Te preliminary results are sown in Fig. 2. our experiment, we set its value as 200. Te value is suitable for tat te camera is installed about 2~3 metres ig from te ground. Tis eigt is commonly used in practical usage. Furtermore, morpological operations including image erosion and dilation are used to eliminate te noise and fill te oles. In tis paper, we firstly use closing operation wit 2*2 template to process foreground regions, and ten exploit opening operation wit 2*2 template to process te regions. Te optimized foreground regions after above operations are sown in Fig. 3. From Fig. 3, we can see tat irrelevant blobs in Fig. 2 are removed. Moreover, from Fig. 3 we can see tat noise in Fig. 2 is disposed and oles in Fig. 2 are filled. Fig. 3. Te optimized results of foreground regions: result by using an area tresold; result by using morpological operations. B. Head Detection In order to detect uman eads in frame images accurately, we use te binary image of foreground regions as mask to extract region of interest (ROI) from initial video frame images. Generally, in te binary image resulting from foreground extraction, foreground regions are wit gray value 1 and background regions are wit gray value 0. Tus, we firstly invert te binary image, convert it from binary to RGB mode, and ten perform bitwise-or operation wit te initial frame image. Finally, we can obtain useful foreground regions including moving uman objects of initial frame image. Te results are sown in Fig. 4. Fig. 2. Te preliminary results of foreground extraction: current frame image; result of VIBE algoritm. Te foreground regions obtained using te above metods usually contain muc noise and a lot of oles, and some blobs wic are not components of moving uman bodies. In order to remove te blobs, we utilize a tresolding formula to judge weter blobs sould be reserved or not as follows: 0, Area( B) Tarea B( i, j) (1) 1, Area( B) Tarea Were Area( B ) is contour area of a blob, and T area is an area tresold determined by prior knowledge of ead size. In Fig. 4. Te results of ROI extraction: te initial frame image; te ROI image including moving uman objects. After te operations stated above, te retained regions are defined as te candidate regions for ead detection. Ten, an off-line trained LBP based Adaboost classifier is used to detect uman eads. Te LBP feature as been proved to be a very good texture descriptor for objects [11]. We evaluate some oter feature descriptors for ead detection during experiments, and finally find tat LBP feature can give te 137

3 best detection accuracy and acieve a quite ig speed. In our experiment, we resize eac sample image of uman ead to 24*24 pixels. And for simplicity s sake, we regard eac resized sample image as a feature window. Eac feature window is split into 3*3 blocks wit te size of 8*8 pixels. Te non-uniform 8-neigborood LBP feature is adopted. Ten we can figure out te LBP descriptor in eac block using integral image metod [9][10], and furter get te LBP descriptor for te wole feature window. After preliminary ead detection, we use two tresolds of ead size to filter some false detected ead objects. Te minimum ead size S min is set to 20*20 because some objects could be smaller tan training samples, and te maximum ead size S max is set to 100*100 according to te video resolution. Te final ead detection results in candidate regions are sown in Fig. 5. Fig. 5. Head detection results in candidate regions C. Head Tracking After ead detection, we take detected uman eads as candidate ead objects to be tracked, and propose a local ead tracking metod based on Meansift algoritm. A counting line sould be specified vertically or orizontally across te frame image. Due to te time elapsed wen people crossing line is very sort, so ead objects just move in a local area around counting line during tis period. According to tis, we add a counting region centred on counting line to assist in tracking as sown in Fig. 6. Fig. 6. Counting line and counting region: orizontal; vertical. Te proposed local ead tracking algoritm is stated as follows: 1) Initialization: In te beginning, te algoritm produces a tracking of ead objects and initializes te tracking to empty. 2) Update: If a ead object is detected in a frame image, it will be determined weter to be added to te tracking or not according to te following rules: a) Out of counting region or not: If te centre of te ead object is out of te counting region, it will not be added to te tracking. Here te centre means te centre of bounding rectangle of te ead object. Te decision rule can be stated as follows: ( x xl x xr ) 0, H ( y yu y yd ) (2) candidate, oterwise. Were H denotes te ead object, x and y respectively represent x-coordinate and y-coordinate of centre of te ead object H, x L and x R respectively represent x-coordinate of left and rigt border of te counting region (if vertical), and y and U y D respectively represent y-coordinate of up and down border of te counting region (if orizontal). b) Already in te tracking or not: If te ead object is in te counting region, it will be determined weter it is already in te tracking or it is a new ead object according to te decision rule as follows: 0, Areaoverlap ( H, H ) T H (3) newobj, oterwise. Were H is eac ead object already in te tracking, Area ( H, H ) means te area of overlap between H and overlap H, and T is a tresold to measure degree of overlap. In our experiment, te value of T is set as 1. 3) Meansift iteration: Te Meansift algoritm is used to calculate new coordinates of eac ead object in te tracking. During tis process, te algoritm compares te new coordinates wit te previous coordinates of te ead object. If te Euclidean distance between tem is less tan a tresold T d or number of iterations is larger tan anoter tresold T n, te algoritm stops te iteration and updates te ead object. Finally te new position of te ead object is added to its route. In our experiment, T d is set to 0.2 and T n is set to 10. 4) Termination: If te tracking is not empty, te algoritm uses crossing-line judgment to determine weter eac ead object in tracking is crossing counting line or not. If te crossing-line condition is satisfied, te algoritm adds count number by 1; oterwise it repeats te process from 2) to 4). D. Crossing-line Judgment After te counting line and region are specified, relative locations between a ead object and counting line and region 138

4 can be determined. Tus, we can use te information to determine te initial moving direction of te ead object as follows: If te counting line and region is vertical: left2 rigt, x _ ini xl DH (4) rigt2 left, x _ ini xr If te counting line and region is orizontal: up 2 down, y _ ini yu DH (5) down2 up, y _ ini yd Were D H is te initial moving direction of te ead object, and x _ ini and y _ ini respectively represent initial x-coordinate and y-coordinate of centre of te ead object. After te initial moving direction of te ead object is determined, we can perform crossing-line judgment using current position of te ead object as rules below: left2 rigt, DH left2rigt and x xr rigt2 left, DH rigt2left and x xl DC (6) up2 down, DH up2down and y yd down2 up, DH down2up and y yu Were D C is te direction for counting people, and x, y represent te current x-coordinate and y-coordinate of centre of te ead object H, respectively. If te ead object satisfies one of cases ed above, we add count number corresponding to D C by 1; oterwise te ead object will be discarded. IV. EXPERIMENTAL RESULTS A. Training Because of lack of famous open-access datasets for ead detection, so we collected some ead images from camera, pedestrian datasets and internet. Tere are 2800 positive samples wit te size of 24*24 pixels, and 3000 negative samples wit different kind of sizes but larger tan tat of positive samples. Some positive and negative samples are sown in Fig. 7. Fig. 7. Some training samples: positive samples; negative samples. Here we use LBP feature based Adaboost classifier to train samples to get ead detector. Te typical process of training can refer to [9]. In our training process, te number of stages of classifier is set to 20, te minimum it rate of eac stage is 0.995, and te maximum false alarm rate of eac stage is 0.5. B. Testing Testing videos are obtained by using an indoor over-ead camera. Eac video as te same resolution of 352*288(CIF). All te experiments run on a computer wit 2GB memory and AMD 2.49 GHz CPU. In order to measure te performance of our metod, we could calculate detection rate, recall, and accuracy troug te following formulas: TP DetectionRate (7) TP FP TP Re call (8) TP FN TP TN Accuracy (9) TP FP TN FN Were TP denotes true positives, FP denotes false positives, TN denotes true negatives, and FN denotes false negatives. In te first experiment, we compared our metod wit HOG based Adaboost classifier [12] and HOG based SVM metod [13] on testing videos. Te comparison results are sown in Table I. Table I. Te comparison wit oter metods SVM-HO G Adaboost- HOG Our metod Detection rate 88.4% 94.1% 98.9% Recall 97.1% 86.5% 99.1% Accuracy 86.1% 82.1% 98.1% Average time per frame 67.8ms 40.7ms 36.4ms Compared wit oter relevant metods, our metod can acieve te igest detection rate, recall and accuracy. Meanwile, our metod costs te minimum time and tus as te best real-time performance. We demonstrate some detection results of mentioned metods in Fig

5 (c) (d) Fig. 8. Experimental results of mentioned metods, and from left to rigt tey are: SVM-HOG, Adaboost-HOG, and our metod: detection results wen people crossing; detection results of one-way walking people; (c) (d) detection results wen crowd density is ig. From te detection results in Fig. 8, we can see tat HOG based SVM metod often gives ig false positive rate, and HOG based Adaboost metod often gives ig false negative rate. But our metod can often provide accurate detection results. In te second experiment, we tested our metod on videos of different scenes. Here we coose 4 kinds of scenes. Te results are sown in Table II. Table II. Experiment results in different scenes Bi-directi onal Hig density Fuzzy scene Carrying tings Real number True positives False positives False negatives Accuracy 98.7% 96.8% 92.3% 97.4% We also provide some people counting results in mentioned scenes in Fig. 9. Te first scene is tat people walk bi-directionally, te second one is tat te crowd as ig density, te tird one is tat ligt canges and te scene is fuzzy sometimes, and te last one is tat people are carrying someting else. From Table II and Fig. 9 we can see tat our metod fit well in different scenes, wic proves tat our metod is quite robust and effective. (c) (d) Fig. 9. People counting results in different scenes: people walk bi-directionally; te crowd as ig density; (c) ligt canges and te scene is fuzzy; (d) people are carrying tings. C. Analysis Compared wit oter reported approaces, our metod can acieve better performance. Te comparison results are sown in Table III. Table III. Te comparison wit oter reported metods Metod Recall Accuracy BFR+MEA [14] 94.3% 94.5% Houg+Optical flow [3] 97.4% 93.5% Face detection [15] 82.6% 87.8% Our metod 98.9% 96.3% Firstly, our metod uses an over-ead camera to acquire videos of walking people in order to divide umans into individuals. Tis approac can avoid overlap of moving people wic frequently occurs in people counting based on face detection, ead-soulder detection and uman detection. Also, we reduce false detection rate troug te foreground extraction and te LBP feature based Adaboost classifier, 140

6 wic can eliminate oter static and irrelevant objects in frame images. Furtermore, our metod produces a tracking of ead objects and updates it all te time, to store track of eac ead object and ensure accurate people counting of te crowd. Compared wit metod proposed in [14], our metod solves overlap problem caused by ead-soulder detection of moving people, and works well wen crowd density is ig. Compared wit metod proposed in [3], our metod as faster ead detection and tracking speed, and solves problem of inaccuracy in detection caused by Houg circle transform. Compared wit metod proposed in [15], our metod mainly uses ead features to represent uman caracteristics, because ead feature is simple and effective for ead tracking. V. CONCLUSIONS Tis paper presents a novel metod for people counting based on ead detection and tracking. Te proposed metod generally consists of four parts: foreground extraction, ead detection, ead tracking, and crossing-line judgment. To solve te problems of ligt variations, background interference and irrelevant objects, we firstly perform foreground extraction and morpological operations to get candidate regions. Ten we apply LBP feature based Adaboost classifier to detect uman eads in te candidate regions. Te detected eads are tracked by a local ead tracking metod based on Meansift algoritm. Finally we use crossing-line judgment based on tracking results and positions of ead objects to determine weter te ead object sould be counted or not. Te proposed metod allows us to count moving people from different directions. Experimental results sow tat our metod can acieve promising people counting accuracy and acceptable computation speed, and is suitable for te real-time applications. Te proposed metod works well in different scenes and various crowd densities. tecnique for background detection and subtraction in video sequences, IEEE International Conference on Acoustics, Speec and Signal Processing, pp , April [8] Cris Stauffer and W.E.L Grimson, Adaptive background mixture models for real-time tracking, Computer Vision and Pattern Recognition, vol. 2, pp , [9] Paul Viola and Micael Jones, Rapid object detection using a boosted cascade of simple features, Computer Vision and Pattern Recognition, vol. 1, pp , [10] Paul Viola and Micael Jones, Robust real-time face detection, International Journal of Computer Vision, vol. 2, pp , [11] T. Ojala, M. Pietikainen, and D. Harwood, A comparative study of texture measures wit classification based on feature distributions, Pattern Recognition, vol. 29, pp , [12] Qiang Zu, Sai Avidan, Mei-Cen Ye, and Kwang-Ting Ceng, Fast uman detection using a cascade of istograms of oriented gradients, Computer Vision and Pattern Recognition, vol. 2, pp , [13] Navneet Dalal and Bill Triggs, Histograms of oriented gradients for uman detection, Computer Vision and Pattern Recognition, vol. 1, pp , [14] Yaowu Hu, Ping Zou, Hao Zou, A new fast and robust metod based on ead detection for people-flow counting system, International Journal of Information Engineering, vol. 1, pp , [15] Tsongyi Cen, Caoo Cen, Dajinn Wang, and Yili Kuo, A people counting system based on face-detection, International Conference on Genetic and Evolutionary Computing, pp , ACKNOWLEDGMENT Tis paper is partly supported by NSFC under grants No and ,as well as te Senzen Municipal Science and Tecnology Innovation Council (Nos. JCYJ and JCYJ ). REFERENCES [1] Teekapun Caroenpong, Human ead detection by using partial ead contour, Proceedings of te Tird International Conference on Knowledge and Smart Tecnologies, pp , [2] Min Zao, Di-ua Sun, and Wan-mei Fan, Hair-color model and adaptive contour templates based ead detection, Proceedings of te 8t World Congress on Intelligent Control and Automation, pp , July [3] Satarupa Mukerjee, BaidyaNatSaa, Iqbal Jamal, Ricard Leclerc, and Nilanjan Ray, A novel framework for automatic passenger counting, IEEE International Conference on Image Processing, pp , [4] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd ed., Prentice Hall, [5] B. K. P. Horn and B. G. Scunck, Determining optical flow, Artif. Intell., vol. 17, pp , [6] Olivier Barnic and Marc Van Droogenbroeck, ViBe: a universal background subtraction algoritm for video sequences, IEEE Transactions on Image Processing, vol. 20, no. 6, pp , June [7] Olivier Barnic and Marc Van Droogenbroeck, ViBe: a powerful 141