, pp.287-292 http://dx.doi.org/10.14257/astl.2016.139.60 Early Skip Decision based on Merge Index of SKIP for HEVC Encoding Jinwoo Jeong, Sungjei Kim, and Yong-Hwan Kim, Intelligent Image Processing Research Center, Korea Eletectronics Technology Institute (KETI), Korea, {jw.jeong, sungjei.kim, yonghwan}@keti.re.kr Abstract. The high efficiency video coding (HEVC) standard has significantly improved the coding efficiency by adopting a number of new coding tools. However enormous computational complexity is introduced by recursive rate-distortion optimization and new coding tools. In this paper, early SKIP decision method is proposed based on merge index of SKIP. There is high possibility that SKIP is selected as optimal mode when merge index of SKIP is 0. Based on this observation, if merge index of SKIP is 0, SKIP is selected as optimal mode and mode decision process for current CU is early terminated. To increase accuracy of early SKIP decision, we also utilize quantized residual of 2Nx2N_MERGE. Experimental results show that the proposed algorithm can significantly reduce encoding time by 42.9% on average with negligible RD performance loss (1.0%) under random access condition. Keywords: High efficiency video coding (HEVC), mode decision, SKIP, merge index 1 Introduction HEVC has adopted various new coding tools such as hierarchical structures of coding unit (CU), prediction unit (PU) and transform unit (TU) [1]. It can achieve 50% bitrate saving compared with previous standard H.264/AVC thanks to the new features. To achieve best coding efficiency, HEVC adopts recursive rate-distortion optimization (RDO). However, RDO introduces enormous computational complexity. Therefore, fast encoder implementation is desirable for real time application. Many approaches have been proposed in order to reduce computational complexity. In this paper, we only focus on early SKIP decision method because the portion of SKIP is very high from 45% to 95% [2]. Therefore, if it is possible to detect early SKIP decision, we can dramatically reduce encoding complexity. Several early SKIP decision methods have been studied [2-4]. Kim et al [3] proposes an early SKIP detection (ESD) algorithm, in which the different motion vectors and coded block flag (CBF) of 2Nx2N_INTER modes are utilized. In [4], Kim et al finds out that SKIP is spatially related. Based on the fact, they propose fast ISSN: 2287-1233 ASTL Copyright 2016 SERSC
SKIP decision by using global and local distribution of SKIP. Hu et al proposes early SKIP mode decision based on Neyman-Pearson theory [2]. This algorithm can reduce encoding time by comparing the rate-distortion cost (RDcost) of SKIP and Neyman- Pearson based predefined thresholds. The previous algorithms can effectively reduce encoding time based on utilizing motion information, CBF and RDcost relationship. To further reduce encoding complexity, we investigate probability that SKIP is selected as optimal mode according to merge index of SKIP. Based on this observation, this paper proposes fast SKIP decision method by utilizing merge index of SKIP. The rest of this paper is organized as follows. In section 2, we firstly represent the SKIP mode distribution according to merge index of SKIP. Section 3 introduces new early SKIP decision method based on merge index of SKIP. Section 4 shows the performance evaluation of the proposed algorithm. At last, the conclusion is drawn in Section 5. Table 1. SKIP mode distribution according to merge index of SKIP when the RDcost of SKIP is lower than the RDcost of 2Nx2N_MERGE. Sequences Traffic (2560x1600) BasketballDrive (1920x1080) BQMall (832x480) Depth merge index of SKIP Level 0 1 2 3 4 0 91.4% 73.5% 63.4% 55.8% 55.3% 1 97.0% 83.3% 75.1% 65.7% 67.3% 2 99.1% 91.1% 83.6% 75.0% 77.0% 3 99.7% 94.8% 89.1% 83.4% 86.3% 0 91.2% 82.5% 72.9% 64.2% 63.8% 1 97.1% 88.3% 79.1% 70.9% 68.7% 2 98.9% 92.8% 86.6% 80.1% 80.0% 3 99.6% 95.9% 92.4% 89.1% 88.4% 0 90.6% 71.0% 53.5% 50.7% 52.4% 1 94.8% 77.3% 65.1% 59.0% 55.7% 2 97.5% 86.2% 75.6% 66.4% 63.9% 3 99.1% 92.6% 83.3% 74.8% 74.8% BlowingBubbles 0 76.5% 66.1% 63.2% 57.9% 33.3% (416x240) 1 91.6% 79.1% 72.0% 55.3% 47.9% 2 96.8% 86.5% 78.6% 64.5% 63.6% 3 98.9% 91.3% 85.9% 79.4% 77.4% Average 95.0% 84.5% 76.2% 68.3% 66.0% 2 The Distribution of SKIP according to Merge Index of SKIP Instead of the median-based approach in H.264/AVC, the motion information of the PU associated with a skipped CU is inferred by block merging. If block merging is to be used, a merge index is sent to identify one of the candidates. The length of candidate list can be between 1 and 5, which is controlled by NumMergeCands. The whole process of adding candidates will stop as soon as the number of candidates reaches NumMergeCands. The candidates consist of ones from spatially neighboring prediction units, a temporal candidate and additional virtual candidates. The best candidate is selected using RDO in candidate list [5]. 288 Copyright 2016 SERSC
In order to investigate the distribution of optimal PU mode according to merge index of SKIP, the reference encoder HM16.9 is used for experiment. We encode four test sequences with various resolution size such as Traffic (2560x1600, 150 frames), BasketballDrive (1920x1080, 500 frames), BQMall (832x480, 600 frames) and BlowingBubbles (416x240, 500 frames) under random access (RA) main configuration [6]. We set NumMergeCands as 5 and quantization parameter (QP) as 32 for the experiment. In order to increase ratio of SKIP mode, we add the following condition that the RDcost of SKIP is lower than the RDcost of 2Nx2N_MERGE mode. Table 1 shows SKIP mode distribution according to merge index of SKIP when the RDcost of SKIP is lower than the RDcost of 2Nx2N_MERGE. The table shows that the ratio of SKIP decreases when merge index of SKIP increases. The ratio of SKIP with merge index 0 is range from 76.5% to 99.7% and 95.0% on average. It shows that the possibility, which SKIP is selected as optimal mode, is very high if merge index of SKIP is 0. We can also see the ratio of SKIP for various depth level. In large depth levels with small CU size, similar motion vectors can be predicted from neighboring blocks directly since the small blocks have strong correlation. Therefore, the portion of SKIP is extremely high more than 98% when CU size is 8x8. 3 Fast Early Skip Decision based on Merge Index of SKIP In this section, we introduce early SKIP decision method based on merge index of SKIP. If the RDcost of SKIP is lower than the RDcost of 2Nx2N_MERGE, we will check weather merge index of SKIP is 0. If merge index of SKIP is 0, SKIP is selected as optimal mode and mode decision process for current CU is early terminated. Start encoding a CU Compute RDcost of MERGE and SKIP N Y N Y Select SKIP as optimal mode Compute RDcost of remaining MODE Compare RDcost and Select optimal mode End encoding a CU Fig. 1. Flowchart of the proposed early SKIP decision algorithm Copyright 2016 SERSC 289
In order to increase accuracy of early SKIP decision, we consider quantized residual of 2Nx2N_MERGE with merge index 0. Note that SKIP and 2Nx2N_MERGE share motion information when merge index of SKIP is identical to merge index of 2Nx2N_MERGE. If the quantized residual of 2Nx2N_MERGE is zero, the possibility to be selected SKIP as optimal mode is very high. Otherwise, the possibility decreases. Therefore, we will check weather quantized residual of 2Nx2N_MERGE with merge index 0 is zero. Finally, if merge index of SKIP is 0 and quantized residual of 2Nx2N_MERGE with merge index 0 is zero, SKIP is selected as optimal mode and mode decision process for current CU is early terminated. Fig. 1 shows the flowchart of our proposed early skip decision algorithm. The early SKIP decision is conducted for a current CU after computing the RDcost of SKIP and 2Nx2N_MERGE. If the RDcost of SKIP is below than the RDcost of 2Nx2N_MERGE, the proposed early skip decision algorithm starts. Otherwise, the RDcosts of all remaining modes are computed. In Fig.1, merge index and CBF mean merge index of SKIP and quantized residual of 2Nx2N_MERGE with merge index 0, respectively. If merge index of SKIP is 0 and CBF of 2Nx2N_MERGE with merge index 0 is zero, computing RDcosts of all remaining modes are skipped and mode decision process for current CU is terminated. 4 Experimental Results To evaluate the performance of the proposed algorithm, it was implemented in reference HEVC test model HM16.9 [7]. The common test conditions defined in JCTVC-L1100 [6] are used as the anchor in experiments. We compare the proposed algorithm with early skip detection [3], which was adopted by HM reference software. The experiments are conducted under RA condition. Table 2 shows the results of proposed early SKIP decision based on merge index (ESDMI) of SKIP. ESDMI1 includes merge index condition and ESDMI2 includes merge index and CBF conditions. ESDMI1 and ESDMI2 can save 43.1% and 42.9% encoding time on average. The loss of coding efficiency is 1.2% and 1.0% respectively. Although ESDMI1 is fastest among three algorithms, it reveals little RD degradation because CBF is not considered. However, this result proves that only merge index condition of SKIP can achieve significant encoding time saving with negligible RD loss. In ESDMI2, CBF condition can reduce RD degradation from 1.2% to 1.0% without loss of encoding time saving. The proposed ESDMI2 outperforms ESD [3] with 10.9% more encoding time saving with 0.8% more BDBR loss. 5 Conclusion This paper proposes early SKIP decision method based on merge index of SKIP. We investigate the distribution of SKIP according to merge index of SKIP. To our knowledge, this is the first work about effect of mode decision according to merge 290 Copyright 2016 SERSC
index of SKIP. There is high possibility that SKIP is selected as optimal mode when merge index of SKIP is 0. To increase accuracy of early SKIP decision, we also utilize quantize residual of 2Nx2N_MERGE. Experimental results show that the proposed algorithm can significantly reduce encoding time by 42.9% on average with negligible RD performance loss (1.0%) under RA condition. Table 2. Results of proposed algorithms compared with recent work under RA condition. ESDMI1 ESDMI2 ESD [3] Sizes Sequences BDBR TS BDBR TS BDBR TS (%) (%) (%) (%) (%) (%) Class A Traffic 1.61 58.34 1.42 58.21 0.20 41.46 PeopleOnStreet 0.87 23.07 0.70 23.03 0.37 17.96 Nebuta 0.27 19.24 0.27 19.59 0.04 16.91 StreamLocomotive 1.89 50.78 1.47 50.42 0.32 37.51 Class B Kimono 1.04 45.86 0.87 45.67 0.30 33.27 ParkScene 1.21 53.20 1.04 53.05 0.21 39.43 Cactus 0.82 45.69 0.69 45.28 0.27 35.57 BasketballDrive 1.71 44.83 1.53 44.21 0.39 34.39 BQTerrace 2.17 55.05 1.98 54.49 0.30 39.63 Class C BasketballDrill 1.59 37.25 1.34 37.06 0.21 26.94 BQMall 1.01 44.57 0.90 44.81 0.37 34.84 PartyScene 0.83 36.05 0.73 35.91 0.21 29.35 RaceHorses 1.54 24.60 1.27 23.78 0.36 18.28 Class D BasketballPass 0.88 30.18 0.77 30.23 0.25 24.35 BQSquare 1.34 50.58 1.24 50.36 0.27 38.34 BlowingBubbles 0.91 37.34 0.80 37.53 0.24 29.65 RaceHorses 1.45 21.04 1.27 21.35 0.41 17.53 Class F BasketballDrillText 1.51 37.43 ChinaSpeed 1.02 32.35 1.29 0.84 36.76 32.70 0.22 0.31 27.10 24.18 SlideEditing 0.20 85.42 0.15 85.28-0.02 56.99 SlideShow 0.44 71.96 0.37 71.93 0.17 48.28 Average 1.16 43.09 1.00 42.94 0.26 32.00 Acknowledgement. This research is supported by Ministry of Culture, Sports and Tourism (MCST) and Korea Creative Content Agency (KOCCA) in the Culture Technology (CT) Research & Development Program 2016. References 1. ITU-T Rec. H.265, High Efficiency Video Coding, ITU-T, March 2013 2. Hu, Q., Zhang, X., Shi, Z., Gao, Z.: Neyman-Pearson-Based Early Mode Decision for HEVC Encoding, in IEEE Transactions on Multimedia, vol. 18, no. 3, pp. 379-391, March 2016. 3. Kim, J., Yang, J., Won, K., Jeon, B.: Early determination of mode decision for HEVC, in Proc. Picture Coding Symposium, 2012, pp.449-452. Copyright 2016 SERSC 291
4. Kim, M., Ling, N., Song, L., Gu, Z.: Fast skip mode decision with rate-distortion optimization for High Efficiency Video Coding, Multimedia and Expo Workshops (ICMEW), 2014 IEEE International Conference on, Chengdu, 2014, pp. 1-6. 5. Helle, P., Helle, P., Oudin, S., Bross, B., Marpe, D., Bici, M. O., Ugur, K., Jung, J., Clare, G., Wiegand, T.: Block Merging for Quadtree-Based Partitioning in HEVC, in IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1720-1731, Dec. 2012. 6. Bossen, F.: Common test conditions and software reference configurations, JCTVC-L1100, January, 2013 7. HEVC Reference Model, [Online] Available: https://hevc.hhi.fraunhofer.de/trac/hevc /browser/tags. 292 Copyright 2016 SERSC