Decompression Method For Massive Compressed Files In Mobile Rich Media Applications

Transcription

1 th IEEE International Conference on Computer and Information Technology (CIT 2010) Decompression Method For Massive Compressed Files In Mobile Rich Media Applications Houchen Li, Zhijie Qiu, Lei Luo School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu Abstract Aiming at the characteristics of mobile terminal equipments, the program of decompressing massive compressed files in mobile rich media applications is investigated in this paper. Firstly, the demand for decompressing is analyzed according to the demand for rich media applications and the special requirements of mobile terminals, so as to design rational decompression program. And then, the optimization of the decompression flow of open source project LiteUnzip, which is to adapt to the characteristics of various mobile platforms, improve the disadvantage of long execution time for single times process, improve decompression efficiency, and renovate LiteUnzip into a high-efficiency decompression program that can be suitable for mobile terminals, is described. Finally, the specific implementation of decompressing massive compressed files in rich media application programs in mobile terminals is achieved. Keywords decompress, interactive application, mobile terminals, massive compressed files, rich media applications, user experience 1. Introduction With the popularity of intelligent cell phone, and the coming of 3G and super 3G age, as well as the promotion by various aspects of the industrial chain, mobile internet is no longer limited to such applications as image, ring, news, and information on WAP, blog, search, and communities have already extended to the mobile internet, and people will experience services such as games, music, video, animation, position service, and business affairs anytime and anywhere[1]. It is obviously that the contents of mobile internet are becoming massive while the application of mobile rich media is one of the most typical service and expression means for massive data on mobile internet. Rich media is different from traditional media and it combines multimedia objects such as graph, image, animation, and audio and video organically in time/space and provides rich expression means and interaction ability [2]. It is mainly characterized in generating dynamic actions based on time or user interaction[3]. Compression and decompression technology will surely be widely applied in the application of mobile rich media to satisfy the demands for transmission and processing of massive multimedia objects. However, embedded equipments such as intelligent cell phone, have limited processing capacity, and the system platforms of such mobile terminals are of many varieties and different system platforms have different characteristics. Compression and decompression method suitable for different system platforms of mobile terminals is a necessity for mobile internet which is now enjoying fast development. Since decompression of compressed files is on the contrary to compression of files, and the application program of mobile rich media, as the client terminal in application of mobile internet, generally has greater demand for decompression than that for compression. Therefore, this paper emphasizes the research on and implementation of decompression program for massive compressed files. The compression program frame can be obtained by making corresponding reverse processing to the following decompression programs[4], for whose details will not be mentioned in this paper. 2. Requirements analysis of decompression 2.1 Analysis of decompression requirements according to the characteristics of mobile terminal platform The mobile terminal platform is not only characterized in poor hardware performance, its embedded operating systems are also diverse and differ /10 $ IEEE DOI /CIT

2 from each other greatly. The unique characteristics of various mobile terminal system platforms have special requirements for decompression of massive compressed files, such as: 1. Symbian OS: although Symbian OS supports multithread in the process, it does not encourage multithread actually but to avoid multithread as far as possible and use an event driven collaborative multitasking model active object instead[5]. One of the reasons is that running of multiple threads may cause the system to come to a stop due to context switching[6]. Besides, the scheduling switch between threads also requires additional overhead of the system. Although the active object stimulates multiple threads in a process, it can only be executed under single thread. For the rich media application program, including graphic user interface[7], the operation in each active object must be completed quickly, otherwise UI events will be blocked and thus to affect user s experience. Therefore, to realize the decompression of massive compressed files in Symbian platform, it is better not to use new thread to accomplish the decompression and not to affect the response to user s operation in the meantime. 2. Brew platform: brew platform supports multithread, but its threads are non-preemptive, and currently running thread must give up its execution right actively, namely present thread should suspend itself actively so as to schedule other threads[8]. Therefore, the thread responsible for decompression should suspend its own thread after completing a segment of decompression flow so as to enable the scheduling of other threads and provide fast response to user s operation[9]. Therefore, the decompression program that can adapt to the diverse characteristics of mobile terminal platform should be able to complete the decompression operation without long blocking other emergency services even not use multithreading technology. So, it requires that the decompression flow can be rationally divided into several smaller program processing procedures. 2.2 Analysis of decompression requirements according to decompression efficiency For compressed file with massive data, the application program will first inspect a certain file (such as a certificate and configuration file, etc.) in the compressed file and then decide whether to obtain other files in the compressed file or to decompress some of the files in the compressed file instead of decompressing all of them according to the inspection results. To support decompression of partial files, it requires that any designated file in the compressed file can be decompressed. To decompress designated files requires traversing the entrance information of all the files in the compressed file and matching each information so as to determine whether the designated file is in the compressed file or not and obtain entrance information of the designated file. Obtaining related information (including entrance information of the file and file name, etc.) of the file can be independent from obtaining data of the file, namely the operation of file decompression data is unnecessary to follow the operation of obtaining entrance information of the file, so that decompression can use the previously obtained and buffered entrance information of the file to avoid repeated entrance information obtaining of the same file which exists in the decompression function library of open source project LiteUnzip[10]. 2.3 Outline of decompression requirements in mobile rich media applications According to the above analysis, the decompression function in the rich media application program of the mobile terminal platform can satisfy the following demands: 1. Decompression flow must be able to be divided into several smaller program processing procedures rationally; 2. Any designated file in the compressed file can be decompressed separately; 3. Related information of each file in the compressed file can be traversed easily; 4. Obtaining related information of the file is independent from obtaining file data. 3. Design of decompression solution Single operation of obtaining data from the compressed file may obtain related information of the file or obtain file data and decompress such data, and both of the processes include file data reading flow and data analyzing and processing flow and both of them are relatively time-consuming. The decompression program is mainly described by the processing means of the two processes. 3.1 Obtaining related information of files To support obtaining related information of a designated file, related information of the file can be obtained by two means: 1. Obtain related information through the index value of the file in the compressed file (if the number of files 2067

3 in the compressed file is n, the range of index value is 0 ~ n-1), and the program flow is as below: Fig 1. Flow of obtaining related information of a file through index value 1 Whether there is related information of the designated file in the program buffer or not; 2Search the location of related information of the designated file in the compressed file from current file information visiting location (original file information visiting location of the program is the location of related information of the first file) in single direction, and if current file information visiting location is after the location of related information of the file to be visited, the searching will be started from the location of related information of the first file until the description information of the designated file is visited; 3 Read the data containing related information of the file from the compressed file; 4 Analyze such data, construct entrance information of the file (which is required for decompressing data of the file, including initial value of crc32 accumulated checksum, time of creation and modification of the file, etc.), and obtain other information such as file name, so as to obtain related information of the file with designated index value. In the meantime, the program will storage related information of the file into program buffer for use when necessary. 2. Obtain related information of the designated file through file name, and the program flow is as below: Fig 2. Flow of obtaining related information of a file through file name 1 If it is the first time to enter the traversal process, the index value of current file is set to be 0; if the index value of current file is greater than or equal to the total number of files in the compressed file less 1, the index value of current file is set to be 0; otherwise, the index value of current file will increase by 1; 2 Execute the above flow of obtaining related information of the file through index value of the file; 3 Match the file name of the file in this index value with the file name of the file required and judge whether the file in this index value is the file required. The above two means of obtaining related information of the file satisfy the demand for traversing related information of all files in the compressed file and make necessary information preparation for decompressing any designated file in the compressed file. In the meantime, the flow of obtaining related information of a file according index value of the file is short and the execution process is simple, not requiring division into smaller processing procedures, and is the unit execution process for obtaining related information of the file, while the process of obtaining related information of a designated file through file name is composed of several unit execution processes. 3.2 Decompression of file data On the premise that related information of the file has been obtained, the entrance information of the file can be obtained, and the data of this file can be decompressed according to the entrance information of the file designated. Since the flows of such key algorithms as decryption algorithm, decompression algorithm, and crc32 algorithm used for decompression are flows of accumulated operation 2068

4 to data one by one, the division of file decompression process is realizable. Fig 3. Flow of decompressing file data block by block according to entrance information of the file 1 Related information of the file designated should be obtained first before obtaining entrance information of the file designated, and then pick up the entrance information of the file; 2 Pick up part of the data that has not been decompressed from the compressed file according to currently valid entrance information of the file. The length of such data picked up can be determined according to the processing capacity of different mobile terminals. Generally, The length of this data is an empirical value determined in specific engineering practice. As long as it will not affect the processing of UI event, the greater the value, the better; 3 Whether all the data of the designated file has been decompressed. 4. Implementation of decompression program Take the decompression of compressed file in ZIP/GZIP format as the example, modify the program framework of the open source project LiteUnzip whose functions are sound, and explain how to improve the decompression flow to satisfy the special demands of the rich media application program on mobile terminal for decompressing massive compressed file. 4.1 Decompression flow of LiteUnzip Fig 4. Decompression flow of LiteUnzip 1 Optional operation types include: obtaining the entrance information of a designated file in the compressed file and decompressing the designated file. Before this procedure, the file to be operated in the compressed file must have been designated; 2 Search from the location of related information of the first file in the compressed file until the description information of the designated file is visited, and obtain the entrance information of this file; 3 Read the file data according to the entrance information of the file obtained, and decode and decompress such data and perform crc32 verification, and store the file data decompressed to the designated buffer; 4 Determine whether other files need to be processed or not, if yes, this file is the designated file in subsequent flows, otherwise, destroy the decompression handle. It is easily found that the operation of obtaining entrance information of the file is repeated during the process of actual file data decompression and the process of obtaining related information of the file only, which causes loss in efficiency. Besides, the actual decompression operation must follow the operation of obtaining entrance information of the file, so that the entrance information of the file previously obtained can not be used for this decompression, which also causes unnecessary efficiency loss. LiteUnzip can only decompress the whole file at one time especially in 2069

5 decompression of large file, which will cause blocking of other emergency services. 4.2 Optimizing LiteUnzip decompression flow with decompression program The modifications to decompression flow of LiteUnzip will be explained below. The improvements are made from such four aspects as obtaining entrance information of the file, buffering of entrance information of the file, and separation of obtaining entrance information of the file from actual decompression flow, and division of operation granularity of decompression flow. operated in the compressed file must have been designated; 2 Obtain the entrance information of the designated file from the compressed file by searching from current file information visiting location in single direction. If current file information visiting location is after the location of related information of the file to be visited, the searching will be started from the location of related information of the first file. After obtaining the entrance information of this file, add such entrance information to the buffer; 3 Read a segment of file data with specified length, and decrypt and decompress such data and perform crc32 verification; 4 Determine whether other files in the compressed file need to be processed or not, if yes, this file is the designated file in subsequent flows, otherwise, destroy the decompression handle and exit the processing flow. 5. Test The decompression program realized is tested on intelligent cell phone below. 5.1 Test conditions Model of the intelligent cell phone is CoolPAD E200, its memory capacity is 128 MB RAM and 256 MB ROM,and its firmware is brew The source data for this test is a compressed file test.zip which is 778,662Byte, and ordering of the information of the files contained in the compressed file is as shown in table 1: Table 1. List of files in the compressed file used for the test File name Size before decompression (Byte) file1.txt 878 2,571 File2.txt 5,027 18,387 File3.txt 10,208 24,725 File4.txt 762,375 1,106,536 File5.txt Decompression test Size after decompression (Byte) Fig 5. Decompression flow of optimized LiteUnzip 1 Determine whether the entrance information of the designated file has been obtained and has existed in the program buffer; the determination here is performed to the designated file, and before which, the file to be File name Table 2. Results of LiteUnzip decompression test Time Maximum Decompr Total required for decompres ession decompressi obtaining sion time times on time (ms) description for single (number information block of of blocks) of the file data (ms) (ms) file1.txt File2.txt File3.txt File4.txt 1 11, ,131 File5.txt

6 Note: if the time required in the table is 0, it means that the time of this procedure is too short and less than 1ms. The program logic designed for the test is: to determine whether file file5.txt is the compressed file, if it is, decompress all the files in the compressed file test.zip in turns. 1. Use the function of LiteUnzip directly to realize the decompression function in the test application program. The test results is shown in table Use the optimized LiteUnzip to realize the decompression function in the test application program. The test results is shown in table 3. Table 3. Results of optimized LiteUnzip decompression test File Time required for obtaining description Maximum decompre ssion time for single Decompre ssion times (number of blocks) Total decompress ion time (ms) information of the file (ms) block of data (ms) file1.txt File2.txt File3.txt File4.txt ,140 File5.txt Note: if the time required in the table is 0, it means that the time of this procedure is too short and less than 1ms. 5.3 Test results analysis For brew platform, the UI thread can only be run after current decompression thread is suspended, then the time required for obtaining description information of the file and decompression time for a single block is the time of blocking of UI thread. According to table 2, LiteUnzip decompresses all the data of the whole file at one time before optimizing, the decompression time for a single block of data is long when decompressing large file. While the decompression time for a single block of data is short when using the optimized LiteUnzip for decompression. Therefore, the decompression program designed above can enable timely response to UI events and other emergency services. Comparing table 2 with table 3, it is obviously that the time required for obtaining the description information of each file in turns increases gradually before optimizing LiteUnzip, while the time required for obtaining the description information of each file is basically the same after optimizing LiteUnzip. It is obviously that execution efficiency of obtaining description information of the file is improved. The time for obtaining description information of the file file5.txt after optimizing LiteUnzip is much shorter than that before the optimization. This is because the file file5.txt should be checked first for existence before decompressing the compressed file test.zip, during which the description information of file file5.txt should be obtained, and the optimized LiteUnzip will buffer the description information. It is predictable that when files in the compressed file are massive, the time for generating the description information of files at the back of the compressed file in order will be long (the worst situation is when current file visiting location is the first file), and such buffering mechanism can improve decompression efficiency remarkably. 6. Conclusion This decompression program solves the problems of low efficiency, serious blocking of other emergency services when the rich media application programs in mobile terminals with limited processing capacities decompress massive compressed files. Through rational arrangement of time-consuming processing, obtained data can be reused as far as possible, long processing procedure can be divided into several smaller execution processes, and better experiences can be provided to the users. 7. References [1] The future development trend of mobile Internet [OL]. aencysxyant [2] Ji-Xian Zhang, LuoLei. An embedded rich media interactive technology based on the expansion of the SVG Tiny 1.2[C] //Proceedings of the International Conference on Computer Science and Software Engineering, 2008: [3] Ji-Xian Zhang, Lei Luo. A method for improving interaction of embedded rich media application[c] //Proceedings of the International Conference on Advanced Computer Theory and Engineering, 2008: [4] Principle and Realization of Compression of ZIP(lz77 compression algorithm)[ol]. [5] Symbian OS [OL]. [6] Steve Babin, Developing Software for Symbian OS: An Introduction to Creating Smartphone Applications in C++ [M]. Beijing: Posts & telecom press, 2006: [7] 3GPP. TS26.142V Dynamic and Interactive Multimedia Scenes(Release 8)[S]. 3GPP 2008 [8] Hong-Wei Cao, Advanced and proficient in BREW [M]. Beijing: Publishing house of electronic Industory, 2009: [9] Dufourd J C, Avaro O, Concolato C. An MPEG standard for rich media services[j]. IEEE Transactions on Multimedia, 2005, 12(4): [10] Jeff Glatt, LiteZip/LiteUnzip[OL]