Log2fs or how to achieve IO/s

Transcription

1 fs or how to achieve IO/s Lazybastard.org September 24, 2010 fs or how to achieve IO/s

2 Just a bunch of random hacks Lazybastard.org September 24, 2010 Just a bunch of random hacks

3 Flash basics Fast random reads Fast somethat-random writes Erase before write Large eraseblocks fs or how to achieve IO/s

4 Blocks and Pages 4K 4K 4K 4K 4K 4K... 4K 4K 4K 4K 4K 4K 256K 256K... fs or how to achieve IO/s

5 Drais card PCIe x4 interface 1 FPGA 64 Flash chips fs or how to achieve IO/s

6 Drais card fs or how to achieve IO/s

7 Drais driver Simple MTD driver 64 queues for requests Does error correction Adds FIO interface fs or how to achieve IO/s

8 FIO interface Adds three new methods to struct mtd info fio read fio write fio erase fs or how to achieve IO/s

9 FIO interface fio read reads exactly 1 page, then calls fio->fi end io fs or how to achieve IO/s

10 FIO interface fio write writes exactly 1 page, then calls fio->fi end io fs or how to achieve IO/s

11 FIO interface fio erase erases exactly 1 block, then calls fio->fi end io fs or how to achieve IO/s

12 wait multiple wait multiple waits for N fios to complete fs or how to achieve IO/s

13 Read Performance Single threaded: 6800 IO/s or 27MB/s 4096 threads: IO/s 610MB/s Scales 22x fs or how to achieve IO/s

14 Write Performance Single threaded: 40 IO/s or 10MB/s 4096 threads: 1859 IO/s or 480MB/s Scales 46x fs or how to achieve IO/s

15 Read performance fs or how to achieve IO/s

16 Write performance fs or how to achieve IO/s

17 Compression in LogFS byte-precise packing indirect blocks contain pointers block headers contain compressed size fs or how to achieve IO/s

18 Alignment many blocks span a page boundary fs or how to achieve IO/s

19 Alignment fs or how to achieve IO/s

20 Alignment uncompressed and compressed blocks are mixed fs or how to achieve IO/s

21 Writes write header and compressed data to buffer occasionally flush buffer fs or how to achieve IO/s

22 Reads read header plus maximal blocksize to cache uncompress fs or how to achieve IO/s

23 Deletions Read header into cache Use compressed size for accounting fs or how to achieve IO/s

24 Cache Cache has a granularity of (MMU)PAGE SIZE fs or how to achieve IO/s

25 Cache Oops! fs or how to achieve IO/s

26 Deletions fs or how to achieve IO/s

27 Deletions fs or how to achieve IO/s

28 Don t mix uncompressed and compressed blocks Align uncompressed blocks fs or how to achieve IO/s

29 Move compressed size to indirect blocks...and a number of other fields...and remove (most) direct pointers from inodes fs or how to achieve IO/s

30 Venti Efficient way to store multiple identical copies Ideal for large universities Horrible for personal computers fs or how to achieve IO/s

31 VentiLog Add a block hashtable Check hashtable before writes Increment refcount when possible fs or how to achieve IO/s

32 BtrLog Add reference count to block pointers copyfile() becomes possible clones become possible fs or how to achieve IO/s

33 Birthday attack LogFS stores directory entries in a hash table. fs or how to achieve IO/s

34 Birthday attack Given N random numbers between 1 and M (N M), what is the probability of having two or more identical numbers? fs or how to achieve IO/s

35 Birthday attack 1 M! N! M N fs or how to achieve IO/s

36 Birthday attack fs or how to achieve IO/s

37 Birthday attack Given N random numbers between 1 and M (N M), what is the probability of having O or more identical numbers (O N O M)? fs or how to achieve IO/s

38 fs or how to achieve IO/s