CSE120 Principles of Operating Systems. Prof Yuanyuan (YY) Zhou I/O & File Systems

Transcription

1 CSE120 Principes of Operating Systems Prof Yuanyuan (YY) Zhou I/O & Fie Systems

2 Fie Systems First we discuss properties of physica disks Structure Performance Scheduing Then we discuss how we buid fie systems on them Fies Directories Sharing Protection Fie System Layouts Fie Buffer Cache Read Ahead 2 11/12/18 CSE 120

3 Disks and the OS Disks are messy physica devices: Errors, bad bocks, missed seeks, etc. The job of the OS is to hide this mess from higher eve software Low-eve device contro (initiate a disk read, etc.) Higher-eve abstractions (fies, databases, etc.) The OS may provide different eves of disk access to different cients Physica disk (surface, cyinder, sector) Logica disk (disk bock #) Logica fie (fie bock, record, or byte #) 3 November 12, 2018 CSE 120

4 Magnet Disk Structure Disk components Patters Surfaces Tracks Sectors Cyinders Arm Heads 4 11/12/18 CSE 120

5 Another View of Disk 5 11/12/18 CSE 120

6 Disk Interaction Specifying disk requests requires a ot of info: Cyinder #, surface #, track #, sector #, transfer size Oder disks required the OS to specify a of this The OS needed to know a disk parameters Modern disks are more compicated Not a sectors are the same size, sectors are remapped, etc. Current disks provide a higher-eve interface (SCSI) The disk exports its data as a ogica array of bocks [0 N] Disk maps ogica bocks to cyinder/surface/track/sector Ony need to specify the ogica bock # to read/write But now the disk parameters are hidden from the OS 6 November 12, 2018 CSE 120

7 Disk Performance Disk request performance depends upon three steps Seek moving the disk arm to the correct cyinder 7 Depends on how fast disk arm can move (increasing very sowy) Rotation waiting for the sector to rotate under the head Depends on rotation rate of disk (increasing, but sowy) Transfer transferring data from surface into disk controer eectronics, sending it back to the host Depends on density (increasing quicky) Anaogy: Baggage caim at airports When the OS uses the disk, it tries to minimize the cost of a of these steps Particuary seeks and rotation November 12, 2018 CSE 120

8 Disk Scheduing (optiona materia) Because seeks are so expensive (miiseconds!), the OS tries to schedue disk requests that are queued waiting for the disk FCFS (do nothing) Reasonabe when oad is ow Long waiting times for ong request queues SSTF (shortest seek time first) Minimize arm movement (seek time), maximize request rate Favors midde bocks SCAN (eevator) Service requests in one direction unti done, then reverse C-SCAN Like SCAN, but ony go in one direction (typewriter) 8 November 12, 2018 CSE 120

9 9 11/12/18 CSE 120

10 Disk Scheduing (2) In genera, uness there are request queues, disk scheduing does not have much impact Important for servers, ess so for PCs Modern disks often do the disk scheduing themseves Disks know their ayout better than OS, can optimize better Ignores, undoes any scheduing done by OS 10 November 12, 2018 CSE 120

11 11 Soid State Disks

12 Advantages Reiabiity in portabe environments and no noise No moving parts Faster start up Does not need spin up Extremey ow read atency No seek time (25 us per page/4kb) Deterministic read performance The performance does not depends on the ocation of data

13 Disadvantage Cost significanty more per unit capacity 3$/GB vs. 0.15$/GB Limited write erase time writes for SLC (MLC is even fewer) high endurance ces may have an 1-5 miion But some fies sti need more Weaver eaving to spread writes a over the disk Sower write speeds because of the erase bocks are becoming arger and arger(1.5 ms per erase) For ow capacity fash SSDs, ow power consumption and heat production when in active use. High capacity SSDs may have significant higher power requirements

14 Typica read and write rates

15 Mixed writes and reads

16 Youtube Video SSD vs HD YXjw /12/18 CSE 120

17 Stages of I/O Request 17 11/12/18 CSE 120

18 Fie Systems Fie systems Impement an abstraction (fies) for secondary storage Organize fies ogicay (directories) Permit sharing of data between processes, peope, and machines Protect data from unwanted access (security) 18 November 12, 2018 CSE 120

19 Fies A fie is data with some properties Contents, size, owner, ast read/write time, protection, etc. A fie can aso have a type Understood by the fie system Bock, character, device, porta, ink, etc. Understood by other parts of the OS or runtime ibraries Executabe, d, souce, object, text, etc. A fie s type can be encoded in its name or contents Windows encodes type in name.com,.exe,.bat,.d,.jpg, etc. Unix encodes type in contents Magic numbers, initia characters (e.g., #! for she scripts) 19 November 12, 2018 CSE 120 Lecture 12 Fie Systems

20 Basic Fie Operations Unix creat(name) open(name, how) read(fd, buf, en) write(fd, buf, en) sync(fd) seek(fd, pos) cose(fd) unink(name) NT CreateFie(name, CREATE) CreateFie(name, OPEN) ReadFie(hande, ) WriteFie(hande, ) FushFieBuffers(hande, ) SetFiePointer(hande, ) CoseHande(hande, ) DeeteFie(name) CopyFie(name) MoveFie(name) 20 November 12, 2018 CSE 120 Lecture 12 Fie Systems

21 Fie Systems Fie systems Impement an abstraction (fies) for secondary storage Organize fies ogicay (directories) Permit sharing of data between processes, peope, and machines Protect data from unwanted access (security) 21 11/12/18 CSE 120

22 Directories Directories serve two purposes For users, they provide a structured way to organize fies For the fie system, they provide a convenient naming interface that aows the impementation to separate ogica fie organization from physica fie pacement on the disk Most fie systems support muti-eve directories Naming hierarchies (/, /usr, /usr/oca/, ) Most fie systems support the notion of a current directory Reative names specified with respect to current directory Absoute names start from the root of directory tree 22 11/12/18 CSE 120

23 Directory Internas A directory is a ist of entries <name, ocation> Name is just the name of the fie or directory Location depends upon how fie is represented on disk List is usuay unordered (effectivey random) Entries usuay sorted by program that reads directory Directories typicay stored in fies Ony need to manage one kind of secondary storage unit 23 11/12/18 CSE 120

24 Basic Directory Operations Unix Directories impemented in fies Use fie ops to create dirs C runtime ibrary provides a higher-eve abstraction for reading directories opendir(name) readdir(dir) seekdir(dir) cosedir(dir) NT Expicit dir operations CreateDirectory(name) RemoveDirectory(name) Very different method for reading directory entries FindFirstFie(pattern) FindNextFie() 24 11/12/18 CSE 120

25 Path Name Transation Let s say you want to open /one/two/three What does the fie system do? Open directory / (we known, can aways find) Search for the entry one, get ocation of one (in dir entry) Open directory one, search for two, get ocation of two Open directory two, search for three, get ocation of three Open fie three Systems spend a ot of time waking directory paths This is why open is separate from read/write OS wi cache prefix ookups for performance /a/b, /a/bb, /a/bbb, etc., a share /a prefix 25 11/12/18 CSE 120

26 Fie Sharing Fie sharing has been around since timesharing Easy to do on a singe machine PCs, workstations, and networks get us there (mosty) Fie sharing is incrediby important for getting work done Basis for communication and synchronization Two key issues when sharing fies Semantics of concurrent access What happens when one process reads whie another writes? What happens when two processes open a fie for writing? Protection 26 11/12/18 CSE 120

27 Protection Fie systems impement some kind of protection system Who can access a fie How they can access it More generay Objects are what, subjects are who, actions are how A protection system dictates whether a given action performed by a given subject on a given object shoud be aowed You can read and/or write your fies, but others cannot You can read /etc/motd, but you cannot write it 27 11/12/18 CSE 120

28 Representing Protection Access Contro Lists (ACL) For each object, maintain a ist of subjects and their permitted actions Capabiities For each subject, maintain a ist of objects and their permitted actions Subjects Objects /one /two /three Aice rw - rw Bob w - r Charie w r rw Capabiity ACL 28 11/12/18 CSE 120

29 Rea ife anaogy In rea ife, which one is a capabiity ist? Keys We have a key to a room, we can open the room In rea ife, what is equivaent to an access contro ist? ID-based ocks: you can enter the room if your ID/fingerprint/face is stored in the set of user who can have access to this room 29 11/12/18 CSE 120

30 Linux Access Rights Mode of access: read, write, execute Three casses of users RWX a) owner access 7 Þ RWX b) group access 6 Þ RWX c) pubic access 1 Þ For a particuar fie (say game) or subdirectory, define an appropriate access. owner group pubic chmod 761 game

31 Issues with Linux Just a singe owner, a singe group and the pubic Pro: Compact enough to fit in just a few bytes Con: Not very expressive Access Contro List: This is a per-fie ist that tes who can access that fie Pro: Highy expressive Con: Harder to represent in a compact way

32 Windows ACLs

33 Googe Drive/Docs Is ACL of Googe Drive/Doc simiar to Windows or Linux? 33 11/12/18 CSE 120

34 Fie System Layout How do fie systems use the disk to store fies? Fie systems define a bock size (e.g., 4KB) Disk space is aocated in granuarity of bocks A Master Bock determines ocation of root directory Aways at a we-known disk ocation Often repicated across disk for reiabiity A free map determines which bocks are free, aocated Usuay a bitmap, one bit per bock on the disk Aso stored on disk, cached in memory for performance Remaining disk bocks used to store fies (and dirs) There are many ways to do this 34 11/12/18 CSE 120

35 Disk Layout Strategies Fies span mutipe disk bocks How do you find a of the bocks for a fie? 1. Contiguous aocation Like memory Fast, simpifies directory access Infexibe, causes fragmentation, needs compaction 2. Linked structure Each bock points to the next, directory points to the first Good for sequentia access, bad for a others 3. Indexed structure (indirection, hierarchy) An index bock contains pointers to many other bocks Handes random better, sti good for sequentia May need mutipe index bocks (inked together) 35 11/12/18 CSE 120

36 Contiguous Aocation Aocate fies contiguousy on disk

37 Contiguous Aocation Pros: Simpe: state required per fie is start bock and size Performance: entire fie can be read with one seek Cons: Fragmentation: externa is bigger probem Usabiity: user needs to know size of fie Hard for a fie to dynamicay grow Used in CDROMs, DVDs

38 Linked List Aocation Each fie is stored as inked ist of bocks First word of each bock points to next bock Rest of disk bock is fie data

39 Linked List Aocation Pros: No space ost to externa fragmentation Disk ony needs to maintain first bock of each fie Cons: Random access is costy Overheads of pointers.

40 MS-DOS fie system Impement a inked ist aocation using a tabe Caed Fie Aocation Tabe (FAT) Take pointer away from bocks, store in this tabe

41 Pros: FAT Discussion Entire bock is avaiabe for data Random access is faster than inked ist. Cons: Many fie seeks uness entire FAT is in memory For 20 GB disk, 1 KB bock size, FAT has 20 miion entries If 4 bytes used per entry Þ 80 MB of main memory required for FS

42 Indexed Aocation Index bock contains pointers to each data bock Pros? Cons?

43 UFS - Unix Fie System

44 Unix Inodes Unix inodes impement an indexed structure for fies Aso store metadata info (protection, timestamps, ength, ref count ) Each inode contains 15 bock pointers First 12 are direct bocks (e.g., 4 KB bocks) Then singe, doube, and tripe indirect (Metadata) (1) (2) (3) 44 11/12/18 CSE 120

45 Unix inodes If data bocks are 4K First 48K reachabe from the inode Next 4MB avaiabe from singe-indirect Next 4GB avaiabe from doube-indirect Next 4TB avaiabe through the tripe-indirect bock Any bock can be found with at most 3 disk accesses

46 Youtube Video Unix inode: A Windows FAT, FAT32 EPUYuQc 46 11/12/18 CSE 120

47 Impementing Directories When a fie is opened, OS uses path name to find dir Directory has information about the fie s disk bocks Whoe fie (contiguous), first bock (inked-ist) or I-node Directory aso has attributes of each fie Directory: map ASCII fie name to fie attributes & ocation 2 options: entries have a attributes, or point to fie I-node

48 Shared Fies If B wants to share a fie owned by C One Soution: copy disk addresses in B s directory entry Probem: modification by one not refected in other user s view

49 Soft vs Hard Links Fie name Inode# Inode Foo.txt 2433 Hard.nk 2433 Inode #2433

50 Hard vs Soft Links Soft.nk /path/to/foo.txt Inode # and then redirects to Inode #2433 at open() time.. Foo.txt 2433 Inode #2433

51 Managing Free Disk Space 2 approaches to keep track of free disk bocks Linked ist and bitmap approach

52 Unix Inodes and Path Search Unix Inodes are not directories Inodes describe where on the disk the bocks for a fie are paced Directories are fies Directories have inodes for themseves, too Directory entries map fie names to inodes To open /one, use Master Bock to find inode for / on disk Open /, ook for entry for one This entry gives the disk bock number for the inode for one Read the inode for one into memory The inode says where first data bock is on disk Read that bock into memory to access the data in the fie 52 11/12/18 CSE 120

53 Fie Buffer Cache Appications exhibit significant ocaity for reading and writing fies Idea: Cache fie bocks in memory to capture ocaity This is caed the fie buffer cache Cache is system wide, used and shared by a processes Reading from the cache makes a disk perform ike memory Even a 4 MB cache can be very effective Issues The fie buffer cache competes with VM (tradeoff here) Like VM, it has imited size Need repacement agorithms again (LRU usuay used) 53 11/12/18 CSE 120

54 Caching Writes 54 On a write, some appications assume that data makes it through the buffer cache and onto the disk As a resut, writes are often sow even with caching Severa ways to compensate for this write-behind Maintain a queue of uncommitted bocks Periodicay fush the queue to disk Unreiabe Battery backed-up RAM (NVRAM) As with write-behind, but maintain queue in NVRAM Expensive Log-structured fie system Aways write next bock after ast bock written Compicated 11/12/18 CSE 120

55 Read Ahead (Prefetch) 55 Many fie systems impement read ahead FS predicts that the process wi request next bock FS goes ahead and requests it from the disk This can happen whie the process is computing on previous bock Overap I/O with execution When the process requests bock, it wi be in cache Compiments the disk cache, which aso is doing read ahead For sequentiay accessed fies (e.g movies, songs) can be a big win Uness bocks for the fie are scattered across the disk Fie systems try to prevent that, though (during aocation) 11/12/18 CSE 120

56 Youtube video Ck 56 11/12/18 CSE 120

57 Summary Fies Operations, access methods Directories Operations, using directories to do path searches Sharing Protection ACLs vs. capabiities Fie System Layouts Unix inodes Fie Buffer Cache Strategies for handing writes Read Ahead 57 11/12/18 CSE 120