Filesystem Fundamentals

The Story So Far

(Almost) everything we've discussed so far occurs in main memory (RAM):

• The PCB and TCB are stored in main memory.
• Virtual address spaces are mapped onto physical address spaces.
• The heap/stack/SDS are part of the process's virtual address space

RAM is nice! It's relatively speedy, and you can store a lot of stuff in there.

....but it's got a major drawback.

The Story So Far

To solve this problem, we introduce stable storage (disks). These devices retain data even after the power to them has been shutoff (i.e. they are persistent).

Now we can turn off the computer without losing important data!

So where are we now?

(Almost) everything we've discussed so far occurs in main memory (RAM):

• The PCB and TCB are stored in main memory.
• Virtual Address spaces are mapped onto physical address spaces.
• The heap is part of the process's virtual address space

Now we've added persistent storage which is large, very slow, and order-dependent.

Today's Questions

• What makes a good filesystem?
   

• How should applications interact with the filesystem?
   

• How do we organize a file on-disk?
   

• How can we present files in a way that makes sense to users?

Goals in Filesystem Design

What makes a filesystem good?

Speed

Unfortunately, we cannot always avoid paying the disk access cost (if we could, we wouldn't need the disk!)

DISK ACCESS

But we can use lots of common systems design tricks to make sure this doesn't affect us too badly!

• Minimize required accesses

• Cache everything

• Interleave I/O with computation

Remember: disk access is  slow !

Unfortunately, we cannot always avoid paying the disk access cost (if we could, we wouldn't need the disk!)

Reliability

Reliability

Our filesystem must provide tools to recover from a crash!

Otherwise we could be irrecoverably stuck in an invalid state.

Usability

The filesystem consists of raw block numbers. User programs are responsible for keeping track of which blocks they use, and for making sure they don't overwrite other program's blocks.

If the filesystem isn't easy to use, users will try to implement their own filesystem (usually poorly)

For example, the following filesystem is easy to implement, and is as fast and as consistent as the user chooses to make it:

Usability

A file is named by the a hash of its contents. All files are in the root directory. Filenames cannot be changed.

Who in their right mind is going to use this??

Filesystem: Is it good?

When analyzing a potential filesystem design, we'll use these three criteria to determine how much we like it:

Speed:

Reliability:

Usability:

How fast is this design? How many disk accesses do we need in order to perform common operations?

Will this design cause application programmers to tear their hair out? Does it require some deep knowledge of the system or is that abstracted away?

Can this design become corrupted if the computer fails (e.g. through sudden power loss), or if small pieces of data are damaged? Can it be recovered? How fast is the recovery procedure?

Today, we mostly focus on speed and usability. We'll talk about reliability next time!

An Aside: Bits, Bytes, Sectors, and Blocks

Sector: Smallest unit that the drive can read, usually 512 bytes large.

Last Time: Sectors

File Design And Layout

Files Are Stored as Data and Metadata

Metadata for all files is stored at a fixed location (something known by the OS) so that they can be accessed easily.

Data is the stuff the user actually cares about. It consists of sectors of data placed on disk.

Examples: file owner, file size, file permissions, creation time, last modified time, location of data blocks.

META

DATA

DATA

Logical layout of a file (not necessarily how it's placed on disk!)

Files Are Stored as Data and Metadata

DATA

Logical layout of a file (not necessarily how it's placed on disk!)

For now, we'll focus on how the file data is laid out on disk.

Assume we already know where the metadata is.

Evaluating File Layouts

Speed and Usability

Evaluating File Layouts

Speed and Usability

We have to allocate some of these data blocks to hold a file.

How do we choose?

Contiguous Allocation

• Files are allocated as a contiguous (adjacent) set of blocks.
• The only location information needed in the file header is the first block and the size.
• How do we keep track of free blocks and allocate them to files?

Contiguous Allocation: Access

How many disk reads do we need to access a particular block?

Contiguous Allocation: Access

How many disk reads do we need to access a particular block?

CPU

How many disk reads to access the first block?

(We always start with the block# of the file header)

Contiguous Allocation: Access

How many disk reads do we need to access a particular block?

CPU

H

How many disk reads to access the first data block?

(We always start with the block# of the file header)

• Read header block
                                                                           

Contiguous Allocation: Access

How many disk reads do we need to access a particular block?

CPU

H

How many disk reads to access the first data block?

(We always start with the block# of the file header)

• Read header block
• Find address of first data block
                                                                               

Contiguous Allocation: Access

How many disk reads do we need to access a particular block?

CPU

H

How many disk reads to access the first data block?

(We always start with the block# of the file header)

• Read header block
• Find address of first data block
• Read first data block
                                                                               

Contiguous Allocation: Access

How many disk reads do we need to access a particular block?

What if I want to do random access?

Let's say I want to read only the third block.

Contiguous Allocation: Assessment

This method is very simple (this is good!)

How fast is sequential access?

How fast is random access?

What if we want to grow a file?

How bad is fragmentation?

Linked Allocation

• The file is stored as a linked list of blocks.
• In the file header, keep a pointer to the first and last block allocated.
• In each block, keep a pointer to the next block.

Linked Allocation: Access

CPU

How many disk reads to access the first block?

(We always start with the block# of the file header)

• Read header block
                                                                         

Linked Allocation: Access

CPU

How many disk reads to access the first block?

(We always start with the block# of the file header)

• Read header block                                                                                                             

H

Linked Allocation: Access

CPU

How many disk reads to access the first block?

(We always start with the block# of the file header)

• Read header block
• Find address of first block
                                                                         

H

Linked Allocation: Access

CPU

How many disk reads to access the first block?

(We always start with the block# of the file header)

• Read header block
• Find address of first block
• Read first block
                                                                        

H

1

Linked Allocation: Access

CPU

How many disk reads to access the first block?

(We always start with the block# of the file header)

• Read header block
• Find address of first block
• Read first block
                                                                         

H

1

What if I want to read the second block from here? (sequential access)

Linked Allocation: Access

CPU

How many disk reads to access the first block?

(We always start with the block# of the file header)

• Read header block
• Find address of first block
• Read first block
• Find address of second block using data in first block
• Read second block
                                                                         

2

1

Linked Allocation: Access

CPU

How many disk reads to access the third block?

(We always start with the block# of the file header)

Random Access

Linked Allocation: Access

CPU

How many disk reads to access the third block?

(We always start with the block# of the file header)

Random Access

• Read header block                                                                     

H

Linked Allocation: Access

CPU

How many disk reads to access the third block?

(We always start with the block# of the file header)

Random Access

• Read header block
• Find address of first block (oh no)
• Read first block                                                                     

H

1

Linked Allocation: Access

CPU

How many disk reads to access the third block?

(We always start with the block# of the file header)

Random Access

• Read header block
• Find address of first block (oh no)
• Read first block
• Find address of second block in first block (oh no oh no)                                                                 

H

1

Linked Allocation: Access

CPU

How many disk reads to access the third block?

(We always start with the block# of the file header)

Random Access

• Read header block
• Find address of first block (oh no)
• Read first block
• Find address of second block in first block (oh no oh no)
• Read second block
• Find address of third block in second block (auuugh)                                                      

H

2

Linked Allocation: Access

CPU

How many disk reads to access the third block?

(We always start with the block# of the file header)

Random Access

• Read header block
• Find address of first block (oh no)
• Read first block
• Find address of second block in first block (oh no oh no)
• Read second block
• Find address of third block in second block (auuugh)    
• Read third block                                                 

H

3

Linked Allocation: Assessment

How fast is sequential access? Is it always good?

How bad is fragmentation?

What if we want to grow the file?

How fast is random access?

What happens if a disk block becomes corrupted? (what sort of problem is this?)

Example: FAT File System

File Allocation Table (FAT)

Started with MS-DOS (Microsoft, late 70s)

Descendants include FATX and exFAT

FAT File System

Advantages

Disadvantages

• Poor random access
  • Requires sequential traversal of linked blocks
• Limited Access Control
  • No file owner or group ID
  • Any user can read/write any file
• No support for hard links
• Volume and file size are limited
  • Example: FAT-32 is limited to 32 bits
  • Assuming 4KB blocks (fairly typical), filesystem cannot be larger than 2TB
  • Individual files cannot be larger than 4GB
• No support for transactional updates (more on this next time)

Direct Allocation

File header points to each data block directly (that's it!)

Direct Allocation: Assessment

How fast is sequential access? How about random access?

How bad is fragmentation?

What if we want to grow the file?

Does this support small files? How about large files?

What if other file metadata takes up most of the space in the header?

User-level Organization

Or: All About Directories

The Story So Far

We know how to get the data associated with a file if we know where its metadata (file header) is. We also know how to identify file headers (by their index in the file header array).

Great! Are we done?

We don't have the ability to name files right now. Which of our three main goals is this a failure in?

The Story So Far

To edit your shell configuration, open file 229601, unless you have Microsoft Word installed, in which case you need to edit file 92135113

We still don't have human-readable names or organization!

This is a big usability issue.

The Simpleton's Guide to File Organization

Use one name space for the entire disk.

• Use a special area of the disk to hold the directory info.
• Directory info consists of <filename, inode #> pairs.
• If one user uses a name, nobody else can.

The Simpleton's Guide to File Organization: Part 2

• Simple improvement: make a separate "special area" for each user.
• Now files only have to be uniquely named per-user.

(Yeah, it's not that great of an improvement)

user1's Directory

user2's Directory

Introducing: Directories

Directories are files that contain mappings from file names to inode numbers.

• The "special reserved area" we saw previously was an example of a directory, albeit a very primitive one.
• The inode number is called the inumber.
• Only the OS can modify directories
  • Ensures that mappings cannot be broken by a malicious user (referee role)
  • User-level code can read directories
• Directories create a name space for files (file names within the same directory have to be unique, but can have same filename in different directories)

Introducing: Directories

Note: the i# in a directory entry may refer to another directory!

The OS keeps a special bit in the inode to determine if the file is a directory or a normal file.

There is a special root directory (usually inumber 0, 1, or 2).

2B

Example directory with 16B entries

14B

So...how do you find the data of a file?

There's an infinite loop in here...or is there?

To find a file's inumber, read the directory that contains the file.

How does the filesystem service this syscall?

CPU

In previous examples, we started knowing the inumber of the file to read. This time, we don't have that.

But we do have....what?

CPU

• Read inode 2 (the root inode)                                                                                                                                                                              

i2

CPU

• Read inode 2 (the root inode)
  
• Use the inode to locate the data in the root directory                                                                                                                                                                             
  

i2

CPU

• Read inode 2 (the root inode)
• Use the inode to locate the data in the root directory
• Read the data in the root directory                                                                                                                                                                            

i2

i2

Contents of Block #1214

B 1214

CPU

• Read inode 2 (the root inode)
• Use the inode to locate the data in the root directory
• Read the data in the root directory
• See that our next path target is "home", which has i# = 11                                                                                                                                                                          

i2

i2

B 1214

Contents of Block #1214

CPU

• Read inode 2 (the root inode)
• Use the inode to locate the data in the root directory
• Read the data in the root directory
• See that our next path target is "home", which has i# = 11
• Read inode 11 (the inode for "/home")                                                                                                                                                                        

i11

B 1214

Contents of Block #1214

CPU

• Read inode 2 (the root inode)
• Use the inode to locate the data in the root directory
• Read the data in the root directory
• See that our next path target is "home", which has i# = 11
• Read inode 11 (the inode for "/home")
• Use the inode to locate the data in the "/home" directory                                                                                                                                                                        

i11

B 1214

Contents of Block #1214

CPU

• Read inode 2 (the root inode)
• Use the inode to locate the data in the root directory
• Read the data in the root directory
• See that our next path target is "home", which has i# = 11
• Read inode 11 (the inode for "/home")
• Use the inode to locate the data in the "/home" directory
• Read the data in the "/home" directory                                                                                                                                                                

i11

B 2772

Contents of Block #2772

CPU

• Read inode 2 (the root inode)
• Use the inode to locate the data in the root directory
• Read the data in the root directory
• See that our next path target is "home", which has i# = 11
• Read inode 11 (the inode for "/home")
• Use the inode to locate the data in the "/home" directory
• Read the data in the "/home" directory
• See that our next path target is "user1", which has i# = 6                                                                                                                                                          

i11

B 2772

Contents of Block #2772

CPU

• Read inode 2 (the root inode)
• Use the inode to locate the data in the root directory
• Read the data in the root directory
• See that our next path target is "home", which has i# = 11
• Read inode 11 (the inode for "/home")
• Use the inode to locate the data in the "/home" directory
• Read the data in the "/home" directory
• See that our next path target is "user1", which has i# = 6
• Read inode 6 (the inode for "/home/user1")                                                                                                                                                     

i6

B 2772

Contents of Block #2772

CPU

• Read inode 2 (the root inode)
• Use the inode to locate the data in the root directory
• Read the data in the root directory
• See that our next path target is "home", which has i# = 11
• Read inode 11 (the inode for "/home")
• Use the inode to locate the data in the "/home" directory
• Read the data in the "/home" directory
• See that our next path target is "user1", which has i# = 6
• Read inode 6 (the inode for "/home/user1")
• Use the inode to locate the data for "/home/user1"                                                                                                                                                      

i6

B 2772

Contents of Block #2772

CPU

• Read inode 2 (the root inode)
• Use the inode to locate the data in the root directory
• Read the data in the root directory
• See that our next path target is "home", which has i# = 11
• Read inode 11 (the inode for "/home")
• Use the inode to locate the data in the "/home" directory
• Read the data in the "/home" directory
• See that our next path target is "user1", which has i# = 6
• Read inode 6 (the inode for "/home/user1")
• Use the inode to locate the data for "/home/user1"
• Read the data for "/home/user1"                                                                                                                                               

i6

B 537

Contents of Block #537

CPU

• Read inode 2 (the root inode)
• Use the inode to locate the data in the root directory
• Read the data in the root directory
• See that our next path target is "home", which has i# = 11
• Read inode 11 (the inode for "/home")
• Use the inode to locate the data in the "/home" directory
• Read the data in the "/home" directory
• See that our next path target is "user1", which has i# = 6
• Read inode 6 (the inode for "/home/user1")
• Use the inode to locate the data for "/home/user1"
• Read the data for "/home/user1"
• We know what inode the file is at now!                                                                                                                                                 

i6

B 537

Record that the requested file has i# = 23 and end the search

Contents of Block #537

How many disk reads was that?

• Read inode 2 (the root inode)
• Use the inode to locate the data in the root directory
• Read the data in the root directory
• See that our next path target is "home", which has i# = 11
• Read inode 11 (the inode for "/home")
• Use the inode to locate the data in the "/home" directory
• Read the data in the "/home" directory
• See that our next path target is "user1", which has i# = 6
• Read inode 6 (the inode for "/home/user1")
• Use the inode to locate the data for "/home/user1"
• Read the data for "/home/user1"
• We know what inode the file is at now!                                                                                                                                                 

6 disk reads just to open the file

We didn't even try to read anything out of the file--that was just an open() call!

Simple optimization: cwd

We can't avoid this disk access...

What We Have

We now know how to do the following:

• Given a file header, find the data blocks of the file
• Given a file header, extend the file (if necessary)
• Find a file header from a human-readable path

More Usability

The UNIX Filesystem API

The UNIX Filesystem API

Armed with what we know about files and directories, let's take a look at the classic UNIX Filesystem API.

You already have an idea of what some of these functions do. Let's look at how some of them are implemented.

Filesystem API Structures

Think about the following scenario:

• Process A opens file F
• Process A reads 5 bytes from F
• Process B opens file F
• Process A reads 5 bytes from F
• Process B reads 5 bytes from F

Filesystem API Structures

"Open /var/logs/installer.log"

Filesystem API Structures

"Open /var/logs/installer.log"

File

Descriptor

On-Disk Data

User Memory

Per-Process Memory

Global System Memory

On Disk

struct file

struct inode

The user gets everything to the left of the thick line and interacts with it via system calls (since they can't directly edit system memory). The OS is responsible for updating things to the right of the line.

+syscall

Creates in-memory data structures used to manage open files.  Returns file descriptor to the caller.

On open(), the OS needs to:

• Check if the file is already opened by another process. If it is not:
  • Find the file
  • Copy information into the system-wide open file table
• Check protection of file against requested mode. If not allowed, abort syscall.
• Increment the open count (number of processes that have this file open).
   
• Create an entry in the process's file table pointing to the entry in the system-wide file table.
• Initialize the current file pointer to the start of the file.
• Return the index into the process's file table.
  • Index used for subsequent read, write, seek, etc. operations.

Close the file.

On close(), the OS needs to:

• Remove the entry for the file in the process's file table
• Decrement the open count in the system-wide file table
• If the open count is zero, remove the entry from the system-wide file table

Read designated bytes from the file

On read(), the OS needs to:

• Determine which blocks correspond to the requested reads, starting at the current position and ending at current position + num_bytes (note: where is current position stored?)
• Dispatch disk reads to the appropriate sectors
• Place the read results into the buffer pointed at by buffer

Creates a new file with some metatdata and a name.

On create(), the OS will:

• Allocate disk space (check quotas, permissions, etc).
• Create metadata for the file in the file header, such as name, location, and file attributes
• Add an entry to the directory containing the file

Creates a hard link--a user-friendly name for some underlying file.

On link(), the OS will:

• Add the entry to the directory with a new name
• Increment counter in file metadata that tracks how many links the file has

This new name points to the same underlying file!

Removes an existing hard link.

To delete() a file, the OS needs to:

• Find directory containing the file
• Remove the file entry from the directory
• Clear file header
• Free disk blocks used by file and file headers

The OS decrements the number of links in the file metadata. If the link count is zero after unlink, the OS can delete the file and all its resources.

Other Common Syscalls

• write() is like read(), but copies the buffer to the appropriate disk sectors
  
   
• seek() updates the current file pointer
  
   
• fsync() does not return (blocks) until all data is written to persistent storage. This will be important for consistency.

Does write() require us to access the disk?

How about seek()?

A. Yes, Yes

B. Yes, No

C. No, Yes

D. No, No

Summary

Without persistent storage, computers are very annoying to use.

Persistent storage requires a different approach to organizing and storing data, due to differences in its behavior (speed, resilience, request ordering). This leads naturally to the idea of a file system.

Big Picture

Big Picture

Data is stored on the disk as a bunch of blocks. A block is the smallest unit the filesystem can read/write. Blocks are identified by their order on the disk (e.g. #3124  is the block after #3123)

Big Picture

Data is stored on the disk as a bunch of blocks. A block is the smallest unit the filesystem can read/write. Blocks are identified by their order on the disk (e.g. #3124  is the block after #3123)

Big Picture

We need some way to describe how the data is organized. These are the metadata blocks.

Once you have the file metadata, you know everything you need to access the file.

Big Picture

We need some way to describe how the data is organized. These are the metadata blocks.

Once you have the file metadata, you know everything you need to access the file.

Big Picture

There are some in-memory structures that the OS uses to track what's happening with the filesystem (e.g. which files are open, synchronization tools).

Big Picture

There are some in-memory structures that the OS uses to track what's happening with the filesystem (e.g. which files are open, synchronization tools).

Big Picture

There are also some per-process pieces of information that need to be tracked in-memory.

Big Picture

There are also some per-process pieces of information that need to be tracked in-memory.

Big Picture

Finally, we need to worry about how a user program accesses all of this!

Big Picture

Finally, we need to worry about how a user program accesses all of this!

Big Picture

A user program cannot just punch through dual-mode execution to start overwriting kernel data structures directly (this would be very bad!)

Big Picture

Operating system provides syscalls which give a controlled way for a user process to request services from the kernel. The syscalls can modify system data (both on disk and in memory).

The application (user) thinks of files in terms of abstractions like "filenames" and "directories". The machinery of the syscall is responsible for mapping these high-level concepts down to particulars (e.g. "block #32750")

Food For Thought

The questions in the following slides are intended to provoke some thought and tie back what you've learned in this section to things you've studied previously in this course, or to really test your knowledge of what you learned in this unit.

• The observation that most files on a system are small was an empirical one. Why might this be generally true? What sort of usage patterns would lead to this? What sort of usage patterns might cause this to be violated?
   
• File headers are stored in a different location than the file data. However, this almost-certainly forces a disk head jump when accessing a file, since we need to read the header and then find its data (which is probably not located on the same sector). Why do we not store the file header in the same physical location as the data, to avoid this jump?
   
• Consider the plot in the "Evaluating File Layouts" slide. There is a plateau in the plot at 4096, which means there are a large number of files (perhaps 2% of the files on the entire system) that are exactly 4KB. What causes these files? (Hint: this plateau occurs on most systems--you can look on the department's Linux systems if you want to explore this further).
   
• The cwd optimization caches a directory's inode (and potentially its data) to avoid having to do disk lookups. However, if users access files at random, this optimization does nothing. What properties of typical usage are we exploiting by caching the cwd?

Food For Thought

• Why do we implement directories like special cases of files? What advantages are there to this? What disadvantages might occur due to this?
   
• Why do we return a file descriptor to the user instead of the full data structure associated with the open file?
   
• How many names/paths can a file have associated with it?