Chapter 12

File Management

Fall 2014

2014 by Greg Ozbirn, UTD, for

use with Stalling's 8th Ed. OS book

Files and File Systems

The file system is one of the most important parts of an
operating system.
Some desirable properties of files include:
Long-term existence: kept on secondary storage
Sharable between processes: for shared access
Structure: some file systems manage the internal
structure of a file, which can be organized in a way
suitable for particular applications

File Operations
Create
creates a new file and adds it to the directory
Delete
remove from the directory and destroyed
Open
allows a process to perform functions on the file
Close
the process is no longer allowed to operate on the file
Read
read all or a portion of the file
Write
add new data or change existing data in the file
3

File Structure
Field:
Basic element of data
Contains a single value
Characterized by its length and data type
Can be fixed length or variable length
Record:
Collection of related fields
Treated as a unit
Example: employee record
Can be variable length if some of the fields are variable
length, or if the number of fields may vary
4

File Structure
File
Collection of similar records
Treated as a single entity
Have unique file names
May restrict access
Database
Collection of related data (one or more files)
Relationships exist among elements
Managed by separate DBMS

Typical File Operations

Retrieve_All to process the whole file

Retrieve_One selects a single record
Retrieve_Next move forward in file
Retrieve_Previous move backward in file
Insert_One may require adding at a particular position
Delete_One may need to preserve sequence
Update_One harder if length changes, preserve sequence
Retrieve_Few records matching a set of criteria

Unix Files
Unix files are simply streams of bytes and do not have
internal structure as far as the operating system is
concerned.
Some OSs such as mainframe systems provide various file
structures the application can use.

File Management System

Set of system software providing file services to users and
applications.
Users and applications access files through the file
management system.
Programmer does not need to develop file management
software for each application.

Objectives for a
File Management System
Meet the data management needs and requirements of the
user (described on next slide).
Guarantee that the data in the file are valid.
Optimize performance.
Provide I/O support for a variety of storage device types.
Minimize or eliminate the potential for lost or destroyed
data.
Provide a standardized set of I/O interface routines
Provide I/O support for multiple users.

Meeting User Requirements

(the first objective listed on previous slide)
Each user:
Should be able to create, delete, read, and change files.
May have controlled access to other users files.
May control what type of accesses are allowed to the
users files.
Should be able to restructure the users files in a form
appropriate to the problem.
Should be able to move data between files.
Should be able to back up and recover the users files in
case of damage.
Should be able to access the users files by using symbolic
names.
10

File System Architecture

The next slide illustrates a file system architecture.
The pieces involved are:
Device driver: communicates directly with hardware
device.
Basic file system: handles physical I/O.
Basic I/O supervisor: responsible for file I/O initiation and
termination, device selection, I/O scheduling, buffer
allocation.
Logical I/O: Enables users and applications to access
records. Provides record-oriented I/O capability.
Access method: access files based on their structure and
method of access.
11

12

File Management Functions

See illustration on next slide:
Identify and locate a selected file.
Use a directory to describe the location of all files plus
their attributes.
Enforce user access control (shared system).
Employ access method on the records.
Provide blocking for file I/O.
Block for output and unblock for input.
Manage secondary storage
Allocate files to available blocks.
Manage free storage of available blocks.
Scheduling of I/O requests
13

14

File Organization and Access

Criteria for choosing a file organization:
Rapid Access
Ease of update
Economy of storage
Simple maintenance
Reliability
There may be trade-offs between criteria.
Priority of criteria depends on how application will use the
file.
15

File Organization and Access

There are a large number of file organizations.
Most actual file organizations are related to one of these
five fundamental file organizations:
Pile
Sequential file
Indexed sequential file
Indexed file
Direct or hashed file

16

Pile

Data are collected in the order they arrive

Purpose is to accumulate a mass of data and save it
Records may have different structures or no structure.
Data access is by exhaustive search
May be useful to store data prior to processing

17

18

Sequential File

Fixed format used for records.

Records are the same length.
All fields the same (order and length).
Field names and lengths are attributes of the file.
One field is the key field
Uniquely identifies the record
Records are stored in key sequence
New records are placed in a log file or transaction file.
Log file is periodically merged with the master file.
Useful in batch processing.
19

20

Indexed Sequential File

File is like a sequential file with key and data.
Index provides a lookup capability to quickly reach the
vicinity of the desired record.
Greatly reduces the time required to access a single record,
while still providing sequential processing.
Additions can be handled by an overflow file, which can
be periodically merged with the main file.

21

22

Indexed File

Uses multiple indexes for different key fields.

Abandons sequential format with single key.
Records are accessed only by their index.
Records can be variable length.
Exhaustive index has pointers to every record.
Partial index only has pointers to records having the field
of interest.
Useful in applications such as an airline reservation system
that need to access particular records quickly, but rarely
need to process them sequentially.
23

24

Direct or Hash File

Turn key value into an address to support direct access of
records in the file.
Gives very rapid access to data.
Useful if records are only accessed one at a time.

25

B-Trees
Indexes are often implemented as B-trees.
The B-tree will have a large branching factor resulting in a
tree of low height.
This results in fewer disk accesses to traverse the tree in a
search.

26

Directories
The directory contains information about the files,
including attributes, location, and ownership.
The directory is itself a file, owned by the OS and
managed by the file management system.
The next slides give information typically found in the
directory for each file.

27

28

29

Directory Operations
Search: find a file in the directory.
Create file: add entry to directory when a file is created.
Delete file: remove entry from directory when file is
deleted.
List directory: all or a portion may be listed.
Update directory: change file attributes stored in directory.

30

Directory Structure
A tree-structure is used for storing the file directory.
At the top is the master directory, beneath it are each user
directory.
Every level may contain files and subdirectories.

31

Directory Naming
Users refer to files by their symbolic names.
Each name must be unique.
In a tree structure, the path to the file helps eliminate
duplicate name problems.
Most systems provide for a working directory, where file
names can be specified relative to it.
Users typically have a default, or home, directory upon
logging in.

32

File Sharing
In a multiuser system, files should be able to be shared
between users.
This presents two issues:
Access rights
Simultaneous access

33

File Sharing Access Rights

When sharing files, we have to have a way of controlling
access to the file.
Typically one user is the owner.
The owner has all of the access rights and may grant rights
to others.

34

File Sharing Access Rights

Access rights may be given to different classes of users.
Typical classes are:
Specific user
Group of users
All:

35

File Sharing Access Rights

Examples of access rights:
None: file is unknown to user.
Knowledge: user can determine file existence and owner.
Execution: can execute the file.
Reading: can read the file.
Appending: can only append to end of file.
Updating: can modify file contents.
Changing protection: can change access rights.
Deletion: can remove the file.
These rights may constitute a hierarchy, with higher level
rights including the lower level ones.
36

File Sharing - Simultaneous Access

When access is granted to append or update a file to more
than one user, the OS or file management system must
enforce discipline.
One approach is to lock the entire file when it is to be
updated.
Another approach is to lock individual records during
update.

37

Record Blocking
Users work with a logical view of a file as a set of records.
The I/O device works with the physical view of a file as a
set of blocks.
Blocking of records is performed when writing data,
unblocking is performed when reading data.

38

Record Blocking
Blocking considerations:
The larger the block, the more records that are passed in a
single I/O.
For sequential access this reduces I/Os, but may result in
unnecessary data transfer for random access.
Typically blocks are of fixed length to simplify the I/O
transfer.

39

Record Blocking
There are three blocking methods:
Fixed: fixed-length records. May cause internal
fragmentation.
Variable-length spanned: variable-length records
which may span blocks. Permits unlimited record size.
May require multiple I/Os.
Variable-length unspanned: variable-length records
which do not span blocks. May cause internal
fragmentation.

40

41

Secondary Storage Management

A file consists of a collection of blocks on the secondary
storage device.
The OS or file management system manages the blocks.
There are two management issues:
Allocating blocks to files.
Maintaining a list of free blocks.

42

File Allocation
File allocation issues:
Whether to allocate the maximum space required all at
once, or to allocate space in portions.
If portions are used, how big should a portion be?
If portions are used, how to keep track of the portions.

43

Preallocation versus Dynamic Allocation

Preallocation requires that the maximum file size be
declared and allocated when the file is created.
May create waste as users over-estimate required size to
ensure space needs are met.
Dynamic allocation allows portions of space to be given to
the file so that it may grow dynamically.

44

Allocation Portion Size

Two basic methods:
Large, variable-length contiguous portion.
Good performance.
May cause fragmentation.
Allocation table is simple, just need pointer to
beginning and length.
Small fixed-sized blocks.
Good flexibility.
Requires more complicated allocation table to keep
track of non-contiguous blocks.
45

File Allocation Methods

Three methods of file allocation:
Contiguous allocation a single contiguous set of blocks is
allocated to a file at the time of file creation. Thus, this is
a preallocation method of a variable-size portion. May
require periodic compaction due to fragmentation. Good
for sequential file access.
Chained allocation allocate individual blocks. Each
block contains a pointer to the next block.
Indexed allocation an index specifies the location of each
block.

46

47

48

49

50

51

52

Free Space Management

It is necessary to know what space is available in order to
perform allocations.
A disk allocation table keeps track of available space.
Techniques:
Bit tables: each bit represents a block
Chained free portions: free portions are chained
together.
Indexing: index keeps track of every free portion as if it
was a file.
Free block list: blocks are numbered and list of free
block numbers is maintained.
53

End of Slides

based on Stalling's official slides

54