Input and Output

Reference: Operating Systems, Fourth Edition,

William Stallings, Chapters 11, 12

             Nick Urbanik <nicku@nicku.org>
Copyright Conditions: GNU FDL (see http://www.gnu.org/licenses/fdl.html  )

A computing department

Input/Output

What is involved in

input/output? How does the operating system manage I/O?

I/O — Enormous Variety

Three main categories: Human readable: video displays, printers, keyboard,

mouse Machine readable: disk, cdrom, tape drives, sensors,

controllers, actuators Communications: network cards, modems

Differences in I/O Devices

Data rate: Speeds of i^{/}o devices vary greatly: Fast: Gigabit Ethernet — 10^{9} bps, Slow: keyboard — 10^{2} bps

Application: os Policies and supporting utilities required depend on application. Disks holding files need file management support. Disks for swap require virtual memory hardware and software.

Complexity of control: A printer has a simple control interface; a disk has a complex control interface.

Unit of transfer: Can transfer a stream of bytes (e.g., keyboard), or large blocks of data (e.g., disk i^{/}o)

Data representation: Different data encoding schemes

Error conditions: Type of errors, way they are reported vary greatly

Techniques of I/O

Programmed i^{/}o: cpu gives i^{/}o command; cpu polls in a loop waiting for i^{/}o to complete (busy waiting)

Interrupt-driven i^{/}o: cpu gives i^{/}o command, continues with some other work, then i^{/}o module sends interrupt to cpu when it is finished.

Direct memory access (dma): A dma controller moves data between memory and i^{/}o device independently of cpu. cpu simply tells dma controller where to move data, and how much, dma controller does the rest. dma controller interrupts cpu when finished.

What is I/O?

Involves transferring data to/from hardware, e.g. Printers, hard disks, mice, displays, cameras, networks,…

Hardware often has support for data transfer independent of main cpu, e.g., dma, cpu on i^{/}o device. cpu will have some involvement Disk I/O is very important, and most effort in improving i^{/}o in os goes into improving disk i^{/}o

I/O (usually) limits execution speed

Most processes perform a burst of:

cpu activity i^{/}o activity cpu activity i^{/}o activity …

Even with maximum priority, most processes are limited by the i^{/}o they perform. We say that most processes are I/O bound

If a process performs much computation, but little i^{/}o, it is said to be CPU bound

PC architecture

OSSI — Input/Output — ver. 1.1 – p. 8/50

I/O ports

Registers in i^{/}o interface, often grouped into: Control registers Status registers Input register Output register

Originally mostly connected to “i^{/}o bus” Use only in, out machine instructions Memory mapped i^{/}o ports more common now PCI cards use high addresses

Memory mapped I/O

i^{/}o ports can be mapped into the same address space as ram, rom. Full range of normal instructions can be used to access i^{/}o ports Fuller address space available

I/O mapped I/O — 1

OSSI — Input/Output — ver. 1.1 – p. 11/50

I/O mapped I/O — 2

The diagram may be just conceptual There is a logically separate data and address bus for i^{/}o ports Some hardware provides a separate bus dedicated to i^{/}o, e.g. pci

Limited special instructions perform input or output to port. On Intel architecture, only 2^{16} different port addresses available for isa devices

I/O interface

An I/O interface is a hardware circuit between some i^{/}o ports and their device controller Often connected to Programmable Interrupt controller through irq line Can have Custom i^{/}o interface, or General purpose i^{/}o interface

Examples of I/O interfaces

Custom interfaces

Specialised to a particular function Examples:

Keyboard Graphic display Disk Network General interfaces

Can plug in many types of device Examples:

Parallel port Serial port usb port pcmcia scsi

I/O controller

Hardware used by complex devices, e.g., disk controller Interprets high level commands received by interface through i^{/}o ports Converts these to complex operations on the hardware Simpler devices have no device controller, e.g., pic and pi^{/}t

Methods of I/O

There are three basic methods of i^{/}o: dma (Direct Memory Access) Interrupt-driven i^{/}o polling

dma

Used on PCI bus and ATA hard disks A special processor called a DMA controller (dmac) takes over from cpu Transfers data rapidly, no need to fetch instructions Called a bus master because it takes over the buses from cpu pc has five isa dma channels, and also special dma controllers built into pci controller

Interrupts

PC supports 15 hardware interrupts Each interrupt is a hardware connection from i^{/}o device to cpu Goes through a pi^{/}t (Programmable interrupt controller) module Interrupts have different priorities Notice that i^{/}o devices usually use interrupts

What is an interrupt? Example

Use serial port (uart) as an example uart = Universal Asynchronous Receiver/Transmitter = an ic Receives/transmits characters one bit at a time. Frame each char with start/stop bit

Interrupt example: UART — 2

When character is finally received by uart:

1. uart sends hardware signal to cpu

2. cpu determines whether priority higher than current task—if so,

3. cpu stores program counter (pc), some other registers on stack

4. Executes a subroutine called an isr (interrupt Service Routine) that allows cpu to copy character(s) from uart to buffer in ram

5. cpu restores PC, registers, continues from where it left off.

Polling

i^{/}o device has a register cpu checks a flag in this register in a loop. cpu may be busy, unable to do other tasks, so waste time But simpler than other i^{/}o methods

Input/Output: how?

In the old days, (ms-dos), application programs would write directly to input/output hardware

Examples: hardware = registers in uart, or dmac, other registers in i^{/}o device

Problems: if program has bug, whole system hangs: unreliable

What if two processes want the same resource?

I/O: how now? — 2

Each hardware device has a device driver. High level command → Device driver → low level actions Application uses same high level commands with different hardware Provides modularity

Advantages of device drivers

Reliability os controls access to the hardware Modular system possible, e.g. Linux has dynamically loading modules Separate kernel from low-level details of hardware. In Linux, device drivers part of the kernel distribution.

Installing a driver (e.g. for nic)

To install the driver for a 3Com 3c59x card in Linux, simply do:

modprobe 3c59x

To install it permanently for eth0,

   Edit /etc/modules.conf

Add the line:

   alias eth0 3c59x

Restart networking on that nic with ifup eth0 Do not reboot!

Command lsmod lists all installed drivers; drivers are usually loadable modules.

Block and character devices

Block devices include disks Provide random access Data transferred in fixed-sized blocks

Character devices include serial devices, printers Transfer data in variable sized chunks Provide sequential access

I/O Buffering — 1

Buffering provides two benefits: Improves performance Allows process to be swapped out of ram while os performs i^{/}o for the process

I/O Buffering — 2

If transfer i^{/}o data directly to or from a slow device to a process, the memory page getting or providing the data must stay in ram

So os could not completely swap out this process

More efficient to use buffering:

perform input transfer before the data is required perform output sometime later than when request is made.

Single Buffer

os requests one block of data before it is needed — read ahead. i^{/}o device fills block Application finds data is there immediately it is required

Double Buffer

Use two memory buffers instead of one.

While os fills one buffer, user process empties the other.

Very important for process that continuously provides data, such as data acquisition, since data may be lost with a single buffer, as the input cannot be paused while emptying the buffer ready for new data

Case Study: dma and Double Buffering — 1

I wrote software some years ago to collect data, analyse the data, graph the data, and write the data to disk

Used dma on the isa bus (A Data Translation DT2821 data acquisition card)

The DT2821 used an interrupt and two dma channels

The device driver initialised the two dma channels with this information:

The direction of data transfer (read or write)

The address of the dma buffer

The size of the dma buffer

Case Study: dma and Double Buffering — 2

The device driver enables the dma controller The dma controller transfers data from DT2821 to the dma buffer

When dma buffer filled, dmac raises an interrupt

The Interrupt Service Routine (isr) starts the other dma channel, then re-programs the dma controller that has just finished.

Buffer in Process or Device Driver?

Should we put the buffers in the process that uses the data or should we put it in the device driver?

For most applications, should put buffers in device driver

Allows process to be swapped out during i^{/}o Avoids problems with special memory needs of special i^{/}o devices:

High speed dma requires the buffer to be

continuous memory, not pages scattered about in ram

Main Effects of Buffering

Device files — 1

Smooth out peaks of i^{/}o demand Allow processes to be swapped in and out, regardless of state of i^{/}o for the process In Linux and Unix, most i^{/}o devices are accessed through a device file

These are in the /dev directory.

Why?

Allows device access to be managed using file access permissions Provides a uniform access method to all i^{/}o devices, similar to those used to access files

Example: a temperature controller device file may give the current temperature when read it

Example: the same write() system call can write data to an ordinary file or send it to a printer by writing to the /dev/lp0 device.

There are two types of device files as mentioned in slide 26: block and character

Each device file has two numbers:

Major number Minor number

These two numbers, and the type, uniquely identify a device and its device file.

$ ls -l /dev/hd[a-d]
brw-rw----     1 root      disk        3,   0 Oct 22 22:53 /dev/hda
brw-rw----     1 root      disk        3,  64 Aug 31  2002 /dev/hdb
brw-rw----     1 root      disk       22,   0 Aug 31  2002 /dev/hdc
brw-rw----     1 root      disk       22,  64 Aug 31  2002 /dev/hdd

Note: major number is on the left, minor number is on the right; for /dev/hdd, major is 22, minor is 64 How do I know what device numbers my hardware should have?

see the file devices.txt that is part of the kernel source code:

$ rpm -ql kernel-doc | grep '/devices.txt'
/usr/share/doc/kernel-doc-2.4.18/devices.txt

How do I create a device file if it is missing (for eaxample, on a rescue disk?)

Use the mknod command. See also the command /dev/MAKEDEV, which also has its manual page.

Monitoring I/O performance

Both vmstat and procinfo provide information about i^{/}o.

Also sar -b

iostat gives details of data transfers to/from each hard disk The command hdparm -tT gives some basic performance parameters for your hard disks. hdparm and hdparm -i gives other info about your hard disks.

netstat -i, ifconfig and ip -statistics

-statistics link give details of network performance

Types of hard disk

Currently have two choices: ata133, or scsi. ata is cheaper, now reasonable performance compared with scsi for a small number of disks. A cost effective method is to:

add a number of ide controllers to a server, Add one disk to each ide channel Use software raid for reliability & speed

scsi controller has processor to reduce load on main cpu, ata adds load to main cpu Coming soon: serial ata and serial scsi, reducing size of cables, increasing data rate (early models are already on the market)

OSSI — Input/Output — ver. 1.1 – p. 39/50

Getting info about I/O devices

Can list all pci devices with lspci Can list all devices with lsdev List all modules that are currently loaded into kernel

with lsmod

The /proc file system contains all you ever wanted to know about your hardware and os See which hard disks are the busiest with iostat

Virtual Filesystem

Linux supports many file systems:

Filesystems for Linux: Ext2, Ext3, ReiserFS Filesystems for other Unixs: sysv, ufs, minix, Veritas VxFS Microsoft filesystems: ms-dos, vfat, ntfs Filesystems for cds and dvds: iso9660, Universal Disk Format (udf) hpfs (os2), hfs (Apple), affs (Amiga), adfs (Acorn) Journalling filesystems from elsewhere: jfs from ibm, xfs from sgi

Same commands and system calls work with all! How does it all work?

OSSI — Input/Output — ver. 1.1 – p. 41/50

Virtual Filesystem (vfs) — 2

vfs is a kernel software layer

handles all system calls to a file system

Provides a common interface to all filesystems

Also handles network filesystems, including nfs, smb (used in Microsoft networking), ncp (Novell)

Also handles special or “virtual” file systems, such as the /proc filesystem, and supports block devices such as /dev/loop0 which allow mounting a file as if it were a disk partition.

Volume Managers

A Volume Manager is a software layer built into an os that allows logical disk partitions to be created from physical disks.

Allows logical partitions to be dynamically resized (without rebooting the computer)

Very useful for large systems with rapidly changing data

Examples: Logical Volume Manager (lvm) from Sistina on Linux, a standard part of the kernel

Can be built on top of software raid, a good choice

Microsoft Dynamic Disks (see http://www.winnetmag.com/
WindowsServer2003/Index.cfm?ArticleID=27085)

Redundant Array of Independent Disks

raid

Allows disks or disk partitions to be combined to appear to be one disk

Aims:

Performance Increase — because data can be transferred from many disks in parallel Redundancy — raid above “level 0” allows one or more disks to fail without losing any data: system continues to be available

Can be implemented in hardware or software Software raid earned a bad name on Microsoft os, but reliable and efficient on Linux

Example: raid and lvm on Linux Server

Have two ide controller cards, two ide interfaces on motherboard

Five 80 GB hard disks, one on each of the six ide interfaces, set as master,

Partition all hard disks the same way:

Example Use of raid and lvm— 2

OSSI — Input/Output — ver. 1.1 – p. 46/50

Example Use of raid and lvm— 3

Combine hda1, hdc1 into raid 1, specify as the /boot partition: md0

install grub into Master Boot Record of both hda and hdc so can still boot if either fails

Combine hde1, hdg1, hdi1 into raid 5 for the swap partition: md1

Combine hda2, hdc2, hde2, hdg2, hdi2 into raid 5

for the root partition /: md2

Combine hda3, hdc3, hde3, hdg3, hdi3 into raid 5

for large flexible lvm storage for /home and /var: md3

Example Use of raid and lvm— 4

Run pvcreate on /dev/md3 to initialise the disks ready to hold lvm information

Use vgcreate to create a logical volume group main

Use vgextend to add the raid device md3 to the volume group

Use lvcreate to create two logical volumes, one for /var and the other for /home.

Format the two logical volumes and start using them.

In the future, if /var is full and /home is not as full, then can dynamically resize both partitions, removing storage from the /home logical volume, and add to the /var logical volume, without needing to reboot or shut down the server

Summary of raid, Volume Managers

The use of raid provides two benefits:

Higher performance (in example, because can read or write to five disks at once through five separate ide interfaces) Redundancy: if any disk fails, server can continue to operate normally, (with reduced disk performance), without data loss, until disk is replaced.

The use of a volume manager, such as lvm on top of raid, adds the benefit of flexibility, allowing system administrator to resize logical volumes, add new disks, re-arrange storage without disrupting service to the customers

References

Daniel P. Bovet and Marco Cesati, Understanding the Linux Kernel, Second Edition, O’Reilly, Dec 2002, Chapter 13: Managing I/O Devices.

William Stallings, Operating Systems, Fourth Edition, Prentice Hall, 2001, Chapters 11 and 12.

Andrew S. Tanenbaum, Modern Operating Systems, Prentice Hall, 2001, Chapter 5, Input/Output.

Gary J. Nutt, Operating Systems: A Modern Perspective, 2nd Edition, Lab Update, Addison-Wesley, 2002, Chapter 5, Device Management.

LVM HOWTO,

http://tldp.org/HOWTO/LVM-HOWTO/