CS6900 Independent study

1
CS6900 Independent study

• Bolin Hsu
• ltbolin_hsu_at_yahoo.comgt

2
Introduction

• Relationship of Linux kernel and Linux device
  drivers
• Implementation of a Linux device driver
• How to write your own device driver
• Bibliography

3
Concepts
4
Objective

• Let the users access the device by file system
  calls
• fdopen(/dev/myrd,O_RDWR)
• write(fd,buffer,sizeof(buffer))
• read(fd,buffer,N)
• close(fd)

5
Challenges

• How to connect the device file /dev/myrd to the
  device
• How to make the system calls access the device

6
Linking of device file and the device
7
Everything in UNIX is a file

• User interact with the device through filesystem
  functions
• open, close, read, write, ioctl, etc
• A device file or a filesystem
• Two interfaces to the device
• Interface to user device file
• Interface to kernel service routine

8
Device file concept

• Device file type
• Character (inode.type 3)
• Block (inode.type 4)
• Device file number
• Major device type (2 floppy disk 3 hard disk,
  etc)
• Minor device number
• Examples
• brw-rw---- 1 root disk 3, 0 May 19 2004 /dev/had

9
Creating the device file

• Create our ram disk device file
• Type is block
• Major number is 100
• Minor number is 0
• mknod /dev/ b 100 0
• Another option is to create device during device
  driver initialization.

10
Connecting system calls to the device driver
11
How Linux kernel uses device drivers
User applications
Libraries
User mode
Kernel mode
File subsystem
Buffer/page cache
Block device driver
Character device driver
Hardware control
12
Block I/O and Page I/O

• Two fundamental I/O type
• Block I/O bread()
• Page I/O brw_page()
• Block I/O read/write a single block. Kernel uses
  it to handle file system superblock and inode.
• Page I/O read/write a page in a file. Used in
  read/write system calls.

13
Page/buffer cache

• A cache is a RAM area
• The cache holds data which usually should be on a
  block device.
• I/O from the cache is faster than the device.
• Kernel can merge and schedule device I/O to
  improve system performance.

14
Sector, block, and buffer cache

• A sector is the basic data transfer unit of a
  block device
• A block is a collection of adjacent sectors
  transferred in one I/O request
• Every block device file has its own block size
• Every block has a corresponding buffer cache

15
How device request uses buffer cache
request
Buffer head
Buffer head
Buffer head
request
16
Device driver API calling sequence
open
close
read
write
device driver API
buffer cache calls
block device switch table
open
release
request
ioctl
check_media_change
Interrupt service ruotine
revalidate
Interrupt vector
device interrupt
17
How read() works

• The read() system call searches the page in the
  page cache. If the page is in the cache and
  valid, it is copied to the user buffer. If no
  valid page is found in the cache, the process
  issues a request for the page and suspend itself.
• When the page becomes available later, the kernel
  wakes up the suspended process. The process then
  copies the page to the user buffer.

18
Trace read() system call

• These functions are called in order
• read()
• sys_read()
• def_blk_fops.read generic_file_read()
• do_generic_file_read()if the page is found in
  the page cache, it is copied to the user buffer.
  If it is not in the cache, call next function.
  This function could suspend the calling process.
• def_blk_aops.readpage blkdev_readpage()
• block_read_full_page()
• blkdev_get_block()

19
How write() works

• The write() system call copies the user buffer to
  the page cache and returns.
• Unlike read(), the write() system call doesnt
  wait for the transfer between page cache and the
  device to complete. There is no data dependency
  in write operation.

20
Trace write() system call

• These functions are called in order
• write()
• sys_write()
• def_blk_fops.write generic_file_write()
• def_blk_aops.prepare_write blkdev_prepare_write()
  def_blk_aops.commit_write blkdev_commit_write()
• block_prepare_write()block_commit_write()

21
How open() works

• open() makes the file subsystem read the inode of
  the file from the device.
• The kernel then checks the inode type and knows
  it is a device file. The kernel then initializes
  the inode structure in RAM to contain file
  operations defined in the device driver. The
  kernel uses the major number of the device to
  find the file operations.
• From this point on, file operation calls to this
  inode will cause the execution of the
  corresponding device driver functions.

22
Trace open() system call

• These functions are called in order
• open()
• sys_open()
• file_open()
• dengry_open()
• def_blk_fops.open blkdev_open()
• do_open()
• get_blkfops()retrieves blkdevsMAJOR.bdops,
  stores in bdev.bd_op

23
The request queue of the device

• Requests to a block device is maintained in a
  request queue of the device.
• BLK_DEFAULT_QUEUE(MAJOR) finds the request queue
  by the major number.
• A request function from the device driver handles
  the request.
• The request function is registered when the queue
  is initializedblk_init_queue(BLK_DEFAULT_QUEUE(M
  AJOR),request_function)

24
Implementation
25
Whats in a device driver

• A device driver is like a server. It has
• A set of functions to services
• Functions to publish the services to the kernel
• Some internal data structures for bookkeeping
  purposes

26
In-kernel or kernel module

• Two ways of adding device driver to the kernel
• Compile into the kernel
• Loadable kernel module
• Benefit of kernel module
• No need to rebuild kernel
• No need to reboot system

27
Block device interface to Linux kernel

• /usr/src/linux/fs/block_dev.c
• struct file_operations def_blk_fops
• open blkdev_open,
• release blkdev_close,
• llseek block_llseek,
• read generic_file_read,
• write generic_file_write,
• mmap generic_file_mmap,
• fsync block_fsync,
• ioctl blkdev_ioctl,
• // Warning nonstandard gcc structure
  initialization

28
File system interface of the block device

• /usr/src/linux/include/linux/fs.h
• struct block_device_operations
• int (open) (struct inode , struct file
  )
• int (release) (struct inode , struct
  file )
• int (ioctl) (struct inode , struct file
  , unsigned, unsigned long)
• int (check_media_change) (kdev_t)
• int (revalidate) (kdev_t)
• struct module owner
• // file system hooks for the block devices

29
Important global array

• read_ahead // how many blockblk_size // device
  sizeblksize_size // block sizehardsect_size //
  sector size
• These arrays are indexed by major number
• Kernel fetches information about the device from
  these arrays

30
Initializing the device driver

• Done in init_module
• Allocate memory
• Initialize the device drivers slots in global
  arrays
• Initialize the request queue of the device driver
• Register the device driver (API)

31
Initialization example

• Int Init_module(void)
• ...
• myrd_storage (char )kmalloc(210241024sizeof
  (char), GFP_KERNEL)
• ...
• read_aheadMAJOR_NR 2 // sectors
• blk_sizeMAJOR_NR kmalloc(sizeof(int),
  GFP_KERNEL)
• blk_sizeMAJOR_NR 2048 // device size
  (kilobytes)
• blksize_sizeMAJOR_NR kmalloc(sizeof(int),
  GFP_KERNEL)
• blksize_sizeMAJOR_NR 512 // block size
  (bytes)
• hardsect_sizeMAJOR_NR kmalloc(sizeof(int),
  GFP_KERNEL)
• hardsect_sizeMAJOR_NR 512 // sector size
  (bytes)
• blk_init_queue(BLK_DEFAULT_QUEUE(MAJOR_NR),
  myrd_request)
• register_blkdev(MAJOR_NR,"myrd",myrd_ops)
• register_disk(NULL, MKDEV(MAJOR_NR, 0), 1,
  myrd_ops, 100)
• ...

32
Cleanup example

• void cleanup_module(void)
• ...
• fsync_dev(MKDEV(MAJOR_NR, 0))
• blk_cleanup_queue(BLK_DEFAULT_QUEUE(MAJOR_NR))
• kfree(hardsect_sizeMAJOR_NR)
• hardsect_sizeMAJOR_NR NULL
• kfree(blk_sizeMAJOR_NR)
• blk_sizeMAJOR_NR NULL
• kfree(blksize_sizeMAJOR_NR)
• blksize_sizeMAJOR_NR NULL
• kfree(myrd_storage)
• unregister_blkdev(MAJOR_NR,"myrd")

33
Servicing a request

• The kernel passes to the request function the
  beginning sector and the number of sectors to
  read.
• The request function can calculate the starting
  location and size bystart_loc sector
  SECTOR_SIZEsize nr_sectors SECTOR_SIZE
• Assume the RAM disk stores the data in an byte
  array data, the memory region to be copied is
  datastart_loc to datastart_locsize-1

34
Request routine example

• void myrd_request(request_queue_t q)
• ...
• while(1)
• INIT_REQUEST // this macro does a return when
  queue is empty
• location myrd_storage CURRENT-gtsector
  MYRD_HARDSECT_SIZE
• size CURRENT-gtcurrent_nr_sectors
  MYRD_HARDSECT_SIZE
• spin_lock(myrd_lock)
• if (CURRENT-gtcmd READ)
• memcpy(CURRENT-gtbuffer, location, size)
• else if (CURRENT-gtcmd WRITE)
• memcpy(location, CURRENT-gtbuffer, size)
• spin_unlock(myrd_lock)
• end_request(1) // update the queue and
  requests

35
Adding the device driver into the Linux source
tree

• Modify the Config.in in the directory
• Let user choose Y/N/M for this questiontristate
  Configure My RAM disk CONFIG_MYRD
• Add your code to the makefile
• obj_(CONFIG_MYRD) myrd.o
• The above line adds the object to obj_m or obj_y,
  depending on the answer in make config

36
Roll your own device driver
37
Helpful courses

• If you dont feel comfortable working with the
  hardware, these two courses are very helpful
• CS3432 Digital circuit design
• CS3434 Computer interfacing

38
A checklist of tasks

• Create the device file
• Define the default file operations
• def_blk_ops
• def_chr_fops
• Initialize the global data arrays
• Register the device driver to the kernel
• Cleanup everything when you are done

39
How do I drive a real device

• The derive driver interface to the user and the
  kernel remain the same.
• The request function will have to interact with
  the device. This is usually done by reading and
  writing to device registers.
• If the device is slow, you may want to use the
  interrupt driven approach. The process sets up an
  I/O operation and suspends. When the I/O
  completes, the device raise an interrupt request
  and the interrupt service routine in the device
  driver is called to handle the completed transfer.

40
How do I drive character device

• Needs to supply the file operations defined by
  the file system.
• No cache is used. Thus no need of the request
  function.
• Needs to initialize the global arrays of the
  kernel and register the driver.
• Still needs to implement interrupt service
  routine.

41
Looking ahead the 2.6.x kernel

• The 2.6 kernel removed several macro used in this
  presentation.
• The 2.6 kernel changed some global bookkeeping
  arrays.
• The 2.6 kernel is preemptive.
• The nonstandard structure initialization was
  removed in 2.6.x

42
Other types of drivers

• Sometimes character and block driver models are
  not suitable
• Network interface card driver
• Bus driver
• PCI, SCSI, etc.
• Need to deal with packets like in networking.
• And more

43
Suggestions to future work

• Build a file system on the ram disk
• Partition the data array into boot block, super
  block, inodes, data blocks.
• Supply mount() and unmount() system calls, plus
  other system calls.
• Port the RAM disk driver to kernel 2.6.x

44
Bibliography

• Linux device drivers, 2ed Alessandro Rubini
  Jonathan Corbet, OReilly, Sebastopol, CA
• Understanding the Linux kernel Daniel P.l Bovet
  Marco Cesati, OReilly, Sebastopol, CA
  ltftp//ftp.oreilly.com/pub/examples/linux/drivers2
  /book_pdf.rargt
• The design of the UNIX operating system Maurice
  J. Bach, Prentice Hall, Englewood Cliffs, New
  Jersey
• Writing a UNIX device driver, 2ed Janet I. Egan,
  Thomas J. Teixeira, John Wiley Sons, Inc.