Memory Addressing

1
Memory Addressing

• Segmentation and paging background
• Segmentation
• Intel segmentation hardware
• Linux use of Intel segmentation - minimal
• Paging
• Intel paging hardware two level
• Linux use of Intel paging three level
  abstraction
• Extended paging (jumbo pages 2MB or 4MB)
• Physical Address Extension (PAE) up to 64GB
  physical
• Memory layout
• Kernel layout in physical memory
• Process virtual memory layout
• Kernel virtual memory layout

2
Segmentation Background

• Motivated by logical use of memory areas
• Code, heap, stack, etc.
• Base offset
• Segment registers (es, cs, ds) contain base
• Actually, they contain Segment Selector
• Index into Segment Descriptor table (which
  contains base)
• Segments variable size (usually large)
• Process as collection of segments
• No notion of linear, contiguous memory
• Similar to multi-stream files (Mac)
• Requires segment descriptor table
• Similar to page table

3
Paging Background

• Motivated by notion of linear, contiguous,
  "virtual" memory (space)
• Every process has it's own "zero" address
• Uniform sized chunks (pages 4K on Intel)
• So called jumbo pages (2MB or 4MB)
• Virtually contiguous pages may be physically
  scattered
• Virtual space may have "holes"
• Page table translates virtual "pages" to physical
  "page frames"

4
Intel Segmentation/Paging

• Intel address terminology
• Logical segment offset
• Linear (virtual) 0 .. 4GB
• Physical - 0 .. 4 GB (64GB with PAE)
• Logical (segmentation) -gt Linear (paging) -gt
  Physical
• Paging can be disabled
• Segmentation required
• Though you can just have one big segment

5
Intel Segmentation Hardware

• Segment registers cs, ss, ds, es, fs, gs
• Contain indices ("selectors") into "segment
  descriptor tables"
• Segment descriptor tables GDT, LDT(s)
• Global and local (per process, in theory)
• Each holds about 8000 descriptors
• Special registers point to tables gdtr, ldtr
• Segment descriptors 8 bytes each
• Base/limit, code/data privileges, type
• Cached in ro registers when seg registers loaded
• Task segment descriptor (TSS)
• Special segment for holding process context
• Segment registers which table? Which descriptor?
• Offset which byte in segment?

6
Segmentation in Linux

• History
• Early versions segmented now paged
• Using "shared" segments simplifies things
• Some RISC chips don't support segmentation
• Linux only uses GDT
• LDTs allocated for Windows emulation
• Each process has TSS and LDT descriptors
• TSS is per-process LDT is shared

7
Linux Descriptor Allocation

• GDT holds 8192 descriptors
• 4 primary, 4 APM, 1 null (index 0), 3 unused
• 8180 left / 2 -gt 4090 processes (limit in 2.2)
• Primary (shared) segments
• Kernel code, kernel data
• User code, user data
• Segments overlap in linear address space
• Difference is access rights (code/date,
  user/kernel)
• 2.4 removes 4K process restriction
• GDT entries are managed, swapped as necessary
• Segment registers changed on kernel entry

8
Intel Paging Hardware

• Page table "maps" linear address space
• Some pages may be invalid
• Address space grows/shrinks (mapping)
• Created by exec()
• New regions mapped by DLLs, mmap(), sbrk()
• Pages (linear) vs. page frames (physical)
• Page may map to different frame after swap
• Page tables stored in kernel data segment
• Intel page size 4K or 4M ("jumbo pages")
• Jumbo pages reduce table size
• Paging "enabled" by writing cr0 register
• Register points to page table (cr3)

9
Intel Two-Level Paging

• "Page table" is actually a two-level tree
• Page Directory (root), Page Tables (children),
  Pages (grandchildren)
• Why two levels?
• 1 level table requires 4MB per process!
• 2 levels unmapped regions dont require
  (secondary) page tables
• Linear address carved into three pieces
• Directory (10), Table (10), Offset (12)
• Entries frame bookkeeping bits
• Bookkeeping bits
• Present, Accessed, Dirty
• Read/Write, User/Supervisor, Page Size
• Extended Paging (Jumbo pages 2MB or 4MB)
• Map entire 4GB address space with just top-level
  Directory
• No need for Tables! (Kernel uses this technique)

10
Three-Level Paging

• How big are page tables on 64 bit arch?
• Sparc, Alpha, Itanium
• Assume 16K pages gt 128 MB per process!
• Better idea three-level paging trees
• Page Global Directory (pgd)
• Page Middle Directory (pmd)
• Page Table (pt)
• Carve linear address into 4 pieces
• Conceptual paging model
• On Intel, page middle directory is compiled out

11
Physical Address Extension (PAE)

• Initially virtual memory exceeds physical
• Over time physical grows to exceed virtual!
• Supports up to 64 GB physical (36 bits)
• Extends kernel, not user, memory access
• Linus Get a 64 bit machine!
• Good for enhancing server performance
• Page Directory Pointer Table (PDPT)
• Third level to paging hierarchy
• Contains only 4 entries!
• Linear addresses still on 32 bits!
• How to access high memory?
• Map, access, remap
• Jumbo pages are 2MB with PAE enabled

12
Aside Caching

• Exploit temporal and spatial locality
• L1 and L2 caches (on chip)
• Cache "lines" (128 bytes on Pentium)
• Kernel developer goals
• Keep frequently used struct fields in same cache
  line!
• Spread lines out for large data structures to
  keep cache utilized
• The "Cache Police
• Intel allows per-page caching policy!
• Linux used write-back for all
• Snooping on SMP machine

13
TLB

• Translation Look-aside Buffer
• Virtual to physical cache
• Must be flushed (invalidated) when address space
  mappings change
• A significant cost of context switch

14
Paging in Linux

• Three-level scheme
• Middle directories "collapsed" w/ macro tricks
• No bits assigned for middle directory
• On context switch (address space change)
• Save/load registers in TSS
• Install new top-level directory (write cr3)
• Flush TLB one entry, one page, all, all
  processors
• Intel recently supports global pages (not
  flushed from TLB)
• Lot's of macros for
• Allocating/deallocating, altering, querying
• Page directories, tables, entries
• Examples
• set_pte(), mk_pte(), pte_young(),
  new_page_tables()

15
Kernel Physical Layout

• Kernel is reserved ("pinned) (non-swappable)
• Stored contiguously modules can be removed
• Usually about 2MB
• Mapped starting at 1MB physical (first 1MB not
  used)
• Appears virtually starting at 3GB
• Some "holes" in low memory
• Zero frame always invalid (BIOS weirdness)
• ISA "i/o aperture" (640K .. 1M)
• Kernel layout symbols (System.map)
• _text (code start) (usually about 1M)
• _etext, _edata, _end
• Remaining frames allocated by kernel memory
  allocator

16
Process Virtual Layout

• 4GB virtual space on a 32 bit system
• Possible to install Linux on systems with more
  than 4GB of physical memory but you must use
  tricks to access "high" memory
• Kernel macro PAGE_OFFSET
• Division between user and kernel regions
• Typically 3GB user, 1 GB kernel (adjustable)
• We will look at details of user space later

17
Kernel Virtual Layout

• Kernel page tables initialized in two stages
• Phase one only maps 8MB (provisional)
• Phase two maps all memory
• Kernel page table (uses jumbo pages)
• swapper_pg_dir (shared by all kernel threads)
• Up to 896 MB of physical mapped by kernel
• So called identity mapped segment
• Allows kernel to be easily addressed virtually or
  physically
• Virtual Physical PAGE_OFFSET
• Remaining 128 MB of space virtually mapped
• High memory mapping virtual kernel memory
  allocation etc.

18
Final Kernel Virtual Layout

• Kernel code
• Operates using linear (virtual) addresses
• Macros to go from physical to virtual and back
• _pa(virual), _va(physical)
• Kernel virtual address space
• Low physical frames (containing kernel) mapped to
  virtual addresses starting at PAGE_OFFSET
• Remaining frames allocated on demand by kernel
  and user processes and mapped to virtual
  addresses

19
Summary

• Intel hybrid segmented/paged architecture
• Segment support used minimally
• Linux has a conceptual 3 level page table
• Kernel is mapped into high memory of every
  process address space but is stored in low
  physical memory