Advanced Operating Systems

1
Advanced Operating Systems
Lecture 12 Process migration

• University of Tehran
• Dept. of EE and Computer Engineering
• By
• Dr. Nasser Yazdani

2
Covered topic

• Process migration, Why? And how.
• References
• Chapter 3 of the text book
• Fred Douglis and John Ousterhout, Transparent
  Process Migration Design Alternatives and the
  Sprite Implementation

3
Outline

• Process in Distributed Systems
• Motivation for migration
• How does migration occur?
• Resource migration
• Agent-based system
• Details of process migration
• Problems

4
Threads in DS

• The goal is to allow blocking calls without
  blocking the whole process
• Multithread clients like browsers, display data
  while getting others.
• Multithread Servers like dispatcher which reads
  incoming requests

5
ClientsX-Window

• The basic organization of the X Window System

6
Client-Side Software for Distribution Transparency

• A possible approach to transparent replication of
  a remote object using a client-side solution.

7
Servers General Design Issues

1. Client-to-server binding using a daemon as in DCE
2. Client-to-server binding using a superserver as
   in UNIX

3.7
8
Object Adapter (1)

• Activation policy adapter or wrapper
• Organization of an object server supporting
  different activation policies.

9
Object Adapter (2)
/ Definitions needed by caller of adapter and
adapter /define TRUEdefine MAX_DATA 65536 /
Definition of general message format /struct
message long source / senders identity
/ long object_id / identifier for the
requested object / long method_id /
identifier for the requested method /
unsigned size / total bytes in list of
parameters / char data / parameters as
sequence of bytes / / General definition of
operation to be called at skeleton of object
/typedef void (METHOD_CALL)(unsigned, char
unsigned, char) long register_object
(METHOD_CALL call) / register an object
/void unrigester_object (long object)id) /
unrigester an object /void invoke_adapter
(message request) / call the adapter /

• The header.h file used by the adapter and any
  program that calls an adapter.

10
Object Adapter (3)
typedef struct thread THREAD / hidden
definition of a thread / thread CREATE_THREAD
(void (body)(long tid), long thread_id)/
Create a thread by giving a pointer to a function
that defines the actual // behavior of the
thread, along with a thread identifier / void
get_msg (unsigned size, char data)void
put_msg(THREAD receiver, unsigned size, char
data)/ Calling get_msg blocks the thread
until of a message has been put into its //
associated buffer. Putting a message in a
thread's buffer is a nonblocking // operation.
/

• The thread.h file used by the adapter for using
  threads.

11
Object Adapter (4)

• The main part of an adapter that implements a
  thread-per-object policy.

12
Motivation

• Key reasons performance and flexibility
• Process migration (strong mobility)
• Improved system-wide performance better
  utilization of system-wide resources
• Idle workstations
• Code migration (weak mobility)
• Shipment of server code to client filling forms
  (reduce communication, no need to pre-link stubs
  with client)
• Ship parts of client application to server
  instead of data from server to client (e.g.,
  databases)
• Improve parallelism agent-based web searches

13
Motivation

• Flexibility
• Dynamic configuration of distributed system
• Clients dont need preinstalled software
  download on demand

14
Migration models

• Process code seg resource seg execution seg
• Weak versus strong mobility
• Weak gt transferred code (program) starts from
  initial the state (Java Applets). Simple
• Strong gt move execution segment
• Sender-initiated versus receiver-initiated
• Sender-initiated (code is with sender)
• Client sending a query to database server
• Client should be pre-registered
• Receiver-initiated
• Java applets
• Receiver can be anonymous

15
Who executes migrated entity?

• Code migration
• Execute in a separate process
• Applets Execute in target process
• Process migration
• Remote cloning
• Migrate the process

16
Models for Code Migration

• Alternatives for code migration.

17
Do Resources Migrate?

• Depends on resource to process binding
• By identifier specific web site, ftp server
• By value Java libraries
• By type printers, local devices
• Depends on type of attachments
• Unattached to any node data files
• Fastened resources (moved only at high cost)
• Database, web sites
• Fixed resources
• Local devices, communication end points

18
Resource Migration Actions
Resource-to machine binding
Unattached Fastened Fixed
By identifier By value By type MV (or GR) CP ( or MV, GR) RB (or GR, CP) GR (or MV) GR (or CP) RB (or GR, CP) GR GR RB (or GR)
Process-to-resource binding

• Actions to be taken with respect to the
  references to local resources when migrating code
  to another machine.
• GR establish global system-wide reference
• MV move the resources
• CP copy the resource
• RB rebind process to locally available resource

19
Migration in Heterogeneous Systems

• Systems can be heterogeneous (different
  architecture, OS)
• Support only weak mobility recompile code, no
  run time information
• Strong mobility recompile code segment,
  transfer execution segment migration stack
• Virtual machines - interpret source (scripts) or
  intermediate code Java

Migration on Only subroutine Or
method Call Migrate stack
20
Cost of migration

• Multiprocessor nondistributed
• loss of the lines associated with the process in
  the processor's instruction a data caches
• Distributed environment
• Moving a process's virtual memory
• Forwarding a process's IPC (local and network)
  messages, informing senders of the process's new
  contact information.
• Moving information of files. the open file table,
  the file descriptor table, the file offset, dirty
  blocks in the buffer cache, c
• Moving the process's user-level state registers,
  stack, c
• Moving the process's kernel-level state pwd,
  pid, signal masks, c

21
Cost of migration (partial migration)

• Migration of the whole process too expensive.
• Move certain aspects of a process
• The remaining portions of the process create
  residual dependencies -- the migrated process
  still relies on the original host to provide the
  services that were not migrated.

22
Migrating Virtual memory

• Freeze and copy migration Suspend or freeze the
  process on the original host, and then to copy
  all of the pages of memory to the new host. Once
  all done, process can be resumed on the new host.
  
• Simple, clean and easy to implement.
• Does not create a residual dependency
• Many pages which are never used may be copied and
  sent over the network If the process is migrated
  several times, this cost adds up
• Do nothing while copying?

23
Migrating Virtual memory

• Precopying The process runs on the original
  host, while the pages are being copied.
• It is clean -- it does not create any residual
  dependencies.
• Copying pages that may never be used.
• Dirty pages must be transferred. Can be more
  expensive
• lazy migration like demand paging.
• It creates residual dependencies

24
Migrating Virtual memory

• Distributed file system a memory-mapped file.
  the process's memory can be migrated simply by
  flushing the dirty blocks and mapping the file
  from a different host.
• Isn't as clean as it may seem

25
Migrating Communication Channels

• If a process migrates, its communications must be
  able to continue.
• Inform "interested" processes of the new location
  of a migrating process.
• Unclean, unnecessary messages, how to know other
  communicating clients?
• link redirection or forwarding at the original
  host of the migrating process.
• Residual dependency and can increase the latency
  involved in sending messages to the migrated
  process, but makes the process of migration
  itself cheaper

26
Process with open files

• Show up at the new host and re-open the files.
  But, in truth, there is a great deal of state
  associated with an open file. Consider the
  system-wide open file table, the cached inodes,
  dirty blocks that may live only in the local
  buffer cache, c.
• fork()'d proceses share the same file offset.
• it is often much easier to leave the process
  dependent on the old host for file service.

27
Migrate kernel state

• It is often easier to leave a migrating process
  dependent on a prior (or berhaps first) host for
  these services.
• Checkpointing and recovery A process's state to
  be saved to a file (much like a persistent
  object) and then a new process to be created
  (restored) based on this checkpoint file. This
  checkpoint file contains all of the "goods"
  including the kernel material.

28
Migrate? Or not migrate?

• Several things to consider
• If the home host suffers from a bursty load, it
  may not make sense to migrate a process -- the
  home host will be free again, soon.
• Processes with significant virtual memory or IPC
  usage or many open files are poor choices for
  migration.
• Historical consideration long running processes
  are better candidates than recent arrivals they
  are likely to continue to run for a long time.
  Short lived processes are likely to complete
  shortly after migration, offering little gain to
  amortize the cost of migration over useful work.

29
Distributed Scheduling Motivation

• Distributed system with N workstations
• Model each w/s as identical, independent M/M/1
  systems
• Utilization u, P(system idle)1-u
• What is the probability that at least one system
  is idle and one job is waiting?

30
Implications

• Probability high for moderate system utilization
• Potential for performance improvement via load
  distribution
• High utilization gt little benefit
• Low utilization gt rarely job waiting
• Distributed scheduling (aka load balancing)
  potentially useful
• What is the performance metric?
• Mean response time
• What is the measure of load?
• Must be easy to measure
• Must reflect performance improvement

31
Design Issues

• Measure of load
• Queue lengths at CPU, CPU utilization
• Types of policies
• Static decisions hardwired into system
• Dynamic uses load information
• Adaptive policy varies according to load
• Preemptive versus non-preemptive
• Centralized versus decentralized
• Stability lgtm gt instability, l1l2ltm1m2gtload
  balance
• Job floats around and load oscillates

32
Components

• Transfer policy when to transfer a process?
• Threshold-based policies are common and easy
• Selection policy which process to transfer?
• Prefer new processes
• Transfer cost should be small compared to
  execution cost
• Select processes with long execution times
• Location policy where to transfer the process?
• Polling, random, nearest neighbor
• Information policy when and from where?
• Demand driven only if sender/receiver,
  time-driven periodic, state-change-driven send
  update if load changes

33
Sender-initiated Policy

• Transfer policy
• Selection policy newly arrived process
• Location policy three variations
• Random may generate lots of transfers gt limit
  max transfers
• Threshold probe n nodes sequentially
• Transfer to first node below threshold, if none,
  keep job
• Shortest poll Np nodes in parallel
• Choose least loaded node below T

34
Receiver-initiated Policy

• Transfer policy If departing process causes load
  lt T, find a process from elsewhere
• Selection policy newly arrived or partially
  executed process
• Location policy
• Threshold probe up to Np other nodes
  sequentially
• Transfer from first one above threshold, if none,
  do nothing
• Shortest poll n nodes in parallel, choose node
  with heaviest load above T

35
Symmetric Policies

• Nodes act as both senders and receivers combine
  previous two policies without change
• Use average load as threshold
• Improved symmetric policy exploit polling
  information
• Two thresholds LT, UT, LT lt UT
• Maintain sender, receiver and OK nodes using
  polling info
• Sender poll first node on receiver list
• Receiver poll first node on sender list

36
Case Study V-System (Stanford)

• State-change driven information policy
• Significant change in CPU/memory utilization is
  broadcast to all other nodes
• M least loaded nodes are receivers, others are
  senders
• Sender-initiated with new job selection policy
• Location policy probe random receiver, if still
  receiver, transfer job, else try another

37
Sprite (Berkeley)

• Workstation environment gt owner is king!
• Centralized information policy coordinator keeps
  info
• State-change driven information policy
• Receiver workstation with no keyboard/mouse
  activity for 30 seconds and active processes lt
  number of processors
• Selection policy manually done by user gt
  workstation becomes sender
• Location policy sender queries coordinator
• WS with foreign process becomes sender if user
  becomes active selection policygt home
  workstation

38
Sprite (contd)

• Sprite process migration
• Facilitated by the Sprite file system
• State transfer
• Swap everything out
• Send page tables and file descriptors to receiver
• Demand page process in
• Only dependencies are communication-related
• Redirect communication from home WS to receiver

39
Overview of Code Migration in D'Agents (1)

• A simple example of a Tel agent in D'Agents
  submitting a script to a remote machine (adapted
  from gray.r95)

proc factorial n if (n ? 1) return 1
fac(1) 1 expr n factorial expr n
1 fac(n) n fac(n 1) set number
tells which factorial to compute set machine
identify the target machine agent_submit
machine procs factorial vars number script
factorial number agent_receive receive
the results (left unspecified for simplicity)
40
Overview of Code Migration in D'Agents (2)

• An example of a Tel agent in D'Agents migrating
  to different machines where it executes the UNIX
  who command (adapted from gray.r95)

all_users machines proc all_users machines
set list "" Create an initially empty list
foreach m machines Consider all hosts in
the set of given machines agent_jump
m Jump to each host set users exec
who Execute the who command append
list users Append the results to the list
return list Return the complete list
when done set machines Initialize the set
of machines to jump toset this_machine Set to
the host that starts the agent Create a
migrating agent by submitting the script to this
machine, from where it will jump to all the
others in machines. agent_submit this_machine
procs all_users -vars machines -script
all_users machines agent_receive receive
the results (left unspecified for simplicity)
41
Agents

• Software agents
• Autonomous process capable of reacting to, and
  initiating changes in its environment, possibly
  in collaboration
• More than a process can act on its own
• Mobile agent
• Capability to move between machines
• Needs support for strong mobility
• Example DAgents (aka Agent TCL)
• Support for heterogeneous systems, uses
  interpreted languages

42
Implementation Issues (1)

• The architecture of the D'Agents system.

• Lowest communication
• Server agent manage-
• ment, comm. Among
• agent, auth.
• RTS start end agents,
• Etc.

43
Implementation Issues (2)

• The parts comprising the state of an agent in
  D'Agents.

Status Description
Global interpreter variables Variables needed by the interpreter of an agent
Global system variables Return codes, error codes, error strings, etc.
Global program variables User-defined global variables in a program
Procedure definitions Definitions of scripts to be executed by an agent
Stack of commands Stack of commands currently being executed
Stack of call frames Stack of activation records, one for each running command
44
Software Agents in Distributed Systems
Property Common to all agents? Description
Autonomous Yes Can act on its own
Reactive Yes Responds timely to changes in its environment
Proactive Yes Initiates actions that affects its environment
Communicative Yes Can exchange information with users and other agents
Continuous No Has a relatively long lifespan
Mobile No Can migrate from one site to another
Adaptive No Capable of learning

• Some important properties by which different
  types of agents can be distinguished.

45
Agent Technology

• The general model of an agent platform (adapted
  from fipa98-mgt).

46
Agent Communication Languages (1)

• Examples of different message types in the FIPA
  ACL fipa98-acl, giving the purpose of a
  message, along with the description of the actual
  message content.

Message purpose Description Message Content
INFORM Inform that a given proposition is true Proposition
QUERY-IF Query whether a given proposition is true Proposition
QUERY-REF Query for a give object Expression
CFP Ask for a proposal Proposal specifics
PROPOSE Provide a proposal Proposal
ACCEPT-PROPOSAL Tell that a given proposal is accepted Proposal ID
REJECT-PROPOSAL Tell that a given proposal is rejected Proposal ID
REQUEST Request that an action be performed Action specification
SUBSCRIBE Subscribe to an information source Reference to source
47
Agent Communication Languages (2)
Field Value
Purpose INFORM
Sender max_at_http//fanclub-beatrix.royalty-spotters.nl7239
Receiver elke_at_iiop//royalty-watcher.uk5623
Language Prolog
Ontology genealogy
Content female(beatrix),parent(beatrix,juliana,bernhard)

• A simple example of a FIPA ACL message sent
  between two agents using Prolog to express
  genealogy information.

48
Next Lecture

• Naming in distributed systems.
• References
• Chapter 4 of the book