vpm_xvcpus

1
vpm_xvcpus
2
Dedicated vs. Shared

• A customer complained that a batch run on Shared
  was twice slower than on Dedicated.
• How much is the overhead of hypervisor ?

• IBM does not mention about hypervisor overhead.

• hypv
• Shows the percentage of physical processor
  consumption spent making hypervisor calls.

3
True meaning of smt_snooze_delay

• Disabling smt_snooze_delay in shared processor
  pool environment
• Does this reduce the latency/overhead of shared
  processor pool ?

Moment of disabling smt_snooze_delay
4
So what was wrong ?

• LPAR sheet showed no folded VPs, but...

5
What is CPU folding ?

• One of the primary merits of micro-partition
• No need to estimate how many CPUs you really need
• However, too many vcpus may
• produce too much context switching
• result in high hyperv
• Thus CPU folding is introduced
• Every second, the kernel scheduler evaluates the
  number of virtual processors that should be
  activated to accommodate the physical utilization
  of the partition. If the number yields a high
  virtual processor utilization, the base number of
  virtual processors required is incremented to
  enable the workload to expand. You can request
  additional virtual processors with the schedo
  command. The value is then used to determine
  whether a virtual processor needs to be enabled
  or disabled, as the scheduler only adjusts the
  number of virtual processors in use each second
  by one.

6
Disabling CPU folding saved the day !

• schedo -o vpm_xvcpus -1
• The batch time was reduced from 7 hrs to 3 hrs
  30mins

CPU folding enabled
folded CPUs
CPU folding disabled
7
Is it normal to disable CPU folding ?

• SAP generally encourages disabling it.
• http//www.saptechno.com/sap-notes.html?viewsapno
  teid1102070
• Use of micro-partitioning (POWER5), logical
  partitioning (LPAR) or SMT is supported with
  Oracle RAC for an SAP installation with specific
  settings of the AIX scheduler. These parameter
  settings have an effect on the latency imposed by
  the AIX hypervisor due to workload management
  among the LPARs in the frame. Most important is
  to turn off CPU folding in AIX. You must be user
  root to implement the settings. Execute the
  following commands as shown below schedo -o
  vpm_xvcpus -1

8
CPU folding is always bad ?

• No, not always.

logical CPU CPU01
logical CPU CPU11
9
So turn it on, or turn it off ?

• My personal experiences indicate
• Limit the raito of vcpuspcpus under 21
• A very ugly case of 101 ratio
• turn off CPU folding when in doubt
• Do not disable smt_snooze_delay
• It dampens the beauty of Shared Processor Pool
• After all, it does not seem to reduce the hypv
  much