To Compress or not to Compress?

1
To Compress or not to Compress?

• Chuck Hopf

2
What is your precious?

• Gollum says every data center has something that
  is precious or hard to come by
• CPU Time
• DASD Space
• Run Time
• IO
• Memory

3
Lots of talk

• On the LISTSERVE does compression use more CPU?
  Does it save DASD space?
• On the LISTSERVE what is the best BUFNO to use
  with MXG

4
Testing the theories

• Built two tests
• COMPRESSNO varying BUFNO from 2 10 15 20
• COMPRESSYES again varying the BUFNO

5
An Epiphany!

• What if you run with COMPRESSNO and send the
  output to PDB as a temporary dataset and then at
  the end, turn on COMPRESSYES and do a PROC COPY
  INDDPDB OUTDDPERMPDB NOCLONE ? That would
  eliminate all of the compression during the
  reading and writing of all of the interim
  datasets but still create a compressed PDB.

6
So there are now 3 Tests!

• TESTNO - COMPRESSNO
• TESTNO/YES - COMPRESSNO but final PDB is
  compressed
• TESTYES COMPRESSYES

7
CPU Time
8
Elapsed Time
9
Low Memory
10
High Memory
11
EXCP DASD
12
DASD IO Time
13
DASD Space
14
DASD Space by DDNAME
15
Conclusions?

• Running with COMPRESSNO and then copying to a
  compressed PDB optimizes permanent DASD space and
  uses very little additional CPU.
• Even better, use the LIBNAME OPTION to turn it on
  where you want
• LIBNAME PDB COMPRESSYES / zOS only /
• Memory requirements increase with BUFNO but are
  not really that bad and BUFNO GT 10 shows very
  little additional benefit

16
Caveats!

• BLKSIZE matters. SAS procs are sometimes built
  with a BLKSIZE of 6160 on WORK. This radically
  affects the IO counts. Use the recommended
  BLKSIZEDASD(OPT) and leave the DCB attributes
  off of SAS datasets.
• REGION may have to be increased use REGION0M
  and be sure you are using the MXG defaults for
  MEMSIZE.
• This all applies to zOS not to ASCII platforms

17
So What About ASCII?

• Using the same data, tests run with SAS 9.2 on
  Win 7 system
• 1.5GB memory
• Dell 4600 P4 2.7GHz

18
ASCII Results
Test BUFNO Elapsed CPU Memory
COMPRESSNO DEFAULT 0645.4 0325.0 95713k
COMPRESSYES DEFAULT 0612.5 0251.6 95721K
COMPRESSNO 16K 0735.1 0356.8 275537K
COMPRESSYES 16K 0557.8 0249.1 179769K
COMPRESSNO 32K 0739.2 0258.0 275537K
COMPRESSYES 32K 0605.4 0251.0 179679K
COMPRESSNO 40K 0828.6 0417.0 275537K
COMPRESSYES 40K 0620.4 0259.1 179769K
COMPRESSNO 80K 0744.1 0402.8 275537K
COMPRESSYES 80K 0559.2 0254.5 179769K
COMPRESSNO 16M 0742.1 0401.1 275537K
COMPRESSYES 16M 0609.2 0253.0 179769K
COMPRESSNO 32M 0743.4 0354.9 275537K
COMPRESSYES 32M 0557.4 0251.1 179769K
COMPRESSNO 64M 0802.7 0358.7 275537K
COMPRESSYES 64M 0637.8 0255.5 179769K
COMPRESSNO 128M 0814.2 0355.0 275537K
COMPRESSYES 128M 0630.0 0258.0 179769K
COMPRESSNO 10 0711.5 0316.1 96259K
COMPRESSYES 10 0556.2 0237.1 96649K
COMPRESSNO 40 0717.5 0320.9 97603K
COMPRESSYES 40 0600.1 0241.1 98892K
COMPRESSNO 80 0713.0 0324.1 99529K
COMPRESSYES 80 0557.6 0236.2 102095K
COMPRESSNO 160 0716.1 0324.0 103379K
COMPRESSYES 160 0544.6 0226.5 108825K
19
Wow!

• COMPRESSYES outperforms COMPRESSNO!
• BUFNO makes some difference but not a lot and
  BUFNO10 looks to be optimal
• Difference is in seconds not minutes
• But there is something we dont understand in
  the memory numbers
• Runs faster under Win 7 than under zOS
• But does not include download time

20
So What Should You Do?

• It Depends on what your precious is
• Running zOS
• Optimal for CPU and DASD is COMPRESSNO with a
  copy to a compressed dataset at the end or by
  setting the compressYES option with a LIBNAME
• Optimal for CPU is COMPRESSNO
• Optimal for DASD is COMPRESSYES
• BUFNO10 is optimal for run time
• Running ASCII
• Optimal for CPU and DASD is COMPRESSYES

21
JCL
// SAMPLE JCL TO RUN BUILDPDB WITH COMPRESSNO
AND COMPRESS AT // THE END USING PROC COPY //S1
EXEC MXGSASV9 //PDB DD
DSNMXG.PDB(1),SPACE(CYL,(500,500)), //
DISP(,CATLG,DELETE) //SPININ DD
DSNMXG.SPIN(0),SPACE(CYL,(500,500)) //
DISP(,CATLG,DELETE) //SPIN DD
DSNMXG.SPIN(1),DISPOLD //CICSTRAN DD
DSNMXG.CICSTRAN(1),SPACE(CYL,(500,500)), //
DISP(,CATLG,DELETE) //DB2ACCT DD
DSNMXG.DB2ACCT(1),SPACE(CYL,(500,500)), //
DISP(,CATLG,DELETE) //SMF DD
DSNYOUR,SMF DATA,DISPSHR //SYSIN DD
OPTIONS COMPRESSNO BUFNO10 LIBNAME PDB
COMPRESSYES LIBNAME SPIN COMPRESSYES LET
SPININSPININ UTILBLDP( MACKEEPX MACRO
_LDB2ACC DB2ACCT.DB2ACCT MACRO _KDB2ACC
COMPRESSYES MACRO _KCICTRN COMPRESSYES
, SPINCNT7, SPINUOW2,
OUTFILEINSTREAM) INCLUDE INSTREAM JCL is
in the 27.10 SOURCLIB as JCLCMPDB
22
Why UTILBLDP?

• Allows you to add data sources to BUILDPDB
  without having to edit the macros in the
  SOURCLIB.
• Allows you to suppress data sources like 110 and
  DB2 and TYPE74 and process them in other jobs
  again without editing the macros.
• Flexibility

23
Example
OPTIONS COMPRESSNO BUFNO10 LIBNAME PDB
COMPRESSYES LIBNAME SPIN COMPRESSYES LET
SPININSPININ UTILBLDP( USERADD42,
SUPPRESS110 DB2, SPINCNT7,
OUTFILEINSTREAM) INCLUDE INSTREAM RUN
24
MXG User Experience

• Running MXG with WPS instead of SAS
• Data from multiple platforms
• Processed under two Virtual products
• Also, Comparison of SAS/PC and WPS on zLinux

25
PC/SAS VMWARE/Windows versus PC/SAS
Hyper-V/Windows (four platforms data, three
installation groups PROD/QA/DEV) Data From
VMWARE(PROD) Hyper-V(PROD) Unix
000530 001056 zOS
000130 000454 zVM/Linux 000307
000808 Windows Servers 024308
093257 Data From VMWARE(QA)
Hyper-V(QA) Unix 000031
000418 ZOS 000127
000246 zVM/Linux 000102
000706 Windows Servers 004124
023419 Data From VMWARE(DEV)
Hyper-V(DEV) Unix 000043
000242 ZOS 000021
000142 zVM/Linux 000108
000334 Windows Servers 000906 003847
Processing of performance Data collected from
Unix, zVM/Linux, zOS and Windows.
26
PC/SAS versus LNX/WPS

• PC/SAS VMWARE/Windows versus WPS zVM/Linux
• PC/SAS VMWARE is taking 24308 to process the
  data from Window Servers for what the WPS
  zVM/Linux environment can do in 13000
  (hhmmss). 
• That is, the Mainframe WPS zVM/Linux is a 45
  improvement over the PC/SAS VMWARE/WIN.
• This is most likely due to the extra bandwidth
  the mainframe has for I/Os compared to the
  Windows environment. 
• The results for Windows would probably be better
  if WIN2008 had been used.

27
PC/SAS versus WPS on z

• PC/SAS under Hyper-V
• WPS under zVM/Linux on z-10

28
Z10 SAS versus WPS

• zOS/SAS versus zOS/WPS to run MXG
• 30 more I/Os for SAS
• TCB for WPS 551,423
• TCB for SAS 551,273
• NOTES
• WPS version 2.4.0.1 and SAS 9.1.3
• MXG from FEB 2009