Express%20Reorg:%20How%20to%20save%20your%20weekend

1
Express ReorgHow to save your weekend

• Laura Rochon
• Ajilon Consulting

2
Laura Rochon

• Started working with CA-IDMS in 1983
• Joined Cullinet in 1985
• Left CA in 1992, has been consulting ever since
• Joined Ajilon Consulting in January 2005
• IUA Board member since 1998
• CQUI President

3
Agenda

• Overview of Unload/Reload
• Overview of Express Reorg
• Express Reorg Execution
• Considerations
• Test results
• Conclusion

4
Overview of Unload/Reload
Unload
Reload
DB
DB
5
Overview of Unload/Reload
Reload
Unload
Unload
SORT1
DBL2
SORT3
DBL3
SORT4
DBL4
SORT2
DBLX
DB
DB
6
Overview of Express Reorg
Unload
Reload
Unload
Reload
Control file
Unload
Reload
DB
DB
DB
DB
DB
7
Overview of Express Reorg

• New functionality
• Add or remove fields from index key
• Change order of set

8
Overview of Express Reorg

• 4 major phases
• SETUP
• Allocates control file
• Specifies what is to be done
• UNLOAD
• Unloads data
• RELOAD
• Reloads data
• CLEANUP
• Delete work files (but not control file)
• Unlock areas

9
Overview of Express Reorg

• Unload
• Area sweep for most records, except
• Record stored VIA system-owned index
• VIA record
• CALC records with duplicate option

10
Overview of Express Reorg

• Unload (contd)
• Records assigned target page
• CALC records
• VIA records
• Direct records

11
Overview of Express Reorg

• REORG slicing
• Based on DIVIDE PROCESSING n WAYS
• Generally, each area is divided symmetrically
• If total size of all areas is less than 200
  pages, 1 slice
• Areas and subareas that contain only system
  indexes not divided and assigned to a slice

12
Overview of Express Reorg

• REORG slicing (contd)
• User-owned indexes are processed with the slice
  of the owner record
• System-owned indexes assigned to index group and
  processed indepently of slices

13
Overview of Express Reorg

• Reorg processing is divided into tasks and
  grouped into phases
• RELOAD phase is divided into
• Reload phases 1 thru 6
• Each reload task in a given task must
  successfully complete before processing can move
  to next Reload phase
• Rebuild phases 1 thru 4

14
Overview of Express Reorg
REORG
Reload
Cleanup
Unload
Unload
Rebuild 1
Reload 1
Reload 3
Reload 4
Reload 6
Rebuild 2
Rebuild 3
Rebuild 4
Cleanup
Reload 2
Reload 5
DB
DB
DB
DB
DB
15
Express Reorg Execution

• Parameters
• JCL
• Work Files
• Output
• Processing
• Job submission

16
Express Reorg Parameters
REORG setup options
SEGMENT source-segment USING source-ssc AREA area
RELOAD INTO target-db USING target-ssc DMCL
dmcl-name DIVIDE PROCESSING n WAYS
REUSE WORKFILES AS
SORTEXIT SHARE NOTIFY notify-interval
17
Express Reorg Parameters
REORG execution options
CLEANUP STOP AFTER
SETUP UNLOAD
RELOAD
NO CREATE UNLOAD WORKFILES
RELOAD
ALL SUBMIT
18
Express Reorg Parameters
REORG other parameters
CLEANUP STATUS REPORT ONLY
19
Express Reorg JCL

• //BCF EXEC PGMIDMSBCF,REGION0M
• //STEPLIB DD DSNidms.loadlib,DISPSHR
• //SYSIDMS DD
• Sysidms parameters
• //SYSLST DD SYSOUT
• //SYSPRINT DD SYSOUT
• //SORTMSG DD SYSOUT
• //SORTWRKnn DD UNITunit,SPACE(CYL,(mmm,nnn))
• //SYSIPT DD
• Reorg parameters
• //RORGCTL DD DSNrorgctl-file,DISPSHR
• //RORGJCL DD DSNjcl-file,DISPSHR

20
Express Reorg JCL

• RORGCTL file
• Describes the specifics of the REORG operation
• Contains status information
• Fixed-length, LRECL4096
• Initialized during setup
• Should not have DISPOLD or DISPNEW

21
Express Reorg JCL

• RORGJCL file
• Contains JCL for parallel jobs
• Job name should be short enough to append 1
  characters
• 1 step IDMSBCF with following syntax
• REORG
• Should have DB files and RORGCTL file

22
Express Reorg Work Files

• The number of work files proportional to number
  of jobs submitted
• Work files can be manually allocated
• Work files allocated automatically by using new
  command CREATE DSMODEL
• Can specify several CREATE DSMODEL

23
Express Reorg Work Files

• REORG uses following DDNAME prefixes
• WU -- unload/reload work files
• WI -- Index Rebuild work files
• WS -- Sort output work files
• WD -- DBKEYS work files

24
Express Reorg Work Files

• CREATE DSMODEL creates a temporary model of data
  set attributes
• Must include information
• Where to create work file
• Space
• Blocksize

25
CREATE DSMODEL
26
Express Reorg Work Files

• Sizing work files
• Very hard to estimate
• Run REORG STOP AFTER UNLOAD with dummied work
  files.

27
Express Reorg Work Files

• Number of files grows exponentially example of
  WU files

Divide n ways No reuse Reuse workfiles
1 11 9
2 32 24
3 63 45
4 104 72
5 155 105
28
Output
29
Output
30
Output
31
Output
32
Output
33
Output
34
Output
35
Output
36
Output
37
Express Reorg Processing

• If Target files same as Source files
• REORG STOP AFTER UNLOAD
• If not SHARE, UNLOCK AREAS
• FORMAT files
• REORG with STOP AFTER CLEANUP SUBMIT
• Backup
• IDMSDBAN (optionally)
• PRINT SPACE (optionally)

38
Express Reorg Processing

• If Target files with different DDnames
• Format new files
• REORG with STOP AFTER CLEANUP
• Backup
• PRINT SPACE
• IDMSDBAN (optionally)

39
Express Reorg Processing

• If Target files different but same DDnames
• Format new files
• REORG with STOP AFTER UNLOAD(REORG jcl and jcl
  in RORGJCL have old DDnames)
• REORG with STOP AFTER CLEANUP and SUBMIT(REORG
  jcl and jcl in RORGJCL have new files)

40
Job Submission

• Automatic job submission
• Manual job submission
• Restarting REORG

41
Automatic Job Submission

• Jobs automatically submitted
• at end of SETUP phase
• If SUBMIT parameter is specified
• REORG will submit
• 1 job for each slice
• 1 job for each index group
• Maximum jobs submitted 2x no slices

42
Manual Job Submission

• Can manually submit REORG job(s)
• Job will examine control file and determine if
  there is work to be done
• If no work, waits until some other job completes
  a task
• If work is needed, it performs that work

43
Restarting REORG

• If a REORG job fails, other jobs will try to
  restart the task that failed
• If problem is not temporary (work file too
  small), problem needs to be corrected, then REORG
  can be submitted with SUBMIT.
• In some severe cases, you might need to restart
  REORG from the beginning

44
Considerations

• Relative REORG performance
• Efficiency of the reorganized database
• Work files

45
Relative REORG performance

• Usually runs in less time than normal
  UNLOAD/RELOAD, however..
• Best to try it out first

46
DB Efficiency

• DB might not be as efficient as reorganized by
  UNLOAD/RELOAD
• REORG reloads in forwards fashion whereas
  UNLOAD/RELOAD reloads in backwards fashion
• REORG can result in more CALC VIA overflow
  especially if long sets

47
Work Files

• Work files HAVE to be on disk
• Hard to estimate their size

48
Tests Results
67.71
TSTDML (46,116 pages)
50.5
58.4
33.54
63.8
64.5
28.16
24.47
24.04
Unload/reload
Reorg 2 ways
Reorg 3 ways
Reorg 4 ways
Reorg 5 ways
49
Tests Results
40.9
1 Area unloaded (120,000 pages going to
147,000 pages) 1 Area affected (45,000
pages)
27.2
29.78
44.0
22.89
Unload/reload
Reorg 2 ways
Reorg 3 ways
50
Tests Results
1 Area unloaded (427,720 pages _at_5064 going to
428,400 pages _at_ 7548) 3 Areas affected
(39,000 pages 160,000 pages 294,000 pages)
168.27
46.9
51.1
89.31
80.23
Unload/reload
Reorg 3 ways
Reorg 5 ways
51
Conclusion

• Can reduce the time to reorganize a database up
  to 50-60
• Easy to use
• Needs lots of disk space