JCL Conversion - Related Documents

 JCLcnv1demo.htm - DEMO conversions, sample JCL, scripts, executions
 JCLcnv2real.htm - comprehensive instructions for REAL conversions
 JCLcnv3aids.htm - conversion AIDS (cross-refs,tips,mass changes,etc)
 JCLcnv4gdg.htm  - GDG file handler from UV Software                 *THIS DOC*
 DATAcnv1.htm    - Data file conversion - Original Documentation
 DATAcnv2.htm    - Data File Conversion - Comprehensive & Complete

GDG handler - Contents by Part

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Contents by Part

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_1 Vancouver Utilities GDG system

Part 1 Contents

The Vancouver Utilities provides GDG (Generation Data Group) processing similar to the mainframe facility. In fact, the VU GDG system provides significantly enhanced features that go well beyond the original implementation provided on the mainframe.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1A1. Vancouver Utilities GDG system - Overview

Vancouver Utility GDG system - Basic Features

1. After mainframe GDG files are converted (from EBCDIC to ASCII, preserving packed/binary fields), they are copied to the testdata/proddata directories & renamed with suffix '_000001' (1st generation using 6 digit#).
   
   
2. When mainframe JCL is converted to Korn shell scripts, GDG files are identified with a trailing '_' underscore only.
   
   
3. The JCL calls Korn shell 'functions' to determine the appropriate generation. 'exportgen0' defines the latest/highest generation (for reading). 'exportgen1' defines the next generation (for writing).
   
   
4. Early versions of exportgen1 incremented the generation# immediately (vs delaying until the job terminated normally as on the mainframe). If a multi-step job terminated abnormally, you were advised to use the 'restart' facility (provided for every step by the VU JCL converter). But if for some reason you wanted to restart from the begining, you had to manually remove any new generations created before the abterm.
   
   
5. The abterm problem was solved in 2007 by an enhanced version of 'exportgen1' and new function 'jobend51' called at Normal End of Job. 'exportgen1' now assigns new generations to the 'jobtmp' directory & at Normal EOJ, function 'jobend51' will move the jobtmp files back to the $RUNDATA directories. This allows you to rerun abterm jobs from the begining , but if you do restart, you will be prompted if it is OK to move any jobtmp files back to the $RUNDATA directories.
   
   
6. JCL converter option g0/g1 is provided to control whether new GDG's are written to the original directory or to the jobtmp directory for restore at end of job, but g1 is now the default.
   
   
7. Early versions of exportgen1 used a hard-coded number of generations to be accumulated before erasing older generations, unless you coded the gdg=... keyword on all exportgen1's for the file, which would be inconvenient.
   
   
8. The current system provides an Indexed control file which exportgen1 accesses to get the desired number of generations for each file.
   
   
9. We provide a utility to automatically create the indexed file by extracting all GDG filenames from all JCL/scripts, sorting,& dropping duplicates. You then edit the control file to modify the default number of generations as desired & run another utility job to load the indexed file.
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1A2. Vancouver Utilities GDG system - Introduction & Overview

Vancouver Utility GDG system - Advanced Features

The indexed control file allows us to provide significant enhancements that go well beyond the original GDG implementation on the mainframe. See Part_9 for examples of these 'advanced features', but here are brief descriptions.

1. You can specify a prior generation of a file to be defined by exportgen0 for the next run only, or until a specified expiry date/time is reached. This avoids having to modify the JCL/scripts. Note that you only have to modify one entry in the control file, whereas you could easily miss some of the possible multiple references in one or more JCL/scripts.
   
   
2. You can specify range processing for a series of prior generations This is a powerful feature that 1 customer is already planning to make good use of. On the mainframe he had 30 tapes that had to be processed one at a time by a batch COBOL program & the operator simply mounted the tapes in turn.
   
   
3. On the unix/linux system, all tape files are automatically converted to generation disc files, so to process the 30 tapes (now GDG disc files) separately, the customer might have to modify the JCL to hard-code the next desired generation# or he might specify via an operator keyin.
   
   
4. With the enhanced GDG system, he edits the control file specifying the low to high range of generation#s to be processed. He can then run the batch job 30 times to process the 30 files (with no changes to JCL/scripts & no operator generation# keyins).
   
   
5. This is possible because 'exportgen0' uses the current next generation#, which is incremented by the 'jobend51' function at normal EOJ. At the high end of the range, processing returns to normal, but an historical record is maintained in the indexed file database, making it easier to reactivate if the same cycle is repeated periodically.
   
   
6. You can specify 'opt=c' (confirm) on control file entries to cause exportgen0 to prompt the operator to confirm the generation# selection, or to enter an alternative 6 digit generation#.
   
   
7. The current system maintains a history of the last 30 accesses by exportgen0 and exportgen1. You can look at the control file (with uvhd) and for each of the last 30 accesses, you can see: date:time, generation# selected, lowest:highest generations on-file, jobname, step#,& program of last access.
   
   
8. Each of the last 30 entries that define access history is identified with one of three status codes: 'G0'=exportgen0(reading current/selected gen), 'G1'=exportgen1(writing next gen), or 'AB'=ABnormal Termination (indicates file not moved from jobtmp/GDG/ back to RUNDATA/data1/).
   
   

   GDG control file history updates optional

As of June 2012, GDG control file history updates is optional depending on environmental variable GDGCTLUPDT, default GDGCTLUPDT=YES (in common_profile). Inhibiting history updates might speed up on very busy system and reduce possibility of file locks & corruption. GDGCTLUPDT=YES is required to use some advanced features & could be useful debugging problems with GDG files.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1B1. JCL conversion - Generation files

GDG JCL Conversion Examples

Mainframe GDG files are implemented via unix/linux Korn shell functions (exportgen0, exportgen1/2/..., exportgen-1/-2/...,& exportgenall). These functions are listed later beginning on page '5J0'. Here is a sample conversion (before & after) of GDG files:

mainframe JCL - BEFORE conversion

 //JGL220   JOB  (1234),'TEST/DEMO MVS JCL CONVERSION'
 //* see www.uvsoftware.ca/jclcnv4gdg.htm
 //* mainframe GDG files converted to functions exportgen0 & exportgen1
 //* UPDATE GL ACCOUNT MASTER WITH GLTRANS
 //STEP020  EXEC PGM=CGL200,COND=(4,LT,STEP010),PARM=&YEAREND
 //GLTRANS  DD DSN=GL.ACCOUNT.TRANS(0),DISP=OLD
 //GLMSOLD  DD DSN=GL.ACCOUNT.MASTER(0),DISP=OLD
 //GLMSNEW  DD DSN=GL.ACCOUNT.MASTER(+1),DISP=(,CATLG,DELETE),
 //            UNIT=DISK,SPACE=(TRK,(50,50),RLSE),
 //            DCB=(MODEL.DSCB,LRECL=128,BLKSIZE=6118,RECFM=FBA)

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1B2. JCL conversion - GDG files

JCL - AFTER conversion to Korn shell script

 #!/bin/ksh
 ##JGL220   JOB  (1234),'TEST/DEMO MVS JCL CONVERSION'
 export jobid2=jgl220 JOBID2=JGL220; scriptpath="$0"; args="$*"
 if [[ -z "$jobid1" ]]; then export jobid1=$jobid2; fi
 for arg in $args; do if [[ "$arg" == *=* ]]; then export $arg; fi; done
 integer JCC=0 SCC=0 LCC=0  # init step status return codes
 autoload jobset51 jobend51 jobabend51 logmsg1 logmsg2 stepctl51
 autoload exportfile exportgen0 exportgen1 exportgen2 exportgenall exportgenx
 . $APPSADM/env/stub.ini  #<-- for control-M (see notes in env/stub.ini)
 jobset51    # call function for JCL/script initialization
 goto
 S0000=A
 # * see www.uvsoftware.ca/jclcnv4gdg.htm
 # * mainframe GDG files converted to functions exportgen0 & exportgen1
 # * UPDATE GL ACCOUNT MASTER WITH GLTRANS
 #1======================= begin step#S0010 CGL200 ========================
 S0010=A
 stepctl51;
 goto
 export JSTEP=S0010; ((XSTEP+=1)); SCC=0; alias goto="";
 logmsg2 "******** Begin Step $JSTEP cgl200 (#$XSTEP) ********"
 ##STEP020  EXEC PGM=CGL200,COND=(4,LT,STEP010),PARM=&YEAREND
 export PROGID=cgl200
 export PARM="$YEAREND"
 exportgen0 0 GLTRANS data1/gl.account.trans_
 exportgen0 0 GLMSOLD data1/gl.account.master_
 exportgen1 +1 GLMSNEW data1/gl.account.master_
 #exportgen1 GDGs written to jobtmp/$jobid2 restored to outdir at Normal EOJ
 logmsg2 "Executing--> cobrun $ANIM $CBLX/cgl200"
 #3----------------------------------------------------------------------
 cobrun $ANIM $CBLX/cgl200
 #4----------------------------------------------------------------------
 SCC=$?; S0010C=$SCC; ((JCC|=SCC)); S0010R=1; alias goto="";
 logmsg2 "SCC = $SCC"
 if ((SCC>4 || SCC<0))
    then logmsg2 "ERR: step#$JSTEP cgl200 abterm $SCC"
    alias goto="<<S9900=\A"; fi
 goto
 #8======================================================================
 S9000=A
 jobend51 #move any new GDG files from jobtmp to intended outdirs
 logmsg2 "JobEnd=Normal, JCC=$JCC, StepsExecuted=$XSTEP, LastStep=$JSTEP"
 exit 0 #ver:20160721 a2b0c0d1e2f0g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1
 #9======================================================================
 S9900=A
 jobabend51 #report GDGs NOT moved from jobtmp/subdirs to outdirs
 logmsg2 "JobEnd=AbTerm, JCC=$JCC,Steps=$XSTEP/$JSTEP" RV ACK
 exit $JCC  # JCL converted to ksh by UVSW jclunix51 20170207:174049

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1B3. JCL conversion - GDG files

GDG files isolated from JCL/scripts

Here are just the GDG files extracted from the JCL & the Korn shell scripts:

GDG files BEFORE conversion

 //GLMSOLD  DD DSN=GL.ACCOUNT.MASTER(0),DISP=OLD
 //GLMSNEW  DD DSN=GL.ACCOUNT.MASTER(+1),DISP=(,CATLG,DELETE),
 //            UNIT=DISK,SPACE=(TRK,(50,50),RLSE),
 //            DCB=(MODEL.DSCB,LRECL=128,BLKSIZE=6118,RECFM=FBA)

GDG files AFTER conversion

 exportgen0 0 GLMSOLD data1/gl.account.master_     #<-- current generation
 exportgen1 +1 GLMSNEW data1/gl.account.master_    #<-- next generation

Notes re GDG file conversions

1. In the mainframe JCL the '(0)' suffix on the DSN indicates the current generation, and the '(+1)' indicates the next generation.
   
   
2. In the converted JCL/script, GDG files are identified with a trailing '_' underscore. Function 'exportgen0' is called to determine the current generation, and function 'exportgen1' to determine the next generation (for COBOL or utility programs).
   
   
3. After the data files were converted from the mainframe to unix/linux, they are renamed as shown below.
   
   
4. After conversion from the mainframe, we might store the 1st generation on unix/linux as gl.account.master_000001. After 4 updates, we would have 5 generations as shown below.
   
   

      data1/gl.account.master_000001     <-- 1st generation after conversion
      data1/gl.account.master_000002
      data1/gl.account.master_000003
      data1/gl.account.master_000004
      data1/gl.account.master_000005     <-- current generation after 4 updates

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1B4. JCL conversion - GDG files

new GDG's written to jobtmp/...

 /home/userxx        <-- homedirs for testing, separate file system for production
 :-----testdata         <-- data dirs (defined as $RUNDATA in JCL/scripts)
 :-----proddata         <-- test or production, different for different users
 :     :------data1       - datafiles
 :     :      :-----gl.account.master_000001 <--input
 :     :      :-----gl.account.master_000002 <-- output from jobtmp/ at EOJ
 :     :------ctl         - GDG control file
 :     :------jobtmp      - temp dir for new generations, etc
 :     :      :-----$JOBID - separate subdirs by jobname
 :     :      :     :-----data1 - new GDGs here until successful EOJ
 :     :      :     :     :-----gl.account.master_000002
 :     :------...         - other subdirs (see JCLcnv1demo.htm#3B3)

The GDG file examples on the previous page would give you the impression that the new generation output would be written directly to the data1/... directory. The files are defined in JCL/scripts using exportgen0 & exportgen1 as follows:

 exportgen0 0 GLMSOLD data1/gl.account.master_      #<-- current generation
 exportgen1 +1 GLMSNEW data1/gl.account.master_     #<-- next generation

But exportgen1 will actually write the new generation to the jobtmp/... directory, until the job reaches Normal termination, where the jobend51 function will restore to the intended directory. The effect of exportgen1 is to generate the following actual 'export':

 export ACTLIST=$RUNDATA/jobtmp/$JOBID/data1/gl.account.acntlist_000002
 #=====================================================================

At successful End of Job, the jobset51 function generates the following move:

 mv $RUNDATA/jobtmp/$JOBID/data1/gl.account.acntlist_000002 $RUNDATA/data1/
 #=========================================================================

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1B5. JCL conversion - GDG files

Generation# reset at _900000

When the generation# reaches _900000, all files in the filename group will automatically be resequenced from _000001. The end of job function 'jobset51' will call script 'gdgreset1' to do this for files exceeding _900000.

We also provide batch job 'gdgreset2' to resequence all files over _900000 back to _000001. Note that if you had a generation file that was updated once a day it would take 2500 years to exceed 900000.

We believe this GDG system is more reliable than the mainframe GDG system because it depends only on the generations currently in the directory & not some other file that could get corrupted.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1C1. JCL conversion - GDG files

sample GDG control file

      gl.account.acntlist_                     gdg=08
      gl.account.master_                       gdg=10
      gl.account.trans_                        gdg=10
      py.payroll.master_                       gdg=26
      py.time.cards_                           gdg=52

1. During conversion, the GDG control file ($RUNLIBS/ctl/gdgctl51) is automatically created by extracting 'exportgen1's from all JCL/scripts.
   
   
2. JCL conversion script 'jcl2ksh51A' calls 'jclgdgctl51' to extract the GDG files into a text file & 'gdgload51' is then called to load the text file into the Indexed file used by exportgen1 to determine the nummber of generations for each file.
   
   
3. You will be prompted for the default number of generations per file.
   
   
4. After initial creation, you can edit the text file to specify the number of generations desired for each file, and then re-run 'gdgload51' to re-load the Indexed file used by exportgen1.
   
   
5. Note that the sub-directory (data1/) is not part of the control file (it is defined in the JCL/script). The JCL conversion initially assigns all files to the data1/ subdir, but you could subsequently assign data files anywhere.
   
   
6. exportgen1 assumes that the lowest level of subdir (usually data1/) is in $RUNDATA (since JCL/scripts perform 'cd $RUNDATA' at the begining).
   
   
7. But you can assign the higher levels of directories anywhere. You might setup your own $SYMBOLS if outside of the initial $RUNDATA directory
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1C2. JCL conversion - GDG files

Initial creation of GDG control-file

The GDG control file is initially created at the end of the JCL conversion super script 'jcl2ksh51A'. It calls 'jclgdgctl51' to extract GDG files into a text file 'gdgctl51'. Script 'gdgload51' is then called to load the text file into the Indexed file (gdgctl51I.dat & gdgctl51I.idx) used by exportgen1 to determine the nummber of generations for each file.

Here is a brief review of the JCL conversion jobs ALREADY EXECUTED in JCLcnv1demo.htm#Part_3 or JCLcnv2real.htm#Part_1.

 #1a. login userxx, or as desired

 #1b. cdl --> $RUNLIBS (/home/userxx/testlibs, /p1/cnv/cnv1/testlibs
           - see directories,profiles,$RUNLIBS,etc at JCLcnv1demo.htm#Part_1

 #2. jcl2ksh51A all       <-- JCL conversion already completed
     ==============        - left all converted scripts in jcl3/...

 #3. uvcopy jclgdgctl51,fild1=jcl3,filo2=ctl/gdgctl51,uop=g07
     ========================================================
     option g07 = default generations, override as desired

 #4. uvcopy jclgdgctl51     <-- same as above (files default as shown)
     ==================       - sample output shown below

 #5. cp ctl/gdgctl51 $RUNDATA/ctl/
     =============================
     - copy from $RUNLIBS/ctl/. to $RUNDATA/ctl/.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1C3. JCL conversion - GDG files

Edit no of generations desired

The JCL conversion defaults the number of generations as 07 for all files, unless you have transferred the LISTCAT reports from the mainframe & extracted the numbers into a control file as described at JCLcnv2real.htm#Part_3

Here is how you could assign the desired number to each file if the LISTCAT reports were not used in the conversion or you need to adjust the numbers.

 #1a. login userxx --> $HOME (/home/userxx/)

 #1b. cdd --> $RUNDATA (/home/userxx/testdata)

 #2. vi ctl/gdgctl51    <-- edit to modify no of generations as desired
     ===============

      gl.account.master_                       gdg=07
      gl.account.trans_                        gdg=15
      py.payroll.master_                       gdg=115

 #3. uvcopy gdgload51,fili1=ctl/gdgctl51,filo1=ctl/gdgctl51I
     =======================================================
     - load Indexed file to supply no of generations to exportgen1

 #3a. uvcopy gdgload51   <-- same but easier (files default as shown above)
      ===============

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1C4. JCL conversion - GDG files

update generations directly in Indexed file

If you have only a few desired updates to genration numbers, you could update the Indexed file directly with 'uvhd' (rather than updating the text file & reloading the Indexed file). Do not attempt to update the Indexed file with vi. 'uvhd' is safer since it will never change the record-size (possible with vi).

Note that when the text file is loaded into an Indexed file, you get 2 partitions (gdgctl51I.dat & gdgctl51I.idx).

 uvhd ctl/gdgctl51I.dat r2048uh1   <-- execute uvhd to display gdgctl file
 ===============================     - options: r2048=record-size, u=update
                                       h1=chars only (omit hex zones/digits)

                   10        20        30        40        50        60
 r#     2 0123456789012345678901234567890123456789012345678901234567890123
     2048 gl.account.master_                                 gdg=07
       64 next=000000(000000:000000)000000:000000 jgl200    170203:084657
      128 170203:084657:01+000004(000001:000003)jgl200   S0020 cgl200
      192 170203:084657:00 000003(000001:000003)jgl200   S0020 cgl200
      256 000000:000000:___000000(000000:000000)_______  _____ _________
      320 000000:000000:___000000(000000:000000)_______  _____ _________
                    ------- 25 lines omitted --------
     1920 000000:000000:___000000(000000:000000)_______  _____ _________
     1984 000000:000000:___000000(000000:000000)_______  _____ _________>.

 --> u 55(2),'26'     <-- update no of generations from '07' to '26'
     ============
 --> ...              <-- could browse to other records & update
 --> q                <-- quit uvhd

Notes re updating Indexed file directly

1. Do not attempt to update the Indexed file with 'vi', it would be very easy to corrupt the file. Inserting or deleting bytes would corrupt the file from that point onwards.
   
   
2. It is safe to update with uvhd, because it will never change the record-size. Of course you must be careful not to change the key (1st 44 bytes filename).
   
   
3. The above shows the GDG control file record for the gl.account.master_ after 1 execution of jgl200.ksh COBOL program cgl200.cbl. See jgl200.ksh listed on page '1B2'. Here are I/O file definitions GLMSOLD/GLMSNEW.
   
   

      exportgen0 0 GLMSOLD data1/gl.account.master_    #<-- current generation
      exportgen1 +1 GLMSNEW data1/gl.account.master_   #<-- next generation

4. File I/O entries are inserted at byte 128 & prior entries are pushed down. This provides history of the last 30 I/O's to each GDG file.
   
   

           date   time (##)  gen# note5  prior#  job     step  program
      128 170203:084657:01+000004(000001:000003)jgl200   S0020 cgl200
      192 170203:084657:00 000003(000001:000003)jgl200   S0020 cgl200

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1C5. JCL conversion - GDG files

Indexed file record format ctl/gdgctl51I.dat

                   10        20        30        40        50        60
 r#     2 0123456789012345678901234567890123456789012345678901234567890123
     2048 gl.account.master_                                 gdg=07
       64 next=000000(000000:000000)000000:000000 jgl200    170203:084657
      128 170203:084657:01+000004(000001:000003)jgl200   S0020 cgl200
      192 170203:084657:00 000003(000001:000003)jgl200   S0020 cgl200
      256 000000:000000:___000000(000000:000000)_______  _____ _________
                    ------- 25 lines omitted --------
     1984 000000:000000:___000000(000000:000000)_______  _____ _________>.

000:049 - gl/account.master_ <--filename with trailing underscore GDG ID 051:056 - gdg=.. - generations for this file 058:062 - opt=c - option for oprtr confirm y or enter alternate gen#

  064:102 - next=... - see advanced functions in Part_9
            next=gen###(lowgen:higen#)yymmdd:HHMMSS
            next=000000(000000:000000)000000:000000
            zeros template/place-holder generated if not used

  128:191 - 1st entry of multiple action history entries
          yymmdd:HHMMSS:g0_123456(123456:123456)JOBNAME  STEP  PROGRAM +
  128:191 170203:084657:01+000004(000001:000003)jgl200   S0020 cgl200
          - stored by exportgen0, exportgen1, jobend51?, jobend52?
          - action entries shifted down & latest action stored here
  190:190 - gdgupok1 '+' flag if moved back from jobtmp/subdir (exportgen1)

  192:255 - update history#2 (shifted from 128-191 by jobend51)
  192:255 170203:084657:00 000003(000001:000003)jgl200   S0020 cgl200

256:319 - update history#3 (shifted from 192-255) 320:1983- --- entries #4-#29 --- 1984:2045- update history#30 2046:2046- '>' marks end of record data 2047:2047- x'0A' Indexed record status byte

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1C6. JCL conversion - GDG files

unload Indexed file, edit no of generations,& reload

If you had updated the Indexed file directly & then wanted to do a mass update with vi, you should unload the Indexed file to the text file, edit,& reload (in order not to lose any updates made directly to the Indexed file).

 #1. login mvstest --> /home/mvstest

 #2. cdd  (alias cdd='cd $RUNDATA') --> /home/mvstest/testdata
     ===

 #3. uvcopy gdgunload51,fili1=ctl/gdgctl51I,typ=ISF,filo1=ctl/gdgctl51,typ=LSTt
     ==========================================================================

 #3a. uvcopy gdgunload51   <-- same as above, files default as shown above
      ==================

 #4. vi ctl/gdgctl51    <-- update the text file
     ===============

     data1/gl.account.acntlist_  gdg=07
     data1/gl.account.master_    gdg=10
     data1/gl.account.trans_     gdg=20
     data1/py.payroll.master_    gdg=26
     data1/py.time.cards_        gdg=52

 #5. uvcopy gdgload51,fili1=ctl/gdgctl51,filo1=ctl/gdgctl51I
     ======================================================
     - load Indexed file to supply file info to JCL converter

 #5a. uvcopy gdgload51    <-- same as above (files default as shown)
      ===============

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

1C7. JCL conversion - GDG files

Alternate location for GDG control file

The location of the GDG control file is determined by the value of $GDGCTL in the common_profile (called by .profile or .bash_profile). Here is line 75 of $APPSADM/env/common_profile_uv.

 if [[ -z "$GDGCTL" ]]; then export GDGCTL=$RUNDATA/ctl; fi #<-- set default
 #=========================================================

 # export GDGCTL=$APPSADM/ctl  #<-- uncomment to override $RUNDATA to $APPSADM
 #===========================

If you have multiple RUNDATA directories you might want to activate 'export GDGCTL=$APPSADM/ctl' to have only 1 GDG control file vs multiple.

If the GDG control file has already been created in $RUNDATA/ctl, you would now copy to $APPSADM as follows:

 cp $RUNDATA/ctl/gdgctl51* $APPSADM/ctl
 ======================================
 - copy text file gdgctl51 and Indexed file (gdgctl51I.dat & gdgctl51I.idx)

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_2 JCL conversion - GDG files

                        Part 2 - Contents

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2A1. JCL conversion - GDG files

GDG logic illustrated

This illustration will show you how we prevent incrementing the generation# on abnormal terminations. We will use the 'jgl200.ksh' test/demo JCL/script. See complete listing at JCLcnv1demo.htm#2E1. Here we will show only lines relevant to explaining the GDG logic.

jgl200.jcl to illustrate GDG logic

 //JGL200   JOB  (1234),'TEST/DEMO MVS JCL conversion - GDG files'
 //* SORT TRANSACTIONS & UPDATE GL ACCOUNT MASTER
 //*
 //STEP010  EXEC PGM=SORT,REGION=2048K,PN=STEPA
 //SORTIN   DD DSN=GL.ACCOUNT.TRAN1,DISP=OLD
 //SORTOUT  DD DSN=GL.ACCOUNT.TRANS(+1),DISP=(,CATLG,DELETE),
 //            DCB=(LRECL=80,BLKSIZE=8000,RECFM=FB),
 //            SPACE=(TRK,(50,50),RLSE),UNIT=DISK
 //SYSIN    DD *
 SORT FIELDS=(1,8,CH,A,69,12,CH,A)
 /*
 //*============================ step# 0020 ============================
 //STEP020  EXEC PGM=CGL200
 //GLTRANS  DD DSN=GL.ACCOUNT.TRANS(+1),DISP=OLD
 //GLMSOLD  DD DSN=GL.ACCOUNT.MASTER(0),DISP=OLD
 //GLMSNEW  DD DSN=GL.ACCOUNT.MASTER(+1),DISP=(,CATLG,DELETE),
 //            UNIT=DISK,SPACE=(TRK,(50,50),RLSE),
 //            DCB=(MODEL.DSCB,LRECL=128,BLKSIZE=6118,RECFM=FBA)

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2A3. JCL conversion - GDG files

jgl200.ksh to illustrate GDG logic

 ##JGL200   JOB  (1234),'TEST/DEMO MVS JCL CONVERSION'
        --- omitting lines not relevant to GDG logic ---
 jobset51    # call function for JCL/script initialization          <--Note1,2
        --- omit step 1 SORT except for I/O files ---
 exportfile  SORTIN data1/gl.account.tran1
 exportgen1 +1 SORTOUT data1/gl.account.trans_                      <--Note3
        --- omit step 1 SORT creates data1/gl.account.trans_
 exportgen1 +1 GLTRANS data1/gl.account.trans_                      <--Note4
 exportgen0 0 GLMSOLD data1/gl.account.master_                      <--Note5
 exportgen1 +1 GLMSNEW data1/gl.account.master_                     <--Note6
 #exportgen1 GDGs written to jobtmp/$jobid2/GDG restored at Normal EOJ
 logmsg2 "Executing--> cobrun $ANIM $CBLX/cgl200"
 #3----------------------------------------------------------------------
 cobrun $ANIM $CBLX/cgl200
 #4----------------------------------------------------------------------
 SCC=$?; S0020C=$SCC; ((JCC|=SCC)); S0020R=1; alias goto="";
 logmsg2 "SCC = $SCC"
 if ((SCC>4 || SCC<0))                                                  <--Note7
    then logmsg2 "ERR: step#$JSTEP cgl200 abterm $SCC"
    alias goto="<<S9900=\A"; fi
 goto
 #8======================================================================
 S9000=A                                                                <--Note8
 jobend51 #move any new GDG files from jobtmp to intended outdirs
 logmsg2 "JobEnd=Normal, JCC=$JCC, StepsExecuted=$XSTEP, LastStep=$JSTEP"
 exit 0 #ver:20160721 a2b0c0d1e2f0g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1
 #9======================================================================
 S9900=A                                                                <--Note9
 jobabend51 #report GDGs NOT moved from jobtmp/subdirs to outdirs
 logmsg2 "JobEnd=AbTerm, JCC=$JCC,Steps=$XSTEP/$JSTEP" RV ACK
 exit $JCC  # JCL converted to ksh by UVSW jclunix51 20160722:144459

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2A3. JCL conversion - GDG files

Notes re jgl200.ksh above

1. 'jobset51' sets up jobtmp/JGL100/GDG/ to hold any new GDG files until normal EOJ. See jobset51 listed on page '11B1'
   
   
2. jobset51 performs 'cd $RUNDATA', so all files are defined in the script relative to $RUNDATA, which is /home/mvstest/testdata/... for the test/demos documented in JCLcnv1demo.htm.
   
   
3. Most lines are omitted for step 1 SORT, except for the I/O files exportgen1 +1 SORTOUT data1/gl.account.trans_ <--Note3 =============================================
   
   
4. Step1 SORT output file is input on step 2 COBOL exportgen1 +1 GLTRANS data1/gl.account.trans_ <--Note4 ============================================= Note that this step2 input is defined the same as the output on step1 - using 'exportgen1 +1' & not 'exportgen 0' as usual for input files
   
   
5. Step2 exportgen0 defines the current generation for input. Assuming 3 generations existing (000001,000002,000003), the result would be:
   
   

    exportgen0 0 GLMSOLD data1/gl.account.master_000003                     <--Note5
    ===========================================================

6. Step2 exportgen1 defines the next generation for output, effectively:
   
   

    exportgen1 +1 GLMSNEW data1/gl.account.master_000004                    <--Note6
    ===========================================================
    #exportgen1 GDGs written to jobtmp/$jobid2/GDG restored at Normal EOJ

7. Return code from the COBOL program is tested & control goes to S9900 on failure(non-zero) & S9000 on success(zero).
   
   
8. At S9000 (Normal EOJ), function 'jobend51' will move GDG files from jobtmp/GDG/ back to $RUNDATA/data1/.
   
   
9. At S9900 (Abnormal EOJ), jobset51 is not called, so any new GDG files are not moved back to $RUNDATA/data1/. This means you can rerun the job without having to remove any new GDGs (so exportgen0 will get intended input).
   
   
10. Note that for a multi-step JCL/script, you might use the restart facility to restart at the step that failed. On a restart jobset51 will prompt you & copy any new GDG files back to $RUNDATA/data1/.
    
    

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2A4. JCL conversion - GDG files

GDG control-file for test/demos

 # gdgctl51 - GDG control file, by Owen Townsend, UV Software, Feb 2017
 #          - to demo specified next gen# & range processing
 #          - see www.uvsoftware.ca/JCLcnv4gdg.htm
 #
 gl.account.acntlist_          gdg=07
 gl.account.master_            gdg=15
 gl.account.trans_             gdg=15

$RUNDATA subdirs BEFORE jgl200.ksh test GDG

 /home/mvstest/testdata:
 drwxrwxr-x    2 mvstest  users        4096 Jun  1 15:03 data1
 drwxrwxr-x    2 mvstest  users        4096 Jun  1 15:03 joblog
 drwxrwxr-x    3 mvstest  users        4096 Jun  1 15:03 jobtmp

 /home/mvstest/testdata/data1/gl*
 -rw-rw-r--    1 mvstest  users        8720 May 31 17:33 gl.account.master_000001
 -rw-rw-r--    1 mvstest  users        8720 May 31 17:33 gl.account.master_000002
 -rw-rw-r--    1 mvstest  users        8720 May 31 17:33 gl.account.master_000003

 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 gl.account.trans_000001
 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 gl.account.trans_000002
 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 gl.account.trans_000003

/home/mvstest/testdata/jobtmp: total 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2B1. JCL conversion - GDG files

run#1 jgl200.ksh & observe GDG results

This assumes you have setup the 'mvstest' user & copied the demo files from /home/uvadm/mvstest/* to /home/mvstest/... as instructed on pages '1P1'-'1P3'.

 #1. login mvstest --> /home/mvstest           (see dtree on page '1A1')

 #2. cdd  (alias cdd='cd $RUNDATA') --> /home/mvstest/testdata

 #3. rm data1/*[4-9]         <-- remove excess generations
     ===============             (from prior testing)

 #4. cp -r data1 data1save   <-- save data1/... files to retore after tests
     =====================

 #5. testdatainit            <-- remove old test output files
     ============                (jobtmp/*, joblog/*, etc)
                               - also prompts to reload the GDG control file

 #5a. uvcopy gdgload51,fili1=ctl/gdgctl51,filo1=ctl/gdgctl51I
     ========================================================
     - testdatainit prompts to reload GDG control file

 #6a. jgl200.ksh                  <-- execute JCL/script WITHOUT LOGGING
      ==========                    - BUT use alternative below to capture joblog

 #6b. joblog jgl200.ksh           <-- BETTER ALTERNATIVE to capture console log file
      =================             - see listing on the next page

 #7a. vi joblog/jgl200.log        <-- view console log
      ====================

 #7b. uvlp12 joblog/jgl200.log    <-- OR, print console log
      ========================

 #8a. ls -l data1                 <-- inspect contents of data1 subdir
      ===========                   - see before above & after below
                                      on successful & failed runs

 #8b. ls -l jobtmp/jgl200/GDG/    <-- inspect contents of jobtmp/...
      ========================      - on good runs, files already moved out
                                    - on failure, new GDG files still there

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2B2. JCL conversion - GDG files

jgl200 run#1 - Terminated Normally

 170203:122616:jgl200: Begin Job=jgl200
 170203:122616:jgl200: /home/mvstest1/testlibs1/jcls/jgl200.ksh
 170203:122616:jgl200: Arguments: jgl200.ksh
 170203:122616:jgl200: ProcessID=14018 RUNDATE=20170203
 170203:122616:jgl200: RUNLIBS=/home/mvstest1/testlibs1
 170203:122616:jgl200: RUNDATA=/home/mvstest1/testdata1
 170203:122616:jgl200: JTMP=jobtmp/jgl200 SYOT=sysout/JGL200
 170203:122616:jgl200: LOGNAME=mvstest1 TESTPROD=T000
 170203:122616:jgl200: HOSTNAME=uvsoft5.uvsoftware.ca
 170203:122616:jgl200: ******** Begin Step S0010 sort (#1) ********
 170203:122616:jgl200: file: SORTIN=data1/gl.account.tran1 fsize=4K
 170203:122616:jgl200: gen1+1 SORTOUT=jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.trans_000004 gens=7
 170203:122616:jgl200: file: SYSIN=/home/mvstest1/testlibs1/parms/pgl200s1 fsize=4K
 170203:122616:jgl200: Executing--> uvsort "fili1=data1/gl.account.tran1,typ=RSF,rcs=80,filo1=...
 170203:122616:JGL200: uvsort fili1=data1/gl.account.tran1,filo1=jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.tra
 170203:122616:JGL200: EOF fili01 20 rds, 1600 size; data1/gl.account.tran1
 170203:122616:JGL200: sort phase complete, used 512 megabytes, sort output begin
 170203:122616:JGL200: EOF filo01 20 wrts, 1600 size; jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.trans_000004
 170203:122616:jgl200: SCC = 0
 170203:122616:JGL200: StepTimes: S0010 sort Begun=12:26:16 End=12:26:16 Elapsed=00:00:00
 170203:122616:jgl200: ******** Begin Step S0020 cgl200 (#2) ********
 170203:122616:jgl200: gen1+1 GLTRANS=jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.trans_000004 gens=7
 170203:122616:jgl200: gen0 GLMSOLD=data1/gl.account.master_000003 insize=16K
 170203:122616:jgl200: gen1+1 GLMSNEW=jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.master_000004 gens=7
 170203:122616:jgl200: Executing--> cobrun -F /home/mvstest1/testlibs1/cblx/cgl200
 170203:122616:jgl200: SCC = 0
 170203:122616:jgl200: moving /home/mvstest1/testdata1/jobtmp/jgl200/GDG/files back to intended directories
 /home/mvstest1/testdata1/data1/gl.account.master_000004
 /home/mvstest1/testdata1/data1/gl.account.trans_000004
 170203:122616:JGL200: StepTimes: S0020 cgl200 Begun=12:26:16 End=12:26:16 Elapsed=00:00:00
 170203:122616:JGL200: Job Times: Begun=12:26:16 NormalEnd=12:26:16 Elapsed=00:00:00
 170203:122616:jgl200: JobEnd=Normal, JCC=0, StepsExecuted=2, LastStep=S0020

$RUNDATA subdirs AFTER jgl200.ksh run OK

 /home/mvstest/testdata/data1
 -rw-rw-r--. 1 mvstest apps 13952 Apr 11  2015 data1/gl.account.master_000001
 -rw-rw-r--. 1 mvstest apps 13952 Apr 11  2015 data1/gl.account.master_000002
 -rw-rw-r--. 1 mvstest apps 13952 Apr 11  2015 data1/gl.account.master_000003
 -rw-rw-r--. 1 mvstest apps 13952 Oct 28 07:52 data1/gl.account.master_000004
 -rw-rw-r--. 1 mvstest apps  1600 Apr 11  2015 data1/gl.account.trans_000001
 -rw-rw-r--. 1 mvstest apps  1600 Apr 11  2015 data1/gl.account.trans_000002
 -rw-rw-r--. 1 mvstest apps  1600 Apr 11  2015 data1/gl.account.trans_000003
 -rw-rw-r--. 1 mvstest apps  1600 Oct 28 07:52 data1/gl.account.trans_000004

 /home/mvstest/testdata/jobtmp:
 drwxrwxr-x    6 mvstest  users        4096 Jun  1 15:03 jgl200
 /home/mvstest/testdata/jobtmp/jgl200:
 total 0   <--Note - no files in jobtmp/jgl200, moved back to data1/ at Normal EOJ

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2C1. JCL conversion - GDG files

jgl200 run#2 - demo Abnormal Termination

We will now cause a run failure & rerun to illustrate that the new generations of GDG files are not stored in the $RUNDATA/data1/ if AbTerm occurs.

 #1. cdl --> /home/mvstest/testlibs  ('cdl' is an alias for 'cd $RUNLIBS')
     ===

 #2. vi jcls/jgl200.ksh     <-- update script to cause FAILURE
     ==================       - step 2 lines 68-75 shown AFTER UPDATE BELOW

      #3----------------------------------------------------------------------
      cobrun $ANIM $CBLX/cgl200
      #4----------------------------------------------------------------------
      # SCC=$?; S0020C=$SCC; ((JCC|=SCC)); S0020R=1; alias goto="";
      SCC=$?; S0020C=$SCC; ((JCC|=SCC)); S0020R=1; alias goto="";
      logmsg2 "SCC = $SCC"
      if ((SCC>4 || SCC<0))
         then logmsg2 "ERR: step#$JSTEP cgl200 abterm $SCC"
         alias goto="<<S9900=\A"; fi
      goto

 #3. cdd  (alias cdd='cd $RUNDATA') --> /home/mvstest/testdata
     ===

 #4. joblog jgl200.ksh        <-- execute with joblogging
     =================            see listing on the next page -->

 #5. vi joblog/jgl200.log     <-- view console log
     ====================

 #6a. ls -l data1/gl*             <-- inspect data1/gl* files
      ===============               - see before above & after below
                                      on successful & failed runs

 #6b. ls -l jobtmp/jgl200/GDG/    <-- inspect contents of jobtmp/...
      ========================      - on good runs, files already moved out
                                    - on failure, new GDG files still there
                                      see listing on the next page -->

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2C2. JCL conversion - GDG files

jgl200 run#2 - Terminated Abnormally

 170203:124014:jgl200: Begin Job=jgl200
 170203:124014:jgl200: /home/mvstest1/testlibs1/jcls/jgl200.ksh
 170203:124014:jgl200: Arguments: jgl200.ksh
 170203:124014:jgl200: ProcessID=14449 RUNDATE=20170203
 170203:124014:jgl200: RUNLIBS=/home/mvstest1/testlibs1
 170203:124014:jgl200: RUNDATA=/home/mvstest1/testdata1
 170203:124014:jgl200: JTMP=jobtmp/jgl200 SYOT=sysout/JGL200
 170203:124014:jgl200: LOGNAME=mvstest1 TESTPROD=T000
 170203:124014:jgl200: HOSTNAME=uvsoft5.uvsoftware.ca
 170203:124014:JGL200: StepTimes: S0000  Begun=12:26:16 End=12:40:14 Elapsed=00:13:58
 170203:124014:jgl200: ******** Begin Step S0010 sort (#1) ********
 170203:124014:jgl200: file: SORTIN=data1/gl.account.tran1 fsize=4K
 170203:124014:jgl200: gen1+1 SORTOUT=jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.trans_000005 gens=7
 170203:124014:jgl200: file: SYSIN=/home/mvstest1/testlibs1/parms/pgl200s1 fsize=4K
 170203:124014:jgl200: Executing--> uvsort "fili1=data1/gl.account.tran1,typ=RSF,rcs=80,filo1=...
 170203:124014:JGL200: uvsort fili1=data1/gl.account.tran1\
                      ,filo1=jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.trans_000005
 170203:124014:JGL200: EOF fili01 20 rds, 1600 size; data1/gl.account.tran1
 170203:124014:JGL200: sort phase complete, used 512 megabytes, sort output begin
 170203:124014:JGL200: EOF filo01 20 wrts, 1600 size; jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.trans_000005
 170203:124014:jgl200: SCC = 0
 170203:124014:JGL200: StepTimes: S0010 sort Begun=12:40:14 End=12:40:14 Elapsed=00:00:00
 170203:124014:jgl200: ******** Begin Step S0020 cgl200 (#2) ********
 170203:124014:jgl200: gen1+1 GLTRANS=jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.trans_000005 gens=7
 170203:124014:jgl200: gen0 GLMSOLD=data1/gl.account.master_000004 insize=16K
 170203:124014:jgl200: gen1+1 GLMSNEW=jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.master_000005 gens=7
 170203:124014:jgl200: Executing--> cobrun -F /home/mvstest1/testlibs1/cblx/cgl200
 170203:124014:jgl200: SCC = 99

 170203:124014:jgl200: ERR: step#S0020 cgl200 abterm 99
 170203:124014:jgl200: jobabend51 ERR: GDG files NOT moved from jobtmp/... to intended dirs
 170203:124014:jgl200: - can rerun OK, since GDGs not added to RUNDATA/dirs
 170203:124014:jgl200: - OR restart at abterm step & reply y to move prompt

 170203:124014:jgl200: jobabend51 ERR: /home/mvstest1/testdata1/jobtmp/jgl200/GDG/files NOT moved back & listed below:
 @home@mvstest1@testdata1@data1@gl.account.master_000005
 @home@mvstest1@testdata1@data1@gl.account.trans_000005

 170203:124014:jgl200: - reply y/n acknowledge GDG msg, but no auto action
 170203:124014:JGL200: StepTimes: S0020 cgl200 Begun=12:40:14 End=12:40:14 Elapsed=00:00:00
 170203:124014:JGL200: Job Times: Begun=12:40:14 ***AbEnd=12:40:14 Elapsed=00:00:00
 170203:124014:jgl200: JobEnd=AbTerm, JCC=99,Steps=2/S0020

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2C3. JCL conversion - GDG files

$RUNDATA subdirs AFTER jgl200.ksh run FAILURE

 testdata:
 drwxrwxr-x    2 mvstest  users        4096 Jun  1 15:11 data1
 drwxrwxr-x    2 mvstest  users        4096 Jun  1 15:11 joblog
 drwxrwxr-x    3 mvstest  users        4096 Jun  1 15:03 jobtmp

 testdata/data1:
 -rw-rw-r--. 1 mvstest apps 13952 Apr 11  2015 data1/gl.account.master_000001
 -rw-rw-r--. 1 mvstest apps 13952 Apr 11  2015 data1/gl.account.master_000002
 -rw-rw-r--. 1 mvstest apps 13952 Apr 11  2015 data1/gl.account.master_000003
 -rw-rw-r--. 1 mvstest apps 13952 Apr 11  2015 data1/gl.account.master_000004
 -rw-rw-r--. 1 mvstest apps  1600 Apr 11  2015 data1/gl.account.trans_000001
 -rw-rw-r--. 1 mvstest apps  1600 Apr 11  2015 data1/gl.account.trans_000002
 -rw-rw-r--. 1 mvstest apps  1600 Apr 11  2015 data1/gl.account.trans_000003
 -rw-rw-r--. 1 mvstest apps  1600 Apr 11  2015 data1/gl.account.trans_000004

jobtmp: total 0 drwxrwxr-x. 5 mvstest apps 37 Oct 28 07:56 jgl200

jobtmp/jgl200: total 8 drwxrwxr-x. 2 mvstest apps 4096 Oct 28 07:56 GDG drwxrwxr-x. 2 mvstest apps 4096 Oct 28 07:56 gtmp drwxrwxr-x. 2 mvstest apps 6 Oct 28 07:56 tmp

jobtmp/jgl200/GDG: total 20 13952 Oct 28 07:56 @home@mvstest@testdata@data1@gl.account.master_000005 1600 Oct 28 07:56 @home@mvstest@testdata@data1@gl.account.trans_000005

Notes re job FAILURE

1. data1/gl... files are unchanged (no new generations seen), - because new generations are written to jobtmp/GDG/... (only restored to data1/... if Normal Termination reached (S9000)
   
   
2. After a failure, you can see the new GDG files ..._000005 are in jobtmp/jgl200/GDG/... - full path names, '/'s replaced by '@'s to allow GDG files in subdirs other than data1/...
   
   

      @home@mvstest@testdata@data1@gl.account.master_000005
      @home@mvstest@testdata@data1@gl.account.trans_000005

3. The new GDGs in jobtmp/GDG/... are not moved back to $RUNDATA/data1/... because job ends Abnormally at S9900 (not moved back to data1/... as at S9000)
   
   
4. After we correct the problem, we can simply rerun the job, no need to worry about removing new GDGs for incomplete jobs.
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2D1. JCL conversion - GDG files

jgl200 run#3 - 1st run AFTER Abnormal Termination

 #1. cdl --> /home/mvstest/testlibs

 #2. vi jcls/jgl200.ksh  <-- update script to remove cause of failure
     ==================    - opposite of causing failure on page '2C1'
                           - step 2 lines 68-75 shown after failure ccorerction

      #3----------------------------------------------------------------------
      cobrun $ANIM $CBLX/cgl200
      #4----------------------------------------------------------------------
      SCC=$?; S0020C=$SCC; ((JCC|=SCC)); S0020R=1; alias goto="";
      # SCC=$?; S0020C=$SCC; ((JCC|=SCC)); S0020R=1; alias goto="";
      logmsg2 "SCC = $SCC"
      if ((SCC>4 || SCC<0))
         then logmsg2 "ERR: step#$JSTEP cgl200 abterm $SCC"
         alias goto="<<S9900=\A"; fi
      goto

 #3. cdd  --> /home/mvstest/testdata
     ===

 #4. joblog jgl200.ksh        <-- execute with joblogging
     =================            see listing on the next page -->

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2D2. JCL conversion - GDG files

jgl200 joblog run#3 - after Abnormal Termination

 170203:144410:jgl200: jobset51 WARN: files in jobtmp/GDG subdirs (prior AbTerm ERR?)
 170203:144410:jgl200: ---- files in jobtmp/jgl200/GDG/*/* listed below:
 jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.master_000005
 jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.trans_000005
 170203:144410:jgl200: If NO restart by step#, GDG files in jobtmp/... will be cleared
 170203:144410:jgl200:    - allows rerun from begin job with no worry about GDGs
 170203:144410:jgl200: If RESTARTing by step#, example--> jobname.ksh start=S0050
 170203:144410:jgl200:    - GDG files in jobtmp/... will NOT be cleared
 170203:144410:jgl200:    - will be available to steps after restart step#
 170203:144410:jgl200:    - will be restored to data1/... subdir at JobEnd=Normal
 170203:144410:jgl200: enter/continue - will clear jobtmp/... if no restart
 170203:144410:jgl200:                - will not clear jobtmp/... if start=...
                                           <-- enter to continue (clearing jobtmp/...)
 170203:144515:jgl200: Begin Job=jgl200
 170203:144515:jgl200: /home/mvstest1/testlibs1/jcls/jgl200.ksh
 170203:144515:jgl200: Arguments: jgl200.ksh
 170203:144515:jgl200: ProcessID=16447 RUNDATE=20170203
 170203:144515:jgl200: RUNLIBS=/home/mvstest1/testlibs1
 170203:144515:jgl200: RUNDATA=/home/mvstest1/testdata1
 170203:144515:jgl200: JTMP=jobtmp/jgl200 SYOT=sysout/JGL200
 170203:144515:jgl200: LOGNAME=mvstest1 TESTPROD=T000
 170203:144515:jgl200: HOSTNAME=uvsoft5.uvsoftware.ca
 170203:144515:JGL200: StepTimes: S0000  Begun=12:40:14 End=14:45:15 Elapsed=02:05:01
 170203:144515:jgl200: ******** Begin Step S0010 sort (#1) ********
 170203:144515:jgl200: file: SORTIN=data1/gl.account.tran1 fsize=4K
 170203:144515:jgl200: gen1+1 SORTOUT=jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.trans_000005 gens=7
 170203:144515:jgl200: file: SYSIN=/home/mvstest1/testlibs1/parms/pgl200s1 fsize=4K
 170203:144515:jgl200: Executing--> uvsort "fili1=data1/gl.account.tran1,typ=RSF,rcs=80,filo1=...
 170203:144515:JGL200: uvsort fili1=data1/gl.account.tran1\
                      ,filo1=jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.trans_000005
 170203:144515:JGL200: EOF fili01 20 rds, 1600 size; data1/gl.account.tran1
 170203:144515:JGL200: sort phase complete, used 512 megabytes, sort output begin
 170203:144515:JGL200: EOF filo01 20 wrts, 1600 size; jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.trans_000005
 170203:144515:jgl200: SCC = 0
 170203:144515:JGL200: StepTimes: S0010 sort Begun=14:45:15 End=14:45:15 Elapsed=00:00:00
 170203:144515:jgl200: ******** Begin Step S0020 cgl200 (#2) ********
 170203:144515:jgl200: gen1+1 GLTRANS=jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.trans_000005 gens=7
 170203:144515:jgl200: gen0 GLMSOLD=data1/gl.account.master_000004 insize=16K
 170203:144515:jgl200: gen1+1 GLMSNEW=jobtmp/jgl200/GDG/@home@mvstest1@testdata1@data1@gl.account.master_000005 gens=7
 170203:144515:jgl200: Executing--> cobrun -F /home/mvstest1/testlibs1/cblx/cgl200
 170203:144515:jgl200: SCC = 0

 170203:144515:jgl200: moving /home/mvstest1/testdata1/jobtmp/jgl200/GDG/files back to intended directories
 /home/mvstest1/testdata1/data1/gl.account.master_000005
 /home/mvstest1/testdata1/data1/gl.account.trans_000005

 170203:144515:JGL200: StepTimes: S0020 cgl200 Begun=14:45:15 End=14:45:15 Elapsed=00:00:00
 170203:144515:JGL200: Job Times: Begun=14:45:15 NormalEnd=14:45:15 Elapsed=00:00:00
 170203:144515:jgl200: JobEnd=Normal, JCC=0, StepsExecuted=2, LastStep=S0020

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2D3. JCL conversion - GDG files

Notes

1. Note the 'files exist in jobtmp/GDG subdirs (from prior Abterm)' warning & the prompts to move back (for restart) or not (rerun from begining).
   
   
2. If the job only creates new generations & does not update indexed files, it is probably easier to rerun the job from the begining, since we know that any new generations have not been moved back to RUNDATA/data1/.
   
   
3. But if job has updated indexed files (in RUNDATA/data1/), then you should restart from the step# in error (after fixing the problem of course). In this case reply 'y' to move the GDGs (from prior steps) back to RUNDATA/data1/.
   
   
4. You might also prefer to restart at the error step if the job was very long, had very big files, created time stamped outputs, etc.
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2E1. JCL conversion - GDG files

gdgctl record - ABterm on run#2 & rerun #3 OK

After an abnormal termination, you can look (with uvhd) at the gdgctl records for any files you are concerned about. Here is control record for the gdg file created by jgl200.ksh after the 3 runs illustrated on the previous pages. (run#1 OK, run#2 ABterm, run#3 OK).

 cdd  (alias cdd='cd $RUNDATA') --> /home/mvstest/testdata
 ===

 uvhd ctl/gdgctl51I.dat r2048h1   <-- display control file with uvhd
 ==============================

                   10        20        30        40        50        60
 r#     2 0123456789012345678901234567890123456789012345678901234567890123
     2048 gl.account.master_                                 gdg=07
       64 next=000000(000000:000000)000000:000000 jgl200    170203:144515
      128 170203:144515:01+000005(000001:000004)jgl200   S0020 cgl200
      192 170203:144515:00 000004(000001:000004)jgl200   S0020 cgl200
      256 170203:124014:01+000005(000001:000004)jgl200   S0020 cgl200
      320 170203:124014:00 000004(000001:000004)jgl200   S0020 cgl200
      384 170203:122616:01+000004(000001:000003)jgl200   S0020 cgl200
      448 170203:122616:00 000003(000001:000003)jgl200   S0020 cgl200
      512 000000:000000:___000000(000000:000000)_______  _____ _________
      576 000000:000000:___000000(000000:000000)_______  _____ _________
                      ------ 24 lines omitted ------
     1920 000000:000000:___000000(000000:000000)_______  _____ _________
     1984 000000:000000:___000000(000000:000000)_______  _____ _________>.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

2E2. JCL conversion - GDG files

notes re gdgctl51I after failure demo

1. Note latest history entries at the top, starting at displacement 128, You can confirm this by dates:times (latest at the top)
   
   

    128 - run#3 output file '+000005' '+' flag for new generation
    192 - run#3 input  file ' 000004' ' ' blank flag existing generation
        - rerun after failure, jobtmp/GDG gen#s 000005 & 000004 cleared
        - recreated & moved to data1/... at Normal EOJ

    256 - run#2 output file '+000005' '+' flag for new generation
    320 - run#2 input  file ' 000004' ' ' blank flag existing generation
        - FAILURE, gen#s 000005 & 000004 left in jobtmp/jgl200/GDG

384 - run#1 output file '+000004' '+' flag for new generation 448 - run#1 input file ' 000003' ' ' blank flag existing generation

2. You can see that run#3 (OK) created gen# 000005, which had been created by run#2 (ABterm), but was not moved back to $RUNDATA/data1/...
   
   
3. You will not have to worry about this, if you can simply rerun the job from the begining. No new GDGs were created since we did not reach the S9000 jobend51 function that moves files back.
   
   
4. Alternatively, you could use the 'restart' option to restart at the failing step, since you are given the option to move the files back to RUNDATA/data1/ at the begining of a restarted job. You will be prompted by the 'jobset51' function called at the begining of all JCL/scripts converted by the Vancouver Utilities JCL converter.
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_5 Vancouver Utilities GDG files - Advanced Features

Part 5 - Contents

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5A1. Vancouver Utilities GDG files - Advanced Features

Advanced Features Optional

The Vancouver Utility GDG system provides significant enhancements that go well beyond the original GDG implementation on the mainframe.

1. You can specify a prior generation of a file to be defined by exportgen0 for the next run only, or until a specified expiry date/time is reached. This avoids having to modify the JCL/scripts. Note that you only have to modify one entry in the control file, whereas you could easily miss some of the possible multiple references in one or more JCL/scripts.
   
   
2. You can specify range processing for a series of prior generations This is a powerful feature that 1 customer is already planning to make good use of. On the mainframe he had 30 tapes that had to be processed one at a time by a batch COBOL program & the operator simply mounted the tapes in turn.
   
   
3. On the unix/linux system, all tape files are automatically converted to generation disc files, so to process the 30 tapes (now GDG disc files) separately, the customer might have to modify the JCL to hard-code the next desired generation# or he might specify via an operator keyin.
   
   
4. With the enhanced GDG system, he edits the control file specifying the low to high range of generation#s to be processed. He can then run the batch job 30 times to process the 30 files (with no changes to JCL/scripts & no operator generation# keyins).
   
   
5. This is possible because 'exportgen0' uses the current next generation#, which is incremented by the 'jobend51' function at normal EOJ. At the high end of the range, processing returns to normal, but an historical record is maintained in the indexed file database, making it easier to reactivate if the same cycle is repeated periodically.
   
   
6. You can specify 'opt=c' (confirm) on control file entries to cause exportgen0 to prompt the operator to confirm the generation# selection, or to enter an alternative 6 digit generation#.
   
   
7. The current system maintains a history of the last 30 accesses by exportgen0 and exportgen1. You can look at the control file (with uvhd) and for each of the last 30 accesses, you can see: date:time, generation# selected, lowest:highest generations on-file, jobname, step#,& program of last access.
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5B1. Vancouver Utilities GDG files - Advanced Features

exportgen0 & gdgctl - Exception Processing

Normally exportgen0 defines the latest generation (file with highest 6 digit suffix). Consider the following data files & the 'exportgen0' example:

data1/gl.account.master_000001 data1/gl.account.master_000002 data1/gl.account.master_000003

 exportgen0 0 GLMSTR data1/gl.account.master_  <-- JCL/script coding to define the file
 ============================================        for the following COBOL program

 export GLMSTR=data1/gl.account.master_000003  <-- effective result
 ============================================

option to process prior generations

To process prior generations, you could temporarily modify the JCL/script, explicitly coding the desired 6 digit suffix. For example we could code:

 export GLMSTR=data1/gl.account.master_000002   <-- script modified with specific#
 ============================================

            - - - OR - - -

 exportfile GLMSTR data1/gl.account.master_000002 <-- same effect & displays filename
 ================================================

1. If you hard-code the 6 digit seq#, you have to change 'exportgen0' to 'export' or 'exportfile' since exportgen0 is valid only for filenames with trailing '_'
   
   
2. 'export' is the underlying command & 'exportfile' is a function which does issue the 'export' command, but in addition displays the filename for the console log.
   
   

   disadvantages of changing the JCL/scripts

3. You have to remember to change them back after your reruns.
   
   
4. Anytime you edit the JCL/scripts, there is the possibility of making mistakes.
   
   
5. There may be multiple 'exportgen0's in multiple JCL/scripts to be modified for the reruns desired.
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5B2. Vancouver Utilities GDG files - Advanced Features

exportgen0 & gdgctl rerun facility

In 2007, a new version of exportgen0 was developed to provide a powerful rerun (prior generation access) capability.

This system uses the same control file ($RUNDATA/ctl/gdgctl51) that carries the desired number of generations per file (processed by exportgen1).

Consider the following few lines of the gdgctl file & the following generations of the account.master file:

      # gdgctl51 - GDG control file, by Owen Townsend, UV Software, Feb 2017
      #          - to demo specified next gen# & range processing
      #          - see www.uvsoftware.ca/JCLcnv4gdg.htm
      #
      gl.account.acntlist_     gdg=07
      gl.account.master_       gdg=15  <-- samples extracted from ctl/gdgctl51
      gl.account.trans_        gdg=20 opt=c

      data1/gl.account.master_000001
      data1/gl.account.master_000002
      data1/gl.account.master_000003         <-- last 5 generations of account.master

If you wanted to rerun a JCL/script using prior generation _000002, you could modify the control file as follows:

      gl.account.acntlist_     gdg=07
      gl.account.master_       gdg=15  next=000002   <-- gen# for rerun
      gl.account.trans_        gdg=20  opt=c

1. It is safest to edit only the ctl/gdgctl51 text file, not the indexed file (ctl/gdgctl51I.dat) used by exportgen0 (& the jobend51 function).
   
   
2. After modifying the text file, you must load it to the indexed file using 'uvcopy gdgload51'
   
   
3. Or you could update the Indexed file directly using 'uvhd' (as shown later on page '5C2').
   
   
4. Or if very careful, you could edit the Indexed file directly using the 'vi' editor since we do have the text file as backup & we can recreate the text file by rerunning jclgdgctl51.
   
   
5. At the end of the 1st JCL/script with exportgen0 reference to account.master, the 'next=000002' is disabled by changing it to 'next=000000'
   
   
6. You do have to remember to clear the original 'next=000002' from the text file version before you reload the indexed file for some other reason.
   
   
7. It might be safer to 1st run the 'gdgunload51' job to convert the indexed file back to a text file for future edits & reload.
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5B3. Vancouver Utilities GDG files - Advanced Features

Expiry Times

The system allows you to specify an expiry time when modifying the gdgctl file. For example:

     gl.account.master_  gdg=15  next=000002_yymmdd:HHMMSS
     ==========================================================

The exception processing (defining prior generations of the file) is disabled if the expiry date has been exceeded when the next JCL/script is run with exportgen0 references to the account.master.

Progressing thru a range of prior generations

The system also allows for successive runs to process thru a range of prior generations. For example:

     gl.account.trans_ gdg=20 opt=c next=000001(000001:000012)
     ===============================================================

This would allow you to load the transaction files for all 12 months, before you do any processing, then process the 12 months on successive runs. (with no changes to JCL/scripts or control file required).

After the 12th run the control file entry would be updated by the jobend51 function executed at Normal EOJ, to:

     gl.account.trans_ gdg=20 opt=c next=000000(000001:000012)
     ===============================================================

IE, the next gen# is set to '000000' to indicate the processing is completed but not lose a record of the manual edits. The next reference by exportgen0 for this file would define the then highest generation# (which might still be '000012').

You could combine expiry times with progression, but it is not necessary, since the progression automatically ends the exception processing when the upper limit is reached. Expiry dates are ignored if a low < high range is specified, but the format would be:

     gl.account.trans_ gdg=20 opt=c next=000001(000001:000012)yymmdd:HHMMSS
     ============================================================================

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5B4. Vancouver Utilities GDG files - Advanced Features

Retaining History in gdgctl

A simplified example of a 1 time access to a prior generation is shown below, followed by the actual format used, which retains some history of your prior generation overrides in the gdgctl file.

     gl.account.master_ gdg=15 next=000002  <-- gen# for next access (1 time)
     ==========================================

     gl.account.master_ gdg=15 next=000000  <-- reset to this at Normal EOJ
     ==========================================

     gl.account.master_ gdg=15 next=000002(000002:000002)yymmdd:HHMMSS <-- optional
     ======================================================================

     gl.account.master_ gdg=15 next=000000(000002:000002)yymmdd:HHMMSS <-- reset
     ==============================************============================

gdgctl illustrating prior gen# over-rides

 gl.account.acntlist_ gdg=07
 gl.account.master_   gdg=15 next=000002(000002:000002)000000:000000 <--note1
 gl.account.trans_    gdg=20 next=000002(000002:000002)070529:235959 <--note2
 py.payroll.master_   gdg=30 op=c                                    <--note3
 py.time.cards_       gdg=52 next=000001(000001:000012)000000:000000 <--note4
 py.time.cards.div2_  gdg=52 next=000001(000001:000012)070529:235959 <--note5

1. 'next=000002' will be reset to 'next=000000' (disabling over-ride) at the end of the 1st JCL/script that references the file with exportgen0.
   
   
2. Specify an 'expiry date' if you want the over-ride to persist for multiple runs until some specified date/time.
   
   
3. Specify 'opt=c' if you want an operator prompt to confirm gen# selection or enter an alternative generation#
   
   
4. The generation# will be incremented each time you run the JCL/script referencing the file. After the 12th run it will be set to 'next=000000(000001:000012)' disabling the over-ride, but leaving some history, in case you wanted to reactivate in future.
   
   

5a. You could also specify a non-zero expiry date/time, but not necessary since the override is automatically disabled at the high end of the range.

5b. Expiry dates are ignored for ranges (when low & high different), but you could code expiry dates with ranges for an historical record.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5C1. Vancouver Utilities GDG files - Advanced Features

using 'uvhd' to display GDG Indexed control-file

 uvhd ctl/gdgctl51I.dat r2048h1  <-- execute uvhd to display gdgctl file
 ==============================    - options: r2048=record-size, u=update
                                      h1=chars only (omit hex zones/digits)

                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
        2048 gl.account.master_                                 gdg=15 opt=c
          64 next=000002(000002:000002)171231:235959 jgl220    170206:092840
         128 170206:092840:01+000006(000001:000005)jgl220   S0010 cgl200
         192 170206:092837:00 000002(000001:000005)jgl220   S0010 cgl200
         256 170206:092800:01+000005(000001:000004)jgl220   S0010 cgl200
         320 170206:092756:00 000002(000001:000004)jgl220   S0010 cgl200
         384 170206:092558:01+000004(000001:000003)jgl220   S0010 cgl200
         448 170206:092536:00 000002(000001:000003)jgl220   S0010 cgl200
         512 000000:000000:___000000(000000:000000)_______  _____ _________
                      --------- 25 lines removed ---------
        1984 000000:000000:___000000(000000:000000)_______  _____ _________>.

                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
        4096 gl.account.trans_                                  gdg=15 opt=c
          64 next=000004(000001:000012)000000:000000 jgl220
         128 170206:092839:00 000003(000001:000012)jgl220   S0010 cgl200
         192 170206:092758:00 000002(000001:000012)jgl220   S0010 cgl200
         256 170206:092544:00 000001(000001:000012)jgl220   S0010 cgl200
         320 000000:000000:___000000(000000:000000)_______  _____ _________
                      --------- 25 lines removed ---------
        1984 000000:000000:___000000(000000:000000)_______  _____ _________>.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5C2. using 'uvhd' to display GDG Indexed control-file

notes re using uvhd to display Indexed file

1. Please see the uvhd display on the previous page. We are showing records 2&3 after loading the indexed file with ctl/gdgctl51. Each 2048 byte record is displayed on 32 lines (of 64 bytes). Here are 1st 2 lines for records 2 & 3.
   
   

        2048 gl.account.master_                                 gdg=15 opt=c
          64 next=000002(000002:000002)171231:235959 jgl220    170206:092840
                 - - - 30 lines omitted - - -

        4096 gl.account.trans_                                  gdg=15 opt=c
          64 next=000004(000001:000012)000000:000000 jgl220
                 - - - 30 lines omitted - - -

2. Only the 1st 2 lines have items that might be updated with uvhd or vi. You might want to modify the number of generations (gdg=15 or gdg=20 above). You might want to add option 'opt=c' to prompt operator to confirm gen# (as shown for the data1/gl.account.trans file).
   
   
3. Some users may have a reason to code the 'next=...' options shown above 'next=000002...' for data1/gl.account.master overrides the normal selection of the highest generation until the expiry date is reached.
   
   
4. 'next=000001(000001:000012)...' on data1/gl.account/trans, will increment the generation# thru a range of previously stored generations from low to high. 'exportgen0' will use the next=... value & 'jobend51/gdgupok1' will increment the generation# at successful EOJ. We will demonstrate later using jgl220.ksh in Part_6.
   
   
5. Note that the last 2 bytes of each record show as '>.'. The '>' marks the end of the record & allows possible updating with the editor. The '.' is actually x'0A' which is the indexed file record status byte for a good record, whereas x'00' would indicate a deleted record. It shows as a '.' because uvhd shows unprintable characters as '.'.
   
   
6. You can see the hex characters by adding option 'h2' to force display in 'vertical hexadecimal', which is automatic if the record contains any non-character bytes. The above was in character because the only non-char was the LineFeed x'0A' which is expected at the end of text records.
   
   
7. On page '5C4' we will show how to use 'vi' to update the Indexed file - if you are very careful.
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5C3. Vancouver Utilities GDG files - Advanced Features

updating gdgctl Indexed file with uvhd

On this page we will show you only the 'uvhd' commands to update the Indexed file. Please see the records displayed on page '5C1'. The records already have the updates shown below because we loaded the file from ctl/gdgctl51 which was already updated with 'vi' before loading the indexed file (the easier way to make extensive updates).

 #1. login mvstest --> /home/mvstest

 #2. cdd  (alias cdd='cd $RUNDATA') --> /home/mvstest/testdata
     ===

 #3. uvhd ctl/gdgctl51I.dat r2048uh1  <-- start uvhd with options
     ===============================      r2048=recsize, u=update
                                          h1=chars only (omit hex zones/digits)

      <--- see uvhd record display on prior page
         - uvhd displays the 1st record & prompts for commands
         - you can reply as shown below:

 #3a. -->    <-- null entry to advance to record #2

 #3b. --> u 64(38),'next=000002(000002:000002)000000:000000'
          ==================================================

easier alternative

Actually the gdgload51 utility inserts zero-filled inactive entries, so you only have to update the 6 digit next gen# as follows:

 #3b. --> u 69(6),'000002'
          ================

 #3c. --> q   <-- quit

Notes

1. If you use uvhd, you must be sure not to modify the keys (1st 50 bytes)
   
   
2. using uvhd is more awkward than an editor, but much safer. uvhd will never change the record size which would corrupt the remainder of the file, since the index partition depends upon all data record being same size. Adding or deleting bytes inadvertently is very easy to do with vi.
   
   
3. Updating the Indexed file directly with 'vi' is dangerous, but it is fast & convenient, and we will show you how on the next page.
   
   
4. After updating with 'vi' you should always run the 'dcheck' utility to ensure the validity of the Indexed file.
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5C4. Vancouver Utilities GDG files - Advanced Features

updating gdgctl Indexed file with vi

 #1. cdd  (alias cdd='cd $RUNDATA') --> /home/mvstest/testdata

 #2. vi ctl/gdgctl51I.dat   <-- start 'vi' on the Indexed file .dat partition
     ====================

 gl.account.master_                                 gdg=07       next=000000(0000
 00:000000)000000:000000 jgl200    170203:144515 170203:144515:01+000005(000001:0
 00004)jgl200   S0020 cgl200     170203:144515:00 000004(000001:000004)jgl200   S
 0020 cgl200     170203:124014:01+000005(000001:000004)jgl200   S0020 cgl200
 170203:124014:00 000004(000001:000004)jgl200   S0020 cgl200     170203:122616:01
 +000004(000001:000003)jgl200   S0020 cgl200     170203:122616:00 000003(000001:0
 00003)jgl200   S0020 cgl200     000000:000000:___000000(000000:000000)_______  _
 ____ _________  000000:000000:___000000(000000:000000)_______  _____ _________
                    ------ omitting 18 lines ------

 #2a. -->vi updates ...

 #2x. -->:wq  <-- write & quit vi editor

 #3. dcheck ctl/gdgctl51I    <-- validate the Indexed file
     ====================      - to ensure you did not screw up with editor

notes re display above

To the 'vi' editor the gdgctl51I.dat records are considered very long lines, since there is only 1 Linefeed at the end of the 2048 byte records. They appear as 26 lines on the screen, since screen lines wrap at 80 bytes. If we printed the original record you would see only the 1st 90 bytes (page-width).

To simulate the screen appearance I ran a uvcopy job 'splitrec1' which inserts a Line-Feed every 80 bytes. I 1st used vi to write 1 record to tmp/gdgctl1.

 uvcopy splitrec1,fili1=tmp/gdgctl1,filo1=tmp/gdgctl2
 ====================================================

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

5C5. Vancouver Utilities GDG files - Advanced Features

Notes re updating gdgctl with 'vi'

 gl.account.trans_                                  gdg=20 opt=c next=000005(0000
 01:000012)000000:000000 JGL220             err4 070621:210051:G0_000003(000001:0
 00012)JGL220   S0010 cgl200err4 070621:210030:G0_000003(000001:000012)JGL220   S
                    ------ omitting 23 lines ------

1. Shown above is the 3rd record of gdgctl51I after loading from gdgctl51. Each 2048 byte record requires 26 lines. The filename appears 1st, in this example 'data1/gl.account.trans. The only items you might need to update are on the 1st 2 lines (+ 6 chars of the 3rd line). The rest is history.
   
   
2. 'gdg=20' is the number of generations to be maintained. 'opt=c' is the option for operator confirmation of gen# selection. 'next=...' is the option to process a series of existing prior generations.
   
   
3. Note 'next=000005(000001:000012)...'. This 'vi' snapshot was captured 4 runs after setting next=000001(000001:000012)...'
   
   
4. If you are very careful you might be able to update with vi. Use only the 'R'eplace command of vi, not insert or delete. Note that 'gdgload51' inserts a '>' at col 2047 to indicate the end of the record. The '>' is followed by '.' which is actually x'0A', valid record status for Indexed file records.
   
   
5. When you rewrite the file, check that the file-size is a multiple of the record-size (256). Or use control-G in the editor to check before writing.
   
   
6. Updating the indexed file is only justified if you always run the 'dcheck' utility to validate & if you maintain the text file version.
   
   
7. If you do much updating with vi, you could periodically run 'gdgunload51' to convert the Indexed file back to the text file backup.
   
   
8. If need be you can recreate the text file by running jclgdgctl51 (see '1C2').
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_6 GDG Advanced Features - Demonstration

Part 6 - Contents

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6A1. GDG Advanced Features - Demonstration

prior GDG & Range processing

Here we will show you how you might use one of the extra features of the Vancouver Utility GDG system to process a series of GDG files already stored in the directory before processing begins.

We can update the gdgctl file to specify the from:to series of generations to be processed 1 a t a time when processing begins. This feature saves having to modify any JCL or copy any files between the multiple runs.

For example, suppose we have already stored 12 months of gl transactions & we now wish to process all 12 successively without having to modify any JCL or copy files.

The job that stores the 12 files would use exportgen1 & assuming that there were no existing files in the data1/ subdir, the result would be as follows:

data files to demo range processing

 /home/mvstest/testdata
 :-----ctl
 :     :-----gdgctl51
 :     :-----gdgctl51I.dat
 :     :-----gdgctl51I.idx
 :     :
 :-----data1
 :     :-----gl.account.master_000001
 :     :-----gl.account.master_000002
 :     :-----gl.account.master_000003
 :     :
 :     :-----gl.account.trans_000001
 :     :-----gl.account.trans_000002
 :     :-----gl.account.trans_000003
 :     :-----gl.account.trans_000004
 :     :-----gl.account.trans_000005
 :     :       - - - etc - - -
 :     :-----gl.account.trans_000012

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6A2. GDG Advanced Features - Demonstration

GDG control-file to demo range processing

 # gdgctl51 - GDG control file, by Owen Townsend, UV Software, Feb 2017
 #          - to demo specified next gen# & range processing
 #          - see www.uvsoftware.ca/JCLcnv4gdg.htm
 #
 gl.account.acntlist_     gdg=07
 gl.account.master_       gdg=15 next=000002(000002:000002)171231:235959 opt=c
 gl.account.trans_        gdg=15 next=000001(000001:000012)000000:000000 opt=c

 uvcopy gdgload51,fili1=ctl/gdgctl51
 ===================================
 - load gdgctl51 to demo select next gen# & increment thru a range

display GDG Indexed file with 'uvhd'

 uvhd /home/mvstest/testdata/ctl/gdgctl51I.dat r2048h1
 =====================================================

                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
        2048 gl.account.master_                                 gdg=15 opt=c
          64 next=000002(000002:000002)171231:235959
         128 000000:000000:___000000(000000:000000)_______  _____ _________
         192 000000:000000:___000000(000000:000000)_______  _____ _________
                     ---------- 25 lines removed -----------
        1984 000000:000000:___000000(000000:000000)_______  _____ _________>.

                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
        4096 gl.account.trans_                                  gdg=15 opt=c
          64 next=000001(000001:000012)000000:000000
         128 000000:000000:___000000(000000:000000)_______  _____ _________
         192 000000:000000:___000000(000000:000000)_______  _____ _________
                     ---------- 25 lines removed -----------
        1984 000000:000000:___000000(000000:000000)_______  _____ _________>.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6A3. GDG Advanced Features - Demonstration

updating gdgctl to process a series of GDG files

Here is the ctl/gdgctl51 entry for gl.account.trans_ expected as we process a series of 12 generations in separate runs of jgl220.ksh.

 gl.account.trans_
 =================

 gdg=15 next=000000(000000:000000)  <-- default next=000000
 =================================      selects highest gen# in directory

 gdg=15 next=000001(000001:000012)  <-- after gdgctl edit for next=...
 =================================    - before any runs

 gdg=15 next=000002(000001:000012)  <-- after 1st run
 =================================

 gdg=15 next=000003(000001:000012)  <-- after 2nd run
 =================================

 gdg=15 next=000012(000001:000012)  <-- after 11th run
 =================================

 gdg=15 next=000000(000001:000012)  <-- after 12th run
 =================================    - next gen# reset to next=000000

1. After the 12th run, the next gen# is set to '000000' which disables the feature, but retains a history/comment, which can easily be activated for the next requirement.
   
   
2. Any runs after next is reset to 000000 will pick up the highest gen# in the directory as normal, which may still be the 12th month.
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6B1. GDG Advanced Features - Demonstration

Demo prior GDG & Range processing

The next few pages will illustrate how you can setup the control file to process multiple generations. You can load the control file, setup to process prior generation of the account.master file & multiple generations of the gl.account.trans_ file with automatic incrementing thru a range generations.

data1/gl* files for GDG range demo

 -rw-rw-r--    1 mvstest  users        8720 May 31 17:33 account.master_000001
 -rw-rw-r--    1 mvstest  users        8720 May 31 17:33 account.master_000002
 -rw-rw-r--    1 mvstest  users        8720 May 31 17:33 account.master_000003

 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 account.trans_000001
 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 account.trans_000002
 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 account.trans_000003
                 - - - will demo only 1st 3 runs - - -
 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 account.trans_000012

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6C1. GDG Advanced Features - Demonstration

JGL220 JCL to demo prior GDG processing

 //JGL220   JOB  (1234),'TEST/DEMO MVS JCL CONVERSION'
 //* UPDATE GL ACCOUNT MASTER WITH GLTRANS
 //* demo exportgen0 input prior master generation (ignore newer gens)
 //* demo exportgen0 input a progression of prior trans files
 //STEP020  EXEC PGM=CGL200,COND=(4,LT,STEP010),PARM=&YEAREND
 //GLMSOLD  DD DSN=GL.ACCOUNT.MASTER(0),DISP=OLD
 //GLTRANS  DD DSN=GL.ACCOUNT.TRANS(0),DISP=OLD
 //GLMSNEW  DD DSN=GL.ACCOUNT.MASTER(+1),DISP=(,CATLG,DELETE),
 //            UNIT=DISK,SPACE=(TRK,(50,50),RLSE),
 //            DCB=(MODEL.DSCB,LRECL=80,BLKSIZE=6118,RECFM=FBA)

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6C2. GDG Advanced Features - Demonstration

JGL220.jcl converted to jgl220.ksh

 #!/bin/ksh
 ##JGL220   JOB  (1234),'TEST/DEMO MVS JCL CONVERSION'
 export jobid2=jgl220 JOBID2=JGL220; scriptpath="$0"; args="$*"
 if [[ -z "$jobid1" ]]; then export jobid1=$jobid2; fi
 for arg in $args; do if [[ "$arg" == *=* ]]; then export $arg; fi; done
 integer JCC=0 SCC=0 LCC=0  # init step status return codes
 autoload jobset51 jobend51 jobabend51 logmsg1 logmsg2 stepctl51
 autoload exportfile exportgen0 exportgen1 exportgen2 exportgenall exportgenx
 . $APPSADM/env/stub.ini  #<-- for control-M (see notes in env/stub.ini)
 jobset51    # call function for JCL/script initialization
 goto
 S0000=A
 # * UPDATE GL ACCOUNT MASTER WITH GLTRANS
 # * demo exportgen0 input prior master generation (ignore newer gens)
 # * demo exportgen0 input a progression of prior gltrans files
 #1======================= begin step#S0010 CGL200 ========================
 S0010=A
 stepctl51;
 goto
 export JSTEP=S0010; ((XSTEP+=1)); SCC=0; alias goto="";
 logmsg2 "******** Begin Step $JSTEP cgl200 (#$XSTEP) ********"
 ##STEP020  EXEC PGM=CGL200,COND=(4,LT,STEP010),PARM=&YEAREND
 export PROGID=cgl200
 export PARM="$YEAREND"
 exportgen0 0 GLMSOLD data1/gl.account.master_
 exportgen0 0 GLTRANS data1/gl.account.trans_
 exportgen1 +1 GLMSNEW data1/gl.account.master_
 #exportgen1 GDGs written to jobtmp/$jobid2 restored to outdir at Normal EOJ
 logmsg2 "Executing--> cobrun $ANIM $CBLX/cgl200"
 #3----------------------------------------------------------------------
 cobrun $ANIM $CBLX/cgl200
 #4----------------------------------------------------------------------
 SCC=$?; S0010C=$SCC; ((JCC|=SCC)); S0010R=1; alias goto="";
 logmsg2 "SCC = $SCC"
 if ((SCC>4 || SCC<0))
    then logmsg2 "ERR: step#$JSTEP cgl200 abterm $SCC"
    alias goto="<<S9900=\A"; fi
 goto
 #/=*@.@. procs expanded by UVSW jclproc51 20160425:145759
 #8======================================================================
 S9000=A
 jobend51 #move any new GDG files from jobtmp to intended outdirs
 logmsg2 "JobEnd=Normal, JCC=$JCC, StepsExecuted=$XSTEP, LastStep=$JSTEP"
 exit 0 #ver:20160425 a2b0c0d1e2f0g1h1i15j0k15l1m4n3o8p0q0r0s2t1u1v0w0x0y1z1
 #9======================================================================
 S9900=A
 jobabend51 #report GDGs NOT moved from jobtmp/subdirs to outdirs
 logmsg2 "JobEnd=AbTerm, JCC=$JCC,Steps=$XSTEP/$JSTEP" RV ACK
 exit $JCC  # JCL converted to ksh by UVSW jclunix51 20160425:160654

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6D1. GDG Advanced Features - Demonstration

setup to demo prior GDG & Range processing

We need to load the control file, to process a prior generation of the gl.account.master_ file & multiple generations of the gl.account.trans_ file with automatic incrementing thru a range generations.

 #1. login mvstest --> /home/mvstest

 #2. cdd  (alias cdd='cd $RUNDATA') --> /home/mvstest/testdata
     ===

 #3. vi ctl/gdgctl51   <-- inspect gdgctl to demo prior GDG & Range
     ===============

      # gdgctl51 - GDG control file, by Owen Townsend, UV Software, Feb 2017
      #          - to demo specified next gen# & range processing
      #          - see www.uvsoftware.ca/JCLcnv4gdg.htm
      #
      gl.account.acntlist_     gdg=07
      gl.account.master_       gdg=15 next=000002(000002:000002)171231:235959 opt=c
      gl.account.trans_        gdg=15 next=000001(000001:000012)000000:000000 opt=c

 #4. uvcopy gdgload51,fili1=ctl/gdgctl51
     ===================================
     - load gdgctl51 to demo generation# selection & increment thru a raange

 #5. testdatainit              <-- remove old temporary & test output files
     ============                  (jobtmp/*, joblog/*, tmp/*, etc)

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6E1. GDG Advanced Features - Demonstration

prior GDG select/increment - Run#1

 #1. jgl220.ksh                <-- execute JCL/script to demo GDG files
     ==========                  - BETTER to use joblog as below

 #1a. joblog jgl220.ksh        <-- alternative to capture console log file
      =================

 #2.  vi joblog/jgl220.log     <-- view console log
      ====================

 #2b. uvlp12 joblog/jgl220.log <-- OR, print console log
      ========================     see listing on the next page -->

 #3.  ls -l data1/gl*          <-- display  data1/gl* files after 1st run
      ===============

 #3a. lslp data1               <-- OR, print data1/gl* files after 1st run
      ==========                      see listing on the next page -->

 #4. Please inspect the console logs & directory listings on the next page
     & observe the following:

observations after 1st run

1. The master file selected was data1/gl.account.master_000002, because the gdgctl file had next=000002... overriding the usual selection of the last generation in the directory which was _000003
   
   
2. The trans file selected was data1/gl.account.trans_000001, because the gdgctl file had next=000001(000001:000012)... & this was the 1st run.
   
   
3. After the 1st run, the trans entry in ctl/gdgctl51I.dat would be: next=000002(000001:000012)...
   
   
4. Note that 'opt=c' coded on the control file entry for data1/gl.account.trans_ causes the following operator prompt to confirm 'y' or enter an alternative 6 digit generation#.
   
   

     gen0: GLTRANS=data1/gl.account.trans_000001     <-- confirm y or alt# ?
     gen0: GLTRANS: generations=000001:000012  <-- enter y(000001) or alt# ?
     070616:155225:JGL220: gen0: GLTRANS=data1/gl.account.trans_000001 insize=1600

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6E2. GDG Advanced Features - Demonstration

jgl220.log after 1st run

 170206:092536:jgl220: Begin Job=jgl220
 170206:092536:jgl220: /home/mvstest/testlibs/jcls/jgl220.ksh
 170206:092536:jgl220: Arguments: jgl220.ksh
 170206:092536:jgl220: ProcessID=9431
 170206:092536:jgl220: RUNDATE=20170206
 170206:092536:jgl220: RUNLIBS=/home/mvstest/testlibs
 170206:092536:jgl220: RUNDATA=/home/mvstest/testdata
 170206:092536:jgl220: JTMP=jobtmp/jgl220
 170206:092536:jgl220: SYOT=sysout/JGL220
 170206:092536:jgl220: LOGNAME=mvstest1 TESTPROD=T000
 170206:092536:jgl220: HOSTNAME=uvsoft5.uvsoftware.ca
 170206:092536:jgl220: ******** Begin Step S0010 cgl200 (#1) ********
 gen0: GLMSOLD=gl.account.master_000002  <-- confirm y or alt# ?
 gen0: GLMSOLD: generations=000001:000003  <-- enter y(000002) or alt# ?
 gen0: GLMSOLD: generations=000001:000003  <-- enter y(000002) or alt# ?
 170206:092544:jgl220: gen0 GLMSOLD=data1/gl.account.master_000002 insize=16K
 gen0: GLTRANS=gl.account.trans_000001  <-- confirm y or alt# ?
 gen0: GLTRANS: generations=000001:000012  <-- enter y(000001) or alt# ?
 gen0: GLTRANS: generations=000001:000012  <-- enter y(000001) or alt# ?
 170206:092558:jgl220: gen0 GLTRANS=data1/gl.account.trans_000001 insize=4K
 170206:092558:jgl220: gen1+1 GLMSNEW=jobtmp/jgl220/GDG/@home@mvstest1@testdata@data1@gl.account.master_000004 gens=15
 170206:092558:jgl220: Executing--> cobrun -F /home/mvstest/testlibs/cblx/cgl200
 170206:092559:jgl220: SCC = 0
 170206:092559:jgl220: moving /home/mvstest/testdata/jobtmp/jgl220/GDG/files back to intended directories
 /home/mvstest/testdata/data1/gl.account.master_000004
 170206:092559:JGL220: StepTimes: S0010 cgl200 Begun=09:25:36 End=09:25:59 Elapsed=00:00:23
 170206:092559:JGL220: Job Times: Begun=09:25:36 NormalEnd=09:25:59 Elapsed=00:00:23
 170206:092559:jgl220: JobEnd=Normal, JCC=0, StepsExecuted=1, LastStep=S0010

data1/gl* files after 1st run

 -rw-rw-r--    1 mvstest  users        8720 May 31 17:33 account.master_000001
 -rw-rw-r--    1 mvstest  users        8720 May 31 17:33 account.master_000002
 -rw-rw-r--    1 mvstest  users        8720 May 31 17:33 account.master_000003
 -rw-rw-r--    1 mvstest  users        8720 Jun  2 07:31 account.master_000004

 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 account.trans_000001
 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 account.trans_000002
 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 account.trans_000003
                  --------- 8 generations not shown ---------
 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 account.trans_000012

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6F1. GDG Advanced Features - Demonstration

prior GDG select/increment - Run#2

 #1. jgl220.ksh                <-- execute JCL/script to demo GDG files
     ==========                  - BETTER to use joblog as below

 #1a. joblog jgl220.ksh        <-- alternative to capture console log file
      =================

 #2a. vi joblog/jgl220.log     <-- view console log
      ====================

 #2b. uvlp12 joblog/jgl220.log <-- OR, print console log
      ========================     see listing on the next page -->

 #3.  ls -l data1/gl*          <-- display data1/gl* files after 2nd run
      ===============

 #3a. lslp data1               <-- OR, print data1/gl* files after 2nd run
      ==========                      see listing on the next page -->

 #4. Please inspect the console logs & directory listings on the next page
     & observe the following:

observations after 2nd run

1. The master file selected was data1/gl.account.master_000002, because the gdgctl file had next=000002... overriding the last generation of _000003. The next=000002 was not reset because the expiration date was 171231.
   
   
2. The trans file selected was data1/gl.account.trans_000002, because the gdgctl file had next=000002(000001:000012)... (next= incremented at end 1st run).
   
   
3. After the 2nd run, the trans entry in ctl/gdgctl51I.dat will be: next=000003(000001:000012)...
   
   
4. There are 12 generations of the gl.account.trans_ file, but we will demo only the 1st 3 runs
   
   

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6F2. GDG Advanced Features - Demonstration

jgl220.log after 2nd run

 170206:092756:jgl220: Begin Job=jgl220
 170206:092756:jgl220: /home/mvstest/testlibs/jcls/jgl220.ksh
 170206:092756:jgl220: Arguments: jgl220.ksh
 170206:092756:jgl220: ProcessID=9565
 170206:092756:jgl220: RUNDATE=20170206
 170206:092756:jgl220: RUNLIBS=/home/mvstest/testlibs
 170206:092756:jgl220: RUNDATA=/home/mvstest/testdata
 170206:092756:jgl220: JTMP=jobtmp/jgl220
 170206:092756:jgl220: SYOT=sysout/JGL220
 170206:092756:jgl220: LOGNAME=mvstest1 TESTPROD=T000
 170206:092756:jgl220: HOSTNAME=uvsoft5.uvsoftware.ca
 170206:092756:JGL220: StepTimes: S0000  Begun=09:25:59 End=09:27:56 Elapsed=00:01:57
 170206:092756:jgl220: ******** Begin Step S0010 cgl200 (#1) ********
 gen0: GLMSOLD=gl.account.master_000002  <-- confirm y or alt# ?
 gen0: GLMSOLD: generations=000001:000004  <-- enter y(000002) or alt# ?
 170206:092758:jgl220: gen0 GLMSOLD=data1/gl.account.master_000002 insize=16K
 gen0: GLTRANS=gl.account.trans_000002  <-- confirm y or alt# ?
 gen0: GLTRANS: generations=000001:000012  <-- enter y(000002) or alt# ?
 170206:092800:jgl220: gen0 GLTRANS=data1/gl.account.trans_000002 insize=4K
 170206:092800:jgl220: gen1+1 GLMSNEW=jobtmp/jgl220/GDG/@home@mvstest1@testdata@data1@gl.account.master_000005 gens=15
 170206:092800:jgl220: Executing--> cobrun -F /home/mvstest/testlibs/cblx/cgl200
 170206:092800:jgl220: SCC = 0
 170206:092800:jgl220: moving /home/mvstest/testdata/jobtmp/jgl220/GDG/files back to intended directories
 /home/mvstest/testdata/data1/gl.account.master_000005
 170206:092800:JGL220: StepTimes: S0010 cgl200 Begun=09:27:56 End=09:28:00 Elapsed=00:00:04
 170206:092800:JGL220: Job Times: Begun=09:27:56 NormalEnd=09:28:00 Elapsed=00:00:04
 170206:092800:jgl220: JobEnd=Normal, JCC=0, StepsExecuted=1, LastStep=S0010

data1/gl* files after 2nd run

 -rw-rw-r--    1 mvstest  users        8720 May 31 17:33 account.master_000001
 -rw-rw-r--    1 mvstest  users        8720 May 31 17:33 account.master_000002
 -rw-rw-r--    1 mvstest  users        8720 May 31 17:33 account.master_000003
 -rw-rw-r--    1 mvstest  users        8720 Jun  2 07:31 account.master_000004
 -rw-rw-r--    1 mvstest  users        8720 Jun  2 07:32 account.master_000005

 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 account.trans_000001
 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 account.trans_000002
 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 account.trans_000003
                  --------- 8 generations not shown ---------
 -rw-rw-r--    1 mvstest  users        1600 May 31 17:33 account.trans_000012

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6G1. GDG Advanced Features - Demonstration

prior GDG select/increment - Run#3

 #1. jgl220.ksh            <-- execute JCL/script to demo GDG files
     ==========

 #1a. joblog jgl220.ksh    <-- alternative to capture console log file
      =================

jgl220.log after 3rd run

 170206:092756:jgl220: Begin Job=jgl220
 170206:092756:jgl220: /home/mvstest/testlibs/jcls/jgl220.ksh
 170206:092756:jgl220: Arguments: jgl220.ksh
 170206:092756:jgl220: ProcessID=9565
 170206:092756:jgl220: RUNDATE=20170206
 170206:092756:jgl220: RUNLIBS=/home/mvstest/testlibs
 170206:092756:jgl220: RUNDATA=/home/mvstest/testdata
 170206:092756:jgl220: JTMP=jobtmp/jgl220
 170206:092756:jgl220: SYOT=sysout/JGL220
 170206:092756:jgl220: LOGNAME=mvstest1 TESTPROD=T000
 170206:092756:jgl220: HOSTNAME=uvsoft5.uvsoftware.ca
 170206:092756:JGL220: StepTimes: S0000  Begun=09:25:59 End=09:27:56 Elapsed=00:01:57
 170206:092756:jgl220: ******** Begin Step S0010 cgl200 (#1) ********
 gen0: GLMSOLD=gl.account.master_000002  <-- confirm y or alt# ?
 gen0: GLMSOLD: generations=000001:000004  <-- enter y(000002) or alt# ?
 170206:092758:jgl220: gen0 GLMSOLD=data1/gl.account.master_000002 insize=16K
 gen0: GLTRANS=gl.account.trans_000002  <-- confirm y or alt# ?
 gen0: GLTRANS: generations=000001:000012  <-- enter y(000002) or alt# ?
 170206:092800:jgl220: gen0 GLTRANS=data1/gl.account.trans_000002 insize=4K
 170206:092800:jgl220: gen1+1 GLMSNEW=jobtmp/jgl220/GDG/@home@mvstest1@testdata@data1@gl.account.master_000005 gens=15
 170206:092800:jgl220: Executing--> cobrun -F /home/mvstest/testlibs/cblx/cgl200
 170206:092800:jgl220: SCC = 0
 170206:092800:jgl220: moving /home/mvstest/testdata/jobtmp/jgl220/GDG/files back to intended directories
 /home/mvstest/testdata/data1/gl.account.master_000005
 170206:092800:JGL220: StepTimes: S0010 cgl200 Begun=09:27:56 End=09:28:00 Elapsed=00:00:04
 170206:092800:JGL220: Job Times: Begun=09:27:56 NormalEnd=09:28:00 Elapsed=00:00:04
 170206:092800:jgl220: JobEnd=Normal, JCC=0, StepsExecuted=1, LastStep=S0010

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6H1. GDG Advanced Features - Demonstration

gdgctl file after 3 runs of jgl220.ksh

 uvhd ctl/gdgctl51I.dat r2048h1   <-- display control file with uvhd
 ==============================

                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
        2048 gl.account.master_                                 gdg=15 opt=c
          64 next=000002(000002:000002)171231:235959 jgl220    170206:092840
         128 170206:092840:01+000006(000001:000005)jgl220   S0010 cgl200
         192 170206:092837:00 000002(000001:000005)jgl220   S0010 cgl200
         256 170206:092800:01+000005(000001:000004)jgl220   S0010 cgl200
         320 170206:092756:00 000002(000001:000004)jgl220   S0010 cgl200
         384 170206:092558:01+000004(000001:000003)jgl220   S0010 cgl200
         448 170206:092536:00 000002(000001:000003)jgl220   S0010 cgl200
         512 000000:000000:___000000(000000:000000)_______  _____ _________
                      --------- 25 lines removed ---------
        1984 000000:000000:___000000(000000:000000)_______  _____ _________>.

                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
        4096 gl.account.trans_                                  gdg=15 opt=c
          64 next=000004(000001:000012)000000:000000 jgl220
         128 170206:092839:00 000003(000001:000012)jgl220   S0010 cgl200
         192 170206:092758:00 000002(000001:000012)jgl220   S0010 cgl200
         256 170206:092544:00 000001(000001:000012)jgl220   S0010 cgl200
         320 000000:000000:___000000(000000:000000)_______  _____ _________
                      --------- 25 lines removed ---------
        1984 000000:000000:___000000(000000:000000)_______  _____ _________>.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

6I1. GDG Advanced Features - Demonstration

Notes re data1/gl.account.master_

1. Note the history of the 3 runs of jgl220.ksh. The latest is inserted at the top, pushing the previous history entries down.
   
   
2. data1/gl.account.master_ has been processed by exportgen0 & exportgen1 on each of 3 runs.
   
   
3. exportgen0 (for gl.account.master_) always selected gen# 000002 because of the 'next=000002...' override. Forcing a specific generation# for the master file thru multiple updates might not be practical, but was done here to demo the facility. The primary demo was to demo processing multiple generations of the gl.account.trans_ file (range processing).
   
   
4. exportgen1 created new generations from 01+000004 (at the bottom) to 01+000006 (at the top).
   
   

   Notes re data1/gl.account.trans_

5. data1/gl.account.trans_ has been processed only by exportgen0 on each of 4 runs.
   
   
6. The 1st 3 runs selected generations 000001, 000002, 000003 due to the 'next=......(000001:000012)...' override.
   
   
7. After 12 runs, next will be reset to 'next=000000(000001:000012)' 'next=000000' means future runs will select highest generation found. '(000001:000012)' is retained but inactive in case you want to reactivate by resetting 'next=000000' to 'next=000001' or higher.
   
   

   GDG control file history updates optional

As of June 2012, GDG control file history updates is optional depending on environmental variable GDGCTLUPDT, default GDGCTLUPDT=YES (in common_profile). Inhibiting history updates might speed up on very busy system and reduce possibility of file locks & corruption.

Of course GDGCTLUPDT=YES is required to use the advanced features documented on the pages above, and could be useful in debugging problems with GDG files.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

part_7 VU GDG files - Caveats

Part 7 - Contents

Here are some situations that may occur at some VSE sites, that require manual editing to enable the Vancouver Utility GDG system to function as intended.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_7 VU GDG files - Caveats

Introduction

The problems discussed in this section are for VSE systems.

Are there any left that have not yet converted to MVS or LUW ?

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

7B1. special procedures for VU GDG system required at some sites

replace $SYMBOLS in gdgctl file

Any $SYMBOLS in the GDG control file ($RUNLIBS/ctl/gdgctl51) need to be replaced by the actual values.

$SYMBOLS in the filenames given to exportgen0/exportgen1 are replaced by the Korn shell, but any $SYMBOLS in filenames extracted by jclgdgctl51 & loaded into the gdgctl file would remain as is.

We need to replace them with actual values so exportgen0/exportgen1 can match to the control file.

sample filenames extracted from JCL/scripts

 $MSTR/e2121653.mtaxe.piscines_           gdg=15
 $MSTR/e2121656.itaxe.facturat_           gdg=15
 $TAPE/ev.f01.28401a19.annexe_            gdg=15
 $TAPE/ev.f01.e211211.diftax_             gdg=15
 $TAPE2/tu.f01.e211801.adrpos_            gdg=15
 $TAPE2/tu.f01.e212-no.eauctr_            gdg=15

after replacing $SYMBOLS with values

 data1/e2121653.mtaxe.piscines_           gdg=15
 data1/e2121656.itaxe.facturat_           gdg=15
 tape/ev.f01.28401a19.annexe_            gdg=15
 tape/ev.f01.e211211.diftax_             gdg=15
 tape2/tu.f01.e211801.adrpos_            gdg=15
 tape2/tu.f01.e212-no.eauctr_            gdg=15

Note that the replacements are very easy to do using vi mass change commands. This editing would be in addition to the procedures shown earlier on page '1C3', so we will extend those operating instruction numbers.

 #5. vi ctl/gdgctl51    <-- edit the GDG control file
     ===============
 #5a. --> :%s/$MSTR/data1/
 #5b. --> :%s/$TAPE2/tape2/    <-- vital to replace $TAPE2 before $TAPE
 #5b. --> :%s/$TAPE/tape/

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

7C1. special procedures for VU GDG system required at some sites

correction for files reassigned

Here is a situation I noticed in a VSE conversion that currently requires manual correction.

VSE sites often defined most files up front & sometimes used SORT OPTION FILNM &/or TDYNASN to reassign the logical name to an alternate used for the SORT or a COBOL program.

This is a problem because the JCL converter assumes that the file is a non-GDG & uses 'exportfile' for the reassignment. This would cause 'file not found' because exportfile would assign the file as is (with trailing underscore). We need 'exportgen0/1' to select the latest or next GDG file with 6 digit sequence#. To illustrate the problem I have extracted a few lines from a typical VSE JCL.

 #!/bin/ksh
 ##JOB 33001D10
          ---------- lines omitted -----------
   .... VSE sites often defined most files up front ....
 exportfile E212001 data1/e2123001.itaxe.banqtaxe
 exportfile E212002 data1/e2123002.itaxe.taxation
 exportgen0 0 E212990 tape/tu.f01.e212-no.soldes_
 exportgen1 +1 E212991 $JTMP/tape/tu.f01.e212-no.soldav_

       ---------- lines omitted -----------
            ....... SORT step .......
 #  cat > $SYS011 <</*EOD
 #  SORT FIELDS=(18,6,A),FORMAT=BI,WORK=4
 #  OPTION FILNM=(E212084,E212990),SORTOUT=020,SORTIN=(014)
 # /*EOD
 exportfile SORTOUT $E212084
 exportfile SORTIN1 $E212990
 ## EXEC SORT
 #3----------------------------------------------------------------------
 uxsort "fili1=$SORTIN1,typ=RSF,rcs=354,filo1=$SORTOUT,keys=(17,6,b,a)"
 #4----------------------------------------------------------------------

 #3----------------------------------------------------------------------
 uxsort "fili1=$E212990,typ=RSF,rcs=354,filo1=$SORTOUT,keys=(17,6,b,a)"
 #4----------------------------------------------------------------------

TDYNASN is a similar situation

 ## EXEC TDYNASN
      ... TDYNASN step reassigned logical filename
 # ASSGN E212990,SYS014,INPUT
 exportfile SYS014 $E212990

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

7D1. special procedures for VU GDG system required at some sites

Write new generation & Readback in same job

There should be no manual changes required to jobs that write new generations of GDG files & read back on a later step within the same job, because the new GDG is written into $JTMP (jobtmp/subdir). 'exportgen1' determines the next generation by examining the files in the original $RUNDATA/subdir & does not get confused by the new generation in the $JTMP/GDG/...

The mainframe JCL would use the (+1) suffix on the new generation creation DSN, and also on the DSN that reads the file back, because the programmer knew that the GDG generation# was not updated until successful EOJ.

This situation is illustrated by demo JCL jcl0/JGL210.jcl, and here are the relevant lines extracted from the mainframe JCL & from the converted script (jcl3/jgl210.ksh):

JCL to demo Write GDG & Readback in 1 job

 //* Demo GDG processing, write gltrans(+1) & read back as (+1)
 //STEP010  EXEC PGM=SORT
 //SORTIN   DD DSN=GL.ACCOUNT.TRAN1,DISP=OLD
 //SORTOUT  DD DSN=GL.ACCOUNT.TRANS(+1),DISP=(,CATLG,DELETE),...
 //SYSIN    DD *
 SORT FIELDS=(1,8,CH,A,69,12,CH,A)
 /*
 //STEP020  EXEC PGM=CGL200,PARM=&YEAREND
 //GLTRANS  DD DSN=GL.ACCOUNT.TRANS(+1),DISP=OLD
 //GLMSOLD  DD DSN=GL.ACCOUNT.MASTER(0),DISP=OLD
 //GLMSNEW  DD DSN=GL.ACCOUNT.MASTER(+1),DISP=(,CATLG,DELETE),...

corresponding lines from converted script

 exportfile SORTIN data1/gl.account.tran1
 exportgen1 +1 SORTOUT $JTMP/data1/gl.account.trans_                    #<--Note1
 #exportgen1 assigns $JTMP/GDG/file, move to data1/file at Normal EOJ
 #3----------------------------------------------------------------------
 uvsort "fili1=$SORTIN,typ=RSF,rcs=80,filo1=$SORTOUT,typ=RSF,rcs=80\
 ,keys=(0,8,c,a,68,12,c,a)"
 #4----------------------------------------------------------------------
 exportgen1 +1 GLTRANS $JTMP/data1/gl.account.trans_                     #<--Note2
 #exportgen1 assign $JTMP/GDG/file, move to data1/file at Normal EOJ
 # 2+ exportgen1's for this file in this JCL/script
 exportgen0 0 GLMSOLD data1/gl.account.master_
 exportgen1 +1 GLMSNEW $JTMP/data1/gl.account.master_
 #exportgen1 assign $JTMP/GDG/file, move to data1/file at Normal EOJ
 #3----------------------------------------------------------------------
 cobrun $ANIM $CBLX/cgl200
 #4----------------------------------------------------------------------

The 'Note1' line above writes a new generation for the sorted file. The 'Note2' line above reads the new GDG back into the COBOL program. They were both codes with the '(+1)' suffix on the DSN's, and are now both coded on the 'exportgen1' function in the Korn shell script.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

7E1. special procedures for VU GDG system required at some sites

problem converting tape files to GDG's

We noticed a problem when converting VSE JCL's with tape files to GDG's.

The VSE JCL carried a comment indication INPUT or OUTPUT, and we added an option to the JCL converter to use the I/O indicator to generate 'exportgen1' for OUTPUT & 'exportgen0' for INPUT. We also coded the tape files in a 'tape/...' subdir. Using the example on the previous page the write & read back file definitions might be:

exportgen1 +1 SORTOUT $JTMP/tape/account.trans_ #Note1 (write)

 exportgen0 0 GLTRANS tape/account.trans_     #Note2 (readback on following step)

correction method #1 (OK)

exportgen1 +1 SORTOUT $JTMP/tape/account.trans_ #Note1 (write)

exportgen1 0 GLTRANS $JTMP/tape/account.trans_ #2 (readback on following step)

Correction method #1 is similar to the example on the previous page, resulting for the fact that the mainframe programmer would have coded both files with suffix '(+1)'.

correction method #2 (NOK)

exportgen1 +1 SORTOUT $JTMP/tape/account.trans_ #Note1 (write)

exportgen0 0 GLTRANS $JTMP/tape/account.trans_ #2 (readback on following step)

Correction method #2 is to leave the function as 'exportgen0' & simply add the '$JTMP/' prefix onto the file.

Note that the '$JTMP/' directory is honoured by exportgen0, but is ignored by exportgen1 when determining the latest generation within a directory. Function 'exportgen1' will 1st remove any '$JTMP/' prefix before it determines the latest generation, but 'exportgen0' will not.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_11 JCL conversion - Generation files

GDG functions & scripts

The functions & scripts are listed on the following pages.

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

11A1. JCL conversion - GDG functions

exportgen0 determine current generation

 # exportgen0  - function to export the latest existing generation of a filename
 #             - replaces mainframe MVS JCL GDG functions
 #             - used by Vancouver Utilities JCL to Korn shell converter
 #             - these functions distributed in /home/uvadm/sfun/...
 #
 #                      ** Change History **
 #
 #Aug19/14 - change $JOBID2 to $jobid2 on #cmts, code alrdy chgd Jun10
 #Jun10/14 - ensure filenames lower-case
 #Jun01/14 - allow AIX COBOL file type prefix, see sample below
 #Dec05/12 - verify directory of subdir/file_ or $RUNDATA/subdir/file_
 #Dec05/12 - old history cleared
 #
 #                  ** summary of GDG functions supplied **
 #
 # exportgen0 0:-50  - determine current generation (highest generation#)
 #                     or any prior generation (-1),(-2),(-3),...(-50)
 # exportgen1 +1:+50 - determine future generations (+1),(+2),(+3),...(+50)
 # exportgen2 +1:+50 - alternate exportgen1, writes direct to $RUNDATYA/subdir
 # exportgenall        - concatenates all existing generations
 # exportfile          - non-GDG, same as 'export', but displays filename
 #
 #                    ** uvcopy jobs used for GDG functions **
 #
 # jclgdgctl51 - creates gdgctl51 by extracting all exportgens from all scripts
 # gdgload1  - loads edited text file ctl/gdgctl51 to indexed file gdgctl51I
 # gdgget51  - gets records from indexed file by key, called by exportgen1
 # gdgget52  - gets records from indexed file by key, called by exportgen0
 # gdgupok1  - updates gdgctl file next=... reset =000000 or increment range
 # gdgupab1  - updates gdgctl file history, files NOT moved to RUNDATA/subdirs
 #
 #                   ** sample usage illustrated **
 #
 #   gl.account.master_000001
 #   gl.account.master_000002  <-- assume $RUNDATA contains these files:
 #   gl.account.master_000003
 #
 #command: exportgen0 0 CUSTMAS data1/gl.account.master_
 #         =============================================
 #result:  export CUSTMAS=data1/gl.account.master_000003  <-- gen0=highest onfile
 #         =============================================
 #
 #                      ** sample control file **
 #
 # gl.account.master_   gdg=10
 # gl.account.trans_    gdg=20  next=000003(000003:000003)_070531:235959
 # py.payroll.master_   gdg=30  opt=c
 # py.time.cards_       gdg=52  next=000001(000001:000012)_000000:000000
 #
 #
 #                      ** AIX COBOL file type **
 #
 # AIX COBOL file type may be coded as keyword cft=...
 # - not coded for non-cobol steps (different version not needed)
 #
 # exportgen0 0 CUSTMAS data1/ar.customer.master_ cft=QSAM
 # =======================================================
 # export CUSTMAS=QSAM-data1/ar.customer.master_000003      <-- result
 # ===================================================
 #
 #                       ** Vital Requirements **
 #
 # export FPATH=$APPSADM/sfun   #<-- must define FPATH in common_profile
 # ==========================      - directory holding these ksh functions
 #
 #                 ** coding for function exportgen0 **
 #
 function exportgen0  # function name here must match physical filename
 {
 if (($# < 3)); then
    logmsg2 "ERR: gen0: $# requires 3 args: gen#=$1, LFD=$2, LBL=$3"
    exit 9; fi
 integer gn=$1;
 if (($gn > 0)); then
    logmsg2 "ERR: gen0: arg1 gen# $gn must be 0,-1,-2,etc"
    exit 9; fi
 #
 # capture DDname & DSNname into named variables
 lfd=$2; lbla=$3;
 #
 # ensure filename lowercase
 ## lbla=$(echo $3 | tr [A-Z] [a-z])
 #Oct18/14 - get err --> tr: extra operand 'm' <-- ?? (#cmt out for now)
 # - on Red Hat running with ksh login shell
 # - OK on bash shell & following OK
 ## lbla=$(echo $3 | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz')
 # - but dont think we need for IMMD
 #
 #Jun01/14 - allow AIX COBOL file type as following keyword cft=...
 if [[ "$4" == cft=* ]]; then
    cft=${4#*=}; cft=${cft}-;
 else cft=""; fi
 #
 #Mar11/09 - change to allow option w2 $RUNDATA prefix on datafiles
 #         - extract subdir/filename from $RUNDATA/subdir/filename
 lbl1=${lbla##*/}      # drop large left to get filename
 lbl2=${lbla%/*}       # drop small right to get $RUNDATA/subdir
 lbl3=${lbl2##*/}      # drop large left to get subdir
 lbl4=$lbl3/$lbl1      # create subdir/filename (to match coding in gdgctl51I)
 lbld=$(echo $lbl4 | tr "/" "_")  # convert any '/' to '_' in DSNname
 #                                # for filename to hold ls of gdg filenames
 #
 #Dec05/12 - verify directory of subdir/file_ or $RUNDATA/subdir/file_
 if [[ ! -d "$lbl2" ]]; then
    logmsg2 "ERR: gen0: $lbl2 directory not found, continue/kill/fix ?"
    read reply; fi
 #
 # verify that filename ends with '_'
 if [[ ! "$lbla" == *_ ]]; then
    logmsg2 "ERR: gen0: filenames should end with '_', continue/kill/fix ?"
    read reply; fi
 #
 # verify $GTMP subdir present, created by jobset51
 if [[ ! -d $GTMP ]]; then
    logmsg2 "ERR: gen0: \$GTMP subdir not found GTMP=$GTMP"; exit 9; fi
 #
 # $GTMP subdir to hold output of 'ls' with all files matching filename_*
 # export GTMP=jobtmp/$jobid2/gtmp <-- defined in jobset51
 # used for ls output file create/readback to determine latest generation:
 #   lslbls=$GTMP/${JSTEP}I_$lbld
 #   lslbls=$RUNDATA/jobtmp/$jobid2/gtmp/${JSTEP}I_$lbld
 # could use GTMP=jobtmp/${jobid2}_$(date +%y%m%d%H%M%S)/gtmp
 #   in alternative code version for time stamped jobtmp subdirs
 #
 # capture current step# (part of ls output filename)
 typeset -RZ4 step="$JSTEP"                # capture step# from calling job
 if [[ -z "$step" ]]; then step=0000; fi   # set 0000 if undefined
 #
 # capture all matching GDG filenames into a file for readback to get highest#
 # - 1st create filename for 'ls' write lslbls=jobtmp/$jobid2/${JSTEP}I_$lbld
 # - with all filenames matching up to the trailing '_'
 lslbls=$GTMP/${step}I_$lbld       # assign filename for ls write
 ls $lbla[0-9][0-9][0-9][0-9][0-9][0-9] >$lslbls 2>/dev/null
 #
 # if no existing files, define ..._000001 filename, warning msg,& early return
 if [[ ! -s $lslbls ]]; then       # if NO existing files ?
    lblgena=${lbla}000001          # create filename for 1st generation
    export $lfd=$lblgena           # export LFD=lbl of next generation
    logmsg2 "ERR: gen0: No existing file, gen_000001 exported, $lfd=$lblgena "
    logmsg2 "ERR: gen0: No existing file, continue/kill/fix ?"
    read reply;                    # June05/03 OT warning added
    return 0; fi                   # return 0 if no files existing
 #
 # call uvcopy job gdgget52 to select control file entry for matching filename
 # - from indexed file ctl/gdgctl51I using filename as key
 # - uvcopy job will write selected entry to a file for reading by this script
 lblxG=$GTMP/${step}G0_$lbld        # create filename to be written by uvcopy
 #
 #Jun11/12 - test $GDGCTLUPDT for alt uvcopy job with noupdate/nolock
 if [[ "$GDGCTLUPDT" == "NO" ]]
 then uvcopy gdgget52n,fili1=$lslbls,filo2=$lblxG,arg1=$lfd,arg2=$lbl1,arg3=$gn,rop=i63
 else uvcopy gdgget52,fili1=$lslbls,filo2=$lblxG,arg1=$lfd,arg2=$lbl1,arg3=$gn,rop=i63
      #===============================================================================
 #Nov16/14 - fix arg2=$lbl4 to arg2=$lbl1 (ctlfile now no subdir)
 fi
 # arg1  = DDN (ex: GLMSTR)
 # arg2  = DSN filename key for lookup gdgctl file (ex: gl.account.master_)
 # arg3  = gen# desired (0,-1,-2,etc)
 # fili1 = $lslbls ls of all matching filenames_* to underscore (+ 6 digits)
 # filo2 = $lblxG output file for gdgctl entry (possible override, prior gen#)
 #         output file has 1 line with 4 fields as follows:
 # ===============================
 # gen#   gdg=... next=... opt=...
 # 000004 gdg=15
 # 000006 gdg=24  next=000006(000001:000012)070529:235959 opt=c
 # ============================================================
 # field1 = gen# determined by gdgget52 (Oct18/08 changed from filename_gen#)
 # field2 = gdg=... no of generations maintained for this file
 # field3 = next=... in gdgctl file (not/used except for debug/audit)
 # field4 = opt=c from gdgctl file, causes prompt for confirm/change
 #Oct18/08 - gdgget52 changed to return just gen# vs filename_gen#
 #         - exportgen0 changed to concat filename_ + gen#
 #
 # read the file written by uvcopy, 1 line with 4 fields (as above)
 exec 3< $lblxG                # open the file
 read -u3 genx gens next opts  # read 1st 4 fields (4th field options)
 exec 3<&-                     # close file
 lblgenx=$lbla$genx            # concat filename_ + gen#
 #
 export $lfd=$cft$lblgenx      # export LFD=LBL for selected filename
 #=======================
 #
 # get filesize & display results for console log
 fsize="";
 if [[ $lblgenx == /* ]]; then lblx=$lblgenx; else lblx=$RUNDATA/$lblgenx; fi
 ## if [[ -f $lblx ]]; then fsize=$(stat -c%s $lblx); fi
 #Nov09/10 - SUN unix does not have stat to get filesize
 #Nov19/10 - use 'du' as portable way to get filesize
 #Oct15/11 - du option 'h' reports in 4K increments (but OK)
 #Jan06/14 - change 'du' to '\du' disable any alias du
 #         - use option -k (-h did not work on AIX)
 \du -k $lblgenx | read fsize dummy
 #
 logmsg1 "gen$gn $lfd=$cft$lblgenx insize=${fsize}K"
 #==================================================
 # return no of generations existing
 gdgsbf=$(wc -l $lslbls); gdgsb=${gdgsbf% *}; gdgs=${gdgsb##* };
 return $gdgs        # return no of generations existing
 }

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

11B1. JCL conversion - GDG functions

exportgen1 determine next generation

 # exportgen1  - function to export NEXT generation of a file (not yet existing)
 #             - replaces mainframe MVS JCL GDG functions
 #             - used by Vancouver Utilities JCL to Korn shell converter
 #             - these functions distributed in /home/uvadm/sfun/...
 #             - must be copied to $APPSADM/sfun, since common_profile FPATH is:
 #               export FPATH=$APPSADM/sfun  <--Function PATH for ksh
 #
 #                          ** Change History **
 #
 #Feb2016  - corrections as noted by Joseph dale, Cothern Computers
 #         - touch & gdg=... optional args 4 & 5 not working
 #         - touch corrected from $lblgenx to $lbljag about line 244
 #         - gdg=... override Indexed file not working
 #         - capture gdg=... moved after uvcopy return gens about line 220
 #Oct18/14 - modify for IMMD new dir structure $DATA/gg/filename
 #           $DATA=$ISS2APPHOME/data/gg (fullpath)
 #         - write newgdg to $JGDG/fullpath('@'s replace '/'s)/filename
 #         - jobend51 will replace '@'s with '/'s to write newgdg
 #         - this allows for the new data/gg/ & for GDG's in other systems
 #         - elimnates need for GDGmkdirs
 #Aug19/14 - change $JOBID2 to $jobid2 on #cmts & line 158 export JTMP=...
 #Jun10/14 - ensure filenames lower-case
 #Jun01/14 - allow AIX COBOL file type prefix, see samples further below
 #May24/12 - modify to allow superdir/subdir/filename (vs subdir/filename)
 #           data/filexx will now default to $RUNDATA/data/filexx
 #Mar14/12 - gdgctl51I located via env-var $GDGCTL vs $RUNDATA
 #         - common_profile default "export GDGCTL=$RUNDATA/ctl"
 #         - could change to "export GDGCTL=$APPSADM/ctl"
 #Dec05/12 - clear prior history
 #Jul2008  - avoids GDG incrementing if job abends
 #         - gets gdg=... values from indexed file $RUNDATA/gdgctl51I.dat/.idx
 #
 #                ** summary of GDG functions supplied **
 #
 # exportgen0 0:-50  - determine current generation (highest generation#)
 #                     or any prior generation (-1),(-2),(-3),...(-50)
 # exportgen1 +1:+50 - determine future generations (+1),(+2),(+3),...(+50)
 # exportgen2 +1:+50 - alternate exportgen1, writes direct to $RUNDATYA/subdir
 # exportgenall      - concatenates all existing generations
 # exportfile        - non-GDG, same as 'export', but displays filename
 #
 #                    ** uvcopy jobs used for GDG functions **
 #
 # jclgdgctl51 - creates gdgctl51 by extracting all exportgens from all scripts
 # gdgload1  - loads edited text file ctl/gdgctl51 to indexed file gdgctl51I
 # gdgget51  - gets records from indexed file by key, called by exportgen1
 # gdgget52  - gets records from indexed file by key, called by exportgen0
 # gdgupok1  - updates gdgctl file next=... reset =000000 or increment range
 # gdgupab1  - updates gdgctl file history, files NOT moved to RUNDATA/subdirs
 #
 #                       ** Vital Requirements **
 #
 # Directory holding these functions must be defined by FPATH (for Korn shell)
 # export FPATH=$APPSADM/sfun       #<-- in common_profile
 # ==========================
 #
 #                    ** command format & sample usage **
 #
 # exportgen1 gen DDname DSname [generations] [touch option] <-- format
 # =========================================================
 # - arg1 gen desired +1,+2,etc (as of Jan 2011)
 # - arg2 Logical filename
 # - arg3 physical filename
 # - arg4 generations to maintain (see default below if absent)
 # - arg5 option to touch (create new empty file)
 # - arg4 & arg5 order may be reversed
 #
 # exportgen1 +1 CUSTMAS data1/gl.account.master_   <-- example
 # ==============================================
 #
 # data1/gl.account.master_000001
 # data1/gl.account.master_000002 <-- existing files
 # data1/gl.account.master_000003
 #
 # export CUSTMAS=$JGDG/gl.account.master_000004     <-- result
 # =============================================
 # $JGDG is jobtmp/jobname/GDG/, example jobtmp/jgl230/GDG/
 #
 #Oct2014 - GDG files now written to $JGDG with full-path
 #        - by coding '/'s as '@'s, for example:
 #
 # jobtmp/jgl230/GDG/@home@mvstest@testdata@data1@gl.account.master_000004
 # =======================================================================
 # at normal EOJ, will be copied to $RUNDATA/data1/gl.account.master_000004
 #
 #                  ** AIX COBOL file type **
 #
 # AIX COBOL file type may be coded as keyword cft=...
 # - not coded for non-cobol steps (different version not needed)
 #
 # exportgen1 +1 CUSTMAS data1/ar.customer.master_ cft=QSAM
 # ========================================================
 # export CUSTMAS=QSAM-data1/ar.customer.master_000004       <-- result
 # ===================================================
 #
 #           ** filename manipulations **
 #
 # lbla  =  arg3 = data/filename        (VU)
 #          arg3 = $DATA/gg/filename    (IMMD)
 # lblp  =  fullpath/filename (prepends $RUNDATA if no leading /)
 # lblf  = filename only
 # dirp  = directories (fullpath to filename)  <-- this or next ?
 # dirpa = directories(fullpath) with '/'s replaced by '@'s
 # lblja  = ${dirpa}@${lblf}             <-- will write to $JGDG (adding gen#)
 # lbljag = $JGDG/${dirpa}@${lblf}$genx  <-- write newgdg in jobtmp/...
 #                                        - for restore by jobend51
 #                  ** coding for: function exportgen1 **
 #
 function exportgen1  # function name here must match physical filename
 {
 # define/create GDG subdirs if not present
 export JTMP=jobtmp/$jobid2
 export JGDG=$JTMP/GDG
 export GTMP=$JTMP/gtmp
 test -d $JTMP || mkdir $JTMP
 test -d $JGDG || mkdir $JGDG
 test -d $GTMP || mkdir $GTMP
 #
 typeset -RZ6 gen0;             # define cur gen# 6 digits Right just, zero fill
 typeset -RZ6 genx;             # define next gen# 6 digits Right just, zero fill
 integer gensdflt=15;   # generations default 15
 integer gens=$gensdflt # init gens to gensdflt
 integer filecnt=0; integer excess=0; integer lblcnt=0;  # clear counters
 #
 if (($# < 3)); then
    logmsg2 "ERR: gen1: $1 requires 3 args: gen=$1, LFD=$2, lbl=$3"
    exit 91; fi
 #
 # capture gen, DDname, DSNname cft=... gdg=...
 gn=$1; lfd=$2; lbla=$3;
 #
 # ensure filename lowercase
 ## lbla=$(echo $lbla | tr [A-Z] [a-z])
 #Oct18/14 - #cmntd out got err --> tr: extra operand 'm' ??
 # - on Red Hat running with ksh login shell
 # - OK on bash shell & following OK
 ## lbla=$(echo $lbla | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz')
 # - but dont think we need for IMMD
 #
 #Jun01/14 - allow AIX COBOL file type as following keyword cft=...
 if [[ "$4" == cft=* ]]; then
    cft=${4#*=}; cft=${cft}-;
 else cft=""; fi
 #
 # verify that filename (arg3 on call) ends with '_'
 if [[ ! "$lbla" == *_ ]]; then
    logmsg2 "ERR: gen1: filenames must end with '_', continue/kill/fix ?"
    read reply; fi
 #
 # ensure fullpathname, if no leading '/' - prepend $RUNDATA
 lblp=$lbla
 if [[ $lblp != /* ]]; then lblp=$RUNDATA/$lbla; fi
 #
 # avoid problem when no matching files present
 # - by creating gen# filename_000000 as nullfile & removing later below
 if [[ ! -e ${lblp}000000 ]]; then touch ${lblp}000000; fi
 # echo "DEBUG1: \$lbla=$lbla \$lblp=$lblp"
 #
 # split fullpathname to directories/filename
 lblf=${lblp##*/}      # drop large left to get filename only
 dirp=${lblp%/*}       # drop small right to get directories only
 #
 #Dec03/12 - verify directory of subdir/file_ or $RUNDATA/subdir/file_
 if [[ ! -d "$dirp" ]]; then
    logmsg2 "ERR: gen1: $ddirp directory not found, continue/kill/fix ?"
    read reply; fi
 #
 ##lbla=$(echo $3 | tr [A-Z] [a-z]) #Jun10/14 ensure filename lower-case
 # convert '/'s to '@'s to write newgdg to $JGDG temp restore by jobend51
 lblja=$(echo $lblp | tr '/' '@')
 # will later create lbljag - with next gen# appended
 #
 # create filename to write all matching lbls (in $Gtmp/...)
 typeset -RZ4 step="$JSTEP"                # capture step# from calling job
 if [[ -z "$step" ]]; then step=0000; fi   # set 0000 if undefined
 lslbls=$GTMP/${step}O_$lblf       # assign filename for ls write
 # use 'ls' to create tmp file of matching filenames
 ls $lblp[0-9][0-9][0-9][0-9][0-9][0-9] >$lslbls 2>/dev/null
 #
 # read back file of filenames created by 'ls' above & use last name
 exec 3< $lslbls                 # open file of 'ls' names
 while read -u3 lblgenr          # read lines of ls filenames
       do  lblgenh=$lblgenr      # save highest generation filename
           ((filecnt+=1))        # count files for excess delete
       done                      #
 exec 3<&-                       # close file
 #
 gen0=${lblgenh##*_}             # drop filename, capture gen# of last filename
 genx=$gen0
 ((genx+=$gn))                   # calc next generation
 #
 #Sep28/09 - display for debugging
 ## echo "DEBUG3: lbla=$lbla gen0=$gen0 genx=$genx"
 ## read reply
 #
 # create filename for newgdg write to $JGDG
 lbljag=$JGDG/${lblja}$genx
 #
 # export for COBOL program, prepending with Cobol File Type if any
 export $lfd=$cft$lbljag     # export LFD=LBL for selected filename
 #======================     # for COBOL or utility program

 #
 # retrieve gdg=... from indexed file using filename key & uvcopy job gdgget51
 lblG=$GTMP/${step}G1_$lblf     # create filename for outfile
 # lblG=$GTMP/${step}G$gn_$lblf     # create filename for outfile
 #Oct18/14 - lblG outfile for uvcopy with new gen# not used, replaced by return code
 #         - removed the $gn_ which disappeared & looked funny
 #
 #Jun11/12 - test $GDGCTLUPDT for alt uvcopy job with noupdate/nolock
 if [[ "$GDGCTLUPDT" == "NO" ]]
 then uvcopy gdgget51n,fili1=$lslbls,filo2=$lblG,arg1=$lblf,arg2=$genx,arg3=$gn,rop=i63
 else uvcopy gdgget51,fili1=$lslbls,filo2=$lblG,arg1=$lblf,arg2=$genx,arg3=$gn,rop=i63
      #===============================================================================
 fi
 gens=$?           # capture gdg=... value from return code
 #
 # override gdg=.., if arg4 or arg5
 if [[ "$4" == gdg=* ]]; then gens=${4#gdg=};
 elif [[ "$5" == gdg=* ]]; then gens=${5#gdg=}; fi
 #
 # verify generations min/max, if invalid set to default
 if (($gens<2)) || (($gens>250)); then gens=$gensdflt; fi
 #
 # delete files to maintain generations spcfd by arg#3 or default (see above)
 ((excess=$filecnt-$gens+1))    # calc number of files to delete
 #note: +1 because filecnt does not include the new generation being created
 # echo "DEBUG4: gens=$gens, filecnt=$filecnt, excess=$excess"
 #
 # if excess, re-read file of filenames, deleting 1st few excess
 if ((excess > 0)); then
    exec 3< $lslbls              # re-open file of 'ls' names
    while read -u3 lblgenr       # read lines of ls filenames
          do ((lblcnt+=1))      # count files for excess delete
             if ((lblcnt <= excess)); then rm -f $lblgenr; fi
          done                   #
    exec 3<&-                    # close file
 fi
 # can now remove the _000000 nullfile if created above
 if [[ ! -s ${lblp}000000 ]]; then rm -f ${lblp}000000; fi
 #
 # touch (create new empty file) if arg4,5,or 6 = 'touch'
 if [[ "$4" == "touch" ]]; then touch $lbljag;
 elif [[ "$5" == "touch" ]]; then touch $lbljag;
 elif [[ "$6" == "touch" ]]; then touch $lbljag; fi
 #
 # display result for console log
 logmsg1 "gen1$gn $lfd=$cft$lbljag gens=$gens"
 #============================================
 # return no of generations existing
 gdgsbf=$(wc -l $lslbls); gdgsb=${gdgsbf% *}; gdgs=${gdgsb##* };
 return $gdgs        # return no of generations existing
 }

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

11C1. JCL conversion - GDG functions

exportgenall concatenate all generations

 # exportgenall - function to export all of the existing generation of a filename
 #              - by Mark Wedge, Armas Solutions, Aug 2002
 #
 #Aug19/14 - change $JOBID2 to $jobid2 on #cmts, code alrdy chgd Jun10
 #Jun01/14 - allow AIX COBOL file type prefix, see sample below
 #
 # All matching files with the 6 digit gen# are concatentated into 1 output file.
 # The output file will have the trailing '_' (ID for generation file)
 # but will have no 6 digit# (preventing any future inclusion as a member)
 #
 # ap/account.trans_000001
 # ap/account.trans_000002   <-- sample input 3 files
 # ap/account.trans_000003
 #
 # ap/account.trans_       <-- output 3 files combined into 1
 #
 #command: exportgenall CUSTMAS ap/account.trans_
 #         ======================================
 #result:  export CUSTMAS=ap/account.trans_
 #         ================================
 #
 # All files 'ap/account.trans_[0-9][0-9][0-9][0-9][0-9][0-9]'
 # will be concatenated in to the base name 'ap/account.trans_'
 # To remove all generations, the 'rm' comamnd will be generated as follows:
 # rm -f ${CUSTMAS}[0-9][0-9][0-9][0-9][0-9][0-9]
 # The concatenated file '_' will remain & might be a backup in case of error
 #
 #                      ** AIX COBOL file type **
 #
 # AIX COBOL file type may be coded as keyword cft=...
 # - not coded for non-cobol steps (different version not needed)
 #
 # exportgenall CUSTMAS ar/customer.master_ cft=QSAM
 # =================================================
 # export CUSTMAS=QSAM-ar/customer.master_            <-- result
 # =======================================
 #
 function exportgenall
 {
 if [[ -n "$1" && -n "$2" ]]; then :
 else logmsg2 "ERR: exportgenall requires: LFD=$1 lbl=$2"; exit 9; fi
 lfd=$1; lbl1=$2
 #
 #Jun01/14 - allow AIX COBOL file type as following keyword cft=...
 if [[ "$3" == cft=* ]]; then
    cft=${3#*=}; cft=${cft}-;
 else cft=""; fi
 #
 typeset -RZ4 step="$JSTEP"                # capture step# from calling job
 if [[ -z "$step" ]]; then step=0000; fi   # set 0000 if undefined
 #
 #Mar11/09 - change to allow option w2 $RUNDATA prefix on datafiles
 #         - extract subdir/filename from $RUNDATA/subdir/filename
 lbl2=${lbl1##*/}      # drop large left to get filename
 lbl3=${lbl1%/*}       # drop small right to get $RUNDATA/subdir
 lbl4=${lbl3##*/}      # drop large left to get subdir
 lbla=$lbl4/$lbl2      # create subdir/filename (to match coding in gdgctl51I)
 lblc=$(echo $lbla | tr "/" "_") #convert any '/' to '_' in DSNname
 #                     # for use as filename of filenames in $JGDG/gtmp
 #
 #Jun05/03 - verify that filename (arg2 on call) ends with '_'
 if [[ ! "$lbla" == *_ ]]; then
    logmsg2 "ERR: gen filenames should end with '_', continue?, need fix?"
    read reply; fi
 #
 cat /dev/null > $lbla
 cat ${lbla}[0-9][0-9][0-9][0-9][0-9][0-9] > $lbla 2>$GTMP/${step}GAERR_$lblc
 #===========================================================================
 #Nov29/11 - redirect stderr to file in gtmp/...
 #
 export $lfd=$cft$lbla  # result export DDNAME=DSNname_
 #====================
 #
 # get filesize & display results for console log
 fsize="";
 if [[ $lbla == /* ]]; then lblb=$lbla; else lblb=$RUNDATA/$lbla; fi
 ##if [[ -f $lblb ]]; then fsize=$(stat -c%s $lblb); fi
 ##Nov09/10 - SUN unix does not have stat to get filesize
 logmsg1 "genall: $lfd=$cft$lbla bytes=$fsize"
 #============================================
 #
 #Aug29/08 - call gdgget52 to add exportgenall 'GA' entry to ctlfile history
 # capture all matching GDG filenames into a file for readback to get low#/high#
 # - 1st create filename for 'ls' write lslbls=jobtmp/$jobid2/${JSTEP}I_$lbla
 # - with all filenames matching up to the trailing '_'
 lslbls=$GTMP/${step}I_$lblc       # assign filename for ls write
 ls $lbla[0-9][0-9][0-9][0-9][0-9][0-9] >$lslbls 2>/dev/null
 #
 # call uvcopy job gdgget52 to select control file entry for matching filename
 # - from indexed file ctl/gdgctl51I using filename as key
 lblxG=$GTMP/${step}GA_$lblc       # create filename to be written by uvcopy
 #Oct18/08 - remove $RUNDATA/ if present due to option e1 in jclunixop51
 lblr=${lbla#$RUNDATA/}            # remove $RUNDATA/ for optn e1 jclunixop51
 #Note - output file not required for exportgenall (simply adding history)
 #
 #Jun11/12 - test $GDGCTLUPDT for alt uvcopy job with noupdate/nolock
 if [[ "$GDGCTLUPDT" == "NO" ]]
 then uvcopy gdgget52n,fili1=$lslbls,filo2=$lblxG,arg1=$lfd,arg2=$lblr,arg3=GA,rop=i63
 else uvcopy gdgget52,fili1=$lslbls,filo2=$lblxG,arg1=$lfd,arg2=$lblr,arg3=GA,rop=i63
      #===============================================================================
 fi
 # arg1  = DDN (ex: GLMSTR)
 # arg2  = DSN filename key for lookup gdgctl file (ex: gl/account.master_)
 # arg3  = 'GA' indicates All Generations, coded in gdgctl (GDG History file)
 #         vs codes 00,01-,02-,etc for exportgen0 & 01+,02+,etc for exportgen1
 # fili1 = $lslbls ls of all matching filenames_* to underscore (+ 6 digits)
 # filo2 = $lblxG output file for gdgctl entry (NOT required for exportgenall)
 #
 return 0
 }

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

11D1. JCL conversion - non-GDG function

exportgenx - report err if gen# out of bounds

 # exportgenx - function to report error in JCL to script conversion
 #            - when GDG JCL not as expected
 #            - see GDG doc at: www.uvsoftware.ca/jclcnv4gdg.htm
 #
 #Aug19/14 - change $JOBID2 to $jobid2 on echo (no active code uses here)
 # exportgen0 - current generation (0) & prior (-1),(-2),(-3)
 # exportgen1 - future generations not yet existing (+1),(+2),etc
 # exportgenx - (this function) errmsg if GDG JCL was not as expected
 # exportgenall - concatenates all existing generations
 #
 # This 'exportgenx' reports an error if script run before manual correction
 #
 function exportgenx
 {
 lfd=$1; lbl=$2          # capture args into named variables
 logmsg2 "ERR: exportgenx - ERR in JCL to UNIX script conversion"
 logmsg2 "- LFD=$lfd, LBL=$lbl."
 logmsg2 "- see GDG doc at: www.uvsoftware.ca/jclcnv4gdg.htm"
 logmsg2 "- correct script $jobid2 & rerun"
 logmsg2 "- enter=continue, or kill job by interrupt (del or ^C)"
 read reply               # wait for operator to enter or kill
 return 9
 }

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

11E1. JCL conversion - non-GDG function

exportfile - exports & display non-GDG filename

 # exportfile - function to export DDname=DSName for Micro Focus external names
 #              DDname (logical name in program) = DSName (physical filename)
 #
 #Jun10/14 - ensure filenames lower-case
 #Jun01/14 - allow AIX COBOL file type prefix, see sample below
 #
 # exportfile CUSTMAS data/ar.customer.master  <-- example of use
 # ==========================================
 # export CUSTMAS=data/ar.customer.master      <-- result
 # ======================================
 #
 # exportfile - display non-GDG filenames on console (& console-log)
 #            - also see GDG functions:
 # exportgen0 - to determine current generation (0) highest gen#
 # exportgen1 - to determine next generation (+1)
 #
 #Jun01/14 - allow AIX COBOL file type prefix
 #         - Cobol File Type code follows exportfile as follows:
 #         - not coded for non-cobol steps (different version not needed)
 #
 # exportfile CUSTMAS data/ar.customer.master cft=QSAM
 # ===================================================
 # export CUSTMAS=QSAM-data/ar.customer.master          <-- result
 # ===========================================
 #
 # export FPATH=$APPSADM/sfun
 # ==========================
 # - functions must be in directory defined by FPATH in profiles as above
 #
 function exportfile
 {
 #
 if [[ -n "$1" && -n "$2" ]]; then :
 else logmsg2 "ERR: exportfile requires: DDname=$1 DSName=$2."
      exit 9; fi
 #
 # echo "DEBUG1: \$1=$1, \$2=$2"
 # capture args into named variables
 lfd=$1; lbl=$2
 #Jun10/14 ensure filename lower-case
 #
 # ensure filename lowercase
 ## lbl=$(echo "$lbl" | tr [A-Z] [a-z])
 #Oct18/14 - get err --> tr: extra operand 'm' <-- ?? (#cmt out for now)
 # - on Red Hat running with ksh login shell
 # - OK on bash shell & following OK
 ## lbla=$(echo $2 | tr 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' 'abcdefghijklmnopqrstuvwxyz')
 # - but dont think we need for IMMD
 # echo "DEBUG2: \$lbl=$lbl"
 #
 #Jun01/14 - allow AIX COBOL file type as following keyword cft=...
 if [[ "$3" == cft=* ]]; then
    cft=${3#*=}; cft=${cft}-;
 else cft=""; fi
 #echo "DEBUG3: \$3=$3, \$cft=$cft"
 #
 export $lfd=$cft$lbl        # export LFD=lbl of highest gen existing
 #===================
 #
 #Oct05/08 - init file if name ends with 'nullfile'
 if [[ $lbl == *nullfile ]]; then >$lbl; fi
 #
 # get filesize & display results for console log
 # - allow for Indexed file by also testing for '.dat' suffix
 #Feb08/15 - allow for PL/1 coding on end filename,recsize(80),type(fixed)
 lbla=${lbl%%,*}
 fsize="";
 if [[ $lbla == /* ]]; then lblc=$lbla; lbld=$lbla.dat;
    else lblc=$RUNDATA/$lbla; lbld=$RUNDATA/$lbla.dat; fi
 ## if [[ -f $lblc ]]; then fsize=$(stat -c%s $lblc);
 ##   elif [[ -f $lbld ]]; then fsize=$(stat -c%s $lbld); fi
 ## SUN has no 'stat', use 'du' without option 'h'
 #
 if [[ -f $lblc ]]; then \du -k $lblc | read fsize dummy
 elif [[ -f $lblc.dat ]]; then \du -k $lblc.dat | read fsize dummy; fi
 if [[ -z "$fsize" ]]; then fsize=0; fi
 logmsg1 "file: $lfd=$cft$lbl fsize=${fsize}K"
 #============================================
 return 0      # return 0 if file was found
 }

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

11F1. JCL conversion - GDG functions

jobset51 - setup function (line 10 of JCL/scripts)

 # jobset51 - setup environment for UNIX scripts converted from MVS JCL
 #          - this KORN shell function stored in $APPSADM/sfun
 #          - called at the begining of each JCL/script
 #          - inserted by JCL converter from ctl/jclunixop51
 #          - uses $RUNLIBS & $RUNDATA (exported in prgmr/oprtr profiles)
 #            to define Libraries & Data dirs for 'test' or 'prod'uction
 #          - see more doc at: www.uvsoftware.ca/jclcnv4gdg.htm
 # jobset51 - use for most Vancouver Utility installations
 # jobset53 - alternative for AIX/IMMD directory structure
 #
 #        - - - sample RUNLIBS/RUNDATA for programer testing - - -
 # export RUNLIBS=/home/mvstest/testlibs #<-- test/demo Libraries
 # export RUNDATA=/home/mvstest/testdata #<-- test/demo Data superdir
 #        - - - sample RUNLIBS/RUNDATA for production operators - - -
 # export RUNLIBS=/p2/apps/prodlibs           #<-- production libraries
 # export RUNDATA=/p2/apps/proddata           #<-- production Data superdir
 #
 # - JCL/scripts are found via $RUNLIBS/jcls in the profile PATH
 # - COBOL program subdir defined here in jobset51 as '$RUNLIBS/cblx'
 #   (this allows programs to have same names as JCL/scripts)
 #
 function jobset51
 {
 cd $RUNDATA              #change to working dir for production (or test)
 cdstatus=$?              # capture status of cd $RUNDATA
 #Oct20/14 - ensure reqquired subdirs present in $RUNDATA
 test -d jobctl   || mkdir jobctl
 test -d joblog   || mkdir joblog
 test -d jobmsgs  || mkdir jobmsgs
 test -d jobtimes || mkdir jobtimes
 if ((cdstatus))          # ERR if cdstatus not zero (OK)
    then logmsg2 "jobset51 ERR: cd \$RUNDATA ($RUNDATA) failed"
         logmsg2 "- investigate, RUNDATA definition in profiles"
         logmsg1 "- enter to exit"; read reply; exit 91; fi
 #
 # cd $RUNDATA means all files are referenced relative to $RUNDATA
 # - allows switching between production & test files by changing $RUNDATA
 # - COBOL programs are found via $CBLX/progname in JCL/scripts
 # - RPG   programs are found via $RPX/progname in JCL/scripts
 export CBLX=$RUNLIBS/cblx    # path for loading COBOL programs
 export RPX=$RUNLIBS/rpgx     # path for loading RPG programs
 #
 if [ -z "$ANIM" ]; then ANIM=-F; fi
 # 'ANIM=-F' inhibits non-numeric field checks (could change to +F ?)
 # - cobol programs are called via ---> cobrun $ANIM $CBLX/progname <---
 #
 # make subdirs for work files, instream data files,& GDG subdirs
 # define jobtmp/subdirs for new GDG files
 umask 002  # ensure perms to create dirs/files 775/664 (group share)
 export JTMP=jobtmp/$jobid2
 export JGDG=$JTMP/GDG
 export GTMP=$JTMP/gtmp
 test -d $JTMP || mkdir $JTMP
 test -d $JGDG || mkdir $JGDG
 test -d $GTMP || mkdir $GTMP
 test -d $JTMP/tmp || mkdir $JTMP/tmp
 # $JTMP/...     - for work files & instream data files
 # $GTMP/...     - for exportgen0,1,etc 'ls' output files
 #                 GDG matching filenames readback to determine latest gen#
 # $JGDG/...     - store new GDGs for move back at Normal EOJ
 # $JTMP/tmp/... - for ADRDSSU DUMP INCLUDE/EXCLUDE, each ssuch step will:
 #               - clear $JTMP/tmp/*, copy to, remove from,
 #                 then move remaining files to output directory tape1/...
 #
 #Note - if multiple copies of same job could be run at same time:
 #       to avoid problems naming temp dirs using jobnames
 # solution#1
 #  - use the 'joblog2' script to run these jobs
 #  - joblog2 ensures any currently running same name job completes
 #    before scheduling the next same name job
 #  - this solution does not require any change here in jobset51
 # solution#2
 # - $JTMP subdir could be date/time stamped (or append process ID)
 #   export JTMP=jobtmp/${jobid2}$(date +%y%m%d%H%M%S) #<-- date/time stamp
 #   export JTMP=jobtmp/${jobid2}$$                    #<-- process ID
 # - If you do this, then 'rm -rf $RUNDATA/jobtmp/*' nightly by cron
 # - see crontab_appsadm1 & script cleantmps at ADMjobs.htm#5B1
 #
 #----------- SYOT/sysout definition ---------------
 export SYOT=sysout/$JOBID2
 # jclunix51 changed to generate SYSOUT files as follows:
 # exportfile SYSOUT $SYOT/${JOBID2}_${JSTEP}_SYSOUT_$(date +%y%m%d_%H%M%S) J20
 # - should setup cron to remove sysout/... files older than 30 days ??
 # make subdir for SYSOUT files (or any file w/o DSN=...)
 if [[ ! -d $SYOT ]]; then mkdir $SYOT; fi
 #
 # $JTMP & $SYOT are vital - ensure created successfully else exit
 if [[ -d $JTMP && -d $SYOT ]]; then :
    else logmsg2 "jobset51 ERR: $JTMP &/or $SYOT failed creation"
         logmsg2 "- investigate: permissions,JTMP,SYOT dirnames changed?"
         logmsg1 "- enter to exit"; read reply; exit 92; fi
 #Note - in code above we create $JTMP & $SYOT if not already existing
 #     - will clean out any prior run $JTMP files (but not $SYOT files)
 # - BUT not on a 'RESTART', see following line of code below (near end)
 # ---> else rm -rf $JTMP/*; fi  <--- see below
 #
 # ensure nullfile present & empty, for use on DD DUMMY
 # - use /dev/null, but some unix variants had problems with that
 >tmp/nullfile    # ensure present & empty
 export NULLFILE=tmp/nullfile
 #Nov15/13 - setup JTMP1 file for miscellaneous uses
 #         - ICETOOL uses tmp1 as output file for COUNT function
 export JTMP1=$JTMP/tmp1
 #
 # Test for any files in $JGDG/subdirs at begin job
 # (prior run abend not moving GDG/subdir/files back to $RUNDATA/subdirs)
 # - will prompt operator to move back or not (rerun from begining)
 lsgdgs=$(ls $JGDG/* 2>/dev/null);  # capture filenames in GDG/subdirs
 if [[ -n "$lsgdgs" ]]; then
    logmsg2 "jobset51 WARN: files in jobtmp/GDG subdirs (prior AbTerm ERR?)"
    logmsg2 "---- files in $JGDG/*/* listed below:"
    echo    "$lsgdgs"
    logmsg2 "If NO restart by step#, GDG files in jobtmp/... will be cleared"
    logmsg2 "   - allows rerun from begin job with no worry about GDGs"
    logmsg2 "If RESTARTing by step#, example--> jobname.ksh start=S0050"
    logmsg2 "   - GDG files in jobtmp/... will NOT be cleared"
    logmsg2 "   - will be available to steps after restart step#"
    logmsg2 "   - will be restored to data1/... subdir at JobEnd=Normal"
    logmsg2 "enter/continue - will clear jobtmp/... if no restart"
    logmsg2 "enter/continue - will not clear jobtmp/... if start=..."
    read reply
 fi
 # initialize step counters
 export JSTEP=S0000 XSTEP=0
 integer JCC=0 SCC=0 LCC=0;
 #
 # Allow for RESTART at any step by start=... argument on command line:
 # --> jobname.ksh start=S0020   <-- restart at step 2
 #Oct20/14 - allow both start=... & restart=...
 test -n "$restart" && start="$restart"
 test -z "$start"   && export start=S0000
 if [[ $start != S[0-9][0-9][0-9][0-9] ]]; then
    logmsg2 "jobset51 ERR: start=$start invalid"; exit 94; fi
 alias goto="<<${start}=\A"
 #
 if [[ $start != S0000 ]]; then
    logmsg2 "WARN: **START** at start=$start";
 else
    # ensure $JGDG/subdirs exist to hold exportgen1 files until Normal EOJ
    # Normal EOJ will move $JGDG/subdir/files back to $RUNDATA/GDG/subdirs
    #Note - create user set of subdirs in 2 places, above for file test
    #     - and here after 'rm -fr $JTMP/*' for non-restart
    rm -fr $JTMP/*;
    test -d $JGDG || mkdir $JGDG
    test -d $GTMP || mkdir $GTMP
    test -d $JTMP/tmp || mkdir $JTMP/tmp
 fi
 #------------------------------------------------------------------------
 #Feb06/12 - add 'stop step' feature
 if [[ -n "$stop" ]]; then
    if [[ $stop != S[0-9][0-9][0-9][0-9] ]]; then
       logmsg2 "jobset51 ERR: stop=$stop invalid"; exit 95; fi
    exec 4>jobctl/$jobid2.ctl
    print -u4 "$jobid2.ksh start=$start stop=$stop $(date +%y%m%d_%H%M%S)"
    exec 4>&-;
    logmsg2 "WARN: **STOP** at: stop=$stop";
 fi
 #
 # Default RUNDATE to system date yyyymmdd, but allow cmd line override
 # --> jobname.ksh RUNDATE=20091120  <-- override system date for this job
 if [[ -z "$RUNDATE" ]]; then RUNDATE=$(date +%Y%m%d); fi; export RUNDATE
 export RUNDTTM=$(date +%y%m%d_%H%M%S)
 export YMD=$(date +%y%m%d)
 #------------------------------------------------------------------------
 # Display various information at begining of JCL/script execution
 # - for the console log, info may be useful to debug problems
 logmsg2 "Begin Job=$jobid2"
 logmsg1 "$scriptpath"
 logmsg1 "Arguments: $args"
 logmsg1 "ProcessID=$$"
 logmsg1 "RUNDATE=$RUNDATE"
 logmsg1 "RUNLIBS=$RUNLIBS"
 logmsg1 "RUNDATA=$RUNDATA"
 logmsg1 "JTMP=$JTMP"
 logmsg1 "SYOT=$SYOT"
 logmsg1 "LOGNAME=$LOGNAME TESTPROD=$TESTPROD"
 logmsg1 "HOSTNAME=$HOSTNAME"
 ## logmsg1 "OBSLOGDIR=$OBSLOGDIR"
 uvtime W1D0 jobbgn    # get start time for later elapsed time calc
 #Jan26/12 - arg2 filename disabled, no dummy
 return 0
 }
 #----------------------------- end jobset51 -----------------------------

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

11G1. JCL conversion - GDG functions

jobend51 - support new GDG features (May 2007+)

 # jobend51 - common function called at Normal EOJ of JCL/scripts
 #          - by Owen Townsend, UV Software, May 23/2007
 #          - this KORN shell function stored in $APPSADM/sfun
 #          - see more doc at: www.uvsoftware.ca/jclcnv4gdg.htm
 #
 #Apr20/15 - add code to test for GDG#s > threshold & resequence
 #Nov10/14 - created from jobend53 IMMD directory structure
 #         - restoring Vancouver Utility directory structure
 #         - retaining support for new exportgen0/exportgen1 functions
 # exportgen1 enhanced to allow for IMMD new dir structure
 # - writes newgdgs to $JGDG (jobtmp/$jobid2/GDG) as following sample:
 #   $JGDG/@home@mvstest@testdata@data1@gl.account.master_000004
 # - full path name with '/'s replaced by '@'s
 # - can now retore '@'s to '/'s & move back to intended dir
 #
 function jobend51
 {
 JTMP=$RUNDATA/jobtmp/$jobid2
 JGDG=$JTMP/GDG
 # move any GDG files in jobtmp/GDG/... back to intended dirs
 # lsgdgs=$(ls $JGDG/* 2>/dev/null);  # capture filenames in GDG/...
 # if [[ -n "$lsgdgs" ]]; then
 ls $JGDG >$JTMP/gdgfiles
 if [[ -s $JTMP/gdgfiles ]]; then
    logmsg2 "moving $JGDG/files back to intended directories"
    for jgdgfpa in $JGDG/*
      { fpa=$(basename $jgdgfpa)
        fpd=$(echo $fpa | tr '@' '/')
        echo $fpd              # show filename on log
        mv -i $JGDG/$fpa $fpd  # move current file
        #====================
      }
 fi
 # call job to update the GDG control file for any prior gen# processing
 # - as indicated by exportgen0 functions within this job
 # - resets next gen# 000000 if indicated & expiry date:time past
 # - increments next gen# if processing range low to high & resets at end
 #Jun11/12 - make optional depending on env-var $GDGCTLUPDT
 if [[ "$GDGCTLUPDT" == "NO" ]]; then :
 else uvcopy gdgupok1
      #==============
 fi
 #Apr20/15 - add code to test for GDG#s > threshold & resequence
 #  $JGDG/@home@mvstest@testdata@data1@gl.account.master_000004
 typeset -RZ6 fpfg         # 6 digits Right justified Zero filled
 exec 3<$JTMP/gdgfiles     # open file of GDG filenames
 while read -u3 fpaf extra
   do fpdf=$(echo $fpaf | tr '@' '/')
      fpd=${fpdf%/*}          # drop filename to get directories
      fpf=$(basename $fpdf)   # drop directories to get filename
      fpfb=${fpf%_*}_         # drop _gdg# & restore trailing '_'
      fpfg=${fpf##*_}         # drop filename_ to get gdg#
      if [[ $fpfg -gt 900000 ]]; then
         gdgreset1 $fpd $fpfb 900000 100001
         #=================================
      fi
   done
 #
 # disable any jobctl file for job ending (change name to allow inspection)
 test -f jobctl/$jobid2.ctl && mv jobctl/$jobid2.ctl jobctl/$jobid2.ctl_old
 #
 #Jan15/12 - call uvtime option D2 to report elapsed time of last step
 uvtime W1D2 steptimes
 uvtime W1D4 jobtimes NormalEnd
 return 0
 }
 #------------------------------ end jobend51 ----------------------------

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

11H1. JCL conversion - GDG functions

jobabend51 - support new GDG features (May 2007+)

 # jobabend51 - function called at Abnormal Termination of JCL/scripts
 #            - by Owen Townsend, UV Software, June 18/2007
 #            - ksh function stored in $APPSADM/sfun
 #            - see doc at: www.uvsoftware.ca/jclcnv4gdg.htm
 # This function called at Abnormal Termination, to report new GDGs
 # NOT moved back from jobtmp/... to $RUNDATA/dir/...
 # also updates the gdgctl file, inserting an AB warning in history entries
 #
 #Nov10/14 - created from jobabend53 IMMD directory structure
 #         - restoring Vancouver Utility directory structure
 #         - retaining support for new exportgen0/exportgen1 functions
 # exportgen1 enhanced to allow for IMMD new dir structure
 # - writes newgdgs to $JGDG (jobtmp/$jobid2/GDG) as following sample:
 #   $JGDG/@immd@tidw@data@gg@pidw.prd.gg.cmperib_000003
 # - full path name with '/'s replaced by '@'s
 # - jobend51 restores '@'s to '/'s & move back to intended dir
 # - jobabend51 reports any GDG files NOT moved back to intended dir
 #
 function jobabend51
 {
 JTMP=$RUNDATA/jobtmp/$jobid2
 JGDG=$JTMP/GDG
 #
 # note if any GDG files in jobtmp/...
 lsgdgs=$(ls $JGDG/* 2>/dev/null);  # capture filenames in GDG/...
 if [[ -n "$lsgdgs" ]]; then
    logmsg2 "jobabend51 ERR: GDG files NOT moved from jobtmp/... to intended dirs" RV
    logmsg2 "- can rerun OK, since GDGs not added to RUNDATA/dirs"
    logmsg2 "- OR restart at abterm step & reply y to move prompt"
 fi
 # report any GDG files in jobtmp/... NOT moved back to intended dirs
 # - also call uvcopy job to update gdgctl file with err entry
 lsjgdg=$(ls $JGDG)
 if [[ -n "$lsjgdg" ]]; then
    logmsg2 "jobabend51 ERR: $JGDG/files NOT moved back & listed below:"
    echo "$lsjgdg"          # list files not moved back
    # update control file - optional depending on env-var $GDGCTLUPDT
    # if [[ "$GDGCTLUPDT" == "NO" ]]; then
    #   uvcopy gdgupab1,fild1=$psd,arg1=$sd  # also lists files
    # fi
    #Oct18/14 - gdgupab1 needs work, can do later, since inhibited for IMMD
 fi
 # ensure operator acknowledges GDG files not moved back
 if [[ -n "$lsgdgs" ]]; then
    logmsg2 "- reply y/n acknowledge GDG msg, but no auto action" RV ACK
 fi
 # disable any jobctl file for job ending (change name to allow insection)
 test -f jobctl/$jobid2.ctl && mv jobctl/$jobid2.ctl jobctl/$jobid2.ctl_old
 uvtime W1D2 steptimes
 uvtime W1D4 jobtimes ***AbEnd
 return 0
 }
 #---------------------------- end jobabend51 --------------------------

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

11A1. JCL conversion - GDG functions

 #!/bin/ksh
 # gdgreset1 - script to resequence gdg#s for 1 file at a time, by Owen Townsend, April 2015
 #           - /home/uvadm/sf/IBM/gdgreset1
 #           - script called by function 'jobend51' (at end all JCL/scripts)
 #           - resequences gdg#s for 'ONE' file by mv/rename within directory
 #
 # gdgreset2 - also see to reset gdg#s for 'ALL' files in directory with GDG# > threshold
 #           - /home/uvadm/pf/IBM/gdgreset2
 #           - resequences gdg#s for all files with gdg# > threshold
 #             by copy/rename to alternate directory for inspection before activation
 #
 # gdgreset1 directory filename_ gdgmax gdgreset     <-- command format
 # =============================================
 #
 # gdgreset1 data1 gl.account.trans_ 900000 100001   <-- command example
 # ===============================================
 #
 # VU GDG files are identified in JCL/scripts by a '_' trailing underscore
 # GDG filenames in $RUNDATA/data1/... have 6 digits following the '_'
 # The number of generations maintained is controlled by gdg=... in control file
 # in $GDGCTL/gdgctl51I.dat/.idx (Indexed file loaded from $GDGCTL/gdgctl51 text file)
 #
 # JCL/scripts call function 'exportgen1' to determine next gen# to create
 # exportgen1 deletes oldest gdg#s depending on gdg=___ in $GDGCTL/gdgctl51I.dat/.idx
 # - for example if gdg=3, adding a 4th file will delete the 1st file
 #
 # gl.account.master_000001
 # gl.account.master_000002       <-- sample GDG file with 3 generations
 # gl.account.master_000003
 #
 # gl.account.master_999999       <-- max gen#, need to reset before reached
 #
 # - would take 3000 years to reach max gen# if 1 new generation per day
 # - would take 3 years if you create 1000 new generations/day
 # - generation#s will be automatically reset at 900000 by jobend51/gdgreset1
 #
 # New generations are created in jobtmp/GDG/... & moved back to data1/...
 # at successful End-Of-Job by function 'jobend51'
 # 'jobend51' also tests for GDG#s > 900000 & calls script 'gdgreset1'
 # to re-sequence gdg#s > 900000 back to 100001, 100002, etc
 #
 # gl.account.master_900001       <-- gdg file with 3 gens > 900000
 # gl.account.master_900002
 # gl.account.master_900003
 #
 # gl.account.master_100001       <-- gdg#s re-sequenced from 100001
 # gl.account.master_100002
 # gl.account.master_100003
 #

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

 # 'gdgreset1' called by 'jobend51' at EOJ will reset gdg#s for all members
 #  in 'ONE' file when the new member gdg# is > 900000
 #
 # Also see 'gdgreset2' to reset gdg#s for 'ALL' files in directory with GDG# > threshold
 # - a uvcopy job in /home/uvadm/pf/IBM/gdgreset2
 #
 # 'gdgreset2' creates a 'script' to copy/rename GDG files with gdg#s > specified threshold
 #  - the script copies all files to a 2nd directory (resequencing if gdg# > threshold)
 #  - you can inspect before renaming the input as data1.old & the output dir back to data1
 #  - you would run gdgreset2 in a controlled manner when no jobs are running
 #
 echo "gdgreset1 - script to resequence gdg#s for 1 file if gdg# > threshold"
 typeset -RZ6 gdgm gdgr gn   # GDG#s 6 digits right justified zero filled
 fpd="$1"; fpfb="$2"; gdgm="$3"; gdgr="$4";
 if [[ -d $fpd && $# -eq 4 ]]; then :
 else echo "usage: gdgreset1 directory file gdgmax gdgreset"
      echo "       ========================================"
      echo "example: gdgreset1 data1 gl.account.trans_ 900000 100001"
      echo "         ==============================================="
      echo " - arg1 must be a directory, arg2 a filename with trailing underscore"
      echo " - arg3 must be threshold for reset"
      echo " - arg4 must be the start# for re-sequencing"
      exit 99; fi
 #
 test -d tmp || mkdir tmp
 integer count=0
 ls $fpd/$fpfb* >tmp/gdgfiles    # create file of filenames in GDG group
 exec 4< tmp/gdgfiles            # open file of filenames
 while read -u4 dfngn extra      # read current member of group
   do fngn=${dfngn##*/}          # drop tmp/
      fn=${fngn%_*}              # drop gdg# to get filename base
      gn=${fngn##_*}             # drop filename base to get gdg#
      # echo "DEBUG1: dfngn=$dfngn fn=$fn gn=$gn"
      if [[ $gn -lt $gdgm ]]; then return 9; fi
      fngn2=${fn}_$gdgr          # assign new sequence#
      mv $fpd/$fngn $fpd/$fngn2  # rename file with new sequence#
      #========================
      echo "gdgreset1 $fpd/$fngn ---> $fngn2"
      ((gdgr+=1))                # increment seq# for next
      ((count+=1))               # count files re-seqeunced
   done
 exec 4<&-   # close file
 echo "gdgreset1 - $count GDG files resequenced"
 return 0

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Part_12 JCL conversion - Generation files

uvcopy jobs used by GDG functions

 12A1< jclgdgctl51   - create ctl/gdgctl51 file from all JCL/scripts
                      in directory jcls (converted to Korn shell scripts)

12C1< gdgunload51 - convert indexed file back to text file for editing/reload

 12D1< gdgget51   - used by exportgen1 to get max generations from the
                   indexed control file (ctl/gdgctl51I)

 12E1< gdgget52      - used by exportgen0 to determine next generation#
                    - checks gdgctl51I for over-ride to prior generation

 12F1< gdgupok1   - update gdgctl file at Normal EOJ
                    - called by jobend51 function

 12G1< gdgupab1   - update gdgctl file at Normal EOJ
                    - called by jobabend51 function

 12H1< gdgreset2  - reset generation#s for all GDG files with generation#s
                    exceeding 900000

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

12B1. JCL conversion - uvcopy jobs called by GDG functions

gdgload51 - load indexed file ctl/gdgctl51I

 # gdgload51 - uvcopy job from UVSI stored in: /home/uvadm/pf/IBM/
 # gdgload51 - load ctl/gdgctl51 to indexed file ctl/gdgctl51I
 #          - by Owen Townsend, UV Software, May 31/2007
 #          - see doc at www.uvsoftware.ca/mvsjcl.htm#Part5
 #             or www.uvsoftware.ca/jclcnv1demo.htm#5G1
 #
 #Feb06/17 - store only filename 0(50), gdg= 51(6), opt= 58(5),& next= 64(40),
 #Oct17/14 - see uvcopy jclgdgctl51 create input for IMMD new dirs
 #Mar14/12 - define gdg file using $GDGCTL (default in profile $RUNDATA/ctl)
 #
 #           ** Op. Instrns. for VU JCL/scripts **
 #
 # Assuming - JCL/conversion completed for user 'mvstest'
 #          - JCLs converted to /home/mvstest/testlibs/jcl3/...
 #          - using jcl3 (all JCL/scripts)
 #          - since will copy jcl3/... to jcls/... 1 at a time for debug/test
 #
 # 1. Login mvstest --> /home/mvstest
 #
 # 2. cdl --> /home/mvstest/testlibs
 #
 # 3. uvcopy jclgdgctl51,fild1=jcl3,filo1=ctl/gdgctl51
 #    ================================================
 #    - prior job to create initital gdgctl51 (text file)
 #
 # 4. cp ctl/gdgctl51 $GDGCTL/ctl/    <-- copy to GDG control directory
 #    ============================
 #  NOTE - GDGCTL defined in profile, either $RUNDATA or $APPSADM:
 #
 # 4a. cp ctl/gdgctl51 $RUNDATA/ctl/  <-- $RUNDATA for programmer testing
 #     =============================    - separate sets of testdata files
 #
 # 4b. cp ctl/gdgctl51 $APPSADM/ctl/  <-- $APPSADM better for production
 #     =============================    - 1 control file for all GDG files
 #                                      - common set of production data files
 #
 # 5. cd $GDGCTL        <-- change to GDG ctlfile directory
 #    ==========
 #
 # 6. vi ctl/gdgctl51   <-- edit before loading Indexed file
 #    ===============     - modify no of generations (see sample below)
 #
 #*7. uvcopy gdgload51,fili1=ctl/gdgctl51,filo1=ctl/gdgctl51I
 #    =======================================================
 #    - load Indexed file for use by exportgen1, exportgen0,& jobend51
 #
 #Note - $GDGCTL in profile defines the 'ctl' control file directory
 #

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

 # The gdgctl file specifies the no of generations to be maintained by the
 # exportgen1 function which writes new generations & deletes oldest generations
 # The gdgctl file also provides prior generation exception processing
 # in conjunction with the exportgen0 function (see next=... examples below)
 #
 #                 ** sample input text file ctl/gdgctl51 **
 #
 # gl/account.acntlist_ gdg=08
 # gl/account.master_   gdg=10 next=000002(000002:000002)000000:000000
 # gl/account.trans_    gdg=20 opt=c next=000001(000001:000012)070531:235959
 # py/payroll.master_   gdg=30
 #
 # jclgdgctl51 creates only filenames & gdg=... with a default no of generations
 # You would then edit the text file modifying generations as desired
 #
 # You would probably not setup any next=... at this time, but it is possible
 #     next=gen###(lowgen:higen#)yymmdd:HHMMSS
 #     next=000000(000000:000000)000000:000000
 # - will be set all zeros by gdgload51 if not coded on input
 # - see field explanations & examples at www.uvsoftware.ca/mvsjcl.htm#Part_5
 #
 #               ** control file record format ctl/gdgctl51I **
 #
 # 000:049 - filename example--> gl/account.master_ <--trailing underscore
 # 051:056 - gdg=..   - generations for this file (default set in jclgdgctl51)
 # 058:062 - opt=c    - option for oprtr confirm y or enter alternate gen#
 # 064:102 - next=... any input verified & adjusted to bytes 064-103
 #           next=gen###(lowgen:higen#)yymmdd:HHMMSS
 #           next=000000(000000:000000)000000:000000
 #           zeros template/place-holder generated if not coded
 # 104:113 - jobname of last JCL/script to access this file
 #         - signal to jobend function to reset next=... as appropriate
 #           (reset next=000000 &/or increment next=... from low to high)
 # 114:126 - date/time yymmdd:HHMMSS file created by exportgen1
 #         - used by gdgupok1 to '+' flag history entry last exportgen1
 #
 # 128:191 - 1st entry of multiple action history entries
 #         - yymmdd:HHMMSS:g0_123456(123456:123456)JOBNAME  S1234 PROGRAM +
 #         - stored by exportgen0, exportgen1, jobend51?, jobend52?
 #         - action entries shifted down & latest action stored here
 # 190:190 - gdgupok1 '+' flag if moved back from jobtmp/subdir (exportgen1)
 #
 # 192:255 - update history#2 (shifted from 128-191 by jobend51)
 # 256:319 - update history#3 (shifted from 192-255)
 # 320:1983-  --- entries #4-#29 ---
 #1984:2045- update history#30
 #2046:2046- '.' period marking end of record data
 #2047:2047- x'0A' Indexed record status byte
 #         - see explanations & examples at www.uvsoftware.ca/mvsjcl.htm#Part_5
 #

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

 opr='load gdgctl text file to Indexed file, shifting keywords over for maxkey'
 was=c5000
 fili1=?${GDGCTL}/gdgctl51,rcs=256,typ=LST
 filo1=?${GDGCTL}/gdgctl51I,rcs=2047,typ=ISF,isk1=0(50)
 #
 # load zeros entries for next gen# & exportgen0 update/history entries
 #         1         2         3         4         5         6         7
 #1234567890123456789012345678901234567890123456789012345678901234567890
 lodv3=k0(100)
 next=gen###(lowgen:higen#)yymmdd:HHMMSS               <-- next gen# override
 next=000000(000000:000000)000000:000000
 next=######(######:######)######:######               <-- verify user coding
 yymmdd:HHMMSS:G1_123456(123456:123456)jobname  S1234 programid  <-- history
 000000:000000:___000000(000000:000000)_______  _____ _________
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 #1234567890123456789012345678901234567890123456789012345678901234567890
 #         1         2         3         4         5         6         7
 @run
        opn    all                    open I/O files
        sxo    128,'0(50)'            open the sort
 #

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

 # begin loop to get records, reformat,& put to the sort
 # - if codes on right (gdg=..., etc) prior to col 51
 #   shift over to allow 50 bytes for the filename key
 man20  get    fili1,a0(200)          get ctlfile record
        skp>   man60
        cmc    a0(1),'#'              comment line ?
        skp=   man20                  yes - bypass, get next
        clr    b0(200),' '            clear reformat area
        mvu    b0(80),a0,' '          move filename until ending blank
 #
 #Feb06/17 - store only filename 0(50), gdg= 51(6), opt= 58(5),& next= 64(40),
 man24  scn    a0(100),' gdg='
        skp!   man30
        mvc    b51(6),ax1             store gdg=...
 #
 man30  scn    a0(100),' opt='
        skp!   man40
        mvc    b58(5),ax1             store gdg=...
 #
 # test for next=... coded on input, ifso verify & shift to byte 64
 # - else insert zeros entry at byte 64
 man40  scn    a0(80),' next='
        skp!   man46
        mvc    n0(40),ax1             isolate user's next=... entry
        cmcp   n0(40),k200(40)        verify users coding with table pattern
        skp!   err1
 man44  mvc    b64(40),n0             restore user entry to byte 64+
        skp    man50
 #
 # - user next=... entry not coded, insert zeros entry at byte 64
 man46  mvc    b64(40),k100           store zeros entry at byte 64+
 #
 # common point to put record to the sort
 man50  sxp    b0(128)                put to the sort
        skp    man20                  repeat loop get/put to sort
 #

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

 # end of input - now execute the sort
 man60  sxs                           execute the sort
 #
 # begin loop to get sorted records & load to Indexed file
 man62  sxg    c0(128)                get record from the sort
        skp>   man90
 #
 # create zeros entries for exportgen0,1,jobend update/history slots
 # - create zero entry only for 1st or for all by move overlapping ??
 man64  mvc    c128(64),k400          create zeros action history entrie#1
        mvc    c192(1920),c128        move overlap to create all 30 entries
        mvc    c2046(1),'>'           mark last data byte of record
        put    filo1,c0(2047)         write record to output file
        skp    man62                  repeat loop get/put sorted recs
 #
 # end of sorted records - close files & eoj
 man90  cls    all
        eoj
 #--------------------------------------------------------------------
 # Err rtns
 err1   msg    n0(40)                show users next=... entry
        msg    k0(40)                show next=... valid format
        msgw   'ERR: next=... format invalid, should fix & reload'
        mvc    b123(4),'err1'        flag record in error
        skp    man44
 #

Goto:   Begin this doc ,  End this doc ,  Index this doc ,  Contents this library ,  UVSI Home-Page

Visitor Counters for ThisYear and LastYear