VSEDATA.doc - Converting Mainframe DATA to Linux [libcnv]

VSEDATA Contents

- Introduction & Overview

Part_1	- Profiles,& Directory Setup required
	- Generating uvcopy jobs to convert All or 1 data file(s)
	- Executing uvcopy jobs to convert All or 1 data file(s)
	- convert varlth mainframe data to fixlth for conversion process
	- translate EBCDIC to ASCII but preserve packed/binary
	- convert fixlth back to varlth while copying to testdata directory
	- Retransfer, Regenerate, Reconvert periodically during conversion
	- using 'gencnv4A' script to regenerate all uvcopy jobs for All datafiles
	- using 'gencnv41' script to regenerate 1 uvcopy job for 1 datafile

Part_2	- generating data conversion jobs for files with FIXED length records
	- generating uvcopy data conversion jobs from copybooks
	- executing the generated uvcopy jobs for All files or 1 file
	- modifications required for files with multiple record types

Part_3	- generation data conversion jobs for files with VARIABLE length records
	- for variable length files (IDXFORMAT8) with multiple record types
	- this file had 100+ record types described by 100+ copybooks
	- you must edit in the auto generated instructions from copybooks
	& add Record-Type test coding to branch to the corresponding instrns

Part_4	- Preparing scripts to copy converted data over to testdata dirs
	- uxcp/uxsort scripts to copy/sort/load indexed variable lth files
	- converting from the fixed length used for conversion
	back to the variable length format expected by COBOL programs

Part_5	- Special purpose uvcopy jobs (R/T statistics & selections)
	- use these as samples/templates to create your jobs for other files
	stat_e2121656.itaxe.facturat - Record Type stats assist R/T coding
	sel1_e2121656.itaxe.facturat - select 1st record of each R/T
	- could run for EBCDIC & ASCII files to verify R/T conversions

Part_6	- JCL/script conversions assisted by Data file conversions
	- LISTCAT report supplies data file characteristics to JCL converter
	(record-sizes, key locations, file types, etc)
	- JCL conversion procedures summarized
	- see VSEJCL.htm for illustrated procedures

Part_7	- Testing aids & recommendations
	- recommended procedures for testing JCL/scripts with datafiles
	- vi tutorial, unix tips for new users, aliases for easy navigation,
	- providing separate TESTDATA dirs for each programmer (optional)
	- joblog script to capture console outputs
	- llr script to display files in multi-level directories
	- alldiff2 script to run unix 'diff' on all file pairs in 2 subdirs

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

Converting Mainframe DATA to unix/linux/windows

VSEDATA Contents (continued)

Part_8	- control files to guide data conversions & COBOL compiles
	- Control files to guide generation of uvcopy jobs
	listcat0 - LISTCAT report from mainframe (multi lines per file)
	listcat2 - 1 line per file extracted info from LISTCAT
	ctlfile3 - listcat2 reformatted with cpy=___ for copybook insert
	ctlfile4 - after copybooknames inserted & unwanted lines dropped
	- Control files for Micro Focus COBOL (supplied in /home/uvadm/ctl/...)
	cobdirectives - Micro Focus COBOL directives
	extfh.cfg - Micro Focus COBOL file handler configuration

Part_9	- Summary of Vancouver Utilities scripts & jobs for data file conversion
	- VU scripts can be found in /home/uvadm/sf/IBM/... ('*' listed here)
	- *cnvdatadirs, vsedatadirs, vselibsdirs
	- *cnvdatainit, testdatainit, vsetestlibsinit
	- gencnv4A, gencnv41,
	- VU uvcopy jobs can be found in /home/uvadm/pf/IBM/... ('*' listed here)
	- *varstat1, catgut41, ctlfile41, ctlfile42, ctlfile43
	- cobmap1, uvdata41, uvdata42, uvdata43, uvdata44 or uvdata46
	(uvdata44 for variable length, uvdata46 for fixed length)

City of Lynn Valley

The procedures documented here are based on the conversion performed at the City of Lynn Valley in 2007. Examples of their JCL & datafilenames are used here with their permission.

VSEDATA vs MVSDATA vs DATAcnv1

In spring 2007, VSEDATA was completely rewritten based on the conversions at City of Lynn Valley.

VSEDATA will now be printed for MVS customers, since it is a much better writeup of data conversion, and since there is not much difference between MVS & VSE data conversions (unlike JCL conversions).

Note that you should use DATAcnv1.htm for your 1st few data file conversions, since the DATAcnv1 procedures are much simpler than VSEDATA procedures.

You will need the VSEDATA conversion procedures (or modified version of), for major conversion projects where you need the automation provided by the control files used in VSEDATA.

The VSEDATA procedures allow us to reconvert all data files with 1 command. You may need to retransfer & reconvert data files several times during a major conversion & certainly on the 'go live' week end.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

Part_1 Converting Mainframe DATA to unix/linux/windows

Generating & Executing Data Conversion jobs - Overview

1A1.	Introduction & Overview
	- Profiles for Testing & Production
1A2.	Directories for Conversion & Testing
	- Environmental variables for location independence
	- $CNVDATA, $TESTDATA, $TESTLIBS, etc
	- aliases for easy navigation (cdc, cdd, cdl)
1A3.	Filename Conversions
	- translate to lower case & drop inapplicable suffixes
1A4.	Master & Work files
	- jobset41 function inserted at begin JCL/scripts
	- makes JCL/scripts independent of location

1B1.	Data Directories for Conversion & Testing
1B2.	Libraries - for Testing

1C1.	Retransfer, Regenerate,& Reconvert periodically during conversion period
	CNVDATA subdirs to be replaced, maintained, or cleared for regens
1C2.	TESTDATA subdirs to be replaced or cleared for retest after regen
1C3.	TESTLIBS subdirs to be replaced, maintained or cleared for retest

1D1.	Pre-Requisites for DATA file conversion
	- Setups & File Transfer from Mainframe
1D2.	Data file conversion - Overview
	- provision for files without 2 byte prefix
1D3.	Generating Data conversion jobs - Overview

1E1.	Preparation to generate data conversion jobs
	- rename files to lowercase & remove inapplicable suffixes
	- create statistics for variable length files (optional)
1E2.	Create Data conversion control file from LISTCAT
1E3.	Editing ctlfile with copybook names
1E4.	Re-Creating ctlfile & recovering copybook names from old ctlfile

1F0.	Generate jobs to convert ALL datafiles (3 sets in pfv1,2, pfx1,2,3, pfy1,2)
1F1.	Generate jobs to convert All files from varlth to fixlth (simplify convert)
	- variable to fixed jobs are stored in subdir pfv2/... for later execution

1F2.	Generate jobs to convert ALL datafiles EBCDIC to ASCII (pfx1,pfx2,pfx3)
	- preserving packed/binary fields
	- driven by the copybooks & a control file to relate the copybooks
	to the datafilenames & to supply record-sizes & key info
	- EBCDIC to ASCII jobs are left in subdir pfx2 (may need manual changes)
	- manually copy to pfx3 & modify jobs for files with multiple record types
	- EBCDIC to ASCII jobs are stored in subdir pfx3/... for later execution

1F3.	Generate jobs to restore original record sizes (pfy1,pfy2)
	- variable lth records were converted to fixed to facilitate conversion
	- now convert fixed lth records back to variable, using original recsizes
	saved in extra space at end of records
	- fixed to variable jobs are stored in subdir pfy2/... for later execution

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

Part 1 Converting Mainframe DATA to unix/linux/windows

Generating & Executing Data Conversions - Overview (cont)

1G0.	Preparation to Execute jobs to convert All data files
	- Pre-Requisites
	- Recommendations for multi record type files
	- Recommendations if NO multi record type files

1G1.	Executing jobs to convert ALL files varlth to fixlth (simplify conversion)
	Executing jobs to convert ALL files from EBCDIC to ASCII

1G2.	Executing jobs to convert ALL files from fixlth back to varlth (as reqd)
	Copying All converted Data files to test directories

1H1.	Generating job to convert 1 Data file (generic filename)
	- varlth to fixlth, EBCDIC to ASCII, fixlth to varlth
1H2.	Generating job to convert 1 file - example using an actual filename

1I1.	Executing jobs to convert 1 data file
	- varlth to fixlth, EBCDIC to ASCII, fixlth to varlth, copy to test dir

1J1.	using script 'gencnv4A' to Re-Generating All conversion jobs
1J2.	Preparations to use gencnv4A (to regenerate All jobs)
1J3.	running the gencnv4A script to regen All conversion jobs
	- auto gens stop at pfx2, to protect edited jobs in pfx3
	- copy jobs for new files from pfx2/... to pfx3/
	if copybooks updated for some files with multi R/T coding
	- you need to merge new convert instrns with old R/T testing
	- see the example given for gencnv41

1K1.	using script 'gencnv41' to Re-Generate 1 conversion jobs
	- add new/updated jobs from pfx2 to pfx3
1K2.	Recover record type test code if jobs regenerated
1K3.	Executing job (created by gencnv41) to convert 1 datafile

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1A1. Converting Mainframe DATA to unix/linux/windows

Profiles for Testing & Production

The profiles are split into 3 parts as shown below. This makes system administration much easier because the sysadm can change critical definitions in 1 place rather than have to change definitions in the home directory profiles of all programmers.

This system provides for TEST & PRODuction environments without the need to to change any filenames or any JCL/scripts when we move data files, JCL, & COBOL programs from test to production.

Environmental variable '$RUNDATA' will point to the test data superdir for programmers or the production data superdir for operators. Environmental variable '$RUNLIBS' will point to the test libraries (of JCL/scripts & COBOL programs) for programmers or the production libraries for operators.

Different programmers could also have different sets of test data simply by changing the defintiions of RUNDATA in their profiles.

stub_profile

copy to each user homedir & rename .profile or .bash_profile

defines only items that vary from user to user
defines RUNLIBS as testlibs for programmers OR prodlibs for operators
defines RUNDATA as testdata for programmers OR proddata for operators
calls common_profile todefine PATHs based on RUNLIBS/RUNDATA

common_profile

called by 'stub_profile'

defines search PATHs to libraries & data based on $RUNLIBS & $RUNDATA

You can see the complete profiles listed in part 1 of ADMjobs.htm. These profiles are provided with the Vancouver Utility pkg in the 'env' subdir. If installed as recommended they will be located at /home/uvadm/env/...

Note	to maintain the integrity of this system, you must: NEVER code absolute pathnames in the JCL/scripts NO filenames starting from 'root' (with a leading '/')

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1A2. Converting Mainframe DATA to unix/linux/windows

Directories for Conversion & Testing

At City of Lynn Valley we have setup directories for conversion, test data, & libraries as follows:

 /tax/cnvdata   - superdir for data conversion subdirs
                - see subdirs listed on page '1B1'

 /tax/testdata  - superdir for testdata subdirs
                - see subdirs listed on page '1B1'

 /tax/testlibs  - superdir for test libraries (JCL,COBOL,etc)
                - see subdirs listed on page '1B2'

To make it easy to change directory locations, we will restrict any absolute references to /tax to 1 place, the stub_profiles define as follows:

export CNVDATA=/tax/cnvdata export TESTDATA=/tax/testdata export TESTLIBS=/tax/testlibs

aliases for easy navigation

To make it easy for the users to change from his login homedir to the the desired superdir, we have provided aliases, defined in the common_profile as follows:

alias cdc='cd $CNVDATA' alias cdd='cd $RUNDATA' alias cdl='cd $RUNLIBS'

$symbols used in documentation

To make this documentation more applicable despite future changes to your directory assignments, we will use the above symbols as much as possible. ($CNVDATA, $TESTDATA, $TESTLIBS - not /tax/cnvdata, /tax/testdata,/tax/testlibs)

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1A3. Converting Mainframe DATA to unix/linux/windows

Filename Conversions

Here are some examples of filenames used on the mainframe & the filenames that will be used on the Linux system.

E2121653.MTAXE.PISCINES.CLUS.VSEPRD <-- filenames on MAINFRAME E2121656.ITAXE.FACTURAT.CLUS.VSEPRD E2122683.ITAXE.DEGREVEM.CLUS.VSEPRD E2123003.ITAXE.TRANSDAM TU.F01.E212-NO.TRPERM

 e2121653.mtaxe.piscines                <-- filenames on linux
 e2121656.itaxe.facturat
 e2122683.itaxe.degrevem
 e2123003.itaxe.transdam
 tu.f01.e212-no.trperm

Filenames on linux will be translated to lower case & suffixes such as 'CLUS.VSEPRD' will be dropped, since they do not apply to unix/linux.

TEST/PRODuction indicators such as PRD/DEV are not required because the Vancouver Utility conversions provide for TEST & PRODuction systems using environmental variables in the profiles of programmers & operators.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1A4. Converting Mainframe DATA to unix/linux/windows

Master & Work files

'Master files' are permanent files that must be present before JCL/scripts are run. 'Work files' are temporary inter-step or inter-job files that can be removed at the end of the job or end of a short cycle (daily, weekly).

We can distinguish between master & work files by running LISTCAT on the mainframe between cycles when we know there is a minimum of work files on disc. We will transfer the LISTCAT report to linux & convert it into a control file which will automatically cause datafiles to be assigned to the 'mstr' or 'wrk' subdir. For example here are a few files from a sample JCL/script.

    exportfile E212002 $MSTR/e2123002.itaxe.taxation
    exportfile E212504 $MSTR/e2125504.mtaxe.paracomp
    exportfile E212556 $WRK/e2125556.ttaxe.b8401j04
    exportfile E212557 $WRK/e2125557.ttaxe.b8401j04
    exportfile E212558 $WRK/e2125558.ttaxe.b8401j04

All converted JCL/scripts call a common function at the begining which sets up several environmental variables. You can see the 'jobset41' function listed on page 3C1 of the JCL conversion doc ('VSEJCL.doc#3C1'), but here are just a few relevant lines:

 function jobset41 {
 cd $RUNDATA                #change to working dir for production (or test)
                            # - depending on RUNDATA def in prgmr/oprtr profiles
 RLX=$RUNLIBS/cblx          #setup path for loading programs
 MSTR=mstr                  #master files
 WRK=wrk                    #interstep temp/work files or see $JTMP
 JTMP=jobtmp/${JOBID2}      #some temp files stored here
 RPTS=rpts/$(date +%y%m%d)  #subdir for today's printer files
 TAPE=tape/$(date +%y%m%d)  #subdir for today's tape files
 }

Having jobset41 change to $RUNDATA has several advantages:

The JCL/scripts are independent of location. They need no changes to run in TEST or PRODuction locations.
The filename definitions are shorter since they omit the front part of the path defined by $RUNDATA.
You can run the JCL/scripts from anywhere. Unix/linux finds the JCL/script by searching the PATH defined in your profile (which uses $RUNLIBS to define the location of the JCL/scripts).
Once the JCL/script is found, the jobset41 function changes to the data file superdir (which is defined by $RUNDATA in your profile).

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1B1. Converting Mainframe DATA to unix/linux/windows

Data Conversion Directories

 /tax                     - superdir for cnvdata, testdata,& testlibs
 :-----cnvdata            - $CNVDATA=/tax/cnvdata ($symbol for PATH independence)
 :     :-----archive      - copy unwanted files here for later deletion
 :     :-----cpys         - COBOL copybooks
 :     :-----ctl          - control files (to guide conversions)
 :     :-----d0           - data files FTP'd from mainframe (variable length)
 :     :-----d0ebc        - filenames lower case, '.clus.vseprd' dropped
 :     :-----d1ebc        - converted to fixed length for conversion
 :     :-----d2asc        - converted to ASCII (packed/binary preserved)
 :     :-----d3ebc        - possible convert back to EBCDIC to verify
 :     :-----errs         - error reports from conversion utilities
 :     :-----maps         - copybooks converted to record layouts
 :     :-----pf           - misc uvcopy jobs
 :     :-----pfv1         - uvcopy jobs to convert varlth to fixlth
 :     :-----pfv2         - copied here for execution
 :     :-----pfx1         - uvcopy skeleton jobs generated from 'maps'
 :     :-----pfx1a        - uvcopy instrns extracted from pfx1 jobs
 :     :-----pfx2         - uvcopy jobs completed, datafilenames (vs cpybknms)
 :     :-----pfx3         - uvcopy jobs copied here for modify/execute
 :     :-----pfx3.bak     - backup before regens to recover modified R/T code
 :     :-----pfx4         - optional jobs to convert back to EBCDIC for verify
 :     :-----pfy1         - uvcopy jobs to restore original rec-sizes
 :     :-----pfy2         - generated in pfy1, copied to pfy2 for execution
 :     :-----sf           - miscellaneous scripts
 :     :-----sfux         - sample scripts using uxcp/uxsort to copy/sort IDXf8v
 :     :-----stats        - reports to assist multi record type file conversions
 :     :-----tmp          - misc files here (keep working dir clean, rm daily)
 :     :-----tmp1         - misc files (remove weekly vs daily)

Note	use script 'cnvdatadirs' to make these directories see /home/uvadm/sf/IBM/cnvdatadirs or listing on page '9A1'

DATA Directories - for Testing

 /tax                     - superdir for cnvdata, testdata,& testlibs
 :-----testdata           - $TESTDATA=/tax/testdata ($symbol PATH independence)
 :     :-----joblog       - captures console logs from 'joblog1' script
 :     :-----jobparms1    - SYSIN data cards from $TESTLIBS/jobparm1 current
 :     :-----jobparms2    - SYSIN data cards from $TESTLIBS/jobparm2 no dates
 :     :-----jobtmp       - temp datafiles, subdirs for each jobname
 :     :-----mstr         - datafiles copied from $CNVDATA/d2asc for testing
 :     :-----rpts         - reports created by JCL/scripts
 :     :-----sysout       - optional reports from COBOL DISPLAYs
 :     :-----tape         - mainframe tape files redirected to disc subdir
 :     :-----tmp          - misc files, sort work files, etc
 :     :-----wrk          - JCL/script data files not identified as 'mstr'

Note	use script 'vsedatadirs' to make these directories see /home/uvadm/sf/IBM/vsedatadirs

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1B2. Converting Mainframe DATA to unix/linux/windows

Libraries - for Testing

 /tax                     - superdir for cnvdata, testdata,& testlibs
 :-----testlibs           - $TESTLIBS=/tax/testdata ($symbol PATH independence)
 :     :-----cbl0         - COBOL source programs FTP'd from mainframe
 :     :-----cbl1         - after cleanup (clear 1-6,73-80)
 :     :-----cbl2         - after base conversion
 :     :-----cbl3         - after optional convert ACCEPT's to READ from file
 :     :-----cbls         - fully converted COBOL source programs
 :     :-----cblst        - COBOL compiler listings
 :     :-----cblx         - compiled COBOL programs for execution
 :     :-----cpy0         - COBOL copybooks FTP'd from mainframe
 :     :-----cpy1         - after cleanup (clear 1-6,73-80)
 :     :-----cpy2         - after base COBOL conversion
 :     :-----cpys         - fully converted COBOL copybooks
 :     :-----ctl          - control files to guide JCL/COBOL conversions
 :     :-----errs         - error reports from JCL/COBOL convert utilities
 :     :-----jcl0         - JCL FTP'd from mainframe
 :     :-----jcl1         - after cleanup (73-80 cleared)
 :     :-----jcl2         - after PROC expansion
 :     :-----jcl3         - after JCL/script conversion
 :     :-----jcls         - JCL/scripts debugged (copy 1 at a time for testing)
 :     :-----jcls.bak     - backup before reconvert to recover modified code
 :     :-----joblog       - console log from joblog1 testing script
 :     :-----jobparm1     - datacards extracted from JCL/scripts date-stamped
 :     :-----jobparm2     - datacards extracted from JCL/scripts no date-stamps
 :     :-----maps         - copybooks converted to record layouts
 :     :-----parm0        - SYSIN control cards if sep modules on mainframe
 :     :-----parms        - SYSIN control cards after cleanup (clear 73-80)
 :     :-----pf           - misc uvcopy jobs
 :     :-----proc0        - PROCs FTP'd from mainframe
 :     :-----procs        - PROCs after cleanup (73-80 cleared)
 :     :-----sf           - misc scripts (written as needed)
 :     :-----sfun         - Korn shell functions used by converted JCL/scripts
 :     :-----sli0         - SLI's FTP'd from mainframe
 :     :-----slis         - SLI's after cleanup (73-80 cleared)
 :     :-----tmp          - misc files here (keep working dir clean, rm daily)
 :     :-----xref         - cross-references (JCL & COBOL)

Note	use script 'vselibsdirs' to make these directories see /home/uvadm/sf/IBM/vselibsdirs

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1C1. Converting Mainframe DATA to unix/linux/windows

Re-Transfer, Re-Generate,& Re-Convert

If there are many updates to the mainframe during the conversion, you will need to periodically retransfer the datafiles & copybooks, regenerate the uvcopy jobs & reconvert the datafiles. (use script 'gencnv4A' to regenerate all uvcopy jobs)

Here is the 'dtree' for the CNVDATA superdir with comments on the right about which subdirs are retransferred, cleared, or remain the same for reconversion. See page '1B1' for content descriptions of these subdirs.

                                 --- indicator meanings ---
                        <--- subdirs retransferred from mainframe
                          0  subdirs that are cleared for regenerate/reconvert
                         -*- subdirs that remain the same or update as noted
 /tax
 :-----cnvdata
 :     :-----archive
 :     :-----cpys       <--- retransfer copybooks from mainframe (or testlibs)
 :     :-----ctl         -*- ctlfiles may remain the same
                             or need manual updates for new files
                             or recreate if new LISTCAT from mainframe ('1E2')
 :     :-----d0         <--- retransfer datafiles from mainframe
 :     :-----d0ebc        0  clear intermediate subdirs before reconvert
 :     :-----d1ebc        0
 :     :-----d2asc        0
 :     :-----d3ebc        0
 :     :-----errs         0
 :     :-----maps        -?-
 :     :-----pf          -*- misc uvcopy jobs (not for datafile conversions)
 :     :-----pfv1         0  uvcopy jobs (varlth to fixlth) will regen ('1F1')
 :     :-----pfv2         0
 :     :-----pfx1         0  uvcopy jobs (EBCDIC to ASCII) will regen ('1F2')
 :     :-----pfx1a        0
 :     :-----pfx2         0
 :     :-----pfx3        -*- uvcopy jobs with possible edits for multi R/T files
                             copied 1 at a time from pfx2 just before test/debug
                             might merge existing R/T code into new pfx2 layouts
 :     :-----pfx3.bak  <---  make new backup before regen, so we can copy a pfx2
                             job to pfx3 & recover the R/T code from pfx3.bak
 :     :-----pfx4        -*- uvcopy jobs for ASCII to EBCDIC reconvert/verify
 :     :-----pfy1         -  uvcopy jobs to restore original rec-sizes
 :     :-----pfy2         -  generated in pfy1, copied to pfy2 for execution
 :     :-----sf          -*- misc scripts, ad hoc, coded as required
 :     :-----sfux        -*- sample scripts using uxcp/uxsort to copy/sort IDXf8v
 :     :-----stats       -?- could regen if desired
 :     :-----tmp          0
 :     :-----tmp1         0

Note	the 'cnvdatainit' script can be used to clear the '0' subdirs see /home/uvadm/sf/IBM/cnvdatainit or listing on page '9B1'

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1C2. Converting Mainframe DATA to unix/linux/windows

TESTDATA subdirs for retesting after reconvert

 :-----testdata
 :     :-----ftp         -?-
 :     :-----jobctl       0
 :     :-----joblog       0
 :     :-----jobparms1   -*- SYSIN datacards, originally embedded in JCL
                             jobparmx1 utility extracts to $TESTLIBS/jobparm1
                             dates appended to allow updates for future runs
                             jobparms1 utility copies current cards each day
 :     :-----jobparms2   -*- SYSIN datacards from $TESTLIBS/jobparm2 no dates
 :     :-----jobtmp       0
 :     :-----mstr        -*- Master datafiles
 :     :-----rpts         0
 :     :-----sysout       0
 :     :-----tape        -?- Tape files (could clear if only outputs ?)
 :     :-----tmp          0
 :     :-----wrk          0

Note	the 'testdatainit' script can be used to clear the '0' subdirs see script at /home/uvadm/sf/IBM/testdatainit

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1C3. Converting Mainframe DATA to unix/linux/windows

TESTLIBS subdirs for retesting after reconvert

 :-----testlibs
 :     :-----cbl0      <--- COBOL programs retransferred from mainframe
 :     :-----cbl1        0  clear intermediate versions for reconvert
 :     :-----cbl2        0
 :     :-----cbl3        0
 :     :-----cbls       -?- clear only if you 1st backup to cbls.bak
                            If many mainframe updates & few prior unix changes
                            'cp cbl3/* cbls' & recover unix changes from cbls.bak
 :     :-----cbls.bak   -?- recover modified programs if mainframe unchanged
                            recover parts of programs if mainframe changed
 :     :-----cblst       0
 :     :-----cblx        0
 :     :-----cpy0      <--- COBOL copybooks retransferred from mainframe
 :     :-----cpy1        0
 :     :-----cpy2        0
 :     :-----cpys       -?- clear only if you 1st backup to cpys.bak
                            If many mainframe updates & few prior unix changes
                            'cp cpy2/* cpys' & recover unix changes from cpys.bak
 :     :-----cpys.bak   -?- recover copybooks modified on unix (unlikely ?)
 :     :-----ctl        -*- datafiles41,42,43,43I will be recreated/updated
                            during mass JCL/script conversion (others unchanged)
 :     :-----errs        0
 :     :-----jcl0      <--- JCL retransferred from mainframe
 :     :-----jcl1        0  clear intermediate versions for reconvert
 :     :-----jcl2        0
 :     :-----jcl3        0
 :     :-----jcls       -*- you will never clear jcls, usually several changes
                            always copy each job from jcl3 just before test/debug
 :     :-----jcls.bak   -?- backup all jcls here before reconversion
                            If both mainframe changes & prior changes on unix
                            'cp jcl3/xxx.ksh jcls' & edit unix changes from here
 :     :-----joblog      0
 :     :-----jobparm1   -?- SYSIN control cards, originally embedded in JCL
                            jobparmx1 utility extracts to jobparm1
                            dates appended to allow updates for future runs
                            jobparms1 utility copies current cards each day
                            from $TESTLIBS/jobparm1 to $TESTDATA/jobparms
 :     :-----jobparm2   -?- SYSIN control cards with no dates
 :     :-----maps        0
 :     :-----parm0      -?- not used for VSE conversion
 :     :-----parms      -?-
 :     :-----pf         -*- misc uvcopy jobs, ad hoc, code as required
 :     :-----proc0      -?- not heavily used, may not need to refresh
 :     :-----procs      -?-
 :     :-----sf         -*- misc scripts, ad hoc, coded as required
 :     :-----sfun       -*- ksh functions from UVSI, not changed by retransfer
 :     :-----sli0       -?- SLI code, might not have changed on mainframe ?
 :     :-----slis       -*-
 :     :-----tmp         0
 :     :-----xref        0  can rerun cross-references

Note	the 'vsetestlibsinit' script can be used to clear the '0' subdirs see script at /home/uvadm/sf/IBM/vsetestlibsinit

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1D1. Converting Mainframe DATA to unix/linux/windows

Pre-Requisites for DATA file conversion

Setups & File Transfer from Mainframe


 #1 - Install Vancouver Utilities as per install.htm


 #2 - Setup user/logins for uvadm, appsadm, conversion team members


 #3 - Setup file systems & directories for conversion as per ADMjobs.htm
    - super directories $CNVDATA, $TESTDATA, $TESTLIBS
    - see all subdirs listed on page '1B1' & '1B2'


 #4 - Transfer data files to be converted to $CNVDATA/d0
    - assumed to be in variable length format
    - records prefixed with 2 byte binary record length (RDW format)
    - use option 'site rdw' on FTP from mainframe
    - convert from RDW when copying from $CNVDATA to $TESTDATA
    - applications may require fixed typ=RSF
      or typ=IDXf8 (variable length for Micro Focus COBOL)


 #5 - Generate LISTCAT reports on mainframe for transfer to unix
    - ideally when mainframe discs contain mostly master files
    - with very few temp/work files that do not need to be converted
    - transfer to unix $CNVDATA/ctl/listcat0


 #6 - Transfer mainframe COBOL copybooks to $CNVDATA/cpys
    - OR (better) copy from $TESTLIBS/cpys/* if already stored there
    - copybooks may need some conversion for compiling (see VSECOBOL.htm)
    - but would probably be OK as is for generating data convert jobs

Provision for manual changes to Auto-Generated jobs

Part 1 presents the operating instructions to both generate & execute the data conversion jobs. If you have no multiple record type files, you may not have to make any manual changes to the auto-generated jobs.

You may need to modify the auto-generated job data conversion uvcopy jobs if your site has data files with multiple record types. These manual changes will be discussed & illustrated with examples later in 'Part_3'.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1D2. Converting Mainframe DATA to unix/linux/windows

Data file conversion - Overview

 MF--->d0------>d0ebc------->d1ebc---------->d2asc----->d3asc----->$TESTDATA/mstr
 #1 FTP #2 copy  #3 var->fix #4 EBCDIC->ASCII  #5 fix->var #6 copy
        rename   uvdata41    uvdata42/uvdata44   uvdata45
        scripts   pfv2/...        pfx3/...      pfy2/...


 #1 - mainframe ---> $CNVDATA/d0
    - data records must be FTP'd in binary mode (else packed fields destroyed)
    - data records are variable length, prefixed with 2 byte binary recsize
    - use option 'site rdw' on FTP from mainframe


 #2 - d0 --------> d0ebc
    - files are copied to d0ebc
    - filenames are translated to lowercase (via renameL script)
    - unwanted suffixes dropped (via rename-X script)
      (.clus.vseprd, .clus.vsedev, etc)


 #3 - d0ebc -------> d1ebc
    - uvcopy jobs generated by uvdata41 guided by control file ctl/ctlfile4
    - variable length converted to fixed length to facilitate conversion
    - 2 byte binary record prefixes dropped
    - files without prefixes must be identified in ctl/ctlfile4 with 'typ=RSF'
    - record length set to next multiple of 64 at least 35 higher
    - recsize info string inserted at end of maxsize data (in 35 bytes)
    - recsize info string: <1=recsz<2=maxsz<3=fixsz<K=dsp(lth)


 #4 - d1ebc --------> d2asc
    - uvdata42 generates uvcopy jobs from copybooks into pfx1/...
    - character fields are translated from EBCDIC to ASCII
    - packed/binary fields are preserved (copied as is), since they
      are the same as on the mainframe (expected by Micro Focus COBOL)
    - uvdata44 inserts datafilenames (vs copybooknames) while copying to pfx2/...
      guided by control file ctl/ctlfile4
      (uvdata44 for variable length, uvdata46 for fixed length)
    - pfx2/* is copied to pfx3/... for editing required for multi record types


 #5 - d2asc --------> d3asc
    - uvcopy jobs generated from control file by uvdata45
    - to restore original record sizes saved in extra space at end records
    - uvdata45 generates into pfy1, copied to pfy2 for execution by uxcopy
    - keyed files are sorted to ensure keys in sequence for loading indexes
      (out of seq could be caused by differences in ASCII collating sequence)


 #6 - d3asc --------> $TESTDATA/mstr
    - files are copied over to $TESTDATA/mstr for testing

provision for files without 2 byte prefix

We assume as the default that all files from the mainframe will have the 2 byte binary record size prefix. Files that do not have the record prefixes must be identified in the control file (ctl/ctlfile4) by adding 'typ=RSF' to their entry. This allows us to have consistent record formats for data conversion.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1D3. Converting Mainframe DATA to unix/linux/windows

Generating data conversion jobs - Overview

There are 3 types of data conversion job generations which are summarized here before we give the detailed operating instructions.

1. Generating jobs to convert variable length to fixed length

   ctl/ctlfile4 --------------> pfv1 --------------> pfv2 ----------------->
                generate jobs         manual copy         execute from pfv2
                  uvdata41

To facilitate EBCDIC to ASCII conversions, mainframe variable length records will be converted to a fixed length format which can be converted back to the original record sizes after conversion by saving the original size in extra space at the end of each record.

2. Generating EBCDIC to ASCII jobs - Overview

 mainframe------>cpys--------->maps---------->pfx1---------->pfx2----------->pfx3
          #1 FTP     #2 cobmap1    #3 uvdata42    #4 uvdata44    #5 copy/edit


 #1 - Mainframe ---> $CNVDATA/cpys
    - FTP in text mode automatically translates to ASCII


 #2 - cpys ------> maps
    - cobmap1 utility converts copybooks to record layouts (cobmaps)


 #3 - maps ------> pfx1
    - uvdata42 utility converts cobmaps to uvcopy job skeletons
    - skeleton jobs are named same as copybooks
    - datafilenames fili1=... & filo1=... are same as copybooknames


 #4 - pfx1 ------> pfx2
    - uvdata44 utility is guided by control file (ctl/ctlfile3)
    - datafilenames & record characteristics: recsizes, key start(length)
    - uvcopy jobnames changed from copybooknames to datafilenames
    - datafilenames inserted on fili1=... & filo1=...
    - indexed file keys added on filo1=...,key1=dsplcmnt(length)


 #5 - pfx2 ------> pfx3
    - uvcopy jobs are copied to pfx3 before any modification & execution
    - modification required for multi record type files
    - insert record type test instructions & skip to auto generated instrns
    - never modify in pfx2 in case of regeneration & overwrite

3. Generating jobs to restore original record sizes

   ctl/ctlfile4 --------------> pfy1 --------------> pfy2 ----------------->
                generate jobs         manual copy         execute from pfy2
                  uvdata45

To facilitate EBCDIC to ASCII conversions, mainframe variable length records are converted to a fixed length format & now we need to convert back to the original record sizes. The original record size was saved in extra space added to each record by the jobs generated by uvdata41 & executed from pfv2.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1E1. Converting Mainframe DATA to unix/linux/windows

Preparation to generate data conversion jobs

Note	you must have completed the pre-requisites on page '1D1' see subdirs required on page '1B1' EBCDIC data FTP'd (binary) from mainframe to $CNVDATA/d0 LISTCAT report FTP'd (ascii) from mainframe to $CNVDATA/ctl COBOL copybooks FTP'd (text/ASCII) from mainframe to $CNVDATA/cpys

Note	For your 1st conversion, you must follow these step by step procedures For subsequent Re-Conversions, you can use the scripts provided: see 'gencnv4A' on pages '1J1' - 1J3 to Re-Generate all conversion jobs see 'uxcopyxx' on '1J4' to Re-Execute all jobs to Re-Convert all files


 #1a. Login as appsadm --> /home/appsadm
 #1b. cdc ---> $CNVDATA    ('cdc' is an alias in common_profile)


 #2. cp d0/* d0ebc
     =============
     - copy datafiles to alt subdir for filename changes


 #3a. renameL d0ebc
      =============
      - translate datafilenames to lower case (standard for unix/linux)


 #3b. rename-X d0ebc .clus.vseprd
      ===========================
      - remove unwanted suffixes (.clus.vseprd, .clus.vsedev, etc)
      - do not apply to unix, do not need test/production indicators
      - test/prod provided by differences in profiles for prgrmrs/oprtrs
        with no need to have different filenames (or JCL/scripts)

create statistics for variable length files (optional)


 #4. create statistics for variable length files in d0ebc (optional)
     - create table summary counts of different recsizes in each file
     - reads the files via the 2 byte binary recsize prefixes


 #4a. mkdir tmp1      <-- make subdir for output reports (1 for file)
      rm tmp1/*         - OR remove all files if tmp1 already exists


 #4b. uvcopyx varstat1 d0ebc tmp1 uop=q0i7,rop=r0
      ===========================================
      - create table summary counts for each recsize in each file


 #4c. uvlpd12 tmp1   <-- print all table summaries in tmp1 subdir
      ============     - optional, could just view using 'vi tmp1/*'

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1E2. Converting Mainframe DATA to unix/linux/windows

create Data conversion control file

 listcat0--------->listcat1-------->listcat2---------->ctlfile3-------->ctlfile4
          catgut41           sort            ctlfile41          cp/edit


 #5. Create data conversion control file from mainframe LISTCAT report
     - LISTCAT report must have been stored as ctl/listcat0
     - see sample listing on page '8A1'
     - will extract filenames, avg-recsize, max-recsize, indexed key start(lth)


 #5a. uvcopy catgut41,fili1=ctl/listcat0,filo1=ctl/listcat1
      =====================================================
      - extract data conversion info from mainframe LISTCAT report


      uvcopy catgut41   <-- same as above (files default as shown above)
      ===============


 #5b. sort -o ctl/listcat2 ctl/listcat1
      =================================
      - sort by data file name
      - see sample listing on page '8B1'


 #5c. uvlp12 ctl/listcat2     <-- list the output file
      ===================       - a few records are shown below

 e2121653.mtaxe.piscines   rca=00082 rcm=00082 key=000(020) typLM_=IDXf1  cntr=00033321
 e2121656.itaxe.facturat   rca=00083 rcm=00245 key=000(036) typLM_=IDXf8v cntr=15946788
 e2123002.itaxe.taxation   rca=00046 rcm=00383 key=000(035) typLM_=IDXf8v cntr=00074292
 e2123004.itaxe.tranjour   rca=00464 rcm=00464 key=000(000) typLM_=RSF    cntr=00003905


 #5d. vi ctl/listcat2         <-- edit the listcat extract
      ===============
      - drop unwanted files (not required for conversion)
      - add any missing files (not on disc when LISTCAT run)


 #5e. uvlp12 ctl/listcat2     <-- re-list the listcat extract report
      ===================
      - store in 3 ring binder for frequent reference during conversion


 #6.  cp ctl/listcat2 $TESTLIBS/ctl
      ============================
      - copy listcat2 from $CNVDATA to $TESTLIBS for later use in JCL conversion
      - see JCL/script conversion summary in 'Part_6'
      - see JCL/script conversions illustrated in VSEJCL.htm

create data conversion control file from LISTCAT


 #7. uvcopy ctlfile41,fili1=ctl/listcat2,filo1=ctl/ctlfile3
      =====================================================
      - convert listcat report into data conversion control file format
      - inserts cpy=________, rcf=fixsz, drops cnt=...
      - see sample listing on page '8C1'

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1E3. Converting Mainframe DATA to unix/linux/windows

Editing ctlfile with copybook names

ctlfile3 was created above with 'cpy=________' slots into which, you must edit the copybooknames corresponding to the datafilenames.

Before we edit the control file, we will copy/rename as ctlfile4, to protect our edited version, in case ctlfile41 is rerun (would overwrite ctlfile3).

If this is the 1st creation, you can simply copy ctlfile3 to ctlfile4, and then edit ctlfile4.

Note	Do NOT do #8 (cp) if you are re-creating ctlfile4 see #8a (alternative) on next page '1E4' to recover copybook names


 #8. cp ctl/ctlfile3 ctl/ctlfile4
     ============================
     - copy/rename control file before editing
     - protects edited file from overwrite if ctlfile41 rerun
     - see sample listing on page '8D1'

Editing control file with copybook names


 #9. vi ctl/ctlfile4      <-- edit data conversion control file
     ===============
     - insert copybook names into the slot provided by cpy=________
       for all files if 1st creation, only new files if recreation from LISTCAT
     - drop files that do not need to be converted
     - add any missing files
     - identify files without 2 byte recsize prefixes by adding 'typ=RSF'
     - here are a few lines before & after adding the copybook names

ctlfile4 sample before & after copybook names

 e2121653.mtaxe.piscines cpy=________ rca=00082 rcm=00082 rcf=00128 key=000(020)
 e2121656.itaxe.facturat cpy=________ rca=00083 rcm=00245 rcf=00320 key=000(036)
 e2122683.itaxe.degrevem cpy=________ rca=00158 rcm=00265 rcf=00320 key=000(037)
 e2127546.mtaxe.donmarch cpy=________ rca=00100 rcm=00245 rcf=00320 key=000(020)

 e2121653.mtaxe.piscines cpy=e212159b rca=00082 rcm=00082 rcf=00128 key=000(020)
 e2121656.itaxe.facturat cpy=e212037b rca=00083 rcm=00245 rcf=00320 key=000(036)
 e2122683.itaxe.degrevem cpy=e212047b rca=00158 rcm=00265 rcf=00320 key=000(037)
 e2127546.mtaxe.donmarch cpy=e212142b rca=00100 rcm=00245 rcf=00320 key=000(020)
 tu.f01.e212-no.trperm   cpy=e221004b rca=00080 rcm=00080 rcf=00128 typ=RSF

Note	the last line (tu.f01...) was added manually, because it was tape only VSAM files are captured on LISTCAT reports you can see the complete control file listed on page '8D1'


 #10. uvlp12 ctl/ctlfile4  <-- list the edited data conversion control file
      ===================    - file in 3 ring binder for reference

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1E4. Converting Mainframe DATA to unix/linux/windows

Alternative for re-creating ctlfile

For re-creations, we would not want to overwrite ctlfile4 & lose the copybooknames that we added (laboriously) on the initial creation.

We will 1st save the existing ctlfile4 as an indexed file (ctlfile4I), that we can lookup to get the copybooknames as we copy ctlfile3 to ctlfile4 using utility ctlfile43.


 #8a. uvcopy ctlfile42,fili1=ctl/ctlfile4,filo1=ctl/ctlfile4I
      =======================================================
      - convert existing ctlfile (w copybooknames) to an indexed file


 #8b. uvcopy ctlfile43,fili1=ctl/ctlfile3,filo2=ctl/ctlfile4,fili3=ctl/ctlfile4I
      ==========================================================================
      - copy ctlfile3 to ctlfile4 recovering copybooknames from ctlfile4I

edit control file with copybook names


 #9a. vi ctl/ctlfile4      <-- edit data conversion control file
      ===============
      - insert copybook names into the slot provided by cpy=________
        for all files if 1st creation, only new files if recreation from LISTCAT
      - drop files that do not need to be converted
      - add any missing files
      - identify files without 2 byte recsize prefixes by adding 'typ=RSF'


 #10a. uvlp12 ctl/ctlfile4  <-- list the edited data conversion control file
       ===================    - file in 3 ring binder for reference

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1F0. Converting Mainframe DATA to unix/linux/windows

generate All jobs to convert All DATA files

Note

you must have completed pre-requisites on pages '1D1' thru '1E3'
create library subdirs: cpys,maps,ctl,pfv1,pfv2,pfx1,pfx2,pfx3,pfy1,pfy2
create data subdirs: d0, d0ebc, d1ebc, d2asc, d3asc
(see directories illustrated on page '1B1')
transfer DATA files, rename as desired (see page '1E1')
create/edit data control file ctl/ctlfile4 (see page '1E2' & '1E3')

We will generate 3 sets of uvcopy jobs to perform 3 major functions:


 #1. Generate jobs to convert variable to fixed length to simplify conversion.
     Generated into pfv1/..., copied to pfv2/... for execution


 #2. Generate jobs to convert EBCDIC to ASCII.
     Generated in pfx1, filenames added in pfx2, copied to pfv2/... for execution
     Manual changes will be required for files with multiple record types.


 #3. Generate jobs to convert fixed length back to variable length
     (using record sizes saved at the end of the fixed length records)
     Generated into pfy1/..., copied to pfy2/... for execution


 #1. ctl/ctlfile4 --------------> pfv1 --------------> pfv2 ----------------->
                  generate jobs         manual copy         execute from pfv2
                    uvdata41       (in case edit, unlikely)


 #2. main----->cpys------->maps-------->pfx1-------->pfx2------->pfx3--------->
     frame FTP     cobmap1     uvdata42     uvdata44     copy/edit    execute
                                                                       from pfx3


 #3. ctl/ctlfile4 --------------> pfy1 --------------> pfy2 ----------------->
                  generate jobs         manual copy         execute from pfy2
                    uvdata45       (in case edit, unlikely)

Note	you must follow these 'step by step' procedures for your first conversion for subsequent conversions, you can use the 'gencnv4A' script to reconvert all data files with 1 command

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1F1. Converting Mainframe DATA to unix/linux/windows

generate jobs to convert variable length to fixed length

   ctl/ctlfile4 --------------> pfv1 --------------> pfv2 ----------------->
                generate jobs         manual copy         execute from pfv2
                  uvdata41       (in case edit, unlikely)

At City of Lynn Valley, the VSAM files from the mainframe were prefixed by a 2 byte binary record size field whether truly variable or not. The tape files did not have these prefixes and the control file entries for these were coded with typ=RSF so the uvdata41 utility could generate the correct job.


 #0a. Login as appsadm --> /home/appsadm
 #0b. cdc ---> $CNVDATA    (/tax/cnvdata at City of Lynn Valley)


 #1.  uvcopy uvdata41,fili1=ctl/ctlfile4,fild2=pfv1
      =============================================
      - generate jobs to convert variable length datafiles to fixed length
      - as specified in control-file


 #1a. uvcopy uvdata41     <-- same as above (files default as above)
      ===============


 #2.  cp pfv1/* pfv2       <-- copy generated jobs to alt subdir
      ==============
      - protects pfv2/* from overwrite if uvdata41 rerun inadvertently


 #3.  vi pfv2/*            <-- edit generated jobs ?
      =========
      - changes to pfv2/* jobs are unlikely to be required
      - vs subsequent pfx2/* jobs that perform EBCDIC to ASCII convert
        (which need changes for multi record type files)

Note	you can probably rerun uvdata41 any time the ctlfile4 is updated to regenerate the pfv1/pfv2 uvcopy jobs (convert variable to fixed) it is highly unlikely you ever need to customize the generated job versus pfx1,2,3 (EBCDIC to ASCII) jobs which need record type coding

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1F2. Converting Mainframe DATA to unix/linux/windows

Generating jobs to convert ALL datafiles EBCDIC to ASCII

 mainframe----->cpys------->maps-------->pfx1-------->pfx2------->pfx3--------->
           FTP      cobmap1     uvdata42     uvdata44     copy/edit    execute
                                                                       from pfx3


 #1. uvcopyx cobmap1 cpys maps uop=q0i7p0
     ====================================
     - convert all copybooks to record layouts (cobmaps)


 #2. rmzf maps    <-- remove any zero files
     =========      - caused by procedure copybooks vs data copybooks


 #3. uvcopyx uvdata42 maps pfx1 uop=q0i7
     ===================================
     - generate skeleton uvcopy jobs from cobmaps
     - skeleton jobs have the same filenames as copybooks
     - skeleton jobs use copybooknames for datafilenames on fili1=... & filo1=...


 #4. uvcopyx uvdata43 pfx1 pfx1a uop=q0i7
     ====================================
     - extract uvcopy instructions from each job
     - extract uvcopy instructions (mvc's for packed fields) from each job
     - in case required for editing into uvcopy jobs in pfx3
     - when primary copybook (in ctlfile) does not have all record redefs


 #5. uvcopy uvdata44,fili1=ctl/ctlfile4,fild2=pfx1,fild3=pfx2,uop=q0i7
     =================================================================
     - complete the uvcopy jobs with actual datafilenames & indexed file keys
       (uvdata44 for variable length, uvdata46 for fixed length)
     - 1 uvcopy job for each line in control-file using specified copybook/map
     - completed jobs named same as datafilenames (vs copybooknames used in pfx1)
     - completed jobs change fili1=... & filo1=... to datafilenames


 #6. cp pfx2/* pfx3  <-- copy to pfx3/ for modification & execution (1st gen)
     ==============    - WARNING if Re-Generating (see note below)

Note	uvcopy jobs must be copied from pfx2 to pfx3 before executing may need modifications for multi record type files see Part_3 for examples of record type modifications

Note	do not copy all pfx2/* to pfx3 if re-generating if you have made any manual edits for multi R/T files (would not want to lose edits ?) could 1st save pfx3 as pfx3bak & then merge jobs with R/T coding ?


 #7. create script to copy/sort/load converted datafile to $TESTDATA/mstr
     - see sample scripts listed in 'Part_4'.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1F3. Converting Mainframe DATA to unix/linux/windows

generate jobs to restore original record sizes

   ctl/ctlfile4 --------------> pfy1 --------------> pfy2 ----------------->
                generate jobs         manual copy         execute from pfy2
                  uvdata45

'uvdata45' will generate jobs in pfy1/pfy2 that will use that extra space info to restore the records to the original size. uvdata45 is driven only by the control file (does not need the copybook info as did uvdata42).

It will generate 4 different job types depending on the file type

variable length indexed (typ=IDXf8v with key)
variable length sequential (also typ=IDXf8v but no key)
fixed length indexed (typ=IDXf1 with key)
fixed length sequential (typ=RSF or typ=RSR for DAM files)

generate jobs to restore original record sizes


 #0a. Login as appsadm --> /home/appsadm
 #0b. cdc ---> $CNVDATA    (/tax/cnvdata at City of Lynn Valley)


 #1.  uvcopy uvdata45,fili1=ctl/ctlfile4,fild2=pfy1
      =============================================
      - generate jobs to restore original record sizes
      - driven by the control-file & original rec-size saved at end record


 #1a. uvcopy uvdata45      <-- same as above (files default as above)
      ===============


 #2.  cp pfy1/* pfy2       <-- copy generated jobs to alt subdir
      ==============
      - protects pfy2/* from overwrite if uvdata45 rerun inadvertently


 #3.  vi pfy2/*            <-- edit generated jobs ?
      =========
      - changes to pfy2/* jobs are unlikely to be required
      - vs the pfx2/* (pfx3/*) jobs that perform EBCDIC to ASCII convert
        (which need changes for multi record type files)

Note	you can probably rerun uvdata45 any time the ctlfile4 is updated to regenerate the pfy1/pfy2 uvcopy jobs (restore original rec-size)

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1G0. Converting Mainframe DATA to unix/linux/windows

Executing jobs to convert ALL data files

Note	these 'execution' instructions out of sequence for sites that have files with multiple record types + packed, binary,or signed numeric fields they will need to modify the auto generated uvcopy jobs before execution see 'Part_3' for instructions to add record type test code to uvcopy jobs

Pre-Requisites


 #1 - You must have performed the pre-requisites listed on page '1D1' thru '1E3'
    - transfer mainframe datafiles to $CNVDATA/d0
    - transfer copybooks to $CNVDATA/cpys
    - generate mainframe LISTCAT reports & transfer to $CNVDATA/ctl/ctlfile0


 #2 - You must have performed the pre-requisites listed on page '1D1' thru '1E3'
    - required before generating the uvcopy jobs
    - rename datafiles to lower case, drop unwanted suffixes (CLUS.VSEPRD, etc)
    - create the control file to guide generation of jobs to convert data
    - generate jobs to reformat varlth files to fixed,


 #3 - You have already generated the uvcopy jobs to convert ALL datafiles
    - as documented on pages '1F0' thru '1F3'
    - or using script 'gencnv4A' to regen after copybook &/or ctlfile changes
    - see gencnv4A Op. Instrns. on page '1J3' & listing on page '9C1'
    - uvcopy jobs to convert files are auto-generated in subdir pfx2/...


 #4 - You have made any manual changes required
    - for files with multiple record types (see 'Part_3')
    - you must copy uvcopy jobs from pfx2/... to pfx3/... before manual edits

Recommendations for multi record type files

auto generated jobs are left in subdir 'pfx2'
copy to pfx3 before adding code to test for multi record types
this protects your edited pfx3 jobs in case auto gens rerun
copy jobs 1 at a time just before your edit session to add R/T tests
provides progress measurement by listing the pfx3 directory (vs pfx2)
see Part_3 for examples of record type modifications

Recommendations if NO multi record type files

I still recommend copying pfx2/... jobs to pfx3 before execution
to be consistent with documentation
and in case some files are later found to have multi record types

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1G1. Converting Mainframe DATA to unix/linux/windows

Executing jobs to convert ALL data files

 MF--->d0------>d0ebc------->d1ebc---------->d2asc----->d3asc----->$TESTDATA/mstr
 #1 FTP #2 copy  #3 var->fix #4 EBCDIC->ASCII  #5 fix->var #6 copy
        rename   uvdata41    uvdata42/uvdata44   uvdata45
        scripts   pfv2/...        pfx3/...      pfy2/...


 #0a. Login as appsadm --> /home/appsadm
 #0b. cdc ---> $CNVDATA    ('cdc' is an alias in common_profile)

Note	steps 2,3a,3b may already have been performed on page '1E1' as part of the procedure to generate the data conversion control file


 #1. cp d0/* d0ebc
     =============
     - copy datafiles to alt subdir for filename changes


 #2a. renameL d0ebc
      =============
      - translate datafilenames to lower case (standard for unix/linux)


 #2b. rename-X d0ebc .clus.vseprd
      ===========================
      - remove unwanted suffixes (.clus.vseprd, .clus.vsedev, etc)
      - do not apply to unix, do not need test/production indicators
      - test/prod provided by differences in profiles for prgrmrs/oprtrs
        with no need to have different filenames (or JCL/scripts)


 #3. uvcopyxx 'pfv2/*'
     =================
     - execute all uvcopy jobs to convert all datafiles from variable to fixed
     - copies all files from d0ebc/... to d1ebc/...
     - dropping the 2 byte binary prefix on varlth records
     - padding records with x'00's to a fixed length spcfd in the ctlfile
     - inserting recsize info string: <1=recsz<2=maxsz<3=fixsz<K=dsp(lth)
       in the space between record maxsz & fixsz spcfd in control file
     - files without 2 byte prefix are identified by 'typ=RSF' in the ctlfile
       which generated an fixlth version uvcopy job (vs varlth default)


 #4. uvcopyxx 'pfx3/*'
     =================
     - execute ALL uvcopy jobs in pfx3/... to convert all data files
     - copies all data files from d1ebc/... to d2asc/...
     - translating char fields from EBCDIC to ASCII
     - preserving any packed/binary fields (same as on mainframe)

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1G2. Converting Mainframe DATA to unix/linux/windows

Restoring original record sizes


 #5. uxcopyxx 'pfy2/*'
     =================
     - execute all jobs to restore original record sizes saved at end record
     - copies all files from d2asc/... to d3asc/...

Reduce the fixed size records used for conversion back to the original
record size (which was saved in extra space at the end of each record).

Convert the format from fixed size to variable length if applicable.

Load indexed files if applicable

Sort datafiles before loading indexed files, even if files were in
sequence on the mainframe, because the ASCII collating sequence could
cause out-of-sequences if the key field contains both char & numeric
(since numerics are now lower than alphas in ASCII).

Variable length (Indexed & Sequential) files will be converted to IDXFORMAT8.

Copying converted Data files to test directories

After data file conversion & before we can begin testing, we need to copy the converted data files from $CNVDATA/d3asc/... to $TESTDATA/mstr/...

This procedure can be repeated any time that testing has destroyed the integrity of the set of co-ordinated data files.


 #6a. rm -f $TESTDATA/mstr/*
      ======================
      - remove old files & any extra files generated thru testing


 #6b. cp -f d3asc/* $TESTDATA/mstr
      ============================
      - copy all converted files for the next round of testing

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1H1. Converting Mainframe DATA to unix/linux/windows

Generating job to convert 1 Data file

Note	this page uses 'datafilename' or 'prefix' vs an actual filename next page repeats these instrns using a filename/prefix example actually easier to generate jobs for ALL datafiles (vs 1 at a time)


 #1a. Login as appsadm --> /home/appsadm
 #1b. cdc ---> $CNVDATA    ('cdc' is an alias in common_profile)


 #2a. uvcopy uvdata41,fili1=ctl/ctlfile4,fild2=pfv1,arg1=datafilename/prefix
      ======================================================================
      - generate uvcopy job to convert 1 variable length datafile to fixed length
      - uvcopy job(s) generated for lines in control file matching arg1(prefix)


 #2b. cp pfv1/datafilename pfv2  - copy to pfv2 for later execution
      =========================


 #3. uvcopy cobmap1,fili1=cpys/copybookname,filo1=maps/copybookname
     ==============================================================
     - convert the specified copybook to a record layout (cobmap)


 #4. uvcopy uvdata42,fili1=maps/copybookname,filo1=pfx1/copybookname
     ===============================================================
     - generate uvcopy job to convert 1 data file from EBCDIC to ASCII
     - generates a skeleton uvcopy job from the record layout (cobmap)


 #5. uvcopy uvdata43,fili1=pfx1/copybookname,filo1=pfx1a/copybookname
     ================================================================
     - extract conversion instructions from specified job
     - write instructions into pfx1a using same jobname as in pfx1
     - probably do NOT need to run uvdata43 for 1 file at a time
       since '1 at a time' generations usually only required for copybooks
       missed when we initially generated jobs from all copybooks


 #6. uvcopy uvdata44,fili1=ctl/ctlfile4,fild2=pfx1,fild3=pfx2,arg1=datfn/prefix
     ==========================================================================
     - complete the uvcopy job with actual datafilenames & indexed file keys
     - jobs generated for control file lines matching arg1 datafilename/prefix
     - jobs in pfx2 named for datafiles in control file (vs copybooks in pfx1)


 #7. cp pfx2/jobname pfx3   <-- copy uvcopy job to execution dir
     ====================     - jobnames same as datafilenames


 #8. vi pfx3/jobname        <-- edit job if required
     ===============          - see examples in 'Part_3'.
     - changes required if file contains multiple record types
     - test record types & skip to instructions generated for redefined records


 #9a. uvcopy uvdata45,fili1=ctl/ctlfile4,fild2=pfy1,arg1=datafilename/prefix
      ======================================================================
      - generate uvcopy job to convert fixed length back to variable length


 #9b. cp pfy1/datafilename pfy2  - copy to pfy2 for later execution
      =========================

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1H2. Converting Mainframe DATA to unix/linux/windows

Generating job to convert 1 file - example

This page repeats prior page using an actual copybookname & datafilename vs the generic copybookname, datafilename/prefix used on prior page.

 e212159b                 <-- copybookname example used on this page
 e2121653.mtaxe.piscines  <-- datafilename example used on this page
 e2121653                 <-- prefix example used on this page
 ========                   - 1st 8 chars is unique at this site


 #2a. uvcopy uvdata41,fili1=ctl/ctlfile4,fild2=pfv1,arg1=e2121653
      ===========================================================
      - generate uvcopy job to convert 1 variable length datafile to fixed length
      - uvcopy job(s) generated for lines in control file matching arg1(prefix)


 #2b. cp pfv1/datafilename pfv2    <-- copy to pfv2 for later execution
      =========================


 #3. uvcopy cobmap1,fili1=cpys/e212159b,filo1=maps/e212159b
     ======================================================
     - convert the specified copybook to a record layout (cobmap)


 #4. uvcopy uvdata42,fili1=maps/e212159b,filo1=pfx1/e212159b
     =======================================================
     - generate uvcopy job to convert 1 data file from EBCDIC to ASCII
     - generates a skeleton uvcopy job from the record layout (cobmap)


 #5. uvcopy uvdata43,fili1=pfx1/e212159b,filo1=pfx1a/e212159b
     ========================================================
     - extract conversion instructions from specified job
     - write instructions into pfx1a using same copybookname as in pfx1
     - probably do NOT need to run uvdata43 for 1 file at a time
       since '1 at a time' generations usually only required for copybooks
       missed when we initially generated jobs from all copybooks


 #6. uvcopy uvdata44,fili1=ctl/ctlfile4,fild2=pfx1,fild3=pfx2,arg1=e2121653
     ======================================================================
     - complete the uvcopy job with actual datafilenames & indexed file keys
     - jobs generated for control file lines matching arg1 datafilename/prefix
     - jobs in pfx2 named for datafiles in control file (vs copybooks in pfx1)


 #7. cp pfx2/e2121653* pfx3   <-- copy uvcopy job to execution dir
     ======================     - uvcopy jobnames same as datafilenames
                                - 1st 8 characters was unique at this site


 #8. vi pfx3/e2121653*        <-- edit job if required
     =================          - see examples in Part_3
     - changes required if file contains multiple record types
     - test record types & skip to instructions generated for redefined records


 #9a. uvcopy uvdata45,fili1=ctl/ctlfile4,fild2=pfy1,arg1=datafilename/prefix
      ======================================================================
      - generate uvcopy job to convert fixed length back to variable length


 #9b. cp pfy1/datafilename pfy2  - copy to pfy2 for later execution
      =========================

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1I1. Converting Mainframe DATA to unix/linux/windows

Executing jobs to convert 1 data file

You must already have performed the pre-requisites listed on page '1G0' (same as the pre-requisites for executing ALL datafiles).

At the begining of the conversion you should use the 'generate ALL' & 'execute ALL' procedures, but you may need to convert 1 data file if some files were missed during the initial conversion or you need to transfer & convert the current datafile from the mainframe.

 MF--->d0------>d0ebc------->d1ebc---------->d2asc----->d3asc----->$TESTDATA/mstr
 #1 FTP #2 copy  #3 var->fix #4 EBCDIC->ASCII  #5 fix->var #6 copy
        rename   uvdata41       uvdata42/44       uvdata45
        scripts    pfv2/...       pfx3/...          pfy2/...


 #0a. Login as appsadm or as yourself
 #0b. cdc ---> $CNVDATA      ('cdc' is an alias in common_profile)


 #1. cp d0/datafilename d0ebc
     ========================
     - copy the datafile to alt subdir for filename changes


 #2a. renameL d0ebc
      =============
      - translate datafilenames to lower case (standard for unix/linux)
      - renameL can be used for 1 file or all files in directory


 #2b. rename-X d0ebc .clus.vseprd
      ===========================
      - remove unwanted suffixes (.clus.vseprd, .clus.vsedev, etc)
      - rename-X can be used for 1 file or all files in directory


 #3. uvcopy pfv2/datafilename
     ========================
     - execute 1 uvcopy job to convert 1 datafile from variable to fixed
     - copies the file from d0ebc/... to d1ebc/...
     - dropping the 2 byte binary prefix on varlth records
     - padding records with x'00's to a fixed length spcfd in the ctlfile
     - inserting recsize info string: <1=recsz<2=maxsz<3=fixsz<K=dsp(lth)
       in the space between record maxsz & fixsz spcfd in control file
     - files without 2 byte prefix are identified by 'typ=RSF' in the ctlfile
       which generated a fixlth version uvcopy job (vs varlth default)


 #4. uvcopy pfx3/datafilename
     ========================
     - execute 1 uvcopy job to convert 1 datafile
     - copies the datafile from d1ebc/... to d2asc/...
     - translating char fields from EBCDIC to ASCII
     - preserving any packed/binary fields (same as on mainframe)

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1I2. Converting Mainframe DATA to unix/linux/windows

Restoring original record sizes


 #5. uxcopy pfy2/datafilename
     ========================
     - execute 1 uxcopy job to restore original record sizes

Reduce the fixed size records used for conversion back to the original
record size (which was saved in extra space at the end of each record).

Convert the format from fixed size to variable length if applicable.

Load indexed files if applicable

Sort datafiles before loading indexed files, even if files were in
sequence on the mainframe, because the ASCII collating sequence could
cause out-of-sequences if the key field contains both char & numeric
(since numerics are now lower than alphas in ASCII).

Variable length (Indexed & Sequential) files will be converted to IDXFORMAT8.

Must execute 'uxcopy' (vs 'uvcopy') to process IDXFORMAT3/IDXFORMAT8 files.

Copying converted Data files to test directories

After data file conversion & before we can begin testing, we need to copy the converted data files from $CNVDATA/d3asc/... to $TESTDATA/mstr/...

Re-copy any time that testing has updated 1 or more files, especially if you know the expected results from the original mainframe datafile.


 #6. cp -f d3asc/datafilename $TESTDATA/mstr
     =======================================
     - copy converted file(s) for the next round of testing

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1J1. Converting Mainframe DATA to unix/linux/windows

Scripts to Re-Generate All Conversion jobs

For the 1st time setup & generation of data conversion jobs, you must follow the step by step procedures documented on pages '1E1' thru '1F3'.

Scripts gencnv41 & gencnv4A are provided to quickly re-generate 1 or All jobs. Use these when changes are made to copybooks during the conversion phase.

 mainframe------>cpys--------->maps---------->pfx1---------->pfx2----------->pfx3
          #1 FTP     #2 cobmap1    #3 uvdata42    #4 uvdata44    #5 copy/edit

 <--- manual ---->|<----------------- automatic -------------->|<--- manual --->

Note that the automatic generation stops at pfx2, to avoid overwriting jobs in pfx3/... which may already have had considerable editing to add code for multiple record types.

You should use gencnv4A even if only a few copybooks have changed, because it is usually almost as fast to regenerate for All as for 1 & some multi- record type files may require instructions from multiple copybooks.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1J2. Converting Mainframe DATA to unix/linux/windows

Preparations to use gencnv4A/gencnv41

We assume you have already completed the 1st time Pre-Requisites on '1D1'.


 #1a. Login as appsadm or as yourself
 #1b. cdc ---> $CNVDATA      ('cdc' is an alias in common_profile)

You could save all existing cnvdata, if you wish to compare new results to old results, or if you are not yet confident in these procedures.


 #2a. cd ..          <-- change above $CNVDATA
 #2b. cp -r cnvdata cnvdata.old  <-- save existing cnvdata (optional)
      =========================
 #2c. cdc            <-- change back into $CNVDATA

Run the 'cnvdatainit' script to clear subdir/files that will be recreated See script listing on page '9B1'. See cnvdata subdirs listed on page '1C1' which indicates which subdirs are cleared, updated, or maintained as is.


 #3a. cnvdatainit   <-- clear subdir/files to be recreated
      ===========

Add new or updated copybooks (from the mainframe) to $CNVDATA/cpys.
Update the control file as shown on page '1E3'. If you are adding a lot of new files, & if the files are now present on the mainframe, you might recreate the LISTCAT report on the mainframe, & re-transfer to linux. You could then recreate ctlfile4 from LISTCAT as documented on page '1E2'. See sample listings of control files on pages '8A1' thru '8D1'.

Note	Alternatively if you are adding very few new files it will faster to add lines to the control file with the editor


 #5a. vi ctl/ctlfile4   <-- add lines for new files to existing ctlfile
      ===============     - easier alternative if very few new files
      - 3 lines shown below, see more on page '1E3' or '8D1'

 e2121653.mtaxe.piscines cpy=e212159b rca=00082 rcm=00082 rcf=00128 key=000(020)
 e2121656.itaxe.facturat cpy=e212037b rca=00083 rcm=00245 rcf=00320 key=000(036)
 tu.f01.e212-no.trperm   cpy=e221004b rca=00080 rcm=00080 rcf=00128 typ=RSF

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1J3. Converting Mainframe DATA to unix/linux/windows

Add new datafiles (from the mainframe) to $CNVDATA/d0/...
Copy datafiles from d0/... to d0ebc, rename to lower case & remove unwanted suffixes such as CLUS.VSEPRD, CLUS.VSEPRD, etc (more details on page '1E1').


 #7a. cp d0/* d0ebc    <-- copy datafiles for filename changes
     ==============


 #7b. renameL d0ebc    <-- translate datafilenames to lower case
      =============


 #7c. rename-X d0ebc .clus.vseprd  <-- remove unwanted suffixes
      ===========================

gencnv4A - regen All jobs to reconvert All files


 #8. gencnv4A ctl/ctlfile4     <-- super-script to regen All conversion jobs
     =====================
     - generate uvcopy jobs to convert all datafiles listed in ctlfile4

add new/updated jobs from pfx2 to pfx3

Auto generations stop at pfx2, to avoid overwriting jobs in pfx3/... which may already have had considerable editing to add code for multiple record types. We usually regenerate all jobs (up to pfx2), even if only a few copybooks have changed since the initial generation.


 cp pfx2/* pfx3    <-- do NOT do this, unless you are certain you need
 ==============        no manual changes (for files with multiple record types)

You need to edit/merge the newly generated instructions from pfx2, with the 'record type testing code' that you had previously developed in pfx3.


 #9. cp -r pfx3 pfx3.bak  <-- backup prior generated jobs, that may have
     ===================      been modified for multi R/T's


 #10. cp pfx2/xxxx pfx3   <-- selective copy of jobs that need updates
      ==================


 #11. vi pfx3/xxxx pfx3   <-- selective update of jobs with multi R/Ts
      =================
      :r pfx3.bak/xxxx      - read in R/T test code from prior jobs

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1J4. Converting Mainframe DATA to unix/linux/windows

Re-Executing All jobs to Re-convert All datafiles

Note	you must have added/updated jobs from pfx2 to pfx3 before proceeding. inserting code for multi record type files (see examples in 'Part_2').

Please see procedures to 'Execute All jobs to convert All datafiles' on pages '1G1' & '1G2', but briefly & without much explanation:


 #1. uvcopyxx 'pfv2/*'  <-- execute all jobs to convert varlth to fixlth
     =================    - as we copy from d0ebc/* to d1ebc/...


 #2. uvcopyxx 'pfx3/*' <-- execute all jobs to convert EBCDIC to ASCII
     =================   - as we copy from d1ebc/* to d2asc/...


 #3. uxcopyxx 'pfy2/*' <-- execute all jobs to restore original record-sizes
     =================   - as we copy from d2asc/* to d3asc/...


 #4. rm -f $TESTDATA/mstr/*
     ======================
     - remove old files & any extra files generated thru testing


 #5. cp -f d3asc/* $TESTDATA/mstr
     ============================
     - copy all converted files for the next round of testing

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1K1. Converting Mainframe DATA to unix/linux/windows

gencnv41 - regen 1 job to reconvert 1 datafile

Note	ensure that ctl/ctlfile4 has the correct info for the file to reconvert see ctlfile update briefly on page '1J2' or extensively '1E1' thru '1E4'.


 #1a. Login as appsadm or as yourself
 #1b. cdc ---> $CNVDATA      ('cdc' is an alias in common_profile)


 #2a. gencnv41 datafilename copybookname   <-- command format for 1 at a time
      ==================================
     - generate uvcopy jobs to convert 1 datafile  at a time (vs All)


 #2b. gencnv41 tu.f01.e212-no.trperm e221004b <-- example for 1 datafile&copybook
      =======================================
     - generate uvcopy job to convert 1 datafile using 1 copybook

add new/updated jobs from pfx2 to pfx3

Auto generations stop at pfx2, to avoid overwriting jobs in pfx3/... which may already have had considerable editing to add code for multiple record types.

We usually regenerate all jobs (up to pfx2), even if only a few copybooks have changed since the initial generation.

Now you need to edit/merge the newly generated instructions from pfx2, with the 'record type testing code' that you had previously developed in pfx3.

I suggest the following procedure. As an example I will use datafile 'tu.f01.e212-no.trperm' & copybook 'e221004b'.


 #3. cp -rf pfx3 pfx3.bak
     ====================
     - backup the entire pfx3 directory
     - many jobs may contain manually added instructions to test record types


 #4. cp pfx2/tu.f01.e212-no.trperm pfx3
     ==================================
     - copy regenerated job to pfx3, overwriting the old job
       (not a problem because we have saved it in pfx3.bak)


 #5. vi pfx3/tu.f01.e212-no.trperm
     =============================
     - edit the regenerated job (with new instructions for new copybook)
     --> position to the R/T test code area (usually line 13)
     --> :r pfx3.bak/tu.f01.e212-no.trperm  <-- read in the old job
     --> delete all but the R/T testing code
     --> :wq                                <-- write & quit

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1K2. Converting Mainframe DATA to unix/linux/windows

alternative extract of prior R/T code

You might find the following an easier alternative for editing the old record type testing code into the new job.


 #5a. cp pfx3.bak/tu.f01.e212-no.trperm tmp/tu.f01.RT
      ===============================================
      - copy the old job to tmp/... (might give it a short name since tmp)


 #5b. vi tmp/tu.f01.RT     <-- edit the copied job in tmp/...
      ================
      --> delete all lines except the 'record type test coding'
      --> :wq         <-- write & quit


 #5c. vi pfx3/tu.f01.e212-no.trperm
      =============================
      - edit the regenerated job (with new instructions for new copybook)
      --> position to the R/T test code area (usually line 13)
      --> :r pfx3/tu.f01.RT  <-- read in the record type test code (only)
      --> :wq                <-- write & quit

Note	see the above example illustrated in detail on Part_2.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

1K3. Converting Mainframe DATA to unix/linux/windows

Executing job (created by gencnv41) to convert 1 datafile

 MF--->d0------>d0ebc------->d1ebc---------->d2asc----->d3asc----->$TESTDATA/mstr
 #1 FTP #2 copy  #3 var->fix #4 EBCDIC->ASCII  #5 fix->var #6 copy
        rename   uvdata41       uvdata42/44       uvdata45
        scripts    pfv2/...       pfx3/...          pfy2/...


 #0a. Login as appsadm or as yourself
 #0b. cdc ---> $CNVDATA      ('cdc' is an alias in common_profile)


 #1. cp d0/e2121653.mtaxe.piscines d0ebc
     ===================================
     - copy the datafile to alt subdir for filename changes


 #2a. renameL d0ebc
      =============
      - translate e2121653.mtaxe.piscines to lower case
      - renameL can be used for 1 file or all files in directory


 #2b. rename-X d0ebc .clus.vseprd
      ===========================
      - remove unwanted suffixes (.clus.vseprd, etc, do not apply to unix)
      - test/prod provided by differences in profiles for prgrmrs/oprtrs
        with no need to have different filenames (or JCL/scripts)
      - rename-X can be used for 1 file or all files in directory


 #3. uvcopy pfv2/e2121653.mtaxe.piscines
     ===================================
     - execute 1 uvcopy job to convert 1 datafile from variable to fixed
     - copies the file from d0ebc/... to d1ebc/...
     - dropping the 2 byte binary prefix on varlth records
     - padding records with x'00's to a fixed length spcfd in the ctlfile
     - inserting recsize info string: <1=recsz<2=maxsz<3=fixsz<K=dsp(lth)
       in the space between record maxsz & fixsz spcfd in control file


 #4. uvcopy pfx3/e2121653.mtaxe.piscines
     ===================================
     - execute 1 uvcopy job to convert 1 datafile
     - copies the datafile from d1ebc/... to d2asc/...
     - translating char fields from EBCDIC to ASCII
     - preserving any packed/binary fields (same as on mainframe)


 #5. uxcopy pfy2/e2121653.mtaxe.piscines
     ===================================
     - execute 1 uxcopy job to restore original record sizes


 #6. cp -f d3asc/e2121653.mtaxe.piscines $TESTDATA/mstr
     ==================================================
     - copy converted file(s) for the next round of testing

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

Part_2 Converting Mainframe DATA to unix/linux/windows

Illustrated uvcopy job Generation - single R/T files

2A1.	Introduction & Overview

2B1.	sample control file to guide uvcopy job generation

2B2.	generate job to convert variable length to fixed length (for conversion)

2B3.	convert copybook to record layout 'cobmap' (from cpys/... to maps/...)

2B4.	convert cobmap to uvcopy skeleton job (from maps/... to pfx1/...)
	- skeleton jobs use copybookname for modulename & I/O datafilenames

2B5.	extract instructions from uvcopy job (from pfx1/... to pfx1a/...)
	- in case required by other uvcopy jobs

2B6.	insert datafilenames into skeleton jobs (vs copybook names)
	- guided by control file relating datafilenames to copybooknames

Illustrated uvcopy job Execution - single R/T files

2C1.	Preparation to execute uvcopy conversion jobs
	- copy/rename files

2C2.	uvhd (hexdump) of sample EBCDIC datafile (in varlth format)

2C3.	convert variable length to fixed length (only for conversion phase)
	- uvhd (hexdump) of sample EBCDIC datafile in fixlth format

2C4.	convert from EBCDIC to ASCII, preserving packed fields
	- uvhd (hexdump) of results

2C5.	copy converted datafile to $TESTDATA/mstr (for testing & later production)
	- converting fixlth back to varlth if required for COBOL programs
	- uvhd (hexdump) of final results

Part 2B - Illustrated Generation & Execution Multi Record Types

2D1.	Introduction to Multi-Record-Type files

2E1.	Begin Illustrated Generations for multi record type files
	- same items as shown above for single R/T files (2B1 - 2B6)

2F1.	Begin Illustrated Executions for multi record type files
	- same items as shown above for single R/T files (2C1 - 2C6)

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2A1. Illustrated uvcopy job generation - simple files

Illustrated uvcopy job generation - simple files

This section is intended to help you understand the generation & execution of uvcopy data conversion jobs. The operating instructions in Part_1 are concise and give no examples of the uvcopy jobs generated or the data converted.

Part_2 will present the illustrated version of the sample generation previously given on page '1H2', and also illustrate its execution.

Part_2 illustrates the generation & execution of a uvcopy job, that required no manual changes because the datafile did not have multiple record types.

See Part 2B ('2D1' - 2F5) for datafiles with multiple record types that do require insertion of record type test code to skip to the automatically generated instructions for redefined records.

See Part_3 for illustrated conversions of files with Variable length records and multiple (100+) record types.

Generating uvcopy job - Overview

 mainframe------>cpys--------->maps---------->pfx1---------->pfx2----------->pfx3
          #1 FTP     #2 cobmap1    #3 uvdata42    #4 uvdata44    #5 copy/edit

Data file conversion - Overview

 MF--->d0------>d0ebc------->d1ebc---------->d2asc----->d3asc----->$TESTDATA/mstr
 #1 FTP #2 copy  #3 var->fix #4 EBCDIC->ASCII  #5 fix->var #6 copy
        rename   uvdata41       uvdata42/44       uvdata45
        scripts    pfv2/...       pfx3/...          pfy2/...

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2B1. Generating jobs to convert simple datafiles (illustrated)

Note	Part 2 generates 1 job at a time with illustrations it is intended to help you understand the process see Part_1 for the efficient generation of all jobs for all data files

Generating jobs to convert 1 datafile

We will illustrate the uvcopy job generation & execution using the following datafile & its corresponding copybook:


 e2121653.mtaxe.piscines  <-- datafilename used for this example
 =======================


 e212159b                 <-- copybookname corresponding to the datafile
 ========

Part_2 assumes you have completed the pre-requisites on pages '1D1' thru '1E3' Page '1E3' prepared the control file which relates the datafilenames to the corresponding copybooknames.

sample control file ctl/ctlfile4

 e2121653.mtaxe.piscines cpy=e212159b rca=00082 rcm=00082 rcf=00128 key=000(020)*
 e2121656.itaxe.facturat cpy=e212037b rca=00083 rcm=00245 rcf=00320 key=000(036)
 e2122683.itaxe.degrevem cpy=e212047b rca=00158 rcm=00265 rcf=00320 key=000(037)
 e2127546.mtaxe.donmarch cpy=e212142b rca=00100 rcm=00245 rcf=00320 key=000(020)
 tu.f01.e212-no.trperm   cpy=e221004b rca=00080 rcm=00080 rcf=00128 typ=RSF

The control file contains entries for all datafiles that need to be converted & the '*' indicates the file selected for our sample conversion.

You can see that this is a fixed length file, since rcs=82 & rcm=82, but most files are transferred from the mainframe in variable length format (2 byte prefix with record size in binary) even though all records are the same size.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2B2. Generating jobs to convert simple datafiles (illustrated)

Generate job to convert varlth data to fixlth data


 #1a. Login as appsadm --> /home/appsadm
 #1b. cdc ---> $CNVDATA    ('cdc' is an alias in common_profile)


 #2a. uvcopy uvdata41,fili1=ctl/ctlfile4,fild2=pfv1,arg1=e2121653
      ===========================================================
      - generate uvcopy job to convert 1 variable length datafile to fixed length
      - uvcopy job(s) generated for lines in control file matching arg1(prefix)


 #2b. cp pfv1/datafilename pfv2
      =========================
      - copy generated uvcopy job to pfv2 for later execution

sample job to convert varlth to fixlth

 # e2121653.mtaxe.piscines - convert var to fix before EBCDIC/ASCII convert
 #  - uvcopy job generated by $UV/pf/IBM/uvdata41 (Dec19/07 chg to typ=RDW)
 #  - drop 2 byte varlth prefix & save recsz info at end of maxsz
 #  - variable length to fixed assumed if control file not 'typ=RSF'
 was=a32768b32768
 fili1=d0ebc/e2121653.mtaxe.piscines,typ=RDWz2,rcs=1024
 filo1=d1ebc/e2121653.mtaxe.piscines,typ=RSF,rcs=00128
 @run
        opn    all
 loop   get    fili1,a0,x'00'      get next varlth record (init area nulls)
        skp>   eof                 (cc set > at EOF)
        mvc    b0(00128),a0        move data to output area
        mvn    $rb,00082           load rgstr 'b' with maxsz
        mvc    bb0(35),'<1=RECSZ<2=00082<3=00128<K=000(020)'  setup recsz info
        mvn    bb3(5),$rv          current recsz stored in $rv by get typ=RDW
        tre    bb0(35)             convert to EBCDIC
        put    filo1,b0            write current record to output file
        skp    loop                repeat loop until EOF
 eof    cls    all                 close files
        eoj                        end job

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2B3. Generating jobs to convert simple datafiles (illustrated)

convert copybook to cobmap to guide next step


 #3. uvcopy cobmap1,fili1=cpys/e212159b,filo1=maps/e212159b
     ======================================================
     - convert the specified copybook to a record layout (cobmap)

 cobmap1  start-end bytes for cobol record fields    200702061501  pg# 0001
 cpys/e212159b                                     RCSZ=00082  bgn  end  lth typ
 *  definition du fichier piscines  (82)
    03  fp-rec-piscines.
      05  fp-role-k               pic x(20).                  0000 0019  020
      05  fp-code-maj             pic s9       comp-3.        0020 0020  001pns 1
      05  fp-dt-crea              pic s9(8)    comp-3.        0021 0025  005pns 8
      05  fp-dt-maj               pic s9(8)    comp-3.        0026 0030  005pns 8
      05  fp-mat1                 pic s9(11)   comp-3.        0031 0036  006pns11
      05  fp-mat2                 pic s9(9)    comp-3.        0037 0041  005pns 9
      05  fp-code-post            pic x(6).                   0042 0047  006
      05  fp-nbr-pisc             pic s9       comp-3.        0048 0048  001pns 1
      05  fp-code-pisc            pic x.                      0049 0049  001
      05  fp-no-semaine           pic s99      comp-3.        0050 0051  002pns 2
      05  fp-no-vendeur           pic s9(3)    comp-3.        0052 0053  002pns 3
      05  fp-annee-fact           pic s9(4)    comp-3.        0054 0056  003pns 4
      05  filler001               pic x(25).                  0057 0081  025
 *RCSZ=00082

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2B4. Generating jobs to convert simple datafiles (illustrated)

Generate job to convert EBCDIC to ASCII & preserve packed


 #4. uvcopy uvdata42,fili1=maps/e212159b,filo1=pfx1/e212159b
     =======================================================
     - generate uvcopy job to convert 1 data file from EBCDIC to ASCII
     - generates a skeleton uvcopy job from the record layout (cobmap)

sample skeleton job to convert EBCDIC to ASCII

 # e212159b - uvcopy EBCDIC to ASCII, preserve packed, fix signs
 opr='e212159b - uvcopy job generated from copybook: E212159B '
 #ctlfile:
 was=a33000b33000
 fili1=?d1ebc/e212159b,rcs=00082,typ=RSF
 filo1=?d2asc/e212159b,rcs=00082,typ=RSF
 @run
        opn    all
 loop   get    fili1,a0
        skp>   eof
        mvc    b0(00082),a0       move rec to outarea before field prcsng
        tra    b0(00082)          translate entire outarea to ASCII
 #      ---                <-- insert R/T tests here for redefined records
 typ159 mvc    b20(22),a20             pns fp-code-maj:fp-mat2
        mvc    b48(1),a48              pns fp-nbr-pisc
        mvc    b50(7),a50              pns fp-no-semaine:fp-annee-fact
 put1   put    filo1,b0
        skp    loop
 eof    cls    all
        eoj

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2B5. Generating jobs to convert simple datafiles (illustrated)

extract instructions in case required by other jobs


 #5. uvcopy uvdata43,fili1=pfx1/e212159b,filo1=pfx1a/e212159b
     ========================================================
     - extract conversion instructions from specified job
     - write instructions into pfx1a using same copybookname as in pfx1
     - probably do NOT need to run uvdata43 for 1 file at a time
       since '1 at a time' generations usually only required for copybooks
       missed when we initially generated jobs from all copybooks

sample output - extracted instructions

 # e212159b
 typ159 mvc    b20(22),a20             pns fp-code-maj:fp-mat2
        mvc    b48(1),a48              pns fp-nbr-pisc
        mvc    b50(7),a50              pns fp-no-semaine:fp-annee-fact
        skp    put1
 #

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2B6. Generating jobs to convert simple datafiles (illustrated)

complete skeleton job with datafilenames (vs copybookname)


 #6. uvcopy uvdata44,fili1=ctl/ctlfile4,fild2=pfx1,fild3=pfx2,arg1=e2121653
     ======================================================================
     - complete the uvcopy job with actual datafilenames & indexed file keys
       (uvdata44 for variable length, uvdata46 for fixed length)
     - jobs generated for control file lines matching arg1 datafilename/prefix
     - jobs in pfx2 named for datafiles in control file (vs copybooks in pfx1)


 #7. cp pfx2/e2121653* pfx3   <-- copy uvcopy job to execution dir
     ======================     - uvcopy jobnames same as datafilenames
                                - 1st 8 characters was unique at this site


 #8. vi pfx3/e2121653*        <-- edit job if required
     =================          - see examples in Part_3
     - changes required if file contains multiple record types
     - test record types & skip to instructions generated for redefined records
     - may need to read instructions for record types not redefined
     - read from pfx1a (instructions only extracted from generated jobs)

complete job to convert EBCDIC to ASCII & preserve packed

 # e2121653.mtaxe.piscines - uvcopy EBCDIC to ASCII, preserve packed, fix signs
 opr='e2121653.mtaxe.piscines - uvcopy job generated from copybook: E212159B '
 #ctlfile: e2121653.mtaxe.piscines cpy=e212159b rca=00082 rcm=00082 rcf=00128 key=000(020)
 was=a33000b33000
 fili1=?d1ebc/e2121653.mtaxe.piscines,rcs=00128,typ=RSF
 filo1=?d2asc/e2121653.mtaxe.piscines,rcs=00128,typ=RSF
 @run
        opn    all
 loop   get    fili1,a0
        skp>   eof
        mvc    b0(00128),a0       move rec to outarea before field prcsng
        tra    b0(00128)          translate entire outarea to ASCII
 #      ---                <-- insert R/T tests here for redefined records
 typ159 mvc    b20(22),a20             pns fp-code-maj:fp-mat2
        mvc    b48(1),a48              pns fp-nbr-pisc
        mvc    b50(7),a50              pns fp-no-semaine:fp-annee-fact
 put1   put    filo1,b0
        skp    loop
 eof    cls    all
        eoj

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2B7. Generating jobs to convert simple datafiles (illustrated)

generate job to restore original record sizes


 #9.  uvcopy uvdata45,fili1=ctl/ctlfile4,fild2=pfy1,arg1=e2121653
      ===========================================================
      - generate job to restore original record sizes
      - driven by the control-file & original rec-size saved at end record

sample job to restore original record sizes

 # e2121653.mtaxe.piscines - convert back to original recsize & load typ=IDXf1 indexed file
 opr='e2121653.mtaxe.piscines - convert to original recsize & load IDXf1 indexed file'
 #ctlfile: e2121653.mtaxe.piscines cpy=e212159b rca=00082 rcm=00082 rcf=00128 key=000(020)
 was=a33000b33000
 fili1=?d2asc/e2121653.mtaxe.piscines,rcs=00128,typ=RSF
 filo1=?d3asc/e2121653.mtaxe.piscines,rcs=00082,typ=IDXf1,isk1=000(020)
 @run
        opn    all
        sxo    00082,'000(020)'
 loop1  get    fili1,a0             get fixed lth input
        skp>   sort
        sxp    a0(00082)            put to sort
        skp    loop1
 sort   sxs                         execute sort
 loop2  sxg    b0(00082)            get record from sort
        skp>   eof
        put    filo1,b0(00082)      write record with orig recsize
        skp    loop2
 eof    cls    all
        eoj

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2C1. Converting Mainframe DATA to unix/linux/windows

Executing 1 job to convert 1 data file - Illustrated

 MF--->d0------>d0ebc------->d1ebc---------->d2asc----->d3asc----->$TESTDATA/mstr
 #1 FTP #2 copy  #3 var->fix #4 EBCDIC->ASCII  #5 fix->var #6 copy
        rename   uvdata41       uvdata42/44       uvdata45
        scripts    pfv2/...       pfx3/...          pfy2/...


 #1a. Login as appsadm or as yourself
 #1b. cdc ---> $CNVDATA      ('cdc' is an alias in common_profile)


 #2. cp d0/e2121653.mtaxe.piscines d0ebc
     ===================================
     - copy the datafile to alt subdir for filename changes


 #3a. renameL d0ebc
      =============
      - translate e2121653.mtaxe.piscines to lower case
      - renameL can be used for 1 file or all files in directory


 #3b. rename-X d0ebc .clus.vseprd
      ===========================
      - remove unwanted suffixes (.clus.vseprd, .clus.vsedev, etc)
      - do not apply to unix, do not need test/production indicators
      - test/prod provided by differences in profiles for prgrmrs/oprtrs
        with no need to have different filenames (or JCL/scripts)
      - rename-X can be used for 1 file or all files in directory

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2C2. Executing jobs to convert simple datafiles (illustrated)

uvhd of original EBCDIC datafile


 uvhd d0ebc/e2121653.mtaxe.piscines r84a
 =======================================

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123

           0 .R015     5  14   4   ..r..*.r...fS.r&......H7K3E2. .....r.
             05FFF44444F44FF444F4441091350923985295200000CFDFCF24000009144444
             0201500000500140004000C1900C1901C63590C0000C872352C00C0C19C00000
          64
             44444444444444444444
             00000000000000000000

Records are in variable length format with 2 byte binary prefixes. Note 1st 2 bytes are x'0052' = 5*16+2 = 82 bytes (data length) + 2 byte prefix = 84 bytes. Don't be confused by the 'R'- just a coincidence that x'52' part of binary recsize is an ASCII 'R'.
Although in variable length format, all records in this file are the same size (84 bytes total).
uvhd options 'r84a' specify recsize & option 'a' to translate the character line to ASCII (zones & digits show the EBCDIC values).

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2C3. Executing jobs to convert simple datafiles (illustrated)

Executing job to convert FTP varlth to fixlth before conversion


 #4. uvcopy pfv2/e2121653.mtaxe.piscines
     ===================================
     - execute 1 uvcopy jobs to convert 1 datafile from variable to fixed
     - copies the file from d0ebc/... to d1ebc/...
     - dropping the 2 byte binary prefix on varlth records
     - padding records with x'00's to a fixed length spcfd in the ctlfile
     - inserting recsize info string: <1=recsz<2=maxsz<3=fixsz<K=dsp(lth)
       in the space between record maxsz & fixsz spcfd in control file

uvhd of EBCDIC datafile - AFTER varlth to fixlth


 uvhd d1ebc/e2121653.mtaxe.piscines r128a
 =======================================

 uvhd filename=/home5/cnvdata/d1ebc/e2121653.mtaxe.piscines
 options=r128a lastmod=2007020615 today=200702101733 print=p3
 rec#=1 rcount=263 filesize=33664 recsize=128 fsize%rsize(remainder)=0

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123

           0 015     5  14   4   ..r..*.r...fS.r&......H7K3E2. .....r.
             FFF44444F44FF444F4441091350923985295200000CFDFCF2400000914444444
             01500000500140004000C1900C1901C63590C0000C872352C00C0C19C0000000

          64                   <1=00082<2=00082<3=00128<K=000(020)...........
             4444444444444444444F7FFFFF4F7FFFFF4F7FFFFF4D7FFF4FFF500000000000
             000000000000000000C1E00082C2E00082C3E00128C2E000D020D00000000000

The variable length recsize prefixes have been dropped off. The x'0052' (on prior page) is gone & 1st 3 bytes are now '015' or x'F0F1F5' in EBCDIC.
The pfv2 job converted the records to fixed size 128 bytes for the conversion process (will later be converted back to 82 bytes).
A 'record info string' has been inserted to assist conversions. '<1=recsz<2=maxsz<3=fixsz<K=dsp(lth)'. '<1=' is the size of this record & could vary from record to record (but all 00082 in this file).

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2C4. Executing jobs to convert simple datafiles (illustrated)

Executing job to convert EBCDIC to ASCII preserving packed


 #5. uvcopy pfx3/e2121653.mtaxe.piscines
     ===================================
     - execute 1 uvcopy job to convert 1 datafile
     - copies the datafile from d1ebc/... to d2asc/...
     - translating char fields from EBCDIC to ASCII
     - preserving any packed/binary fields (same as on mainframe)

uvhd - AFTER convert to ASCII (packed preserved)


 uvhd d2asc/e2121653.mtaxe.piscines r128
 =======================================

 uvhd filename=/home5/cnvdata/d2asc/e2121653.mtaxe.piscines
 options=r128 lastmod=2007020615 today=200702101732 iprint=i3
 rec#=1 rcount=263 filesize=33664 recsize=128 fsize%rsize(remainder)=0

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123

           0 015     5  14   4   ....0\.. 1..S%.P,.....H7K3E2, .......
             3332222232233222322210913509239852952000004343432200000912222222
             01500000500140004000C1900C1901C63590C0000C87B352C00C0C19C0000000
          64                   <1=00082<2=00082<3=00128<K=000(020)...........
             2222222222222222223333333333333333333333333433332333200000000000
             000000000000000000C1D00082C2D00082C3D00128CBD0008020900000000000

                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123

         128 015     6  48   1   ....0\..Q ..T.%E\.....H7K2K5.C......l
             3332222232233222322210913500521851245000004343431400000062222222
             01500000600480001000C1900C2010C54255C0000C87B2B5C30C0C20C0000000
          64                   <1=00082<2=00082<3=00128<K=000(020)...........
             2222222222222222223333333333333333333333333433332333200000000000
             000000000000000000C1D00082C2D00082C3D00128CBD0008020900000000000

The pfx3 job has translated to ASCII (except packed fields). Note that the 1st byte '0' is now x'30' (ASCII) - compared to the prior uvhd of x'F0' (EBCDIC).

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2C5. Executing jobs to convert simple datafiles (illustrated)

execute job to restore original record size after conversion


 #6. uxcopy pfy2/e2121653.mtaxe.piscines
     ===================================
     - execute 1 uxcopy job to restore original record sizes

uvhd - AFTER restoring original record size


 uvhd d3asc/e2121653.mtaxe.piscines r83
 ======================================

 uvhd filename=/home5/laval4/cnvdata/d3asc/e2121653.mtaxe.piscines.dat
 options= lastmod=2007050819 today=20070509214108 print=p2
 rec#=1 rcount=263 filesize=21829 recsize=83 fsize%rsize(remainder)=0
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 015     5  14   4   ....0\.. 1..S%.P,.....H7K3E2, .......
             3332222232233222322210913509239852952000004343432200000912222222
             01500000500140004000C1900C1901C63590C0000C87B352C00C0C19C0000000
          64                   .
             2222222222222222220
             000000000000000000A

                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
          83 015     6  48   1   ....0\..Q ..T.%E\.....H7K2K5.C......l
             3332222232233222322210913500521851245000004343431400000062222222
             01500000600480001000C1900C2010C54255C0000C87B2B5C30C0C20C0000000
          64                   .
             2222222222222222220
             000000000000000000A

Note

compare this to the uvhd d2asc/... shown on the previous page
we have dropped '<1=00082<2=00082<3=00128<K=000(020)' at end each record
records are now logically 82 bytes, but the x'0A' ISAM flag makes it 83
this is a fixed record length file, so file type is IDXf1
typ=IDXF8v files are interesting since recsize can vary record to record

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

Part 2B Converting Mainframe DATA to unix/linux/windows

generating jobs to convert Multi-Record-Type files

Part 3 will illustrate the generation & execution of a uvcopy job to convert a datafile with multi-record-types. Part 2 has already illustrated the generation & execution for a single record type file, so we will omit steps that are the same as for Part 2.

2D1.	Introduction & Overview

2E1.	sample control file to guide uvcopy job generation

2E2.	generate job to convert variable length to fixed length (for conversion)
	- omitted since same as '2E2'

2E3.	convert copybook to record layout 'cobmap' (from cpys/... to maps/...)

2E4.	convert cobmap to uvcopy skeleton job (from maps/... to pfx1/...)
	- skeleton jobs use copybookname for modulename & I/O datafilenames

2E5.	extract instructions from uvcopy job (from pfx1/... to pfx1a/...)
	- in case required by other uvcopy jobs
	- omitted since same as '2E5'

2E6.	insert datafilenames into skeleton jobs (vs copybook names)
	- guided by control file relating datafilenames to copybooknames
	- edit the job to insert record type testing instructions
	to skip to the auto generated packed field preserve instructions

2E7.	listing of completed uvcopy job with datafilenames & record type tests

Executing jobs to convert multi record type file

2F1.	Preparation to execute uvcopy conversion jobs
	- copy/rename files

2F2.	uvhd (hexdump) of sample EBCDIC datafile (in varlth format)

 2F3  convert variable length to fixed length (only for conversion phase)
      - omitted since this file was transferred from mainframe in Fixed format
      - vs Part 2A sample which was in variable format (with 2 byte prefixes)
      - even though all records were in fact the same size

2F4.	convert from EBCDIC to ASCII, preserving packed fields
	- uvhd (hexdump) of results

2F5.	copy converted datafile to $TESTDATA/mstr (for testing & later production)
	- converting fixlth back to varlth if required for COBOL programs
	- uvhd (hexdump) of final results

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2D1. Generating jobs to convert Multi-Record-Type files

Note	Part 2 generates 1 job at a time with illustrations it is intended to help you understand the process see Part_1 for the efficient generation of all jobs for all data files

Illustrated uvcopy job generation - Multi R/T files

Part 2A illustrated the generation & execution of a uvcopy job, that required no manual changes because the datafile did not have multiple record types.

This Part 2B will illustrate generation & execution for datafiles with multiple record types that do require insertion of record type test code to skip to the automatically generated instructions for redefined records.

Generating uvcopy job - Overview

 mainframe------>cpys--------->maps---------->pfx1---------->pfx2----------->pfx3
          #1 FTP     #2 cobmap1    #3 uvdata42    #4 uvdata44    #5 copy/edit

Data file conversion - Overview

 MF--->d0------>d0ebc------->d1ebc---------->d2asc----->d3asc----->$TESTDATA/mstr
 #1 FTP #2 copy  #3 var->fix #4 EBCDIC->ASCII  #5 fix->var #6 copy
        rename   uvdata41       uvdata42/44       uvdata45
        scripts    pfv2/...       pfx3/...          pfy2/...

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2E1. Generating jobs to convert Multi-Record-Type files

Generating job to convert a multi R/T datafile

We will illustrate the uvcopy job generation & execution using the following datafile & its corresponding copybook:

 tu.f01.e212-no.trperm  <-- datafilename used for this example
                          - contains 2 record types ('1' or '2' in 1st byte)
                          - with packed fields in different locations

 e221004b               <-- copybookname corresponding to the datafile

Part 2B assumes you have completed the pre-requisites on pages '1D1' thru '1E3' Page '1E3' prepared the control file which relates the datafilenames to the corresponding copybooknames.

sample control file ctl/ctlfile4

 e2121653.mtaxe.piscines cpy=e212159b rca=00082 rcm=00082 rcf=00128 key=000(020)
 e2121656.itaxe.facturat cpy=e212037b rca=00083 rcm=00245 rcf=00320 key=000(036)
 e2122683.itaxe.degrevem cpy=e212047b rca=00158 rcm=00265 rcf=00320 key=000(037)
 e2127546.mtaxe.donmarch cpy=e212142b rca=00100 rcm=00245 rcf=00320 key=000(020)
 tu.f01.e212-no.trperm   cpy=e221004b rca=00080 rcm=00080 rcf=00128 typ=RSF     *

Note that ctlfile4 contains entries for all datafiles to be converted & we have indicated our sample file with a '*' on the right side.

You can see that this is a fixed length file (since rca=80 & rcm=80). The 'typ=RSF' (Fixed) indicates that the format (for mainframe transfer) is also fixed, vs most other files which are transferred in variable length format even though the records are fixed length (as indicated by rca=rcm).

In future we may add illustrated samples for a true 'variable-length' datafile.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2E2. Generating jobs to convert Multi-Record-Type files

convert copybook to cobmap to guide next step


 #3. uvcopy cobmap1,fili1=cpys/e221004b,filo1=maps/e221004b
     ======================================================
     - convert the specified copybook to a record layout (cobmap)

 cobmap1  start-end bytes for cobol record fields    200702170943  pg# 0001
 cpys/e221004b                  mt-travaux-perman  RCSZ=00080  bgn  end  lth typ

 * fichier travaux permanents
  01  mt-travaux-permanents.
      05 mt-cd-dos            pic 9.                          0000 0000  001 n  1
      05 mt-exq.
              10 mt-exv       pic 99.                         0001 0002  002 n  2
              10 mt-anq       pic x.                          0003 0003  001
 *
 * r/t 1 - identified by '1' in 1st byte (mt-cd-dos)
      05 mt-rec-1.
          10 mt-cd-imp.
              15 mt-regl      pic x(6).                       0004 0009  006
              15 mt-serv      pic xx.                         0010 0011  002
          10 mt-division      pic 9.                          0012 0012  001 n  1
          10 mt-terme         pic 9999.                       0013 0016  004 n  4
          10 mt-perio         pic 99.                         0017 0018  002 n  2
          10 mt-mes-cont      pic 9(10)v999  comp-3.          0019 0025  007pn 13
          10 mt-taux          pic 9(4)v9(5)  comp-3.          0026 0030  005pn  9
          10 filler001        pic x(8).                       0031 0038  008
          10 mt-cd-anc        pic 9.                          0039 0039  001 n  1
          10 filler002        pic x(40).                      0040 0079  040
 *
 * r/t 2 - identified by '2' in 1st byte (mt-cd-dos)
      05 mt-rec-2 redefines mt-rec-1.
          10 mt-cd-imp-2.
               15 mt-regl-2   pic x(6).                       0004 0009  006
               15 mt-serv-2   pic xx.                         0010 0011  002
          10 mt-mod-2         pic x.                          0012 0012  001
          10 mt-rol-2.
               15 mt-ex2  pic 99.                             0013 0014  002 n  2
               15 filler003  pic x.                           0015 0015  001
               15 mt-qt2  pic x.                              0016 0016  001
               15 filler004  pic x(13).                       0017 0029  013
          10 mt-matricu-2     pic x(17).                      0030 0046  017
          10 mt-mesur-2       pic 9(8)v999 comp-3.            0047 0052  006pn 11
          10 filler005        pic x(27).                      0053 0079  027
 ***************** end of e221004b.cpy **********************
 *RCSZ=00080

Note	the packed fields are in different locations in R/T '1' & R/T '2' see the 'mvc' instructions generated in the uvcopy job on the next page

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2E3. Generating jobs to convert Multi-Record-Type files

Generate job to convert EBCDIC to ASCII & preserve packed


 #4. uvcopy uvdata42,fili1=maps/e221004b,filo1=pfx1/e221004b
     =======================================================
     - generate uvcopy job to convert 1 data file from EBCDIC to ASCII
     - generates a skeleton uvcopy job from the record layout (cobmap)

sample skeleton job to convert EBCDIC to ASCII

 # e221004b - uvcopy EBCDIC to ASCII, preserve packed, fix signs
 opr='e221004b - uvcopy job generated from copybook: E221004B '
 #ctlfile:
 was=a33000b33000
 fili1=?d1ebc/e221004b,rcs=00080,typ=RSF
 filo1=?d2asc/e221004b,rcs=00080,typ=RSF
 @run
        opn    all
 loop   get    fili1,a0
        skp>   eof
        mvc    b0(00080),a0       move rec to outarea before field prcsng
        tra    b0(00080)          translate entire outarea to ASCII
 #      ---                <-- insert R/T tests here for redefined records
 typ004 mvc    b19(12),a19             pn  mt-mes-cont:mt-taux
        skp    put1
 #      ---
 typ___ mvc    b47(6),a47              pn  mt-mesur-2
 put1   put    filo1,b0
        skp    loop
 eof    cls    all
        eoj

Notes re translation & packed field preservation

The entire record is translated to ASCII by the 'tra' instrn on line 11
The packed fields are preserved (same for Micro Focus as on the mainframe) by 'mvc'ing them from input to output to undo the prior translation.
Note the 'mvc's reflect the different locations of the packed fields in the 2 different record types:

    typ004 mvc    b19(12),a19        <-- preserves packed fields for R/T '1'

    typ___ mvc    b47(6),a47         <-- preserves packed fields for R/T '2'

Note that adjacent packed fields are combined to minimize the number of packed fields required. You can see from the copybook map on page '2E3' that R/T '1' had 2 packed fields at 19(7) & 26(5). Note that these are combined into 1 mvc instruction of 12 bytes.

    typ004 mvc    b19(12),a19        <-- preserves 2 adjacent packed fields

    typ004 mvc    b19(7),a19         <-- same as above (without combining)
           mvc    b26(5),a19

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2E4. Generating jobs to convert Multi-Record-Type files

complete skeleton job with datafilenames (vs copybookname)


 #6. uvcopy uvdata44,fili1=ctl/ctlfile4,fild2=pfx1,fild3=pfx2,arg1=tu.f01.e212-no.trperm
     ======================================================================
     - complete the uvcopy job with actual datafilenames & indexed file keys
     - jobs generated for control file lines matching arg1 datafilename/prefix
     - jobs in pfx2 named for datafiles in control file (vs copybooks in pfx1)


 #7. cp pfx2/tu.f01.e212-no.trperm pfx3 <-- copy uvcopy job to execution dir
     ==================================   - uvcopy jobnames same as datafilenames


 #8. vi pfx3/tu.f01.e212-no.trperm      <-- edit job as required
     =============================
     - changes required if file contains multiple record types
     - test record types & skip to instructions generated for redefined records
     - may need to read instructions for record types not redefined
     - read from pfx1a (instructions only extracted from generated jobs)

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2E5. Generating jobs to convert Multi-Record-Type files

complete job to convert EBCDIC to ASCII & preserve packed

 # tu.f01.e212-no.trperm - uvcopy EBCDIC to ASCII, preserve packed, fix signs
 opr='tu.f01.e212-no.trperm - uvcopy job generated from copybook: E221004B '
 #ctlfile: tu.f01.e212-no.trperm cpy=e221004b rca=00080 rcm=00080 rcf=00128 typ=RSF
 was=a33000b33000
 fili1=?d1ebc/tu.f01.e212-no.trperm,rcs=00128,typ=RSF
 filo1=?d2asc/tu.f01.e212-no.trperm,rcs=00128,typ=RSF
 @run
        opn    all
 loop   get    fili1,a0
        skp>   eof
        mvc    b0(00128),a0       move rec to outarea before field prcsng
        tra    b0(00128)          translate entire outarea to ASCII
 #      ---                <-- insert R/T tests here for redefined records
 #Feb06/07 - R/T tests added by Owen Townsend                          <--*--
        cmc    b0(1),'1'                                               <--*--
        skp=   typ004                                                  <--*--
        cmc    b0(1),'2'                                               <--*--
        skp=   typ005                                                  <--*--
        msg    b0(80)                                                  <--*--
        msgw   'R/T col1 not 1/2 - enter to trnslt only'               <--*--
        skp    put1                                                    <--*--
 #
 typ004 mvc    b19(12),a19             pn  mt-mes-cont:mt-taux
        skp    put1
 #      ---
 typ005 mvc    b47(6),a47              pn  mt-mesur-2                  <--*--
 #
 put1   put    filo1,b0
        skp    loop
 eof    cls    all
        eoj

Notes re manually added R/T test code

We have identified the manually edited R/T testing instructions with <--*-- on the right hand side.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2F1. Executing jobs to convert Multi-Record-Type files

Note	Part 2 generates 1 job at a time with illustrations it is intended to help you understand the process see Part_1 for the efficient generation of all jobs for all data files

Executing jobs to convert 1 data file - Illustrated

 MF--->d0------>d0ebc------->d1ebc---------->d2asc----->d3asc----->$TESTDATA/mstr
 #1 FTP #2 copy  #3 var->fix #4 EBCDIC->ASCII  #5 fix->var #6 copy
        rename   uvdata41       uvdata42/44       uvdata45
        scripts    pfv2/...       pfx3/...          pfy2/...


 #1a. Login as appsadm or as yourself
 #1b. cdc ---> $CNVDATA      ('cdc' is an alias in common_profile)


 #2. cp d0/tu.f01.e212-no.trperm d0ebc
     =================================
     - copy the datafile to alt subdir for filename changes


 #3a. renameL d0ebc
      =============
      - translate tu.f01.e212-no.trperm to lower case
      - renameL can be used for 1 file or all files in directory


 #3b. rename-X d0ebc .clus.vseprd
      ===========================
      - remove unwanted suffixes (.clus.vseprd, .clus.vsedev, etc)
      - do not apply to unix, do not need test/production indicators
      - test/prod provided by differences in profiles for prgrmrs/oprtrs
        with no need to have different filenames (or JCL/scripts)
      - rename-X can be used for 1 file or all files in directory


 #4. uvhd d0ebc/tu.f01.e212-no.trperm r82a
     =====================================
     - display original EBCDIC file before conversion (for later compare)

Note	I captured the 'uvhd' printouts below using the 'p'rint command the 1st record shown is R/T '1' I searched for the first R/T '2' via --> s 0(1),'2' and printed it with --> p1 all printed records are collected in tmp/filename.yymmdd.hhmmssP

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2F2. Executing jobs to convert Multi-Record-Type files

uvhd of original EBCDIC datafile


 #4. uvhd d0ebc/tu.f01.e212-no.trperm r82a
     =====================================

---> p1 <-- uvhd command to print 1 record (writes to file tmp/...)

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 101 /08285041201320..........T.        1
             FFF46FFFFFFFFFFFFFF00000000205344444444F444444444444444444444444
             1010108285041201320000000F0934F000000001000000000000000000000000
          64
             4444444444444444
             0000000000000000

---> s 0(1),'2' <-- uvhd command to 's'earch for the first '2' in byte 0

                      10        20        30        40        50        60
 r#     3636 0123456789012345678901234567890123456789012345678901234567890123
      290800 201 L00608002015     2   2                     ...X..
             FFF4DFFFFFFFFFFF44444F444F44444444444444444444400053944444444444
             2010300608002015000002000200000000000000000000000086F00000000000
          64
             4444444444444444
             0000000000000000

---> p1 <-- uvhd command to print 1 record (appends to file tmp/...)

---> q <-- uvhd command to quit


 uvlp12 tmp/tu.f01.e212-no.trperm.0702171244P
 ============================================
 - print the file written by uvhd (collected output of all 'p'rint commands)

Notes

We have printed 1 record of each type. Note that the 1st has a '1' (EBCDIC x'F1') in the byte 0 (column 1). The 2nd type has a '2' (EBCDIC x'F2') in byte 0.
We used the uvhd search command --> s 0(1),'2' <-- to find the first record of R/T '2' - at record # 3636 within the file.
Notice the 2 packed fields in R/T '1' at 19(7) & 26(5). Notice the 1 packed field in R/T '2' at 47(6).
Relate the packed fields to the uvcopy job shown on pages '2E4' or '2E6'. The 2 packed fields are combined into 1 instruction:

    typ004 mvc    b19(12),a19             pn  mt-mes-cont:mt-taux

Also note that this file is in Fixed length format & does not have the variable length 2 byte prefix (as did the sample file in Part 2A).

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2F4. Executing jobs to convert Multi-Record-Type files

Executing job to convert EBCDIC to ASCII preserving packed


 #5. uvcopy pfx3/tu.f01.e212-no.trperm
     =================================
     - execute 1 uvcopy job to convert 1 datafile
     - copies the datafile from d1ebc/... to d2asc/...
     - translating char fields from EBCDIC to ASCII
     - preserving any packed/binary fields (same as on mainframe)

uvhd - AFTER convert to ASCII (packed preserved)


 uvhd d2asc/tu.f01.e212-no.trperm r128
 =====================================

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 101 /08285041201320........).T?        1
             3332233333333333333000000002053222222223222222222222222222222222
             1010F08285041201320000000F0934F000000001000000000000000000000000
          64                 <1=00080<2=00080<3=00128<K=000(000).............
             2222222222222222333333333333333333333333343333233320000000000000
             0000000000000000C1D00080C2D00080C3D00128CBD000800090000000000000

---> s 0(1),'2' <-- uvhd command to 's'earch for the first '2' in byte 0

                      10        20        30        40        50        60
 r#     3636 0123456789012345678901234567890123456789012345678901234567890123
      465280 201 L00608002015     2   2                     ...X6.
             3332433333333333222223222322222222222222222222200053922222222222
             2010C00608002015000002000200000000000000000000000086F00000000000
          64                 <1=00080<2=00080<3=00128<K=000(000).............
             2222222222222222333333333333333333333333343333233320000000000000
             0000000000000000C1D00080C2D00080C3D00128CBD000800090000000000000

As on '2F1' we have collected 1 sample record of each type, using the 's'earch command to find the first occurrence of R/T '2' at r# 3636.
Note that the packed fields have been preserved at 19(12) for R/T '1' & 47(6) for R/T '2'. The packed field 'zones' & 'digits' are the same as in the EBCDIC file on page '2F1'.

Do not be misled by the character line which looks different because of the 'a' option used for the EBCDIC file has translated to ASCII so we can read the character line.

For example on page '2F1' the sign byte of the 2nd packed field in R/T 1 is x'3F' which shows as a '.' on the character line because x'3F' does not translate to a printable character in ASCII.

But the ASCII record on this page shows the x'3F' as a '?' (in ASCII). IE - the values in packed fields are preserved & it is just a coincidence when those values coincide with printable characters in ASCII.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

2F5. Executing jobs to convert Multi-Record-Type files

Copying converted Data files to test directories

After data file conversion & before we can begin testing, we need to copy the converted data files from $CNVDATA/d2asc/... to $TESTDATA/mstr/... We have already made a script for each file & stored in $CNVDATA/sfload/...

These scripts will reformat/sort/load variable & indexed files as they copy records from $CNVDATA/d2asc/... to $TESTDATA/mstr/... Please see Part_4 for some samples of these uxcp/uxsort scripts.

Executing 1 script to reformat/sort/copy 1 file to test dir


 #6. sfload/script-name
     ==================
     - execute 1 script in subdir sfload/...
     - to copy 1 data file from $CNVDATA/d2asc/... to $TESTDATA/mstr/...
     - converting fixed length used for conversions back to variable length
       if appropriate (NOT for this file)
     - sorting/loading indexed files as applicable


 #6a. sfload/copyRSF_tu.f01.e212-no.trperm
      ====================================
      - copy/sort/load Fixed length Indexed file


 #7. uvhd $TESTDATA/mstr/tu.f01.e212-no.trperm r80
     =============================================

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 101 /08285041201320........).T?        1
             3332233333333333333000000002053222222223222222222222222222222222
             1010F08285041201320000000F0934F000000001000000000000000000000000
          64
             2222222222222222
             0000000000000000

---> s 0(1),'2' <-- uvhd command to 's'earch for the first '2' in byte 0

                      10        20        30        40        50        60
 r#     3636 0123456789012345678901234567890123456789012345678901234567890123
      290800 201 L00608002015     2   2                     ...X6.
             3332433333333333222223222322222222222222222222200053922222222222
             2010C00608002015000002000200000000000000000000000086F00000000000
          64
             2222222222222222
             0000000000000000

The records have been restored to the original size (80 bytes). The record info string '<1=...etc...' has been dropped off.
They would be reconverted to variable format if applicable (not this file). Note that variable format for Micro Focus COBOL is different than for the mainframe - see pages I1-I8 at uvcp.htm#I1.
The file has been copied over to $TESTDATA/mstr for testing JCL & COBOL.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

Part_3 Converting Mainframe DATA to unix/linux/windows

Illustrated uvcopy Generation - Variable Length files

3A1.	Introduction & Overview

3B1.	sample control file to guide uvcopy job generation

3B2.	generate job to convert variable length to fixed length (for conversion)

3B3.	convert copybook to record layout 'cobmap' (from cpys/... to maps/...)

3B4.	convert cobmap to uvcopy skeleton job (from maps/... to pfx1/...)
	- skeleton jobs use copybookname for modulename & I/O datafilenames

3B5.	extract instructions from uvcopy job (from pfx1/... to pfx1a/...)
	- in case required by other uvcopy jobs

3B6.	insert datafilenames into skeleton jobs (vs copybook names)
	- guided by control file relating datafilenames to copybooknames

3B7.	create table summary of Record Types in the data file
	- to assist in coding record type testing instructions

3B8.	conversion job after editing for multi-record-type tests

3B9.	generate job to restore original record sizes in variable length file
	- using record size info saved earlier at the end of each record

Illustrated uxcopy Execution - Variable length files

3C1.	Preparation to execute uvcopy conversion jobs
	- copy/rename files

3C2.	uvhd (hexdump) of sample EBCDIC datafile (in FTP varlth format)

3C3.	execute job to convert FTP variable length to fixed length
	- for conversion phase, will be restored to variable after conversion

3C4.	uvhd (hexdump) of sample EBCDIC datafile in fixlth format

3C5.	execute job to convert EBCDIC to ASCII, preserving packed fields

3C6.	uvhd (hexdump) of ASCII output file

3C7.	execute job to restore original record size & converting to variable
	- will be copied to $TESTDATA/mstr for testing

3C8.	uvhd (hexdump) of final results variable length indexed file (IDXFORMAT8)

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3A1. Illustrated conversions - Variable length multi R/T files

Note	Part 3 generates 1 job at a time with illustrations it is intended to help you understand the process see Part_1 for the efficient generation of all jobs for all data files

Generating jobs to convert Variable length Multi R/T files

Part_2 illustrated conversion job generation & execution for a simple file with fixed record lengths & only 1 record type in the file. No manual changes were required.

Part_2 also illustrated conversion job generation & execution for a file with 2 record types (still fixed record length). Some manual changes required to insert record type tests & branch to the banks of auto-generated instructions.

This Part_3 will illustrate conversion & execution for a very complex file with over 100 record types & with variable length records (IDXFORMAT8).

Generating uvcopy job - Overview

 mainframe------>cpys--------->maps---------->pfx1---------->pfx2----------->pfx3
          #1 FTP     #2 cobmap1    #3 uvdata42    #4 uvdata44    #5 copy/edit

Data file conversion - Overview

 MF--->d0------>d0ebc------->d1ebc---------->d2asc----->d3asc----->$TESTDATA/mstr
 #1 FTP #2 copy  #3 var->fix #4 EBCDIC->ASCII  #5 fix->var #6 copy
        rename   uvdata41       uvdata42/44       uvdata45
        scripts    pfv2/...       pfx3/...          pfy2/...

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3B1. Generating jobs to convert Variable length Multi R/T files

Generating job to convert 1 datafile

We will illustrate the uvcopy job generation & execution using the following datafile & its corresponding copybook:


 e2121656.itaxe.facturat  <-- datafilename used for this example
 =======================


 e212037b                 <-- the 1st copybook used to describe this file
 ========                   - over 100 different R/T's & copybooks
 e212041b                   - will show only a few of the copybooks involved
 e212042b

Part_3 assumes you have completed the pre-requisites on pages '1D1' thru '1E3' Page '1E3' prepared the control file which relates the datafilenames to the corresponding copybooknames.

sample control file ctl/ctlfile4

 e2121653.mtaxe.piscines cpy=e212159b rca=00082 rcm=00082 rcf=00128 key=000(020)
 e2121656.itaxe.facturat cpy=e212037b rca=00083 rcm=00245 rcf=00320 key=000(036)*
 e2122683.itaxe.degrevem cpy=e212047b rca=00158 rcm=00265 rcf=00320 key=000(037)
 e2127546.mtaxe.donmarch cpy=e212142b rca=00100 rcm=00245 rcf=00320 key=000(020)
 tu.f01.e212-no.trperm   cpy=e221004b rca=00080 rcm=00080 rcf=00128 typ=RSF

The control file contains entries for all datafiles that need to be converted & the '*' indicates the file selected for our sample conversion.

You can see that this is a variable length file, since rca=83 & rcm=245, but most files are transferred from the mainframe in variable length format (2 byte prefix with record size in binary) whether fixed or variable.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3B2. Generating jobs to convert Variable length Multi R/T files

Generate job to convert FTP varlth data to fixlth data

The mainframe FTP program inserts a 2 byte binary record size prefix at the begining of each record. We will remove it for conversion, but save it at the end of the record, so we can later restore original record sizes.


 #1a. Login as appsadm --> /home/appsadm
 #1b. cdc ---> $CNVDATA    ('cdc' is an alias in common_profile)


 #2a. uvcopy uvdata41,fili1=ctl/ctlfile4,fild2=pfv1,arg1=e2121656
      ===========================================================
      - generate uvcopy job to convert 1 variable length datafile to fixed length
      - uvcopy job(s) generated for lines in control file matching arg1(prefix)


 #2b. cp pfv1/e2121656.itaxe.facturat pfv2
      ====================================
      - copy generated uvcopy job to pfv2 for later execution

sample job to convert varlth to fixlth

 # e2121656.itaxe.facturat - convert var to fix before EBCDIC/ASCII convert
 #  - uvcopy job generated by $UV/pf/IBM/uvdata41 (Dec19/07 chg to typ=RDW)
 #  - drop 2 byte varlth prefix & save recsz info at end of maxsz
 #  - variable length to fixed assumed if control file not 'typ=RSF'
 was=a32768b32768
 fili1=d0ebc/e2121656.itaxe.facturat,typ=RDWz2,rcs=1024
 filo1=d1ebc/e2121656.itaxe.facturat,typ=RSF,rcs=00320
 @run
        opn    all
 loop   get    fili1,a0,x'00'      get next varlth record (init area nulls)
        skp>   eof                 (cc set > at EOF)
        mvc    b0(00320),a0        move data to output area
        mvn    $rb,00245           load rgstr 'b' with maxsz
        mvc    bb0(35),'<1=RECSZ<2=00245<3=00320<K=000(036)'  setup recsz info
        mvn    bb3(5),$rv          current recsz stored in $rv by get typ=RDW
        tre    bb0(35)             convert to EBCDIC
        put    filo1,b0            write current record to output file
        skp    loop                repeat loop until EOF
 eof    cls    all                 close files
        eoj                        end job

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3B3. Generating jobs to convert Variable length Multi R/T files

convert copybook to cobmap to guide next step


 #3. uvcopy cobmap1,fili1=cpys/e212037b,filo1=maps/e212037b
     ======================================================
     - convert the specified copybook to a record layout (cobmap)

 cobmap1  start-end bytes for cobol record fields    200702271519  pg# 0001
 cpys/e212037b                  if-facturation     RCSZ=00239  bgn  end  lth typ

 * niveaux 01 du fichier e212656  (facturation et sommaires)
 * longueur: min = (46), max = (239)
  01  if-facturation.
      05  if-cle-1.
          10  if-cd-fich          pic s9(3)    comp-3.        0000 0001  002pns 3
          10  if-cle              pic x(32).                  0002 0033  032
          10  if-occ              pic s999     comp-3.        0034 0035  002pns 3
      05  filler001               pic x(203).                 0036 0238  203
 *RCSZ=00239

  01  if-600-xx                   pic x(83).                  0000 0082  083
 *RCSZ=00083

  01  if-600-degrev-fonc          pic x(121).                 0000 0120  121
 *RCSZ=00121

  01  if-600-tp.
      05  filler002               pic x(51).                  0000 0050  051
      05  if-600-tp-nbr           pic s99      comp-3.        0051 0052  002pns 2
  05 if-600-tp-groupe pic x(31) occurs 0 to 6 depending on if 0053 0083  031
 *RCSZ=00053

  01  if-600-eau.
      05  filler003               pic x(51).                  0000 0050  051
      05  if-600-eau-nbr          pic s99      comp-3.        0051 0052  002pns 2
  05 if-600-eau-groupe pic x(15) occurs 0 to 12 depending on  0053 0067  015
 *RCSZ=00053

  01  if-600-tarif                pic x(151).                 0000 0150  151
 *RCSZ=00151

  01  if-600-99                   pic x(65).                  0000 0064  065
 *RCSZ=00065

  01  if-700-sommaire-val-mnt     pic x(66).                  0000 0065  066
 *RCSZ=00066

  01  if-700-sommaire-mnt         pic x(46).                  0000 0045  046
 *RCSZ=00046

Note	many more copybooks required to define the 100+ record types in this file uvcopy conversion instructions are auto-generated for each copybook see ahead where corresponding instrns are read into primary uvcopy job

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3B4. Generating jobs to convert Variable length Multi R/T files

Generate job to convert EBCDIC to ASCII & preserve packed


 #4. uvcopy uvdata42,fili1=maps/e212037b,filo1=pfx1/e212037b
     =======================================================
     - generate uvcopy job to convert 1 data file from EBCDIC to ASCII
     - generates a skeleton uvcopy job from the record layout (cobmap)

sample skeleton job to convert EBCDIC to ASCII

 # e212037b - uvcopy EBCDIC to ASCII, preserve packed, fix signs
 opr='e212037b - uvcopy job generated from copybook: E212037B '
 #ctlfile:
 was=a33000b33000
 fili1=?d1ebc/e212037b,rcs=00239,typ=RSF
 filo1=?d2asc/e212037b,rcs=00239,typ=RSF
 @run
        opn    all
 loop   get    fili1,a0
        skp>   eof
        mvc    b0(00239),a0       move rec to outarea before field prcsng
        tra    b0(00239)          translate entire outarea to ASCII
 #      ---                <-- insert R/T tests here for redefined records
 typ037 mvc    b0(2),a0                pns if-cd-fich
        mvc    b34(2),a34              pns if-occ
        skp    put1
 #      ---
 typ___ mvc    b51(2),a51              pns if-600-tp-nbr
        skp    put1
 #      ---
 typ___ mvc    b51(2),a51              pns if-600-eau-nbr
        skp    put1
 #      ---
 put1   put    filo1,b0
        skp    loop
 eof    cls    all
        eoj

This is the uvcopy job generated from the primary copybook for this file
There are 100+ copybooks required to describe the 100+ record types & each copybook is converted into uvcopy jobs in subdir pfx1.
Just the instructions are extracted from each job in pfx1 & stored in pf1a for easier editing into the primary job.
You must then insert record type testing instructions (where marked above) to branch to the banks of auto-generated instructions.
A partial listing of the final result is shown later on page '3B8'

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3B5. Generating jobs to convert Variable length Multi R/T files

extract instructions in case required by other jobs


 #5. uvcopyxx uvdata43 pfx1 pfx1a uop=q0i7
     =====================================
     - extract conversion instructions from all generated jobs
     - write instructions into pfx1a using same copybookname as in pfx1

sample output - extracted instructions

 # e212041b
 typ041 mvc    b0(23),a0           pns if-600-99-cd-fich-k:if-600-99-dtdepot-k
        mvc    b24(27),a24         pns if-600-99-dteff-k:if-600-99-periode-au
        mvc    b51(14),a51         pns if-600-99-cd-gl:if-600-99-mnt-total
        skp    put1

 # e212042b
 typ042 mvc    b0(2),a0            pns if-700-xx-cd-fich-k
        mvc    b14(10),a14         pns if-700-xx-c-exer-k:if-700-xx-dtdepot-k
        mvc    b25(11),a25         pns if-700-xx-genretx-k:if-700-xx-occ-k
        mvc    b36(17),a36         pns if-700-xx-nbr:if-700-xx-base-imp
        mvc    b46(20),a46         pns if-700-xx-base-imp-dec:if-700-xx-var-total
        skp    put1

We are showing only 2 of 100+ instruction banks generated from copybooks
You must edit them into the primary job as required
You must then add record type testing code to branch to the appropriate instruction bank.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3B6. Generating jobs to convert Variable length Multi R/T files

complete skeleton job with datafilenames (vs copybookname)


 #6. uvcopy uvdata44,fili1=ctl/ctlfile4,fild2=pfx1,fild3=pfx2,arg1=e2121656
     ======================================================================
     - complete the uvcopy job with actual datafilenames & indexed file keys
       (uvdata44 for variable length, uvdata46 for fixed length)
     - jobs generated for control file lines matching arg1 datafilename/prefix
     - jobs in pfx2 named for datafiles in control file (vs copybooks in pfx1)


 #7. cp pfx2/e2121656* pfx3   <-- copy uvcopy job to execution dir
     ======================     - uvcopy jobnames same as datafilenames
                                - 1st 8 characters was unique at this site


 #8. vi pfx3/e2121656*        <-- edit job if required
     =================
     - changes required if file contains multiple record types
     - test record types & skip to instructions generated for redefined records
     - may need to read instructions for record types not redefined
     - read from pfx1a (instructions only extracted from generated jobs)

complete job to convert EBCDIC to ASCII & preserve packed

 # e2121656.itaxe.facturat - uvcopy EBCDIC to ASCII, preserve packed, fix signs
 opr='e2121656.itaxe.facturat - uvcopy job generated from copybook: E212037B '
 #ctlfile: e2121656.itaxe.facturat cpy=e212037b rca=00078 rcm=00239 rcf=00320 key=000(036)
 was=a33000b33000
 fili1=?d1ebc/e2121656.itaxe.facturat,rcs=00320,typ=RSF
 filo1=?d2asc/e2121656.itaxe.facturat,rcs=00320,typ=RSF
 @run
        opn    all
 loop   get    fili1,a0
        skp>   eof
        mvc    b0(00320),a0       move rec to outarea before field prcsng
        tra    b0(00320)          translate entire outarea to ASCII
 #      ---                <-- insert R/T tests here for redefined records
 typ037 mvc    b0(2),a0                pns if-cd-fich
        mvc    b34(2),a34              pns if-occ
        skp    put1
 #      ---
 typ___ mvc    b51(2),a51              pns if-600-tp-nbr
        skp    put1
 #      ---
 typ___ mvc    b51(2),a51              pns if-600-eau-nbr
        skp    put1
 #      ---
 put1   put    filo1,b0
        skp    loop
 eof    cls    all
        eoj

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3B7. Generating jobs to convert Variable length Multi R/T files

create table summary of R/T's in data to assist coding

Since we knew there were over 100 different record types in this file, we wrote a uvcopy job to create a table summary report, which will be a big help coding the record type test instructions. See the table summary job listed on page '5A1'. The table summary report is shown below.

 stat_e2121656.itaxe.facturat2  2007/05/11_13:01:36  600 code analysis
 tbl#001 pg#001     -argument-
 line#  count    %  sys typ occ

     1   2,782   9  600 010
     2   1,920   6  600 011
     3   1,913   6  600 012
          ----- etc -----
    44     343   1  600 093
    45     602   2  600 098
    46   4,451  15  600 099

        28,216*100   *TOTAL*

 stat_e2121656.itaxe.facturat2  2007/05/11_13:01:36  700 code analysis
 tbl#002 pg#001     -argument-
 line#  count    %  sys typ cat occ

     1   4,015  11  700 000
     2   2,285   6  700 010
     3   1,169   3  700 011
          ----- etc -----
    43     214   0  700 091
    44     340   1  700 092
    45   1,005   2  700 093

        33,899*100   *TOTAL*

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3B8. Generating jobs to convert Variable length Multi R/T files

conversion job after editing for multi record types

 # e2121656.itaxe.facturat - uvcopy EBCDIC to ASCII, preserve packed, fix signs
 opr='e2121656.itaxe.facturat - uvcopy job generated from copybook: E212037B '
 #ctlfile: e2121656.itaxe.facturat cpy=e212037b rca=00078 rcm=00245 rcf=00320 key=000(036)
 was=a33000b33000
 fili1=?d1ebc/e2121656.itaxe.facturat,rcs=00320,typ=RSF
 filo1=?d2asc/e2121656.itaxe.facturat,rcs=00320,typ=RSF
 @run
         opn    all
 loop    get    fili1,a0
         skp>   eof
         mvc    b0(00320),a0       move rec to outarea before field prcsng
         tra    b0(00320)          translate entire outarea to ASCII
 #-----------------------------------------------------------------------
 #       ---                <-- insert R/T tests here for redefined records
 logic   cmn    a0(2p),600         system 600 ?
         skp=   typ600
         cmn    a0(2p),700         system 700 ?
         skp=   typ700
         skp    syserr
 sys600  cmn    a32(2p),10         type 10 ?
         skp=   typ6010
         cmn    a32(2p),11         type 11 ?
         skp=   typ6011
         --- etc --------          <-- 46 Record Types for system 600
 sys700  cmn    a32(2p),0          type 00 ?
         skp=   typ7000
         cmn    a32(2p),10         type 10 ?
         skp=   typ7011
         --- etc --------          <-- 45 Record Types for system 700
 #-----------------------------------------------------------------------
 typ6010 mvc    b0(2),a0                pns if-cd-fich
         mvc    b34(2),a34              pns if-occ
         skp    put1
 typ6011 mvc    b51(2),a51              pns if-600-tp-nbr
         skp    put1
 #
 put1    put    filo1,b0
         skp    loop
 eof     cls    all
         eoj
 #-----------------------------------------------------------------------
 #Note - many record types not included as redefines in the principal copybook
 # - code was edited in here from corresponding uvcopy jobs gen'd from copybooks
 typ7000 mvc    b0(23),a0           pns if-600-xx-cd-fich-k:if-600-xx-dtdepot-k
         mvc    b24(27),a24         pns if-600-xx-dteff-k:if-600-xx-periode-au
         mvc    b51(2),a51          pns if-600-xx-cat-av
         mvc    b54(15),a54         pns if-600-xx-basimp-av:if-600-xx-cat-ap
         mvc    b70(13),a70         pns if-600-xx-basimp-ap:if-600-xx-mntcal-ap
         skp    put1
 typ7010 mvc    b0(23),a0           pns if-600-30-cd-fich-k:if-600-30-dtdepot-k
         mvc    b24(29),a24         pns if-600-30-dteff-k:if-600-30-nbr-tp
         skp    put1

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3B9. Generating jobs to convert Variable length Multi R/T files

generate job to restore original record sizes


 #9.  uvcopy uvdata45,fili1=ctl/ctlfile4,fild2=pfy1,arg1=e2121656
      ===========================================================
      - generate job to restore original record sizes
      - driven by the control-file & original rec-size saved at end record

sample job to restore original record sizes

 # e2121656.itaxe.facturat - convert back to original recsize & load typ=IDXf8v indexed file
 opr='e2121656.itaxe.facturat - convert to original recsize & load IDXf8v indexed file'
 #ctlfile: e2121656.itaxe.facturat cpy=e212037b rca=00078 rcm=00245 rcf=00320 key=000(036)
 was=a33000b33000
 fili1=?d2asc/e2121656.itaxe.facturat,rcs=00320,typ=RSF
 filo1=?d3asc/e2121656.itaxe.facturat,rcs=00245,typ=IDXf8v36,isk1=000(036)
 @run
        opn    all
        sxo    00320,'000(036)'
 loop1  get    fili1,a0             get fixed lth input
        skp>   sort
        sxp    a0(00320)            put to sort
        skp    loop1
 sort   sxs                         execute sort
 loop2  sxg    b0(00320)            get record from sort
        skp>   eof
        scn    b0(00320),'<1='      scan for original recsize
        skp=   1
        can    '<1=... recsize not found, job cancelled'
        mvn    $rb,bx3(5)           load original recsize to rgstr b
        put    filo1,b0($rb00245)   write record with orig recsize
        skp    loop2
 eof    cls    all
        eoj

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3C1. Executing jobs to convert Variable length Multi R/T files

Note	Part 3 executes 1 job at a time with illustrations it is intended to help you understand the process see Part_1 for the efficient execution of all jobs for all data files

Executing jobs to convert a variable length IDXFORMAT8 file

 MF--->d0------>d0ebc------->d1ebc---------->d2asc----->d3asc----->$TESTDATA/mstr
 #1 FTP #2 copy  #3 var->fix #4 EBCDIC->ASCII  #5 fix->var #6 copy
        rename   uvdata41       uvdata42/44       uvdata45
        scripts    pfv2/...       pfx3/...          pfy2/...


 #1a. Login as appsadm or as yourself
 #1b. cdc ---> $CNVDATA      ('cdc' is an alias in common_profile)


 #2. cp d0/e2121656.itaxe.facturat d0ebc
     ===================================
     - copy the datafile to alt subdir for filename changes


 #3a. renameL d0ebc
      =============
      - translate e2121656.itaxe.facturat to lower case
      - renameL can be used for 1 file or all files in directory


 #3b. rename-X d0ebc .clus.vseprd
      ===========================
      - remove unwanted suffixes (.clus.vseprd, .clus.vsedev, etc)
      - do not apply to unix, do not need test/production indicators
      - test/prod provided by differences in profiles for prgrmrs/oprtrs
        with no need to have different filenames (or JCL/scripts)
      - rename-X can be used for 1 file or all files in directory

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3C2. Executing jobs to convert Variable length Multi R/T files

uvhd of original variable length EBCDIC datafile


 uvhd d0ebc/e2121656.itaxe.facturat r256a
 ========================================

                      10        20        30        40        50        60
 r#     1 0123456789012345678901234567890123456789012345678901234567890123
          ||<--- begin rec#1 --->
        0 .S-..r..../.hp........-..1..-....*......Q.%......./.... ........
          0560093000628710000000611F00611205000000516000000062100400000000
          030C79C1C01880C0000C2000C12000CC4C9C0C2011C0000C2013C0C0000000C0

                  end rec#1 -->||<--- begin rec#2 --->
       64 .......R..............S-..r..../.hp........-..1..-....*......Q.%
          0000000D00100000000000560093000628710000000611F00611205030000516
          0000C0C9000400C00000C030C79C1C01880C0000C2000C12000CC4C9C0C2011C

                                       end rec#2 -->||<--- begin rec#3 --->
      128 ......./.... ..............*R..............A-..r..../.hp........
          0000000621004000000000000005D00100000000000460093000628710000000
          0000C2013C0C0000000C00000C0C9000400C00000C010C79C1C01880C0000C20

                                          end rec#3 -->||<--- begin rec#4 --->
      192 -..1..-....*..r...Q.%..-..../.....&-..........S-..r...tB.g&*....
          611F006112050999005160061100621000561000000000560093007438550000
          00C12000CC4C9C9C2011C2000C2013C0010000C00000C030C79C1C42570C0000

Records are in variable length format with 2 byte binary prefixes. Note 1st 2 bytes are x'0053' = 5*16+3 = 83 bytes (data length) + 2 byte prefix = 85 bytes. Don't be confused by the 'S' just a coincidence that x'53' part of binary recsize is an ASCII 'S'.
uvhd options 'r256a' specify recsize & option 'a' to translate the character line to ASCII (zones & digits show the EBCDIC values).
We use the default recsize 256 for variable length records, knowing that we will see several records or parts of records in each 256 byte block.
Note that I have inserted '||' above the start of each variable length record (which is its record size in binary).
I have inserted <--- begin rec#... above the begining of each record data. Note that the 1st 2 bytes are x'600C' (packed field with value 600+). This is 'system code' 600 & a 2nd system code in the file is 700.
Each system code has about 45 record types found in a 2 byte packed field in bytes 32 & 33 (1st record type above is x'090C' (lined up above 34 & 35 due to the record size in the 1st 2 bytes).

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3C3. Executing jobs to convert Variable length Multi R/T files

Executing job to convert FTP varlth to fixed for conversion


 #4. uvcopy pfv2/e2121656.itaxe.facturat
     ===================================
     - execute 1 uvcopy jobs to convert 1 datafile from variable to fixed
     - copies the file from d0ebc/... to d1ebc/...
     - dropping the 2 byte binary prefix on varlth records
     - padding records with x'00's to a fixed length spcfd in the ctlfile
     - inserting recsize info string: <1=recsz<2=maxsz<3=fixsz<K=dsp(lth)
       in the space between record maxsz & fixsz spcfd in control file

Note	please see the uvhd hexdump of the output file on the next page --->

Notes re uvhd on next page

The variable length recsize prefixes have been dropped off.
The pfv2 job converted the records to fixed size 320 bytes for the conversion process.
A 'record info string' has been inserted to assist conversions. '<1=recsz<2=maxsz<3=fixsz<K=dsp(lth)'. '<1=.....' is the size of each record & varies from record to record, as you can see in the uvhd on the next page --->
After conversion the 'record info' is used to restore each record to its original size (see the pfy2/... uvcopy jobs shown further ahead).

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3C4. Executing jobs to convert Variable length Multi R/T files

uvhd of EBCDIC datafile - AFTER varlth to fixlth


 uvhd d1ebc/e2121656.itaxe.facturat r320a
 ========================================
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 -..r..../.hp........-..1..-....*......Q.%......./.... ..........
             60093000628710000000611F0061120500000051600000006210040000000000
             0C79C1C01880C0000C2000C12000CC4C9C0C2011C0000C2013C0C0000000C000
          64 .....R..........................................................
             00000D0010000000000000000000000000000000000000000000000000000000
             00C0C9000400C00000C000000000000000000000000000000000000000000000
         128 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             0000000000000000000000000000000000000000000000000000000000000000
         192 .....................................................<1=00083<2=
             000000000000000000000000000000000000000000000000000004F7FFFFF4F7
             00000000000000000000000000000000000000000000000000000C1E00083C2E
         256 00245<3=00320<K=000(036)........................................
             FFFFF4F7FFFFF4D7FFF4FFF50000000000000000000000000000000000000000
             00245C3E00320C2E000D036D0000000000000000000000000000000000000000
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
         640 -..r..../.hp........-..1..-....*..r...Q.%..-..../.....&-........
             60093000628710000000611F0061120509990051600611006210005610000000
             0C79C1C01880C0000C2000C12000CC4C9C9C2011C2000C2013C0010000C00000
          64 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             C000000000000000000000000000000000000000000000000000000000000000
         128 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             0000000000000000000000000000000000000000000000000000000000000000
         192 .....................................................<1=00065<2=
             000000000000000000000000000000000000000000000000000004F7FFFFF4F7
             00000000000000000000000000000000000000000000000000000C1E00065C2E
         256 00245<3=00320<K=000(036)........................................
             FFFFF4F7FFFFF4D7FFF4FFF50000000000000000000000000000000000000000
             00245C3E00320C2E000D036D0000000000000000000000000000000000000000
                      10        20        30        40        50        60
 r#        6 0123456789012345678901234567890123456789012345678901234567890123
        1600 -..r...tB.g&*.......-..1..-...........Q........./...%L1212700A..
             60093007438550000000611F00611100000000512000000062106DFFFFFFFC00
             0C79C1C42570C0000C2000C12000CC0C3C0C2012C0000C2013C0C31212700100
          64 ..R ...q@..<......&.L1212700B....bh...q@..<.....i..L1212700C....
             00540217700400000050DFFFFFFFC0000880217700400000600DFFFFFFFC0000
             0120C071C21C2C00370C312127002000128C071C21C2C00390C3121270030002
         128 .....q@..<......&.L12127001.....q@..q@..<......X.L12127002....-o
             130217700400000350DFFFFFFFF0000077217700400000158DFFFFFFFF000069
             36C071C21C2C00400C312127001000081C071C21C2C00078C312127002000006
         192 ...q@..<........L12127003....Er...q@..<.....k........<1=00239<2=
             0217700400000200DFFFFFFFF00004702177004000009200000004F7FFFFF4F7
             C071C21C2C00130C312127003000052C071C21C2C00025C000000C1E00239C2E
         256 00245<3=00320<K=000(036)........................................
             FFFFF4F7FFFFF4D7FFF4FFF50000000000000000000000000000000000000000
             00245C3E00320C2E000D036D0000000000000000000000000000000000000000

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3C5. Executing jobs to convert Variable length Multi R/T files

Executing job to convert EBCDIC to ASCII (preserving packed)


 #5. uvcopy pfx3/e2121656.itaxe.facturat
     ===================================
     - execute 1 uvcopy job to convert 1 datafile
     - copies the datafile from d1ebc/... to d2asc/...
     - translating char fields from EBCDIC to ASCII
     - preserving any packed/binary fields (same as on mainframe)

Note	please see the uvhd hexdump of the output file on the next page --->

Notes re uvhd on next page

The pfx3 job has translated to ASCII (except packed fields).
Note that I am showing records #1, #3,& #6 to illustrate different types.
All 3 records shown are system 600, but the secondary record type (in bytes 32-33) varies - x'090C', x'099C', x'030C' for these 3.
The 1st 2 records shown are very similar, but the 3rd record type is dramatically different.
The records are now in ASCII, but the packed fields are the same, so the difference is not as great as for all character data. You can see that the 'info string' is in ASCII since '<1' is x'3C30' whereas it was '4CF1' in the previous uvhd of EBCDIC data.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3C6. Executing jobs to convert Variable length Multi R/T files

uvhd - AFTER convert to ASCII (packed preserved)


 uvhd d2asc/e2121656.itaxe.facturat r320
 =======================================
                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 `...<...a(.p........`..1..`..,.\......Q.l.......a#... ..........
             6009300062871000000061130061120500000051600000006210020000000000
             0C79C1C01880C0000C2000C12000CC4C9C0C2011C0000C2013C0C0000000C000
          64 .....R..........................................................
             0000050010000000000000000000000000000000000000000000000000000000
             00C0C2000400C00000C000000000000000000000000000000000000000000000
         128 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             0000000000000000000000000000000000000000000000000000000000000000
         192 .....................................................<1=00083<2=
             0000000000000000000000000000000000000000000000000000033333333333
             00000000000000000000000000000000000000000000000000000C1D00083C2D
         256 00245<3=00320<K=000(036)........................................
             3333333333333343333233320000000000000000000000000000000000000000
             00245C3D00320CBD000803690000000000000000000000000000000000000000
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
         640 `...<...a(.p........`..1..`..,.\......Q.l..`....a#....P`........
             6009300062871000000061130061120509990051600611006210005610000000
             0C79C1C01880C0000C2000C12000CC4C9C9C2011C2000C2013C0010000C00000
          64 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             C000000000000000000000000000000000000000000000000000000000000000
         128 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             0000000000000000000000000000000000000000000000000000000000000000
         192 .....................................................<1=00065<2=
             0000000000000000000000000000000000000000000000000000033333333333
             00000000000000000000000000000000000000000000000000000C1D00065C2D
         256 00245<3=00320<K=000(036)........................................
             3333333333333343333233320000000000000000000000000000000000000000
             00245C3D00320CBD000803690000000000000000000000000000000000000000
                      10        20        30        40        50        60
 r#        6 0123456789012345678901234567890123456789012345678901234567890123
        1600 `...<..tB5.P\.......`..1..`...........Q.,.......a#..lL1212700A..
             6009300743855000000061130061110000000051200000006210643333333400
             0C79C1C42570C0000C2000C12000CC0C3C0C2012C0000C2013C0CC1212700100
          64 ..R@. .q|..L......P.L1212700B....... .q|..L.....i..L1212700C....
             0054021770040000005043333333400008802177004000006004333333340000
             0120C071C21C2C00370CC12127002000128C071C21C2C00390CC121270030002
         128 .6. .q|..L.....0P.L12127001.....q| .q|..L......X.L12127002....`.
             1302177004000003504333333330000077217700400000158433333333000069
             36C071C21C2C00400CC12127001000081C071C21C2C00078CC12127002000006
         192 . .q|..L.....#..L12127003....Er. .q|..L......%.......<1=00239<2=
             0217700400000200433333333000047021770040000092000000033333333333
             C071C21C2C00130CC12127003000052C071C21C2C00025C000000C1D00239C2D
         256 00245<3=00320<K=000(036)........................................
             3333333333333343333233320000000000000000000000000000000000000000
             00245C3D00320CBD000803690000000000000000000000000000000000000000

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3C7. Executing jobs to convert Variable length Multi R/T files

execute job to restore original varlth record size


 #6. uxcopy pfy2/e2121656.itaxe.facturat
     ===================================
     - execute 1 uxcopy job to restore original record sizes

Note	please see the uvhd hexdump of the output file on the next page --->

Notes re uvhd on next page

Compare this to the uvhd d2asc/... shown on the previous page Note that we specify option 'v' to tell uvhd that the file is now IDXFORMAT8 (vs the fixed length 320 byte records used for conversion)
We have dropped the 'info string' from the end of each record & restored the record sizes as specified by '<1=.....' at the end of each record.
Note that IDXFORMAT8 files have several system control records interspersed among the data records, but we only show the 1st such record. The 1st record is 1024 bytes for Indexed files, but only 128 bytes for sequential.
The system control record is identified by x'3_' in the zone of 1st byte. The remainder x'_3FE' is the record length = 1022 + 2 = 1024 total.
Data records are identified by x'4_' in the zone of 1st byte. The remainder x'_053' (on 1st data record) is the record length x'0053' = 83 + 2 = 85 total.
Note the record numbering is different due to the system record. The prior uvhd data records (1,3,6) are now (2,4,7).
Note that the record sizes for the 3 data records x'0053'(83), x'0041'(65),& x'00EF'(243) are the same s previously shown in the original EBCDIC file shown on page '3C2'.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

3C9. Executing jobs to convert Variable length Multi R/T files

uvhd - AFTER restoring original record size


 uvhd d3asc/e2121656.itaxe.facturat v   <-- option 'v' for IDXf8 (or IDXf3) files
 ====================================

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 3.......0705101546504407051015465911.>.......................$..
             3F0000003333333333333333333333333333030000000000000000000F000200
             3E00000007051015465044070510154659110E02001800001060000005000400
          64 ............................................X"8*X"8*.........u-.
             000000FA0000000000000000000000000000000000FA5232523200000000072E
             00000023000022100000000000000000000000000024828A828A0000000005D0
         128 .....u-.........................................................
             0000072E000000000000000000000000000000000000000F0000000000000000
             000005D04244010400000040000000C0000000000000003E0000000000000000
              ----- bytes 192:960 not shown for system control record -----
         960 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             0000000000000000000000000000000000000000000000000000000000000000

                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
        3320 @S`...<...a(.p........`..1..`..,.\......Q.l.......a#... ........
             4560093000628710000000611300611205000000516000000062100200000000
             030C79C1C01880C0000C2000C12000CC4C9C0C2011C0000C2013C0C0000000C0
          64 .......R................
             000000050010000000000000
             0000C0C2000400C00000C000

                      10        20        30        40        50        60
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
        3496 @A`...<...a(.p........`..1..`..,.\......Q.l..`....a#....P`......
             4460093000628710000000611300611205099900516006110062100056100000
             010C79C1C01880C0000C2000C12000CC4C9C9C2011C2000C2013C0010000C000
          64 ....
             0000
             00C0
                      10        20        30        40        50        60
 r#        7 0123456789012345678901234567890123456789012345678901234567890123
        3744 @.`...<..tB5.P\.......`..1..`...........Q.,.......a#..lL1212700A
             4E60093007438550000000611300611100000000512000000062106433333334
             0F0C79C1C42570C0000C2000C12000CC0C3C0C2012C0000C2013C0CC12127001
          64 ....R@. .q|..L......P.L1212700B....... .q|..L.....i..L1212700C..
             0000540217700400000050433333334000088021770040000060043333333400
             000120C071C21C2C00370CC12127002000128C071C21C2C00390CC1212700300
         128 ...6. .q|..L.....0P.L12127001.....q| .q|..L......X.L12127002....
             0013021770040000035043333333300000772177004000001584333333330000
             0236C071C21C2C00400CC12127001000081C071C21C2C00078CC121270020000
         192 `.. .q|..L.....#..L12127003....Er. .q|..L......%....
             6902177004000002004333333330000470217700400000920000
             06C071C21C2C00130CC12127003000052C071C21C2C00025C000

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

Part_4 Converting Mainframe DATA to unix/linux/windows

scripts to copy/sort/load data files of various types

4A1.	copyRSF_tu.f01.e212-no.trperm
	- sample script to copy data to typ=RSF (Record Sequential Fixed)
	- typ=RSF are non-indexed & have no extensions
	- typ=RSF allows files > 2 gig

4B1.	sortIDXf1_e2121653.mtaxe.piscines
	- sample script to copy data to typ=IDXf1 (Indexed Fixed length)
	- typ=IDXf1 are Indexed & have 2 parts (.dat & .idx)
	- typ=IDXf1 does NOT allow files > 2 gig

4C1.	sortIDXf8_e2121656.itaxe.facturat
	- sample script to copy data to typ=IDXF8 (Indexed Variable length)
	- typ=IDXf8 are Indexed & but have only 1 part (.dat includes index)
	- typ=IDXf8 allows files > 2 gig
	- this script creates IDXf8v varlth format, BUT records all same size
	(need uvcopy job pfy2/... to restore original size from end of record)

4D1.	uxcpIDXf8_facturat
	- sample script to copy an IDXf8v file passing thru varlth record sizes
	- copies from d3asc/... files already restored to original varlth size
	by uvcopy jobs in pfy2/...

These scripts are listed on the following pages & should provide you with examples/templates for creating scripts to copy/sort/modify other files as required.

Background re these sample scripts

These scripts are from the City of Lynn Valley conversion, May 2007 To facilitate data conversion, we converted all mainframe files to fixed length format, extending all records to carry conversion info at the end. After conversion we must restore files back to original fixed or variable.

We can easily restore 'fixed length' files back to original record size (& create indexed files) using 'uxcp/uxsort' scripts. But for the 'variable length' files we need the power of 'uxcopy' to use the saved record size from the end of the record to write out each record in its original size.

Note	As of May2007 we don't need these scripts to restore original record sizes 'uvdata45' now generates uxcopy jobs to restore original record sizes uxcopyxx 'pfy2/' now executes all generated jobs to restore rec-sizes see uvdata45 on page '1F3' & uxcopyxx 'pfy2/' on page '1G2'

Note	these scripts no longer essential, but left here as examples of using uxcp & uxsort to copy/sort/reformat variable length records

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

4A1. scripts to copy/sort/load data files of various types

sample script to create typ=RSF (non-indexed)

 # copyRSF_tu.f01.e212-no.trperm
 #  - copy file from $CNVDATA/d2asc to $TESTDATA/mstr
 #  - scripts stored in $CNVDATA/sfload, execute as follows:
 #
 #    sfload/copyRSF_tu.f01.e212-no.trperm
 #    ====================================
 # - create typ=RSF, non-indexed, no extension (vs index files)
 # - could use either 'uvcp' or 'uxcp' for typ=RSF files
 # - typ=RSF files have no extensions (vs IDXf1,3 which have .dat & .idx)
 # - this example simply reduces record size from 128 to 80
 #
 uxcp "fili1=$CNVDATA/d2asc/tu.f01.e212-no.trperm,typ=RSF,rcs=128\
 ,filo1=$RUNDATA/tape2/tu.f01.e212-no.trperm_000001,rcs=80,typ=RSF"
 #
 # ------------------- background re these sample scripts ------------------
 # - sample scripts from City of Lynn Valley conversion, May 2007
 # - uxcp/uxsort scripts in sfload/... replaced by uvcopy jobs in pfy2/...
 # - which restore original record sizes (saved at end of conversion record)
 # - sfload scripts now examples using uxcp/uxsort to copy various file types
 # - sfload scripts would work for typ=RSF & typ=IDXf1 (fixed record sizes)
 #   but not for IDXF8v (must use pfy2/... jobs to restore recsize from endrec)
 #
 # - IDXf8 indexed files allow files > 2 gig, have only 1 part (.dat),
 #   & indexes are contained within the 1 part (extnsn .dat)
 # - vs typ=IDXf1 or typ=ISF files that have 2 parts .dat & .idx
 # - typ=IDXf8 (or IDXf3) can only be created by 'uxsort' (not 'uvsort')
 # ---------------------------------------------------------------------------

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

4B1. scripts to copy/sort/load data files of various types

sample script to create typ=IDXf1 (Indexed, Fixed)

 # sortIDXf1_e2121653.mtaxe.piscines
 #  - sort&load indexed while copying from $CNVDATA/d2asc to $TESTDATA/mstr
 #  - scripts stored in $CNVDATA/sfload, execute as follows:
 #
 #    sfload/sortIDXf1_e2121653.mtaxe.piscines
 #    ========================================
 #
 # - this uxsort script creates a fixed length Indexed file
 #   typ=IDXf1 (IDXFORMAT1 vs IDXFORMAT3 or IDXFORMAT8)
 # - we sort because the EBCDIC to ASCII conversion could create out of sequence
 #   if there is mixed alpha & numerics in the key (numerics now low vs high)
 #
 uxsort "fili1=$CNVDATA/d2asc/e2121653.mtaxe.piscines,typ=RSF,rcs=128\
 ,filo1=$RUNDATA/mstr/e2121653.mtaxe.piscines,rcs=82,typ=IDXf1\
 ,isk1=0(20),key1=0(20)"
 #
 # ------------------- background re these sample scripts ------------------
 # - sample scripts from City of Lynn Valley conversion, May 2007
 # - uxcp/uxsort scripts in sfload/... replaced by uvcopy jobs in pfy2/...
 # - which restore original record sizes (saved at end of conversion record)
 # - sfload scripts now examples using uxcp/uxsort to copy various file types
 # - sfload scripts would work for typ=RSF & typ=IDXf1 (fixed record sizes)
 #   but not for IDXF8v (must use pfy2/... jobs to restore recsize from endrec)
 #
 # - IDXf8 indexed files allow files > 2 gig, have only 1 part (.dat),
 #   & indexes are contained within the 1 part (extnsn .dat)
 # - vs typ=IDXf1 or typ=ISF files that have 2 parts .dat & .idx
 # - typ=IDXf8 (or IDXf3) can only be created by 'uxsort' (not 'uvsort')
 # - typ=ISF files are the same as typ=IDXf1
 #   typ=ISF use the D-ISAM file handler (vs Micro Focus EXTFH file handler)
 #   typ=ISF files can be created by both uvsort uxsort (typ=IDX only by uxsort)
 # ---------------------------------------------------------------------------

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

4C1. scripts to copy/sort/load data files of various types

sample script to create typ=IDXf8v (but all same size)

 # sortIDXf8_e2121656.itaxe.facturat
 #  - sort&load indexed file while copying from $CNVDATA/d2asc to $TESTDATA/mstr
 #  - scripts stored in $CNVDATA/sfload, execute as follows:
 #
 #    sfload/sortIDXf8_e2121656.itaxe.facturat
 #    ========================================
 #
 # - create typ=IDXf8v (Micro Focus COBOL IDXFORMAT8 variable length Indexed)
 #
 #Note - this script will NOT restore original record size saved at end record
 #       (you must run uvcopy job pfy2/e2121656.itaxe.facturat)
 #     - this uxsort would write all records at the max size 245 (option y8)
 #     - would not run without option y8, default would pass fili1=...,rcs=320
 #       to output causing an errmsg since 320 > rcs=245 spcfd on filo1=...
 #
 uxsort "fili1=$CNVDATA/d2asc/e2121656.itaxe.facturat,typ=RSF,rcs=320\
 ,filo1=$RUNDATA/mstr/e2121656.itaxe.facturat,rcs=245,typ=IDXf8v36y8\
 ,isk1=0(36),key1=0(36)"
 #
 # ------------------- background re these sample scripts ------------------
 # - sample scripts from City of Lynn Valley conversion, May 2007
 # - uxcp/uxsort scripts in sfload/... replaced by uvcopy jobs in pfy2/...
 # - which restore original record sizes (saved at end of conversion record)
 # - sfload scripts now examples using uxcp/uxsort to copy various file types
 # - sfload scripts would work for typ=RSF & typ=IDXf1 (fixed record sizes)
 #   but not for IDXF8v (must use pfy2/... jobs to restore recsize from endrec)
 #
 # - IDXf8 indexed files allow files > 2 gig, have only 1 part (.dat),
 #   & indexes are contained within the 1 part (extnsn .dat)
 # - vs typ=IDXf1 or typ=ISF files that have 2 parts .dat & .idx
 # - typ=IDXf8 (or IDXf3) can only be created by 'uxsort' (not 'uvsort')
 # ---------------------------------------------------------------------------

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

4D1. scripts to copy/sort/load data files of various types

sample script, copy typ=IDXf8v, varlth recsize pass thru

 # uxcpIDXf8_facturat - sample uxcp to copy an indexed variable length file
 #
 # - input file d3asc/e2121656.itaxe.facturat already variable length
 #   (created by uvcopy pfy2/... job that converted d2asc/... fixed to variable)
 # - compare this script, that copies d3asc/... variable to variable
 #   to other scripts in sfload/... that copy d2asc/... fixed to variable format
 #   (but cannot restore original recsize, which requires uvcopy pfy2/... jobs)
 #
 #    sfload/uxcpIDXf8_facturat
 #    =========================
 #    - copies $CNVDATA/d3asc/e2121656.itaxe.facturat to $TESTDATA/mstr
 #
 #Note - this script is unnecessary since file already variable could use 'cp'
 #
 #    cp $CNVDATA/d3asc/e2121656.itaxe.facturat $TESTDATA/mstr
 #    ========================================================
 #
 uxcp "fili1=$CNVDATA/d3asc/e2121656.itaxe.facturat,rcs=245,typ=IDXf8v36\
 ,filo1=$RUNDATA/mstr/e2121656.itaxe.facturat,rcs=245,typ=IDXf8v36,isk1=0(36)\
 ,sel1=0(2):0x600C,mvn=0(2p):601"
 #
 #Note - since this is just a sample, we have added a 'sel1' & 'mvn' instructions
 #     - to select only record types 600 & change them to 601 in output file
 #
 # ------------------- background re these sample scripts ------------------
 # - sample scripts from City of Lynn Valley conversion, May 2007
 # - uxcp/uxsort scripts in sfload/... replaced by uvcopy jobs in pfy2/...
 # - which restore original record sizes (saved at end of conversion record)
 # - sfload scripts now examples using uxcp/uxsort to copy various file types
 # - sfload scripts would work for typ=RSF & typ=IDXf1 (fixed record sizes)
 #   but not for IDXF8v (must use pfy2/... jobs to restore recsize from endrec)
 #
 # - IDXf8 indexed files allow files > 2 gig, have only 1 part (.dat),
 #   & indexes are contained within the 1 part (extnsn .dat)
 # - vs typ=IDXf1 or typ=ISF files that have 2 parts .dat & .idx
 # - typ=IDXf8 (or IDXf3) can only be created by 'uxsort' (not 'uvsort')
 # ---------------------------------------------------------------------------

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

4E1. scripts to copy/sort/load data files of various types

uvcopy job to restore original varlth IDXf8 recsize

 # e2121656.itaxe.facturat - convert back to original recsize & load typ=IDXf8v indexed file
 opr='e2121656.itaxe.facturat - convert to original recsize & load IDXf8v indexed file'
 #ctlfile: e2121656.itaxe.facturat cpy=e212037b rca=00078 rcm=00245 rcf=00320 key=000(036)
 was=a33000b33000
 fili1=?d2asc/e2121656.itaxe.facturat,rcs=00320,typ=RSF
 filo1=?d3asc/e2121656.itaxe.facturat,rcs=00245,typ=IDXf8v36,isk1=000(036)
 @run
        opn    all
        sxo    00320,'000(036)'
 loop1  get    fili1,a0             get fixed lth input
        skp>   sort
        sxp    a0(00320)            put to sort
        skp    loop1
 sort   sxs                         execute sort
 loop2  sxg    b0(00320)            get record from sort
        skp>   eof
        scn    b0(00320),'<1='      scan for original recsize
        skp=   1
        can    '<1=... recsize not found, job cancelled'
        mvn    $rb,bx3(5)           load original recsize to rgstr b
        put    filo1,b0($rb00245)   write record with orig recsize
        skp    loop2
 eof    cls    all
        eoj

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

Part_5 Converting Mainframe DATA to unix/linux/windows

special purpose uvcopy jobs used in data file conversions

5A1.	stat_e2121656.itaxe.facturat - Record Type counts for a file
	- sample uvcopy job used to gather statistics on 'record type' codes
	- the R/T stats will help you code the instructions to test R/T's &
	skip to instruction groups automatically generated from the copybooks.

5B1.	sel1_e2121656.itaxe.facturat - select 1st record of each R/T
	- could run for EBCDIC & ASCII files to verify R/T conversions

Note	you can use these as samples/templates to create your jobs for other files

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

5A1. special purpose uvcopy jobs used in data file conversions

sample job to collect R/T statistics

 # stat_e2121656.itaxe.facturat
 # - table analysis of record types in e2121656.itaxe.facturat
 # - by Owen Townsend UVSI, Dec05/05
 #
 # uvcopy pf/stat_e2121656.itaxe.taxation,fili1=d1ebc/ie2121656.itaxe.facturat\
 # ============================================================================
 #                                      ,filo1=stats/stat_e2121656.rpt
 #                                      ==============================
 # uvcopy pf/stat_e2121656*     <-- same as above (files default as shown)
 # ========================
 #
 # uvlp12 stats/stat_e2121656*  <-- print report
 # ===========================
 #
 #          ** sample report **
 #
 # stat_e2121656  2007/01/24_09:06:50  tax-type code analysis
 # tbl#001 pg#001     -argument-
 # line#  count    %  system-code tax-type
 #     1   2,782   4  600 010
 #     2   1,920   3  600 011
 #     3   1,913   3  600 012
 #      ---------etc ----------
 #    88   1,691   2  700 067
 #    89   1,032   1  700 068
 #    90     935   1  700 069
 #        62,115*100   *TOTAL*
 #
 opr='$jobname - table analysis of record types in master file'
 rop=r1
 fili1=?d1ebc/e2121656.itaxe.facturat,typ=RSF,rcs=320
 filo1=?stats/$jobname.rpt,typ=LSTt,rcs=128
 @run
        opn     all
 #
 # begin loop to get records & build table in memory until EOF
 man20  get     fili1,a0               get next record
        skp>    man90                  (cc set > at EOF)
        clr     b0(10),' '             clear w/a for table argument
        mvn     b0(3),a0(2p)           system code
        mvn     b4(3),a32(2p)          tax type
 man30  tblt1f1 b0(10),'system-code tax-type'
        skp     man20                  return to get next
 #
 # EOF - dump table, close files,& end job
 man90  tbpt1s1 filo1,'e2121656.itaxe.facturat tax-type code analysis'
        cls     all
        eoj

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

5B1. special purpose uvcopy jobs used in data file conversions

sample job to select 1st record of each type

 # sel1_e2121656.itaxe.facturat
 # - select 1st record for each type from e2121656.itaxe.facturat
 # - also create table summary counts by system code,type,category
 # - by Owen Townsend, Jan 24/2007
 #
 # uvcopy pf/sel1_e2121656.itaxe.facturat,fili1=d1ebc/e2121656.itaxe.facturat\
 # ===========================================================================
 # ,filo2=tmp1/sel1_e2121656.itaxe.facturat\
 # =========================================
 # ,filo3=stats/sel1_e2121656.itaxe.facturat.rpt2
 # ==============================================
 #
 # uvcopy pf/sel1_e2121656*       <-- same as above (files default as shown)
 # ========================
 #
 # uvlp12 stats/sel1_e2121656*    <-- print table summary
 # ===========================
 #
 # uvhd tmp1/sel1_e2121656* r320a  <-- investigate output file (1st of each R/T)
 # ==============================
 #
 # uvsort "fili1=tmp1/sel1_e2121656*,rcs=320,typ=RSF,filo1=tmp1/sel1_e2121656*\
 # ============================================================================
 # ,key1=0(2p),key2=32(2p),key3=25(2p),key4=27(2p)"
 # ================================================
 # - sort output file to key sequence
 #
 # uvhd tmp1/sel1_e2121656_rt1s r320a  <-- investigate sorted output file
 # ==================================
 #
 # note - stats table was sorted in memory before being dumped to output file
 #
 was=d20000  # allow 20,000 bytes for table (1000 entries of 20 bytes each)
 fili1=?d1ebc/e2121656.itaxe.facturat,typ=RSF,rcs=320
 filo2=?tmp1/sel1_e2121656.itaxe.facturat,typ=RSF,rcs=320
 filo3=?stats/$jobname.rpt2,typ=LSTt,rcs=128
 @run
        opn    all                     open files
        clr    d0(20000),'~'           clear table to all tildes
 #

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

 # begin loop to get & process records until EOF
 man20  get    fili1,a0                get next record
        skp>   man90                   (cc set > at EOF)
 #
 # test system-code & setup table lookup key accordingly
 man22  clr    c0(20),' '              clear table keys format area
        mvn    c0(3),a0(2p)            unpack system-code
        cmc    c0(3),'600'             system-code 600 ?
        skp=   man30
        cmc    c0(3),'700'             system-code 700 ?
        skp=   man34
        skp    man40                   goto common table lookup
 #
 # system code 600 - append tax-type
 man30  mvn    c4(3),a32(2p)           append tax-type for 600
        skp    man40                   goto common table lookup
 #
 # system code 700 - append tax-type & category
 man34  mvn    c4(3),a25(2p)           append tax-type (for 700)
        mvn    c8(3),a27(2p)           append category
        skp    man40                   goto common table lookup
 #
 # common point to lookup table
 man40  lokz1  dd0(20),dd0(11),c0(11)  lookup table by codes formatted above
        skp=   man44
 #
 # nomatch - create new entry with count 1 & write out 1st record
 # - $rd will point to end of table (all tildes)
 man42  mvc    dd0(20),c0(11)          store new entry
        mvn    dd12(7),1               init count to 1
        put    filo2,a0                write out 1st rec of this type
        skp    man20                   return to get next record
 #
 # match - increment record count in matching table entry
 # - $rd will point to the matching table entry
 man44  add    dd12(7),1               increment count
        skp    man20                   return to get next record
 #
 # EOF - sort & dump table to output file & close files
 man90  srtn1  d0(20),d0(11),1000      sort table by keys
        wtbe   filo3,d0(20),d0(20)     dump table to outfile
        cls    all
        eoj

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

Part_6 Converting Mainframe DATA to unix/linux/windows

JCL/script conversions assisted by Data file conversions

6A1.	Overview, how LISTCAT info used to improve JCL/script conversion

6B1.	Control file samples of listcat3,& datafiles43

6C1.	Creating listcat3I from listcat2 (explanation)

6D1.	Operating Instructions to update listcat3, reload,& reconvert all JCL

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

6A1. JCL/script conversion assisted by Datafile conversion

LISTCAT info improves JCL/script conversion

JCL/script conversion is separately documented in VSEJCL.htm, but we will summarize JCL/script conversion here, because it can be enhanced by using the LISTCAT info created here in the data conversions.

Page '1E2' shows how record-sizes & indexed file key locations are extracted from the mainframe LISTCAT report into a control file (ctl/listcat2), which has 1 line per datafile with file info coded as keywords (rca=... rcm=... key=...).

We will now show you how the LISTCAT info can be used to improve the JCL/script conversions. Please refer to the 'jcl2ksh4A' script listed on page '9D1'. 'jcl2ksh4A' is intended for fast & easy reconversions of all JCL. For the initial conversion, you should use the step by step method documented in VSEJCL.htm.

Note that step#3 in jcl2ksh4A is utility job 'jcldata41', which extracts all DLBLs/TLBLs from all JCL. These are sorted & duplicates dropped resulting in 1 line per unique datafilename found in all JCL. The output is written to $TESTLIBS/ctl/datafiles41.

If required you can create ctl/datafiles42 which is sorted with datafiles41 to create indexed file datafiles43I (looked up by the JCL converter).

The JCL converter looks 1st in ctl/listcat3I, 2nd in ctl/datafiles43I for matches on datafiles to get file characteristics (recsize, keys, etc).

To help you understand the process, the next page will show you a few lines of the 2 control files.

If you are not already familiar with the subdirs in $CNVDATA & $TESTLIBS, please review them on pages '1B1' & '1B2'.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

6B1. JCL/script conversion assisted by Datafile conversion

ctl/listcat3I - file info #1 for JCL converter

listcat3I is created by transferring the mainframe LISTCAT report, processing as shown on page '1E2' (here in VSEDATA.doc) , & loading into an indexed file as shown on page 4J2 in VSEJCL.htm#4J2.

 e2121653.mtaxe.piscines        rca=00082 rcm=00082 key=000(020) typLMC=IDXf1
 e2121656.itaxe.facturat        rca=00083 rcm=00245 key=000(036) typLMC=IDXf8v
 e2121853.mtaxe.b8409d35        rca=00024 rcm=00024 key=000(011) typLW_=IDXf1
 e2122683.itaxe.degrevem        rca=00158 rcm=00265 key=000(037) typLMC=IDXf8v
 e2122941.mtaxe.permispn        rca=00045 rcm=00045 key=000(022) typLMC=IDXf1
 e2122942.mtaxe.droitmut        rca=00030 rcm=00030 key=000(011) typLMC=IDXf1
 e2123001.itaxe.banqtaxe        rca=00126 rcm=00383 key=000(036) typLMC=IDXf8v
 e2123002.itaxe.taxation        rca=00046 rcm=00383 key=000(035) typLMC=IDXf8v
 e2123003.itaxe.transdam        rca=00464 rcm=00464 key=000(000) typLMC=RSR
 e2123004.itaxe.tranjour        rca=00464 rcm=00464 key=000(000) typLW_=RSF
 e2123005.mtaxe.tpcalcul        rca=00081 rcm=00081 key=000(029) typLMC=IDXf1
 e2123086.itaxe.encapost        rca=00080 rcm=00080 key=000(000) typLW_=RSF
 e2123691.itaxe.indexenc        rca=00036 rcm=00036 key=000(036) typLMC=IDXf1
 e2123692.itaxe.encaisse        rca=00146 rcm=00426 key=000(022) typLMC=IDXf8v
 e2123693.itaxe.tranksds        rca=00168 rcm=00453 key=000(017) typLMC=IDXf8v

ctl/datafiles43I - file info #2 for JCL converter

uvcopy jcldata41 extracts all DLBLs/TLBLs & IDCAMS info
sorts & reduces to 1 line per unique filename & writes ctl/datafiles41
datafiles41 + datafiles42 sorted together & loaded to ctl/datafiles43I
you may code overrides in datafiles42, else create null file for uvsort
see procedures on page 4K2 of VSEJCL.htm#4K2.

 e1133081.tfadm.b8306j02        rca=00464 rcm=00464 key=000(000) typJWV=RSF
 e2121653.mtaxe.piscines        rca=_____ rcm=_____ key=000(000) typJWV=RSF
 e2121656.itaxe.facturat        rca=00083 rcm=00245 key=000(036) typJWV=IDXf8v
 e2122680.ttaxe.b8401d30        rca=03180 rcm=03180 key=000(000) typJWV=RSF
 e2122683.itaxe.degrevem        rca=00158 rcm=00265 key=000(037) typJWV=IDXf8v
 e2122951.mtaxe.matrenov        rca=_____ rcm=_____ key=000(000) typJWV=RSF
 e2122958.ttaxe.b8409d36        rca=00236 rcm=00236 key=000(000) typJWV=RSF
 e2123001.itaxe.banqtaxe        rca=00126 rcm=00383 key=000(036) typJWV=IDXf8v
 e2123002.itaxe.taxation        rca=_____ rcm=_____ key=000(000) typJWV=RSF
 e2123003.itaxe.transdam        rca=_____ rcm=_____ key=000(000) typJWD=RSF
 e2123004.itaxe.tranjour        rca=00464 rcm=00464 key=000(000) typJWV=RSF
 e2123005.mtaxe.tpcalcul        rca=00081 rcm=00081 key=000(029) typJWV=IDXf1
 e2123010.ttaxe.sommtrie        rca=00464 rcm=00464 key=000(000) typJWV=RSF
 e2123011.ttaxe.rembours        rca=00198 rcm=00360 key=000(000) typJWV=IDXf8v
 e2123013.ttaxe.c3001d31        rca=00360 rcm=00360 key=000(000) typJWV=RSF
 e2123013.ttaxe.rembtrie        rca=00198 rcm=00360 key=000(000) typJWV=IDXf8v
 tu.f01.e211801.adrpos          rca=_____ rcm=_____ key=000(000) typJWT=RSF
 tu.f01.e212-no.eauctr          rca=_____ rcm=_____ key=000(000) typJWT=RSF
 tu.f01.e212-no.trperm          rca=_____ rcm=_____ key=000(000) typJWT=RSF
 tu.f02.bkfaver.90-m07          rca=_____ rcm=_____ key=000(000) typJWT=RSF

The JCL converter uses the 2 Indexed files above to get file characteristics. 'listcat3I' has priority over 'datafiles43I'. The 'M' in 'typLM...' assigns these files to $MSTR/..., else to $WRK/..., but TLBL files to $TAPE/...

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

6C1. JCL/script conversion assisted by Datafile conversion

creating listcat3I from listcat2

Most of these procedures are automatic, but we provide you an opportunity to enhance the JCL conversion by manually updating the listcat2/3/3I control file to add datafiles that are missing from LISTCAT or to update file characteristics that you want to change for unix/linux.

You might only need to transfer & extract file info from LISTCAT once at the begining of the conversion, but you might update listcat3 & reload listcat3I several times to improve the conversion results.

We assume here that you have completed the procedures shown on page '1E2' to extract file info from LISTCAT. Those procedures may have already transferred listcat2 from $CNVDATA/ctl/... to $TESTLIBS/ctl/..., but we will repeat that step again in the instructions below.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

6D1. JCL/script conversion assisted by Datafile conversion

update/load listcat3I & reconvert JCL/scripts


 #1a. Login as yourself or appsadm
 #1b. cdl ---> $TESTLIBS


 #2. cp $CNVDATA/ctl/listcat2 ctl
     ============================
     - copy LISTCAT info from data conversion superdir to JCL conversion dir


 #3. cp ctl/listcat2 ctl/listcat3
     ============================
     - copy/rename to protect future edits in listcat3
     - data convert procedures copy to listcat2, never to listcat3
     - only these JCL procedures will copy listcat2 to listcat3 & edit as reqd


 #4. vi ctl/listcat3
     ===============
     - update LISTCAT info as desired
     - add files missing when LISTCAT created on mainframe
     - modify/append keyword info on existing files as desired

Note

you probably would not update on the initial conversion
but you might update several times during conversion
then rerun jcl2ksh4A to get new file info into all JCL/scripts
updates get into jcl3/... not into jcls/... (already debugged)
but early in conversion most JCL/scripts still in jcl3/...
since we copy 1 at a time from jcl3/... to jcls/... when ready to debug


 #5. uvcopy loadcat3I,fili1=ctl/listcat3,filo1=ctl/listcat3I
     =======================================================
     - load LISTCAT info into Indexed file used by JCL converter


 #5a. uvcopy loadcat3I   <-- same but easier (files default as shown above)
      ================


 #6. jcl2ksh4A jcl2 jcl3
     ===================
     - convert all JCL from jcl0/... thru several steps to scripts in jcl3/...
     - looks up listcat3I (1st) & datafiles43I (2nd) to get file recsize, etc


 #7. cp jcl3/??? jcls
     ================
     - you will copy JCL/scripts 1 at a time to jcls/... (in the PATH)
       just before you are ready to test/debug each script
     - this allows us to reconvert anytime to jcl3/... to improve conversions
       after updating listcat3 & reloading listcat3I (as shown above)

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

Part_7 Converting Mainframe DATA to unix/linux/windows

Testing aids & recommendations

7A1.	'vi' tutorial

7B1.	Unix/Linux tips for new users

7C1.	aliases to provide y/n prompts for rm, cp, mv
	& to make it easy to change to desired directories

7D1.	Providing separate datafiles for each programmer (optional)

7E1.	testdatainit - script to init output data dirs before test runs

7F1.	joblog1 - script to capture console log from 1 JCL/script

7G1.	llr - script to display output files in multi-level subdirs

7H1.	alldiff2 - script to run diff for all files in 2 subdirs
	- great for verifying JCL conversion improvements
	after fine tuning conversion control files (such as ctl/listcat3)

Note	7D1, 7E1,& 7F1 are great tips to make your JCL/script testing easy.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7A1. Testing & Debugging Aids

'vi' tutorial

'vi' 2 modes of operation - COMMAND or INSERT

COMMAND mode - cursor movement, read/write files, etc
INSERT mode - enter text (insert or replace) - ESCAPE to return to command mode

It is most important to understand that vi has TWO modes of operation. Switch to INSERT mode using commands such as the following:

i=insert, a=append, R=replace, o=open new line, cw=change word.

Switch back to command mode using 'ESCAPE'. Make it a habit to hit ESCAPE as soon as you finish text entry. If in doubt (COMMAND or INSERT mode) - hit ESCAPE, it will do no harm if already in command mode (will beep).

starting 'vi' to edit a file


 vi filename     - starts vi, reads the file,& displays 1st screen
 ===========       (or blank screen if creating a new file)

CURSOR MOVEMENT

                                 k
                                 |
                           h <-- . --> l
                                 |
                                 j

 'h'  - move cursor left
 'j'  - move cursor down
 'k'  - move cursor up
 'l'  - move cursor right

 ^d   - down 1/2 screen
 ^u   -  up  1/2 screen

 :1   - goto line# 1
 :500 - goto line# 500
 G    - goto end-of-file

 0    - move cursor to begining of current line
 $    - move cursor to end of current line

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7A2. vi editor - command summary (minimum essentials)

INSERT text (ESCAPE to end insert)

i	enter insert mode at cursor location

a	append text after cursor location
A	append text onto the end of the current line

r	replace 1 character under the cursor automatically returns to command mode (no need to ESCAPE)

R	replace (overwrite) text until ESCAPE

cw	change word (from cursor to end of word) ESCAPE to end change & adjust line to new word length

o	open (insert) a new line BELOW the cursor
O	open (insert) a new line ABOVE the cursor

DELETE commands

x	delete character (under cursor)

dd	delete current line

D	delete from the cursor to end of line

5dd - delete 5 lines (current & next 4 lines below cursor)

COPY lines (YANK & PUT)

yy	yank current line (into current buffer) move cursor to line above intended insert point
p	put (insert) buffer to line below cursor

 5yy - yank 5 lines into current buffer
     - move cursor to line above intended insert point
 p   - put (insert) buffer (5 lines) below cursor

MOVE lines (DELETE & PUT)

dd	delete current line (& store in current buffer) move cursor to line above intended insert point
p	put (insert) buffer to line below cursor

 5dd - delete 5 lines (& store in current buffer)
     - move cursor to line above intended insert point
 p   - put (insert) buffer (5 lines) below cursor

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7A3. vi editor - command summary (minimum essentials)

search commands

 /xxx - search for xxx anywhere in the file
      - search will wrap around EOF back to cursor location

n	repeat previous search (next)

substitution commands

 :1,$ s/xxx/yyy    - substitute 'xxx' with 'yyy'
                   - from 1st line to last line (1,$)

 :%s/xxx/yyy       - substitute 'xxx' with 'yyy'
                   - on ALL lines, '%' same as '1,$'

 :%s/xxx/yyy/g     - substitute 'xxx' with 'yyy'
                   - ALL lines ('%') & ALL occurrences on each line (/g)

Global command - to delete, etc

 :g/xxx/d          - on ALL lines (g=global), Delete lines with 'xxx'

 :g/xxx/p          - on ALL lines (g=global), Print lines with 'xxx'

miscellaneous commands

u	undo the last change command

U	Undo all changes to the current line (since cursor move to it)

 .  - repeat the previous change command

J	Join line below cursor onto current line

 :r file2 - read a file into the workspace
            (following the line with the cursor)

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7A4. vi editor - command summary (minimum essentials)

mark command

m[a-z]

mark current line as a,b,c,... (any 1 of 26 alphas)

ma	mark current line as mark 'a' can move cursor to some other line
mb	mark new current line as mark 'b'

 'a         - address mark 'a' (via prefix of single quote)

 'a,'b      - address a range of lines from previously marked a & b

 'a,'b m .  - move lines from mark 'a' thru mark 'b' to follow current line
            - deletes original lines

 'a,'b co . - copy lines from mark 'a' thru mark 'b' to follow current line
            - preserves original lines

WRITE & QUIT

 :w  - write workspace back to the file named on the vi entry command

 :w filename  - write workspace to a different file

 :q  - quit vi (warning if changes made without writing)

:wq - write & quit

ZZ	write & quit (same as :wq)

:q! - quit without writing

 :10,20 w filex  - write lines 10-20 to filex

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7B1. Tips for users new to Unix/Linux

tip#1 - stay in your 'working directory'

Your working directory should contain only subdirectories, and all files should reside within sub-directories. In the following example, your working directory should be 'testlibs'

 /p1
 :-----testlibs         <-- working directory
 :     :-----cbl0         - COBOL source from mainframe
 :     :-----cbls         - COBOL converted for Micro Focus
 :     :-----cblx         - compile output (.int .idy .cbl .err)
 :     :-----cpys         - COBOL copybooks
 :     :-----jcl0         - JCL from mainframe
 :     :-----jcls         - JCL converted to Unix/Linux scripts

You should work 1 level above your files & address your files through a sub- directory. In the example above, you would edit your programs as follows:

 cd /p1/testlibs        <-- change to working directory
 cdl                    <-- alias 'cdl' changes to $RUNLIBS (defined in profile)


 vi cbls/apay100.cbl    <-- address program thru subdir
 ===================

advantages of working above subdirs

Helps maintain a clean well organized workplace - Since all files are in subdirs, you can probably list all your subdirs on 1 screen, which reminds you of your subdir organization.
Protects your files, even the infamous 'rm *' would do no harm at the working directory level, since all files are in subdirs.
Garbage files (accidentally created) are highlighted & can be easily removed or moved to their proper subdir. When you list your subdirs, any files (non-subdirs) will stick out like a sore thumb.
Subdirs allow you to maintain the same filename for data & programs in the various stages of conversion, or archiving, etc.
Many operations require addressing multiple files in multiple subdirs and it is then best to work 1 level above the subdirs. For example:


    mfcbl1 car100.cbl        <-- compile 1 COBOL program
    =================          - script assumes program is in subdir 'cbls'


    cnvMF41 cbl0/CAR100.cbl  <-- convert program from cbl0->cbl1->cbl2->cbls
    =======================    - script assumes cbl1,cbl2,cbls are below you


    jcl2ksh41 jcl0/JAR100.jcl  <-- convert JCL from jcl0->jcl1->jcl2->jcl3->jcls
    ========================= - script assumes jcl1,jcl2,jcl3,jcls are below you

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7B1a. Tips for new users to Unix/Linux

example - editing & compiling programs

Normally you will only edit 1 program at a time because you will usually then compile the program & the compile script must run above the subdirs because it needs to reference source, copybooks,& compiled libraries.

It makes no sense to change into the subdir, edit 1 program,& then change back to compile the program. That would use 4 commands vs only 2 commands when you work above the subdirs.

working IN subdirs - WRONG


 #1. cd cbls                 <-- change into subdir
     =======


 #2. vi car100.cbl           <-- edit program (specify just filename)
     =============               (but overall less efficient)


 #3. cd ..                   <-- change back out (to compile program)
     =====


 #4. mfcbl1 car100.cbl       <-- compile program
     =================

working above subdirs - RIGHT


 #1. vi cbls/car100.cbl      <-- edit program (specify subdir/filename)
     ==================          (but overall more efficient)


 #2. mfcbl1 car100.cbl       <-- compile program
     =================

When you move onto the next program, your keystrokes are minimized by using your shell history & changing only the program name.


 #3. vi cbls/car200.cbl      <-- edit 2nd program
     ==================
 #4. mfcbl1 car200.cbl       <-- compile 2nd program
     =================

Exception to the Rule

When you have extensive editing to do on multiple files in the same subdir, of course you would change into the directory, do the edits,& change back.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7B2. Tips for new users to Unix/Linux

tip#2 - setup 'tmp' sub-dir in your working directory

I recommend setting up a 'tmp' subdirectory in your various working directories, and use it for any temporary files, rather than clutter up your working directory (keep your working directory clean, subdirs only).

examples of using 'tmp' subdir

Use 'tmp' whenever you want to redirect output to a temporary file so that you can then use the editor to explore the output. For example we might use 'grep' to find all programs using upsi as follows:


 #1a. grep 'UPSI' cbls/* >tmp/upsi    <-- extract lines with UPSI


 #1b. vi tmp/upsi                     <-- review results with editor


 #1c. uvlp12 tmp/upsi                 <-- print results

Editing parts of programs & JCL/scripts into other programs/scripts For example, to copy a paragraph from 1 apay100.cbl to apay200.cbl:


 #2a. vi cbls/apay100.cbl       - edit program apay100.cbl
        ma                     - mark start of desired paragraph (mark 'a')
        mb                     - mark end of desired paragraph   (mark 'b')
        :'a,'b w tmp/chkdigit  - write paragraph out to tmp subdir
                                 (from mark 'a' to mark 'b')
        :q                     - quit edit apay100


 #2b. vi cbls/apay200.cbl       - edit program apay200.cbl
        --                     - move to desired point
        :r tmp/chkdigit        - read paragraph into apay200.cbl
        :wq                    - write/quit apay200

Don't confuse our 'tmp' subdir with /tmp which is a system wide directory.

 /tmp                      - system temporary directory
 /home/joe/tmp             - Joe's temporary directory
 /p1/testlibs/tmp          - temp subdir for apay libraries

You would clean out your 'tmp' subdir at least once a day, or you could setup a 'crontab' to do this automatically.

If multiple programmers working in shared directory (such as /home/testlibs) & desire their own tmp subdir, I suggest they append their initials (tmpaa, tmpbb, etc). Cron job can still clean out via 'rm tmp*/*'.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7B3. CNVaids: Tips for new users to Unix/Linux

File Naming Conventions

It is a good idea to append a digit (1,2,3,etc) on the end of filenames. For example: 'customer1', 'sales1', etc. This provides several long lasting advantages thru out the life of our applications:

It makes our file & table names 'unique' & avoids confusion with other similar names.
We can reliably use 'grep' & 'find' to search for all instances of a desired file/table name in all our directories.
We can use 'vi' to '/' search for all instances of a desired file/table name when editing our scripts & programs.
It makes it easy to create names for related files/tables. 'customer2' or 'sales2' might be a variant, save, or backup copy. Or we might use 'customer1a', 'sales1a', etc.

Misspell filenames Intentionally

It is a good idea to intentionally misspell filenames for the same reasons as explained above (appending a digit (1,2,3) on the end of filenames). Misspelling also makes them unique & easy to find with vi,grep,find,etc. For example 'tmp' vs 'temp', 'wrk' vs 'work', etc.

common prefixes to identify related files

Common prefixes make it easy to find related files with 1 command. For example if I wanted to find all instances where my utilities are called in the scripts directory: ('uv' is common to uvcopy,uvsort,uvlist,uvqrpg,uvhd,etc).


 grep 'uv' scripts/*
 ===================


 grep 'uv' sf/*/*  (uvadm scripts are sf/adm/* sf/demo/* sf/util/* sf/IBM/*)
 ================

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7C1. Testing & Debugging Aids

Recommended Aliases in profiles

Here are the 'alias' commands extracted from the recommended profiles which are listed begining on pages 3B0 thru 3B5 of VSEJCL.htm#3B0.

 # alias UNIX commands to prompt for overwrite (highly recommended)
 # - use \rm, \mv, \cp, when you have many files & know what you are doing
 # - '\' tells UNIX to ignore the alias & use native UNIX command
 alias rm='rm -i'              # confirm removes
 alias mv='mv -i'              # confirm renames
 alias cp='cp -i'              # confirm copy overwrites
 alias l='ls -l'               # saves a lot of keystrokes
 alias cdd='cd $RUNDATA'       # quick access to data dir
 alias cdl='cd $RUNLIBS'       # quick access to libs (same as cd)
 alias cdc='cd $CNVDATA'       # quick access to data conversion superdir

The 1st 3 (rm='rm -i',etc) are recommended for unix/linux beginners & experts since it is so easy to wipe out files unintentionally. When you do have multiple files to remove (using the '*' wildcard), you can disable the prompt by preceding the command with a backslash or using option '-f'.

cd aliases to RUNDATA, RUNLIBS,& CNVDATA

These are a great convenience for Your programmers & operators, because the recommended directory design has libraries & data in different filesystems & the paths can be long & awkward if you had to key them often.

If you need to see definitions of RUNDATA,RUNLIBS,CNVDATA, please see the profiles discussed on page '1A1' & the directories discussed on page '1B1'.

 export RUNLIBS=/p1/testlibs     #<-- defs in .profile or .bash_profile
 export RUNDATA=/p1/testdata
 export CNVDATA=/p1/cnvdata
         --- OR ---
 export RUNLIBS=/p2/prodlibs
 export RUNDATA=/p2/prodlibs
 export CNVDATA=/p2/cnvdata

 alias cdd='cd $RUNDATA'         #<-- defs in common_profile
 alias cdl='cd $RUNLIBS'
 alias cdc='cd $CNVDATA'

With the above aliases in your profile, you can switch between your libraries & data with 3 character commands:

 cdd     <-- change to your data files superdir
 cdl     <-- change to your library files superdir
 cdc     <-- change to your data conversion superdir

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7D1. Testing & Debugging Aids

providing separate Datafiles for each programmer

Our profiles make it easy for you to setup a separate TESTDATA directory in your homedir. You can switch between sharing the common TESTDATA or using your own set in your home dir, simply by changing the definition of TESTDATA in your profile, and logging off & back on to make your changes effective.

See the profiles documented begining at ADMjobs.htm#1C0 We will assume that the stub_profile has already been copied to your homedir & renamed as .profile (for ksh/unix) or .bash_profile (for bash/linux).

You will need to study the profiles to fully understand this, but here are relevant lines from the stub_profile.

relevant lines from stub_profile

export RUNLIBS=/p1/testlibs export RUNDATA=/p1/testdata export CNVDATA=/p1/cnvdata # above are the stub_profile defs recommended for programmers # - operators would define RUNLIBS=/p2/prodlibs & PRODDATA=/p2/proddata # ## export TESTLIBS=$HOME/testlibs ## export TESTDATA=$HOME/testdata ## export CNVDATA=$HOME/cnvdata # above are the defs recommended for programmers who wish to have # their own set of Libraries & Data in their home directory

The '##' lines are normally commented out as shown when all programmers are sharing the LIBS & DATA directories.

You can uncomment just the '## export TESTDATA=$HOME/testdata' line & setup you own testdata superdir within your home dir as follows:


 #1. Login as yourself ---> your homedir


 #2. mkdir testdata    <-- setup a 'testdata' directory within your homedir
     ==============


 #3. cp -r $TESTDATA/* testdata
     ==========================
     - copy all subdirs from the common TESTDATA to your own testdata


 #4. vi .bash_profile     - edit your profile (.profile for ksh)
     ================
     --> uncomment the '## export TESTDATA=$HOME/testdata' line

Logoff & back on to make your profile change effective
Run your JCL/script tests as usual, all data I/O will be to the testdata directory within your homedir. The 'cdd' command will now take you there.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7E1. Testing & Debugging Aids

testdatainit - clear output subdirs before testing

You can use the 'testdatainit' script before running your JCL/script tests, to clear out any old outputs & enable you to more easily examine the outputs of your current test runs.

You can run testdatainit anytime, if you have changed your profile (as shown above) to redefine TESTDATA as a subdir in your homedir, or if you know that no other programmers are currently testing & do not need to retain any existing outputs.

You can see the 'testdatainit' script at /home/uvadm/sf/IBM/testdatainit. Here are the TESTDATA subdirs with a '0' beside the subdirs that are cleared by testdatainit.

TESTDATA subdirs cleared by testdatainit

 $TESTDATA
 :     :-----ftp         -?-
 :     :-----jobctl       0
 :     :-----joblog       0
 :     :-----jobparms1   -*- SYSIN data cards, originally embedded in JCL
                             jobparmx1 utility extracts to $TESTLIBS/jobparm1
                             dates appended to allow updates for future runs
                             jobparms1 utility copies current cards each day
                             from $TESTLIBS/jobparm1 dropping date stamps
 :     :-----jobparms2   -*- SYSIN data cards that do not need updates
 :     :-----jobtmp       0
 :     :-----mstr        -*- Master datafiles
 :     :-----rpts         0
 :     :-----sysout       0
 :     :-----tape        -?- Tape files (might clear if only outputs)
 :     :-----tmp          0
 :     :-----wrk          0


 #1. cdd                    <-- change to $TESTDATA superdir


 #2. testdatainit            <-- run the testdatainit script
     ===========            <-- will prompt y/n to proceed or not
     --> y                  <-- to proceed or 'n' to cancel


 #3. joblog1 28401d23.ksh   <-- run JCL/script being test/debugged
     ====================


 #4. llr rpts               <-- display report files (see '7G1')
     ========


 #4. llr jobtmp             <-- display temp/work/inter-step files
     ==========

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7F1. Testing & Debugging Aids

joblog1 - capture logs via 'tee'

Console logging is highly recommended (especially for production) since it captures everything that happens on the screen, including operator commands & replies to prompts. See console logging in ADMjobs.htm.

But until you get console logging activated, you could use the joblog1 script (provided in /home/uvadm/sf/IBM/...). For example:


 #1. cdd                 <-- change to $TESTDATA superdir


 #2. joblog1 jar100.ksh  <-- use joblog1 to run script jar100.ksh
     ==================      & capture console log in joblog/jar100.log


 #3. uvlp12 joblog/jar100.log  <-- print the log
     ========================

The script capture the JCL/script screen displays into a file using the unix/linux 'tee' command. The log filename is created by dropping the '.ksh' extension from the jobname & then appending '.log'.

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7F2. Testing & Debugging Aids

joblog1

 # joblog1 - run a JCL/script & capture a log file (via tee)
 #         - names the log file by dropping the .ksh & appending .log
 # - writes the logfile into subdir 'joblog' (creates it if not present)
 # - prompts for command to view/print (vi,cat,more,uvlp12,etc)
 #   &/or optionally (save) with a date/time stamp
 #Feb12/10 - reduce to 1 version (joblog1)
 #         - writes to joblog/... wherever you are
 #
 # This script intended only for test/debug when console logging not activated
 # - Console logging is better because it captures everything that happens,
 #   including operator commands & replies to prompts.
 # - to activate console logging, see: www.uvsoftware.ca/admjobs.htm#Part_6
 #
 sf="$1"     # capture the script filename with extension (jclname.ksh)
 if [[ ! -f $RUNLIBS/jcls/$sf ]]
    then
    echo "usage: joblog1 jclname.ksh [args] <-- file not found $RUNLIBS/jcls/$sf"
    echo "       ========================="
    exit; fi
 #
 # setup joblog directory pathname & make joblog subdir if not existing
 #Feb12/10 - use joblog/ wherever you are
 # ld=$RUNLIBS/joblog    # setup joblog directory pathname
 ld=joblog               # setup joblog directory pathname
 if [[ ! -d $ld ]]; then mkdir $ld; fi
 #
 # create logfilename by dropping .ksh & appending .log
 f=${sf%\.*}           # drop extension .ksh from JCL/script filename
 lf=$f.log             # add extension .log to create logfilename
 #
 $sf $2 2>&1 | tee $ld/$lf
 #========================
 echo "enter command to view, print, and/or save logfile"
 echo "logfile: $ld/$lf"
 echo "--> vi,cat,more,uvlp12,etc, and/or 'save' to date/time stamp"
 read reply
 if [[ "$reply" == *save* ]]; then
    cp $ld/$lf $ld/${lf}_$(date +%y%m%d_%H%M%S); fi
 cmd=${reply%save*}
 if [[ -n "$cmd" ]]; then $cmd $ld/$lf; fi
 exit 0

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7G1. Testing & Debugging Aids

llr - display all files in multilevel subdirs

The 'llr' script is convenient to examine the contents of multi-level subdirs, such as $RPTS, $JTMP, or the entire $TESTDATA.


 #1. Login as yourself ---> your homedir


 #2. mkdir tmp        <-- make a tmp dir if not already existing
     =========


 #3. llr > tmp/llr    <-- run the 'llr' script & capture output
     =============


 #4. uvlp12 tmp/llr   <-- print results (or vi, more, etc)
     ==============

sample llr of TESTDATA after running 1 job

 drwxrwxr-x    2 lynnv3   users        4096 Feb 13 08:43 joblog
 drwxrwxr-x    2 lynnv3   users       16384 Jan 14 14:12 jobparms1
 drwxrwxr-x    2 lynnv3   users       16384 Jan 14 14:12 jobparms2
 drwxrwxr-x    3 lynnv3   users        4096 Feb 13 08:43 jobtmp
 drwxrwxr-x    2 lynnv3   users        4096 Feb 12 11:23 mstr
 drwxrwxr-x    3 lynnv3   users        4096 Feb 13 08:43 rpts
 drwxrwxr-x    3 lynnv3   users        4096 Feb 13 08:43 sysout
 drwxrwxr-x    5 lynnv3   users        4096 Feb 13 08:08 tape
 drwxrwxr-x    2 lynnv3   users        4096 Feb 16 16:55 tmp
 drwxrwxr-x    2 lynnv3   users        4096 Feb 13 08:43 wrk

 ./joblog:
 -rw-rw-r--    1 lynnv3   users        5743 Feb 13 08:43 28401d23x.log

 ./jobparms1:
 -rw-rw-r--    1 lynnv3   users         251 Jan 14 14:12 24502R01_S0010_E173001C
 -rw-rw-r--    1 lynnv3   users         190 Jan 14 14:12 24516D03_S0010_E173001C
                    - - - - - - - etc - - - - - -
 -rw-rw-r--    1 lynnv3   users         323 Jan 14 14:12 33001R01_S0010_E173001C
 -rw-rw-r--    1 lynnv3   users         327 Jan 14 14:12 33001R02_S0010_E173001C

 ./jobtmp:
 drwxrwxr-x    2 lynnv3   users        4096 Feb 13 08:43 28401D23

 ./jobtmp/28401D23:
 -rw-rw-r--    1 lynnv3   users          80 Feb 13 08:43 jobbgn
 -rw-rw-r--    1 lynnv3   users          80 Feb 13 08:43 jobend
 -rw-rw-r--    1 lynnv3   users          70 Feb 13 08:43 jobtimes
 -rw-rw-r--    1 lynnv3   users       45220 Feb 13 08:43 S0060_E212640C

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

 ./mstr:
 -rw-rw-r--    1 lynnv3   users       21829 Feb 12 08:46 e2121653.mtaxe.piscines.dat
 -rw-rw-r--    1 lynnv3   users       10240 Feb 12 08:46 e2121653.mtaxe.piscines.idx
 -rw-rw-r--    1 lynnv3   users     3003880 Feb 12 08:46 e2123002.itaxe.taxation.dat
 -rw-rw-r--    1 lynnv3   users      108112 Feb 12 08:46 e2123003.itaxe.transdam
 -rw-rw-r--    1 lynnv3   users    25032392 Feb 12 08:46 e2125504.mtaxe.paracomp.dat
 -rw-rw-r--    1 lynnv3   users        1416 Feb 12 08:46 e2125553.ttaxe.b8401j04
 -rw-rw-r--    1 lynnv3   users         210 Feb 12 08:46 e2125554.ttaxe.b8401j04
 -rw-rw-r--    1 lynnv3   users         630 Feb 12 08:46 e2125555.ttaxe.b8401j04
 -rw-rw-r--    1 lynnv3   users      800000 Feb 12 11:23 tu.f01.e212-no.trperm

 ./rpts:
 drwxrwxr-x    2 lynnv3   users        4096 Feb 13 08:43 070213

 ./rpts/070213:
 -rw-rw-r--    1 lynnv3   users       40033 Feb 13 08:43 28401D23_S0060_E212640C_084349.lvl
 -rw-rw-r--    1 lynnv3   users        2437 Feb 13 08:43 28401D23_S0060_E212640C.hex

 ./sysout:
 drwxrwxr-x    2 lynnv3   users        4096 Feb 13 08:43 28401D23

./sysout/28401D23:

 ./tape:
 drwxrwxr-x    2 lynnv3   users        4096 Feb  8 08:02 070208
 drwxrwxr-x    2 lynnv3   users        4096 Feb 12 09:06 070212
 drwxrwxr-x    2 lynnv3   users        4096 Feb 13 08:08 070213

./tape/070213:

 ./tmp:
 ----------    1 lynnv3   users           0 Feb 13 08:43 SORTtmptest

 ./wrk:
 -rw-rw-r--    1 lynnv3   users           0 Feb 13 08:43 nullfile

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

7H1. Testing & Debugging Aids

alldiff2 - diff report for all files in 2 dirs

The 'alldiff2' script will run the unix 'diff' utility on all pairs of files in 2 subdirs. The unix 'diff' command is a marvelous thing. It shows you the differences between any 2 text files.

I recommend you use 'alldiff2' whenever you make changes to the JCL converter or the control files.

First save the existing scripts as jcl3.old, remake jcl3, rerun the converter, & then run alldiff2 to verify that the changes made are the changes you intended & nothing else has gotten screwed up.

You can use the 'newold' script to change the name & make the new subdir.


 #1a. Login as yourself or appsadm
 #1b. cdl ---> $TESTLIBS

Modify the control files to enhance conversions. See Part_6 for updating ctl/ctlfile3 & reloading ctl/ctlfile3I (used by the JCL converter). Or see ctl/jclunixop41 listed at VSEJCL.htm#3E1.


 #3. newold jcl3   <-- script to 'mv jcl3 jcl3.old' & 'mkdir jcl3'
     ===========


 #4. jclxx41 jcl2 jcl3       <-- reconvert all JCL in jcl2 to ksh in jcl3
     =================


 #5. alldiff2 jcl3.old jcl3  <-- run alldiff2 script
     ======================    - saves jcl3.dif in the tmp subdir


 #6. uvlp12 tmp/jcl3.dif     <-- print the jcl3.dif file
     ===================       - or vi, more, etc

sample jcl3.dif

Here is a sample dif file that might be the result of updating ctl/ctlfile3 & reloading ctl/ctlfile3I before reconverting the JCL.

 266c266
 < ,typ=RSF,rcs=487"
 ---
 > ,typ=IDXf8v,rcs=487"
 772c772
 < ,typ=IDXf3v104,rcs=307,isk1=0(34)"
 ---
 > ,typ=IDXf8v,rcs=307,isk1=0(34)"
 diff file# 18 - jcl3.old/... vs jcl3/28401a01.ksh

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

Part_8 Converting Mainframe DATA to unix/linux/windows

control files to guide data conversions & COBOL compiles

Control Files to guide uvcopy job generations

8A1.	LISTCAT sample from the mainframe
	- multiple lines per file, much extraneous info

8B1.	listcat2 - relevant info extracted from LISTCAT
	- 1 line per file & sorted by filename
	- filename, record sizes, key start(length), count, type

8C1.	ctlfile3 - listcat2 reformatted with cpy=________
	- for manual editing to insert corresponding copybooknames

8D1.	ctlfile4 - ctlfile after copybook names added
	- also dropping lines for files that do not need to be converted

8E1.	datafiles41,42,43,43I - control file used by JCL converter
	JCL converter uses listcat3I as primary source of file info
	& datafiles43I as secondary source if no filename match in listcat3I

Control files for Micro Focus COBOL

8P1.	contents for section 2 of Part 8.

8Q1.	Micro Focus COBOL Compile Options control file
	- supplied at /home/uvadm/ctl/cobdirectives

8R1.	File Handler Configuration file for Micro Focus COBOL
	- supplied at /home/uvadm/ctl/extfh.cfg

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

8A1. Control Files to guide uvcopy job generations

LISTCAT sample from the mainframe

Note	multiple lines per file, much extraneous info relevant info is extracted by 'catgut41' (see page '1E2')

 CLUSTER ------- E2121656.ITAXE.FACTURAT.CLUS.VSEPRD
    HISTORY
      OWNER-IDENT-------(NULL)     CREATION--------2006.347
      RELEASE----------------2     EXPIRATION------0000.000
    PROTECTION----------(NULL)
    ASSOCIATIONS
      DATA-----E2121656.ITAXE.FACTURAT.DATA.VSEPRD
      INDEX----E2121656.ITAXE.FACTURAT.INDX.VSEPRD
  DATA ------- E2121656.ITAXE.FACTURAT.DATA.VSEPRD
    HISTORY
      OWNER-IDENT-------(NULL)     CREATION--------2006.347
      RELEASE----------------2     EXPIRATION------0000.000
    PROTECTION----------(NULL)
    ASSOCIATIONS
      CLUSTER--E2121656.ITAXE.FACTURAT.CLUS.VSEPRD
    ATTRIBUTES
      KEYLEN----36  AVGLRECL--83  BUFSPACE-----14336  CISIZE--6144
      RKP--0  MAXLRECL-239  EXCPEXIT----(NULL)  CI/CA----105
      SHROPTNS(2,3)  SPEED  SUBALLOC  NOERASE  INDEXED  NOWRITECHK  NOIMBED  NOREPLICAT
      UNORDERED  NOREUSE  NONSPANNED
    STATISTICS
      REC-TOTAL-15946788  SPLITS-CI--0  EXCPS-178347
      REC-DELETED0  SPLITS-CA--0  EXTENTS----7
      REC-INSERTED-----0  FREESPACE-%CI----0  SYSTEM-TIMESTAMP:
      REC-UPDATED---6747  FREESPACE-%CA----0    2006.347   22:31:34
      REC-RETRIEVED---15757426  FREESPC-BYTES---84510720    X'BFD986FAC7F7C700'
    ALLOCATION
      SPACE-TYPECYLINDER
      SPACE-PRI440  USECLASS-PRI-----0  HI-ALLOC-RBA--1793433600
      SPACE-SEC380  USECLASS-SEC-----0  HI-USED-RBA---1708922880
    VOLUME
      VOLSERPRD314  PHYREC-SIZE---6144  HI-ALLOC-RBA--1509580800  EXTENT-NUMBER----6
      DEVTYPE-3380  PHYRECS/TRK7  HI-USED-RBA---1708922880  EXTENT-TYPE--X'00'

NOTE	many lines omitted (until begining of next CLUSTER) then lines shown ONLY where desired information is extracted CLUSTER, KEYLEN, RKP, AVGLRECL, MAXLRECL, REC-TOTAL

 CLUSTER ------- E2122940.TTAXE.B8401D57.CLUS.VSEPRD
      KEYLEN---------0   AVGLRECL----------50   BUFSPACE--------4096   CISIZE----------2048
      RKP------------0   MAXLRECL----------50   EXCPEXIT------(NULL)   CI/CA------------270
      REC-TOTAL----102   SPLITS-CI----------0   EXCPS----------8

 CLUSTER ------- E2122941.MTAXE.PERMISPN.CLUS.VSEPRD
      KEYLEN--------22   AVGLRECL----------45   BUFSPACE-------13824   CISIZE----------6144
      RKP------------0   MAXLRECL----------45   EXCPEXIT------(NULL)   CI/CA------------105
      REC-TOTAL----537   SPLITS-CI----------0   EXCPS--------250

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

8B1. Control Files to guide uvcopy job generations

listcat2 - relevant info extracted from LISTCAT

Note	LISTCAT info gathered onto 1 line per file & sorted by filename datafilename, recsize(avg), recsize(max), key start(length), count

 e1133081.tfadm.b8306j02   rca=00464 rcm=00464 key=000(000) typLM_=RSF    cntr=00001518
 e1133085.ifadm.ajusteme   rca=00080 rcm=00080 key=000(000) typLM_=RSF    cntr=00001518
 e1133086.ifadm.rembours   rca=00160 rcm=00160 key=000(000) typLM_=RSF    cntr=00000000
 e1133087.ifadm.fourniss   rca=00160 rcm=00160 key=000(000) typLM_=RSF    cntr=00000000
 e2115227.meval.dtaxbdm$   rca=00250 rcm=00250 key=000(000) typLM_=RSF    cntr=00000646
 e2119113.ieval.b4598j01   rca=00018 rcm=00018 key=000(000) typLM_=RSF    cntr=00000001
 e2119351.ieval.txdrues$   rca=00075 rcm=00250 key=000(018) typLM_=IDXf8v cntr=00005096
 e2120817.ttaxe.mutation   rca=00171 rcm=00171 key=000(000) typLM_=RSF    cntr=00000000
 e2120914.ttaxe.b8401j04   rca=01000 rcm=01000 key=000(000) typLM_=RSF    cntr=00000012
 e2121653.mtaxe.piscines   rca=00082 rcm=00082 key=000(020) typLM_=IDXf1  cntr=00033321
 e2121656.itaxe.facturat   rca=00083 rcm=00245 key=000(036) typLM_=IDXf8v cntr=15946788
 e2121853.mtaxe.b8409d35   rca=00024 rcm=00024 key=000(011) typLM_=IDXf1  cntr=00002895
 e2122683.itaxe.degrevem   rca=00158 rcm=00265 key=000(037) typLM_=IDXf8v cntr=00244227
 e2122940.ttaxe.b8401d57   rca=00050 rcm=00050 key=000(000) typLM_=RSF    cntr=00000102
 e2122941.mtaxe.permispn   rca=00045 rcm=00045 key=000(022) typLM_=IDXf1  cntr=00000537
 e2122942.mtaxe.droitmut   rca=00030 rcm=00030 key=000(011) typLM_=IDXf1  cntr=00006597
 e2122945.mtaxe.crtconst   rca=00275 rcm=00275 key=000(015) typLM_=IDXf1  cntr=00002077
 e2122951.mtaxe.matrenov   rca=00050 rcm=00050 key=000(022) typLM_=IDXf1  cntr=00090433
 e2122956.mtaxe.crtrenov   rca=00270 rcm=00270 key=000(026) typLM_=IDXf1  cntr=00015254
 e2123001.itaxe.banqtaxe   rca=00126 rcm=00383 key=000(036) typLM_=IDXf8v cntr=03036729
 e2123002.itaxe.taxation   rca=00046 rcm=00383 key=000(035) typLM_=IDXf8v cntr=00074292
 e2123004.itaxe.tranjour   rca=00464 rcm=00464 key=000(000) typLM_=RSF    cntr=00003905
 e2123005.mtaxe.tpcalcul   rca=00081 rcm=00081 key=000(029) typLM_=IDXf1  cntr=00648978
 e2123046.ttaxe.majexclu   rca=00082 rcm=00082 key=000(000) typLM_=RSF    cntr=00000000
 e2123086.itaxe.encapost   rca=00080 rcm=00080 key=000(000) typLM_=RSF    cntr=00000000
 e2123691.itaxe.indexenc   rca=00036 rcm=00036 key=000(036) typLM_=IDXf1  cntr=00001397
 e2123692.itaxe.encaisse   rca=00146 rcm=00426 key=000(022) typLM_=IDXf8v cntr=01970304
 e2123693.itaxe.tranksds   rca=00168 rcm=00453 key=000(017) typLM_=IDXf8v cntr=00000001
 e2124200.ttaxe.c0705a03   rca=00153 rcm=00153 key=001(003) typLM_=IDXf1  cntr=00000002
 e2124201.ttaxe.c0715a01   rca=00032 rcm=00032 key=001(011) typLM_=IDXf1  cntr=00018399
 e2124202.ttaxe.c0715a08   rca=00040 rcm=00040 key=000(017) typLM_=IDXf1  cntr=00186159
 e2124203.ttaxe.c0715a03   rca=00305 rcm=00305 key=001(017) typLM_=IDXf1  cntr=00186159
 e2124696.itaxe.renovati   rca=00117 rcm=00117 key=000(036) typLM_=IDXf1  cntr=00011128
 e2124777.ttaxe.b8401d31   rca=03900 rcm=03900 key=000(000) typLM_=RSF    cntr=00006659
 e2124923.itaxe.c3001d36   rca=00450 rcm=00450 key=000(000) typLM_=RSF    cntr=00015853
 e2124972.mtaxe.crehypcp   rca=00251 rcm=00251 key=000(056) typLM_=IDXf1  cntr=00218055
 e2124973.ttaxe.b8401d63   rca=00251 rcm=00251 key=000(000) typLM_=RSF    cntr=00024463
 e2124974.ttaxe.b8401d63   rca=00251 rcm=00251 key=000(000) typLM_=RSF    cntr=00000000
 e2124977.mtaxe.b8401d64   rca=00234 rcm=00234 key=000(000) typLM_=RSF    cntr=00000000
 e2124985.ttaxe.b859d064   rca=00234 rcm=00234 key=000(000) typLM_=RSF    cntr=00000371
 e2124998.ttaxe.transde1   rca=00464 rcm=00464 key=000(000) typLM_=RSF    cntr=00002946
 e2125504.mtaxe.paracomp   rca=00104 rcm=00307 key=000(034) typLM_=IDXf8v cntr=18956924
 e2125519.ttaxe.b8401a01   rca=00423 rcm=00487 key=000(000) typLM_=IDXf8v cntr=00438605
 e2125520.mtaxe.indxpara   rca=00066 rcm=00272 key=000(034) typLM_=IDXf8v cntr=03334795
 e2125521.ttaxe.b8401a01   rca=00307 rcm=00307 key=000(000) typLM_=RSF    cntr=00441101
 e2125526.ttaxe.b8401a01   rca=00423 rcm=00487 key=000(000) typLM_=IDXf8v cntr=00438605
 e2125704.itaxe.traneige   rca=00039 rcm=00039 key=000(011) typLM_=IDXf1  cntr=00056766
 e2125860.ttaxe.b8401j08   rca=00060 rcm=00060 key=000(000) typLM_=RSF    cntr=00000020
 e2126722.itaxe.saispisc   rca=00080 rcm=00080 key=000(000) typLM_=RSF    cntr=00000064
 e2126723.itaxe.saiseauc   rca=00080 rcm=00080 key=000(000) typLM_=RSF    cntr=00000000
 e2126724.itaxe.saisietp   rca=00080 rcm=00080 key=000(000) typLM_=RSF    cntr=00000000
 e2126725.itaxe.saisexcl   rca=00080 rcm=00080 key=000(000) typLM_=RSF    cntr=00000000
 e2126726.itaxe.saisfrvt   rca=00080 rcm=00080 key=000(000) typLM_=RSF    cntr=00000000
 e2126727.itaxe.saisborn   rca=00080 rcm=00080 key=000(000) typLM_=RSF    cntr=00000000
 e2126728.itaxe.adrgenie   rca=00314 rcm=00314 key=000(007) typLM_=IDXf1  cntr=00000001
 e2126729.itaxe.conresid   rca=00264 rcm=00402 key=000(011) typLM_=IDXf8v cntr=00001214
 e2126732.itaxe.saisadmi   rca=00464 rcm=00464 key=000(000) typLM_=RSF    cntr=00000000
 e2126753.mtaxe.aqullcdr   rca=00024 rcm=00024 key=000(000) typLM_=RSF    cntr=00002895
 e2126863.mtaxe.matadmis   rca=00050 rcm=00050 key=000(022) typLM_=IDXf1  cntr=00080464
 e2126864.ttaxe.b8409d42   rca=00021 rcm=00021 key=000(000) typLM_=RSF    cntr=00000924
 e2126998.ttaxe.b8401d59   rca=00021 rcm=00021 key=000(000) typLM_=RSF    cntr=00079965
 e2126998.ttaxe.matramod   rca=00022 rcm=00022 key=000(000) typLM_=RSF    cntr=00000049
 e2127546.mtaxe.donmarch   rca=00100 rcm=00245 key=000(020) typLM_=IDXf8v cntr=03299561
 e2127590.itaxe.adrposta   rca=00235 rcm=00307 key=000(000) typLM_=IDXf8v cntr=00000022
 e2127591.ttaxe.b8401a01   rca=00235 rcm=00307 key=000(000) typLM_=IDXf8v cntr=00000000
 e2127594.itaxe.majdonma   rca=00250 rcm=00250 key=000(000) typLM_=RSF    cntr=00000000
 e2127762.itaxe.tervagta   rca=00031 rcm=00031 key=000(026) typLM_=IDXf1  cntr=00011193
 e2128598.ttaxe.b8401j04   rca=00295 rcm=00295 key=000(000) typLM_=RSF    cntr=00000038
 e2129765.itaxe.factextc   rca=03900 rcm=03900 key=000(000) typLM_=RSF    cntr=00033352
 e2129772.ttaxe.b8409d02   rca=00033 rcm=00033 key=000(000) typLM_=RSF    cntr=00004535
 e2129816.itaxe.saischyp   rca=00059 rcm=00059 key=000(002) typLM_=IDXf1  cntr=00000001
 e2130112.ieval.rrtaxloc   rca=00406 rcm=00406 key=000(000) typLM_=RSF    cntr=00000808
 e2137102.ieval.dftaxloc   rca=00350 rcm=00350 key=000(000) typLM_=RSF    cntr=00000808
 e2146013.ttaxe.b8401a33   rca=00118 rcm=00118 key=000(000) typLM_=RSF    cntr=00000013

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

8C1. Control Files to guide uvcopy job generations

ctlfile3 - listcat2 reformatted for editing

Note	'cpy=________' inserted for editing with corresponding copybooknames

 e2121653.mtaxe.piscines cpy=________ rca=00082 rcm=00082 rcf=00128 key=000(020)
 e2121656.itaxe.facturat cpy=________ rca=00078 rcm=00245 rcf=00320 key=000(036)
 e2122683.itaxe.degrevem cpy=________ rca=00158 rcm=00265 rcf=00320 key=000(037)
 e2122941.mtaxe.permispn cpy=________ rca=00045 rcm=00045 rcf=00128 key=000(022)
 e2122942.mtaxe.droitmut cpy=________ rca=00030 rcm=00030 rcf=00128 key=000(011)
 e2122945.mtaxe.crtconst cpy=________ rca=00275 rcm=00275 rcf=00320 key=000(015)
 e2122951.mtaxe.matrenov cpy=________ rca=00050 rcm=00050 rcf=00128 key=000(022)
 e2122956.mtaxe.crtrenov cpy=________ rca=00270 rcm=00270 rcf=00320 key=000(026)
 e2123001.itaxe.banqtaxe cpy=________ rca=00066 rcm=00383 rcf=00448 key=000(036)
 e2123002.itaxe.taxation cpy=________ rca=00046 rcm=00383 rcf=00448 key=000(035)
 e2123005.mtaxe.tpcalcul cpy=________ rca=00081 rcm=00081 rcf=00128 key=000(029)
 e2123691.itaxe.indexenc cpy=________ rca=00036 rcm=00036 rcf=00128 key=000(036)
 e2123692.itaxe.encaisse cpy=________ rca=00300 rcm=00426 rcf=00512 key=000(022)
 e2123693.itaxe.tranksds cpy=________ rca=00168 rcm=00453 rcf=00512 key=000(017)
 e2124696.itaxe.renovati cpy=________ rca=00117 rcm=00117 rcf=00192 key=000(036)
 e2125504.mtaxe.paracomp cpy=________ rca=00104 rcm=00307 rcf=00384 key=000(034)
 e2125520.mtaxe.indxpara cpy=________ rca=00066 rcm=00272 rcf=00320 key=000(034)
 e2126863.mtaxe.matadmis cpy=________ rca=00050 rcm=00050 rcf=00128 key=000(022)
 e2127546.mtaxe.donmarch cpy=________ rca=00152 rcm=00245 rcf=00320 key=000(020)
 e2146011.mtaxe.mapaqagr cpy=________ rca=00160 rcm=00160 rcf=00256 key=000(036)

Note	ctlfile3 is created from listcat2 by utility 'ctlfile41' (page '1E2') then lines were dropped for datafiles not requiring conversion

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

8D1. Control Files to guide uvcopy job generations

ctlfile4 - ctlfile3 after sorting & editing

Note	copybook names edited into the cpy=________ slots lines dropped for files that do not need to be converted

 e2121653.mtaxe.piscines cpy=e212159b rca=00082 rcm=00082 rcf=00128 key=000(020)
 e2121656.itaxe.facturat cpy=e212037b rca=00078 rcm=00245 rcf=00320 key=000(036)
 e2122683.itaxe.degrevem cpy=e212047b rca=00158 rcm=00265 rcf=00320 key=000(037)
 e2122941.mtaxe.permispn cpy=e212941b rca=00045 rcm=00045 rcf=00128 key=000(022)
 e2122942.mtaxe.droitmut cpy=e212942b rca=00030 rcm=00030 rcf=00128 key=000(011)
 e2122945.mtaxe.crtconst cpy=e212909b rca=00275 rcm=00275 rcf=00320 key=000(015)
 e2122951.mtaxe.matrenov cpy=e212910b rca=00050 rcm=00050 rcf=00128 key=000(022)
 e2122956.mtaxe.crtrenov cpy=e212912b rca=00270 rcm=00270 rcf=00320 key=000(026)
 e2123001.itaxe.banqtaxe cpy=e212001b rca=00066 rcm=00383 rcf=00448 key=000(036)
 e2123002.itaxe.taxation cpy=e212007b rca=00046 rcm=00383 rcf=00448 key=000(035)
 e2123003.itaxe.transdam cpy=e212024b rca=00464 rcm=00464 rcf=00512 typ=RSR
 e2123005.mtaxe.tpcalcul cpy=e212025b rca=00081 rcm=00081 rcf=00128 key=000(029)
 e2123691.itaxe.indexenc cpy=e212156b rca=00036 rcm=00036 rcf=00128 key=000(036)
 e2123692.itaxe.encaisse cpy=e212055b rca=00300 rcm=00426 rcf=00512 key=000(022)
 e2123693.itaxe.tranksds cpy=e212056b rca=00168 rcm=00453 rcf=00512 key=000(017)
 e2124696.itaxe.renovati cpy=e212061b rca=00117 rcm=00117 rcf=00192 key=000(036)
 e2125504.mtaxe.paracomp cpy=e212138b rca=00104 rcm=00307 rcf=00384 key=000(034)
 e2125520.mtaxe.indxpara cpy=e212136b rca=00066 rcm=00272 rcf=00320 key=000(034)
 e2126863.mtaxe.matadmis cpy=e212863b rca=00050 rcm=00050 rcf=00128 key=000(022)
 e2127546.mtaxe.donmarch cpy=e212142b rca=00152 rcm=00245 rcf=00320 key=000(020)
 e2146011.mtaxe.mapaqagr cpy=e214011b rca=00160 rcm=00160 rcf=00256 key=000(036)
 tu.f01.e211801.adrpos   cpy=e212105b rca=00250 rcm=00307 rcf=00384 typ=IDXf8v
 tu.f01.e212-no.trperm   cpy=e221004b rca=00080 rcm=00080 rcf=00128 typ=RSF
 tu.f01.e212-no.eauctr   cpy=compteau rca=00606 rcm=00606 rcf=00640 typ=RSF

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

8E1. Control Files to guide uvcopy job generations

datafiles41,42,43 - control file used by JCL converter

Note	see VSEJCL.htm for details on how this file is created & updated the JCL converter looks 1st in listcat3I, 2nd in datafiles43I you can assign files to $MSTR/... by changing 'typJW...' to 'typJM...'

 e2121653.mtaxe.piscines        rca=_____ rcm=_____ key=000(000) typJWV=RSF    cntf=00020
 e2121656.itaxe.facturat        rca=00083 rcm=00245 key=000(036) typJWV=IDXf8v cntf=00035
 e2121657.ttaxe.b8401a01        rca=00236 rcm=00236 key=000(000) typJWV=RSF    cntf=00004
 e2121657.ttaxe.b8401a04        rca=00157 rcm=00307 key=000(000) typJWV=IDXf8v cntf=00004
 e2122683.itaxe.degrevem        rca=00158 rcm=00265 key=000(037) typJWV=IDXf8v cntf=00014
 e2122684.ttaxe.b8401a19        rca=00062 rcm=00118 key=000(023) typJWV=IDXf8v cntf=00004
 e2122938.ttaxe.c3001d46        rca=00101 rcm=00101 key=000(000) typJWV=RSF    cntf=00004
 e2122939.ttaxe.b8401d57        rca=_____ rcm=_____ key=000(000) typJW_=RSF    cntf=00001
 e2122940.ttaxe.b8401d57        rca=00050 rcm=00050 key=000(000) typJWV=RSF    cntf=00003
 e2122941.mtaxe.permispn        rca=_____ rcm=_____ key=000(000) typJWV=RSF    cntf=00006
 e2122958.ttaxe.b8409d36        rca=00236 rcm=00236 key=000(000) typJWV=RSF    cntf=00004
 e2123001.itaxe.banqtaxe        rca=00126 rcm=00383 key=000(036) typJWV=IDXf8v cntf=00033
 e2123002.itaxe.taxation        rca=_____ rcm=_____ key=000(000) typJWV=RSF    cntf=00059
 e2123003.itaxe.transdam        rca=00464 rcm=00464 key=000(000) typJMD=RSR
 e2123004.itaxe.tranjour        rca=00464 rcm=00464 key=000(000) typJWV=RSF    cntf=00030
 e2123005.mtaxe.tpcalcul        rca=00081 rcm=00081 key=000(029) typJWV=IDXf1  cntf=00011

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

8P1. Conversion Control files for Data conversion & Micro Focus COBOL

Control files for Micro Focus COBOL

8Q1.	Micro Focus COBOL Compile Options control file
	- supplied at /home/uvadm/ctl/cobdirectives

8R1.	File Handler Configuration file for Micro Focus COBOL
	- supplied at /home/uvadm/ctl/extfh.cfg

You must compile the COBOL programs with these directives & file handler configurations to be compatible with the Data Conversions.

These files are supplied in /home/uvadm/ctl/..., and you must copy them to $RUNLIBS/ctl/... to be effective, since the profiles & compile scripts use $RUNLIBS/ctl/ in the path to these files

The 'extfh.cfg' file handler configuration is critical:

IDXNAMETYPE=2

specifies .dat & .idx extensions for indexed files
IDXFORMAT8 files have only 1 part (.dat)
but the .dat part does include the index as well as data

IDXFORMAT=8

specifies the default file format for variable length
or indexed files

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

8Q1. Conversion Control files, Conversion Scripts,& Compile Scripts

COBOL Directives for Server-Express

 # cobdirectives - Directives for Micro Focus COBOL Server Express
 #               - for batch compiles on Unix/Linux
 #                 of COBOL programs migrated from an IBM VSE mainframe
 # This file is defined in the compile scripts mfcbl1 & mfcblA as follows:
 #
 # export COBOPT=$RUNLIBS/ctl/cobdirectives
 # ========================================
 #
 -C VSC2(2)
 -C CASE
 -C FILESHARE
 -C IBMCOMP
 -C INDD(SYSIN)
 -C LIST()
 # -C OUTDD(SYSOUT)   <-- disable so all displays goto console
 -C PERFORM-TYPE(OSVS)
 -C SPZERO
 -C NOSEG
 -C NOTRUNC
 -C XREF
 -C SIGN(ASCII)
 -C SOURCEFORMAT(FIXED)
 #-------------------------------------------------------------------------
 # -C LIST() causes mfcbl1 & mfcblA scripts to leave source listing in cblst
 # -C SIGN(ASCII) <-- default recommended, SIGN(EBCDIC) not recommended
 # -C OUTDD(SYSOUT) requires fixpack40.02_11 on SX 4.0 SP1
 # -C ANS85       <-- might use for MVS, use -C VSC2(2) for VSE
 # -C SETTINGS(COL3)
 # -C OSVS
 #--------------------------------------------------------------------------
 #               ** compile 'options' vs directives **
 #
 # Compile 'options' (such -a -P -x etc) cant be spcfd here
 # but rather in the compile script on the 'cob' command line
 # -a for animation (generate .int & .idy)
 # -g for native code (.gnt)
 # -x compile to executable (not required for batch, use cobrun .int)
 # -P to create the listing in subdir cblst/progname.lst
 #------------------------------------------------------------------
 #          ** File Handler Configuration vs directives **
 #
 # See extfh.cfg - File Handler Configuration file for Micro Focus COBOL
 # - stored at $RUNLIBS/ctl/extfh.cfg, sample contents might be:
 #
 # [XFH-DEFAULT]  - following options apply to all files (unless specified dif)
 #  IDXNAMETYPE=2     - '.dat' ext on data partition (& .idx on index part)
 #  IDXFORMAT=8       - create all indexed files as IDX8 (allows > 2 gig)
 #  FILEPOINTERSIZE=8 - allow IDX8 files > 256 Terrabytes (don't need)

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

8R1. Conversion Control files, Conversion Scripts,& Compile Scripts

extfh.cfg - file handler config for MF COBOL

 # extfh.cfg - File Handler Configuration file for Micro Focus COBOL
 # - see https://supportline.microfocus.com/Documentation/books/sx40sp/fh...
 # - see www.uvsoftware.ca/vsecobol.htm & www.uvsoftware.ca/mvscobol.htm
 # location of this file defined for COBOL by EXTFH, in profiles, for example:
 #        export RUNLIBS=/home/mvstest/testlibs
 #        export EXTFH=$RUNLIBS/ctl/extfh.cfg
 #
 #---------------------------------------------------------------------------
 # [XFH-DEFAULT]      - following options apply to all files
 #                    - unless over-ridden by [INTERNAL:EXTNAME] (see below)
 #  IDXNAMETYPE=2     - means data partitions have '.dat' extension
 #                    - index partitions always have '.idx' extension
 #                    - IDXFORMAT=8 files have only 1 partition
 #                      & it will have '.dat' extension if IDXNAMETYPE=2
 #                      & does include the indexes (in spite of its name)
 #
 #  IDXFORMAT=1       - create all indexed files as IDX1 (fixed lth < 2 gig)
 #  IDXFORMAT=8       - create all indexed files as IDX8 (allows > 2 gig)
 #    default         - IDXFORMAT unspecified would create fixed lth IDXFORMAT1,
 #                      & variable length as IDXFORMAT3 ? or IDXFORMAT8 ?
 #  $SET IDXFORMAT(3) - alternative way to set IDXFORMAT
 #                    - insert $SET IDX... in COBOL source ('$' in col 7)
 #
 #  FILEPOINTERSIZE=8 - allow IDX8 files > 256 Terrabytes (don't need)
 #
 #---------------------------------------------------------------------------
 # [INTERNAL:EXTNAME] - following options apply to a specific file ID by EXTNAME
 #                      on: 'select cobolFDname assign external EXTNAME'
 #                    - applies only to file defined by EXTNAME matching env-var
 #                      export EXTNAME=pathname (in JCL/script prior to cobrun)
 #                                 - - - examples - - -
 # [INTERNAL:E212853] - example for file whose EXTNAME=E212853 (select external)
 #  IDXFORMAT=1       - force IDXFORMAT1 vs any [XFH-DEFAULT] IDXFORMAT_
 # [INTERNAL:E212002] - example for file whose EXTNAME=E212002 (select external)
 #  IDXFORMAT=8       - force IDXFORMAT8 vs any [XFH-DEFAULT] IDXFORMAT_
 #
 #---------------------------------------------------------------------------
 [XFH-DEFAULT]
 IDXNAMETYPE=2
 # IDXFORMAT=8
 #---------------------------------------------------------------------------

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

Part_9 Converting Mainframe DATA to unix/linux/windows

Vancouver Utility Scripts used in data file conversions

9A1.	cnvdatadirs - script for initial setup of CNVDATA subdirs
	vsedatadirs - for initial setup of TESTDATA subdirs
	vselibsdirs - for initial setup of TESTLIBS subdirs

9B1.	cnvdatainit - script to clear some CNVDATA subdirs before regen/reconvert
	testdatainit - script to clear some TESTDATA subdirs before regen/reconvert
	vsetestlibsinit - script to clear some TESTLIBS subdirs before regen/reconvert

9C1.	gencnv4A - script to regenerate all uvcopy jobs to reconvert all files
9C2.	gencnv41 - script to regenrate 1 uvcopy job to reconvert 1 file

9D1.	jcl2ksh4A - script to reconvert all JCL from jcl0/... to ksh in jcl3/...

Note	scripts with hyperlinks are listed on following pages scripts without hyperlinks con be found in /home/uvadm/sf/IBM/...

Vancouver Utility jobs used in data file conversions

9P1.	contents for section 2 of Part 9.

9Q1.	varstat1 - summary table of record-size stats in variable length file
	catgut41 - extract 1 line file info from multi line LISTCAT report
	ctlfile41 - create data convert ctlfile, insert cpy=___ into listcat info
	ctlfile42 - load ctlfile into Indexed file to recover copybook names
	ctlfile43 - recover copybooknames into new ctlfile
	cobmap1 - convert copybooks into record layouts
	uvdata41 - generate uvcopy jobs to convert varlth to fixlth for convert
	uvdata42 - generate uvcopy jobs for data convert from copybookmaps
	uvdata43 - extract instructions from uvcopy jobs for insert to others
	uvdata44 - insert datafilenames into jobs generated from copybooks
	- uvdata44 for variable length records
	uvdata46 - for fixed length records

Note	scripts with hyperlinks are listed on following pages scripts without hyperlinks con be found in /home/uvadm/sf/IBM/...

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

9A1. Utilities & Scripts used in data file conversions

cnvdatadirs - setup subdirs in $CNVDATA for reconvert

 #!/bin/ksh
 # cnvdatadirs - script to make sub-dirs for data conversions
 #             - see DATAcnv1.doc & VSEDATA.doc
 #
 mkdir d1ebc  # EBCDIC data file input loaded here
 mkdir d2asc  # files converted to ASCII with same record layout
 mkdir d3unpk # files converted to ASCII & packed fields unpacked (optional)
 mkdir d4pipe # data files converted to '|' pipe delimited format
 mkdir cpys   # COBOL copybooks supplied or generated by signscanA/sign2cpyA
 mkdir ctl    # control files for datafile conversions
 mkdir maps   # record layouts generated from copybooks (start,end,length,type)
 mkdir pfx1   # uvcopy jobs generated by gencnvA, convert EBCDIC to ASCII
 mkdir pfx2   # uvcopy jobs completed with datafilenames (vs copybooknames)
 mkdir pfx3   # uvcopy jobs from pfx2, modified for multi R/T, executed from here
 mkdir pfp1   # uvcopy jobs generated by genpipe1, convert to pipe delimited
 mkdir pfp2   # uvcopy jobs copied from pfp1 & modified for multi R/Ts, etc
 mkdir pfp3   # uvcopy jobs from pfp2, modified for multi R/T, executed from here
 mkdir pf     # special uvcopy jobs for varlth records, occurs depending, etc
 mkdir sf     # Script Files subdir for misc (as required) scripts
 mkdir sqls   # SQL loader control files generated from copybooks
 mkdir signs  # record layout reports generated by signscan1/signscanA
 mkdir tmp    # temporary miscellaneous intermediate files created by
 exit 0

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

9B1. Utilities & Scripts used in data file conversions

cnvdatainit - clear subdirs in $CNVDATA for reconvert

 # cnvdatainit - clear some subdirs in $CNVDATA/cnvdata
 #             - by Owen Townsend, Feb2007
 # clear all files that will be recreated when you:
 # 1. regenerate all uvcopy conversion jobs from cpys/... up to pfx2/...
 #    - you must copy jobs for new files to pfx3 (& modify for multi R/T's ?)
 #    - you might have to merge R/T test code from old jobs into newer jobs
 #      whose instructions have chnaged due to updated copybooks
 # 2. then rerun all jobs to convert all files from subdir d0/... to d2asc/...
 #
 echo "clear CNVDATA subdirs before regen uvcopy jobs & reconvert data files"
 echo "clear d0ebc,d1ebc,d2asc,d3ebc,errs,maps,pfv1,pfv2,pfx1,pfx1a,pfx2,tmp"
 reply=x;
 until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "- clear CNVDATA subdirs before regen&reconvert y/n ?"
       read reply; done
 if [[ "$reply" != "y" ]]; then echo "$0 aborted"; exit; fi
 cd $CNVDATA
 rm -f d0ebc/* d1ebc/* d2asc/* d3ebc/*
 rm -f errs/*
 rm -f maps/*
 rm -f pfv1/* pfv2/*
 rm -f pfx1/* pfx1a/* pfx2/*
 rm -f tmp/* tmp1/*
 echo "cleared: d0ebc,d1ebc,d2asc,d3ebc,errs,maps,pfv1,pfv2,pfx1,pfx1a,pfx2,tmp"
 exit 0
 #

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

9C1. Utilities & Scripts used in data file conversions

gencnv4A - generate jobs to convert ALL data files

 # gencnv4A - generate uvcopy jobs to convert ALL data files EBCDIC to ASCII
 #          - by Owen Townsend, UV Software, Jan 7/2007
 #          - see explanations at www.uvsoftware.ca/vsedata.htm
 #
 #                        ** pre-requisites **
 #
 # 1. setup conversion superdir ($CNVDATA)
 #    mkdir /p3/cnvdata          <-- separate file system for high volumes, OR
 #    mkdir /home/userxx/cnvdata <-- subdir in homedir for low volume & testing
 #
 # 2. setup profile with export & alias for CNVDATA
 #    export CNVDATA=/home/userxx/cnvdata
 #    alias cdc='cd $CNVDATA'
 #
 # 3. Logoff/logon to activate exports & aliases & change to $CNVDATA
 #    cdc        <-- change to $CNVDATA (using alias 'cdc')
 #
 # 4. setup subdirs in $CNVDATA
 #    mkdir cpys ctl maps pfv1 pfv2 pfx1 pfx2 pfx3 <-- to generate convert jobs
 #    mkdir d0ebc d1ebc d2asc                      <-- for data files
 #
 # 5. store EBCDIC data files to be converted in 'd0ebc' & rename to lower case
 # 5a. cp ???/* d0ebc
 # 5b. renameL d0ebc         <-- ensure datafilenames Lower-case
 #
 # 6. store copybooks in subdir 'cpys' & drop any extension
 # 6a. cp $TESTLIBS/cpys/* cpys    <-- copy copybooks from testlibs
 # 6b. rename-X cpys .cpy          <-- drop extensions
 #
 # 7. create control file to relate datafilenames to copybook names
 #    - select desired files from ctl/listcat2 (info extracted from mainframe)
 #    - or create manually as shown in the samples below
 #      (see more at www.uvsoftware.ca/vsedata.htm)
 #    - either way you need to edit/insert cpy=copybooknames
 #
 # 7a. uvcopy ctlfile23,fili1=ctl/listcat2,filo1=ctl/ctlfile3
 # 7b. cp ctl/ctlfile3 ctl/ctlfile4 <-- copy/rename to protect edited version
 # 7c. vi ctl/ctlfile4   <-- edit to drop unwanted files
 #                       <-- edit to add copybooknames (see samples below)
 #
 #                ** sample listcat2 file **
 #
 # e2121653.mtaxe.piscines        rca=00082 rcm=00082 key=000(020) cnt=00033321
 # e2121656.itaxe.facturat        rca=00083 rcm=00239 key=000(036) cnt=15946788
 # e2123002.itaxe.taxation        rca=00046 rcm=00383 key=000(035) cnt=00074292
 #
 #                ** sample ctlfile4 file **
 #
 #e2121653.mtaxe.piscines cpy=e212159b rca=00082 rcm=00082 rcf=00128 key=000(020)
 #e2121656.itaxe.facturat cpy=e212037b rca=00083 rcm=00239 rcf=00320 key=000(036)
 #e2123002.itaxe.taxation cpy=e212007b rca=00046 rcm=00383 rcf=00448 key=000(035)
 #
 echo "generate uvcopy jobs to convert all files from copybooks & control file"
 cf="$1";
 if [[ ! -f "$cf" ]]; then
    echo "usage: gencnv4A ctl/ctlfile <--arg1 = data conversion control-file"
    echo "       ===================="
    echo "example: gencnv4A ctl/ctlfile4 <-- ctl/ctlfile4 is usual control file"
    echo "         ====================="
    echo "- ctlfile must have datafilenames,copybooks,recsizes,keys"
    exit 1; fi
 #
 reply=x; until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "generate uvcopy jobs to convert all files y/n ?"; read reply; done
 if [[ "$reply" = "n" ]]; then echo "reply=$reply, exiting"; exit 1; fi
 #
 if [[ -d cpys && -d maps && -d pfv1 && -d pfv2 && -d pfx1 && -d d0ebc ]]; then :
    else echo "must have subdirs: cpys,maps,ctl,pfv1/2,pfx1,2,3,d0ebc";exit 9;fi
 #
 echo "enter to: generate jobs to convert variable length to fixed length"; read
 uvcopy uvdata41,fili1=$cf,fild2=pfv1
 #===================================
 # - generate uvcopy jobs to convert variable length datafiles to fixed length
 # - 1 uvcopy job generated (in pfv1) for each line in control-file
 reply=x; until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "- copy pfv1/* to pfv2 ? y/n (unusual to need changes in pfv2)"
 read reply; done
 if [[ "$reply" = "y" ]]; then cp -f pfv1/* pfv2;
 else echo "cp pfv1/* pfv2 bypassed, do manually before execute from pfv2"; fi
 #
 echo "enter to: convert all copybooks to cobmaps (record layouts)"; read
 uvcopyx cobmap1 cpys maps uop=q0i7p0
 #===================================
 rmzf maps   # remove any zero files (from procedure vs data copybooks)
 #
 echo "enter to: generate skeleton uvcopy jobs from cobmaps"; read
 uvcopyx uvdata42 maps pfx1 uop=q0i7
 #==================================
 # - generate uvcopy skeleton jobs from copybooks
 #
 echo "enter to: extract uvcopy instructions from each job"; read
 uvcopyx uvdata43 pfx1 pfx1a uop=q0i7
 #===================================
 # - extract uvcopy instructions (mvs's for packed fields) from each job
 # - in case required for editing into uvcopy jobs in pfx3
 # - when primary copybook (in ctlfile) does not have all record redefs
 #
 echo "enter to: complete uvcopy jobs with datafilenames & indexed keys"; read
 uvcopy uvdata44,fili1=$cf,fild2=pfx1,fild3=pfx2,uop=q0i7
 #=======================================================
 # - complete the uvcopy jobs with actual filenames & indexed file keys
 # - 1 pfx2 uvcopy job for each line in control-file using specified copybook/map
 #
 echo "enter to: generate jobs to restore original recsize for varlth files";read
 uvcopy uvdata45,fili1=$cf,fild2=pfy1,uop=q0i7
 #============================================
 # - generate jobs to restore varlth recsizes, saved at end of fixlth records
 # - 1 uvcopy job generated (in pfy1) for each line in control-file
 reply=x; until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "- copy pfy1/* to pfy2 ? y/n (unusual to need changes in pfy2)"
 read reply; done
 if [[ "$reply" = "y" ]]; then cp -f pfy1/* pfy2;
 else echo "cp pfy1/* pfy2 bypassed, do manually before execute from pfy2"; fi
 #
 echo "enter to: display recommended procedures for editing & executing"; read
 echo "           ** recommendations for manual editing & executing **"
 echo " "
 echo "1. cp pfv1/??? pfv2  <-- copy generated var-fix jobs to execution dir"
 echo "                       - selective copy/update ? (usually OK to copy all)"
 echo "                       - see prompt to copy all above & reply y/n ?"
 echo "2. uvcopyxx 'pfv2/*' <-- execute all jobs to convert d0ebc/* to d1ebc"
 echo " "
 echo "3. cp pfx2/* pfx3    <-- copy EBCDIC-->ASCII jobs to execution dir"
 echo "                       - ONLY if no multi record type files (unlikely ?)"
 echo "              - AND if no jobs need instrns from alternate copybook/jobs"
 echo "                ELSE copy 1 at a time from pfx2 to pfx3 for edit/execute"
 echo " "
 echo "3a. cp pfx2/xxx pfx3 <-- copy jobs 1 at a time for edit & execute"
 echo " "
 echo "3b. uvcopy pfx3/xxx  <-- execute uvcopy jobs 1 at a time"
 echo " "
 echo "4. uvcopyxx 'pfx3/*' <-- execute all uvcopy jobs to convert all files"
 echo "                       - use for re-conversions after all jobs debugged"
 echo "5. cp pfy1/??? pfy2  <-- copy generated varlth restore jobs for execution"
 echo "                       - selective copy/update ? (usually OK to copy all)"
 echo "                       - see prompt to copy all above & reply y/n ?"
 echo "6. uvcopyxx 'pfy2/*' <-- execute all jobs to restore varlth recsizes"
 echo " "
 exit 0

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

9C2. Utilities & Scripts used in data file conversions

gencnv41 - generate job to convert 1 data file

 # gencnv41 - generate uvcopy job to convert 1 data file from EBCDIC to ASCII
 #          - by Owen Townsend, UV Software, Jan 7/2007
 #          - see explanations at www.uvsoftware.ca/vsedata.htm
 #          - note related job 'gencnv4A' to generate all uvcopy jobs
 #
 # gencnv4A - generate ALL uvcopy jobs to convert ALL datafiles
 #          - recommended for initial use
 #*gencnv41 - (this job) to generate uvcopy jobs to convert 1 datafile
 #          - recommended for files added after initial selection
 #
 #                ** pre-requisites (manual commands) **
 #
 # 1. setup conversion superdir ($CNVDATA)
 #    mkdir /p3/cnvdata          <-- separate file system for high volumes, OR
 #    mkdir /home/userxx/cnvdata <-- subdir in homedir for low volume & testing
 #
 # 2. setup profile with export & alias for CNVDATA
 #    export CNVDATA=/home/userxx/cnvdata
 #    alias cdc='cd $CNVDATA'
 #
 # 3. Logoff/logon to activate exports & aliases & change to $CNVDATA
 #    cdc        <-- change to $CNVDATA (using alias 'cdc')
 #
 # 4. setup subdirs in $CNVDATA
 #    mkdir cpys ctl maps pfv1 pfv2 pfx1 pfx2 pfx3 <-- to generate convert jobs
 #    mkdir d0ebc d1ebc d2asc                      <-- for data files
 #
 # 5. store EBCDIC data files to be converted in 'd0ebc' & rename to lower case
 # 5a. cp ???/* d0ebc
 # 5b. renameL d0ebc         <-- ensure datafilenames Lower-case
 #
 # 6. store copybooks in subdir 'cpys' & drop any extension
 # 6a. cp $TESTLIBS/cpys/* cpys    <-- copy copybooks from testlibs
 # 6b. rename-X cpys .cpy          <-- drop extensions
 #
 # 7. create control file to relate datafilenames to copybook names
 #    - select desired files from ctl/listcat2 (info extracted from mainframe)
 #    - or create manually as shown in the samples below
 #      (see more at www.uvsoftware.ca/vsedata.htm)
 #    - either way you need to edit/insert cpy=copybooknames
 #
 # 7a. uvcopy ctlfile23,fili1=ctl/listcat2,filo1=ctl/ctlfile3
 # 7b. cp ctl/ctlfile3 ctl/ctlfile4 <-- copy/rename to protect edited version
 # 7c. vi ctl/ctlfile4   <-- edit to drop unwanted files
 #                       <-- edit to add copybooknames (see samples below)
 #
 #                ** sample ctlfile4 file **
 #
 #e2121653.mtaxe.piscines cpy=e212159b rca=00082 rcm=00082 rcf=00128 key=000(020)
 #e2121656.itaxe.facturat cpy=e212037b rca=00083 rcm=00239 rcf=00320 key=000(036)
 #e2123002.itaxe.taxation cpy=e212007b rca=00046 rcm=00383 rcf=00448 key=000(035)
 #e2123003.itaxe.transdam cpy=e212024b rca=00446 rcm=00464 rcf=00512 key=000(000)
 #e2125504.mtaxe.paracomp cpy=e212138b rca=00104 rcm=00307 rcf=00384 key=000(034)
 #
 echo "generate uvcopy job to convert 1 file from EBCDIC to ASCII"
 echo "- uvcopy job generated from copybooks & control file"
 echo "- arg1 must specify datafilename within the 'd0ebc' subdir"
 echo "- arg2 must specify copybookname within the 'cpys' subdir"
 echo "- ctl/ctlfile4 must contain all datafilenames, copybooks, recsizes, keys"
 data="$1"; copy="$2"; cf=ctl/ctlfile4;
 if [[ -f d0ebc/$data && -f cpys/$copy ]]; then :
 else echo "usage: gencnv41 datafile copybook"
      echo "       =========================="
      echo "       - arg1=datafile(in d0ebc), arg2=copybook(in cpys)"
      echo "example: gencnv41 e2122683.itaxe.degrevem e212047b"
      echo "         ========================================="
      exit 1; fi
 #
 if [[ -d cpys && -d maps && -d pfv1 && -d pfx1 && -d pfx2 && -d pfy1 ]]; then :
    else echo "need subdirs: cpys,maps,ctl,pfv1/2,pfx1,2,3,pfy1,2";exit 9;fi
 #
 echo "enter to: generate 1 job convert variable length to fixed length"; read
 uvcopy uvdata41,fili1=$cf,fild2=pfv1,arg1=$data
 #==============================================
 # - generate 1 uvcopy job to convert variable length datafile to fixed length
 # - as specified in control-file
 #
 echo "enter to: convert 1 copybook to cobmap (record layout)"; read
 uvcopy cobmap1,fili1=cpys/$copy,filo1=maps/$copy,uop=q0i7p0
 #==========================================================
 #
 echo "enter to: generate 1 skeleton uvcopy job from cobmap"; read
 uvcopy uvdata42,fili1=maps/$copy,filo1=pfx1/$copy,uop=q0i7
 #=========================================================
 #
 echo "enter to: complete uvcopy job with datafilenames & indexed keys"; read
 uvcopy uvdata44,fili1=$cf,fild2=pfx1,fild3=pfx2,uop=q0i7,arg1=$data
 #==================================================================
 # - complete the uvcopy job with actual filenames & indexed file keys
 #
 echo "enter to: gen job to cnvt fixlth conversion file back to orig varlth";read
 uvcopy uvdata45,fili1=$cf,fild2=pfy1,arg1=$data
 #==============================================
 # generate job to covnert fixlth conversion file back to original varlth
 #
 echo "enter to: display recommended procedures for editing & executing"; read
 echo "Recommendations - for manual editing & executing"
 echo "                - using 'xxxxx' to represent datafilename"
 echo " "
 echo "1. cp pfv1/xxxxx pfv2  <-- copy generated var-fix job to execution dir"
 echo "2. uvcopy pfv2/xxxxx   <-- execute 1 job to convert d0ebc/* to d1ebc"
 echo "                         - if file is variable (2 byte recsize prefix)"
 echo " "
 echo "3. cp pfx2/xxxxx pfx3 <-- copy EBCDIC-->ASCII job to execution dir"
 echo " "
 echo "4. vi pfx3/xxxxx      <-- edit job if multiple record types, OR"
 echo "                        - instrns required from pfx1a (other copybooks)"
 echo " "
 echo "5. uvcopy pfx3/xxxxx  <-- execute 1 uvcopy job"
 echo "                        - to convert d1ebc/xxxxx to d2asc/xxxxx"
 echo "6. cp pfy1/xxxxx pfy2 <-- copy generated restore varlth job for execute"
 echo "7. uvcopy pfv2/xxxxx  <-- execute job to restore original varlth recsize"
 exit 0

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

9D1. Utilities & Scripts used in data file conversions

jcl2ksh4A - reconvert all JCL from jcl0 to ksh in jcl3

 #!/bin/ksh
 # jcl2ksh4A - convert All VSE JCL to Korn shell scripts
 #           - see page 1Q1 VSEJCL.doc or www.uvsoftware.ca/vsejcl.htm#1Q1
 # Oct2009 - option q1 in ctl/jclunixop41 applies TOPNODES replace table
 #
 if [[ -d jcl0 && -d jcl1 && -d jcl2 && -d jcl3 && -d proc0 && -d procs\
    && -d parm0 && -d parms && -d sli0 && -d slis && "$1" = "all"\
    && -f ctl/jclunixop41 && -f ctl/datafiles42 ]]; then :
 else echo "usage: jcl2ksh4A all"
      echo "       ============="
      echo "subdirs reqd: jcl0,jcl1,jcl2,jcl3,proc0,procs,sli0,slis,parm0,parms"
      echo "control files reqd: ctl/jclunixop41 & ctl/datafiles42 (may be null)"
      exit 1; fi
 #
 echo "jcl2ksh4A to perform all steps of JCL conversion to Korn shell scripts"
 echo "jcl0--------->jcl1------------->jcl2----------->jcl3---------->jcls"
 echo "     cleanup      Pro/SLIcExpand      convert         manual       "
 echo "- all steps reqd if you retransfer JCL, PROCs,or Parms"
 echo "- run 'jclxx41' to reconvert jcl2-->jcl3 after converter/option changes"
 echo "for 1st conversion - see step by step in 1Q1-1Q5 of MVSJCL.doc"
 echo "jcl2ksh4A can omit cleanup proc0-->procs & sli0-->slis"
 echo "- saves time if no new/modified PROCs, INCLUDEs, or control cards"
 echo "cleanup PROCs, INCLUDEs,& CTLCARDS y/n ? "
 reply=x; read reply
 until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "cleanup PROCs, INCLUDEs,& CTLCARDS - y/n ?"; read reply; done
 if [[ "$reply" = "y" ]]; then
 #
 reply=x; until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "OK to remove all files from procs & SLIs ? y/n"; read reply; done
 if [[ "$reply" = "y" ]]; then rm -f procs/*; rm -f slis/*; fi
 #
 echo "clear cols 73-80 of slis y/n ?"; reply=x; g=g8; read reply
 until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "clear cols 73-80 of slis y/n ?"; read reply; done
    if [[ "$reply" = "n" ]]; then g=g0; fi
    uvcopyx cleanup proc0 procs uop=q0i7g8n1,arg1=.proc  #<-- cleanup PROCs
    #==================================================
    uvcopyx cleanup sli0 slis uop=q0i7n1$g,arg1=.sli     #<-- cleanup SLIs
    #===============================================
    #Aug03/06 - $g is g8 to clear 73-80 or g0 to NOT clear (set at prompt above)
 fi
 echo "will now cleanup jcl0--->jcl1 & expand PROCs jcl1--->jcl2"
 reply=x; until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "OK to remove all files from jcl1 & jcl2 ? y/n"; read reply; done
 if [[ "$reply" = "y" ]]; then rm -f jcl1/*; rm -f jcl2/*; fi
 #
 # 1. cleanup JCL, strip CR's, clear cols 73-80,& shorten to last nonblank
 uvcopyx cleanup jcl0 jcl1 uop=q0i7g8n1,arg1=.jcl     #<-- cleanup JCL
 #===============================================
 echo "cleanup complete, enter to perform PROC expansion"; read reply
 #
 # 2. expand procs as we copy JCL from subdir jcl1 to subdir jcl2
 jclpx41 jcl1 jcl2 procs slis
 #===============================
 echo "PROC expansion complete, enter to generate data control file"; read reply
 #
 # 3. generate data control file for JCL converter
 #
 uvcopy jcldata41,fild1=jcl2,filo2=ctl/datafiles41
 #================================================
 echo "jcldata41 has extracted datafilenames from all jcl2 DLBLs/TLBLs & IDCAMS"
 echo " - 1 line for each filename with recsizes, index keys, file type on right side"
 echo " - file names & info written into ctl/datafiles41"
 #
 # 4. explain control files & allow user to stop & create ctl/datafiles42
 #
 echo "                  ** control files used by JCL converter **"
 echo "listcat3I - datafile info derived from the mainframe LISTCAT report"
 echo "          - file types, record sizes,& indexed key location(lengths)"
 echo "          - creation documented at VSEDATA.doc#1E2 & VSEJCL.doc#4J3   "
 echo " "
 echo "datafiles43I - datafile info extracted from all JCL to be converted"
 echo "             - filenames from DLBLs, TLBLs,& IDCAMS"
 echo "             - IDCAMS supplies file types, record sizes,& indexed keys "
 echo "             - creation documented at VSEJCL.doc#4K2"
 echo "                       ** listcat3I - creation **"
 echo "listcat0 - mainframe LISTCAT transferred to \$CNVDATA/ctl/listcat0"
 echo "listcat1 - LISTCAT info extracted & summarized to 1 line per datafile"
 echo "lisctat2 - sorted by filename & copied to \$TESTLIBS/ctl/listcat2"
 echo "listcat3 - copied from listcat2 & possibly edited to add missing info"
 echo "         - all listcat files presumed as Masters, may change some to Work"
 echo "listcat3I - listcat3 loaded into Indexed file for use by JCL converter"
 echo "          - primary source, JCL converter uses this info if match found"
 echo "                      ** datafiles43I - creation **"
 echo "datafiles41 - datafile info automatically extracted from all JCL"
 echo "            - from DLBLs,TLBLs,IDCAMS (file types,rec-sizes,indexed keys)"
 echo "datafiles42 - optional manual created file in same format as datafiles41"
 echo "            - to add to or modify info in datafiles41"
 echo "            - info on matching filenames here overrides datafiles41 info"
 echo "            - must create a null file if no override info required"
 echo "datafiles43I - Indexed file used by JCL converter"
 echo "             - created by sorting datafiles41 & datafiles42 together"
 echo "             - dropping datafiles41 filenames matched by datafiles42"
 echo "             - loading Indexed file keyed on datafilename"
 #
 reply=x;
 until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "enter 'y' - to continue using files described above "
       echo "enter 'n' - to stop & add info to ctl/datafiles42 "
       echo "- can create nullfile ctl/datafiles42 for 1st JCL conversion"
       read reply; done
 if [[ "$reply" != "y" ]]; then echo "job ended by request"; exit 91; fi
 #
 # 5. sort datafiles41 & datafiles42 (may be null), loads datafiles43I
 #
 sortload43I   #<-- script sorts df41+df42, loads Indexed datafiles43I
 #==========
 if (($?)); then
    echo "sortload43I failed, did you make ctl/datafiles42 (nullfile OK)"
    exit 99; fi
 echo " "
 echo "- enter to convert JCL to ksh "; read reply
 #
 # 6. convert expanded JCL to Korn shell scripts (uses ctl/datafiles42)
 #
 jclxx41 jcl2 jcl3
 #================
 #
 # 7. copy scripts from jcl3 to jcls 1 by 1 as you test/debug
 echo "all JCLs converted from jcl0 -> jcl1 -> jcl2 -> jcl3 (Not to jcls)"
 echo "copy scripts from jcl3 to jcls (in PATH) 1 by 1 as you test/debug"
 echo " "
 # 8. Extract jobparms from JCL/scripts - optional
 echo "Extract jobparms from JCL/scripts - optional ?"
 echo "- see uvcopy jobparmx1 at: www.uvsoftware.ca/vsejcl.htm#4L1"
 #
 echo "uvlp12 errs/procNF.rpt     <-- print summary report of PROCs Not Found"
 echo "uvlp12 errs/includeNF.rpt  <-- print summary report INCLUDEs Not Found"
 echo "uvlp12 errs/sliNF.rpt      <-- print summary report SLIs Not Found"
 echo "uvlp12 errs/parmNF.rpt     <-- print summary report of PARMs Not Found"
 exit 0

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

9P1. Converting Mainframe DATA to unix/linux/windows

Vancouver Utility jobs used in data file conversions

9Q1.	varstat1 - summary table of record-size stats in variable length file

      catgut41  - extract 1 line file info from multi line LISTCAT report

      ctlfile41 - create data convert ctlfile, insert cpy=___ into listcat info

      ctlfile42 - load ctlfile into Indexed file to recover copybook names

      ctlfile43 - recover copybooknames into new ctlfile

      cobmap1   - convert copybooks into record layouts

      uvdata41  - generate uvcopy jobs to convert varlth to fixlth for convert

      uvdata42  - generate uvcopy jobs for data convert from copybookmaps

      uvdata43  - extract instructions from uvcopy jobs for insert to others

      uvdata44  - insert datafilenames into jobs generated from copybooks
                - uvdata44 for variable length records
      uvdata46  - for fixed length records

Note	scripts with hyperlinks are listed on following pages scripts without hyperlinks con be found in /home/uvadm/sf/IBM/...

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

9Q1. Vancouver Utility jobs used in data file conversions

varstat1 - summary table of record-size stats in varlth file

 # varstat1 - create summary table of record-sizes used in variable file
 #          - for variable lth records with 4 byte hdrs (standard RDW)
 #          - by Owen Townsend, Dec 12, 2007
 #Dec19/07 - replace getvr2 subrtn with file typ=RDW & RDWz2 (optional)
 #         - see varstat2 for the old 'getvr2' subrtn
 #Jan20/10 - name was changed previously from varstat1 to RDWstat1
 #         - creating duplicate to avoid confusion ?
 #
 #           ** create report for 1 file (for testing) **
 #
 # uvcopy RDWstat1,fili1=d0ebc/datafilename,filo2=tmp1/reportname
 # ==============================================================
 #
 #           ** create reports for all files in directory **
 #
 # 1. cd $CNVDATA     <-- change to conversion superdir
 #                     - subdir d0ebc contains EBCDIC var lth files
 #
 # 2. mkdir tmp1      <-- make tmp1 subdir if not existing
 # 2a. rm -f tmp1/*   <-- OR remove all files if tmp1 already exists
 #
 # 3. uvcopyx RDWstat1 d0ebc tmp1 uop=q0i7,rop=r0
 #    ===========================================
 #    - create table summary recsize counts for all files in d0ebc subdir
 #    - output reports in tmp1 with same names as datafiles in d0ebc
 #
 # 4. uvlpd12 tmp1     <-- print all reports in tmp subdir
 #    ============
 #
 #                  ** sample report **
 #
 # RDWstat1  2006/12/17_18:15:17  record-sizes in d0ebc/E2121656
 # tbl#001 pg#001     -argument-
 # line#  count    %  record-sizes
 #     1  10,552  16  00046
 #     2   4,451   7  00065
 #     3  23,347  37  00066
 #     4     367   0  00068
 #     5  21,010  33  00083
 #       - - - etc - - -
 #    18       3   0  00218
 #    19     441   0  00233
 #    20     813   1  00239
 #        62,115*100   *TOTAL*
 #

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page

 # This job designed for variable lth records typ=RDW & typ=RDWz2
 # typ=RDW (standard RDW) - 4 byte record prefixes
 #         - binary recsize in 1st 2 bytes (3rd & 4th x'0000' unused)
 #         - binary recsize of data (includes the 4 byte prefix)
 #         - blk/rec lth binary BIG-END format, need 's'witch option for INTEL
 # typ=RDWz2 (non-std RDW) - 2 byte record prefixes (2 nulls omitted)
 #         - binary recsize does not include the 2 byte prefix
 #
 opr='$jobname - create summary table of record-sizes used in variable file'
 opr='uop=q1z0 - option defaults (message only, see uop=... in job)'
 opr='      z0 - RDW 4 byte prefix, recsize 1st 2 BIG-END binary'
 opr='      z2 - RDW 2 byte prefix, recsize 1st 2 BIG-END binary'
 uop=q1z0      # option defaults
 rop=r1        # option to prompt for report disposition
 was=a16384
 fili1=?d0ebc/input,rcs=8192,typ=RDW        #Dec19/07 typ=RDW replaces getvr
 filo1=?tmp1/$jobname.rpt,typ=LSTt,rcs=128
 @run
 #Dec19/07 - replace subrtn getvr with typ=RDW & RDWz2
 #         - for nonstd 2 byte prefix vs std 4 bytes
 # if uop=z2, append file option z2 for typ=RDWz2
        tsb    o26(1),x'02'            uop z2 for nonstd RDW ?
        skp!   1
        cata8  $fili1+180(15),'z2'
        opn    all
 #
 # begin loop to get records & build summary table of recsizes
 # - to be dumped at end of file
 man20  get    fili1,a0                 get next RDW record data
        skp>   eof                      (cc set > at EOF)
        mvn    c0(5),$rv                cnvrt binary recsize to numeric
        tblt1f1 c0(5),'record-sizes'    build table in memory
        skp    man20                    repeat get loop til EOF
 #
 # EOF - close files & end job
 eof    mvfv1  f0(80),'record-sizes in $fili1'
        tbpt1s1 filo1,f0(50)            dump table from memory to file
        cls    all
        eoj
 #

Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page