JCLcnv1demo.htm - DEMO conversions, sample JCL, scripts, executions
JCLcnv2real.htm - comprehensive instructions for REAL conversions
JCLcnv3aids.htm - conversion AIDS (cross-refs,tips,mass changes,etc)
JCLcnv4gdg.htm - GDG file handler from UV Software
DATAcnv1.htm - Data file conversion - Original Documentation <--This Doc**
- datafilenames same as copybooknames for easy conversion
DATAcnv2.htm - Data File Conversion - Comprehensive & Complete
- datafilenames mainframe standards aaaaa.bbbbb.cccc,etc
| Part_0 | - Mainframe DATA Conversion Plans |
| - Plan A, convert Sequential & Indexed files retaining existing layouts | |
| to get off the mainframe as soon as possible | |
| - Plan B, provides conversion of flat files to RDBMS tables | |
| - Plan C, follow Plan A & use Plan B for desired files on your schedule | |
| (after initial conversion to get off the mainframe asap). |
| Part_1 | - Introduction & Overview |
| - Summary of utilities used in this book | |
| - setting up subdirs required & loading test/demo files | |
| - using uvhd to investigate data files to be converted | |
| - Recommendations for converting your own files |
| Part_2 | - scanning data to determine record layout |
| - signscanA, sign2cpyA, cobmapA | |
| - use only when COBOL copybook not available | |
| - most sites using these utilities will already have the copybooks | |
| - skip Part 2 & start in Part 3 when COBOL copybooks available |
| Part_3 | - EBCDIC to ASCII translation with packed field preservation |
| - Illustration of EBCDIC data file & corresponding COBOL copybook | |
| - GENERATING uvcopy jobs to convert data files | |
| (cobmap1, cobmapA, uvdata51, gencnvA) | |
| - Sample generated job with explanations | |
| - Executing uvcopy jobs to convert EBCDIC to ASCII & preserve packed | |
| - Character translation & packed field preservation | |
| - Zoned sign conversion from EBCDIC to ASCII conventions | |
| - Manual changes required (if multiple record types) | |
| - Demo job after inserting record type test instructions | |
| - 'uvhdcob' displays data beside the COBOL copybook fieldnames |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| Part_4 | - Converting data to pipe-delimited text files for loading RDBMS |
| - Packed/binary/numeric fields convert to zero suppressed numeric fields | |
| with leading separate signs & decimal points inserted. | |
| - GENERATING uvcopy jobs to convert data to a '|' pipe delimited text | |
| code listing for sample generated job | |
| - EXECUTING generated jobs to convert data to pipe delimited text | |
| listings of test/demo input/output files | |
| - MODIFYING generated jobs to convert data to pipe delimited text | |
| - Options for field delimiters, input EBCDIC vs ASCII, RISC vs Intel | |
| - 'listpipe1' lists '|' delimited output file to verify conversion. | |
| (copybook fieldnames listed beside the data field contents) | |
| - 'sqlcreate1' generates SQL scripts to create & load table | |
| - script 'gencnvD2' generates all conversion jobs when copybook present | |
| (cobmap1, uvdata51, genpipe3, genfixd3, sqlcreate1). | |
| - 'genrfmA' generate jobs to reformat records based on 2 copybooks | |
| example to unpack all packed/binary fields | |
| copy/rename alternate copybook with all comp-3 & comp-4 removed | |
| - 'genfixd2' generate jobs to convert pipe|delimited to fixed-field | |
| - 'genpipe1' expand copybooks with item occurs, demo for armaster | |
| - 'cpyoccurs1' expand copybooks for 'group occurs', demo for custran1 |
| Part_5 | - Special situations requiring manual changes to generated jobs |
| - Multi-Record-Type files (copybooks with redefined records) | |
| require manual coding to test the record type & skip to auto gen code | |
| - Converting Multi Record Type files & Splitting to Separate files | |
| - manual changes for 'occurs' not allowed in SQL databases | |
| - Generating SQL Loader script to load Relational databases |
| Part_6 | - Conversions for real datafilenames (vs same as copybooknames parts3,4,5) |
| Generating & Executing ALL jobs to convert ALL data filesin directory | |
| vs '1 file at a time' documented in previous parts | |
| - recommended for high volume file conversions that need to be repeated | |
| during testing & final conversion | |
| - Generating a 'control-file' to relate copybooks to datafilenames | |
| - The control file can also supply Indexed file types & key locations | |
| - uvcopy uvdata52 inserts real datafilenames by control-file lookuip | |
| as it copies pfx1/... jobs (generated by uvdata51) to pfx2/... | |
| - Then copy from pfx2 to pfx3 & modify for files with multi record types | |
| - always execute jobs from pfx3 to allow for later control file addition | |
| - test files provided so you can run demos to get familiar with procedures |
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| Part_7 | - test EBCDIC/ASCII Translate tables used by Vancouver Utilities |
| and conversions from Fixed-field to Delimited & back to Fixed-field | |
| - bytes256 - test-file for translates & delimits 256 bytes x'00'-x'FF' | |
| - Ebytes - conversion reults considering bytes256 as an EBCDIC file | |
| - Abytes - conversion reults considering bytes256 as an ASCII file | |
| - uvcmpFA1 compare original fixed-field to reconverted from delimited | |
| hex display record pair with '*'s flagging differences | |
| - ebc2asc.c & asc2ebc.c EBCDIC/ASCII translate tables | |
| - trantestEAE - alternate test Vancouver Utilities EBCDIC/ASCII translates. | |
| - uvcopy job generates 256 lines showing EBCDIC/ASCII codes for each byte | |
| - makebytes256 - uvcopy job to create 256 bytes with values x'00' - x'FF' | |
| - trtneut.c - neutral translate table, C source code & uvhd hex representation |
| Part_8 | - Variable Length Record Files |
| - RDW (Record Descriptor Word) variable length files | |
| - option 'quote site RDW' FTP from mainframe (operating on unix/linux) | |
| - option 'LOCSITE RDW' FTP to Unix/Linux (operating on mainframe) | |
| - option 'OUTFIL FNAMES=SORTOUT,FTOV' SORT option to convert Fixed to Variable | |
| - investigating RDW files with uvhd | |
| - converting EBCDIC RDW files to ASCII using uvhd, uvcp,& uvcopy varfix11 | |
| - creating table summaries of record sizes found in RDW files | |
| varstat1 - uvcopy job to table summarize record sizes in RDW files |
| Part_9 | - Verifying Conversions & Generating Test Files |
| - converting ASCII files back to EBCDIC to compare to original | |
| - data file comparison utility 'uvcmp1' | |
| - Verifying Data Conversion with 'uvhdcob' to verify Numeric fields | |
| - Verifying Data Conversion with 'genverify1' | |
| (generate uvcopy jobs to Verify all Packed & Numeric fields) | |
| - generate jobs to accumulate all numeric fields in a file | |
| 'genacum2' generates uvcopy jobs from the copybook record layout | |
| (could use to verify conversions or check COBOL report totals) | |
| - Generating test data files automatically from COBOL copybooks |
| Part_10 | - summary of scripts used for data conversions |
| - Korn shell scripts for unix/linux |
Owen Townsend, UV Software, 4667 Hoskins Rd., North Vancouver BC, V7K2R3
Tel: 604-980-5434 Fax: 604-980-5404
mailto:owen@uvsoftware.ca
https://www.uvsoftware.ca
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
UV Software offers 2 strategies for converting mainframe data files for use on Unix/Linux systems. We are concerned here with the Sequential & Indexed files used on the mainframe, not with existing mainframe DBMS's. Software for converting mainframe DBMS tables is usually supplied by the vendor.
UV Software supplies powerful utilities to convert the Sequential & Indexed files, automatically from the COBOL copybooks, allowing for complex files, with multiple record types, occurs, etc.
Plan 'A' is to retain existing record layouts, which allows you to convert quickly, since COBOL program logic needs no changes.
Plan 'B' is to convert data files to pipe delimited text files for loading RDBMS tables. Any packed/binary fields are unpacked & edited with signs & decimal points as required for loading RDBMS tables. We also automatically generate SQL Loader scripts from the COBOL copybooks.
See sample conversions for plan A (layouts unchanged, preserve packed fields) at https://www.uvsoftware.ca/datacnv1.htm#Part_3
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
See sample conversions for plan B (unpack to delimited files to load RDBMSs) at https://www.uvsoftware.ca/datacnv1.htm#Part_4
We recommend plan A to get off the mainframe as quickly as possible. Then you can convert files to RDBMS tables depending on your priorities & timetable.
SQLconvert.htm - convert flat files to pipe delimited for Microsoft SQL server
- driven by COBOL copybooks
- also generate scripts to create & load tables
- generates jobs for all copybooks & all data files
(vs this DATAcnv1.doc for 1 copybook/file at a time)
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 1A1. | Data Conversion Overview |
| 1A3. | Summary of utilities used in this book |
| 1B1. | test/demo files on uvadm.zip distribution archive |
| customer installation of mvstest1, 2, or 3 | |
| copy desired files to separate login or to your homedir |
| 1B2. | export RUNLIBS/RUNDATA/CNVDATA in .bash_profile |
| depending on which of mvstest1,2,3 desired | |
| aliases for easy change to testlibs1,testdata1,cnvdata1 |
| 1B3. | Setup for DataFile Conversion |
| 1B4. | conversion subdirs illustrated (dtree report) |
| 1B5. | subdir contents & intended use |
| 1C1. | Data file investigation with uvhd |
| uvhd d1ebc/citytax1 r128a |
1D1. Using 'uvhd' to investigate EBCDIC files that you wish to convert to ASCII.
- determine/verify record size & look for packed fields & zoned signs
- look for complex files (multi record types, variable length records, etc)
| 1E1. | Recommendations for converting your own files |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The power & flexibility of these procedures are necessarily accompanied by some underlying complexity, but we want you to know at the outset, that there are master scripts to simplify these procedures.
We recommend you follow the step by step procedures to convert the test/demo file & perhaps your first data file. This will help you understand how these conversions work & allow you to modify the procedures for complex files (multi record types or variable length records).
For most files, you can use the 'gencnvD1' or 'gencnvD2' master scripts to generate all conversion jobs with 1 command. Converting any 1 data file is then simplified to the following (using 'citytax1' demo file for illustration):
#1. Copy the data file to the expected input subdir
(supplied demo file dat1/citytax1 may be copied to cnvdata1/d1ebc).
#2. gencnvD1 citytax1 r128d4 <-- generate All conversion jobs for demo file
========================
#3. uvcopy pfx3/citytax1 <-- execute EBCDIC to ASCII conversion
==================== - from d1ebc/citytax1 to d2asc/citytax1
(preserves packed, fixes zoned signs)
#4. uvcopy pfp3/citytax1 <-- execute conversion to pipe delimited text
==================== - from d2asc/citytax1 to d4pipe/citytax1
#5. Converted outputs are left in following subdirs: #5a. d2asc/citytax1 - ASCII file (same layout as input) #5b. d4pipe/citytax1 - all text pipe delimited file (for loading RDB's) #5c. sqlTC1/citytax1 - script to Create database Table #5d. sqlTL1/citytax1 - script to Load database Table
If you already have the COBOL copybook (datafile record description), you would use master script 'gencnvD2' which saves the 1st 2 of the 6 steps in gencnvD1 (scanning the data file & generating the copybook).
These jobs are simplified by setting up several subdirs within a superdir, which allows us to maintain the same file name as we generate jobs & convert data thru the various steps.
You don't have to modify the generated jobs if you copy the data file to the expected input subdir (d1ebc), and rename same as copybook. After conversion you can copy the converted from the output subdir (d2asc), and rename back to your desired datafilename.
But is is a simple matter to modify the generated conversion jobs to specify your I/O data pathnames. You might do this for large files, especially if you need several re-conversions during your conversion project.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
https://uvsoftware.ca/jclcnv2real.htm#Part_3
UV Software recommends a minimum 1 week training course with the initial installation of the Vancouver Utilities. With this training, you should be able to convert most EBCDIC files. UV Software provides customers with a generous amount of free support, so please call if you can use some help with a complex file. UV Software can also be contracted to convert the unusually complex files if the problem is not easily resolved via telephone and emails.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Here are the data conversion test/demo files provided on the Vancouver Utilities distribution archive (usually uvadm.zip or uvadm.tar.gz) - assuming you have setup user uvadm & unzipped into /home/uvadm/...
/home/uvadm/mvstest/mvstest0 - original JCL/COBOL/demo files (downgraded)
/home/uvadm/mvstest/mvstest1 - cnvdata1 datafilenames same as copybooknames
/home/uvadm/mvstest/mvstest2 - cnvdata2 datafilenames mainframe standards
/home/uvadm/mvstest/mvstest3 - AIX COBOL conversions (vs Micro Focus)
/home/uvadm/mvstest/mvstest0/testlibs0 <-- JCL/COBOL/DATA test/demo files
/home/uvadm/mvstest/mvstest0/testdata0 - original versions DOWNGRADED
/home/uvadm/mvstest/mvstest0/cnvdata0 - use mvstest1,mvstest2,mvstest3
* /home/uvadm/mvstest/mvstest1/testlibs1 <-- JCL/COBOL/DATA test/demo files ***
* /home/uvadm/mvstest/mvstest1/testdata1 - for DATAcnv1.doc
* /home/uvadm/mvstest/mvstest1/cnvdata1 or www.uvsoftware.ca/datacnv1.htm
- data-file-names same as copybook-names
/home/uvadm/mvstest/mvstest2/testlibs2 <-- JCL/COBOL/DATA test/demo files
/home/uvadm/mvstest/mvstest2/testdata2 - for DATAcnv2.doc
/home/uvadm/mvstest/mvstest2/cnvdata2 - or www.uvsoftware.ca/datacnv2.htm
- data-file-names IBM standards aaa.bbb.ccc,etc
After customers install uvadm.zip/tar, they can setup a separate user login for the desired set of test files (mvstest1,2) & copy to that homedir.
---> cp -r /home/uvadm/mvstest/mvstest1 /home/mvstest1/. <-- for this DATAcnv1.doc
---> cp -r /home/uvadm/mvstest/mvstest2 /home/mvstest2/.
===================================================
OR copy mvstest1,2 to YOUR HOMEDIR from /home/uvadm/mvstest1,2
---> cp -r /home/uvadm/mvstest/mvstest1 /home/userxx/.
---> cp -r /home/uvadm/mvstest/mvstest2 /home/userxx/.
================================================
OR copy the subdirs of mvstest1,2 to YOUR HOMEDIR (example for mvstest1)
---> cp -r /home/uvadm/mvstest/mvstest1/cnvdata1 /home/userxx/.
---> cp -r /home/uvadm/mvstest/mvstest1/testlibs1 /home/userxx/.
---> cp -r /home/uvadm/mvstest/mvstest1/testdata1 /home/userxx/.
===========================================================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Define CNVDATA,RUNDATA,RUNLIBS in .bash_profile, depending on which of mvstest1,2 you wish to test/demo - example for mvstest1 as a separate login.
export RUNLIBS=$HOME/mvstest1/testlibs1
export RUNDATA=$HOME/mvstest1/testdata1
export CNVDATA=$HOME/mvstest1/cnvdata1
OR if you copied just testlibs1,testdat1,cnvdata1 to your homedir:
cp -r /home/uvadm/mvstest/mvstest1/testlibs1 $HOME/. cp -r /home/uvadm/mvstest/mvstest1/testdata1 $HOME/. cp -r /home/uvadm/mvstest/mvstest1/cnvdata1 $HOME/.
Then setup the exports in your .bash_profile as follows. This is the default in the .bash_profile supplied.
export RUNLIBS=$HOME/testlibs1 export RUNDATA=$HOME/testdata1 export CNVDATA=$HOME/cnvdata1
The supplied .bash_profile & .bashrc include the following Aliases that make it easy to change to testlibs, testdat, or cnvdata.
alias cdl='cd $RUNDATA' alias cdl='cd $RUNDATA' alias cdl='cd $RUNDATA'
cdc <-- easy change to $HOME/cnvdata (/home/userxx/cnvdata) ===
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
I recommend you setup a separate login user account to perform these data conversions. I suggest userid 'mvstest1' which will be used in this document. The conversion directory will then be /home/mvstest1/cnvdata1.
If you cannot setup a separate userid, you could use a subdirectory within your home directory (but this may not be advisable if there is not enough space for your projected requirements).
Setting up a separate userid might be better if you do not want to modify your current .profile. Note that you must modify the .profile to setup PATH, etc to run the Vancouver Utility scripts & uvcopy jobs. See install.htm.
Sub-directories are made for the different types of files, so we can keep the same name as the original datafile & copybook, as we convert copybooks to record layouts, to uvcopy jobs. For example our demo file is named 'citytax1' & the same name is used as we convert thru several subdirs:
d1ebc ---------> signs ---------> cpys ---------> maps ----------> pfx1
signscan2 sign2cpy1 cobmap1 uvdata51
Keeping the same name thruout the various subdirs greatly simplifies our operating instructions. This also means you must drop any extensions on your datafile or copybook names.
Preferably setup a separate userid 'cnvdata1' to perform these conversions, But if not possible make directory 'cnvdata1' within your home dir or elsewhere. For Windows, make directory C:\cnvdata1.
#1. login mvstest1 --> /home/mvstest1 #2. mkdir cnvdata1 #3. cd cnvdata1 --> /home/mvstest1/cnvdata1
#4. cnvdatadirs <-- script to create sub-dirs (listed on next page)
===========
- - - OR - - -
#1. login youruserid ---> yourhomedir (/home/yourid) #2. mkdir cnvdata1 <-- make conversion superdir within your homedir #3. cd cnvdata1 <-- change into conversion superdir
#4. cnvdatadirs <-- script to create sub-dirs (listed on next page)
===========
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
/home/uvadm/mvstest/mvstest1 <-- location in Vancouver Utilities distribution /home/mvstest1 <-- OR setup user mvstest1 /home/userxx OR setup cnvdata1 within your $HOME dir :-----cnvdata1 <-- cnvdata1 superdir for DATA conversion : :-----archive - backup to reload d1ebc/... copybooknames or datafilenames : : :-----d1ebc_cpy_names <-- for Parts3,4,5 : : :-----d1ebc_data_names <-- for Part6 : :-----cpys - copybooks : :-----ctl - control files : :-----d0ebc - EBCDIC data files, copy here 1st in case changes before d1ebc : :-----d1ebc - EBCDIC data files (named same as copybooks for part3,4,5) : :-----d2asc - ASCII data files (converted by uvcopy uvdata51) : :-----d3ebc - convert back to EBCDIC by uvdata31 : :-----d3asc - reformatted data files (see genrfmA) : :-----d4pipe - pipe delimited to load DB tables (by genpipe1) : :-----d5fix - convert pipe|delimited back to fixed field (genfixd2) : :-----listpipe - pipe|delimited listed vertically with copybook fieldnames : :-----maps - copybooks with record layouts on right side : :-----pfx1 - uvcopy jobs to convert EBCDIC to ASCII (gen by uvdata51) : :-----pfx2 - with data filenames inserted (vs copybook names) : :-----pfx3 - copied here for modify/execute : :-----pfy1 - uvcopy jobs to convert ASCII to EBCDIC (gen by uvdata31) : :-----pfp1 - uvcopy jobs to create pipe delimited (gen by genpipe1) : :-----pfq1 - uvcopy jobs to convert pipe|delimited to fixed field (genfixd2) : :-----pfg1 - uvcopy jobs to create test data via copybooks (gentest1) : :-----pf - misc uvcopy jobs : :-----sf - misc scripts : :-----sqlTC1 - scripts to Create database tables (gen from copybooks) : :-----sqlTL1 - scripts to Load database tables (gen from copybooks) : :-----tmp - temp files (keep working directory clean) : : :-----testdata1 <-- testdata1 superdir for later testing : :-----data1 : :-----joblog : :-----jobmsgs : :-----jobtmp : :-----sysout : : :-----testlibs1 <-- testlibs1 superdir for COBOL & JCL conversion : :-----cbls - see JCLcnv1demo.htm#3B2 : :-----cblx : :-----cpys : :-----ctl : :-----jcls : :-----maps : :-----parms : :-----pf : :-----procs : :-----sf : :-----sfun : :-----xref
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
cnvdatadirs <-- script to create the sub-dirs listed below ===========
| archive |
|
| d0ebc |
|
| d0ebc |
|
| d1ebc |
|
| d2asc |
|
| d3asc |
|
| d4pipe |
|
| cpys |
|
| maps |
|
| pfx1 |
|
| pfx2 |
|
| Note |
|
| pfx3 |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| pfp1 |
|
| pfp2 |
|
| pfp3 |
|
| pf |
|
| signs |
|
| sqls |
|
| tmp |
|
ctl - control-files (used in Part_6) ctl/datacpy51 - skeleton control-file generated from mainframe data-file-names ctl/datacpy52 - control-file with copybooknames coded beside datafilenames
You will not need subdir 'signs' if you have the COBOL copybooks & do not need to scan the data (signscan2) to automatically create the copybooks. Note that signscan2 is not foolproof & may require correction.
| mkdir signs |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
You will need the following subdirs if you wish to create unpacked data records. The previous page assumed you wished only to convert the EBCDIC data to pipe delimited ASCII text records for loading into SQL databases.
| mkdir cpyu |
|
| mkdir mapu |
|
| Note |
| mkdir mapsI |
|
| mkdir pfr1 |
|
| mkdir pfr2 |
|
As you run the various scripts & uvcopy jobs, you should stay in the working directory (/home/mvstest1/cnvdata1/ or whatever you call it). Do NOT change into subdirs, since you must be back at the working directory to run the next step.
Address the files thru 1 level of subdir for various view/print commands. For example, to display the EBCDIC to ASCII conversion job:
more pfx1/citytax1 <-- address files thru 1 level of subdir ==================
cd pfx1 <-- do not change into & out of subdirs more citytax1 - would waste 3 commands vs 1 cd .. - since next step requires working directory level
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
You can demonstrate the data conversion scripts, by setting up the directories suggested on the previous pages,& loading the supplied test files.
cd cnvdata1 - change to superdir setup on prior pages ========== (with subdirs: d1ebc,d2asc,d4pipe,cpys,maps,etc)
uvhd d1ebc/citytax1 r128as7 - inspect citytax1 demo file
=========================== - option 'r' (r128) specifies record size
- option 'a' translates characters to Ascii
- option 's7' spaces between scale & data below
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
FFFFFFFF44DDCD4CCDDE444444444444444FFFF4CDECD4DDCC4444444444ECDC
1013014000168508559800000000000000018150266550961400000000005153
64 OUVER BCV9S1H1..CK..Wb....g...qq.00014906A970530
DEECD44444444444CCEFEFCF004D005880008000770FFFFFFFFCFFFFFF444444
64559000000000002359218101320072C0047C0111C000149061970530000000
--> i3 <-- immediate print 1st 3 records == --> q <-- quit uvhd
When you have a new EBCDIC file to be converted, the first thing you need to do is investigate the file (using 'uvhd') to determine or verify the correct record size & to check for packed fields, zoned signs, and complex files (multiple record types, variable length records, etc).
Your first objective is to determine the 'CORRECT RECORD SIZE'. In this case I already knew it & specified it on the command line (see 'r128' above). If you don't specify any option 'r', the record size defaults to 256.
'uvhd' makes it easy to determine the correct record size. When displaying record #1, uvhd will suggest the next lower & next higher record-size. If you have it wrong you will see the data shifting left or right as you browse thru the file (just hit enter to advance to the next record).
Note that you cannot use 'vi' or most other unix tools on EBCDIC files, or ASCII files with packed fields, or any file without linefeeds (most mainframe files do not have linefeeds so the entire file appears in vi as 1 long line).
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
When you have a new EBCDIC file to be converted, you should investigate the file using 'uvhd' since you cannot use 'vi' with EBCDIC files. Your 1st concern is to determine the correct record size or verify the size given to you by the source or copybook. Uvhd defaults record size to 256 & it is usually easy to determine record size by looking for data repetition.
| Note |
|
'uvhd' displays data files in vertical hex. Data is displayed in 64 byte segments on 3 lines (characters, hex zones,& hex digits). Use option 'a' with EBCDIC files to translate the character line to ASCII while still showing the EBCDIC data as hex zones & hex digits.
For the citytax1 demo file we know the record size is 128 which is specified via option 'r' (r128). We will also specify option 'a' to translate the character line to ASCII. If we also specify 's2' to space between the 3 line groups of characters, zones,& digits the option string will be 'r128as2'.
uvhd d1ebc/citytax1 r128as2 ===========================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
FFFFFFFF44DDCD4CCDDE444444444444444FFFF4CDECD4DDCC4444444444ECDC
1013014000168508559800000000000000018150266550961400000000005153
64 OUVER BCV9S1H1..CK..Wb....g...qq.00014906A970530
DEECD44444444444CCEFEFCF004D005880008000770FFFFFFFFCFFFFFF444444
64559000000000002359218101320072C0047C0111C000149061970530000000
rec#=1 recs=10 rsiz=128 fsiz=1280 d1ebc/citytax1 enter command--> i3 <-- immediate print 1st 3 records
'uvhd' displays the 1st record in the file & waits for you to enter a command. Please see uvhd.htm for the complete command repertoire or enter '?' for the Help screens. In this case we entered 'i3' to immediately print the 1st 3 records.
Note the packed fields in bytes 88-105 & the zoned sign in byte 115. You can calculate byte positions by adding the scale position (from top line) to the offset of the current row of data. For example packed fields begin at scale position 24 + offset 64 = byte 88 (zero relative) = column 89 (1 relative).
If you have long records & have a large screen (WinDos allows 60 lines/screen) use the 'm' option of uvhd to records up to 1000 bytes on 1 screen, for example:
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The 'immediate printing' (i3 above) feature of uvhd is very handy, but if it does not work on your computer, you can use the 'print' command (p3 vs i3). The 'print' command stores the formatted file in the 'tmp' subdir, named the same as the datafile with a date/time suffix. You could print separately:
uvlp12 tmp/citytax1.yymmddhhmmssP =================================
For the 'i' command, uvhd calls the 'uvlp12' script (or 'uvlpr12' if WinDos). The printer for 'uvlp12' is defined by env-var 'UVLPDEST'. The WinDos script 'uvlpr12.bat' redirects output to 'LPT1'. See more on page C1 of uvhd.doc.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Before we leave the discussion of using 'uvhd' with EBCDIC files, we should show you what happens if you don't supply any options to uvhd:
uvhd d1ebc/citytax1 <-- uvhd with No Options ===================
1 2 3 4 5 6
1 0123456789012345678901234567890123456789012345678901234567890123 <--scale
0 ........@@....@.....@@@@@@@@@@@@@@@....@.....@....@@@@@@@@@@.... <--chars
FFFFFFFF44DDCD4CCDDE444444444444444FFFF4CDECD4DDCC4444444444ECDC <--zones
1013014000168508559800000000000000018150266550961400000000005153 <--digits
64 .....@@@@@@@@@@@..............W.........qq......................
DEECD44444444444CCEFEFCF0888005880008000770FFFFFFFFCFFFFFFFFFFFF
645590000000000023592181029C0072C0047C0111C000149061970530191809
128 ........@@......@.....@@@@@@@@@@@@@....@....@......@......@@....
FFFFFFFF44CCDDCC4CDDED4444444444444FFFF4CCEE4DCDCCD4EEDCCE44ECDC
1013992300756975029665000000000000012500512307554590239553005153
192 .....@@@@@@@@@@@..............W.........xRL.....................
DEECD44444444444CCEFDFEF0888005880008800754FFFFFFFFDFFFFFFFFFFFF
645590000000000023553161029C0072C0047C0182C000146261980601190156
uvhd d1ebc/citytax1 a <-- uvhd with option 'a' =====================
10 20 30 40 50 60
1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC <--chars
FFFFFFFF44DDCD4CCDDE444444444444444FFFF4CDECD4DDCC4444444444ECDC <--zones
1013014000168508559800000000000000018150266550961400000000005153 <--digits
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'citytax2' is a multi-record-type file with 3 record types 'H','T',or 'P' in byte 8 (Header records, Tax Records,& Payment records).
The record size is 128 (same as citytax1), but we will use '512' in the example below to illustrate some points.
uvhd d1ebc/citytax2 r128as3 <-- display citytax2 (see options below) ===========================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 H December 31, 1998
44444444C4C88898894FF64FFFF4444444444444444444444444444444444444
000000008045354259031B019980000000000000000000000000000000000000
64
4444444444444444444444444444444444444444444444444444444444444444
0000000000000000000000000000000000000000000000000000000000000000
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
FFFFFFFF44DDCD4CCDDE444444444444444FFFF4CDECD4DDCC4444444444ECDC
1013014000168508559800000000000000018150266550961400000000005153
64 OUVER BCV9S1H1..CK..Wb....g...qq.00014906A970530
DEECD44444444444CCEFEFCF004D005880008000770FFFFFFFFCFFFFFF444444
64559000000000002359218101320072C0047C0111C000149061970530000000
10 20 30 40 50 60
r# 3 0123456789012345678901234567890123456789012345678901234567890123
256 10130140P 1999...A....A....A....A....A....A....A....A....A
FFFFFFFFD4444444FFFF00042000420004200042000420004200042000420004
101301407000000019990001C0001C0001C0001C0001C0001C0001C0001C0001
64 ....A....A....A.
2000420004200042444444444444444444444444444444444444444444444444
C0001C0001C0001C000000000000000000000000000000000000000000000000
uvhd d1ebc/citytax2 r128as3 <-- note options explained below ===========================
| option r128 |
|
| option a |
|
| option s3 |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| Note |
|
| 2A1. | 'signscanA' will help you determine the record layout of EBCDIC data files |
| when no COBOL copybook is available. Script signscanA runs uvcopy job | |
| 'signscan2' which scans for packed fields, zoned signs, date fields, and | |
| right blank filled fields (such as Name & Address). |
| 2B1. | Scan logic for various field types (unpacked, zoned/signed, packed) |
| 2B4. | Tips to improve scan results |
| 2C1. | 'sign2cpyA' converts the signscanA record layout to a COBOL copybook which |
| used by following scripts to perform various conversions. | |
| If a copybook is supplied with the EBCDIC data file, you can start the | |
| conversion at 'Part_3' & skip this Part_2. |
| 2D1. | 'cobmapA' converts the copybook automatically created by signscanA & |
| sign2cpyA into a record layout, which may then need manual corrections | |
| since the signscan method is not perfect. |
| 2E1. | 'sign2mapA' script combines signscan2, sign2cpy1,& cobmap1 |
| - the easy way to generate the cobmap from the data file with 1 command. |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Please see page '1B4' for our operating environment, changed into cnvdata1 superdir with various subdirs (d1ebc,d2asc,...,signs,...,etc). Note that most of these procedures are uvcopy jobs which have long commands, but we have provided scripts to allow shorter commands. By convention the uvcopy jobs have a digit suffix (1) & scripts have an UPPER case alpha (A).
#1a. uvcopy signscan2,fili1=d1ebc/citytax1,filo1=signs/citytax1,uop=r128
===================================================================
#1b. signscanA citytax1 r128 <-- use script equivalent for shorter command
======================= - ksh for unix/linux or .bat file for Win/Dos
uop=r0000s0t500z5b2x10f1 - option defaults
r???? - record size must agree with data file
r8192 - record size max (increase via was= below)
s0 - skip count before testing starts
t500 - terminate count (end testing)
z5 - Zoned field minimum lth for detection
b2 - pic x fields Ended by 2 EBCDIC blanks 0x4040
x10 - consolidate pic x fields with start bytes
within 10 of 1st of contiguous pic x fields
f1 - only 1 file output (final report)
f3 - 2nd file output before start byte dups reduced
to entry with Highest occurrences
f7 - 3rd file output (pack&zoned only) for debug
121101:144530:signscan2: uvcopy ver=20121015 pf=/home/uvadm/pf/IBM/signscan2
uvcopy LNX L64 license=20121015_99V_930630 site=UV_Software
User OPtion (uop) defaults = q1r0s0t500z5b2x10f1r128
--> null to accept or re-specify (1 or more) --> r128 <-- ENTER REC SIZE
121101:144531:signscan2: EOF fili01 rds=10 size=1280: d1ebc/citytax1s
121101:144531:signscan2: EOF fili01 rds=10 size=1280: d1ebc/citytax1s
121101:144531:signscan2: EOF filo01 wrts=20 size=690: signs/citytax1s
EOJ, Output File written to: signs/citytax1s
default command = null, OR enter: more,print,uvlpr12,edit,null -->
You must enter at least the record size option 'r128' in this case. Note the other option defaults & enter any you wish to change. For example if your date format was yymmdd & you wanted to test the 1st 5000 records you would specify options as --> r128d4t5000
The options are shown on the 3rd line of the output report (see next page) and in this example they would appear as:
# default options=q1b2c1d0e99f80g33r0s0t200y4z15r128d4t5000 ================================================***********
The default options on the left are overridden by any options you enter which appear here on the right (underlined by '*'s vs '='s).
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1a. uvcopy signscan2,fili1=d1ebc/citytax1,filo1=signs/citytax1,uop=r128
===================================================================
#1b. signscanA citytax1 r128 <-- use script equivalent for shorter command
======================= - allows options for record-size, etc
| NOTE |
|
uvhd d1ebc/citytax1 r128a <-- uvhd option 'a' (translate chars to ASCII) =========================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
FFFFFFFF44DDCD4CCDDE444444444444444FFFF4CDECD4DDCC4444444444ECDC
1013014000168508559800000000000000018150266550961400000000005153
64 OUVER BCV9S1H1..CK..Wb....g...qq.00014906A970530
DEECD44444444444CCEFEFCF004D005880008000770FFFFFFFFCFFFFFF444444
64559000000000002359218101320072C0047C0111C000149061970530000000
#2a. vi signs/citytax1 <-- inspect output with vi
=================
#2b. uvlp12 signs/citytax1 <-- list output with uvlp12
===================== (uvlpr12 for Windows/DOS batch script)
# signscan2 - Determine EBCDIC data file layout 2012/11/01_14:42:14
# recsize=00128,datafile=d1ebc/citytax1s
# options=q1r0s0t500z5b2x10f1r128
# total-records=00000010,records-sampled=00000010
#start(length) end count (p=packed,z=+/- signs {A-I,}J-R)
.00000(00008z) 00007 0000010
.00008(00002x) 00009 0000000
.00010(00025x) 00034 0000010
.00035(00005z) 00039 0000002
.00040(00020x) 00059 0000009
.00060(00020x) 00079 0000010
.00080(00008x) 00087 0000010
.00088(00004b) 00091 0000010
.00092(00005p) 00096 0000006
.00097(00005p) 00101 0000010
.00102(00005p) 00106 0000010
.00107(00009z) 00115 0000008
.00116(00006z) 00121 0000008
.00122(00006x) 00127 0000000
.00128(00000x) 00127 0000010
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
** signscan2 report field-layout
1 2 3 4
01234567890123456789012345678901234567890
# signscan2 - Determine EBCDIC data file layout 2012/11/01_14:42:14
# recsize=00128,datafile=d1ebc/citytax1s
# options=q1r0s0t500z5b2x10f1r128
# total-records=00000010,records-sampled=00000010
#start(length) end count (p=packed,z=+/- signs {A-I,}J-R)
.00000(00008z) 00007 0000010
.00008(00002x) 00009 0000000
.00010(00025x) 00034 0000010
.00035(00005z) 00039 0000002
.00040(00020x) 00059 0000009
.00060(00020x) 00079 0000010
.00080(00008x) 00087 0000010
.00088(00004b) 00091 0000010
.00092(00005p) 00096 0000006
.00097(00005p) 00101 0000010
.00102(00005p) 00106 0000010
.00107(00009z) 00115 0000008
.00116(00006z) 00121 0000008
.00122(00006x) 00127 0000000
.00128(00000x) 00127 0000010
00(1) - '.' identifies an active entry (vs '#' comment) 01(5) - end position of signed field 07(5) - start position of signed field 13(5) - length of signed field 18(1) - field type (p,z,x,d) see descriptions below: 21(7) - count of records with this signed field detected
'p' - packed fields detected by scan for x'0C' or x'0D'
- reliable since thee codes would not otherwise occur in EBCDIC files
'z' - zoned fields detected by scan for contiguous numerics 5 or more
- may have units zoned signs x'C0' or x'D0' '{' or '}' +0 or -0
or 'A'-'I' or 'J'-'R' for +1 thru +9 or -1 thru -9
- you will have to split contiguous zoned fields if no signs
'x' - character field detected by scanning for blank fill on right side
- detects fields such as name & address
- field start assumed to be end of prior field which might not be
correct if no packed, zoned, or other blank filled on left
'b' - 'binary' assumed if field starting with < x'40' does not have following
valid packed sign within 10 bytes
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We are primarily concerned with accurate EBCDIC to ASCII file conversion preserving any packed fields & correcting any EBCDIC zoned signs to ASCII equivalents. Negatives '}JKLMNOPQR' to 'pqrstuvwxy', and positives '{ABCDEFGHI' to '0123456789'.
We only need to care about the accurate detection of packed & signed fields (type 'p' & 'z'). We don't need to worry about accurate boundaries for type 'x' fields since the entire record will be translated from EBCDIC to ASCII & then the packed & signed fields will be overlaid.
But if you also want to generate an accurate copybook (that reflects the application field boundaries), then you can modify the signscan results before conversion to copybook (or modify the copybook fields after).
After conversion to copybook (by sign2cpy1), you can assign meaningful names if desired. For the 1st 3 fields of the signscan2 results above, the default names would be f00000 (8 bytes), f00008 (2 bytes), & f00010 (25 bytes). Using your application knowledge, you might rename as 'rollno', 'flags',& 'owner-name'.
But remember if all you want is an accurate conversion, you don't need to do any alphanumeric field splitting & renaming. You only need to care about the accurate detection of packed & signed fields (types 'p' & 'z').
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'signscan2' will detect all packed & zoned signed fields that are logically possible to detect. However there are some data patterns where it is impossible to detect the boundaries for certain. You should understand & watch for the following possible situations.
Unpacked signed fields are detected by the sign codes in the units position.
--> Signed +0 thru +9 are x'C0' thru x'C9' displayed as '{ABCDEFGHI'. --> Signed -0 thru -9 are x'D0' thru x'D9' displayed as '}JKLMNOPQR'
See the EBCDIC file display on page '2A2', using uvhd with option 'a' to translate the character line to ASCII, but still displaying the zones & digits in hexadecimal.
+0 x'C0' '{' and -0 x'D0' '}' are unique but A-I & J-R are not unique.
If an unsigned field immediately precedes a signed field, we will not know where the signed field ends. For example consider the following file:
product# price qty amount
unsigned signed signed
hammer 0000495000000A000000049E <-- 4.95 * 1 = 4.95
wrench 0000995000000C000000298E <-- 9.95 * 3 = 29.85
The amount field is easily determined to be 9 digits long since the preceding field is also a signed field with unique codes in the units position. However we don't know for certain how long the 'qty' field is, since the preceding field is unsigned & has no unique codes in the zone of the units.
The good news here is that it does not matter, since the translations of unsigned & signed fields are the same except for the units position of the signed fields. In this example, signscan2 would report the price & qty as 1 signed field 14 digits long.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Packed fields are identified by the sign codes in the right hand byte:
--> Negative fields will have x'0D' thru x'9D' in the right hand byte. --> Positive fields will have x'0C' thru x'9C' or x'0F' thru x'9F'.
x'_C' & x'_D' are unique, but some of the x'_F' bytes are not unique. x'4F' is ']', x'5F' is '^', x'6F' is '?', x'7F' is '"'.
If a packed field only had 1 of these last 4 codes in the sign byte (in all records tested), we could not uniquely identify it as a packed sign byte. But this is highly unlikely. We might increase our terminate count to test more records in the file.
The left boundary of a packed field is determined by scanning left until we reach a byte whose zone or digit is not 0 thru 9. This could be the sign byte of another packed field or any unpacked numeric, alpha, or special character.
If we encounter a x'40', we would not know for sure whether it is 2 digits of the packed field 4 & 0, or it could be an EBCDIC character blank. But if we encounter 2 blanks (x'4040'), we will assume this is the end of the packed field (a small possibility of ending too soon).
If a packed field always had 1 x'40' on its left side, we might report the length as 1 longer than it should be.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 2C1. | sign2cpy1 - convert sign report to copybook |
| 2D1. | cobmap1 - convert copybook to cobmap |
| 3D1. | uvhdcob - display data field contents beside copybook field names |
See the uvhdcob example on page '3D1'. The original auto assigned fieldnames (f00000, f00008, f00010, etc) have already been replaced with meaningful names, but that is not necessary to check field boundaries.
Using uvhdcob, you can quickly spot any packed or signed fields with incorrect boundaries.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Pre-requisite is uvcopy job 'signscan2' which created sign report file by scanning an EBCDIC data file for packed & zoned signs. This job will convert the sign report to a COBOL copybook which may be used as input for several Vancouver Utility jobs (see further below).
#1a. uvcopy sign2cpy1,fili1=signs/citytax1,filo1=cpys/citytax1
=========================================================
#1b. sign2cpyA citytax1 <-- use script equivalent for shorter command
================== - ksh for unix/linux or .bat file for Win/Dos
# signscan2 - Determine EBCDIC data file layout 2012/11/01_14:42:14
# recsize=00128,datafile=d1ebc/citytax1s options=q1r0s0t500z5b2x10f1r128
# total-records=00000010,records-sampled=00000010
#start(length) end count (p=packed,z=+/- signs {A-I,}J-R)
.00000(00008z) 00007 0000010
.00008(00002x) 00009 0000000
.00010(00025x) 00034 0000010
.00035(00005z) 00039 0000002
.00040(00020x) 00059 0000009
.00060(00020x) 00079 0000010
.00080(00008x) 00087 0000010
.00088(00004b) 00091 0000010
.00092(00005p) 00096 0000006
.00097(00005p) 00101 0000010
.00102(00005p) 00106 0000010
.00107(00009z) 00115 0000008
.00116(00006z) 00121 0000008
.00122(00006x) 00127 0000000
.00128(00000x) 00127 0000010
******* OUTPUT: copybook for: citytax1s
******* generated from signscan2 report 2012/11/01_14:42:21
01 citytax1s.
10 f00000 pic s9(00008).
10 f00008 pic x(00002).
10 f00010 pic x(00025).
10 f00035 pic s9(00005).
10 f00040 pic x(00020).
10 f00060 pic x(00020).
10 f00080 pic x(00008).
10 f00088 pic x(00004).
10 f00092 pic s9(00009) comp-3.
10 f00097 pic s9(00009) comp-3.
10 f00102 pic s9(00009) comp-3.
10 f00107 pic s9(00009).
10 f00116 pic s9(00006).
10 f00122 pic x(00006).
Before you attempt to make corrections & enhancements to the copybook, you should run cobmap1/cobmapA which converts the copybook to a 'cobmap' showing field start/end/length/type on the right side. Please see next page --->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1a. uvcopy cobmap1,fili1=cpys/citytax1,filo1=maps/citytax1
======================================================
#1b. cobmapA citytax1 <-- use script equivalent for shorter command
================ - ksh for unix/linux or .bat file for Win/Dos
#2a. more maps/citytax1 - display copybook map for review & enhancement
==================
#2b. uvlp12 maps/citytax1 - or list copybook map with UV script for lasers
==================== (uvlpr12 for Windows/DOS batch script)
cobmap1 COBOL record layout 2012/11/01_14:42:22 pg# 0001
cpys/citytax1s citytax1s RCSZ=00128 bgn end lth typ
* copybook for: citytax1s
* generated from signscan2 report 2012/11/01_14:42:21
01 citytax1s.
10 f00000 pic s9(00008). 0000 0007 008 ns 8
10 f00008 pic x(00002). 0008 0009 002
10 f00010 pic x(00025). 0010 0034 025
10 f00035 pic s9(00005). 0035 0039 005 ns 5
10 f00040 pic x(00020). 0040 0059 020
10 f00060 pic x(00020). 0060 0079 020
10 f00080 pic x(00008). 0080 0087 008
10 f00088 pic x(00004). 0088 0091 004
10 f00092 pic s9(00009) comp-3. 0092 0096 005pns 9
10 f00097 pic s9(00009) comp-3. 0097 0101 005pns 9
10 f00102 pic s9(00009) comp-3. 0102 0106 005pns 9
10 f00107 pic s9(00009). 0107 0115 009 ns 9
10 f00116 pic s9(00006). 0116 0121 006 ns 6
10 f00122 pic x(00006). 0122 0127 006
*RCSZ=00128
#3. vi cpys/citytax1 - modify the generated copybook (see next page)
================
Since the following conversions are based upon this automatically scan generated copybook, you should inspect it & if possible modify it to improve the results of the following data file conversions.
Please see the example on the next page --->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
01 citytax1s.
10 f00000 pic s9(00008). 0000 0007 008 ns 8
10 f00008 pic x(00002). 0008 0009 002
10 f00010 pic x(00025). 0010 0034 025
10 f00035 pic s9(00005). 0035 0039 005 ns 5
10 f00040 pic x(00020). 0040 0059 020
10 f00060 pic x(00020). 0060 0079 020
10 f00080 pic x(00008). 0080 0087 008
10 f00088 pic x(00004). 0088 0091 004
10 f00092 pic s9(00009) comp-3. 0092 0096 005pns 9
10 f00097 pic s9(00009) comp-3. 0097 0101 005pns 9
10 f00102 pic s9(00009) comp-3. 0102 0106 005pns 9
10 f00107 pic s9(00009). 0107 0115 009 ns 9
10 f00116 pic s9(00006). 0116 0121 006 ns 6
10 f00122 pic x(00006). 0122 0127 006
*RCSZ=00128
#3. vi cpys/citytax1 - modify the copybook (not the cobmap)
================
cobmap1 COBOL record layout 2012/12/04_14:55:51 pg# 0001
cpys/citytax1 citytax1 RCSZ=00128 bgn end lth typ
* citytax1 copybook for uv demos www.uvsoftware.ca/datacnv1.
01 citytax1.
10 folio pic x(00010). 0000 0009 010
10 name pic x(00025). 0010 0034 025
10 address pic x(00025). 0035 0059 025
10 city pic x(00020). 0060 0079 020
10 province pic x(00002). 0080 0081 002
10 zip pic x(00006). 0082 0087 006
10 post-date pic s9(00007) comp-4. 0088 0091 004bns 7
10 land-value pic s9(00009) comp-3. 0092 0096 005pns 9
10 improv-value pic s9(00009) comp-3. 0097 0101 005pns 9
10 face-value pic s9(00009) comp-3. 0102 0106 005pns 9
10 maint-tax pic s9(00007)v99. 0107 0115 009 ns 9
10 purchase-date pic x(00006). 0116 0121 006
10 filler001 pic x(00006). 0122 0127 006
*RCSZ=00128
Note that you should plan your changes using the 'cobmap' listing, but be sure to modify the copybook (not the cobmap). Then rerun the cobmapA script to regenerate & re-list the cobmap for the new copybook to verify that your changes result in the same record size.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'sign2mapA' script combines signscan2, sign2cpy1,& cobmap1, the easy way to generate the cobmap from the data file with 1 command.
sign2mapA citytax1 <-- generate cobmap from datafile ==================
#!/bin/ksh
# sign2mapA - Korn shell script from UVSI stored in: /home/uvadm/sf/IBM/
# sign2mapA - scan EBCDIC data file for packed/zoned signs, dates,& blanks
# - to determine record layout to assist conversion to ASCII
# - then convert signscan2 report to COBOL copybook
# - then convert copybook to cobmap
# (record layout start/end/lth on right side)
#
# This script combines scripts signscanA, sign2cpyA, & cobmapA
# - makes it easier to run uvcopy jobs signscan2, sign2cpy1,& cobmap1
#
if [[ -f d1ebc/"$1" && -n "$2" ]]; then :
else echo "usage: sign2mapA datafilename recsize"
echo " =============================="
echo " - argg1 must be datafile in d1ebc/..., arg2 must be rec-size"
echo " - output copybook in cpys/..., cobmap in maps/..."
exit 1; fi
#
uvcopy signscan2,fili1=d1ebc/$1,filo1=signs/$1,uop=r$2
#=====================================================
uvcopy sign2cpy1,fili1=signs/$1,filo1=cpys/$1,uop=r$2
#====================================================
uvcopy cobmap1,fili1=cpys/$1,filo1=maps/$1,rop=r1
#================================================
exit 0
#
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| Note |
|
| Part3 |
|
| 3A1. | EBCDIC to ASCII translation with packed field preservation |
| - uvcopy jobs & scripts | |
| 3A2. | Directories required & pre-requisites |
| - datafiles renamed same as copybooknames makes conversions easy to run | |
| - convert all filenames to lower case & remove extensions | |
| 3A3. | Illustration - EBCDIC data file & corresponding COBOL copybook |
| 3B1. | Preparation to run or re-run test/demo conversions (for self-training) |
| cnvdatainit - to clear previous test/demo outputs | |
| Restore d1ebc/... from archive/d1ebc_cpy_names or archive/d1ebc_data_names | |
| cp archive/d1ebc_cpy_names/* d1ebc/. <-- for Part3,4,5 (vs Part6) |
| 3B2. | 'Generating' uvcopy jobs to convert EBCDIC to ASCII & preserve packed |
| using uvcopy jobs (cobmap1 & uvdata51) OR script (gencnvA) | |
| - alternate uvdata51p & gencnvAp to ensure packed sign x'_C' for AIX COBOL | |
| 3B3. | Operating Instructions to generate the conversion jobs from copybooks |
| 3B4. | Sample generated job with explanations |
| 3B5. | Alternative conversion for AIX COBOL with uvdata51p (vs uvdata51) to ensure |
| packed field + signs x'_C' vs existing x'_F' signs in mainframe files |
| 3C1. | 'Executing' uvcopy jobs to convert EBCDIC to ASCII & preserve packed |
| 3C2. | Character translation & packed field preservation |
| 3C3. | Zoned sign conversion from EBCDIC to ASCII conventions |
| 3C4. | Manual changes required (if multiple record types) |
| 3C5. | Demo job after inserting record type test instructions |
| 3D1. | 'uvhdcob' displays the data beside the COBOL copybook fieldnames making |
| it easy to see if the copybook is consistent with the data. I recommend | |
| you list out the 1st few records with both uvhd & uvhdcob to verify the | |
| layout & resolve any problems if file is supplied by outside sources. |
| 3E1. | gencnvE - convert ASCII data files back to EBCDIC |
| - opposite of gencnvA |
uvcmpFE1 - might then compare converted file to the original EBCDIC file
- uvcmpFE1 is 1 of several scripts using the 'uvcmp1' utility
- see doc in CMPjobs.htm
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The Vancouver Utilities have jobs to convert mainframe data files for use on Unix, Linux, or Windows.
A 'uvcopy job' is generated (for each data file) automatically from the COBOL copybook. The uvcopy job has 1 instruction to translate the entire record from EBCDIC to ASCII. Then we generate instructions to preserve any packed/binary fields which are the same on unix/linux/windows.
We assume here that the data files are intended for use by Micro Focus COBOL which provides for packed/binary fields exactly the same as on the mainframe.
| cobmap1 |
|
| uvdata51 |
|
| uvdata51p |
|
| Note |
|
| cobmapA |
|
| gencnvA |
|
| gencnvAp |
|
| gencnvD2 |
|
| gencnvD2p |
|
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
/home/userxx <-- conversions within your $HOME dir :-----cnvdata1 <-- cnvdata1 superdir for DATA conversion : :-----archive - backup to reload d1ebc/... copybooknames or datafilenames : : :-----d1ebc_cpy_names <-- test/demo files for Parts3,4,5 : : : :-----armaster : : : :-----arsales : : : :-----artrans : : : :-----citytax1 : : : :-----citytax2 : : : :-----glmaster : : : :-----gltran : : : :-----vendormas : : : :-----vendorpaymas : : :-----d1ebc_data_names <-- test/demo files for Part6 : : : :-----ar.customer.armaster : : : :-----ar.customer.arsales : : : :-----ar.customer.artrans : : : :-----ar.customer.phone.list : : : :-----city.tax.assess : : : :-----city.tax.payments : : : :-----gl.account.glmaster.g0001v00 : : : :-----gl.account.glmaster.g0002v00 : : : :-----gl.account.gltran.g0001v00 : : : :-----gl.account.gltran.g0002v00 : : : :-----vendor.master.names : : : :-----vendor.master.payments : :-----cpys - copybooks : :-----ctl - control files : :-----d0ebc - EBCDIC data files, copy here 1st in case changes before d1ebc : :-----d1ebc - EBCDIC data files (named same as copybooks for part3,4,5) : :-----d2asc - ASCII data files (converted by uvcopy uvdata51) : :-----d3ebc - convert back to EBCDIC by uvdata31 : :-----d3asc - reformatted data files (see genrfmA) : :-----d4pipe - pipe delimited to load DB tables (by genpipe1) : :-----d5fix - convert pipe|delimited back to fixed field (genfixd2) : :-----listpipe - pipe|delimited listed vertically with copybook fieldnames : :-----maps - copybooks with record layouts on right side : :-----pfx1 - uvcopy jobs to convert EBCDIC to ASCII (gen by uvdata51) : :-----pfx2 - with data filenames inserted (vs copybook names) : :-----pfx3 - copied here for modify/execute : :-----pfy1 - uvcopy jobs to convert ASCII to EBCDIC (gen by uvdata31) : :-----pfp1 - uvcopy jobs to create pipe delimited (gen by genpipe1) : :-----pfq1 - uvcopy jobs to convert pipe|delimited to fixed field (genfixd2) : :-----pfg1 - uvcopy jobs to create test data via copybooks (gentest1) : :-----pf - misc uvcopy jobs : :-----sf - misc scripts : :-----signs - record layouts created by signscan2 : :-----sqlTC1 - scripts to Create database tables (gen from copybooks) : :-----sqlTL1 - scripts to Load database tables (gen from copybooks) : :-----tmp - temp files (keep working directory clean) : :-----ctl - control-files (Part_6 for real datafilenames vs copybooknames) : : :-----datacpy51 - skeleton control-file generated from mainframe datafilenames : : :-----datacpy52 - control-file with copybooknames coded beside datafilenames
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Datafiles are renamed the same as copybooknames to make conversion scripts easy to run. You only need to specify 1 filename to the conversion scripts. Various conversion jobs can be generated with the same name because they are in different subdirs. See 'Part_6' to convert real mainframe datafilenames aaa.bbb.ccc,etc
| Note |
|
#1. renameL cpys <-- convert all copybooknames to Lower case
============
#2. renameL d1ebc <-- enssure Lower case
=============
#3. mv d1ebc/datafilename d1ebc/copybookname
========================================
- change datafilenames to match copybooknames (without .cpy extension)
We will illustrate the conversion using our demo data file 'd1ebc/citytax1' and the corresponding COBOL copybook 'cpys/citytax1.cpy'. We need to use the 'uvhd' utility to display the data file since it is EBCDIC with packed fields.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd d1ebc/citytax1 r128a <-- uvhd option 'a' (translate chars to ASCII) =========================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
FFFFFFFF44DDCD4CCDDE444444444444444FFFF4CDECD4DDCC4444444444ECDC
1013014000168508559800000000000000018150266550961400000000005153
64 OUVER BCV9S1H1..CK..Wb....g...qq.00014906A970530
DEECD44444444444CCEFEFCF004D005880008000770FFFFFFFFCFFFFFF444444
64559000000000002359218101320072C0047C0111C000149061970530000000
cobmap1 COBOL record layout 2012/02/19_09:30:13 pg# 0001
mf/cpys/citytax1.cpy citytax1 RCSZ=00128 bgn end lth typ
01 citytax1.
10 folio pic x(00010). 0000 0009 010
10 name pic x(00025). 0010 0034 025
10 address pic x(00025). 0035 0059 025
10 city pic x(00020). 0060 0079 020
10 province pic x(00002). 0080 0081 002
10 zip pic x(00006). 0082 0087 006
10 post-date pic s9(00007) comp-4. 0088 0091 004bns 7
10 land-value pic s9(00009) comp-3. 0092 0096 005pns 9
10 improv-value pic s9(00009) comp-3. 0097 0101 005pns 9
10 face-value pic s9(00009) comp-3. 0102 0106 005pns 9
10 maint-tax pic s9(00007)v99. 0107 0115 009 ns 9
10 purchase-date pic x(00006). 0116 0121 006
10 filler001 pic x(00006). 0122 0127 006
*RCSZ=00128
# citytax1 - uvcopy job, EBCDIC to ASCII, preserve packed, fix signs
opr='citytax1 - uvcopy code generated from copybook: citytax1.cpy '
was=a33000b33000
fili1=?d1ebc/citytax1,rcs=00128,typ=RSF
filo1=?d2asc/citytax1,rcs=00128,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00128),a0 move rec to outarea before field prcsng
tra b0(00128) translate entire outarea to ASCII
# --- <-- insert R/T tests here for redefined records
mvc b88(4),a88 bns post-date
mvc b92(15),a92 pns land-value:face-value
trt b107(9),$trtsea ns maint-tax
put1 put filo1,b0
skp loop
eof cls all
eoj
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We assume you have installed Vancouver Utilities on your computer & copied the test/demo files from /home/uvadm/mvstest/mvstest1/* to separate userid mvstest1 or to subdir ccnvdata1 within your homedir (see page '1B1' & 1B2).
cp -r /home/uvadm/mvstest/mvstest1/cnvdata1 /home/userxx/. ===========================================================
Alternativly, UV Software can provide you with a guest userid/password, so you can test the data file conversions without having to install on your own computer.
#0a. login mvstest1 --> /home/mvstest1
#0b. cd cnvdata1 --> /home/mvstest1/cnvdata1
---------------- OR ---------------
#0a. login youruserid ---> yourhomedir (/home/yourid)
#0b. cd cnvdata1 <-- change into conversion dir in your homedir
---------------- OR ---------------
#0a. ssh uvsoft99@uvsoftware.ca <-- login with assigned userid/password
#0b. cd cnvdata1 <-- change into conversion dir on UV Software
#0c. cdc <-- cdc alias for 'cd cnvdata1'
If you have already run the test/demo conversions & you or others wish to re-run, you will want to clear the outputs of prior test/demo conversions, for easier inspection of your new outputs & avoid confusion with prior outputs.
#1. cnvdatainit <-- run script to clear any old outputs
=========== - d2asc,d4pipe,pfx1,pfx2,pfx3,pfp1,etc
- see script listed on page '10K1' or in $UV/sf/IBM/...
#2. rm -f d1ebc/* <-- clear d1ebc/... in case retest Part3 after Part6
=============
#3. cp archive/d1ebc_cpy_names/* d1ebc/. <-- for Part3,4,5
====================================
- refresh d1ebc/... files named same as copybooks (for Parts3,4,5)
| Note |
|
#3a. cp archive/d1ebc_data_names/* d1ebc/. <-- Alternative for Part6
===================================== - do NOT do for Parts3,4,5
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvcopy cobmap1 uvcopy uvdata51
copybook --------------> cobmap ---------------> uvcopy job (EBCDIC to ASCII)
cpys/citytax1.cpy maps/citytax1 pfx1/citytax1
We will present 2 ways to generate the EBCDIC to ASCII conversion job - run the 2 uvcopy jobs separately (cobmap1 & uvdata51), OR run the script (gencnvA) which runs the 2 jobs within the script.
#1a. uvcopy cobmap1,fili1=cpys/citytax1.cpy,filo1=maps/citytax1
==========================================================
- convert any 1 copybook to a 'cobmap' (record layout)
- - - OR (shorter command, same effect) - - -
#1b. cobmapA citytax1.cpy <-- script to convert any 1 copybook to cobmap
==================== - same as above, but shorter command
cobmap1 COBOL record layout 2012/02/19_09:30:13 pg# 0001
mf/cpys/citytax1.cpy citytax1 RCSZ=00128 bgn end lth typ
* citytax1 copybook
01 citytax1.
10 folio pic x(00010). 0000 0009 010
10 name pic x(00025). 0010 0034 025
10 address pic x(00025). 0035 0059 025
10 city pic x(00020). 0060 0079 020
10 province pic x(00002). 0080 0081 002
10 zip pic x(00006). 0082 0087 006
10 post-date pic s9(00007) comp-4. 0088 0091 004bns 7
10 land-value pic s9(00009) comp-3. 0092 0096 005pns 9
10 improv-value pic s9(00009) comp-3. 0097 0101 005pns 9
10 face-value pic s9(00009) comp-3. 0102 0106 005pns 9
10 maint-tax pic s9(00007)v99. 0107 0115 009 ns 9
10 purchase-date pic x(00006). 0116 0121 006
10 filler001 pic x(00006). 0122 0127 006
*RCSZ=00128
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#2a. uvcopy uvdata51,fili1=maps/citytax1,filo1=pfx1/citytax1 <-- uvcopy command
=======================================================
- - - OR (shorter command, same output & more) - - -
#2b. gencnvA citytax1.cpy <-- script equivalent of above (shorter command)
==================== - ksh for unix/linux or .bat file for Windows/Dos
- see script listed on page '10C2'
| Note |
|
| Note |
|
# citytax1 - uvcopy job, EBCDIC to ASCII, preserve packed, fix signs
opr='citytax1 - uvcopy code generated from copybook: citytax1.cpy '
uop=q0
was=a33000b33000
fili1=?d1ebc/citytax1,rcs=00128,typ=RSF
filo1=?d2asc/citytax1,rcs=00128,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00128),a0 move rec to outarea before field prcsng
tra b0(00128) translate entire outarea to ASCII
# --- <-- insert R/T tests here for redefined records
mvc b88(4),a88 bns post-date
mvc b92(15),a92 pns land-value:face-value
trt b107(9),$trtsea ns maint-tax
put1 put filo1,b0
skp loop
eof cls all
eoj
The uvcopy jobs are automatically generated by gencnvA/uvdata51 based on the copybook record definitions which define the locations of packed & zoned signed fields. These jobs are generated from a template into which we insert filenames, record sizes,& the instructions to preserve the packed fields & correct the zoned signs.
In the 'citytax1' example there were only 2 instructions inserted to preserve packed & correct signs, because the 4 packed fields were contiguous. But for some real life data files, there could be hundreds of packed & zoned signed fields. The number of mvc/trt instructions inserted depends on how contiguous the packed/zoned fields are. See detailed explanations on the next page --->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#01 # uvcopy job to translate EBCDIC to ASCII, preserve packed, fix zoned signs #02 opr='jobname=JOBNAME - pfx2 name=datafilename' #03 opr='copybook=citytax1 - pfx1 name=copybookname' #04 uop=q0,was=a33000b33000 #05 fili1=?d1ebc/citytax1,rcs=00128,typ=RSF #06 filo1=?d2asc/citytax1,rcs=00128,typ=RSF #07 @run #08 opn all #09 loop get fili1,a0 #10 skp> eof #11 mvc b0(00128),a0 move rec to outarea before field prcsng #12 tra b0(00128) translate entire outarea to ASCII #13 # --- <-- insert R/T tests here for redefined records #14 mvc b88(4),a88 bns post-date #15 mvc b92(15),a92 pns land-value:face-value #16 trt b107(9),$trtsea ns maint-tax #17 put1 put filo1,b0 #18 skp loop #19 eof cls all #20 eoj
The uvcopy job generated by 'gencnvA/uvdata51' is re-listed above with line#s so we can give a brief explanation of the more relevant instructions. See all instructions documented in uvcopy3.htm.
Lines 5&6 declare the I/O (fili1/filo1) filenames. The generation has
inserted the correct filenames & record sizes.
Line 8 (opn) opens all files.
Line 9 (get) is the beginning of the record processing loop. 'get' reads
the next record from fili1 into area 'a' starting at byte '0' (a0).
Line 10 (skp>) will skip to eof at End Of File, because the 'get' will set
the condition code '>' when there are no more records.
Line 11 (mvc) moves the record to output area 'b' starting at byte '0' for
a length of 128 bytes(recsize), from input area 'a' starting at byte '0'.
Line 12 (tra) translates to ASCII, entire record in area 'b' from byte '0'.
Line 13 is a #comment indicating where you can add code to test Record Types.
Line 14 (mvc) moves the binary field from the input area 'a' beginning at
byte 88 to the output area 'b' beginning at byte 88 for a length of (4),
Line 15 (mvc) moves the packed fields from the input area 'a' beginning at
byte 92 to the output area 'b' beginning at byte 92 for a length of (15),
which is the total of 3 packed fields (5+5+5).
Line 16 (trt) translates the zoned signed field b107(9) using translate
table 'trtsea', which translates Ebcdic zoned signs to Ascii zoned signs.
Line 17 (put) writes to the output file from area 'b' starting from byte '0'
for the record length declared on the filo1= (line 6).
Line 18 'skp' returns to get the next record & repeat loop until EOF
Line 19 closes the files & line 20 ends the job.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Use alternates uvdata51p or gencnvAp to ensure packed signs x'_C' for AIX COBOL vs uvdata51/gencnvA which do not change any existing x'_F' signs. But uvdata51p generates 1 mvn instruction per packed field vs uvdata51 which groups all contiguous packed fields together in 1 mvc instruction.
#2a. uvcopy uvdata51p,fili1=maps/citytax1,filo1=pfx1/citytax1 <-- uvcopy command
========================================================
#2b. gencnvAp citytax1.cpy <-- script equivalent of above (shorter command)
=====================
#01 # uvcopy job to translate EBCDIC to ASCII, preserve packed, fix zoned signs #02 opr='jobname=JOBNAME - pfx2 name=datafilename' #03 opr='copybook=citytax1 - pfx1 name=copybookname' #04 uop=q0,was=a33000b33000 #05 fili1=?d1ebc/citytax1,rcs=00128,typ=RSF #06 filo1=?d2asc/citytax1,rcs=00128,typ=RSF #07 @run #08 opn all #09 loop get fili1,a0 #10 skp> eof #11 mvc b0(00128),a0 move rec to outarea before field prcsng #12 tra b0(00128) translate entire outarea to ASCII #13 # --- <-- insert R/T tests here for redefined records #14 mvc b88(4),a88 bns post-date #15 mvn b92(5p),a92(5p) pns land-value #16 mvn b97(5p),a97(5p) pns improv-value #17 mvn b102(5p),a102(5p) pns face-value #18 trt b107(9),$trtsea ns maint-tax #19 put1 put filo1,b0 #20 skp loop #21 eof cls all #22 eoj
Compare this job to the job listed on the previous page. It is the same except that 3 separate 'mvn' instructions are generated for the 3 packed fields vs uvdata51 which combined the 3 contiguous 5 byte packed fields into 1 x 15 byte 'mvc' instruction.
uvdata51p is provided to ensure packed field signs are x'_C' vs x'_F' existing in many packed fields in files transferred from a mainframe. This was provided for AIX COBOL vs Micro Focus COBOL.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#3. cp pfx1/citytax1 pfx3 <-- copy auto-generated job to pfx3
===================== - before any manual changes & execution
#4. vi pfx3/citytax1 <-- make any manual changes required
================ (if multiple record types)
| Note |
|
#1a. uvcopy pfx3/citytax1,fili1=d1ebc/citytax1,filo1=d2asc/citytax1
==============================================================
#1b. uvcopy pfx3/citytax1 <-- easier (filenames default as shown above)
====================
#2. uvhd d1ebc/citytax1 r128a <-- display EBCDIC input file
=========================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
FFFFFFFF44DDCD4CCDDE444444444444444FFFF4CDECD4DDCC4444444444ECDC
1013014000168508559800000000000000018150266550961400000000005153
64 OUVER BCV9S1H1..CK..Wb....g...qq.00014906A970530
DEECD44444444444CCEFEFCF004D005880008000770FFFFFFFFCFFFFFF444444
64559000000000002359218101320072C0047C0111C000149061970530000000
#3. uvhd d2asc/citytax1 r128 <-- display ASCII output file
========================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
3333333322444424445522222222222222233332445442544422222222225444
1013014000AF8E085E29000000000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H1..C...W.........qq.000149061970530
455452222222222244535343004D005880008000770333333333333333222222
F5652000000000002369318101320072C0047C0111C000149061970530000000
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Zoned (unpacked) signs must be converted from the mainframe codes to the codes accepted by Micro Focus COBOL on Unix/Linux/Windows systems.
trt b107(9),$trtsea
=====================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Here is a sample sign conversion based on record #8 of the demo file citytax1. I am showing only bytes 64-128 of these 128 byte records & have replaced some bytes surrounding the signed field in bytes 107-115 with '...'. Note that 'R' EBCDIC -9 x'D9' is converted to 'y' ASCII -9 x'79'.
64 OUVER ALT6C4J2.Ri....Y@..........00000909R............
DEECD44444444444CDEFCFDF0589000570000000090FFFFFFFFDFFFFFFFFFFFF
645590000000000013363412029C0039C0000C0099C000009099980601190157
64 OUVER ALT6C4J2.R.....Y|..........00000909y............
4554522222222222445343430589000570000000090333333337333333333333
F5652000000000001C4634A2029C0039C0000C0099C000009099980601190157
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Manual changes may be required to the auto-generated uvcopy job, before you get the optimum results. The most common reason is to allow for multiple record types (with different layouts).
I recommend you do NOT modify the auto-generated jobs in subdir 'pfx1', but first copy them to subdir 'pfx3' before making any manual changes. This also protects your manual changes in pfx3 in case of regenerating (after changes to the copybook.).
| Note |
Regardless of whether changes are required or not, I suggest you always copy the jobs from pfx1 to pfx3, before executing (from pfx3). Often you wont realize you need modifications until after you inspect the output data.
#3. cp pfx1/citytax1 pfx3 - copy auto-generated job to pfx3
===================== before any manual changes &/or execution
#4. vi pfx3/citytax1 - make any manual changes required
================ (record type tests, see example page '5B2')
The most common change required is to insert code to test record types for files with redefined records (different layouts in the same file). Our 'citytax1' test file does not have multiple record types and the generated job requires no changes before execution.
But for illustration purposes, let us suppose it had a header record with no packed or zoned fields. It is a simple matter to insert 2 instructions to test for this & bypass the packed/zoned instructions which would destroy parts of our all character header record. We will assume that the header record is identified with an 'H' in byte 8 (zero relative).
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# citytax1 - uvcopy job, EBCDIC to ASCII, preserve packed, fix signs
opr='citytax1 - uvcopy code generated from copybook: citytax1.cpy '
uop=q0
was=a33000b33000
fili1=?d1ebc/citytax1,rcs=00128,typ=RSF
filo1=?d2asc/citytax1,rcs=00128,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00128),a0 move rec to outarea before field prcsng
tra b0(00128) translate entire outarea to ASCII
#------------------------------------------------------------------------
# --- <-- insert R/T tests here for redefined records
cmc b8(1),'H' Header record ? (no packed/zoned fields)
skp= put1 ifso - bypass packed/zoned mvc/trt
#------------------------------------------------------------------------
mvc b88(4),a88 bns post-date
mvc b92(15),a92 pns land-value:face-value
trt b107(9),$trtsea ns maint-tax
put1 put filo1,b0
skp loop
eof cls all
eoj
We have inserted instructions to compare (cmc) byte 8 to 'H' & if =, to skip to the 'put1' label, bypassing the mvc/trt which would destroy part of our header record. I have also added the '#------' lines to emphasize our inserted instructions, but you would not need to bother with these.
See the Operating Instructions to execute the uvcopy job on the next page --->
Note that we did not need to modify the 'citytax1' job before executing since the citytax1 datafile does not in fact have any header record.
For an example of a multi record type file (with 3 record types) & the coding changes required, please see the 'citytax2' demo beginning on page '5A1'.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'uvhdcob' is similar to 'uvhd', but also lists the copybook fieldnames beside the data field contents. You must have run cobmap1/cobmapA to convert the copybook to the 'cobmap' required by uvhdcob. To display our citytax1 demo file, we will specify option 'r128' for recsize + option 'a' to translate the characters line to ASCII.
uvhdcob datafile cobmap [options] <-- command format
=================================
#1. uvhdcob d1ebc/citytax1 maps/citytax1 a <-- display citytax1 EBCDIC file
====================================== 'a' option translate to ASCII
d1ebc/citytax1 rec#=1 recs=10 rsz=128 fsiz=1280 fptr=0 cobmap=maps/citytax1 today=0306092049 lastmod=02120515 rec# 1 fieldname occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 folio 0 9 10130140 002 name 10 34 JOHN HENRY 003 address 35 59 1815 BOWEN ROAD 004 city 60 79 VANCOUVER 005 province 80 81 BC 006 zip 82 87 V9S1H1 007 post-date 88 91pns 0082898C 008 land-value 92 96pns 000057828C 009 improv-value 97 101pns 000004870C 010 face-value 102 106pns 000171710C 011 maint-tax 107 115 ns 00014906A 012 purchase-date 116 121 970530 013 filler001 122 127 191809 rec#=1 of 10,null=next,r#=rec,s=search,u=update,p=print,i=iprint,w=write ,t=tally,c=chkseq,e=exit to uvhd,q=quit,?=help -->
enter command --> p1 <-- print 1st record enter command --> q <-- quit
Note you can use uvhdcob on either the EBCDIC file or the ASCII file. For the EBCDIC file, specify option 'a' to translate to ASCII (as above). For the ASCII file, omit the 'a' option. You might want to specify option 'm' to increase the 'maximum' lines per screen if you have records > 320 bytes.
#1a. uvhdcob d2asc/citytax1 maps/citytax1 m60 <-- ASCII data + fieldnames
======================================== - 60 lines/scrn or recsize
#1b. --> i3f1 <-- immediate print 1st 3 records, formfeed each page
====
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1a. mkdir pfy1 <-- make subdir for ASCII to EBCDIC jobs
==========
#1b. mkdir d3ebc <-- make subdir for output datafile
===========
#2. gencnvE citytax1.cpy <-- generate ASCII to EBCDIC conversion job
====================
#3. uvcopy pfy1/citytax1 <-- execute ASCII to EBCDIC job
====================
#4. uvcmpFE1 d1ebc/citytax1 d3ebc/citytax1 r128y1
=============================================
- compare original EBCDIC file to file converted to ASCII & back to EBCDIC
- see uvcmpFE1 script documented at CMPjobs.htm#1F1
- 'r128' is option for record-sizes
- 'y1' is option to translate character line (of vertical hexadecimal)
to ASCII (so you can read it on the unix/linux machines)
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 4A1. | Jobs/Scripts to generate pipe delimit & SQL table create/load |
| 4A2. | Directories required for pipe-delimit & SQL table create/load |
| 4A3. | Illustration converting data to pipe-delimited text files for loading RDBMS |
| Packed/binary/numeric fields converted to zero suppressed numeric fields | |
| with leading separate signs & decimal points inserted. |
In Oct 2014, we developed genpipe3/genfixd3 (alternates to genpipe1/genfixd2), to remove the restriction of 100 bytes max field length (& also faster). The new jobs (genpipe3/genfixd3) are documented begining on pages '4M1' & '4N1'. The original jobs (genpipe1/genfixd2) are documented first as follows and this documentation contains several items relevant to both methods.
| 4B1. | 'genpipe1' utility to convert fixed-field data to a '|' pipe delimited text. |
| - also see genpipe3 '4M1' to increase max field length from 100 to 32,767 | |
| 4B2. | - EBCDIC to ASCII conversion before pipe delimit (could combine) |
| 4B3. | - Command line instruction & arguments for genpipe1 (pipe delimit job) |
| 4B3. | - OPtions for genpipe1, prompts to allow change |
| 4B5. | - listing of generated job to create pipe|delimited file |
| 4B6. | - combining EBCDIC to ASCII & pipe|delimit to save execution time |
| - also see genfixd3 '4N1' to convert pipe delimited back to fixed layout |
| 4C1. | 'Executing' the generated jobs to convert data to pipe delimited text. |
| 4C2. | listings of test/demo input/output files |
| 4D1. | 'Modifying' the generated jobs to convert data to pipe delimited text. |
| - options for different field delimiters, etc | |
| 4D2. | Option to input EBCDIC file directly |
| 4D3. | Option s0/s1 for Intel or RISC machines |
| 4E1. | 'listpipe1' lists a '|' delimited output file to verify conversion. |
| The copybook fieldnames are listed beside the data field contents. |
| 4F1. | Converting Data Files to ASCII Delimited for loading DB tables |
| - generate SQL scripts to create & load table | |
| 4F2. | Oracle SQL demo - CREATE & LOAD table |
| 4F3. | Oracle 'batch-file' to create table |
| 4F4. | Oracle SQL script to LOAD table |
| 4F5. | create & load table using SQL scripts |
| - using SQL*PLUS to verify table loaded OK | |
| - using SQL script to list table rows |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 4G1. | script 'gencnvD1' to generate all conversion jobs including the scan data |
| jobs that assist in creating copybooks for sites that do not have copybooks | |
| (signscan2, sign2cpy1, cobmap1, uvdata51, genpipe1, sqlcreate1). | |
| - gencnvD1 is NOT practical, scanning for layout usually needs changes |
| 4H1. | script 'gencnvD2' generates all conversion jobs for any 1 copybook |
| (cobmap1, uvdata51, genpipe3, genfixd3, sqlcreate1). | |
| - updated Oct2014 to run genpipe3 (vs genpipe1) | |
| - this script saves time (all jobs generated with minimal effort) | |
| - usually need no manual changes (unless file has multiple record types) |
| 4H4. | script 'gencnvD9' generates ALL conversion jobs for ALL copybooks in cpys/ |
| - great time saver when you have many files to convert | |
| script 'uvcopypfx1' executes pfx1/... & pfp1/... jobs for 1 file | |
| script 'uvcopypfx2' same as uvcopypfx1, but includes uvhd & uvhdcob displays | |
| script 'uvcopypfx9' executes pfx1/... & pfp1/... jobs for ALL files |
| 4I1. | 'genrfmA' generate jobs to reformat records based on 2 copybooks |
| Example to unpack all packed/binary fields to numeric zoned fields | |
| - Use genrfmA when the application for which the data file is intended | |
| does not allow packed fields. |
| 4J1. | Convrting Delimited format back to Fixed-field format |
| 'genfixd2' generates jobs to convert 'delimited' format to 'fixed field' | |
| formats described by COBOL copybooks. This job is useful when you need to | |
| convert delimited files from DB2 'export's (on UNIX) to fixed field format | |
| that was created by DB2 'unload's (on the mainframe). Note that mainframe | |
| 'UNLOAD' creates 'Fixed-field', but unix 'EXPORT' creates 'Delimited'. |
| 4J2. | We will demonstrate using 'citytax1' demo copybook/cobmap & data file. |
| You could repeat this demo in your homedir assuming Vancouver Utilities | |
| installed & your profile updated as described at | |
| 'https://uvsoftware.ca/jclcnv1demo.htm#Part_1' |
| 4J3. | maps/citytax1 - sample copybook/map to demo genfixd2 & genpipe1 |
| 4J4. | pfq1/citytax1 - sample job to convert Delimited to Fixed-field |
| generated by genfixd2 from copybook/map | |
| 4J5. | pfp1/citytax1 - sample job to convert Fixed-field to Delimited |
| generated by genpipe1 from copybook/map |
| 4J6. | Copy pfq1 & pfp1 jobs to pfq2 & pfp2 before any modifiactions |
| Modify pfp2/citytax1 subdirs for this demo |
| 4J7. | Execute job to convert 'pipe|delimited' to 'Fixed-field' |
| d4pipe/citytax1 - sample input Pipe|Delimited | |
| d5fix/citytax1 - sample output Fixed-Field record |
| 4J8. | Execute COBOL program to process Fixed-Field format |
| - we will simply copy file for this demo | |
| Execute job to convert 'Fixed-Field' to 'pipe|delimited' | |
| d6fix/citytax1 - sample input Fixed-Field record | |
| d7pipe/citytax1 - sample output Pipe|Delimited | |
| Execute 'diff' to prove delimited output same as original input |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 4J9. | Demo BIG-end/little-end binary & file compare utility 'uvcmp1' |
| - Re-execute delimited to fixed with option 's0' to create Little-End | |
| - Compare to prior Big-end file | |
| - compare report highlights differences via '*'s & vertical hex |
| 4J10. | Understanding BIG-end & Little-end binary |
| - illustrate converting hex to decimal, proving same value as input |
| 4K1. | genpipe1 will expand 'item occurs' to pipe|delimit occurs fields |
| - demo using copybook armaster.cpy with occurs 12 for monthly sales |
| 4L1. | cpyoccurs1 will expand group occurs for genpipe1 pipe|delimit |
| - demo using copybook custran1 with group occurs 5 for 3 pics |
In Oct 2014, we developed genpipe3/genfixd3 (alternates to genpipe1/genfixd2), to remove the restriction of 100 bytes max field length (& also faster).
| 4M1. | 'genpipe3' utiltity to convert data to a '|' pipe delimited text. |
| - Op. Instrns. & options | |
| 4M2. | sample copybook/cobmap input maps/citytax1 |
| - field type conversion rules | |
| 4M3. | sample generated job pfp1/citytax1 |
| 4M4. | sample job execution & results |
| - input d2asc/citytax1 (uvhd display) | |
| - output d4pipe/citytax1 (pipe|delimited text, numeric fields edited) |
| 4N1. | 'genfixd3' utiltity to convert delimited files back to fixed-field |
| - Op. Instrns. & options | |
| 4N2. | sample copybook/cobmap input maps/citytax1 |
| - field type conversion rules | |
| 4N3. | sample generated job pfp1/citytax1 |
| 4N4. | sample job execution & results |
| - input d4pipe/citytax1 (pipe|delimited text, numeric fields edited) | |
| - output d5fix/citytax1 (uvhd display) |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| cobmap1 |
|
| uvdata51 |
|
| uvdata51p |
|
| genpipe1 |
|
| sqlcreate1 |
|
| Note |
|
| cobmapA |
|
| gencnvA |
|
| gencnvAp |
|
| genpipeA |
|
| sqlcreateA |
|
| gencnvD2 |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| gencnvD2 |
|
gencnvD2 citytax1.cpy <-- script to generate all jobs + batch-file + loader-file ===================== - from the demo copybook 'cpys/citytax1'
| Note |
|
| gencnvD9 |
|
uvcopypfx1 citytax1 <-- executes pfx1/citytax1 & pfp1/citytax1 for d1ebc/citytax1
=================== - also generates listpipe/citytax1 to check pipe delimited
- 1st 3 records listed vertically with copybook fieldnames
uvcopypfx2 citytax1 <-- same as above, but also creates uvhd & uvhdcob displays
=================== - in subdirs uvhd_d1ebc, uvhd_d2asc, uvhdcob (d2asc only)
(makes subdirs if not present)
uvcopypfx9 all <-- executes ALL pfx1/... & pfp1/... for ALL datafiles d1ebc/...
============== - also generates listpipe/... to check pipe delimited
- 1st 3 records listed vertically with copybook fieldnames
| copysub2 |
|
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
/home/uvadm/mvstest/mvstest1 <-- location in Vancouver Utilities distribution /home/mvstest1 <-- OR setup user mvstest1 /home/userxx OR setup cnvdata1 within your $HOME dir :-----cnvdata1 <-- cnvdata1 superdir for DATA conversion : :-----archive - backup to reload d1ebc/... copybooknames or datafilenames : : :-----d1ebc_cpy_names <-- for Parts3,4,5 : : :-----d1ebc_data_names <-- for Part6 : :-----cpys - copybooks : :-----ctl - control files : :-----d1ebc - EBCDIC data files : :-----d2asc - ASCII data files (converted by uvcopy uvdata51) : :-----d3ebc - convert back to EBCDIC by uvdata31 : :-----d3asc - reformatted data files (see genrfmA) : :-----d4pipe - pipe delimited to load DB tables (by genpipe1) : :-----d5fix - convert pipe|delimited back to fixed field (genfixd2) : :-----listpipe - pipe|delimited listed vertically with copybook fieldnames : :-----maps - copybooks with record layouts on right side : :-----pfx1 - uvcopy jobs to convert EBCDIC to ASCII (gen by uvdata51) : :-----pfx2 - with data filenames inserted (vs copybook names) : :-----pfx3 - copied here for modify/execute : :-----pfy1 - uvcopy jobs to convert ASCII to EBCDIC (gen by uvdata31) : :-----pfp1 - uvcopy jobs to create pipe delimited (gen by genpipe1) : :-----pfq1 - uvcopy jobs to convert pipe|delimited to fixed field (genfixd2) : :-----pfg1 - uvcopy jobs to create test data via copybooks (gentest1) : :-----pf - misc uvcopy jobs : :-----sf - misc scripts : :-----signs - record layouts created by signscan2 : :-----sqlTC1 - scripts to Create database tables (gen from copybooks) : :-----sqlTL1 - scripts to Load database tables (gen from copybooks) : :-----tmp - temp files (keep working directory clean)
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd d2asc/citytax1 r128 <-- display sample input data with 'uvhd'
======================== - note packed fields in bytes 88-106
- will show only 1st 2 of 10 records in the file
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
3333333322444424445522222222222222233332445442544422222222225444
1013014000AF8E085E29000000000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H1......W.........qq.000149061970530191809
4554522222222222445353430888005880008000770333333333333333333333
F56520000000000023693181029C0072C0047C0111C000149061970530191809
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
128 10139923 GEORGE BROWN 1250 EAST PENDER STREET VANC
3333333322444544245454222222222222233332445525444452555445225444
101399230075F275022F7E0000000000000125005134005E45203425540061E3
64 OUVER BCV5L1W1......W.........xRL00014626q980601190156
4554522222222222445343530888005880008800754333333337333333333333
F5652000000000002365C171029C0072C0047C0182C000146261980601190156
folio|name|address|city|province|zip|post-date|land-value|land-value|improv-value|face-value|maint-tax|purchase-date|purchase-d 10130140|JOHN HENRY|1815 BOWEN ROAD|VANCOUVER|BC|V9S1H1|+82898|+57828|+4870|+171710|+1490.61|970530| | 10139923|GEORGE BROWN|1250 EAST PENDER ST.|VANCOUVER|BC|V5L1W1|+82898|+57828|+4878|+178524|-1462.61|980601| | 10147615|BONNIE SMITH|44430 YALE ROAD WEST|VANCOUVER|BC|V2P6J1|+121395|+39623|+0|+51914|+376.92|950601| | 10149304|TONY TATTERSAL|3350 S.W. MARINE DRIVE|VANCOUVER|BC|V6N3Y9|+50594|+76800|+0|+208391|+1925.03|930601| | 10154688|MEL RAHAL|1110 - 625 HOWE STREET|VANCOUVER|BC|V6C2T6|+82898|+44880|+0|+125038|+1154.79|961126| | 10173320|SAM SMITH|2381 ROGERS AVE|VANCOUVER|BC|V3K5Y2|+50498|+60600|+0|+169086|+1562.00|970530| | 10204700|JOHN JENSEN|12611-100 ST|VANCOUVER|AL|T8V4H2|+122396|+15720|+0|+43744|-404.40|961126| | 10223240|BOB ANDERSON|7215 ARGYLL ROAD|VANCOUVER|AL|T6C4J2|+52899|+3597|+0|+9990|-90.99|980601| | 10308685|TOM PERRY|3932 - 3A ST. N.W.|VANCOUVER|AL|T2E6R4|+70192|+68155|+0|+125773|+1065.10|920601| | 10318833|HARRY SMIRNOFF|BOX 308, STN J|VANCOUVER|AL|T2A4X6|+11399|+54541|+42046|+93502|+761.46|970530| |
The output file will be an all text ASCII file and can be easily listed with the 'uvlp' scripts or other unix utilities. We will later illustrate another utility 'sqlcreate1' to generate SQL scripts to create & load tables from the pipe delimted files into Relational Data Base tables (see page '4F1').
The 1st record with copybook fieldnames was generated by genpipe1 option 'w1'. Illustrated only here, option default w0 will be used on following generations.
Above also shows '+' signs on numeric signed fields by option 'e2'. Default option e1 will show only '-' signs if negative (omitting '+' if positive).
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvcopy cobmap1 uvcopy genpipe1
copybook --------------> cobmap ---------------> uvcopy job (to pipe delimit)
cpys/citytax1.cpy maps/citytax1 pfp1/citytax1
We will present 2 ways to generate the pipe delimit job - run the 2 uvcopy jobs separately (cobmap1 & genpipe1), OR run the script genpipeA which runs the 2 jobs within the script. The only input required by genpipeA is the COBOL copybook for the data file.
#1. genpipeA citytax1.cpy <-- script to generate pipe|delimit job
=====================
| Note |
|
#1a. uvcopy cobmap1,fili1=cpys/citytax1.cpy,filo1=maps/citytax1
==========================================================
- convert any 1 copybook to a 'cobmap' (record layout)
#1b. uvcopy uvdata51,fili1=maps/citytax1,filo1=pfx1/citytax1
=======================================================
- convert cobmap to uvcopy job to convert EBCDIC to ASCII
#1c. uvcopy genpipe1,fili1=maps/citytax1,filo1=pfp1/citytax1
=======================================================
- convert data file to pipe|delimited (unpacks/edits packed fields)
cobmap1 COBOL record layout 2012/02/19_09:30:13 pg# 0001
mf/cpys/citytax1.cpy citytax1 RCSZ=00128 bgn end lth typ
* citytax1 copybook
01 citytax1.
10 folio pic x(00010). 0000 0009 010
10 name pic x(00025). 0010 0034 025
10 address pic x(00025). 0035 0059 025
10 city pic x(00020). 0060 0079 020
10 province pic x(00002). 0080 0081 002
10 zip pic x(00006). 0082 0087 006
10 post-date pic s9(00007) comp-4. 0088 0091 004bns 7
10 land-value pic s9(00009) comp-3. 0092 0096 005pns 9
10 improv-value pic s9(00009) comp-3. 0097 0101 005pns 9
10 face-value pic s9(00009) comp-3. 0102 0106 005pns 9
10 maint-tax pic s9(00007)v99. 0107 0115 009 ns 9
10 purchase-date pic x(00006). 0116 0121 006
10 filler001 pic x(00006). 0122 0127 006
*RCSZ=00128
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1b. uvcopy uvdata51,fili1=maps/citytax1,filo1=pfx1/citytax1
=======================================================
# citytax1 - uvcopy job, EBCDIC to ASCII, preserve packed, fix signs
opr='citytax1 - uvcopy code generated from copybook: citytax1.cpy '
uop=q0
was=a33000b33000
fili1=?d1ebc/citytax1,rcs=00128,typ=RSF
filo1=?d2asc/citytax1,rcs=00128,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00128),a0 move rec to outarea before field prcsng
tra b0(00128) translate entire outarea to ASCII
# --- <-- insert R/T tests here for redefined records
mvc b88(4),a88 bns post-date
mvc b92(15),a92 pns land-value:face-value
trt b107(9),$trtsea ns maint-tax
put1 put filo1,b0
skp loop
eof cls all
eoj
The generated job (listed above) would be executed as follows
2b. uvcopy pfx1/citytax1
====================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| Note |
|
#3a. uvcopy genpipe1,fili1=maps/citytax1,filo1=pfp1/citytax1\
========================================================
,arg1=delimiter,arg2=enclose-character,uop=translate-option
===========================================================
,arg1=7C,arg2=20,uop=a0 <-- default arguments & options
=======================
- arg1 & arg2 must be specified in hex (if specified)
,arg1=7C - delimiter '|' x'7C' (the default, no need to specify)
,arg1=2C - alternate delimiter ',' x'2C' - see example page '4D1'
,arg1=FC - example for umlat x'FC' (u with 2 dots above)
arg2 - enclose char for A/N fields, numeric fields not enclosed
,arg2=20 - field enclose char (blank x'20' default for none)
,arg2=22 - alternate enclose-char " x'22' - see example page '4D1'
,uop=a0 - do NOT translate input data (a0/off is the default)
,uop=a1 - translate input from EBCDIC to ASCII (see page '4D2')
#3b. uvcopy genpipe1,fili1=maps/citytax1,filo1=pfp1/citytax1
=======================================================
- only need to specify I/O files if default delimiter & no translate OK
#3c. genpipeA citytax1.cpy <-- easy script equivalent of above (shorter command)
===================== - use this if defaults OK for arg1, arg2, uop
- combines cobmap1 & genpipe1
In Oct 2014, we developed genpipe3/genfixd3 (alternates to genpipe1/genfixd2), to remove the restriction of 100 bytes max field length (& also faster). The new jobs (genpipe3/genfixd3) are documented begining on pages '4M1' & '4N1'.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| Note |
|
genpipe1 - generate uvcopy code to convert files to | delimited format
- default delimiter is pipe "|", modify via arg1=xx (in hex)
- arg1=2C for comma, arg1=FC for umlat (u with 2 dots over)
arg2 may specify a data enclose character for A/N fields (vs numeric)
- must be specified as 2 hex digits (arg2=22 for double quote)
uop=a0d0e1f1m0p0s0w0 - default options
a0 - file already translated from EBCDIC to ASCII by uvdata51
a1 - translate data from EBCDIC to ASCII here in genpipe1 job
- must transfer any packed field instrns from the uvdata51
(subdir pfx1) to this genpipe1 job (subdir pfp1)
d0 - do not test/convert any date/time fields
d1 - convert date/time zero/blank input to blank/null output
d2 - convert date/time zero/blank input to zeros output
d4 - if Macrosoft date8 - insert bal cnvrt 00000000 to |null|
e1 - edit signed fields default: leading "-" only if negative
e2 - edit signed fields with + if pos (as well as - if neg)
f0 - drop fieldnames begining with "filler"
f1 - retain fieldnames begining with "filler"
m1 - inhibit msg "field length > 100" (filler usually not problem)
p1 - convert invalid packed fields to blank
s0 - auto Binary Switch (Big-END AIX/SUN, little-end Intel)
- use option s1/s2 only if generating code for other machines
s1 - force no switch (default on BIG-end machines (AIX/SUN)
s2 - force switch ends (default on Little-end machines (Intel)
w1 - generate code to write fieldnames as 1st output record
131021:121828:genpipe1: uvcopy ver=20131008 pf=/home/uvadm/pf/IBM/genpipe1
uvcopy LNX H64 license=20131008_99V_930630 site=UV_Software
User OPtion (uop) defaults = q1a0d0e1f1m0p0s0w0
-->null to accept or enter/override -->
You may enter 1 or more options to override the defaults, for example:
--> e2w1 <-- e2 will insert '+' signs if positive (as well as '-' if negative)
- w1 will generate 1st record with copybook fieldnames
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# citytax1 - uvcopy job to convert EBCDIC/ASCII data to delimited text file
opr='citytax1 - uvcopy code generated from copybook: citytax1.cpy '
rop=j200000r1 #Run OPtions: increase instrn storage & prompt display output
was=a33000b33000c999000d33000e33000u16384v16384 #increase u&v if fields > 163
fili1=?d2asc/citytax1,rcs=00128,typ=RSF
filo1=?d4pipe/citytax1,rcs=00384,typ=LSTtd3
@run
opn all
cat j0(4000),'folio|name|address|city|province|zip|post-date|land-value|'
cat j0(4000),'improv-value|face-value|maint-tax|purchase-date|filler001|'
put filo1,j0(4000) write fieldnames as 1st record
loop get fili1,a0
skp> eof
# area a input, see 'get' instrn
# area b translated to ASCII, in case mainframe EBCDIC file (with packed?)
# area c data fields fixed 100 bytes apart in prep for var | delimit instrn
# area d output, see 'put' instrn at end
mvc b0(00128),a0 move input area a to area b
### tra b0(00128) uncomment for EBCDIC input (or option a1)
# --- <-- & insert instrns from uvdata51 if packed fields
# --- <-- insert R/T tests if redef records
mvc c0(10),b0(10) #1 folio
mvc c100(25),b10(25) #2 name
mvc c200(25),b35(25) #3 address
mvc c300(20),b60(20) #4 city
mvc c400(2),b80(2) #5 province
mvc c500(6),b82(6) #6 zip
edta1 c600(9),a88(4bs),'-zzzzzzzzz9' #7 post-date
edta1 c700(11),a92(5p),'-zzzzzzzzz9' #8 land-value
edta1 c800(11),a97(5p),'-zzzzzzzzz9' #9 improv-value
edta1 c900(11),a102(5p),'-zzzzzzzzz9' #10 face-value
edta1 c1000(11),b107(9),'-zzzzzzz.99' #11 maint-tax
mvc c1100(6),b116(6) #12 purchase-date
mvc c1200(6),b122(6) #13 filler001
rep c0(01300),'|','!' #comment speedup if no '|'s in data
varb1 d0(01300),c0(100),0013,'|' #insert '|'s between fields & squeeze
trt d0(01300),$trtchr #ensure all ascii chars x'20'-x'7E'
clr c0(01300),' '
put1 put filo1,d0
skp loop
#
eof cls all
eoj
@pf2=genpipe1.sub
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
You could combine the 2 generated conversion jobs above - EBCDIC to ASCII (on page '4B2') & pipe|delimit (on page '4B5'). This would save execution time when you have very large files to convert.
Note that the generated pipe-delimit job contains the following ##commented out instruction:
## tra b0(00128) uncomment for EBCDIC input (or option a1) # --- <-- & insert instrns from uvdata51 if packed fields # --- <-- insert R/T tests if redef records
Note that the generated EBCDIC to ASCII job contained the 3 folowing instructions (to preserve packed/binary fields & correct zoned signs).
mvc b88(4),a88 bns post-date
mvc b92(15),a92 pns land-value:face-value
trt b107(9),$trtsea ns maint-tax
So you could save the EBCDIC to ASCII preliminary step if you:
The result would be as follows (showing a few surrounding instrucitons)
mvc b0(00128),a0 move input area a to area b
tra b0(00128) uncomment for EBCDIC input (or option a1)
# --- <-- & insert instrns from uvdata51 if packed fields
# --- <-- insert R/T tests if redef records
# following 3 instrns transferred from EBCDIC to ASCII job to save execution time
mvc b88(4),a88 bns post-date
mvc b92(15),a92 pns land-value:face-value
trt b107(9),$trtsea ns maint-tax
#
mvc c0(10),b0(10) #1 folio
mvc c100(25),b10(25) #2 name
mvc c200(25),b35(25) #3 address
... see all on page '4B5' ...
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
In case manual changes are later required, I recommend you copy the jobs from subdir pfp1/... (generation) to subdir pfp2/... (for execution). This will protect any manual changes you might make in pfp2/... in case you later run mass re-generations to pfp1/...
Manual changes will be required for files with multiple record types/layouts in the same file. 'citytax1' has only 1 record type, so we will execute it now.
#1. cp pfp1/citytax1 pfp2 - copy auto-generated job to pfp2, before
===================== optional insert datafilenames vs copybooknames
#2. cp pfp2/citytax1 pfp3 - copy to pfp3
===================== before any manual changes &/or execution
#2. vi pfp3/citytax1 - make any manual changes required
================ (record type tests, see example page '5B3')
#3a. uvcopy pfp3/citytax1,fili1=d2asc/citytax1,filo1=d4pipe/citytax1
===============================================================
#3b. uvcopy pfp3/citytax1 <-- easier (filenames default as shown above)
====================
#4. uvhd d2asc/citytax1 r128
========================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
3333333322444424445522222222222222233332445442544422222222225444
1013014000AF8E085E29000000000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H1......W.........qq.000149061970530191809
4554522222222222445353430888005880008000770333333333333333333333
F56520000000000023693181029C0072C0047C0111C000149061970530191809
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
128 10139923 GEORGE BROWN 1250 EAST PENDER STREET VANC
3333333322444544245454222222222222233332445525444452555445225444
101399230075F275022F7E0000000000000125005134005E45203425540061E3
64 OUVER BCV5L1W1......W.........xRL00014626q980601190156
4554522222222222445343530888005880008800754333333337333333333333
F5652000000000002365C171029C0072C0047C0182C000146261980601190156
Only 1st 2 records of input file are shown above since we must use uvhd (vs vi/lp/uvlp12/etc) because the data contains packed fields & no linefeeds.
We will show all records of the output file on the next page, since the output data is now a text file with linefeeds & easily printed or displayed.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#5. more d4pipe/citytax1 <-- examine output (or vi/cat/lp/uvlp14/etc)
====================
10130140|JOHN HENRY|1815 BOWEN ROAD|VANCOUVER|BC|V9S1H1|+82898|+57828|+4870|+171710|+1490.61|1997/05/30|191809| 10139923|GEORGE BROWN|1250 EAST PENDER STREET|VANCOUVER|BC|V5L1W1|+82898|+57828|+4878|+178524|-1462.61|1998/06/01|190156| 10147615|BONNIE SMITH|44430 YALE ROAD WEST|VANCOUVER|BC|V2P6J1|+121395|+39623|+0|+51914|+376.92|1995/06/01|201031| 10149304|TONY TATTERSAL|3350 S.W. MARINE DRIVE|VANCOUVER|BC|V6N3Y9|+50594|+76800|+0|+208391|+1925.03|1993/06/01|201512| 10154688|MEL RAHAL|1110 - 625 HOWE STREET|VANCOUVER|BC|V6C2T6|+82898|+44880|+0|+125038|+1154.79|1996/11/26|082351| 10173320|SAM SMITH|2381 ROGERS AVE|VANCOUVER|BC|V3K5Y2|+50498|+60600|+0|+169086|+1562.00|1997/05/30|191810| 10204700|JOHN JENSEN|12611-100 ST|VANCOUVER|AL|T8V4H2|+122396|+15720|+0|+43744|-404.40|1996/11/26|081702| 10223240|BOB ANDERSON|7215 ARGYLL ROAD|VANCOUVER|AL|T6C4J2|+52899|+3597|+0|+9990|-90.99|1998/06/01|190157| 10308685|TOM PERRY|3932 - 3A ST. N.W.|VANCOUVER|AL|T2E6R4|+70192|+68155|+0|+125773|+1065.10|1992/06/01|221650| 10318833|HARRY SMIRNOFF|BOX 308, STN J|VANCOUVER|AL|T2A4X6|+11399|+54541|+42046|+93502|+761.46|1997/05/30|191811|
The output file is now an all text ASCII file and can be easily listed with the 'uvlp' scripts or other unix utilities. The input file requires a utility such as uvhd to display the packed fields.
#6. uvlp14L d4pipe/citytax1 e1 <-- list 1st page (e1=end after 1 page)
==========================
#6a. uvlp20L d4pipe/citytax1 e1 <-- use uvlp20L for long records
========================== - prints 200 chars per line Landscape
- - - OR for WindowsDOS, use 'uvlpr...' scripts - - -
#6. uvlpr14L d4pipe/citytax1 e1 <-- list 1st page (e1=end after 1 page)
===========================
#6a. uvlpr20L d4pipe/citytax1 e1 <-- use uvlp20L for long records
=========================== - prints 200 chars per line Landscape
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
arg1 may specify an alternate field separator (vs '|' delimiter). If used, arg1 must be specified as 2 digit hexadecimal value. For example: arg1=2C defines a comma ',' & arg1=FC defines an umlat (u with 2 dots above).
arg2 may specify a character to enclose alpha-numeric data fields. If used, arg2 must be specified as 2 digit hexadecimal value. For example: arg2=22 for '"' double quote, arg2=5E for '^' caret.
Here is the command to regenerate the delimit job specifying the separator as a comma x'2C' & the data enclose character as a double quote x'22'. Followed by the command to execute the job, followed by a display of 1st few records.
#1. uvcopy genpipe1,fili1=maps/citytax1,filo1=pfp1/citytax1a,arg1=2C,arg2=22
========================================================================
#2. cp pfp1/citytax1a pfp2 - copy to standard execution pfp2 subdir
======================
#4. uvcopy pfp2/citytax1a,fili1=d2asc/citytax1,filo1=d4pipe/citytax1
================================================================
"10130140 ","JOHN HENRY ","1815 BOWEN ROAD ","VANCOUVER ","BC" ,"V9S1H1",+82898,+57828,+4870,+171710,+1490.61,0000/00/00,"191809", "10139923 ","GEORGE BROWN ","1250 EAST PENDER STREET ","VANCOUVER ","BC" ,"V5L1W1",+82898,+57828,+4878,+178524,-1462.61,0000/00/00,"190156", "10147615 ","BONNIE SMITH ","44430 YALE ROAD WEST ","VANCOUVER ","BC" ,"V2P6J1",+121395,+39623,+0,+51914,+376.92,0000/00/00,"201031", "10149304 ","TONY TATTERSAL ","3350 S.W. MARINE DRIVE ","VANCOUVER ","BC" ,"V6N3Y9",+50594,+76800,+0,+208391,+1925.03,0000/00/00,"201512", "10154688 ","MEL RAHAL ","1110 - 625 HOWE STREET ","VANCOUVER ","BC" ,"V6C2T6",+82898,+44880,+0,+125038,+1154.79,0000/00/00,"082351", "10173320 ","SAM SMITH ","2381 ROGERS AVE ","VANCOUVER ","BC" ,"V3K5Y2",+50498,+60600,+0,+169086,+1562.00,0000/00/00,"191810",
I have split the records (after "BC") so you can see the numeric fields (at the end of the record) are not enclosed in the specified enclose character. COBOL type 'x' & unpacked unsigned type '9' are enclosed, while type '9' signed or packed or binary fields are not enclosed.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Conversion from EBCDIC to ASCII is normally a pre-requisite to pipe-delimiting. For large files, it may be worth combining the EBCDIC to ASCII instructions into the pipe delimit job.
You might do this to save time & space for large files where you only needed the pipe delimited output. The genpipeA script generates a uvcopy job to read the ASCII file by default. If you want the generated job to read the original EBCDIC file you could run the genpipe1 job with the 'a1' option as shown below.
#1. uvcopy genpipe1,fili1=maps/citytax1,filo1=pfp1/citytax1,uop=a1 <-- generate
==============================================================
Or if already generated, you can simply uncomment the following line:
### tra b0(00128) uncomment for EBCDIC input (or option a1)
You also need to transfer the instructions (that preserve packed fields & correct zoned signs) from the EBCDIC to ASCII conversion job to the pipe delimit job at the point marked by:
# --- <-- insert instrns from uvdata51 if packed fields
From our EBCDIC to ASCII job pfx2/citytax1 listed on page '3B2', these instructions are:
mvc b88(19),a88 pns taxduedate:landbldgtax
trt b107(9),$trtsea ns f00107
Insert these instructions into the pipe delimit job (listed on page '4B5') after the '# ----' indicator line.
The execute command would then specify the EBCDIC file as input (vs the ASCII file on the execute command on the previous page).
#2a. uvcopy pfp2/citytax1,fili1=d1ebc/citytax1,filo1=d4pipe/citytax1
===========================*****===============================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Note that genpipe1 generates edit instructions with or without option 's' depending on the machine architecture - BIG-end or little-end (AIX,SUN,etc or INTEL). On page '4B5' the binary field post-date was generated as follows (on my INTEL machine).
edta1 c600(9),a88(4bs),'+zzzzzzzzz9' #7 post-date
Prior to April 2013, you had to specify user option 's0' or 's1' depending on your machine to cause edit instructions to be generated with or without instruction option 's'. See user option 'uop=s0' or 'uop=s1'prompts on page '4B4'
Now genpipe1 tests for BIG-end/little-end coded within uvcopy when uvcopy is compiled on AIX/SUN/etc vs INTEL machines.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1a. uvcopy listpipe1,fili1=d4pipe/citytax1,fili2=maps/citytax1
,filo1=listpipe/citytax1
==========================================================
#1b. listpipeA citytax1 <-- script equivalent of above (shorter command)
================== - ksh for unix/linux or .bat file for Win/Dos
uop=q1f1l1s3 - default options
f1 - FormFeed after each record, f2 every 2nd rec, etc
l1 - list data recs horizontally before vertically
s3 - stop count default 1st 3 records in file
User OPtion (uop) defaults = q1f1l1s3
null to accept or re-specify (1 or more) --> <-- null reply lists 3 records
report=tmp/citytax1, enter: vi/more/lp/etc
--> more <-- display output (shown below)
listpipe1 - list '|' delimited files with copybook fieldnames
- to verify files created by genpipe1 (see DATAcnv1.htm#Part_4)
datafile=d4pipe/filename? copybookmap=maps/filename? datetime=2003/06/09_20:26:34
SEQ# FIELDNAME COBOL PICTURE DATA
datafile = d4pipe/citytax1, record# = 1, field count = 13
10130140|JOHN HENRY|1815 BOWEN ROAD|VANCOUVER|BC|V9S1H1|+82898|+57828|+4870|+171 710|+1490.61|1997/05/30|191809|
001 folio x(00010). 10130140 002 name x(00025). JOHN HENRY 003 address x(00025). 1815 BOWEN ROAD 004 city x(00020). VANCOUVER 005 province x(00002). BC 006 zip x(00006). V9S1H1 007 post-date s9(00007) comp-4. +82898 008 land-value s9(00009) comp-3. +57828 009 improv-value s9(00009) comp-3. +4870 010 face-value s9(00009) comp-3. +171710 011 maint-tax s9(00007)v99. +1490.61 012 purchase-date x(00006). 1997/05/30 013 filler001 x(00006). 191809
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
cobmap1 COBOL record layout 2012/02/19_09:30:13 pg# 0001
mf/cpys/citytax1.cpy citytax1 RCSZ=00128 bgn end lth typ
* citytax1 copybook
01 citytax1.
10 folio pic x(00010). 0000 0009 010
10 name pic x(00025). 0010 0034 025
10 address pic x(00025). 0035 0059 025
10 city pic x(00020). 0060 0079 020
10 province pic x(00002). 0080 0081 002
10 zip pic x(00006). 0082 0087 006
10 post-date pic s9(00007) comp-4. 0088 0091 004bns 7
10 land-value pic s9(00009) comp-3. 0092 0096 005pns 9
10 improv-value pic s9(00009) comp-3. 0097 0101 005pns 9
10 face-value pic s9(00009) comp-3. 0102 0106 005pns 9
10 maint-tax pic s9(00007)v99. 0107 0115 009 ns 9
10 purchase-date pic x(00006). 0116 0121 006
10 filler001 pic x(00006). 0122 0127 006
*RCSZ=00128
#1. uvcopy cobmap1,fili1=cpys/citytax1.cpy,filo1=maps/citytax1
==========================================================
- convert copybook to 'cobmap' (record layout)
#2. uvcopy sqlcreate1,fili1=maps/citytax1,filo1=sqlTC1/create_citytax1.sql
======================================================================
,filo2=sqlTL1/load_citytax1.ctl
====================================================================
- convert cobmap to SQL scripts to create & load table
- - - OR use script 'sqlcreateA' (combines above into 1 short command)
#12. sqlcreateA citytax1 <-- script equivalent of above (shorter command)
=================== - ksh for unix/linux or .bat file for Win/Dos
#3a. more sqlTC1/create_citytax1.sql <-- display batch-file output
===============================
#3b. more sqlTL1/load_citytax1.ctl <-- display SQL script output
=============================
See the output files listed on the next page --->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. Login demo1 --> /home/demo1
===========
#2. sqlplus demo1/demo1pw <-- start SQL*Plus
=====================
#3. SQL> create table citytax1 (folio char (10), name char (25),
address char (25), city char (20), province char (02),
zip char (06), post_date date, land_value number (09),
improv_value number (09), face_value number (09),
maint_tax number (09,2), purchase_date date);
#4. SQL> describe citytax1
=================
Name Null? Type ----------------------------------------- -------- ---------------------------- FOLIO CHAR(10) NAME CHAR(25) ADDRESS CHAR(25) CITY CHAR(20) PROVINCE CHAR(2) ZIP CHAR(6) POST_DATE DATE LAND_VALUE NUMBER(9) IMPROV_VALUE NUMBER(9) FACE_VALUE NUMBER(9) MAINT_TAX NUMBER(9,2) PURCHASE_DATE DATE
#5. SQL> exit <-- exit to unix shell to run SQLLDR
====
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Creating tables manually (as shown above) would be very laborious for files with many fields. UV Software provides uvcopy job 'sqlcreate1' to automatically generate an Oracle batch-file to create the table. The job also generates the SQL*LOADER script to load the table. See 'sqlcreate1' fully documented in DATAcnv1.htm#Part_4.
/* citytax1.sql - SQL batch file to create a table */ /* - generated from COBOL copybook 20131021:123340 */ /* www.uvsoftware.ca/datacnv1.htm#Part_4 & sqldemo.htm#Part_4 */ /* */ /* sqlplus user/pass @sqlTC2/citytax1.sql */ /* ====================================== */ /* */ DROP TABLE citytax1; CREATE TABLE citytax1 ( folio char (0010) , --#001 x(00010). name char (0025) , --#002 x(00025). address char (0025) , --#003 x(00025). city char (0020) , --#004 x(00020). province char (0002) , --#005 x(00002). zip char (0006) , --#006 x(00006). post_date number (07) , --#007 s9(00007) comp-4. land_value number (09) , --#008 s9(00009) comp-3. improv_value number (09) , --#009 s9(00009) comp-3. face_value number (09) , --#010 s9(00009) comp-3. maint_tax number (09,2) , --#011 s9(00007)v99. purchase_date char (0006) , --#012 x(00006). filler001 char (0006) --#013 x(00006). ); exit;
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
-- citytax1.ctl - SQL*LOADER script to load Oracle table -- - generated from COBOL copybook 20131021:123340 -- www.uvsoftware.ca/datacnv1.htm#Part_4 & sqldemo.htm#Part_4 -- -- sqlldr user/pass control=sqlTL2/citytax1.ctl -- ============================================ -- load data infile 'd4pipe/citytax1' into table citytax1 fields terminated by '|' optionally enclosed by '"' ( folio , --char (0010) , --#001 x(00010). name , --char (0025) , --#002 x(00025). address , --char (0025) , --#003 x(00025). city , --char (0020) , --#004 x(00020). province , --char (0002) , --#005 x(00002). zip , --char (0006) , --#006 x(00006). post_date , --number (07) , --#007 s9(00007) comp-4. land_value , --number (09) , --#008 s9(00009) comp-3. improv_value , --number (09) , --#009 s9(00009) comp-3. face_value , --number (09) , --#010 s9(00009) comp-3. maint_tax , --number (09,2) , --#011 s9(00007)v99. purchase_date , --char (0006) , --#012 x(00006). filler001 --char (0006) --#013 x(00006). )
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. Login demo1 --> /home/demo1
===========
#2. sqlplus user/pass @sqlTC1/create_citytax1.sql
=============================================
- run SQL 'batch-file' to create the table
#3. sqlldr user/pass control=sqlTL1/load_citytax1.ctl
=================================================
- run SQL*LOADER script to load the table
#4. sqlplus demo1/demo1pw <-- start SQL*Plus
=====================
#5. SQL> select * from citytax1 <-- SELECT all rows/fields from table
====================== - headings & 1st 2 rows shown below:
FOLIO NAME ADDRESS CITY PR ZIP POST_DATE LAND_VALUE IMPROV_VALUE FACE_VALUE MAINT_TAX PURCHASE_D ============================================================================
10130140 JOHN HENRY 1815 BOWEN ROAD
VANCOUVER BC V9S1H1 82898 57828 4870 171710
1490.61 1997/05/30
10139923 GEORGE BROWN 1250 EAST PENDER STREET VANCOUVER BC V5L1W1 82898 57828 4878 178524 -1462.61 1998/06/01
/* select_all_citytax1.sql - batch file to select all rows from citytax1 */ /* - batch files useful for complex repeated selects */ select * from citytax1; exit;
You could use the batch-file listed above as an alternative to logging in to SQL*PLUS to list the table rows:
#6. SQL> exit <-- exit from SQL*PLUS (back to Unix/Linux prompt)
#7. sqlplus user/pass @sqlTC1/select_all_citytax1.sql
=================================================
- run SQL 'batch-file' to create the table
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'gencnvD1' - script to generate all conversion jobs when copybooks ABSENT.
- signscan2, sign2cpy1, cobmap1, uvdata51, genpipe1, sqlcreate1
'gencnvD2' - script to generate all conversion jobs when copybooks PRESENT.
- cobmap1, uvdata51, uvdata31, genpipe3, genfixd3, sqlcreate1
- updated Oct2014 to run genpipe3/genfixd3 (vs genpipe1/genfixd2)
| gencnvD2p |
|
'gencnvA' - script to generate most commonly used conversion jobs
- cobmap1, uvdata51, genpipe3, genfixd3
- updated Oct2014 to run genpipe3 (vs genpipe1)
I suggest you convert your 1st file using the individual scripts or uvcopy jobs (as described earlier in this documentation), to gain a good understanding of these procedures. Running them 1 by 1 allows you to examine the various outputs & check for any errors.
Use 'gencnvD1' when no COBOL copybook is available & you need to scan the datafile to create the copybook. Use alternate script 'gencnvD2' when the COBOL copybook is available (skips signscan2 & sign2cpy1 jobs).
The input EBCDIC datafile must be stored in the d1ebc subdir & the outputs will be named the same as the input datafile & will be stored in separate subdirs (signs, cpys, maps, pfx1,pfx2,pfx3 pfp1,pfp2,pfp3,& sqls).
| Note |
|
SQLconvert.htm - convert flat files to pipe delimited for Microsoft SQL server
- driven by COBOL copybooks
- also generate scripts to create & load tables
- generates jobs for all copybooks & all data files
(vs this DATAcnv1.doc for 1 copybook/file at a time)
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Script 'gencnvD2' runs conversion GENERATION jobs based on CopyBooks. (cobmap1, uvdata51, uvdaaata31, genpipe3, genfixd3, sqlcreate1, gentest1). gencnvD2 was updated Oct2014 to run genpipe3/genfixd3 (vs genpipe1/genfixd2) See listing of gencnvD2 on page '10D1'. See subdirs required on page '1B4'.
#1. gencnvD2 file-name <-- command format
==================
#1a. gencnvD2 citytax1.cpy <-- example for test/demo file
=====================
Regardless of whether changes are required or not, I suggest you always copy the jobs from pfx1 to pfx3 & from pfp1 to pfp3, before executing. Often you wont realize you need modifications until after you inspect the output data.
#2a. cp pfx1/citytax1 pfx3 - copy auto-generated EBCDIC to ASCII job
===================== before any manual changes &/or execution
#2b. cp pfy1/citytax1 pfy3 - copy auto-generated ASCII to EBCDIC job
#2c. cp pfp1/citytax1 pfp3 - copy auto-gen pipe delimit job
#2d. cp pfq1/citytax1 pfq3 - copy auto-gen pipe delimit to Fixed
#2e. cp pfg1/citytax1 pfq3 - copy job to create test data from copybook
#3a. vi pfx3/citytax1 - make any manual changes required
================ (for record type tests, see page '5D2')
#3b. vi pfy3/citytax1 - make manual changes ASCII to EBCDIC job
#3c. vi pfp3/citytax1 - make manual changes to pipe delim job
#3d. vi pfq3/citytax1 - make manual changes to delim to fixed job
#3e. vi pfg3/citytax1 - make manual changes to create test data job
#4. uvcopy pfx3/citytax1 <-- converts d1ebc/citytax1 to d2asc/citytax1
==================== (EBCDIC to ASCII, preserving packed, fixing signs)
#5. uvcopy pfy3/citytax1 <-- converts d2asc/citytax1 to d3ebc/citytax1
==================== (ASCII back to EBCDIC)
#6. uvcopy pfp3/citytax1 <-- converts d2asc/citytax1 to d4pipe/citytax1
==================== (converts ASCII file to delimited text file)
#7. uvcopy pfq3/citytax1 <-- converts d4pipe/citytax1 to d5fix/citytax1
==================== (converts pipe|delimited to Fixed-field records)
#8. uvcopy pfq3/citytax1 <-- generates test data file from copybook
==================== A/N fields A,B,C,etc numeric fields 1,2,3,etc
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. vi cpys/sample1 - copybook supplied
- must be stored in cpys/...
#2. vi maps/sample1 - cobmap generated from copybook by cobmap1
- field start/end/length on right side
#3. vi pfx1/sample1 - EBCDIC to ASCII conversion job
generated by uvdata51
#4. vi pfy1/sample1 - ASCII to EBCDIC conversion job
generated by uvdata31
#5. vi pfp1/sample1 - job to convert ASCII file to text delimited
generated by genpipe1
#6. vi pfq1/sample1 - job to convert pipe|delimited to Fixed-field
generated by genfixd2
#7. vi pfg1/sample1 - job to generate test data file from copybook
generated by gentest1
#8. vi sqlTC1/sample1.sql - batch file to create table
- generated by sqlcreate1
#9. vi sqlTL1/sample1.ctl - SQL script to load table
- generated by sqlcreate1
#1. uvcopy gentest1,fili1=maps/sample1,filo1=pfg1/sample1,uop=e1
=============================================================
- generate job to create testfile
#2. vi pfg1/sample1 - inspect job to crate test file
===============
#3. uvcopy pfg1/sample1,filo1=d1ebc/sample1
=======================================
- execute job to create testfile from copybook
#4. uvhd d1ebc/sample1 r592a <-- inspect generated testfile
========================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. uvcopy pfx1/sample1 <-- convert EBCDIC to ASCII
===================
#2. uvcopy pfy1/sample1 <-- convert ASCII to EBCDIC
=================== - to return file to mainframe
#3. uvcopy pfp1/sample1 <-- create pipe|delimited data
=================== (to load RDBMS table)
#4. uvcopy pfq1/sample1 <-- convert pipe|delimited back to fixed field EBCDIC
=================== - to return file to mainframe
#5. listpipeA sample1 <-- convert pipe|delimited to vertical format
================= with copybook fieldnames
#1. uvhd d1ebc/sample1 r592a - original EBCDIC input
======================== OR testfile generated by gentest1 above
#2. uvhd d2asc/sample1 r592 - inspect ASCII output from pfx1/sample1
=======================
#3. uvhd d3ebc/sample1 r592a - inspect EBCDIC output from pfy1/sample1
========================
#4. vi d4pipe/sample1 - inspect delimited output from pfp1/sample1
=================
#4a. listpipeA sample1 - create vertical listing of delimited output with
================= copybook fieldnames beside data contents
#4b. vi listpipe/sample1 - inspect vertical listing of delimited output
===================
#5. uvhd d5fix/sample1 r592a - inspect Fixed-field converted from pipe|delimited
========================
Please see the gencnvD2 script listed on page '10D1' Linux, '10D2' Windows .bat. You can modify the various options as appropriate for your site. For example to use a semicolon as the delimiter, you would add ',arg1=\;' to line 38. If you want to generate 1st record with fieldnames, add ',uop=w1', for example:
uvcopy genpipe1,fili1=maps/$f,filo1=pfp1/$f,arg1=\;,uop=w1 ==========================================================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| gencnvD9 |
|
uvcopypfx1 citytax1 <-- executes pfx1/citytax1 & pfp1/citytax1 for d1ebc/citytax1
=================== - also generates listpipe/citytax1 to check pipe delimited
- 1st 3 records listed vertically with copybook fieldnames
uvcopypfx2 citytax1 <-- same as above, but also creates uvhd & uvhdcob displays
=================== - in subdirs uvhd_d1ebc, uvhd_d2asc, uvhdcob (d2asc only)
(makes subdirs if not present)
uvcopypfx9 all <-- executes ALL pfx1/... & pfp1/... for ALL datafiles d1ebc/...
============== - also generates listpipe/... to check pipe delimited
- 1st 3 records listed vertically with copybook fieldnames
Generating & executing the various data conversion jobs for 1 or ALL files is made easy by maintaining the same filename in multiple subdirs, but this does make it difficult to gather the results in 1 directory. You might want to do this send the results to somebody via an email atachment.
'copysub2' will gather all jobs & data for 1 file, from multiple subdirs to one output directory. It appends _subdir names on the filename, allowing cohabitation in 1 directory.
Here is an example to gather all results for 'citytax1'. Assume userxx has just completed generating & executing his data conversion jobs in his superdir /home/userxx/cnvdata1/... with the various subdirs cpys,maps,pfx1,etc.
#1. cd .. <-- change above cnvdata1
#2. mkdir tmp1 <-- make temp subdir at same level as cnvdata1
#3. copysub2 citytax1 tmp1 <-- collects all citytax1 files into tmp1
====================== - made possible by appending subdir names
#4. ls tmp1 <-- list files collected
=======
citytax1_cpys citytax1_maps citytax1_d1ebc citytax1_d2asc
citytax1_pfx1 citytax1_pfp1 ...etc...
#5. zip -r tmp1.zip tmp1 <-- zip files for email attachment etc
====================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'genrfmA' reformats data records based on old & new versions of the copybook. For example, we could unpack all packed/binary fields by copy/rename/modify the copybook removing all comp-3 & comp-4 on field pictures.
Prior to Nov2014 alternate copybook was copied to cpyu/... with same name. After Nov2014, copy/rename in same dir (cpys/citytax1u.cpy in our example).
:---cnvdata1 <-- superdir 'cnvdata1' within your homedir or mvstest1
** Library subdirs **
: :---cpys/citytax1.cpy <-- original copybook
: :---cpys/citytax1u.cpy <-- copybook with 'comp-3's & 'comp-4's removed
: :---maps/citytax1 <-- original record layout (cobmap)
: :---maps/citytax1u <-- unpacked record layout (cobmap)
: :---mapsI/citytax1 <-- original cobmap loaded to Indexed file for lookup
: :---pfr1/citytax1u <-- uvcopy job generated by step4 of genrfmA (reform2)
** Data subdirs **
: :---d2asc/citytax1 <-- input data file ASCII (may have packed/binary fields)
: :---d3asc/citytax1u <-- output file (all unpacked characters)
#1. Make directories as shown above #2. Store input files required - we will demo using following:
#2a. d2asc/citytax1 <-- data file with packed/binary fields (to be unpacked)
==============
#2b. cpys/citytax1 <-- copybook corresponding to above data file
=============
#3. cp cpys/citytax1.cpy cpys/citytax1u.cpy <-- copy/rename for alternate layout
=======================================
#4. vi cpys/citytax1u.cpy <-- edit copy to remove all 'comp-3's & 'comp-4's
=====================
| Note |
|
#5a. uvcopy cobmap1,fili1=cpys/citytax1.cpy,filo1=maps/citytax1 <-- gen INPUT cobmap
==========================================================
#5b. more maps/citytax1 <-- inspect original layout
#6a. uvcopy cobmap1,fili1=cpys/citytax1u.cpy,filo1=maps/citytax1u <-- gen OUTPUT cobmap
============================================================
#6b. more maps/citytax1u <-- inspect unpacked layout
#7. genrfmA citytax1 citytax1u <-- generate reformat job pfr1/citytax1
========================== - will re-gen 2 pages ahead & execute 3 pages ahead
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
cobmap1 COBOL record layout 2012/12/04_14:55:51 pg# 0001
cpys/citytax1.cpy citytax1 RCSZ=00128 bgn end lth typ
* citytax1 copybook for uv demos www.uvsoftware.ca/datacnv1.
01 citytax1.
10 folio pic x(00010). 0000 0009 010
10 name pic x(00025). 0010 0034 025
10 address pic x(00025). 0035 0059 025
10 city pic x(00020). 0060 0079 020
10 province pic x(00002). 0080 0081 002
10 zip pic x(00006). 0082 0087 006
10 post-date pic s9(00007) comp-4. 0088 0091 004bns 7
10 land-value pic s9(00009) comp-3. 0092 0096 005pns 9
10 improv-value pic s9(00009) comp-3. 0097 0101 005pns 9
10 face-value pic s9(00009) comp-3. 0102 0106 005pns 9
10 maint-tax pic s9(00007)v99. 0107 0115 009 ns 9
10 purchase-date pic x(00006). 0116 0121 006
10 filler001 pic x(00006). 0122 0127 006
*RCSZ=00128
cobmap1 start-end bytes for cobol record fields 200306092029 pg# 0001
cpys/citytax1u.cpy citytax1u RCSZ=00143 bgn end lth typ
* citytax1u copybook
01 citytax1u.
10 folio pic x(00010). 0000 0009 010
10 name pic x(00025). 0010 0034 025
10 address pic x(00025). 0035 0059 025
10 city pic x(00020). 0060 0079 020
10 province pic x(00002). 0080 0081 002
10 zip pic x(00006). 0082 0087 006
10 post-date pic s9(00007) . 0088 0094 007 ns 7
10 land-value pic s9(00009) . 0095 0103 009 ns 9
10 improv-value pic s9(00009) . 0104 0112 009 ns 9
10 face-value pic s9(00009) . 0113 0121 009 ns 9
10 maint-tax pic s9(00007)v99. 0122 0130 009 ns 9
10 purchase-date pic x(00006). 0131 0136 006
10 filler001 pic x(00006). 0137 0142 006
*RCSZ=00143 0143
Note that the record size has increased from 128 to 143, due to unpacking. Since there was one 4 byte field (which is now 7) & three 5 byte fields (which become 9), the output is (1*3 + 3*4) = 3+12 = 15 bytes larger. The new record is 128 + 15 = 143 bytes, confirmed by the cobmap above. The conversion job option t1 (default) will add 1 more for a terminating LF.
#7. genrfmA citytax1 citytax1u <-- generate uvcopy job to unpack packed fields
==========================
Please see the genrfmA output listed on the next page --->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#7. genrfmA citytax1 citytax1u <-- generate uvcopy job to unpack packed fields
==========================
opr='JOBNAME citytax1u - generated by cobmap1,reform2,uvdata52'
uop=q1d2m2t0z0g0n4p0s0w50x1y0q1 <-- options from reform2 gen
was=a9000b9000
fili1=?d2asc/citytax1,rcs=00128,typ=RSF
filo1=?d3asc/citytax1u,rcs=00143,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
clr b0(00143),' ' init output area to all blanks
# --- <-- insert R/T tests if redef records
mvc b0(88),a0(88) folio : zip
bal rfmbn,'a00088b07c07d00088e04f07','post-date'
bal rfmpn,'a00095b09c09d00092e05f09','land-value'
bal rfmpn,'a00104b09c09d00097e05f09','improv-value'
bal rfmpn,'a00113b09c09d00102e05f09','face-value'
mvc b122(9),a107(9) maint-tax
bal rfmnx,'a00131b06c06d00116e06f06','purchase-date'
mvc b137(6),a122(6) filler001
#
put1 put filo1,b0
skp loop
eof cls all
eoj
@pf2=reform2.sub
Since this DATAcnv1.doc is intended to be the easiest to use document for file conversions, I wont go into detail here about the generated code. If you wish more explanation & discussion about possible modifications for complex files, please see REFORMjobs.htm.
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Manual changes may be required to the auto-generated uvcopy job (usually to allow for multiple record types/layouts in the same file).
Regardless of whether changes are required or not, I suggest you always copy the jobs from pfr1 to pfr2, before executing (from pfr2). Often you wont realize you need modifications until after you inspect the output data.
#8. cp pfr1/citytax1u pfr2 - copy auto-generated job to pfr2
====================== before any manual changes &/or execution
#9. vi pfr2/citytax1u - make any manual changes required
================= (for record type tests, see page '5D2')
#10a. uvcopy pfr2/citytax1u,fili1=d2asc/citytax1,filo1=d3asc/citytax1u
================================================================
#10b. uvcopy pfr2/citytax1u <-- easier (filenames default as shown above)
=====================
Please see next page for sample data I/O listed with uvhd --->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#11. uvhd /home/mvstest1/cnvdata1/d2asc/citytax1 r128
================================================
rec#=1 rsize=128 fptr=0 fsize=1280 rcount=10
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
3333333322444424445522222222222222233332445442544422222222225444
1013014000AF8E085E29000000000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H1..C...W.........qq.000149061970530
455452222222222244535343004D005880008000770333333333333333222222
F5652000000000002369318101320072C0047C0111C000149061970530000000
#12. uvhd /home/mvstest1/cnvdata1/d3asc/citytax1u r143
=================================================
rec#=1 rsize=143 fptr=0 fsize=1430 rcount=10
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
3333333322444424445522222222222222233332445442544422222222225444
1013014000AF8E085E29000000000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H10082898000057828000004870000171710000149
4554522222222222445353433333333333333333333333333333333333333333
F565200000000000236931810082898000057828000004870000171710000149
128 061970530
333333333222222
061970530000000
Note that the record size has increased from 128 to 143, due to unpacking. Since there was one 4 byte field (which is now 7) & three 5 byte fields (which become 9), the output is (1*3 + 3*4) = 3+12 = 15 bytes larger.
Note option 't' to add record terminators if desired: Option t0 (default) does not append any record terminator Option t1 would add 1 byte for LF x'0A' only (unix). Option t2 would add 2 bytes for CR/LF x'0D0A' (dos/windows).
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'genfixd2' will generate jobs to convert 'delimited formats' to 'fixed field' formats described by COBOL copybooks. This job is useful when you need to convert delimited files from DB2 'export's (on UNIX) to the fixed field format that was created by DB2 'unload's (on the mainframe). Note that mainframe 'UNLOAD' creates 'Fixed-field', but unix 'EXPORT' creates 'delimited format.
On the mainframe, you might have:
Using 'genfixd2' on unix, you could:
Yes, this is 5 steps vs 3 steps, but it saves you from having to rewite the COBOL program to process delimited format vs fixed format.
We will demonstrate using the 'citytax1' demo copybook/cobmap & data file. You could repeat this demo in your homedir assuming Vancouver Utilities installed & your profile updated as described at https://uvsoftware.ca/jclcnv1demo.htm#Part_1
#1. Login userxx --> /home/userxx #2. mkdir cnvdata1 #3. cd cnvdata1 (or alias 'cdc') --> home/userxx/cnvdata1
#4. cnvdatadirs <-- setup subdirs (see next page)
===========
#5. mkdir d6fix d7pipe pfq2 pfp2
============================
- make subdirs for this demo (not made by cnvdatadirs)
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
/home/userxx <-- etup cnvdata1 withing your $HOME dir :-----cnvdata1 <-- cnvdata1 superdir for DATA conversion : :-----cpys - copybooks : :-----ctl - control files : :-----d1ebc - EBCDIC data files : :-----d2asc - ASCII data files (converted by uvcopy uvdata51) : :-----d3ebc - convert back to EBCDIC by uvdata31 : :-----d4pipe - pipe delimited to load DB tables (by genpipe1) : :-----d5fix - convert pipe|delimited to fixed field (genfixd2) : :--*--d6fix - output of COBOL or SORT or cp (for this demo) : :--*--d7pipe - convert back to pipe|delimited to import back to DB2 : :-----maps - copybooks with record layouts on right side : :-----pfx1 - uvcopy jobs to convert EBCDIC to ASCII (gen by uvdata51) : :-----pfx2 - with data filenames inserted (vs copybook names) : :-----pfp1 - uvcopy jobs to create pipe delimited (gen by genpipe1) : :--*--pfp2 - uvcopy jobs to create pipe delimited (gen by genpipe1) : :-----pfq1 - uvcopy jobs to convert pipe|delimited to fixed field (genfixd2) : :--*--pfq2 - uvcopy jobs to convert pipe|delimited to fixed field (genfixd2) : :-----tmp - temp files (keep working directory clean)
| Note |
|
#5. cp $UV/mvstest1/cnvdata1/d4pipe/citytax1 d4pipe/citytax1
========================================================
- copy demo datafile, here are 1st few records
ctx-folio|ctx-name|ctx-address|ctx-city|ctx-province|ctx-zip|ctx-post-date|ctx-land-value|ctx-imp-value|ctx-face-value|ctx-main 10130140|JOHN HENRY|1815 BOWEN ROAD|VANCOUVER|BC|V9S1H1|82898|57828|4870|171710|14906.10|970530| | 10139923|GEORGE BROWN|1250 EAST PENDER ST.|VANCOUVER|BC|V5L1W1|82898|57828|4878|178524|-14626.10|980601| | 10147615|BONNIE SMITH|44430 YALE ROAD WEST|VANCOUVER|BC|V2P6J1|121395|39623|0|51914|3769.20|950601| | 10149304|TONY TATTERSAL|3350 S.W. MARINE DRIVE|VANCOUVER|BC|V6N3Y9|50594|76800|0|208391|19250.30|930601| | 10154688|MEL RAHAL|1110 - 625 HOWE STREET|VANCOUVER|BC|V6C2T6|82898|44880|0|125038|11547.90|961126| | 10173320|SAM SMITH|2381 ROGERS AVE|VANCOUVER|BC|V3K5Y2|50498|60600|0|169086|15620.00|970530| | 10204700|JOHN JENSEN|12611-100 ST|VANCOUVER|AL|T8V4H2|122396|15720|0|43744|-4044.00|961126| | 10223240|BOB ANDERSON|7215 ARGYLL ROAD|VANCOUVER|AL|T6C4J2|52899|3597|0|9990|-909.90|980601| | 10308685|TOM PERRY|3932 - 3A ST. N.W.|VANCOUVER|AL|T2E6R4|70192|68155|0|125773|10651.00|920601| | 10318833|HARRY SMIRNOFF|BOX 308, STN J|VANCOUVER|AL|T2A4X6|11399|54541|42046|93502|7614.60|970530| |
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#6. cp $UV/mvstest1/cnvdata1/cpys/citytax1.cpy cpys/citytax1
========================================================
- copy demo copybook (drop .cpy suffix if present)
- here is copybook - AFTER conversion to cobmap
* citytax1 copybook for UV demos www.uvsoftware.ca/datacnv1.htm
01 citytax1.
10 folio pic x(00010).
10 name pic x(00025).
10 address pic x(00025).
10 city pic x(00020).
10 province pic x(00002).
10 zip pic x(00006).
10 post-date pic s9(00007) comp-4.
10 land-value pic s9(00009) comp-3.
10 improv-value pic s9(00009) comp-3.
10 face-value pic s9(00009) comp-3.
10 maint-tax pic s9(00007)V99.
10 purchase-date pic x(00006).
10 filler pic x(00006).
In Oct 2014, we developed genpipe3/genfixd3 (alternates to genpipe1/genfixd2), to remove the restriction of 100 bytes max field length (& also faster). The new jobs (genpipe3/genfixd3) are documented begining on pages '4M1' & '4N1'.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. Login userxx --> /home/userxx #2. cdc --> home/userxx/cnvdata1
#3a. gencnvD2 citytax1.cpy <-- easiest to use script
=====================
- generate all data conversion jobs, all named 'citytax1'
- runs cobmap1,uvdata51,uvdata31,genpipe3,genfixd3,gentest1,sqlcreate1
- updated Oct2014 to run genpipe3/genfixd3 (vs genpipe1/genfixd2)
#3b. uvcopy genfixd2,fili1=maps/citytax1,filo1=pfq1/citytax1
=======================================================
- generate pfq1/citytax1 to undelimit d2asc/citytax1
- using older 'fxd' instructions generated by genfixd2
vs newer 'ftd' instructions generated by genfixd3 (see '4N1')
# citytax1 - uvcopy job to convert pipe delimited to Fixed-field format
opr='citytax1 - generated by genfixd2 from copybook: citytax1 '
opr='uop=q1e0s1 - option defaults'
opr=' e0 - output ASCII (do not translate char/zoned to EBCDIC)'
opr=' e1 - output EBCDIC (translate char/zoned fields to EBCDIC)'
opr=' s1 - switch binary field ends if running on Intel (little-end)'
opr=' s0 - use s0 if running on BIG-end machine (such as AIX)'
uop=q1e0s1 # option defaults
was=a33000b33000c999000d33000e33000
fili1=?d4pipe/citytax1,rcs=00183,typ=LST
filo1=?d5fix/citytax1,rcs=00128,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
fix c0(100),a0(00183),0015,'|'
# --- <-- insert R/T tests if redef records
clr d0(00128),x'20' init to x'20' or x'40' if optn e1 EBCDIC
fxd d0(10c),c0,'folio'
fxd d10(25c),c100,'name'
fxd d35(25c),c200,'address'
fxd d60(20c),c300,'city'
fxd d80(2c),c400,'province'
fxd d82(6c),c500,'zip'
fxd d88(4b),c600,'post-date'
fxd d92(5p),c700,'land-value'
fxd d97(5p),c800,'improv-value'
fxd d102(5p),c900,'face-value'
fxd d107(9z),c1000,'maint-tax'
fxd d116(6c),c1100,'purchase-date'
fxd d122(6c),c1200,'filler001'
put1 put filo1,d0
skp loop
eof cls all
eoj
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# citytax1 - uvcopy job to convert EBCDIC/ASCII data to delimited text file
opr='citytax1 - uvcopy code generated from copybook: citytax1.cpy '
rop=j200000r1 #Run OPtions: increase instrn storage & prompt display output
was=a64000b64000c999000d64000e64000u64000v64000 #increase u&v if fields > 163
fili1=?d2asc/citytax1,rcs=00128,typ=RSF #<-- will change d2asc/ to d6fix/
filo1=?d4pipe/citytax1,rcs=00384,typ=LSTtd3 #<-- will change d4pipe/ to d7pipe
@run
opn all
loop get fili1,a0
skp> eof
# area a input, see 'get' instrn
# area b translated to ASCII, in case mainframe EBCDIC file (with packed?)
# area c data fields fixed 100 bytes apart in prep for var | delimit instrn
# area d output, see 'put' instrn at end
mvc b0(00128),a0 move input area a to area b
### tra b0(00128) uncomment for EBCDIC input (or option a1)
# --- <-- & insert instrns from uvdata51 if packed fields
# --- <-- insert R/T tests if redef records
mvc c0(10),b0(10) #1 folio
mvc c100(25),b10(25) #2 name
mvc c200(25),b35(25) #3 address
mvc c300(20),b60(20) #4 city
mvc c400(2),b80(2) #5 province
mvc c500(6),b82(6) #6 zip
edta1 c600(9),a88(4bs),'-zzzzzzzzz9' #7 post-date
edta1 c700(11),a92(5p),'-zzzzzzzzz9' #8 land-value
edta1 c800(11),a97(5p),'-zzzzzzzzz9' #9 improv-value
edta1 c900(11),a102(5p),'-zzzzzzzzz9' #10 face-value
edta1 c1000(11),b107(9),'-zzzzzzz.99' #11 maint-tax
mvc c1100(6),b116(6) #12 purchase-date
mvc c1200(6),b122(6) #13 filler001
rep c0(01300),'|','!' #comment speedup if no '|'s in data
varb1 d0(01300),c0(100),0013,'|' #insert '|'s between fields & squeeze
trt d0(01300),$trtchr #ensure all ascii chars x'20'-x'7E'
clr c0(01300),' '
put1 put filo1,d0
skp loop
#
eof cls all
eoj
@pf2=genpipe1.sub
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
For my demo files datafilenames & copybooknames are the same, but you will want to modify yuor generated jobs. Always copy auto-generated job to 2nd subdir to avoid losing your edits in case jobs regenerated
#4a. cp pfq1/citytax1 pfq2 <-- copy before edit
=====================
#4b. vi pfq2/citytyax1 <-- insert your datafilenames vs copybooknames
================= on fili1=... & filo1=...
#5a. cp pfp1/citytax1 pfp2 <-- copy before edit
=====================
#5b. vi pfp2/citytyax1 <-- insert your datafilenames vs copybooknames
================= on fili1=... & filo1=...
| Note |
|
fili1=?d2asc/citytax1,rcs=00128,typ=RSF
filo1=?d4pipe/citytax1,rcs=00384,typ=LSTtd3
fili1=?d6fix/citytax1,rcs=00128,typ=RSF
filo1=?d7pipe/citytax1,rcs=00384,typ=LSTtd3
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. Login userxx --> /home/userxx #2. cdc --> home/userxx/cnvdata1
#3. Execute JCL/script calling db2 to EXPORT delimited file from DB2 table.
| Note |
10130140|JOHN HENRY|1815 BOWEN ROAD|VANCOUVER|BC|V9S1H1|82898|57828|4870|171710|1490.61|970530| | 10139923|GEORGE BROWN|1250 EAST PENDER ST.|VANCOUVER|BC|V5L1W1|82898|57828|4878|178524|-1462.61|980601| | 10147615|BONNIE SMITH|44430 YALE ROAD WEST|VANCOUVER|BC|V2P6J1|121395|39623|0|51914|376.92|950601| | 10149304|TONY TATTERSAL|3350 S.W. MARINE DRIVE|VANCOUVER|BC|V6N3Y9|50594|76800|0|208391|1925.03|930601| | 10154688|MEL RAHAL|1110 - 625 HOWE STREET|VANCOUVER|BC|V6C2T6|82898|44880|0|125038|1154.79|961126| | 10173320|SAM SMITH|2381 ROGERS AVE|VANCOUVER|BC|V3K5Y2|50498|60600|0|169086|1562.00|970530| | 10204700|JOHN JENSEN|12611-100 ST|VANCOUVER|AL|T8V4H2|122396|15720|0|43744|-404.40|961126| | 10223240|BOB ANDERSON|7215 ARGYLL ROAD|VANCOUVER|AL|T6C4J2|52899|3597|0|9990|-90.99|980601| | 10308685|TOM PERRY|3932 - 3A ST. N.W.|VANCOUVER|AL|T2E6R4|70192|68155|0|125773|1065.10|920601| | 10318833|HARRY SMIRNOFF|BOX 308, STN J|VANCOUVER|AL|T2A4X6|11399|54541|42046|93502|761.46|970530| |
#4a. uvcopy pfq2/citytax1,uop=e0s1
=============================
- convert d4pipe/citytax1 to d5fix/citytax1
- option uop=e0 to output ASCII & NOT translate to EBCDIC
- option uop=s1 converts binary fields to BIG-end format (mainframe format)
#5. uvhd d5fix/citytax1 r128
========================
rec#=1 rsize=128 fptr=0 fsize=1280 rcount=10
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
3333333322444424445522222222222222233332445442544422222222225444
1013014000AF8E085E29000000000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H1..C...W.........qq.001490610970530
455452222222222244535343004D005880008000770333333333333333222222
F5652000000000002369318101320072C0047C0111C001490610970530000000
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
6. EXECUTE your COBOL program here to process d5fix/... to d6fix/...
=================================================================
| Note |
|
#7a. cp d5fix/citytax1 d6fix/citytax1
================================
- copy demo file (vs executing your COBOL program)
#7a. uvcopy pfp2/citytax1
====================
- convert fixed d6fix/citytax1 to delim d7pipe/citytax1
#8. head d7pipe/citytax1
=====================
10130140|JOHN HENRY|1815 BOWEN ROAD|VANCOUVER|BC|V9S1H1|82898|57828|4870|171710|1490.61|970530| | 10139923|GEORGE BROWN|1250 EAST PENDER ST.|VANCOUVER|BC|V5L1W1|82898|57828|4878|178524|-1462.61|980601| | 10147615|BONNIE SMITH|44430 YALE ROAD WEST|VANCOUVER|BC|V2P6J1|121395|39623|0|51914|376.92|950601| | 10149304|TONY TATTERSAL|3350 S.W. MARINE DRIVE|VANCOUVER|BC|V6N3Y9|50594|76800|0|208391|1925.03|930601| | 10154688|MEL RAHAL|1110 - 625 HOWE STREET|VANCOUVER|BC|V6C2T6|82898|44880|0|125038|1154.79|961126| | 10173320|SAM SMITH|2381 ROGERS AVE|VANCOUVER|BC|V3K5Y2|50498|60600|0|169086|1562.00|970530| | 10204700|JOHN JENSEN|12611-100 ST|VANCOUVER|AL|T8V4H2|122396|15720|0|43744|-404.40|961126| | 10223240|BOB ANDERSON|7215 ARGYLL ROAD|VANCOUVER|AL|T6C4J2|52899|3597|0|9990|-90.99|980601| | 10308685|TOM PERRY|3932 - 3A ST. N.W.|VANCOUVER|AL|T2E6R4|70192|68155|0|125773|1065.10|920601| | 10318833|HARRY SMIRNOFF|BOX 308, STN J|VANCOUVER|AL|T2A4X6|11399|54541|42046|93502|761.46|970530| |
#9. diff d7pipe/citytax1 d4pipe/citytax1
====================================
- could compare cobol output in delim format to original input delim
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. uvcopy pfq2/citytax1,fili1=d4pipe/citytax1,filo1=d5fix/citytax1s0,uop=s0
========================================================================
- 'uop=s0' specifies little-end option on the command-line
- OR, could enter at the options prompt, as shown below --> s0 <--
uop=q1e0s1 - option defaults
e0 - output ASCII (do NOT translate char/zoned to EBCDIC)
e1 - output EBCDIC (translate char/zoned fields to EBCDIC)
s1 - switch binary field ends if running on Intel (little-end)
s0 - use s0 if running on BIG-end machine (such as AIX)
User OPtion (uop) defaults = q1e0s1s0
-->null to accept or enter/override --> s0 <-- ENTER option for little-end
| Note |
|
#2. uvcmpFA1 d5fix/citytax1 d5fix/citytax1s0 r128
=============================================
- compare prior BIG-end output to this little-end output
uvcmp1 - compare 2 files, print mismatched records, '*' flag diffs
2012/02/19_16:15:20 uop=q1p30r256s6t500000u3x2y0q1r128
recsize reccount file-size typ Report=rptcmp/citytax1
1: 128 10 1,280 RSF File1=d5fix/citytax1
2: 128 10 1,280 RSF File2=d5fix/citytax1s0
1 2 3 4 5 6
f#record#byte# 0123456789012345678901234567890123456789012345678901234567890123
===============================================================================
1 1 0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
3333333322444424445522222222222222233332445442544422222222225444
1013014000AF8E085E29000000000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H1..C...W.........qq.000149061970530
455452222222222244535343004D005880008000770333333333333333222222
F5652000000000002369318101320072C0047C0111C000149061970530000000
****
2 1 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
3333333322444424445522222222222222233332445442544422222222225444
1013014000AF8E085E29000000000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H1.C....W.........qq.000149061970530
455452222222222244535343D400005880008000770333333333333333222222
F5652000000000002369318123100072C0047C0111C000149061970530000000
****
==================== EOF or StopPrint/StopRead count reached ============== F1Count=10, F2Count=10, StopPrint=6, StopRead=500000 F1Reads=10, MisMatches=10, MisMatsPrinted=6, Recsize=128
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
To help you understaand BIG/little end binary, we have collected relevant items from the previous paages:
| 4J3. | copybook/map, only the binary field 'post-date' at byte 88(4b) |
10 post-date pic s9(00007) comp-4. 0088 0091 004bns 7
| 4J7. | 1st line of Pipe|Delimited file, note binary field value is '+82898' |
10130140|JOHN HENRY|1815 BOWEN ROAD|VANCOUVER|BC|V9S1H1|+82898|57828|4870|171710|1490.61|970530| |
| 4J9. | file-comparison report, showing bytes 64-127 with difference in bytes 88-91 |
64 OUVER BCV9S1H1..C...W.........qq.000149061970530
455452222222222244535343004D005880008000770333333333333333222222
F5652000000000002369318101320072C0047C0111C000149061970530000000
****
64 OUVER BCV9S1H1.C....W.........qq.000149061970530
455452222222222244535343D400005880008000770333333333333333222222
F5652000000000002369318123100072C0047C0111C000149061970530000000
****
We can express the difference field in horizontal hex as follows:
| x'000143D2' |
|
| x'D2430100' |
|
Can convert BIG-end binary to decimal as follows (working from right to left):
2 * 00001 = 00002
D * 00016 = 00208
3 * 00256 = 00768
4 * 04096 = 16384
1 * 65536 = 65536
=================
88898
genpipe1 generates edit instructions with or without option 's' depending on the machine architecture - BIG-end or little-end (AIX,SUN,etc or INTEL). On page '4B4' the binary field post-date was generated as follows on my INTEL.
edta1 c600(9),a88(4bs),'+zzzzzzzzz9' #7 post-date
See more discussion on page '4D3'.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
genpipe1 will expand 'item occurs' (occurs & pic on same statement), but will not expand 'group occurs'. See following '4L1' cpyoccurs1, which would be run before cobmap1 & genpipe1.
Here we will demo genpipe1 with occurs, using the armaster copybook, which has occurs 12 for this year & last year sales. This assumes you are operating in the cnvdata1 directory with subdirs as shown on page '4A2'.
#1. genpipeA armaster.cpy <-- easiest way to generate all jobs required for pipe delim
===================== (cobmap1, uvdata51, genpipe1)
- easier than run separately as follows:
#1a. uvcopy cobmap1,fili1=cpys/armaster.cpy,filo1=maps/armaster
==========================================================
- generate cobmap layout (from copybook)
#1b. uvcopy uvdata51,fili1=maps/armaster,filo1=pfx1/armaster
======================================================
- generate uvcopy job to convert EBCDIC data to ASCII (from cobmap layout)
- optional, not required if data file already ASCII
#1c. uvcopy genpipe1,fili1=maps/armaster,filo1=pfp1/armaster
======================================================
- generate uvcopy job to convert ASCII data to pipe delimited
#2. uvcopy pfx1/armaster <-- execute uvcopy job to convert EBCDIC to ASCII
==================== - optional, not required if data file already ASCII
#3. uvcopy pfp1/armaster <-- execute uvcopy job to convert ASCII to pipe delim
====================
cobmap1 COBOL record layout 2013/05/19_10:32:12 pg# 0001
cpys/armaster.cpy RCSZ=00256 bgn end lth typ
* armaster - cobol copy book for armaster file (see uvtrain.
10 cm-num pic 9(6). 0000 0005 006 n 6
10 cm-delete pic x(4). 0006 0009 004
10 cm-name pic x(25). 0010 0034 025
10 cm-adrs pic x(25). 0035 0059 025
10 cm-city pic x(16). 0060 0075 016
10 filler001 pic x. 0076 0076 001
10 cm-prov pic x(2). 0077 0078 002
10 filler002 pic x. 0079 0079 001
10 cm-postal pic x(10). 0080 0089 010
10 cm-phone pic x(12). 0090 0101 012
10 cm-contact pic x(18). 0102 0119 018
10 cm-thisyr-sales pic s9(7)v99 comp-3 occurs 12. 0120 0124 005pns 9
10 cm-lastyr-sales pic s9(7)v99 comp-3 occurs 12. 0180 0184 005pns 9
10 filler003 pic x(16). 0240 0255 016
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# armaster - uvcopy job to convert EBCDIC/ASCII data to delimited text file
opr='armaster - uvcopy code generated from copybook: armaster.cpy '
rop=j200000r1 #Run OPtions: increase instrn storage & prompt display output
was=a33000b33000c999000d33000e33000u16384v16384 #increase u&v if fields > 163
fili1=?d2asc/armaster,rcs=00256,typ=RSF
filo1=?d4pipe/armaster,rcs=00768,typ=LSTtd3
@run
opn all
loop get fili1,a0
skp> eof
# area a input, see 'get' instrn
# area b translated to ASCII, in case mainframe EBCDIC file (with packed?)
# area c data fields fixed 100 bytes apart in prep for var | delimit instrn
# area d output, see 'put' instrn at end
mvc b0(00256),a0 move input area a to area b
### tra b0(00256) uncomment for EBCDIC input (or option a1)
# --- <-- & insert instrns from uvdata51 if packed fields
# --- <-- insert R/T tests if redef records
mvc c0(6),b0(6) #1 cm-num
mvc c100(4),b6(4) #2 cm-delete
mvc c200(25),b10(25) #3 cm-name
mvc c300(25),b35(25) #4 cm-adrs
mvc c400(16),b60(16) #5 cm-city
mvc c500(1),b76(1) #6 filler001
mvc c600(2),b77(2) #7 cm-prov
mvc c700(1),b79(1) #8 filler002
mvc c800(10),b80(10) #9 cm-postal
mvc c900(12),b90(12) #10 cm-phone
mvc c1000(18),b102(18) #11 cm-contact
edta1 c1100(11),a120(5p),'+zzzzzzz.99' #12 cm-thisyr-sales occurs 12
edta1 c1200(11),a125(5p),'+zzzzzzz.99' #13 cm-thisyr-sales occurs 12
# --- 9 fields omitted ---
edta1 c2200(11),a175(5p),'+zzzzzzz.99' #23 cm-thisyr-sales occurs 12
#
edta1 c2300(11),a180(5p),'+zzzzzzz.99' #24 cm-lastyr-sales occurs 12
edta1 c2400(11),a185(5p),'+zzzzzzz.99' #25 cm-lastyr-sales occurs 12
# --- 9 fields omitted ---
edta1 c3400(11),a235(5p),'+zzzzzzz.99' #35 cm-lastyr-sales occurs 12
#
mvc c3500(16),b240(16) #36 filler003
rep c0(03600),'|','!' #comment speedup if no '|'s in data
varb1 d0(03600),c0(100),0036,'|' #insert '|'s between fields & squeeze
trt d0(03600),$trtchr #ensure all ascii chars x'20'-x'7E'
clr c0(03600),' '
put1 put filo1,d0
skp loop
#
eof cls all
eoj
@pf2=genpipe1.sub
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#2. uvcopy pfx1/armaster <-- execute uvcopy job to convert EBCDIC to ASCII
==================== - optional, not required if data file already ASCII
#3. uvcopy pfp1/armaster <-- execute uvcopy job to convert ASCII to pipe delim
====================
uvhd /home/mvstest1/ALLcnvdata1/d2asc/armaster ==============================================
rec#=1 rsize=256 fptr=0 fsize=8192 rcount=32
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 EVERGREEN MOTORS LTD. 1815 BOWEN ROAD NANA
3333332222454545444244545524542222233332445442544422222222224444
130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
64 IMO BC V9S1H1 250-754-5531 LARRY WRENCH ..4V|...
4442222222222442535343222233323332333324455525544442222201357000
9DF00000000002306931810000250D754D55310C12290725E38000000246C000
128 .........W0....`........)X}..f3.....\.................4V}...f...
0000000005300016000000002570063100095000000000000000013570016000
0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0056C00
192 .E|...V}.......................f.....<........f.C 19950531
0470005700000000880000000018000680001300000000694233333333222222
35C0046D0000C0023C0000C0083C0056D0012C0000C0016D3019950531000000
130140| |EVERGREEN MOTORS LTD.|1815 BOWEN ROAD|NANAIMO| |BC| |V9S1H1 |250-754-5531|LARRY WRENCH |+12345.67|+.00|+.00|-2573.00|+5146.00|+.00 |-295.87|+2663.31|+4049.45|+.00|+.00|+.00 |-12345.67|+156.60|+34.57|-45.67|+.00|+28.38 |+.00|+81.38|-56.68|+11.23|+.00|-16.69 |C 19950531|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
genpipe1 (see prior pages '4K1'-'4K3' expands 'item occurs' but not expand 'group occurs'. For group occurs, run 'cpyoccurs1' on the copybook before cobmap1 & genpipe1.
Here we will demo cpyoccurs1, using the custran1 copybook, which has group occurs 5 for 3 items. This assumes you are operating in the cnvdata1 directory with subdirs as shown on page '4A2'.
#1. mkdir cpys2 <-- make subdir for output of 'cpyoccurs1'
===========
#2. uvcopy cpyoccurs1,fili1=cpys/custran1,filo1=cpys2/custran1
=========================================================
- copy the copybook, expanding group occurs
#3. uvcopy cobmap1,fili1=cpys2/custran1,filo1=maps/custran1
=======================================================
- generate cobmap layout (from copybook with expanded group occurs)
#4. uvcopy uvdata51,fili1=maps/custran1,filo1=pfx1/custran1
=======================================================
- generate uvcopy job to convert EBCDIC data to ASCII (from cobmap layout)
- optional, not required if data file already ASCII
#5. uvcopy genpipe1,fili1=maps/custran1,filo1=pfp1/custran1
=======================================================
- generate uvcopy job to convert ASCII data to pipe delimited
#2. uvcopy pfx1/custran1 <-- execute uvcopy job to convert EBCDIC to ASCII
==================== - optional, not required if data file already ASCII
#3. uvcopy pfp1/custran1 <-- execute uvcopy job to convert ASCII to pipe delim
====================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
cobmap1 COBOL record layout 2013/05/19_15:56:08 pg# 0001
cpys/custran1 RCSZ=00256 bgn end lth typ
10 cmt-num pic 9(6). 0000 0005 006 n 6
10 cmt-delete pic x(4). 0006 0009 004
10 cmt-name pic x(25). 0010 0034 025
10 cmt-adrs pic x(25). 0035 0059 025
10 cmt-city pic x(16). 0060 0075 016
10 filler001 pic x. 0076 0076 001
10 cmt-prov pic x(2). 0077 0078 002
10 filler002 pic x. 0079 0079 001
10 cmt-postal pic x(10). 0080 0089 010
10 cmt-phone pic x(12). 0090 0101 012
10 cmt-contact pic x(18). 0102 0119 018
*BGNOCCURSM:c-p:00005*00010=00050:00120-00169:1:
10 cmt-trans occurs 5.
15 cmt-tran-type pic x. 0120 0120 001
15 cmt-tran-ref pic 9(4). 0121 0124 004 n 4
15 cmt-tran-amt pic s9(7)v99 comp-3. 0125 0129 005pns 9
*ENDOCCURSM:c-p:00005*00010=00050:00120-00169:1:
10 filler003 pic x(86). 0170 0255 086
cobmap1 COBOL record layout 2013/05/19_14:04:02 pg# 0001
cpys2/custran1 RCSZ=00256 bgn end lth typ
10 cmt-num pic 9(6). 0000 0005 006 n 6
10 cmt-delete pic x(4). 0006 0009 004
10 cmt-name pic x(25). 0010 0034 025
10 cmt-adrs pic x(25). 0035 0059 025
10 cmt-city pic x(16). 0060 0075 016
10 filler001 pic x. 0076 0076 001
10 cmt-prov pic x(2). 0077 0078 002
10 filler002 pic x. 0079 0079 001
10 cmt-postal pic x(10). 0080 0089 010
10 cmt-phone pic x(12). 0090 0101 012
10 cmt-contact pic x(18). 0102 0119 018
* 10 cmt-trans occurs 5.
15 cmt-tran-type001 pic x. 0120 0120 001
15 cmt-tran-ref001 pic 9(4). 0121 0124 004 n 4
15 cmt-tran-amt001 pic s9(7)v99 comp-3. 0125 0129 005pns 9
15 cmt-tran-type002 pic x. 0130 0130 001
15 cmt-tran-ref002 pic 9(4). 0131 0134 004 n 4
15 cmt-tran-amt002 pic s9(7)v99 comp-3. 0135 0139 005pns 9
15 cmt-tran-type003 pic x. 0140 0140 001
15 cmt-tran-ref003 pic 9(4). 0141 0144 004 n 4
15 cmt-tran-amt003 pic s9(7)v99 comp-3. 0145 0149 005pns 9
15 cmt-tran-type004 pic x. 0150 0150 001
15 cmt-tran-ref004 pic 9(4). 0151 0154 004 n 4
15 cmt-tran-amt004 pic s9(7)v99 comp-3. 0155 0159 005pns 9
15 cmt-tran-type005 pic x. 0160 0160 001
15 cmt-tran-ref005 pic 9(4). 0161 0164 004 n 4
15 cmt-tran-amt005 pic s9(7)v99 comp-3. 0165 0169 005pns 9
10 filler003 pic x(86). 0170 0255 086
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# custran1 - uvcopy job to convert EBCDIC/ASCII data to delimited text file
opr='custran1 - uvcopy code generated from copybook: custran1 '
rop=j200000r1 #Run OPtions: increase instrn storage & prompt display output
was=a33000b33000c999000d33000e33000u16384v16384 #increase u&v if fields > 163
fili1=?d2asc/custran1,rcs=00256,typ=RSF
filo1=?d4pipe/custran1,rcs=00768,typ=LSTtd3
@run
opn all
loop get fili1,a0
skp> eof
# area a input, see 'get' instrn
# area b translated to ASCII, in case mainframe EBCDIC file (with packed?)
# area c data fields fixed 100 bytes apart in prep for var | delimit instrn
# area d output, see 'put' instrn at end
mvc b0(00256),a0 move input area a to area b
### tra b0(00256) uncomment for EBCDIC input (or option a1)
mvc c0(6),b0(6) #1 cmt-num
mvc c100(4),b6(4) #2 cmt-delete
mvc c200(25),b10(25) #3 cmt-name
mvc c300(25),b35(25) #4 cmt-adrs
mvc c400(16),b60(16) #5 cmt-city
mvc c500(1),b76(1) #6 filler001
mvc c600(2),b77(2) #7 cmt-prov
mvc c700(1),b79(1) #8 filler002
mvc c800(10),b80(10) #9 cmt-postal
mvc c900(12),b90(12) #10 cmt-phone
mvc c1000(18),b102(18) #11 cmt-contact
mvc c1100(1),b120(1) #12 cmt-tran-type001
mvc c1200(4),b121(4) #13 cmt-tran-ref001
edta1 c1300(11),a125(5p),'+zzzzzzz.99' #14 cmt-tran-amt001
mvc c1400(1),b130(1) #15 cmt-tran-type002
mvc c1500(4),b131(4) #16 cmt-tran-ref002
edta1 c1600(11),a135(5p),'+zzzzzzz.99' #17 cmt-tran-amt002
mvc c1700(1),b140(1) #18 cmt-tran-type003
mvc c1800(4),b141(4) #19 cmt-tran-ref003
edta1 c1900(11),a145(5p),'+zzzzzzz.99' #20 cmt-tran-amt003
mvc c2000(1),b150(1) #21 cmt-tran-type004
mvc c2100(4),b151(4) #22 cmt-tran-ref004
edta1 c2200(11),a155(5p),'+zzzzzzz.99' #23 cmt-tran-amt004
mvc c2300(1),b160(1) #24 cmt-tran-type005
mvc c2400(4),b161(4) #25 cmt-tran-ref005
edta1 c2500(11),a165(5p),'+zzzzzzz.99' #26 cmt-tran-amt005
mvc c2600(86),b170(86) #27 filler003
rep c0(02700),'|','!' #comment speedup if no '|'s in data
varb1 d0(02700),c0(100),0027,'|' #insert '|'s between fields & squeeze
trt d0(02700),$trtchr #ensure all ascii chars x'20'-x'7E'
clr c0(02700),' '
put1 put filo1,d0
skp loop
eof cls all
eoj
@pf2=genpipe1.sub
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#2. uvcopy pfx1/custran1 <-- execute uvcopy job to convert EBCDIC to ASCII
==================== - optional, not required if data file already ASCII
#3. uvcopy pfp1/custran1 <-- execute uvcopy job to convert ASCII to pipe delim
====================
uvhd /home/mvstest1/ALLcnvdata1/d2asc/custran1 ==============================================
rec#=1 rsize=256 fptr=0 fsize=8192 rcount=32
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 EVERGREEN MOTORS LTD. 1815 BOWEN ROAD NANA
3333332222454545444244545524542222233332445442544422222222224444
130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
64 IMO BC V9S1H1 250-754-5531 LARRY WRENCH A0101..4
4442222222222442535343222233323332333324455525544442222243333013
9DF00000000002306931810000250D754D55310C12290725E380000010101024
128 V|B0102...`.C0103....\ 0000.....E0105..4V}
5743333001604333300095233330000043333013572222222222222222222222
6C201020540C301030444C000000000C501050246D0000000000000000000000
192
2222222222222222222222222222222222222222222222222222222222222222
0000000000000000000000000000000000000000000000000000000000000000
130140| |EVERGREEN MOTORS LTD.|1815 BOWEN ROAD|NANAIMO| |BC| |V9S1H1 |250-754-5531| LARRY WRENCH |A|0101|+12345.67|B|0102|+5146.00|C|0103|+4049.45| |0000|+.00|E|0105|-12345.67| |
132588| |GEECOE GENERATOR SERVICES|UNIT 170 - 2851 SIMPSON|RICHMOND| |BC| |V6X2R2 |604-278-4488| HARRY LIGHT |A|0201|+46.80| |0000|+.00|C|0203|+37.08|D|0204|+53.60| |0000|+.00| |
139923| |JOHNSTONE BOILER & TANKS|1250 EAST PENDER STREET|VANCOUVER| |BC| |V5L1W1 |604-320-1845|GEORGE BROWN |A|0301|+15.28| |0000|+.00| |0000|+.00| |0000|+.00|E|0305|+144.96| |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'genpipe3' was developed in Oct 2014 to increase the max field length from 100 allowed by 'genpipe1' to 32,767 and genpipe3 is also much faster.
This is similar to the genpipe1 demo on pages '4B3' - '4B5'. The easiest way to run genpipe3 is to use script 'gencnvA' or 'gencnvD2' which include the pre-requisite cobmap as well as genfixd3 (undelimit). These scripts have been updated to run the new genpipe3/genfixd3 jobs vs the old genpipe1/genfixd2 jobs (max field limit 100 bytes).
gencnvA citytax1.cpy <-- script runs cobmap1,uvdata51,genpipe3,genfixd3 ====================
gencnvD2 citytax1.cpy <-- script runs cobmap1,uvdata51,genpipe3,genfixd3 ===================== + sqlcreateA
| Note |
|
uvcopy cobmap1,fili1=cpys/citytax1.cpy,filo1=maps/citytax1 ==========================================================
uvcopy genpipe3,fili1=maps/citytax1,filo1=pfp1/citytax1 =======================================================
uop=a0e2s0 - default options
a0 - file already translated from EBCDIC to ASCII by uvdata51
a1 - translate data from EBCDIC to ASCII here in genpipe3 job
- must transfer any packed field instrns from the uvdata51
(subdir pfx1) to this genpipe3 job (subdir pfp1)
e1 - edit signed fields with leading "-" only if negative
e2 - edit signed fields with + if pos (as well as - if neg)
s0 - auto Binary Switch (Big-END AIX/SUN, little-end Intel)
- use option s1/s2 only if generating code for other machines
s1 - force no switch (default on BIG-end machines (AIX/SUN)
s2 - force switch ends (default on Little-end machines (Intel)
User OPtion (uop) defaults = q1a0e2s0
-->null to accept or enter/override -->
141026:172434:genpipe3: EOF fili01 rds=19 size=1257: maps/citytax1
141026:172434:genpipe3: EOF filo01 wrts=38 size=1923: pfp1/citytax1
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
cobmap1 COBOL record layout 2014/10/07_12:42:02 pg# 0001 cpys/citytax1.cpy citytax1 RCSZ=00128 bgn end lth typ
* citytax1 copybook for uv demos www.uvsoftware.ca/datacnv1.
01 citytax1.
10 folio pic x(00010). 0000 0009 010
10 name pic x(00025). 0010 0034 025
10 address pic x(00025). 0035 0059 025
10 city pic x(00020). 0060 0079 020
10 province pic x(00002). 0080 0081 002
10 zip pic x(00006). 0082 0087 006
10 post-date pic s9(00007) comp-4. 0088 0091 004bns 7
10 land-value pic s9(00009) comp-3. 0092 0096 005pns 9
10 improv-value pic s9(00009) comp-3. 0097 0101 005pns 9
10 face-value pic s9(00009) comp-3. 0102 0106 005pns 9
10 maint-tax pic s9(00007)v99. 0107 0115 009 ns 9
10 purchase-date pic 9(00006). 0116 0121 006 n 6
10 filler001 pic x(00006). 0122 0127 006
*RCSZ=00128
Character pic x =============== - any bytes < space x'20' are converted to spaces - trailing spaces are removed - any delimiter (default arg1=|) in existing data is replaced by arg2=! (default)
Numeric pic 9 (zoned, unsigned) =============================== - any non-numerics replaced by zeros '0's
Packed, Binary, Numeric signed ============================== - converted to binary (dropping any non-digits) - edited with leading +/- sign - decimal point inserted depending on pic (...9v9...)
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# citytax1 - uvcopy job to convert EBCDIC/ASCII data to delimited text file
opr='citytax1 - uvcopy code generated from copybook: citytax1.cpy '
rop=j200000 #Run OPtions: increase instrn storage
arg1=|,arg2=! # arg1=delimiter, arg2=delimiter replacement if in existing data
# # arg1=x.. for hex control chars, x09 for tab, etc
was=a64000b64000c64000d64000e64000u64000v64000
fili1=?d2asc/citytax1,rcs=00128,typ=RSF
filo1=?d4pipe/citytax1,rcs=64000,typ=LSTtd3
@run
opn all
loop get fili1,a0
skp> eof
# area a input, see 'get' instrn
# area b translated to ASCII, in case mainframe EBCDIC file (with packed?)
# area c built field by field by 'ftd' instruction (allows > 100 bytes/field)
#Oct2014 - vs prior fixed 100 bytes apart for var | delimit instrn
mvc b0(00128),a0 move input area a to area b
### tra b0(00128) uncomment for EBCDIC input (or option a1)
# --- <-- & insert instrns from uvdata51 if packed fields
# --- <-- insert R/T tests if redef records
ftd b0(10c) #1 folio
ftd b10(25c) #2 name
ftd b35(25c) #3 address
ftd b60(20c) #4 city
ftd b80(2c) #5 province
ftd b82(6c) #6 zip
ftd a88(4bs),'+zzzzzz9' #7 post-date
ftd a92(5p),'+zzzzzzzz9' #8 land-value
ftd a97(5p),'+zzzzzzzz9' #9 improv-value
ftd a102(5p),'+zzzzzzzz9' #10 face-value
ftd b107(9z),'+zzzzzzz.99' #11 maint-tax
ftd b116(6n) #12 purchase-date
ftd b122(6c) #13 filler001
put1 put filo1,c0
clr c0($rc64000),' '
skp loop
eof cls all
eoj
Line 4 of the generated job defines the delimiter (default arg1=|). You can change as desired & may specify in hexadecimal. For example to specify the delimiter as the 'tab' character, use 'arg1=x09').
For each field, trailing blanks are eliminated, the delimiter is inserted after the last non-blank. Control characters (bytes < x'20') are also eliminated. You may inhibit drop ctlchars by Run option 'w1' OR instruction option 'c1'
--> rop=w1 #<-- run option 'w1' inhibit drop ctlchars all fields --> ftdc1 b10(25c) #<-- instrn option c1 inhibit drop ctlchars for 1 field
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvcopy pfp1/citytax1 ====================
citytax1 - uvcopy code generated from copybook: citytax1.cpy d2asc/citytax1 = default fili01 - null accept or reenter ? d4pipe/citytax1 = default filo01 - null accept or reenter ? 141026:175313:citytax1: EOF fili01 rds=10 size=1280: d2asc/citytax1 141026:175313:citytax1: EOF filo01 wrts=10 size=1051: d4pipe/citytax1
uvhd d2asc/citytax1 r128
========================
rec#=1 rsize=128 fptr=0 fsize=1280 rcount=10
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
3333333322444424445522222222222222233332445442544422222222225444
1013014000AF8E085E29000F00000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H1..C...W.........qq.000149061970530
455452222222222244535343004D005880008000770333333333333333222222
F5652000000000002369318101320072C0047C0111C000149061970530000000
rec#=2 rsize=128 fptr=128 fsize=1280 rcount=10
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
128 10139923 GEORGE BROWN 1250 EAST PENDER ST. VANC
3333333322444544245454222222222222233332445525444452552222225444
101399230075F275022F7E0000000000000125005134005E452034E0000061E3
64 OUVER BCV5L1W1..C...W.........xRL00014626q980601
455452222222222244534353004D005880008800754333333337333333222222
F5652000000000002365C17101320072C0047C0182C000146261980601000000
cat d4pipe/citytax1 ===================
10130140|JOHN HENRY|1815 BOWEN ROAD|VANCOUVER|BC|V9S1H1|+82898|+57828|+4870|+171710|+1490.61|970530| | 10139923|GEORGE BROWN|1250 EAST PENDER ST.|VANCOUVER|BC|V5L1W1|+82898|+57828|+4878|+178524|-1462.61|980601| | 10147615|BONNIE SMITH|44430 YALE ROAD WEST|VANCOUVER|BC|V2P6J1|+121395|+39623|+0|+51914|+376.92|950601| | 10149304|TONY TATTERSAL|3350 S.W. MARINE DRIVE|VANCOUVER|BC|V6N3Y9|+50594|+76800|+0|+208391|+1925.03|930601| | 10154688|MEL RAHAL|1110 - 625 HOWE STREET|VANCOUVER|BC|V6C2T6|+82898|+44880|+0|+125038|+1154.79|961126| | 10173320|SAM SMITH|2381 ROGERS AVE|VANCOUVER|BC|V3K5Y2|+50498|+60600|+0|+169086|+1562.00|970530| | 10204700|JOHN JENSEN|12611-100 ST|VANCOUVER|AL|T8V4H2|+122396|+15720|+0|+43744|-404.40|961126| | 10223240|BOB ANDERSON|7215 ARGYLL ROAD|VANCOUVER|AL|T6C4J2|+52899|+3597|+0|+9990|-90.99|980601| | 10308685|TOM PERRY|3932 - 3A ST. N.W.|VANCOUVER|AL|T2E6R4|+70192|+68155|+0|+125773|+1065.10|920601| | 10318833|HARRY SMIRNOFF|BOX 308, STN J|VANCOUVER|AL|T2A4X6|+11399|+54541|+42046|+93502|+761.46|970530| |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'genfixd3' was developed in Oct 2014 to increase the max field length from 100 allowed by 'genfixd2' to 32,767 and genfixd3 is also faster.
This is similar to the genfixd2 demo on pages '4J1' - '4J10'. The easiest way to run genfixd3 is to use script 'gencnvA' or 'gencnvD2' which include the pre-requisite cobmap as well as genfixd3 (undelimit). These scripts have been updated to run the new genpipe3/genfixd3 jobs vs the old genpipe1/genfixd2 jobs (max field limit 100 bytes).
gencnvA citytax1.cpy <-- script runs cobmap1,uvdata51,genpipe3,genfixd3 ====================
gencnvD2 citytax1.cpy <-- script runs cobmap1,uvdata51,genpipe3,genfixd3 ===================== + sqlcreateA
| Note |
|
uvcopy cobmap1,fili1=cpys/citytax1.cpy,filo1=maps/citytax1 ==========================================================
uvcopy genfixd3,fili1=maps/citytax1,filo1=pfq1/citytax1 =======================================================
uop=q1e0s1t0 - default options (e0s1 generated job, t0 at gen time)
e0 - output ASCII (do not translate char/zoned to EBCDIC)
e1 - output EBCDIC (translate char/zoned fields to EBCDIC)
s1 - switch binary field ends if running on Intel (little-end)
- for mainframe files converted on Intel machines
s0 - use s0 if running on BIGend machine (such as AIX)
---------- above for GENERATED job, following for generation time
t0 - output file typ=RSF (no LineFeed)
t1 - output file typ=RST (LineFeed in last byte of recsize)
t2 - output file typ=RSR (LineFeed appended to recsize)
User OPtion (uop) defaults = q1e0s1t0
-->null to accept or enter/override --> <-- null accept filenames on cmd line
141027:111042:genfixd3: EOF fili01 rds=19 size=1257: maps/citytax1
141027:111042:genfixd3: EOF filo01 wrts=36 size=1398: pfq1/citytax1
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
cobmap1 COBOL record layout 2014/10/07_12:42:02 pg# 0001 cpys/citytax1.cpy citytax1 RCSZ=00128 bgn end lth typ
* citytax1 copybook for uv demos www.uvsoftware.ca/datacnv1.
01 citytax1.
10 folio pic x(00010). 0000 0009 010
10 name pic x(00025). 0010 0034 025
10 address pic x(00025). 0035 0059 025
10 city pic x(00020). 0060 0079 020
10 province pic x(00002). 0080 0081 002
10 zip pic x(00006). 0082 0087 006
10 post-date pic s9(00007) comp-4. 0088 0091 004bns 7
10 land-value pic s9(00009) comp-3. 0092 0096 005pns 9
10 improv-value pic s9(00009) comp-3. 0097 0101 005pns 9
10 face-value pic s9(00009) comp-3. 0102 0106 005pns 9
10 maint-tax pic s9(00007)v99. 0107 0115 009 ns 9
10 purchase-date pic 9(00006). 0116 0121 006 n 6
10 filler001 pic x(00006). 0122 0127 006
*RCSZ=00128
Character pic x =============== - output to field defined by copybook depending on displacement,& length - space filled if input data length < copybook field length - truncated if input data length > copybook field length
Numeric pic 9 (zoned, unsigned) =============================== - same as Character pic x Same length as original field in fixed-field record that created delimited - assuming that delimited input was created by genpipe3 - which converted any non-numerics to zeros '0's
Packed, Binary, Numeric signed ============================== - converted to binary (dropping any non-digits) - output to field defined by copybook depending on type, displacement,& length
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# citytax1 - uvcopy job to convert pipe delimited to Fixed-field format
opr='citytax1 - generated by genfixd3 from copybook: citytax1.cpy '
opr='uop=q1e0s1 - option defaults'
opr=' e0 - output ASCII (do not translate char/zoned to EBCDIC)'
opr=' e1 - output EBCDIC (translate char/zoned fields to EBCDIC)'
opr=' s1 - switch binary field ends if running on Intel (little-end)'
opr=' s0 - use s0 if running on BIG-end machine (such as AIX)'
uop=q1e0s1 # User option defaults
arg1=| # delimiter default, arg1=x.. for hex (arg1=x09 for tab)
was=a64000b64000c64000d64000e64000
fili1=?d4pipe/citytax1,rcs=00220,typ=LST
filo1=?d5fix/citytax1,rcs=00128,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
# --- <-- insert R/T tests if redef records
clr d0(00128),x'20' init to x'20' or x'40' if optn e1 EBCDIC
dtf d0(10c),c0,'folio'
dtf d10(25c),c0,'name'
dtf d35(25c),c0,'address'
dtf d60(20c),c0,'city'
dtf d80(2c),c0,'province'
dtf d82(6c),c0,'zip'
dtf d88(4b),c0,'post-date'
dtf d92(5p),c0,'land-value'
dtf d97(5p),c0,'improv-value'
dtf d102(5p),c0,'face-value'
dtf d107(9z),c0,'maint-tax'
dtf d116(6z),c0,'purchase-date'
dtf d122(6c),c0,'filler001'
put1 put filo1,d0
skp loop
eof cls all
eoj
Line 9 of the generated job defines the delimiter (default arg1=|). You may specify in hexadecimal. For example to specify the 'tab' character
arg1=x09 #<-- change delimiter to tab (hex '09')
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvcopy pfq1/citytax1 ====================
uop=q1e0s1 - option defaults
e0 - output ASCII (do not translate char/zoned to EBCDIC)
e1 - output EBCDIC (translate char/zoned fields to EBCDIC)
s1 - switch binary field ends if running on Intel (little-end)
s0 - use s0 if running on BIG-end machine (such as AIX)
User OPtion (uop) defaults = q1e0s1
-->null to accept or enter/override --> <-- null accept default filenames
d4pipe/citytax1 = default fili01 - null accept or reenter ?
d5fix/citytax1 = default filo01 - null accept or reenter ?
141027:111144:citytax1: EOF fili01 rds=10 size=1047: d4pipe/citytax1
141027:111144:citytax1: EOF filo01 wrts=10 size=1280: d5fix/citytax1
cat d4pipe/citytax1 ===================
10130140|JOHN HENRY|1815 BOWEN ROAD|VANCOUVER|BC|V9S1H1|+82898|+57828|+4870|+171710|+1490.61|970530| | 10139923|GEORGE BROWN|1250 EAST PENDER ST.|VANCOUVER|BC|V5L1W1|+82898|+57828|+4878|+178524|-1462.61|980601| | 10147615|BONNIE SMITH|44430 YALE ROAD WEST|VANCOUVER|BC|V2P6J1|+121395|+39623|+0|+51914|+376.92|950601| | 10149304|TONY TATTERSAL|3350 S.W. MARINE DRIVE|VANCOUVER|BC|V6N3Y9|+50594|+76800|+0|+208391|+1925.03|930601| | 10154688|MEL RAHAL|1110 - 625 HOWE STREET|VANCOUVER|BC|V6C2T6|+82898|+44880|+0|+125038|+1154.79|961126| | 10173320|SAM SMITH|2381 ROGERS AVE|VANCOUVER|BC|V3K5Y2|+50498|+60600|+0|+169086|+1562.00|970530| | 10204700|JOHN JENSEN|12611-100 ST|VANCOUVER|AL|T8V4H2|+122396|+15720|+0|+43744|-404.40|961126| | 10223240|BOB ANDERSON|7215 ARGYLL ROAD|VANCOUVER|AL|T6C4J2|+52899|+3597|+0|+9990|-90.99|980601| | 10308685|TOM PERRY|3932 - 3A ST. N.W.|VANCOUVER|AL|T2E6R4|+70192|+68155|+0|+125773|+1065.10|920601| | 10318833|HARRY SMIRNOFF|BOX 308, STN J|VANCOUVER|AL|T2A4X6|+11399|+54541|+42046|+93502|+761.46|970530| |
uvhd d5fix/citytax1 r128
========================
rec#=1 rsize=128 fptr=0 fsize=1280 rcount=10
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
3333333322444424445522222222222222233332445442544422222222225444
1013014000AF8E085E29000F00000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H1..C...W.........qq.000149061970530
455452222222222244535343004D005880008000770333333333333333222222
F5652000000000002369318101320072C0047C0111C000149061970530000000
rec#=2 rsize=128 fptr=128 fsize=1280 rcount=10
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
128 10139923 GEORGE BROWN 1250 EAST PENDER ST. VANC
3333333322444544245454222222222222233332445525444452552222225444
101399230075F275022F7E0000000000000125005134005E452034E0000061E3
64 OUVER BCV5L1W1..C...W.........xRL00014626q980601
455452222222222244534353004D005880008800754333333337333333222222
F5652000000000002365C17101320072C0047C0182C000146261980601000000
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 5A1. | Multi Record Type test/demo datafile & copybook |
| cpys/citytax2.cpy - copybook with redefined records | |
| d1ebc/citytax2 - uvhd hexdump of test file with multi record types |
| 5B1. | Generating uvcopy jobs to convert Multi Record Type files |
| - inserting manual code to test the record type & skip to the proper | |
| set of conversion instructions which are automatically generated | |
| from the copybook with redefined records. |
| 5C1. | Converting Multi Record Type files & Splitting to Separate files |
| - a minor change to the manual coding (above), to write separate files |
| 5D1. | Converting Separated Record Types to Pipe Delimited for loading DB tables |
| - 1st modify the copybook to change any occurs to separate fields | |
| since SQL DataBases do not allow occurs. |
| 5E1. | Generating SQL Loader script to load Relational databases |
| - for each of the record types split from a Multi Record Type file |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
cobmap1 start-end bytes for cobol record fields 200403021116 pg# 0001
cpys/citytax2.cpy citytax2 RCSZ=00128 bgn end lth typ
* citytax2 copybook for redefined multi record type file
* tax record (type 'T')
01 citytax2.
10 folio pic x(00008). 0000 0007 008
10 rectype pic x. 0008 0008 001
10 filler001 pic x. 0009 0009 001
10 name pic x(00025). 0010 0034 025
10 address pic x(00025). 0035 0059 025
10 city pic x(00020). 0060 0079 020
10 province pic x(00002). 0080 0081 002
10 zip pic x(00006). 0082 0087 006
10 post-date pic s9(00007) comp-4. 0088 0091 004bns 7
10 land-value pic s9(00009) comp-3. 0092 0096 005pns 9
10 improv-value pic s9(00009) comp-3. 0097 0101 005pns 9
10 face-value pic s9(00009) comp-3. 0102 0106 005pns 9
10 maint-tax pic s9(00007)v99. 0107 0115 009 ns 9
10 purchase-date pic x(00006). 0116 0121 006
10 filler002 pic x(00006). 0122 0127 006
*
* monthly payment record (type 'P')
*RCSZ=00128 0128
01 mthpayrec redefines citytax2.
10 folio pic x(00008). 0000 0007 008
10 rectype pic x. 0008 0008 001
10 filler003 pic x(7). 0009 0015 007
10 tax-year pic 9(4). 0016 0019 004 n 4
10 mthpayments pic s9(9) comp-3 occurs 12. 0020 0024 005pns 9
10 filler004 pic x(48). 0080 0127 048
*
* file header record (type 'H')
*RCSZ=00128 0128
01 headerrec redefines citytax2.
10 folio pic x(00008). 0000 0007 008
10 rectype pic x. 0008 0008 001
10 filler005 pic x. 0009 0009 001
10 year-end pic x(30). 0010 0039 030
10 filler006 pic x(88). 0040 0127 088
************** end of citytax2 copybook ********************
*RCSZ=00128 0128
| Note |
|
uvcopy cobmap1,fili1=cpys/citytax2.cpy,filo1=maps/citytax2 ==========================================================
cobmapA citytax2.cpy <-- script to simplify running cobmap1 (above) ====================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd d1ebc/citytax2 r128a <-- 'a' option to translate char lines to ASCII =========================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 H December 31, 1998
44444444C4C88898894FF64FFFF4444444444444444444444444444444444444
000000008045354259031B019980000000000000000000000000000000000000
64
4444444444444444444444444444444444444444444444444444444444444444
0000000000000000000000000000000000000000000000000000000000000000
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
128 10130140T JOHN HENRY 1815 BOWEN ROAD VANC
FFFFFFFFE4DDCD4CCDDE444444444444444FFFF4CDECD4DDCC4444444444ECDC
1013014030168508559800000000000000018150266550961400000000005153
64 OUVER BCV9S1H1.bi...Wb....g...qq.00014906A970530191809
DEECD44444444444CCEFEFCF0888005880008000770FFFFFFFFCFFFFFFFFFFFF
645590000000000023592181029C0072C0047C0111C000149061970530191809
10 20 30 40 50 60
r# 3 0123456789012345678901234567890123456789012345678901234567890123
256 10130140P 1999...A....A....A....A....A....A....A....A....A
FFFFFFFFD4444444FFFF00042000420004200042000420004200042000420004
101301407000000019990001C0001C0001C0001C0001C0001C0001C0001C0001
64 ....A....A....A.
2000420004200042444444444444444444444444444444444444444444444444
C0001C0001C0001C000000000000000000000000000000000000000000000000
Please relate the 3 record types here (H,T,P in byte 8 above) to the redefined records in the copybook listed on the previous page.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1a. uvcopy uvdata51,fili1=maps/citytax2,filo1=pfx1/citytax2 <-- uvcopy command
======================================================
#1b. gencnvA citytax2.cpy <-- script equivalent of above (shorter command)
==================== - ksh for unix/linux or .bat file for Win/Dos
# uvcopy job to translate EBCDIC to ASCII, preserve packed, fix zoned signs
opr='jobname=JOBNAME - pfx2 name=datafilename'
opr='copybook=citytax2 - pfx1 name=copybookname'
uop=q0,was=a33000b33000
fili1=?d1ebc/citytax2,rcs=00128,typ=RSF
filo1=?d2asc/citytax2,rcs=00128,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00128),a0 move rec to outarea before field prcsng
tra b0(00128) translate entire outarea to ASCII
# --- <-- insert R/T tests here for redefined records
mvc b88(4),a88 bns post-date
mvc b92(15),a92 pns land-value:face-value
trt b107(9),$trtsea ns maint-tax
skp put1
# --- redef, R/T test?
typ__ mvc b20(60),a20 pns mthpayments
skp put1
# --- redef, R/T test?
put1 put filo1,b0
skp loop
eof cls all
eoj
uvcopy jobs generated from copybooks with redefined records require manual coding to test the record type & skip to the proper set of conversion instructions.
Please see this job re-listed on the next page, AFTER record type test coding has been inserted --->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Before modifying, copy the auto generated job to an alternate subdir, for protection in case the generation job gets rerun in future.
cp pfx1/citytax2 pfx3 <-- copy auto-generated job to alternate subdir =====================
vi pfx3/citytax2 <-- modify the job as shown below: ================
opr='JOBNAME citytax2 - genby: cobmap1,uvdata51,uvdata52'
uop=q0
was=a33000b33000
fili1=?d1ebc/citytax2,rcs=00128,typ=RSF
filo1=?d2asc/citytax2,rcs=00128,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00128),a0 move rec to outarea before field prcsng
tra b0(00128) translate entire outarea to ASCII
#------------------------------------------------------------------------
# test Record Type & skip to appropriate instrns to preserve packed & signed
cmc b8(1),'H' Header rec (no packed or signed)
skp= put1
cmc b8(1),'T' Tax rec ?
skp= tax1
cmc b8(1),'P' Payment rec ?
skp= pay1
msg b0(80) show record with invalid type
msgw 'Error Rec Type invalid - enter to bypass'
skp loop
#-------------------------------------------------------------------------
tax1 mvc b88(4),a88 bns post-date
mvc b92(15),a92 pns land-value:face-value
trt b107(9),$trtsea ns maint-tax
skp put1 skip to output
#
pay1 mvc b20(60),a20 pns mthpayments
# ---
put1 put filo1,b0
skp loop
eof cls all
eoj
Please compare this to the previous page (BEFORE changes for multi R/T's).
| See | 'uvcopy3.doc' for the complete uvcopy instruction set available. |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#2a. uvcopy pfx3/citytax2,fili1=d1ebc/citytax2,filo1=d2asc/citytax2
==============================================================
#2b. uvcopy pfx3/citytax2 <-- easier (filenames default as shown above)
====================
uvhd d2asc/citytax2 r128 ========================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 H December 31, 1998
2222222242466666672332233332222222222222222222222222222222222222
00000000804535D252031C019980000000000000000000000000000000000000
64
2222222222222222222222222222222222222222222222222222222222222222
0000000000000000000000000000000000000000000000000000000000000000
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
128 10130140T JOHN HENRY 1815 BOWEN ROAD VANC
3333333352444424445522222222222222233332445442544422222222225444
1013014040AF8E085E29000000000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H1......W.........qq.000149061970530191809
4554522222222222445353430888005880008000770333333333333333333333
F56520000000000023693181029C0072C0047C0111C000149061970530191809
10 20 30 40 50 60
r# 3 0123456789012345678901234567890123456789012345678901234567890123
256 10130140P 1999...A,...A,...A,...A,...A,...A,...A,...A,...A
3333333352222222333300042000420004200042000420004200042000420004
101301400000000019990001C0001C0001C0001C0001C0001C0001C0001C0001
64 ,...A,...A,...A,
2000420004200042222222222222222222222222222222222222222222222222
C0001C0001C0001C000000000000000000000000000000000000000000000000
Please relate these 3 ASCII data records to the original EBCDIC records shown on page '5A2'. Note that option 'a' was used on page '5A2' to translate the character line to ASCII, but not used here since this is ASCII. You should also see that the zones & digits are EBCDIC on page '5A2', but are ASCII here. For example EBCDIC blanks are x'40', but ASCII blanks are x'20'.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Splitting different record types to separate files is a minor change to the uvcopy conversion job listed on page '5B2', since that already includes the code to test the record types. First we will copy & rename in order not to destroy our job that converts record types into 1 output file. We will append an 's' to the original jobname to signify 'split'.
cp pfx3/citytax2 pfx3/citytax2s <-- copy/rename previous job =============================== vi pfx3/citytax2s <-- modify the job as shown below: =================
opr='citytax2s - genby: cobmap1,uvdata51 convert EBCDIC to ASCII'
opr=' - preserving packed fields & correcting signs of zoned fields'
opr=' - record type tests added for redefined records'
opr=' - alternate version to split different rec types to different files'
uop=q0
was=a33000b33000
fili1=?d1ebc/citytax2,rcs=00128,typ=RSF #input EBCDIC 3 rec types
filo1=?d2asc/citytax2a,rcs=00128,typ=RSF #output #1 tax recs
filo2=?d2asc/citytax2b,rcs=00128,typ=RSF #output #2 payment recs <--
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00128),a0 move rec to outarea before field prcsng
tra b0(00128) translate entire outarea to ASCII
#------------------------------------------------------------------------
# test Record Type & skip to appropriate instrns to preserve packed&signed
cmc b8(1),'H' Header rec (no packed or signed)
skp= loop drop Header rec <--
cmc b8(1),'T' Tax rec ?
skp= tax1
cmc b8(1),'P' Payment rec ?
skp= pay1
msg b0(80) show record with invalid type
msgw 'Error Rec Type invalid - enter to bypass'
skp loop
#-----------------------------------------------------------------
tax1 mvc b88(4),a88 bns post-date
mvc b92(15),a92 pns land-value:face-value
trt b107(9),$trtsea ns maint-tax
put filo1,b0 write tax rec to file#1
skp loop return to get next
pay1 mvc b20(60),a20 pns mthpayments
put filo2,b0 write payment rec to file#2 <--
skp loop return to get next
eof cls all
eoj
Please compare this to page '5B2' (before changes to write separate files). We have marked the few changes required with '<--' on the right side above.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvcopy pfx3/citytax2s <-- convert & split MRTs to separate files =====================
uvhd d2asc/citytax2a r128 <-- display Tax records (1st 2 of 10) =========================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140T JOHN HENRY 1815 BOWEN ROAD VANC
3333333352444424445522222222222222233332445442544422222222225444
1013014040AF8E085E29000000000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H1......W.........qq.000149061970530191809
4554522222222222445353430888005880008000770333333333333333333333
F56520000000000023693181029C0072C0047C0111C000149061970530191809
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
128 10139923T GEORGE BROWN 1250 EAST PENDER STREET VANC
3333333352444544245454222222222222233332445525444452555445225444
101399234075F275022F7E0000000000000125005134005E45203425540061E3
64 OUVER BCV5L1W1......W.........xRL00014626q980601190156
4554522222222222445343530888005880008800754333333337333333333333
F5652000000000002365C171029C0072C0047C0182C000146261980601190156
uvhd d2asc/citytax2b r128 <-- display Payment records (1st 2 of 10) =========================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140P 1999...A,...A,...A,...A,...A,...A,...A,...A,...A
3333333352222222333300042000420004200042000420004200042000420004
101301400000000019990001C0001C0001C0001C0001C0001C0001C0001C0001
64 ,...A,...A,...A,
2000420004200042222222222222222222222222222222222222222222222222
C0001C0001C0001C000000000000000000000000000000000000000000000000
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
128 10139923P 1999...B....B....B....B....B....B....B....B....B
3333333352222222333300048000480004800048000480004800048000480004
101399230000000019990002C0002C0002C0002C0002C0002C0002C0002C0002
64 ....B....B....B.
8000480004800048222222222222222222222222222222222222222222222222
C0002C0002C0002C000000000000000000000000000000000000000000000000
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
In the section above, we have split the data records to separate files, and we will now split the copybook redefined records to separate copybooks, so we can use them to process the separate data files.
vi cpys/citytax2.cpy <-- split redefined records to separate copybooks ====================
#1. write Tax record as cpys/citytax2a #2. write Payment record as cpys/citytax2b #3. ignore the Header/Date record (not required)
We will not illustrate the Tax record cpys/citytax2a, since it is the same as cpys/citytax1.cpy, already illustrated previously.
We will illustrate the Payment record, which requires special consideration since it contains recurring data defined by OCCURS (pic s9(9) occurs 12). SQL databases do not allow for 'occurs', so we must define these fields individually. The easiest way to do this is to modify the copybook & then generate the pipe delimit job.
cobmap1 start-end bytes for cobol record fields 200306090839 pg# 0001
cpys/citytax2b mthpayrec RCSZ=00128 bgn end lth typ
* citytax2b copybook
* monthly payment record (type 'P')
* - isolated out of citytax2 (multi rec type)
* to convert mthpay recs to pipe delimited
* - occurs 12 manually split to 12 separate fields
* since occurs not allowed in sql data bases
01 mthpayrec.
10 folio pic x(00008). 0000 0007 008
10 rectype pic x. 0008 0008 001
10 filler001 pic x(7). 0009 0015 007
10 tax-year pic 9(4). 0016 0019 004 n 4
* 10 mthpayments pic s9(9) comp-3 occurs 12.
* original split to sep fields sql does not allow occurs
10 mthpayments-01 pic s9(9) comp-3. 0020 0024 005pns 9
10 mthpayments-02 pic s9(9) comp-3. 0025 0029 005pns 9
10 mthpayments-03 pic s9(9) comp-3. 0030 0034 005pns 9
10 mthpayments-04 pic s9(9) comp-3. 0035 0039 005pns 9
10 mthpayments-05 pic s9(9) comp-3. 0040 0044 005pns 9
10 mthpayments-06 pic s9(9) comp-3. 0045 0049 005pns 9
10 mthpayments-07 pic s9(9) comp-3. 0050 0054 005pns 9
10 mthpayments-08 pic s9(9) comp-3. 0055 0059 005pns 9
10 mthpayments-09 pic s9(9) comp-3. 0060 0064 005pns 9
10 mthpayments-10 pic s9(9) comp-3. 0065 0069 005pns 9
10 mthpayments-11 pic s9(9) comp-3. 0070 0074 005pns 9
10 mthpayments-12 pic s9(9) comp-3. 0075 0079 005pns 9
10 filler002 pic x(48). 0080 0127 048
************** end of citytax2b copybook *******************
*RCSZ=00128 0128
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1a. uvcopy genpipe1,fili1=maps/citytax2b,filo1=pfp1/citytax2b,uop=a0 <-hard way
================================================================
#1b. genpipeA citytax2b <-- easy script equivalent of above (shorter command)
================== - no-ext(ksh) for unix/linux or .bat for Win/Dos
opr='JOBNAME citytax2b - generated by cobmap1,genpipe1,uvdata52'
# - uvcopy job to convert & fix data fields 100 bytes apart
# for compress & '|' delimit (see 'var' instrn below)
rop=j200000r1 #Run OPtions: increase instrn storage & prompt display output
was=a33000b33000c999000d33000e33000
fili1=?d2asc/citytax2b,rcs=00128,typ=RSFm10000
filo1=?d4pipe/citytax2b,rcs=9000,typ=LSTtd3 #<-- DOS option d3 for CR/LF
@run
opn all
loop get fili1,a0
skp> eof
# area a input, see 'get' instrn
# area b translated to ASCII, in case mainframe EBCDIC file (with packed?)
# area c data fields fixed 100 bytes apart in prep for var | delimit instrn
# area d output, see 'put' instrn at end
mvc b0(00128),a0 move input area a to area b
### tra b0(00128) uncomment for EBCDIC input (or option a1)
# --- <-- & insert instrns from uvdata51 if packed fields
# --- <-- insert R/T tests if redef records
mvc c0(8),b0(8) #1 folio
mvc c100(1),b8(1) #2 rectype
mvc c200(7),b9(7) #3 filler001
mvc c300(4),b16(4) #4 tax-year
edta1 c400(11),b20(5p),'+zzzzzzzzz9' #5 mthpayments-01
edta1 c500(11),b25(5p),'+zzzzzzzzz9' #6 mthpayments-02
edta1 c600(11),b30(5p),'+zzzzzzzzz9' #7 mthpayments-03
edta1 c700(11),b35(5p),'+zzzzzzzzz9' #8 mthpayments-04
edta1 c800(11),b40(5p),'+zzzzzzzzz9' #9 mthpayments-05
edta1 c900(11),b45(5p),'+zzzzzzzzz9' #10 mthpayments-06
edta1 c1000(11),b50(5p),'+zzzzzzzzz9' #11 mthpayments-07
edta1 c1100(11),b55(5p),'+zzzzzzzzz9' #12 mthpayments-08
edta1 c1200(11),b60(5p),'+zzzzzzzzz9' #13 mthpayments-09
edta1 c1300(11),b65(5p),'+zzzzzzzzz9' #14 mthpayments-10
edta1 c1400(11),b70(5p),'+zzzzzzzzz9' #15 mthpayments-11
edta1 c1500(11),b75(5p),'+zzzzzzzzz9' #16 mthpayments-12
mvc c1600(48),b80(48) #17 filler002
var d0(32000),c0(100),0017,'|'
trt d0(32000),$trtchr
clr c0(001700),' '
put1 put filo1,d0
skp loop
eof cls all
eoj
@pf2=genpipe1.sub
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#2a. uvcopy pfp1/citytax2b,fili1=d2asc/citytax2b,filo1=d4pipe/citytax2b
==================================================================
#2b. uvcopy pfp1/citytax2b <-- easier (filenames default as shown above)
=====================
#3a. more d4pipe/citytax2b <-- display output
=====================
#3a. uvlp12 d4pipe/citytax2b <-- list the output file
======================= (uvlpr12 for Windows/DOS batch script)
10130140|P||1999|+412|+412|+412|+412|+412|+412|+412|+412|+412|+412|+412|+412|| 10139923|P||1999|+428|+428|+428|+428|+428|+428|+428|+428|+428|+428|+428|+428|| 10147615|P||1999|+124|+124|+124|+124|+124|+124|+124|+124|+124|+124|+124|+124|| 10149304|P||1999|+500|+500|+500|+500|+500|+500|+500|+500|+500|+500|+500|+500|| 10154688|P||1999|+300|+300|+300|+300|+300|+300|+300|+300|+300|+300|+300|+300|| 10173320|P||1999|+405|+405|+405|+405|+405|+405|+405|+405|+405|+405|+405|+405|| 10204700|P||1999|+104|+104|+104|+104|+104|+104|+104|+104|+104|+104|+104|+104|| 10223240|P||1999|+23|+23|+23|+23|+23|+23|+23|+23|+23|+23|+23|+23|| 10308685|P||1999|+301|+301|+301|+301|+301|+301|+301|+301|+301|+301|+301|+301|| 10318833|P||1999|+224|+224|+224|+224|+224|+224|+224|+224|+224|+224|+224|+224||
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1a. uvcopy listpipe1,fili1=d4pipe/citytax2b,fili2=maps/citytax2b,filo1=tmp/citytax2b
================================================================================
#1b. listpipeA citytax2b <-- script equivalent of above (shorter command)
=================== - ksh for unix/linux or .bat file for Win/Dos
listpipe1 - list '|' delimited files with copybook fieldnames
- to verify files created by genpipe1 (see DATAcnv1.htm#Part_4)
datafile=d4pipe/filename? copybookmap=maps/filename? datetime=2003/06/09_13:05:26
SEQ# FIELDNAME COBOL PICTURE DATA
datafile = d4pipe/citytax2b, record# = 1, field count = 17
10130140|P||1999|+412|+412|+412|+412|+412|+412|+412|+412|+412|+412|+412|+412||
001 folio x(00008). 10130140 002 rectype x. P 003 filler001 x(7). 004 tax-year 9(4). 1999 005 mthpayments-01 s9(9) comp-3. +412 006 mthpayments-02 s9(9) comp-3. +412 007 mthpayments-03 s9(9) comp-3. +412 008 mthpayments-04 s9(9) comp-3. +412 009 mthpayments-05 s9(9) comp-3. +412 010 mthpayments-06 s9(9) comp-3. +412 011 mthpayments-07 s9(9) comp-3. +412 012 mthpayments-08 s9(9) comp-3. +412 013 mthpayments-09 s9(9) comp-3. +412 014 mthpayments-10 s9(9) comp-3. +412 015 mthpayments-11 s9(9) comp-3. +412 016 mthpayments-12 s9(9) comp-3. +412 017 filler002 x(48).
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. uvcopy cobmap1,fili1=cpys/citytax2b.cpy,filo1=maps/citytax2b
========================================================
- convert copybook to 'cobmap' (record layout)
#2. uvcopy sqlcreate1,fili1=maps/citytax2b,filo1=sqlTC1/create_citytax2b.sql
========================================================================
,filo2=sqlTL1/load_citytax2b.ctl
=====================================================================
- convert cobmap to SQL scripts to create & load table
- - - OR use script 'sqlcreateA' (combines above into 1 short command)
#12. sqlcreateA citytax2b <-- script equivalent of above (shorter command)
==================== - ksh for unix/linux or .bat file for Win/Dos
#3a. more sqlTC1/create_citytax2b.sql <-- display batch-file output
================================
#3b. more sqlTL1/load_citytax2b.ctl <-- display SQL script output
==============================
cobmap1 start-end bytes for cobol record fields 200306090839 pg# 0001
cpys/citytax2b mthpayrec RCSZ=00128 bgn end lth typ
* citytax2b copybook monthly payment record (type 'P')
* - occurs 12 manually split to 12 separate fields
01 mthpayrec.
10 folio pic x(00008). 0000 0007 008
10 rectype pic x. 0008 0008 001
10 filler001 pic x(7). 0009 0015 007
10 tax-year pic 9(4). 0016 0019 004 n 4
* 10 mthpayments pic s9(9) comp-3 occurs 12.
* original split to sep fields sql does not allow occurs
10 mthpayments-01 pic s9(9) comp-3. 0020 0024 005pns 9
10 mthpayments-02 pic s9(9) comp-3. 0025 0029 005pns 9
10 mthpayments-03 pic s9(9) comp-3. 0030 0034 005pns 9
10 mthpayments-04 pic s9(9) comp-3. 0035 0039 005pns 9
10 mthpayments-05 pic s9(9) comp-3. 0040 0044 005pns 9
10 mthpayments-06 pic s9(9) comp-3. 0045 0049 005pns 9
10 mthpayments-07 pic s9(9) comp-3. 0050 0054 005pns 9
10 mthpayments-08 pic s9(9) comp-3. 0055 0059 005pns 9
10 mthpayments-09 pic s9(9) comp-3. 0060 0064 005pns 9
10 mthpayments-10 pic s9(9) comp-3. 0065 0069 005pns 9
10 mthpayments-11 pic s9(9) comp-3. 0070 0074 005pns 9
10 mthpayments-12 pic s9(9) comp-3. 0075 0079 005pns 9
10 filler002 pic x(48). 0080 0127 048
************** end of citytax2b copybook *******************
*RCSZ=00128 0128
See the output files listed on the next page --->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
/* create_citytax2b.sql - SQL batch file to create a table */ /* - generated from COBOL copybook 20090220:094034 */ /* see www.uvsoftware.ca/datacnv1.htm#Part_4 & sqldemo.htm#Part_4 */ /* */ /* sqlplus user/pass @sqlTC1/create_citytax2b.sql */ /* ============================================= */ DROP TABLE citytax2b; CREATE TABLE citytax2b ( folio char (0008) , --#001 x(00008). rectype char (0001) , --#002 x. tax_year number (04) , --#003 9(4). mthpayments_01 number (09) , --#004 s9(9) comp-3. mthpayments_02 number (09) , --#005 s9(9) comp-3. mthpayments_03 number (09) , --#006 s9(9) comp-3. mthpayments_04 number (09) , --#007 s9(9) comp-3. mthpayments_05 number (09) , --#008 s9(9) comp-3. mthpayments_06 number (09) , --#009 s9(9) comp-3. mthpayments_07 number (09) , --#010 s9(9) comp-3. mthpayments_08 number (09) , --#011 s9(9) comp-3. mthpayments_09 number (09) , --#012 s9(9) comp-3. mthpayments_10 number (09) , --#013 s9(9) comp-3. mthpayments_11 number (09) , --#014 s9(9) comp-3. mthpayments_12 number (09) --#015 s9(9) comp-3. ); exit;
-- load_citytax2b.ctl - SQL*LOADER script to load Oracle table -- - generated from COBOL copybook 20081221:080752 -- see www.uvsoftware.ca/datacnv1.htm#Part_4 & sqldemo.htm#Part_4 -- -- sqlldr user/pass control=sqlTL1/load_citytax2b.ctl -- ================================================== load data infile 'd4pipe/citytax2b' into table citytax2b fields terminated by '|' optionally enclosed by '"' ( folio , --char (0008) , --#001 x(00008). rectype , --char (0001) , --#002 x. tax_year , --number (04) , --#003 9(4). mthpayments_01 , --number (09) , --#004 s9(9) comp-3. mthpayments_02 , --number (09) , --#005 s9(9) comp-3. mthpayments_03 , --number (09) , --#006 s9(9) comp-3. mthpayments_04 , --number (09) , --#007 s9(9) comp-3. mthpayments_05 , --number (09) , --#008 s9(9) comp-3. mthpayments_06 , --number (09) , --#009 s9(9) comp-3. mthpayments_07 , --number (09) , --#010 s9(9) comp-3. mthpayments_08 , --number (09) , --#011 s9(9) comp-3. mthpayments_09 , --number (09) , --#012 s9(9) comp-3. mthpayments_10 , --number (09) , --#013 s9(9) comp-3. mthpayments_11 , --number (09) , --#014 s9(9) comp-3. mthpayments_12 --number (09) --#015 s9(9) comp-3. )
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. Login demo1 --> /home/demo1
===========
#2. sqlplus user/pass @sqlTC1/create_citytax2b.sql
==============================================
- run SQL 'batch-file' to create the table
#3. sqlldr user/pass control=sqlTL1/load_citytax2b.ctl
==================================================
- run SQL*LOADER script to load the table
#4. sqlplus demo1/demo1pw <-- start SQL*Plus
=====================
#5. SQL> select * from citytax2b <-- SELECT all rows/fields from table
====================== - headings & 1st 2 rows shown below:
>>>>> to be supplied
/* select_all_citytax2b.sql - batch file to select all rows from citytax2b */ /* - batch files useful for complex repeated selects */ select * from citytax2b; exit;
You could use the batch-file listed above as an alternative to logging in to SQL*PLUS to list the table rows:
#6. SQL> exit <-- exit from SQL*PLUS (back to Unix/Linux prompt)
#7. sqlplus user/pass @sqls/select_all_citytax2b.sql
================================================
- run SQL 'batch-file' to create the table
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| Note |
|
| 6A1. | Conversions for real data-file-names (vs copy-book-names as in Part3) |
| Converting ALL files in the Directory (vs 1 at a time in previous parts). | |
| - Introduction & overview |
| 6B1. | Generating & Executing Data File Conversions - Preparation |
| - VU installed, profile setup, conversion directories setup | |
| May demo in homedir, cnvdata1 superdir with 15 subdirs | |
| (cpys,ctl,d1ebc,d2asc,d4pipe,maps,pfx2,pfx2,pfx3,pfp1,tmp) | |
| 6B2. | Test/Demo files supplied |
| 6B3. | Directories required for data file covnersions |
| 6B4. | FTP data from Mainframe to Unix/Linux |
| 6B5. | Re-Naming files as required |
| 6C1. | Illustrated Operating Instructions (vs concise at 6H1+). |
| Demo Conversion using supplied test files | |
| - copy demo datafiles & copybooks to d1ebc/ & cpys/ | |
| - rename mainframe GDG files to VU convention & lower case |
| 6C2. | Generate control file from datafilenames in d1ebc/ |
| - inserting copybooknames method#1 - manual edit |
| 6C3. | Inserting copybooknames method#2 |
| - automatic if copybooknames embedded in datafilenames |
| 6C4. | Using control file - to generate data conversion jobs |
| - script gencnvC (runs cobmap1,uvdata51,& uvdata52) |
| 6C5. | Sample outputs for demo file gl.account.gltran |
| maps/gltran - cobmap (record layout) generated from copybook | |
| pfx1/gltran - conversion job generated from cobmap | |
| pfx2/gltran - conversion job with datafilenames (vs copybooknames) |
| 6C6. | Inspect & Edit generated jobs |
| - editing usually required only for files with multi-record-types |
| 6C7. | Execute jobs to convert datafiles EBCDIC to ASCII (d2asc --> d3ebc) |
| - execute all jobs to convert all datafiles in directory | |
| - with 1 command 'uvcoypfxgA' (console log shown) | |
| - repeats conversion jobs for GDG files for all members in the set |
| 6C8. | Investigate converted data files |
| - with uvhd &/or uvhdcob if any packed/binary fields |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 6D1. | Modifying control-file rcs=... typ=... Appending rcso=... typo=... |
| Might use to change output files to typ=LST (with LineFeeds) to | |
| allow use of unix commands (vi,more,lp,etc), but also need to | |
| change COBOL programs to ORGANIZATION LINE SEQUENTIAL. |
| 6E1. | Adding new files to be converted 1 at a time (vs all in directory). |
| 6F1. | Generating jobs to convert fixed layout datafiles to 'Pipe|Delmited' |
| - gencnvC2, script to generate pipe|delimit jobs |
| 6F2. | Sample pipe|delimit job pfp2/gl.account.gltran_ |
| 6F3. | Execute pipe|delimit jobs for all files in directory |
| 6G1. | Copy converted data files from $CNVDATA to $RUNDATA for testing |
| 6G2. | Execute a few VU test/demo JCL/scripts to prove datafile conversions OK |
| 6H1. | Concise Operating Instructions |
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Up to this point, we have been converting 1 file at a time and we have been renaming the datafiles the same as the copybooks to simplify the procedures to generate the data converison jobs from the copybooks.
Using the same name for the copybook, datafile,& all generated jobs is possible because we have different subdirs for different file types (cpys/..., maps/..., pfx1/..., pfp1/... d1ebc/..., d2asc/..., d4pipe/...).
This Part 6 will show you how to perform these conversions for all files in the directory (vs one at a time documented previously), and to insert the actual data filenames in the generated jobs (vs copybooknames).
A key part of this plan is use a control file to relate the data file names to the copybook names. The control file method allows for the distinct possibility of having multiple datafiles using the same copybook.
ar.customer.armaster cpy=____________ rcs=_____ typ=_____ ar.sales.arsales cpy=____________ rcs=_____ typ=_____ gl.account.glmaster_ cpy=____________ rcs=_____ typ=_____ dups=003
ar.customer.armaster cpy=armaster.cpy rcs=_____ typ=_____ ar.sales.arsales cpy=arsales.cpy rcs=_____ typ=_____ gl.account.glmaster_ cpy=glmaster.cpy rcs=_____ typ=_____ dups=003
You will probably have to insert the copybooknames manually with the editor, but after that the conversion jobs are generated automatically from the copybooks & they can all be executed with 1 command. Exceptions would be for files with multiple record types (redefined records in the copybook). In those usually rare instances you need to add code to the generated jobs to select the appopriate bank of auto generated converison isntructions.
Some sites (admittedly rare) embed the copybooknames within the datafilenames and for these sites, we have provided a job to automatically insert the copybooknames in the control file (as illustrated above). This means there would might be no manual editing to connvert the datafiles at these sites.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Here is an illustrated demo of generating & executing all jobs to convert all data files in the test/demo directories supplied by UV Software. We suggest you perform this demo to better understand these procedures, before using them with your own datafiles.
We assume the following has been performed:
#3a. Login userxx --> /home/userxx
#3b. vi .profile OR .bash_profile
===========
export CNVDATA=$HOME/cnvdata1 # Conversion superdir & subdirs export RUNDATA=$HOME/testdata1 # data for testing in homedir export RUNLIBS=$HOME/testlibs1 # COBOL, copybooks,& JCL/scripts
#4a. exit #4b. Login userxx --> /home/userxx Re-login to make exports effective
#4c. mkdir cnvdata1 testdata1 testlibs1
==================================
#5a. cdc --> alias cdc='cd $CNVDATA' --> /home/userxx/cnvdata1
#5b. cnvdatadirs <-- setup subdirs for data conversion
===========
#6a. cdd --> /home/userxx/testdata1
#6b. mvsdatadirs <-- setup subdirs for JCL/script executions
===========
#6a. cdl --> /home/userxx/testlibs1
#6b. mvslibsdirs <-- setup subdirs for copybooks,COBOL,& JCL conversion
===========
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
/home/uvadm/mvstest/mvstest1 <-- location in Vancouver Utilities distribution /home/mvstest1 <-- OR setup user mvstest1 /home/userxx OR setup cnvdata1 within your $HOME dir :-----cnvdata1 <-- cnvdata1 superdir for DATA conversion : :-----archive - backup to reload d1ebc/... copybooknames or datafilenames : : :-----d1ebc_cpy_names <-- for Parts3,4,5 : : :-----d1ebc_data_names <-- for Part6 :-----cnvdata1 : :-----cpys <-- copybooks <--- copy from $UV/mvstest1/cnvdata1/cpys/* : :-----ctl - control files : :-----d0ebc <-- EBCDIC data files <-- copy $UV/mvstest1/cnvdata1/d0ebc/* : :-----d1ebc - EBCDIC data files renamed to VU conventions : :-----d2asc - ASCII data files (converted by uvcopy uvdata51) : :-----d3ebc - convert back to EBCDIC by uvdata31 : :-----d4pipe - pipe delimited to load DB tables (by genpipe1) : :-----maps - copybooks with record layouts on right side : :-----pfx1 - uvcopy jobs to convert EBCDIC to ASCII (gen by uvdata51) : :-----pfx2 - with data filenames inserted (vs copybook names) : :-----pfx3 - copied here for modify/execute : :-----pfp1 - uvcopy jobs to create pipe delimited (gen by genpipe1) : :-----pfp2 - pipe delim jobs with datafilenames inserted : :-----pfp3 - copy here before editing for multi-record-type files : :-----sqlTC1 - scripts to Create database tables (gen from copybooks) : :-----sqlTL1 - scripts to Load database tables (gen from copybooks) : :-----tmp - temp files (keep working directory clean) : : :-----testdata1 : :-----ctl : :-----data <-- copy d2asc/* here to test with JCL/scripts : :-----jobctl : :-----joblog : :-----jobmsgs : :-----jobtimes : :-----jobtmp : :-----sysout : : :-----testlibs1 : :-----cbl0,cbl1,cbl2,cbls,cblx : :-----cpy0,cpy1,cpy2,cpys : :-----ctl : :-----jcl0,jcl1,jcl2,jcl3,jcls : :-----parm0,parms : :-----proc0,procs
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
If you have already run the test/demo conversions & you or others wish to re-run, you will want to clear the outputs of prior test/demo conversions, for easier inspection of your new outputs & avoid confusion with prior outputs.
#1. cnvdatainit <-- run script to clear any old outputs
=========== - d2asc,d4pipe,pfx1,pfx2,pfx3,pfp1,etc
- see script listed on page '10K1' or in $UV/sf/IBM/...
#2. rm -f d1ebc/* <-- clear d1ebc/... in case retest Part3 after Part6
=============
#3. cp archive/d1ebc_data_names/* d1ebc/. <-- for Part6 real datafilenames
====================================
- refresh d1ebc/... files named same as copybooks (for Parts3,4,5)
archive
|-- d1ebc_cpy_names/ <-- copybooknames for Part3,4,5
| |-- armaster
| |-- arsales
| |-- artrans
| |-- citytax1
| |-- citytax2
| |-- glmaster
| |-- gltran
| |-- vendormas
| |-- vendorpaymas
| `
`-- d1ebc_data_names/ <-- datafilenames for Part6 & DATAcnv2.doc
|-- ar.customer.armaster
|-- ar.customer.arsales
|-- ar.customer.artrans
|-- ar.customer.phone.list
|-- city.tax.assess
|-- city.tax.payments
|-- gl.account.glmaster.g0001v00
|-- gl.account.glmaster.g0002v00
|-- gl.account.gltran.g0001v00
|-- gl.account.gltran.g0002v00
|-- vendor.master.names
`-- vendor.master.payments
cpys-------->maps---------->pfx1-------------->pfx2----------->pfx3--------->
cobmap1 uvdata51 uvdata52 copy/edit execute
convert copybooks change copybooknames
to uvcopy jobs to datafilenames
Mainframe---->d0ebc---------->d1ebc------------->d2asc------->$RUNDATA/data
FTP rename GDG/lower EBCDIC->ASCII copy files to test dirs
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
For demos in Part 6, we will use testfiles supplied with Vancouver Utilities, but for your covnersion, you will need to FTP files from the mainframe & convert as illustrated with the demo files in the rest of this Part 6.
If you had no packed/binary fields you could FTP in TEXT MODE which automatically translates to ASCII, but the CR/LF makes records 2 bytes longer, & you either have to remove to match COBOL programs, or modify program SELECTs to 'ORGANIZATION LINE SEQUENTIAL' which allows for the CR/LF & also allows you to use unix system utilities (vi,more,lp,cat,etc) that required LineFeeds.
But most mainframes have a lot of files with packed/binary fields & it is probably best to FTP all files in BINARY MODE & use the VU conversion methods, which generate data conversion jobs automatically from the COBOL copybooks, to preserve the packed/binary fields (same for unix COBOLs as mainframe COBOLs). Note that unpacked signed fields require zoned sign conversion. Mainframe units byte positives are coded '{ABCDEFGHI' & negatives '}JKLMNOPQR'. Unix COBOL positives are coded just '0123456789' & negatives 'pqrstuvwxy'. The VU methods aallow for this & an alternate version could be used with FTP text/ASCII files.
Variable-> fixed BCDIC->ASCII copy/rename
MAINFRAME -----> d0ebc ------> d1ebc ----------> d2asc ---------> $RUNDATA/data
FTP/BINARY ASCII to d2asc pfx1/uvcopy jobs cp/renameL
#1. Login --> your homedir
=====
#2. cdc --> $HOME/$CNVDATA
===
#3. cd d0ebc <-- change into the data subdir
========
#4. ftp xxx.xxx.xxx.xxx <-- FTP to mainframe IP#
===================
#4a. userid--> ..... <-- enter userid
#4b. passwd--> ..... <-- enter password
#4c. ascii <-- translate EBCDIC to ASCII & insert CR/LF
---OR--- (ascii is usually the default)
#4c. binary <-- ensure binary transfer
#4d. quote site RDW <-- required if any variable length files
#4e. cd ... <-- change to data files directory ??
#4f. get 'XXX.XXX.XXX' <-- get desired data files
#4g. get '...etc....' - 'single quotes' may be required
#5. Might need to change permissions on files FTP'd from mainframe
#5a. chmod 664 * <-- files should be 664
===========
| See | '8A2' for 'LOCSITE RDW' option to FTP to unix via JCL on mainframe |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
On some systems FTP'd files may be enclosed in single quotes, may be UPPER case, may have embedded blanks or '$' signs. ON unix we need to remove single quotes, remove any embedded blanks or '$' signs, & lower case is strongly recommended.
#5. mv \'XXX.XXX.XXX\' xxx.xxx.xxx <-- the hard way
================================ - see #6. the easy way
Imagine how awkward manual renaaming via 'mv' commands would be if you have hundreds of files to be renamed. Vancouver Utilities has many 'rename' scripts to make mass changes to filenames easy.
#6a. renameQQ data <-- easy way to remove Quotes from All files in subdir
=============
#6b. renameL data <-- easy way to translate names to Lower case
============
#6c. renameBU data <-- easy way to convert Blanks to '_' Underscores
=============
#6d. renameDU data <-- easy way to convert '$' signs to '_' Underscores
=============
After converting GDG files, you must rename them using the VU GDG file naming conventions - appending '_000001' for the 1st generation. For example:
cp $CNVDATA/d2asc/AR.CUSTOMER.MASTER(0) $RUNDATA/data/ar.customer.master_000001 ===============================================================================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
If your data files do not have packed/binary fields, you can use FTP ASCII vs binary which requires UV Software DATA conversions driven by copybooks to translate only the character fields, but preserve the packed/binary fields which would be destroyed by EBCDIC to ASCII translation.
But FTP ASCII adds 2 bytes (Carriage-Return & Line-Feed) to each record, which means the record-sizes will not agree with the converted COBOL programs & utilities in JCL/scripts (SORT,IDCAMS,IEBGENER,etc).
You can use the ASCII files with CR/LF if you change the COBOL program SELECTs from 'organization RECORD sequential' to 'organization LINE sequential' and you do not have to change the record size in the COBOL programs.
The Vancouver Utilities (uvsort,uvcp,etc) can also accept the ASCII files if you change 'typ=RSF' to 'typ=LST' which means the the record size is determiend by the LF (LineFeed) & allows the record-size to remain the same as long as there are not more than the 2 extra bytes for CR/LF.
If the record size was any greater the record would be split into 2 records. You could increase the record sizes in the JCL to allow a saftey margin.
Here is the JCL/script for a SORT of 50 byte fixed length records, with the original #commented out & the changes on a following line + a note about the change. It is a good idea to make notes such as these, for other team members (and youself when you come back to it at a later time).
#3------------------------------------------------------------------------------ #Sep08/13/OT - change file type & rec-size for FTP ASCII files & LINE sequential # uvsort "fili1=$SORTIN,typ=RSF,rcs=50,filo1=$SORTOUT,typ=RSF,rcs=50,keys=(0,5,c,a)" uvsort "fili1=$SORTIN,typ=LSTt,rcs=80,filo1=$SORTOUT,typ=LSTt,rcs=80,keys=(0,5,c,a)" #4------------------------------------------------------------------------------
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#0a. Login userxx --> /home/userxx
#0b. cdc --> alias cdc='cd $CNVDATA' --> /home/userxx/cnvdata1
#1a. cp $UV/mvstest1/cnvdata1/d0ebc/* d0ebc
======================================
- copy VU supplied test data files to your homedir/cnvdata1/d0ebc/
AR.CUSTOMER.ARMASTER
AR.SALES.ARSALES
GL.ACCOUNT.GLMASTER.G0001V00 <-- GDG files in mainframe format
GL.ACCOUNT.GLMASTER.G0002V00
GL.ACCOUNT.GLMASTER.G0003V00
GL.ACCOUNT.GLTRAN
GL.ACCOUNT.GLTRAN.G0001V00
GL.ACCOUNT.GLTRAN.G0002V00
GL.ACCOUNT.GLTRAN.G0003V00
#1b. cp $UV/mvstest1/cnvdata1/cpys/* cpys
====================================
- copy VU supplied copybooks to your homedir/cnvdata1/cpys/
-rw-rw-r-- 1 mvstest1 apps 720 Aug 18 13:50 armaster.cpy
-rw-rw-r-- 1 mvstest1 apps 687 Aug 18 13:53 arsales.cpy
-rw-rw-r-- 1 mvstest1 apps 791 Aug 18 13:54 citytax1.cpy
-rw-rw-r-- 1 mvstest1 apps 1795 Aug 18 13:51 citytax2.cpy
-rw-rw-r-- 1 mvstest1 apps 499 Aug 18 13:52 glmaster.cpy
-rw-rw-r-- 1 mvstest1 apps 548 Aug 24 11:03 gltran.cpy
#2. cp d0ebc/* d1ebc/ <-- copy to d1ebc before rename (save original)
=================
#3a. renameGDG d1ebc <-- rename GDG files from mainframe convention to UV
=============== .G0001V00 -> _000001, .G0002V00 -> _000002, etc
#3b. renameL d1ebc <-- rename to lowercase
=============
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
ar.customer.armaster
ar.sales.arsales
gl.account.glmaster_000001 <-- GDG files in VU/GDG format
gl.account.glmaster_000002
gl.account.glmaster_000003
gl.account.gltran <-- non-gdg file
gl.account.gltran_000001
gl.account.gltran_000002 <-- GDG files
gl.account.gltran_000003
#4. uvcopy data2cpy51,fild1=d1ebc,filo2=ctl/datacpy51
=================================================
- create ctl/datacpy51 from all datafilenames in d1ebc/...
- reducing GDG filenames to 1 entry per GDG set
#4a. uvcopy data2cpy51 <-- same as above, filenames default as shown above
=================
--> cat <-- reply 'cat' at the prompt for output file viewing
ar.customer.armaster cpy=____________ rcs=_____ typ=_____ ar.sales.arsales cpy=____________ rcs=_____ typ=_____ citytax1 cpy=____________ rcs=_____ typ=_____ citytax2 cpy=____________ rcs=_____ typ=_____ gl.account.glmaster_ cpy=____________ rcs=_____ typ=_____ dups=003 gl.account.gltran cpy=____________ rcs=_____ typ=_____ gl.account.gltran_ cpy=____________ rcs=_____ typ=_____ dups=003
#5. cp ctl/datacpy51 ctl/datacpy52
==============================
- copy/rename to prevent edited file overwrite if data2cpy51 rerun
#6. vi ctl/datacpy52 <-- edit/insert copybook names as shown below
================
ar.customer.armaster cpy=armaster.cpy rcs=_____ typ=_____ ar.sales.arsales cpy=arsales.cpy rcs=_____ typ=_____ citytax1 cpy=citytax1.cpy rcs=_____ typ=_____ citytax2 cpy=citytax2.cpy rcs=_____ typ=_____ gl.account.glmaster_ cpy=glmaster.cpy rcs=_____ typ=_____ dups=003 gl.account.gltran cpy=gltran.cpy rcs=_____ typ=_____ gl.account.gltran_ cpy=gltran.cpy rcs=_____ typ=_____ dups=003
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
IF copybooknames are embedded in datafilenames (at some sites), you can use 'data2cpy52' to extract copybooknames from datafilenames into the control-file. (Unique to some sites, but a great convenience).
'data2cpy52' would be run following 'data2cpy51' & it assumes that the copybookname is the last segment in the filename, following the last '.' & ended by a blank or the '_' indicating a GDG file.
#5. uvcopy data2cpy52,fili1=ctl/datacpy51,filo1=ctl/datacpy52
=========================================================
- automatic insert copybooknames from datafilenames
#5a. uvcopy data2cpy52 <-- same, files default as above
=================
#6. vi ctl/datacpy52 <-- inspect ctlfile with copybook names
================ - make corrections if auto code not perfect
ar.customer.armaster cpy=armaster.cpy rcs=_____ typ=_____ ar.sales.arsales cpy=arsales.cpy rcs=_____ typ=_____ citytax1 cpy=citytax1.cpy rcs=_____ typ=_____ citytax2 cpy=citytax2.cpy rcs=_____ typ=_____ gl.account.glmaster_ cpy=glmaster.cpy rcs=_____ typ=_____ dups=003 gl.account.gltran cpy=gltran.cpy rcs=_____ typ=_____ gl.account.gltran_ cpy=gltran.cpy rcs=_____ typ=_____ dups=003
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#7. gencnvC ctl/datacpy52 <-- generate data conversion jobs
=====================
- script 'gencnvC' runs 3 uvcopy jobs (cobmap1, uvdata51,& uvdata52)
- see script listed at https://uvsoftware.ca/datacnv1.htm#10E1
----- OR could run separately as follows -----
#7a. uvcopyx cobmap1 cpys maps uop=q0i7
==================================
- convert cobol copybooks to maps (record layouts)
#7b. uvcopyx uvdata51 maps pfx1 uop=q0i7
===================================
- generates uvcopy data conversion jobs from copybookmaps
- codes fili1=... & filo1=... & jobnames sames as copybooknames
#7c. cp $UV/IBM/skeleton2 pfx1
=========================
- copy uvcopy job skeleton for datafiles without copybooks
- OK for files that have no packed/binary fields
- could modify control file typ=LST (LineFeeds OK if no packed/binary)
-rw-rw-r-- 1 mvstest1 apps 743 Aug 18 15:03 armaster
-rw-rw-r-- 1 mvstest1 apps 725 Aug 18 15:03 arsales
-rw-rw-r-- 1 mvstest1 apps 811 Aug 18 15:03 citytax1
-rw-rw-r-- 1 mvstest1 apps 1033 Aug 18 15:03 citytax2
-rw-rw-r-- 1 mvstest1 apps 680 Aug 18 15:03 glmaster
-rw-rw-r-- 1 mvstest1 apps 673 Aug 18 15:03 gltran
-rw-rw-r-- 1 mvstest1 apps 943 Aug 18 15:03 skeleton2
#7d. uvcopy uvdata52,fili1=ctl/datacpy52,fild2=pfx1,fild3=pfx2,uop=q0i7
==================================================================
- copies pfx1/* to pfx2/... inserting datafilenames (vs copybooknames)
- also renames the jobs for the datafilenames (vs copybooknames
- since could have multiple datafiles for same copybook (esp. for GDG)
-rw-rw-r-- 1 mvstest1 apps 780 Aug 18 15:03 ar.customer.armaster
-rw-rw-r-- 1 mvstest1 apps 752 Aug 18 15:03 ar.sales.arsales
-rw-rw-r-- 1 mvstest1 apps 812 Aug 18 15:03 citytax1
-rw-rw-r-- 1 mvstest1 apps 1034 Aug 18 15:03 citytax2
-rw-rw-r-- 1 mvstest1 apps 717 Aug 18 15:03 gl.account.glmaster_
-rw-rw-r-- 1 mvstest1 apps 708 Aug 18 15:03 gl.account.gltran
-rw-rw-r-- 1 mvstest1 apps 708 Aug 18 15:03 gl.account.gltran_
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
01 gltran.
10 glt-acntno pic 9(00008). 0000 0007 008 n 8
10 glt-type pic x(00002). 0008 0009 002
10 glt-description pic x(00030). 0010 0039 030
10 glt-amount pic s9(00010). 0040 0049 010 ns10
10 glt-status pic x(00001). 0050 0050 001
10 glt-client pic x(00004). 0051 0054 004
10 glt-province pic x(00002). 0055 0056 002
10 glt-trace pic x(00012). 0057 0068 012
10 glt-filler1 pic x(00011). 0069 0079 011
*RCSZ=00080
# uvcopy job to translate EBCDIC to ASCII, preserve packed, fix zoned signs
opr='jobname=JOBNAME - pfx2 name=datafilename'
opr='copybook=gltran - pfx1 name=copybookname'
uop=q0,was=a64000b64000
fili1=?d1ebc/gltran,rcs=00080,typ=RSF
filo1=?d2asc/gltran,rcs=00080,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00080),a0 move rec to outarea before field prcsng
tra b0(00080) translate entire outarea to ASCII
# --- <-- insert R/T tests here for redefined records
trt b40(10),$trtsea ns at-amount
put1 put filo1,b0
skp loop
eof cls all
eoj
# uvcopy job to translate EBCDIC to ASCII, preserve packed, fix zoned signs
opr='jobname=gl.account.gltran_ - pfx2 name=datafilename'
opr='copybook=gltran - pfx1 name=copybookname'
uop=q0,was=a64000b64000
fili1=?d1ebc/gl.account.gltran_,rcs=00080,typ=RSF
filo1=?d2asc/gl.account.gltran_,rcs=00080,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00080),a0 move rec to outarea before field prcsng
tra b0(00080) translate entire outarea to ASCII
# --- <-- insert R/T tests here for redefined records
trt b40(10),$trtsea ns at-amount
put1 put filo1,b0
skp loop
eof cls all
eoj
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
It may be necessary to make some manual changes to auto-generated jobs. Most common reason is to allow for files with multiple record-types. You need to insert code to test R/T & skip to corresponding bank of instructions already automatically generated from redefined records in copybook. Here is the example from https://uvsoftware.ca/datacnv1.htm#5A1 - 5C2
#8. cp pfx2/* pfx3 <-- copy auto-generated jobs before any editing
============== - to avoid losing manual changes if generation rerun
#9. vi pfx3/... <-- make any manual changes required
=========== - as discussed above for multi-record-type files
# uvcopy job to translate EBCDIC to ASCII, preserve packed, fix zoned signs
opr='jobname=citytax2 - pfx2 name=datafilename'
opr='copybook=citytax2.cpy - pfx1 name=copybookname'
uop=q0,was=a64000b64000
fili1=?d1ebc/citytax2,rcs=00128,typ=RSF
filo1=?d2asc/citytax2,rcs=00128,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00128),a0 move rec to outarea before field prcsng
tra b0(00128) translate entire outarea to ASCII
# --- <-- insert R/T tests here for redefined records
# Next 9 instructions inserted manually to test multi-record-types
cmc b8(1),'H' Header record
skp= put1
cmc b8(1),'T' Tax record
skp= typT
cmc b8(1),'P' Payment record
skp= typP
msg b0(64) Error - show 1st 64 bytes
msgw 'Invalid record type byte 8 not H,T,P - enter to bypass'
skp loop
#
typT mvc b88(4),a88 bns post-date
mvc b92(15),a92 pns land-value:face-value
trt b107(9),$trtsea ns maint-tax
skp put1
# ---
typP mvc b20(60),a20 pns mthpayments
skp put1
# ---
put1 put filo1,b0
skp loop
eof cls all
eoj
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#10. uvcopypfxgA pfx3 d1ebc d2asc <-- script executes all jobs in pfx3
============================ - converting files from d1ebc to d2asc
- execute all jobs in pfx3/...
- to convert all files from d1ebc/... to d2asc/...
- Repeats jobs for all members of GDG file groups
- see script 'uvcopypfxgA' listed on page '10F1'
- see console log below illustrating GDG file conversions
jobname=ar.customer.armaster - pfx2 name=datafilename
copybook=armaster.cpy - pfx1 name=copybookname
140824:103046:ar.customer.ar: uvcopy ver=20140813 pf=pfx3/ar.customer.armaster
uvcopy LNX H64 license=140813_99V_930630 site=UV_Software
140824:103046:ar.customer.ar: EOF fili01 rds=32 size=8192: d1ebc/ar.customer.armaster
140824:103046:ar.customer.ar: EOF filo01 wrts=32 size=8192: d2asc/ar.customer.armaster
--- omitting log for 5 files ---
jobname=gl.account.gltran_ - pfx2 name=datafilename copybook=gltran - pfx1 name=copybookname 140824:103046:gl.account.glt: uvcopy ver=20140813 pf=pfx3/gl.account.gltran_ uvcopy LNX H64 license=140813_99V_930630 site=UV_Software 140824:103046:gl.account.glt: EOF fili01 rds=20 size=1600: d1ebc/gl.account.gltran_000001 140824:103046:gl.account.glt: EOF filo01 wrts=20 size=1600: d2asc/gl.account.gltran_000001
jobname=gl.account.gltran_ - pfx2 name=datafilename copybook=gltran - pfx1 name=copybookname 140824:103046:gl.account.glt: uvcopy ver=20140813 pf=pfx3/gl.account.gltran_ uvcopy LNX H64 license=140813_99V_930630 site=UV_Software 140824:103046:gl.account.glt: EOF fili01 rds=20 size=1600: d1ebc/gl.account.gltran_000002 140824:103046:gl.account.glt: EOF filo01 wrts=20 size=1600: d2asc/gl.account.gltran_000002
| Note |
|
uvcopyxx 'pfx3/*' =================
Being able to kickoff all the data-file covnersions with 1 command is very handy since you may need to repeat teh conversions several times before testing complete and of course on the go-live weekend.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#11. l d2asc <-- list output files converted to ASCII
=======
-rw-rw-r-- 1 uvtest apps 8192 Aug 27 10:23 ar.customer.armaster -rw-rw-r-- 1 uvtest apps 1280 Aug 27 10:23 ar.sales.arsales -rw-rw-r-- 1 uvtest apps 1280 Aug 27 10:23 citytax1 -rw-rw-r-- 1 uvtest apps 2048 Aug 27 10:23 citytax2 -rw-rw-r-- 1 uvtest apps 13952 Aug 27 10:23 gl.account.glmaster_000001 -rw-rw-r-- 1 uvtest apps 13952 Aug 27 10:23 gl.account.glmaster_000002 -rw-rw-r-- 1 uvtest apps 13952 Aug 27 10:23 gl.account.glmaster_000003 -rw-rw-r-- 1 uvtest apps 1600 Aug 27 10:23 gl.account.gltran -rw-rw-r-- 1 uvtest apps 1600 Aug 27 10:23 gl.account.gltran_000001 -rw-rw-r-- 1 uvtest apps 1600 Aug 27 10:23 gl.account.gltran_000002 -rw-rw-r-- 1 uvtest apps 1600 Aug 27 10:23 gl.account.gltran_000003
#12a. uvhd d2asc/citytax1 r128 <-- inspect file with uvhd
========================
rec#=1 rsize=128 fptr=0 fsize=1280 rcount=10
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
3333333322444424445522222222222222233332445442544422222222225444
1013014000AF8E085E29000000000000000181502F75E02F14000000000061E3
64 OUVER BCV9S1H1..C...W.........qq.000149061970530
455452222222222244535343004D005880008000770333333333333333222222
F5652000000000002369318101320072C0047C0111C000149061970530000000
#12b. uvhdcob d2asc/citytax1 maps/citytax1 <-- uvhdcob to see COBOL fieldnames
====================================
rec#=1 rcount=10 rsize=128 fsize=1280 fptr=0 rec# 1 occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 ctx-folio 0 9 10130140 002 ctx-name 10 34 JOHN HENRY 003 ctx-address 35 59 1815 BOWEN ROAD 004 ctx-city 60 79 VANCOUVER 005 ctx-province 80 81 BC 006 ctx-zip 82 87 V9S1H1 007 ctx-post-date 88 91bns 000143D2 (82898 dcml) 008 ctx-land-value 92 96pns 000057828C 009 ctx-imp-value 97 101pns 000004870C 010 ctx-face-value 102 106pns 000171710C 011 ctx-maint-tax 107 115 ns 000149061 012 ctx-purch-date 116 121 970530 013 filler001 122 127
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The control file is generated with rcs=_____ & typ=______ which is OK, because the record-size is determined by the copybook & the file type is defaulted to 'typ=RSF' (mainframe files FTP'd binary mode).
If desired, you can MODIFY rcs=_____ & typ=_____ and ADD rcso=... & typo=... I suggest you complete the 1st conversion using the control file as generated, & then as an experiment, you can modify the control file,& re-convert to note the differences in the generated jobs.
ar.sales.arsales1 cpy=arsales.cpy rcs=_____ typ=LST ar.sales.arsales2 cpy=arsales.cpy rcs=_____ typ=_____ typo=LST ar.sales.arsales3 cpy=arsales.cpy rcs=_____ typ=_____ typo=RST ar.sales.arsales4 cpy=arsales.cpy rcs=_____ typ=_____ rcso=80 typo=RST
Changing control file ar.sales.arsales1 from 'typ=_____' to 'typ=LST' would have the following BEFORE/AFTER effect in the generated job (pfx2/ar.sales.arsales1).
fili1=?d1ebc/ar.sales.arsales,rcs=00064,typ=RSF <-- BEFORE changing filo1=?d2asc/ar.sales.arsales,rcs=00064,typ=RSF control file
fili1=?d1ebc/ar.sales.arsales,rcs=00064,typ=LST <-- AFTER change to typ=LST filo1=?d2asc/ar.sales.arsales,rcs=00064,typ=LST
You might think of doing this for files that are FTP'd in ASCII mode (vs binary) to allow for the LineFeed appended by FTP ASCII, but you would also have to disable the 'tra' (TRanslate to Ascii) instruction in the generated uvcopy job. These conversions are intended to be used when ALL files are FTP'd in binary mode to allow for packed/binary fields.
| typ=RSF |
|
| typ=LST |
|
| typ=RST |
|
| typ=LSTt |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Adding 'typo=LST' in the control file would change output filo1 to 'typ=LST' in the generated job (pfx2/ar.sales.arsales2) as follows:
fili1=?d1ebc/ar.sales.arsales2,rcs=00064,typ=RSF filo1=?d2asc/ar.sales.arsales2,rcs=00064,typ=LST <-- control file typo=LST
You might append 'typo=LST' to change output file type to Line Sequential (LineFeed terminated) to allow use of unix utilities (vi, more, lp, etc). You would also need to change the COBOL programs using this file. You could add 1 to the record-size to allow for the extra byte (LineFeed), but much better to change the file select to ORGANIZATION LINE SEQUENTIAL (could leave record-size as is).
You would also need to change any uvsort's & uvcp's in the JCL/scripts from typ=RSF to typ=LST. BUT, you could do this automatically by copying the $CNVDATA/ctl/datacpy52 file to $RUNLIBS/ctl/add/... and reconverting the JCL/scripts using jcldata51A & jclxx51 (jclxx53 for AIX).
Adding 'typo=RST' in the control file would change filo1 output to 'typ=RST' in the generated job (pfx2/ar.sales.arsales3) as follows:
fili1=?d1ebc/ar.sales.arsales3,rcs=00064,typ=RSF filo1=?d2asc/ar.sales.arsales3,rcs=00064,typ=RST <-- control file typo=RST
You could use 'typo=RST' if you know that the last byte of the record is unused. 'typ=RST' inserts a LineFeed in last byte of the specified record-size vs appending a LineFeed onto the specified record-size.
Appending 'rcso=80 typo=RST' in the control file would have the following effect in the generated job (pfx2/ar.sales.arsales4).
fili1=?d1ebc/ar.sales.arsales4,rcs=00064,typ=RSF filo1=?d2asc/ar.sales.arsales4,rcs=00080,typ=RST <-- rcso=80 & typo=RST
This example shows you could change the output record-size, but you probably would not, because COBOL programs & uvsorts would also have to be changed.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
This Part 6 documents generating conversion jobs for ALL files in the directory (vs previous Parts 3,4,& 5 for 1 ata time). This page will show you how you might make 1 at a time additions to the operating procedures written for all files in the directory.
Let's assume you need to add 'PY.PAYROLL.PYMASTER' file to the files already converted in this Part 6.
#0a. Login userxx --> /home/userxx
#0b. cdc --> alias cdc='cd $CNVDATA' --> /home/userxx/cnvdata1
1a. FTP the new data-file to $CNVDATA/d0ebc/PY.PAYROLL.PYMASTER
1b. FTP the corresponding copybook to $CNVDATA/cpys/pymaster.cpy
#2. cp d0ebc/PY.PAYROLL.PYMASTER d1ebc/ <-- copy to d1ebc
===================================
#3a. renameGDG d1ebc <-- if GDG file, rename to UV convention
=============== .G0001V00 -> _000001, etc
#3b. renameL d1ebc <-- rename to lowercase
=============
| Note |
|
On page '6C2', we generated the control file for all files in directory d1ebc/ but to add 1 file, we can just add a new line to the existing control file.
#6. vi ctl/datacpy52 <-- add entry for py.payroll.pymaster
================
ar.customer.armaster cpy=armaster.cpy rcs=_____ typ=_____ ar.sales.arsales cpy=arsales.cpy rcs=_____ typ=_____ citytax1 cpy=citytax1.cpy rcs=_____ typ=_____ citytax2 cpy=citytax2.cpy rcs=_____ typ=_____ gl.account.glmaster_ cpy=glmaster.cpy rcs=_____ typ=_____ dups=003 gl.account.gltran cpy=gltran.cpy rcs=_____ typ=_____ gl.account.gltran_ cpy=gltran.cpy rcs=_____ typ=_____ dups=003 py.payroll.pymaster cpy=pymaster.cpy rcs=00512 typ=RSF
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#7. gencnvC ctl/datacpy52 <-- generate data conversion jobs
=====================
- script 'gencnvC' runs 3 uvcopy jobs (cobmap1, uvdata51,& uvdata52)
- see script listed at https://uvsoftware.ca/datacnv1.htm#10E1
| Note |
|
You would now copy just the new generated job for the new file from pfx2 to pfx3 and edit if required (for multi-record-types).
#8. cp pfx2/py.payroll.pymaster pfx3 <-- copy new job before any editing
================================
#9. vi pfx3/py.payroll.pymster <-- make any manual changes required
==========================
#10. uvcopy pfx3/py.payroll.pymaster
===============================
- execute just 1 job in pfx3/...
- to convert just 1 file from d1ebc/... to d2asc/...
- jobs for GDG files are repeated depending on generations present
| Note |
|
The above procedure added the new file entry to the existing control file & re-generated conversion jobs for all datafiles. Alternatively you could make a separate control-file with just 1 entry to generate just 1 new job. (As noted above regenerating all to pfx2/... does no harm as long as you had copied pfx2/* to pfx3/... before any manual edits).
#6a. vi ctl/datacpy52a <-- create alternate ctlfile with just 1 new entry
=================
py.payroll.pymaster cpy=pymaster.cpy rcs=00512 typ=RSF
#7a. gencnvC ctl/datacopy52a
=======================
- generate just the 1 new job (as specified in aalternate ctlfile)
- 'gencnvC' runs 3 utilities (cobmap1,uvdata51, uvdata52) see page '6C4'
#8. #9, #10 - run the new job same as above
=======================================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Script 'gencnvC2' is similar to gencnvC (EBCDIC to ASCII conversions), but generates jobs to create pipe|delimited files. 'gencnvC' is a pre-requisite, since the input to the pipe|delimit jobs is the ASCII files from gencnvC. gencnvC2 uses the same control file as gencnvC.
ar.customer.armaster cpy=armaster.cpy rcs=_____ typ=_____ ar.sales.arsales cpy=arsales.cpy rcs=_____ typ=_____ citytax1 cpy=citytax1.cpy rcs=_____ typ=_____ citytax2 cpy=citytax2.cpy rcs=_____ typ=_____ gl.account.glmaster_ cpy=glmaster.cpy rcs=_____ typ=_____ dups=003 gl.account.gltran cpy=gltran.cpy rcs=_____ typ=_____ gl.account.gltran_ cpy=gltran.cpy rcs=_____ typ=_____ dups=003
#1. gencnvC2 ctl/datacpy52 <-- generate pipe|delimit jobs
======================
cpys-------->maps---------->pfp1-------------->pfp2----------->pfp3--------->
cobmap1 genpipe1 uvdata52 copy/edit execute
convert copybooks change copybooknames
to pipe|delimit jobs to datafilenames
via control-file
#2. cp pfp2/* pfp3/ <-- copy to pfp3/ before any edits
===============
#3. vi pfp3/... <-- modify for multi-record-type files (if any)
===========
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# gltran - uvcopy job to convert EBCDIC/ASCII data to delimited text file
opr='gltran - uvcopy code generated from copybook: gltran '
rop=j200000 #Run OPtions: increase instrn storage
was=a64000b64000c999000d64000e64000u64000v64000 #increase u&v if fields > 163
fili1=?d2asc/gl.account.gltran_,rcs=00080,typ=RSF
filo1=?d4pipe/gl.account.gltran_,rcs=00240,typ=LSTtd3
@run
opn all
loop get fili1,a0
skp> eof
# area a input, see 'get' instrn
# area b translated to ASCII, in case mainframe EBCDIC file (with packed?)
# area c data fields fixed 100 bytes apart in prep for var | delimit instrn
# area d output, see 'put' instrn at end
mvc b0(00080),a0 move input area a to area b
### tra b0(00080) uncomment for EBCDIC input (or option a1)
# --- <-- & insert instrns from uvdata51 if packed fields
# --- <-- insert R/T tests if redef records
mvc c0(8),b0(8) #1 glt-acntno
mvc c100(2),b8(2) #2 glt-type
mvc c200(30),b10(30) #3 glt-description
edta1 c300(12),b40(10),'-zzzzzzzzz9' #4 glt-amount
mvc c400(1),b50(1) #5 glt-status
mvc c500(4),b51(4) #6 glt-client
mvc c600(2),b55(2) #7 glt-province
mvc c700(12),b57(12) #8 glt-trace
mvc c800(11),b69(11) #9 glt-filler1
rep c0(00900),'|','!' #comment speedup if no '|'s in data
varb1 d0(00900),c0(100),0009,'|' #insert '|'s between fields & squeeze
trt d0(00900),$trtchr #ensure all ascii chars x'20'-x'7E'
clr c0(00900),' '
put1 put filo1,d0
skp loop
#
eof cls all
eoj
@pf2=genpipe1.sub
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#4. uvcopypfxgA pfp3 d2asc d4pipe <-- script to execute all jobs in pfp3
============================= - converting from d2asc/* to d4pipe/...
#5. vi d4pipe/gl.account.gltran_000001
==================================
- inspect same pipe|delimited file
00011100|T|Bank Fee|-1000| |1234|BJ|040428| | 00011100|T|EMJ Sony Tape Drive|-123141| |1234|BC|000871040428| | 00011100|T|Telephone|-5598| |1234|BC|000882040428| | 00011100|T|FEDEX|-1091| |1234|BC|000870040428| | 00011100|T|VISA payment|-400000| |1234|BC|000884040428| | 00011100|T|Staples office supplies|-12275| |1234|BC|000874040428| | 00011100|T|Datex office rent|-32100| |1234|BC|000885040428| |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
If desired you could copy your converted datafiles to $testdata1/data & execute some of the demo JCL/scripts supplied in $UV/mvstest1/testlibs1. On page '6B1' we setup testdata1 & testlibs1 (with empty subdirs). We only need to store datafiles in testdata1/data & GDG control file in testdata1/ctl.
BUT, to execute, we will need several subdirs in testlibs1 (jcls,cblx,etc). The COBOL & JCL conversions are documented extensively in https://www.uvsoftware.ca/jclcnv1demo.htm#Part_3 so we will take a shortcut here & simply copy from the supplied $UV/mvstest1/testlibs1.
#1. Login --> /home/userxx
#2. cdl --> /home/userxx/testlibs1
==============================
#3. cp -r $UV/mvstest1/testlibs1/* . (shortcut for data conversion demos)
================================
- copy all supplied COBOL & JCL libraries
- programs already compiled for Micro Focus COBOL
- JCL/scripts converted for Micro Focus COBOL
- you would need to convert for AIX COBOL
- see https://www.uvsoftware.ca/jclcnv1demo.htm#Part_3
#4. cdd --> /home/userxx/testdata1
#5. cp /home/userxx/cnvdata1/d2asc/* data
=====================================
- copy datafiles (converted above 6C1 - 6C7) to $RUNDATA/data/...
#6. cp $UV/mvstest1/testdata1/ctl/* ctl
==================================
- copy supplied GDG control file gdgctl51
#7. testdatainit
============
- good idea to init $RUNDATA/jobtmp before testing
- also reloads Indexed file version gdgctl51I.dat/.idx from gdgctl51
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
You could run a few of the demo JCL/scripts, see more details & console logs in https://www.uvsoftware.ca/jclcnv1demo.htm#Part_4.
#1. Login --> /home/userxx
#2. cdl --> /home/userxx/testlibs1
===============================
#3. l jcls <-- list JCL/scripts available
====== - previously converted in JCLcnv1demo.htm#Part_3
#3a. jar100.ksh - 1 step COBOL #3b. jar200.ksh - SORT + COBOL #3c. jgl100.ksh - GDG files with COBOL #3d. jgl200.ksh - GDG files + SORT + COBOL #3e. jgl230.ksh - SORT + COBOL + COBOL #3f. jgl320.ksh - IEBGENER + IDCAMS + IF/THEN/ELSE
#4. joblog jar100.ksh <-- execute a script using files converted
=================
140827:155237:jar100: Begin Job=jar100 140827:155237:jar100: /home/uvtest/testlibs1/jcls/jar100.ksh 140827:155237:jar100: Arguments: jar100.ksh 140827:155238:jar100: RUNDATE=20140827 140827:155238:jar100: RUNLIBS=/home/uvtest/testlibs1 140827:155238:jar100: RUNDATA=/home/uvtest/testdata1 140827:155238:jar100: JTMP=jobtmp/jar100 140827:155238:jar100: SYOT=sysout/jar100 140827:155238:jar100: LOGNAME=uvtest TESTPROD=T000 140827:155238:jar100: HOSTNAME=uvsoft4 140827:155238:jar100: ******** Begin Step S0010 car100 (#1) ******** 140827:155238:jar100: file: CUSTMAS=data/ar.customer.armaster fsize=8K 140827:155238:jar100: file: NALIST=data/ar.customer.nameadrs.list100 fsize=K 140827:155238:jar100: file: SYSOUT=sysout/jar100/jar100_s0010_sysout_140827_155238 fsize=K 140827:155238:jar100: Executing--> cobrun -F /home/uvtest/testlibs1/cblx/car100 140827:155238:JAR100: StepTimes: S0010 car100 Begun=15:52:38 End=15:52:38 Elapsed=00:00:00 140827:155238:JAR100: Job Times: Begun=15:52:38 NormalEnd=15:52:38 Elapsed=00:00:00 140827:155238:jar100: JobEnd=Normal, StepsExecuted=1, LastStep=S0010
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
On pages '6C1' - '6C8', we presented the "Illustrated Operating Instructions" using VU supplied test/demo datafiles & copybooks. We hope this gave you a good understanding of these procedures.
For your own datafile conversions, you might like a more "Concise" version of the operating procedures, which is presented on the following pages. We assume the following has been performed:
/home/userxx <-- your homedir :-----cnvdata1 <-- cnvdata1 superdir for DATA conversion : :-----cpys <-- copybooks <-- FTP from mainframe : :-----ctl - control files : :-----d0ebc <-- EBCDIC data files <-- FTP from mainframe : :-----d1ebc - EBCDIC data files : :-----d2asc - ASCII data files (converted by uvcopy uvdata51) : :-----d3ebc - convert back to EBCDIC by uvdata31 : :-----d4pipe - pipe delimited to load DB tables (by genpipe1) : :-----maps - copybooks with record layouts on right side : :-----pfx1 - uvcopy jobs to convert EBCDIC to ASCII (gen by uvdata51) : :-----pfx2 - with data filenames inserted (vs copybook names) : :-----pfx3 - copied here for modify/execute : :-----pfp1 - uvcopy jobs to create pipe delimited (gen by genpipe1) : :-----pf - misc uvcopy jobs : :-----sf - misc scripts : :-----sqlTC1 - scripts to Create database tables (gen from copybooks) : :-----sqlTL1 - scripts to Load database tables (gen from copybooks) : :-----tmp - temp files (keep working directory clean)
cpys-------->maps---------->pfx1-------------->pfx2----------->pfx3--------->
cobmap1 uvdata51 uvdata52 copy/edit execute
convert copybooks change copybooknames
to uvcopy jobs to datafilenames
Mainframe---->d0ebc---------->d1ebc------------->d2asc------->$RUNDATA/data
FTP rename GDG/lower EBCDIC->ASCII copy files to test dirs
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#0a. Login userxx --> /home/userxx
#0b. cdc --> alias cdc='cd $CNVDATA' --> /home/userxx/cnvdata1
#1a. cp $UV/mvstest1/cnvdata1/d0ebc/* d0ebc
======================================
- copy VU supplied test data files to your homedir/cnvdata1/d0ebc/
- OR FTP mainframe datafiles for your real conversion
#1b. cp $UV/mvstest1/cnvdata1/cpys/* cpys
====================================
- copy VU supplied copybooks to your homedir/cnvdata1/cpys/
- OR FTP mainframe copybooks for your real conversion
#2. cp d0ebc/* d1ebc/ <-- copy to d1ebc before rename (save original)
=================
#3a. renameGDG d1ebc <-- rename GDG files from mainframe convention to UV
=============== .G0001V00 -> _000001, .G0002V00 -> _000002, etc
#3b. renameL d1ebc <-- rename to lowercase
=============
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#4. uvcopy data2cpy51,fild1=d1ebc,filo2=ctl/datacpy51
=================================================
- create ctl/datacpy51 from all datafilenames in d1ebc/...
- reducing GDG filenames to 1 entry per GDG set
#4a. uvcopy data2cpy51 <-- same as above, filenames default as shown above
=================
#5. cp ctl/datacpy51 ctl/datacpy52
==============================
- copy/rename to prevent edited file overwrite if data2cpy51 rerun
#6. vi ctl/datacpy52 <-- edit/insert copybook names as shown below
================
IF copybooknames are embedded in datafilenames (at some sites), you can use 'data2cpy52' to extract copybooknames from datafilenames into the control-file. (Unique to some sites, but a great convenience).
'data2cpy52' would be run following 'data2cpy51' & it assumes that the copybookname is the last segment in the filename, following the last '.' & ended by a blank or the '_' indicating a GDG file.
#5. uvcopy data2cpy52,fili1=ctl/datacpy51,filo1=ctl/datacpy52
=========================================================
- automatic insert copybooknames from datafilenames
#5a. uvcopy data2cpy52 <-- same, files default as above
=================
#6. vi ctl/datacpy52 <-- inspect ctlfile with copybook names
================ - make corrections if auto code not perfect
ar.customer.armaster cpy=armaster.cpy rcs=_____ typ=_____ ar.sales.arsales cpy=arsales.cpy rcs=_____ typ=_____ citytax1 cpy=citytax1.cpy rcs=_____ typ=_____ citytax2 cpy=citytax2.cpy rcs=_____ typ=_____ gl.account.glmaster_ cpy=glmaster.cpy rcs=_____ typ=_____ dups=003 gl.account.gltran cpy=gltran.cpy rcs=_____ typ=_____ gl.account.gltran_ cpy=gltran.cpy rcs=_____ typ=_____ dups=003
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#7. gencnvC ctl/datacpy52 <-- generate data conversion jobs
=====================
- script 'gencnvC' runs 3 uvcopy jobs (cobmap1, uvdata51,& uvdata52)
- see script listed at https://uvsoftware.ca/datacnv1.htm#10E1
----- OR could run separately as follows -----
#7a. uvcopyx cobmap1 cpys maps uop=q0i7
==================================
- convert cobol copybooks to maps (record layouts)
#7b. uvcopyx uvdata51 maps pfx1 uop=q0i7
===================================
- generates uvcopy data conversion jobs from copybookmaps
- codes fili1=... & filo1=... & jobnames sames as copybooknames
#7c. cp $UV/IBM/skeleton2 pfx1
=========================
- copy uvcopy job skeleton for datafiles without copybooks
- OK for files that have no packed/binary fields
- could modify control file typ=LST (LineFeeds OK if no packed/binary)
#7d. uvcopy uvdata52,fili1=ctl/datacpy52,fild2=pfx1,fild3=pfx2,uop=q0i7
==================================================================
- copies pfx1/* to pfx2/... inserting datafilenames (vs copybooknames)
- also renames the jobs for the datafilenames (vs copybooknames
- since could have multiple datafiles for same copybook (esp. for GDG)
It may be necessary to make some manual changes to auto-generated jobs, to allow for any files with multiple record-types, for variable-length files,& corrections to for missing copybooks (job generated from skeleton2).
#8. cp pfx2/* pfx3 <-- copy auto-generated jobs before any editing
==============
#9. vi pfx3/... <-- make any manual changes required
===========
#10. uvcopypfxgA pfx3 d1ebc d2asc
============================
- execute all jobs in pfx3/...
- to convert all files from d1ebc/... to d2asc/...
- jobs for GDG files are repeated depending on generations present
- see console log below illustrating GDG file conversions
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Script 'gencnvC2' is similar to gencnvC (EBCDIC to ASCII conversions), but generates jobs to create pipe|delimited files. 'gencnvC' is a pre-requisite, since the input to the pipe|delimit jobs is the ASCII files from gencnvC. gencnvC2 uses the same control file as gencnvC.
#11. gencnvC2 ctl/datacpy52 <-- generate pipe|delimit jobs
======================
cpys-------->maps---------->pfp1-------------->pfp2----------->pfp3--------->
cobmap1 genpipe1 uvdata52 copy/edit execute
convert copybooks change copybooknames
to pipe|delimit jobs to datafilenames
via control-file
#12. cp pfp2/* pfp3/ <-- copy to pfp3/ before any edits
===============
#13. vi pfp3/... <-- modify for multi-record-type files (if any)
===========
#14. uvcopypfxgA pfp3 d2asc d4pipe <-- script to execute all jobs in pfp3
============================= - converting from d2asc/* to d4pipe/...
#15. vi d4pipe/gl.account.gltran_000001
==================================
- inspect same pipe|delimited file
00011100|T|Bank Fee|-1000| |1234|BJ|040428| | 00011100|T|EMJ Sony Tape Drive|-123141| |1234|BC|000871040428| | 00011100|T|Telephone|-5598| |1234|BC|000882040428| | 00011100|T|FEDEX|-1091| |1234|BC|000870040428| | 00011100|T|VISA payment|-400000| |1234|BC|000884040428| | 00011100|T|Staples office supplies|-12275| |1234|BC|000874040428| | 00011100|T|Datex office rent|-32100| |1234|BC|000885040428| |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 7A1. | bytes256 - test-file for translates & delimits |
| 7B1. | preparation for tests - make directories |
| - can make directory in your homedir with multiple subdirs | |
| - get test file from $UV/dat1/bytes256 | |
| - get copybook from $UV/mf/cpys/bytes256 & convert to cobmap (layout) |
| 7C1. | Op. Instrns. to test translate EBCDIC to ASCII & back to EBCDIC |
| 7D1. | RESULTS - d1ebc(EBCDIC), d2asc(ASCII), d3ebc(back to EBCDIC) |
| 7D2. | RESULTS - compare original EBCDIC file before translate to ASCII |
| & final EBCDIC file reconverted from ASCII |
| 7E1. | uvcopy jobs to translate EBCDIC to ASCII & ASCII to EBCDIC |
| - generated automatically from copybook |
| 7F1. | Op. Instrns. to convert EBCDIC Fixed to Delimited & back to Fixed |
| 7G1. | RESULTS - pipe|delimted file d4pipe/Ebytes, ugly for vi |
| - use listpipeA to list record broken into 80 byte segments | |
| & then listing delimited fields vertically | |
| 7G2. | RESULTS - uvhd hex before & after converting fixed to delimited & back |
| 7G3. | RESULTS - uvcmpFA1 comparing original fixed-field to reconverted fixed file |
| - hex displays with '*'s flagging differences |
| 7H1. | Op. Instrns. to test translate ASCII to EBCDIC & back to EBCDIC |
| 7I1. | RESULTS - d1ebc(EBCDIC), d2asc(ASCII), d3ebc(back to EBCDIC) |
| 7I2. | RESULTS - compare original EBCDIC file before translate to ASCII |
| & final EBCDIC file reconverted from ASCII |
| 7J1. | Op. Instrns. to convert EBCDIC Fixed to Delimited & back to Fixed |
| 7K1. | RESULTS - pipe|delimted file d4pipe/Abytes, ugly for vi |
| - use listpipeA to list record broken into 80 byte segments | |
| & then listing delimited fields vertically | |
| 7K2. | RESULTS - uvhd hex before & after converting fixed to delimited & back |
| 7K3. | RESULTS - uvcmpFA1 comparing original fixed-field to reconverted fixed file |
| - hex displays with '*'s flagging differences | |
| 7K4. | Notes re differences before/after delimiting ASCII files |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 7L1. | Checking Data File Conversions |
| checking data file conversions with 'uvhd' | |
| uvhd check ar.customer.armaster - EBCDIC file |
| 7L2. | uvhd check ar.customer.armaster - ASCII file |
| checking EBCDIC to ASCII conversion |
| 7L3. | compress/uncompress with 'tar' |
| 7L4. | compress/uncompress with 'zip' |
| 7L5. | Notes re: Micro Focus COBOL variable length records |
| 7L6. | checking data file conversions with 'uvhdcob' |
| uvhdcob check ar.customer.armaster - EBCDIC file |
| 7L7. | uvhdcob check ar.customer.armaster - ASCII file |
| 7L8. | uvhdcob check city.tax.payments - EBCDIC file |
| uvhdcob check city.tax.payments - ASCII file |
| 7M1. | ebc2asc.c - EBCDIC to ASCII translate table |
| 7M2. | asc2ebc.c - ASCII to EBCDIC translate table |
| 7M3. | ebc1141asc.c - ASCII to EBCDIC translate table for codepage 1141 |
| - option rop=t1 for uvcopy tre instruction |
| 7N1. | trantestEAE - test translate EBCDIC to ASCII & back to EBCDIC |
| - alternate way to test Vancouver Utilities EBCDIC/ASCII translates. | |
| - uvcopy job to translate EBCDIC to ASCII & back to EBCDIC. | |
| - no data file input, adds binary to create each character of 256 possible | |
| - translates each from EBCDIC to ASCII & back to EBCDIC. | |
| - shows results in hexadecimal & character | |
| 7N2. | trantestESE uvcopy job listing that creates above report |
| 7O1. | makebytes256 - uvcopy job to create 256 bytes with values x'00' - x'FF' |
| 7O3. | alternate way to create bytes256 by compiling neutral traanslate table |
| & extracting the 256 bytes with unix utility 'dd' | |
| 7O4. | trtneut.c - neutral translate table, C source code & uvhd hex representation |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We will use $UV/dat1/bytes256 to test EBCDIC/ASCII translates & delimit/undelimit. 'bytes256' is a file of 1 256 byte record containing all possible combinations of 8 bits from '00000000' to '11111111' or in hexadecimal from x'00' to x'FF'.
uvhd /home/uvadm/dat1/bytes256 r256
===================================
records=1 rsize=256 fsize=256
rec#=1 rsize=256 fptr=0 fsize=256 rcount=1
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ................................ !"#$%&'()*+,-./0123456789:;<=>?
0000000000000000111111111111111122222222222222223333333333333333
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
64 @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^abcdefghijklmnopqrstuvwxyz{|}~.
4444444444444444555555555555555566666666666666667777777777777777
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
128 ................................................................
88888888888888889999999999999999AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBB
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
192 ................................................................
CCCCCCCCCCCCCCCCDDDDDDDDDDDDDDDDEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFF
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
We will copy $UV/dat1/bytes256, renaming as Ebytes to test EBCDIC to ASCII translates and renaming as Abytes to test ASCII to EBCDIC translates.
We will use 'Ebytes' to translate EBCDIC to ASCII & back to EBCDIC, then compare to prove the re-translated EBCDIC characters are the same as the original EBCDIC. Note that EBCDIC characters are x'40' to x'F9' & the other bytes in x'00'-x'3F' will not all compare the same, but should not matter.
We will use 'Abytes' to translate ASCII to EBCDIC & back to ASCII, then compare to prove the re-translated ASCII characters are the same as the original EBCDIC. Note that ASCII characters are x'00' to x'7F' & the other bytes in x'80'-x'FF' will not all compare the same, but should not matter.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
You can see the results of my tests in $UV/mvstest1/cnvtran/... You can repeat my tests by setting up a subdir (cnvtran or whatever) in your homedir and proceeding as follows:
#1. Login userxx --> /home/userxx
#2. mkdir cnvtran <-- setup directory for tests
=============
#3. cd cnvtran <-- chnage into directory
==========
#4. cnvdatadirs <-- make subdirs required
===========
| Note |
|
/home/userxx <-- setup cnvtran withing your $HOME dir :-----cnvtran <-- cnvtran superdir for DATA conversion : :-----cpys - copybooks : :-----d1ebc - EBCDIC data files : :-----d2asc - ASCII data files (converted by uvcopy uvdata51) : :-----d3ebc - convert back to EBCDIC by uvdata31 : :-----d4pipe - pipe delimited to load DB tables (by genpipe1) : :-----d5fix - convert pipe|delimited back to fixed field (genfixd3) : :-----listpipe - pipe|delimited listed vertically with copybook fieldnames : :-----maps - copybooks with record layouts on right side : :-----pfx1 - uvcopy jobs to convert EBCDIC to ASCII (gen by uvdata51) : :-----pfy1 - uvcopy jobs to convert ASCII to EBCDIC (gen by uvdata31) : :-----pfp1 - uvcopy jobs to create pipe delimited (gen by genpipe1) : :-----pfq1 - uvcopy jobs to convert pipe|delimited to fixed field (genfixd3) : :-----tmp - temp files (keep working directory clean)
#5. Get supplied datafile 1 record of 256 bytes x'00' - x'FF'
#5a. cp $UV/dat1/bytes256 d1ebc/Ebytes <-- copy/rename to consider
================================= as an EBCDIC file
#5b. cp $UV/dat1/bytes256 d2asc/Abytes <-- copy/rename to consider
================================= as an ASCII file
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#6. Get supplied copybook for bytes256 - 16 fields of pic x(16).
#6a. cp cpys/bytes256 cpys/Ebytes <-- for EBCDIC file conversions
============================
#6b. cp cpys/bytes256 cpys/Abytes <-- for ASCII file conversions
============================
#7. uvcopy cobmap1,fili1=cpys/Abytes.cpy,filo1=maps/Abytes
======================================================
- convert copybook to cobmap (record layout)
to generate various uvcopy jobs to translate & delimit
- could omit this since it is included in following 'gencnvA' script
cobmap1 COBOL record layout 2014/10/22_15:57:54 pg# 0001
cpys/Abytes bytes256 RCSZ=00256 bgn end lth typ
* bytes256 - copybook to test translations ebcdic to ascii &
* - by owen townsend, uv software, oct22/2014
* copybook for datafiles bytes256, Ebytes,& Abytes
* - bytes256 datafile is 256 bytes total from x'00' to x'FF'
* - this copybook will describe as 16 fields of 16 bytes eac
* use script 'gencnvA' to generate uvcopy jobs to convert:
* - ebcdic to ascii (pfx1/...), ascii to ebcdic (pfy1/...)
* - fixed-field to delimited (pfp1), delimited to fixed-fiel
01 bytes256.
10 b00-0f pic x(16). 0000 0015 016
10 b10-1f pic x(16). 0016 0031 016
10 b20-2f pic x(16). 0032 0047 016
10 b30-3f pic x(16). 0048 0063 016
10 b40-4f pic x(16). 0064 0079 016
10 b50-5f pic x(16). 0080 0095 016
10 b60-6f pic x(16). 0096 0111 016
10 b70-7f pic x(16). 0112 0127 016
10 b80-8f pic x(16). 0128 0143 016
10 b90-9f pic x(16). 0144 0159 016
10 ba0-af pic x(16). 0160 0175 016
10 bb0-bf pic x(16). 0176 0191 016
10 bc0-cf pic x(16). 0192 0207 016
10 bd0-df pic x(16). 0208 0223 016
10 be0-ef pic x(16). 0224 0239 016
10 bf0-ff pic x(16). 0240 0255 016
******** end copybook bytes256 ****************
*RCSZ=00256
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We will generate jobs to translate EBCDIC to ASCII & back to EBCDIC, execute, & prove the re-converted EBCDIC file same as original EBCDIC file. We will also generate pipe delimit/undelimit jobs, execute them, & prove the EBCDIC characters x'40' - x'FF' are the same.
| Note |
|
#1. gencnvA Ebytes <-- generates all conversion jobs
============== pfx1,pfy1,pfq1,pfp1
- see #comments doc in $UV/sf/IBM/gencnvA
#2. uvcopy pfx1/Ebytes <-- execute EBCDIC to ASCII, d1ebc --> d2asc
==================
#3. uvcopy pfy1/Ebytes <-- execute ASCII to EBCDIC, d2asc --> d3ebc
==================
#4. uvcmpFE1 d1ebc/Ebytes d3ebc/Ebytes r256
=======================================
- compare original EBCDIC file to reconverted file to see any diff
- x'40' - x'FF' (EBCDIC characters) are the same
- x'00' - x'3F' does have differences, but should not matter
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#5. use uvhd to display results in vertical hexadecimal
#5a. uvhd d1ebc/Ebytes r256a <-- note option 'a' show chars in ASCII
======================= - zones & digits remain EBCDIC codes
--> p1 <-- capture in text file tmp/Ebytes_date/timeP original EBCDIC
#5b. uvhd d2asc/Ebytes r256 <-- show converted to ASCII in hexadecimal
=======================
--> p1 <-- capture in text file tmp/Ebytes_date/timeP ASCII version
#5c. uvhd d3ebc/Ebytes r256a <-- note option 'a' show chars in ASCII
======================= - zones & digits remain EBCDIC codes
--> p1 <-- capture in text file tmp/Ebytes_date/timeP reconverted file
#5d. vi tmp/* <-- investigate uvhd hex displays
======== - satisfy yourself reconversion same as original
#6. OR see uvhd hex-rep files renamed & saved in subdir uvhd/...
#6a. vi uvhd/uvhd_Ebytes_d1ebc
=========================
#6b. vi uvhd/uvhd_Ebytes_d2asc
=========================
#6c. vi uvhd/uvhd_Ebytes_d3ebc
=========================
#7. uvcmpFE1 d1ebc/Ebytes d3ebc/Ebytes r256
=======================================
- compare original EBCDIC file to reconverted file to see any diff
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd d1ebc/Ebytes r256a <-- option 'a' character line ASCII so you can read
======================= - zones & digits lines are EBCDIC codes
mvstest1/cnvtran/uvhd/uvhd_Ebytes_d1ebc
rec#=1 rsize=256 fptr=0 fsize=256 rcount=1
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ................................................................
0000000000000000111111111111111122222222222222223333333333333333
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
64 ABCDEFGHI..<(+|&QRSTUVWXY!$*);^-/.cdefghi.,%_>?p.rs.vwxy`:#@'="
4444444444444444555555555555555566666666666666667777777777777777
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
128 .abcdefghi.......jklmnopqr.......~stuvwxyz...[...............]..
88888888888888889999999999999999AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBB
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
192 {ABCDEFGHI......}JKLMNOPQR......\.STUVWXYZ......0123456789......
CCCCCCCCCCCCCCCCDDDDDDDDDDDDDDDDEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFF
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
uvhd d2asc/Ebytes r256 <-- all 3 line groups ASCII (character/zones/digits)
====================== mvstest1/cnvtran/uvhd/uvhd_Ebytes_d2asc
rec#=1 rsize=256 fptr=0 fsize=256 rcount=1
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ................................ !"#$...()*+,...01.3456.89:;..>.
0000000700000000111111011111111122222011222220003313333033331131
0123496F89ABCDEF0123458789ABCDEF01234A7B89ABC5670163456489AB45EA
64 ABCDEFGHI..<(+|&QRSTUVWXY!$*);^-/.cdefghi.,%_>?p.rs.vwxy`:#@'="
2444444444A23227255555555522223522C6666666A225337C77C77776324232
01234567892EC8BC612345678914A9BEDF234567896C5FEF0923867890A307D2
128 .abcdefghi.......jklmnopqr.......~stuvwxyz...[...............]..
86666666668888889666666777999999A777777777AAA5AABBBBBBBBBBBBB5BB
0123456789ABCDEF0ABCDEF012ABCDEF0E3456789AABCBEF0123456789ABCDEF
192 {ABCDEFGHI......}JKLMNOPQR......\.STUVWXYZ......0123456789......
7444444444CCCCCC7444444555DDDDDD5E55555555EEEEEE3333333333FFFFFF
B123456789ABCDEFDABCDEF012ABCDEFC13456789AABCDEF0123456789ABCDEF
uvhd d3ebc/Ebytes r256a <-- option 'a' character line ASCII so you can read
======================= mvstest1/cnvtran/uvhd/uvhd_Ebytes_d3ebc
rec#=1 rsize=256 fptr=0 fsize=256 rcount=1
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ................................ !"#$...()*+,...01.3456.89:;..>.
000030201020000011113312113211114577522245546222FF3FFFF3FF753363
012375E7655BCDEF0123CD6689F7CDEF0AFBB567DDCEBDEF0123456789AECDEF
64 ABCDEFGHI..<(+|&QRSTUVWXY!$*);^-/.cdefghiw,%_>?p.rs.vwxy`:#@'="
4CCCCCCCCC4444445DDEEEEEEE5555556668888888A66666979A7AAAA7777777
0123456789ABCDEF0892345678ABCDEF01234567896BCDEF7192456789ABCDEF
128 .abcdefghi.......jklmnopqr.......~stuvwxyz...[...............]..
88888888888888889999999999999999AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBB
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
192 {ABCDEFGHI......}JKLMNOPQR......\.STUVWXYZ......0123456789......
CCCCCCCCCCCCCCCCDDDDDDDDDDDDDDDDEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFF
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
4. uvcmpFE1 d1ebc/Ebytes d3ebc/Ebytes r256
======================================= rptcmp/Ebytes_d1ebc_d3ebc
uvcmp1 - compare 2 files, print mismatched records, '*' flag diffs
2014/12/24_20:19:46 uop=q1p30r256s6t1000000u3x2y0q1y1r256u0
recsize reccount file-size typ Report=rptcmp/Ebytes
1: 256 1 256 RSF File1=d1ebc/Ebytes
2: 256 1 256 RSF File2=d3ebc/Ebytes
1 2 3 4 5 6
f#record#byte# 0123456789012345678901234567890123456789012345678901234567890123
===============================================================================
1 1 0 ................................ !"#$...()*+,...01.3456.89:;..>.
0000000000000000111111111111111122222222222222223333333333333333
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
* * *** ** * ** ***** ***** ** **** **** *
64 ABCDEFGHI..<(+|&QRSTUVWXY!$*);^-/.cdefghi.,%_>?p.rs.vwxy`:#@'="
4444444444444444555555555555555566666666666666667777777777777777
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
********* ********* ******** * ** ****
128 .abcdefghi.......jklmnopqr.......~stuvwxyz...[...............]..
88888888888888889999999999999999AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBB
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
192 {ABCDEFGHI......}JKLMNOPQR......\.STUVWXYZ......0123456789......
CCCCCCCCCCCCCCCCDDDDDDDDDDDDDDDDEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFF
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
2 1 ................................ !"#$...()*+,...01.3456.89:;..>.
000030201020000011113312113211114577522245546222FF3FFFF3FF753363
012375E7655BCDEF0123CD6689F7CDEF0AFBB567DDCEBDEF0123456789AECDEF
* * *** ** * ** ***** ***** ** **** **** *
64 ABCDEFGHI..<(+|&QRSTUVWXY!$*);^-/.cdefghiw,%_>?p.rs.vwxy`:#@'="
4CCCCCCCCC4444445DDEEEEEEE5555556668888888A66666979A7AAAA7777777
0123456789ABCDEF0892345678ABCDEF01234567896BCDEF7192456789ABCDEF
********* ********* ******** * ** ****
128 .abcdefghi.......jklmnopqr.......~stuvwxyz...[...............]..
88888888888888889999999999999999AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBB
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
192 {ABCDEFGHI......}JKLMNOPQR......\.STUVWXYZ......0123456789......
CCCCCCCCCCCCCCCCDDDDDDDDDDDDDDDDEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFF
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
==================== EOF or StopPrint/StopRead count reached ============== F1Count=1, F2Count=1, StopPrint=6, StopRead=1000000 F1Reads=1, MisMatches=1, MisMatsPrinted=1, Recsize=256
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# uvcopy job to translate EBCDIC to ASCII, preserve packed, fix zoned signs
opr='jobname=JOBNAME - pfx2 name=datafilename'
opr='copybook=Ebytes - pfx1 name=copybookname'
uop=q0,was=a64000b64000
fili1=?d1ebc/Ebytes,rcs=00256,typ=RSF
filo1=?d2asc/Ebytes,rcs=00256,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00256),a0 move rec to outarea before field prcsng
tra b0(00256) translate entire outarea to ASCII
# --- <-- insert R/T tests here for redefined records
put1 put filo1,b0
skp loop
eof cls all
eoj
# Ebytes - uvcopy job, ASCII to EBCDIC preserve packed, fix signs
opr='Ebytes - uvcopy code generated from copybook: Ebytes'
uop=q0
was=a33000b33000
fili1=?d2asc/Ebytes,rcs=00256,typ=RSF
filo1=?d3ebc/Ebytes,rcs=00256,typ=RSF
@run
opn all
loop get fili1,a0
skp> eof
mvc b0(00256),a0 move rec to outarea before field prcsng
tre b0(00256) translate entire outarea to EBCDIC
# --- <-- insert R/T tests here for redefined records
put1 put filo1,b0
skp loop
eof cls all
eoj
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Using input d2asc/Ebytes created above in translate tests
| Note |
|
#1. uvcopy pfp1/Ebytes,fili1=d2asc/Ebytes,filo1=d2pipe/Ebytes
=========================================================
- convert ASCII fixed-field (origianlly EBCDIC) to ASCII pipe|delimited
#1a. uvcopy pfp1/Ebytes <-- same, files default as above
==================
#2. vi d4pipe/Ebytes <-- inspect pipe|delimited file with vi
================ - ugly, try listpipe below
#3. listpipeA Ebytes <-- create vertical listing of pipe|delim fields
================
#3a. vi listpipe/Ebytes <-- view listpipe results
==================
#4. uvcopy pfq1/Ebytes,fili1=d2pipe/Ebytes,filo1=d5fix/Ebytes
=========================================================
- convert ASCII pipe|delimited back to fixed-field ASCII
#4a. uvcopy pfq1/Ebytes <-- same, files default as above
==================
#5. uvhd d5fix/Ebytes r256 <-- inspect results re-converted back to fixed-field
======================
#6. uvcmpFA1 d2asc/Ebytes d5fix/Ebytes r256
=======================================
- compare original fixed-field to reconverted fixed-field from pipe|delimited
#7. vi rptcmp/Ebytes <-- view compare results
================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#2. vi d4pipe/Ebytes <-- inspect pipe|delimited file with vi
================ - not good, try listpipe below
| Note |
|
#3. listpipeA Ebytes <-- create vertical listing of pipe|delim fields
================
#3a. vi listpipe/Ebytes <-- view listpipe results
==================
listpipe1 - list '|' delimited files (from genpipe1) with copybook fieldnames DataFile=d4pipe/Ebytes, RecSize=00256, RecCount=1 CopyBook=maps/Ebytes, DateTime=2014/12/24_20:16:59 Seq# FieldName OffSet COBOL picture DataFieldContents DataFile = d4pipe/Ebytes, Record# = 1 of 1, Fields = 16
| | !"#$ ()*+,|01 3456 89:; >| ABCDEFGHI .<(+!|&QRSTUVWXY!$*);^|-/ cd
efghi ,%_>?|p rs vwxy`:#@'="| abcdefghi| jklmnopqr| ~stuvwxyz [| ]
|{ABCDEFGHI|}JKLMNOPQR|\ STUVWXYZ|0123456789|
001 b00-0f 000 x(16).
002 b10-1f 016 x(16).
003 b20-2f 032 x(16). !"#$ ()*+,
004 b30-3f 048 x(16). 01 3456 89:; >
005 b40-4f 064 x(16). ABCDEFGHI .<(+!
006 b50-5f 080 x(16). &QRSTUVWXY!$*);^
007 b60-6f 096 x(16). -/ cdefghi ,%_>?
008 b70-7f 112 x(16). p rs vwxy`:#@'="
009 b80-8f 128 x(16). abcdefghi
010 b90-9f 144 x(16). jklmnopqr
011 ba0-af 160 x(16). ~stuvwxyz [
012 bb0-bf 176 x(16). ]
013 bc0-cf 192 x(16). {ABCDEFGHI
014 bd0-df 208 x(16). }JKLMNOPQR
015 be0-ef 224 x(16). \ STUVWXYZ
016 bf0-ff 240 x(16). 0123456789
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd d2asc/Ebytes r256
====================== mvstest1/cnvtran/uvhd/uvhd_Ebytes_d2asc
rec#=1 rsize=256 fptr=0 fsize=256 rcount=1
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ................................ !"#$...()*+,...01.3456.89:;..>.
0000000700000000111111011111111122222011222220003313333033331131
0123496F89ABCDEF0123458789ABCDEF01234A7B89ABC5670163456489AB45EA
64 ABCDEFGHI..<(+|&QRSTUVWXY!$*);^-/.cdefghi.,%_>?p.rs.vwxy`:#@'="
2444444444A23227255555555522223522C6666666A225337C77C77776324232
01234567892EC8BC612345678914A9BEDF234567896C5FEF0923867890A307D2
128 .abcdefghi.......jklmnopqr.......~stuvwxyz...[...............]..
86666666668888889666666777999999A777777777AAA5AABBBBBBBBBBBBB5BB
0123456789ABCDEF0ABCDEF012ABCDEF0E3456789AABCBEF0123456789ABCDEF
192 {ABCDEFGHI......}JKLMNOPQR......\.STUVWXYZ......0123456789......
7444444444CCCCCC7444444555DDDDDD5E55555555EEEEEE3333333333FFFFFF
B123456789ABCDEFDABCDEF012ABCDEFC13456789AABCDEF0123456789ABCDEF
uvhd d5fix/Ebytes r256
====================== mvstest1/cnvtran/uvhd/uvhd_Ebytes_d5fix
rec#=1 rsize=256 fptr=0 fsize=256 rcount=1
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 !"#$ ()*+, 01 3456 89:; >
2222222222222222222222222222222222222222222222223323333233332232
000000000000000000000000000000000123400089ABC0000103456089AB00E0
64 ABCDEFGHI .<(+!&QRSTUVWXY!$*);^-/ cdefghi ,%_>?p rs vwxy`:#@'="
2444444444223222255555555522223522266666662225337277277776324232
01234567890EC8B1612345678914A9BEDF034567890C5FEF0023067890A307D2
128 abcdefghi jklmnopqr ~stuvwxyz [ ]
2666666666222222266666677722222227777777772225222222222222222522
01234567890000000ABCDEF0120000000E3456789A000B000000000000000D00
192 {ABCDEFGHI }JKLMNOPQR \ STUVWXYZ 0123456789
7444444444222222744444455522222252555555552222223333333333222222
B123456789000000DABCDEF012000000C03456789A0000000123456789000000
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvcmpFA1 d2asc/Ebytes d5fix/Ebytes r256
======================================= rptcmp/Ebytes_d2asc_d5fix
uvcmp1 - compare 2 files, print mismatched records, '*' flag diffs
2014/12/24_20:20:31 uop=q1p30r256s6t1000000u3x2y0q1r256u0
recsize reccount file-size typ Report=rptcmp/Ebytes
1: 256 1 256 RSF File1=d2asc/Ebytes
2: 256 1 256 RSF File2=d5fix/Ebytes
1 2 3 4 5 6
f#record#byte# 0123456789012345678901234567890123456789012345678901234567890123
===============================================================================
1 1 0 ................................ !"#$...()*+,...01.3456.89:;..>.
0000000700000000111111011111111122222011222220003313333033331131
0123496F89ABCDEF0123458789ABCDEF01234A7B89ABC5670163456489AB45EA
******************************** *** *** * * ** *
64 ABCDEFGHI..<(+|&QRSTUVWXY!$*);^-/.cdefghi.,%_>?p.rs.vwxy`:#@'="
2444444444A23227255555555522223522C6666666A225337C77C77776324232
01234567892EC8BC612345678914A9BEDF234567896C5FEF0923867890A307D2
* * * * * *
128 .abcdefghi.......jklmnopqr.......~stuvwxyz...[...............]..
86666666668888889666666777999999A777777777AAA5AABBBBBBBBBBBBB5BB
0123456789ABCDEF0ABCDEF012ABCDEF0E3456789AABCBEF0123456789ABCDEF
* ******* ******* *** *************** **
192 {ABCDEFGHI......}JKLMNOPQR......\.STUVWXYZ......0123456789......
7444444444CCCCCC7444444555DDDDDD5E55555555EEEEEE3333333333FFFFFF
B123456789ABCDEFDABCDEF012ABCDEFC13456789AABCDEF0123456789ABCDEF
****** ****** * ****** ******
2 1 !"#$ ()*+, 01 3456 89:; >
2222222222222222222222222222222222222222222222223323333233332232
000000000000000000000000000000000123400089ABC0000103456089AB00E0
******************************** *** *** * * ** *
64 ABCDEFGHI .<(+!&QRSTUVWXY!$*);^-/ cdefghi ,%_>?p rs vwxy`:#@'="
2444444444223222255555555522223522266666662225337277277776324232
01234567890EC8B1612345678914A9BEDF034567890C5FEF0023067890A307D2
* * * * * *
128 abcdefghi jklmnopqr ~stuvwxyz [ ]
2666666666222222266666677722222227777777772225222222222222222522
01234567890000000ABCDEF0120000000E3456789A000B000000000000000D00
* ******* ******* *** *************** **
192 {ABCDEFGHI }JKLMNOPQR \ STUVWXYZ 0123456789
7444444444222222744444455522222252555555552222223333333333222222
B123456789000000DABCDEF012000000C03456789A0000000123456789000000
****** ****** * ****** ******
==================== EOF or StopPrint/StopRead count reached ============== F1Count=1, F2Count=1, StopPrint=6, StopRead=1000000 F1Reads=1, MisMatches=1, MisMatsPrinted=1, Recsize=256
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We will generate jobs to translate ASCII to EBCDIC & back to ASCII, execute, & prove the re-converted ASCII file same as original ASCII file. We will also generate pipe delimit/undelimit jobs, execute them, & prove the ASCII characters x'20' - x'7E' are the same.
| Note |
|
#1. gencnvA Abytes <-- generates all conversion jobs
============== pfx1,pfy1,pfq1,pfp1
- see #comments doc in $UV/sf/IBM/gencnvA
#2. uvcopy pfy1/Abytes <-- execute ASCII to EBCDIC, d2asc --> d3ebc
==================
#3. uvcopy pfx1/Abytes,fili1=d3ebc/Abytes,filo1=d4asc/Abytes
========================================================
- execute EBCDIC to ASCII, d3ebc --> d4asc
- must specify fili1=d3ebc/... & filo1=d4asc/..., since
defaults are fili1=d1ebc/... & filo1=d2asc/...
#4. uvcmpFA1 d2asc/Abytes d4asc/Abytes r256
=======================================
- compare original EBCDIC file to reconverted file to see any diff
- x'20' - x'7E' (EBCDIC characters) are the same
- x'00' - x'1F' & x'7F' - x'FF' have differences, but should not matter
- could modify translate tables to make more similar if required
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#5. use uvhd to display results in vertical hexadecimal
#5a. uvhd d2asc/Abytes r256 <-- show ASCII input file in hexadecimal
=======================
--> p1 <-- capture in text file tmp/Abytes_date/timeP ASCII input
#5b. uvhd d3ebc/Abytes r256a <-- show EBCDIC output file in hexadecimal
======================= - note option 'a' show chars in ASCII
- zones & digits remain EBCDIC codes
--> p1 <-- capture in text file tmp/Abytes_date/timeP EBCDIC output
#5c. uvhd d4asc/Abytes r256 <-- show reconverted ASCII file in hexadecimal
======================
--> p1 <-- capture in text file tmp/Abytes_date/timeP reconverted file
#5d. vi tmp/* <-- investigate uvhd hex displays
======== - satisfy yourself reconversion same as original
- at least in the ASCII character range x'20' - x'7E'
#6. OR see uvhd hex-rep files renamed & saved in subdir uvhd/...
#6a. vi uvhd/uvhd_Abytes_d2asc
=========================
#6b. vi uvhd/uvhd_Abytes_d3ebc
=========================
#6c. vi uvhd/uvhd_Abytes_d4asc
=========================
#7. uvcmpFA1 d2asc/Abytes d4asc/Abytes r256
=======================================
- compare original ASCII to after conversion to EBCDIC & back to ASCII
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd d2asc/Abytes r256 mvstest1/cnvtran/uvhd/uvhd_Abytes_d2asc
======================
rec#=1 rsize=256 fptr=0 fsize=256 rcount=1
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ................................ !"#$%&'()*+,-./0123456789:;<=>?
0000000000000000111111111111111122222222222222223333333333333333
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
64 @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^abcdefghijklmnopqrstuvwxyz{|}~.
4444444444444444555555555555555566666666666666667777777777777777
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
128 ................................................................
88888888888888889999999999999999AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBB
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
192 ................................................................
CCCCCCCCCCCCCCCCDDDDDDDDDDDDDDDDEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFF
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
uvhd d3ebc/Abytes r256a mvstest1/cnvtran/uvhd/uvhd_Abytes_d3ebc
=======================
rec#=1 rsize=256 fptr=0 fsize=256 rcount=1
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ................................ !"#$%&'()*+,-./0123456789:;<=>?
000032221020000011113332113211114577565745546646FFFFFFFFFF754766
01237DEF655BCDEF0123CD2689F7CDEF0AFBBC0DDDCEB0B10123456789AECEEF
64 @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^abcdefghijklmnopqrstuvwxyz{|}~.
7CCCCCCCCCDDDDDDDDDEEEEEEEEAEB567888888888999999999AAAAAAAAC4DA0
C12345678912345678923456789D0DFD9123456789123456789234567890F017
128 .abcdefghi.......jklmnopqr.......~.tuvwxyz...[...............]..
88888888888888889999999999999999AA4AAAAAAAAAAAAABBBBBBBBBBBBBBBB
0123456789ABCDEF0123456789ABCDEF01A3456789ABCDEF0123456789ABCDEF
192 {A.CDEFG........}JKLMNOPQR......\.STUVWXYZ......0123456789......
CC6CCCCC77CCCCCCDDDDDDDDDDDDDDDDEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFF
0123456741ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
uvhd d4asc/Abytes r256 mvstest1/cnvtran/uvhd/uvhd_Abytes_d4asc
======================
rec#=1 rsize=256 fptr=0 fsize=256 rcount=1
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ................................ !"#$%&'()*+,-./0123456789:;<=>?
0000000000000000111111111111111122222222222222223333333333333333
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
64 @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^abcdefghijklmnopqrstuvwxyz{|}~.
4444444444444444555555555555555566666666666666667777777777777777
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
128 ................................................................
88888888888888889999999999999999AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBB
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
192 ................................................................
CCCCCCCCCCCCCCCCDDDDDDDDDDDDDDDDEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFF
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#7. uvcmpFA1 d2asc/Abytes d4asc/Abytes r256
=======================================
uvcmp1 - compare 2 files, print mismatched records, '*' flag diffs
2014/12/24_20:24:34 uop=q1p30r256s6t1000000u3x2y0q1r256u0
recsize reccount file-size typ Report=rptcmp/Abytes
1: 256 1 256 RSF File1=d2asc/Abytes
2: 256 1 256 RSF File2=d4asc/Abytes
1 2 3 4 5 6
f#record#byte# 0123456789012345678901234567890123456789012345678901234567890123
===============================================================================
1 1 0 ................................ !"#$%&'()*+,-./0123456789:;<=>?
0000000000000000111111111111111122222222222222223333333333333333
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
64 @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^abcdefghijklmnopqrstuvwxyz{|}~.
4444444444444444555555555555555566666666666666667777777777777777
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
128 ................................................................
88888888888888889999999999999999AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBB
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
********* ********* * ******* * *
192 ................................................................
CCCCCCCCCCCCCCCCDDDDDDDDDDDDDDDDEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFF
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
** ***** ********** * ******** **********
2 1 ................................ !"#$%&'()*+,-./0123456789:;<=>?
0000000000000000111111111111111122222222222222223333333333333333
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
64 @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^abcdefghijklmnopqrstuvwxyz{|}~.
4444444444444444555555555555555566666666666666667777777777777777
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
128 .abcdefghi.......jklmnopqr.......~.tuvwxyz...[...............]..
86666666668888889666666777999999A7A7777777AAA5AABBBBBBBBBBBBB5BB
0123456789ABCDEF0ABCDEF012ABCDEF0E2456789AABCBEF0123456789ABCDEF
********* ********* * ******* * *
192 {A.CDEFG........}JKLMNOPQR......\.STUVWXYZ......0123456789......
74C44444CCCCCCCC7444444555DDDDDD5E55555555EEEEEE3333333333FFFFFF
B123456789ABCDEFDABCDEF012ABCDEFC13456789AABCDEF0123456789ABCDEF
** ***** ********** * ******** **********
==================== EOF or StopPrint/StopRead count reached ============== F1Count=1, F2Count=1, StopPrint=6, StopRead=1000000 F1Reads=1, MisMatches=1, MisMatsPrinted=1, Recsize=256
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Using input d2asc/Abytes created above in translate tests
| Note |
|
#1. uvcopy pfp1/Abytes,fili1=d2asc/Abytes,filo1=d2pipe/Abytes
=========================================================
- convert ASCII fixed-field to ASCII pipe|delimited
#1a. uvcopy pfp1/Abytes <-- same, files default as above
==================
#2. uvcopy pfq1/Abytes,fili1=d4pipe/Abytes,filo1=d5fix/Abytes
=========================================================
- convert ASCII pipe|delimited back to fixed-field ASCII
#2a. uvcopy pfq1/Abytes <-- same, files default as above
==================
#3. listpipeA Abytes <-- create vertical listing of pipe|delim fields
================
#3a. vi listpipe/Abytes <-- view results
==================
#4. uvcmpFA1 d2asc/Abytes d5fix/Abytes r256
=======================================
- compare original fixed-field to reconverted fixed-field from pipe|delimited
#4a. vi rptcmp/Abytes <-- view results
================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#2. vi d4pipe/Abytes <-- inspect pipe|delimited file with vi
================ - not good, try listpipe below
| Note |
|
#3. listpipeA Abytes <-- create vertical listing of pipe|delim fields
================
#3a. vi listpipe/Abytes <-- view listpipe results
==================
listpipe1 - list '|' delimited files (from genpipe1) with copybook fieldnames DataFile=d4pipe/Abytes, RecSize=00256, RecCount=1 CopyBook=maps/Abytes, DateTime=2014/12/24_20:27:16 Seq# FieldName OffSet COBOL picture DataFieldContents DataFile = d4pipe/Abytes, Record# = 1 of 1, Fields = 16
| | !"#$%&'()*+,-./|0123456789:;<=>?|@ABCDEFGHIJKLMNO|PQRSTUVWXYZ[\]^_|`abcdefg
hijklmno|pqrstuvwxyz{!}~ | | | | | | | | |
001 b00-0f 000 x(16).
002 b10-1f 016 x(16).
003 b20-2f 032 x(16). !"#$%&'()*+,-./
004 b30-3f 048 x(16). 0123456789:;<=>?
005 b40-4f 064 x(16). @ABCDEFGHIJKLMNO
006 b50-5f 080 x(16). PQRSTUVWXYZ[\]^_
007 b60-6f 096 x(16). `abcdefghijklmno
008 b70-7f 112 x(16). pqrstuvwxyz{!}~
009 b80-8f 128 x(16).
010 b90-9f 144 x(16).
011 ba0-af 160 x(16).
012 bb0-bf 176 x(16).
013 bc0-cf 192 x(16).
014 bd0-df 208 x(16).
015 be0-ef 224 x(16).
016 bf0-ff 240 x(16).
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd d2asc/Abytes r256 mvstest1/cnvtran/uvhd_Abytes_d2asc
======================
rec#=1 rsize=256 fptr=0 fsize=256 rcount=1
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ................................ !"#$%&'()*+,-./0123456789:;<=>?
0000000000000000111111111111111122222222222222223333333333333333
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
64 @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^abcdefghijklmnopqrstuvwxyz{|}~.
4444444444444444555555555555555566666666666666667777777777777777
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
128 ................................................................
88888888888888889999999999999999AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBB
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
192 ................................................................
CCCCCCCCCCCCCCCCDDDDDDDDDDDDDDDDEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFF
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
uvhd d5fix/Abytes r256 mvstest1/cnvtran/uvhd_Abytes_d5fix
======================
rec#=1 rsize=256 fptr=0 fsize=256 rcount=1
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 !"#$%&'()*+,-./0123456789:;<=>?
2222222222222222222222222222222222222222222222223333333333333333
000000000000000000000000000000000123456789ABCDEF0123456789ABCDEF
64 @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^abcdefghijklmnopqrstuvwxyz{!}~
4444444444444444555555555555555566666666666666667777777777772772
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789AB1DE0
128
2222222222222222222222222222222222222222222222222222222222222222
0000000000000000000000000000000000000000000000000000000000000000
192
2222222222222222222222222222222222222222222222222222222222222222
0000000000000000000000000000000000000000000000000000000000000000
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvcmpFA1 d2asc/Ebytes d5fix/Ebytes r256 =======================================
uvcmp1 - compare 2 files, print mismatched records, '*' flag diffs
2014/12/24_20:27:40 uop=q1p30r256s6t1000000u3x2y0q1r256u0
recsize reccount file-size typ Report=rptcmp/Abytes
1: 256 1 256 RSF File1=d2asc/Abytes
2: 256 1 256 RSF File2=d5fix/Abytes
1 2 3 4 5 6
f#record#byte# 0123456789012345678901234567890123456789012345678901234567890123
===============================================================================
1 1 0 ................................ !"#$%&'()*+,-./0123456789:;<=>?
0000000000000000111111111111111122222222222222223333333333333333
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
********************************
64 @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^abcdefghijklmnopqrstuvwxyz{|}~.
4444444444444444555555555555555566666666666666667777777777777777
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
* *
128 ................................................................
88888888888888889999999999999999AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBB
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
****************************************************************
192 ................................................................
CCCCCCCCCCCCCCCCDDDDDDDDDDDDDDDDEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFF
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
****************************************************************
2 1 !"#$%&'()*+,-./0123456789:;<=>?
2222222222222222222222222222222222222222222222223333333333333333
000000000000000000000000000000000123456789ABCDEF0123456789ABCDEF
********************************
64 @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^abcdefghijklmnopqrstuvwxyz{!}~
4444444444444444555555555555555566666666666666667777777777772772
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789AB1DE0
* *
128
2222222222222222222222222222222222222222222222222222222222222222
0000000000000000000000000000000000000000000000000000000000000000
****************************************************************
192
2222222222222222222222222222222222222222222222222222222222222222
0000000000000000000000000000000000000000000000000000000000000000
****************************************************************
==================== EOF or StopPrint/StopRead count reached ============== F1Count=1, F2Count=1, StopPrint=6, StopRead=1000000 F1Reads=1, MisMatches=1, MisMatsPrinted=1, Recsize=256
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The above uvcmpFA1 comparison of ASCII files before & after pipe|delimiting shows 2 differences.
In the before record x'7C' is a '|', but in the after record we see a '!'.
This is because it would be a problem if we converted a '|' in existing data to a '|' in the output file - it would shift following fields when attempting to extract data from the pipe|delimited file.
The solution is to convert any '|'s in existing data to '!'s. This is the default, but you can specify arg2=X when you create the delimited file. For example, to change the default '!' to '@' you would run the delimt job on page '7J1' as follows:
#1. uvcopy pfp1/Abytes,arg2=@
=========================
The other difference is insignificant, the x'7F' DELete character is converted to space x'20'.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
First we will check data file conversions with 'uvhd' (vertical hexadecimal dumps of data records) & later we will check with 'uvhdcob' which shows the COBOL copybook names beside the data field contents.
You can also convert the files back to EBCDIC & compare to the original FTP. See http://uvsoftware.ca/datacnv1.htm#Part_9. Utility 'uvdat55' (page 9A2) copies conversion jobs from pfx4/* to pfx5/... reversing the translate. Then use utility 'uvcmp1' to compare the 2 EBCDIC files showing any mismatched record pairs in character & hexadecimal with differences highlighted by '*'s (see page http://uvsoftware.ca/datacnv1.htm#9B2).
#0a. Login userxx --> /home/userxx/ #0b. cdc (alias cdc='cd $CNVDATA') --> /home/userxx/cnvdata1/
#1. uvhd d1ebc/ar.customer.armaster r256a <-- check EBCDIC file
=====================================
- option 'r' (r256) for Record size
- option 'a' to translate character line1 to ASCII so we can read
- zones & digits on lines 2&3 (of each 64 byte segment) show EBCDIC values
records=32 rsize=256 fsize=8192
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 EVERGREEN MOTORS LTD. 1815 BOWEN ROAD NANA
FFFFFF4444CECDCDCCD4DDEDDE4DEC44444FFFF4CDECD4DDCC4444444444DCDC
130140000055597955504636920334B000018150266550961400000000005151
64 IMO BC V9S1H1 250-754-5531 LARRY WRENCH ...V@...
CDD4444444444CC4EFEFCF4444FFF6FFF6FFFF4DCDDE4EDCDCC4444401357000
946000000000023059218100002500754055310319980695538000000246C000
128 .........W.....-.........X'..f.....m*..................V'...f...
0000000005300016000000002570063100095000000000000000013570016000
0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0056C00
192 ......f.........c..............f..............f.C 19950531
000000890000000088000000001800068000130000000069C4FFFFFFFF444444
00C0026C0000C0023C0000C0083C0056D0012C0000C0016D3019950531000000
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#2. uvhd data1/ar.customer.armaster r257 <-- check ASCII file
==================================== - Indexed IDXFORMAT1 (fixed-length)
- option 'r257' Recsize, ONE greater for delete flag at end each record
- option 'a' omitted since now already ASCII
- zones & digits on lines 2&3 (of each 64 byte segment) show ASCII values
records=32 rsize=257 fsize=8224
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 130140 EVERGREEN MOTORS LTD. 1815 BOWEN ROAD NANA
3333332222454545444244545524542222233332445442544422222222224444
130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
64 IMO BC V9S1H1 250-754-5531 LARRY WRENCH ..4V|...
4442222222222442535343222233323332333324455525544442222201357000
9DF00000000002306931810000250D754D55310C12290725E38000000246C000
128 .........W0....`........)X}..f3.....\.................4V}...f...
0000000005300016000000002570063100095000000000000000013570016000
0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0056C00
192 ...............................f.....<........f.C 19950531
0000008900000000880000000018000680001300000000694233333333222222
00C0026C0000C0023C0000C0083C0056D0012C0000C0016D3019950531000000
256 .
0
A
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. uvhd d1ebc/city.tax.payments z4a
================================
- option 'z4' RDW variable length files with 4 byte record prefixes
- option 'a' to translate character line (zones/digits show EBCDIC values)
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 .T.. H December 31, 1998
050044444444C4C88898894FF64FFFF444444444444444444444444444444444
0400000000008045354259031B01998000000000000000000000000000000000
64
44444444444444444444
00000000000000000000
rec#=1 rsize=84 fptr=0 fsize=1680 records=16
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
84 .d..10130140T JOHN HENRY 1815 BOWEN ROAD
0800FFFFFFFFE4DDCD4CCDDE444444444444444FFFF4CDECD4DDCC4444444444
0400101301403016850855980000000000000001815026655096140000000000
64 VANCOUVER BCV9S1H1..CK..Wb....g......000149061970530
ECDCDEECD44444444444CCEFEFCF004D005880008000770FFFFFFFFFFFFFFF44
515364559000000000002359218101320072C0047C0111C00014906197053000
128
4444
0000
rec#=2 rsize=132 fptr=84 fsize=1680 records=16
10 20 30 40 50 60
r# 3 0123456789012345678901234567890123456789012345678901234567890123
216 .T..10130140P 1999...A....A....A....A....A....A....A....A.
0500FFFFFFFFD4444444FFFF0004200042000420004200042000420004200042
0400101301407000000019990001C0001C0001C0001C0001C0001C0001C0001C
64 ...A....A....A....A.
00042000420004200042
0001C0001C0001C0001C
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#2. uvhd data1/city.tax.payments v5
===============================
- option 'v5' Micro Focus COBOL variable length files with 2 byte record
prefixes (4 if record-size > 4095) & 128 byte File-Header record
- option 'v4' would show only the data records (default if just 'v')
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 0~......170329072520.................>..........................
3700000033333333333300000000000000000300000000000000000008000800
0E00000017032907252000000000000000000E01000000001000000000000000
64 ................................................................
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
rec#=1 rsize=128 fptr=0 fsize=1808 records=17
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
128 @P H December 31, 1998
4522222222424666666723322333322222222222222222222222222222222222
0000000000804535D252031C0199800000000000000000000000000000000000
64 ..
22222222222222222200
00000000000000000000
rec#=2 rsize=84 fptr=128 fsize=1808 records=17
10 20 30 40 50 60
r# 3 0123456789012345678901234567890123456789012345678901234567890123
212 @.10130140T JOHN HENRY 1815 BOWEN ROAD VA
4833333333524444244455222222222222222333324454425444222222222254
001013014040AF8E085E29000000000000000181502F75E02F14000000000061
64 NCOUVER BCV9S1H1..C...W.........qq.000149061970530
44455452222222222244535343004D0058800080007703333333333333332222
E3F5652000000000002369318101320072C0047C0111C0001490619705300000
128 ..
2200
0000
rec#=3 rsize=132 fptr=212 fsize=1808 records=17
10 20 30 40 50 60
r# 4 0123456789012345678901234567890123456789012345678901234567890123
344 @P10130140P 1999...A,...A,...A,...A,...A,...A,...A,...A,..
4533333333522222223333000420004200042000420004200042000420004200
00101301400000000019990001C0001C0001C0001C0001C0001C0001C0001C00
64 .A,...A,...A,...A,..
04200042000420004200
01C0001C0001C0001C00
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Now we will check data file conversions with 'uvhdcob' which shows the COBOL copybook field-names beside the data field contents. uvhdcob makes it easy to see if anything has gone wrong in the conversion, such as the wrong COBOL copybook, which would cause the data field contents to be misaligned with the field-names.
For uvhdcob, you must specify the COBOL copybook as the 2nd argument following the datafilename & any options as the 3rd argument. For uvhdcob, you do not need to specify option 'r' (Record-size) since that is known from the copybook.
#1. uvhdcob d1ebc/ar.customer.armaster maps/armaster a
==================================================
- can omit option 'r' (Record-size) since known from copybook
- need option 'a' to translate character line to Ascii
now=20170329:1051 uvhdcob d1ebc/ar.customer.armaster m25a version=20161001 copybook=maps/armaster rec#=1 rcount=32 rsize=256 fsize=8192 fptr=0 rec# 1 occurs bgn end typ <------ data (hex if typ=p/b) -----> 001 armaster 0 0 002 cm-cust 0 5 n 130140 003 cm-delete 6 9 004 cm-name 10 34 EVERGREEN MOTORS LTD. 005 cm-adrs 35 59 1815 BOWEN ROAD 006 cm-city 60 75 NANAIMO 007 filler001 76 76 008 cm-prov 77 78 BC 009 filler002 79 79 010 cm-postal 80 89 V9S1H1 011 cm-telephone 90 101 250-754-5531 012 cm-contact 102 119 LARRY WRENCH 013 cm-thisyr-sales 1/12 120 124pns 001234567C 014 cm-lastyr-sales 1/12 180 184pns 001234567D 015 filler003 240 255 C 19950531 rec#=1 rcount=32 rsize=256 fsize=8192 fptr=0
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#2. uvhdcob data1/ar.customer.armaster maps/armaster <-- check ASCII file
================================================ - Indexed IDXFORMAT1
- uvhdcob needs no options for fixed length ASCII files
now=20170329:1052 uvhdcob data1/ar.customer.armaster.dat m25 version=20161001 copybook=maps/armaster rec#=1 rcount=32 rsize=257 fsize=8224 fptr=0 rec# 1 occurs bgn end typ <------ data (hex if typ=p/b) -----> 001 armaster 0 0 002 cm-cust 0 5 n 130140 003 cm-delete 6 9 004 cm-name 10 34 EVERGREEN MOTORS LTD. 005 cm-adrs 35 59 1815 BOWEN ROAD 006 cm-city 60 75 NANAIMO 007 filler001 76 76 008 cm-prov 77 78 BC 009 filler002 79 79 010 cm-postal 80 89 V9S1H1 011 cm-telephone 90 101 250-754-5531 012 cm-contact 102 119 LARRY WRENCH 013 cm-thisyr-sales 1/12 120 124pns 001234567C 014 cm-lastyr-sales 1/12 180 184pns 001234567D 015 filler003 240 255 C 19950531 rec#=1 rcount=32 rsize=257 fsize=8224 fptr=0
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#1. uvhdcob d1ebc/city.tax.payments maps/citytax2 z4a
=================================================
- option 'z4' RDW variable length files with 4 byte record prefixes
- option 'a' to translate character line (zones/digits show EBCDIC values)
----> output to be supplied
#2. uvhdcob data1/city.tax.payments maps/citytax2 v5
================================================
- option 'v5' Micro Focus COBOL variable length files with 2 byte record
prefixes (4 if record-size > 4095) & 128 byte File-Header record
----> output to be supplied
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
This is the EBCDIC to ASCII translate table used by the Vancouver Utiltiies (uvcopy, uvsort, uvcp, etc).
/* ebc2asc.c - EBCDIC to ASCII translation table */
/* - Owen Townsend, UV Software, June 1995 */
/* similar to ISO 8859-1, but defines some control chars undefined by ISO 8859*/
unsigned char ebc2asc[256] = {
0x00,0x01,0x02,0x03,0x04,0x09,0x06,0x7f,0x08,0x09,0x0a,0x0b,0x0c,0x0d,0x0e,0x0f,
0x10,0x11,0x12,0x13,0x14,0x15,0x08,0x17,0x18,0x19,0x1a,0x1b,0x1c,0x1d,0x1e,0x1f,
0x20,0x21,0x22,0x23,0x24,0x0a,0x17,0x1b,0x28,0x29,0x2a,0x2b,0x2c,0x05,0x06,0x07,
0x30,0x31,0x16,0x33,0x34,0x35,0x36,0x04,0x38,0x39,0x3a,0x3b,0x14,0x15,0x3e,0x1a,
' ',0x41,0x42,0x43,0x44,0x45,0x46,0x47,0x48,0x49,0xA2, '.', '<', '(', '+', '|',
'&',0x51,0x52,0x53,0x54,0x55,0x56,0x57,0x58,0x59, '!', '$', '*', ')', ';', '^',
'-', '/',0xC2,0x63,0x64,0x65,0x66,0x67,0x68,0x69,0xA6, ',', '%', '_', '>', '?',
0x70,0xC9,0x72,0x73,0xC8,0x76,0x77,0x78,0x79, '`', ':', '#', '@',0x27, '=', '"',
0x80, 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i',0x8a,0x8b,0x8c,0x8d,0x8e,0x8f,
0x90, 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r',0x9a,0x9b,0x9c,0x9d,0x9e,0x9f,
0xa0, '~', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z',0xaa,0xab,0xac,'[',0xae,0xaf,
0xb0,0xb1,0xb2,0xb3,0xb4,0xb5,0xb6,0xb7,0xb8,0xb9,0xba,0xbb,0xbc,']',0xbe,0xbf,
'{', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I',0xca,0xcb,0xcc,0xcd,0xce,0xcf,
'}', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R',0xda,0xdb,0xdc,0xdd,0xde,0xdf,
0x5c,0xe1, 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z',0xea,0xeb,0xec,0xed,0xee,0xef,
'0', '1', '2', '3', '4', '5', '6', '7', '8', '9',0xfa,0xfb,0xfc,0xfd,0xfe,0xff
};
/* EBCDIC '|' - ARC x'6A', Oracle x'BB', Simotime x'4F' */
/* Oracle '|' = x'BB', '[' = x'4A', ']' = x'5A', '!' = x'4F' */
/* Simotime '|' = x'4F', '[' = x'BA', ']' = x'BB', '!' = x'5A' */
/*Sep02/12 - add accented characters: A circumflex x'62'-x'C2', */
/* E acute x'71'-x'C9', E grave x'74'-x'C8' */
/*Mar14/13 - change x'4F' from ']' to '|' for Cothern */
/*Apr22/13 - change x'6A' from '|' to x'A6' (simotime alt '|') */
/*Dec24/14 - add '[' & ']' 0xad & 0xbd code page 1047 IMMD */
/* - 0x4A EBCDIC cent sign to 0xA2 ASCII cent (vs '[') */
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
This is the ASCII to EBCDIC translate table used by the Vancouver Utiltiies (uvcopy, uvsort, uvcp, etc).
/* asc2ebc.c - ASCII to EBCDIC translation table */
/* - Owen Townsend, UV Software, June 1995 */
/* similar to ISO 8859-1, but defines some control chars undefined by ISO 8859*/
unsigned char asc2ebc[256] = {
0x00,0x01,0x02,0x03,0x37,0x2d,0x2e,0x2f,0x16,0x05,0x25,0x0b,0x0c,0x0d,0x0e,0x0f,
0x10,0x11,0x12,0x13,0x3c,0x3d,0x32,0x26,0x18,0x19,0x3f,0x27,0x1c,0x1d,0x1e,0x1f,
0x40,0x5a,0x7f,0x7b,0x5b,0x6c,0x50,0x7d,0x4d,0x5d,0x5c,0x4e,0x6b,0x60,0x4b,0x61,
0xf0,0xf1,0xf2,0xf3,0xf4,0xf5,0xf6,0xf7,0xf8,0xf9,0x7a,0x5e,0x4c,0x7e,0x6e,0x6f,
0x7c,0xc1,0xc2,0xc3,0xc4,0xc5,0xc6,0xc7,0xc8,0xc9,0xd1,0xd2,0xd3,0xd4,0xd5,0xd6,
0xd7,0xd8,0xd9,0xe2,0xe3,0xe4,0xe5,0xe6,0xe7,0xe8,0xe9,0xAD,0xe0,0xBD,0x5f,0x6d,
0x79,0x81,0x82,0x83,0x84,0x85,0x86,0x87,0x88,0x89,0x91,0x92,0x93,0x94,0x95,0x96,
0x97,0x98,0x99,0xa2,0xa3,0xa4,0xa5,0xa6,0xa7,0xa8,0xa9,0xc0,0x4F,0xd0,0xa1,0x07,
0x80,0x81,0x82,0x83,0x84,0x85,0x86,0x87,0x88,0x89,0x8A,0x8B,0x8C,0x8D,0x8E,0x8F,
0x90,0x91,0x92,0x93,0x94,0x95,0x96,0x97,0x98,0x99,0x9A,0x9B,0x9C,0x9D,0x9E,0x9F,
0xA0,0xA1,0x4A,0xA3,0xA4,0xA5,0xA6,0xA7,0xA8,0xA9,0xAA,0xAB,0xAC,0xAD,0xAE,0xAF,
0xB0,0xB1,0xB2,0xB3,0xB4,0xB5,0xB6,0xB7,0xB8,0xB9,0xBA,0xBB,0xBC,0xBD,0xBE,0xBF,
0xC0,0xC1,0x62,0xC3,0xC4,0xC5,0xC6,0xC7,0x74,0x71,0xCA,0xCB,0xCC,0xCD,0xCE,0xCF,
0xD0,0xD1,0xD2,0xD3,0xD4,0xD5,0xD6,0xD7,0xD8,0xD9,0xDA,0xDB,0xDC,0xDD,0xDE,0xDF,
0xE0,0xE1,0xE2,0xE3,0xE4,0xE5,0xE6,0xE7,0xE8,0xE9,0xEA,0xEB,0xEC,0xED,0xEE,0xEF,
0xF0,0xF1,0xF2,0xF3,0xF4,0xF5,0xF6,0xF7,0xF8,0xF9,0xFA,0xFB,0xFC,0xFD,0xFE,0xFF
};
/* Nov2002 - asc2ebc.c modified x'80'- x'FF' now neutral */ /* - asc2ebc2.c created all x'40' in x'80'-x'FF' */ /*Sep02/12 - add accented chars: A circumflex x'C2'-x'62', */ /* E acute x'C9'-x'71', E grave x'C8'-x'74' */ /*Oct14/14 - convert '|' x'7C' to x'4F (vs x'6A') */ /* to match Mar14/13 ebc2asc x'4F' to '|' x'7C' */ /*Dec24/14 - change x'5B' to x'AD' '[' & x'5D' to x'BD' ']' */ /* - see reverse changes in ebc2asc.c */
/* see following notes in 'ebc2asc.c' applicable to this 'asc2ebc.c' */ /*Mar14/13 - change x'4F' from ']' to '|' for Cothern */ /*Apr22/13 - change x'6A' from '|' to x'A6' (simotime alt '|') */ /* EBCDIC '|' - ARC x'6A', Oracle x'BB', Simotime x'4F' */ /* Oracle '|' = x'BB', '[' = x'4A', ']' = x'5A', '!' = x'4F' */ /* Simotime '|' = x'4F', '[' = x'BA', ']' = x'BB', '!' = x'5A' */ /*Dec24/14 - add '[' & ']' 0xad & 0xbd code page 1047 IMMD */ /* - see reverse changes above in this asc2ebc.c */ /* - 0xA2 ASCII cent sign to 0x4A EBCDIC cent (vs '[') */
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
/* ebc1141asc.c - translate table, EBCDIC-1141 to ASCII ISO8859-1 */
/* - stored at $UV/srcf/ebc1141asc.c */
/* - Jan23/10, for Exigen, by Owen Townsend, UV Software */
/* - copied/renamed/modified ebc2asc.c to ebc1141asc.c */
/* - code page 1141 does not specify x'00' - x'3F' */
/* - these codes x'00'-x'3F' same as ebc2asc.c originally from A/R */
unsigned char ebc1141asc[256] = {
0x00,0x01,0x02,0x03,0x9C,0x09,0x86,0x7f,0x97,0x8D,0x0a,0x0b,0x0c,0x0d,0x0e,0x0f,
0x10,0x11,0x12,0x13,0x9D,0x0A,0x08,0x87,0x18,0x19,0x92,0x8F,0x1c,0x1d,0x1e,0x1f,
0x80,0x81,0x82,0x83,0x84,0x85,0x17,0x1b,0x88,0x89,0x8a,0x8b,0x8c,0x05,0x06,0x07,
0x90,0x91,0x16,0x93,0x94,0x95,0x96,0x04,0x38,0x39,0x3a,0x3b,0x14,0x15,0x9e,0x1a,
0x20,0xA0,0xE2,0x7B,0xE0,0xE1,0xE3,0xE5,0xE7,0xF1,0xC4,0x2E,0x3C,0x28,0x2B,0x21,
0x26,0xE9,0xEA,0xEB,0xE8,0xED,0xEE,0xEF,0xEC,0x7E,0xDC,0x24,0x2A,0x29,0x3B,0x5E,
0x2D,0x2F,0xC2,0x5B,0xC0,0xC1,0xC3,0xC5,0xC7,0xD1,0xF6,0x2C,0x25,0x5F,0x3E,0x3F,
0xF8,0xC9,0xCA,0xCB,0xC8,0xCD,0xCE,0xCF,0xCC,0x60,0x3A,0x23,0xA7,0x27,0x3D,0x22,
0xDB,'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i' ,0xAB,0xBB,0xF0,0xFD,0xFE,0xB1,
0xB0,'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 0xAA,0xBA,0xE6,0xB8,0xC6,0xA4,
0xB5,0xDF,'s', 't', 'u', 'v', 'w', 'x', 'y', 'z', 0xA1,0xBF,0xD0,0xDD,0xDE,0XAE,
0xA2,0xA3,0xA5,0xB7,0xA9,0x40,0xB6,0xBC,0xBD,0xBE,0xAC,0x7C,0xAF,0xA8,0xB4,0xD7,
0xE4, 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I',0xAD,0xF4,0xA6,0xF2,0xF3,0xF5,
0xFC, 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R',0xB9,0xFB,0x7D,0xF9,0xFA,0xFF,
0xD6,0xF7, 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z',0xB2,0xD4,0x5C,0xD2,0xD3,0xD5,
'0', '1', '2', '3', '4', '5', '6', '7', '8', '9',0xB3,0xDB,0x5D,0xD9,0xDA,0x9F
};
/* see uvcopy job 'trantestE2A' to show results translating each of 256 characters*/ /* - from EBCDIC to ASCII & back to EBCDIC */ /* uvcopy trantestE2A <-- using ebc2asc.c original A/R table */ /* uvcopy trantestE2A,uop=t1 <-- using ebc1141asc.c (this code page 1141 table) */ /* ========================= */ /* uvcopy trantestE2A,rop=t1 <-- OR use Run OPtion (vs User OPtion) */ /* ========================= */ /* export UVCOPYROP=t1 <-- could set codepage 1141 option for */ /* =================== all uvcopy jobs until logout */
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Here is an alternate way to test Vancouver Utilities EBCDIC/ASCII translates. "trantestEAE' is a uvcopy job to translate EBCDIC to ASCII & back to EBCDIC. It requires no data file input, it adds binary to create each character of the 256 possible & translates each from EBCDIC to ASCII & back to EBCDIC.
uvcopy trantestEAE <-- generate 256 lines E-->A-->E ================== uvcopy trantestEAE,uop=t1 <-- option to use codepage 1141 ========================= uvcopy trantestEAE,rop=t1 <-- OR can use Run OPtion (vs User OPtion) ========================= export UVCOPYROP=t1 <-- could set codepage 1141 option for =================== all uvcopy jobs until logout
| trantestEAE |
|
x'00' --> x'00' --> x'00' c'^@' ---> c'^@' ---> c'^@'
x'01' --> x'01' --> x'01' c'^A' ---> c'^A' ---> c'^A'
x'02' --> x'02' --> x'02' c'^B' ---> c'^B' ---> c'^B'
x'03' --> x'03' --> x'03' c'^C' ---> c'^C' ---> c'^C'
x'04' --> x'04' --> x'37' c'^D' ---> c'^D' ---> c'7'
x'05' --> x'09' --> x'05' c'^E' ---> c'^I' ---> c'^E'
x'06' --> x'06' --> x'2E' c'^F' ---> c'^F' ---> c'.'
x'07' --> x'7F' --> x'07' c'^G' ---> c'' ---> c'^G'
x'08' --> x'08' --> x'16' c'^H' ---> c'^H' ---> c'^V'
x'09' --> x'09' --> x'05' c'^I' ---> c'^I' ---> c'^E'
x'0A' --> x'0A' --> x'25' c'^J' ---> c'^J' ---> c'%'
x'0B' --> x'0B' --> x'0B' c'^K' ---> c'^K' ---> c'^K'
x'0C' --> x'0C' --> x'0C' c'^L' ---> c'^L' ---> c'^L'
x'0D' --> x'0D' --> x'0D' c'^M' ---> c'^M' ---> c'^M'
x'0E' --> x'0E' --> x'0E' c'^N' ---> c'^N' ---> c'^N'
x'0F' --> x'0F' --> x'0F' c'^O' ---> c'^O' ---> c'^O'
x'10' --> x'10' --> x'10' c'^P' ---> c'^P' ---> c'^P'
x'11' --> x'11' --> x'11' c'^Q' ---> c'^Q' ---> c'^Q'
x'12' --> x'12' --> x'12' c'^R' ---> c'^R' ---> c'^R'
x'13' --> x'13' --> x'13' c'^S' ---> c'^S' ---> c'^S'
x'14' --> x'14' --> x'3C' c'^T' ---> c'^T' ---> c'<'
x'15' --> x'15' --> x'3D' c'^U' ---> c'^U' ---> c'='
x'16' --> x'08' --> x'16' c'^V' ---> c'^H' ---> c'^V'
x'17' --> x'17' --> x'26' c'^W' ---> c'^W' ---> c'&'
x'18' --> x'18' --> x'18' c'^X' ---> c'^X' ---> c'^X'
x'19' --> x'19' --> x'19' c'^Y' ---> c'^Y' ---> c'^Y'
x'1A' --> x'1A' --> x'3F' c'^Z' ---> c'^Z' ---> c'?'
x'1B' --> x'1B' --> x'27' c'^[' ---> c'^[' ---> c'''
x'1C' --> x'1C' --> x'1C' c'^\' ---> c'^\' ---> c'^\'
x'1D' --> x'1D' --> x'1D' c'^]' ---> c'^]' ---> c'^]'
x'1E' --> x'1E' --> x'1E' c'^^' ---> c'^^' ---> c'^^'
x'1F' --> x'1F' --> x'1F' c'^_' ---> c'^_' ---> c'^_'
x'20' --> x'20' --> x'40' c' ' ---> c' ' ---> c'@'
x'21' --> x'21' --> x'5A' c'!' ---> c'!' ---> c'Z'
x'22' --> x'22' --> x'7F' c'"' ---> c'"' ---> c''
x'23' --> x'23' --> x'7B' c'#' ---> c'#' ---> c'{'
x'24' --> x'24' --> x'5B' c'$' ---> c'$' ---> c'['
x'25' --> x'0A' --> x'25' c'%' ---> c'^J' ---> c'%'
x'26' --> x'17' --> x'26' c'&' ---> c'^W' ---> c'&'
x'27' --> x'1B' --> x'27' c''' ---> c'^[' ---> c'''
x'28' --> x'28' --> x'4D' c'(' ---> c'(' ---> c'M'
x'29' --> x'29' --> x'5D' c')' ---> c')' ---> c']'
x'2A' --> x'2A' --> x'5C' c'*' ---> c'*' ---> c'\'
x'2B' --> x'2B' --> x'4E' c'+' ---> c'+' ---> c'N'
x'2C' --> x'2C' --> x'6B' c',' ---> c',' ---> c'k'
x'2D' --> x'05' --> x'2D' c'-' ---> c'^E' ---> c'-'
x'2E' --> x'06' --> x'2E' c'.' ---> c'^F' ---> c'.'
x'2F' --> x'07' --> x'2F' c'/' ---> c'^G' ---> c'/'
x'30' --> x'30' --> x'F0' c'0' ---> c'0' ---> c'ð'
x'31' --> x'31' --> x'F1' c'1' ---> c'1' ---> c'ñ'
x'32' --> x'16' --> x'32' c'2' ---> c'^V' ---> c'2'
x'33' --> x'33' --> x'F3' c'3' ---> c'3' ---> c'ó'
x'34' --> x'34' --> x'F4' c'4' ---> c'4' ---> c'ô'
x'35' --> x'35' --> x'F5' c'5' ---> c'5' ---> c'õ'
x'36' --> x'36' --> x'F6' c'6' ---> c'6' ---> c'ö'
x'37' --> x'04' --> x'37' c'7' ---> c'^D' ---> c'7'
x'38' --> x'38' --> x'F8' c'8' ---> c'8' ---> c'ø'
x'39' --> x'39' --> x'F9' c'9' ---> c'9' ---> c'ù'
x'3A' --> x'3A' --> x'7A' c':' ---> c':' ---> c'z'
x'3B' --> x'3B' --> x'5E' c';' ---> c';' ---> c'^'
x'3C' --> x'14' --> x'3C' c'<' ---> c'^T' ---> c'<'
x'3D' --> x'15' --> x'3D' c'=' ---> c'^U' ---> c'='
x'3E' --> x'3E' --> x'6E' c'>' ---> c'>' ---> c'n'
x'3F' --> x'1A' --> x'3F' c'?' ---> c'^Z' ---> c'?'
x'40' --> x'20' --> x'40' c'@' ---> c' ' ---> c'@'
x'41' --> x'41' --> x'C1' c'A' ---> c'A' ---> c'Á'
x'42' --> x'42' --> x'C2' c'B' ---> c'B' ---> c'Â'
x'43' --> x'43' --> x'C3' c'C' ---> c'C' ---> c'Ã'
x'44' --> x'44' --> x'C4' c'D' ---> c'D' ---> c'Ä'
x'45' --> x'45' --> x'C5' c'E' ---> c'E' ---> c'Å'
x'46' --> x'46' --> x'C6' c'F' ---> c'F' ---> c'Æ'
x'47' --> x'47' --> x'C7' c'G' ---> c'G' ---> c'Ç'
x'48' --> x'48' --> x'C8' c'H' ---> c'H' ---> c'È'
x'49' --> x'49' --> x'C9' c'I' ---> c'I' ---> c'É'
x'4A' --> x'A2' --> x'4A' c'J' ---> c'¢' ---> c'J'
x'4B' --> x'2E' --> x'4B' c'K' ---> c'.' ---> c'K'
x'4C' --> x'3C' --> x'4C' c'L' ---> c'<' ---> c'L'
x'4D' --> x'28' --> x'4D' c'M' ---> c'(' ---> c'M'
x'4E' --> x'2B' --> x'4E' c'N' ---> c'+' ---> c'N'
x'4F' --> x'7C' --> x'4F' c'O' ---> c'|' ---> c'O'
x'50' --> x'26' --> x'50' c'P' ---> c'&' ---> c'P'
x'51' --> x'51' --> x'D8' c'Q' ---> c'Q' ---> c'Ø'
x'52' --> x'52' --> x'D9' c'R' ---> c'R' ---> c'Ù'
x'53' --> x'53' --> x'E2' c'S' ---> c'S' ---> c'â'
x'54' --> x'54' --> x'E3' c'T' ---> c'T' ---> c'ã'
x'55' --> x'55' --> x'E4' c'U' ---> c'U' ---> c'ä'
x'56' --> x'56' --> x'E5' c'V' ---> c'V' ---> c'å'
x'57' --> x'57' --> x'E6' c'W' ---> c'W' ---> c'æ'
x'58' --> x'58' --> x'E7' c'X' ---> c'X' ---> c'ç'
x'59' --> x'59' --> x'E8' c'Y' ---> c'Y' ---> c'è'
x'5A' --> x'21' --> x'5A' c'Z' ---> c'!' ---> c'Z'
x'5B' --> x'24' --> x'5B' c'[' ---> c'$' ---> c'['
x'5C' --> x'2A' --> x'5C' c'\' ---> c'*' ---> c'\'
x'5D' --> x'29' --> x'5D' c']' ---> c')' ---> c']'
x'5E' --> x'3B' --> x'5E' c'^' ---> c';' ---> c'^'
x'5F' --> x'5E' --> x'5F' c'_' ---> c'^' ---> c'_'
x'60' --> x'2D' --> x'60' c'`' ---> c'-' ---> c'`'
x'61' --> x'2F' --> x'61' c'a' ---> c'/' ---> c'a'
x'62' --> x'C2' --> x'62' c'b' ---> c'Â' ---> c'b'
x'63' --> x'63' --> x'83' c'c' ---> c'c' ---> c'ƒ'
x'64' --> x'64' --> x'84' c'd' ---> c'd' ---> c'„'
x'65' --> x'65' --> x'85' c'e' ---> c'e' ---> c'…'
x'66' --> x'66' --> x'86' c'f' ---> c'f' ---> c'†'
x'67' --> x'67' --> x'87' c'g' ---> c'g' ---> c'‡'
x'68' --> x'68' --> x'88' c'h' ---> c'h' ---> c'ˆ'
x'69' --> x'69' --> x'89' c'i' ---> c'i' ---> c'‰'
x'6A' --> x'A6' --> x'A6' c'j' ---> c'¦' ---> c'¦'
x'6B' --> x'2C' --> x'6B' c'k' ---> c',' ---> c'k'
x'6C' --> x'25' --> x'6C' c'l' ---> c'%' ---> c'l'
x'6D' --> x'5F' --> x'6D' c'm' ---> c'_' ---> c'm'
x'6E' --> x'3E' --> x'6E' c'n' ---> c'>' ---> c'n'
x'6F' --> x'3F' --> x'6F' c'o' ---> c'?' ---> c'o'
x'70' --> x'70' --> x'97' c'p' ---> c'p' ---> c'—'
x'71' --> x'C9' --> x'71' c'q' ---> c'É' ---> c'q'
x'72' --> x'72' --> x'99' c'r' ---> c'r' ---> c'™'
x'73' --> x'73' --> x'A2' c's' ---> c's' ---> c'¢'
x'74' --> x'C8' --> x'74' c't' ---> c'È' ---> c't'
x'75' --> x'76' --> x'A5' c'u' ---> c'v' ---> c'¥'
x'76' --> x'77' --> x'A6' c'v' ---> c'w' ---> c'¦'
x'77' --> x'78' --> x'A7' c'w' ---> c'x' ---> c'§'
x'78' --> x'79' --> x'A8' c'x' ---> c'y' ---> c'¨'
x'79' --> x'60' --> x'79' c'y' ---> c'`' ---> c'y'
x'7A' --> x'3A' --> x'7A' c'z' ---> c':' ---> c'z'
x'7B' --> x'23' --> x'7B' c'{' ---> c'#' ---> c'{'
x'7C' --> x'40' --> x'7C' c'|' ---> c'@' ---> c'|'
x'7D' --> x'27' --> x'7D' c'}' ---> c''' ---> c'}'
x'7E' --> x'3D' --> x'7E' c'~' ---> c'=' ---> c'~'
x'7F' --> x'22' --> x'7F' c'' ---> c'"' ---> c''
x'80' --> x'80' --> x'80' c'€' ---> c'€' ---> c'€'
x'81' --> x'61' --> x'81' c'' ---> c'a' ---> c''
x'82' --> x'62' --> x'82' c'‚' ---> c'b' ---> c'‚'
x'83' --> x'63' --> x'83' c'ƒ' ---> c'c' ---> c'ƒ'
x'84' --> x'64' --> x'84' c'„' ---> c'd' ---> c'„'
x'85' --> x'65' --> x'85' c'…' ---> c'e' ---> c'…'
x'86' --> x'66' --> x'86' c'†' ---> c'f' ---> c'†'
x'87' --> x'67' --> x'87' c'‡' ---> c'g' ---> c'‡'
x'88' --> x'68' --> x'88' c'ˆ' ---> c'h' ---> c'ˆ'
x'89' --> x'69' --> x'89' c'‰' ---> c'i' ---> c'‰'
x'8A' --> x'8A' --> x'8A' c'Š' ---> c'Š' ---> c'Š'
x'8B' --> x'8B' --> x'8B' c'‹' ---> c'‹' ---> c'‹'
x'8C' --> x'8C' --> x'8C' c'Œ' ---> c'Œ' ---> c'Œ'
x'8D' --> x'8D' --> x'8D' c'' ---> c'' ---> c''
x'8E' --> x'8E' --> x'8E' c'Ž' ---> c'Ž' ---> c'Ž'
x'8F' --> x'8F' --> x'8F' c'' ---> c'' ---> c''
x'90' --> x'90' --> x'90' c'' ---> c'' ---> c''
x'91' --> x'6A' --> x'91' c'‘' ---> c'j' ---> c'‘'
x'92' --> x'6B' --> x'92' c'’' ---> c'k' ---> c'’'
x'93' --> x'6C' --> x'93' c'“' ---> c'l' ---> c'“'
x'94' --> x'6D' --> x'94' c'”' ---> c'm' ---> c'”'
x'95' --> x'6E' --> x'95' c'•' ---> c'n' ---> c'•'
x'96' --> x'6F' --> x'96' c'–' ---> c'o' ---> c'–'
x'97' --> x'70' --> x'97' c'—' ---> c'p' ---> c'—'
x'98' --> x'71' --> x'98' c'˜' ---> c'q' ---> c'˜'
x'99' --> x'72' --> x'99' c'™' ---> c'r' ---> c'™'
x'9A' --> x'9A' --> x'9A' c'š' ---> c'š' ---> c'š'
x'9B' --> x'9B' --> x'9B' c'›' ---> c'›' ---> c'›'
x'9C' --> x'9C' --> x'9C' c'œ' ---> c'œ' ---> c'œ'
x'9D' --> x'9D' --> x'9D' c'' ---> c'' ---> c''
x'9E' --> x'9E' --> x'9E' c'ž' ---> c'ž' ---> c'ž'
x'9F' --> x'9F' --> x'9F' c'Ÿ' ---> c'Ÿ' ---> c'Ÿ'
x'A0' --> x'A0' --> x'A0' c' ' ---> c' ' ---> c' '
x'A1' --> x'7E' --> x'A1' c'¡' ---> c'~' ---> c'¡'
x'A2' --> x'73' --> x'A2' c'¢' ---> c's' ---> c'¢'
x'A3' --> x'74' --> x'A3' c'£' ---> c't' ---> c'£'
x'A4' --> x'75' --> x'A4' c'¤' ---> c'u' ---> c'¤'
x'A5' --> x'76' --> x'A5' c'¥' ---> c'v' ---> c'¥'
x'A6' --> x'77' --> x'A6' c'¦' ---> c'w' ---> c'¦'
x'A7' --> x'78' --> x'A7' c'§' ---> c'x' ---> c'§'
x'A8' --> x'79' --> x'A8' c'¨' ---> c'y' ---> c'¨'
x'A9' --> x'7A' --> x'A9' c'©' ---> c'z' ---> c'©'
x'AA' --> x'AA' --> x'AA' c'ª' ---> c'ª' ---> c'ª'
x'AB' --> x'AB' --> x'AB' c'«' ---> c'«' ---> c'«'
x'AC' --> x'AC' --> x'AC' c'¬' ---> c'¬' ---> c'¬'
x'AD' --> x'5B' --> x'AD' c'' ---> c'[' ---> c''
x'AE' --> x'AE' --> x'AE' c'®' ---> c'®' ---> c'®'
x'AF' --> x'AF' --> x'AF' c'¯' ---> c'¯' ---> c'¯'
x'B0' --> x'B0' --> x'B0' c'°' ---> c'°' ---> c'°'
x'B1' --> x'B1' --> x'B1' c'±' ---> c'±' ---> c'±'
x'B2' --> x'B2' --> x'B2' c'²' ---> c'²' ---> c'²'
x'B3' --> x'B3' --> x'B3' c'³' ---> c'³' ---> c'³'
x'B4' --> x'B4' --> x'B4' c'´' ---> c'´' ---> c'´'
x'B5' --> x'B5' --> x'B5' c'µ' ---> c'µ' ---> c'µ'
x'B6' --> x'B6' --> x'B6' c'¶' ---> c'¶' ---> c'¶'
x'B7' --> x'B7' --> x'B7' c'·' ---> c'·' ---> c'·'
x'B8' --> x'B8' --> x'B8' c'¸' ---> c'¸' ---> c'¸'
x'B9' --> x'B9' --> x'B9' c'¹' ---> c'¹' ---> c'¹'
x'BA' --> x'BA' --> x'BA' c'º' ---> c'º' ---> c'º'
x'BB' --> x'BB' --> x'BB' c'»' ---> c'»' ---> c'»'
x'BC' --> x'BC' --> x'BC' c'¼' ---> c'¼' ---> c'¼'
x'BD' --> x'5D' --> x'BD' c'½' ---> c']' ---> c'½'
x'BE' --> x'BE' --> x'BE' c'¾' ---> c'¾' ---> c'¾'
x'BF' --> x'BF' --> x'BF' c'¿' ---> c'¿' ---> c'¿'
x'C0' --> x'7B' --> x'C0' c'À' ---> c'{' ---> c'À'
x'C1' --> x'41' --> x'C1' c'Á' ---> c'A' ---> c'Á'
x'C2' --> x'42' --> x'C2' c'Â' ---> c'B' ---> c'Â'
x'C3' --> x'43' --> x'C3' c'Ã' ---> c'C' ---> c'Ã'
x'C4' --> x'44' --> x'C4' c'Ä' ---> c'D' ---> c'Ä'
x'C5' --> x'45' --> x'C5' c'Å' ---> c'E' ---> c'Å'
x'C6' --> x'46' --> x'C6' c'Æ' ---> c'F' ---> c'Æ'
x'C7' --> x'47' --> x'C7' c'Ç' ---> c'G' ---> c'Ç'
x'C8' --> x'48' --> x'C8' c'È' ---> c'H' ---> c'È'
x'C9' --> x'49' --> x'C9' c'É' ---> c'I' ---> c'É' x'CA' --> x'CA' --> x'CA' c'Ê' ---> c'Ê' ---> c'Ê' x'CB' --> x'CB' --> x'CB' c'Ë' ---> c'Ë' ---> c'Ë' x'CC' --> x'CC' --> x'CC' c'Ì' ---> c'Ì' ---> c'Ì' x'CD' --> x'CD' --> x'CD' c'Í' ---> c'Í' ---> c'Í' x'CE' --> x'CE' --> x'CE' c'Î' ---> c'Î' ---> c'Î' x'CF' --> x'CF' --> x'CF' c'Ï' ---> c'Ï' ---> c'Ï' x'D0' --> x'7D' --> x'D0' c'Ð' ---> c'}' ---> c'Ð' x'D1' --> x'4A' --> x'D1' c'Ñ' ---> c'J' ---> c'Ñ' x'D2' --> x'4B' --> x'D2' c'Ò' ---> c'K' ---> c'Ò' x'D3' --> x'4C' --> x'D3' c'Ó' ---> c'L' ---> c'Ó' x'D4' --> x'4D' --> x'D4' c'Ô' ---> c'M' ---> c'Ô' x'D5' --> x'4E' --> x'D5' c'Õ' ---> c'N' ---> c'Õ' x'D6' --> x'4F' --> x'D6' c'Ö' ---> c'O' ---> c'Ö' x'D7' --> x'50' --> x'D7' c'×' ---> c'P' ---> c'×' x'D8' --> x'51' --> x'D8' c'Ø' ---> c'Q' ---> c'Ø' x'D9' --> x'52' --> x'D9' c'Ù' ---> c'R' ---> c'Ù' x'DA' --> x'DA' --> x'DA' c'Ú' ---> c'Ú' ---> c'Ú' x'DB' --> x'DB' --> x'DB' c'Û' ---> c'Û' ---> c'Û' x'DC' --> x'DC' --> x'DC' c'Ü' ---> c'Ü' ---> c'Ü' x'DD' --> x'DD' --> x'DD' c'Ý' ---> c'Ý' ---> c'Ý' x'DE' --> x'DE' --> x'DE' c'Þ' ---> c'Þ' ---> c'Þ' x'DF' --> x'DF' --> x'DF' c'ß' ---> c'ß' ---> c'ß' x'E0' --> x'5C' --> x'E0' c'à' ---> c'\' ---> c'à' x'E1' --> x'E1' --> x'E1' c'á' ---> c'á' ---> c'á' x'E2' --> x'53' --> x'E2' c'â' ---> c'S' ---> c'â' x'E3' --> x'54' --> x'E3' c'ã' ---> c'T' ---> c'ã' x'E4' --> x'55' --> x'E4' c'ä' ---> c'U' ---> c'ä' x'E5' --> x'56' --> x'E5' c'å' ---> c'V' ---> c'å' x'E6' --> x'57' --> x'E6' c'æ' ---> c'W' ---> c'æ' x'E7' --> x'58' --> x'E7' c'ç' ---> c'X' ---> c'ç' x'E8' --> x'59' --> x'E8' c'è' ---> c'Y' ---> c'è' x'E9' --> x'5A' --> x'E9' c'é' ---> c'Z' ---> c'é' x'EA' --> x'EA' --> x'EA' c'ê' ---> c'ê' ---> c'ê' x'EB' --> x'EB' --> x'EB' c'ë' ---> c'ë' ---> c'ë' x'EC' --> x'EC' --> x'EC' c'ì' ---> c'ì' ---> c'ì' x'ED' --> x'ED' --> x'ED' c'í' ---> c'í' ---> c'í' x'EE' --> x'EE' --> x'EE' c'î' ---> c'î' ---> c'î' x'EF' --> x'EF' --> x'EF' c'ï' ---> c'ï' ---> c'ï' x'F0' --> x'30' --> x'F0' c'ð' ---> c'0' ---> c'ð' x'F1' --> x'31' --> x'F1' c'ñ' ---> c'1' ---> c'ñ' x'F2' --> x'32' --> x'F2' c'ò' ---> c'2' ---> c'ò' x'F3' --> x'33' --> x'F3' c'ó' ---> c'3' ---> c'ó' x'F4' --> x'34' --> x'F4' c'ô' ---> c'4' ---> c'ô' x'F5' --> x'35' --> x'F5' c'õ' ---> c'5' ---> c'õ' x'F6' --> x'36' --> x'F6' c'ö' ---> c'6' ---> c'ö' x'F7' --> x'37' --> x'F7' c'÷' ---> c'7' ---> c'÷' x'F8' --> x'38' --> x'F8' c'ø' ---> c'8' ---> c'ø' x'F9' --> x'39' --> x'F9' c'ù' ---> c'9' ---> c'ù' x'FA' --> x'FA' --> x'FA' c'ú' ---> c'ú' ---> c'ú' x'FB' --> x'FB' --> x'FB' c'û' ---> c'û' ---> c'û' x'FC' --> x'FC' --> x'FC' c'ü' ---> c'ü' ---> c'ü' x'FD' --> x'FD' --> x'FD' c'ý' ---> c'ý' ---> c'ý' x'FE' --> x'FE' --> x'FE' c'þ' ---> c'þ' ---> c'þ' x'FF' --> x'FF' --> x'FF' c'ÿ' ---> c'ÿ' ---> c'ÿ'
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# trantestEAE - test translate EBCDIC to ASCII & ASCII to EBCDIC
# - for Vancouver Utilities uvcopy
# - by Owen Townsend Oct/2014
#
# uvcopy trantestEAE
# ==================
# - generates 256 lines translate results for each character
# - see template further below
#
opr='$jobname - test EBCDIC to ASCII translate for Vancouver Utilities'
opr=' - generates 256 lines translate results for each character'
filo1=?tmp/trantestEAE_results,rcs=80,typ=LSTt
#
# load table of page headings
lodv1=h0(100)
$jobname - test uvcopy translate EBCDIC to ASCII & back to EBCDIC
- generates 256 lines translate results for each character
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# load table with template
# 1 2 3 4 5 6
#123456789012345678901234567890123456789012345678901234567890
lodv1=k0(100)
x'00' --> x'00' --> x'00' c'_' ---> c'_' ---> c'_'
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@run
opn all
wtbe filo1,h0(100),h0(100) print headings
#
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# begin loop to print 256 lines
man20 mvc a0(80),k0 template
mvn a32(1b),$ca1 current value x'00' --> x'FF'
mvn a42(1b),$ca1 copy value for tra
tra a42(1) translate EBCDIC to ASCII
mvc a52(1),a42 copy tra for tre
tre a52(1) translate ASCII to EBCDIC
hxc a02(2),a32(1) convert EBCDIC char to hex rep
hxc a12(2),a42(1) convert ASCII char to hex rep
hxc a22(2),a52(1) convert retran EBCDIC to hex rep
#
# convert control chars < x'20' to '^X' format, x'01' --> '^A' etc
cmc a52(1),x'20' control char ?
skp=> 2
orc a52(1),x'40' convert x'01' to x'41' etc
ins a52(40),'^' insert '^'
#
cmc a42(1),x'20' control char ?
skp=> 2
orc a42(1),x'40' convert x'01' to x'41' etc
ins a42(40),'^' insert '^'
#
cmc a32(1),x'20' control char ?
skp=> 2
orc a32(1),x'40' convert x'01' to x'41' etc
ins a32(40),'^' insert '^'
#
# write to report, increment,& return for next char until 256
put filo1,a0(80) print
add $ca1,1 increment to next char of 256
cmn $ca1,256 end 256 ?
skp< man20
#
# End 256 chars - close file & end job
man90 cls all
eoj
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
In our preparation above (on page '7B1'), we simply copied a supplied file from $UV/dat1/bytes256. But you could create as follows using a supplied uvcopy job 'makebytes256' (listed on the next page).
#1. Login userxx --> /home/userxx #2. mkdir cnvtran <-- setup directory for tests #3. cd cnvtran <-- chnage into directory #4. cnvdatadirs <-- make subdirs required #5a. cp $UV/dat1/bytes256 d1ebc/Ebytes <-- copy/rename to consider #5b. cp $UV/dat1/bytes256 d2asc/Abytes <-- copy/rename to consider
Instead of #5a & 5b above, you could run uvcopy job 'makebytes256'
#5. uvcopy makebytes256,filo1=tmp/bytes256
======================================
#5a. uvcopy makebytes256 <-- same, outputfile defaults as above
===================
#6a. cp $UV/dat1/bytes256 d1ebc/Ebytes <-- copy/rename to consider as EBCDIC
=================================
#6b. cp $UV/dat1/bytes256 d2asc/Abytes <-- copy/rename to consider as ASCII
=================================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# makebytes256 - uvcopy job to create bytes256
# - 1 record file of all possible bit combinations in 1 byte
# - by Owen Townsend, Oct 2014
#
# uvcopy makebytes256,filo1=tmp/bytes256
# ======================================
# - generate 256 byte record containing all bit combinations in a byte
#
# uvcopy makebytes256 <-- same, output file defaults as above
# ===================
#
opr='$jobname - generate 256 bytes containing all bit combinations in a byte'
filo1=?tmp/bytes256,rcs=256,typ=RSF
@run
# use loop to increment 1 byte x'00' - x'FF', appending on output record
# - will increment $rb (register 'b') & append to area 'b' via $rb
# - initially binary zero when uvcopy begins
# - note that bb0 is 'area b' offset by 'register b'
# - note that (1b) is length 1 byte, but the 'b' means 'binary' vs char
man20 mvn bb0(1b),$rb append current $rb value to area 'b'
add $rb,1 increment register 'b' ($rb)
cmn $rb,256 reached max value ?
skp< man20 repeat loop until 256
#
man90 opn filo1 open outfile
put filo1,b0(256) write record to outfile
cls filo1 close output file
eoj end job
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Here is an alternate way to create bytes256 from a neutral translate table. VU supplies translate table at $UV/srcf/trtneut.c You could copy it to your homedir/tmp, compile,& extract the 256 bytes as follows:
#1. Login userxx --> /home/userxx
#2. mkdir src bin data <-- make subdirs
#3. cp $UV/srcf/trtneut.c src
#4. cc -c trtneut.c -o bin/trtneut.o <-- compiles to trtneut.o
================================
#5. uvhd bin/trtneut.o <-- investigate object code
================== - note 64 byte header & extra training bytes
#6. dd if=bin/trtneut.o of=data/bytes256 bs=64 skip=1 count=4
=========================================================
extract the 256 bytes x'00' - x'FF' from object code
#7. uvhd data/bytes256 <-- investigate 256 bytes extracted
================== - confirm 1st 64 dropped & excess dropped
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#8. vi src/trtneut.c
================
/* trtneut.c - translate table - neutral, use as basis to create others */
unsigned char trtneut[256] = {
0x00,0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09,0x0A,0x0B,0x0C,0x0D,0x0E,0x0F,
0x10,0x11,0x12,0x13,0x14,0x15,0x16,0x17,0x18,0x19,0x1A,0x1B,0x1C,0x1D,0x1E,0x1F,
0x20,0x21,0x22,0x23,0x24,0x25,0x26,0x27,0x28,0x29,0x2A,0x2B,0x2C,0x2D,0x2E,0x2F,
0x30,0x31,0x32,0x33,0x34,0x35,0x36,0x37,0x38,0x39,0x3A,0x3B,0x3C,0x3D,0x3E,0x3F,
0x40,0x41,0x42,0x43,0x44,0x45,0x46,0x47,0x48,0x49,0x4A,0x4B,0x4C,0x4D,0x4E,0x4F,
0x50,0x51,0x52,0x53,0x54,0x55,0x56,0x57,0x58,0x59,0x5A,0x5B,0x5C,0x5D,0x5E,0x5F,
0x60,0x61,0x62,0x63,0x64,0x65,0x66,0x67,0x68,0x69,0x6A,0x6B,0x6C,0x6D,0x6E,0x6F,
0x70,0x71,0x72,0x73,0x74,0x75,0x76,0x77,0x78,0x79,0x7A,0x7B,0x7C,0x7D,0x7E,0x7F,
0x80,0x81,0x82,0x83,0x84,0x85,0x86,0x87,0x88,0x89,0x8A,0x8B,0x8C,0x8D,0x8E,0x8F,
0x90,0x91,0x92,0x93,0x94,0x95,0x96,0x97,0x98,0x99,0x9A,0x9B,0x9C,0x9D,0x9E,0x9F,
0xA0,0xA1,0xA2,0xA3,0xA4,0xA5,0xA6,0xA7,0xA8,0xA9,0xAA,0xAB,0xAC,0xAD,0xAE,0xAF,
0xB0,0xB1,0xB2,0xB3,0xB4,0xB5,0xB6,0xB7,0xB8,0xB9,0xBA,0xBB,0xBC,0xBD,0xBE,0xBF,
0xC0,0xC1,0xC2,0xC3,0xC4,0xC5,0xC6,0xC7,0xC8,0xC9,0xCA,0xCB,0xCC,0xCD,0xCE,0xCF,
0xD0,0xD1,0xD2,0xD3,0xD4,0xD5,0xD6,0xD7,0xD8,0xD9,0xDA,0xDB,0xDC,0xDD,0xDE,0xDF,
0xE0,0xE1,0xE2,0xE3,0xE4,0xE5,0xE6,0xE7,0xE8,0xE9,0xEA,0xEB,0xEC,0xED,0xEE,0xEF,
0xF0,0xF1,0xF2,0xF3,0xF4,0xF5,0xF6,0xF7,0xF8,0xF9,0xFA,0xFB,0xFC,0xFD,0xFE,0xFF
};
#9. uvhd data/bytes256 <-- investigate 256 bytes extracted
================== - confirm 1st 64 dropped & excess dropped
rec#=1 rsize=256 fptr=0 fsize=256 rcount=1
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ................................ !"#$%&'()*+,-./0123456789:;<=>?
0000000000000000111111111111111122222222222222223333333333333333
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
64 @ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^abcdefghijklmnopqrstuvwxyz{|}~.
4444444444444444555555555555555566666666666666667777777777777777
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
128 ................................................................
88888888888888889999999999999999AAAAAAAAAAAAAAAABBBBBBBBBBBBBBBB
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
192 ................................................................
CCCCCCCCCCCCCCCCDDDDDDDDDDDDDDDDEEEEEEEEEEEEEEEEFFFFFFFFFFFFFFFF
0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF0123456789ABCDEF
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 8A1. | Introduction to Variable Length RDW files |
| use FTP option 'quote site RDW' to transfer variable length files | |
| from the mainframe to unix/linux. | |
| 8A2. |
| 8B1. | test/demo files supplied by UV Software |
| 8B2. | testRDWe test file to demo RDW files |
| 8C1. | Investigating RDW files with 'uvhd' |
| 8D1. | converting EBCDIC RDW files to ASCII - with 'uvhd' |
| - uvhd interactive utility, easy to use |
| 8E1. | converting EBCDIC RDW files to ASCII - with 'uvcp' |
| 8F1. | converting EBCDIC RDW files to ASCII - with 'uvcopy varfix11' |
| - varfix11 batch utility, better for high volume conversions |
| 8G1. | creating table summary stats of record sizes in variable length files |
| uvcopy job 'varstat1' Operating Instructions & sample report |
| 8X0. | Listings of uvcopy jobs used in Part 8 |
| - varstat1, varfix11, | |
| - LST2RDW1, RDW2LST1, varfix11 |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
RDW (Record Descriptor Word) variable length files are often used to FTP variable length EBCDIC files from the mainframe to unix/linux/windows systems. uvcp provides for 2 types of RDW files - standard & non-standard (option z2).
We will supply the variable length data files used for the demos here, but if you are FTP'ing variable length files from your mainframe, please note that you must use FTP option 'quote site RDW' to transfer variable length files from the mainframe to unix/linux.
#1. Login --> your homedir #2. cd d0ebc <-- change into the data subdir
#3. ftp xxx.xxx.xxx.xxx <-- FTP to mainframe IP#
===================
#3a. userid--> ..... <-- enter userid
#3b. passwd--> ..... <-- enter password
#3c. binary <-- ensure binary transfer
#3d. quote site RDW <-- required if any variable length files
#3e. get XXX.XXX.XXX <-- get desired data files
#3f. get ...etc.... - 'single quotes' may be required
'quote site RDW' is reqquired for FTP to insert the 4 byte record prefixes with the record-length in binary. Without the RDW record profixes, we can not determine where 1 record ends & the next begins.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Before FTP'ing files from mainframe to Unix/Linux, we recommend converting the Fixed-Length files to Variable-Length so we can use the RDW option. Then, when we receive the files on unix/linux, we will immediately know the record-sizes of all files & also know which were originally Fixed because their minimum & maximum record sizes will be the same. Here is a skeleton JCL that could be used to convert Fixed to Variable on the mainframe.
//* JCL to convert Fixed-Length to Variable-Length
//SORT001 EXEC PGM=SORT
//SYSUDUMP DD SYSOUT=K,HOLD=YES
//SYSOUT DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//SORTIN DD DSN=AAAAA.BBBBB.CCC.DDDDD,
// DISP=(SHR,KEEP,KEEP)
//SORTOUT DD DSN=AAAAA.BBBBB.CCC.DDDDD.X,
// DISP=(NEW,CATLG,DELETE),UNIT=SYSDA,
// SPACE=(TRK,(0500,0500),RLSE),
// DCB=(RECFM=VB,LRECL=32000)
//SYSIN DD *
SORT FIELDS=COPY
OUTFIL FNAMES=SORTOUT,FTOV
/*
//*Note - 'FTOV' means Fixed TO Variable
//* - 'VTOF' could also convert Variable TO Fixed
You could then run FTP with RDW on the mainframe with the following JCL, or you could run FTP on Unix/Linux as shown on the next page.
//* JCL to FTP Variable-Length with RDW to Unix/Linux //ACHP01B EXEC PGM=FTP,PARM='(TRAN TRTAUGE1' //OUTPUT DD SYSOUT=* //SYSPRINT DD SYSOUT=* //INPUT DD * 30.252.99.11 21 (exit userid password cd /p2/cnv1/cnvdata1/d0ebc/ pwd type i LOCSITE RDW PUT 'AAAAA.BBBBB.CCC.DDDDD.X' AAAAA.BBBBB.CCC.DDDDD QUIT /*
Also see https://uvsoftware.ca/datacnv2.htm#5C1 - 5C6, which show how you can automatically generate the JCL to convert Fixed to Variable from a control-file list of filenames to be transferred to Unix/Linux. 'genF2Vjcl1' is a Vancouver Utility which runs on unix/linux, so you need to FTP the generated JCL to the mainframe for execution.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| dat1/testLST |
|
| dat1/testRDW |
|
| dat1/testRDWe |
|
We will illustrate how to convert RDW EBCDIC files to text or fixed length files using: uvhd, uvcp,& uvcopy.
We will 1st investigate the testRDWe file using 'uvhd'. We can not use unix tools (vi, lp, etc) since the file is EBCDIC & contains binary record sizes.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd dat1/testRDWe as3 <-- examine the RDW test file
====================== - option 'a' translates character line to ASCII
- option 's3' to space between scale & 3 line groups
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ....DELL00 - Dell Inc. ....HP0000 - Hewlett Packard....IBM000 -
0100CCDDFF464C8994C984440100CDFFFF464C8A98AA4D8898980200CCDFFF46
080045330000045330953B000C008700000008563533071321940C0092400000
64 International Business Machines....MFC000 - Micro Focus COBOL
4C9A8998A899894CAA898AA4D888898A0200DCCFFF464D88994C98AA4CDCDD44
0953595139651302429552204138955200004630000004939606634203626300
128 ....MS0000 - Microsoft Corp.....REDHAT - Red Hat Linux ....SUN0
0100DEFFFF464D8899A98A4C99940100DCCCCE464D884C8A4D89AA440200EEDF
0C0042000000049396266303697B0C0095481300095408130395470004002450
192 00 - Sun Microsystems Ltd ....UVSI00 - UV Software Inc.
FF464EA94D8899AAAA89A4DA84440200EEECFF464EE4E98AA8984C984444
00000245049396282354203340000000452900000450266361950953B000
The entire file is only 252 bytes & contains 8 short variable length records:
You cannot display RDW files with 'vi', because vi cannot handle binary, there are no LineFeeds to separate the records, so the entire file appears as 1 long line (unless there just happened to be a x'0A' in the length field, which would be interpreted as a LineFeed by vi).
OPtion 'z' would tell uvhd to look for the 'RDW' record prefixes & show 1 record at a time, but we did not specify option z above, and the default is to show any file in 256 byte blocks (4 groups of 3 64 byte lines for characters, zones,& digits).
See the next page where we will specify option 'z' to show RDW files 1 record at a time. You can press enter to browse forward until EOF reached. Then you could enter '1' to return to the begining of the file.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvhd dat1/testRDWe z4a <-- display RDWz4 file using option 'z'
====================== - option 'a' to display character line in ASCII
- null entries browse forward until EOF reached
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ....DELL10 - Dell Inc.
0100CCDDFF464C8994C98444
080045331000045330953B00
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
24 ....HP0010 - Hewlett Packard
0100CDFFFF464C8A98AA4D889898
0C00870010000856353307132194
10 20 30 40 50 60
r# 3 0123456789012345678901234567890123456789012345678901234567890123
52 ....IBM010 - International Business Machines
0200CCDFFF464C9A8998A899894CAA898AA4D888898A
0C009240100009535951396513024295522041389552
10 20 30 40 50 60
r# 4 0123456789012345678901234567890123456789012345678901234567890123
96 ....MFC010 - Micro Focus COBOL
0200DCCFFF464D88994C98AA4CDCDD44
00004630100004939606634203626300
10 20 30 40 50 60
r# 5 0123456789012345678901234567890123456789012345678901234567890123
128 ....MS0010 - Microsoft Corp.
0100DEFFFF464D8899A98A4C9994
0C0042001000049396266303697B
10 20 30 40 50 60
r# 6 0123456789012345678901234567890123456789012345678901234567890123
156 ....REDH10 - Red Hat Linux
0100DCCCFF464D884C8A4D89AA44
0C00954810000954081303954700
10 20 30 40 50 60
r# 7 0123456789012345678901234567890123456789012345678901234567890123
184 ....SUN010 - Sun Microsystems Ltd
0200EEDFFF464EA94D8899AAAA89A4DA8444
040024501000024504939628235420334000
10 20 30 40 50 60
r# 8 0123456789012345678901234567890123456789012345678901234567890123
220 ....UVSI10 - UV Software Inc.
0200EEECFF464EE4E98AA8984C984444
0000452910000450266361950953B000
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
On this page we will show you how to convert EBCDIC RDWz4 files to ASCII text using 'uvhd'. On following pages we will do the same with 'uvcp' & 'uvcopy'.
We will specify uvhd options 'za3p4y7' which mean:
| z4 |
|
| a3 |
|
| a1 |
|
| a2 |
|
| p4 |
|
| y7 |
|
| y1 |
|
| y2 |
|
| y4 |
|
uvhd dat1/testRDWe z4a3p4y7 <-- display 1st record & wait for command ===========================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 ....DELL10 - Dell Inc.
0100CCDDFF464C8994C98444
080045331000045330953B00
---> w99999 <-- write all records (8) to tmp/testRDWe_yymmdd_hhmmssW
- output filename will be date/time stamped
- On Dec 18, 20007 at 12:15 tmp/testRDWe_071218_121500W
---> q <--- quit uvhd
vi tmp/testRDWe_071218_121500W ============================== vi tmp/*00W <-- shortcut to display desired file ===========
DELL00 - Dell Inc.
HP0000 - Hewlett Packard
IBM000 - International Business Machines
MFC000 - Micro Focus COBOL
MS0000 - Microsoft Corp.
REDHAT - Red Hat Linux
SUN000 - Sun Microsystems Ltd
UVSI00 - UV Software Inc.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We will now use 'uvcp' to perform the conversion from EBCDIC RDWz4 to ASCII.
Instead of variable length text, we will create fixed length output records that are required for most COBOL input files. For this demo, we will convert to fixed 64 byte records & insert a LineFeed in the last byte so we can have it both ways (fixed length for COBOL, but LFs allow investigation with vi).
uvcp "fili1=dat1/testRDWe,rcs=64,typ=RDWz4,filo1=tmp/testz4,typ=RST,tra=0(64)" ==============================================================================
| tra=0(64) |
|
| typ=RST |
|
uvhd tmp/testz4 r64h2 <-- display output file with uvhd
===================== - option r64 for record size
- option 'h2' for hex display to see LineFeed
- only 1st record shown below
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 DELL10 - Dell Inc. .
4444332224666246622222222222222222222222222222222222222222222220
45CC100D045CC09E3E000000000000000000000000000000000000000000000A
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'varfix11' is a uvcopy utility job to convert an RDW variable length file to a specified fixed length (large enough to hold largest variable record).
On your mainframe migration, you may have EBCDIC RDW & BDW/RDW files to be transferred to & converted on your unix/linux system.
For a mainframe migration you would not want to use the interactive 'uvhd' for hundreds of files. Here we will illustrate a batch utility 'varfix11' that you could use to convert varlth RDW EBCDIC files to ASCII fixed records.
Note that this job can not be used for EBCDIC files with packed/binary fields. For those you need the procedures documented in 'Part_3' of this DATAcnv1.doc or in JCLcnv2real.htm#Part_3.
uvcopy varfix11,fili1=dat1/testRDWe,filo1=tmp/testRDWaf,uop=a1b1h1r64t1 =======================================================================
| Note |
|
uop=a0b0h1r2048t0 - option defaults
a0 - no translate (input EBCDIC with packed/binary fields)
a1 - translate from EBCDIC to ASCII
b0 - do NOT convert nulls to blanks
b1 - DO convert nulls to ASCII blanks
b2 - DO convert nulls to EBCDIC blanks
h0 - drop the 4 byte binary record-size headers
h1 - replace 4 byte binary recsize hdr with numeric chars
r2048 - output fixed records 2048 bytes
r8192 - max output fixed size 8192 bytes
t0 - do NOT insert LineFeed in last byte of record
t1 - DO insert LineFeed in last byte of record
null to accept or re-specify (1 or more) -->
071128:172717:varfix11: EOF fili01 rds=16 size=252: dat1/testRDWe
071128:172717:varfix11: EOF filo01 wrts=8 size=512: tmp/testRDWaf
cat tmp/testRDWaf <-- display output file =================
0024DELL10 - Dell Inc. 0028HP0010 - Hewlett Packard 0044IBM010 - International Business Machines 0032MFC010 - Micro Focus COBOL 0028MS0010 - Microsoft Corp. 0028REDH10 - Red Hat Linux 0036SUN010 - Sun Microsystems Ltd 0032UVSI10 - UV Software Inc.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'varstat1' is a uvcopy job that you can run to create a summary table of record-sizes found in all records of a variable file typ=RDWz4 or typ=RDWz2 (variable length records with 4 byte or 2 byte prefixes).
#1. cd $UV <-- change to /home/uvadm
#2. uvcopy varstat1,fili1=d0ebc/testRDW,filo2=rpts/testRDW_recsizes
===============================================================
#3. vi rpts/testRDW_recsizes <-- inspect report
========================
varstat1 2007/12/19_18:10:29 record-sizes in dat1/testRDW tbl#001 pg#001 -argument- line# count % record-sizes
1 1 12 00020
2 3 37 00024
3 2 25 00028
4 1 12 00032
5 1 12 00040
8*100 *TOTAL*
#1. cd $CMPDATA <-- change to file compare superdir
#2. mkdir tmp1 <-- make tmp1 subdir if not existing #2a. rm -f tmp1/* <-- OR remove all files if tmp1 already exists
#3. uvcopyx varstat1 d0ebc tmp1 uop=q0i7,rop=r0
===========================================
- create table summary recsize counts for all files in d0ebc subdir
- output reports in tmp1 with same names as datafiles in d0ebc
#4. uvlpd12 tmp1 <-- print all reports in tmp subdir
============
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The 'uvcopy jobs' used in this section are listed on the following pages. You might need to modify them for complex variable length conversions.
| 8X1. | varstat1 - create table summary of record sizes in variable length files |
| - listed further below |
| 8X2. | varfix11 - convert variable length BDW/RDW files to Fixed Length files |
| - might be used to convert mainframe EBCDIC files to ASCII | |
| - BUT not if packed/binary present (see Part_3). |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# varstat1 - create summary table of record-sizes used in variable file # - for variable lth records with 4 byte hdrs (standard RDW) # - by Owen Townsend, Dec 12, 2007 #Dec19/07 - replace getvr2 subrtn with file typ=RDWz4 & RDWz2 (optional) # # ** create report for 1 file (for testing) ** # # uvcopy varstat1,fili1=d0ebc/datafilename,filo2=tmp1/reportname # ============================================================== # # ** create reports for all files in directory ** # # 1. cd $CNVDATA <-- change to conversion superdir # - subdir d0ebc contains EBCDIC var lth files # # 2. mkdir tmp1 <-- make tmp1 subdir if not existing # 2a. rm -f tmp1/* <-- OR remove all files if tmp1 already exists # # 3. uvcopyx varstat1 d0ebc tmp1 uop=q0i7,rop=r0 # =========================================== # - create table summary recsize counts for all files in d0ebc subdir # - output reports in tmp1 with same names as datafiles in d0ebc # # 4. uvlpd12 tmp1 <-- print all reports in tmp subdir # ============ # # ** sample report ** # # varstat1 2006/12/17_18:15:17 record-sizes in d0ebc/E2121656 # tbl#001 pg#001 -argument- # line# count % record-sizes # 1 10,552 16 00046 # 2 4,451 7 00065 # 3 23,347 37 00066 # 4 367 0 00068 # 5 21,010 33 00083 # - - - etc - - - # 18 3 0 00218 # 19 441 0 00233 # 20 813 1 00239 # 62,115*100 *TOTAL* #
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# This job designed for variable lth records typ=RDW & typ=RDWz2
# typ=RDW (standard RDW) - 4 byte record prefixes
# - binary recsize in 1st 2 bytes (3rd & 4th x'0000' unused)
# - binary recsize of data (includes the 4 byte prefix)
# - blk/rec lth binary BIG-END format, need 's'witch option for INTEL
# typ=RDWz2 (non-std RDW) - 2 byte record prefixes (2 nulls omitted)
# - binary recsize does not include the 2 byte prefix
#
opr='$jobname - create summary table of record-sizes used in variable file'
opr='uop=q1z0 - option defaults (message only, see uop=... in job)'
opr=' z0 - RDW 4 byte prefix, recsize 1st 2 BIG-END binary'
opr=' z2 - RDW 2 byte prefix, recsize 1st 2 BIG-END binary'
uop=q1z0 # option defaults
rop=r1 # option to prompt for report disposition
was=a16384
fili1=?d0ebc/input,rcs=8192,typ=RDW #Dec19/07 typ=RDW replaces getvr
filo1=?tmp1/$jobname.rpt,typ=LSTt,rcs=128
@run
#Dec19/07 - replace subrtn getvr with typ=RDW & RDWz2
# - for nonstd 2 byte prefix vs std 4 bytes
# if uop=z2, append file option z2 for typ=RDWz2
tsb o26(1),x'02' uop z2 for nonstd RDW ?
skp! 1
cata8 $fili1+180(15),'z2'
opn all
#
# begin loop to get records & build summary table of recsizes
# - to be dumped at end of file
man20 get fili1,a0 get next RDW record data
skp> eof (cc set > at EOF)
mvn c0(5),$rv cnvrt binary recsize to numeric
tblt1f1 c0(5),'record-sizes' build table in memory
skp man20 repeat get loop until EOF
#
# EOF - close files & end job
eof mvfv1 f0(80),'record-sizes in $fili1'
tbpt1s1 filo1,f0(50) dump table from memory to file
cls all
eoj
#
# getvr - subrtn to get records from IBM std variable length file
#Dec19/07 - getvr replaced by file typ=RDW & typ=RDWz2
# - subrtn saved (for interest) in $UV/pf/util/getvr
## folwng instrns modified (see above) for typ=RDW
##man20 bal getvr set rgstrs a & b to next record
## mvn c0(5),a0(2bs) cnvrt binary recsize to numeric
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# varfix11 - convert RDW variable length file test Aug 24/05 # - convert RDW variable length file to fixed length records # - prior to 2nd job that will convert EBCDIC to ASCII # OR compare 2 files uvcmpVA1/uvcmpVE1 (fixlth max 4096) # - by Owen Townsend, UV Software # - originally for Laval, Dec 2005 # #Dec19/07 - replace subrtn getvr with typ=RDW & add option z2 # for nonstd 2 byte prefix vs std 4 bytes (binary recsize 1st 2) # - subrtn code preserved in $UV/pf/util/getv for interest # # Enhanced Oct 2007 for Sungard, options added # - a1=translate to ASCII, h1=cnvt hdr to numerics, r4096=fixed output recsize # - b1=convert nulls to ASCII blanks, t1=terminate with LineFeed # # ** Operating Instructions ** # # uvcopy varfix11,fili1=d0ebc/infile,filo1=d1ebc/outfile,uop=a1h1r4096 # ==================================================================== # - convert data file from variable to fixed 4096 byte records # # option a1 - translate from EBCDIC to ASCII # option h1 - replace 4 byte binary recsize hdr with recsize numerics # option r4096 - output fixed records 4096 bytes # - large enough to hold largest variable lth record # # This 1 uvcopy job 'varfix11' can be used for all data files # - to convert from variable to fixed # - drops 4 byte variable recsize headers, or retain converted to numerics # - null fill records to fixed recsize spcfd by option r # # ** uses for varfix11 ** # # Convert variable length to fixed length to facilitate: # # 1. CONVERTING datafiles from EBCDIC to ASCII (preserving packed/binary) # - could convert back to variable length after conversion # - see examples in VSEDATA.doc # # 2. COMPARING 2 datafiles # - see script uvcmpVE1 to compare 2 varlth EBCDIC datafiles # - this varfix11 used to convert both files to fixed max lth in tmp1/tmp2 # - followed by utility 'uvcmp1' to compare & create report in rpts/... #
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# ** sample variable-length RDW files ** # # uvhd dat1/testRDWe a <-- display demo file EBCDIC varlth RDWz4 # ==================== - option 'a' to show char lines in ASCII # (zones & digits lines show EBCDIC codes) # # 10 20 30 40 50 60 # r# 1 0123456789012345678901234567890123456789012345678901234567890123 # 0 ....DELL10 - Dell Inc. ....HP0010 - Hewlett Packard....IBM010 - # 0100CCDDFF464C8994C984440100CDFFFF464C8A98AA4D8898980200CCDFFF46 # 080045331000045330953B000C008700100008563533071321940C0092401000 # 64 International Business Machines....MFC010 - Micro Focus COBOL # 4C9A8998A899894CAA898AA4D888898A0200DCCFFF464D88994C98AA4CDCDD44 # 0953595139651302429552204138955200004630100004939606634203626300 # 128 ....MS0010 - Microsoft Corp.....REDH10 - Red Hat Linux ....SUN0 # 0100DEFFFF464D8899A98A4C99940100DCCCFF464D884C8A4D89AA440200EEDF # 0C0042001000049396266303697B0C0095481000095408130395470004002450 # 192 10 - Sun Microsystems Ltd ....UVSI10 - UV Software Inc. # FF464EA94D8899AAAA89A4DA84440200EEECFF464EE4E98AA8984C984444 # 10000245049396282354203340000000452910000450266361950953B000 # # uvhd dat0/testRDWe z4a1 <-- option 'z4' to process RDWz4 files # ======================= - 'a1' to translate char line to ASCII # - hexadecimal zones & digits show EBCDIC values # # 10 20 30 40 50 60 # r# 1 0123456789012345678901234567890123456789012345678901234567890123 # 0 ....DELL10 - Dell Inc. # 0100CCDDFF464C8994C98444 # 080045331000045330953B00 # # Note - uvhd displays 1st record & prompts for commands # - enter w999 to write all records & q to quit: # # --> w999a1t6r64 <-- Write all records to the tmp/... subdir # - filename date/time stamped ex: tmp/testRDWe_yymmdd_hhmmssW # - On Nov 17/07 3:30 PM might be: tmp/testRDWe_071117_153059W # a1 - translate output records to ASCII # t6 - insert LineFeed terminators after last non-blank # (t6=t2+t4, t2=LineFeed, t4=insert after last non-blank) # r64 - max size for output records # - 'y7' to insert CR+LF after last non-blank # --> q <-- quit uvhd # # cat tmp/*59W <-- display output text records # ============ - use *59W to save a lot of keystrokes # # DELL10 - Dell Inc. # HP0010 - Hewlett Packard # IBM010 - International Business Machines # MFC010 - Micro Focus COBOL # MS0010 - Microsoft Corp. # REDH10 - Red Hat Linux # SUN010 - Sun Microsystems Ltd # UVSI10 - UV Software Inc. #
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
opr='$jobname - convert variable length file to fixed length records'
opr='uop=a0b0h1r4096t0z4 - option defaults'
opr=' a0 - no translate (for EBCDIC packed/binary fields)'
opr=' a1 - translate from EBCDIC to ASCII'
opr=' b0 - do NOT convert nulls to blanks'
opr=' b1 - DO convert nulls to ASCII blanks'
opr=' b2 - DO convert nulls to EBCDIC blanks'
opr=' h0 - drop the 4 byte binary record-size headers'
opr=' h1 - replace 4 byte binary recsize hdr with numeric chars'
opr=' r4096 - output fixed records 4096 bytes'
opr=' r8192 - max output fixed size 8192 bytes'
opr=' t0 - do NOT insert LineFeed in last byte of record'
opr=' t1 - DO insert LineFeed in last byte of record'
opr=' z4 - RDW 4 byte prefix, recsize 1st 2 BIG-END binary'
opr=' z2 - RDW 2 byte prefix, recsize 1st 2 BIG-END binary'
uop=q1a0b0h1r4096t0z4 # option defaults
was=a32768b32768c32768 # increase areas a,b,c from dflt 1024
fili1=?d0ebc/filename,rcs=8192,typ=RDWz4 # uop z2 will change to option z2
filo1=?d1ebc/filename,rcs=8192,typ=RSF
@run
#Dec19/07 - replace subrtn getvr with typ=RDWz4 & typ=RDWz2
# - typ=RDWz2 for nonstd 2 byte prefix vs std 4 bytes
# if uop=z2, change file option z4 to z2
tsb o26(1),x'02' uop z2 for nonstd RDW ?
skp! 1
cata8 $fili1+180(15),'z2'
#
opn all open files
mvn $rf,$uopbr load fixed outsize in rgstr 'f'
mvn $rg,$rf transfer to $rg
sub $rg,1 -1 byte to insert LF in last byte
#
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# begin loop to read variable length records until EOF
# - using get typ=RDW to get var lth records depending on size in prefix
# - write fixed length records padded to max 8192 with EBCDIC blanks
##man20 bal getvr <--Dec19/07 change to get file typ=RDW
man20 get fili1,b0 get next record depending on varlth prfx
skp> eof
#
# test option to translate EBCDIC to ASCII
tsb o1(1),x'01' trnslt EBCDIC to ASCII ?
skp! 1
tra b0($rf8192)
#
# presume option h1 to replace 4 byte recsize hdr with numerics
## mvn c0(4),a0(2bs) <--Dec19/07 chg to $rv
mvn c0(4),$rv cnvt binary recsize to ASCII numerics
tsb o1(1),x'01' trnslt EBCDIC to ASCII ?
skp= 1
tre c0(4) no - trnslt ASCII recsize to EBCDIC
mvc c4($rf8192),b0 move data to follow recsize
#
# test option h0/h1 drop recsizehdr or replace with numerics ?
tsb o8(1),x'01' replace recsize hdr w numerics ?
skp= 1
mvc c0($rf8192),b0 no - drop recsize hdr
#
# test option b1 to convert nulls to ASCII blanks
man40 tsb o2(1),x'01' convert nulls to ASCII blanks ?
skp! 1
rep c0($rf8192),x'00',x'20'
# test option b2 to convert nulls to EBCDIC blanks
tsb o2(1),x'02' convert nulls to EBCDIC blanks ?
skp! 1
rep c0($rf8192),x'00',x'40'
#
# test option t1 to insert Line-Feed Terminator in last byte of record
tsb o20(1),x'01' insert LF in last byte ?
skp! 1
mvc cg0(1),x'0A' Yes - insert LF in last byte
#
# write current record to output file & return to get next
man48 put filo1,c0($rf8192) write record to file#1 out
skp man20 return to get next record
#
# EOF - close files & end job
eof cls all
eoj
#
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# getvr - subrtn to get RDW variable length records
#Dec19/07 - getvr subrtn no longer required
# - getvr replaced by file typ=RDW & typ=RDWz2
# - this subrtn saved (for interest) in $UV/pf/util/getvr
#
# - RDW format with no BDW
# - record length in 1st 2 bytes of 4 with last 2 x'0000'
# - record length includes data length + 4 bytes for RDW prefix
# - blk/rec lth in binary big end format, need 's'witch option for INTEL
#
# - read record into area 'a' (1st 4 bytes will be RDW)
# - sets rgstr 'a' to record length (including the 4 byte RDW)
# - sets rgstr 'b' to record length (excluding the 4 byte RDW)
# - return with cc > at EOF
#
# note - see alternate job BDW2LST1 to handle BDW+RDW
# - includes subrtn 'getvbr' to process BDW+RDW
#
# read 4 byte RDW to get record length (-4 for following data)
getvr get fili1,a0(4) read 4 byte block lth prefix
skp> getvr9
cmc a2(2),x'0000' verify RDW (bytes 3&4 null)
skp! getvre1
add $ca9,1 count blocks (in case errmsg)
mvn $ra,a0(2bs) store rec lth (switch ends INTEL)
mvn $rb,$ra copy to $rb for subtract 4
sub $rb,4 -4 for following data length
skp<= getvre1 (err if 0 or negative)
#
# now read the data following 4 byte RDW
getvr4 get fili1,a4($rb8192) read block (lth in rgstr c)
skp> getvr9
#
# exit with cc = for normal, or cc > for EOF
getvr8 ret=
getvr9 ret>
#
# error rtn for invalid block or record prefix
getvre1 msgv1 'rec prefix 3&4 not x0000, reclth=$ra'
mvf g0(80),'1st 32 bytes='
hxc g13(64),a0(32) convert 1st 32 bytes to hex
msg g0(80) display block prefix
can 'unrecoverable error - job cancelled'
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 9A1. | Converting ASCII files back to EBCDIC (for 1 file at a time) |
| - uvcopy job 'uvdata55' | |
| - sample conversion for demo file 'citytax1' |
| 9B1. | Data File Comparison utility (uvcmp1) |
| - use to verify EBCDIC to ASCII file conversions, by converting | |
| ASCII file back to EBCDIC & comparing to original EBCDIC file |
| 9C1. | Generate ASCII-->EBCDIC jobs - for ALL data-files (vs 1 at a time) |
| - instructions are shorter & simpler than for 1 at a time |
| 9D1. | Verifying Data Conversion with 'uvhdcob' |
| - by verifing packed & numeric fields |
| 9E1. | Verifying Data Conversion with 'genverify1' |
| - generate uvcopy jobs to Verify all Packed & Numeric fields |
| 9F1. | Demonstrate 'genverify1' verifies ALL packed/numeric fields of 'occurs' |
| vs 'uvhdcob' which only verify the 1st field of any 'occurs' | |
| using armaster test file which has 12 months sales for this year & last. |
| 9G1. | genacum2 - generate uvcopy jobs to accumulate all numeric fields in a file. |
| - generated automatically from the 'cobmap' record layout | |
| (which is generated from the COBOL copybook). |
| 9H1. | gentest1 - generate test data files automatically |
| - based on COBOL copybooks | |
| - inserts cobol copybook fieldnames in type X fields in every 10th record | |
| - otherwise, inserts 'A's in 1st field, 'B's in 2nd, etc | |
| - inserts current record# in numeric & packed fields |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Part_3 documented generating & executing uvcopy jobs to convert EBCDIC to ASCII - 1 job at a time. Part_6 documented the same thing for ALL copybooks & ALL data files in a dirctory.
We will now show you how to generate & execute jobs to convert the ASCII files back to EBCDIC (& then compare the reconverted files to the original mainframe files).
You might do this to verify your conversions &/or you might need the ASCII to EBCDIC conversions to return some files to the maifnrame if you can not switch all applications over to unix/linux at the same time.
We could generate the ASCII to EBCDIC jobs using 'uvdata31' the opposite of 'uvdata51' that we used to generate EBCDIC to ASCII, but it is much better to use 'uvdata55' to convert our already generated EBCDIC-->ASCII jobs to ASCII-->EBCDIC jobs.
Besides saving steps, the big advantage is that this method saves having to duplicate any manual coding required for files with multiple Record Types. With the addition of uvdata55, pfx4,& uvcmp1all, our flow charts will then be:
cpys -------> maps --------> pfx1 --------> pfx2 ---------> pfx3 -------> pfx4
cobmap1 uvdata51 uvdata52 copy/edit uvdata55
Main -----> d1ebc ----------> d2asc ----------> d3ebc --------> rptcmp frame FTP uvcopyxx 'pfx3/*' uvcopyxx 'pfx4/*' uvcmp1all mismatch rpts
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#0a. Login as mvstest1 ---> /home/mvstest1
#0b. Login as yourself --> your homedir
#0c. cd $CNVDATA <-- change to superdir with subdirs cpys,d1ebc,d2asc,etc
===========
#0d. cdc <-- same, alias cdc='cd $CNVDATA'
==== export CNVDATA=$HOME/cnvdata1 <-- in profile
#1. mkdir pfx4 <-- make subdir for ASCII-->EBCDIC jobs
========== (if not already existing)
#2. uvcopy uvdata55,fili1=pfx4/citytax1,filo1=pfx5/citytax1
=======================================================
# citytax1 - uvcopy job, ASCII to EBCDIC preserve packed, fix signs #01
opr='citytax1 - uvcopy code generated from copybook: citytax1.cpy ' #02
uop=q0 #03
was=a33000b33000 #04
fili1=?d2asc/citytax1,rcs=00128,typ=RSF #05
filo1=?d3ebc/citytax1,rcs=00128,typ=RSF #06
@run #07
opn all #08
loop get fili1,a0 #09
skp> eof #10
mvc b0(00128),a0 move rec to outarea before field prcsng #11
tre b0(00128) translate entire outarea to EBCDIC #12
# --- <-- insert R/T tests here for redefined records #13
mvc b88(4),a88 bns post-date #14
mvc b92(15),a92 pns land-value:face-value #15
trt b107(9),$trtsae ns maint-tax #16
put1 put filo1,b0 #17
skp loop #18
eof cls all #19
eoj #20
Compare this ASCII to EBCDIC job with the original EBCDIC to ASCII job job listed on page '3B2'. Aside from directory & file names, there are only 2 instructions different as follows:
tra b0(00128) translate entire outarea to ASCII #12
tre b0(00128) translate entire outarea to EBCDIC #12
trt b107(9),$trtsea ns maint-tax #16
trt b107(9),$trtsae ns maint-tax #16
Note that $trtsea is a table to translate Signs Ebcdic to Ascii and that $trtsae is a table to translate Signs Ascii to Ebcdic
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#4a. uvcopy pfx4/citytax1,fili1=d2asc/citytax1,filo1=d3ebc/citytax1
==============================================================
#4b. uvcopy pfx4/citytax1 <-- easier (filenames default as shown above)
====================
#3. uvhd d2asc/citytax1 r128a <-- display ASCII input file
=========================
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
128 10139923 GEORGE BROWN 1250 EAST PENDER STREET VANC
3333333322444544245454222222222222233332445525444452555445225444
101399230075F275022F7E0000000000000125005134005E45203425540061E3
64 OUVER BCV5L1W1......W.........xRL00014626q980601190156
4554522222222222445343530888005880008800754333333337333333333333
F5652000000000002365C171029C0072C0047C0182C000146261980601190156
^--- ASCII -1
#4. uvhd d3ebc/citytax1 r128 <-- display EBCDIC output file
==========================
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
128 10139923 GEORGE BROWN 1250 EAST PENDER STREET VANC
FFFFFFFF44CCDDCC4CDDED4444444444444FFFF4CCEE4DCDCCD4EEDCCE44ECDC
1013992300756975029665000000000000012500512307554590239553005153
64 OUVER BCV5L1W1.bi...Wb....g...yR<00014626J980601190156
DEECD44444444444CCEFDFEF0888005880008800754FFFFFFFFDFFFFFFFFFFFF
645590000000000023553161029C0072C0047C0182C000146261980601190156
^--- EBCDIC -1
We have shown record #2 because it has a negative zoned sign at byte 115 (the units position of the field in 107-115).
In ASCII a -1 is a 'q' or x'71'. The normal translate EBCDIC to ASCII (tre) would translate it to an EBCDIC 'q' x'98' which would be wrong, so the 'trt' instruction is used with special translate table '$trtsae' to translate Signs Ascii to Ebcdic. This table translates the 'q' to a 'J' (-1 in EBCDIC).
See more details about zoned sign conversion on page '3C2'
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Pages '3C1' thru '3C6' documented converting EBCDIC to ASCII.
Pages '9A1' thru '9A3' documented converting ASCII to EBCDIC.
You could use the ASCII to EBCDIC conversion to verify the accuracy of the EBCDIC to ASCII conversion, by comparing the output of 2nd conversion to the input of the 1st. The 3 test/demo files involved are:
| d1ebc/citytax1 |
|
| d2asc/citytax1 |
|
| d3ebc/citytax1 |
|
We can't use the unix/linux 'diff' utility because these mainframe style data files are fixed length without the LineFeeds required by diff.
We might try the unix/linux 'cmp' utility. We will use the '-l' option, otherwise it quits at the 1st mismatch.
cmp -l d1ebc/citytax1 d3ebc/citytax1 <-- compare 2 data files ======================================
Byte# file1 file2 116 301 361 372 302 362 500 303 363 628 311 371 756 300 360
The 'cmp' utility shows differences in 3 columns - byte#, 1st file byte contents, 2nd file byte contents. The byte contents are in 'octal' which is an anachronism but there seems to be no hexadecimal option.
A much better solution is to use a Vancouver Utility uvcopy job 'uvcmp1':
uvcopy uvcmp1,fili1=d1ebc/citytax1,fili2=d3ebc/citytax1 =========================================================
See the output report on the next page --->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
uvcmp1 - compare 2 files "*" diffs (uops=q1e0p30r256s5t500000v2x0e1r128) recsize reccount file-size typ filename 1: 128 10 1,280 RSF d1ebc/citytax1 2: 128 10 1,280 RSF d3ebc/citytax1 2005/04/21_17:24:39 1 2 3 4 5 6 f#rec#byte# 0123456789012345678901234567890123456789012345678901234567890123 ============================================================================
1 1 0 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
FFFFFFFF44DDCD4CCDDE444444444444444FFFF4CDECD4DDCC4444444444ECDC
1013014000168508559800000000000000018150266550961400000000005153
64 OUVER BCV9S1H1 bi Wb g qq 00014906A970530191809
DEECD44444444444CCEFEFCF0888005880008000770FFFFFFFFCFFFFFFFFFFFF
645590000000000023592181029C0072C0047C0111C000149061970530191809
*
2 1 10130140 JOHN HENRY 1815 BOWEN ROAD VANC
FFFFFFFF44DDCD4CCDDE444444444444444FFFF4CDECD4DDCC4444444444ECDC
1013014000168508559800000000000000018150266550961400000000005153
64 OUVER BCV9S1H1 bi Wb g qq 000149061970530191809
DEECD44444444444CCEFEFCF0888005880008000770FFFFFFFFFFFFFFFFFFFFF
645590000000000023592181029C0072C0047C0111C000149061970530191809
*
1 5 0 10154688 MEL RAHAL 1110 - 625 HOWE STREET VANC
FFFFFFFF44DCD4DCCCD4444444444444444FFFF464FFF4CDEC4EEDCCE444ECDC
1015468800453091813000000000000000011100006250866502395530005153
64 OUVER BCV6C2T6 bi Dh 00011547I961126082351
DEECD44444444444CCEFCFEF0888004800000000208FFFFFFFFCFFFFFFFFFFFF
645590000000000023563236029C0048C0000C0153C000115479961126082351
*
2 5 10154688 MEL RAHAL 1110 - 625 HOWE STREET VANC
FFFFFFFF44DCD4DCCCD4444444444444444FFFF464FFF4CDEC4EEDCCE444ECDC
1015468800453091813000000000000000011100006250866502395530005153
64 OUVER BCV6C2T6 bi Dh 000115479961126082351
DEECD44444444444CCEFCFEF0888004800000000208FFFFFFFFFFFFFFFFFFFFF
645590000000000023563236029C0048C0000C0153C000115479961126082351
*
The differences shown are not a problem - they are caused by translating zoned signs from EBCDIC to ASCII & back, because there are no + zones in ASCII. The original + zones in EBCDIC x'C_' are lost (not needed) in ASCII where normal numeric zones x'3_' are the only + zones. When we convert back we get the normal numeric zones in EBCDIC x'F_', which cause the mismatches (but these are not errors).
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Actually, the instructions to generate & execute jobs for for ALL data-files are shorter & simpler than for 1 at a time.
#0a. Login as mvstest1 ---> /home/mvstest1
#0b. Login as yourself --> your homedir
#0c. cd $CNVDATA <-- change to superdir with subdirs cpys,d1ebc,d2asc,etc
===========
#0d. cdc <-- same, alias cdc='cd $CNVDATA'
==== export CNVDATA=$HOME/cnvdata1 <-- in profile
#1. mkdir pfx4 <-- make subdir for ASCII-->EBCDIC jobs
========== (if not already existing)
#2. uvcopyx uvdata55 pfx3 pfx4 uop=q0i7 <-- generate ALL jobs
===================================
#3. uvcopyxx 'pfx4/*' <-- execute ALL jobs
=================
#4. mkdir rptcmp <-- make subdir for mismatch reprots
============
#5. uvcmp1all d1ebc d3ebc <-- compare all original files to reconverted files
=====================
#6a. vi rptcmp/* <-- inspect all mismatch reports
===========
#6b. uvlpd12 rptcmp <-- print all mismatch reports
==============
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
You can use the 'v' command of uvhdcob to verify numeric fields (packed & unpacked) in the converted data files. Here is an example for d2asc/citytax1. First we need to update the file with some intentional errors:
#1. Login mvstest1 or yourself --> /home/mvstest1 or your homedir $HOME
#2. cdc <-- alias cdc='cd $CNVDATA' (CNVDATA=$HOME/cnvdata1)
=== - change to data conversion superdir to perform data conversion
#3. uvhdcob d2asc/citytax1 maps/citytax1 u <-- Update citytax1 demo file
====================================== - create intentional errs
- uvhdcob displays 1st record (as follows) & prompts for comamnds
rec# 1 fieldname occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 folio 0 9 10130140 002 name 10 34 JOHN HENRY 003 address 35 59 1815 BOWEN ROAD 004 city 60 79 VANCOUVER 005 province 80 81 BC 006 zip 82 87 V9S1H1 007 post-date 88 91bns 0082898C 008 land-value 92 96pns 000057828C 009 improv-value 97 101pns 000004870C 010 face-value 102 106pns 000171710C 011 maint-tax 107 115 ns 000149061 012 purchase-date 116 121 970530 013 filler001 122 127 191809
--> 3 <-- advance to record#3
--> u 92(1),x'EE' <-- Update 1st byte of 'land-value' with bad Packed data
--> u 107(1),'X' <-- Update 1st byte of 'maint-tax' with bad Numeric data
--> 6 <-- advance to record#6
--> u 88(1),x'EE' <-- Update 1st byte of 'post-date' with bad Binary data
--> u 115(1),'X' <-- Update last byte of 'maint-tax' with bad Numeric data
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
On the prior page we updated d2asc/citytax1 with BAD data in Packed & Numeric fields in records #3 & #6. Now we will demo the uvhdcob 'v' verify command.
#4. uvhdcob d2asc/citytax1 maps/citytax1 u <-- Update citytax1 demo file
=============================================
--> <-- uvhdcob displays 1st record (not shown) & prompts for command
--> v <-- enter 'v' to search/verify (until bad numeric fields found)
- will display rec#3 (1st record with bad data) as follows:
rec# 3 fieldname occurs bgn end typ<------ data (hex if typ=p/b) -----> 014 folio 0 9 10147615 015 name 10 34 BONNIE SMITH 016 address 35 59 44430 YALE ROAD WEST 017 city 60 79 VANCOUVER 018 province 80 81 BC 019 zip 82 87 V2P6J1 020 post-date 88 91bns 0121395C 021 land-value 92 96pns EE0039623C <-- Bad Packed 022 improv-value 97 101pns 000000000C 023 face-value 102 106pns 000051914C 024 maint-tax 107 115 ns X00037692 583030303337363932<-BadN 025 purchase-date 116 121 950601 026 filler001 122 127 201031
v --> 3 recs verified, 3 ERRs found in rec# 3, 1st at byte# 92
2 packed & 1 numeric bad signs/digits, 0 unprintables in char fields
---> vv <-- 'vv' command to continue verify
rec# 6 fieldname occurs bgn end typ<------ data (hex if typ=p/b) -----> 014 folio 0 9 10173320 015 name 10 34 SAM SMITH 016 address 35 59 2381 ROGERS AVE 017 city 60 79 VANCOUVER 018 province 80 81 BC 019 zip 82 87 V3K5Y2 020 post-date 88 91bns EE50498C <-- Bad Binary 021 land-value 92 96pns 000060600C 022 improv-value 97 101pns 000000000C 023 face-value 102 106pns 000169086C 024 maint-tax 107 115 ns 00015620X 303030313536323058<-BadN 025 purchase-date 116 121 970530 026 filler001 122 127 191810
vv --> 3 recs verified, 3 ERRs found in rec# 6, 1st at byte# 88
2 packed & 1 numeric bad signs/digits, 0 unprintables in char fields
---> vv <-- 'vv' command to continue verify
vv --> 4 records verified, No errors found
*** EOF reached (enter -1 to see last rec)
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Verifying Numeric & Packed fields is the best way to detect any discrepancy between the copybook layouts & the actual data - especially effective for files with packed fields, where the unique sign codes must be in the last byte of each packed field.
We will demonstrate using the 'citytax1' file shown on the previous page where we used 'uvhdcob' to Verify Packed & Numeric fields.
Why do we need 'genverify1' if we can use uvhdcob ? - because uvhdcob displays & verifies ONLY the 1st field of any OCCURS. uvhdcob would not be conclusive for files such as armaster which has occurs 12 for this year & last year sales.
#1. Login mvstest1 or yourself --> /home/mvstest1 or your homedir $HOME
#2. cdc <-- alias cdc='cd $CNVDATA' (CNVDATA=$HOME/cnvdata1)
=== - change to data conversion superdir to perform data conversion
#3. mkdir pfv1 pfv2 pfv3 rptver <-- make subdirs for outputs
===========================
#4. uvcopy cobmap1,fili1=cpys/citytax1.cpy,filo1=maps/citytax1
==========================================================
- convert copybook to 'cobmap' (record layout)
#5. uvcopy genverify1,fili1=maps/citytax1,filo1=pfv1/citytax1
=========================================================
- convert cobmap to uvcopy job to verify numeric/packed fields
| Note |
|
#4a. uvcopyx cobmap1 cpys maps uop=q0i7p0
====================================
- convert all COBOL copybooks to layouts
#5a. uvcopyx genverify1 maps pfv1 uop=q0i7f0
=======================================
- convert all cobmaps to data verify jobs
#6. uvcopy uvdata52,fili1=ctl/ctlfile54,fild2=pfv1,fild3=pfv2,uop=r1s1t0
====================================================================
- insert datafilenames (vs copybook names)
- see options explained within $UV/pf/IBM/uvdata52
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
cobmap1 start-end bytes for cobol record fields 200306092029 pg# 0001
cpys/citytax1.cpy citytax1 RCSZ=00128 bgn end lth typ
01 citytax1.
10 folio pic x(00010). 0000 0009 010
10 name pic x(00025). 0010 0034 025
10 address pic x(00025). 0035 0059 025
10 city pic x(00020). 0060 0079 020
10 province pic x(00002). 0080 0081 002
10 zip pic x(00006). 0082 0087 006
10 post-date pic s9(00007) comp-4. 0088 0091 004bns 7
10 land-value pic s9(00009) comp-3. 0092 0096 005pns 9
10 improv-value pic s9(00009) comp-3. 0097 0101 005pns 9
10 face-value pic s9(00009) comp-3. 0102 0106 005pns 9
10 maint-tax pic s9(00007)v99. 0107 0115 009 ns 9
10 purchase-date pic x(00006). 0116 0121 006
10 filler001 pic x(00006). 0122 0127 006
*RCSZ=00128 0128
# uvcopy job to verify packed or zoned numeric fields
opr='jobname=JOBNAME - pfv2 name=datafilename'
opr='copybook=citytax1.cpy - maps/pfv1 name=copybookname'
arg1=JOBNAME,arg2=citytax1
uop=q0,was=a33000t80000
fili1=?d2asc/citytax1,rcs=00128,typ=RSF
filo1=?rptver/citytax1,rcs=128,typ=LSTt
lodv1=h0(100)
DataFile=DATAFILE CopyBook=COPYBOOK $datetime
LthPS <--HexData-- <--NumData-- Rec# Byte FieldName-->
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@run
opn all
repf1 h0(100),'DATAFILE',$fili1
repf4 h0(100),'COPYBOOK',$arg2
wtbe filo1,h0(100),h0(100)
loop get fili1,a0
skp> eof
vnft3 a92(5p),1,'land-value'
vnft2 a97(5p),1,'improv-value'
vnft2 a102(5p),1,'face-value'
vnft2 a107(9z),1,'maint-tax'
wtbe filo1,t0(80),t0(80)
skp loop
#
eof cls all
eoj
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#6. uvcopy pfv1/citytax1 <-- execute job to verify Packed/Numeric fields
====================
#7. cat rptver/citytax1 <-- display report (created in rptver/citytax1)
===================
DataFile=d2asc/citytax1 CopyBook=citytax1 2012/10/16_09:57:01 LthPS <--HexData-- <--NumData-- Rec# Byte FieldName-->
005p EE0039623C 0003 0092 land-value 009 583030303337 X00037692 0003 0107 maint-tax
005p 000169086E 0006 0102 face-value 009 303030313536 00015620X 0006 0107 maint-tax
rec# 3 fieldname occurs bgn end typ<------ data (hex if typ=p/b) -----> 014 folio 0 9 10147615 015 name 10 34 BONNIE SMITH 016 address 35 59 44430 YALE ROAD WEST 017 city 60 79 VANCOUVER 018 province 80 81 BC 019 zip 82 87 V2P6J1 020 post-date 88 91bns 0121395C 021 land-value 92 96pns EE0039623C <-- Bad Packed 022 improv-value 97 101pns 000000000C 023 face-value 102 106pns 000051914C 024 maint-tax 107 115 ns X00037692 583030303337363932<-BadN 025 purchase-date 116 121 950601 026 filler001 122 127 201031
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Example#1 (citytax1) had no occurs fields. For example#2 we will use 'armaster' which has occurs fields for 12 months sales this year & last. Here are the illustrated procedures to generate & execute the 'genverify1' job for armaster.
#1. Login mvstest1 or yourself --> /home/mvstest1 or your homedir $HOME
#2. cdc <-- alias cdc='cd $CNVDATA' (CNVDATA=$HOME/cnvdata1)
=== - change to data conversion superdir to perform data conversion
#3. uvcopy cobmap1,fili1=cpys/armaster.cpy,filo1=maps/armaster
==========================================================
- convert copybook to 'cobmap' (record layout)
cobmap1 start-end bytes for cobol record fields 200902200736 pg# 0001
cpys/armaster.cpy RCSZ=00256 bgn end lth typ
* armaster - cobol copy book for armaster file (see uvtrain.
10 cm-num pic 9(6). 0000 0005 006 n 6
10 cm-delete pic x(4). 0006 0009 004
10 cm-name pic x(25). 0010 0034 025
10 cm-adrs pic x(25). 0035 0059 025
10 cm-city pic x(16). 0060 0075 016
10 filler001 pic x. 0076 0076 001
10 cm-prov pic x(2). 0077 0078 002
10 filler002 pic x. 0079 0079 001
10 cm-postal pic x(10). 0080 0089 010
10 cm-phone pic x(12). 0090 0101 012
10 cm-contact pic x(18). 0102 0119 018
10 cm-thisyr-sales pic s9(7)v99 comp-3 occurs 12. 0120 0124 005pns 9
10 cm-lastyr-sales pic s9(7)v99 comp-3 occurs 12. 0180 0184 005pns 9
10 filler003 pic x(16). 0240 0255 016
*RCSZ=00256
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
We want to demonstrate that 'genverify1' generates uvcopy jobs that will verify ALL packed/numeric fields vs 'uvhdcob' which can only verify the 1st field of any 'occurs'.
So, before we demo 'genverify1' (on the next page), we will 1st show you the 'uvhdcob' verify on this page.
#4. uvhdcob d2asc/cutsmas1 maps/armaster <-- execute uvhdcob for armaster
====================================
| Note |
|
--> v <-- search & Verify character,numeric,& packed fields
rec# 10 fieldname occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 cm-num 0 5 n 15468X 313534363858 <-BadN 002 cm-delete 6 9 003 cm-name 10 34 TAURUS RESOURCES 004 cm-adrs 35 59 1110 - 625 HOWE STREET 005 cm-city 60 75 VANCOUVER 006 filler001 76 76 007 cm-prov 77 78 B. 42EE <-BadC 008 filler002 79 79 009 cm-postal 80 89 V6C2T6 010 cm-phone 90 101 604 685 8565 011 cm-contact 102 119 MEL RAHAL 012 cm-thisyr-sales 012 120 124pns EE0123456C <-- Bad Packed 013 cm-lastyr-sales 012 180 184pns 000000000C 014 filler003 240 255 C 19980331
v --> 10 recs verified, 4 ERRs found in rec# 10, 1st at byte# 0
2 packed & 1 numeric bad signs/digits, 1 unprintables in char fields
--> vv <-- continue Verify
rec# 20 fieldname occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 cm-num 0 5 n 2343XX 323334335858 <-BadN 002 cm-delete 6 9 003 cm-name 10 34 TERRITORIAL REWIND 004 cm-adrs 35 59 BOX 1648 005 cm-city 60 75 WHITEHORSE 006 filler001 76 76 007 cm-prov 77 78 YK 008 filler002 79 79 009 cm-postal 80 89 Y1A... 593141EEEEEE <-BadC 010 cm-phone 90 101 403-512-3948 011 cm-contact 102 119 012 cm-thisyr-sales 012 120 124pns 000000000C 013 cm-lastyr-sales 012 180 184pns 1E2E3E4E5E <-- Bad Packed 014 filler003 240 255 20021130
vv --> 10 recs verified, 10 ERRs found in rec# 20, 1st at byte# 0
5 packed & 2 numeric bad signs/digits, 3 unprintables in char fields
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#5. uvcopy genverify1,fili1=maps/armaster,filo1=pfv1/armaster
=========================================================
- convert cobmap to uvcopy job to verify numeric/packed fields
# uvcopy job to verify packed or zoned numeric fields
opr='jobname=JOBNAME - pfv2 name=datafilename'
opr='copybook=armaster - maps/pfv1 name=copybookname'
uop=q0,was=a33000t80000
fili1=?d2asc/armaster,rcs=00256,typ=RSF
filo1=?rptver/armaster,rcs=128,typ=LSTt
@run
opn all
loop get fili1,a0
skp> eof
vnft3 a0(6z),1,'cm-num'
vnft2 a120(5p),12,'cm-thisyr-sales'
vnft2 a180(5p),12,'cm-lastyr-sales'
wtbe filo1,t0(80),t0(80)
skp loop
eof cls all
eoj
#5. uvcopy pfv1/armaster <-- execute verify job for armaster
====================
#6. uvlp12 rptver/armaster <-- print report
======================
lth(p=packed) hex-representation numeric-data rec# occurs# field-name =================================================================
06 313534363858 15468X 010 cm-num 05p EE0123456C 010 01 cm-thisyr-sales 05p- E01234567D 010 12 cm-lastyr-sales
06 323334335858 2343XX 020 cm-num 05p EE0012345C 020 03 cm-thisyr-sales 05p 1E2E3E4E5E 020 01 cm-lastyr-sales 05p 00000000EE 020 12 cm-lastyr-sales
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'genacum2' is a Vancouver Utility to generate uvcopy jobs to accumulate all numeric fields (packed/zoned/binary) in 1 or all data files. The jobs are generated automatically from the 'cobmap' record layout (which is generated from the COBOL copybook).
You might use these reports to verify data file conversions, OR to get a 2nd opinion on suspicious looking totals on your COBOL reports.
We will demonstrate using the 'armaster' copybook & datafile, which has occurs fields for 12 months sales this year & last. Here are the illustrated procedures to generate & execute the 'genacum2' job for armaster.
#1. Login mvstest1 or yourself --> /home/mvstest1 or your homedir $HOME
#2. cdc <-- alias cdc='cd $CNVDATA' (CNVDATA=$HOME/cnvdata1)
=== - change to data conversion superdir to perform data conversion
#3. mkdir pfa1 pfa2 pfa3 <-- make subdirs for genacum2 jobs
====================
/home/mvstest1/cnvdata <-- or use your homedir with directory cnvdata1
/home/yourhomedir/cnvdata1
:-----cpys/ <-- copybooks
: :----armaster.cpy - copybook 'armaster' for datafile 'armaster'
:-----d2asc/ <-- data files (previously converted to ASCII)
: :-----armaster - datafile 'armaster'
:-----maps/ <-- record layouts (cobmaps) created from copybooks
: :-----armaster - cobmap 'armaster'
:-----pfa1/ <-- uvcopy jobs generated from cobmaps by 'genacum2'
: :-----armaster - uvcopy job 'armaster' to add all fields in datafile
:-----pfa2/ <-- reserved in case you use uvdata52 insert datafilenames
:-----pfa3/ <-- uvcopy jobs copied here before execution
: : - modified if required (if multiple record types)
:-----rptacu/ <-- reports written here
: :-----armaster - reports named same as copybook & datafile
(run uvdata52 to change to datafilenames)
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#4. uvcopy cobmap1,fili1=cpys/armaster.cpy,filo1=maps/armaster
==========================================================
- convert copybook to 'cobmap' (record layout)
cobmap1 start-end bytes for cobol record fields 200902200736 pg# 0001
cpys/armaster.cpy RCSZ=00256 bgn end lth typ
* armaster - cobol copy book for armaster file (see uvtrain.
10 cm-num pic 9(6). 0000 0005 006 n 6
10 cm-delete pic x(4). 0006 0009 004
10 cm-name pic x(25). 0010 0034 025
10 cm-adrs pic x(25). 0035 0059 025
10 cm-city pic x(16). 0060 0075 016
10 filler001 pic x. 0076 0076 001
10 cm-prov pic x(2). 0077 0078 002
10 filler002 pic x. 0079 0079 001
10 cm-postal pic x(10). 0080 0089 010
10 cm-phone pic x(12). 0090 0101 012
10 cm-contact pic x(18). 0102 0119 018
10 cm-thisyr-sales pic s9(7)v99 comp-3 occurs 12. 0120 0124 005pns 9
10 cm-lastyr-sales pic s9(7)v99 comp-3 occurs 12. 0180 0184 005pns 9
10 filler003 pic x(16). 0240 0255 016
*RCSZ=00256
On the next page, we will run genacum2 to generate a job to add all fields in test file d2asc/armaster (based on the copybook converted to a cobmap as shown on the previous page).
But first (to understand the datafile), you might run 'uvhdcob' which displays the data field contents beside the COBOL copybook fieldnames.
#5. uvhdcob d2asc/cutsmas1 maps/armaster <-- execute uvhdcob for armaster
====================================
rec# 1 fieldname occurs bgn end typ<------ data (hex if typ=p/b) -----> 001 cm-num 0 5 n 130140 002 cm-delete 6 9 003 cm-name 10 34 EVERGREEN MOTORS LTD. 004 cm-adrs 35 59 1815 BOWEN ROAD 005 cm-city 60 75 NANAIMO 006 filler001 76 76 007 cm-prov 77 78 BC 008 filler002 79 79 009 cm-postal 80 89 V9S1H1 010 cm-phone 90 101 250-754-5531 011 cm-contact 102 119 LARRY WRENCH 012 cm-thisyr-sales 012 120 124pns 001234567C 013 cm-lastyr-sales 012 180 184pns 001234567D 014 filler003 240 255 C 19950531
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#6. uvcopy genacum2,fili1=maps/armaster,filo1=pfa1/armaster,uop=f0
==============================================================
- convert cobmap to uvcopy job to accumulate All numeric fields
uop=f2m1 - option defaults (for generated jobs)
f0 - Input file default typ=RSF fixed lth (in generated job)
f2 - Input typ=RDWz2 (2 byte BIG-end binary recsize hdrs)
f4 - Input file typ=RDWz4 (4 byte hdrs, bytes 3&4 nulls)
m1 - read copybook record layouts from maps/... in $CNVDATA
m2 - read copybook record layouts from maps/... in $RUNLIBS
User OPtion (uop) defaults = q1f2m1
-->null to accept or enter/override --> f0 <-- enter 'f0' for typ=RSF
| Note |
|
# uvcopy job to accumulate all numeric fields in a datafile
opr='jobname=JOBNAME - pfa2 name=datafilename'
opr='copybook=armaster.cpy - maps/pfa1 name=copybookname'
uop=q0i3,rop=r1 # minimize startup msgs, prompt for report disposition
was=a33000c200000t100000 # 33K maxrec, 200K cpybkmap, 100K acum tbl
fili1=?d2asc/armaster,rcs=00256,typ=RSF
filo1=?rptacu/armaster,rcs=128,typ=LSTt
fili2=${CNVDATA}/maps/armaster,rcs=128,typ=LST
@run
opn all
rtb fili2,c0(100),c0(100) read cobmap into area c table
clr t0(100000),x'00' clear area t for accumulator table
loop get fili1,a0 get next record into area 'a'
skp> eof
add $cb3,1 count all record types
# --- --- <-- insert any Record Type tests here
mvn $rt,0 init register t for current record
mvn $rs,0 init register s for current record
add ts9800(8b),1 count 1st (or only) R/T in copybook
bal acum,'0,6,z,1,cm-num'
bal acum,'120,5,p,12,cm-thisyr-sales'
bal acum,'180,5,p,12,cm-lastyr-sales'
skp loop
eof bal dump dump tables of fieldnames & acummulators
cls all
eoj
@pf2=acum2.sub
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#7. uvcopy pfa1/armaster <-- execute job to add all numeric fields armaster
====================
#8. uvlp12 rptacu/armaster <-- print report
======================
genacum2 RCSZ=00256 DataFile=d2asc/armaster 2009/02/26_09:22:03 copybook=cpys/armaster.cpy line# Field-Name occurs start lth type field-total
1 cm-num 0005 006 n 7,259,353
2 cm-thisyr-sales 12 0124 005 pns 99,697.93
3 cm-lastyr-sales 12 0184 005 pns 45,440.85
Record count 32 ** Hash-Total 21,773,231 **
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The above procedures were for 1 file at a time. For major projects we would want to generate jobs for all copybooks & all datafiles. Actually the operating instructions are shorter & easier to key.
#1. Login mvstest1 or yourself --> /home/mvstest1 or your homedir $HOME
#2. cdc <-- alias cdc='cd $CNVDATA' (CNVDATA=$HOME/cnvdata1)
=== - change to data conversion superdir to perform data conversion
#3. uvcopyx cobmap1 cpys maps uop=q0i7p0
====================================
- convert COBOL copybooks to layouts (field start/end on right)
#4. uvcopyx genacum2 maps pfa1 uop=q0i7f0
=====================================
- convert cobmaps to uvcopy jobs to accumulate all numeric fields
- I/O datafiles coded as copybook names
#5. vi ctl/ctlfile54 <-- edit control file to relate copybooks to datafiles
================
# ctlfile54 - sample control file to relate copybooknames & datafilenames
ar/customer.master cpy=armaster rcs=00256 typ=RSF
gl/account.master_ cpy=glmaster rcs=00128 typ=RDWz2
ap/vendor.master cpy=vendmas rcs=00256 rcs=257 keys=0(10)
#6. uvcopy uvdata52,fili1=ctl/ctlfile54,fild2=pfa1,fild3=pfa2,uop=r1s1t0
====================================================================
- insert datafilenames (vs copybook names)
- option r1 = override fili1 recsize with recsize from control file
s1 = set input typ=... from control file
t0 = do not change output file type (typ=LSTt from genverify1)
#7. cp pfa2/* pfa3
==============
- copy completed uvcopy data conversion jobs to execution subdir
#8. vi pfa3/* <-- edit for files with multiple record Types
========= - need to insert R/T test code & skip to
corresponding bank of auot generated code
#9. uvcopyxx 'pfa3/*' <-- execute all jobs to acum all num-fields in all files
=================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
'gentest1' will generate uvcopy jobs to generate test data files based on the COBOL copybooks. The cobol copybooks are 1st converted to the 'cobmap' record layouts by the 'cobmap1' utility.
The generated uvcopy jobs are named after the copybooks and they call sub- routines to generate data for each field. Each picture field in the copybook generates a 'bal' to a subrtn, depending on field type (type X, packed, binary, signed/unsigned numeric). For example:
bal genx,'a00035b0025','address' <-- sample call to subrtn =================================
This calls subrtn 'genx' for a type X field starting at byte 35 (option a00035) that is 25 bytes long (option b0025), whose fieldname is 'address'.
Here is a summary of the subrtns called & a description of the actions performed for the various field types:
| genx |
|
| gen9 |
|
| gens |
|
| genp |
|
| genb |
|
Calling subrtns keeps the generated uvcopy jobs short since most of the field processing code is in these sub-routines. The subrtn module is called via the following code, which you can see at the end of the sample job listed on the next page.
@pf2=gentest1.sub <-- code to include subfunctions in uvcopy jobs
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
cobmap1 start-end bytes for cobol record fields 200408220851 pg# 0001
cpys/citytax1.cpy citytax1 RCSZ=00128 bgn end lth typ
* citytax1 copybook
01 citytax1.
10 folio pic x(00010). 0000 0009 010
10 name pic x(00025). 0010 0034 025
10 address pic x(00025). 0035 0059 025
10 city pic x(00020). 0060 0079 020
10 province pic x(00002). 0080 0081 002
10 zip pic x(00006). 0082 0087 006
10 post-date pic s9(00007) comp-4. 0088 0091 004bns 7
10 land-value pic s9(00009) comp-3. 0092 0096 005pns 9
10 improv-value pic s9(00009) comp-3. 0097 0101 005pns 9
10 face-value pic s9(00009) comp-3. 0102 0106 005pns 9
10 maint-tax pic s9(00007)v99. 0107 0115 009 ns 9
10 purchase-date pic x(00006). 0116 0121 006
10 filler001 pic x(00006). 0122 0127 006
*RCSZ=00128 0128
opr='JOBNAME citytax1 - generated by cobmap1 & gentest1'
opr='uop=c100 - option "c" default record output count = 100'
opr=' - may change at prompt (ex: enter "c2000" for 2000 records)'
uop=q1c100 # option defaults
was=a9000b9000
filo1=?tmp/citytax1,rcs=00128,typ=RSF
@run
opn all
bal initj inits for job (setup A-Z 0-9 patterns)
# begin loop to generate fields & output each record
loop bal initr inits for record (counts records,etc)
clr b0(00128),' ' init output area to all blanks
# --- <-- insert R/T tests if redef records
bal genx,'a00000b0010','folio'
bal genx,'a00010b0025','name'
bal genx,'a00035b0025','address'
bal genx,'a00060b0020','city'
bal genx,'a00080b0002','province'
bal genx,'a00082b0006','zip'
bal genb,'a00088b0004','post-date'
bal genp,'a00092b0005','land-value'
bal genp,'a00097b0005','improv-value'
bal genp,'a00102b0005','face-value'
bal gens,'a00107b0009','maint-tax'
bal genx,'a00116b0006','purchase-date'
bal genx,'a00122b0006','filler001'
#
put1 put filo1,b0 output current record
cmn $ca1,$uopbc reached output Count ?
skp< loop
eof cls all
eoj
@pf2=gentest1.sub
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
1. uvcopy cobmap1,fili1=cpys/citytax1.cpy,filo1=maps/citytax1
==========================================================
- convert the COBOL copybook into a 'cobmap' (record layout)
2. uvcopy gentest1,fili1=maps/citytax1,filo1=pfg1/citytax1
=======================================================
- generate the uvcopy job from the cobmap
3. uvcopy pfg1/citytax1,filo1=tmp/citytax1 <-- execute the generated job
=======================================
3a. uvcopy pfg1/citytax1 <-- same (output file defaults as shown above)
====================
| uop=c100 |
|
User OPtion (uop) defaults = q1c100 null to accept or re-specify (1 or more) --> c500 <-- may change default
tmp/citytax1 = default filo01 - null accept or reenter --> <-- null accept
citytax1:040823:105849: EOF filo01 0 rds, 100 wrts, 12800 size; tmp/citytax1
4. uvhd filename=tmp/citytax1 r128 <-- examine the output
===============================
Please see the 'uvhd' output display shown on the next page --->
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Here is the 'uvhd' command repeated from the previous page, but with option 's7' added to space out the display to make it easier to read.
4. uvhd filename=tmp/citytax1 r128s7 <-- examine the output
=================================
10 20 30 40 50 60
r# 1 0123456789012345678901234567890123456789012345678901234567890123
0 folio name address city
6666622222666622222222222222222222266676772222222222222222226677
6FC9F00000E1D500000000000000000000014425330000000000000000003949
64 przip ...................000000001purchafiller
2222222222222222777672220001000010000100001333333333777666666667
000000000000000002A90000000C0000C0000C0000C00000000105238169CC52
10 20 30 40 50 60
r# 2 0123456789012345678901234567890123456789012345678901234567890123
128 AAAAAAAAAABBBBB000002BBBBBBBBBBBBBBCCCCC000002CCCCCCCCCCCCCCDDDD
4444444444444443333334444444444444444444333333444444444444444444
1111111111222220000022222222222222233333000002333333333333334444
64 D000002DDDDDDDDDEEFFFFFF...,....,....,....,000000002GGGGGGHHHHHH
4333333444444444444444440002000020000200002333333333444444444444
400000244444444455666666000C0000C0000C0000C000000002777777888888
10 20 30 40 50 60
r# 3 0123456789012345678901234567890123456789012345678901234567890123
256 AAAAAAAAAABBBBB000003BBBBBBBBBBBBBBCCCCC000003CCCCCCCCCCCCCCDDDD
4444444444444443333334444444444444444444333333444444444444444444
1111111111222220000032222222222222233333000003333333333333334444
64 D000003DDDDDDDDDEEFFFFFF...=....=....=....=00000000sGGGGGGHHHHHH
4333333444444444444444440003000030000300003333333337444444444444
400000344444444455666666000D0000D0000D0000D000000003777777888888
Please relate this output to the copybook & the rules listed previously:
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
The 'gentestA' script is provided to make it easier to run the gentest1 utility. The script includes both the cobmap1 & gentest1 uvcopy jobs, reducing the operating instructions to 1 line:
gentestA citytax1 - generates uvcopy job --> pfg1/citytax1 =================
uvcopy pfg1/citytax1,filo1=tmp/citytax1 <-- execute the generated job =======================================
#1. uvcopyx cobmap1 cpys maps uop=q0i7p0 <-- create cobmaps for all copybooks
====================================
#2. uvcopyx gentest1 maps pfg1 uop=q0i7 <-- create data gen jobs for all maps
===================================
#3. uvcopyX 'pfg1/*' <-- execute all data generate jobs
================
You might want to generate all test data generation jobs as shown in #1 & #2 above, but it is unlikely you would want to execute all at once because they would be generated in the 'tmp' subdir & all default to 100 records.
The uvcopy code for gentest1 & gentest1.sub is not shown here or on the web site documentation, but if you have Vancouver Utilities installed, you may find the code as follows:
/home/uvadm/pf/IBM/gentest1 - uvcopy code to generate the uvcopy jobs
to generate the test data files from copybooks
/home/uvadm/pf/IBM/gentest1.sub - uvcopy code for the sub-routines called
by the generated uvcopy jobs
Note that the generated jobs would be in your sub-directories. If you had followed the recommended setup procedures (setting up userid 'cnvdata1'), this would be:
/home/mvstest1/cnvdata1/pfg1/xxxxxx
Each of the generated jobs loads the subrtns via the code found on the last line of each generated job, which is:
@pf2=gentest1.sub <-- loads the subrtn code
The subrtns will be loaded from /home/uvadm/pf/IBM/gentest1.sub. If you want to modify the subrtns, you should 1st copy to your own 'pf' directory & place your directory ahead of the 'uvadm' pf directory in the PFPATH variable that uvcopy uses to load uvcopy job code. Do not modify in the uvadm pf directory, because any later update install of Vancouver Utilities would overwrite your modified code.
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
10A1. Summary of scripts used in this book (DATAcnv1.doc)
(gencnvD,signscanA,sign2cpyA,cobmapA,gencnvA,genrfmA,genpipeA,listpipeA)
10B1. Summary of uvcopy jobs used in this book
(cobmap1,uvdata51,uvdata52,genpipe1,sqlcreate1,signscan2,sign2cpy1)
| Note |
|
| 10C1. | *cobmapA - listing of script to convert 1 copybook to a 'cobmap' |
| 10C2. | *gencnvA - script to generate job to convert 1 data file EBCDIC to ASCII |
| - combines jobs to create cobmap, convert EBCDIC to ASCII, | |
| & convert ASCII file to pipe|delimited in d4pipe/... |
| 10D1. | *gencnvD2 - script to generate all conversion jobs from 1 copybook |
| - cobmap, uvcopy jobs EBCDIC to ASCII, pipe-delimit, | |
| pipe-delimit to Fixed-field, create SQL table, load SQL table |
| 10D2. | *gencnvD2.bat - same for Windows/DOS |
| 10E1. | *gencnvC - script to generate EBCDIC to ASCII jobs from directory |
| of filenames in d1ebc/..., using a control-file | |
| edited with copybook names corresponding to datafilenames | |
| (auto-generated if copybooknames embeded in datafilenames) |
| 10F1. | *uvcopypfxgA - Execute all uvcopy jobs in a directory(arg1) to convert all |
| files from 1 directory(arg2) to a 2nd directory(arg3) |
| 10G2. | *uvcopypfx2 - execute pfx1/..., pfp1/...,& listpipe jobs for any 1 file |
| - EBCDIC to ASCII & pipe|delimit jobs | |
| - also creates uvhd & uvhdcob displays | |
| in subdirs uvhd_d1ebc, uvhd_d2asc, uvhdcob (d2asc only) | |
| - also see uvcopypfx9 to execute all jobs for all files |
| 10H1. | *genrfmA - generate jobs to reformat records based on 2 copybooks |
| example to unpack all packed/binary fields to numeric zoned fields |
| 10I1. | *copysub2 - collect related files from multiple subdirs to 1 output dir |
| appending subdir names, so common filenames not overwritten |
| 10J1. | *uvcmp1all - script to compare all files in 2 directories |
| - use to compare reconverted files to original mainframe files | |
| - to verify conversions | |
| & in case all applications can go live at the same time |
| 10K1. | cnvdatainit - script to clear ouputs of previous test/demos before re-runs |
| - clears d2asc,d4pipe,pfx1,pfx2,pfx3,pfp1,etc |
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
Only a few '*' are listed on the following pages, but you can view or print any of them from /home/uvadm/sf/IBM/...
| 10C1. | *cobmapA - listing of script to convert 1 copybook to a 'cobmap' |
| 10C2. | *gencnvA - script to generate job to convert 1 data file EBCDIC to ASCII |
| - combines jobs to create cobmap, convert EBCDIC to ASCII, | |
| & convert ASCII file to pipe|delimited in d4pipe/... |
gencnvAp - ensure packed sign x'_C' for AIX COBOL
gencnvD1 - generate all conversion jobs (for sites with no COBOL copybooks)
- gencnvD1 is not practical, since generating copybooks is not perfect
- you usually need to do manual corrections after signscan2 & sign2cpy1
- so run signscanA & sign2cpyA separately & then run gencnvD2 (below)
| 10D1. | *gencnvD2 - generate all conversion jobs from COBOL copybook |
| - cobmap1, uvdata51, genpipe1, genfixd2, sqlcreate1 (Oracle) |
gencnvD2p - same for AIX COBOL (ensure packed signs x'_C' vs x'_F'
| 10D2. | *gencnvD2.bat - same for Windows/DOS |
gencnvD9 - generates ALL conversion jobs for ALL copybooks in cpys/
- generates jobs in pfx1, pfp1, pfy1, pfq1, pfg1, sqlTC1, sqlTL1
gencnvE - generate uvcopy job to translate ASCII file back to EBCDIC
- opposite of gencnvA
- could then compare to original using 'uvcmpFE1' (script for uvcmp1)
signscanA - determine record layout when no COBOL copybook available
- script signscanA runs uvcopy job signscan2 which scans for
packed fields, zoned signs, date fields, and right blank filled
fields (such as Name & Address).
sign2cpyA - converts the signscanA record layout to a COBOL copybook which
is used by following scripts to perform various conversions.
genpipeA - generates uvcopy jobs to convert data to a '|' pipe delimited
ASCII text file. Packed & numeric fields will be converted to zero
suppressed numeric fields with leading sign +/-.
- Pipe delimited format is used for loading Relational data bases.
- also generates pipe|delimited back to Fixed-field (as of Feb2012)
listpipeA - by default lists the 1st 3 records of the pipe delimited file
first as is (1 long line), then field by field with copybook names
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
| 10E1. | *gencnvC - script to generate EBCDIC to ASCII jobs from directory |
| of filenames in d1ebc/..., using a control-file | |
| edited with copybook names corresponding to datafilenames | |
| (auto-generated if copybooknames embeded in datafilenames) |
| 10E2. | gencnvC2 - script to generate pipe|delimit jobs from directory |
| 10F1. | *uvcopypfxgA - Execute all uvcopy jobs in a directory(arg1) to convert all |
| files from 1 directory(arg2) to a 2nd directory(arg3) |
uvcopypfx1 - execute pfx1/..., pfp1/...,& listpipe jobs for any 1 file
- EBCDIC to ASCII & pipe|delimit jobs
| 10G2. | *uvcopypfx2 - execute pfx1/..., pfp1/...,& listpipe jobs for any 1 file |
| - EBCDIC to ASCII & pipe|delimit jobs | |
| - also creates uvhd & uvhdcob displays | |
| in subdirs uvhd_d1ebc, uvhd_d2asc, uvhdcob (d2asc only) | |
| - also see uvcopypfx9 to execute all jobs for all files |
uvcopypfx9 - execute ALL pfx1/* & pfp1/* jobs for ALL datafiles in d1ebc/*
| 10H1. | *genrfmA - generate jobs to reformat records based on 2 copybooks |
| example to unpack all packed/binary fields to numeric zoned fields |
| 10I1. | *copysub2 - script to gather all jobs & data for 1 file, from subdirs to 1 outdir |
| - appends _subdir names on filename, allowing cohabitation in 1 dir | |
| - might use to send results to other parties for inspection |
| Note |
|
| 10J1. | *uvcmp1all - compare all files in 2 subdirs |
| - use to compare reconverted EBCDIC files to original mainframe files | |
| - use for files with packed/binary fields | |
| & fixed length files without the LineFeeds required by unix 'diff' |
alldiff2 - run unix 'diff' for all pairs of files in 2 directories
- great for proving mass changes OK & no unwanted changes
alldiff2 pfx1.old pfx1 <-- creates diff reports in tmp/pfx1.dif
======================
alldiff2sub - run 'alldiff2' script for all pairs of subdirs in 2 superddirs
alldiff2 cnvdata1.old cnvdata1 tmp1
=================================
- create alldiff2 reports tmp1/cpys.dif, maps.dif, pfx1.dif, pfp1.dif, etc
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
No uvcopy jobs are listed here, but you can view or print any of them from /home/uvadm/sf/IBM/...
| signscan2 |
|
| sign2cpy1 |
|
| data2cpy51 |
|
| data2cpy52 |
|
| cobmap1 |
|
| uvdata51 |
|
| uvdata51p |
|
| uvdata52 |
|
| genpipe1 |
|
| listpipe1 |
|
| sqlcreate1 |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#!/bin/ksh
# cobmapA - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
# cobmapA - generate record layout from COBOL copybook
#
# This script makes it easier to run cobmap1 uvcopy job
# /home/uvadm/sf/cobmapA - script files Vancouver Utility subdir 'sf'
# /home/uvadm/pf/cobmapA - uvcopy jobs in subdir 'pf' (parameter files)
#
if [[ ! -f cpys/"$1" ]]; then
echo "usage: cobmapA copybookname [options]"
echo " =============================="
echo " - arg1 must be a file within subdir 'cpys' (specify filename only)"
echo " - output to subdir 'maps' with same filename"
echo " - arg2 options (example a1 to shift free-format over 6 cols)"
exit 1; fi
#
uvcopy cobmap1,fili1=cpys/$1,filo1=maps/$1,uop=q0i7$2,rop=r1
#===========================================================
echo "record layout created, you may view or print as follows:"
echo "more maps/$1 - display output"
echo "uvlp12 maps/$1 - print output (at 12cpi with UV laser printer script)"
exit 0
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#!/bin/ksh
# gencnvA - Generate uvcopy jobs to convert 1 data file EBCDIC to ASCII,
# ASCII to EBCDIC, fixed to delimited,& delimited to fixed
# - see www.uvsoftware.ca/datacnv1.htm#3B2
#Oct2014 - change genpipe1 to genpipe3 for fields > 100 bytes
# - change genfixd2 to genfixd3 for fields > 100 bytes
#
# gencnvA must run from superdir containing subdirs cpys,maps,pfx1,pfp1,pfq1,pfy1
# The copybooks (in cpys/...) are used by following utilties to generate uvcopy jobs
# in the following subdirs as follows:
#
# utility subdir ------- purpose -------
# cobmap1 - maps - record layouts (cobmaps) used to generate following jobs:
# uvdata51 - pfx1 - uvcopy jobs translate EBCDIC to ASCII
# uvdata31 - pfy1 - uvcopy jobs translate ASCII to EBCDIC
# genpipe3 - pfp1 - uvcopy jobs convert ASCII datafiles to delimited (default '|' pipe)
# genfixd3 - pfq1 - uvcopy jobs convert delimited files to fixed format (as per copybook)
#
#Note - gencnvA intended to generate jobs to process 1 file at a time easily
# - requires datafile to be renamed same as copybook (simplifies procedures)
# Also see www.uvsoftware.ca/datacnv1.htm#Part_6
# - to generate jobs for all copybooks
# - to process all datafiles in directory (using original datafilenames)
# - requires a 'control file' to relate copybookname to datafilename
# - utility 'uvdata52' reads control file & may generate multiple jobs
# to convert multiple datafiles for 1 copybook (often the case)
#
echo "gencnvA using: cpys,maps,pfx1,pfp1,pfq1,pfy1"
f1="$1"; # capture arg1 filename (should not have .cpy)
if [[ -f cpys/"$f1" && -d maps && -d pfp1 && -d pfq1 && -d pfy1 ]]; then :
else echo "usage: gencnvA CopyBookName"
echo " ===================="
echo "arg1 ($f1) not found in subdir cpys/..."
echo "- OR subdir missing - must have cpys,maps,pfx1,pfp1,pfq1,pfy1"
echo "must specify any .cpy extension if present"
echo " - but any .cpy extension removed on output files"
exit 9; fi
f2=${f1%.*} # remove any extension
#
uvcopy cobmap1,fili1=cpys/$f1,filo1=maps/$f2,uop=q0i7p0 # copybook to cobmap
#======================================================
uvcopy uvdata51,fili1=maps/$f2,filo1=pfx1/$f2,uop=q0i7 # EBCDIC to ASCII
#=====================================================
uvcopy genpipe3,fili1=maps/$f2,filo1=pfp1/$f2,uop=q0i7 # delimit fields > 100
#=====================================================
uvcopy genfixd3,fili1=maps/$f2,filo1=pfq1/$f2,uop=q0i7 # undelimit fields > 100
#=====================================================
uvcopy uvdata31,fili1=maps/$f2,filo1=pfy1/$f2,uop=q0i7 # ASCII to EBCDIC
#=====================================================
exit 0
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# gencnvD2 - generate all conversion jobs for any 1 copybook
# - cobmap1,uvdata51,uvdata31,genpipe1,genfixd2,sqlload1
# - by Owen Townsend, June10/2012,
# - see www.uvsoftware.ca/datacnv1.htm#4H1
#Oct07/14 - change genpipe1 to genpipe3 & genfixd2 to genfixd3
# - to allow pipe delimit/undelimit fields > 100 bytes
#
# gencnvD2 - generate all jobs using sqlcreate1 to create/load tables for Oracle
# gencnvD2p - same but calls uvdata51p to ensure packed signs x'_C' for AIX COBOL
# gencnvD3 - same as gencnvD2, but call sqlcreate2 for SQL Server vs Oracle
# gencnvD9 - same as gencnvD2, but generatesjobs for ALL copybooks in cpys/...
#
# gencnvD2 citytax1 <-- example (from DATAcnv1.doc)
# =================
#
# if [[ -d "$CNVDATA" ]]; then cd $CNVDATA fi
#Aug13/2014 - above disabled, sometimes you want to run elsewhere
#
fx="$1";
f=${fx%.*}; # remove any extensions
if [[ -f cpys/"$fx" && -d maps && -d pfx1 && -d pfy1 && -d pfp1 && -d pfq1 \
&& -d pfg1 && -d sqlTC1 && -d sqlTL1 ]]; then :
else echo "usage: gencnvD2 filename"
echo " ================="
echo "example: gencnvD2 custmas.cpy"
echo " ===================="
echo "- arg1 must be a file in subdir cpys/..."
echo "- any extension in cpys/...cpy will be removed for all other dirs"
echo "- must have outdirs: maps,pfx1,pfy1,pfp1,pfq1,pfg1,sqlTC1,sqlTL1"
exit 1; fi
#
uvcopy cobmap1,fili1=cpys/$fx,filo1=maps/$f,uop=q0i7p0
#=====================================================
uvcopy uvdata51,fili1=maps/$f,filo1=pfx1/$f,uop=q0i7
#===================================================
uvcopy uvdata31,fili1=maps/$f,filo1=pfy1/$f,uop=q0i7
#===================================================
uvcopy genpipe3,fili1=maps/$f,filo1=pfp1/$f,uop=q0i7 #Oct2014 fields > 100
#===================================================
uvcopy genfixd3,fili1=maps/$f,filo1=pfq1/$f,uop=q0i7 #Oct2014 fields > 100
#===================================================
uvcopy gentest1,fili1=maps/$f,filo1=pfg1/$f,uop=q0i7
#===================================================
uvcopy sqlcreate1,fili1=maps/$f,filo1=sqlTC1/$f.sql,filo2=sqlTL1/$f.ctl,uop=q0i7
#===============================================================================
exit 0
#
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
@echo off
rem gencnvD2.bat - generate conversion jobs for Windows systems
rem - cobmap1,uvdata51,genpipe1,sqlload1
rem - assumes COBOL copybook available in subdir cpys
rem - by Owen Townsend, Feb 24/2004
rem - see EBCDICjobs.doc or www.uvsoftware.ca/datacnv1.htm
rem
rem gencnvD2 filename <-- command format
rem =================
rem
rem gencnvD2 citytax1 <-- example (from EBCDICjobs.doc)
rem =================
rem
if "%1" == "" goto error
if exist dat1ebc\"%1" goto process
:error
echo "arg1 not a file"
echo "usage: gencnvD2 filename"
echo " ================="
echo "example: gencnvD2 citytax1"
echo " ================="
goto end
:process
uvcopy cobmap1,fili1=cpys/%1,filo1=maps/%1,uop=q0p0
uvcopy uvdata51,fili1=maps/%1,filo1=pfx1/%1,uop=q0
uvcopy genpipe1,fili1=maps/%1,filo1=pfp1/%1,uop=q0,rop=r0
uvcopy sqlcreate1,fili1=maps/%1,filo1=sqlTC1/create_%1.sql,filo2=sqlTL1/load_%1.ctl
:end
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#!/bin/ksh # gencnvC - Generate Data Conversion jobs for all copybooks in directory maps/ # - using control file of datafilenames & copybooknames # - by Owen Townsend, UV Software, July 2014 # - run from $CNVDATA with subdirs: ctl,cpys,maps,pfp1,pfp2,d2asc,d4pipe # # ** sample control file: datafilenames & copybooknames ** # # ar.customer.armaster cpy=____________ rcs=_____ typ=_____ # ar.sales.arsales cpy=____________ rcs=_____ typ=_____ # citytax1 cpy=____________ rcs=_____ typ=_____ # gl.account.glmaster_ cpy=____________ rcs=_____ typ=_____ dups=003 # gl.account.gltran_ cpy=____________ rcs=_____ typ=_____ dups=003 # # ** Pre-Requisite - Creating control file ctl/datacpy52 ** # # Control file must have been created as follows (3 ways); # (1. all manually, or 2. partial automated, or 3. fully automated) # # 1. vi ctl/datacpy52 <-- all manual insert copybooknames with editor # ================ - IF datafiles not on unix/linux ssystem yet # # --------- OR 2a,2b,2c ------------ # # 2a. uvcopy data2cpy51,fild1=d1ebc,filo1=ctl/data2cpy51 # ================================================== # IF datafiles have been FTP'd to unix/linux & stored in $CNVDATA/d1ebc/... # - create skeleton control file by extracting datafilenames from d1ebc/... # # 2b. cp ctl/data2cpy51 ctl/data2cpy52 # ================================ # - copy/rename so final edited filename always ctl/data2cpy52 # # 2c. vi ctl/data2cpy52 <-- edit/insert copybooknames in skeleton ctlfile # ================= # # --------- OR 3a,3b ------------ # # 3a. uvcopy datacpy52,fili1=ctl/datacpy51,filo1=ctl/datacpy52 # ======================================================== # IF copybooknames embedded in datafilenames - UNIQUE to some sites # - extract copybooknames from datafilenames, code on cpy=_____________ # # 3b. vi ctl/datacpy52 <-- inspect & fix if data2cpy52 not perfect # ================ # # ** ctl/datacpy52 result from method 1,2,OR 3 ** # # ar.customer.armaster cpy=armaster.cpy rcs=_____ typ=_____ # ar.sales.arsales cpy=arsales.cpy rcs=_____ typ=_____ # citytax1 cpy=citytax1.cpy rcs=_____ typ=_____ # gl.account.glmaster_ cpy=glmaster.cpy rcs=_____ typ=_____ dups=003 # gl.account.gltran_ cpy=gltran.cpy rcs=_____ typ=_____ dups=003 # # 4. gencnvC ctl/datacpy52 <-- run this script to generate data convert jobs # =====================
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
if [[ ! -f "$1" ]]; then
echo "usage: gencnvC control-file"
echo " ===================="
echo "example: gencnvC ctl/datacpy52"
echo " ====================="
exit 91; fi
#
if [[ -d cpys && -d maps && -d pfx1 && -d pfx2 && -d ctl ]]; then :
else echo "mandatory subdirs: cpys maps pfx1 pfx2 ctl"; exit 92; fi
echo "gencnvC process ALL cpys to maps,pfx1,pfx2 "
echo "- enter for step1, convert copybooks to record layout maps"; read
#
uvcopyx cobmap1 cpys maps uop=q0i7p0
#===================================
rmzf maps # remove any zero maps caused by procedure copybooks
#========
echo "enter to generate uvcopy jobs (in pfx1/...) from copybook maps"; read
uvcopyx uvdata51 maps pfx1 uop=q0i7
#==================================
#
echo "enter to copy \$UV/pf/skeleton2 to pfx1/ for files without copybooks"; read
cp $UV/pf/IBM/skeleton2 pfx1
#===========================
#
echo "enter to copy pfx1/* to pfx2/, replacing copybooknames with datafilenames"
echo "- using control-file of datafilenames & corresponding copybooknames"; read
uvcopy uvdata52,fili1=$1,fild2=pfx1,fild3=pfx2,uop=q0i7f1
#========================================================
exit 0
#
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# ** edit generated jobs (for multi-record-types) ** # # It may be necessary to make some manual changes to auto-generated jobs. # # A. Allow for any files with multiple record-types # - insert code to test R/T & skip to corresponding bank of instructions # already automatically generated from redefined records in copybook # - see example of inserting R/T test code for citytax2 file # at 'http://uvsoftware.ca/datacnv1.htm#5A1' - 5C2 # # B. Allow for any files with variable length records # - change file typ=RSF to typ=RDWz4 # # C. Corrections for missing copybooks (generates job from uvcopy skeleton) # - translates entire record to ASCII assuming no packed/binary fields # - could manually insert instrucitons to preserve packed/binary fields # # Note - should copy pfx2/* jobs to pfx/... before any editing # - so you do not lose manual changes if automatic generations rerun # # #6. cp pfx2/* pfx3 <-- copy auto-generted jobs before any editing # ============== # # #7. vi pfx3/... <-- make any manual changes required # =========== - as discusses above A,B,C # # ** Execute jobs to convert datafiles d1ebc --> d2asc ** # # #8. uvcopypfxgA pfx3 d1ebc d2asc # ============================ # - exewcute all jobs in pfx3/... # - to convert all files from d1ebc/... to d2asc/... # - jobs for GDG files are repeated depending on generations present #
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#!/bin/ksh # uvcopypfxgA - Execute directory of uvcopy jobs # - Data Conversion jobs in pfx3/... or pfp3/..., etc # - converts all members of GDG file sets # - by Owen Townsend, UV Software, July 2014 # # Special treatment for jobs with GDG files (filenames IF by trailing '_') # - Re-Executes the job for all members in the data input directory # uvcopy jobs in pfx2/pfx3 (or pfp2/pfp3,etc) were generated as follows: # # 1. uvcopy data2cpy51,fild1=d1ebc,filo1=ctl/datacpy51 # # ** control file created from directory of datafilenames ** # ar.customer.armaster cpy=____________ rcs=_____ typ=_____ # gl.account.glmaster_ cpy=____________ rcs=_____ typ=_____ dups=003 # # 2. cp ctl/datacpy51 ctl/datacpy52 <-- copy/rename # 3. vi ctl/datacpy52 <-- edit with copybooknames # # ** control file after inserting copybooknames ** # ar.customer.armaster cpy=armaster.cpy rcs=_____ typ=_____ # gl.account.glmaster_ cpy=glmaster.cpy rcs=_____ typ=_____ dups=003 # # --- OR - if copybooknames embedded in datafilenames (at some sites) # - data2cpy52 will transfer copybookname to cpy=____________ # # 2a. uvcopy data2cpy52,fili1=ctl/datacpy51,filo1=ctl/datacpy52 # 3a. vi ctl/datacpy52 <-- inspect, correct if auto not perfect # # 4. uvcopyx cobmap1 cpys maps uop=q0i7p0 # - convert copybooks to record-layouts # 5. uvcopyx uvdata51 maps pfx1 uop=q0i7 # - convert cobmaps to data conversion jobs in pfx1/... # 6. uvcopy uvdata52,fili1=ctl/datacpy52,fild2=pfx1,fild3=pfx2,uop=q0i7f1 # - replace copybooknames with datafilenames in pfx2/... # - via control file of datafilenames & copybooknames # # 4a. gencnvC - script alternative, runs #4,5,& 6 # # 7. cp pfx2/* pfx3 <-- copy to pfx3 before any edits for multi R/T files # # 8. vi pfx3/... <-- modify jobs for any files with record redefinitions # # 8. uvcopypfxgA pfx3 d1ebc d2asc # ============================ # - run this script to execute all data conversion jobs in pfx2/... # converting files from d1ebc/... to d2asc/... #
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
pfd="$1"; d1="$2"; d2="$3";
if [[ -d "$pfd" && -d "$d1" && -d "$d2" ]]; then :
else echo "usage: uvcopypfxgA pfjobs indir outdir"
echo " ==============================="
echo "example#1: uvcopypfxgA pfx3 d1ebc d2asc"
echo " ============================"
echo "example#2: uvcopypfxgA pfp3 d2asc d4pipe"
echo " ============================="
exit 91; fi
#
echo "$0 execute jobs in $1 to convert files from $2 to $3"
echo "- Repeats jobs for all members of GDG file groups"
echo "- enter to proceed"; read reply
#
for pfduvj in $pfd/*
{ uvj=$(basename $pfduvj)
if [[ $uvj == *_ ]]; then
for d1fn in $d1/${uvj}*
{ fn=$(basename $d1fn)
uvcopy $pfd/$uvj,fili1=$d1/$fn,filo1=$d2/$fn
#===========================================
}
else
fn=$uvj
uvcopy $pfd/$uvj,fili1=$d1/$fn,filo1=$d2/$fn
#===========================================
fi
}
exit 0
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#!/bin/ksh
# genrfmA - Korn shell script from UVSI stored in: /home/uvadm/sf/IBM/
# genrfmA - Generate uvcopy job to reformat a data file
# based on old & new versions of the copy-books
# - unpack packed fields, or reformat records for any reason
# - see REFORMjobs.doc or DATAcnv1.doc#4I1
#
# genrfmA original_copybook modified_copybook
# ===========================================
#
# This script executes the multi-steps to generate the uvcopy job
# - For use when you only have 1 or 2 files to do
# - For mass conversions you should the 'ALL FILES IN DIRECTORY'
# operating instructions documented in section 'G' of REFORMjobs.doc
#
# Run from cnvdata superdir with subdirs: cpys,maps,mapsI,pfr1
# Two input files must be present:
# 1. original copybook & modified copybook in subdir cpys
#
# This script runs 4 uvcopy jobs:
# 1. cobmap1 converts original copybook from cpys to cobmap in maps
# 2. cobmap1 converts modified copybook from cpys to cobmap in maps
# 3. reform1 loads original copybook to indexed file mapsI/original copybook
# with fieldnames as keys
# 4. reform2 reads modified copybook, looks up indexed file of origional fields
# & writes out uvcopy job to move/unpack data fields
#
# Alternate similar scripts:
# genrfmB - executes additional step 'uvdata52', which inserts the actual
# datafilename (vs copybookname) using a control file
# genrfmD - same as genrfmB for sites with duplicate fieldnames in copybooks
#
cb1="$1"; cb2="$2";
echo "genrfmA using: cpys,maps,mapsI,pfr1"
if [[ -f cpys/$cb1 && -f cpys/$cb2 && -d maps && -d mapsI && -d pfr1 ]]
then :
else echo "usage: genrfmA copybook1 copybook2"
echo " ==========================="
echo "- arg1 ($cb1) or arg2 ($cb2) not found in cpys"
echo "- or subdirs missing (cpys,maps,mapsI,pfr1)"
exit 9; fi
#
echo "#1 - convert original copybook to cobmap cpys/arg1 --> maps/arg1"
echo " - enter to proceed"; read reply
uvcopy cobmap1,fili1=cpys/$cb1,filo1=maps/$cb1,uop=q0i7p0
echo "#2 - convert modified copybook to cobmap cpys/arg2 --> maps/arg2"
echo " - enter to proceed"; read reply
uvcopy cobmap1,fili1=cpys/$cb2,filo1=maps/$cb2,uop=q0i7p0
echo "#3 - load original cobmap to Indexed file maps/arg1 --> mapsI/arg1"
echo " - enter to proceed"; read reply
uvcopy reform1,fili1=maps/$cb1,filo1=mapsI/$cb1,uop=q0i7
echo "#4 - generate reformat job maps/arg2 + mapsI/arg1 --> pfr1/arg2"
echo " - enter to proceed"; read reply
uvcopy reform2,fili1=maps/$cb2,filo1=pfr1/$cb2,filr1=mapsI/$cb1,uop=q1
exit 0
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# uvcopypfx2 - execute EBCDIC to ASCII & pipe|delimit jobs for 1 file
# - also creates listpipe/... report datafields with copybook names
# - also creates uvhd & uvhdcob reports for d1ebc & d2asc file
# - by Owen Townsend, UV Software, Oct19/2013
# - see www.uvsoftware.ca/datacnv1.htm#Part_4
# uvcopypfx1 - same, but omits uvhd & uvhdcob reports
# uvcopypfx9 - executes ALL (vs 1) jobs in pfx1/ pfp1/ + listpipe reports
#
# gencnvD2 - must have been run to create conversion jobs in pfx1/ & pfx2/
# via cobmap1,uvdata51,uvdata31,genpipe1,genfixd2,sqlcreate1
# from copybook in cpys/xxx & maps/xxx
#
# if [[ -d "$CNVDATA" ]]; then cd $CNVDATA fi
#Aug13/2014 - above disabled, sometimes you want to run elsewhere
#
echo "execute EBCDIC to ASCII & pipe|delimit jobs for 1 specified file"
echo "- also list 1st 3 pipe delim records vertically with copybook names"
f="$1";
if [[ -f pfx1/"$f" && -f pfp1/"$f" && -f d1ebc/"$f" && -d d2asc && -d d4pipe ]]
then :
else echo "usage: uvcopypfx2 filename"
echo " ==================="
echo "example: uvcopypfx2 citytax1"
echo " ==================="
echo "- arg1 must be a file in subdir d1ebc, pfx1, pfp1, etc"
echo "- must have outdirs: d2asc, d4pipe, listpipe, uvhd_d1ebc, etc"
exit 1; fi
#
uvcopy pfx1/$f,uop=q0i31
#=======================
uvcopy pfp1/$f,uop=q0i31
#=======================
uvcopy listpipe1,fili1=d4pipe/$f,filo1=listpipe/$f,uop=q0i31
#===========================================================
# get rec-size from maps for uvhd & create uvhd & uvhdcob reports
test -d uvhd_d1ebc || mkdir uvhd_d1ebc
test -d uvhd_d2asc || mkdir uvhd_d2asc
test -d uvhdcob || mkdir uvhdcob
rcsz1=$(grep '\*RCSZ=' maps/$f); rcsz2=${rcsz1##*=}
echo -e "\n\nq" | uvhd d1ebc/$f ak4r${rcsz2} >uvhd_d1ebc/$f
echo -e "\n\nq" | uvhd d2asc/$f k4r${rcsz2} >uvhd_d2asc/$f
echo -e "\n\nq" | uvhdcob d2asc/$f maps/$f f1k4m500 >uvhdcob/$f
echo "head d4pipe/$f.dat <-- inspect pipe delimited report"
echo "more listpipe/$f <-- inspect listpipe report (1st 3 records)"
echo "more uvhd*/$f <-- inspect uvhd_d1ebc,uvhd_d2asc,& uvhdcob rpts"
exit 0
#
| Note |
|
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#!/bin/ksh
# copysub2 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
# copysub2 - copy filex from multiple subdirs in a superdir to an outdir
# - by Owen Townsend, UV Software, Oct19/2013
# copysub2 - copies filename specified with '*' wildcard appended
# to allow for any different extensions in different subdirs
# copysub1 <-- alternate script that copies exact filename specified
#
# Renames filex by apending '_' + subdirname, for example:
# Given: cnvdata/cpys/filex, cnvdata/maps/filex, cnvdata/pfx1/filex
# 1. cd ...above... cnvdata/...
# 2. mkdir tmp1
# 3. copysub2 filex cnvdata tmp1
# ===========================
# 4. ls tmp1 --> tmp1/filex_cpys, tmp1/filex_maps, tmp1/filex_pfx1
#
# Useful for collecting data conversion files from multi subdirs in cnvdata/...
# - since same filename used in multi subdirs
# - output files identified by subdir-name appended
# - then might zip files & send to clients as 1 archive
#
echo "copy filex from multi subdirs in superdir to outdir"
echo "- rename/append subdirs: filex_subdir1, filex_subdir2, etc"
f="$1"; d1="$2"; d2="$3";
if [[ -d "$2" && -d "$3" ]]; then :
else echo "usage: copysub2 filex superdir outdir"
echo " =============================="
echo " - arg1 must be a filename"
echo " - arg2 & arg3 must be directories"
exit 1; fi
x=0;
for d1sd in $d1/*
{ sd=$(basename $d1sd)
sdf=${f}_$sd
ls $d1/$sd/$f >/dev/null 2>/dev/null
if ((!$?)); then
cp $d1/$sd/$f $d2/$sdf
let x=x+1
echo "copy file# $x $d1/$sd/$f to $d2/$sdf"
fi
}
echo "total $x files copied from $d1/.../$f to $d2/..."
exit
#Oct27/14 - removed '*' from $f on ls & cp aabove
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
#!/bin/ksh
# uvcmp1all - script to compare all files in 2 directories
# - by Owen Towsnend, Feb 2010 at Everis
# calls 'uvcmp1' for fixed length files w/o LineFeeds
# - shows record pairs in vertical hexadecimal
# with mismatches flagged with '*'s
# uses Indexed control file to get record size (ctl/ctlfile4I)
# - same control file as used for EBCDIC to ASCII conversion
#
echo "uvcmp1all - compare all files in 2 directories, using uvcmp1"
echo "- to show record pairs in vertical hexadecimal with '*' mismatches"
echo "- for files with packed/binary & no LineFeeds (use diff for text files)"
d1="$1"; d2="$2";
if [[ -d "$1" && -d "$2" ]]; then :
else echo "usage: uvcmp1all dir1 dir2 [options]"
echo " ============================="
echo "ex1: uvcmp1all d2asc.old d2asc"
echo " ========================="
echo "ex2: uvcmp1all d1ebc d3ebc y1"
echo " ========================"
echo "- option y1 for EBCDIC not req'd if dirname contains *ebc*"
exit 99; fi
echo "uvcmp1 $d1/... $d2/..."
echo "- will write mismatch reports to subdir rptcmp/..."
reply=x;
until [[ "$reply" = "y" || "$reply" = "n" ]]
do echo "- clear all old mismatch reports from rptcmp/... y/n ?"
read reply; done
if [[ "$reply" == "y" ]]; then rm -f rptcmp/*; echo "rptcmp/* removed"; fi
x=0;y=0;
for d1f in $d1/*
{ let x=x+1
f=$(basename $d1f)
if [[ $f == *.idx ]]; then continue; fi
if [[ ! -f $d2/$f ]]; then
echo "bypassing nomatch in $d2 for $d1/$f"
continue; fi
#
# use uvcopy job to get recsize & key options from control file
# - will write to: /tmp/${LOGNAME}.uvcmp
# remove any '.dat' (not coded in control file)
fx=${f%.dat}
uvcopy uvcmpget2,filr1=${CNVDATA}/ctl/ctlfile4I\
,filo1=/tmp/${LOGNAME}.uvcmp,arg1=$fx
if (( $? != 0))
then echo "datafilename not in control file"; exit 2;fi
#
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# open the file written by uvcopy & retrieve options into $variables
exec 3< /tmp/${LOGNAME}.uvcmp # open file written by uvcopy
read -u3 copyname ctlops other # read file contents into variables
exec 3<&- # close file
#
# if dirname contains 'ebc' add option y1 to trnslt chars to ASCII
if [[ $d1 == *ebc* ]]; then ctlops=${ctlops}y1; fi
#
echo "file# $x $d1/$f $d2/$f"
uvcopy uvcmp1,fili1=$d1/$f,fili2=$d2/$f,rop=r0,uop=q0i7$ctlops
#=============================================================
if (( $? != 0)); then ((y+=1)); fi
}
echo "$y mismatches of $x total files compared in $d1 & $d2"
echo "vi rptcmp/* <-- view all mismatch reports"
echo "==========="
echo "uvlpd12 rptcmp <-- print all mismatch reports"
echo "=============="
exit 0
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page
# cnvdatainit - clear outputs of prior test/demos before re-runs
# - by Owen Townsend, Feb2007, update Aug2021
#
# Clear files that will be created when you re-run test/demo conversions
# - see www.uvsoftware.ca/datacnv1.htm#3B1
# Assuming - Vancouver Utilities installed at /home/uvadm/...
# - you have setup userid mvstest1 & changed to /home/mvstest1/cnvdata1
# - copied /home/uvadm/mvstest/mvstest1/* to /home/mvstest1/...
# (OR copy mvstest1/* to your own homedir & setup exports in .bash_profile)
# - have already run some conversions & now want to rerun with output dirs cleared
# - after running this cnvdatainit, restore EBCDIC datafile input as follows:
# #1 ----> cp archive/d1ebc_cpy_name/* d1ebc/. <-- datafilenames same as copybooknames
# OR#2--> cp archive/d1ebc_data_name/* d1ebc/. <-- real datafilenames aaa.bbb.ccc.etc
# #2 for Part_6 of DATAcnv1.doc or for all conversions in DATAcnv2.doc
#
# OR, you have converted some of your own files & already saved results
# - and you now want to convert a new set of data files
# - run this cnvdatainit script (clears all data subdirs & generated uvcopy jobs)
# - copy new set of data files to d0ebc (or d1ebc if already lowercase names)
# - copy new set of copybooks to cnvdata1/cpys/...
#
echo "clear CNVDATA subdirs before regen uvcopy jobs & reconvert data files"
echo "clear d0ebc,d1ebc,d2asc,d3ebc,maps,pfx1,pfx2,pfx3,etc,tmp,tmp1,etc"
reply=x;
until [[ "$reply" = "y" || "$reply" = "n" ]]
do echo "- clear CNVDATA subdirs before reconvert y/n ?"
read reply; done
if [[ "$reply" != "y" ]]; then echo "$0 aborted"; exit; fi
cd $CNVDATA
rm -f d[0-9]*/* # clear d0ebc,d1ebc,d2asc,d3asc,d4pipe,d5fix
rm -f listpipe/*
rm -f maps/* # clear record layouts (converted from cpys/...)
rm -f pf[a-z][0-9]/* # clear pfx1,pfx2,etc,pfy1,pfp1,etc
rm -f tmp*/* # clear tmp,tmp1,tmp2,etc files
echo "cleared: d0ebc,d1ebc,d2asc,d3ebc,maps,pfx1,pfx2,pfx3,etc,tmp,tmp1,etc"
exit 0
#
Goto: Begin this doc , End this doc , Index this doc , Contents this library , UVSI Home-Page