Goto:   Begin this document ,  End this document ,  UVSI Home-Page

uvsort - sort utility for Unix/Linux - CONTENTS (continued)

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

uvsort - sort utility for Unix/Linux - CONTENTS (continued)

uvhd - free sample of Vancouver utilities

'uvhd' is a free sample of Vancouver utilities, that you can download from https://uvsoftware.ca/libuvhd.htm. uvhd is a file investigation utility, especially relevant for files with packed/binary fields. It displays any file in vertical hexadecimal & prompts for commands to browse, search, select, update, translate, etc. uvhd has 25 help screens and is all my customers favourite tool.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

A1. uvsort Overview

'uvsort' is a powerful utility that provides mainframe sorting capability to UNIX systems. It is about 20% faster than Micro Focus 'mfsort'. It is not as fast as the unix system sort, but the unix system sort is totally unsatisfactory for mainframe file types (migrated to unix systems) because:

1. unix sort cannot sort fixed length record files without LineFeeds
2. unix sort cannot sort on packed decimal fields
3. unix sort cannot sort Indexed files (Micro Focus compatible)
4. unix sort does not have the select/delete/reformat capabilities of uvsort or mfsort (both uvsort & mfsort have capabilties similar to mainframe DFSORT or SYNCSORT)
   

uvsort provides for 600 record selects/deletes for input time & a further 600 record selects/deletes for output time. These select/deletes may be based on multiple and/or conditions comparing record fields to constants. uvsort also provides for 600 instructions for reformatting records at both input time & output time (move,add,subtract,multiply,compare,skip,etc).

uvsort is very similar to the uvcp utility with the addition of sort keys. Keys may be specified as separate parameters or as a string, for example:

        key1=0(8),key2=20(15),key3=30(5pd)
                      -or-
        keys=(0,8,c,a,20,15,c,a,30,5,p,d)

Up to 9 keys are allowed in format 1 & up to 15 keys in format 2.

This uvsort section will repeat some documentation for file declarations which are similar to uvcp but not identical (re multifile input).

This uvsort section will not repeat any documentation for the processing instructions (add,sub,mpy,mvc,etc) since they are identical to the uvcp utility & have already been documented in the uvcp section.

uvsort pricing

uvsort is sold as part of the Vancouver Utiltities package class B, which includes all utilities except tof the JCL converter. The class B price is $4800 US, but see https://uvsoftware.ca/uvprices.htm#B3 for all details including support options.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

A2. uvsort Overview

uvsort prelimnary examples

Here are some preliminary examples, before presenting all the details for the many parameters & options documented on the following pages.

 #1. uvsort "fili1=dat1/sales2,rcs=64,typ=RST,filo1=tmp/s2,rcs=64,key1=30(6),sel1=10(2):21"
     ======================================================================================
     - sort sales2 demo file by product#, selecting only salesman #21
     - output 'typ=RSF' Record Sequential Fixed, recsize=64, no linefeed
     - could specify 'typ=RST' to force Linefeed into the last 64th byte
     - 'key1=30(6)' is product# in bytes 30-35 (zero relative) columns 31-36 (one relative)
     - see https://uvsoftware.ca/uvsort.htm#L3 for sales file layout & results expected
     - this example has only 1 select, but page 'L3' has 6 selects (600 maximum)

 #2. uvsort "fili1=dat1/custmas1,rcs=256,typ=RSF,filo1=tmp/cm1,typ=ISF,key1=0(6),isk1=0(6)"
     ======================================================================================
     - sort & load an Indexed sequential file (C-ISAM & Micro Focus COBOL compatible)
     - 'key1=0(6)' is the sort key (cust# 1st 6 bytes)
     - 'isk1=0(6)' Indexed Sequential Key to load output file (1st 6 bytes)
     - 'rcs=256' input recsize, output will be 257 (Indexed file status byte added)
     - see https://uvsoftware.ca/uvsort.htm#L4 for customer demo file layout

 #3. uvsort "fili1=dat1/custmas1,rcs=256,typ=ISF,filo1=tmp/custphones,rcs=80,typ=LSTt\
            ,key1=10(25),clr=35(45),mvc=35(12)90"
     =================================================================================
     - sort customer master by customer-name & reformat into a telephone list
     - converts Indexed file 'typ=ISF' to a text file 'typ=LSTt'
     - input fixed 256 byte records, output max 80 LineFeed after last non-blank
     - '\' continues script to a 2nd line
     - 'key1=10(25)' sorts on customer-name cols 11-35 (zero relative 10-34)
     - 'clr=35(45)' blanks the address, before telephone# moved here
     - 'mvc=35(12)90' moves the telephone# from cols 91-112 to cols 36-47
     - see https://uvsoftware.ca/uvsort.htm#L6 for demo file results expected

 #4. uvsort "fili1=dat1/sales1,rcs=64,typ=RST,filo1=tmp/salecustomers,key1u1=0(6)"
     =============================================================================
     - sort sales file with data reduction option 'u1' of 'key1u1=0(6)'
     - unique key option retains only 1st record of duplicate key set (cust#s)
     - see https://uvsoftware.ca/uvsort.htm#C5 for duplicate drop & select options

 #5. uvsort "fili1=dat1/sales1,rcs=64,typ=RST,filo1=tmp/salesman_,key1f1=10(2)"
     ==========================================================================
     - sort sales file with option 'f1' of 'key1f1=10(2)' (salesman cols 11-12)
     - option 'f1' creates a separate output file for each salesman
     - appending the sort key (salesman#) on to output filename
     - see https://uvsoftware.ca/uvsort.htm#C7 for demo file results expected

 #6. uvsort "fili1=dat1/sales,typ=RSF,rcs=100,filo1=tmp/sales,key1f1=0(6),rop=x20000000"
     ===================================================================================
      This illustrates Run OPtion 'x' (rop=x____) to split output into multiple
      files of a Max number of records (except for last output file).
      'rop=x20000000' 20,000,000 records * 100 = 2,000,000,000 bytes
      Assuming a 5 gig input file, the output files would be:
      sales_001 (2 gig), sales_002 (2 gig), sales_003 (1 gig)
      rop 'x#' is used in conjunction with 'key' option 'f1' key1f1=...rop=x...

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

A3. uvsort: similar sort facility is also provided in uvcopy

alternate version 'uxsort' for IDXFORMAT3/8 files

'uxsort' is an alternate version of uvsort that includes support for Micro Focus COBOL IDXFORMAT3/8 files (Indexed Sequential Variable length records). You must execute uxsort to read or write 'typ=IDX' files. Note that Fixed length Indexed files (compatible with Micro Focus COBOL IDXFORMAT1) are supported by both uvsort & uxsort.

'uxsort' must be compiled with the Micro Focus 'cob' command. Please see install.doc which describes the use of the 'cobccuv' script which makes it easy to compile Vancouver Utility programs using the 'cob' command.

similar sort facility also provided in uvcopy

Uvcopy also includes the uvsort facility, implemented in a more powerful manner, but not as convenient for simple sorts, because uvsort can specify all parameters on the command line.

In uvcopy, the sort is implemented as several instructions for the various sort phases, which provides a lot of flexibility.

For example, you can reformat the records as you are giving them to the sort, then execute the sort,& then load an indexed file as you are retrieving the records from the sort (possibly appending sequence#s to make unique keys).

See the sort instructions documented at the end of uvcopy3.doc (sxo,sxp,sxs,sxg,sxc).

For some particularly good examples of using the sort within uvcopy please see the following jobs in other sections of the documentation.

   index  - in section TEXTjobs.doc
          - create an index from a document containing keywords & page ejects

   permute - also in the TEXTjobs section.
           - create a permuted index from a table of contents

DFSORT/SYNCSORT to uvsort by JCL converter

The JCL converter automatically connerts mainframe SORT steps to uvsort. DFSORT/SYNCSORT syntax is converted to the more compact uvsort equivalents. INCLUDE/OMIT is converted to sel/del, INREC/OUTREC FIELDS are converted to uvsort equivalents. uvsort provides up to 600 selects/deletes on input and another 600 on output. Up to 600 FIELD reformats are also allowed at both input & output time. See example of converting JCL with SORT OMIT to Korn shell script uvsort delete at https://www.uvsoftware.ca/JCLcnv1demo.htm#2B1.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

A4. Enhancements to uvsort

Jan2004 Enhancements to uvsort

uvsort now allows you to specify up to 20 input files & up to 61 output files. On input you may specify record selects/deletes/reformats that are applied to all records. On output you may specify record selects/deletes/reformats that are different for different output files.

A maximum of 20 input & 61 output files apply when you specifically name the desired files (fili1=...,fili2=...,fili20=...,filo1=...,filo2=...,filo60=...).

An alternative to specifying multiple input filenames is to use the new run option 'f2', to input all files from a specified directory, OR all filenames matching a specified prefix &/or suffix. See option 'f2' on page 'F5'.

For output files, you may specify option 'f1' on key1 (key1f1) to output multiple separate files for duplicate record key sets based on key1. This is a unique & powerful feature that can be used to write out separate files for each customer#,salesman#,etc. See option 'f1' on page 'C5'.

The output filenames are automatically created in the specified output directory, with an optional prefix, followed by the current key1 data value. These enhancements make uvsort even more powerful than SYNCSORT.

See option 'o#' to select/delete records to different output files page 'G3'. See option 'o#' to execute instructions on different output files page 'I2'. See SUM function to accumulate fields into 1 record for duplicate key sets page 'H1'.

Oct2018 Enhancement

The 'XSUM' functionality of SYNCSORT is now provided by declaring output file #61. File #61 will receive records deleted by 'sum=none' or sum=dsp(lth) acum field/ Intended for mainframe conversions. You might define $SORTIN,$SORTOUT,$SORTXSUM in the JCL/script & uvsort would be:

 uvsort "fili1=$SORTIN,rcs=...,typ=...,filo1=$SORTOUT,filo61=$SORTXSUM"
 =====================================================****************=

Nov 2004 Enhancements

uvsort now maintains the original file record sequence. This is effective when multiple records exist for the specified sort keys. This feature is implemented by automatically adding an extra low order sort field with an input record sequence number. For best performance, uvsort uses the 'qsort' algorithm which does not guarantee original record sequence.

sort field type 'g'

Sort field type 'g' changes the character collation sequence. The exact sequence is shown on page C3 of https://www.uvsoftware.ca/uvsort.htm#C3. This was requested by customers converting DB2 UDB applications from the mainframe to unix/linux. Other sites might use this if they want UPPER & lower case letters to sort together (vs the separate ranges of ASCII).

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

A5. Enhancements to uvsort

uvsort Max Memory Increased

You can now specify up to 2 gigabytes of memory for uvsort. The same sort is also built in to some other utilities (uvcopy,uvqrpg,uxcopy,uxqrpg). The default Max Memory is now 512 megabytes (vs 64 megabytes). Sort times may not be much different for small files, but could be dramatically improved for large files (especially if over 12 gigabytes).

The number of workfiles has also been increased from 50 to 100. These work files are created in the 'tmp'subdir while reading the input file(s). These sorted workfiles are then merged during the sort output phase. If (WorkFiles * MaxMemory) < filesize, the sort will be efficient, there will be no need for intermediate sorts & merges.

You may increase the Max Memory via run option 'm'. For example 'rop=640' to specify 640 megabytes (vs the 512 meg default):

 uvsort "fili1=bigfile,typ=RSF,rcs=351,filo1=sorted,key1=71(20),rop=m640"
 ========================================================================

max filename lengths increased

The maximum filename lengths for uvcopy,uvqrpg,uxcopy,& uxqrpg have been increased from 80 bytes to 180 bytes. Note that uvsort,uvcp,uxsort,& uxcp already allowed 200 bytes for maximum filename length.

Feb 2019 - edit instruction added

In Feb 2019, we added an edit instruction, for example, here is our test script:

 # testsortedt1 - test uvsort edit instrn Feb27/2019
 #  - script to test/demo 'edt' instruction, stored in $UV/sf/adm/testsortedt1
 uvsort "fili1=dat1/sales2,rcs=100,typ=LSTt,filo1=tmp1/sales2,edt=71(13)53(9):z;zzz;zzz.99-"
 #==========================================================================================
 exit 0

You can test as follows:

 #1. Login as uvadm or yourself if your profile setup for Vancouver Utilities
 #2. mkdir tmp1       <-- make output directory (if not existing)
 #3. cat dat1/sales2  <-- display input file
 #4. testsortedt1     <-- execute script
     ============
 #5. cat tmp1/sales2  <-- display output file

See edt documentation at https://uvsoftware.ca/uvcp.htm#M3 (instructions common to both uvsort & uvcp, documented in uvcp.doc).

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

B1. uvsort FUNCTION SUMMARY

 fili1=infilename           - declare input filename
 fili20=...                 - up to 20 input files may be declared
 fili1=dir/*                - input files from directory option (rop=f2)

 typ=LST,RSF,RST,RSV,ISF,IDX,STL - declare file type
 typ=LSTt                          & possible options appended to file type

 rcs=9999                   - declare record size

 filo1=outfilename          - declare output filename
 filo60=...                 - up to 60 output files may be declared
 filo61=...                 - receives any deleted records 'XSUM' equivalent
 filo1=dir/*                - multi-file output by option 'f1' on key1 breaks

 iskx=start(length)         - declare indexed file keys

 keys=(0,8,c,a,30,5,p,d)    - declare sort keys (format 1)
         -or-
 key1=0(8),key2=30(5pd)     - declare sort keys (alternate format)
                            - options to delete or select duplicate keys

 rop=abc...xyz              - declare misc run options

 sel=start(lth):constant     - select records with spcfd value in spcfd cols
 del=start(lth):constant     - delete records with spcfd value in spcfd cols

prm updates Oct 2013

uvsort now allows any combination of prm=... & command line args, examples:

 uvsort prm=pfile1      <-- all commands in pfile1 (searches $PFPATH)
 uvsort pfile1          <-- same as above

 uvsort prm=pfile1,cmd1=aaa,cmd2=bbb,cmd3=ccc,etc,prm=pfile2
 uvsort      pfile1,cmd1=aaa,cmd2=bbb,cmd3=ccc,etc,prm=pfile2
 - could have all 3 combinations, more practical examples follow:

 uvsort "fili1=dat1/sales2,rcs=64,typ=RST,filo1=tmp/s2,key1=30(6),del=13(2)<12"
        - "del=13(2)<12" drops records older than 2012
        - might put in prm=file if used on multiple sorts thruout scripts

uvsort "fili1=dat1/sales2,rcs=64,typ=RST,filo1=tmp/s2,key1=30(6),prm=sortdropold"

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

B2. uvsort INSTRUCTION SUMMARY

      logical:  mvc, mvn, edt, clr
   arithmetic:  add, sub, mpy, div, pac, unp, seq
 compare/skip:  cmc, cmn, skp
 scan/replace:  rep, scn, sqz
   translates:  tra, tre, trl, tru, trp
        other:  anc, orc, chx, hxc, tim
          I/O:  put

 The instructions are the same as for the 'uvcp' utility
 - please see instructions documented at uvsoftware.ca/uvcp.htm#M0

 Also see areas 'x' & 'y' documented at uvsoftware.ca/uvcp.htm#O1
 - containing system info such as dates & times that you could insert
   into output records.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

C1. uvsort: sort KEY specification

sort key format #1 - up to 28 keys allowed

keys=(displacement,length,data-type,sequence, ... etc )

keys=(dsp1,lth1,typ1,seq1,dsp2,lth2,typ2,seq2,----etc----)

 keys=(0,8,c,a)            - sortkey = cols 1-8, character, ascending

 keys=(0,8)                - same (character,ascending is the default)

 keys=(0,8,c,a,228,3,p,d)  - sortkey #1 = cols 1-8 character, ascending
                           - sortkey #2 = cols 229-231 packed, descending

sort key format #2 - up to 9 keys allowed

key#=displacement(length)

key1=dsp1(lth1),key2=dsp2(lth2),key3=dsp3(lth3),etc

 key1=0(8ca)            - sortkey #1 is cols 1-8
                        - character ascending

 key1=0(8)              - sortkey #1 is cols 1-8
                        - character ascending is the default

 key2=228(3pd)          - sortkey #2 is cols 229-231
                        - packed decimal, descending

Sort key format#2 allows fewer sort fields, but more options, which will be described on the following pages.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

C2. uvsort: sort key specification

sort field data types

sort field sequence codes

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

C3. uvsort: Sort Key Specifications

Sort Sequence for field type 'g'

 uvsort "fili1=infile,rcs=80,typ=RSF,filo1=outfile,key2=20(30g)"
 ============================================================*==

Field type 'g' can be used to sort to the sequence shown in the table below:

 00-1F - control characters, reassigned to x'FF' to sort high
 20-2F - B _ - , ; : ! ? / . ` ^ ~ ' " (  ('B' is Blank)
 30-3F - ) [ ] { } @ $ * \ & # % + < = >
 40-4F - | 0 1 2 3 4 5 6 7 8 9 a A a A b  (2nd 'a A' with diaeresis)
 50-5F - B c C d D e E f F g G h H i I j
 60-6F - J k K l L m M n N o O o O p P q  (2nd o O with diaeresis)
 70-7F - Q r R s S s t T u U u U v V w W  (2nd 's' sharp, 2nd 'u U' diaeresis)
 80-85 - x X y Y z Z
 86-FE - not used
       - some German characters reasigned to lower sort sequences
 FF    - null & control characters reassigned to sort high

ASCII ISO 8859-1 sort sequence

For comparison the normal (default) character sort sequence is shown below:

 00-1F - control characters (null,soh,stx,etc,...)
 20-2F - B !  " # $ % & ' () * + , - .  /   ('B' is Blank)
 30-3F - 0 1 2 3 4 5 6 7 8 9 : ; < = > ?
 40-4F - @ A B C D E F G H I J K L M N O
 50-5F - P Q R S T U V W X Y Z [ \ ] ^ _
 60-6F - ` a b c d e f g h i j k l m n o
 70-7F - p q r s t u v w x y z { | } ~ D    ('D' is DEL char)
 80-9F - extended control characters
 AF-FF - extended characters for European languages
       - not used in US ASCII ISO 646 (subset of ISO 8859)
       - a few characters German language characters are shown
       - field type 'g' reassigns these to lower values (see above)
 0xC4 - 'a' with diaeresis
 0xD6 - 'O' with diaeresis
 0xDC - 'U' with diaeresis
 0xDF - 'S' sharp
 0xE4 - 'A' with diaeresis
 0xF6 - 'o' with diaeresis
 0xFC - 'u' with diaeresis

uvcopy option rop=t1 for codepage 1141

uvcopy has Run OPtion 'rop=t1' to use odepage 1141 for EBCDIC to ASCII data conversions. See https://uvsoftware.ca/datacnv1.htm#7M3.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

C4. uvsort: Sort Key Specifications

Sort Sequence field type 's'

Field type 's' can be used to sort lowercase alphas a-z & UPPERcase A-Z together as shown in the table below:

00-3F - same as shown for ASCII ISO 8859-1 on previous page

   ----- default 8859-1 sort sequence -----
 40-4F - @ A B C D E F G H I J K L M N O    <-- UPPERcase
 50-5F - P Q R S T U V W X Y Z [ \ ] ^ _
 60-6F - ` a b c d e f g h i j k l m n o    <-- lowercase
 70-7F - p q r s t u v w x y z { | } ~ D

   ----- field type 's' lowercase/UPPERcase together -----
 40-4F - @ a A b B c C d D e E f F g G h
 50-5F - H i I j J k K l L m M n N o O p    <--lower/UPPER merged
 60-6F - P q Q r R s S t T u U v V w W x
 70-7F - X y Y z Z [ \ ] ^ _ ` { | } ~ D

80-FF - extended control characters sequence unchanged

test sort sequence type 's'

You can test type 's' sort sequence as follows:

 uvsort "fili1=tf/testsortseq,rcs=8,typ=LSTt,filo1=tmp/s2,key1=0(8s)"
 =================================================================*==

 The input file is actually 94 records of 1 byte each
 (defined as 8 byte records & 8 byte sort field above).
 Here are the 94 characters in the input file sequence
 - shown with 26 characters per line to save space

 Z Y X W V U T S R Q P O N M L K J I H G F E D C B A
 z y x w v u t s r q p o n m l k j i h g f e d c b a
 9 8 7 6 5 4 3 2 1 0 ~ } | { ` _ ^ ] \ [ ? > = < ; :
 / .  - , + *) ( ' & % $ # " !

 The output file should be as follows:
 - but 1 character per record

! " # $ % & ' () * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? a A b B c C d D e E f F g G h H i I j J k K l L m M n N o O p P q Q r R s S t T u U v V w W x X y Y z Z [ \ ] ^ _ ` { | } ~

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

C5. Sort Key Options - Unique Keys (key#u1/2) - Drop Duplicate Keys

option 'u' (key#u1/2/4) - drop duplicate records

Option 'u' in the 5th byte of 'key#u' specifies 'unique keys' (data reduction) which outputs only 1 record (or none) for the keys at the specified level '#'.

run option 's' - insert dup count at specified column#

ex#1 - option 'u1' keep only 1st of dup sets

ex#2 - option 'u4' drop all dup sets

 #1. ls cobsdir >tmp/cobslist
 #2. ls cobxdir >tmp/cobxlist

 #3. uvsort "fili1=tmp/cobslist,fili2=tmp/cobxlist,rcs=80,typ=LSTt\
             filo1=tmp/misslist,key1u4=0(20)"
                               ============

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

C6. Sort Key Options (key#s1/2) - SELECT records for DUPLICATE Keys

option 's' (key#s1/2) - select duplicate records

Option 's' in the 5th byte of 'key#s' specifies selection of records for duplicate key sets at the specified key level#. IE - drop all records that had no duplicate keys at the specified key level#.

run option 's' - insert dup count at specified column#

ex#3 - option 's2' - select last rec of dup set

 uvsort "fili1=dat1/sales1,rcs=64,typ=RSF,filo1=sales1dups,\
 ==========================================================
 rcs=80,typ=LSTt,key1s2=0(6),key2=13(6),rop=s70"
 ===============================================
                     ^^                 ^^^^^^^

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

C7. uvsort: key1f1 option to write multiple files

key1f1 option to write multiple files

This is a unique & powerful feature that can be used to write out separate files for each customer#, salesman#, state/province, etc. Whenever a control break occurs in key1, a new file is opened whose filename is taken from the declared filo1=... with the new key1 data appended.

For example, we can sort the dat1/sales3 demo file by salesman# (bytes 10-11) & write separate files to the tmp/ subdir for each salesman# encountered.

 uvsort "fili1=dat1/sales3,typ=RSF,rcs=64,filo1=tmp/sm*,key1f1=10(2)"
 ====================================================================

1st 6 of 20 records in dat1/sales3 demo file

130140 21 940802 IN111001 HAM001 00020+ 0000001 00000020+ 130140 21 940802 IN111001 SCR012 00021+ 0001001 00021021+ 130140 21 940802 IN111001 CHR001 00022+ 0002001 00044022+ 139923 35 950802 IN111002 TAB013 00023+ 0003001 00069023+ 139923 35 950807 IN111002 TAB013 00024+ 0004001 00096024+ 150825 44 960804 IN1122 HAM001 00025+ 0005001 00125025+

Option f1 on key1f1 will write a separate output file for each salesman#, in the directory specified by filo1 & with any specified prefix:

Rules for option f1 (key1f1)

1. Option f1 can only be coded on key1 (key1f1=...)
   
   
2. Only 1 output file may be declared, for example: filo1=tmp/sm*
   
   
3. The output filename must include a directory (at least 1 '/') & optionally may include a filename prefix (on which key1 data will be appended).
   
   
4. You may append an '*' on the directory/filenameprefix*, but this is just a visual reminder. Only the option 'f1' on key1f1 is required.
   
   
5. There is no limit on the number of output files that could be created but you would use this option where you knew the outputs are manageable.
   
   
6. You might use this option when you need to send out separate files to each salesman, or each customer, etc.
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

C8. Year 2000 field type (y) - Illustrated Demonstration

    key1=30(6y)     - field type 'y' specifies Year2000 windowed sort
    =========^=
                    - 1st 2 digits assumed to be the year (no century)
                    - the default window is '50', meaning:
                      years 50-99 are considered 1950-1999,
                      years 00-49 are considered 2000-2049.

Year2000 sort demo

Note: You can run the following Y2K sort demo using the supplied test file.

 uvsort "fili1=tf/testY2K,rcs=80,typ=LSTt,filo1=tmp1,key1=30(6y)
 ===============================================================

INPUT file (uvadm/tf/testY2K)

          1         2         3         4         5
 12345678901234567890123456789012345678901234567890
                             purchase        expiry  <-- sort field 31-36
 cust#   product description   yymmdd policy yymmdd

00000 test Year2000 date sort 000101 0-year 000101 11111 test Year2000 date sort 100101 1-year 110101 22222 test Year2000 date sort 200101 2-year 220101 33333 test Year2000 date sort 301231 3-year 331231 44444 test Year2000 date sort 400101 4-year 440101 55555 test Year2000 date sort 500101 5-year 550101 66666 test Year2000 date sort 601231 6-year 661231 77777 test Year2000 date sort 700101 7-year 770101 88888 test Year2000 date sort 800101 8-year 880101 99999 test Year2000 date sort 901231 9-year 991231

Y2Ksort OUTPUT results

 55555 test Year2000 date sort 500101 5-year 550101  <-- Y2K windowed sort
 66666 test Year2000 date sort 601231 6-year 661231      (default window 50)
 77777 test Year2000 date sort 700101 7-year 770101
 88888 test Year2000 date sort 800101 8-year 880101
 99999 test Year2000 date sort 901231 9-year 991231
 00000 test Year2000 date sort 000101 0-year 000101
 11111 test Year2000 date sort 100101 1-year 110101
 22222 test Year2000 date sort 200101 2-year 220101
 33333 test Year2000 date sort 301231 3-year 331231
 44444 test Year2000 date sort 400101 4-year 440101

---> Rerun with run options to modify window value & with sliding option

 uvsort "fili1=tf/testY2K,rcs=80,typ=LSTt,filo1=tmp1,key1=30(6y),rop=y70z1"
 ==========================================================================
                                                              ^  ^^^^^^^^^
 ---> Try 'key1=30(6yd)' for descending sort on type y fields.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

C9. Year 2000 sort demo - Options to change window & specify sliding

The default window is '50', you can change with run options 'y__' & 'z1'. NOTE - run options are specified on the 'rop' command.

Y2K demo sort with run options y35z1

<--- Please see the input file listed on the previous page

---> Rerun with run options to modify window value & with sliding option

 uvsort "fili1=tf/testY2K,rcs=80,typ=LSTt,filo1=tmp1,key1=44(6y),rop=y35z1"
 ==========================================================================
                                                              ^  ^^^^^^^^^
 cat tmp1       - display sort output file
 ========
                  ** output results **

33333 test Year2000 date sort 301231 3-year 331231 44444 test Year2000 date sort 400101 4-year 440101 55555 test Year2000 date sort 500101 5-year 550101 66666 test Year2000 date sort 601231 6-year 661231 77777 test Year2000 date sort 700101 7-year 770101 88888 test Year2000 date sort 800101 8-year 880101 99999 test Year2000 date sort 901231 9-year 991231 00000 test Year2000 date sort 000101 0-year 000101 11111 test Year2000 date sort 100101 1-year 110101 22222 test Year2000 date sort 200101 2-year 220101

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

D1. uvsort: FILE DEFINITION PARAMETERS

      fili1=, typ=, rcs=, [isk#=,]  filo1=, [typ=,] [rcs=,] [isk#=]

The 'XSUM' functionality of SYNCSORT is now provided by declaring output file #61. File #61 will receive records deleted by 'sum=none' or sum=dsp(lth) acum field/ Intended for mainframe conversions. You might define $SORTIN,$SORTOUT,$SORTXSUM in the JCL/script & uvsort would be:

 uvsort "fili1=$SORTIN,rcs=...,typ=...,filo1=$SORTOUT,filo61=$SORTXSUM"
 =====================================================****************=

        isk1=0(8)         - declare 1st key (bytes 1-8)
        isk2=59(15)       - declare 2nd key (bytes 61-75)

        isk1p1=0(8)       - declare key#1 part#1
        isk1p2=20(6)      - declare key#1 part#2
        isk1p3=30(4)      - declare key#1 part#3

fili1/filo1 example

 uvsort "fili1=xx,typ=RSF,rcs=80,filo1=yy,typ=ISF,isk1=0(8),key1=0(8)"
 =====================================================================

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

D2. Indexed file definition - additional options

alternate format for ISAM keys

uvsort "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isk1=0(8),isk2=59(5),key1=0(8)" uvsort "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isks=(0,8,59,5),key1=0(8)"

ISAM option for DUPS/NODUPS

uvsort "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isks=(0,8,40,20),key1=0(8)" uvsort "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isks=(0,8n,40,20d),key1=0(8)"

• the above illustrates the defaults - nodups on the 1st key,
  and dups allowed on secondary keys.
• You only need to code a 'd' on the primary key for dups,
  or an 'n' on a secondary key for nodups.

uvsort "fili1=x,typ=RSF,rcs=80,filo1=y,typ=ISF,isks=(0,8,40,20),key1=0(8)"

• the above specifies nodups allowed on key #2
  (key#1 defaults to nodups & key#2 to dups allowed)

 uvsort "fili1=x,typ=RSF,rcs=80,\
       filo1=y,typ=ISF,isk1=0(8),isk2=40(20n),key1=0(8)"

• could also specify the 'd' or the 'n' on the original isk format

do not need to specify ISAM keys for input files

uvsort "fili1=x,rcs=80,typ=ISF,isks=(0,8,40,20),filo1=y,typ=RST,key1=0(8)" uvsort "fili1=x,rcs=80,typ=ISF,filo1=y,typ=RST,key1=0(8)"

• the above are the same, you never need to specify ISAM keys
  for input files (only for output file creation).

option 'k' for key of reference (read sequence)

uvsort "fili1=x,rcs=80,typ=ISFk2,filo1=y,typ=RST,key1=0(8)"

• the above option 'k2' of file 'typ=ISFk2' specifies the key
  of reference as key #2, to cause record retrieval in sequence
  by the 2nd key.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

E1. uvsort file TYP codes

File type codes are 3 upper case alpha + misc options (a-z), for example:

       uvsort "fili1=test1,typ=RSF,rcs=80,filo1=test2,typ=LST,key1=0(8)"
       ====================*******====================*******===========

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

E2. uvsort file TYP codes

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

E3. TYP codes for Indexed files

typ=ISF Indexed Sequential Fixed files

 uvsort "fili1=dat1/sales2,typ=LST,rcs=64,filo1=xx,typ=ISF,isk1=0(6),key1=0(6)"
 ==============================***=====================***=====================

The example above will convert a text input file to an Indexed Sequential Fixed record size file with key in 1st 6 bytes. This is an easy way to create indexed files - just use the editor to create the keys & data fields, with no need to worry about record sizes or key sequences, then use uvsort to create the Indexed file of fixed data records with .idx partition.

export ISDATEXT=.dat

This applies only to D-ISAM typ=ISF Indexed files. You may export ISDATEXT=".dat" if you wish to use '.dat' extensions on D-ISAM files read or written by uvcp (+ uvsort + uvcopy). The default is no extension on data partition, which is compatible with Micro Focus COBOL. Add to /etc/profile if your site always needs '.dat'. Add the export to /etc/profile if your site always needs '.dat'.

typ=RSV MF COBOL file type IDXFORMAT3

 uvsort "fili1=indir/paymast,typ=RSV,rcs=4000,\
 ==============================================
         filo1=outdir/paymast,typ=ISF,rcs=200,isk1=0(6),key1=0(6)"
         =========================================================

The above would convert a Record Sequential Variable (Micro Focus IDXFORMAT3) data file to an Indexed Sequential Fixed record length file. The 'ISF' file is compatible with MF COBOL fixed length Indexed files & C-ISAM files.

Please see pages 'N1' thru 'N3' for more details & examples --->

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

E4. file TYP codes for fili1/filo1 (continued)

typ=IDX Indexed Sequential Variable files

'typ=IDX' declares Indexed Sequential Variable length record files compatible with Micro Focus COBOL IDXFORMAT1,3,8 files. See the 'File Handling' documentation for Micro Focus Object or Server Express COBOL.

The Micro Focus COBOL 'EXTFH' file handler is compiled into an alternate version of uvsort called 'uxsort' using the 'cobccuv' script as explained in install.doc.

'uxsort' supports the same file types as uvsort with typ=IDX in addition. Two versions of the program are required since only Micro Focus COBOL users (server express 2.2+) could compile 'uxsort' whereas any user can compile 'uvsort' which uses D-ISAM to support Indexed Sequential FIXED record length files.

Note that 'typ=RSV' (Record Sequential Variable with no indexed keys) files are supported by both uvsort & uxsort. 'uxsort' must be used when either input or output or both files are typ=IDX.

example - sort text file & load Indexed Variable

 uxsort "fili1=dat1/idxfL,typ=LST,rcs=256\
 =========================================
        ,filo1=tmp/idxf3v,typ=IDXf3v60y4,rcs=252,isk1=0(6d),key1=0(6)"
        ==============================================================

This would read text file dat1/idxfL & build an Indexed Sequential Variable file tmp/idxf3v whose key is the 1st 6 bytes of each record. The output records could vary from 60 to 252 bytes depending on the last nonblank character of input record. 'rcs=252' sets the maximum size & option 'v60' sets the minimum. The records will actually vary from 64 to 256 allowing for the record header (see pages 'N1' to 'N3' which explain the record format).

You can run this example using a test file provided (uvadm/dat1/idxfL).

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

E5. uvsort: file TYP OPTIONS

File options may be appended to the 3 byte file typ code - lower case letters a-z with possible numeric suffixes - for example:

   uvsort "fili1=text1,typ=LSTb1c2,rcs=80,filo1=text2,typ=LSTt3,key1=0(20)"
   ===========================****===========================**============

options for file typ=LST... (text files)

Notes re options for typ=LST...

1. You normally need no options for typ=LST, since the defaults are what you would expect for text files - stop read on LineFeed x'0A' & append LF x'0A' on output writes (after the last non-blank).
   
   
2. File typ options (like most Vancouver Utility options) are "binary" and may be combined. For example, typ=LSTt3 would terminate output records with both a CR & an LF x'0D' + x'0A' (MSDOS file).
   
   
3. Output file typ options may be omitted & defaulted to input options, for example:
   
   

   uvsort "fili1=text1,typ=LST,rcs=80,filo1=text2,key1=0(20)"
   ==========================================================

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

E6. file TYP OPTIONS (continued)

          ** file typ options applicable to multiple file types **
 a   - append to the file's existing contents (vs default erasing existing contents)

Option 'j' useful when you are processing all files in directory (with uvcopyx) And you want the output files in a different case than input files.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

E7. file TYP OPTIONS (continued)

file typ options unique to typ=RDW files

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

E8. file TYP OPTIONS (continued)

file typ options unique to typ=RSV & typ=RSR

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

E9. file TYP OPTIONS (continued)

file typ options for FIXED-length Indexed files

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

E10. file TYP OPTIONS (continued)

file typ options for VARIABLE-length Indexed files

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

E11. file TYP OPTIONS (continued)

output record size defaults for variable length files

The output record size for IDXf3/IDXf8 variable length records defaults to the corresponding input record size. The record size is passed thru the sort by appending it on the end of the record & using it on the output phase.

This works well if both input & output are variable length (typ=IDXf3v/IDXf8v). If input was fixed (typ=RSF/ISF/IDXf1) & output was variable (typ=IDXf3/IDXf8v), then output records would be in the variable length format, but all same size.

See pages 'N1' thru 'O3' for examples of RSV/IDX & options f,v,x,y --->

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

F1. uvsort: 'rop' - RUN OPTIONS

'rop' is used to declare various options for the entire run. Do not confuse with file options which apply only to a file (fili1/filo1) & which are appended to the file typ code (example: typ=LSTt). Do not confuse with instruction options which are appended to instruction opcodes (see the uvcp documentation).

 uvsort "fili1=...,typ=...,rcs=...,filo1=xxx,key1=...,rop=m1024v0"
 =================================================================
 - example, option 'm' 1024 meg MAXMEMORY vs default 512 meg
   and option 'v0' to inhibit sort progress messages

 export UVSORTROP=m1024v0
 ========================
 - alternate way to specify run options via Environmental Variable 'UVSORTROP'
 - could define in your profile to specify options desired for all uvsort's

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

F2. uvsort: rop options - continued

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

F3. uvsort: rop options - continued

 Note - Year2000 options 'y' & 'z' removed in 2019 (no longer relevant)
 y70   - set Year2000 window for field type 'y' on sort keys
         & also for any sel1y/del1y selects/deletes
       - option 'y__' assumes 1st 2 bytes of field is the year (no century)
       - 'y70'  means that years 70-99 = 1970-1999, 00-69 = 2000-2069
 z     - option 'z1' specifies sliding windows, for example in 1998
 y70z1 - window value '68' meaning 68-99 = 1968-1999, 00-67 = 2000-2067

uvsort options for performance

Most sort times are determined by input & output times. Intermediate sort merges are required only for files larger than 6.4 gigabytes, assuming the default sort memory of 512 megs & 100 sort work files. As the input file is read, each 512 meg portion is sorted & written to up to 100 work files in the tmp subdir. These are then merged during the output phase.

See options 'm' & 'w' (above) for sort memory & number of work files (default 512 megs & 100 work files max). There is no advantage increasing the memory unless the file is > 6.4 gig.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

F4. uvsort: rop options - continued

rop options - complete examples

 #1. uvsort "fili1=bigfile,typ=RSF,rcs=351,filo1=bigsorted,key1=71(20),rop=m640"
     ==================================================================********=

rop=m640 - assign 640 megs for internal sort memory (vs 512 megs default)

 #3. uvsort "fili1=xx,typ=RSF,rcs=256,filo1=yy,key1=0(8),rop=b8000,i4000"
     ====================================================***************

     rop=b8000 - bypass the 1st 8000 records in the input file
         i4000 - stop input after 4000 records (8001-12000)

 #3. uvsort "fili1=dat1/sales,typ=RSF,rcs=100,filo1=tmp/sales,key1f1=0(6),rop=x20000000"
     ===================================================================================
      This illustrates Run OPtion 'x' (rop=x____) to split output into multiple
      files of a Max number of records (except for last output file).

      For example, given a large file (5 gigabytes) of 100 byte records,
      You might need to split into multiple files of 2 gig each, for transfer
      to an older system with the 2 gig file limit.

       Use 'rop=x20000000' 20,000,000 records * 100 = 2,000,000,000 bytes

       output: sales_001 (2 gig), sales_002 (2 gig), sales_003 (1 gig)

 #4. uvsort "fili1=warmas1a,typ=RSF,rcs=64,filo1=wm2,key1=44(6y),rop=y70z1"
     ============================================================*********=

     rop=y70z1 - set Year2000 sliding window '70' for sort field type 'y'
                 & also for any sel1y/del1y selects/deletes

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

F5. uvsort: rop options - continued

rop=f2 input all files from directory

Run option f2 allow you to input all files from a directory, or all files matching a specified prefix &/or suffix.

The fili1 pathname must include at least 1 '/' & exactly 1 '*' (a directory & optionally a prefix &/or suffix)

This is the reverse of option 'f1' on key1f1 (see page 'C7'), which writes multiple separate files to a directory based on key1 control breaks.

For example, we could recombine the multiple salesman files that were created on the key1f1 demo on page 'C7'.

example rop=f2 input all files in subdir

 uvsort "fili1=tmp/sm*,typ=RSF,rcs=64,filo1=tmp/allsm,key1=10(2),rop=f2"
 =======================================================================
 - input all files with matching pprefix 'sm'

UVSORTROP - env-var to specify uvsort Run OPtions

Environmental Variable 'UVSORTROP' can be defined in your profile to specify options for all uvsort executions (vs appending ,rop=... on the uvsort command). There are only a few options that you might want to specify this way, for example:

 export UVSORTROP=m640n1v0
 #========================
 # m640 - allocate 640 megabytes for uvsort memory space
 # n1 - inhibit waiting for any reply to uvsort prompts
 # v0 - inhibit sort progress messages (large sorts > 12 MB)

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

F6. uvsort: rop options - continued

environmental variable for sortwork directory

 export SORTDIR=/u2/sortwork   #<-- change sortworkdir from default ./tmp/
 ===========================

'SORTDIR' is an optional environmental variable that may be used to assign the sortwork directory. You could code this in your .profile or .bash_profile or better in the common_profile for all users. /u2 would be an alterate file system with lots of space. Be sure permissions are 777 or 775 if in same group as users executing uvsort.

The default sortwork directory is 'tmp' within the current directory (wherever uvsort is executed - not the system '/tmp' subdir). The sortwork directory will be created if it does not already exist.

Note that sort work (merge) files are not created if the data file size is less than the memory assigned to uvsort via run option 'm' (see page 'F2'). The default is 'rop=m512' which assigns 512 megabytes to internal sort work You could increase up to 2 gigabytes, but you wont see much improvement unless your file is larger than about 6 gigabytes, which is the point when uvsort would need to do multi-pass merges. m640 seems to be max effective.

uvsort creates sort work files using the process ID as part of the filename. The process ID is converted to hexadecimal, for example:

      tmp/SORT008466.001   <-- porcess ID 8466
         ---- etc ---      <-- 10 merge files reqquired to sort 4 gig file
      tmp/SORT008466.010

If uvsort fails for whatever reason (hardware, software, or no space left in filesystem where tmp subdir located), then the SORT work files will be left in the tmp subdir & you might want to remove them before rerunning the sort. uvsort automatically removes SORT work files only if more than 2 days old (in case multiple uvsorts are running at the same time).

 find tmp/SORT* -mtime +2 -exec rm -f {} \;
 ==========================================

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

F7. uvsort: rop options - continued

clear SORTDIR by nightly cron

As noted above, uvsort failures can leave sort work/merge files in the SORTDIR (default ./tmp/... of current dir). There is a chance that new sortwork files might match old ones if the process-id wraps, since the process-id is part of sortwork filenames (as illustrated above).

So, I recommend you clear the SORTDIR by cron at some point when you know no sorts will be running. This will be easier if you assign SORTDIR to a fixed location (vs ./tmp/... in current dir of current user). For example:

 export SORTDIR=/var/sortwork   #<-- in common_profile
 ============================   # sortwork perms 775 & group matches users

 00 3 * * * rm -f /var/sortwork/*   #<-- remove sortwork files at 3 AM nightly
 ================================

Above in crontab file for root, see sample crontab file at https://uvsoftware.ca/admjobs.htm#5F2

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

F8. uvsort: rop options - continued

HEADER & TRAILER options rop=h & rop=t

In May 2018, the HEADER option was enhanced to allow any number (vs only 1) and the TRAILER option was added to allow any number of records. Here is a script ($UV/sf/adm/testsortHT1) to test the HEADER & TRAILER options.

 # testsortHT1 - test uvsort HEADER & TRAILER
 #             - by Owen Townsend, UV Software, May23/2018
 uvsort "fili1=dat1/test20,rcs=80,typ=LSTt,filo1=tmp1/test20HT1,key1=5(25),rop=h3t3"
 #==================================================================================
 cat tmp1/test20HT1

test file dat1/test20 - BEFORE testsortHT1

0001 tf/test100 one --- 0002 to 0019 omitted to save space --- 0020 tf/test100 twenty

 testsortHT1       <-- execute script $UV/sf/testsortHT1
 ===========

test file tmp1/test20HT1 - AFTER testsortHT1

 0001 tf/test100 one
 0002 tf/test100 two        <-- rop=h3 preserves 1st 3 records
 0003 tf/test100 three
                              - space in this doc (not in test output)
 0008 tf/test100 eight
 0011 tf/test100 eleven
 0015 tf/test100 fifteen
 0005 tf/test100 five
 0004 tf/test100 four
 0014 tf/test100 fourteen   <-- other records sorted by key1=5(25)
 0009 tf/test100 nine         - records 4 to 17 sorted to alpha sequence
 0007 tf/test100 seven
 0017 tf/test100 seventeen
 0006 tf/test100 six
 0016 tf/test100 sixteen
 0010 tf/test100 ten
 0013 tf/test100 thirteen
 0012 tf/test100 twelve
                              - space in this doc (not in test output)
 0018 tf/test100 eightteen
 0019 tf/test100 nineteen   <-- rop=t3 preserves last 3 records
 0020 tf/test100 twenty

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

G1. uvsort: sel/del - SELECT/DELETE RECORDS

      sel1=start(length):dataconstant

      sel1=10(2):21          - select records with '21' in cols 11-12 (slsmn)
      sel2=30(3):HAM           AND 'HAM' in cols 31-33 (product code)
      sel3=38(6)>000025        AND >'000025' in cols 49-54 (quantity)
                               ('AND' condition if 4th byte increases)

      sel1=10(2):21,sel2=30(3):HAM,sel3=38(6)>000025  <-- code contigously
      ==============================================      with ',' separators

      sel1=10(2):21          - select records with '21' in cols 11-12 (slsmn)
      sel1=10(2):44            OR with '44' in cols 11-12
      sel1=10(2):85            OR with '85' in cols 11-12
                               (OR condition if 4th byte reverts to '1')

      del1=90(2)>55          - delete if cols 91-92 > '55'
      del2=99(1)!0xff          AND if column 100 not = x'ff'

'sel' & 'del' are mutually exclusive within Inputs & within Outputs. You can have all dels for Input files & all sels for Output files, but do not mix sels/dels within Inputs or Outputs. You can have a maximum of 500 sels/dels.

The AND/OR condition is indicated by the 4th byte, sel1, sel2, etc to sel8 max. The 'AND' condition is indicated as long as 4th byte increases (sel1,sel2,etc). The 'OR' condition is indicated whenever 4th byte reverts to '1'.

select to multiple output files

You can select to multiple output files using option 'o#' where '#' is the number of the output file declared (filo1,filo2,... max 60). You can use the 'SAVE' option to select all non-selected records to the last file#, for example:

      sel1o1=0(1):A         select records with 'A' in column 1 to filo1
      sel1o2=0(1):B         select records with 'B' in column 1 to filo2
      sel1o3=0(1):C         select records with 'C' in column 1 to filo3
      sel1o4=0(4):SAVE      select all non-selected records to filo4

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

G2. uvsort: sel/del - SELECT/DELETE records

sel/del example

          1         2         3         4         5         6
 1234567890123456789012345678901234567890123456789012345678901234
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 940802 IN111001  HAM001  000020 0000001 000000020
 139923    35 950802 IN111002  TAB013  000023 0003001 000069023
 400002    85 010812 CR245     HAX129  00001v 0006001 00009601v

 #1. Salesman 21,44,& 65 (columns 10-11)
 #2. Product hammers ('HAM' in cols 31-33)
 #3. sales QTY 10 or more (columns 39-44)

Solution as follows:

uvsort "fili1=dat1/sales2,rcs=64,typ=RSF,filo1=tmp/s2,key1=10(2),key2=30(6)\ ,sel1=10(2):21,sel2=30(3):HAM,sel3=38(6z)>9\ ,sel1=10(2):44,sel2=30(3):HAM,sel3=38(6z)>9\ ,sel1=10(2):65,sel2=30(3):HAM,sel3=38(6z)>9"

SORT INCLUDE/OMIT conversions to uvsort sel/del

The JCL converter converts mainframe SORT steps to the uvsort equivalent. Any 'INCLUDE/OMIT's are converted to 'sel/del's. Complex SORTs with multiple mixed INCLUDEs & OMITs may require manual changes and a warning is given. Please see several examples at the end of this uvsort documentation.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

G3. uvsort: sel/del (continued) - option 'o'

option 'o' & 'o#' on sel/del

Option 'o' on sel/del functions cause the record selection to be performed at output time (vs input time). Without any number attached these selects/ deletes will be applied to all output files.

Option 'o#' may specify the output file# to which the selects/deletes are to be applied. For example, to select different salesmen to separate files:

 uvsort "fili1=dat1/sales3,rcs=64,typ=RSF,key1=10(2)\
 ===================================================
 ,filo1=tmp/sm21,filo2=tmp/sm35,filo3=tmp/sm44\
 =============================================
 ,sel1o1=10(2):21,sel1o2=10(2):35,sel1o3=10(2):44"
 =================================================

This is somewhat similar to option 'f1' on key1f1 already demonstrated on page 'C7', but this can be much more selective. For example, we can add additional conditions to select by product & quantity as well as salesman#:

 ,sel1o1=10(2):21,sel2o1=30(3):HAM,sel3o1=38(6)>000009\
 =====================================================
 ,sel1o2=10(2):35,sel2o2=30(3):HAM,sel3o2=38(6)>000009\
 =====================================================
 ,sel1o3=10(2):44,sel2=30(3):HAM,sel3=38(6)>000009\
 =================================================

Rules for option 'o' on sel/del

1. Output files (filo1=...,filo2=...,etc) must have been declared to match the option 'o#'s specified (o1,o2,...,o61).
   
   
2. The option 'o#'s must appear in sequence, but in fact any out of sequence o#s will be included with the current highest o# yet encountered.
   
   
3. This means if you inadvertently omit the o# option as shown above for the sel2 & sel3 on the 3rd line, the conditions are still properly applied to filo3 as declared on the sel1o3.
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

G4. uvsort: sel/del - SELECT/DELETE RECORDS

option 'u1' - allow coding ' 's as '_'s

Option 'u1' allows you to code blanks as underscores, which is convenient because blanks are not allowed in the uvcp command string. The alternative would be to code blanks in hexadecimal (' ' = 0x20). Here are some examples:

option 'u2' - allow coding ','s as '@'s

Option 'u2' allows you to code commas as at-signs, which is convenient because commas are used to separate commands in the command string and therefore can not be coded as constants.

examples - coding ' ' as '_' & ',' ar '@'

 sel1u1=0(1):_,sel1u2=0(1):@,sel...
 ==================================
 - select record if 1st byte blank ' ' OR 1st byte comma ','

 sel1u1=0(1):_,sel2u2=1(1):@,sel...
 ==================================
 - select record if 1st byte blank ' ' AND 2nd byte comma ','

 sel1u3=0(2):_@,sel...
 ==================================
 - select record if 1st byte blank ' ' AND 2nd byte comma ','
 - coding a 2 byte constant (vs separate 1 byte constants)
 - note option u3 = (u1 + u2)

option 'y' for Year2000 windowed date field selections

     sel1y=20(6)>980630   - select records with dates in cols 21-26
                            from 19980701 up to 20491231

Option 'y' assumes 1st 2 bytes of specified field is the 2 digit year regardless of field length (yymmdd, yymm, yy).

The default window is '50', may be changed by run option 'y' to specify the window value,& option 'z1' for sliding windows.

 Note - run options are specified on the 'rop' command (ex: rop=y50z1)
     rop=y50  -------- means 50-99 = 1950-1999,& 00-49 = 2000-2049
     rop=y50z1 in 1998 means 48-99 = 1948-1999,& 00-47 = 2000-2047
     sel1y=20(2)>98,rop=y70z1  - select records with years in cols 21-22
                                 from 1998 to 2067 (rop=y70z1) if now 1998

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

H1. uvsort: SUM Function

 sum1=38(6z),sum2=53(9z)        <-- sum example

The 'sum' function accumulates fields from multiple records into 1 record for each duplicate key set. Consider the following complete uvsort example:

 uvsort="fili1=dat1/sales2,rcs=64,typ=RSF,filo1=tmp/sales2,key1=0(6)\
 ===================================================================
 ,sum1=38(6z),sum2=53(9z)"
 =========================

input file

          1         2         3         4         5         6
 1234567890123456789012345678901234567890123456789012345678901234
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 940802 IN111001  HAM001  000020 0000001 000000020
 130140    21 940802 IN111001  SCR012  000021 0001001 000021021
 130140    21 940802 IN111001  CHR001  000022 0002001 000044022
 139923    35 950802 IN111002  TAB013  000023 0003001 000069023
 139923    35 950807 IN111002  TAB013  000024 0004001 000096024
 150825    44 960804 IN1122    HAM001  000025 0005001 000125025
 150825    44 960804 IN1122    HAX129  000026 0006001 000156026

                     ** output file **
                                       (sum1)           (sum2)
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 940802 IN111001  HAM001  000063 0000001 000065063
 139923    35 950802 IN111002  TAB013  000047 0003001 000165047
 150825    44 960804 IN1122    HAM001  000051 0005001 000281051

Note that qty & amount have been accumulated into 1 record for each customer#. By default the SUM totals go into the FIRST record of the dup key record set. If you wish to use the LAST record of the dup key set, specify run option 't2'. For example:

 uvsort="fili1=dat1/sales2,rcs=64,typ=RSF,filo1=tmp/sales2,key1=0(6)\
 ===================================================================
 ,sum1=38(6z),sum2=53(9z),rop=t2"
 =========================******

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

I1. uvsort Instructions

This uvsort.doc documents only functions (fili1,filo1,keys,sel/del,etc), but most instructions (mvc,mvn,edt,add,rep,etc) are documented in uvcp.doc.

      logical:  mvc, mvn, edt, clr
   arithmetic:  add, sub, mpy, div, pac, unp, seq
 compare/skip:  cmc, cmn, skp
 scan/replace:  rep, scn, sqz
   translates:  tra, tre, trl, tru, trp
        other:  anc, orc, chx, hxc, tim

Some instructions support 'options' (appended to the instruction code), and these were also documented in uvcp.doc. Instruction option 'o' is unique to uvsort & may be coded on most instructions. Option 'o' identifies instructions to be executed at sort 'output time' vs sort 'input time' (the default).

I2. uvsort Instructions

option 'o' example

 mvno=70(9)53,divo=70(9)38(6)
 ============================

The complete uvsort coding might be:

 uvsort="fili1=dat1/sales2,rcs=64,typ=LSTt,filo1=tmp/sales2,rcs=80,key1=0(6)\
 ===========================================================================
 ,sum1=38(6z),sum2=53(9z),mvno=64(9)53,divo=64(9)38(6)"
 =========================****************************

The 'sum' functions accumulate the qty & amount into 1 record for each customer. The 'mvno' moves the accumulated amount to the end of the record and the 'divo' calculates the average sale (total sales/total qty). The 'o' option is essential to force these instructions to be executed at sort output time (vs the default sort input time).

example input file

          1         2         3         4         5         6         7         8
 12345678901234567890123456789012345678901234567890123456789012345678901234567890
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================================
 130140    21 940802 IN111001  HAM001  000020 0000001 000000020
 130140    21 940802 IN111001  SCR012  000021 0001001 000021021
 130140    21 940802 IN111001  CHR001  000022 0002001 000044022
 139923    35 950802 IN111002  TAB013  000023 0003001 000069023
 139923    35 950807 IN111002  TAB013  000024 0004001 000096024

                     ** example output file **
                                       (sum1)           (sum2)
 cust#   slmn YYMMDD invoice  product    qty   price    amount   avg amount
 ===============================================================================
 130140    21 940802 IN111001  CHR001  000063 0002001 000065063  000001032
 139923    35 950807 IN111002  TAB013  000047 0004001 000165047  000003511

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

I3. uvsort Instructions

option 'o#' on instructions for filo1,filo2,etc

Instructions coded with option 'o', but without a number (as on prior page) are applied to all output files.

Instructions coded with o1,o2,o3,...o61 are applied to the corresponding output file (filo1,filo2,filo3,...filo61).

Option 'o' on instructions could be combined with option 'o' on selects/deletes. For example, we will append the salesman's name on the output records as we select the different salesmen to separate files.

 uvsort "fili1=dat1/sales3,rcs=64,typ=RSF,key1=10(2)\
 ===================================================
 ,filo1=tmp/sm21,filo2=tmp/sm35,filo3=tmp/sm44\
 =============================================
 ,sel1o1=10(2):21,sel1o2=10(2):35,sel1o3=10(2):44\
 ================================================
 ,mvco1=64(4):John,mvco2=64(5):Henry,mvco3=64(6):George"
 =======================================================

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

I4. uvsort Instructions

seq - sequence number

Most instrucitons are documented in uvcp.htm, but 'seq' is documented here in uvsort.doc, because it is most applicable to uvsort.

      seq=displacement(length)    <-- instruction format

      seq=75(5)     - sequence# in 75-79 (cols 76-80)
                    - 5 bytes zoned numeric (unpacked)
      seq=75(5p)    - 5 bytes packed
      seq=76(5b)    - 4 bytes binary

      seqo=75(5)    - sequence# at Output time (vs input time)
                    - more applicable to sorts
      seqo1=75(5)   - seq# Output file #1
      seqo2=75(5)   - seq# Output file #2
      seqo3=75(5)   - seq# Output file #3

      seqo1c1=75(5)   - seq# Outfile#1 using counter#1 (default)
      seqo2c2=75(5)   - seq# Outfile#2 using counter#2 (default)

      seqo1c1=75(5)   - seq# Outfile#1 using counter#1 (default)
      seqo1c2=80(5)   - 2nd seq# in Outfile#1 using counter#2
      seqo1c2=85(5)   - can have multiple seq#s in 1 file

      seqo1i10=75(5)  - Increment by 10 (vs 1 default)

      seqo1b1000i10=75(5) - Begin at 1000 & Increment by 10

 uvsort "fili1=tf/test100,rcs=80,typ=LSTt,filo1=tmp/t1,key1=0(4cd),seqo=35(5)"
 =============================================================================
 - sequence# in 35-39 (cols 36-40)

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

I5. uvsort Instructions

seq - demo using Test File tf/test100

Here is a test/demo of the 'seq' (sequence number instruction), using test file tf/test100. You can run this test/demo as follows:

 #1. Login uvadm --> /home/uvadm

 #2. more tf/test100     <-- display input file
     ===============

      0001 tf/test100 one
      0002 tf/test100 two
      0003 tf/test100 three
        --- 94 records omitted ---
      0098 tf/test100 ninety eight
      0099 tf/test100 ninety nine
      0100 tf/test100 one hundred

 #3. uvsort "fili1=tf/test100,rcs=80,typ=LSTt,filo1=tmp/t1,key1=0(4cd),seqo=35(5)"
     =============================================================================
     - sort on 1st 4 bytes Char Descending
     - sequence# in 35-39 (columns 36-40)

 #4. more tmp/t1        <-- display output file
     ===========

      0100 tf/test100 one hundred        00001
      0099 tf/test100 ninety nine        00002
      0098 tf/test100 ninety eight       00003
             --- 94 records omitted ---
      0003 tf/test100 three              00098
      0002 tf/test100 two                00099
      0001 tf/test100 one                00100

test other seq# instructions

 #5. seq=35(4),seqo=40(4)   <-- seq# 35-39 at Input time
     ====================     - and  40-44 at Output time
                              - 1st 3 records shown below

      0100 tf/test100 one hundred        0100 0001
      0099 tf/test100 ninety nine        0099 0002
      0098 tf/test100 ninety eight       0098 0003

 #6. seqob1000i10=40(6)     <-- seq# 40-45 at Output time
     ==================       - Begin from 1000, Incrementing by 10
                              - 1st 3 records shown below

      0100 tf/test100 one hundred             001010
      0099 tf/test100 ninety nine             001020
      0098 tf/test100 ninety eight            001030

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

J1. uvsort: I/O file declaration - 3 methods

1. declaring filenames on the COMMAND LINE
   
   

    uvsort "fili1=dat1/sales2,rcs=64,typ=RSF,filo1=tmp/sales2,key1=10(8)"
    ========***************==================***************============

2. declaring input filenames via the QUERY option
   
   

    uvsort "fili1=?,rcs=64,typ=RSF,filo1=tmp/allsales,key1=10(8)"
    ========*******=============================================

• the '?' will cause the filename to be solicited from the operator
• would be used in a script when the filename varies from run to run

3. declaring Filenames in a PARAMETER FILE
   
   

    uvsort pf/demo/uvsortsplit1  <-- execute instructions in parameter file
    ===========================

 # uvsortsplit1 - split sales to separate files for specified salesman
 #              - output separate file for salesmen 21,35,44,65,85
 #              - select sales only if extended amount > $ 500.00
 #              - see doc at https://www.uvsoftware.ca/uvsort.htm#J1
 #
 # Compare to 'uvsortsplit2' using 'f1' option on 'key1f1=...'
 # - allows unlimited number of output files with 1 line of code
 # But this 'uvsortsplit1' more flexible if you do not want all salesmen
 # - or if sort key not salesman#
 #
 # export PFD=/home/uvadm/pf/demo  <-- Vancouver Utility Parameter File Demos
 # uvsort $PFD/uvsortsplit2        <-- execute uvsort on this parameter file
 #
 fili1=dat1/sales2,rcs=64,typ=RST
 key1=10(8),key2=53(9z)
 filo1=tmp1/sales2_21,rcs=64,typ=RST,sel1o1=10(2):21,sel2o1=53(9)>000050000
 filo2=tmp1/sales2_35,sel1o2=10(2):35,sel2o2=53(9)>000050000
 filo3=tmp1/sales2_44,sel1o3=10(2):44,sel2o3=53(9)>000050000
 filo4=tmp1/sales2_65,sel1o4=10(2):65,sel2o4=53(9)>000050000
 filo5=tmp1/sales2_85,sel1o5=10(2):85,sel2o5=53(9)>000050000

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

J2. uvsort: I/O file declaration - 3 methods

notes re Multiple Files in uvsort Parameter File

A parameter file would be used when the command line would be too long to be comfortable with, especially if the command line is to be entered from the UNIX prompt, and to a lesser extent when the command is within a script file. Use a parameter file for complex sorts with many sels/dels &/or output files.

Alternatively, you could use the sort facility within the 'uvcopy' utility (vs lengthy complex uvsorts). The uvcopy instructions are more powerful & easier to code but not as compact as uvsort fucntions such as sel/del.

Alternative for Multiple Output Files

 # uvsortsplit2 - demo uvsort split sales to separate files for each salesman
 #              - method #2 using 'f1' option on 'key1f1=...'
 #              - allows unlimited number of output files with 1 line of code
 #              - see doc at https://www.uvsoftware.ca/uvsort.htm#C6 & J2
 #
 # Compare to 'uvsortsplit1' which specifies salesman# & multiple output files
 # (but more flexible if you do not want all slsmn or if sort key not slsmn#)
 #
 # export PFD=/home/uvadm/pf/demo  <-- Vancouver Utility Parameter File Demos
 # uvsort $PFD/uvsortsplit2        <-- execute uvsort on this parameter file
 #
 fili1=dat1/sales2,typ=RST,rcs=64,filo1=tmp1/sales_*,key1f1=10(2)
 #===============================================================
 #
 # Alternative to executing uvsort on this parameter file
 # - can enter uvsort command at shell prompt as follows:
 #
 # uvsort "fili1=dat1/sales2,typ=RST,rcs=64,filo1=tmp1/sales_*,key1f1=10(2)"
 #==========================================================================

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

K1. uvsort: MULTIPLE INPUT FILES - 3 methods

1. declaring multiple input files on the COMMAND LINE
   
   

    uvsort "fili1=sales1,fili2=sales2,rcs=128,typ=RSF,filo1=sales,key1=10(8)"
    ========*************************========================================

• up to 20 input files may be declared (fili1,fili2,etc to fili8)

2. run option rop=f2 to input all files from directory - or all files matching a specified prefix &/or suffix
   
   

    uvsort "fili1=dat1/sales*,rcs=64,typ=RSF,filo1=allsales,key1=10(8),rop=f2"
    ==============***********==========================================******=

3. declaring multiple input files via the '??' QUERY OPTION
   
   

    uvsort "fili1=??,rcs=64,typ=RSF,filo1=allsales,key1=10(8)"
    ========********==========================================

• the '??' causes uvsort to query the operator for the filenames
• enter null to end the query process
• the '??' method can be used either on an operator entered command
  line or from within a script to query the operator

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

L0. uvsort examples

Demo file '/home/uvadm/dat1/sales1' for examples 1-3

          1         2         3         4         5         6
 1234567890123456789012345678901234567890123456789012345678901234
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 990802 IN111001  HAM001  000020 0000101 000002020
 130140    21 000802 IN111001  SCR012  000021 0001001 000021021
 130140    21 010802 IN111001  CHR001  000022 0010001 000220022
 139923    35 980802 IN111002  TAB013  000023 0000201 000004623
 139923    35 990807 IN111002  VAR200  000024 0002001 000048024
 150825    44 010804 IN1122    HAM001  000025 0020001 000500025
 150825    44 020804 IN1122    HAX129  000026 0000301 000007826
 201120    44 030807 CR5234    BBQ001  000027 0003001 000081027
 223240    65 010816 CR955     HAM001  000028 0030001 000840028
 223240    65 020816 IN441     BBQ001  000029 0000101 000002929
 308685    21 020812 IN6605    SAW051  00001p 0000101 00000101p
 308685    21 030812 IN6605    BIT075  00001q 0001001 00001101q
 308685    21 040812 CR8835    TAB013  00001r 0010001 00012001r
 315512    44 040805 IN2251    HAM001  00001s 0000201 00000261s
 315512    44 050805 IN2251    SAW051  00001t 0002001 00002801t
 315512    44 060805 IN2255    BIT075  00001u 0020001 00030001u
 400002    85 990812 CR245     HAX129  00001v 0000101 00000161v
 406082    35 010815 IN33001   BBQ001  00001w 0001001 00001701w
 406082    35 020815 IN33001   TAB013  00001x 0010001 00018001x
 406082    65 030816 IN441     VAR200  00001y 0000101 00000191y

Notes re demo files

'dat1/sales1' is a typical mainframe file - fixed record length & no LineFeeds. This means you cannot view/print using the unix vi, cat, more, lp, etc.

 vi dat1/sales1   <-- try to view with 'vi'
 ==============

'vi' will report the entire file as 1 line & the records we know to be 64 bytes will be all run together on the 80 byte screen lines.

 uvhd dat1/sales1 r256  <-- use 'uvhd' to display files w/o LineFeeds
 =====================

                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 130140    21 940802 IN111001  HAM001  00002{ 0000001 00000002{
      64 130140    21 940802 IN111001  SCR012  00002A 0001001 00002102A
     128 139923    35 950802 IN111002  CHR001  00002B 0002001 00004402B
     192 139923    35 950802 IN111002  TAB013  00002C 0003001 00006902C

The record size is actually 64, but was specified as 256, which explains why you see the 1st 4 records in alignment (256/4 = 64).

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

L1. uvsort examples

uvsort ex#1 - simple sort

uvsort ex#1 - solution

 uvsort "fili1=dat1/sales1,rcs=64,typ=RSF,filo1=tmp/s1,typ=RST,key1=30(6)"
 =========================================================================

output file

 201120    44 970807 CR5234    BBQ001  00002G 0007001 00018902G
 406082    35 020815 IN33001   BBQ001  00001P 0007001 00011901P
 223240    65 980816 IN441     BIT001  00002I 0009001 00026102I
 139923    35 950802 IN111002  CHR001  00002B 0002001 00004402B
 130140    21 940802 IN111001  HAM001  00002{ 0000001 00000002{
 150825    44 960804 IN1122    HAM001  00002E 0005001 00012502E
 223240    65 980816 CR955     HAM001  00002H 0008001 00022402H
 315512    44 000805 IN2251    HAM001  00001L 0003001 00003901L
 400002    85 010812 CR245     HAX129  00001O 0006001 00009601O
 150825    44 960804 IN1122    PLN012  00002F 0006001 00015602F
 308685    21 990812 IN6605    SAW001  00001} 0000001 00000001}
 315512    44 000805 IN2251    SAW051  00001M 0004001 00005601M
 130140    21 940802 IN111001  SCR012  00002A 0001001 00002102A
 139923    35 950802 IN111002  TAB013  00002C 0003001 00006902C
 308685    21 990812 CR8835    TAB013  00001K 0002001 00002401K
 406089    35 020815 IN33001   TAB013  00001Q 0008001 00014401Q
 139923    35 950807 CR5421    VAR200  00002D 0004001 00009602D
 406082    65 020816 IN441     VAR200  00001R 0009001 00017101R
 308685    21 990812 IN6605    WID115  00001J 0001001 00001101J
 315512    44 000805 IN2255    WID115  00001N 0005001 00007501N

                         ** Notes **
 1. Note that the file is now in sequence by product code bytes 30-35
    (or columns 31-36).

2. Note that we specified 'typ=RST' for the output filo1=tmp/s1, to insert LineFeeds in the last byte, so we can inspect the file with vi,cat,more,etc & have each record on a separate line, (vs run together across width of your screen).
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

L2. uvsort examples

uvsort ex#2 - simple sort

uvsort ex#2 - solution

 uvsort "fili1=dat1/sales1,rcs=64,typ=RSF,filo1=tmp/s1,typ=LSTt\
 ================================================================
 ,key1=10(2),key2=30(6)"
 =======================

The command is shown on 2 lines for this documentation & but you can enter it all on 1 line or you can use the '\' to complete the command on a 2nd line.

output file

 130140    21 940802 IN111001  HAM001  00002{ 0000001 00000002{
 308685    21 990812 IN6605    SAW001  00001} 0000001 00000001}
 130140    21 940802 IN111001  SCR012  00002A 0001001 00002102A
              - - - - - some records omitted - - - - -
 223240    65 980816 CR955     HAM001  00002H 0008001 00022402H
 406082    65 020816 IN441     VAR200  00001R 0009001 00017101R
 400002    85 010812 CR245     HAX129  00001O 0006001 00009601O

Notes

1. Note that the records are in sequence 1st by key1=10(2) (salesman), 2nd by key2=30(6) (product#).
   
   
2. The input typ=RSF means 'Record Sequential Fixed' (no LineFeeds).
   
   
3. The output file type is now changed to 'typ=LSTt' which means 'Line Sequential Terminated (with LineFeeds)'.
   
   
4. The 't' option on 'LSTt' truncates trailing blanks. The LineFeed is inserted after the right hand non-blank.
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

L3. uvsort examples

uvsort ex#3 - selecting records

 #1. Salesman 21,44,& 65 (columns 10-11)

 #2. Product hammers ('HAM' in cols 31-33)

uvsort solution

 uvsort "fili1=dat1/sales2,rcs=64,typ=RST,filo1=tmp/sales2,key1=10(2),key2=30(6)\
 ================================================================================
 ,sel1=10(2):21,sel2=30(3):HAM,sel1=10(2):44,sel2=30(3):HAM\
 ===========================================================
 ,sel1=10(2):65,sel2=30(3):HAM"
 ==============================

output file

 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 990802 IN111001  HAM001  000020 0000101 000002020
 150825    44 010804 IN1122    HAM001  000025 0020001 000500025
 315512    44 040805 IN2251    HAM001  00001s 0000201 00000261s
 223240    65 010816 CR955     HAM001  000028 0030001 000840028

Notes

1. We have used dat1/sales2 as the input file for this example, so you will find it easier to check the outputs with 'vi', because sales2 has LineFeeds in the last byte.
   
   
2. 'typ=RST' indicates LF in last byte of rcs=64. The LF will be removed when read in but reinserted on output since 'typ=RST' is also in effect for output (unless typ=... re-specified).
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

L4. uvsort examples

uvsort ex#4 - loading indexed (ISAM) files

           001-006 - customer#
           009-009 - Delete flag 'D'
           011-035 - customer name
           036-060 - address
           061-076 - city
           078-079 - province
           081-090 - postal code
           091-102 - phone#
           103-120 - contact name
           121-180 - this year sales 12 x 5 byte packed fields
           181-240 - last year sales 12 x 5 byte packed fields
           241-256 - unused (no LineFeed at end record)

    uvhd dat1/custmas1 r256  - execute uvhd to display 1st record
    =======================

                       1         2         3         4         5         6
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
             3333332222454545444244545524542222233332445442544422222222224444
             130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
          64 IMO          BC V9S1H1    604-754-5531                  ..4V|...
             4442222222222442535343222233323332333322222222222222222201357000
             9DF00000000002306931810000604D754D55310000000000000000000246C000
         128 .........W0....`........)X}..f3.....\.................4V}...f...
             0000000005300016000000002570063100095000000000000000013570016000
             0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0056C00
         192 ...............................f.....<........f.C
             0000008900000000880000000018000680001300000000694222222222222222
             00C0026C0000C0023C0000C0083C0056D0012C0000C0016D3000000000000000

uvsort solution

 uvsort "fili1=dat1/custmas1,rcs=256,typ=RSF,filo1=tmp/custmas1,typ=ISF,\
 =======================================================================
 isk1=0(6),isk2=60(16),key1=0(6),key2=60(16)"
 ===========================================

The sort keys are coded as key1,key2,etc. 'isk' is the code for 'Indexed Sequential Key', which could be different than the sort keys, but usually the same.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

L5. uvsort examples

uvsort ex#5 - Re-Organizing Indexed files

Example #2 (on the previous page) sorted a 'sequential' file & loaded an 'indexed' file. This example will input an indexed file, sort,& write back to the same file.

Writing back to the same file could be a problem if your machine just happened to crash at while writing the output, so you should ensure that a backup has preceded a job such as this.

Sorting & reloading an indexed file should be done periodically to reorganize the indexed file & drop any deleted records. Deleted records could be both physical & logical.

'Reorganizing' means to put the data partition records back into sequence to improve sequential access times. Randomly added records will be at the end of the data partition & reading, sorting,& reloading will put them back into physical sequence depending on their key fields.

Physical deletes are usually performed by COBOL program 'delete' commands. The micro Focus COBOL file handler changes the delete flag (an extra byte at end of each record) from x'0A' to x'00'. The uvsort program wont see these records because the D-ISAM file handler (compatible with COBOL file handler) drops them automatically.

Logical deletes are probably also performed by a COBOL program, but rather than use the 'delete' command, the program sets a data byte to some agreed upon value that other programs will check. In our uvsort example below we will assume that a 'D' in byte 8 (column 9) identifies a logically deleted record. uvsort will drop it due to the 'del=8(1):D'.

Indexed file sort/reload demo

 #1. cp dat1/custmast* tmp      - copy custmast & custmast.idx to tmp subdir
     =====================

 #2. uvsort "fili1=tmp/custmast,rcs=256,typ=ISF,filo1=tmp/custmast\
     =============================================================
     ,key1=0(6),isk1=0(6d),del=8(1):D"
     =================================

Console Log

uvsort:050330:115707: uvsort fili1=tmp/custmast,filo1=tmp/custmast uvsort:050330:115707: EOF fili01 30 rds, 8224 size; tmp/custmast uvsort:050330:115707: DISAM: recs=30,rcsz=256,keys=7; tmp/custmast uvsort:050330:115707: sort phase complete, used 512 megabytes, sort output begin uvsort:050330:115707: EOF filo01 27 wrts, 0 size; tmp/custmast uvsort:050330:115707: DISAM: recs=27,rcsz=256,keys=1; tmp/custmast

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

L6. uvsort examples

uvsort ex#6 - reformatting records

           001-006 - customer#
           009-009 - delete flag (D or X)
           011-035 - customer name
           036-047 - phone# (from 091-102 of input)

uvsort solution

 uvsort "fili1=dat1/custmas1,rcs=256,typ=RSF,filo1=tmp/phones,typ=LSTt,rcs=80\
 ============================================================================
 clr=35(45):0x20,mvc=35(12)90,key1=35(12)"
 =========================================

sample output file

 313720    MONITREX ENGINEERING LTD
 306959  X 356582 ALBERTA LTD.          -   -
 208060    E&L TRUCKING              403-231-7312
 201120    ALLTYPE RENTAL LTD.       403-246-5274
 308685    FOOTHILLS ELECTRIC        403-296-4856
 211140  D FORD NEW HOLLAND          403-312-2760
 318833    TOP NOTCH CONSTRUCTION    403-385-2965
 224700    OTTO MOBILES WESTERN LTD. 403-467-3197
 223240  X NICHOLSON CHEVROLET LTD.  403-467-4257
 234300    TERRITORIAL REWIND        403-512-3948
 204700    CASE POWER EQUIPMENT      403-538-1743
 231550    SPARROW ELECTRIC CO. LTD. 403-742-6139
 301120  D ALBERTA GAS CHEMICALS LTD 403-952-2475
 401210    COAST RANGE CONSTRUCTION  604 681 2626

We might have dropped the deleted records (X or D in column 9), by adding the following to the uvsort command string:

 ,del1=8(1):D,del1=8(1):X
 ========================

The 'mvc=35(12)90' is performed during input, so the sort key is specified as 'key1=35(12)'. The 'key=...' is specified following the 'mvc=...', but in fact the command sequence does not matter. You can perform 'mvc's at output time using the 'o' option (mvco=... see page 'I3').

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

L7. uvsort examples

uvsort ex#7 - convert text file to fixed length Indexed file

COBOL programs (originally from a mainframe) often require Fixed length records & Indexed files. It is difficult to create such a file with the usual unix utilities. here is an example of creating such a file with the 'vi' editor & then using 'uvsort' to load into an Indexed file with the reqqquired record-size & Indexed key.

          BBQ010  BAR-B-Q
          CHR015  LAWN CHAIR
          HAM010  CLAW HAMMER

          --- column layout 1 relative ---
          001-006 - product code
          008-008 - reserve for delete flag (D)
          011-030 - product description
          031-039 - sales (to be initialized to zeros)

uvsort solution

 uvsort "fili1=dat1/products,rcs=80,typ=LST,filo1=tmp/prodmaster,rcs=40,typ=ISF\
        ,isk1=0(6),key1=0(6),mvn=30(9z):0"
 ===============================================================================

sample output file

          BBQ010   BAR-B-Q              000000000
          CHR015   LAWN CHAIR           000000000
          HAM010   CLAW HAMMER          000000000

1. Of course we might have several more data fields in real situation.
   
   
2. 'mvn=30(9):0' initializes a zoned (unpacked) field of 9 bytes could use 'mvn=30(5p)' to initialize a packed 9 digit field
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

M1. rebuilding Indexed files from the data partition of corrupted file s

Microfocus COBOL Indexed files (IDXFORMAT1)

 Data    partition filename: 'filename'     <-- NO .dat extension
                             'filename.dat'   - OR with .dat if ISDATEXT=".dat"
 Indexed partition filename: 'filename.idx' <-- Indexed part always ".idx"

C-ISAM data partition filename: 'filename.dat' <-- C-ISAM convention

No control area at begining of data records (vs MBP ISAM files)

Data records are 1 byte larger than record size loaded because a LineFeed is appended to each data record. The LineFeed is converted to a x'00' if the record is deleted

recovering Microfocus data records

The following will demonstrate the process by loading an Indexed file, pretending that the Index is corrupted,& then rebuilding the Indexed file from the data partition only.

 #0. Create an Indexed file for recovery test/demo

     uvcp "fili1=dat1/custmas2,rcs=128,typ=RSF,filo1=tmp/cm2,typ=ISF,isk1=0(6)"
     ==========================================================================

 #1. Copy the data partition to a save file

     cp tmp/cm2.dat tmp/cm2.sav
     ==========================

 #2. erase the original filename.dat & filename.idx

     rm tmp/cm2.dat tmp/cm2.idx
     ==========================

 #3. sort/reload the Indexed file from the data partition

     uvsort "fili1=tmp/cm2.sav,rcs=129,typ=RSF,filo1=tmp/cm2,rcs=128,typ=ISF\
     =======================================================================
            ,isk1=0(6),key1=0(6),del=128:0x00"
            =================================

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

M2. rebuilding Indexed files from the data partition of corrupted files

10b - MBP COBOL Indexed files

      Data    partition filename is: 'filename.DAT'
      Indexed partition filename is: 'filename.KEY'

588 byte binary control area at begining of data records (Microfocus ISAM files do not have).

Data records are 2 bytes larger than record size loaded because a 2 byte binary record size is prepended to each data record. The record size is converted to a x'0000' if the record is deleted.

recovering MBP data records from an Indexed file

The following will demonstrate the process by loading an Indexed file, pretending that the Index is corrupted,& then rebuilding the Indexed file from the data partition only.

 #0. Create an Indexed file for recovery test/demo

     uvcp "fili1=dat1/custmas2,rcs=128,typ=RSF,filo1=tmp/cm2,typ=ISF,isk1=0(6)"
     ==========================================================================

 #1. Copy the data partition to a save file

     cp tmp/cm2.DAT tmp/cm2.sav1
     ===========================

 #2. erase the original filename.DAT & filename.KEY

     rm tmp/cm2.DAT tmp/cm2.KEY
     ==========================

 #3. drop the control area from the begining of the data partition
    (using the UNIX 'dd' utility)

     dd if=tmp/cm2.sav1 of=tmp/cm2.sav2 bs=588 skip=1
     ================================================

 #4. sort/reload the Indexed file from the data partition

     uvsort "fili1=tmp/cm2.sav2,rcs=130,typ=RSF,filo1=tmp/cm2,rcs=128,typ=ISF\
     ========================================================================
             ,isk1=0(6),key1=0(6),del=0(2):0x0000,mvc=0(128)2"
              ================================================

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

N1. Record Sequential Variable length files (typ=RSV)

As of Nov 2002 'typ=RSV' supports OUTPUT files as well as INPUT files. 'typ=RSV' provides compatibility with Micro Focus COBOL IDXFORMAT3 Variable length files, but only for Sequential. Indexed variable length support is provided by an alternate version of uvsort (uxsort linked with EXTFH).

For complete documentation on these files, please see the 'FILE STRUCTURES' chapter in Micro Focus Object COBOL or Server Express COBOL, or the web site https://supportline.microfocus.com/documentation/books/ocds4140/fhfile.htm.

'typ=RSV' files have 128 byte file header. Each record has 2 or 4 byte record header/prefix which has x'40' in the first nibble to identify a data record, followed by the record size in the next 12 bits or 28 bits, depending on max recsize <= 4095 or > 4095 bytes.

uvcp/uvsort RSV Output Implementation

1. The maximum rcsz is specified by 'rcs=____' & the minimum rcsz is specified by option 'v' of 'typ=RSVv____'. Record size determined by scanning backward to last non-blank & round up to a multiple of 4 (includes 2/4 byte rec hdr).
   
   
2. If minimum rcsz (option v) is not specified, all output records will be written with maximum record size & blank padded as necessary.
   
   
3. 'uvsort' allows any combination of INPUT/OUTPUT file types (LST,RSF,ISF,RSV) & allows you to convert files from any type to any other type. For example you can create test files by using 'vi' to create text records & then use uvcp/uvsort to convert the text file (typ=LST) to a typ=RSV output file.
   
   

   uvcp/uvsort typ=RSV examples

 #1. uvcp "fili1=dat1/testRSV1,rcs=256,typ=LST,filo1=tmp/a,rcs=254,typ=RSV"
     ======================================================================

• reads text file (typ=LST) of records less than 256 bytes
• writes Rec Seq Variable (typ=RSV) records
• all records will be 256 bytes (2 byte recsize hdr + 254 bytes data)

 #2. uvcp "fili1=dat1/testRSV1,rcs=256,typ=LST,filo1=tmp/b,rcs=254,typ=RSVv62"
     =========================================================================

    Same as ex#1 but writes typ=RSV records varying from 128 to 256
    - 064 bytes (2 hdr + 062 data) if bytes 062+ all blank
      ??? bytes (2 hdr + ??? data) depending on last nonblank (multiple of 4)
    - 256 bytes (2 hdr + 254 data) is maximum

 #3. uvcp "fili1=dat1/testRSV1,rcs=256,typ=LST,filo1=tmp/c,rcs=5000,typ=RSVv400"
     ===========================================================================

    This example writes 4 byte record headers (vs 2 bytes in above examples)
    because the maximum record size is > 4095 bytes (rcs=5000).
    Record sizes vary from 404 to 5004 depending on last nonblank
    -  404 bytes (4 hdr +  400 data) if bytes 400+ all blank
      ???? bytes (4 hdr + ???? data) depending on last nonblank (multiple of 4)
    - 5004 bytes (4 hdr + 5000 data) is maximum

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

N2. Record Sequential Variable length files (typ=RSV)

using uvcp to create a typ=RSV file

 #2. uvcp "fili1=dat1/testRSV3,rcs=256,typ=LST,filo1=tmp/test3,rcs=254,typ=RSVv80"
    =============================================================================

input text file (3 records)

 #1 testRSV3 - text to create Rec Seqntl Variable(RSV) file to test uvcp & uvsort
 #2 uvcp "fili1=dat1/testRSV3,rcs=256,typ=LST,filo1=tmp/test3,rcs=254,typ=RSVv80"
 #3 output records vary from min(84) to max(256) depending on last nonblank-----|
 -----this record will be 164 (data 160 + 2 byte rechdr + 2 round to mult 4)---->

output RSV file (displayed with uvhd)

 uvhd tmp/test3 v      <-- display file using uvhd with option 'v'
 ================

                       1         2         3         4         5         6
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 0~......030114102019.................>.......................P..
             370000003333333333330000000000000000030000000000000000000F000500
             0E00000003011410201900000000000000000E0100000000100000000E000000
          64 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             0000000000000000000000000000000000000000000000000000000000000000
                       1         2         3         4         5         6
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
         128 @P#1 testRSV3 - text to create Rec Seqntl Variable (RSV) file to
             4523276775553222767727626766762566256767625676666622555226666276
             00310453423630D0458404F032514502530351E4C0612912C5082369069C504F
          64  test uvcp/uvsort
             27677277672777677222
             0453405630F563F24000
                       1         2         3         4         5         6
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
         212 @P#2 uvcp "fili1=dat1/testRSV3,rcs=256,typ=LST,filo1=tmp/test3,r
             4523277672266663366732767755532767333327773455266663376727677327
             0032056300269C91D4141F45342363C233D256C490DC34C69CF1D4D0F45343C2
          64 cs=254,typ=RSVv80"
             67333327773555733222
             33D254C490D236680200
                       1         2         3         4         5         6
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
         296 @.#3 output records vary from min(84) to max(256) depending on l
             4A23267777727666767276772676626662332276266723332266766666626626
             00330F540540253F24306129062FD0D9E884904F0D188256904505E49E70FE0C
          64 ast nonblank-----|-----this record will be 164 (data 160 + 2 byt
             6772666666662222272222276672766676276662662333226676233322232677
             1340EFE2C1EBDDDDDCDDDDD48930253F24079CC025016408414101600B020294
         128 e rechdr + 2 round to mult 4)---->
             627666672223276766276267672322222322
             502538420B0202F5E404F0D5C4049DDDDE00

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

N3. Record Sequential Variable length files (typ=RSV)

File Header record for typ=RSV (& typ=IDX) files

                       1         2         3         4         5         6
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 0~......021125110120.................>.......................~..
             370000003333333333330000000000000000030000000000000000000F000700
             0E00000002112511012000000000000000000E0100000000100000000E000E00
          64 ................................................................
             0000000000000000000000000000000000000000000000000000000000000000
             0000000000000000000000000000000000000000000000000000000000000000

00-00 - x'30' identifies the file header record (vs x'40' for data records)

 01-01 - data size of this record x'7E' = (7*16+14*1) = 112+14 = 126
       - total size is 128 including the 2 byte record header

 00-03 - If max rcsz > 4095, the record header would be 4 bytes x'3000007C'
       - 7*16 + 12*1) = 112+12 = 124 + 4 byte header = 128 total record size

08-19 - file creation date/time stamp (yymmddHHMMSS)

36-36 - reserved value x'3E' required for Micro Focus COBOL IDXFORMAT3 files

39-39 - file ORGanization x'01' = Sequential

48-48 - Recording Mode x'01' = Variable format

 54-57 - Maximum record length
       - In this example x'000000FE' = (15*16+14*1) = 240+14 = 254

 58-61 - Minimum record length
       - In this example x'0000007E' = (07*16+14*1) = 112+14 = 126

typ=IDX files same as typ=RSV with index added

The file structure described above for typ=RSV files also applies to typ=IDX files which include an indexed partition & some extra system records in the data partition. You must use the alternate program 'uxsort' (vs 'uvsort) if either input or output or both files are typ=IDX.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

O1. 'uxsort' test/demo Indexed Seqntl Variable length files

test/demo uxsort for Indexed Seqntl Variable length files

'uxsort' supports Indexed Sequential Variable length record files & must be compiled with the Micro Focus COBOL 'cob' command as shown on page D1 of install.htm. You can demo uxsort using test files provided as shown below:

dat1/idxtest - test file to demo uxsort

 0010.0060 idxtest text file to create Var Lth Indexed files.
 0020.0064 - input to COBOL,uxcp,uxcopy to create IDXf1,3,8 files
 0030.0065 - 01-04=ISAM key, 05=separator, 06-09=size, 10-400 text
 0040.0067 - output record varies from 64 to 256, depending on 06-09
 0050.0080 - here is an 80 byte record------------------------------------------>
 0060.0160 - here is a 160 byte record...........................................
 0070.0240 - here is a 240 byte record (last record in file).....................
 ===============================================================================>

uxsort - read text (typ=LST), write Indexed file (typ=IDXf3)

 #1. uxsort "fili1=dat1/idxtest,rcs=256,typ=LST\
     ===========================================
             ,filo1=tmp/idxf3v60,rcs=254,typ=IDXf3v60,isk1=0(4),key1=0(4)"
             ============================================================

 #2. uvhd tmp/idxf3v60.dat v    <-- examine output data partition with 'uvhd'
     =======================      - need option 'v' for IDXf3 files

uxsort - read Indexed (typ=IDXf3), write text (typ=LST)

 #3. uxsort "fili1=tmp/idxf3v60,rcs=256,typ=IDXf3,filo1=tmp/idxf3v60vL,typ=LSTt,key1=0(4)"
     ============================================================================

 #4. vi tmp/idxf3v60L           <-- examine output file with 'vi'
     ================

This converts the typ=IDXf3 file back to a text file (typ=LSTt) which should be the same as the original file used in step #1. There is no need to specify the indexed keys or the 'v' option for input files.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

O2. 'uxsort' test/demo Indexed Seqntl Variable length files

uvhd hexdump of IDXf3 varlth indexed file

 #1a. uvhd tmp/IDXf3v60.dat v   <-- examine output data partition with 'uvhd'
      =======================     - uvhd option 'v' for IDXf3 or IDXf8 files

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 0~......0705141326133507051413261335.>.......................<..
             370000003333333333333333333333333333030000000000000000000F000300
             0E00000007051413261335070514132613350E0200000000100000000E000C00
          64 ............................................X"8*................
             0000000000000000000000000000000000000000000052320000000000000000
             00000000000020100000000000000000000000000000828A0000000000000000
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
         384 @<0010.0060 idxtest text file to create Var Lth Indexed files...
             4333332333326677677276772666627626766762567247624666766266667200
             0C0010E00600948453404584069C504F032514506120C4809E45854069C53E00
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
         448 @@0020.0064 - input to COBOL,uxcp,uxcopy to create IDXf1,3,8 fil
             4433332333322266777276244444277672776677276267667624456323232666
             000020E00640D09E05404F03F2FCC5830C583F0904F0325145094861C3C8069C
          64 es..
             6700
             5300
                      10        20        30        40        50        60
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
         516 @A0030.0065 - 01-04=ISAM key, 05=separator, 06-09=size, 10-400 t
             4433332333322233233345442667223337676767672233233376762233233327
             010030E00650D001D04D931D0B59C005D3501214F2C006D09D39A5C010D40004
          64 ext.
             6770
             5840
                      10        20        30        40        50        60
 r#        5 0123456789012345678901234567890123456789012345678901234567890123
         584 @C0040.0067 - output record varies from 64 to 256, depending on
             4433332333322267777727666762767667267662332762333226676666662662
             030040E00670D0F540540253F240612953062FD06404F0256C04505E49E70FE0
          64 06-09...
             33233000
             06D09000
                      10        20        30        40        50        60
 r#        6 0123456789012345678901234567890123456789012345678901234567890123
         656 @P0050.0080 - here is an 80 byte record-------------------------
             4533332333322266762672662332677627666762222222222222222222222222
             000050E00800D0852509301E080029450253F24DDDDDDDDDDDDDDDDDDDDDDDDD
          64 ----------------->..
             22222222222222222300
             DDDDDDDDDDDDDDDDDE00
                ----- last 3 records omitted to keep this on 1 page -----

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

O3. 'uxsort' test/demo Indexed Seqntl Variable length files

create IDXf8 varlth indexed file & examine with uvhd

 #1. uxsort "fili1=dat1/idxtest,rcs=256,typ=LST\
     ==========================================
             ,filo1=tmp/IDXf8v60,rcs=254,typ=IDXf8v60,isk1=0(4),key1=0(4)"
             =============================================================

 #2. uvhd tmp/IDXf8v60.dat v  <-- examine output data partition with 'uvhd'
     =======================

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 3.......0705141522382607051415223827.>.......................<..
             3F0000003333333333333333333333333333030000000000000000000F000300
             3E00000007051415223826070514152238270E0200180000106000000E000C00
          64 ............................................X"8*X"8*............
             0000000000000000000000000000000000000000000052325232000000000011
             00000007000022100000000000000000000000000008828A828A000000000000
         128 ................................................................
             00000011000000000000000000000000000000000000000F0000000000000000
             000000004244010400000040000000C0000000000000003E0000000000000000
                     ----- bytes 192-960 removed (all x'00's) -----
         960 ................................................................
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890123
        3328 @<0010.0060 idxtest text file to create Var Lth Indexed files...
             4333332333326677677276772666627626766762567247624666766266667200
             0C0010E00600948453404584069C504F032514506120C4809E45854069C53E00
                      10        20        30        40        50        60
 r#        3 0123456789012345678901234567890123456789012345678901234567890123
        3392 @@0020.0064 - input to COBOL,uxcp,uxcopy to create IDXf1,3,8 fil
             4433332333322266777276244444277672776677276267667624456323232666
             000020E00640D09E05404F03F2FCC5830C583F0904F0325145094861C3C8069C
          64 es..
             6700
             5300
                      10        20        30        40        50        60
 r#        4 0123456789012345678901234567890123456789012345678901234567890123
        3464 @A0030.0065 - 01-04=ISAM key, 05=separator, 06-09=size, 10-400 t
             4433332333322233233345442667223337676767672233233376762233233327
             010030E00650D001D04D931D0B59C005D3501214F2C006D09D39A5C010D40004
          64 ext.
             6770
             5840
                      10        20        30        40        50        60
 r#        5 0123456789012345678901234567890123456789012345678901234567890123
        3536 @C0040.0067 - output record varies from 64 to 256, depending on
             4433332333322267777727666762767667267662332762333226676666662662
             030040E00670D0F540540253F240612953062FD06404F0256C04505E49E70FE0
          64 06-09...
             33233000
             06D09000
                    ----- records 6,7,8 not shown to fit 1 page -----

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

O4. 'uxsort' test/demo Indexed Seqntl Variable length files

IDXORMAT3/IDXFORMAT8 file header layout

The Vancouver Utilities (uxcopy,uxsort,uxqrpg,uxcp,uvhd) are compatible with Micro Focus COBOL variable length sequential & Indexed files (IDXFORMAT3 & 8).

We will document the most important fields in the file header for IDXf3/IDXf8. If desired, you can find all fields defined on the Micro Focus website.

Note that you can investigate the file headers using 'uvhd' with option 'v' for IDXFORMAT3 or IDXFORMAT8. See sample on the previous page.

 000-003 - file header record length
         - 1st 4 bits x'30' identifies a system record
         - next 12 bits is header length if max record length is < 4095
         - next 28 bits is header length if max record length is => 4095

008-014 - creation date/time yymmddhhmmsscc (indexed files only)

039-039 - organization (1=seqntl, 2=indexed, 3=relative) 043-043 - file type (3=IDXFORMAT3, 8=IDXFORMAT8) 048-048 - recording mode (0=fixed, 1=variable)

 054-057 - maximum record length (big end format)
         - example recsize 80 = x'00000050'
 058-061 - minimum record length

 068-071 - record count (4 bytes Big-End Binary)
         - Indexed files only (not in variable length non-indexed file hdrs)

128-135 - offset to End-of-File (8 bytes binary big end format)

 140-141  - number of keys in Indexed file

record header layout

 000-000 - 1st 4 bits identifies the record type
 x'1_'   - system record - duplicate occurrence record
 x'2_'   - deleted record
 x'3_'   - system record
 x'4_'   - user data record
 x'5_'   - reduced user data record
 x'6_'   - pointer record
 x'7_'   - user data record referenced by a pointer record
 x'8_'   - reduced user data record referenced by a pointer record

000-001 - next 12 bits is the record length if max recsize < 4095 000-003 - next 28 bits is the record length if max recsize =? 4095 002-... - record data if max recsize < 4095 004-... - record data if max recsize => 4095

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

O5. 'uxsort' test/demo Indexed Seqntl Variable length files

Test typ=IDX file options

See page 'E6' which documents file options 'v' 'x' 'y' 'z' to modify the default behavior of the Micro Focus COBOL EXTFH file handler. Note that these tests are the same as #1 on the previous page with the addition of options x,y,or z on the output file typ=IDXv60__?__

scripts to test typ=IDX file options

The next few pages will list some scripts used at UV Software to test/demo using uxsort to write IDXFORMAT3/IDXFORMAT8 files & read them back to verify.

These scripts are supplied in /home/uvadm/sf/demo/..., which is in the PATH of the supplied profile. You can run these at your site. See instructions & explanations embedded within the scripts.

can use 'uxcp' scripts to read back

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

O6. 'uxsort' test/demo Indexed Seqntl Variable length files

 # uxsortIDXf3w - write various IDXFORMAT3 files to test uxsort
 #              - by Owen Townsend, Jan28/06, updated May14/07
 #              - IDXFORMAT8 files created by Micro Focus COBOL EXTFH
 #              - file handler used by uxcp, uxsort, uxcopy, uxqrpg
 #
 #*uxsortIDXf3w - write various IDXf8 files for read back by uxcpIDXf3r
 #                   ** can use 'uxcp' scripts to read back **
 # uxcpIDXf3r   - read back various IDXf8 files written by uxsortIDXf3w
 #              - converts output files back to text records
 # uxcpIDXf3c   - copy various IDXf8 files written by uxsortIDXf3w
 #              - writes output files with same types & record-sizes
 #
 # OPTIONS: f1=IDXFORMAT1, f3=IDXFORMAT3, f8=IDXFORMAT8 (see www.microfocus.com)
 # x1=noindex, x2=indexed; y1=fixed, y2=variable, y4=scanback to last non-blank
 # v=min recsize, (maxsize declared by rcs=___)
 #
 # 1. typ=LST-->IDXf8 indexed, varfmt but all 252+4 since no option v minsize
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8,typ=IDXf8,isk1=0(4)"
 #===============================================================================
 #
 # 2. typ=LST-->IDXf8v60 indexed, variable recsize pass thru from input text recs
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60,typ=IDXf8v60,isk1=0(4)"
 #===============================================================================
 #
 # 3. typ=LST-->IDXf8v60y4 indexed variable option y4 recsize at last nonblank
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60y4,typ=IDXf8v60y4,isk1=0(4)"
 #===============================================================================
 #
 # 4. typ=LST-->IDXf8x1 noindex(x1), fixed 252, no IDX filehdr, same as typ=RSF
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8x1,typ=IDXf8x1,isk1=0(4)"
 #===============================================================================
 #note - must read back using typ=RSF, since no index & fixed lth (no v)
 #
 # 5. typ=LST-->IDXf8v60x1 noindex(x1), varfmt, but all 252+4
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60x1,typ=IDXf8v60x1,isk1=0(4)"
 #===============================================================================
 #
 # 6. typ=LST-->IDXf8v60x1y4 noindex(x1), variable 60-252, depending on LNB (y4)
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60x1y4,typ=IDXf8v60x1y4,isk1=0(4)"
 #===============================================================================
 #
 echo "You can examine output files with 'uvhd', example for output#1 above"
 echo " "
 echo "uvhd tmp1/idxf8.dat v   <-- use 'v' option for IDXf8 files"
 echo "====================="
 #
 #Note - script may bomb on uxcp #6 if you only have 1 Micro Focus COBOL license
 #     - But you can add following to your profile to wait 1 minute (vs bomb)
 #
 # export ASLMFRETRY="5,15"
 # ========================

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

O7. 'uxsort' test/demo Indexed Seqntl Variable length files

 # uxsortIDXf8w - write various IDXFORMAT files to test uxsort
 #            - by Owen Townsend, Jan28/06, updated May14/07
 #            - IDXFORMAT8 files created by Micro Focus COBOL EXTFH
 #            - file handler used by uxcp, uxsort, uxcopy, uxqrpg
 #
 #*uxsortIDXf8w - write various IDXf8 files for read back by uxcpIDXf8r
 #                   ** can use 'uxcp' scripts to read back **
 # uxcpIDXf8r   - read back various IDXf8 files written by uxsortIDXf8w
 #              - converts output files back to text records
 # uxcpIDXf8c   - copy various IDXf8 files written by uxsortIDXf8w
 #              - writes output files with same types & record-sizes
 #
 # OPTIONS: f1=IDXFORMAT1, f3=IDXFORMAT3, f8=IDXFORMAT8 (see www.microfocus.com)
 # x1=noindex, x2=indexed; y1=fixed, y2=variable, y4=scanback to last non-blank
 # v=min recsize, (maxsize declared by rcs=___)
 #
 # 1. typ=LST-->IDXf8 indexed, varfmt but all 252+4 since no option v minsize
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8,typ=IDXf8,isk1=0(4)"
 #===========================================================================
 #
 # 2. typ=LST-->IDXf8v60 indexed, variable recsize pass thru from input text recs
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60,typ=IDXf8v60,isk1=0(4)"
 #===============================================================================
 #
 # 3. typ=LST-->IDXf8v60y4 indexed variable option y4 recsize at last nonblank
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60y4,typ=IDXf8v60y4,isk1=0(4)"
 #===============================================================================
 #
 # 4. typ=LST-->IDXf8x1 noindex(x1), fixed 252, no IDX filehdr, same as typ=RSF
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8x1,typ=IDXf8x1,isk1=0(4)"
 #===============================================================================
 #note - must read back using typ=RSF, since no index & fixed lth (no v)
 #
 # 5. typ=LST-->IDXf8v60x1 noindex(x1), varfmt, but all 252+4
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60x1,typ=IDXf8v60x1,isk1=0(4)"
 #===============================================================================
 #
 # 6. typ=LST-->IDXf8v60x1y4 noindex(x1), variable 60-252, depending on LNB (y4)
 uxsort "fili1=dat1/idxtest,typ=LST,rcs=252,filo1=tmp1/idxf8v60x1y4,typ=IDXf8v60x1y4,isk1=0(4)"
 #===============================================================================
 #
 echo "You can examine output files with 'uvhd', example for output#1 above"
 echo " "
 echo "uvhd tmp1/idxf8.dat v   <-- use 'v' option for IDXf8 files"
 echo "====================="
 #
 #Note - script may bomb on uxcp #6 if you only have 1 Micro Focus COBOL license
 #     - But you can add following to your profile to wait 1 minute (vs bomb)
 #
 # export ASLMFRETRY="5,15"
 # ========================

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

O8. 'uxsort' test/demo Indexed Seqntl Variable length files

input text file for uxsortIDXf3/uxsortIDXf8

 0010.0060 idxtest text file to create Var Lth Indexed files.
 0020.0064 - input to COBOL,uxcp,uxcopy to create IDXf1,3,8 files
 0030.0065 - 01-04=ISAM key, 05=separator, 06-09=size, 10-400 text
 0040.0067 - output record varies from 64 to 256, depending on 06-09
 0050.0080 - here is an 80 byte record------------------------------------------>
 0060.0160 - here is a 160 byte record...........................................
 0070.0240 - here is a 240 byte record (last record in file).....................
 ===============================================================================>

tmp1 subdir listings after uxsortIDXf3w

 #1. -rw-rw-r--    1 uvadm    users     2176 May 14 13:50 idxf3.dat
     -rw-rw-r--    1 uvadm    users     3072 May 14 13:50 idxf3.idx

 #2. -rw-rw-r--    1 uvadm    users     1148 May 14 13:50 idxf3v60.dat
     -rw-rw-r--    1 uvadm    users     3072 May 14 13:50 idxf3v60.idx

 #3. -rw-rw-r--    1 uvadm    users     1148 May 14 13:50 idxf3v60y4.dat
     -rw-rw-r--    1 uvadm    users     3072 May 14 13:50 idxf3v60y4.idx

 #4. -rw-rw-r--    1 uvadm    users     1920 May 14 13:50 idxf3x1

 #5. -rw-rw-r--    1 uvadm    users      892 May 14 13:50 idxf3v60x1

 #6. -rw-rw-r--    1 uvadm    users      892 May 14 13:50 idxf3v60x1y4

tmp1 subdir listings after uxsortIDXf8w

 #1. -rw-rw-r--    1 uvadm    users     4864 May 14 15:26 idxf8.dat

 #2. -rw-rw-r--    1 uvadm    users     4112 May 14 15:26 idxf8v60.dat

 #3. -rw-rw-r--    1 uvadm    users     4112 May 14 15:26 idxf8v60y4.dat

 #4. -rw-rw-r--    1 uvadm    users     1764 May 14 15:26 idxf8x1

 #5. -rw-rw-r--    1 uvadm    users      892 May 14 15:26 idxf8v60x1

 #6. -rw-rw-r--    1 uvadm    users      892 May 14 15:26 idxf8v60x1y4

Please see 'uvhd' of IDXF3v60 on page 'O2' & IDXf8v60 on page 'O3'.

Please see uvcopy6.htm for IDXf3/IDXf8 test/demos (similar to the above), but for 'uxcopy' vs 'uxcp' or 'uxsort'.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

P1. uvsort: RDW (Record Descriptor Word) variable length files

RDW (Record Descriptor Word) variable length files are often used to FTP variable length EBCDIC files from the mainframe to unix/linux/windows systems. uvsort provides for the following types of RDW files:

test/demo files supplied in /home/uvadm/dat1

 uvhd dat1/testRDW z4   <-- investigate RDW file (using uvhd)
 ====================     - only 1st of 8 records shown below

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 ....DELL10 - Dell Inc.
             010044443322246662466222
             080045CC100D045CC09E3E00

Here in uvsort, we will give an example of sorting the ASCII testRDW file to a text file, and then sorting the EBCDIC testRDWe file & translating to ASCII at the same time.

sort RDW ASCII file to a text file

 uvsort "fili1=dat1/testRDW,rcs=64,typ=RDWz4,filo1=tmp/testz4,typ=LSTt,key1=0(6)"
 ================================================================================

text records sorted from RDW file

DELL00 - Dell Inc. HP0000 - Hewlett Packard IBM000 - International Business Machines MFC000 - Micro Focus COBOL MS0000 - Microsoft Corp. REDHAT - Red Hat Linux SUN000 - Sun Microsystems Ltd UVSI00 - UV Software Inc.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

P2. RDW (Record Descriptor Word) variable length files

ex#2 sort RDW EBCDIC file translating to ASCII

Instead of variable length text, we will create fixed length output records that are required for most COBOL input files. For this demo, we will convert to fixed 64 byte records & insert a LineFeed in the last byte so we can have it both ways (fixed length for COBOL, but LFs allow investigation with vi).

 uvsort "fili1=dat1/testRDWe,rcs=64,typ=RDWz4,filo1=tmp/testz4,typ=RST,tra=0(64)"
 ================================================================================

confirm conversion OK with uvhd

 uvhd tmp/testz4 r64h2     <-- display output file with uvhd
 =====================       - option r64 for record size
                             - option 'h2' for hex display to see LineFeed
                             - only 1st record shown below

                      10        20        30        40        50        60
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 DELL10 - Dell Inc.                                             .
             4444332224666246622222222222222222222222222222222222222222222220
             45CC100D045CC09E3E000000000000000000000000000000000000000000000A

Confirmations shown by uvhd

1. Binary record prefixes have been dropped off, because output typ=RST extracts the data record only & omits the typ=RDW record prefix.
   
   
2. LineFeeds in last byte of each record (x'0A's).
   
   
3. Output is ASCII 1st char 'D' is x'44' here vs x'C4' on page 'P1' Translation performed by instruction 'tra=0(64)'.
   
   
4. See pages uvcp.htm for other instructions you can use to reformat data (mvc, clr, rep, tre, tru, trl, trp, sel, del, etc...).
   
   

   see DATAcnv1 for more RDW examples

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

Q1. uvsort - Data file Utility

file type STL

File 'typ=STL' is the STandard Language file system used by AIX COBOL, added to uvsort in February 2016. These are complex file formats with file headers & record headers. There are 3 subtypes as follows:

File typ=STL Indexed & Relative is provided for input files only. You can copy them to output file types (LSTt,RSF,RST,RSV,RDW,ISF,IDXf1,& STLs Sequential).

converting STL Sequential,Indexed,& Relative to typ=LSTt

Here are examples sorting the 3 demo files provided in $UV/dat1/... We will convert them to 'typ=LSTt' which can be easily checked with vi.

 uvsort "fili1=dat1/testSTLs,typ=STLs,rcs=4096,filo1=tmp/teststls,typ=LSTt,key1=0(8)"
 ====================================================================================
 - sort 'typ=STLs' Sequential to 'typ=LSTt' Line Sequential

 uvsort "fili1=dat1/testSTLi,typ=STLi,rcs=4096,filo1=tmp/teststli,typ=LSTt,key1=0(8)"
 ====================================================================================
 - sort 'typ=STLi' Indexed to 'typ=LSTt' Line Sequential

 uvsort "fili1=dat1/testSTLr,typ=STLr,rcs=4096,filo1=tmp/teststlr,typ=LSTt,key1=0(8)"
 ====================================================================================
 - sort 'typ=STLr' Relative to 'typ=LSTt' Line Sequential

We specify 'rcs=4096' or a size you know is larger than the largest record expected. Record size specified by rcs=... will init I/O areas to blanks, The actual record size is specified in the file header &/or as a prefix on each record & could vary from record to record.

Output 'typ=LSTt' inserts a LineFeed x'0A' after the last nonblank, truncating records depending on data present. So the output records could vary from 4096 bytes down to 1 byte.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

Q2. file typ=STL

sorting STL Indexed to typ=RSF

 uvsort "fili1=dat1/testSTLi,typ=STLi,rcs=4096,filo1=tmp/teststli,typ=RSF,key1=0(8)"
 ===================================================================================
 - sort 'typ=STLi' Indexed to 'typ=RSF' Record Sequential Fixed

 In this case the output records would all be 4096 bytes (blank filled on right).
 - regardless of whether Input records are variable or fixed.

 uvsort "fili1=dat1/testSTLi,typ=STLi,rcs=4096,filo1=tmp/teststli,typ=RSF,rcs=200,key1=0(8)"
 ===========================================================================================
 - can specify record size on output side (after filo1=...) as desired

 uvsort "fili1=dat1/testSTLi,typ=STLi,rcs=200,filo1=tmp/teststli,typ=RST,key1=0(8)"
 ==================================================================================
 - or can specify on input side if you know records are <= 200
 - this example specifies output 'typ=RST' which inserts a LineFeed in last byte
   which allows you to use unix tools (vi,etc) on the output file
   (if you know the last byte is unused for data)

sorting STL Indexed (or Relative) to STL Sequential

 uvcp "fili1=dat1/testSTLi,typ=STLi,rcs=4096,filo1=tmp/teststls,typ=STLs,key1=0(8)"
 ==================================================================================
 - sort 'typ=STLi' Indexed to 'typ=STLs' Seqquential
 - output record size is determined by input record size & could be variable

 uvcp "fili1=dat1/testSTLi,typ=STLi,rcs=4096,filo1=tmp/teststls,typ=STLs,rcs=200,key1=0(8)"
 ==========================================================================================
 - But if you specify a record size on the output side (after filo1=...)
   then the output file size will be fixed

STL file formats

See STL file formats documented at https://uvsoftware.ca/uvhd.htm#5J1 or at https://uvsoftware.ca/uvcp.htm#W1

'uvhd' allows you to see the file header records & the record prefixes but 'uvsort' & 'uvcp' will extract only the data records.

 uvhd dat1/testSTLs      <-- will show file header (112 bytes)
 ==================

 uvhd dat1/testSTLs x2   <-- option 'x2' for STL Sequential
 =====================     - use 'x4' for Indexed & 'x8' for Relative
 - file header & system records (for Indexes) will be bypassed
 - you will see only the data records

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

S1. uvsort - unix sort utility with mainframe functionality

CoSort to uvsort converter

In May 2010 we developed a 'CoSort' to 'uvsort' converter. 'cosort2uvsort1' will convert unix/linux CoSort parameters to the equivalent uvsort parameters.

Here is our sample CoSort 'cosort21' & the converted equivalent uvsort 'uvsort21'. On the next page we will present the operating instructions to convert 1 at a time or all CoSorts in a directory.

sample CoSort

 /sort
 /infiles=(SORTIN)
 /length=64
 /field=(custno,  position=1, size=6)
 /field=(slsmn,   position=11, size=2)
 /field=(idate,   position=14, size=6)
 /field=(inv,     position=21, size=8)
 /field=(product, position=31, size=6)
 /field=(qty,     position=39, size=6, numeric)
 /field=(price,   position=46, size=7, numeric)
 /field=(amount,  position=54, size=9, numeric)
 /condition=(oldsale, test=(idate gt "500101" and idate lt "991232"))
 /condition=(slsmn21, test=(slsmn eq "21" and amount gt 20000))
 /condition=(slsmn44, test=(slsmn eq "44" and amount gt 20000))
 /condition=(slsmn65, test=(slsmn eq "65" and amount gt 20000))
 /key=(slsmn,  ascending)
 /key=(amount, descending)
 /omit=(condition=oldsale)
 /outfile=(SORTOF01) /include=(condition=slsmn21)
 /outfile=(SORTOF02) /include=(condition=slsmn44)
 /outfile=(SORTOF03) /include=(condition=slsmn65)

equivalent uvsort

 # cosort21 - uvsort replacing COSORT
 #         - converted at UV_Software on 2010/05/01_16:41:56
 #         - call this parameter file as follows:
 # export SORTIN=...      #<-- define input file
 # export SORTOUT=...     #<-- define output file
 # export PF=...          #<-- define ParamterFile dirctory (in profiles)
 # uvsort $PF/cosort21   #<-- execute uvsort on this parameter file
 # ====================
 fili01=${SORTIN},typ=RSF,rcs=00064
 filo01=${SORTOF01},typ=RSF,rcs=64
 filo02=${SORTOF02},typ=RSF,rcs=64
 filo03=${SORTOF03},typ=RSF,rcs=64
 keys=(10,2,c,a,53,9,c,a)
 del1i00=13(6c)>500101,del2i00=13(6c)<991232
 sel1o01=10(2c):21,sel2o01=53(9z)>20000
 sel1o02=10(2c):44,sel2o02=53(9z)>20000
 sel1o03=10(2c):65,sel2o03=53(9z)>20000

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

S2. CoSort to uvsort converter

converting CoSort to uvsort

If you have the May 2010 version of Vancouver Utilities, you could convert the sample pf/demo/cosort21 to pf/demo/uvsort21, and then execute it on the supplied test file dat1/sales2. The relevant subdirs & files are:

 /home/uvadm/
 :-----bin/                   <-- Vancouver Utility executable programs
 :     :-----uvsort
 :-----pf/                    <-- VU supplied Parameter Files
 :-----pf/IBM/                <-- conversion jobs
 :     :-----cosort2uvsort1   <-- CoSort to uvsort converter
 :-----pf/demo/               <-- demo Parameter Files
 :     :-----cosort21         <-- sample CoSort parameters
 :     :-----uvsort21         <-- converted equivalent uvsort
 :-----dat1/                  <-- Data files subdir
 :     :-----sales2           <-- data file input for converted CoSort
 :-----tmp/                   <-- Temp subdir for output files
 :     :-----sales11
 :     :-----sales44          <-- 3 output files from uvsort21
 :     :-----sales65

 #1. Login uvadm --> /home/uvadm

 #2. uvcopy cosort2uvsort1,fili1=pf/demo/cosort21,filo1=pf/demo/uvsort21
     ===================================================================
     - convert sample CoSort to uvsort equivalent parameters

 #3a. export SORTIN1=dat1/sales2  <-- export input file for uvsort21
      ==========================
 #3b. export OUTDIR=tmp           <-- export output subdir for uvsort21
      =================

 #4. uvsort pf/demo/uvsort21      <-- execute uvsort on parameter file uvsort21
     =======================

You could shorten the 'uvsort' command line if you export a $SYMBOL for the Parameter Files subdir, for example:

 #3c. export PF=/home/uvadm/pf/demo/
      ==============================

 #4a. uvsort $PF/uvsort21      <-- execute uvsort on parameter file uvsort21
      ===================

You would code 'PF' in your profile to define your own Parameter Files subdir. This would allow you to run your uvsorts from any location without having to enter long pathnames.

If you have hundreds of CoSorts, you could convert all CoSorts to equivalent uvsorts as follows:

 uvcopyx cosort2uvsort1 cosortdir uvsortdir uop=q0i7
 ===================================================

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

T1. Sort Timings - uvsort vs mfsort vs unix sort

In this section I will present a sample uvsort & equivalent mfsort, unix sort, and document how I performed some timings between them.

sample uvsort

 # uvsort2 - sample uvsort (UV Software Vancouver Utilities)
 #         - for comparison to 'mfsort' & unix system 'sort'
 #         - by Owen Townsend, UV Software, July 19/2015
 #         - this script stored at /home/uvadm/sf/demo/uvsort2 (in uvadm PATH)
 # See www.uvsoftware.ca/uvsort.htm#T1 doc re demo uvsort vs MFSORT timings
 # - uvsort.htm#T2 shows how to setup big file for sort timings
 echo "uvsort2, input data1/cmbig, Begun at: $(date)"
 uvsort "fili1=data1/cmbig,rcs=256,typ=RSF,filo1=data1/cmbiguvsort,key1=0(6)"
 #============================================================================
 echo "uvsort2, output data1/cmbiguvsort, Ended at: $(date)"

equivalent mfsort

 # mfsort2 - sample MFSORT (Micro Focus standalone sort utility)
 #         - by Owen Townsend, UV Software, July 02/2010
 #         - this script stored at /home/uvadm/sf/demo/mfsort2 (in uvadm PATH)
 # See www.uvsoftware.ca/uvsort.htm#T1 doc re demo MFSORT vs uvsort timings
 # - uvsort.htm#T2 shows how to setup big file for sort timings
 echo "mfsort2, input data1/cmbig, Begun at: $(date)"
 mfsort "sort fields(1,6,ch,a) \
         use data1/cmbig org sq record f,256 \
         give data1/cmbigmfsort"
 echo "mfsort2, output data1/cmbigmf, Ended at: $(date)"

equivalent unix system sort

 # unixsort2 - sample unix system sort for Timing comparison to:
 #             Vancouver Utilities 'uvsort' &  Micro Focus sort 'mfsort'
 #           - by Owen Townsend, UV Software, July 19/2015
 #           - this script stored at /home/uvadm/sf/demo/unixsort2
 # See www.uvsoftware.ca/uvsort.htm#T1 doc re demo uvsort vs unixsort & mfsort
 # - uvsort.htm#T2 shows how to setup big file for sort timings
 echo "unixsort2, input data1/cmbig, Begun at: $(date)"
 sort -k '1.1,1.6' -o data1/cmbigunixsort data1/cmbig
 #===================================================
 echo "unixsort2, output data1/cmbigunixsort, Ended at: $(date)"
 #
 #Timings for 4 gig sort on HP Z420 workstation July 19/2015
 # unix sort = 34 seconds, mfsort = 64 seconds, uvsort = 57 seconds
 # unix sort is faster, but cannot sort mainframe type files because:
 #
 # 1. unix sort cannot sort fixed length record files without LineFeeds
 # 2. unix sort cannot sort on packed decimal fields
 # 3. unix sort cannot sort Indexed files (Micro Focus compatible)
 # 4. unix sort does not have the select/delete/reformat capabilities of
 #    uvsort or mfsort (similar to the mainframe DFSORT or SYNCSORT)

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

T2. sort time comparison - uvsort vs mfsort

script to make BIG file

 #!/bin/ksh
 # makebig1 - make a big file by concatenating spcfd no of times
 echo "makebig1 - make a big file by concatenating spcfd no of times"
 f1="$1"; f2="$2"; integer n="$3"
 if [[ -f "$f1" && -n "$f2" && -n "$n" ]]; then :
    else echo "usage: makebig1 file1 file2 ##  <-- arg1/2=filenames, arg3 number"
         echo "       ======================="
         exit 9; fi
 # if output file does not exist, create empty file now
 if [[ ! -f $f2 ]]; then >$f2; fi
 integer i=0
 while ((i < n))
   do cat $f1 >> $f2
      ((i+=1))
   done
 echo "completed $n concats of $f1 to $f2"
 exit 0

directories for sort time tests

For these tests, I created a separate user 'testbig' with homedir in file system /home4/... where I had lots of space to create & sort BIG files. Here are the relevant directories & files:

 /home/testbig
 :-----data1               <-- subdir for input files
 :     :-----cm1           <-- 8K demo file copied from /home/uvadm/data1/custmas1
 :     :                     - 32 records of 256 bytes = 8192
 :     :-----cmbig         <-- 1 Gig file created by concatenating cm1 125,000 times
 :     :                     - 8192 * 125,000 = 1,024,000,000
 :     :-----cmbiguvsort   <-- output file from uvsort
 :     :-----cmbigmfsort   <-- output file from mfsort
 :     :                     - see scripts mfsort2 & uvsort2 listed on prior page

 uvhd data1/custmas1 r256 <-- use 'uvhd' to display 1st record of test file
 ========================   - customer N&A with 24 * 5 byte packed sales history

                       1         2         3         4         5         6
 r#        1 0123456789012345678901234567890123456789012345678901234567890123
           0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
             3333332222454545444244545524542222233332445442544422222222224444
             130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
          64 IMO          BC V9S1H1    604-754-5531                  ..4V|...
             4442222222222442535343222233323332333322222222222222222201357000
             9DF00000000002306931810000604D754D55310000000000000000000246C000
         128 .........W0....`........)X}..f3.....\.................4V}...f...
             0000000005300016000000002570063100095000000000000000013570016000
             0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0056C00
         192 ...............................f.....<........f.C
             0000008900000000880000000018000680001300000000694222222222222222
             00C0026C0000C0023C0000C0083C0056D0012C0000C0016D3000000000000000

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

T3. sort time comparison - uvsort vs mfsort

Hardware/Software for sort timing tests

• HP Z420 workstation with 16 gig memory 64 bit & 4 1 TB 7200rpm discs
• Red Hat Enterprise 7.1
• Vancouver utilities uvsort version 20150712 (July 12/2015)

• Micro Focus Visual Cobol for Eclipse 2.2 update 2
• mfsort/mfjsort 3.0.00

make 4 gig file & perform timing tests

1. First setup a user with homedir in a file system with lots of space. We setup userid 'testbig' at /home4/testbig & subdirs as illustrated above. You could setup the subdirs in your homedir if enough space in /home/... Our biggest test is an 8 gig file so you need 24 gig (input, output, work).
   
   

 #2. Login as testbig --> /home4/testbig (or login as yourself)
     ================

 #3. cp /home/uvadm/data1/custmas1 data1/cm1 <-- copy 8K demo file
     =======================================   - 32 records * 256 = 8192

 #4. makebig1 data1/cm1 data1/cmbig 500000  <-- run script to make big file
     =====================================    - 8192 * 500000 = 4,096,000,000

 #5a. uvsort2    <-- run uvsort script (see listing on page 'T1' above)
      =======      - see timings below

 #5b. uvsort "fili1=data1/cmbig,rcs=256,typ=RSF,filo1=data1/cmbiguvsort,key1=0(6)"
      ============================================================================
      - alternative, run uvsort directly, all prameters on cmd line (vs script)

150712:114819:BIGSORT1: uvsort fili1=data1/cmbig,filo1=data1/cmbiguvsort 150712:114819:BIGSORT1: inrecs = 1000000 fili01 data1/cmbig 150712:114822:BIGSORT1: inrecs = 2000000 fili01 data1/cmbig --- 12 lines omitted --- 150712:114841:BIGSORT1: inrecs = 15000000 fili01 data1/cmbig 150712:114844:BIGSORT1: inrecs = 16000000 fili01 data1/cmbig 150712:114844:BIGSORT1: EOF fili01 16000000 rds, 4096000000 size; data1/cmbig 150712:114845:BIGSORT1: sort phase complete, used 512 megabytes, sort output begin 150712:114846:BIGSORT1: outrecs = 1000000 filo01 data1/cmbiguvsort 150712:114848:BIGSORT1: outrecs = 2000000 filo01 data1/cmbiguvsort --- 12 lines omitted --- 150712:114915:BIGSORT1: outrecs = 15000000 filo01 data1/cmbiguvsort 150712:114917:BIGSORT1: outrecs = 16000000 filo01 data1/cmbiguvsort 150712:114917:BIGSORT1: EOF filo01 16000000 wrts, 4096000000 size; data1/cmbiguvsort uvsort start 2015/07/12_11:48:19 end 11:49:17 ELAPSED TIME 00:58

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

T4. sort time comparison - uvsort vs mfsort

 #6a. mfsort2    <-- run MFSORT script (see listing on page 'T1' above)
      =======

 #6b. mfsort "sort fields(1,6,ch,a) \
              use data1/cmbig org sq record f,256 \
              give data1/cmbigmfsort"
      ==============================================
      - alternative, run mfsort directly, all prameters on cmd line (vs script)

           Micro Focus MFJSORT Utility 3.0.00

           sort fields(1,6,ch,a)         use data1/cmbig org sq record f,256         give
 SORT204I: ********************* Sort Statistics *********************
 SORT205I: INPUT  file 'data1/cmbig'
                           32000000 record(s) input.
                           32000000 record(s) used.
 SORT206I: OUTPUT file 'data1/cmbigmfsort'
                           32000000 record(s) used.
                           32000000 record(s) output.
 SORT399I: End of Micro Focus MFJSORT Utility.

mfsort2, input data1/cmbig, Begun at: Sun Jul 12 10:51:29 PDT 2015 mfsort2, output data2/cmbigmfsort, Ended at: Sun Jul 12 10:52:34 PDT 2015

specify directory for sort work files

 export TMPDIR=/tmp  <-- optional variable for 'mfsort' work files
 ==================    - defaults to current directory if unspecified

 export SORTDIR=/tmp  <-- optional variable for 'uvsort' work files
 ===================    - defaults to ./tmp/ (tmp subdir in current directory)

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

T5. sort time comparison - uvsort vs mfsort

sort times for 8 GIG file

 #1. makebig1 data1/cm1 data1/cmbig 1000000  <-- run script to make big file
     ======================================    - 8192 * 1000000 = 8,192,000,000

 #2. uvsort2    <-- run uvsort script (see listing on page 'T1' above)
     =======      - console display log similar to 4 gig test above
                  - time = 3 minutes, 54 seconds

 #3. mfsort2    <-- run MFSORT script (see listing on page 'T1' above)
     =======      - console display log similar to 4 gig test above
                  - time = 4 minutes, 57 seconds

conclusions

uvsort is 20% faster than mfsort, which is probably not enough to justify buying the Vancouver Utilities if that were the only reason.

'uvsort' is just 1 part of the Vancouver Utilities, which are designed to replace mainframe utilities on Unix/Linux/windows systems (mainframe utilities such as SORT,IDCAMS,IEBGENER,FILEAID,EASYTRIEVE,etc).

Be sure to try out the 1 FREE utility 'uvhd', which you may download from www.uvsoftware.ca/freestuff.htm. uvhd is all my customers favorite utility.

'uvhd' is a an interactive file investigation utility with 25 help screens. It displays files with packed/binary fields in vertical hexadecimal & prompts for commands to browse, search, select, update, print, etc.

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

U1. JCL conversion Mainframe SORT to uvsort

SORT INCLUDE/OMIT converted to uvsort sel/del

The JCL converter converts mainframe SORT steps to the uvsort equivalent. Any 'INCLUDE/OMIT's are converted to 'sel/del's. Complex SORTs with multiple mixed INCLUDEs & OMITs may require manual changes and a warning is given.

Very complex SORT INCLUDE/OMIT's with multiple mixed AND/OR's might be better rewritten in uvcopy, which has the power to do whatever kind of data manipulation and record selection desired. For example, uvcopy has table load & lookup instructions that can dramatically shorten voluminous selection codes and simplify maintenace.

We will present several examples of SORT INCLUDE/OMIT AND/OR conversions - 1st requiring no change, some requiring changes,& some illustrating splitting the record selections out to a 2nd step (rewritten in uvcopy).

These examples will be presented in the next section V1-V9, but first you should understand the mainframe JCL conversion, which is fully documented in MVSJCLtest2.htm#Part_9, with a short version here on pages U2-U5.

summary of SORT INCLUDE AND/OR examples

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

U2. JCL conversion Mainframe SORT to uvsort

JCL conversion - brief summary

See complete details of JCL conversion at JCLcnv1demo.htm, but here is a summary of the preparations required for JCL conversion:

1. Install Vancouver Utilities as per install.htm#B1 - B6.
2. Setup users uvadm & appsadm as per install.htm#B1 - B6.
3. Compile Vancouver Utilities as per install.htm#C1 - C3.
4. Setup your userid & profile to use VU as per JCLcnv1demo.htm#3C1.
5. Understand stub & common profiles at JCLcnv1demo.htm#1C1 & 1C2.
6. Setup directories required for JCL conversion JCLcnv1demo.htm#Part_3.
7. Copy demo JCL & data files as per JCLcnv1demo.htm#3D1.
8. Convert JCL to ksh as per JCLcnv1demo.htm#3F1.
   

   alternative to 6,7,8

You need more files than are copied by steps 6,7,8 above, so you need to copy the complete directories of JCL conversion demo files as follows. Then you do not have to convert JCL to scripts, the converted scripts will be copied. Assuming you are logged in as yourself & have setup testlibs & testdata in your homedir:

 #6a. cp -r /home/uvadm/mvstest/mvstest1/testlibs1/* testlibs
      =======================================================

 #6b. cp -r /home/uvadm/mvstest/mvstest1/testdata1/* testdata
      =======================================================

2nd alternative to 6,7,8

Actually you could run the following demos by logging in as uvadm and proceeding as follows:

6. Update the uvadm .bash_profile (was copied from supplied stub_profile). Look for following lines & change as shown below:
   
   

      export RUNLIBS=$HOME/testlibs             <-- as supplied
      export RUNDATA=$HOME/testdata

      export RUNLIBS=$HOME/mvstest/mvstest1/testlibs1  <-- change to this
      export RUNDATA=$HOME/mvstest/mvstest1/testdata1

7. Logoff & back on to make new defs of RUNLIBS & RUNDATA effective You can then change to testlibs1 or testdata1 using alliases:
   
   

cdl <-- alias to change to testlibs1 cdd <-- alias to change to testdata1

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

U3. JCL conversion Mainframe SORT to uvsort

Directories relevant to SORT INCLUDE/OMIT conversion demos

Here are the relevant subdirs required to demo conversion of SORT INCLUDE/OMIT's to uvsort sel/del's - selected from the diagrams at JCLcnv1demo.htm#3B2 & 3B3.

 /home/userxx
 :-----testlibs        <-- libraries for JCL conversions
 :     :------jcl0       - original mainframe JCL with SORT INCLUDEs AND/ORs
 :     :------jcls       - converted to Korn shell scripts
 :-----testdata        <-- data files for testing JCL/scripts
 :     :------data       - data files to test/execute Korn shell scripts
 :     :------etc

It would be better to setup superdirs testlibs & testdata with relevant subdirs and copy the files required for the JCL/SOPRT conversions to ksh/uvsort. But if you have login to /home/uvadm, you could run the test/demos there. This requires 'export RUNLIBS=/home/uvadm/mvstest/testlibs' and 'export RUNDATA=/home/uvadm/mvstest/testdata' as shown on the previous page. The relevant subdirs are illustrated below:

Library subdirs for JCL conversions

 /home/uvadm/mvstest/
 :-----testlibs     <-- libraries for JCL conversion to Korn shell scripts
 :     :------cbls       - several directories not relevant to these demos
 :     :------etc          not shown here
 :     :
 :     :------jcl0      <-- original mainframe JCL
 :     :      :----JAR410.JCL,...JAR440.JCL  <-- demo SORT INCLUDE convert
 :     :
 :     :------jcl1,jcl2,jcl3 - intermediate subdirs used in JCL conversion
 :     :
 :     :------jcls      <-- converted to Korn shell scripts
 :     :      :----jar410.ksh,...jar440.ksh  <-- converted to uvsort sel/del
 :     :
 :     :------sf        <-- modified scripts (SORT INCLUDE complex AND/OR's)
 :     :      :           - store in sf/... to avoid overwrite on JCL reconvert
 :     :      :----jar420.ksh,...jar444.ksh  <-- see listings ahead
 :     :------pf        <-- uvcopy jobs to replace complex uvsort sel/del's
 :     :      :           - called as 2nd step in script (after uvsort step)
 :     :      :----jar441.uvc,...jar444.uvc  <-- see listings ahead

Data subdirs for test/demo executions

 /home/uvadm/mvstest/
 :-----testdata      <-- data files for testing JCL/scripts
 :     :------data        - datafiles
 :     :
 :     :------ctl         - GDG control file
 :     :------joblog,jobmsgs,jobtimes,jobtmp,sysout (less relevant subdirs)
 :     :------tmp         - tmp subdir for uvsort & misc use
 :     :------etc         - several less relevant subdirs not shown

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

U4. JCL conversion Mainframe SORT to uvsort

Performing JCL conversion

If you followed instructions on page 'U2', you could execute all steps of JCL conversion as follows: (see complete doc at JCLcnv1demo.htm#3F1).

 jcl2ksh51A all    <-- perform all steps of JCL conversion to ksh scripts
 ==============

 jcl0 -------> jcl1 -----------> jcl2 -------------> jcl3 ------------> jcls
      cleanup       PROC expand      convert to ksh      manual copy 1 at a time
                                                         when ready to test each

JCL conversion is a multi-step process thru several intermediate directories as shown above. The jcl2ksh51A script stops at jcl3/... & we recommend (for your real conversions) that you then copy 1 at a time from jcl3 to jcls when you are ready to debug each script. But for these test/demos we will copy all jcl3/* to jcls/... (jcls/ is in the profile search PATH).

 cp jcl3/* jcls    <-- copy all converted JCL/scripts to search PATH jcls/
 ==============

But if you use the 1st or 2nd alternatives at the bottom of page 'U2', you do not need to convert or copy jcl3 to jcls, because the alternatives copy all subdirs including the converted jcls/...

Here is the 1st of 4 supplied JCLs to demo conversion of SORT/INCLUDEs to uvsort/selects.

JAR410.JCL - 1st of 4 SORT/INCLUDE demos

 //JAR410   JOB  'test SORT INCLUDE AND/OR'
 //STEP1    EXEC PGM=SORT
 //SORTIN   DD DSN=AR.SALES.ITEMS,DISP=SHR
 //SORTOUT  DD DSN=AR.SALES.ITEMS1,DISP=NEW,LRECL=64
 //SYSIN    DD *
 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman & product
 INCLUDE COND=(11,2,CH,EQ,C'21',AND,31,6,CH,EQ,C'HAM001',OR,
                11,2,CH,EQ,C'44',AND,31,6,CH,EQ,C'SAW051')
 /*

See the converted script equivalent listed below. Only this 1st JCL/script is listed with all lines shown. All 8 demo JCLs are listed in the next section 'V0' - 'V9', but only the uvsort lines are extracted & listed for the scripts (all you need to compare original SORT/INCLUDEs to converted uvsort/selects).

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

U5. JCL conversion Mainframe SORT to uvsort

jar410.ksh - 1st of 8 uvsort/select demos

 #!/bin/ksh
 ##JAR410   JOB  'test SORT INCLUDE AND/OR'
 #export start=S0000   #<-- may uncomment & insert restart step#
 export jobid2=jar410 JOBID2=JAR410; scriptpath="$0"; args="$*"
 if [[ -z "$jobid1" ]]; then export jobid1=$jobid2; fi
 for arg in $args; do if [[ "$arg" == *=* ]]; then export $arg; fi; done
 integer JCC=0 SCC=0 LCC=0  # init step status return codes
 autoload jobset51 jobend51 jobabend51 logmsg1 logmsg2 stepctl51 evalcp1
 autoload exportfile exportfil1 exportfil2
 autoload exportgen0 exportgen1 exportgen2 exportgenall exportgenx
 jobset51    # call function for JCL/script initialization
 goto
 S0000=A
 #1======================= begin step#S0010 SORT      =======================
 S0010=A
 alias goto="";
 stepctl51;
 goto
 export JSTEP=S0010; ((XSTEP+=1)); SCC=0; SMAX=0;
 logmsg2 "********** Begin Step $JSTEP/$XSTEP sort      ***********"
 ##S0010    EXEC PGM=SORT
 export PROGID=sort
 export PARM=""
 exportfile  SORTIN data1/ar.sales.items #SHR
 exportfile  SORTOUT data1/ar.sales.items1 #NEW
 exportfile SYSIN $JTMP/${JSTEP}_${PROGID}_SYSIN
 cat > $SYSIN <</*
 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman & product
 INCLUDE COND=(11,2,CH,EQ,C'21',AND,31,6,CH,EQ,C'HAM001',OR,
                11,2,CH,EQ,C'44',AND,31,6,CH,EQ,C'SAW051')
 /*
 logmsg2 "Executing--> uvsort \"fili1=\$SORTIN,typ=RSF,rcs=64,filo1=\$SORTO..."
 #3----------------------------------------------------------------------
 uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64,keys=(10,2,c,a,30,6,c,a)\
 ,sel1=10(2c):21,sel2=30(6c):HAM001,sel1=10(2c):44,sel2=30(6c):SAW051"
 #4----------------------------------------------------------------------
      ----- step status test & EOJ Normal/AbNormal code omitted -----

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

U6. JCL conversion Mainframe SORT to uvsort

Executing jcls/... to demo SORT/INCLUDE conversions

You can execute any of the SORT/INCLUDE demo JCL/scripts (listed on page 'U1' or 'V0') as follows, using jar410.ksh for example:

 #1a. Login as yourself if you copied testlibs/testdata to your homedir as on 'U2'.
 #1b. Or Login as uvadm for 2nd alternative documented on page 'U2'.

 #2. cdl   <-- alias to change to $RUNLIBS ($HOME/testlibs/)

 #3a. vi jcl0/JAR410.JCL  <-- inspect original mainframe JCL (before conversion)

 #3b. vi jcls/jar410.ksh  <-- inspect converted script equivalent
                            - see listings ahead on pages 'U5' & 'U6'

 #4. cdd   <-- alias to change to $RUNDATA ($HOME/testdata/)

 #5a. vi data1/ar.sales.items  <-- inspect input data file
                                - will not work, because no LineFeeds

 #5b. uvhd data1/ar.sales.items r64 <-- inspect input data file
                                     - use uvhd to inspect data w/o LineFeeds

 #6. jar410.ksh    <-- execute jcl/script 1st demo for SORT/INCLUDEs
     ==========        (converted to uvsort/selects)

 #7. vi data1/ar.sales.items1 <-- inspect output uvsort/selects (items1 vs items)
     =======================   - works if uvsort filo1 typ=RSF changed to typ=RST

 #7b. uvhd data1/ar.sales.items1 r64  <-- use uvhd if vi does not work
      =============================

Notes re uvsort record selections

1. There are 20 records in the input file & there should be 2 records in the output file, see listings on the next page.
   
   
2. You should be able to verify the selections by comparing the SORT/INCLUDEs listed on page 'U4' (or the uvsort/selects on page 'U5') to the 20 input records listed below.
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

U7. JCL conversion Mainframe SORT to uvsort

input data for uvsort/select demos

 #4. cdd   <-- alias for cd $RUNDATA (RUNDATA=/home/testdata1,testdata2,etc)
             - instructions repeated from page above

 #5c. cat data1/ar.sales.items  <-- display input file
      =======================

          1         2         3         4         5         6
 1234567890123456789012345678901234567890123456789012345678901234
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 990802 IN111001  HAM001  000020 0000101 000002020
 130140    21 000802 IN111001  SCR012  000021 0001001 000021021
 130140    21 010802 IN111001  CHR001  000022 0010001 000220022
 139923    35 980802 IN111002  TAB013  000023 0000201 000004623
 139923    35 990807 IN111002  VAR200  000024 0002001 000048024
 150825    44 010804 IN1122    HAM001  000025 0020001 000500025
 150825    44 020804 IN1122    HAX129  000026 0000301 000007826
 201120    44 030807 CR5234    BBQ001  000027 0003001 000081027
 223240    65 010816 CR955     HAM001  000028 0030001 000840028
 223240    65 020816 IN441     BBQ001  000029 0000101 000002929
 308685    21 020812 IN6605    SAW051  00001p 0000101 00000101p
 308685    21 030812 IN6605    BIT075  00001q 0001001 00001101q
 308685    21 040812 CR8835    TAB013  00001r 0010001 00012001r
 315512    44 040805 IN2251    HAM001  00001s 0000201 00000261s
 315512    44 050805 IN2251    SAW051  00001t 0002001 00002801t
 315512    44 060805 IN2255    BIT075  00001u 0020001 00030001u
 400002    85 990812 CR245     HAX129  00001v 0000101 00000161v
 406082    35 010815 IN33001   BBQ001  00001w 0001001 00001701w
 406082    35 020815 IN33001   TAB013  00001x 0010001 00018001x
 406082    65 030816 IN441     VAR200  00001y 0000101 00000191y

output data from jar410.ksh (1st of 8 demos)

 #6. jar410.ksh  <-- execute 1st demo SORT/INCLUDEs uvsort/selects
     ==========

 #7. cat data1/ar.sales.items1  <-- display selected records
     ========================

          1         2         3         4         5         6
 1234567890123456789012345678901234567890123456789012345678901234
 cust#   slmn YYMMDD invoice  product    qty   price    amount
 ================================================================
 130140    21 990802 IN111001  HAM001  000020 0000101 000002020
 315512    44 050805 IN2251    SAW051  00001t 0002001 00002801t

• only 2 records selected by jar410.ksh
• see count results for all 8 demo jobs on the page below

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

U8. JCL conversion Mainframe SORT to uvsort

running other SORT/INCLUDE demos

1. You can run any of the demo SORT/INCLUDE uvsort/selects listed on page 'U1' But better to use the next section 'V0' - 'V9' as your guide, where you can see the original SORT/INCLUDEs & converted uvsort/selects.
   
   
2. Note that only jar441.ksh required no manual changes after automatic conversion. Auto conversion is from jcl0 --> jcl1 --> jcl2 --> jcl3 but then we copied all jcl3/* --> jcls, which would overwrite any manual changes that had been made in jcls/...
   
   
3. For the above reason I stored the manually corrected scripts (jar420-jar444) in subdir sf/... and since sf/ is in the search PATH ahead of jcls/... you will execute the modified scripts when you enter just the name:
   
   

      jar420.ksh   <-- executes $RUNLIBS/sf/jar420.ksh,
      ==========     - because sf/ precedes jcls/ in the profile $PATH

executing uncorrected vs corrected scripts

4. You can run both uncorrected & corrected scripts as follows (assuming you are in $HOME/testlibs/ with both sf/ & jcls/ below you).
   
   

      jcls/jar420.ksh  <-- run un-corrected script
      ===============

      sf/jar420.ksh    <-- run corrected script
      =============
      jar420.ksh       <-- also runs corrected script
      ==========         - since sf/ in $PATH ahead of jcls/

5. For un-corrected scripts in jcls/... you should get following results: jar410=02, jar420=08, jar430=09, jar440=13,
   
   
6. For corrected scripts in sf/... you should get following results: jar410=02, jar420=03, jar430=05, jar440=04, jar441=04, jar442=04, jar443=04, jar444=04.
   
   
7. Note, there are NO scripts jar441.ksh,jar442.ksh,jar443.ksh,jar444.ksh in jcls/... because there are NO JCLs JAR441.JCL - JAR444.JCL in jcl0/...
   
   
8. jar441.ksh - jar444.ksh are manually created versions of jar440.ksh to illustrate alternate solutions for manually corrected SORT/INCLUDEs with multiple mixed AND/ORs in ((parens)). See listings on pages 'V5' - 'V9'.
   
   
9. Of course jar440.ksh - jar445.ksh all get same results (4 records) because they are alternate solutions of the same sselections.
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

V0. JCL conversion Mainframe SORT to uvsort

SORT INCLUDE/OMIT converted to uvsort sel/del

The JCL converter converts mainframe SORT steps to the uvsort equivalent. Any 'INCLUDE/OMIT's are converted to 'sel/del's. Complex SORTs with multiple mixed INCLUDEs & OMITs may require manual changes and a warning is given.

Very complex SORT INCLUDE/OMIT's with multiple mixed AND/OR's might be better rewritten in uvcopy, which has the power to do whatever kind of data manipulation and record selection desired. For example, uvcopy has table load & lookup instructions that can dramatically shorten voluminous selection codes and simplify maintenace.

We will present several examples of SORT INCLUDE/OMIT AND/OR conversions - 1st requiring no change, some requiring changes,& some illustrating splitting the record selections out to a 2nd step (rewritten in uvcopy).

This section assumes you already understand the JCL/script & SORT/uvsort conversions and are testing the JCL/scripts & may need to modify some of the script/uvsort/sel/del's converted from the original SORT/INCLUDE/AND/OR's. To understand the JCL/script conversions see the prior section 'U1' - U8.

summary of SORT INCLUDE AND/OR examples

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

V1. JCL/SORT/INCLUDE conversions to script/uvsort/select

The following SORT/uvsort conversion examples will show only the relevant lines of INCLUDEs & converted selects - so we can focus on any corrections that may be required for complex INCLUDEs with multiple mixed AND/ORs. See all lines of the 1st sample JCL/script SORT/uvsort on pages 'U4' & 'U5'.

JAR410.JCL - SORT INCLUDE AND/OR's requiring NO changes

 //JAR410   JOB  'test SORT INCLUDE AND/OR'
 //STEP1    EXEC PGM=SORT
 //SORTIN   DD DSN=AR.SALES.ITEMS,DISP=SHR
 //SORTOUT  DD DSN=AR.SALES.ITEMS1,DISP=NEW,LRECL=64
 //SYSIN    DD *
 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman & product
 INCLUDE COND=(11,2,CH,EQ,C'21',AND,31,6,CH,EQ,C'HAM001',OR,
                11,2,CH,EQ,C'44',AND,31,6,CH,EQ,C'SAW051')
 /*

jar410.ksh - script/uvsort equivalent selects

 exportfile  SORTIN data1/ar.sales.items
 exportfile  SORTOUT data1/ar.sales.items1
 exportfile SYSIN $JTMP/${JSTEP}_${PROGID}_SYSIN
 cat > $SYSIN <</*
 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman $ product
 INCLUDE COND=(11,2,CH,EQ,C'21',AND,31,6,CH,EQ,C'HAM001',OR,
                11,2,CH,EQ,C'44',AND,31,6,CH,EQ,C'SAW051')
 /*
 #3----------------------------------------------------------------------
 uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 ,keys=(10,2,c,a,30,6,c,a)\
 ,sel1=10(2c):21,sel2=30(6c):HAM001,sel1=10(2c):44,sel2=30(6c):SAW051"
 #4----------------------------------------------------------------------

Changes for INCLUDEs with mixed AND/ORs

1. No changes required for INCLUDE/OMITs without mixed AND/ORs.
   
   
2. No changes required for simpler INCLUDE/OMITs with mixed AND/ORs (jar410 above has mixed AND/ORs but requires no changes).
   
   
3. An #ERR/warning is generated when changes are required (see jar420 below, which does require a change).
   
   
4. You can execute any of these 8 demo jobs (jar410.ksh - jar444.ksh) as documented on page 'U6' & you should get the record counts listed on page 'U8'.
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

V2. JCL/SORT/INCLUDE conversions to script/uvsort/select

JAR420.JCL - SORT INCLUDE AND/OR's requiring 1 change

 //JAR420   JOB  'test SORT INCLUDE AND/OR'
 //STEP1    EXEC PGM=SORT
 //SORTIN   DD DSN=AR.SALES.ITEMS,DISP=SHR
 //SORTOUT  DD DSN=AR.SALES.ITEMS2,DISP=NEW,LRECL=64
 //SYSIN    DD *
 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman & product
 INCLUDE COND=((11,2,CH,EQ,C'21',OR,11,2,CH,EQ,C'44'),
           AND,(31,6,CH,EQ,C'HAM001'))
 /*

jar420.ksh - script/uvsort/select BEFORE 1 change

 exportfile  SORTIN data1/ar.sales.items
 exportfile  SORTOUT data1/ar.sales.items2
 exportfile SYSIN $JTMP/${JSTEP}_${PROGID}_SYSIN
 cat > $SYSIN <</*
 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman $ product
 INCLUDE COND=((11,2,CH,EQ,C'21',OR,11,2,CH,EQ,C'44'),
           AND,(31,6,CH,EQ,C'HAM001'))
 /*
 #ERR! OR/AND mixed, may need fix, see www.uvsoftware.ca/uvsort.htm#V0
 #3----------------------------------------------------------------------
 uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 ,keys=(10,2,c,a,30,6,c,a)\
 ,sel1=10(2c):21,sel1=10(2c):44,sel2=30(6c):HAM001"
 #4----------------------------------------------------------------------

jar420.ksh - uvsort/select AFTER 1 change

 #-----------------------------------------------------------------------
 # original above, corrected below for mixed OR/AND problem
 # (JCL converter does not respect multiple parens in INCLUDE COND=((..etc..))
 uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RST,rcs=46\
 ,keys=(10,2,c,a,30,6,c,a)\
 ,sel1=10(2c):21,sel2=30(6c):HAM001,sel1=10(2c):44,sel2=30(6c):HAM001"
 #---------------^^^^^^^^^^^^^^^^^^^----------------------------------

((parens)) not considered in mixed AND/ORs

1. 'OR' causes uvsort to reset to sel1, 'AND' causes increment sel2,sel3,etc.
   
   
2. So the OR & 2nd AND convert OK, but the AND must be re-coded with the 1st CONDition. I have underlined the inserted AND (sel2)
   
   
3. For these test/demos I changed uvsort fileo1=$SORTOUT,typ=RSF to 'typ=RST', which inserts LineFeeds in output records for easier inspection with vi/cat.
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

V3. JCL/SORT/INCLUDE conversions to script/uvsort/select

JAR430.JCL - SORT INCLUDE AND/OR's requiring a FEW changes

 //JAR430   JOB  'test SORT INCLUDE AND/OR'
 //STEP1    EXEC PGM=SORT
 //SORTIN   DD DSN=AR.SALES.ITEMS,DISP=SHR
 //SORTOUT  DD DSN=AR.SALES.ITEMS3,DISP=NEW,LRECL=64
 //SYSIN    DD *
 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman & product
 INCLUDE COND=((11,2,CH,EQ,C'21',OR,11,2,CH,EQ,C'44'),
          AND,(31,6,CH,EQ,C'HAM001',OR,31,6,CH,EQ,C'SAW051'))
 /*

jar430.ksh - script/uvsort/select BEFORE a few changes

 exportfile  SORTIN data1/ar.sales.items
 exportfile  SORTOUT data1/ar.sales.items3
 exportfile SYSIN $JTMP/${JSTEP}_${PROGID}_SYSIN
 cat > $SYSIN <</*
 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman $ product
 INCLUDE COND=((11,2,CH,EQ,C'21',OR,11,2,CH,EQ,C'44'),
          AND,(31,6,CH,EQ,C'HAM001',OR,31,6,CH,EQ,C'SAW051'))
 /*
 #ERR! AND/OR mixed, may need fix, see www.uvsoftware.ca/uvsort.htm#V0
 #3----------------------------------------------------------------------
 uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 ,keys=(10,2,c,a,30,6,c,a)\
 ,sel1=10(2c):21,sel1=10(2c):44,sel2=30(6c):HAM001,sel1=30(6c):SAW051"
 #4----------------------------------------------------------------------

jar430.ksh - script/uvsort/select AFTER a few changes

 #-----------------------------------------------------------------------
 # original above, corrected below for mixed OR/AND problem
 # (JCL converter does not respect multiple parens in INCLUDE COND=((..etc..))
 uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RST,rcs=64\
 ,keys=(10,2,c,a,30,6,c,a)\
 ,sel1=10(2c):21,sel2=30(6c):HAM001,sel1=10(2c):21,sel2=30(6c):SAW051\
 ,sel1=10(2c):44,sel2=30(6c):HAM001,sel1=10(2c):44,sel2=30(6c):SAW051"
 # slsmn '21' (1st sel line) must have sel2 for both HAM001 & SAW051
 # slsmn '44' (2nd sel line) must have sel2 for both HAM001 & SAW051
 # For long complex INCLUDE/OMITs with multiple AND/OR's
 # - recommend re-write in uvcopy, using table load/lookup if many codes
 # - see sf/jar441.ksh & pf/jar441.uvc, sf/jar442.ksh & pf/jar442.uvc, etc
 #4----------------------------------------------------------------------

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

V4. JCL/SORT/INCLUDE conversions to script/uvsort/select

JAR440.JCL - SORT INCLUDE AND/OR's requiring MANY changes

 //JAR440   JOB  'test SORT INCLUDE AND/OR'
 //STEP1    EXEC PGM=SORT
 //SORTIN   DD DSN=AR.SALES.ITEMS,DISP=SHR
 //SORTOUT  DD DSN=AR.SALES.ITEMS4,DISP=NEW,LRECL=64
 //SYSIN    DD *
 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman & product
 INCLUDE COND=((11,2,CH,EQ,C'21',OR,11,2,CH,EQ,C'44'),
          AND,(31,6,CH,EQ,C'BBQ001',OR,31,6,CH,EQ,C'BIT075',
             OR,31,6,CH,EQ,C'HAM051',OR,31,6,CH,EQ,C'HAX129',
             OR,31,6,CH,EQ,C'TAB013',OR,31,6,CH,EQ,C'VAR200'),
           AND,(39,6,ZD,GT,19))
 /*

jar440.ksh - script/uvsort/select BEFORE many changes

 exportfile  SORTIN data1/ar.sales.items
 exportfile  SORTOUT data1/ar.sales.items4
 exportfile SYSIN $JTMP/${JSTEP}_${PROGID}_SYSIN
 cat > $SYSIN <</*
 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman $ product
 INCLUDE COND=((11,2,CH,EQ,C'21',OR,11,2,CH,EQ,C'44'),
          AND,(31,6,CH,EQ,C'BBQ001',OR,31,6,CH,EQ,C'BIT075',
             OR,31,6,CH,EQ,C'HAM051',OR,31,6,CH,EQ,C'HAX129',
             OR,31,6,CH,EQ,C'TAB013',OR,31,6,CH,EQ,C'VAR200'),
           AND,(39,6,ZD,GT,19))
 /*
 #ERR! OR/AND mixed, may need fix, see www.uvsoftware.ca/uvsort.htm#V0
 #3----------------------------------------------------------------------
 uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RSF,rcs=64\
 ,keys=(10,2,c,a,30,6,c,a)\
 ,sel1=10(2c):21,sel1=10(2c):44\
 ,sel2=30(6c):BBQ001,sel1=30(6c):BIT075,sel1=30(6c):HAM051\
 ,sel1=30(6c):HAX129,sel1=30(6c):TAB013,sel1=30(6c):VAR200\
 ,sel2=38(6z)>19"
 #4----------------------------------------------------------------------

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

jar440.ksh - uvsort/select AFTER many changes

 #-----------------------------------------------------------------------
 # original above, corrected below, for mixed OR/AND problem
 uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT,typ=RST,rcs=64\
 ,keys=(10,2,c,a,30,6,c,a)\
 ,sel1=10(2c):21,sel2=30(6c):BBQ001,sel3=38(6z)>19
 ,sel1=10(2c):21,sel2=30(6c):BIT075,sel3=38(6z)>19
 ,sel1=10(2c):21,sel2=30(6c):HAM001,sel3=38(6z)>19
 ,sel1=10(2c):21,sel2=30(6c):HAX129,sel3=38(6z)>19
 ,sel1=10(2c):21,sel2=30(6c):TAB013,sel3=38(6z)>19
 ,sel1=10(2c):21,sel2=30(6c):VAR200,sel3=38(6z)>19
 ,sel1=10(2c):44,sel2=30(6c):BBQ001,sel3=38(6z)>19
 ,sel1=10(2c):44,sel2=30(6c):BIT075,sel3=38(6z)>19
 ,sel1=10(2c):44,sel2=30(6c):HAM001,sel3=38(6z)>19
 ,sel1=10(2c):44,sel2=30(6c):HAX129,sel3=38(6z)>19
 ,sel1=10(2c):44,sel2=30(6c):TAB013,sel3=38(6z)>19
 ,sel1=10(2c):44,sel2=30(6c):VAR200,sel3=38(6z)>19"
 # Must add many 'sel's since converter did not respect (parens) & many AND/OR's
 # - each salesman selecting 6 products selling more than 19 units
 # For long complex INCLUDE/OMITs with multiple AND/OR's
 # - recommend re-write in uvcopy, using table load/lookup if many codes
 # - see sf/jar441.ksh & pf/jar441.uvc, sf/jar442.ksh & pf/jar442.uvc, etc
 #4----------------------------------------------------------------------

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

V5. JCL/SORT/INCLUDE conversions to script/uvsort/select

jar441.ksh - jar400 alt, step2 call uvcopy cmc record selects

 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman $ product
 INCLUDE COND=((11,2,CH,EQ,C'21',OR,11,2,CH,EQ,C'44'),
          AND,(31,6,CH,EQ,C'BBQ001',OR,31,6,CH,EQ,C'BIT075',
             OR,31,6,CH,EQ,C'HAM051',OR,31,6,CH,EQ,C'HAX129',
             OR,31,6,CH,EQ,C'TAB013',OR,31,6,CH,EQ,C'VAR200'),
           AND,(39,6,ZD,GT,19))
 /*
 #3----------------------------------------------------------------------
 uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT1,typ=RST,rcs=64\
 ,keys=(10,2,c,a,30,6,c,a)"
 #4----------------------------------------------------------------------
 # -- non-relevant lines omitted --
 #1======================= begin step#S0012 uvcopy ========================
 S0012=A
 #Note - uvcopy step inserted manually to perform record selection after uvsort
 # - because original mainframe SORT had mixed AND/OR's with multiple (parens)
 # - requiring too much corrective code in uvsort, can recode in uvcopy shorter
 # - uvcopy parameter file stored in $RUNLIBS/pf/... & called in this 2nd step
 stepctl51;
 goto
 export JSTEP=S0012; ((XSTEP+=1)); SCC=0; LCC=0; alias goto="";
 logmsg2 "******** Begin Step $JSTEP uvcopy (#$XSTEP) ********"
 export PROGID=uvcopy
 exportfile  SORTOUT1 tmp/ar.sales.items4
 exportfile  SORTOUT data1/ar.sales.items4
 #3----------------------------------------------------------------------
 uvcopy $RUNLIBS/pf/jar441.uvc,fili1=$SORTOUT1,filo1=$SORTOUT
 #4----------------------------------------------------------------------

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

jar441.uvc - 2nd step uvcopy, cmc/skp for each value

 # jar441.uvc - uvcopy job to perform record selection following uvsort
 # - separate parameter file called from the JCL/script after uvsort
 # - see script sf/jar441.ksh that calls this uvcopy job
 # sf/jar441.ksh & pf/jar441.uvc replace auto-gen jcls/jar440.ksh
 # - from jcl0/JAR440.JCL which had mixed AND/OR's with multiple parens
 fili1=${SORTOUT1},typ=RST,rcs=64
 filo1=${SORTOUT},typ=RST,rcs=64    'typ=RST' inserts LineFeeds, allows 'vi'
 @run
        opn    all                   open files
 man20  get    fili1,a0              get record
        skp>   man90                 (cc set > at EOF)
 #
        cmn    a38(6),19             sale units > 19 ?
        skp<=  man20
 #
        cmc    a10(2),'21'           salesman 21 or 44 ?
        skp=   man30
        cmc    a10(2),'44'
        skp!   man20
 #
 man30  cmc    a30(6),'BBQ001'       any 1 of 6 products ?
        skp=   man60
        cmc    a30(6),'BIT075'
        skp=   man60
        cmc    a30(6),'HAM001'
        skp=   man60
        cmc    a30(6),'HAX129'
        skp=   man60
        cmc    a30(6),'TAB013'
        skp=   man60
        cmc    a30(6),'VAR200'
        skp!   man20
 #
 man60  put    filo1,a0              write selected record
        skp    man20                 return to get next
 man90  cls    all                   close files
        eoj                          end job

Notes re uvcopy instructions

1. See all uvcopy instructions documented at uvcopy3.htm.
   
   
2. This solution uses 'cmn' (CoMpare Numeric), 'cmc' (CoMpare Character), & 'skp' (SKiP depending on condition =!<>)
   
   
3. Compare this to following shorter solutions jar442.uvc & jar443.uvc.
   
   
4. jar442.uvc uses 'tst' which compares multiple values with 1 instruction to reduce the number of instructions required.
   
   
5. jar443.uvc uses 'lod' & 'lok' (table load & table lookup) to reduce instructions & also to improve maintenance (easier to update tables).
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

V6. JCL/SORT/INCLUDE conversions to script/uvsort/select

jar442.ksh - jar400 alt, step2 call uvcopy tst record selects

 exportfile  SORTIN data1/ar.sales.items
 exportfile  SORTOUT1 tmp/ar.sales.items4
 exportfile SYSIN $JTMP/${JSTEP}_${PROGID}_SYSIN
 cat > $SYSIN <</*
 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman $ product
 INCLUDE COND=((11,2,CH,EQ,C'21',OR,11,2,CH,EQ,C'44'),
          AND,(31,6,CH,EQ,C'BBQ001',OR,31,6,CH,EQ,C'BIT075',
             OR,31,6,CH,EQ,C'HAM051',OR,31,6,CH,EQ,C'HAX129',
             OR,31,6,CH,EQ,C'TAB013',OR,31,6,CH,EQ,C'VAR200'),
           AND,(39,6,ZD,GT,19))
 /*
 #3----------------------------------------------------------------------
 uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT1,typ=RST,rcs=64\
 ,keys=(10,2,c,a,30,6,c,a)"
 #4----------------------------------------------------------------------
 # -- non-relevant lines omitted --
 #1======================= begin step#S0012 uvcopy ========================
 S0012=A
 #Note - uvcopy step inserted manually to perform record selection after uvsort
 # - sf/jar442.ksh same as sf/jar441, but calls pf/jar442.uvc (vs pf/jar441.uvc)
 # - pf/jar442 performs same selections as jar441, but in less code using 'tst'
 #
 # -- non-relevant lines omitted --
 export PROGID=uvcopy
 exportfile  SORTOUT1 tmp/ar.sales.items4
 exportfile  SORTOUT data1/ar.sales.items4
 #3----------------------------------------------------------------------
 uvcopy $RUNLIBS/pf/jar442.uvc,fili1=$SORTOUT1,filo1=$SORTOUT
 #4----------------------------------------------------------------------

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

jar442.uvc - uvcopy test multi-values with 1 'tst' instrn

 # jar442.uvc - uvcopy job to perform record selection following uvsort
 # - separate parameter file called from the JCL/script after uvsort
 # - see script sf/jar442.ksh that calls this uvcopy job
 # sf/jar442.ksh & pf/jar442.uvc replace auto-gen jcls/jar440.ksh
 # - from jcl0/JAR440.JCL which had mixed AND/OR's with multiple parens
 # pf/jar442.uvc same function as pf/jar441.uvc
 # - and demos use of 'tst' instrn to save many lines of code
 fili1=${SORTOUT1},typ=RST,rcs=64
 filo1=${SORTOUT},typ=RST,rcs=64    'typ=RST' inserts LineFeeds, allows 'vi'
 @run
        opn    all                   open files
 man20  get    fili1,a0              get record
        skp>   man90                 (cc set > at EOF)
 #
        cmn    a38(6),19             sale units > 19 ?
        skp<=  man20
 #
        tstl2  a10(2),'2144'         salesman 21 or 44 ?
        skp!   man20
 #
 man30  tstl6  a30(6),'BBQ001BIT075HAM001HAX129TAB013VAR200'
        skp!   man20
 #
 man60  put    filo1,a0              write selected record
        skp    man20                 return to get next
 man90  cls    all                   close files
        eoj                          end job

Notes re uvcopy instructions

1. jar442.uvc uses 'tst' which compares multiple values with 1 instruction to reduce the number of instructions required.
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

V7. JCL/SORT/INCLUDE conversions to script/uvsort/select

jar443.ksh - jar400 alt, step2 call uvcopy lod/lok selects

 exportfile  SORTIN data1/ar.sales.items
 exportfile  SORTOUT1 tmp/ar.sales.items4
 exportfile SYSIN $JTMP/${JSTEP}_${PROGID}_SYSIN
 cat > $SYSIN <</*
 SORT FIELDS=(11,2,CH,A,31,6,CH,A)   Sort on salesman $ product
 INCLUDE COND=((11,2,CH,EQ,C'21',OR,11,2,CH,EQ,C'44'),
          AND,(31,6,CH,EQ,C'BBQ001',OR,31,6,CH,EQ,C'BIT075',
             OR,31,6,CH,EQ,C'HAM051',OR,31,6,CH,EQ,C'HAX129',
             OR,31,6,CH,EQ,C'TAB013',OR,31,6,CH,EQ,C'VAR200'),
           AND,(39,6,ZD,GT,19))
 /*
 #3----------------------------------------------------------------------
 uvsort "fili1=$SORTIN,typ=RSF,rcs=64,filo1=$SORTOUT1,typ=RST,rcs=64\
 ,keys=(10,2,c,a,30,6,c,a)"
 #4----------------------------------------------------------------------
 # -- non-relevant lines omitted --
 #1======================= begin step#S0012 uvcopy ========================
 #Note - uvcopy step inserted manually to perform record selection after uvsort
 # - sf/jar443.ksh same as sf/jar442, but calls pf/jar443.uvc (vs pf/jar442.uvc)
 # - pf/jar443 performs same selections as jar441, but in less code using lod/lok
 # - 'lod' loads table for lookup by 'lok', saves code & easier to maintain
 #
 # -- non-relevant lines omitted --
 export PROGID=uvcopy
 exportfile  SORTOUT1 tmp/ar.sales.items4
 exportfile  SORTOUT data1/ar.sales.items4
 #3----------------------------------------------------------------------
 uvcopy $RUNLIBS/pf/jar443.uvc,fili1=$SORTOUT1,filo1=$SORTOUT
 #4----------------------------------------------------------------------

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

jar443.uvc - uvcopy table load/lookup

 # jar443.uvc - uvcopy job to perform record selection following uvsort
 # - separate parameter file called from the JCL/script after uvsort
 # - see script sf/jar443.ksh that calls this uvcopy job
 # sf/jar443.ksh & pf/jar443.uvc replace auto-gen jcls/jar440.ksh
 # - from jcl0/JAR440.JCL which had mixed AND/OR's with multiple parens
 # pf/jar443.uvc same function as pf/jar442.uvc
 # - and demos use of lod/lok to load tables & lookup tables
 # - saves coding & for easier maintenance
 # - great when you have dozens or hundreds of codes to select
 fili1=${SORTOUT1},typ=RST,rcs=64
 filo1=${SORTOUT},typ=RST,rcs=64    'typ=RST' inserts LineFeeds, allows 'vi'
 lod=c0(6)      <-- load table of 6 byte product codes for 'lok'-up
 BBQ001
 BIT075
 HAM001
 HAX129
 TAB013
 VAR200
 ~~~~~~
 #Note - following #commented out 3 lines same as 8 lines above
 #     - save code by loading all 6 x 6 byte entries as 1 36 byte entry
 #     - but 'lok'up using 6 byte entry length
 # lod=c0(36)
 # BBQ001BIT075HAM001HAX129TAB013VAR200
 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 @run
        opn    all                   open files
 man20  get    fili1,a0              get record
        skp>   man90                 (cc set > at EOF)
 #
        cmn    a38(6),19             sale units > 19 ?
        skp<=  man20
 #
        tstl2  a10(2),'2144'         salesman 21 or 44 ?
        skp!   man20
 #
 man30  lok    c0(6),c0(6),a30(6)   lookup table of product codes
        skp!   man20
 #
 man60  put    filo1,a0              write selected record
        skp    man20                 return to get next
 man90  cls    all                   close files
        eoj                          end job

1. jar443.uvc uses 'lod' & 'lok' (table load & table lookup) to reduce instructions & also to improve maintenance (easier to update tables).
   
   
2. Could easily add dozens or hundreds of more values to be selected. The solution above uses 1 line for each table value (for clarity). Note 'lod=c0(6)' loads 1 value on each line (for lookup by the 'lok').
   
   
3. Also see the next solution jar444.uvc which 'lod's multiple values on each line to reduce lines required. Note 'lod=c0(36)' loads 6 values on each line, but uses same lookup 'lok c0(6),c0(6),a30(6)'. Op1 defines the table entry length & op2 the compare length to op3. Op1 & op2 are same here 'c0(6)', but this allows multi column tables.
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

V8. JCL/SORT/INCLUDE conversions to script/uvsort/select

jar444.ksh - jar400 alt, 1 step call uvcopy sort+select

 #1======================= begin step#S0012 uvcopy ========================
 export PROGID=uvcopy
 exportfile  SORTIN  data1/ar.sales.items
 exportfile  SORTOUT data1/ar.sales.items4
 #3----------------------------------------------------------------------
 uvcopy $RUNLIBS/pf/jar444.uvc,fili1=$SORTIN,filo1=$SORTOUT
 #4----------------------------------------------------------------------

jar444.uvc - uvcopy sort & select combined

 # jar444.uvc - uvcopy job to perform record selection following uvsort
 # - separate parameter file called from the JCL/script after uvsort
 # - see script sf/jar444.ksh that calls this uvcopy job
 # sf/jar444.ksh & pf/jar444.uvc replace auto-gen jcls/jar440.ksh
 # - from jcl0/JAR440.JCL which had mixed AND/OR's with multiple parens
 # pf/jar444.uvc performs both the sort & the record selection
 # - demos how to code a sort within a uvcopy job
 # sf/jar444.ksh now only 1 step to call this uvcopy job (remove the uvsort)
 fili1=${SORTOUT1},typ=RST,rcs=64
 filo1=${SORTOUT},typ=RST,rcs=64    'typ=RST' inserts LineFeeds, allows 'vi'
 lod=c0(36)
 BBQ001BIT075HAM001HAX129TAB013VAR200
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 # - above saves code, by 'lod'ing all 6 x 6 byte entries as 1 36 byte entry
 # - but 'lok'up below, using 6 byte entry length
 @run
        opn    all                   open files
        sxo    64,'10(2),30(6)'  <-- open sort, recsize 64, keys slsmn&product
 #
 man20  get    fili1,a0              get record
        skp>   man40                 at EOF, go execute sort
        cmn    a38(6),19             sale units > 19 ?
        skp<=  man20
        tstl2  a10(2),'2144'         salesman 21 or 44 ?
        skp!   man20
        lok    c0(6),c0(6),a30(6)    lookup table of product codes
        skp!   man20
        sxp    a0(64)            <-- put to the sort
        skp    man20                 return to get next record
 #
 man40  sxs                      <-- execute the sort
 #
 man50  sxg    b0(64)            <-- get record from sort
        skp>   man90                 (cc set > at end sorted recs)
        put    filo1,b0              write selected record
        skp    man50                 return to get next from sort
 #
 man90  cls    all                   close files
        eoj                          end job

1. The sort is performed by sxo,sxp,sxs,& sxg (see '<--' markers).
   
   
2. This job has same record selection (table load/lookup) as prior jar443.uvc) but uses 'lod=c0(36)' to load 6 values per line vs 1 'lod=c0(6)'.
   
   

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

V9. JCL/SORT/INCLUDE conversions to script/uvsort/select

jar445.ksh - same as jar443.ksh, but calls jar445.uvc

• jar445.ksh not shown here, see page 'V7'

jar445.uvc - jar443.ksh alt, read table from external file

 # jar445.uvc - uvcopy job to perform record selection following uvsort
 # - separate parameter file called from the JCL/script after uvsort
 # - see script sf/jar445.ksh that calls this uvcopy job pf/jar445.uvc
 # - demo load/lookup tables to save coding & enable easier maintenance
 # - great when you have dozens or hundreds of codes to select
 # jar445.uvc - alternate coding for jar443.uvc
 # jar443.uvc - demo'd lookup tables using 'lod' load values coded in job
 # jar445.uvc - same demo, but using 'rtb' to Read TaBle from external file
 fili1=${SORTOUT1},typ=RST,rcs=64
 filo1=${SORTOUT},typ=RST,rcs=64
 fili2=data1/ar.product.select,typ=LST,rcs=80  #<-- products to be selected
 # - fili2 read into pf/jar445.uvc by 'rtbc1' (after 'opn' below)
 # BBQ001   BAR-B-Q
 # BIT075   DRILL BIT       <-- sample records shown here as #comments
 # HAM001   CLAW HAMMER
 # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 #      rtbc1  fili2,c0(80),c0(40)  <-- Read TaBle of product codes
 # - reads 80 bytes max, stores 1st 40 bytes in table entries
 #      lok    c0(40),c0(6),a30(6)  <-- see LOoKup instrn below
 # - looks up table of 40 byte entries of table in area 'c'
 # - compares 1st 6 bytes c0(6) to bytes 30-35 a30(6) of input record area 'a'
 @run
        opn    all                      open files
        rtbc1  fili2,c0(80),c0(40)  <-- Read TaBle of product codes
 #
 man20  get    fili1,a0                 get record
        skp>   man90                    (cc set > at EOF)
        cmn    a38(6),19                sale units > 19 ?
        skp<=  man20
        tstl2  a10(2),'2144'            salesman 21 or 44 ?
        skp!   man20
 #
 man30  lok    c0(40),c0(6),a30(6)  <-- lookup table of product codes
        skp!   man20
 #
 man60  put    filo1,a0                 write selected record
        skp    man20                    return to get next
 #
 man90  cls    all                      close files
        eoj                             end job

Goto:   Begin this document ,  End this document ,  UVSI Home-Page

Visitor Counters for ThisYear and LastYear