UVdemos.doc - Contents by Part#


Part_0 Introduction, Overview, Intended Audience
- intended for any Unix/Linux user (vs mainframe conversion users)
- demo Vancouver Utility capabilities far beyond unix/linux system utilities
- Pre-Requisites to run the UVdemos tutorials

Part_1 Installing Vancouver utilities, setup uvadm & appsadm admin accounts
- setup user profiles to execute Vancouver Utilities

Part_2 Basic Vancouver Utilities are uvhd,uvsort,uvcp,& uvlist (vs uvcopy)
 Brief descriptions of uvhd,uvsort,uvcp,& uvlist, followed by tutorials
 illustrating how they might help you solve your data problems.
uvhd - investigate problem files, scripts with CRs, high-bit chars, etc
uvcp - translate EBCDIC mainframe files to ASCII & insert LineFeeds.
 uvsort demo - sort with record selection (impossible with unix system sort)
uvlist - list text files on laser printers, options for cpi,lpi,Landscape,Duplex
uvcopy - powerful data manipulation utility, power of assembler w/o complexity

Part_3 SelectJobs - select lines from 1 or all files in a directory, identified by
Multiple patterns & conditions may be on same line or anywhere in the file.
May identify Begin & End group lines to find desired patterns in between.
Like multiple greps in 1 pass, creates reports named from files & patterns.
Demo files provided include US Congress & Canadian Members of Parliament

Part_4 TableJobs - create table summaries of counts & values in text files
- tabled in memory & dumped to an output file at EOF (option to display)
- pre-programmed jobs using the 'tbl' instruction of the uvcopy utility
table3d - scan all files in directory building summary tables of target-words
- identified by preceding key-words (examples: IP#s, userids, passwords)
table2 - alternative for fixed-field files (text,fixed/variable,RDW)
- using byte displacements & lengths to specify fields to be summarized

Part_5 - most useful shell scripts at UV Software
- useful to anybody who works with files & directories on the command line
- can run with demo files provided & check results to expected results listed
- can download uvscripts.zip from the website (if Utilities not installed)
- Count Files,Lines,& KB in 1 or all subdirs of directories
- rename files, removeCRs, alldiff2/3, uvcmpFA1, chmod1/2, dtree
- scripts to list/move/remove files older/newer than x days

Part_6 Vancouver Utility 'help' menus - handy reminders of the most useful Utilities
- just enter 'uvhelp' at your shell prompt & scroll thru the list.
 Use whenever you have a problem that you think may have a Vancouver Utility
 solution, but can't remember the exact command.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

UVdemos - Contents continued


Part_7 uvcopy & uvfix. 'uvcopy' is the most powerful Vancouver Utility. A can do anything
data manipulation utility - the power of assembler without the complexity. uvcopy
interprets instruction files for pre-programmed jobs such as SelectJobs & TableJObs.
'uvfix1' & 'uvfixA' are the easy way to use the power of uvcopy instructions
without writing uvcopy jobs.
'uvfix1' example - convert .csv file to fixed-fields
'uvfixA' example - translate directory of SQL scripts to lowercase &
shorten maninframe FTP 80 column records back to last non-blank.

Part_7 Expanding $VARIABLES using the uvcopy 'evs' instruction
Demo jobs to illustrate the 'evs' instruction, see '7F1' - 7F5
testevs1 - hard-coded variables expanded ($HOME,$LOGNAME,etc)
testevs2 - prompts you to enter your strings with $Variables
expandevs1 - copy a file expanding $VARIABLEs
- critical converting mainframe JCL unloading DB2 tables
for operation on unix/linux systems.

Part_8 Search/replace pre-programmed jobs
scan1d - scan all files in a directory for 1 pattern
- may qualify with 2 other patterns that may or may not be present
rep1d - search all files in directory, replacing 1 pattern with an alternate
  while copying to a 2nd directory
- may qualify with 2 other patterns that may or may not be present
rep2d - search all files in directory, replacing using a search/replace table
  with unlimited number of search & replacement strings
- may qualify search/replaces with 1 present & 1 absent

Part_9 Demos to illustrate how any site can incorporate the many powerful features
 of the Vancouver Utility JCL conversion scripts into their scripts.
- GDG files, testcc(test step status), joblogs, jobmsgs, Restart, Keywords=Values
- autoload functions in separate library defined by export FPATH=$APPSADM/sfun/...
  (jobset51,jobend51,jobabend51,logmsg1,logmsg1,exportgen0,exportgen1)
- $RUNDATA & $RUNLIBS in profiles allows multi sets of DATA vs Scripts/Programs
  allows different programmers to access their own set of testdata (no onflicts)
  OR allows testing changes to scripts/programs with duplicated set of DATA
- above possible since jobset51 function at begining of scripts does 'cd $RUNDATA'
  and scripts address programs & subscripts relative to $RUNLIBS.
- Run job scripts from anywhere, possible since scripts do 'cd $RUNDATA'
  and all DATA files relative to $RUNDATA (no absolute paths files).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

UVdemos - Contents continued


Part_11 Games written in the "C" program language
- fun way to learn C programming
insult2 - insult generator
- generates insults using, using random numbers to select insult words
  from 3 columns of insult adjectives & nouns.
- 2 insult files provided - insults_Shakespeare & insults_Owen
  & you can create your own insults file
- C source code listing for the insult generator
  interesting program for young programmers learning C
lotto1 - generate random numbers for lottery tickets
- generate specified number of random number in a range
  of any 2 numbers (lower & upper range).
- could be useful generating random numbers for real work applications
 C programs to test limits of 16 bit shorts, 32 bit ints,& 64 bit longs
- test64a, test64b, test64c, testmaxints testio1
- binary integers, BigEnd/LittleEnd binary
- simple C programs to read/write data files

Part_12 Games & Education - uvcopy jobs (some parallels to C programs)

Part_13 Vancouver Utility Big Time Savers
- Vancouver Utilities can save you a lot of time when you need to create
  & modify text files of scripts, programs, documentation, control files, etc.
- Some operations are difficult to do with unix system editors & utilities.
  For example, addressing fields by column# hard to do with vi, etc,
  but easy to do with uvcopy & the many pre-programmed jobs using uvcopy.
- The examples might remind you of similar problems you have encountered.

Part_20 Complex conversions considered too long for presentation earlier.
'10B1' - removing CRs x'0D's from scripts (full explanations vs '2B1' brief)
'10C1' - convert UTF-8 Accents to unaccented equivalents
'10D1' - CanadaMPscsv2txt - convert Canadian MPs from csv to fixed field text
'10E1' - UScongresscsv2txt - convert US Senators/Representatives csv to fixed

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_0 UVdemos.doc - Vancouver Utilities for Unix/Linux users

Introduction & Overview

This document is intended to make you aware of the most useful utilities supplied in the Vancouver Utilities package. You can run various tutorials using pre-programmed jobs with supplied test data files.

We will illustrate Vancouver Utility capabilities that are not possible with the standard unix/linux system utilities & we hope you can see how these capabilities could apply to your particular data processing projects.

These utilities are 'GENERAL PURPOSE' - they can be used with any data-file & can process all files in a directory. For our demos, we will use files such as FTP scripts, SQL scripts,& JCL, but we hope you can realize how these utilities could apply to your files (data, scripts,& programming languages).

We will minimize using JCL as testdata (used in mainframe conversion demos) and maximize the use testdata such as FTP & SQL scripts, which are familiar to most data processing professionals.

The intention here is to give you a short (1 or 2 page) introduction to the various utilities available, and then give you a link to the complete documentation which could be in various other books.

If you have a problem with your files & are wondering how it could be solved with Vancouver Utilities, we invite you to email us, explain the problem, and we will help you solve it (as our time allows).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_1 UVdemos.doc - Vancouver Utilities for Unix/Linux users

Part 1 - Contents


1A1. Vancouver Utility Administrator home directory /home/uvadm
- directories & contents briefly described
1A2.  Install Vancouver Utilities (if not already installed)

1B1. Setup user logins and/or modifying profiles to execute Vancouver Utilities
- user profiles call the common_profile stored in /home/appsadm/env/...
1B2.  .bash_profile for userxx
1B3.  common_profile_abc for VU users
1B4. .vimrc - set options for vim (listing from $UV/env/vimrc)
- Be SURE to copy .vimrc to your homedir, GREATLY ENHANCES vim experience.
- Highlights search words & line of current cursor position
- Restores cursor to last line position on your previous vim of that file.
1B5.  Advantages of stub_profile,& common_profile
1B6. Working-Directory - Best Practices
1B7.  tmp/ subdir in Working-Directory

1C1. setup userxx account to run demos
1C2. copy demo files to your $HOME/demo/...
1C3. dtree of files most relevant to UVdemos

1D1. demo/dat1/... data files for SelectJobs & TableJobs
... nameadrs,CanadaMPs,CanadaProvinces,UScongress,USstates
1D2. demo/dat2/... sales3 demo uvsort, custmas1/2 demo uvcmpFA1
1D3. demo/ftps/... FTP scripts (4 short samples)
1D4. demo/sqls/... SQL scripts (8 short samples)
1D5. demo/sf/... bash scripts (llsx & llsy)
demo/pf/... uvcopy jobs (removeCR_uvcopy)
1D6. demo/jcl2/... JCL (just 1 of 6 samples listed)

1E1. Print UVdemos.doc as a guide to work your way thru the tutorials to leave
your screen free to enter commands & compare results. We provide script
'uvlp13D' to print Duplex at 13 cpi with margins for 3 hole paper.
First export UVLPDEST=-dlaserxx where laserxx is a laser printer near you.
See complete instructions on page '1E1'.

Install Vancouver Utilities

Before you begin these Vancouver Utility UVdemos tutorials, you must install the package as documented at www.uvsoftware.ca/install.htm. Here are important pages:

Page B2
  • setup Vancouver Utilities administrator account
  • commonly /home/uvadm but could be elsewhere & export UV=location
Page B3
  • Download Vancouver Utilities with login & password supplied & unzip
Page B4
  • setup appsadm account to hold the common_profile called by user profiles
  • so not lost when new versions of Vancouver Utilities installed
Page C1
  • compile the Vancouver Utility C programs

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A1. UVdemos - Vancouver Utilities for Unix/Linux users

Vancouver Utilities homedir contents

 /home/uvadm           <-- $UV
 :-----bin               - binaries (uvcopy,uvsort,uvcp,uvhd,uvlist,etc)
 :-----ctl               - control files for various purposes
 :-----dat1              - test data files to demo various Vancouver Utilities`
 :-----doc               - Vancouver Utilities documentation (text)
 :-----dochtml           - documentation in HTML (same as on www.uvsoftware.ca)
 :-----demo            <-- demo files for this TestDemo tutorial
 :     :                 - to be copied to your $HOME/demo/...
 :     :-----dat1        - data files for uvsort,uvcopy,uvcp,etc
 :     :-----jcl0        - mainframe JCL samples (8)
 :     :-----jcl2        - JCLs cleaned up, lower cased
 :     :-----jcl3        - JCLs after conversion to Korn shell scripts
 :     :-----ftps        - SYSIN control cards for various purposes
 :     :-----pf          - Parameter Files for uvcopy
 :     :-----rpts        - reports created by various demos (selects,etc)
 :     :-----sf          - Script Files for bash or ksh
 :     :-----sqls        - SQL scripts for demo table analysis
 :     :-----stats       - outputs from table3d table summaries
 :     :-----tf          - Test Files (similar to dat1/...)
 :     :-----tmp         - temporary files, outputs from demos
 :     :-----tmp1        - may clear tmp files before demos to see only new outputs
 :-----env             <-- profiles for users, PATHs to utilities & other software
 :-----hdr               - hdr files for C compiles
 :-----htmlcode          - merged into dochtml when text converted to HTML
 :-----lib               - libraries for C compiles (subfunctions,DISAM,etc)
 :-----mvstest           - test/demos for MVS JCL/COBOL MainFrame conversions
 :-----mf              <-- small versions of MainFrame testdata files
 :-----pf              <-- Parameter Files for uvcopy & uvqrpg
 :     :-----adm         - administrative jobs
 :     :-----demo        - demo jobs
 :     :-----IBM         - IBM mainframe conversion jobs
 :     :-----util        - utility jobs
 :-----sf              <-- Script Files
 :     :-----adm         - administrative scripts
 :     :-----demo        - demo scripts
 :     :-----IBM         - IBM mainframe conversion scripts
 :     :-----util        - utility scripts
 :-----sfun              - ksh functions used in converted JCL/scripts
 :-----src             <-- Vancouver Utilities C source code
 :-----srcf              - C source for various sub-functions
 :-----tf                - test files for various examples in doc
 :-----tmp               - tmp subdir (test/demo outputs)
 :-----vsetest           - test/demos for VSE JCL/COBOL MainFrame conversions

Note that the uvadm subdirs for 'sf' (Script Files) & 'pf' (uvcopy Parameter Files or uvcopy jobs) are further sub-directoried as shown above.


Note - see https://uvsoftware.ca/install.htm to Install Vancouver Utilities
- see a brief summary on the next page

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A2. UVdemos - Installing Vancouver Utilities

Before you can run the UVdemos tutorials, you must install the Vancouver Utilities (if not already installed). See instructions at uvsoftware.ca/install.htm, but here is a brief summary.

Installing Vancouver Utilities - Summary

  1. Setup user account 'uvadm' to serve as Vancouver Utilities administrator. Usual home directory is /home/uvadm, but could be elsewhere if you change the environmental variable 'export UV=/home/uvadm' in the common_profile.

  2. We have suggested setting up group 'apps' for uvadm & other users who will be working with Vancouver Utilities.

  3. Download the Vancouver Utilities from the UV Software website into the uvadm home directory,& unzip or untar (from archives uvadm.zip or uvadm.tar.gz).

  4. You may need to modify the profiles as required for your site. We recommend setting up user 'appsadm' to store your modified versions of the profiles, so you do not lose them if you install a new version of Vancouver Utilities.

  5. Setup user 'appsadm' & copy /home/uvadm/env/* to /home/appsadm/env Then modify .profiles or .bash_profiles to call the common_profile from /home/appsadm/env/common_profile. See the stub_profile & common_profile listed on the following pages.

  6. The common_profile gives you (the system administrator) 1 place to control program & data-file paths for all users on your system.

  7. You should rename the supplied common_profile_uv to common_profile_xxx where 'xxx' identifies your organization (company ID).

  8. Setting up 'appsadm' (separate from uvadm) saves your customized version of the common_profile from being lost or over-written when you install new versions of 'uvadm'. All stub_profiles (.bash_profile or .profile) should call the comon_profile from /home/appsadm/env/common_profile (not from /home/uvadm/env/common_profile).

  9. The C source utility programs are stored in $UV/src & are compiled into $UV/bin. You may need to recompile the C source programs for some versions of Linux (see uvsoftware.ca/install.htm#C1 - C7).

See complete install & compile instructions at uvsoftware.ca/install.htm.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B1. 'profiles' provided in /home/uvadm/env

'profiles' provided in /home/uvadm/env

 /home/uvadm/env         <-- profiles provided here
 :-----stub_profile_uv     - copy/rename to .profile (ksh) or .bash_profile (bash)
 :                         - defines RUNLIBS/RUNDATA for programmers & operators
 :-----common_profile_uv <-- common profile (called by stub_profile)
 :                         - defines PATH's etc using $RUNLIBS/$RUNDATA
 /home/appsadm/env       <-- setup user 'appsadm' & copy from /home/uvadm/env/*
 :-----stub_profile_ABC    - customize & copy to homedirs .profile or .bash_profile
 :-----common_profile_ABC  - common profile (called by stub_profile)

We recommend setting up an application administrator 'appsadm', copying /home/uvadm/env/* to /home/appsadm/env, and customizing the profiles there depending on the site locations of libraries & data.

Do NOT customize profiles in /home/uvadm/env/... because they would be overwritten when a new version of Vancouver Utilities is installed.

We recommend the concept of 'stub' & 'common' profiles. The shell profile in each user's homedir is a 'stub' that calls 'common_profile' which are stored in /home/appsadm/env/...

  1. The supplied 'stub_profile' is copied to homedirs, renamed as '.bash_profile' for bash shell (Linux) & '.profile' for Korn shell (Unix), and then modified depending on whether the user is a programmer or operator. - defines RUNLIBS as testlibs for programmers OR prodlibs for operators - defines RUNDATA as testdata for programmers OR proddata for operators

  2. 'common_profile' then defines the 'PATH's using $RUNLIBS,$COBDIR,$UV,etc For example: export PATH=$PATH:$RUNLIBS/jcls (converted JCL/scripts). Defines software superdirs (uvadm, COBDIR, ORACLE_BASE, ORACLE_HOME, etc)

  3. '$RUNDATA' determines data-file locations indirectly as follows: $RUNDATA defines the superdir housing all data-files.

Note that stub profiles must call 'common_profile' using '.' (dot execution), which means the 'export's made in the common_profile will still be effective on return to the users profile.

This system is a big advantage for any site with multiple users, it means the sysadmin can update common_profile once in 1 place & those changes are effective for all users.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B2. 'profiles' provided in /home/uvadm/env

stub_profile_uv OR bash_profile_uv

 # bash_profile_uv - bash_profile for Vancouver Utilities (same as stub_profile_uv)
 #                 - stub_profile calls 'common_profile'
 #                 - by Owen Townsend, update Feb 2020
 #
 # bash/stub_profile_ABC - users should copy/rename their version of stub_profile
 #                         store their master copy in /home/appsadm/env/...
 #                         copy to user homedirs, renaming .profile or .bash_profile
 # common_profile_ABC - copy/rename their common_profile to /home/appsadm/env/...
 #                      called by their stub_profiles from $APPSADM/env/...
 #
 # bash_profile & common_profile - distributed in $UV/env/...
 # - copy to $APPSADM/env/... (/home/appsadm/env/...) & modify for your site
 # - do not modify profiles in $UV because new versions of uvadm would overwrite
 # - see stub/common profile listings at uvsoftware.ca/install.htm#A6 & A7
 #
 #           ** define RUNLIBS/RUNDATA/CNVDATA & call common_profile **
 #
 # stub_profile defines RUNLIBS/RUNDATA for common_profile to define PATHs to libs & data
 # --> Modify definitions depending on your situation:
 #
 # 1. Mainframe JCL/COBOL/DATA Migrations - libs/data in homedir for testing/training
 #    export RUNLIBS=$HOME/testlibs1 RUNDATA=$HOME/testdata1 CNVDATA=$HOME/cnvdata1
 #    =============================================================================
 # 2. Mainframe JCL/COBOL/DATA Migrations - conversion/production, separate file systems
 #    export RUNLIBS=/p1/apps/testlibs1 RUNDATA=/p2/apps/testdata1 CNVDATA=/p3/apps/cnvdata1
 #    =============================================================================
 #    - appended digit '1' for future possible alternates testlibs2/testdata2/cnvdata2,etc
 #    - RUNDATA could be defined differently for different programmers for test phase
 #    - see uvsoftware.ca/jclcnv1demo.htm#1B3 RUNLIBS/RUNDATA defines for migrations
 #
 # 3. Vancouver Utilities Demos/Tutorials - libs/data in $HOME/demo, default Feb2020+
 export RUNLIBS=$HOME/demo RUNDATA=$HOME/demo CNVDATA=$HOME/demo
 #==============================================================
 # - may define RUNLIBS/RUNDATA/CNVDATA the same for small projects (not major migrations)
 # - see uvsoftware.ca/uvdemos.htm#1B5 more about RUNLIBS/RUNDATA stub/common profiles
 #
 # 4. Programmer testing/development projects, Example track US Election candidates
 #    export RUNLIBS=$HOME/demo/election1 RUNDATA=$HOME/demo/election1 CNVDATA=$HOME/demo/election1
 #    =============================================================================================
 #    - 1st demo election1/, then election2/, then election3, See uvsoftware.ca/uvdemos.htm#Part_9
 #
 export APPSADM=/home/appsadm #Jan2020 APPSADM def in .profile_bash (for WSL Windows Subsystem on Linux)
 CALLER=$(cat /proc/$PPID/comm)
 echo "Executing--> \$HOME=$HOME/.bash_profile (copied/renamed from \$APPSADM/env/bash_profile_uv)"
 echo " - Vancouver Utilities stub_profile in login homedir, will call common_profile"
 echo " - LOGNAME=$LOGNAME HOME=$HOME PWD=$PWD APPSADM=$APPSADM CALLER=$CALLER"
 echo "Calling--> . $APPSADM/env/common_profile_uv"
 #
 . /home/appsadm/env/common_profile_uv   # common_profile called from /home/appsadm/env/...
 #====================================   # - NOT from /home/uvadm/env/...
 # - must setup appsadm to store common_profile, so not lost when uvadm updated
 # - see more at www.uvsoftware.ca/install.htm#A4
 echo "HOSTNAME=$HOSTNAME LOGNAME=$LOGNAME APPSADM=$APPSADM"
 echo "RUNLIBS=$RUNLIBS RUNDATA=$RUNDATA"
 #
 #       ** misc items that user may need to override common_profile defs **
 # export TERM=linux       # TERM - modify depending on your terminal
 # stty erase '^?'         # erase char - modify depending on your terminal
 # stty intr '^C'          # interrupt ^C, (probably already default ?)
 # export UVLPDEST="-dlp0" # default destination for uvlp(uvlist) scripts
 #                         # change to a printer near you & un-comment
 #
 #                       ** user aliases, etc **
 # alias l='ls -l'         # save keystrokes on very often used commands
 # - see common_profile for several more aliases
 # - add more here depending on user preferences
 #
 #                     ** TEST or PRODuction **
 # export TESTPROD=P000   # P___ for PRODuction
 export TESTPROD=T000     # T___ for TEST
 # - PRODuction profiles TESTPROD=P*, developer TEST profiles TESTPROD=T*
 # - JCL/scripts can test $TESTPROD to control various differences desired
 # - used to determine if programmer 'T'esting or 'P'roduction
 # - bytes 2,3,4 of P/T___ reserved for future use as required
 #   if [[ "$TESTPROD" == P* ]]  <-- test only 1st byte for Test/Prod
 #   if [[ "$TESTPROD" != T* ]]  <-- assume Production if not Test
 #Note - Test/Prod code relevant only to mainframe migration JCL/scripts sites
 #     - migration sites would move this code up prior to calling common_profile
 #       so common_profile could modify PATH,etc depending on Test/Production
 #
 #                   ** Console Logging - optional **
 # - uncomment 9 '##' lines below to activate console logging
 # - must setup subdirs matching $LOGNAME in $LOGDIR/log1/...,log2/...,log3/...
 #   (usually LOGDIR=$APPSADM in common_profile)
 # - subdirs log1,log2,log3 hold logfiles for: current file, month, lastmonth
 # - see details at www.uvsoftware.ca/admjobs.htm#Part_6
 # - console logging for production operators to capture entire logon session
 # - programmers can use the 'joblog1' script to capture log for 1 job at a time
  ## login1 || exit 2          # exit here if 2nd login
  ## logfixA $LOGNAME          # process log1 file to log2 (to allow read/print)
  ## echo "--> logview   <-- execute logview script to see prior console logs"
  ## echo "logging requires .bashrc/.kshrc with PS1='<@$HOST1:$LOGNAME:$PWD >'"
  ## echo "logging requires $LOGNAME subdirs in \$LOGDIR/log1 & log2"
  ## if [[ -d $LOGDIR/log1/$LOGNAME && ( -f .kshrc || -f .bashrc) ]]; then
  ##    echo "script $LOGDIR/log1/$LOGNAME/$(date +%y%m%d_%H%M%S)"
  ##    exec script $LOGDIR/log1/$LOGNAME/$(date +%y%m%d_%H%M%S)
  ## fi
 # 'exec script' must be the last non-comment line in the profile
 # 'script' disables aliases & umask 002 - put in .bashrc/.kshrc to be effective
 # ==============================
 # cp $APPSADM/env/bashrc .bashrc  # copy to your homedir restoring correct name
 # ==============================
 # After uvadm installed at $UV (/home/uvadm or /opt/uvsw/uvadm)
 # - setup appsadm at $APPSADM (/home/appsadm or /opt/uvsw/appsadm)
 # - copy $UV/env/* $APPSADM/env
 # - modify $APPSADM/env/common_profile & stub_profile for your site
 # - copy $APPSADM.env/stub_profile to user .profiles
 # Then all user profiles call common_profile from $APPSADM/env/...
 # to prevent loss of customized common_profile when new version uvadm installed

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B3. 'profiles' provided in /home/uvadm/env

common_profile_uv

 # common_profile_uv  - for Vancouver Utilities by Owen Townsend, updated March 2020
 #                    - to be '.' dot executed by user .profile or .bash_profile
 #                    - see www.uvsoftware.ca/install.htm
 #
 # common_profile_uv  - UV Software's suggested common_profile
 #                    - defines search PATHs based on $RUNLIBS & $RUNDATA
 #                      which must be defined in user's .profile or .bash_profile
 #                      before calling this commmon_profile
 #                    - copy to $APPSADM/env (/home/appsadm/env) for site customizations
 # common_profile_ABC - users should copy/rename their version of common_profile
 #                      & store their version in /home/appsadm/env/...
 #
 # bash_profile_uv  - also copy to $APPSADM/env (/home/appsadm/env) for site customizations
 # bash_profile_ABC - might append suffix to identify your customized versions
 #                  - then copy to user homedirs & renamed as .profile or .bash_profile
 # stub_profile_uv  - alternate name for bash_profile_uv (same contents)
 #  - stub_profile defines RUNLIBS/RUNDATA/CNVDATA for common_profile to modify PATHs
 #  - users may define depending on their current project (migrations,testing,development)
 #    export RUNLIBS=$HOME/testlibs RUNDATA=$HOME/testdata CNVDATA=$HOME/cnvdata
 #    export RUNLIBS=$HOME/demo RUNDATA=$HOME/demo CNVDATA=$HOME/demo
 #
 # bash_profile & common_profile - distributed in $UV/env/... (usually /home/uvadm/env)
 # - copy to $APPSADM/env/... (/home/appsadm/env/...) & modify for your site
 # - do not modify profiles in $UV because new versions of uvadm would overwrite
 # - see stub/common profile listings at uvsoftware.ca/install.htm#A6 & A7
 #
 #                  ** begin code for common_profile **
 echo " "
 echo "Executing--> \$APPSADM/env/common_profile_uv (APPSADM=$APPSADM)"
 CALLER=$(cat /proc/$PPID/comm)
 echo "LOGNAME=$LOGNAME HOME=$HOME APPSADM=$APPSADM CALLER=$CALLER"
 #
 export UV=/home/uvadm           # UV homedir symbol used below
 export APPSADM=/home/appsadm    # 1st def in .bash_profile (redef here for saftey)
 export LOGDIR=$APPSADM          # console logging subdirs log1,log2,log3
 #Jan2020 - also define APPSADM in .profile_bash (for Windows Subsystem on Windows WSL)
 #
 # setup PATH for Vancouver Utilities programs & scripts (uvadm & appsadm)
 # - append onto system PATH, using symbols defined above ($UV, $APPSADM, etc)
 export PATH=$PATH:./sf:$HOME/sf:$APPSADM/sf:$RUNLIBS/sf:$RUNLIBS/jcls
 export PATH=$PATH:$UV/sf/adm:$UV/sf/demo:$UV/sf/util:$UV/sf/IBM
 export PATH=$PATH:$HOME/bin:$APPSADM/bin:$UV/bin:$UV/help
 #Note - APPSADSM appears before UV so user modified scripts/jobs in APPSADM
 #       can be stored in $APPSADM & be found prior to original versions in $UV
 #     - $UV/sf subdirectoried to adm,demo,util,IBM
 #
 # setup 'PFPATH' for uvcopy & uvqrpg interpreter to find Parameter Files (jobs)
 export PFPATH=./pf,$RUNLIBS/pf,$RUNLIBS/pfs
 export PFPATH=$PFPATH,$HOME/pf,$UV/pf/adm,$UV/pf/demo,$UV/pf/util,$UV/pf/IBM
 # - use symbol $UV (defined above) to shorten PFPATH definition
 # - UV/pf/... follows RUNLIBS,APPSADM,HOME to allow user duplicate names
 # - uvcopy accepts ',' delimiters as well as ':' in case of SFU on Windows
 #
 # setup PATH & FPATH for JCL/scripts converted from mainframe Vancouver Utils
 # - see www.uvsoftware.ca/jclcnv1demo.htm or www.uvsoftware.ca/vsejcl.htm
 export PATH=$PATH:$RUNLIBS/jcls:$RUNLIBS/jts:$RUNLIBS/jus
 #
 #Jun2018 - adding user written db2 utility scripts to the PATH
 #        - named as mainframe utilities - dsntiaul,dsnutilb,dsnuproc,etc
 export PATH=$PATH:$RUNLIBS/db2s/
 #
 # FPATH - defines directory of Korn shell functions (called by some VU scripts)
 #       - examples: jobset12, exportgen0, exportgen1,
 # export FPATH=$UV/sfun       # functions distributed in /home/uvadm/sfun/...
 export FPATH=$APPSADM/sfun    # copy to /home/appsadm/sfun/... for possible customization
 # export FPATH=$RUNLIBS/sfun  # OR to $RUNLIBS for more flexibility if required
 #Feb2020 - FPATH changed back to $APPSADM
 #
 #Mar14/12 - define 'GDGCTL' location of gdgctl51I.dat/.idx
 # - see doc at www.uvsoftware.ca/jclcnv4gdg.htm#5G1
 if [[ -z "$GDGCTL" ]]; then export GDGCTL=$RUNDATA/ctl; fi #<-- set default
 # - see GDG control file discussed at www.uvsoftware.ca/jclcnv4gdg.htm#5A2
 #
 # Define CTLMAPDIR for uvhdcob (display COBOL copybook fieldnames beside data fields)
 # - see www.uvsoftware.ca/uvhdcob.htm#Part_5
 # - for uvhdc1 script, $UV/ctl/ctlfile_uvhdc1, $UV/mf/maps/copybooks
 export CTLMAPDIR=$HOME/mf/maps  #<-- uvhdc1 demos /home/uvadm/dat1/... & /home/uvadm/maps/...
 # export CTLMAPDIR=$RUNLIBS/maps #<-- comment out above defaults this for uvhdc2
 export COBMAPDIR=$RUNLIBS/maps  # for uvhdcob (display data with fieldnames)
 export UVHDCOBROP=m45           # uvhdcob display 45 lines
 # export UVHDROP=l64            # uvhd display 64 chars/line - default
 export UVHDROP=l100             # uvhd display 100 chars/line (if screen allows)
 #
 # Indexed file extension controls for Vancouver Utilities
 export ISDATEXT=".dat" # .dat/.idx Indexed files for uvsort,uvcopy,uvcp,etc
 #                      # uvsort,etc expects .dat on data partition of ISAM files
 #                      # COBOL equivalent is 'IDXNAMETYPE=2' in $EXTFH/extfh.cfg
 #                      # ISDATEXT new way to control DISAM .dat extension Apr2010
 export DISAMEXT="dat"  # DISAMEXT old way prior to Apr2010
 #                      # - omit both or set null if you want NO .dat extension
 #
 # printer destinations for VU laser printing scripts
 # - modify UVLPDEST to the network printer closest to you
 export UVLPDEST="-dMS610USB"    # default dest for uvlp(uvlist) scripts
 export UVLPOPTN="-onobanner"    # for unix/linux (SFU does not allow)
 export UVHDPRINT=uvlp16         # script for uvhd 'i' immediate print command
 export UVHDPWIDE=uvlp14L        # script for uvhd 'iprint' Landscape 100 chs/line
 #-----------------------------------------------------------------------------
 #
 #                  ** TERM, erase, interupt, etc **
 stty erase '^?'           # erase char - modify depending on your terminal
 #                         #  '^?' for linux/at386, '^H' for vt100,ansi,xterm
 # stty -icrnl             # ensure CR x'0D' omitted & only LF x'0A' inserted
 # stty intr '^C'          # interrupt ^C, (probably already default ?)
 #
 #                 ** UV Recommended items **
 umask 002                     # permissions 775 dirs, 664 files
 ulimit -f 25000000            # set max filesize to 25 gig
 set -o ignoreeof              # disallow logoff via control D (use exit)
 trm=$(tty)                    # capture terminal device for PS1
 export trmv=${trm#/dev/}      # remove prefix /dev/
 export HOSTNAME               # should already be set
 export HOST1=${HOSTNAME%%.*}  # extract 1st segment of $HOSTNAME
 export PS1='<@$HOST1:$LOGNAME:$PWD> '
 export EDITOR=vi              # for Korn shell history
 export VISUAL=vi              # for Korn shell history
 export HISTSIZE=5000;         # Korn shell history file size
 export TD8=$(date +%Y%m%d)
 export TD6=$(date +%y%m%d)
 export EM=$HOME/em            # convenience for Owen (EMail directory)
 export EMTD6=$HOME/em/$TD6    # convenience for Owen (EMail directory)
 export AA=$APPSADM            # convenience since $APPSADM often used
 export LC_ALL=C               # disable UTF8 in errmsgs for C compiler
 #                             # sort alpha ABC-Zabc-z vs AaBbCc-Zz
 #
 #                      ** aliases **
 # alias commands to prompt for overwrite (highly recommended)
 # - use option '-f' when you have many files  (rm -f tmp/*, etc)
 alias rm='rm -i'              # confirm removes
 alias mv='mv -i'              # confirm renames
 alias cp='cp -i'              # confirm copy overwrites
 alias l='ls -l'               # save keystrokes
 alias lsd='ls -l $1 | grep ^d'   # list directories only
 alias vi='vim'                # use vim for Linux
 # alias more='less'             # alias to 'less' if preferred
 alias grep='grep -nHd skip'   # ensure filename & line# on matching lines
 alias uname='uname -a'        # ensure -a on uname (All info)
 alias cdl='cd $RUNLIBS'       # quick access to LIBS superdir
 alias cdlc='cd $RUNLIBS/ctl'  # quick access to LIBS/control-files
 alias cdd='cd $RUNDATA'       # DATA superdir
 alias cdc='cd $CNVDATA'       # data CONVERSION superdir
 alias cdk='cd $CMPDATA'       # data COMPARISON superdir
 alias cdm='cd $RUNLIBS/jclmods'  # quick access to Alternate RUNLIBS
 alias cde='cd $EM'            # EMail directory
 alias cdem='cd $EM/$TD6'      # EMail directory for today
 alias mdem='mkdir $EM/$TD6; cd $EM/$TD6'  # make Email dir & change into it
 alias mkem='mkdir $EM/$TD6; cd $EM/$TD6; touch ${TD6}a; vi ${TD6}a;'
 # aliases - ineffective if console logging activated (in user stub profile)
 #         - ifso, place aliases in .bashrc (or .kshrc, for ksh)
 #
 #-------------------------------------------------------------------------
 # Verify that critical environmental variables have been defined
 # (by stub_profile or this common_profile)
 if [[ "$UV" = "" || "$APPSADM" = "" ]]; then
    echo "UV=$UV or APPSADM=$APPSADM not defined"
    echo "- enter to continue"; read $reply; fi
 if [[ "$RUNLIBS" = "" || "$RUNDATA" = "" ]]; then
    echo "RUNLIBS=$RUNLIBS or RUNDATA=$RUNDATA not defined"
    echo "- enter to continue"; read $reply; fi
 #Dec15/10 - set LOGMSGACK, activate ACK option in logmsg2 in JCL/scripts
 export LOGMSGACK=n
 #-------------------------------------------------------------------------
 #
 #        ** Micro Focus COBOL 2.2 update2 Eclipse on RHEL 7 June2015 **
 # export COBDIR=/opt/microfocus/VisualCOBOL
 # export JAVA_HOME=/usr/local/java32
 # export PATH=$COBDIR/bin:$JAVA_HOME/bin:$PATH
 # export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$COBDIR/lib:$JAVA_HOME/lib
 # export COBCPY=$COBDIR/cpylib
 # export CLASSPATH=$COBDIR/lib/mfcobol.jar:$COBDIR/lib/mfcobolrts.jar:$COBDIR/lib/mfsqljvm.jar
 # export COBMODE=64
 # export EXTFH=$UV/ctl/extfh.cfg   # file handler options IDXNAMETYPE=2 FILEMAXSIZE=8
 # export CBLX=$RUNLIBS/cblx    # path for loading COBOL programs
 #
 #                       ** AIX COBOL **
 # set default file type for JCL converter to AIX COBOL
 # - other code at http://www.uvsoftware.ca/admjobs.htm#1C3 or $UV/env/archive/
 # export COBRTOPT=FILESYS=QSAM
 # converted JCL/scripts allow override via cft=XXX, for example:
 # exportfile CUSTMAS data1/ar.custmas.master #cft=QSAM  <-- as generated
 # exportfile CUSTMAS data1/ar.custmas.master cft=STL    <-- uncomment & change type
 #
 #                  ** Microsoft SQL Server **
 # see www.uvsoftware.ca/sqldemo.htm#Part_6
 # export PATH=$PATH:/opt/mssql-tools/bin
 # export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/microsoft/msodbcsql/lib64
 # export ODBCSQL="ODBC Driver 13 for SQL Server"
 # export DATABASE=testdb   # $(DATABASE) used in table create & load scripts
 # export ODBCINI=/etc/odbc.ini
 # export ODBCSYSINI=/etc/  #<-- Directory with ODBC config (not File odbcinst.ini)
 #
 #            ** GNU COBOL testing Oct 2019 **
 # export GCHOME=/home/gcobol
 # export GCDIR=/home/gcobol/cob
 #
 #         ** optional for WSL (Windows Subsystem for Linux) **
 #  Example#1 - export WINUSER using wslpath to get windows %USERPROFILE% (in C:\USERS\...)
 ## WINUSER=$(wslpath $(cmd.exe /C "echo %USERPROFILE%"))
 ## export WINUSER=$(echo $WINUSER | tr -d '\r')
 ## echo "WINUSER=$WINUSER"
 # Example#2 - setup variables for Both Windows & Linux using 'WSLENV' (translates path differences)
 ## C:\uvadm> set UVADM=C:\uvadm    <-- set variable in windows
 ##           set WSLENV=UVADM/p    <-- WSLENV sets up variable UVADM for Both Windows & Linux
 #  C:\uvadm> echo %UVADM%          --> shows value "C:\uvadm" as expected
 #  C:\uvadm> wsl                   <-- run WSL (or bash)
 #  /mnt/c/uvadm>                   <-- now running Linux, prompt changed as per common_profile
 #  /mnt/c/uvadm> echo $UVADM       <-- test, see if same variable now shows Linux path
 #  /mnt/c/uvadm> /mnt/c/uvadm      --> proves $UVADM on Linux equivalent of %UVADM% on Windows
 #
 #             ** define directories for uvcopy mailx1 or mutt1 **
 # export MAILDATA=maildata        #<-- input data files
 # export MAILMSGS=mailmsgs        #<-- MSG files created for input to mailx utility
 # export MAILSCRIPTS=mailscripts  #<-- scripts created to execute mailx utility
 #
 #        ** defines for QuikJobs & Easytrieves (converted to uvcopy) **
 # export QJS=$RUNLIBS/qjs
 # export EZTS=$RUNLIBS/ezts
 #
 #            ** optional software **
 # - Micro Focus COBOL, AIX COBOL, GNU COBOL, COBOL-IT
 # - Microsoft SQL Server Oracle mySQL Morada RPG
 # - WSL (Windows Subsystem for Linux)
 #
 #Note - code removed for seldom used items, Feb 2016
 # - but prior version saved in $UV/env/archive/common_profile_uv_20160215
 # - contains items that may need to be recovered, such as:
 # - Micro Focus COBOL, COBOL-IT, Oracle, MySQL, Morada RPG, SQL Server
 # - see listings at www.uvsoftware.ca/admjobs.htm#1C1 & 1C2,1C3,etc
 #------------------------- end of common_profile ---------------------------

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B4. Vancouver Utilities Installation Guide

.vimrc - set options for vim

Note
  • Be SURE to copy .vimrc to your homedir, GREATLY ENHANCES vim experience.
  • Highlights your search words
  • Highlights your current line (cursor position)
  • Restores cursor to last line position on your previous vim of that file.
  • Increases max lines per buffer from 50 to 1000 lines
    (used when Yanking & Putting buffers 'a' - 'z')

.vimrc file contents

 " vimrc - setups for vi/vim, stored in /home/uvadm/env/vimrc
 "       - copy to your homedir & rename as .vimrc
 "
 " set HighLighting for search matches & current cursor line
 set hlsearch     "<-- HighLights search words (yellow)
 set cursorline   "<-- Highlights cursor line (light blue)
 "
 " restore cusrsor to previous position when reopening a file
 if has("autocmd")
   au BufReadPost * if line("'\"") > 0 && line("'\"") <= line("$")
     \| exe "normal! g'\"" | endif
 endif
 "
 " increase yank max lines per buffer a-z, from default (50 lines) to 1000 lines
 set viminfo='20,<1000   "note max 20 files & max 1000 lines
 " -------------------- end of .vimrc ----------------------

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B5. profiles for Vancouver Utilities on Unix/Linuxs

Advantages of RUNLIBS/RUNDATA & stub_profile/common_profile

  1. You never need to change scripts between TESTING & PRODUCTION.

  2. Programmers execute scripts from $TESTLIBS/scripts that access data from $TESTDATA/... because their profiles set RUNLIBS/RUNDATA as TESTLIBS/TESTDATA

  3. Operators execute scripts from $PRODLIBS/jcls that access data from $PRODDATA/... because their profiles set RUNLIBS/RUNDATA as PRODLIBS/PRODDATA

  4. This system could allow testing & production on the same machine. They are usually on separate machines, and this system provides great flexibility for both these situations.

  5. Different programmers can have their own set of TESTDATA simply by changing the definition of RUNDATA in their profile.

  6. On production machines, different operators could operate on different sets of data for different companies on the same machine, using the same scripts & programs with no changes required.

  7. The system administrator (appsadm) can make changes in 1 place to affect all users vs having to update profiles in all user homedirs.

  8. The sysadmin would control any updates to common_profile. On test systems the programmers would control updates to their stub_profile.

  9. Programmers have the flexibility required. They can select different sets of testdata by redefining RUNDATA in their profile

    RUNLIBS determine PATHs to programs,scripts,jcls,etc

The PATH to JCL/scripts depends on $RUNLIBS (testlibs or prodlibs), example:


 export PATH=$PATH:$RUNLIBS/jcls    <-- PATH to JCL/scripts (in common_profile)
 ===============================

RUNDATA determine PATHs to DATA files

'$RUNDATA' determines data-file locations indirectly as follows:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B6. Recommended Command Line Working Procedures

Working-Directory - Best Practices

For these UVdemos, you will be asked to copy $UV/demo/* to your $HOME/demo/... & change into demo/ which will be your 'working-directory' for the demos. Here are a few subdirs/files to illustrate Working-Directory Best-Practices.

 /home/uvadm/demo/              <-- demo files supplied
 /home/userxx/demo/             <-- demo files copied to your homedir
 |-- dat1/                      <-- data files (just 2 for illustration)
 |   |-- CanadaMPs
 |   `-- UScongress
 |-- sf/                        <-- scripts (just 2 for illustration
 |   |-- select_CanadaMPs
 |   |-- select_UScongress
 |-- selects/                   <-- output files from selection exercises
 |   |-- dat1_CanadaMPs_BC
 |   `-- dat1_UScongress_AK
 |-- tmp/                       <-- misc output files from various exercises
 |   |-- CanadaMPs_sorted_Prov
 |   |-- SORTtmptest
  1. A working-directory with all files in subdirs keeps your working sapce clean & well organized - your subdirs will probably fit on 1 screen, which makes it easy for you to review your project.

  2. You should stay in the working directory - do not change into the subdirs to investigate/edit your files.

  3. Don't waste time by changing into a subdir, looking at 1 file,& changing back out. Of course OK, if you intend to inspect/edit multiple files, but not for 1 or 2.

  4. Staying in your working-directory protects your files, even the infamous 'rm *' will do no harm when all files are in subdirs.

  5. Working-Directory with subdirs allows you to use the same filename for applications that process the same datafile thru multiple stages. For example JCL & COBOL conversions require multi-step processing.

  6. Many utilities automatically write their output to the tmp/... directory & will make ./tmp/ if it does not exist.

  7. uvsort defaults to using ./tmp for merge files & tests presence & write access by writing tmp/SORTtmptest before attempting the sort (you can export SORTDIR=... in the common_profile to change the default).

  8. Garbage files (accidentally created) in the working-direcotry, rather then the intended subdir will stand out, so you will notice them, investigate the reason & correct the problem.

  9. Files created in the working directory can occur when there is a mistake in assigning a logical $VARIABLE filename to the physical subdir/filename. You would not notice if you were operating in the subdir with hundreds of files.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1B7. Recommended Command Line Working Procedures

tmp/ subdir in Working-Directory

A 'tmp/' subdir in in our working-directory has many uses & benefits. Do not confuse our ./tmp/ subdir in our working-directory with the unix system '/tmp/' at the top of the root file system.

  1. Several Vancouver Utility programs & scripts expect a tmp subdir and some will make it if not present.

  2. 'uvsort' expects a ./tmp/ subdir for merge files unless you define an alternate in the common_profile via 'export SORTDIR=...'

  3. 'uvhd' will make a tmp/ dir (if not present) to store an index for variable length & text files, so you can directly access any record by its record#. uvhd also uses the tmp/ dir for outputs from record write or print selections.

  4. Several Vancouver utility scripts output to the tmp/ subdir. For example 'alldiff2' runs the unix 'diff' command for all files in 2 directories, and writes the output to tmp/... using the 2nd directory-name.dif. Then you can use the editor to investigate the diff report (more flexible than cat/more).

  5. Redirecting large outputs of unix utilities such as 'grep' & then investigating with 'vi' is much more flexible than piping to more or less.

  6. You can use the tmp/ dir for cut & paste between files - edit the 1st file, write out the desired lines to tmp/xxx. Then edit the 2nd file & read the tmp/xxx file into the desired location.

  7. You can use the tmp/... directory like the unix system '|' command that pipes the output of 1 comamnd directly into a 2nd command, when you want to inspect or modify the 1st output before inputting to the 2nd command.

  8. Using a tmp/ subdir for any misc file outputs keeps your working-directory clean & well organized - no garbage file clutter, your subdirs can probably fit on 1 screen.

  9. Any files showing up on your working-directory list will signal an error to be investigated & removed. For example an incorrect variable assignment (export LOGICALNAME=subdir/filename) will cause a program to write the file into the working-directory rather than the intended subdir.

  10. We suggest your clean out your tmp/* dir frequently (at least daily) so you can easily see your current tmp/files vs yesterdays.

  11. These concepts protect your files, even the infamous 'rm *' will do no harm - because you stay in the working directory which has no files.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1C1. UVdemos - Vancouver Utilities for Unix/Linux users

setup user account to run demos

Here are instructions to setup a user account to run the demos. You could run the demos in the Vancouver Utilities admin account /home/uvadm/... But it is better to setup a user account & copy the demo files into your directories.


 #1. login as 'root'

 #2a. useradd -m -g apps -s /bin/bash userxx
      ======================================
      - use option '-s' to specify login shell as 'bash'
        OR use #2b. below for 'ksh'

 #2b. useradd -m -g apps -s /bin/ksh userxx
      =====================================
      - use option '-s' to specify login shell as 'ksh' (Korn shell)

 #3. passwd userxx           <-- setup password desired
     ==============

 #4. chmod 755 /home/userxx  <-- allow copying between your user accounts
     ======================    - optional

 #5. cp $UV/env/stub_profile_uv /home/userxx/.bash_profile
     =====================================================
     - copy supplied profile to homedir renaming to '.bash_profile'
     - calls the common profile from $APPSADM/env/common_profile_uv
     - see profiles listed above on pages '1B2' & '1B3'

 #6. exit    - exit from root
     ====

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1C2. UVdemos - Vancouver Utilities for Unix/Linux users

Alternative - setup demo/ dir in existing home dir

Alternatively (to setting up new user to run demos), you could use your existing login account - Make sub-directory $HOME/demo & copy $UV/demo/* to your $HOME/demo/...


 #1. login your_existing_userid --> your homedir
     ==========================

 #2. mkdir demo                <-- make demo/ dir in your homedir
     ==========                  - /home/userxx/demo

copy demo files to your $HOME/demo


 #3. cp -r $UV/demo/* demo  <-- copy $UV/demo/* files to your demo/...
     =====================

 #4. tree2 demo    <-- investigate demo directories/files using tree2 script
     ==========      - calls the 'tree' utility to draw diagrams
                       illustrating the directory/files structure

See the 'tree' diagram on the next page showing relevant files for demos.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1C4. UVdemos - Vancouver Utilities for Unix/Linux users

demo directories/files illustrated by 'tree2'

 demo
 |-- 1README
 |-- archive
 |-- backup
 |   |-- dat0,dat1,dat2, see files below
 |   |
 |-- dat0                        <-- demo files captured from internet
 |   |-- CanadaMPs.csv             - before reformatting in dat1/...
 |   |-- custmas0
 |   |-- UScities.tab
 |   |-- UScongress.csv
 |   |-- USstates.csv
 |   |-- vendormas0
 |-- dat1                        <-- demo files set1
 |   |-- CanadaMPs                 - Canadian MPs
 |   |-- CanadaProvinces
 |   |-- nameadrs1
 |   |-- UScandidates            <-- US Democratic 2020 candidates
 |   |-- UScandidateups
 |   |-- UScities
 |   |-- UScongress              <-- US congress house members
 |   |-- USstates
 |-- dat2                        <-- demo files set2
 |   |-- accents1                  - various files for various demos
 |   |-- accents2
 |   |-- accents3
 |   |-- ar.custmas
 |   |-- ar.custmas.tellist
 |   |-- ar.sales.items
 |   |-- ar.sales.products
 |   |-- custmas1
 |   |-- custmas2
 |   |-- IPreplace.tbl
 |   |-- nameadrs2
 |   |-- namesadrs0e
 |   |-- namesadrs1f
 |   |-- namesadrs2t
 |   |-- py.payroll.master
 |   |-- sales2
 |   |-- sales3
 |   |-- sales3.cpy
 |   |-- sales4
 |   |-- tabtest1
 |   |-- trt256
 |   |-- vendormas2
 |   |-- vendormas3

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 |-- election1                  <-- election1 for quick test project
 |   |-- candidates               - with no plan for future sustainability
 |   |-- candidateups
 |   |-- sf                     <-- see 'Part_9' demo files
 |   |   |-- loadcandidates       - based on US elections 2020
 |   |   |-- sortcandidates
 |   |   |-- sortloadcandidates
 |   |   |-- unloadcandidates
 |   |-- tmp
 |   |
 |-- election2                   <-- election2 well planned test/develop project for
 |   |-- data1                     - ongoing flexibility & plans for other projects
 |   |   |-- candidates          <-- demo files based on US 2020 elections
 |   |   |-- candidates.dat
 |   |   |-- candidates.idx
 |   |   |-- candidatesun
 |   |   |-- candidateups
 |   |   |-- candidateups.fixed
 |   |-- pf                      <-- uvcopy jobs (Parameter Files)
 |   |   |-- fixcandidateups
 |   |-- sf                      <-- Script Files for Korn/Bash shells
 |   |   |-- loadcandidates
 |   |   |-- sortcandidates
 |   |   |-- sortloadcandidates
 |   |   |-- unloadcandidates
 |   |-- slogs                   <--logs captured by 'slog' script
 |   |   |-- ...                    executing target scripts
 |   |-- tmp

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 |-- errs
 |   |-- jclunix51.err
 |   |-- jclunix51.ERR
 |   |-- parmNF.rpt
 |-- ftps                        <-- ftp scripts
 |   |-- ftpdemo1a
 |   |-- ftpdemo1b
 |   |-- ftpget1
 |   |-- ftpput1
 |-- jcl2                        <-- mainframe JCLs before conversion
 |   |-- jar100.jcl
 |   |-- jar200.jcl
 |   |-- jgl100.jcl
 |   |-- jgl200.jcl
 |   |-- jgl230.jcl
 |   |-- jgl232.jcl
 |   |-- jgl320.jcl
 |-- jcl3                        <-- scripts converted from mainframe JCLs
 |   |-- jar100.ksh
 |   |-- jar200.ksh
 |   |-- jgl100.ksh
 |   |-- jgl200.ksh
 |   |-- jgl230.ksh
 |   |-- jgl232.ksh
 |   |-- jgl320.ksh
 |-- jcl3.old                    <-- used for 'alldiff2' script to diff
 |   |-- jar100.ksh                  all files in 2 directies
 |   |-- jar200.ksh                - confirm changes as desired & no unwanted
 |   |-- jgl100.ksh
 |   |-- jgl200.ksh
 |   |-- jgl230.ksh
 |   |-- jgl232.ksh
 |   |-- jgl320.ksh

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 |-- pf                          <-- uvcopy jobs (Parameter Files)
 |   |-- CanadaMPscsv2txt          - convert .csv file to fixed field text
 |   |-- CanadaMPscsv2txtUTF8      - etc
 |   |-- removeCR_uvcopy
 |   |-- UScitiescsv2txt
 |   |-- UScongresscsv2txt
 |   |-- USstatescsv2txt
 |   |
 |-- rptcmp                      <-- comparison reports for packed/binary files
 |   |-- ...                         (cannot use diff)
 |-- rpts                        <-- misc reports
 |   |-- ...
 |-- selects                     <-- outputs from various record select utilities
 |   |-- ...
 |-- sf                          <-- Script Files for Korn/Bash shells
 |   |-- copyjcl3
 |   |-- fixaccents2
 |   |-- fixaccents2x
 |   |-- llsx
 |   |-- llsx.bak
 |   |-- llsy
 |   |-- select_CanadaMPs
 |   |-- select_nameadrs1
 |   |-- select_UScongress
 |   |-- table2_CanadaMPs
 |   |-- table2_UScongress
 |-- sql0                        <-- mainframe SQL scripts before conversion to Linux
 |   |-- 'SELECT.CUSTOMERS.ALL'
 |   |-- 'SELECT.CUSTOMERS.BASIC'
 |   |-- 'SELECT.CUSTOMERS.SALES10000'
 |   |-- 'SELECT.CUSTOMERS.SALES20000'
 |   |-- 'SELECT.EMPLOYEE.DEPT123'
 |   |-- 'SELECT.EMPLOYEE.SEATTLE'
 |   |-- 'SELECT.ORDERS.BIG'
 |   |-- 'SELECT.PATIENTS.OVER70'
 |   |-- 'SELECT.PATIENTS.OVER80'
 |   |-- 'SELECT.PATIENTS.OVER90'
 |   |-- 'SELECT.STUDENTS.FAILING'
 |-- sqls                        <-- SQL scripts after conversion to Linux
 |   |-- select_customers_all.sql
 |   |-- select_customers_basic.sql
 |   |-- select_customers_sales10000.sql
 |   |-- select_customers_sales20000.sql
 |   |-- select_employee_dept123.sql
 |   |-- select_employee_seattle.sql
 |   |-- select_orders_big.sql
 |   |-- select_patients_over70.sql
 |   |-- select_patients_over80.sql
 |   |-- select_patients_over90.sql
 |   |-- select_students_failing.sql
 |-- stats
 |-- tmp
 |-- tmp1
 |-- tmp2

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D1. Demo files for Tutorials

dat1/nameadrs1 - test/demo file for SelectJobs

 Owen Townsend       4667 Hoskins Rd     North Vancouver, BC Canada V7K2R3
 Jenny Townsend      21 Canada Place     Vancouver, BC       Canada V5P3V8
 John Horgan         123 Townsend Ave.   Townsend City, BC   Canada V1P2G3
 John Townsend       2121 Owen Drive.    St. John, NB        Canada M5W1E6
 John Owens          24 Owen Avenue      Owenville, ON       Canada M5K3B3
 Donald Trump        1600 Pennsylvania   Washingtom, DC      USA 00001
 Bill Gates          1 Microsoft Way     Seattle, WA         USA 98052-6399
 Peter Townsend      24 Johnson Road     London, England     UK EC1A 1HQ

dat1/CanadaMPs - 334 members - samples of all parties

 010  Anderson           David           Cypress Hills/Grasslands      SK Conservative     2015
 011  Angus              Charlie         Timmins/James Bay             ON NDP              2015
 023  Barsalou-Duval     Xavier          Pierre-Boucher/Les Patriotes/ QC Bloc Quebecois   2015
 032  Bernier            Maxime          Beauce                        QC People's Party   2015
 202  May                Elizabeth       Saanich/Gulf Islands          BC Green Party      2015
 328  Wilson-Raybould    Jody            Vancouver Granville           BC Independent      2015

dat1/CanadaProvinces

      01  AB  Alberta
      02  BC  British Columbia
      03  MB  Manitoba
      04  NB  New Brunswick
      05  NL  Newfoundland and Labrador
      06  NT  Northwest Territories
      07  NS  Nova Scotia
      08  NU  Nunavut
      09  ON  Ontario
      10  PE  Prince Edward Island
      11  QC  Quebec
      12  SK  Saskatchewan
      13  YT  Yukon

dat1/UScongress - 1st & last 3 of 539 members

 001  Abraham             Ralph          19540916  M  LA  rep  Republican
 002  Adams               Alma           19460527  F  NC  rep  Democrat
              ---------- omitting 535 lines ----------
 538  Young               Todd           19720824  M  IN  sen  Republican
 539  Zeldin              Lee            19800130  M  NY  rep  Republican

dat1/USstates - 1st 3 & last 3 of 50 states

      01  AL  Alabama
      02  AK  Alaska
        -------- omitting 46 records --------
      49  WI  Wisconsin
      50  WY  Wyoming

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D2. Demo files for Tutorials

dat2/sales2 - sales file with zone signs

          1         2         3         4         5         6
 1234567890123456789012345678901234567890123456789012345678901234
 CUST#    SLM  DATE   INV#     PROD#     QTY   PRICE    AMOUNT
 130140    21 990802 IN111001  HAM001  000020 0000101 000002020
 130140    21 000802 IN111001  SCR012  000021 0001001 000021021
 130140    21 010802 IN111001  CHR001  000022 0010001 000220022
 139923    35 980802 IN111002  TAB013  000023 0000201 000004623
 139923    35 990807 IN111002  VAR200  000024 0002001 000048024
 150825    44 010804 IN1122    HAM001  000025 0020001 000500025
 150825    44 020804 IN1122    HAX129  000026 0000301 000007826
 201120    44 030807 CR5234    BBQ001  000027 0003001 000081027
 223240    65 010816 CR955     HAM001  000028 0030001 000840028
 223240    65 020816 IN441     BBQ001  000029 0000101 000002929
 308685    21 020812 IN6605    SAW051  00001p 0000101 00000101p
 308685    21 030812 IN6605    BIT075  00001q 0001001 00001101q
 308685    21 040812 CR8835    TAB013  00001r 0010001 00012001r
 315512    44 040805 IN2251    HAM001  00001s 0000201 00000261s
 315512    44 050805 IN2251    SAW051  00001t 0002001 00002801t
 315512    44 060805 IN2255    BIT075  00001u 0020001 00030001u
 400002    85 990812 CR245     HAX129  00001v 0000101 00000161v
 406082    35 010815 IN33001   BBQ001  00001w 0001001 00001701w
 406082    35 020815 IN33001   TAB013  00001x 0010001 00018001x
 406082    65 030816 IN441     VAR200  00001y 0000101 00000191y

dat2/sales3 - sales file with separate signs

          1         2         3         4         5         6
 1234567890123456789012345678901234567890123456789012345678901234
 CUST#    SLM  DATE   INV#     PROD#     QTY   PRICE    AMOUNT
 130140    21 940802 IN111001  HAM001  00020+ 0000001 00000020+
 130140    21 940803 IN111001  SCR012  00021+ 0001001 00021021+
 130140    21 940804 IN111001  CHR001  00022+ 0002001 00044022+
 139923    35 950802 IN111002  TAB013  00023+ 0003001 00069023+
 139923    35 950807 IN111002  TAB013  00024+ 0004001 00096024+
 150825    44 960804 IN1122    HAM001  00025+ 0005001 00125025+
 150825    44 960805 IN1122    HAX129  00026+ 0006001 00156026+
 201120    44 970807 CR5234    BBQ001  00027+ 0007001 00189027+
 223240    65 980816 CR955     HAM001  00028+ 0008001 00224028+
 223240    65 980817 IN441     BBQ001  00029+ 0009001 00261029+
 308685    21 990812 IN6605    SAW051  00010- 0000001 00000010-
 308685    21 990813 IN6605    BIT075  00011- 0001001 00011011-
 308685    21 990814 CR8835    TAB013  00012- 0002001 00024012-
 315512    44 000805 IN2251    HAM001  00013- 0003001 00039013-
 315512    44 000806 IN2251    SAW051  00014- 0004001 00056014-
 315512    44 000807 IN2255    BIT075  00015- 0005001 00075015-
 400002    85 010812 CR245     HAX129  00016- 0006001 00096016-
 406082    35 020815 IN33001   BBQ001  00017- 0007001 00119017-
 406082    35 020816 IN33001   TAB013  00018- 0008001 00144018-
 406082    65 020816 IN441     HAM001  00019- 0009001 00171019-

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D3. Demo files for Tutorials


 cat ftps/*      <-- show FTP demo input files
 ===========
      # ftps/ftpdemo1a
      open 192.168.0.4
      user user01 user01pw
      lcd data1
      get /home/userxx/ar/sales.items
      get /home/userxx/gl/account.tran1
      quit
      # ftps/ftpdemo1b
      open 192.168.0.5
      user user02 user02pw
      lcd data1
      put ar.sales.items   C:\AR\SALES.ITEMS
      put gl.account.tran1 D:\GL\ACCOUNT.TRAN1
      quit
      # ftps/ftpget1
      open 192.168.0.6
      user user03 user03pw
      lcd data1
      binary
      cd  /home/userxx/gl
      get gl.acnt.master
      get gl.acnt.trans
      quit
      # ftps/ftpput1
      open 192.168.0.6
      user user03 user03pw
      binary
      lcd data1
      cd /home/userxx/data1
      put gl.acnt.master
      put gl.acnt.trans
      quit

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D4. Demo files for Tutorials

demo input files - SQL scripts


 cat sqls/*      <-- show SQL demo input files
 ==========
      /* customers_all */
         SELECT * FROM Customers;
      /* customers_basic */
         SELECT Name, Phone, Address, City, Zip
                FROM Customers;
      /* customers_sales10000 */
         SELECT custno, Name, Phone, Address, City, Zip,
         TYsales, LYsales, FROM
         Customers WHERE LYsales >10000;
      /* customers_sales20000 */
         SELECT custno, Name, Phone, Address, City, Zip,
         TYsales, LYsales, FROM
         Customers
         WHERE LYsales >20000;
      /* employee_dept123 */
         SELECT EMP_ID, NAME, ADDRESS, CITY, ZIP, DEPT FROM
         EMPLOYEE WHERE DEPT = '123' AND STATUS = 'FULL';
      /* employee_seattle */
         SELECT EMP_ID, LAST_NAME FROM EMPLOYEE
         WHERE CITY = 'Seattle' ORDER BY EMP_ID;
      /* orders_big */
         SELECT item FROM Orders
         WHERE quantity > 500;
      /* patients_over70 */
         SELECT Name, Age FROM Patients WHERE Age > 70
         GROUP BY Age ORDER BY Name;
      /* patients_over80 */
         SELECT Name, Age FROM Patients WHERE Age > 80
         GROUP BY Age ORDER BY Name;
      /* patients_over90 */
         SELECT Name, Age FROM Patients
         WHERE Age > 90 GROUP BY Age ORDER BY Name;
      /* students_failing */
         CREATE VIEW Failing_Students AS
         SELECT S_NAME, Student_ID FROM
         STUDENT WHERE GPA < 30;

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D5. Demo files for Tutorials

demo input files - sf


 cat sf/llsx     <-- show BAD script for uvhd demo
 ===========
      #!/bin/bash
      # llsx - List files in directory, in Size order, Biggest 1st
      echo "llsx - list directory sorted by file-size (biggest files 1st)"
      if  [[ -z "$1" ]] ; then dir="."; else dir=$1; fi
      ls -lS $dir | more
      #=================
      exit 0
      #Note - 'llsx' is a BAD script, But 'vi' will not show the problem !

 cat sf/llsy     <-- show GOOD script for uvhd demo
 ==========
      #!/bin/bash
      # llsy - List files in directory, in Size order, Biggest 1st
      echo "llsy - list directory sorted by file-size (biggest files 1st)"
      if  [[ -z "$1" ]] ; then dir="."; else dir=$1; fi
      ls -lS $dir | more
      #=================
      exit 0
      #Note - 'llsy' is the GOOD script vs 'llsx' the BAD script
Note
  • You can not see any significant difference between BAD & GOOD scripts
    using the usual unix/linux tools (cat, more, vi, lp, etc)
  • You can see the problem using 'uvhd' (display hexadecimal) see '10B1'

demo input files - pf

 # removeCR_uvcopy - copy a file removing any CR's
 #                 - by Owen Townsend, UV Software, Feb 21/2019
 # - documented at uvsoftware.ca/uvdemos.htm
 # - demo Vancouver Utilities usefulness to any unix/linux user
 #
 # uvcopy removeCR_uvcopy    <-- execution command
 # ======================
 opr='$jobname - copy a file removing any Carriage Returns'
 fili1=?sf/llsx,rcs=256,typ=LST        #<-- declare input file
 filo1=?tmp/llsx,rcs=256,typ=LSTtp775  #<-- declare output file
 @run
         opn     all                    open files
 # begin loop to read/process/write records until EOF
 man20   get     fili1,a0               get next record into area 'a'
         skp>    man90                  (cc set > at EOF)
         mvc     b0(256),a0             copy input area 'a' to output area 'b'
         rep     b0(256),x'0D',''       remove CRs
         put     filo1,b0               write record to output file
         skp     man20                  return to get next record
 # EOF - close files & end job
 man90   cls     all
         eoj

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1D6. Demo files for Tutorials

demo input files - JCL


 cat jcl2/jar200.jcl   <-- sample JCL
 ===================
 //JAR200   JOB  (1234),'TEST/DEMO MVS JCL CONVERT'
 //* SORT AR.SALES.ITEMS BY PRODUCT CODE FOR LISTING
 //STEP010  EXEC PGM=SORT,REGION=2048K
 //SORTIN   DD DSN=AR.SALES.ITEMS,DISP=SHR
 //SORTOUT  DD DSN=&&jar200_TEMPSLS,DISP=(NEW,PASS),
 //            DCB=(LRECL=64,BLKSIZE=6400,RECFM=FB),
 //            SPACE=(TRK,(50,50),RLSE),UNIT=DISK
 //SYSIN    DD *
 SORT FIELDS=(31,6,CH,A,1,6,CH,A)
 OMIT COND=((11,2,CH,EQ,C'65'),OR,(COND=(11,2,CH,EQ,C'85'))
 /*
 //* LIST AR.SALES.ITEMS IN PRODUCT CODE SEQUENCE
 //STEP020  EXEC PGM=CAR200,COND=(4,LT,STEP010),PARM=2006
 //SALEDTL  DD DSN=&&jar200_TEMPSLS,DISP=(OLD,PASS)
 //CUTMAS  DD DSN=AR.CUSTOMER.MASTER.INDEXED,DISP=SHR
 //SALELST  DD DSN=AR.SALES.LIST,DISP=(,CATLG,DELETE),
 //            UNIT=DISK,SPACE=(TRK,(25,25),RLSE),
 //            DCB=(MODEL.DSCB,LRECL=133,BLKSIZE=6118,RECFM=FBA)

 cat jcl2/jar200.jcl   <-- sample JCL for SelectJobs
 ===================     - find SORTs with Both AND & OR CONDitions
 //JGL232   JOB  'alt version of jgl230 to test selectlgfd1 AND 7 OR in same sort'
 //* SORT TRANS, UPDATE GLMASTER,& list GLMASTER
 //* SORT GL TRANSACTIONS BEFORE UPDATING ACCOUNT MASTER
 //*May21/2018 - correct INCLUDE column 55 to 56 to match data
 //STEP010  EXEC PGM=SORT
 //SORTIN   DD DSN=GL.ACCOUNT.TRAN1,DISP=OLD
 //SORTOUT  DD DSN=GL.ACCOUNT.TRANS(+1),DISP=(,CATLG,DELETE),
 //            DCB=(LRECL=80,BLKSIZE=8000,RECFM=FB),
 //            SPACE=(TRK,(50,50),RLSE),UNIT=DISK
 //SYSIN    DD *
 SORT FIELDS=(1,8,CH,A,69,12,CH,A)
 INCLUDE COND=(((9,1,CH,EQ,C'T'),OR,(COND=(9,1,CH,EQ,C'X'))),
                                 AND,(COND=(56,2,CH,EQ,C'AB')))
 /* (only 1st step of jgl232)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1E1. UVdemos - Vancouver Utilities for Unix/Linux users

Hard-copy guide to work thru tutorials

We suggest you print this documentation to guide you as you work your way thru the tutorials. This will leave your screen free to enter the commands & compare your results to the expected results shown in the hard-copy. Also useful to highlight the commands you intend to enter & make notes in the margins.

If you have a laser printer that is Duplex capable, you can use the 'uvlp13D' script to print Duplex at 13 cpi, which allows 95 characters across on 8 1/2 paper with margins to allow for 3 hole punched paper for mounting in a 3 ring binder.

Note
  • you need to "export UVLPDEST=-dlaserxx" where 'laserxx' is a printer near you
  • OR you can update the value set in the common_profile
  • OR override the common_profile definition by defining in your .bash_profile
    after calling the common_profile.

 #1. login userxx --> /home/userxx (or login uvadm if you have password)

 #2. cd $UV --> /home/uvadm

 #3. export UVLPDEST=-dlaserxx  <-- set destination to a printer near you
     =========================

 #4. uvlp13D doc/UVdemos.doc    <-- print documentation at 13 cpi Duplex (180 pages)
     =======================

Note that we have provided Parts 3,4,5 as separate documents, since we think these may be the sections that many sites will find most useful & that can be used immediately without a lot of study.


 uvlp13D doc/SelectJobs.doc  <-- Part3 SelectJobs (28 pages)
 ==========================

 uvlp13D doc/TableJobs.doc   <-- Part3 SelectJobs (32 pages)
 =========================

 uvlp13D doc/UVscripts.doc   <-- Part3 SelectJobs (33 pages)
 =========================

If interested, see uvsoftware.ca/uvlist.htm for many more scripts to print text files on laser printers at various cpi,lpi,simplex,duplex,margins. For example, mainframe conversions use 'uvlp13LD' to print 132 columns wide by 66 lines deep Landscape & Duplex on 8 1/2 paper.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_2. UVdemos Vancouver Utility Programming Aids

Part 2 - Basic Vancouver Utilities


2A1. Basic Vancouver Utilities are uvhd,uvsort,uvcp,& uvlist
(vs Advanced Utilities uvcopy & pre-programmed jobs based on uvcopy).
We will give brief descriptions of uvhd,uvsort,uvcp,& uvlist, followed
by tutorials illustrating how they might help you solve data problems.

2B1. uvhd demo#1 - removing CRs (x'0D's) from scripts (CRs prevent execution)
This is the short version of the in-depth version on pages '10B1' - 10B11,
which gives full explanations & multiple ways to correct the problems
for 1 script or all scripts in a directory.

2C1. uvhd demo#2 - investigating problem files
 Files with high-bit characters may cause problems for older applications
Example - converting French accents (UTF-8) to unaccented equivalents
 Only a Brief Intro here, see the full presentation later in Part_20

2D1. uvhd demo#3 - investigating unknown files
example - your Linux site receives an EBCDIC file from a mainframe site
- how to identify with uvhd & determine record-size
- mainframe files do not have LineFeeds to mark end of records

2E1. uvcp demo - translate EBCDIC records to ASCII
- and insert LineFeeds at same time.

2F1. uvsort demo - sort with record selection
uvsort has the functionality of mainframe SORT (record select,reformat,etc).
We will demo sorting dat2/sales3 on salesman & product, selecting records with
sales amount over $1000 (amount field cols 54-62 8 digits & sign).

2G1. uvlist - list ascii text files piping the output to a laser printer.
Many scripts/options are provided for laser printers. For example you can list
132 column reports on 8 1/2 width paper Landscape & Duplex using the 'uvlp13LD'
Page headings identify filename, date:time, userid,& page#.

2G2. 'uvhelp' menu & the 'v31' sub-menu to see the various uvlp... scripts available

2G3. 'listall1' script to print small files, multiple files per page
- starts a new page if next file will not fit on current page
- options to print 1st so many files & 1st no of lines per file
- output file in tmp1/... (using input filename with any '/'s changed to '_'s)
  then you can print with scripts such as 'uvlp13D' or use file any way you wish

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part 2 - Basic Utilities Contents Continued


2H1. uvcopy demo - removing CRs from scripts, uvcopy eqivalent of uvhd '2B1' demo.
uvcopy is the most powerful Vancouver Utility. It is a 'general purpose'
(can do anything) data manipulation utility. It gives you the power of assembler
without the complexity. See over 100 instructions at uvcopy3.htm.

suggest - could do demos in Parts 3,4,5 before Part 2

If you are new to these utilities, you might want to initially skim Part 2 and first do the demos in Parts 3,4,5 - because the tutorials in Parts 3,4,5 demonstrate scripts that any unix/linux user will find very useful immediately. (Selecting Files, Creating Table Summaries, Counting Files,Lines,KB in directories).

Suggest you start with Part_5 since that extends the Unix knwoledge you already have re listing & counting files & directories.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2A1. Basic Vancouver Utilities - Brief Descriptions

These brief descriptions of uvhd,uvcp,uvsort,& uvlist, will be followed by examples/tutorials illustrating how they might help you solve data problems.

uvhd

'uvhd' - file investigation utility that displays any file in vertical hexadecimal, and prompts for commands to browse,search,replace,update,select,print,count,etc. uvhd allows you to see characters not shown by text editors (control characters & high-bit characters that can cause problems in text files). uvhd is interactive with 25 help screens, and is the favorite utility of all my customers. See uvhd documentation at https://www.uvsoftware.ca/uvhd.htm.

uvcp

'uvcp' - command line data utility that can copy files, converting record formats between: fixed, variable, sequential, indexed, text, RDW, etc). uvcp provides record select, reformat, translate, etc. uvcp (& uvsort,uvcopy,etc) are compatible with Micro Focus COBOL & AIX COBOL fixed & variable length sequential, Indexed,& Relative file formats. See uvcp documentation at https://uvsoftware.ca/uvcp.htm.

uvsort

 'uvsort' - sort utility for Unix/Linux with mainframe capabilities. The Unix
 system sort can not sort mainframe type files that do not have Linefeeds & may
 have packed decimal sort fields. uvsort has the functionality of mainframe SORT
 - INCLUDE,OMIT,INREC,BUILD,OVERLAY,REFORMAT,FINDREP,SUM,OUTFIL,FNAMES,OUTREC.
 Up to 30 named input files or unlimited via all files in a directory
 Max 60 named output files (with different record selections & reformats)
 or unlimited output files by control break on sort key fields.
 See full doc at uvsoftware.ca/uvsort.htm.

uvlist

'uvlist' - utility for listing text files on laser printers. uvlist scripts are provided to print files at various cpi,lpi,margins,etc. For example, 'uvlp13LD' will print mainframe reports (132 cols x 66 lines) on 8 1/2 by 11 at 13 cpi, 8 lpi, Landscape, Duplex with margins for 3 hole paper. uvlist is great for listing scripts, programs, etc, because the page headings tell you vital info such as: filename, date, userid, filesize, page#, etc. See uvlist documentation at uvsoftware.ca/uvlist.htm.

6. uvcopy

'uvcopy' - the most powerful Vancouver Utility. It is a 'general purpose' (can do anything) data manipulation utility. It gives you the power of assembler without the complexity. It interprets the instruction files for pre-programmed jobs such as SelectJobs & TableJObs. You do not need to know the uvcopy instruction language to run the pre-programmed jobs. But if you want to use it for your own customized applications, you can see the over 100 instructions at uvcopy3.htm. And be sure to see scripts uvfix1 & uvfixA in Part_7, the easy way to use the power of uvcopy instructions without writing uvcopy jobs. See uvcopy documentation at uvsoftware.ca/uvcopy3.htm.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2B1. Basic Vancouver Utilities

uvhd - file investigation utility

uvhd is a file investigation utility that displays any file in vertical hexadecimal, and prompts for commands to browse,search,replace,update,select,print,count,etc. uvhd allows you to see characters not shown by text editors (control characters & high-bit characters that can cause problems in text files). uvhd is interactive with 25 help screens, and is the favorite utility of all my customers. See uvhd documentation at uvsoftware.ca/uvhd.htm.

uvhd demo#1 - find/fix problem CR's in scripts

CR's (Carriage Returns) can cause scripts to fail - a baffling problem, because you can not see anything wrong with the usual unix tools (vi,cat,more,etc). We will demo the problem using the 'llsx' script provided in /home/uvadm/demo/sf/llsx. We assume you have already copied the demo files to your homedir /home/userxx/demo/... as directed on page '1C2'.


 #0a. login userxx --> /home/userxx
 #0b. cd demo      --> /home/userxx/demo

 #0c. cp backup/sf/llsx sf  <-- ensure the bad version of llsx is in demo/sf/...
      ====================      (might have already been fixed by a prior demo)

 #1. sf/llsx sf   <-- execute the demo script (to list files in sf/...)
     ==========     - get following error:

     "bash: sf/llsx: /bin/bash^M: bad interpreter: No such file"
     ===========================================================
      - 1st instinct is to investigate script with 'vi' as follows:

 #2. vi sf/llsx   <-- investigate with 'vi' to see what's wrong
     ==========     - or could use 'cat' or 'more'
      #!/bin/bash
      # llsx - List files in directory, in Size order, Biggest 1st
      ls -lS "$1" | more
      We can not see anything wrong using unix tools, so let's try uvhd.

 #3. uvhd sf/llsx t   <-- using option 't' (for Text files)
     ==============
      rec#=1 rsize=13 fptr=0 fsize=244 records=7
                      10
      r#   1 0123456789012
           0 #!/bin/bash..
             2226662667600
             31F29EF2138DA

You can see the x'0D' (in horizontal hex vs the 'vertical hex' of uvhd) This is the equivalent of the '^M' shown in the error message above. We could now fix the problem using uvhd as shown on the next page.


 #3a. --> q       <-- reply 'q' to quit
          ===       - need to rerun with option 'u' for update

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2B2. uvhd demo#1 - find/fix problem CR's in scripts

Fix problem - Remove Carriage Returns from scripts


 #4. uvhd sf/llsx tu  <-- run with option 't' & 'u' to allow Update
     ===============
      r#    1 0123456789012
            0 #!/bin/bash..
              2226662667600
              31F29EF2138DA

 #4a. --> v99 x'0D',x'20'  <-- search/replace x'0D' with x'20' (space)
          ===============
          v99 x'0D',' '    <-- could code space as ' ' vs x'20'

 #4b. --> 1  <-- enter 1 to return to 1st record in file (to compare)
          ===
      r#    1 012345678901
            0 #!/bin/bash.
              2226662667620  <-- Note CR x'0D' replaced with x'20' (space)
              31F29EF21380A

 #4c. --> q       <-- reply 'q' to quit
          ===

 #5. sf/llsx sf   <-- retry llsx (after removing CR's x'0D's)
     ==========

# llsx - list directory sorted by file-size (biggest files 1st) -rwxrwxr-x 1 uvadm apps 566 Aug 21 14:31 fixaccents2 -rwxrwxr-x 1 uvadm apps 489 Aug 21 14:31 copyjcl3 -rwxrwxr-x 1 uvadm apps 296 Aug 21 14:31 llsy -rwxrwxr-x 1 uvadm apps 244 Aug 21 14:31 llsx

Notes re Updating with uvhd

  1. Must enter option 'u' on the command line if intent to update

  2. 'v' is command to search/replace, 99 of 'v99' is the number of records to process, but '99' has special meaning to process the entire file, resets file pointer to 1st record & processes all records until EOF.

  3. You could remove the CR by replacing with null as follows:

 #4a. --> v99 x'0D',''   <-- search/replace x'0D' with null ''
          ============
  1. Replacing with null works in this case, but should normally be avoided, because uvhd updates "in place", so rewritten records will be shifted left, inserting a blank at the end of the record, following the LineFeed x'0A', so we will get a blank ' ' at the begining of all records after the update - not a problem for scripts, because scripts bypass blanks, But could be a problem for data files.

 #5. This is the short version of the in-depth version on pages '10B1' - 10B11, which
     gives full explanations & multiple ways to correct the problems for 1 script or
     all scripts in a directory.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2C1. uvhd demo#2 - finding High-Bit characters in text files

high-bit characters may cause problems for old applications

Files with high-bit characters may cause problems for older applications. For Example - files with UTF-8 characters input to legacy applications, not designed to handle them. This demo will show you how to identify them with uvhd & convert them to unaccented characters with uvhd or uvcp.

Brief Intro here, More demos later

The original documentation of High-Bit Accented character conversions was moved to '10C1' in Part 10, because it was too long & complex for presentation early in the UVdemos documentation. Here is a quick demo of using 'uvhd' to see the hex coding of some French accents, see all French accents later in Part_20.

dat2/accents3 - UTF-8 accents, short test file


 #0a. login userxx --> /home/userxx
 #0b. cd demo      --> /home/userxx/demo

 #1. cat dat2/accents3      <-- display short test file with French accents
     =================
      déjà vu Thérèse

 #2. uvhd dat2/accents3 t   <-- use uvhd to see hex chars of French accents
     ====================
 rec#=1 rsize=20 fptr=0 fsize=20 records=1
                      10        2
 r#        1 01234567890123456789
           0 d..j.. vu Th..r..se.
             6CA6CA277256CA7CA760
             439A300650483923835A

uvhd shows the French accents are coded in 2 bytes (UTF-8 coding). Here are hte hex codes using horizontal hex vs uvhd vertical hex:

      accent grave a ---> à ---> x'C3A0' (vs unaccented a x'61')
      accent acute e ---> é ---> x'C3A9' (vs unaccented e x'65')
      accent grave e ---> è ---> x'C3A8' (vs unaccented e x'65')

If you have a printed copy of this documentation, you will probably see that the UTF-8 accented chars print differently than the screen display, probably because the printer is not setup for UTF-8 characters.

See the full documentation converting French Accents to unaccented equivalents later in Part_20.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2D1. Basic Vancouver Utilities

uvhd demo#3 - investigating unknown files

'uvhd' should be the 1st tool you think of when you receive a file of unknown format. This example assumes that your organization has both mainframes & unix/linux. Someone sends you a mainframe file in EBCDIC & you attempt to investigate with vi. We have provided such a file in demo/dat2/namesadrs0e. Try the following:


 #0a. login userxx --> /home/userxx
 #0b. cd demo      --> /home/usrxx/demo

 #1. vi dat2/namesadrs0e
     ===================
 ðððñðð@@@@Ö¦…•@㖦•¢…------ etc, 'vi' display all garbage ----->

Now let's try uvhd. 'uvhd' expects you to specify file-type options following the filename. But we do not yet know the file-type & the default is to assume ASCII with record-size 256.


 #2. uvhd dat2/namesadrs0e
     =====================
                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 ......@@@@....@........@@@@@@@....@.......@..@@@@@.....@........
         FFFFFF4444DA894E9A9A8984444444FFFF4C9A989A4D844444D99A84E8989AA8
         0001000000665503665255400000004667086229520940000056938051536455

uvhd displays in 'vertical hexadecimal' 3 line groups with characters on 1st line, hex zones on the 2nd line, & hex digits on the 3rd line. Here we are showing only the 1st 64 bytes (vs actual 256 bytes in 4 groups of 3 lines of 64 bytes each).

The display is still garbage, but we can see that it is an EBCDIC file. The 1st byte is x'F0' (here in horizontal-hex vs vertical-hex above) & x'F0' should be recognized by most professionals as an EBCDIC zero. And bytes 6-9 are x'40's which are EBCDIC blanks.

uvhd display record & line length defaults


 #2a. uvhd dat2/namesadrs0e r256l64  <-- same as #2 above
      =============================    - option 'r' Record length default 256
                                       - option 'l' Display length default 64

The default record length is 256 bytes if you do not specify option 'r###' or option 't' for text files (lines terminated by LineFeed x'0A'). The default display line length is 64 bytes if you do not specify option 'l###'

You can change the defaults by exporting UVHDROP in the profile, for example:


 export UVHDROP=r500l100  <-- change uvhd Run OPtions for record lth to 500
 =======================      & for display line lth to 100

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2D2. uvhd demo - investigating unknown files

uvhd option 'a' translate EBCDIC to ASCII

The next step should be option 'a' which tells uvhd to translate the character line to ASCII (but uvhd will still display EBCDIC codes for zone & digit lines).


 #3. uvhd dat2/namesadrs0e a    <-- option 'a' translates EBCDIC to ASCII
     =======================
                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 000100    Owen Townsend       4667 Hoskins Rd     North Vancouve
         FFFFFF4444DA894E9A9A8984444444FFFF4C9A989A4D844444D99A84E8989AA8
         0001000000665503665255400000004667086229520940000056938051536455
      64 r     BC V5P3V8 Canada    000200    John Horgan         1234 Gov
         944444CC4EFDFEF4C898884444FFFFFF444499894C99889444444444FFFF4C9A
         9000002305573580315141000000020000001685086971500000000012340765
     128 ernment St. Victoria            BC V1P2G3 Canada    000300    Ju
         899989A4EA44E88A9988444444444444CC4EFDFCF4C898884444FFFFFF4444DA
         5954553023B05933699100000000000023051727303151410000000300000014
     192 stin Trudeau      24 Sussex Drive     Ottawa              ON K1Y
         AA894E9A888A444444FF4EAAA8A4C98A844444DAA8A844444444444444DD4DFE
         2395039445140000002402422570499550000063316100000000000000650218

Now you should be able to see that the record-size is 90 bytes because data repeats at 90 bytes - 1st record id# 000100 & 2nd record id# 00020 is 90 bytes higher (64 on 1st line + 26 on 2nd line) (scales are zero-relative).

  1. uvhd option 'a' translates the character line of the 3 line vertical hexadecimal groups from EBCDIC to ASCII, so you can read it before actual data translated.

  2. The actual data is unchanged, the zones & digits lines are still EBCDIC. Example#1 1st byte is '0' & below you see the zone/digit as x'F0' (in horizontal hex vs vertical hex in display above). Example#2 'O' of Owen Townsend is x'D6' in EBCDIC (would be x'4F' in ASCII).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2D3. uvhd demo - investigating unknown files

uvhd options a=EBCDIC->ASCII, r=RecordLth, l=LineLth

From the above uvhd investigations using the default record length of 64, we have determined the the actual record length is 90 bytes, so we can now give uvhd the correct record-size as option 'r90' & the display size as option 'l90'.


 #4. uvhd dat2/namesadrs0e ar90l90
     =============================
 rec#=1 rsize=90 fptr=0 fsize=450 records=5
                 10        20        30        40        50        60        70        80        9
 r#   1 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
      0 000100    Owen Townsend       4667 Hoskins Rd     North Vancouver     BC V5P3V8 Canada
        FFFFFF4444DA894E9A9A8984444444FFFF4C9A989A4D844444D99A84E8989AA8944444CC4EFDFEF4C898884444
        000100000066550366525540000000466708622952094000005693805153645590000023055735803151410000
 rec#=2 rsize=90 fptr=90 fsize=450 records=5
                 10        20        30        40        50        60        70        80        9
 r#   2 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
     90 000200    John Horgan         1234 Government St. Victoria            BC V1P2G3 Canada
        FFFFFF4444D9894C99889444444444FFFF4C9A899989A4EA44E88A9988444444444444CC4EFDFCF4C898884444
        000200000016850869715000000000123407655954553023B05933699100000000000023051727303151410000
 rec#=3 rsize=90 fptr=180 fsize=450 records=5
                 10        20        30        40        50        60        70        80        9
 r#   3 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
    180 000300    Justin Trudeau      24 Sussex Drive     Ottawa              ON K1Y2L6 Canada
        FFFFFF4444DAAA894E9A888A444444FF4EAAA8A4C98A844444DAA8A844444444444444DD4DFEFDF4C898884444
        000300000014239503944514000000240242257049955000006331610000000000000065021823603151410000
 rec#=4 rsize=90 fptr=270 fsize=450 records=5
                 10        20        30        40        50        60        70        80        9
 r#   4 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
    270 000400    Donald Trump        1600 Pennsylvannia  Washington          DC 00001 USA
        FFFFFF4444C998984E9A9944444444FFFF4D899AA9A8998844E8A8898A994444444444CC4FFFFF4EEC44444444
        000400000046513403944700000000160007555283515591006128957365000000000043000001042100000000
 rec#=5 rsize=90 fptr=360 fsize=450 records=5
                 10        20        30        40        50        60        70        80        9
 r#   5 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
    360 000500    Bill Gates          1 Microsoft Way     Redmond             WA 98052 USA
        FFFFFF4444C8994C8A8A4444444444F4D8899A98A4E8A44444D8899984444444444444EC4FFFFF4EEC44444444
        000500000029330713520000000000104939626630618000009544654000000000000061098052042100000000

Now that we know the record size of our file, we can process with 'uvcp' (as on page '2E1').

updating packed/binary files with uvhd

 Also see page '5M3' for another uvhd demo to update packed/binary files with uvhd
 - hard to do with 'vi' & dangerous because it is too easy to add or delete bytes
 with 'vi', which would destroy these fixed length packed/binary files.

The 'u' update command for 'uvhd' will never update outside the field defined & never shift records as is possible with 'vi' or with the 'v' command for uvhd shown in #4a. on page '2B2'.

Page '5M3' updated custmas2 to make a few bytes different in record #10 of 32 records before comparing with custmas1 to demo 'uvcmpFA1' compare packed/binary files (not possible with unix 'diff').

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2E1. Basic Vancouver Utilities

uvcp demo - translate to ASCII (& insert LineFeeds)


 #5. uvcp "fili1=dat2/namesadrs0e,typ=RSF,rcs=90,filo1=tmp1/namesadrs1f,tra=0(90)"
     =============================================================================
      - copies dat2/namesadrs0e to tmp1/namesadrs1f, translating EBCDIC to ASCII

uvcp - translate to ASCII & insert LineFeeds at same time


 #6. uvcp "fili1=dat2/namesadrs0e,typ=RSF,rcs=90,tra=0(90),filo1=tmp1/namesadrs2t,typ=LSTt"
     ======================================================================================

 #7. uvhd tmp1/namesadrs2t t
     =======================
     rec#=1 rsize=87 fptr=0 fsize=427 records=5
                 10        20        30        40        50        60        70        80
 r#   1 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456
      0 000100    Owen Townsend       4667 Hoskins Rd     North Vancouver     BC V7K2R3 Canada.
        333333222247662567676662222222333324676667256222224677625666677672222244253535324666660
        0001000000F75E04F7E35E40000000466708F3B9E302400000EF248061E3F565200000230650368031E141A
     rec#=2 rsize=87 fptr=87 fsize=427 records=5
                 10        20        30        40        50        60        70        80
 r#   2 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456
     87 000200    John Horgan         1234 Government St. Victoria            BC V1P2G3 Canada.
        333333222266662467666222222222333324676766667257225667676622222222222244253534324666660
        0002000000AF8E08F271E000000000123407F652ED5E4034E06934F291000000000000230610273031E141A

Notes re translate to ASCII & insert LineFeeds

  1. We are showing only 1st 2 records in the ASCII file with LineFeeds inserted. Compare to the original EBCDIC file shown on the previous page '2D3'.

  2. Confirm that this is ASCII, a zero is now x'30' in ASCII vs x'F0' in EBCDIC, letter 'O' of 'Owen' is now x'4F' in ASCII vs x'D6 in EBCDIC.

  3. Note that records are terminated with a LineFeed x'0A' after the last nonblank, vs the original EBCDIC file records exactly 90 bytes with no termiantors.

  4. Input file 'typ=RSF' means 'Record Sequential Fixed' & Output file 'typ=LST' means 'Line Sequential Termianted'.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2F1. uvsort demo - sort with record selection

'uvsort' is a sort utility for Unix/Linux with mainframe capabilities. The Unix system sort can not sort mainframe type files that do not have Linefeeds & may have packed decimal sort fields. uvsort has the functionality of mainframe SORT (record select,reformat,etc). Full doc at uvsoftware.ca/uvsort.htm.

We will demo sorting dat2/sales3 on salesman & product, selecting records with sales amount over $1000 (amount field cols 54-62 8 digits & sign).

dat2/sales3 - input file

          1         2         3         4         5         6
 1234567890123456789012345678901234567890123456789012345678901234
 CUST#    SLM  DATE   INV#     PROD#     QTY   PRICE    AMOUNT
 130140    21 940802 IN111001  HAM001  00020+ 0000001 00000020+
 130140    21 940803 IN111001  SCR012  00021+ 0001001 00021021+
 130140    21 940804 IN111001  CHR001  00022+ 0002001 00044022+
 139923    35 950802 IN111002  TAB013  00023+ 0003001 00069023+
 139923    35 950807 IN111002  TAB013  00024+ 0004001 00096024+
 150825    44 960804 IN1122    HAM001  00025+ 0005001 00125025+
 150825    44 960805 IN1122    HAX129  00026+ 0006001 00156026+
 201120    44 970807 CR5234    BBQ001  00027+ 0007001 00189027+
 223240    65 980816 CR955     HAM001  00028+ 0008001 00224028+
 223240    65 980817 IN441     BBQ001  00029+ 0009001 00261029+
 308685    21 990812 IN6605    SAW051  00010- 0000001 00000010-
 308685    21 990813 IN6605    BIT075  00011- 0001001 00011011-
 308685    21 990814 CR8835    TAB013  00012- 0002001 00024012-
 315512    44 000805 IN2251    HAM001  00013- 0003001 00039013-
 315512    44 000806 IN2251    SAW051  00014- 0004001 00056014-
 315512    44 000807 IN2255    BIT075  00015- 0005001 00075015-
 400002    85 010812 CR245     HAX129  00016- 0006001 00096016-
 406082    35 020815 IN33001   BBQ001  00017- 0007001 00119017-
 406082    35 020816 IN33001   TAB013  00018- 0008001 00144018-
 406082    65 020816 IN441     HAM001  00019- 0009001 00171019-

 uvsort "fili1=dat2/sales3,rcs=64,typ=LST,filo1=tmp/s3,key1=10(2),key2=30(6),sel1=53(6)>001000"
 ==============================================================================================

tmp/s3 - output file

 406082    35 020815 IN33001   BBQ001  00017- 0007001 00119017-
 406082    35 020816 IN33001   TAB013  00018- 0008001 00144018-
 201120    44 970807 CR5234    BBQ001  00027+ 0007001 00189027+
 150825    44 960804 IN1122    HAM001  00025+ 0005001 00125025+
 150825    44 960805 IN1122    HAX129  00026+ 0006001 00156026+
 223240    65 980817 IN441     BBQ001  00029+ 0009001 00261029+
 406082    65 020816 IN441     HAM001  00019- 0009001 00171019-
 223240    65 980816 CR955     HAM001  00028+ 0008001 00224028+
 Keys & field selection locations are specified as zero relative displacements.
 For example 1st sort key in columns 11-12 is specified as 'key1=10(2)'
 - displacement 10 & length 2.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2F2. uvsort demo - sort with record selection


 uvsort "fili1=dat2/sales3,rcs=64,typ=LST,filo1=tmp/s3,key1=10(2),key2=30(6),sel1=53(6)>001000"
 ==============================================================================================
  - uvsort demo

 sort -key=1.11,1.12 -key=1.31,1.36 dat2/sales3 -o tmp/s3
 ========================================================
  - unix sort equivalent, wwithout record selection

uvsort advantages vs unix system sort

  1. uvsort has the functionality of mainframe SORT (record select,reformat,etc). (unix sort has no record selection or record reformat capability).

  2. We think the uvsort key syntax is more intuitive 'key1=10(2)' vs '-key=1.11,1.12'

  3. uvsort can sort several file types: text, fixed length with no LineFeeds, variable (RDW/VSAM 4 byte header with record length in binary), STL (STandard Language file system used by AIX COBOL).

  4. Note that unix sort can sort only text files (with LineFeeds).

  5. uvsort allows for 20 input files &/or all files in a directory Unix sort also allows multi file input & all files in directory

  6. uvsort allows up to 60 output files (with different record selections)

  7. uvsort allows sorting on packed decimal fields

  8. uvsort provides an extensive instruction set to reformat record fields - mvc,mvn,clr,add,sub,mpy,div,rep,sqz,cmc,cmn,skp

  9. uvsort provides translate options (ASCII,EBCDIC,UPPER,lower)

See full documentation at uvsoftware.ca/uvsort.htm.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2G1. uvlist - Program Description

uvlist will list ascii text files (stdout by default) but with the main intention of piping to a laser printer. Many scripts/options are provided for laser printers. For example you can list 132 column reports on 8 1/2 width paper using the 'uvlp18' script which includes the option to print at 18 characters per inch. Script 'uvlp13LD' will print 132 columns across & 66 lines down Landscape & Duplex.

uvlist will help you get organized & stay organized by allowing you to file your important listings in standard 8 1/2 x 11 3 ring binders & by ensuring that all listings have page headings that identify the file name & the last modification date.

Uvlist saves you the frustration that can occur when you pick up a listing without page headings to identify where it came from & when it was created.

My preference for printing scripts, programs, parameter files, etc is 'uvlp13D' which prints Duplex at 13 cpi, which allows 95 characters across on 8 1/2 paper with margins to allow for 3 hole punched paper. For example, I would print the uvlist documentation as follows & note the page headings created:


 uvlp13D doc/uvlist.doc  <-- print uvlist documentation at 13 cpi Duplex
 ======================    - 1st page headings shown below

 /home/uvadm/doc/uvlist.doc now=190822:1606 uvadm pg#  1
 =======================================================
 Note that the page headings include the vital information
 - filename, date:time printed, userid, page#

'uvlp13D' is just 1 of over 40 scripts with various option combinations. The actual 'uvlist' command coded in the uvlp13D script is as follows:


 uvlist $1 p60$2 a2d1c13n-220$3 | lp $UVLPOPTN $UVLPDEST
 #======================================================

You can see the options described at uvsoftware.ca/uvlist.htm

Note the 'lp' destination option '$UVLPDEST', which is defined in the common_profile & you can override in your .bash_profile defining a network printer near your location.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2G2. uvlist - Vancouver Utility for Laser Printing

'uvlist' scripts available

Best way to see the various uvlp... scripts available is to use the 'uvhelp' menu & the 'v31' sub-menu as follows:


 #1. Login userxx --> /home/userxx

 #2. uvhelp   <-- uvhelp menu (documented in Part_6), shows many sub-menus
     ======     - v11,v12,v13,v14,v15, v21-v26, v31,v32,etc,... v92
                - scroll down to find desired sub-menu & quit main menu

 #3. v31      <-- sub-menu for uvlist, uvlp... scripts,& listall1
     ===
uvlp12
  • print at 12 cpi (90 chars on 8 1/2" wide)
uvlp13
  • print at 13 cpi (100 chars on 8 1/2" wide)
  • my favorite for printing scripts & short programs
uvlp13D
  • print Duplex at 12cpi
  • recommended for Vancouver Utility documentation
uvlp14
  • print at 14 cpi (110 chars on 8 1/2" wide)
uvlp16
  • print at 14 cpi (120 chars on 8 1/2" wide)
uvlp18
  • print at 18 cpi (135 chars on 8 1/2" wide)
uvlp12L
  • print LANDSCAPE at 12 cpi (128 chars on 11" wide)
uvlp13L
  • print LANDSCAPE at 12 cpi (135 chars on 11" wide)
uvlp14L
  • print LANDSCAPE at 14 cpi (150 chars on 11" wide)
uvlp18L
  • print LANDSCAPE at 18 cpi (180 chars on 11" wide)
uvlp20L
  • print LANDSCAPE at 18 cpi (200 chars on 11" wide)
uvlp12LD
  • LANDSCAPE & DUPLEX at 12 cpi (90 chars & 60 lines)
uvlp13LD
  • Landscape Duplex, 13 cpi, 8 lpi
uvlp13LA
  • Landscape Automatic
  • Simplex if lines < 1000, Duplex if lines > 1000
uvlpd12
  • print all files in a Directory at 12 cpi
uvlpd13
  • print all files in a Directory at 13 cpi
listall1
  • script to print multiple small files, combines as many
    small files as fits on a page & knows when next file
    won't fit & goes to newpage

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2G3. uvlist - Vancouver Utility for Laser Printing

listall1 - print multiple small files - demo


 listall1 archive/uvbriefs    <-- Example #1 (no report shown)
 =========================
  - list ALL LINES of ALL files in directory: archive/uvbriefs/
  - options default to omit directory file list & print all lines

 listall1 archive/uvbriefs 'uv*' d1f3l3  <-- Example#2 (with report)
 ======================================
  - list files in archive/uvbriefs/ (arg1) with prefix 'uv' (arg2)
  - arg3 option 'd1' print directory, option 'f3' print 1st 3 files,
    & option 'l3' print only 1st 3 lines of each file

output report - in tmp1/archive_uvbriefs_uv_d1f3l3

 File#   Lines Directory/Filename
     1       8 archive/uvbriefs//uvcopy
     2       6 archive/uvbriefs//uvcp
     3       6 archive/uvbriefs//uvhd
     4       7 archive/uvbriefs//uvlist
     5       8 archive/uvbriefs//uvsort
     5 files,      35 total lines in  files in directory archive/uvbriefs/
 FILE# 1 - archive/uvbriefs//uvcopy
 'uvcopy' - the most powerful Vancouver Utility. It is a 'general purpose'
 (can do anything) data manipulation utility. It gives you the power of assembler
 without the complexity. It interprets the instruction files for pre-programmed
           --- 3 lines printed of 8 total ---
 FILE# 2 - archive/uvbriefs//uvcp
 'uvcp' - command line data utility that can copy files, converting record
 formats between: fixed, variable, sequential, indexed, text, RDW, etc).
 uvcp provides record select, reformat, translate, etc.
           --- 3 lines printed of 6 total ---
 FILE# 3 - archive/uvbriefs//uvhd
 'uvhd' - file investigation utility that displays any file in vertical hexadecimal,
 and prompts for commands to browse,search,replace,update,select,print,count,etc.
 uvhd allows you to see characters not shown by text editors (control characters
           --- 3 lines printed of 6 total ---

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2H1. uvcopy - most pwerful Vancouver Utility

'uvcopy' is the most powerful Vancouver Utility. It is a 'general purpose' (can do anything) data manipulation utility. It gives you the power of assembler without the complexity. It interprets the instruction files for pre-programmed jobs such as SelectJobs & TableJObs. You do not need to know the uvcopy instruction language to run the pre-programmed jobs. But if you want to use it for your own customized applications, you can see the over 100 instructions at uvcopy3.htm. And be sure to see scripts uvfix1 & uvfixA in Part_7, the easy way to use the power of uvcopy instructions without writing uvcopy jobs.

uvcopy demo - removing CRs from scripts

Page '2B1' explained the problem of Carriage Returns causing scripts to fail, and a simple solution using uvhd. Here is the uvcopy job equivalent.

      # removeCR_uvcopy - copy a file removing any CR's
      #                 - by Owen Townsend, UV Software, Feb 21/2019
      # uvcopy pf/removeCR_uvcopy    <-- execution command
      # =========================
      opr='$jobname - copy a file removing any Carriage Returns'
      fili1=?sf/llsx,rcs=256,typ=LST        #<-- declare input file
      filo1=?tmp/llsx,rcs=256,typ=LSTtp775  #<-- declare output file
      @run
              opn     all                    open files
      # begin loop to read/process/write records until EOF
      man20   get     fili1,a0               get next record into area 'a'
              skp>    man90                  (cc set > at EOF)
              mvc     b0(256),a0             copy input area 'a' to output area 'b'
      #===============================
              rep     b0(256),x'0D',''       remove CRs
      #===============================
              put     filo1,b0               write record to output file
              skp     man20                  return to get next record
      # EOF - close files & end job
      man90   cls     all
              eoj

The solution is only the 1 line --> rep b0(256),x'0D','' <--, but uvcopy requires the framework of file definitions & get/put instruction loop.

We provide a pre-programmed uvcopy job 'uvfix1' that prompts for instructions & saves you from having to write the uvcopy file I/O framework. So you would enter only the 'rep' instruction at the prompt:


 --> rep b0(256),x'0D',''
     ====================

See the over 100 uvcopy instructions documented at uvcopy3.htm.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_3. Vancouver Utility Programming Aids - SelectJobs

SelectJobs Benefits to Programmers & Analysts

SelectJobs are invaluable to IT professionals at any Unix/Linux/Windows site because:

  1. They are much more powerful than grep because they can search for multiple patterns with 1 pass, using AND & OR relationships.

  2. You can specify a series of up to 9 patterns that must be present or not present (AND relationships) and you can have multiple such series (OR relationships).

  3. You can specify that the multiple patterns must be found or not found on the same line or on any line in the file, or on 1 or any line between a begining & ending line, that are themselves identified by 1 or multiple patterns.

  4. You can use these tools to investigate problems in text files of any kind: data files, scripts (SQLs, JCLs, FTPs, etc),& any programming language.

  5. You can use Selectjobs to research the impacts of planned changes to your scripts & programs.

  6. SelectJobs create reports in subdir selects/... of the current working directory, automatically named from the file or directory concatenated with patterns used. This allows you to review past searches & easily modify & repeat the search.

  7. The search reports include all relevant info (files,dirs,search patterns,date/time, user,host,etc) to help you document problems & plan proposed changes.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

SelectJobs (UVdemos Part 3) - Contents


3A1. Select Jobs - Introduction & Practical Examples

3B1. Vancouver Utilities - subdirs relevant to SelectJobs

3B2. Copy $UV/demo/* to demo/ subdir in your homedir
- so you can run SELECT job demos from your demo/... subdir

3B3. Setup check-list - Are you Ready to run the test/demos ?

3C1. selectlf1 - select lines from 1 file identified by up to 9 patterns
matching and/or not-matching datafile contents
- First Example to serve as basis for pattern definition Rules

3D1. Test/Demo selectlf1 to illustrate Pattern Rules (common to all select jobs)
- 10 sample patterns for use with testfile dat1/nameadrs1
  (testdata only 8 records so you can easily check selections correct or not)
- prefix patterns with '+' for presence (default) & '-' for absence
- may specify multiple patterns for 1 selection via sequence#
  example: arg1=+1@Townsend:+2@Canada,-3@Owen
- may restrict search area: arg1=@0[20]Townsend:@20[60]Owen
- output reports created in ./selects/... subdir
  named by concatenating input directory & patterns

3D3. select_nameadrs1 script to run above 10 test/demos automatically (vs keyins)

3D4. Special Character substitutions
- use '^' circumflex for space & '%' percent for commas
- required when patterns coded on command line arg1=...
- not required if you enter patterns at the prompt
 Reports automatically written to selects/ subdir
- named by concatenating input filename & patterns

3E1. Test/Demo selectlf1 using dat1/CanadaMPs Canadian Members of Parliament (335 records)
- higher volume demo file (vs 8 record dat1/nameadrs1)
- a few suggested patterns, then make up your own patterns

3F1. selectlfd1 - select lines from all files in a directory
- same selection rules as for selectlf1

3G1. selectlgfd1 - select line Groups from all Files in a Directory
- same selection rules as for selectlf1
- begin group patterns specified via arg1=...
- end group patterns specified via arg2=...
- optional group qualify patterns specified via arg3=...
(selects all lines of group if arg3 absent)

3G2. selectlgfd2 - same as selectlgfd1, but arg3=... searches for patterns
on any line in the group (vs on the same line)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part 3 SelectJobs - Contents (continued)


3H1. selectlgfd1 - Example#2 select JCL SYSINs with AND & OR conditions
- We need to inspect SORT INCLUDE/OMIT CONDitions with both "AND"s & "OR"s
  which may need manual corrections.

3H2. selectlgfd2 - Example#2 select JCL SYSINs with AND & OR conditions (BETTER)
- selectlgfd2 can select ONLY the SYSINs with BOTH ANDs & ORs
  vs selectlgfd1 which could only select SYSINs with EITHER AND or OR

3I1. selectfdd1 - select Files from 1 directory to a 2nd directory
- based on multiple patterns on the same line or any line
  Note difference 'selectfdd1' (vs prior selectlf1,selectlfd1,selectlgfd1/2)
- selectfdd1 selects the entire file to an output directory
  vs prior select jobs that created reports in the selects/... directory

3J1. Select Demos using demo file of US Congress Representatives & Senators
- csv file downloaded & converted to fixed field locations to allow
  precise selections & selections based on '>' & '<' conditions.
- samples of the 539 records with a column scale for field locations.
- sample selections with results expected that you can run for self training.

SelectJobs same as UVdemos Part 3

SelectJobs.htm is the same as UVdemos.htm#Part_3. SelectJobs was duplicated as a separate document because it is 1 of the most useful parts of UVdemos.


TableJobs.htm (same as UVdemos.htm#Part_4) is also a separate document.

UVscripts.htm (same as UVdemos.htm#Part_5) is also a separate document.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3A1. Vancouver Utility Programming Aids - Selectjobs

SelectJobs - Introduction & Overview

Select Jobs (selectlf1,selectlfd1,selectlgfd1,etc)are pre-programmed uvcopy jobs to select lines from 1 or all files in a directory, lines identified by up to 9 patterns, matching and/or not-matching datafile contents.

You may prefix patterns with '+' for presence & '-' for absence. You may specify multiple patterns for 1 selection via a sequence#, for example:


 arg1=+1@Townsend:+2@Canada,-3@Owen   <-- multi conditions +present & -absent
 ==================================

You may restrict the search area by displacement[length], for example:


 arg1=+1@0[20]Townsend:+2@20[60]Owen  <-- Townsend in cols 1-20 & Owen in 21-80
 ===================================

This document provides demo files & several suggested search patterns. Please run the demos & check your results to the expected.

We think you will appreciate that these select jobs are much more powerful than 'grep' because they can search for multiple conditional patterns with 1 pass. After the demos & understanding how the patterns & conditions are specified, please try out these select jobs on your files.

You might have a directory of thousands of scripts & you need to select some scripts (for investigation & correction) based on some combination of patterns which may be on the same line or on any line in the script.

Use 'selectlfd1' if the multiple patterns must be on the same line, or use 'selectlgfd2' if the multiple patterns could be anywhere in the file, or if the patterns must be between 2 other lines - start search & stop search lines which can also be identified by up to 9 patterns

For example, we can use selectlgfd2 to search SQL 'SELECT' statements which can be multi-lines, but known to end with a ';' semi-colon. arg1=SELECT,arg2=; & arg3=desired search patterns between arg1 & arg2 (begin & end).

If you have a problem with your files & are wondering how it could be solved with Vancouver Utilities, we invite you to email us, explain the problem, and we will help you solve it (as our time allows).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3A2. SelectJobs - Introduction & Overview

Example to Illustrate SelectJob Capabilities vs grep

JCL conversions of complex SORTs with INCLUDE/OMIT AND/OR conditions may need manual corrections & need to be inspected. I wanted to scan hundreds of JCLs extracting only the SORT SYSIN parameters with both "AND" & "OR" CONDitions.

'grep' can not do this because the ANDs & ORs could be on different lines. Here is the command for 'selectlgfd2' which can look for //SYSIN to /* groups with both "AND" & "OR" in between.


 uvcopy selectlgfd2,fild1=jcl2,arg1=@//SYSIN,arg2=/*,arg3=1@COND:2@AND:3@OR%
 ============================================================================
  - scan for //SYSIN to /* line groups with "AND", "OR", "CONDitions"
  - creates report in subdir selects/... named from directory + patterns as shown below:
 # Report: selects/jcl2_SYSIN_1@COND:2@AND:3@OR%
 # ---> uvcopy selectlgfd2,fild1=jcl2,arg1=//SYSIN,arg2=/*,arg3=1@COND:2@AND:3@OR%
 #      =========================================================================
 # format: uvcopy selectlgfd2,fild1=directory,arg1=...,arg2=...,arg3=...,arg4=...,uop=abcdgi
 # arg1 patterns identify begin-group line, arg2 patterns identify end-group line
 # arg3 patterns qualify group selection, arg4 patterns qualify lines to output/print
 #  - code '^' circumflex for ' ' spaces and '%' percent for ',' commas in patterns
 #  - may prefix patterns: '-' for absence, '+' for presence (default)
 #  - may restrict search area: arg2=@0[2]/*  <-- "/*" must be in 1st 2 bytes
 #  - multiple-condition-series: arg3=1@SORT:2@COND:3@AND:4@OR <-- ALL patterns must match
 # Date=2019/06/09_10:29:01, Site=UV_Software, Host=uvsoft5, User=uvadm, Options=
 #===============================================================================
 //SYSIN    DD *
 SORT FIELDS=(1,8,CH,A,69,12,CH,A)
 INCLUDE COND=(((9,1,CH,EQ,C'T'),OR,(COND=(9,1,CH,EQ,C'X'))),
                                 AND,(COND=(56,2,CH,EQ,C'AB')))
 /*
                          EOG: #1, 5 lines selected from jcl2/jgl232
                          EOF:     5 selected from 35 in jcl2/jgl232
 EOD: 7 files, 168 lines, selected 1 groups, 5 lines to selects/jcl2_SYSIN_1@COND:2@AND:3@OR%

SelectJob Advantages

  1. Can select lines based on up to 9 patterns that must be present or absent.

  2. Can select groups of lines (identified by patterns on a start-line & an end-line), that include multiple patterns on any line between the start & end lines.

  3. SelectJobs create reports in subdir selects/... automatically named from the file or directory concatenated with patterns used. This allows you to review your past searches & see the command used for easy modify & repeat.

  4. You may restrict pattern searches to record areas, for example "arg1=1@0[20]Owen,2@40[40]Vancouver" searches for "Owen" anywhere in 1st 20 bytes and "Vancouver" anywhere in columns 41-80. For the JCL/SORT example we could have "arg1=@0[7]//SYSIN,arg2=@0[2]/*" since we know these patterns start in column 1.

  5. SelectJobs can use '>' & '<' conditions, but only if the field is in a fixed position. See examples later in this documentation on page '3E3'.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B1. Vancouver Utility Programming Aids - Selectjobs

Vancouver Utilities - subdirs relevant to SelectJobs

Here is the Vancouver Utilities homedir showing only the most relevant subdirs required for the SelectJob demos documented in this section of UVdemos.htm.

You could run the demos logged in as 'uvadm', but if you are sharing the system with other uvadm users, it would be better to copy the demo/subdirs/files to your own homedir. See the copy instructions below & an expanded view of the demo/... files on the next page.

 /home/uvadm             <-- $UV
 :-----bin                 - UV binaries (uvcopy,uvsort,uvcp,uvhd,uvlist,etc)
 :-----dat1                - test data files to demo various Vancouver Utilities
 :     :-----...         <-- 200 testdata files for all uvcopy job demos
 :     :                   - SelectJobs testdata files copied to demo/dat1/...
 :-----doc                 - Vancouver Utilities documentation (text)
 :     :----UVdemos.doc    - this documentation (1 of 150 files)
 :     :----SelectJobs.doc - Part3 of UVdemos duplicated as a separate document
 :-----dochtml             - documentation in HTML (uploaded to www.uvsoftware.ca)
 :-***-demo              <-- demo files for TestDemo tutorials, copy to your homedir -->
 :     :                   - copy /home/uvadm/demo/* to your /home/userxx/demo/...
 :     :-----dat1          - data files for SelectJob demos, see details on next page
 :     :-----ftps          - SYSIN control cards for various purposes
 :     :-----jcl2          - mainframe JCL samples (8)
 :     :-----sqls          - SQL scripts for SelectJob demos, see next page -->
 :-----pf                <-- Parameter Files for uvcopy
 :     :-----util              - uvcopy SelectJobs
 :     :     :-----selectlf1     - select Lines from 1 File
 :     :     :-----selectlfd1    - select Lines from all Files in a directory
 :     :     :-----selectlgfd1   - select Line Groups from all Files in directory
 :     :     :-----selectlgfd2   - select Line Groups from all Files in directory
 :     :     :-----selectfdd1    - select Files from 1 Directory to a 2nd Directory
 :     :-----adm               - uvcopy jobs for uvadm admin
 :     :-----demo              - uvcopy jobs for various demos
 :     :-----IBM               - uvcopy jobs for Mainframe conversions
 :-----sf                <-- Script Files (bash or Korn shell)
 :     :-----adm               - scripts for uvadm admin
 :     :-----demo              - demo scripts
 :     :-----util              - utility scripts
 :     :-----IBM               - scripts for mainframe conversions
 :-----src               <-- Vancouver Utilities C source code (uvcopy,uvsort,etc)
 :-----tf                  - test files for various demos (similar to dat1/...)
 :-----tmp                 - tmp subdir (test/demo outputs)
 :-----tmp1                - misc, outdir for some utility scripts (like listall1)
 :-----tmp2

Please copy the demo/... files to your homedir to avoid conflict with other users of Vancouver Utilities & to avoid losing your files when new versions of Vancouver Utilities are installed.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B2. Vancouver Utility Programming Aids - Selectjobs

copy $UV/demo to your $HOME/demo/...


 #1. Login userxx --> /home/userxx
 #2. mkdir demo   <-- make directory for copy of demo files in your homedir

 #3. cp -r /home/uvadm/demo/* demo/   <-- copy demo files to your homedir
     ==============================
 #4. cd demo    <-- change into demo/ directory
     =======

SelectJobs demo files copied to your homedir

Here is an expanded view of the most relevant subdirs/files required for the SelectJob demos (omitting subdirs/files used for other parts of UVdemos.doc).

 /home/userxx/demo/              - copy to your homedir, cd demo/ & run jobs there
 :-----dat1                    <-- testdata files for demos
 :     :-----nameadrs1           - 8 line testdatafile to demo pattern rules
 :     :-----CanadaMPs.csv       - CAnadian Members of Parliament Names & constituencies
 :     :-----CanadaMPs           - text files in fixed fields extracted from .csv file
 :     :-----UScongress.csv
 :     :-----UScongress
 :     :
 :-----jcl2                    <-- JCLs to demo selectlgfd1/2
 :     :-----jar200.jcl
 :     :-----...etc...
 :-----pf                      <-- uvcopy job Parameter Files, see note below
 :     :-----                    - you could create your own uvcopy jobs here
 :     :
 :-----selects                 <-- output directory for select job reports
 :     :-----dat1_nameadrs1_+1@Townsend:+2@Canada:-3@Owen
 :     :-----...                 - sample above from selectlf1 1st demo
 :     :-----...                 - concatenation of directory_file_patterns
 :-----sqls                    <-- SQL scripts for SelectJob demos
 :     :-----select_Customers_all
 :     :-----select_Customers_sales
 :     :-----...etc...
 :-----tmp
 :-----tmp1                    <-- tmp dirs for temp files, sorts, etc
 :-----tmp2

uvcopy job Locations


 uvcopy selectlf1,fili1=dat1/nameadrs1,arg1=+1@Townsend:-2@Owen
 ==============================================================
 - sample uvcopy execution of 'selectlf1' Parameter File (uvcopy job)
 - How does uvcopy know where to find its Parameter File of instructions ?

uvcopy looks for the Parameter/Instruction File via $PFPATH, similar to scripts using $PATH. The SELECT jobs are stored in /home/uvadm/pf/util/... (as shown on the previous page) & the common_profile includes:


 export PFPATH=$UV/pf/demo:$UV/pf/IBM:$UV/pf/util:$HOME/pf:---etc---
 ====================================================================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B3. Vancouver Utility Programming Aids - Selectjobs

Setup check-list - Ready to run test/demos ?

We will present a 'setup check-list' here, since you may be reading this in the separate SelectJobs.doc vs UVdemos.doc which includes SelectJobs as Part3.

See UVdemos.htm for the complete setup procedures, but here is a check-list of the steps required before you can execute these test/demos.

  1. Install Vancouver Utilities. See complete documenation at install.htm. Also see the install summary at UVdemos.htm#1A3.

  2. Setup your user login account if not already done. See UVdemos.htm#1C1.

  3. Setup subdir demo/ in your homedir & copy $UV/demo/* to your $HOME/demo/... See the short version here on page '3B2' &/or the long version with file-lists file-lists at UVdemos.htm#1C2 & file-contents at UVdemos.htm#1D1 - 1D6.

    testdata file contents

Here in SelectJobs, we will usually show just a sample of the relevant testdata before each test/demo execution. See UVdemos.htm#1D1 - 1D6 to see all files & all contents of the testdatafiles.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3C1. Programming Aids - SelectJobs

dat1/nameadrs1 - testdata file for 1st demo

          1         2         3         4         5         6         7         8
 12345678901234567890123456789012345678901234567890123456789012345678901234567890
 Owen Townsend       4667 Hoskins Rd     North Vancouver, BC Canada V7K2R3
 Jenny Townsend      21 Canada Place     Vancouver, BC       Canada V5P3V8
 John Horgan         123 Townsend Ave.   Townsend City, BC   Canada V1P2G3
 John Townsend       2121 Owen Drive.    St. John, NB        Canada M5W1E6
 John Owens          24 Townsend Dr.     Owenville, ON       Canada M5K3B3
 Donald Trump        1600 Pennsylvannia  Washingtom, DC      USA 00001
 Bill Gates          1 Microsoft Way     Seattle, WA         USA 98052-6399
 Peter Townsend      24 Johnson Road     London, England     UK EC1A 1HQ
Note
  • "Owen" & "Townsend" are repeated in various name & address fields
    so we can practise selecting various combinations from various fields

selectlf1 - 1st Example for Pattern Rules following


 #0b. Login uvadm  --> /home/uvadm  <-- could use uvadm if you are only user
 #0a. Login userxx --> /home/userxx <-- better/safer to copy files to your homedir
 #1.  cd demo            <-- change into demo/ directory
 #2a. rm -f selects/*    <-- remove old selection reports
 #2b. rm -f tmp*/*       <-- remove old temporary files (from tmp/, tmp1/, tmp2/)

 #2. uvcopy selectlf1,fili1=dat1/nameadrs1,arg1=+1@Townsend:+2@Canada:-3@Owen
     ========================================================================
     - select lines from dat1/nameadrs1 with "Townsend" & "Canada", but NOT "Owen"

 #2a. cat    <-- End of Job prompt for command to view output file
      ===      - enter command only (cat,more,vi,etc), no need for filename
               - Result (2 lines) as follows:
 Jenny Townsend      21 Canada Place     Vancouver, BC       Canada V5P3V8
 John Horgan         123 Townsend Ave.   Townsend City, BC   Canada V1P2G3
Note
  • Can you confirm that the 2 lines reported (Jenny Townsend & John Horgan)
    are the only 2 of the 8 input lines that satisfy the selection patterns ???
  • See the dat1/nameadrs1 testdatafile listed at the top of this page.
Note
  • output reports are created in the ./selects/ subdir of current directory
  • named by concatenating input directory + filename + arg1 patterns
  • any characters invalid in filenames such as '/' converted to '_' underscores
  • You may view separately at any time with vi,cat,more,lp,uvlp12,etc

 #3a. vi selects/dat1_nameadrs1_+1@Townsend:+2@Canada:-3@Owen  <-- using 'vi'
      =======================================================

 #3b. cat selects/dat1_nameadrs1_+1@Townsend:+2@Canada:-3@Owen  <-- using 'cat'
      ========================================================
      - 'cat' is convenient when you know the result is less than a screenfull

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3C2. Programming Aids - SelectJobs

Output Report from selectlf1

The prior page showed only the 2 lines selected from the 8 line input file.

Actually, selectlf1 creates a Report with command parameters used, which are useful when you need to review the results or rerun the selection with different parameters. Here is the full report, with the selected lines & stats at the end

selects/dat1_nameadrs1_+1@Townsend:+2@Canada:-3@Owen

 # Report: selects/dat1_nameadrs1_+1@Townsend:+2@Canada:-3@Owen
 # ---> uvcopy selectlf1,fili1=dat1/nameadrs1,arg1=+1@Townsend:+2@Canada:-3@Owen
 #      =========================================================================
 #format: uvcopy selectlf1,fili1=inputfile,arg1=pattern1:pattern2,etc,uop=i0/i1
 # arg1 - line selection conditions (up to 9 patterns to match or not-match)
 # - code '^' circumflex for ' ' spaces and '%' percent for ',' commas in patterns
 # - may prefix patterns with '-' for absence, '+' for presence (default)
 # - may restrict search area, example: arg1=@0[20]Townsend:@20[60]Owen
 # - multiple conditions: arg1=+1@Townsend:+2@Canada,-1@Owen,-2@BC
 # - if no seq#s (+1,+2,etc) default selects if any +pattern & no -pattern
 # - combination ex: arg1=+1@0[20]Townsend:+2@60[60]Canada:-@0[20]Owen
 # Date=2019/05/12_16:14:14, Site=UV_Software, Host=uvsoft5, User=uvadm, Options=
 #===============================================================================
 Jenny Townsend      21 Canada Place     Vancouver, BC       Canada V5P3V8
 John Horgan         123 Townsend Ave.   Townsend City, BC   Canada V1P2G3
 # EOF - 2 lines selected from dat1/nameadrs1

Report Format for SelectJobs

  1. 1st line identifies the report filename (in selects/... subdir by default)

  2. 2nd line shows the uvcopy SelectJob command line arguments

  3. Lines 3-11 are SELECT job format rules & help info

  4. Line 12 shows current date, sitename, hostname, username,& options

  5. Lines 14+ are the selected data lines (only 2 lines in this example)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3D1. Select Jobs - selectlf1, selectlfd1, selectlgfd1

Test/Demos to illustrate Pattern Rules

The Pattern Rules are common to all Select Jobs (selectlf1,selectlfd1,selectlgfd1), but we will use selectlf1 for most examples below. You may run the commands below & confirm your results match our documented selection counts.

Patterns may be entered on the command line via arg1=... OR at the prompt, They are best entered on the command line for easy rerun from command history.

dat1/nameadrs1 - compare demo outputs to these 8 records

          1         2         3         4         5         6         7         8
 12345678901234567890123456789012345678901234567890123456789012345678901234567890
 Owen Townsend       4667 Hoskins Rd     North Vancouver, BC Canada V7K2R3
 Jenny Townsend      21 Canada Place     Vancouver, BC       Canada V5P3V8
 John Horgan         123 Townsend Ave.   Townsend City, BC   Canada V1P2G3
 John Townsend       2121 Owen Drive.    St. John, NB        Canada M5W1E6
 John Owens          24 Townsend Dr.     Owenville, ON       Canada M5K3B3
 Donald Trump        1600 Pennsylvannia  Washingtom, DC      USA 00001
 Bill Gates          1 Microsoft Way     Seattle, WA         USA 98052-6399
 Peter Townsend      24 Johnson Road     London, England     UK EC1A 1HQ

 #1. uvcopy selectlf1,fili1=dat1/nameadrs1,arg1=Townsend:Owen
     ========================================================
     - select lines with "Townsend" OR "Owen" anywhere on the line
      --> enter patterns if not on command line arg1=...
      selects/dat1_nameadrs1_Townsend:Owen default filo02 -->     <-- null accept
      default command null, OR enter: vi,cat,more,etc     --> cat <-- enter cat
           ---------------- 6 of 8 lines are selected --------------
     Owen Townsend       4667 Hoskins Rd     North Vancouver, BC Canada V7K2R3
     Jenny Townsend      21 Canada Place     Vancouver, BC       Canada V5P3V8
     John Horgan         123 Townsend Ave.   Townsend City, BC   Canada V1P2G3
     John Townsend       2121 Owen Drive.    St. John, NB        Canada M5W1E6
     John Owens          24 Townsend Dr.     Owenville, ON       Canada M5K3B3
     Peter Townsend      24 Johnson Road     London, England     UK EC1A 1HQ
Note
  • #1 above specifies the input file (fili1=dat1/nameadrs1)
  • But, we will omit on following examples, since it is the default
  • Here is the revised command, the prompts,& the recommend replies:
  • null to accept the I/O files & 'cat' to display the selected records

 #1. uvcopy selectlf1,arg1=Townsend:Owen  <-- can omit fili1=dat1/nameadrs1 (default)
     ===================================
     - select lines with "Townsend" OR "Owen" anywhere on the line
      dat1/nameadrs1 = default fili01                     -->     <-- null accept
      --> enter patterns if not on command line arg1=...
      selects/dat1_nameadrs1_Townsend:Owen default filo02 -->     <-- null accept
      default command null, OR enter: vi,cat,more,etc     --> cat <-- enter cat
Note
  • We will not show these prompts & replies on the following examples
  • But please enter the 2 nulls & 'cat' when prompted

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3D2. Select Jobs - Test/Demos to illustrate Pattern Rules


 #2. uvcopy selectlf1,arg1=+Townsend:+Canada:-Owen
     =============================================
     - select lines with "Townsend" OR "Canada", but NOT "Owen"   <-- ERROR, INVALID

You may code 1st byte of pattern as '+' (default) to indicate the pattern must be present and '-' to indicate the pattern must be absent (NOT found). But, this example is INVALID, cannot mix '+' & '-' conditions, unless the '-' conditions are at the end of a conditional series, for example: +1...,+2...,-3...,-4...


 #3. uvcopy selectlf1,arg1=+1@Townsend:+2@Canada:-3@Owen <-- CONDITION SERIES
     ===================================================
     - select lines with "Townsend" AND "Canada", but NOT "Owen"
           ---------------- 2 of 8 lines are selected --------------
     Jenny Townsend      21 Canada Place     Vancouver, BC       Canada V5P3V8
     John Horgan         123 Townsend Ave.   Townsend City, BC   Canada V1P2G3

The '+1' indicates the First condition & '+2' the Second condition of a series (9 max). Match assumed if any '+' pattern found, then negated if any '-' pattern found. You must code '+' patterns prior to any '-' patterns in a series 1/2/3. You could omit the '+' condition prefixes since that is the default


 #4. arg1=+1@0[60]Townsend:+2@60[20]Canada:-3@0[20]Owen  <-- RESTRICTED SEARCH AREAS
     ==================================================
     - match if "Townsend" in 0-59 & "Canada" in 60-79, But not if "Owen" in 0-19.
           ---------------- 3 of 8 lines are selected --------------
     Jenny Townsend      21 Canada Place     Vancouver, BC       Canada V5P3V8
     John Horgan         123 Townsend Ave.   Townsend City, BC   Canada V1P2G3
     John Townsend       2121 Owen Drive.    St. John, NB        Canada M5W1E6

You may specify "Restricted Search Areas" by coding the Displacement[Length] following the '@' identifier.


 #5. arg1=+1@Townsend:+2@Canada:-3@Owen:+1@Townsend:+2@UK:-3@Owen <-- MULTI-SERIES 1,2,3,1,2,3
     ===========================================================
     - match if "Townsend" AND "Canada" & NOT "Owen"
     - AND   if "Townsend" AND "UK" & NOT "Owen"
           ---------------- 2 of 8 lines are selected --------------
     Jenny Townsend      21 Canada Place     Vancouver, BC       Canada V5P3V8
     John Horgan         123 Townsend Ave.   Townsend City, BC   Canada V1P2G3
     Peter Townsend      24 Johnson Road     London, England     UK EC1A 1HQ

 #6. uvcopy selectlf1,arg1=-Townsend           <-- NOT present
     ===============================
     - select lines WITHOUT "Townsend" (anywhere in record)
           ---------------- 3 of 8 lines are selected --------------
     John Owens          24 Townsend Dr.     Owenville, ON       Canada M5K3B3
     Donald Trump        1600 Pennsylvannia  Washingtom, DC      USA 00001
     Bill Gates          1 Microsoft Way     Seattle, WA         USA 98052-6399

 #7. uvcopy selectlf1,arg1=-1@Townsend:-2@Owen  <-- SERIES NOT present
     =========================================
     - select lines WITHOUT "Townsend" & without "Owen"
           ---------------- 3 of 8 lines are selected --------------
     Donald Trump        1600 Pennsylvannia  Washingtom, DC      USA 00001
     Bill Gates          1 Microsoft Way     Seattle, WA         USA 98052-6399

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3D3. Select Jobs - Test/Demos to illustrate Pattern Rules

script to run 11 selectlf1 test/demos

 # select_nameadrs1 - script stored at $UV/demo/sf/select_nameadrs1
 #                  - to test uvcopy $UV/pf/util/selectlf1
 #                  - by Owen Townsend, UV Software, May13/2019
 # - run selectlf1 with various selection patterns, reports collectedin ./selects/...
 #              --------- testdatafile -----------
 # Owen Townsend       4667 Hoskins Rd     North Vancouver, BC Canada V7K2R3
 # Jenny Townsend      21 Canada Place     Vancouver, BC       Canada V5P3V8
 # John Horgan         123 Townsend Ave.   Townsend City, BC   Canada V1P2G3
 # John Townsend       2121 Owen Drive.    St. John, NB        Canada M5W1E6
 # John Owens          24 Owen Avenue      Owenville, ON       Canada M5K3B3
 # Donald Trump        1600 Pennsylvannia  Washingtom, DC      USA 00001
 # Bill Gates          1 Microsoft Way     Seattle, WA         USA 98052-6399
 # Peter Townsend      24 Johnson Road     London, England     UK EC1A 1HQ
 #
 export UVCOPYROP=q0i7r0  # inhibit prompts
 # default fili1=dat1/nameadrs1 (omitted here to save space)
 uvcopy selectlf1,arg1=Townsend:Owen                  #1 - any match
 uvcopy selectlf1,arg1=+1@Townsend:+2@Canada:-3@Owen  #2 - multi-condition match
 uvcopy selectlf1,arg1=+1@0[20]Townsend:+2@60[20]Canada:-3@0[60]Owen #3(1) - restricted areas
 uvcopy selectlf1,arg1=+1@0[20]Townsend:+2@60[20]Canada:-3@0[60]Owen\
 :+1@0[20]Townsend:+2@60[20]UK:-3@0[60]Owen           #5 - multi-condition-series
 uvcopy selectlf1,arg1=-Townsend                      #6 - NOT condition
 uvcopy selectlf1,arg1=-1@Townsend:-2@Owen            #7 - NOT multi-conditions
 uvcopy selectlf1,arg1=@0[4]>Owen                     #8 - > or < condition
 # following will be rejected with errmsgs
 uvcopy selectlf1,arg1=+Townsend:+Canada:-Owen        #1 ERR: mixed +/-, - not end series
 uvcopy selectlf1,arg1=-Townsend:+Owen                #2 ERR: mixed #1 '-' & any '+'
 uvcopy selectlf1,arg1=-Townsend:-owen                #3 ERR: multi #1 '-'s
 uvcopy selectlf1,arg1=-1@Townsend:+2@Owen            #4 ERR: #1 '-' with any '+'

executing sf/adm/select_nameadrs1


 #0b. Login uvadm  --> /home/uvadm  <-- could use uvadm if you are only user
 #0a. Login userxx --> /home/userxx <-- better/safer to copy files to your homedir

 #1. rm -f selects/*    <-- remove old selection reports

 #2. select_nameadrs1      <-- execute script (no arguments required)
     ================        - will get multiple prompts for input file accept/change
      dat1/nameadrs1 = default fili01                     -->     <-- null accept
      --> enter patterns if not on command line arg1=...
      selects/dat1_nameadrs1_Townsend:Owen default filo02 -->     <-- null accept
      default command null, OR enter: vi,cat,more,etc     --> cat <-- enter cat

For each of the 11 runs in the script, you can reply null to accept the default filenames and then reply "cat" to see the output records selected.

OR, you could reply null to all prompts & then inspect output reports in selects/...


 #3. vi selects/*       <-- view all 11 output reports
     ============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3D4. Select Jobs - Test/Demos to illustrate Pattern Rules

Special Characters in arg1 patterns

  1. You can not code ' ' spaces in the arg1 patterns (for testsst2 & select jobs below), because these jobs convert spaces to tildes when storing patterns in the table, & because the uvcopy command line does not allow embedded spaces. BUT, you may code ' ' spaces as '^' circumflexes, which will be converted to ' ' spaces.

  2. You can not code ',' commas in arg1 patterns, because the uvcopy command line uses ',' commas to separate arguments. BUT, you may code commas as '%' percents, which will be converted to ',' commas.

    reports written to selects/... subdir

  3. By default, the SELECT jobs create their output reports in ./selects/... (subdir in the current directory).

  4. The output report filename defaults to a concatenation of the input directory, and the arg1/2/3 patterns, with '/'s converted to '_' underscores. Note in the above example, the report is 'selects/jcl2_SYSIN_1@cond:2@and:3@or'.

  5. If desired, you could specify an alternate output report filename, but the default is very convenient & creates unique filenames identifying the directory searched & the search patterns used.

  6. Note the 14 line report prefix which includes the Report: filename, the uvcopy command that created it, several lines of "help" information, and the current date=..., Site=..., Host=..., User=...,& Options=...

    Try selectlf1 with YOUR PATTERNS

Please try out selectlf1 with your own patterns, You might vary the suggested patterns listed on the previous page in the 'select_nameadrs1' script. See if you can find a pattern that does not give the expected results.


 uvcopy selectlf1,fili1=dat1/nameadrs1,arg1=...YOURPATTERNS...
 =============================================================

Try selectlf1 with YOUR DATA files & PATTERNS

Then try out selectlf1 with your own files & patterns.


 #1. cd your working directory (above your directory of files)

 #2. mkdir selects   <-- make directory for output reports
     =============

 #3. uvcopy selectlf1,fili1=yourdir/YOURDATAFILE,arg1=...YOURPATTERNS...
     ===================================================================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3E1. Select Jobs - *selectlf1, selectlfd1, selectlgfd1

dat1/CanadaMPs - larger test file to demo Select Jobs

$UV/dat1/CanadaMPs is the CAnadian Members of Parliament data file that we can use for tutorials & you can use for self training using these Select Jobs.

We downloaded a spreadsheet, extracted a csv file (dat1/CanadaMPs.csv) & converted to a fixed layout so we could define specific fields required for some selections. See '10D1' if you are interested to see the uvcopy job (demo/pf/CanadaMPscsv2txt).

Here are the samples of the 335 records with a column scale in case you wish to test select job searches with restricted areas.

samples of Canadian MPs .csv converted to fixed fields

 Title,First Name,Last Name,Constituency,Province/Territory,Political Affiliation,Start Date
 ,Ziad,Aboultaif,Edmonton Manning,Alberta,Conservative,10/19/2015
 ,Dan,Albas,Central Okanagan/Similkameen/Nicola,British Columbia,Conservative,10/19/2015
          1         2         3         4         5         6         7         8         9       100
 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
               ----------------- First 2 of 335 MP's ------------------
 001  Aboultaif          Ziad            Edmonton Manning              AB Conservative     2015
 002  Albas              Dan             Central Okanagan/Similkameen/ BC Conservative     2015
               --------------- samples of all parties --------------
 010  Anderson           David           Cypress Hills/Grasslands      SK Conservative     2015
 011  Angus              Charlie         Timmins/James Bay             ON NDP              2015
 023  Barsalou-Duval     Xavier          Pierre-Boucher/Les Patriotes/ QC Bloc Quebecois   2015
 032  Bernier            Maxime          Beauce                        QC People's Party   2015
 202  May                Elizabeth       Saanich/Gulf Islands          BC Green Party      2015
 328  Wilson-Raybould    Jody            Vancouver Granville           BC Independent      2015
               ----------------- Last of 335 MP's ------------------
 335  Zimmer             Bob             Prince George/Peace River/Nor BC Conservative     2015

sample selection from CAnadian Members of Parliament


 uvcopy selectlf1,fili1=dat1/CanadaMPs,arg1=1@AB:2@Liberal
 ========================================================
  - select "Liberal" party members from "AB" (Alberta)
  - report created in selects/... named by concatenating directory + patterns
 # Report: selects/dat1_CanadaMPs_1@AB:2@Liberal
 # ---> uvcopy selectlf1,fili1=dat1/CanadaMPs,arg1=1@AB:2@Liberal
 #      =========================================================================
 #format: uvcopy selectlf1,fili1=inputfile,arg1=pattern1:pattern2,etc,uop=i0/i1
 # arg1 - line selection conditions (up to 9 patterns to match or not-match)
 # - code '^' circumflex for ' ' spaces and '%' percent for ',' commas in patterns
 # - may prefix patterns with '-' for absence, '+' for presence (default)
 # - may restrict search area, example: arg1=@0[20]Townsend:@20[60]Owen
 # - multiple conditions: arg1=+1@Townsend:+2@Canada,-1@Owen,-2@BC
 # - combination ex: arg1=+1@0[20]Townsend:+2@60[60]Canada:-@0[20]Owen
 # Date=2019/07/06_16:32:34, Site=UV_Software, Host=uvsoft5, User=uvadm, Options=
 #===============================================================================
 042  Boissonnault       Randy           Edmonton Centre               AB Liberal          2015
 140  Hehr               Kent            Calgary Centre                AB Liberal          2015
 294  Sohi               Amarjeet        Edmonton Mill Woods           AB Liberal          2015
 # EOF - 3 lines selected from dat1/CanadaMPs

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3E2. Select Jobs - *selectlf1, selectlfd1, selectlgfd1

suggested patterns for searching dat1/CanadaMPs

Here are some suggested search patterns you might run on the CAnadian Members of Parliament file (for tutorials & self training on Select Jobs).

Here are 1st 3 of the 335 records with a column scale in case you wish to test select job searches with restricted areas. Note scale is 1 relative, but you must code the zero relative values for restricted area searches.

          1         2         3         4         5         6         7         8         9       100
 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 001  Aboultaif          Ziad            Edmonton Manning              AB Conservative     2015
 002  Albas              Dan             Central Okanagan/Similkameen/ BC Conservative     2015
 003  Albrecht           Harold          Kitchener/Conestoga           ON Conservative     2015

search CanadaMPs MP file for Liberals in Vancouver


 uvcopy selectlf1,fili1=dat1/CanadaMPs,arg1=1@70[2]BC:2@Liberal:3@Vancouver
 ==========================================================================
 - select MP's from "BC" who are "Liberal"s from "Vancouver" --> result 5 records
 117  Fry                Hedy            Vancouver Centre              BC Liberal          2015
 128  Goldsmith-Jones    Pamela          West Vancouver/Sunshine Coast BC Liberal          2015
 223  Murray             Joyce           Vancouver Quadra              BC Liberal          2015
 272  Sajjan             Harjit S.       Vancouver South               BC Liberal          2015
 327  Wilkinson          Jonathan        North Vancouver               BC Liberal          2015
 # EOF - 5 lines selected from dat1/CanadaMPs

Other searches showing only number of matches expected


 uvcopy selectlf1,fili1=dat1/CanadaMPs,arg1=AB
 =============================================
 - select all 'MP's from Alberta --> result = 34 records
 - see if you get the same result ?

 uvcopy selectlf1,fili1=dat1/CanadaMPs,arg1=@70[2]AB
 ===================================================
 - restrict search for "AB" to columns 71-72 (in case AB occurring elsewhere)
 - we got the same number of records (34), so "AB" did not occur elsewhere
 - Next, let's try case insensitive to see if "ab" does occur elsewhere

 uvcopy selectlf1,fili1=dat1/CanadaMPs,arg1=ab
 =============================================
 - search for "ab" vs "AB", will not work, since default is case sensitive
 --> gets 15 unrelated records with "ab" as parts of names

 uvcopy selectlf1,fili1=dat1/CanadaMPs,arg1=ab,uop=s1  - case insensitive option 's1'
 ====================================================  - "ab" same as "AB"
 - option 's1' (case insensitive) gets --> 52 records (18 more)

 uvcopy selectlf1,fili1=dat1/CanadaMPs,arg1=...
 ==============================================
 - continue testing with your own patterns

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3E3. Select Jobs - *selectlf1, selectlfd1, selectlgfd1

searching with > & < conditions

Searching for > & < can only be used on fields with fixed-positions. We will demo using dat1/CanadaMPs which has the year elected in columns 91-94 (zero relative 90-93). Here are 1st 3 of 335 records with a column scale.

          1         2         3         4         5         6         7         8         9       100
 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 001  Aboultaif          Ziad            Edmonton Manning              AB Conservative     2015
 002  Albas              Dan             Central Okanagan/Similkameen/ BC Conservative     2015
 003  Albrecht           Harold          Kitchener/Conestoga           ON Conservative     2015

 #1. uvcopy "selectlf1,fili1=dat1/CanadaMPs,arg1=1@Conservative:2@90[4]>2015"
     ========================================================================
     - search for Conservatives elected since 2015
     - must enclose command in quotes when '<' or '>' used

 #2. cat selects/dat1_CanadaMPs_1@Conservative:2@90[4]_2015
     ======================================================
     - display output report (or could have replied just 'cat' at the prompt)
 022  Barrett            Michael         Leeds/Grenville/Thousand Isla ON Conservative     2018
 030  Benzen             Bob             Calgary Heritage              AB Conservative     2017
 074  Davidson           Scot            York/Simcoe                   ON Conservative     2019
 102  Falk               Rosemarie       Battlefords/Lloydminster      SK Conservative     2017
 164  Kusie              Stephanie       Calgary Midnapore             AB Conservative     2017
 182  Lloyd              Dane            Sturgeon River/Parkland       AB Conservative     2017
 197  Martel             Richard         Chicoutimi/Le Fjord           QC Conservative     2018
 222  Motz               Glen            Medicine Hat/Cardston/Warner  AB Conservative     2016
 # EOF - 8 lines selected from dat1/CanadaMPs

Try some other commands such as:


 uvcopy "selectlf1,fili1=dat1/CanadaMPs,arg1=1@Green:2@90[4]>2015:1@NDP:2@90[4]>2015"
 ====================================================================================
 195  Manly              Paul            Nanaimo/Ladysmith             BC Green Party      2019
 293  Singh              Jagmeet         Burnaby South                 BC NDP              2019

 uvcopy "selectlf1,fili1=dat1/CanadaMPs,arg1=1@Green:2@90[4]>2015:3@90[4]<2020"
 ==============================================================================
 195  Manly              Paul            Nanaimo/Ladysmith             BC Green Party      2019

Notes

  1. You must enclose command in quotes when '<' or '>' used

  2. Output files (named from filename & arg1 concatenated) will have any '>' & '<' replaced by '_' underscores

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3F1. SelectJobs: *selectlf1* ,selectlfd1,selectlgfd1,selectlgfd2,selectfdd1

selectlfd1 - SELECT from all files in a directory

'selectlfd1' will select lines from all files in a directory, vs 'selectlf1' which selects lines for only 1 file at a time. We will demo using a directory of SQL scripts $UV/demo/sqls/... (copied to /home/userxx/demo/sqls/...)

4 SQL "customer" scripts in sqls/...

/* select_customers_all */ SELECT * FROM Customers;

 /* select_customers_basic */
     SELECT Name, Phone, Address, City, Zip
            FROM Customers;

/* select_customers_sales10000 */ SELECT custno, Name, Phone, Address, City, Zip, TYsales, LYsales, FROM Customers WHERE LYsales >10000;

/* select_customers_sales20000 */ SELECT custno, Name, Phone, Address, City, Zip, TYsales, LYsales, FROM Customers WHERE LYsales >20000;

select "customer" references from all scripts in directory


 uvcopy selectlfd1,fild1=sqls,arg1=1@SELECT:2@FROM:3@Customers,uop=s1
 ====================================================================
 - select lines with "SELECT","FROM",& "Customers" from all files in sqls/*
 - best to use option 's1' case insensitive
 # Report: selects/sqls_1@SELECT:2@FROM:3@Customers
 # ---> uvcopy selectlfd1,fild1=sqls,arg1=1@SELECT:2@FROM:3@Customers,uop=s1
 #      ====================================================================
 #format: uvcopy selectlf1,fili1=inputfile,arg1=pattern1:pattern2,etc,uop=i0/i1
 # Date=2019/06/03_17:06:11, Site=UV_Software, Host=uvsoft5, User=uvadm, Options=s1
 #=================================================================================
     SELECT * FROM Customers;
                               EOF - 1 lines selected from sqls/select_customers_all
 # EOD - 1 lines selected from 11 files in directory sqls

PROBLEM Notes

  1. File#1 was the only file with all 3 words on the same line (as required by selectlfd1). It would be better to select the Multi-Line SQL statement vs 1 matching line.

  2. See alternative jobs 'selectlgfd1' on the next page to select line groups from arg1="SELECT" until arg2=";" ending SQL statements qualified by arg3 match to 1 or multiple words on 1 line. Or see 'selectlgfd2' which can qualify by matching multiple words on any line between arg1 & arg3.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3G1. SelectJobs: selectlf1,selectlfd1, *selectlgfd1*, selectlgfd2,selectfdd1

selectlgfd1 - Multi-Condition-Series on any ONE line

'selectlgfd1' will select line groups from arg1=... until arg2=... qualified by arg3 matches to 1 or multiple words on 1 line. Here is an example where arg1=SELECT, & arg2=; (begin & end of SQL SELECT statement), qualified by "Customers" & "LYsales", which must appear on the same line for selectlgfd1 (vs selectlgfd2 any line).


 uvcopy "selectlgfd1,fild1=sqls,arg1=SELECT^,arg2=;,arg3=1@Customers:2@LYsales,uop=s1"
 =====================================================================================
 # # Report: selects/sqls_SELECT_1@Customers:2@LYsales
 # # ---> uvcopy selectlgfd1,fild1=sqls,arg1=SELECT,arg2=;,arg3=1@Customers:2@LYsales
 # #      ===========================================================================
 # # format: uvcopy selectlgfd1,fild1=directory,arg1=...,arg2=...,arg3=...,arg4=...,uop=abcdgi
 # # arg1 patterns identify begin-group line, arg2 patterns identify end-group line
 # # arg3 patterns qualify group selection, arg4 patterns qualify lines to output/print
 # #  - code '^' circumflex for ' ' spaces and '%' percent for ',' commas in patterns
 # #  - may prefix patterns: '-' for absence, '+' for presence (default)
 # #  - may restrict search area: arg2=@0[2]/*  <-- "/*" must be in 1st 2 bytes
 # #  - multiple-condition-series: arg3=1@SORT:2@COND:3@AND:4@OR <-- ALL patterns must match
 # # Date=2019/06/03_12:40:18, Site=UV_Software, Host=uvsoft5, User=uvadm, Options=
 # #===============================================================================
 #     SELECT custno, Name, Phone, Address, City, Zip,
 #     TYsales, LYsales, FROM
 #     Customers WHERE LYsales >10000;
 #                          EOG: #1, 3 lines selected from sqls/select_customers_sales10000
 #                          EOF:     3 selected from 4 in sqls/select_customers_sales10000
 # EOD: 11 files, 36 lines, selected 1 groups, 3 lines to selects/sqls_SELECT_1@Customers:2@LYsales_s1

Notes Re: *selectlfgfd1 (& selectlgfd2) vs prior selectlf1/selectlfd1

  1. This 'selectlgfd1' shows the complete SQL statement (from "SELECT" to ending ";") (vs 'selectlf1' & 'selectlfd1' which showed only the lines with the search argument)

  2. 'selectlgf1' qualifies the group selection by arg3 patterns, but multiple conditional series patterns (1,2,3,etc) must appear on the same line.

  3. For the 'selectlgfd1' above, we get only 1 of 4 statements (SELECT FROM Customers) because only file#3 had both "Customers" & "LYsales" on the same line.

  4. See next page example 'selectlgfd2', which can qualify the group selection by arg3 conditional series patterns (1,2,3,etc) using multiple words that can appear on "ANY LINE between the arg1/arg2 begin/end patterns".

  5. Note difference between selectlgfd1 & selectlgfd2: 'selectlgfd1' arg3 qualifiers must appear on SAME LINE between arg1/arg2 patterns. 'selectlgfd2' arg3 qualifiers can appear on ANY LINE between arg1/arg2 patterns.

  6. Note that we need to enclose the uvcopy "selectlgfd1...arg2=;,..." command in quotes when the arguments include characters such as ';'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3G2. SelectJobs: selectlf1,selectlfd1,selectlgfd1, *selectlgfd2*, selectfdd1

selectlgfd2 - Multi-Condition-Series on MULTI lines (vs 1 line)

'selectlgf2' will select line groups from arg1=... until arg2=... qualified by arg3 matches to 1 or multiple words on ANY line in the Group. Here is an example where arg1=SELECT, & arg2=; (begin & end of SQL SELECT statement), qualified by "Customers" & "LYsales", which may appear on ANY line (vs selectlgfd1 one line).


 uvcopy "selectlgfd2,fild1=sqls,arg1=SELECT^,arg2=;,arg3=1@Customers:2@LYsales,uop=s1"
 =====================================================================================
 # # Report: selects/sqls_SELECT_1@Customers:2@LYsales
 # # ---> uvcopy selectlgfd2,fild1=sqls,arg1=SELECT,arg2=;,arg3=1@Customers:2@LYsales,uop=s1
 # #      ==================================================================================
 # # format: uvcopy selectlgfd2,fild1=directory,arg1=...,arg2=...,arg3=...,arg4=...,uop=abcdgi
 # # arg1 patterns identify begin-group line, arg2 patterns identify end-group line
 # # arg3 patterns qualify group selection, arg4 patterns qualify lines to output/print
 # #  - code '^' circumflex for ' ' spaces and '%' percent for ',' commas in patterns
 # #  - may prefix patterns: '-' for absence, '+' for presence (default)
 # #  - may restrict search area: arg2=@0[2]/*  <-- "/*" must be in 1st 2 bytes
 # #  - multiple-condition-series: arg3=1@SORT:2@COND:3@AND:4@OR <-- ALL patterns must match
 # # Date=2019/06/03_12:43:54, Site=UV_Software, Host=uvsoft5, User=uvadm, Options=
 # #===============================================================================
 #     SELECT custno, Name, Phone, Address, City, Zip,
 #     TYsales, LYsales, FROM
 #     Customers WHERE LYsales >10000;
 #                          EOG: #1, 3 lines selected from sqls/select_customers_sales10000
 #                          EOF:     3 selected from 4 in sqls/select_customers_sales10000
 #     SELECT custno, Name, Phone, Address, City, Zip,
 #     TYsales, LYsales, FROM
 #     Customers
 #     WHERE LYsales >20000;
 #                          EOG: #1, 4 lines selected from sqls/select_customers_sales20000
 #                          EOF:     4 selected from 5 in sqls/select_customers_sales20000
 # EOD: 11 files, 36 lines, selected 2 groups, 7 lines to selects/sqls_SELECT_1@Customers:2@LYsales_s1

Notes Re: *selectlfgfd2 (& selectlgfd1) vs prior selectlf1/selectlfd1

  1. Both selectlgfd1 & *selectlgfd2 shows complete SQL statements ("SELECT" to ending ";") (vs 'selectlf1' & 'selectlfd1' which showed only the lines with the search argument).

  2. 'selectlgf1' qualifies the group selection by arg3 patterns, but multiple conditional series patterns (1,2,3,etc) must appear on the SAME line. For 'selectlgfd1', we got only 1 of 4 statements because only file#3 had both "Customers" & "LYsales" on SAME line.

  3. This 'selectlgfd2' can qualify the group selection by arg3 conditional series patterns using multiple words that can appear on ANY line between the arg1/arg2 begin/end patterns.

  4. Note difference between selectlgfd1 & selectlgfd2: 'selectlgfd1' arg3 qualifiers must appear on SAME LINE between arg1/arg2 patterns. 'selectlgfd2' arg3 qualifiers can appear on ANY LINE between arg1/arg2 patterns.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3H1. SelectJobs: selectlf1,selectlfd1, *selectlgfd1*, selectlgfd2,selectfdd1

selectlgfd1 - select JCL SYSINs with AND & OR conditions

We need to inspect SORT INCLUDE/OMIT CONDitions with both "AND"s & "OR"s which may need manual corrections. Here is the command line, followed by the report.


 uvcopy selectlgfd1,fild1=jcl2,arg1=//SYSIN,arg2=/*,arg3=1@COND:2@AND:1@COND:2@OR
 ================================================================================
 # Report: selects/jcl2_SYSIN_1@COND:2@AND:1@COND:2@OR%
 # ---> uvcopy selectlgfd1,fild1=jcl2,arg1=//SYSIN,arg2=/*,arg3=1@COND:2@AND:1@COND:2@OR%
 #      =========================================================================
 # format: uvcopy selectlgfd1,fild1=directory,arg1=...,arg2=...,arg3=...,arg4=...,uop=abcdgi
 # arg1 patterns identify begin-group line, arg2 patterns identify end-group line
 # arg3 patterns qualify group selection, arg4 patterns qualify lines to output/print
 #  - code '^' circumflex for ' ' spaces and '%' percent for ',' commas in patterns
 #  - may prefix patterns: '-' for absence, '+' for presence (default)
 #  - may restrict search area: arg2=@0[2]/*  <-- "/*" must be in 1st 2 bytes
 #  - multiple-condition-series: arg3=1@SORT:2@COND:3@AND:4@OR <-- ALL patterns must match
 # Date=2019/06/09_10:26:58, Site=UV_Software, Host=uvsoft5, User=uvadm, Options=
 #===============================================================================
 //SYSIN    DD *
 SORT FIELDS=(31,6,CH,A,1,6,CH,A)
 OMIT COND=((11,2,CH,EQ,C'65'),OR,(COND=(11,2,CH,EQ,C'85'))
 /*
                          EOG: #1, 4 lines selected from jcl2/jar200
                          EOF:     4 selected from 23 in jcl2/jar200
 //SYSIN    DD *
 SORT FIELDS=(1,8,CH,A,69,12,CH,A)
 INCLUDE COND=((9,1,CH,EQ,C'T'),
           AND,(COND=(56,2,CH,EQ,C'BC'))
 /*
                          EOG: #1, 5 lines selected from jcl2/jgl230
                          EOF:     5 selected from 34 in jcl2/jgl230
 //SYSIN    DD *
 SORT FIELDS=(1,8,CH,A,69,12,CH,A)
 INCLUDE COND=(((9,1,CH,EQ,C'T'),OR,(COND=(9,1,CH,EQ,C'X'))),
                                 AND,(COND=(56,2,CH,EQ,C'AB')))
 /*
                          EOG: #1, 5 lines selected from jcl2/jgl232
                          EOF:     5 selected from 35 in jcl2/jgl232
 EOD: 7 files, 168 lines, selected 3 groups, 13 lines to selects/jcl2_SYSIN_1@COND:2@AND:1@COND:2@OR

Notes re tts & selectlgfd1 demo

  1. demo/jcl2/* contains 7 demo JCLs with 5 SYSINs (3 of them sorts). 'arg1=//SYSIN' defines the group starting line & arg2=/*' defines the group ending line.
  2. 'arg3=1@COND:2@AND:1@COND:2@OR' defines the select group conditions. This arg3 selects groups with lines that contain ("COND" & "AND") OR ("COND" & "OR"). The pattern rules are the same as for the 'sst' instruction (preceding).
  3. Better if we could select ONLY the groups with BOTH AND & OR, since it is only those that need correction, But 'selectlgfd1' (using 'tts' instruction) can not do this, Because its multi conditions apply to single lines & not to all lines in the group
  4. See 'selectlgfd2 (using the 'ttx' instruction), which searches for all lines in the group to satisfy multi condition series patterns (1,2,3,etc).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3H2. SelectJobs: selectlf1,selectlfd1,selectlgfd1, *selectlgfd2*, selectfdd1

selectlgfd2 - select JCL SYSINs with AND & OR conditions (BETTER)

'selectlgfd2' can do a BETTER job of selecting ONLY the SYSINs with BOTH ANDs & ORs (vs 'selectlgfd1 which could only select SYSINs with EITHER AND or OR)


 uvcopy selectlgfd2,fild1=jcl2,arg1=1@//SYSIN,arg2=/*,arg3=1@COND:2@AND:3@OR%
 ============================================================================
 # Report: selects/jcl2_SYSIN_1@COND:2@AND:3@OR%
 # ---> uvcopy selectlgfd2,fild1=jcl2,arg1=//SYSIN,arg2=/*,arg3=1@COND:2@AND:3@OR%
 #      =========================================================================
 # format: uvcopy selectlgfd2,fild1=directory,arg1=...,arg2=...,arg3=...,arg4=...,uop=abcdgi
 # arg1 patterns identify begin-group line, arg2 patterns identify end-group line
 # arg3 patterns qualify group selection, arg4 patterns qualify lines to output/print
 #  - code '^' circumflex for ' ' spaces and '%' percent for ',' commas in patterns
 #  - may prefix patterns: '-' for absence, '+' for presence (default)
 #  - may restrict search area: arg2=@0[2]/*  <-- "/*" must be in 1st 2 bytes
 #  - multiple-condition-series: arg3=1@SORT:2@COND:3@AND:4@OR <-- ALL patterns must match
 # Date=2019/06/09_10:29:01, Site=UV_Software, Host=uvsoft5, User=uvadm, Options=
 #===============================================================================
 //SYSIN    DD *
 SORT FIELDS=(1,8,CH,A,69,12,CH,A)
 INCLUDE COND=(((9,1,CH,EQ,C'T'),OR,(COND=(9,1,CH,EQ,C'X'))),
                                 AND,(COND=(56,2,CH,EQ,C'AB')))
 /*
                          EOG: #1, 5 lines selected from jcl2/jgl232
                          EOF:     5 selected from 35 in jcl2/jgl232
 EOD: 7 files, 168 lines, selected 1 groups, 5 lines to selects/jcl2_SYSIN_1@COND:2@AND:3@OR%

Notes re ttx vs tts & selectlgfd2 vs selectlgfd1

  1. Compare this selectlgfd2 (using the 'ttx' instruction) search result to the previous selectlgfd1 (using the 'tts' instruction).

  2. selectlgfd1/tts selection was 'arg3=1@COND:2@AND:1@COND:2@OR', which selected groups with (COND & AND) or (COND & OR). The result would include extra groups that do not need correction - ONLY groups with BOTH (AND & OR) may need correction.

  3. selectlgfd2/ttx selection is 'arg3=1@COND:2@AND:3@OR', which selects ONLY groups with BOTH ("AND" & "OR"), ie only those that may need correction.

  4. The 'tts' instruction), bases multi condition series patterns (1,2,3,etc) only on patterns found on any 1 line in the group.

  5. The 'ttx' instruction), bases multi condition series patterns (1,2,3,etc) on patterns found on any line in the group.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3I1. SelectJobs: selectlf1,selectlfd1,selectlgfd1,selectlgfd2, *selectfdd1*

select ENTIRE FILE to a 2nd directory - arg1=patterns on 1 line

You could use 'selectfdd1' to select the entire SQL script. 'selectfdd1' selects entire files from the input directory to an output directory.

 'selectfdd1' can be use with BOTH multi-pattern-conditional-series-selection methods
 - based on arg1=multi-patterns on any 1 line AND/OR on arg2=multi-patterns on any line
   in the entire file.

Example#1 will test the arg1 method which selects multi-patterns on ONE line. (Also see Ex#2 on the next page which selects multi-patterns on any line in the file)

selectfdd1 Example#1 - multi-patterns on 1 line (arg1)


 #1. mkdir tmp2 OR rm -f tmp2/*   <-- make an output dir OR clear existing dir
     ==========================

 #2. uvcopy "selectfdd1,fild1=sqls,fild2=tmp2,arg1=1@Customers:2@LYsales,uop=s1t1"
     =========================================****================================
      - select files from sqls/* referencing "Customers" & "LYsales"
        (appearing on ONE line in the file)

stats Report written to selects/...

 # Report: selects/sqls_tmp2_sqls_1@customers:2@lysales
 # ---> uvcopy selectfdd1,fild1=sqls,fild2=tmp2,arg1=1@customers:2@lysales,uop=s1t1
 #      ===========================================================================
 #format: uvcopy selectfdd1,fild1=indir,fild2=outdir,arg1=LinePatterns,arg2=FilePatterns
 # Date=2019/06/03_20:56:02, Site=UV_Software, Host=uvsoft5, User=uvadm, Options=s1t1
 #============----===================================================================
   EOF - 4 lines written to tmp2/select_customers_sales10000
 **EOD Input  - 36 lines  read from 11 files in directory sqls
 **EOD Output - 4 lines written to 1 files in directory tmp2

 #3. cat tmp2/*     <-- display output files
     ==========

/* select_customers_sales10000 */ SELECT custno, Name, Phone, Address, City, Zip, TYsales, LYsales, FROM Customers WHERE LYsales >10000;

Notes Re selectfdd1 selections by *arg1 or arg2

  1. Using "arg1=1@Customers:2@LYsales" we get only 1 of the 2 files that have both "Customers" & "LYsales" - because the other file has these patterns on different lines.

  2. See Ex#2 on the next page which selects multi-patterns on any line in the file. - coding search args on arg2=... finds multi-patterns on any line in the file (vs arg1=... which looks for multi-patterns on the same line).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3I2. SelectJobs: selectlf1,selectlfd1,selectlgfd1,selectlgfd2, *selectfdd1*

select ENTIRE FILE to a 2nd directory - arg2=patterns on ANY line

 'selectfdd1' can be use with BOTH multi-pattern-conditional-series-selection methods
 - based on arg1=multi-patterns on any 1 line AND/OR on arg2=multi-patterns on any line
   in the entire file.

Example#1 (on previous page) used 'arg1=...' which looks for multi-patterns on ONE line. This Example#2 uses 'arg2=...' which looks for multi-patterns on any line in the file.

selectfdd1 Example#2 - multi-patterns on ANY line (arg2)


 #1. mkdir tmp2 OR rm -f tmp2/*   <-- make an output dir OR clear existing dir
     ==========================

 #2. uvcopy "selectfdd1,fild1=sqls,fild2=tmp2,arg2=1@select:2@customers:3@lysales,uop=s1t1"
     =========================================****=========================================
      - select files from sqls/* referencing "Customers" & "LYsales"
        (appearing on ANY line in the file)

stats Report written to selects/...

 # Report: selects/sqls_tmp2_sqls_1@select:2@customers:3@lysales
 # ---> uvcopy selectfdd1,fild1=sqls,fild2=tmp2,arg2=1@select:2@customers:3@lysales,uop=s1t1
 #      ====================================================================================
 #format: uvcopy selectfdd1,fild1=indir,fild2=outdir,arg1=LinePatterns,arg2=FilePatterns
 # Date=2019/06/03_20:34:37, Site=UV_Software, Host=uvsoft5, User=uvadm, Options=s1t1
 #===================================================================================
   EOF - 4 lines written to tmp2/select_customers_sales10000
   EOF - 5 lines written to tmp2/select_customers_sales20000
 **EOD Input  - 36 lines  read from 11 files in directory sqls
 **EOD Output - 9 lines written to 2 files in directory tmp2

 #3. cat tmp2/*     <-- display output files
     ==========

/* select_customers_sales10000 */ SELECT custno, Name, Phone, Address, City, Zip, TYsales, LYsales, FROM Customers WHERE LYsales >10000;

/* select_customers_sales20000 */ SELECT custno, Name, Phone, Address, City, Zip, TYsales, LYsales, FROM Customers WHERE LYsales >20000;

Notes selections by arg1 or *arg2 & options 's1t1'

  1. Example#2 using arg2=..., gets both the files that have "Customers" & "LYsales" vs prior Example#1 which got only 1 file with "Customers" & "LYsales" on same line.

  2. To use option s1 with selectlgfd2, you need to also specify option 't1' which translates the data file to lower case when loaded into memory.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3J1. Vancouver Utility Programming Aids - Selectjobs

Select Demos using dat1/UScongress

$UV/dat1/UScongress is a data file of US Governament Representatives & Senators that we will use for demos & that you can use for self training using SelectJobs.

We downloaded a spreadsheet, extracted a csv file (dat1/UScongress.csv) & converted to a fixed layout so we could define specific fields required for some selections. See '10E1' if you are interested to see the uvcopy job (demo/pf/UScongresscsv2txt).

Here are the samples of the 539 records with a column scale in case you wish to test select job searches with restricted areas.

samples US Reps & Senators .csv converted to fixed field

 Abraham,Ralph,,,,Ralph Lee Abraham,1954-09-16,M,rep,LA,5,,Republican,
 Adams,Alma,,,,Alma S. Adams,1946-05-27,F,rep,NC,12,,Democrat,
 Aderholt,Robert,B.,,,Robert B. Aderholt,1965-07-22,M,rep,AL,4,,Republican,
          1         2         3         4         5         6         7         8
 12345678901234567890123456789012345678901234567890123456789012345678901234567890
 001  Abraham             Ralph          19540916  M  LA  rep  Republican
 002  Adams               Alma           19460527  F  NC  rep  Democrat
 003  Aderholt            Robert         19650722  M  AL  rep  Republican
              ---------- omitting 533 lines ----------
 537  Young               Don            19330609  M  AK  rep  Republican
 538  Young               Todd           19720824  M  IN  sen  Republican
 539  Zeldin              Lee            19800130  M  NY  rep  Republican

sample selection from UScongress testfile


 uvcopy selectlf1,fili1=dat1/UScongress,arg1=1@57[3]sen:2@53[2]OH
 ================================================================
  - select "senators" from OHio
  - report created in selects/... named by concatenating file + patterns
 # Report: selects/dat1_UScongress_1@57[3]sen:2@53[2]OH
 # ---> uvcopy selectlf1,fili1=dat1/UScongress,arg1=1@57[3]sen:2@53[2]OH
 #      =========================================================================
 #format: uvcopy selectlf1,fili1=inputfile,arg1=pattern1:pattern2,etc,uop=i0/i1
 # arg1 - line selection conditions (up to 9 patterns to match or not-match)
 # - code '^' circumflex for ' ' spaces and '%' percent for ',' commas in patterns
 # - may prefix patterns with '-' for absence, '+' for presence (default)
 # - may restrict search area, example: arg1=@0[20]Townsend:@20[60]Owen
 # - multiple conditions: arg1=+1@Townsend:+2@Canada,-1@Owen,-2@BC
 # - if no seq#s (+1,+2,etc) default selects if any +pattern & no -pattern
 # - combination ex: arg1=+1@0[20]Townsend:+2@60[60]Canada:-@0[20]Owen
 # Date=2019/07/07_10:29:30, Site=UV_Software, Host=uvsoft5, User=uvadm, Options=
 #===============================================================================
 050  Brown               Sherrod        19521109  M  OH  sen  Democrat
 384  Portman             Robert         19551219  M  OH  sen  Republican
 # EOF - 2 lines selected from dat1/UScongress

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3J2. Select Demos using selectlf1 & UScongress file

suggested patterns for searching dat1/UScongress

Here are some suggested search patterns you might run on the UScongress testfile for tutorials & self training on Select Jobs.

Here are 1st 3 of the 539 records with a column scale in case you wish to test select job searches with restricted areas. Note scale is 1 relative, but you must code the zero relative values for restricted area searches.

 #        1         2         3         4         5         6         7         8
 #2345678901234567890123456789012345678901234567890123456789012345678901234567890
 001  Abraham             Ralph          19540916  M  LA  rep  Republican
 002  Adams               Alma           19460527  F  NC  rep  Democrat
 003  Aderholt            Robert         19650722  M  AL  rep  Republican

search UScongress file for Female Democrats from teXas


 uvcopy selectlf1,fili1=dat1/UScongress,arg1=1@50[1]F:2@62[15]Democrat:3@53[2]TX
 ===============================================================================
 - select Female Democrats from TeXas
 # Report: selects/dat1_UScongress_1@50[1]F:2@62[15]Democrat:3@53[2]TX
 # ---> uvcopy selectlf1,fili1=dat1/UScongress,arg1=1@50[1]F:2@62[15]Democrat:3@53[2]TX
 #      =========================================================================
 # --- omitting 9 lines of HELP info & Date,Site,Host,User (same as prior page) ---
 #===============================================================================
 149  Escobar             Veronica       19690915  F  TX  rep  Democrat
 160  Fletcher            Lizzie         19750213  F  TX  rep  Democrat
 174  Garcia              Sylvia         19500906  F  TX  rep  Democrat
 237  Jackson Lee         Sheila         19500112  F  TX  rep  Democrat
 242  Johnson             Eddie          19351203  F  TX  rep  Democrat
 # EOF - 5 lines selected from dat1/UScongress

search UScongress file for Male Replublicans from CAlifornia


 uvcopy selectlf1,fili1=dat1/UScongress,arg1=1@50[1]M:2@62[15]Republican:3@53[2]CA
 =================================================================================
 - select Male Republicans from CAlifornia
 # Report: selects/dat1_UScongress_1@50[1]M:2@62[15]Republican:3@53[2]CA
 # ---> uvcopy selectlf1,fili1=dat1/UScongress,arg1=1@50[1]M:2@62[15]Republican:3@53[2]CA
 #      =========================================================================
 # --- omitting 9 lines of HELP info & Date,Site,Host,User (same as prior page) ---
 #===============================================================================
 061  Calvert             Ken            19530608  M  CA  rep  Republican
 098  Cook                Paul           19430303  M  CA  rep  Republican
 232  Hunter              Duncan         19761207  M  CA  rep  Republican
 275  LaMalfa             Doug           19600702  M  CA  rep  Republican
 318  McCarthy            Kevin          19650126  M  CA  rep  Republican
 320  McClintock          Tom            19560710  M  CA  rep  Republican
 358  Nunes               Devin          19731001  M  CA  rep  Republican
 # EOF - 7 lines selected from dat1/UScongress

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3J3. Select Demos using selectlf1 & UScongress file

more examples searching dat1/UScongress

Here are 1st 3 of the 539 records with a column scale so you can verify the field locations specified in the examples below. Note scale is 1 relative, but you must code the zero relative values for restricted area searches.

 #        1         2         3         4         5         6         7         8
 #2345678901234567890123456789012345678901234567890123456789012345678901234567890
 001  Abraham             Ralph          19540916  M  LA  rep  Republican
 002  Adams               Alma           19460527  F  NC  rep  Democrat
 003  Aderholt            Robert         19650722  M  AL  rep  Republican

search UScongress file for Female Democrats born after 1983


 uvcopy "selectlf1,fili1=dat1/UScongress,arg1=1@50[1]F:2@40[4]>1983"
 ===================================================================
 - select Female Democrats born after 1983
 # Report: selects/dat1_UScongress_1@50[1]F:2@40[4]_1983
 # ---> uvcopy selectlf1,fili1=dat1/UScongress,arg1=1@50[1]F:2@40[4]>1983
 #      =========================================================================
 # --- omitting 9 lines of HELP info & Date,Site,Host,User (same as prior page) ---
 #===============================================================================
 156  Finkenauer          Abby           19881227  F  IA  rep  Democrat
 219  Hill                Katie          19870825  F  CA  rep  Democrat
 359  Ocasio-Cortez       Alexandria     19891013  F  NY  rep  Democrat
 492  Torres Small        Xochitl        19841115  F  NM  rep  Democrat
 497  Underwood           Lauren         19861004  F  IL  rep  Democrat
 # EOF - 5 lines selected from dat1/UScongress

search UScongress file for Male Republicans born after 1983


 uvcopy "selectlf1,fili1=dat1/UScongress,arg1=1@50[1]M:2@40[4]>1983"
 ===================================================================
 - select Male Republicans born after 1983
 # Report: selects/dat1_UScongress_1@50[1]M:2@40[4]_1983
 # ---> uvcopy selectlf1,fili1=dat1/UScongress,arg1=1@50[1]M:2@40[4]>1983
 #      =========================================================================
 # --- omitting 9 lines of HELP info & Date,Site,Host,User (same as prior page) ---
 #===============================================================================
 112  Crenshaw            Dan            19840314  M  TX  rep  Republican
 170  Gallagher           Mike           19840303  M  WI  rep  Republican
 182  Gonzalez            Anthony        19840919  M  OH  rep  Republican
 485  Timmons             William        19840430  M  SC  rep  Republican
 # EOF - 4 lines selected from dat1/UScongress

Notes

  1. You must enclose command in quotes when '<' or '>' used

  2. Output files (named from filename & arg1 concatenated) will have any '>' & '<' replaced by '_' underscores since >/< are invalid in filenames.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_4. Vancouver Utility Programming Aids - TableJobs

TableJobs Benefits to Programmers & Analysts

TableJobs are invaluable to IT professionals at any Unix/Linux/Windows site because:

  1. TableJobs build summary tables of counts & values to be dumped to a report file at EOF. There are various TableJobs for various file-types (text,fixed,variable), but we think you will find 'table3d' for text files most useful to supply you with statistics from directories of programs, scripts, parameter files, etc.

  2. You specify a Key-Word preceding the Target-Word to be table summarized. For example, given a directory of JCLs, you could specify KeyWord "PGM" preceding the program name in statements such as "// EXEC PGM=programxx".

  3. For FTP scripts, you could table the IP#s following the KeyWord "OPEN", or table the userids & passwords following the KeyWord "USER". For SQL scripts you could table the table-name following the KeyWord "FROM".

  4. You can also create table summary reports from fixed or variable length data-files which may have packed/binary fields (which can be tabled).

  5. You can use Tablejobs to research the impacts of planned changes to your scripts & programs.

  6. TableJobs create reports in subdir stats/... of the current working directory, automatically named from the file or directory concatenated with Keywords used. This allows you to review table history & easily modify & repeat.

  7. The table reports include all relevant info (files,dirs,Keywords,date/time, user,host,etc) to help you document problems & plan proposed changes.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

TableJobs - Contents


4A1. table3d - create table summary counts of desired items in text files
- pre-programmed job using uvcopy (powerful data manipulation utility)
- scan all files in directory building table statistics to dump at EOF
- table3d example#1 - counts of IP#s in FTP scripts

4B1. Check-List - Ready to run TableJobs test/demos ?
- Vancouver Utilities installed, Unix userid setup,
- setup your user account & profile to run TableJob demos
- OR setup subdir $HOME/demo in your existing account

4B2. Vancouver Utility files in $UV=/home/uvadm/...
- highlighting subdirs/files relevant to TableJobs

4B3. Files relevant to TableJobs already isolated in /home/uvadm/demo/...
- copy $UV/demo/* subdirs/files to your $HOME/demo/...

4C1. table3d user options --> uop=a0b3c0d0f0j0l0m0p1s0t0x0w1 <-- defaults
Many options allow you to define the items to be tabled compensating
for complex situations in your data files.

4D1. Most useful options - option 'w#'
Option w specifies the word# offset of the TargetWord from the KeyWord.
It defaults to 'w1' meaning the TargetWord to be tabled & counted is
the 1st word following the search KeyWord specified by arg1=...
Demo table3d with option w2 to summarize 2nd filenames on FTP puts

4D2. Most useful options - option 'y#'
Option 'y#' specifies the number of keywords to be tabled (default 1).
Demo table3d option y2 to summarize both userid & passwords following 'user'

4E1. Test files to demo table3d
- JCLs, FTP scripts, SQL scripts,

4E2. Table Summarize program names in JCL
Key-Word PGM, Target-Word program-name following PGM=...

4F1. Writing specific uvcopy jobs vs Pre-Programmed TableJobs

4F2. tableIP - custom written table summary demo (only 14 instructions)
- creates same table summary as table3d
- but only for 1 specific summary (vs general purpose table3d)
4F3.  Notes re uvcopy table summarize demo job 'tableIP'

4G1. Creating table summaries of SQL TABLEs in SQL scripts
Key-Word FROM, Target-Word table-name following FROM ...
4G2.  SQL scripts listed - test files for table3d demos

4G3. summarize SQL TABLEs in SELECT ... FROM TableName - 1st attempt
- result will be missing some TableNames
4G4.  Problem if Target-Word (table-name) not on same line as Key_Word (FROM)
 Could use utility 'combine1d' to combine SQL statements prior to table3d
4G5.  Problem Solution - SELECT stmnts combined onto 1 line
- by utility job 'combine1d'
4G6.  Rerun table3d inputting SELECT stmnts combined onto 1 line
- to create correct summary of SQL TABLEs

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

TableJobs - Contents (continued)


4H1. 'table2' - table summaries by fixed position fields displacement(length)
- versus table3 searc for Key-Words & table following Target-Words
- Demos using demo file of US Government Representatives & Senators
- downloaded Excell file, extracted csv file,& converted to fixed layout

4H2. table2 - Notes & Benefits

4H3. table2 demo Execution & Console-Log
- illustrated console-log prompts for options, arguments, table fieldnames,
  & prompt to view output report

4I1. 4 Table Summaries from dat1/UScongress
You can run various table summaries from the dat1/UScongress demo file.
We show a few of the shorter reports, followed by script sf/UScongress_stats,
which runs 13 reports that we thought were interesting.
First report - table US Representatives/Senators by Party

4I2. table US Representatives/Senators by House & Party

4I3. table US Representatives/Senators by Party & Sex

4I4. table US Representatives/Senators by State & Party

4J1. script to run multiple stats on dat1/UScongress Representatives & Senators

4K1. Table2 Summaries of Canadian Members of Parliament dat1/CanadaMPs
- samples of the 335 records with a column scale to determine field locations.

4K2. 3 Table Summaries run from dat1/CanadaMPs
1st CagovMPs table by Party

4K3. table CA MPs by House & Party
table CA MPs by Year-Elected

4K4. script sf/CanadaMPs to run multiple stats on dat1/CanadaMPs

TableJobs same as UVdemos Part 4

TableJobs.htm is the same as UVdemos.htm#Part_4. TableJobs was duplicated as a separate document because it is 1 of the most useful parts of UVdemos.

SelectJobs.htm (same as UVdemos.htm#Part_3) is also a separate document.


UVscripts.htm (same as UVdemos.htm#Part_5) is also a separate document.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4A1. TableJobs - create Table Summaries via Keywords/Targetwords

TableJobs build summary tables of counts & values to be dumped to a report file at EOF. There are various TableJobs for various file-types (text,fixed,variable), but we think you will find 'table3d' most useful to quickly supply you with statistics from directories of programs, scripts, parameter files, etc.

For example, 'table3d' can read all files in the ftps/ directory, searching for KeyWord "open" & count occurrences of the following TargetWord (the IP#).

Example#1 - table summarize all IP#s in FTP scripts


 uvcopy table3d,fild1=ftps,arg1=open
 ===================================
  - read ftps/* search for "open" & count occurrences of following IP#
  - output report created in subdir stats/... & automatically named by concatenating
    directory + keywords separated by underscores

table3d report created in stats/...

 # /home/uvadm/demo/stats/ftps_open  <-- This report created by uvcopy:
 # uvcopy table3d,fild1=ftps,arg1=open
 # =====================================================================
 # - scan all files in directory for a keyword & table counts of following word
 # InDir=ftps  Keyword=open  Qual1=   Qual2=
 # WordSepsBlank=  Options=q1a0b3c0d0f0j0l0m0p1s0t0x0w1
 # - default selects following word, use option w2 to table 2nd word, etc
 # - specify arg1=Keyword,arg2=Qualifier-present,arg2=Qualifier-absent
 # &\,$=/>-<:(.');_*[|]"<--- arg4=punctuation to blank for word separation
 # abcdefghjklpqrsuALPRQ<--- equivalents, easier to enter, may enter "all"
 # Date=2019/06/17_12:18:57, Site=UV_Software, Host=uvsoft5, User=uvadm
 #===============================================================================
 #---> uvcopy table3d,fild1=ftps,arg1=open
 table3d  2019/06/17_12:19:01  Counts by Targetword following specified Keyword
 tbl#0001  tblt1f7 e0(48)         argument
 line#   1strec#  %      count  target-word
     1         2  25         1   192.168.0.4
     2         2  25         1   192.168.0.5
     3         2  50         2   192.168.0.6
                 100         4*  *TOTAL*

Notes, Benefits of table3d reports

  1. Output reports are automatically created in subdir stats/... named by concatenating the input directory & keywords separated by '_' underscores. The filename is coded on the 1st line of the report.

  2. The uvcopy command used to create the report is coded on the 2nd line, so you can look back at previous runs & repeat them (with modifications as desired).

  3. The reports are date/time stamped with all relevent info (site,host,user,options).

  4. See other examples in the tutorials ahead - userids & passwords in FTP ftps, programs called by all JCLs, table-names used by all SQL scripts, etc.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B1. Vancouver Utility Programming Aids - TableJobs

Setup check-list - Ready to run TableJob test/demos ?

We will present a 'setup check-list' here, since you may be reading this in the separate TableJobs.doc vs UVdemos.doc which includes TableJobs as Part4.

See UVdemos.htm for the complete setup procedures, but here is a check-list of the steps required before you can execute these test/demos.

  1. Install Vancouver Utilities. See complete documenation at install.htm. Also see the install summary at UVdemos.htm#1A3.

  2. Setup your user login account if not already done. See UVdemos.htm#1C1. Setup your user profile (.bash_profile or .profile for ksh). User profile in homedir calling common_profile in /home/appsadm/env/...

  3. Setup subdir demo/ in your homedir & copy $UV/demo/* to your $HOME/demo/... See instructions on page '4B3'

  4. Here we list only the datafiles from $UV/dat1/... relevant to TableJobs You can see all the demo/dat1/... files at UVdemos.doc#1C2.

    testdata file contents

  5. Here in TableJobs, we will usually show just a sample of the relevant testdata files before each test/demo execution. See UVdemos.htm#1D1 - 1D6 to see all files & all contents of the testdatafiles.

  6. The next page '4B2' will show files in $UV (usually /home/uvadm) that are relevant to TableJobs.

  7. Page '4B3' gives instructions to copy TableJob testfiles to your $HOME/demo/... (shown here as: /home/userxx/demo/...)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B2. Vancouver Utility Programming Aids - TableJobs

Vancouver Utilities - subdirs relevant to TableJobs

Here is the Vancouver Utilities homedir showing only the most relevant subdirs required for the TableJob demos documented in this section of UVdemos.htm.

You could run the demos logged in as 'uvadm', but if you are sharing the system with other uvadm users, it would be better to copy the demo/subdirs/files to your own homedir. See the copy instructions below & an expanded view of the demo/... files on the next page.

 /home/uvadm             <-- $UV
 :-----bin                 - UV binaries (uvcopy,uvsort,uvcp,uvhd,uvlist,etc)
 :-----dat1                - test data files to demo various Vancouver Utilities
 :     :-----...         <-- 200+ testdata files for all uvcopy job demos
 :     :                   - files for TableJobs already copied to demo/dat1/...
 :     :                   - see next page --->
 :-----doc                 - Vancouver Utilities documentation (text)
 :     :-----UVdemos.doc   - this documentation (1 of 150 files)
 :     :----TableJobs.doc  - Part4 of UVdemos duplicated as a separate document
 :-----dochtml             - documentation in HTML (uploaded to www.uvsoftware.ca)
 :-***-demo              <-- demo files for TestDemo tutorials, copy to your homedir -->
 :     :                   - copy /home/uvadm/demo/* to your /home/userxx/demo/...
 :     :-----dat1          - data files for TableJob demos, see details on next page
 :     :-----jcl2          - mainframe JCL samples (8)
 :     :-----ftps         - SYSIN control cards for various purposes
 :     :-----sqls          - SQL scripts for TableJob demos, see next page -->
 :-----pf                <-- Parameter Files for uvcopy
 :     :-----util            uvcopy TableJobs
 :     :     :-----table1    - table field# in a delimited file
 :     :     :-----table2    - table fixed position field by dsplcmnt(length)
 :     :     :-----table3    - table Target-Word following specified Key-Word for 1 file
 :     :     :-----table3d   - table Target-Word following specified Key-Word
 :     :     :-----            for ALL files in a Directory
 :     :-----adm             - uvcopy jobs for uvadm admin
 :     :-----demo            - uvcopy jobs for various demos
 :     :-----IBM             - uvcopy jobs for Mainframe conversions
 :-----sf                <-- Script Files (bash or Korn shell)
 :     :-----adm             - scripts for uvadm admin
 :     :-----demo            - demo scripts
 :     :-----util            - utility scripts
 :     :-----IBM             - scripts for mainframe conversions
 :-----src               <-- Vancouver Utilities C source code (uvcopy,uvsort,etc)
 :-----tf                  - test files for various demos (similar to dat1/...)
 :-----tmp                 - tmp subdir (test/demo outputs)
 :-----tmp1                - misc, outdir for some utility scripts (like listall1)
 :-----tmp2

Please copy the demo/... files to your homedir to avoid conflict with other users of Vancouver Utilities & to avoid losing your files when new versions of Vancouver Utilities are installed.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B3. Vancouver Utility Programming Aids - TableJobs

copy $UV/demo to your $HOME/demo/...


 #1. Login userxx --> /home/userxx

 #2. mkdir demo   <-- make directory for copy of demo files in your homedir

 #3. cp -r /home/uvadm/demo/* demo/   <-- copy demo files to your homedir
     ==============================

 #4. cd demo    <-- change into demo/ directory
     =======

TableJobs demo files copied to your homedir

Here is an expanded view of the most relevant subdirs/files required for the TableJob demos (omitting subdirs/files used for other parts of UVdemos.doc).

 /home/userxx/demo/              - copy to your homedir, cd demo/ & run jobs there
 :-----dat1                    <-- testdata files for demos
 :     :-----CanadaMPs.csv       - CAnadian Members of Parliament (csv file)
 :     :-----CanadaMPs           - desired fields extracted to fixed positions
 :     :-----UScongress.csv    <-- US Congress Representatives & Senators (csv file)
 :     :-----UScongress          - desired fields extracted to fixed positions
 :     :-----UScities.tab      <-- US top 300 cities & populations (tab separated)
 :     :-----UScitiesb           - desired fields extracted to fixed positions
 :     :-----USstates.csv      <-- US top 300 cities & populations (csv file)
 :     :-----UScitiesb           - desired fields extracted to fixed positions
 :     :
 :-----ftps                    <-- FTP scripts
 :     :ftpdemo1a
 :     :ftpdemo1b
 :     :ftpget1
 :     :ftpput1
 :-----jcl2
 :     :-----jar200.jcl
 :     :-----...etc...
 :-----pf                      <-- uvcopy job Parameter Files, see note below
 :     :-----                    - you could create your own uvcopy jobs here
 :     :
 :-----stats                   <-- output directory for TableJob Reports
 :     :-----ftps_open          - table summary of IP#s in FTP scripts
 :     :-----ftps_user          - userids & passswords in FTP scripts
 :     :-----sqls_FROM           - Table-Names in SQL scripts
 :     :-----CanadaMPs_Province  - Canadian MPs tabled by Province
 :     :-----...                 - Report-Names created from Directory + Keyword
 :     :-----...
 :-----sqls                    <-- SQL scripts for TableJob demos
 :     :-----customers_all
 :     :-----customers_basic
 :     :-----...etc...
 :-----tmp
 :-----tmp1                    <-- tmp dirs for temp files, sorts, etc
 :-----tmp2

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4C1. TableJobs - Table Summary Counts from Keywords/Targetwords

table3d options

The 1st example on page '4A1' did not show the options screen displayed before table3d is executed & the report written to the stats/ directory. Here are the options which we will omit from following examples.


 uvcopy table3d,fild1=dir,arg1=keyword,arg2=qual1,arg3=qual2,arg4=chars2blank,uop=...
 ====================================================================================
 uop=a0b3c0d0f0j0l0m0p1s0t0x0w1 - option defaults
     a1                     - "&" variable end "#", insert space after "#" force endword
     a2                     - discard variables not ending with "#"
       b3                   - assume 1 blank before & after keyword (default)
       b1                   - assume 1 blank before & no blank after
       b2                   - assume no blank before & 1 blank after
       b0                   - assume no blank before or after
         c1                 - bypass comments in COBOL programs (* column 7)
         c2                 - bypass comments in shell scripts (# column 1)
         c4                 - bypass comments in JCL //* cols 1-3
           d1               - insert blank before "$", allow arg1=$
             f0             - do not prepend/append table entry with filename
             f1             - prepend table argument with filename
             f2             - append table argument with filename
               j1           - insert blank before search word in data
               j2           - insert blank after search word in data
               j3           - insert blank & after search word in data
                 l0         - translate lower case before matching patterns
                   m1       - look for multiple keywords on 1 line
                   m2       - look for multiple keywords & prepend targetword
                   m4       - look for multiple keywords & append targetword
                     p1     - qualifier#1 present & qualifier#2 absent
                     p2     - qualifier#1 present & qualifier#2 present
                     p4     - qualifier#1 absent  & qualifier#2 absent
                       s1   - case insensitive translate patterns to lowercase
                         t1 - translate data to lower-case (to match patterns)
                         t2 - translate data to UPPER-case
                     w1     - target word is 1st word following keyword
                     w2     - target word is 2nd word following keyword
                     w0     - target word is same as keyword (partial def)
                   x1       - table w# word (search will ignore keyword)
                            - specified keyword will be inserted in outfilename
                 y#         - table multiple target words (4 max)
                 y3         - would table target word + following 2 words
               z1           - convert x"A3" to "@" in data for keyword "@"
 arg4 special chars to separate words, use with option w# offset from search word
 ------> &\,$=/>-<:(.`);_*[|]" <--- arg4 punctuation chars to blank
 ------> abcdefghjklpqrsuALPRQ <--- OR enter equivalents OR enter "all"
 User OPtion defaults=q1a0b3c0d0f0j0l0m0p1s0t0x0w1 (null accept or enter overrides)
 enter arg1 search keyword ------------>  <-- prompt inhibited if arg1 on cmdline
 enter arg2 qualifier#1 (or null) ----->
 enter arg3 qualifier#2 (or null) ----->
 enter arg4 chars to blank before word sep (comma,equal,etc) -->
 EOJ, Output File written to: stats/ftps_open
 default command = null, OR enter: vi,cat,more,lp,uvlp12,etc -->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4D1. TableJobs - Table Summary Counts from Keywords/Targetwords

most useful options - option 'w#'

Option 'w' is probably the most useful options you are likely to need. Option w specifies the word# offset of the TargetWord from the KeyWord. It defaults to 'w1' meaning the TargetWord to be tabled & counted is the 1st word following the search KeyWord specified by arg1=...

Here is an example using option w2 (2nd word following search KeyWord) to table summarize the output filenames on FTP 'put' commands. First we will show the relevant FTP script (parm file)

# ftps/ftpdemo1b open 192.168.0.5 user user02 user02pw lcd data1 put ar.sales.items C:\AR\SALES.ITEMS put gl.account.tran1 D:\GL\ACCOUNT.TRAN1

Example#2 - option 'w#' TargetWord offset from KeyWord


 #2. uvcopy table3d,fild1=ftps,arg1=put,uop=w2
     ==========================================
      - search ftps/* for keyword 'put' & table 2nd word following
Note
  • you will get prompts for unused arguments, enter null to continue
  • but I suggest reply 'cat' for the last prompt (to display output report)
      ---> cat <-- reply 'cat' to display report

 #2a. vi stats/ftps_put_w2  <-- OR, enter separate command if job already ended
      =====================
 # /home/uvadm/demo/stats/ftps_put_w2  <-- This report created by uvcopy:
 # uvcopy table3d,fild1=ftps,arg1=put,uop=w2
 # =====================================================================
 # - scan all files in directory for a keyword & table counts of following word
 # InDir=ftps  Keyword=put  Qual1=   Qual2=
 # WordSepsBlank=  Options=q1a0b3c0d0f0j0l0m0p1s0t0x0w1w2
 # - default selects following word, use option w2 to table 2nd word, etc
 # - specify arg1=Keyword,arg2=Qualifier-present,arg2=Qualifier-absent
 # &\,$=/>-<:(.');_*[|]"<--- arg4=punctuation to blank for word separation
 # abcdefghjklpqrsuALPRQ<--- equivalents, easier to enter, may enter "all"
 # Date=2019/06/17_15:11:03, Site=UV_Software, Host=uvsoft5, User=uvadm
 #===============================================================================
 #---> uvcopy table3d,fild1=ftps,arg1=put,uop=w2
 table3d  2019/06/17_15:11:05  Counts by Targetword following specified Keyword
 tbl#0001  tblt1f7 e0(48)         argument
 line#   1strec#  %      count  target-word
     1         6  50         1   C:\AR\SALES.ITEMS
     2         7  50         1   D:\GL\ACCOUNT.TRAN1
                 100         2*  *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4D2. TableJobs - Table Summary Counts from Keywords/Targetwords

most useful options - option 'y#'

Option 'y#' specifies the number of keywords to be tabled (default 1). For example, we can table summarize both userid & password following keyword 'user'. Here is a sample using the FTP ftps/* but showing only 1st file:

# ftps/ftpdemo1a open 192.168.0.4 user user01 user01pw lcd data1 get /home/userxx/ar/sales.items get /home/userxx/gl/account.tran1

Example#3 - option 'y' no of words to table


 #3. uvcopy table3d,fild1=ftps,arg1=user,uop=y2
     ===========================================
     - search ftps/* for keyword 'user' & table following 2 words
     --> cat <-- enter at last prompt to display report
 # /home/uvadm/demo/stats/ftps_user_y2  <-- This report created by uvcopy:
 # uvcopy table3d,fild1=ftps,arg1=user,uop=y2
 # =====================================================================
 # - scan all files in directory for a keyword & table counts of following word
 # InDir=ftps  Keyword=user  Qual1=   Qual2=
 # WordSepsBlank=  Options=q1a0b3c0d0f0j0l0m0p1s0t0x0w1y2
 # - default selects following word, use option w2 to table 2nd word, etc
 # - specify arg1=Keyword,arg2=Qualifier-present,arg2=Qualifier-absent
 # &\,$=/>-<:(.');_*[|]"<--- arg4=punctuation to blank for word separation
 # abcdefghjklpqrsuALPRQ<--- equivalents, easier to enter, may enter "all"
 # Date=2019/06/17_15:31:41, Site=UV_Software, Host=uvsoft5, User=uvadm
 #===============================================================================
 #---> uvcopy table3d,fild1=ftps,arg1=user,uop=y2
 table3d  2019/06/17_15:31:44  Counts by Targetword following specified Keyword
 tbl#0001  tblt1f7 e0(48)         argument
 line#   1strec#  %      count  target-word
     1         3  25         1  user01 user01pw
     2         3  25         1  user02 user02pw
     3         3  50         2  user03 user03pw
                 100         4*  *TOTAL*

Notes

  1. Note that any options entered are concatenated onto the output filename (ftps_user_y2 for the example above). The 1st line of the report file shows you the full path name of the report file (see above).

  2. For these demos, we are coding all arguments & options on the command-line, but you could enter only 'uvcopy table3d' & enter relevant info at the prompts. When we enter all relevant info on the cmdline, we can just make null replies to all following prompts.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4E1. Vancouver Utility Programming Aids - TableJobs

table3d - create summary table of programs in JCL

This the 4th demo of table3d, now using JCL as input


 #40a. login userxx --> /home/userxx
 #40b. cd demo      --> /home/usrxx/demo

 #1. l    <-- list files in current directory /home/userxx/demo
     ===
      drwxrwxr-x 2 userxx apps 4096 Feb 13 15:01 jcl2
      drwxrwxr-x 2 userxx apps 4096 Feb 13 15:01 jcl3
      drwxrwxr-x 2 userxx apps 4096 Feb 13 06:47 ftps
      drwxrwxr-x 2 userxx apps 4096 Feb 13 06:47 rpts
      drwxrwxr-x 2 userxx apps 4096 Feb 13 06:47 sqls
      drwxrwxr-x 2 userxx apps 4096 Feb 13 15:04 stats

 #2. l jcl2   <-- list jcl2/* input files for 3rd demo of table3d
     ======
      -rw-rw-r-- 1 userxx apps  480 Feb 13 15:01 jar100.jcl
      -rw-rw-r-- 1 userxx apps  613 Feb 13 15:01 jar120.jcl
      -rw-rw-r-- 1 userxx apps  984 Feb 13 15:01 jar200.jcl
      -rw-rw-r-- 1 userxx apps  777 Feb 13 15:01 jgl100.jcl  <-- listed below
      -rw-rw-r-- 1 userxx apps 1238 Feb 13 15:01 jgl200.jcl
      -rw-rw-r-- 1 userxx apps  617 Feb 13 15:01 jgl220.jcl
      -rw-rw-r-- 1 userxx apps 1477 Feb 13 15:01 jgl230.jcl
      -rw-rw-r-- 1 userxx apps  932 Feb 13 15:01 jgl320.jcl

 #3. cat jcl2/jar200.jcl
     ===================
      //JAR200   JOB  (1234),'TEST/DEMO MVS JCL CONVERT'
      //* SORT AR.SALES.ITEMS BY PRODUCT CODE FOR LISTING
      //STEP010  EXEC PGM=SORT,REGION=2048K                      <--Note PGM=SORT
      //SORTIN   DD DSN=AR.SALES.ITEMS,DISP=SHR
      //SORTOUT  DD DSN=&&jar200_TEMPSLS,DISP=(NEW,PASS),...
      //SYSIN    DD *
      SORT FIELDS=(31,6,CH,A,1,6,CH,A)
      OMIT COND=((11,2,CH,EQ,C'65'),OR,(COND=(11,2,CH,EQ,C'85'))
      /*
      //*=================== step# 0020 ========================
      //* LIST AR.SALES.ITEMS IN PRODUCT CODE SEQUENCE
      //STEP020  EXEC PGM=CAR200,COND=(4,LT,STEP010),PARM=2006   <--Note PGM=CAR200
      //SALEDTL  DD DSN=&&jar200_TEMPSLS,DISP=(OLD,PASS)
      //CUSTMAS  DD DSN=AR.CUSTOMER.MASTER.INDEXED,DISP=SHR
      //SALELST  DD DSN=AR.SALES.LIST,DISP=(,CATLG,DELETE),...

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4E2. Vancouver Utility Programming Aids - TableJobs

table summarize PGM names in JCL


 #4. uvcopy table3d,fild1=jcl2,arg1=PGM,arg4=ec,uop=w1q0i7
     =====================================================
     - read following notes before executing above command line

Notes re arguments & options

We will use 'PGM' as the keyword to table summarize the following word (program-name).

Re: 'arg4=ec', arg4 specifies characters to blank before word separation. Some special characters are a problem when entered using arg4=... on the command line so we allow you to use an alpha character to represent & show the equivalents as follows:

&\,$=/-:(.`);_*|" <--- arg4=special chars to blank abcdefhklpqrsuAPQ <--- equivalents, easier to enter

 //STEP010  EXEC PGM=CAR100,PARM=2008

For the above example, we need to blank the '=' & the ',' to separate the words. We used 'arg4=ec' vs 'arg4==,', because the ',' would confuse command parsing.

You could omit arg4=... from command line & enter actual characters at the prompt, but we prefer entering on the command line for this documentation.

Re: options 'uop=w1q0i7'. 'w1' means the target-word is 1st word following the keyword & we did not need to specify since it is the default.

Option 'q0' inhibits prompting for options (not needed if on the command line).

Option 'i7' inhibits displaying the 'opr' commands describing options, etc.

now execute the table3d command


 #5. uvcopy table3d,fild1=jcl2,arg1=PGM,arg4=ec,uop=q0i7
     ===================================================
 enter arg2 qualifier#1 (null=disable) ----->
 enter arg3 qualifier#2 (null=disable) ----->
 enter strings to be removed (use ":" sep if multiple) --->
 190216:160322:table3d: EOF fili01 rds=34 size=1477: demo/jcl2/jgl230.jcl hits=3
 190216:160322:table3d: EOF fili01 rds=23 size=984: demo/jcl2/jar200.jcl hits=2
 190216:160322:table3d: EOF fili01 rds=11 size=480: demo/jcl2/jar100.jcl hits=1
 190216:160322:table3d: EOF fili01 rds=17 size=777: demo/jcl2/jgl100.jcl hits=1
 190216:160322:table3d: EOF fili01 rds=26 size=1238: demo/jcl2/jgl200.jcl hits=2
 190216:160322:table3d: EOF fili01 rds=22 size=932: demo/jcl2/jgl320.jcl hits=2
 190216:160322:table3d: EOF fild01 rds=8 size=4096: demo/jcl2 hits=11
 190216:160322:table3d: EOF filo01 wrts=20 size=1322: stats/jcl2_PGM_ec_w1q0i7
 EOJ, Output File written to: stats/jcl2_PGM_ec_w1q0i7
 default command = null, OR enter: more,print,uvlpr12,edit,null -->

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4E3. Vancouver Utility Programming Aids - TableJobs

table3d output report for 3rd demo JCL programs


 #5a. --> cat <-- suggest 'cat' to view outputs (when known to be small)
          ===   - alternative, null reply & use #6. vi stats/jcl2_PGM_ec_q0i7
 # /home/uvadm/demo/stats/jcl2_PGM_ec_w1q0i7  <-- This report created by uvcopy:
 # uvcopy table3d,fild1=jcl2,arg1=PGM,arg4=ec,uop=w1q0i7
 # =====================================================================
 # - scan all files in directory for a keyword & table counts of following word
 # InDir=jcl2  Keyword=PGM  Qual1=   Qual2=
 # WordSepsBlank==,  Options=q1a0b3c0d0f0j0l0m0p1s0t0x0w1w1q0i7
 # - default selects following word, use option w2 to table 2nd word, etc
 # - specify arg1=Keyword,arg2=Qualifier-present,arg2=Qualifier-absent
 # &\,$=/>-<:(.');_*[|]"<--- arg4=punctuation to blank for word separation
 # abcdefghjklpqrsuALPRQ<--- equivalents, easier to enter, may enter "all"
 # Date=2019/06/17_16:06:56, Site=UV_Software, Host=uvsoft5, User=uvadm
 #===============================================================================
 #---> uvcopy table3d,fild1=jcl2,arg1=PGM,arg4=ec,uop=w1q0i7
 table3d  2019/06/17_16:06:59  Counts by Targetword following specified Keyword
 tbl#0001  tblt1f7 e0(48)         argument
 line#   1strec#  %      count  target-word
     1         3   7         1   CAR100
     2        15   7         1   CAR200
     3        26  21         3   CGL100
     4        17  21         3   CGL200
     5        12   7         1   IDCAMS
     6         6   7         1   IEBGENER
     7         5  28         4   SORT
                 100        14*  *TOTAL*

 #6. vi stats/jcl2_PGM_ec_q0i7  <-- view report separately after running table3d
     =========================
Note
  • the report is named for the directory + keyword + separators + options
  • so you do not have to make up a unique name to avoid overwriting prior report

General Purpose Utilities

The example above used JCL to test/demo table3d. You may not have JCL, but we hope you can see how table3d might be used with whatever files, scripts,& programming languages you have.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4F1. Vancouver Utility Programming Aids - TableJobs

Writing specific uvcopy jobs vs Pre-Programmed TableJobs

Pre-programmed jobs make it easy to apply the power of uvcopy to your data files - analyzing, selecting, searching/replacing, etc with more options & qualifications than are provided by the standard unix/linux system utilities.

You do not need to know the uvcopy instruction set to use general purpose pre-programmed jobs such as 'table3d', but we hope you will eventually learn the uvcopy instruction set, in case you find situations where you need more specific custom processing.

You can see the instruction file for 'table3d' at /home/uvadm/pf/util/table3d. It has over 250 instructions because it processes all files in a directory & provides many options to search for & qualify keywords & build summary tables of following targetwords.

We will show you here a much simpler table summary job (tableIP) with only 14 essential instructions that creates the same table summary as did the previous 'table3d', but it is written specifically to table summarize IP#s following the word 'open' (vs the general purpose pre-programmed table3d which accepts arguments to summarize any desired item following any desired keyword for all files in directory).

'tableIP' is stored at /home/uvadm/pf/demo/tableIP & you can run as follows:


 #0a. login userxx --> /home/userxx
 #0b. cd demo      --> /home/usrxx/demo

 #1. cat ftps/* >tmp1/ftps_all   <-- concat all ftps/* into 1 file for tableIP
     ===========================

 #2. uvcopy tableIP,fili1=tmp1/ftps_all,filo1=stats/ftps_ftp_tableIP
     =================================================================
     - run demo job to summarize IP#s in 1 file & create table same as previous table3d
 tableIP  2019/06/17_16:22:09  table summarize TCP/IP#s in FTP scripts
 tbl#0001  tblt1   ax6(16)        argument
 line#   1strec#  %      count  FTP IP#
     1         2  25         1  192.168.0.4
     2         9  25         1  192.168.0.5
     3        17  50         2  192.168.0.6
                 100         4*  *TOTAL*

Since this reads only 1 file (vs all files in directory), we concatenate the ftps/* files used for the previous table3d demo into 1 file stored in tmp1/ftps_all for input to uvcopy tableIP.

See the 'tableIP' demo job listed on the next page:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4F2. Writing customized uvcopy job vs Pre-Programmed TableJobs

tableIP - simplified table summary demo

 # tableIP - table summarize counts of unique TCP/IP#s in FTP scripts
 #         - uvcopy demo job, by Owen Townsend, UV Software, June 2019
 # - demo job used in documentation www.uvsoftware.ca/uvdemos.htm#2C1
 # - Normally use pre-programmed job 'table3d' to crate summary table counts
 # - This illustrates writing the uvcopy code for a custom table summary
 # - to keep it simple we will 1st concat all parm files into 1 tmp file
 #
 # cat ftps/* >tmp1/ftps_all
 #
 # uvcopy tableIP,fili1=tmp1/ftps_all,filo1=stats/ftps_ftp_tableIP
 # =================================================================
 # - read input fili1=..., build table of IP#s, write to output filo1=... at EOF
 #
 # uvcopy tableIP    <-- same as above, I/O files default as above
 # ==============
 #
 # 'tableIP' is a simplified version of 'table3d' general purpose pre-programmed job
 #   - to create table summaries of ANY item from ALL files in a directory
 #   - without needing to write any uvcopy instructions
 # 'tableIP' builds a table of 1 specific item from only 1 file
 #   - intended to demo the power of uvcopy & encourage you to learn uvcopy
 #   - will scan for lines with word 'open' known to precede the IP#
 #     open 192.168.0.4   <-- scan for lines with 'open' known to precede the IP#
 #
 opr='$jobname - table summarize TCP/IP#s in FTP scripts'
 rop=r1  # option for EOF prompt to view output file
 fili1=?tmp1/ftps_all,rcs=256,typ=LST
 filo1=?stats/ftps_tableIP,rcs=256,typ=LSTt
 @run
         opn     all
 # begin loop to get records, build table until EOF
 man20   get     fili1,a0               get next record into area 'a'
         skp>    man90                  (cc set > at EOF)
         sqzl1c1 a0(80),' '             ensure column 1 blank & 1 blank between words
         scn     a0(80),' open '        scan for ' open ' preceding IP#
         skp!    man20                  if not found - return to get next record
         tblt1   ax6(16),'FTP IP#'      build table counts in memory
         skp     man20                  return to get next record
 # EOF - dump table to output file, close files, end job
 man90   tbpt1  filo1,'table summarize TCP/IP#s in FTP scripts'
         cls    all
         eoj
 #            ** output - stats/ftps_tableIP **
 # tableIP  2019/06/17_16:22:09  table summarize TCP/IP#s in FTP scripts
 # tbl#0001  tblt1   ax6(16)        argument
 # line#   1strec#  %      count  FTP IP#
 #     1         2  25         1  192.168.0.4
 #     2         9  25         1  192.168.0.5
 #     3        17  50         2  192.168.0.6
 #                 100         4*  *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4F3. Writing customized uvcopy job vs Pre-Programmed TableJobs

Notes re uvcopy demo job 'tableIP'

  1. You could use this demo job to learn a lot about the uvcopy instruction set.

  2. Some uvcopy instructions are similar to the IBM 360 Z/OS instruction set, but uvcopy instructions are interpretive, no compile required, just edit & run.

  3. uvcopy gives you the power of assembler in an easy to use framework. In particular, this example will illustrate the power of index registers. uvcopy operands consist of an area (a-z), optionally an index register (a-z), and a displacement. For example op1 of 'tbl' is 'ax6' which is area 'a' + the displacement in register 'x' + the displacement coded ('6') in this example.

  4. 'scn' finds the word ' open ' & stores its displacement in index register 'x', which is then used as by 'tbl' to address the following word for tabling.

  5. All instructions are documented at uvsoftware.ca#uvcopy3.htm. The 1st page is an index with links to the various instructions. Here are direct links to some of the more interesting instructions in tableIP.
      uvsoftware.ca#uvcopy3.htm#sqz
      uvsoftware.ca#uvcopy3.htm#scn
      uvsoftware.ca#uvcopy3.htm#tbl
      uvsoftware.ca#uvcopy3.htm#tbp
  1. Here are a few notes about these 4 instructions use in tableIP & you can click on the links to see the complete doc on any of these.

 #7a. sqzl1c1 a0(80),' '             ensure column 1 blank & 1 blank between words
      =================
      - option 'l1' of sqzl1c1 ensures 1 blank in column 1
      - option 'c1' of sqzl1c1 ensures 1 blank between words
        so we can know that the word following ' open ' is exactly 6 bytes higher

 #7b. scn     a0(80),' open '        scan for ' open ' preceding IP#
      =======================
      - scn stores the zero relative displacement of ' open ' in register x.

 #7c. tblt1   ax6(16),'FTP IP#'      build table counts in memory
      ========================
      - tblt1 builds table#1 based on the 16 bytes located in area 'a'
        + displacement in register 'x' + 6 = area 'a' + 0 + 6
      - in this example, register 'x' will be 0 for ' open 192.168.0.4 ' since 'open'
        was 1st word on the line & sqzl1c1 has ensured the leading ' ' matching ' open '

 #7d. tbpt1  filo1,'table summarize TCP/IP#s in FTP scripts'
      ======================================================
      - tbpt1 (at EOF) dumps (Prints) the table to the output file (filo1=... above)
      - operand 2 supplies an extra heading for the table.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4G1. Vancouver Utility Programming Aids - TableJobs

table3d - create summaries of TABLEs in SQL scripts


 #0a. login userxx --> /home/userxx
 #0b. cd demo      --> /home/usrxx/demo

 #1a. l        <-- list subdirs in /home/userxx/demo
      ===
      drwxrwxr-x 2 userxx apps 4096 Feb 16 17:16 ctl
      drwxrwxr-x 2 userxx apps 4096 Feb 16 17:16 jcl2
      drwxrwxr-x 2 userxx apps 4096 Feb 16 17:16 jcl3
      drwxrwxr-x 2 userxx apps 4096 Feb 16 17:16 ftps
      drwxrwxr-x 2 userxx apps 4096 Feb 16 17:16 rpts
      drwxrwxr-x 2 userxx apps 4096 Feb 16 17:16 sqls     <-- SQL scripts
      drwxrwxr-x 2 userxx apps 4096 Feb 16 17:16 stats
      drwxrwxr-x 2 userxx apps 4096 Feb 16 17:16 tmp
      drwxrwxr-x 2 userxx apps 4096 Feb 16 17:16 tmp1

 #1b. l sqls     <-- list files in sqls/...
 ===========
      -rw-rw-r-- 1 userxx apps  56 Feb 16 17:16 select_customers_all    <-- simple example
      -rw-rw-r-- 1 userxx apps  97 Feb 16 17:16 select_customers_basic
      -rw-rw-r-- 1 userxx apps 152 Feb 16 17:16 select_customers_WA     <-- PROBLEM
      -rw-rw-r-- 1 userxx apps 136 Feb 16 17:16 select_employee_dept123
      -rw-rw-r-- 1 userxx apps 115 Feb 16 17:16 select_employee_seattle
      -rw-rw-r-- 1 userxx apps  79 Feb 16 17:16 select_orders_big
      -rw-rw-r-- 1 userxx apps 306 Feb 16 17:16 select_patients_old
      -rw-rw-r-- 1 userxx apps 129 Feb 16 17:16 select_students_failing

We can use the known keyword 'FROM' to summarize the following SQL TABLE-name. For example here is the SELECT stmnt from the 1st of the 8 listed on the enxt page.

      SELECT * FROM Customers;

PROBLEM in SQL scripts for table3d

table3d expects the target-word (to be summarized) to follow the known key-word on the same line, as above, but consider the SELECT in the 3rd SQL script:

       SELECT custno, Name, Birthday, Phone,
       Address, City, Zip, LYsales,Discount, FROM   <-- PROBLEM
       Customers WHERE LYsales >5000;

Our solution is to precede 'table3d' with 'combine1d' which will combine the SQL SELECT statements onto 1 line using the ';' semi-colons that end SQL statements.

Note that combine1d is a general purpose utility & could be used for other file types that you may have. 'arg1=...' defines optional statement begin patterns (up to 8 words) and 'arg4=..." defines the END patterns (up to 8 words, not necessarily 1 character).

'arg2=...' defines up to 8 words that must be present for the stmnt to be selected and 'arg3=...' defines up to 8 words that must be absent for the stmnt to be selected.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4G2. Creating table summaries of SQL TABLEs in SQL scripts

SQL scripts listed - test files for table3d


 #1c. cat sqls/*  <-- display contents of files in sqls/...
      ==========
      /* select_customers_all */
         SELECT * FROM Customers;
      /* select_customers_basic */
         SELECT Name, Phone, Address, City, Zip
                FROM Customers;
      /* select_customers_sales10000 */
         SELECT custno, Name, Phone, Address, City, Zip,
         TYsales, LYsales, FROM
         Customers WHERE LYsales >10000;
      /* select_customers_sales20000 */
         SELECT custno, Name, Phone, Address, City, Zip,
         TYsales, LYsales, FROM
         Customers
         WHERE LYsales >20000;
      /* select_employee_dept123 */
         SELECT EMP_ID, NAME, ADDRESS, CITY, ZIP, DEPT FROM
         EMPLOYEE WHERE DEPT = '123' AND STATUS = 'FULL';
      /* select_employee_seattle */
         SELECT EMP_ID, LAST_NAME FROM EMPLOYEE
         WHERE CITY = 'Seattle' ORDER BY EMP_ID;
      /* select_orders_big */
         SELECT item FROM Orders
         WHERE quantity > 500;
      /* select_patients_over70 */
         SELECT Name, Age FROM Patients WHERE Age > 70
         GROUP BY Age ORDER BY Name;
      /* select_patients_over80 */
         SELECT Name, Age FROM Patients WHERE Age > 80
         GROUP BY Age ORDER BY Name;
      /* select_patients_over90 */
         SELECT Name, Age FROM Patients
         WHERE Age > 90 GROUP BY Age ORDER BY Name;
      /* select_students_failing */
         CREATE VIEW Failing_Students AS
         SELECT S_NAME, Student_ID FROM
         STUDENT WHERE GPA < 30;

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4G3. Creating table summaries of SQL TABLEs in SQL scripts

table3d summarize SQL TABLEs - 1st attempt


 #2. uvcopy table3d,fild1=sqls,arg1=FROM,uop=q0i7
     ============================================
      enter arg2 qualifier#1 (null=disable) ----->
      enter arg3 qualifier#2 (null=disable) ----->
      enter arg4 chars to blank before word sep (comma,equal,etc) -->
      190617:173959:table3d: EOF fili01 rds=3 size=77: sqls/select_orders_big hits=1
      190617:173959:table3d: EOF fili01 rds=3 size=109: sqls/select_patients_over80 hits=1
      190617:173959:table3d: EOF fili01 rds=2 size=55: sqls/select_customers_all hits=1
      190617:173959:table3d: EOF fili01 rds=3 size=97: sqls/select_customers_basic hits=1
      190617:173959:table3d: EOF fili01 rds=3 size=115: sqls/select_employee_seattle hits=1
      190617:173959:table3d: EOF fili01 rds=3 size=110: sqls/select_patients_over70 hits=1
      190617:173959:table3d: EOF fili01 rds=5 size=152: sqls/select_customers_sales20000
      190617:173959:table3d: EOF fili01 rds=3 size=110: sqls/select_patients_over90 hits=1
      190617:173959:table3d: EOF fili01 rds=4 size=147: sqls/select_customers_sales10000
      190617:173959:table3d: EOF fili01 rds=4 size=126: sqls/select_students_failing
      190617:173959:table3d: EOF fili01 rds=3 size=136: sqls/select_employee_dept123
      190617:173959:table3d: EOF fild01 rds=13 size=4096: sqls hits=7
      190617:173959:table3d: EOF filo01 wrts=17 size=1240: stats/sqls_FROM_q0i7
      EOJ, Output File written to: stats/sqls_FROM_q0i7
      default command = null, OR enter: more,print,uvlpr12,edit,null --> cat
      # /home/uvadm/demo/stats/sqls_FROM  <-- This report created by uvcopy:
      # uvcopy table3d,fild1=sqls,arg1=FROM
      # =====================================================================
      # - scan all files in directory for a keyword & table counts of following word
      # InDir=sqls  Keyword=FROM  Qual1=   Qual2=
      # WordSepsBlank=  Options=q1a0b3c0d0f0j0l0m0p1s0t0x0w1
      # - default selects following word, use option w2 to table 2nd word, etc
      # - specify arg1=Keyword,arg2=Qualifier-present,arg2=Qualifier-absent
      # &\,$=/>-<:(.');_*[|]"<--- arg4=punctuation to blank for word separation
      # abcdefghjklpqrsuALPRQ<--- equivalents, easier to enter, may enter "all"
      # Date=2019/06/17_11:45:06, Site=UV_Software, Host=uvsoft5, User=uvadm
      #===============================================================================
      #---> uvcopy table3d,fild1=sqls,arg1=FROM
      table3d  2019/06/17_11:45:09  Counts by Targetword following specified Keyword
      tbl#0001  tblt1f7 e0(48)         argument
      line#   1strec#  %      count  target-word
          1         2  28         2   Customers;
          2         2  14         1   EMPLOYEE
          3         2  14         1   Orders
          4         2  42         3   Patients
                      100         7*  *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4G4. Creating table summaries of SQL TABLEs in SQL scripts

PROBLEM - some TABLE names missing ?

If you compare our 1st attempt above to the 8 sql files listed on page '4G2', you will see that there are some missing. This 1st attempt found only 4 TABLE-names, & 6 occurrences, but we know there are 5 TABLE-names & 8 occurrences.

'table3d' expects the target-word to be summarized (SQL TABLEname in this case) to directly follow the keyword ('FROM' in this case), BUT in 4 of the 8 sql files, the keyword 'FROM' is on the line preceding the line with the target-word (TABLEname).

Solution - combine SQL statements onto 1 line

We will use the uvcopy 'combine1d' utility to combine SQL statements onto 1 line. 'arg1=SELECT' will start combining lines until the 'arg4=;' end-of-stmnt marker found and 'arg2=FROM' must occur somewhere within the combined statement.

We will use 'combine1d' to combine the SQL stmnts from all files in sqls/... and write the output combined stmnts into the tmp1/... directory as follows:


 #2. uvcopy "combine1d,fild1=sqls,fild2=tmp1,arg1=SELECT,arg2=FROM,arg4=;,uop=d1r1"
     ==============================================================================
      - copy all files in sqls/* to tmp1/... combining multi-line SQL stmnts onto 1 line
      - we use option 'd1' to drop /*...comments...*/ & option 'r1' to remove the ';' marker
      uop=d1r0t0 - option defaults
          d1     - drop /* ...comments... */
            r1   - remove stmt end pattern
              t0 - no case translation (t1 lower, t2 UPPER)
 190617:172639:combine1d: EOF filo02 wrts=1 size=24: tmp1/select_customers_all
 190617:172639:combine1d: EOF filo02 wrts=1 size=54: tmp1/select_customers_basic
 190617:172639:combine1d: EOF filo02 wrts=1 size=102: tmp1/select_customers_sales10000
 190617:172639:combine1d: EOF filo02 wrts=1 size=102: tmp1/select_customers_sales20000
 190617:172639:combine1d: EOF filo02 wrts=1 size=99: tmp1/select_employee_dept123
 190617:172639:combine1d: EOF filo02 wrts=1 size=78: tmp1/select_employee_seattle
 190617:172639:combine1d: EOF filo02 wrts=1 size=45: tmp1/select_orders_big
 190617:172639:combine1d: EOF filo02 wrts=1 size=73: tmp1/select_patients_over70
 190617:172639:combine1d: EOF filo02 wrts=1 size=73: tmp1/select_patients_over80
 190617:172639:combine1d: EOF filo02 wrts=1 size=73: tmp1/select_patients_over90
 190617:172639:combine1d: EOF filo02 wrts=1 size=54: tmp1/select_students_failing
 190617:172639:combine1d: EOF fild01 size=4096: sqls
 190617:172639:combine1d: EOF fild02 size=4096: tmp1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4G5. Creating table summaries of SQL TABLEs in SQL scripts

Problem Solution - Combine SELECT stmnts onto 1 line


 #2a. cat tmp1/*    <-- list contents of all output files from combine1d
      ==========
      SELECT * FROM Customers
      SELECT Name, Phone, Address, City, Zip FROM Customers
      SELECT custno, Name, Phone, Address, City, Zip, TYsales, LYsales, FROM Customers WHERE LYsales >10000
      SELECT custno, Name, Phone, Address, City, Zip, TYsales, LYsales, FROM Customers WHERE LYsales >20000
      SELECT EMP_ID, NAME, ADDRESS, CITY, ZIP, DEPT FROM EMPLOYEE WHERE DEPT = '123' AND STATUS = 'FULL'
      SELECT EMP_ID, LAST_NAME FROM EMPLOYEE WHERE CITY = 'Seattle' ORDER BY EMP_ID
      SELECT item FROM Orders WHERE quantity > 500
      SELECT Name, Age FROM Patients WHERE Age > 70 GROUP BY Age ORDER BY Name
      SELECT Name, Age FROM Patients WHERE Age > 80 GROUP BY Age ORDER BY Name
      SELECT Name, Age FROM Patients WHERE Age > 90 GROUP BY Age ORDER BY Name
      SELECT S_NAME, Student_ID FROM STUDENT WHERE GPA < 30
Note
  • options default to d1 (drop /*...comments...*/) & r1 (remove end stmnt ';')
  • you might rerun using option d0 (to NOT drop /*...cmts...*/)
  • which would report SELECT stmnt TABLEs found in /*...cmts...*/

rerun table3d inputting SELECT stmnts combined onto 1 line

We will specify the input directory as tmp1/... (output from combine1d above), and the report output of table3d defaults to stats/... Note that table3d names the output report by concatenating the input directory (now tmp1) + the keyword, so in this case the output report will be 'stats/tmp1_FROM'. You could name the output file by adding 'filo1=sqls_FROM' to the command below:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4G6. Creating table summaries of SQL TABLEs in SQL scripts

rerun table3d to create correct summary of SQL TABLEs


 #3. uvcopy table3d,fild1=tmp1,arg1=FROM,uop=q0i7
     ============================================
      enter arg2 qualifier#1 (null=disable) ----->
      enter arg3 qualifier#2 (null=disable) ----->
      enter arg4 chars to blank before word sep (comma,equal,etc) -->
      190617:173015:table3d: EOF fili01 rds=1 size=45: tmp1/select_orders_big hits=1
      190617:173015:table3d: EOF fili01 rds=1 size=73: tmp1/select_patients_over80 hits=1
      190617:173015:table3d: EOF fili01 rds=1 size=24: tmp1/select_customers_all hits=1
      190617:173015:table3d: EOF fili01 rds=1 size=54: tmp1/select_customers_basic hits=1
      190617:173015:table3d: EOF fili01 rds=1 size=78: tmp1/select_employee_seattle hits=1
      190617:173015:table3d: EOF fili01 rds=1 size=73: tmp1/select_patients_over70 hits=1
      190617:173015:table3d: EOF fili01 rds=1 size=102: tmp1/select_customers_sales20000 hits=1
      190617:173015:table3d: EOF fili01 rds=1 size=73: tmp1/select_patients_over90 hits=1
      190617:173015:table3d: EOF fili01 rds=1 size=102: tmp1/select_customers_sales10000 hits=1
      190617:173015:table3d: EOF fili01 rds=1 size=54: tmp1/select_students_failing hits=1
      190617:173015:table3d: EOF fili01 rds=1 size=99: tmp1/select_employee_dept123 hits=1
      190617:173015:table3d: EOF fild01 rds=13 size=4096: tmp1 hits=11
      190617:173015:table3d: EOF filo01 wrts=18 size=1279: stats/tmp1_FROM_q0i7
      EOJ, Output File written to: stats/tmp1_FROM_q0i7
      default command = null, OR enter: vi,cat,more,lp,uvlp12,etc -->
      # /home/uvadm/demo/stats/tmp1_FROM  <-- This report created by uvcopy:
      # uvcopy table3d,fild1=tmp1,arg1=FROM
      # =====================================================================
      # - scan all files in directory for a keyword & table counts of following word
      # InDir=tmp1  Keyword=FROM  Qual1=   Qual2=
      # WordSepsBlank=  Options=q1a0b3c0d0f0j0l0m0p1s0t0x0w1
      # - default selects following word, use option w2 to table 2nd word, etc
      # - specify arg1=Keyword,arg2=Qualifier-present,arg2=Qualifier-absent
      # &\,$=/>-<:(.');_*[|]"<--- arg4=punctuation to blank for word separation
      # abcdefghjklpqrsuALPRQ<--- equivalents, easier to enter, may enter "all"
      # Date=2019/06/17_11:52:30, Site=UV_Software, Host=uvsoft5, User=uvadm
      #===============================================================================
      #---> uvcopy table3d,fild1=tmp1,arg1=FROM
      table3d  2019/06/17_11:52:34  Counts by Targetword following specified Keyword
      tbl#0001  tblt1f7 e0(48)         argument
      line#   1strec#  %      count  target-word
          1         1  36         4   Customers
          2         1  18         2   EMPLOYEE
          3         1   9         1   Orders
          4         1  27         3   Patients
          5         1   9         1   STUDENT
                      100        11*  *TOTAL*

Now you can see that we are reporting TABLE-names found in all 8 SQL scripts (3 have Customers, 2 have EMPLOYEE,& others have only 1 TABLE-name in the script).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4H1. Vancouver Utility Programming Aids - TableJobs

table2 - fixed position fields by displacement(length)

'table2' will build summary tables based on fixed position fields versus 'table3d' which searched records for a Key-Word & tabled the following Target-Word.

We will use $UV/demo/dat1/UScongress (US Government Representatives & Senators) to demonstrate table2. We downloaded an Excell file, extracted a csv file converted to a fixed column layout with uvcopy UScongresscsv2txt. You can see that uvcopy job at UVdemos.htm#10E1.

Here are samples of the 539 records with a column scale, since table2 expects arguments specified by record displacements & lengths (vs table3 which expected Key-Words to table following Target-Words).

samples US Reps & Senators .csv converted to fixed field

Abraham,Ralph,,,,Ralph Lee Abraham,1954-09-16,M,rep,LA,5,,Republican, Adams,Alma,,,,Alma S. Adams,1946-05-27,F,rep,NC,12,,Democrat, Aderholt,Robert,B.,,,Robert B. Aderholt,1965-07-22,M,rep,AL,4,,Republican,

 #        1         2         3         4         5         6         7         8
 #2345678901234567890123456789012345678901234567890123456789012345678901234567890
 001  Abraham             Ralph          19540916  M  LA  rep  Republican
 002  Adams               Alma           19460527  F  NC  rep  Democrat
 003  Aderholt            Robert         19650722  M  AL  rep  Republican
              ---------- omitting 533 lines ----------
 537  Young               Don            19330609  M  AK  rep  Republican
 538  Young               Todd           19720824  M  IN  sen  Republican
 539  Zeldin              Lee            19800130  M  NY  rep  Republican

table2 1st Example for discussion below


 uvcopy table2,fili1=dat1/UScongress,uop=a62b15,arg1=Party,arg2u=Members_by_Party
 ================================================================================
 # Report=/home/uvadm/demo/stats/UScongress_Party_a62b15
 # uvcopy table2,fili1=dat1/UScongress,uop=a62b15,arg1=Party,arg2u=Members_by_Party
 # ===================================================================================
 # - read a file, building a table of counts & values by 1 or 2 arguments, dump at EOF
 # InPutFile=dat1/UScongress
 # Options=a62b15   Title=Members by Party
 # Date=2019/07/05_16:25:17, Site=UV_Software, Host=uvsoft5, User=uvadm
 #====================================================================================
 table2  2019/07/05_16:25:17  Members by Party uop=a62b15
 tbl#0001  tblo1t1  n0($rm48)   <----- argument ----->
 line#   1strec#  %      count  Party
     1         2  52       284  Democrat
     2       265   0         2  Independent
     3         1  46       253  Republican
                 100       539*  *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4H2. Vancouver Utility Programming Aids - TableJobs

table2 - Notes & Benefits

  1. 'table2' allows you to create table summaries for desired fields in your datafiles with fixed length fields - in contrast to the preceding 'table3' & 'table3d' which create table summaries based on key-words known to precede desired target-words.

  2. table2 will prompt for the file type, can be text (with LineFeeds), or fixed length records without LineFeeds, or variable (such as typ=RDW).

  3. Reports are created in subdir stats/... of your current working directory, so it is best to stay in the working directory with your datafile & stats sub-directories below you (ie - do not change into the datafile directories).

  4. Reports are automatically named from the datafilename, fieldnames,& field locations. For example in the sample report above the Report name (on 1st line of report) is: "Report=/home/uvadm/demo/stats/UScongress_Party_a62b15"

  5. This is convenient, you can run many reports without overwriting previous reports, and then use them to research problems or to assess the impacts of planned changes.

  6. The reports include all relevant information (inputs, fieldnames, locations, options, date/time, site, host,& userid).

  7. We recommend you specify all parameters on the command line, but you could wait & reply to the prompts for filename, field location displacement & length, fieldname,& options. Here is our sample command with all parameters:

     uvcopy table2,fili1=dat1/UScongress,uop=a62b15,arg1=Party,arg2u=Members_by_Party
     ================================================================================
  1. Specifying all parameters on the command line makes it easy to use the command search history to recall, modify,& repeat desired commands.

  2. 'arg1=...' specifies the description for the heading above the table field-data & 'arg2=...' specifies an optional Title=... to be included in the report header. If specified on the command line, any spaces between words must be coded as '_' underscores & specify option 'u' on arg1u=... & arg2u=...

  3. Note that the main objective of these demos is to give you the knowledge to run these table utilities on your own data-files to achieve your objectives.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4H3. Vancouver Utility Programming Aids - TableJobs

table2 Execution & Console-Log

The sample above showed the command line & the report created, and now we will illustrate the console-log prompts for options, arguments, table fieldnames, & prompt to view output report.


 uvcopy table2,fili1=dat1/UScongress,uop=a62b15,arg1=Party,arg2u=Members_by_Party
 ================================================================================

Console Log - options & prompts

 uop=a0b0c0d0e0f0g0h0j0n0p0r0x1 - option defaults
 uop=a0                 - argument displacement (0 relative)
 uop=  b0               - argument length
 uop=    c0             - argument displacement (0 relative)
 uop=      d0           - argument length
 uop=        e0         - acum#1 dsplcmnt (omit c,d,e,f for counts only)
 uop=          f0       - acum#1 length
 uop=            g0     - acum#2 displacement (0 relative)
 uop=              h0   - acum#2 length
 uop=            j1     - edit all acums as qty (commas, no decimal)
 uop=            j2     - edit all acums as $ (commas & decimal point)
 uop=            j__    - table formats j1-j12 see uvcopy.doc tbf f1-f12
 uop=          n0       - argument n0=unpacked, n1=packed, n2=binary
 uop=        p0         - acum-fields p0=unpacked, p1=packed, p2=binary
 uop=          r8192    - input record size (max 8192)
 uop=               x1  - bypass any #comments in data file
 uop=              note - if file type LST specify r > largest possible
 uop=a30b6c38d6e53f9g3r64x1 <-- ENTER this for demo with dat1/sales3 file
 product#;quantity;amount   <-- ENTER this for demo Field Headings
 Summary of Qty & Amt by Product#  <-- ENTER this for demo Report Heading

User OPtion (uop) defaults = q1a0b0c0d0e0f0g0h0j0n0p0r0x1a62b15 -->null to accept or enter/override --> recsize (option r) unspcfd, default 2048 OK text files, may continue acum field#1 (option f) length unspcfd, may continue to count only enter argument & acum fieldnames with semicolon separators -->

 LST = typ default fili01 - null accept or enter new typ ?
 190705:174132:table2: EOF fili01 rds=539 size=38781: dat1/UScongress
 190705:174132:table2: EOF filo01 wrts=11 size=886: stats/UScongress_Party_a62b15
 EOJ, Output File written to: stats/UScongress_Party_a62b15
 default command = more, OR enter: vi,cat,more,lp,uvlp12,etc --> more <--
Note
  • table2 option prompts for command (optional) to view output report
  • you could reply 'more' to display (shortened version below)
 table2  2019/07/05_17:41:32  Members by Party uop=a62b15
 tbl#0001  tblo1t1  n0($rm48)   <----- argument ----->
 line#   1strec#  %      count  Party
     1         2  52       284  Democrat
     2       265   0         2  Independent
     3         1  46       253  Republican
                 100       539*  *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4I1. Table2 Summaries of US Representatives & Senators

4 Table Summaries from dat1/UScongress

You can run various table summaries from the dat1/UScongress demo file. We will show a few of the shorter reports, followed by script sf/UScongress_stats, which runs 13 reports that we thought were interesting.

Here is a reminder of the UScongress file layout, in case you want to try making up your own commands to run table summaries on other field combinations.

 #        1         2         3         4         5         6         7         8
 #2345678901234567890123456789012345678901234567890123456789012345678901234567890
 001  Abraham             Ralph          19540916  M  LA  rep  Republican
 002  Adams               Alma           19460527  F  NC  rep  Democrat
 003  Aderholt            Robert         19650722  M  AL  rep  Republican

table US Represenatives/Senators by Party


 #1. uvcopy table2,fili1=dat1/UScongress,uop=a62b15,arg1=Party,arg2u=Members_by_Party
     ================================================================================
 # Report=/home/uvadm/demo/stats/UScongress_Party_a62b15
 # uvcopy table2,fili1=dat1/UScongress,uop=a62b15,arg1=Party,arg2u=Members_by_Party
 # ===================================================================================
 # - read a file, building a table of counts & values by 1 or 2 arguments, dump at EOF
 # InPutFile=dat1/UScongress
 # Options=a62b15   Title=Members by Party
 # Date=2019/07/05_16:25:17, Site=UV_Software, Host=uvsoft5, User=uvadm
 #====================================================================================
 table2  2019/07/05_16:25:17  Members by Party uop=a62b15
 tbl#0001  tblo1t1  n0($rm48)   <----- argument ----->
 line#   1strec#  %      count  Party
     1         2  52       284  Democrat
     2       265   0         2  Independent
     3         1  46       253  Republican
                 100       539*  *TOTAL*

Notes

  1. You must enter the field displacement zero relative. In the above example the the options for Party field location are defined as 'uop=a62b15'. Option 'a62' defines the displacemnt & option 'b15' defines the length of the Party field.

  2. From the scale, you can see that the Party field begins in column 63 'one relative' which is 62 'zero relative'.

  3. The 1st column of the scale is '#' rather than '1', because option 'x1' allows records with '#' in column 1 to be bypassed (avoiding bad data in table). Allows you to add scales & other info in your data files if you so choose.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4I2. Table2 Summaries of US Representatives & Senators

table US Represenatives/Senators by House & Party


 #2. uvcopy table2,fili1=dat1/UScongress,uop=a57b3c62d15,arg1=House+Party,arg2u=Members_by_House+Party
     ==================================================================================================
 # Report=/home/uvadm/demo/stats/UScongress_House+Party_a57b3c62d15
 # uvcopy table2,fili1=dat1/UScongress,uop=a57b3c62d15,arg1=House+Party,arg2u=Members_by_House+Party
 # ===================================================================================
 # - read a file, building a table of counts & values by 1 or 2 arguments, dump at EOF
 # InPutFile=dat1/UScongress
 # Options=a57b20   Title=Members by House+Party
 # Date=2019/07/05_16:25:16, Site=UV_Software, Host=uvsoft5, User=uvadm
 #====================================================================================
 table2  2019/07/05_16:25:17  Members by House+Party uop=a57b20
 tbl#0001  tblo1t1  n0($rm48)   <----- argument ----->
 line#   1strec#  %      count  House+Party
     1         2  44       239  rep  Democrat
     2         1  37       200  rep  Republican
     3        18   8        45  sen  Democrat
     4       265   0         2  sen  Independent
     5         5   9        53  sen  Republican
                 100       539*  *TOTAL*

Notes

  1. This example illustrates using 2 table arguments for House (sen or rep) & Party. Re: options 'uop=a57b3c62d15' - a57b3 specifies the House sen/rep in cols 58-60, a57 displacemnt & b3 length - c62d15 specifies the Party in columns 63-77, c62 displacment & d15 length

  2. arg1=House+Party specifies the field headings to appear directly above the data fields tabled on line 11 of the report. arg2u=Members_by_House+Party is an optional heading that will be inserted on line 9 of the report.

  3. We did not need to use option u on arg1 since no spaces in "House+Party", but did need option u on arg2u since we coded space with '_' underscores.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4I3. Table2 Summaries of US Representatives & Senators

table US Represenatives/Senators by Party & Sex


 #3. uvcopy table2,fili1=dat1/UScongress,uop=a62b15c50d1,arg1=Party+Sex,arg2u=Members_by_Party+Sex
     =============================================================================================
 # Report=/home/uvadm/demo/stats/UScongress_Party+Sex_a62b15c50d1
 # uvcopy table2,fili1=dat1/UScongress,uop=a62b15c50d1,arg1=Party+Sex,arg2u=Members_by_Party+Sex
 # ===================================================================================
 # - read a file, building a table of counts & values by 1 or 2 arguments, dump at EOF
 # InPutFile=dat1/UScongress
 # Options=a62b15c50d1   Title=Members by Party+Sex
 # Date=2019/07/05_16:25:17, Site=UV_Software, Host=uvsoft5, User=uvadm
 #====================================================================================
 table2  2019/07/05_16:25:17  Members by Party+Sex uop=a62b15c50d1
 tbl#0001  tblo1t1  n0($rm48)   <----- argument ----->
 line#   1strec#  %      count  Party+Sex
     1         2  20       108  Democrat        F
     2         4  32       176  Democrat        M
     3       265   0         2  Independent     M
     4         9   4        23  Republican      F
     5         1  42       230  Republican      M
                 100       539*  *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4I4. Table2 Summaries of US Representatives & Senators

table US Represenatives/Senators by State & Party


 #4. uvcopy table2,fili1=dat1/UScongress,uop=a53b2c62d15,arg1=State+Party,arg2u=Members_by_State+Party
     =================================================================================================
 # Report=/home/uvadm/demo/stats/UScongress_State+Party_a53b2c62d15
 # uvcopy table2,fili1=dat1/UScongress,uop=a53b2c62d15,arg1=State+Party,arg2u=Members_by_State+Party
 # ===================================================================================
 # - read a file, building a table of counts & values by 1 or 2 arguments, dump at EOF
 # InPutFile=dat1/UScongress
 # Options=a53b2c62d15   Title=Members by State+Party
 # Date=2019/07/05_16:25:22, Site=UV_Software, Host=uvsoft5, User=uvadm
 #====================================================================================
 table2  2019/07/05_16:25:22  Members by State+Party uop=a53b2c62d15
 tbl#0001  tblo1t1  n0($rm48)   <----- argument ----->
 line#   1strec#  %      count  State+Party
     1       346   0         3  AK Republican
     2       246   0         2  AL Democrat
     3         3   1         7  AL Republican
     4        40   1         6  AR Republican
     5         9   0         1  AS Republican
         ------------- 83 lines omitted -------------
    89        18   0         4  WI Democrat
    90       141   1         6  WI Republican
    91       309   0         1  WV Democrat
    92        63   0         4  WV Republican
    93        21   0         3  WY Republican
                 100       539*  *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4J1. Table2 Summaries of US Representatives & Senators

script to run 13 table summaries on dat1/UScongress

 #!/bin/ksh
 # UScongress_stats - run table2 to create multi stats/reports from dat1/UScongress demo file
 #                  - by owen Townsend, UV Software, Jul04/2019
 # See details at uvsoftware.ca/tablejobs.htm or uvsoftware.ca/uvdemos.htm
 export UVCOPYROP=q0i63r0   # minimize console prompts
 uvcopy table2,fili1=dat1/UScongress,uop=a40b4,arg1=BirthYear,arg2u=Members_by_BirthYear
 uvcopy table2,fili1=dat1/UScongress,uop=a57b3,arg1=House,arg2u=Members_by_House
 uvcopy table2,fili1=dat1/UScongress,uop=a57b20,arg1=House+Party,arg2u=Members_by_House+Party
 uvcopy table2,fili1=dat1/UScongress,uop=a57b3c50d1,arg1=House+Sex,arg2u=Members_by_House+Sex
 uvcopy table2,fili1=dat1/UScongress,uop=a62b15,arg1=Party,arg2u=Members_by_Party
 uvcopy table2,fili1=dat1/UScongress,uop=a62b15c50d1,arg1=Party+Sex,arg2u=Members_by_Party+Sex
 uvcopy table2,fili1=dat1/UScongress,uop=a62b15c53d2,arg1=Party+State,arg2u=Members_by_Party+State
 uvcopy table2,fili1=dat1/UScongress,uop=a50b1,arg1=Sex,arg2u=Members_by_Sex
 uvcopy table2,fili1=dat1/UScongress,uop=a50b1c57d3,arg1=Sex+House,arg2u=Members_by_Sex+House
 uvcopy table2,fili1=dat1/UScongress,uop=a50b1c62d15,arg1=Sex+Party,arg2u=Members_by_Sex+Party
 uvcopy table2,fili1=dat1/UScongress,uop=a53b2,arg1=State,arg2u=Members_by_State
 uvcopy table2,fili1=dat1/UScongress,uop=a53b2c57d3,arg1=State+House,arg2u=Members_by_State+House
 uvcopy table2,fili1=dat1/UScongress,uop=a53b2c62d15,arg1=State+Party,arg2u=Members_by_State+Party
 echo "Output Reports in $PWD/stats - listed with line counts"
 llc stats
 #========
 File#   Lines
     1      69 stats/UScongress_BirthYear_a40b4
     2      15 stats/UScongress_House_a57b3
     3      18 stats/UScongress_House+Party_a57b3c62d15
     4      17 stats/UScongress_House+Sex_a57b3c50d1
     5      16 stats/UScongress_Party_a62b15
     6      18 stats/UScongress_Party+Sex_a62b15c50d1
     7     106 stats/UScongress_Party+State_a62b15c53d2
     8      15 stats/UScongress_Sex_a50b1
     9      17 stats/UScongress_Sex+House_a50b1c57d3
    10      18 stats/UScongress_Sex+Party_a50b1c62d15
    11      69 stats/UScongress_State_a53b2
    12     119 stats/UScongress_State+House_a53b2c57d3
    13     106 stats/UScongress_State+Party_a53b2c62d15
    13     603 total lines in    13 files in directory stats

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4K1. Vancouver Utility Programming Aids - TableJobs

Table2 Summaries of Canadian Members of Parliament

'table2' will build summary tables based on fixed position fields versus 'table3d' which searched records for a Key-Word & tabled the following Target-Word.

We will use $UV/demo/dat1/CanadaMPs (CAnadian Members of Parliament) to demo table2. We downloaded a spreadsheet, extracted a csv file (dat1/CanadaMPs.csv) & converted to a fixed layout so we could define specific fields required for some selections. See UVdemos.htm#10D1 if interested to see the uvcopy job (demo/pf/CanadaMPscsv2txt).

Here are the samples of the 335 records with a column scale to help you determine field locations. You must specify zero relative displacement with option 'a' and length with option 'b'. For example, use 'uop=a70b2' for the Province in columns 71-72.

samples of Canadian MPs .csv converted to fixed fields

Title,First Name,Last Name,Constituency,Province/Territory,Political Affiliation,Start Date ,Ziad,Aboultaif,Edmonton Manning,Alberta,Conservative,10/19/2015 ,Dan,Albas,Central Okanagan/Similkameen/Nicola,British Columbia,Conservative,10/19/2015

          1         2         3         4         5         6         7         8         9       100
 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
               ----------------- First 2 of 335 MP's ------------------
 001  Aboultaif          Ziad            Edmonton Manning              AB Conservative     2015
 002  Albas              Dan             Central Okanagan/Similkameen/ BC Conservative     2015
               --------------- samples of all parties --------------
 010  Anderson           David           Cypress Hills/Grasslands      SK Conservative     2015
 011  Angus              Charlie         Timmins/James Bay             ON NDP              2015
 023  Barsalou-Duval     Xavier          Pierre-Boucher/Les Patriotes/ QC Bloc Quebecois   2015
 032  Bernier            Maxime          Beauce                        QC People's Party   2015
 202  May                Elizabeth       Saanich/Gulf Islands          BC Green Party      2015
 328  Wilson-Raybould    Jody            Vancouver Granville           BC Independent      2015
               ----------------- Last 2 of 335 MP's ------------------
 334  Zahid              Salma           Scarborough Centre            ON Liberal          2015
 335  Zimmer             Bob             Prince George/Peace River/Nor BC Conservative     2015

We will build 3 tables: by Province cols 71-72, Party cols 74-88,& YearElected cols 91-94

Here are the 3 commands we will use, see results on the followng pages:


 #1. uvcopy table2,fili1=dat1/CanadaMPs,uop=a70b2,arg1=Province,arg2u=CDN_MPs_by_Province
     ====================================================================================
 #2. uvcopy table2,fili1=dat1/CanadaMPs,uop=a73b17,arg1=Party,arg2u=CDN_MPs_by_Party
     ===============================================================================
 #3. uvcopy table2,fili1=dat1/CanadaMPs,uop=a90b4,arg1=YearElected,arg2u=CDN_MPs_by_YearElected
     ==========================================================================================

Notes

  1. We enter all arguments on the command line, but you could omit & enter at the prompts

  2. For spaces within arguments, you must code them as '_' undercores & use option 'u' - for example 'arg3u=CDN_MPs_by_Province'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4K2. Table2 Summaries of Canadian Members of Parliament

3 Table Summaries from dat1/CanadaMPs


 #1. uvcopy table2,fili1=dat1/CanadaMPs,uop=a70b2,arg1=Province,arg2u=CDN_MPs_by_Province
     ====================================================================================

table CA MPs by Province

 # Report=/home/uvadm/demo/stats/CanadaMPs_Province_a70b2
 # uvcopy table2,fili1=dat1/CanadaMPs,uop=a70b2,arg1=Province,arg2u=CDN_MPs_by_Province
 # ====================================================================================
 # - read a file, building a table of counts & values by 1 or 2 arguments, dump at EOF
 # InPutFile=dat1/CanadaMPs
 # Options=a70b2   Title=CDN MPs by Province
 # Date=2019/07/06_11:42:06, Site=UV_Software, Host=uvsoft5, User=uvadm
 #====================================================================================
 table2  2019/07/06_11:42:08  CDN MPs by Province uop=a70b2
 tbl#0001  tblo1t1  n0($rm48)   <----- argument ----->
 line#   1strec#  %      count  Province
     1         1  10        34  AB
     2         2  12        41  BC
     3        15   4        14  MB
     4        13   2        10  NB
     5       147   2         7  NL
     6        59   2        10  NS
     7       212   0         1  NT
     8       310   0         1  NU
     9         3  36       121  ON
    10        58   1         4  PE
    11         8  22        77  QC
    12        10   4        14  SK
    13        19   0         1  YT
                 100       335*  *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4K3. Table2 Summaries of Canadian Members of Parliament

table CA MPs by House & Party


 #2. uvcopy table2,fili1=dat1/CanadaMPs,uop=a73b17,arg1=Party,arg2u=CDN_MPs_by_Party
     ===============================================================================
 # Report=/home/uvadm/demo/stats/CanadaMPs_Party_a73b17
 # uvcopy table2,fili1=dat1/CanadaMPs,uop=a73b17,arg1=Party,arg2u=CDN_MPs_by_Party
 # ===================================================================================
 # - read a file, building a table of counts & values by 1 or 2 arguments, dump at EOF
 # InPutFile=dat1/CanadaMPs
 # Options=a73b17   Title=CDN MPs by Party
 # Date=2019/07/06_11:42:08, Site=UV_Software, Host=uvsoft5, User=uvadm
 #====================================================================================
 table2  2019/07/06_11:42:09  CDN MPs by Party uop=a73b17
 tbl#0001  tblo1t1  n0($rm48)   <----- argument ----->
 line#   1strec#  %      count  Party
     1        23   2        10  Bloc Quebecois
     2       325   0         1  Co-operative Co
     3         1  28        96  Conservative
     4       195   0         2  Green Party
     5        52   2         7  Independent
     6         4  52       177  Liberal
     7        11  12        41  NDP
     8        32   0         1  People's Party
                 100       335*  *TOTAL*

table CA MPs by Year-Elected


 #3. uvcopy table2,fili1=dat1/CanadaMPs,uop=a90b4,arg1=YearElected,arg2u=CDN_MPs_by_YearElected
     ==========================================================================================
 # Report=/home/uvadm/demo/stats/CanadaMPs_YearElected_a90b4
 # uvcopy table2,fili1=dat1/CanadaMPs,uop=a90b4,arg1=YearElected,arg2u=CDN_MPs_by_YearElected
 # ===================================================================================
 # - read a file, building a table of counts & values by 1 or 2 arguments, dump at EOF
 # InPutFile=dat1/CanadaMPs
 # Options=a90b4   Title=CDN MPs by YearElected
 # Date=2019/07/06_11:42:09, Site=UV_Software, Host=uvsoft5, User=uvadm
 #====================================================================================
 table2  2019/07/06_11:42:10  CDN MPs by YearElected uop=a90b4
 tbl#0001  tblo1t1  n0($rm48)   <----- argument ----->
 line#   1strec#  %      count  YearElected
     1         1  94       317  2015
     2       222   0         1  2016
     3        30   3        11  2017
     4        22   0         2  2018
     5        27   1         4  2019
                 100       335*  *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4K4. Table2 Summaries of Canadian Members of Parliament

script to run multiple stats on dat1/CanadaMPs

 #!/bin/ksh
 # CandaMPs_stats - run table2 to create multi stats/reports from dat1/CanadaMPs demo file
 #                - by owen Townsend, UV Software, Jul06/2019
 # See details at uvsoftware.ca/tablejobs.htm or uvsoftware.ca/uvdemos.htm
 export UVCOPYROP=q0i63r0   # minimize console prompts
 #
 uvcopy table2,fili1=dat1/CanadaMPs,uop=a70b2,arg1=Province,arg2u=CDN_MPs_by_Province
 #==================================================================================
 #
 uvcopy table2,fili1=dat1/CanadaMPs,uop=a73b17,arg1=Party,arg2u=CDN_MPs_by_Party
 #==============================================================================
 #
 uvcopy table2,fili1=dat1/CanadaMPs,uop=a90b4,arg1=YearElected,arg2u=CDN_MPs_by_YearElected
 #=========================================================================================
 echo "Output Reports in $PWD/stats/... - listed below with line counts"
 llc stats
 #========
 File#   Lines
     1      21 demo/stats/CanadaMPs_Party_a73b17
     2      26 demo/stats/CanadaMPs_Province_a70b2
     3      18 demo/stats/CanadaMPs_YearElected_a90b4
     4      65 total lines in    3 files in directory demo/stats

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_5 UVdemos.doc - Vancouver Utilities for Unix/Linux users

Part 5 most useful UVscripts

Part5 documents some of the most useful shell scripts used at UV Software. Our emphasis here is to illustrate how useful these scripts can be to anybody who works with files & directories on the command line.

We will demo just a few of the most useful scripts here. You can see many more listed in scripts1.htm and if the Vancouver Utilities are installed at your site, you can see hundreds more in $UV/sf/... 4 subdirs (adm,demo,util,IBM).

And if you have the Utiltiies installed you can run the scripts with the demo files provided & check your results to the expected results listed.

UVdemos Part5 Contents


5A1. Initial Example (cfd) to get you interested in the cfd... scripts.

5B1. Setups & Checklist required to run UVscripts.
- Vancouver Utilities installed
- user login setup with stub profile calling common_profile in $UV/env/
- demo files copied to demo/ subdir in your homedir

5C1. Setup check-list - Ready to run UVscripts test/demos ?
See UVdemos.htm#Part_1 for complete setup procedures, but here is a short
list of the steps required before you can execute these UVscript test/demos.

5C2. test data files to demo UVscripts
- CanadaMPs, CanadaProvinces, UScongress, USstates

5D1. 'cfd' scripts for Counting Files,Lines,KB in 1 Directory.
Test/Demos you can execute & compare your results to our results.

9 cfd... scripts as follows

      1. cfl   - Count Lines in 1 File
      2. cfd   - Count Files,Lines,KB in 1 Directory
      3. cfdt  - Totals-Only version of Count Files,Lines,& KiloBytes in a Directory
      4. cfdpf - Count Files in a Directory with a Pattern [or not] in filenames
      5. cfdpl - Count Files in a Directory with a Pattern [or not] on any line in any file
      6. cfdd  - Count Files,Lines,& KB in a ALL Sub-Dirs in a Super-Directory
      7. cfddt - Count Files,Lines,& KB in a ALL Sub-Dirs in a Super-Directory Totals-Only
      8. cfddf - Count Files,Lines,KB in ALL Sub-Dirs in a Super-Dir + 1st few files
      9. cfdmm - List Directory: File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#, Dir/Filename

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

UVdemos Part5 Contents - continued


5E1. cfhelp - Help screen to remind you of the 9 cfd scripts,
- just enter 'cfhelp' at your prompt

5E2. Vancouver Utility 'uvhelp' main menu & 'v12' sub-menu also include the help
menus for the cfd... scripts to count Lines & Files.

5F1. cftest - script to run all 9 cfd scripts in batch mode
- reports collected in subdir rpts/... of current working directory

5G1. Recommendations using cfd... scripts on your own directories

5H1. Listings of cfd... scripts. If interested to see listings of all 9 scripts
please see uvsoftware.ca/scripts1.htm#Part_5.

5H2. Listing here of 1 script 'cfd' to count Files,lines,KB in 1 directory

5I1. Directory listing scripts based on 'ls -l' with various options for sorting
sequence by date/time, filesize, etc - llm,lla,lld,llr,llt,llu,etc.

5J1. 'rename' scripts - can save you a lot of manual keying when importing a lot
 of long awkward filenames from other systems (such as mainframes).
 Here are just a few of the 27 scripts available:
renameL - rename all filenames in a directory to lower case
renameU - rename all filenames in a directory to UPPER case
renameX - rename all files, changing 1 specified extension to an alternate
rename-X - remove specified extension from all files in directory
rename+X - add a specified extension to all files in directory

5K1. Remove/Insert Carriage Returns (removeCR & insertCR)
- often a problem when importing/exporting files from/to other systems.
 Special problem receiving scripts from other systems or by email
- the invisible CRs prevent scripts from executing on Unix/Linux systems

5L1. 'alldiff' scripts report differences between all files in directories
 Invaluable when making major changes to your software & datafiles
- save your programs & files before changes & reconversions
- run alldiff2 to confirm changes as intended & no unintended changes.
 'alldiff3' allows you to ignore differences due to date/time stamps, etc
- may add any unique pattern to a table of differences to be ignored.

5M1. uvcmpFA1 - compare files with packed/binary files & without LineFeeds
- not possible with unix 'diff' & the 'alldiff2/3' scripts.
 Several versions of uvcmp... for ASCII,EBCDIC,fixed/variable length files.
 Displays unmatched record pairs in vertical hexadecimal with '*'s flags on
 unmatched bytes. Demo using sales records with 24 5 byte packed fields.

5M3. Using uvhd to update packed/binary files - how we created custmas2 with
4 bytes different in only 1 record of 32, for the uvcmpFA1 compare demo.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

UVdemos Part5 Contents - continued


5N1. 'chmod' scripts to correct file permissions in directory trees.
- automatically set permissions as desired on directories & files
  Essential for directories shared by multiple users
- directories must be 775 & files must be 664

5O1. lsold/mvold/rmold scripts to list/move/remove files older than x days.
 lsnew/mvnew/rmnew scripts to list/move/remove files newer than x days.
- can find files anywhere in your directories & list/move/remove as desired

5P1. 'dtree' script to draw directory tree illustrations.
- very useful for documentation (note usage in this documentation).

UVscripts.htm is the same as UVdemos.htm#Part_5. UVscripts was duplicated as a separate document because it is 1 of the most useful parts of UVdemos.

SelectJobs.htm (same as UVdemos.htm#Part_3) is also a separate document.


TableJobs.htm (same as UVdemos.htm#Part_4) is also a separate document.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5A1. UVscripts - Introduction & Overview

Initial Example to get you interested


 cfdmm dat1 - report Files,Lines,KB, Min/Max RecSize & RecNumbers in directory dat1/...
 ==========
 cfdmm dat1 - list File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#,Dir/Filename
  File#   Lines      KB MinRsiz MaxRsiz MinRnum MaxRnum  Directory/FileName
      1     335      32      95      95       1     335  dat1/CanadaMPs
      2      13       4      14      34      13       5  dat1/CanadaProvinces
      3       8       4      70      75       6       7  dat1/nameadrs1
      4     305      28      75      82       2       1  dat1/UScities
      5     539      40      71      74       2     422  dat1/UScongress
      6      50       4      13      23      15      40  dat1/USstates
      6    1250     112      13      95                  *Totals* in dat1
                            Report left in --> /home/uvadm/demo/rpts/dat1_cfdmm

UVscripts Benefits to Programmers & Analysts

  1. The 9 cfd... scripts automatically create their reports in subdir rpts/... & the reports are automatically named from the directory & script name. Any pattern arguments will be included in the rpts/filename, For example the report created by 'cfdp dat1 Canada*' will be 'rpts/dat1_cfdp_Canada_'.

  2. The reports include all relevant information - command line used, input directory, any patterns, date/time, full path-name of report created.

  3. The above conventions are convenient, you can run many reports without overwriting previous reports, and then use them to research problems or to assess the impacts of planned changes.

  4. The main objective of these demos is to give you the knowledge to run the scripts on your own directories to achieve your own objectives.

    UVscripts same as UVdemos Part 5

UVscripts.htm is the same as UVdemos.htm#Part_5. UVscripts was duplicated as a separate document because it is 1 of the most useful parts of UVdemos.

Because UVscripts.doc is also a separate document from UVdemos.doc, the next 2 pages will duplicate the setup procedure for users to copy the demo files to a demo/ subdir within their homedir (rather than working in the $UV homedir.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5B1. Vancouver Utility Programming Aids - UVscripts

Vancouver Utilities - subdirs relevant to UVscripts

Here is the Vancouver Utilities homedir showing only the most relevant subdirs required for the UVscript demos documented in this section of UVdemos.htm.

You could run the demos logged in as 'uvadm', but if you are sharing the system with other uvadm users, it would be better to copy the demo/subdirs/files to your own homedir. See the copy instructions below & an expanded view of the demo/... files on the next page.

 /home/uvadm             <-- $UV
 :-----bin                 - UV binaries (uvcopy,uvsort,uvcp,uvhd,uvlist,etc)
 :-----dat1                - test data files to demo various Vancouver Utilities
 :     :-----...         <-- 200 testdata files for all uvcopy job demos
 :     :                   - UVdemos testdata files copied to demo/dat1/...
 :-----doc                 - Vancouver Utilities documentation (text)
 :     :-----UVdemos.doc   - this documentation (1 of 150 files)
 :     :-----UVscripts.doc - Part5 of UVdemos duplicated as a separate document
 :-----dochtml             - documentation in HTML (uploaded to www.uvsoftware.ca)
 :-***-demo              <-- demo files for TestDemo tutorials, copy to your homedir -->
 :     :                   - copy /home/uvadm/demo/* to your /home/userxx/demo/...
 :     :-----dat1          - data files for UVscript demos
 :     :--. ..etc...       - see details on the next page
 :-----pf               <-- Parameter Files for uvcopy
 :     :-----adm               - uvcopy jobs for uvadm admin
 :     :-----demo              - uvcopy jobs for various demos
 :     :-----util              - uvcopy jobs for users
 :     :-----IBM               - uvcopy jobs for Mainframe conversions
 :-----sf                <-- Script Files (bash or Korn shell)
 :     :-----adm               - scripts for uvadm admin
 :     :-----demo              - demo scripts
 :     :-----util     <---***--- includes scripts (cfd,cfdd,etc) in this UVscripts.doc
 :     :-----IBM               - scripts for mainframe conversions
 :-----src               <-- Vancouver Utilities C source code (uvcopy,uvsort,etc)
 :-----tf                  - test files for various demos (similar to dat1/...)
 :-----tmp                 - tmp subdir (test/demo outputs)
 :-----tmp1                - misc, outdir for some utility scripts (like listall1)

Please copy the demo/... files to your homedir to avoid conflict with other users of Vancouver Utilities & to avoid losing your files when new versions of Vancouver Utilities are installed.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5B2. Vancouver Utility Programming Aids - UVscripts

UVscripts demo files copied to your homedir

Here is an expanded view of the most relevant subdirs/files required for the UVscript demos (omitting subdirs/files used for other parts of UVdemos.doc).

 /home/userxx/demo/             <-- your homedir demo directory
 :-----archive
 :     :---...misc...
 :-----backup                   <-- backup ParentDir
 :     :-----dat0                 - files in subdirs same as below
 :     :-----dat1
 :     :-----dat2
 :-----dat0                     <-- originals converted to dat1 fixed formats
 :     :-----CanadaMPs.csv
 :     :-----UScities.tab
 :     :-----UScongress.csv
 :     :-----USstates.csv
 :-----dat1                     <-- PRIMARY DEMO FILES for UVscripts
 :     :-----CanadaMPs
 :     :-----CanadaProvinces
 :     :-----nameadrs1
 :     :-----UScities
 :     :-----UScongress
 :     :-----USstates
 :-----dat2                     <-- N/A to UVscripts
 :     :-----accents1             - used for other parts of UVdemos
 :     :-----ar.custmas
 :     :-----ar.custmas.tellist
 :     :-----                     - several files omitted
 :     :-----nameadrs2
 :     :-----sales3
 :     :-----vendormas2
 :-----ftps
 :-----jcl0
 :-----jcl2   - N/A to UVscripts
 :-----jcl3
 :-----pf
 :     :-----CanadaMPscsv2txt   <-- uvcopy to convert .csv to fixed fields
 :     :-----UScitiescsv2txt
 :     :-----UScongresscsv2txt
 :     :-----USstatescsv2txt
 :-----rpts
 :     :-----dat1_cfd           <-- Reports created by UVscripts
 :     :----- ...                 - named from directory + scriptname
 :     :----- ...
 :-----sf    - scripts for other UVdemos
 :     :----- ... UVscripts are in $UV/sf/util/... (see prior page)
 :-----sql0  - N/A to UVscripts
 :-----sqls  - for other UVdemos
 :-----tmp

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5C1. Vancouver Utility Programming Aids - TableJobs

Setup check-list - Ready to run UVscripts test/demos ?

We will present a 'setup check-list' here, since you may be reading this in the separate UVscripts.doc vs UVdemos.doc which includes UVscripts as Part5. See UVdemos.htm#Part_1 for complete setup procedures, but here is a check-list of the steps required before you can execute these UVscript test/demos.

  1. Install Vancouver Utilities. See complete documenation at install.htm or the install summary in UVdemos.htm#1A3.

  2. Setup your user login account if not already done. See UVdemos.htm#1C1. Setup your user profile (.bash_profile or .profile for ksh). User profile in homedir calling common_profile in /home/appsadm/env/...

  3. Setup subdir demo/ in your homedir & copy $UV/demo/* to your $HOME/demo/... See instructions below. See the demo file directories on the previous page.

    copy $UV/demo to your $HOME/demo/...


 #1. Login userxx --> /home/userxx
 #2. mkdir demo   <-- make directory for copy of demo files in your homedir

 #3. cp -r /home/uvadm/demo/* demo/   <-- copy demo files to your homedir
     ==============================

 #4. cd demo    <-- change into demo/ directory
     =======      - see demo/... subdirs & files on prior paage
 :-----dat1     <-- PRIMARY FILES for UVscripts demos
 :     :-----CanadaMPs
 :     :-----CanadaProvinces
 :     :-----nameadrs1
 :     :-----UScities
 :     :-----UScongress
 :     :-----USstates

 #5. vi dat1/*   <-- look at demo file contents if you wish
     =========

Most of the cfd* scripts are not concerned with the file contents, but rather with counting the files, lines,& KB of the files. Some scripts such as 'cfdpf' selects files based on a pattern in the filenames. Only 1 script (cfdpl) selects files based on a pattern in the contents.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5C2. Demo files for Tutorials

dat1/CanadaMPs - 334 members - samples of all parties

 010  Anderson           David           Cypress Hills/Grasslands      SK Conservative     2015
 011  Angus              Charlie         Timmins/James Bay             ON NDP              2015
 023  Barsalou-Duval     Xavier          Pierre-Boucher/Les Patriotes/ QC Bloc Quebecois   2015
 032  Bernier            Maxime          Beauce                        QC People's Party   2015
 202  May                Elizabeth       Saanich/Gulf Islands          BC Green Party      2015
 328  Wilson-Raybould    Jody            Vancouver Granville           BC Independent      2015

dat1/CanadaProvinces

      01  AB  Alberta
      02  BC  British Columbia
      03  MB  Manitoba
      04  NB  New Brunswick
      05  NL  Newfoundland and Labrador
      06  NT  Northwest Territories
      07  NS  Nova Scotia
      08  NU  Nunavut
      09  ON  Ontario
      10  PE  Prince Edward Island
      11  QC  Quebec
      12  SK  Saskatchewan
      13  YT  Yukon

dat1/UScongress - 1st 5 & last of 539 members

 001  Abraham             Ralph          19540916  M  LA  rep  Republican
 002  Adams               Alma           19460527  F  NC  rep  Democrat
 003  Aderholt            Robert         19650722  M  AL  rep  Republican
 004  Aguilar             Pete           19790619  M  CA  rep  Democrat
 005  Alexander           Lamar          19400703  M  TN  sen  Republican
              ---------- omitting 533 lines ----------
 539  Zeldin              Lee            19800130  M  NY  rep  Republican

dat1/USstates - 1st 5 & last of 50 states

 01  AL  Alabama
 02  AK  Alaska
 03  AZ  Arizona
 04  AR  Arkansas
 05  CA  California
        -------- omitting 44 records --------
 50  WY  Wyoming

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D1. Counting Files,Lines,& KB in directories

Run test/demos for 9 cfd* scripts


 #0a. Login userxx ---> /home/userxx
 #0b. cd demo      <--- change into the demo/directory

1. cfl - Count Lines & KB in 1 File


 #1. cfl $UV/src/uvcopy.c  <-- count lines in 1 file (uvcopy source program)
     ====================
      cfl /home/userxx/src/uvcopy.c - count lines in 1 file
       24400 lines,   1072 KB in /home/userxx/src/uvcopy.c
                           Report also left in
                           /home/userxx/demo/rpts/_home_userxx_src_uvcopy.c_cfl

2. cfd - Count Files,lines,KB in 1 directory


 #2. cfd dat1   <-- Count Files,Lines,& KiloBytes in the dat1/ directory
     ========
      cfd dat1 - Count Files,Lines,KB in Directory
      File#    Lines      KB  Directory/Filename 20190726:1027
           1     335      32  dat1/CanadaMPs
           2      13       4  dat1/CanadaProvinces
           3       8       4  dat1/nameadrs1
           4     305      28  dat1/UScities
           5     539      40  dat1/UScongress
           6      50       4  dat1/USstates
      ******    1250     112  *Totals* in Directory /home/userxx/demo/dat1
                               Report also in --> /home/userxx/demo/rpts/dat1_cfd

3. cfdt - Count Files,lines,KB in 1 directory (Totals-Only)


 #3. cfdt dat1   <-- Count Files,Lines,& KiloBytes in dat1/... Totals-Only
     =========
      cfdt dat1 - Count Files,Lines,& KB in Directory - Totals-Only
       Files   Lines      KB  Directory/Filename 20190726:1027
           6    1250     112  *Totals-Only* in Directory /home/userxx/demo/dat1
                               Report also in --> /home/userxx/demo/rpts/dat1_cfdt

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D2. Counting Files,Lines,& KB in directories

4. cfdpf - Count Files,Lines,KB in Dir with/without Pattern in filenames


 #4. cfdpf dat1 'Canada*'   <-- Count Files,Lines,KB for filenames begining with 'Canada'
     ====================
      cfdpf dat1 Canada* - Count Files,Lines,KB in Directory with Pattern [not] in filenames
      File#    Lines      KB  Directory/Filename 20190726:1027
           1     335      32  dat1/CanadaMPs
           2      13       4  dat1/CanadaProvinces
      ******     348      36  *Totals* in dat1 with pattern "Canada*"
                               Report also in --> /home/userxx/demo/rpts/dat1_cfdpf_Canada_

** 5. cfdpl - Count Files,Lines,KB with/without Pattern any line any file in dir **


 #5. cfdpl dat1 'Washington'   <-- Count Files,Lines,KB with 'Washington' on any line in any file
     =======================
      cfdpl dat1 Washington - Count Files,Lines,KB in Directory with Pattern [not] on any line in file
      File#    Lines      KB  Directory/Filename 20190726:1027
           1     305      28  dat1/UScities
           2      50       4  dat1/USstates
      ******     355      32  *Totals* in /home/userxx/demo/dat1 with  pattern "Washington"
                               Report also in --> /home/userxx/demo/rpts/dat1_cfdpl_Washington

6. cfdd - count Files,Lines,& KB in all subdirs of a superdir


 #6. cfdd $UV/sf    <-- Count Files,Lines,& KB in all Sub-Dirs of $UV/sf/...
     ===========
      cfdd  /home/userxx/sf - Count Files,Lines,& KB in SubDirs of SuperDir
        Dir#   Files   Lines      KB  SubDir/ParentDirectory  20190726:1027
           1     227    6502     944  /home/userxx/sf/adm
           2      86    2501     352  /home/userxx/sf/demo
           3     308   17102    1436  /home/userxx/sf/IBM
           4     541   21323    2248  /home/userxx/sf/util
      ******    1162   47428    4980  *Totals* for SubDirs of SuperDir /home/userxx/sf/...
                                       Report also in --> /home/userxx/demo/rpts/_home_userxx_sf_cfdd

7. cfddt - count Files,Lines,& KB all subdirs of superdir (Totals-Only)


 #7. cfddt $UV/sf   <-- Count Files,Lines,& KB in all Sub-Dirs of $UV/sf/... Totals-Only
     ============
      cfddt /home/userxx/sf - Count Files,Lines,KB in SubDirs of Super-Dir - Totals-Only
        Dirs    Files   Lines      KB  SubDir/ParentDir 20190726:1027
           4     1162   47428    4980  *Totals* for SubDirs of SuperDir /home/userxx/sf/...
                                        Report also in --> /home/userxx/demo/rpts/_home_userxx_sf_cfddt

8. cfddf - Count Files,Lines,KB in SubDirs of Super-Dir + 1st few files

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D3. Counting Files,Lines,& KB in directories


 #8. cfddf backup 2   <-- cfddf - for all Sub-Dirs in backup Dir + 1st 2 files of each subddir
     ==============
      cfddf backup 2 - Count Files,Lines,KB in SubDirs of SuperDir + 1st few files
        Dir#   Files   Lines      KB  SubDir/ParentDirectory  20190726:1027
           1       8    1299     268  backup/archive
                   1    1299      28  backup/archive/CanadaMPs.csv
                   2    1299       4  backup/archive/ftpsall
           2       6    1250     116  backup/dat1
                   1    1250      32  backup/dat1/CanadaMPs
                   2    1250       4  backup/dat1/CAprovinces
           3      24     211     108  backup/dat2
                   1     211       4  backup/dat2/accents1
                   2     211       4  backup/dat2/accents2
      ******      38    2760     492  *Totals* for SubDirs of SuperDir /home/userxx/demo/backup/...
                                       Report also in --> /home/userxx/demo/rpts/backup_cfddf

9. cfdmm - list directory: File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#


 #9. cfdmm dat1  <-- show Record Minsize,Maxsize,MinRec#,MaxRec# for all files in directory
     ==========
     cfdmm dat1 - list File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#,Dir/Filename
      File#   Lines      KB MinRsiz MaxRsiz MinRnum MaxRnum  Directory/FileName
          1     335      32      95      95       1     335  dat1/CanadaMPs
          2      13       4      14      34      13       5  dat1/CanadaProvinces
          3       8       4      70      75       6       7  dat1/nameadrs1
          4     305      28      75      82       2       1  dat1/UScities
          5     539      40      71      74       2     422  dat1/UScongress
          6      50       4      13      23      15      40  dat1/USstates
          6    1250     112      13      95                  *Totals* in dat1
                            Report left in --> /home/uvadm/demo/rpts/dat1_cfdmm

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5E1. Counting Files,Lines,& KB in directories

cfhelp - Help screen for cfd... scripts

If you need a reminder of the 9 cfd scripts, just enter 'cfhelp' at your prompt.


 cfhelp    <-- display help screen reminders of cfd... scripts
 ======      - display as follows:
           ------- cfd... scripts & demo/dirs to count Files,Lines,KB -------
 1. cfl   $UV/src/uvcopy.c  # Count Lines & KB in the uvcopy utility program
 2. cfd   dat1              # Count Files,lines,KB in 1 directory (dat1)
 3. cfdt  dat1              # Totals-Only of cfd dat1 (6 files counts combined onto 1 line)
 4. cfdpf dat1 'Canada*'    # Counts for dat1/... filenames begining with 'Canada'
 5. cfdpl dat1 'Washington' # Counts for dat1/... files with 'Washington' on any line of the file
 6. cfdd  $UV/sf            # count Files,Lines,& KB for the 4 subdirs (adm,demo,util,IBM) of /home/userxx/sf
 7. cfddt $UV/sf            # Totals-Only of cfdd /home/userxx/sf (4 subdir counts combined onto 1 line)
 8. cfddf backup 2          # Counts for backup/... (archive,dat1,dat2) + 1st 2 files of each subdir
 9. cfdmm                   # Directory-Stats: files,lines,Minsize,Maxsize,Minrec#,Maxrec#
           ----- add 'export UVEXPERT=y' to your profile to inhibit these displays -----

In fact, you will get a prompt after you run a cfd... script, asking if you want to see this help screen - reply 'y' to display or null to ignore.

And when you know them & don't need them, you can inhibit them by adding 'export UVEXPERT=y' to your .bash_profile to inhibit future displays.

Help screens for each script (vs all 9 above)

You can see a help screen for each script, by entering the script-name only, without required arguments (omit directory). For example, if we enter just 'cfl'


 cfl     <-- enter script name without arguments (directory)
 ===       - displays ERROR msg, Correct Format,& sample report as follows:
 cfl - count lines in 1 file
 cfl filename                  <-- Usage ERROR, arg1 MUST be a FILE
 ============
 cfl /home/userxx/src/uvcopy.c  <-- Correct Format & Sample Report
 ============================
  24400 lines,   1072 KB in src/uvcopy.c
                         Report--> /home/userxx/rpts/src_uvcopy.c_cfl

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5E2. Counting Files,Lines,& KB in directories

v12 - help menu for counting scripts

The 'v12' help menu is mentioned here because it also includes help menus for the cfd... scripts to count Lines & Files. v12 is just 1 of many help menus displayed by the 'uvhelp' main menu.

'help' screens are stored at $UV/help/... & it is in the profile PATH, so you can display the help/menu screen anytime anywhere, for example:


 uvhelp   <-- display main menu of help screens
 ======     - determine desired sub-menu code 'v12' in this case,
            - then quit the uvhelp main menu, & enter the sub-menu code 'v12'

 v12      <-- display sub-menu of 'counting lines & files' (shown below)
 ====
Note
  • 'v12' is just 1 of 29 sub-menu's listed by uvhelp main-menu
  • see all sub-menu's listed at uvsoftware.ca/uvhelp.htm
  • other sub-menus refer to other scripts listed in this scripts1.htm

v12 - counting Lines,Files,& KB in Directories

  1. cfl - Count Lines in 1 File
  2. cfd - Count Files,Lines,KB in 1 Directory
  3. cfdt - Totals-Only version of Count Files,Lines,& KiloBytes in a Directory
  4. cfdpf - Count Files in a Directory with a Pattern [or not] in filenames
  5. cfdpl - Count Files in a Directory with a Pattern [or not] on any line in any file
  6. cfdd - Count Files,Lines,& KB in a ALL Sub-Dirs in a Super-Directory
  7. cfddt - Count Files,Lines,& KB in a ALL Sub-Dirs in a Super-Directory Totals-Only
  8. cfddf - Count Files,Lines,KB in ALL Sub-Dirs in a Super-Dir + 1st few files
  9. cfdmm - List Directory: File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#, Dir/Filename

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5F1. Counting Files,Lines,& KB in directories

cftest - run all 9 cfd scripts in batch mode

'cftest' is a script to run all 9 cfd scripts in batch mode. All reports will be collected in subdir rpts/... of current working directory. You will be prompted to remove existing reports (reply to remove, null to keep).

 #!/bin/ksh
 # cftest - run the 9 scripts to count Files,Lines,KB in directories
 #        - by Owen Townsend, UV Software, July 2019
 # cp -r $UV/demo $HOME  <-- assumes you have copied demo dir to your homedir
 # cd $HOME/demo         <-- ensure you are in the demo subdir
 export UVEXPERT=y   # inhibit prompt for cfhelp at end each cfs... script
 echo "cftest - run 9 scripts to count Files,Lines,KB in directories"
 echo "cp -r $UV/demo $HOME  <-- assumes you copied demo dir to your homedir"
 echo "cd $HOME/demo         <-- ensure you are in the demo subdir"
 reply=x
 until [[ "$reply" == "y" || "$reply" == "n" ]]
   do echo " - reports will be collected in rpts/..., OK to remove all old rpts/* ? y/n"
      read reply; done
 if [[ "$reply" == "y" ]]; then rm -f rpts/*; fi
 cfl   $UV/src/uvcopy.c
 cfd   dat1
 cfdt  dat1
 cfdpl dat1 'Washington'
 cfdd  $UV/sf
 cfddt $UV/sf
 cfdpf dat1 'Canada*'
 cfddf backup 2
 cfdmm dat1
 echo "reports collected in rpts/* listed as folows:"
 cfd rpts    #<-- list reports with line counts
 echo " "
 echo "enter to see report contents via more, then spacebar for each report"; read reply
 more rpts/*
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5G1. Counting Files,Lines,& KB in directories

Recommendations using cfd scripts on your own directories

  1. Try the cfd... scripts on your own directories. Try cfd,cfdt,cfd[f,cfdpl,cfdmm on directories of files only & cfdd,cfddt,cfddf on super-directories that contain multiple sub-directories.

  2. The cfd reports are automatically created in subdir rpts/... of your current working directory, so you must stay above the directories you are reporting.

  3. The cfd scripts will create subdir 'rpts' if it does not already exist in your current working directory & you would not want rpts to be created among your files. IE - it is best if your current working directory contains only directories & all your files are within their appropriate directories.

  4. The same argument applies to the './tmp' subdir that is automatically created by many of the Vancouver Utility scripts. We are not using the system '/tmp'. Many scripts create a tmp/ subdir in the current working directory if it does not exist - to create an 'ls' list of files to ensure processing alphabetically.

  5. Reports are automatically named from the directory & script name that created it. For example the report created by 'cfd dat1' will be 'rpts/dat1_cfd'.

  6. Any pattern arguments will be included in the rpts/filename, For example the report created by 'cfdp dat1 Canada*' will be 'rpts/dat1_cfdp_Canada_'. Any characters not allowed in filenames (such as '*') are converted to '_'.

  7. Actually, the last line of the report gives you the full pathname of the report. For example '/home/userxx/demo/rpts/dat1_cfdpf_Canada_'

  8. The above conventions are convenient, you can run many reports without overwriting previous reports, and then use them to research problems or to assess the impacts of planned changes.

  9. The reports include all relevant information - command line used, input directory, any patterns, date/time, full path-name of report created.

  10. Note that the main objective of these demos is to give you the knowledge to run these scripts on your own directories to achieve your objectives.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5H1. Counting Files,Lines,& KB in directories

Listings of cfd... scripts

If you are interested to see listings of the cfd... scripts, please see uvsoftware.ca/scripts1.htm#Part_5. You can click on the following links, but be warned that the "+" links will take you to separate document scripts1.htm.

To avoid losing your place in this UVdemos.doc, it is best to 'right-click' on the desired links & open in 'new-tab'. Then you will not get lost in scripts1.doc without an easy way back.


5B1+ cfl - Count Lines in 1 File

5C1+ cfd - Count Files,Lines,KB in 1 Directory

5D1+ cfdt - Totals-Only version of Count Files,Lines,& KiloBytes in a Directory

5E1+ cfdpf - Count Files in a Directory with a Pattern [or not] in filenames

5F1+ cfdpl - Count Files in a Directory with a Pattern [or not] on any line in any file

5G1+ cfdd - Count Files,Lines,& KB in a ALL Sub-Dirs in a Super-Directory

5H1+ cfddt - Count Files,Lines,& KB in a ALL Sub-Dirs in a Super-Directory Totals-Only

5I1+ cfddf - Count Files,Lines,KB in ALL Sub-Dirs in a Super-Dir + 1st few files

5J1+ cfdmm - List Directory: File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#, Dir/Filename

In case you are curious, we will list just 1 of the 9 scripts on the following page:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5H2. scripts to manipulate text files

cfd - Count Files,lines,KB in 1 directory

 #!/bin/ksh
 # cfd - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # cfd - Count Files,Lines,KB in 1 Directory
 #     - by Owen Townsend, UV Software, updated July 2019
 echo "cfd - Count Files,Lines,& KiloBytes in a Directory"
 d1="$1";
 if [[ ! -d "$d1" ]]; then
    echo "cfd directory   <-- Usage ERROR, arg1 MUST be a DIRECTORY"
    echo "============="
    sleep 2
    echo "cfd dat1        <-- Correct Format & Sample Report"
    echo "========"
    echo "cfd - Count Files,Lines,& KiloBytes in a Directory"
    echo "File#    Lines      KB  Directory/Filename 20190722:0819"
    echo "     1     335      32  dat1/CanadaMPs"
    echo "     2      13       4  dat1/CAprovinces"
    echo "     3       8       4  dat1/nameadrs2"
    echo "     4     305      28  dat1/UScities"
    echo "     5     539      40  dat1/UScongress"
    echo "     6      50       4  dat1/USstates"
    echo "******    1250     112  *Totals* in Directory /home/userxx/demo/dat1"
    exit 1; fi
 #
 typeset -R6 files lines tlines fkb fkbt
 test -d rpts || mkdir rpts
 d2=$(echo $d1 | tr '/' '_')
 echo "File#    Lines      KB  Directory/Filename $(date +%Y%m%d:%H%M)" >rpts/${d2}_cfd
 for d1f in $d1/*
   { ((files+=1))
     lines=$(wc -l <$d1f)   # get line count
     ((tlines+=lines))      # count Total lines
     fdus=$(\du -s $d1f | tr '\t' ' '); fkb=${fdus%% *};
     if [[ -d $d1f ]]; then
        echo "$files  $lines  $fkb  ${d1f}/" >>rpts/${d2}_cfd
        else echo "$files  $lines  $fkb  $d1f" >>rpts/${d2}_cfd; fi
     ((fkbt+=fkb))
   }
 if [[ "$d1" == /* ]]; then fd1=$d1; else fd1=$PWD/$d1; fi
 echo "******  $tlines  $fkbt  *Totals* in Directory $fd1" >>rpts/${d2}_cfd
 echo "    - report left in --> $PWD/rpts/${d2}_cfd"
 more rpts/${d2}_cfd
 sleep 1
 cfhelp  # display help screen summary of 8 scripts to count Files,Lines,KB in directories"
 #-------# export UVEXPERT=n to inhibit
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5I1. Vancouver Utility scripts to list files in directories

List files using unix 'ls' options & more

l
  • 'l' (alias l='ls -l') saves keystrokes for this commonly used command
llm
  • List Long files in Directory & pipe to more (sorted by filename)
  • like 'ls -l | more', but allows argument directory (default current)
  • saves keystrokes & adds functionality to this often repeated command
lla
  • same as llm, but lists All files including hidden .files
llc
  • list all filenames in directory with file & line counts
lld
  • list directories only (omit files)
llr
  • list files & subdirs Recursively (all the way down)
llt
  • list files sorted by last modification/creation date/time
lltr
  • same as llt, but with Reverse sort (oldest 1st)
lls
  • list files sorted by Size, largest 1st
llsr
  • same as lls, but with Reverse sort, smallest 1st
llu
  • list files Unsorted, as stored in directory
spreadA
  • list filenames 4-up across
  • great for high volume scripts, programs, copybooks, etc

sample output for 'llc'


 llc $UV/sf/util           <-- list Vancouver Utility script filenames
 ===============             - with File & Line counts (25 files/screen)
 File#   Lines Directory/Filename
     1      43 sf/util//acum1
     2      22 sf/util//all2lower
     3      22 sf/util//all2upper
   454      52 sf/util//wrtbig
   454 files,   16680 total lines in  files in directory

more info on UV scripts

You can see a help screen for each script, by entering the script-name only, without its required arguments (omit the directory). See these scripts listed on pages uvsoftware.ca/scripts1.htm#3A1 - 3H1.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5I2. Vancouver Utility scripts to list files in directories

llt - sample of 20 scripts to list files

Here is just 1 of the scripts to list files in directories (based on unix 'ls' with various options). See more via the '+' links on the following page.

 #!/bin/ksh
 # llt - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # llt  - list directory long in date sequence with Newest first
 #      - pipe to more (show 1 screen & wait for null entry)
 #
 #usage with '*' wildcard patterns
 # llt dir/abc*      <-- does not work (will match only 1st occurrence)
 # llt 'dir/abc*'    <-- does work (enclose patterns in single quotes)
 #
 echo "llt - list directory in date sequence with Newest files 1ST"
 if (($# > 1));then
    echo "usage: llt directory"
    echo "       ============="
    echo "example: llt 'jcls/map*'"
    echo "         ==============="
    echo "- enclose any *wildcard patterns in single quotes"
    exit 1; fi
 #
 if [[ -z "$1" ]] ; then  d="."; else d=$1; fi
 ls -lt  $d | more
 #================
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5I3. Vancouver Utility scripts to list files in directories

All scripts to list files in directories

You can see All scripts at the "+" links in separate document scripts1.htm. To avoid losing your place in this UVdemos.doc, it is best to 'right-click' on the desired links & open in 'new-tab'. Then you will not get lost in scripts1.doc without an easy way back.


3A1+ llc - list files with File & Line Counts

3B1+ llm - list long & pipe to more
- similar to alias llm='ls -l | more' (but a script vs an alias)
- saves keystrokes for this often repeated command

3B2+ lla - same as llm, but lists All files including hidden .files

3C1+ llt - list files, with the latest First

3C2+ lls - list files in Size order

3C3+ llu - list files Unsorted

3D1+ lld - list directories only (omit files)

3E1+ llr - list files & subdirs Recursively (all the way down)

3F1+ lslp - list files & print immediately
- prints using 'uvlp12' - laser printing utility

3G1+ spreadA - list filenames 4-up across
- great for high volume scripts, programs, copybooks, etc

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5J1. Vancouver Utility scripts to Rename all files in directory

Rename scripts

'rename' scripts - can save you a lot of manual keying when you want to make mass changes to filenames in a directory, often the case when you import a lot of long awkward filenames from other systems (such as mainframes).

We have an example in our demo files - demo/sql0/... files are from a mainframe and we modified them as shown in demo/sqls/... with the rename scripts. Here are the 1st 2 of the 11 filenames before & after renaming

         'SELECT.CUSTOMERS.ALL'
         'SELECT.CUSTOMERS.BASIC'
   --- 1st 2 before & after renaming ---
         select_customers_all.sql
         select_customers_basic.sql
  1. convert UPPER-case to lower-case
  2. remove the single quotes
  3. convert '.' period separators to '_' underscore separators
  4. add extension 'sql'

    Operating Procedures

Note that the 'rename' scripts rename the files within the existing directory. Since we wanted to preserve the originals so we could repeat the demos, we 1st copied the files from sql0/... to sql1/... before we renamed the scripts.


 #0a. Login userxx --> /home/userxx/
 #0b. cd demo -------> /home/userxx/demo/

 #0c. cp sql0/* sqls/    <-- copy files to 2nd dir (if originals must be preserved)
      ===============

 #1. renameL sqls        <-- rename all UPPER-case to lower-case
     ============

 #2. rename-QQ sqls      <-- remove single opening & closing quotes
     ==============

 #3. renameCC sqls       <-- convert separators '.' periods to '_' underscores
     =============

 #4. rename+X sqls .sql  <-- add extension '.sql' to all files
     ==================

alternate method to rename files

Also see 'uvfix1' at UVdemos.htm#7D1 which creates a script to do the same renaming as shown here by the 'rename...' scripts. uvfix1 requires some knowledge of the uvcopy instruction set (vs the easy to use rename scripts), but could be useful for really complex situations.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5J2. Vancouver Utility scripts to Rename all files in directory

RenameL - 1 example of 37 in sf/util/...

Here is just 1 example of the 37 rename... scripts in $UV/util/...

 #!/bin/ksh
 # renameL - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # renameL - rename an entire directory of filenames to lower case
 #
 echo "rename all filenames in subdir to lower case"
 if [ -d "$1" ]; then :
    else echo "usage: renameL directory"
         echo "       ================="
         echo " - arg1 must be a directory"
         exit 1; fi
 #
 echo "will rename all files in $1 to lower case OK ? y/n"; read reply
 x=0; y=0
 for i in $1/*
 do let x=x+1
    f=${i##*/}
    typeset -l g=$f
    if [[ $g != $f ]]; then
       mv -i $1/$f $1/$g
       #================
       let y=y+1
       echo "file# $y (of $x) $1/$f - renamed to: $1/$g"
    fi
 done
 echo "total $y files in ${1}, $x renamed to lower case"
 exit 0

How to know there are 37 rename... scripts

I determined there were 37 rename... scripts by using the 'cfdpf' script documented at UVdemos.htm#5D2 as follows:


 cfdpf $UV/sf/util 'rename*'   <-- count files begining with 'rename'
 ===========================
 File#    Lines      KB  Directory/Filename 20190728:0955
      1      17       4  /home/uvadm/sf/util/rename1R
                            --- omitting 36 lines ---
     37      32       4  /home/uvadm/sf/util/rename-XX3
 ******    1322     152  *Totals* in /home/uvadm/sf/util with pattern "rename*"

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5J3. Vancouver Utility scripts to Rename all files in directory

Rename Script Listings - ALL

You can see All scripts at the "+" links in separate document scripts1.htm. To avoid losing your place in this UVdemos.doc, it is best to 'right-click' on the desired links & open in 'new-tab'. Then you will not get lost in scripts1.doc without an easy way back.


1B1+ renameL - translate filenames to lower case
1B2+ renameU - translate filenames to UPPER case
1B3+ renameLsd - translate filenames to lower case in all subdirs of a superdir

1C1+ renameP - replace existing prefix with alternate
1C2+ rename-P - remove any specified prefix
1C3+ rename-P. - remove specified prefix up to 1st '.'
1C4+ rename+P - add any specified prefix

1D1+ renameX - change any specified extension to any specified alternate
1D2+ rename.X - change specified .extension to any specified .alternate
1D3+ rename-X - remove any specified extension
1D4+ rename-.X - remove any specified extension back to last '.' (default)
- may specify an alternate extension (vs '.' default)
1D5+ rename+X - add any specified extension

1E1+ rename-AA - remove Any pattern Anywhere in filename
1E2+ renameAA - replace Any pattern with any Alternate

1F1+ renameB2U - change any embedded blanks to underscores
1F2+ renameD2U - change any '$' dollar signs to '_' underscores

1G1+ rename-QQ - remove any 'Quotes'
1G2+ rename-PP - remove any (Parenthesis)
1G3+ rename2PP - rename to whatever is in (Parenthesis)

1H1+ rename-TS - remove any timestamp _HHMMSS ('_' + 6 digits)
- preserves anything following _###### (.dat, .txt, etc)

1I1+ renameGDG - change mainframe GDG suffixes to Vancouver Utility gdg suffixes
- change '.G####V00' to '_00####'
1I2+ renameGDG1 - change '.G####V00' to '_000001' (keep only latest as 000001)

1J1+ renameCC - convert all occurrences of 1 character to a 2nd character
- example - convert all '.'s in filenames to '_'s

You can see a help screen for each script, by entering the script-name only, without its required arguments (omit the directory).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5K1. Vancouver Utility scripts

removeCR - remove Carriage Returns from text files

Carriage Returns x'0D' can be a problem on unix/linux systems especially for scripts that will not work with carriage returns. You might have this problem when receiving scripts from other systems or via email. The error-message is "bash: srriptname: /bin/bash^M: bad interpreter"

Note that the 'CR's are invisible using text editors, but you can see them with the 'uvhd' utility - as was demonstrated briefly on pages '2B1' - 2B2, & in-depth on pages '10B1' - 10B11. Here is just the 1st line of a script shown with uvhd.


 #5. uvhd sf/llsx t  <-- rerun with option 't' separate lines on LineFeeds x'0A'
     ==============
     rec#=1 rsize=13 fptr=0 fsize=244 records=7
                      10
 r#        1 0123456789012
           0 #!/bin/bash..
             2226662667600    <-- CR x'0D' causes problems
             31F29EF2138DA

removeCR - remove Carriage Returns from text files

 #!/bin/ksh
 # removeCR - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # removeCR - remove CR's Carriage Returns (x'0D')
 #          - by copy to tmp/... & prompt to copy back
 #          - by Owen Townsend, UV Software, Feb 2014
 #
 echo "removeCR  - copy a file to tmp/... removing CRs & nulls, prompt to copy back"
 echo "removeCR1 - alternate saves file in tmp/ before copy back removing CRs"
 df="$1";
 if [[ -f "$df" && -d tmp ]]; then :
 else echo "usage: removeCR subdir/file"
      echo "       ===================="
      echo " - arg1 must be a text file (with LFs & CRs)"
      echo " - current working directory must also contain a tmp/ subdir"
      echo " - will copy to tmp/ removing CRs & prompt to copy back"
      echo " - also see insertCR to insert CR's (create CR/LF)"
      exit 1; fi
 #
 f=$(basename $df)
 tr -d '\r' <$df >tmp/$f
 #==========================
 linesin=$(wc -l $df); sizein=$(du -bs $df);
 linesout=$(wc -l tmp/$f); sizeout=$(du -b tmp/$f);
 echo "input  lines & bytes:    $linesin $sizein"
 echo "output lines & bytes:    $linesout $sizeout"
 until [[ "$reply" = "y" || "$reply" = "n" ]]
    do echo "copy back overwriting original y/n ?"; read reply; done
 if [[ "$reply" = "y" ]]; then
    cp tmp/$f $df; echo "copied back to $df"; ls -l $df
 else echo "copy back aborted"; fi
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5L1. Vancouver Utility scripts

alldiff2 & alldiff3 diff between all files in 2 directories

'alldiff2' repeats the unix system 'diff' utility for each pair of files found in two directories. This is great for confirming results of mass changes to directories of scripts, programs, or any kind of text files.

Before you make mass changes to directories of text files, it is a good idea to 1st save the original directory as '.old', perform the changes,& then run alldiff2 on the 2 directories.

This is really important when modifying utility programs that make the mass changes to hundreds or thousands of files (such scripts, programs, etc). alldiff2 & alldiff3 can make the difference between your project's success or failure, since they can confirm that you have made only the intended changes & no unintended changes.

We will illustrate alldiff2 using the 'jcl3' & jcl3.old' directories in $UV/demo/... Assume that we have modified the JCL converter & are ready to run the new version to convert all JCLs in jcl2/... to JCL/scripts in jcl3/... We proceed as follows:

Alternative #1 - if you have JCL converter

Even if you have the JCL converter installed, you can not run #1a,#1b,#1c in the demo/ directory because it requires other files not supplied in the demo directory. So use Alternative #2 to copy demo/archive/jcl3.old to demo/jcl3.old, then alldiff2


 #1a. mv jcl3 jcl3.old   <-- change name of existing converter outputs
      ================       Note - do NOT run #1a,#1b,#1c (use #2)

 #1b. mkdir jcl3         <-- make new empty directory for new outputs
      ==========

 #1c. jclxx51 jcl2 jcl3  <-- re-convert JCL with modified version of JCL converter
      =================

Alternative #2 - copy jcl3.old from archive for alldiff2 with jcl3


 #2. cp -r archive/jcl3.old .  <-- copy jcl3.old from archive
     ========================

run alldiff2 & investigate diff report


 #3. alldiff2 jcl3.old jcl3  <-- run alldiff2 to create tmp/jcl3.dif
     ======================

 #4. vi tmp/jcl3.dif         <-- investigate the dif report
     ===============

See tmp/jcl3.dif listed on the enxt page:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5L2. alldiff2 & alldiff3 for all files in 2 directories

alldiff2 report - tmp/jcl3.dif

 39c39
 < exit $JCC #jclunix51 version=20190119, converted 20190728:145940 sitename=UV Software
 ---
 > exit $JCC #jclunix51 version=20190119, converted 20190728:155658 sitename=UV Software
 diff file# 1 - jcl3.old/... vs jcl3/jar100.ksh
 56c56
 < exportfile  SALELST data1/ar.sales.list #(,CATLG,DELETE)
 ---
 > exportfile  SALELST data1/ar.sales.list200 #(,CATLG,DELETE)
 68c68
 < exit $JCC #jclunix51 version=20190119, converted 20190728:145940 sitename=UV Software
 ---
 > exit $JCC #jclunix51 version=20190119, converted 20190728:155658 sitename=UV Software
 diff file# 2 - jcl3.old/... vs jcl3/jar200.ksh
 46c46
 < exit $JCC #jclunix51 version=20190119, converted 20190728:145940 sitename=UV Software
 ---
 > exit $JCC #jclunix51 version=20190119, converted 20190728:155658 sitename=UV Software
 diff file# 3 - jcl3.old/... vs jcl3/jgl100.ksh
       ------ omitting reports for file#s 3,4,5,6 (similar to all except #2) ------
 70c70
 < exit $JCC #jclunix51 version=20190119, converted 20190728:145940 sitename=UV Software
 ---
 > exit $JCC #jclunix51 version=20190119, converted 20190728:155658 sitename=UV Software
 diff file# 7 - jcl3.old/... vs jcl3/jgl320.ksh
 7 different of 7 files compared jcl3.old to jcl3

Significant Differences only in file#2

You can see that the only significant differences are in file #2. The differences in the other 7 files are due only to the date/time stamps inserted by the JCL converter.

See alternative script 'alldiff3' which adds another step to alldiff2 to eliminate such differences using a table of keywords such as 'version'. When you run alldiff3 you will be prompted for other keywords to cause elimination, &/or you can add your keywords to the table for future alldiff3s.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5L3. alldiff2 & alldiff3 for all files in 2 directories

run alldiff3 to drop insignicant differences

The operating procedures are similar to the above - just change the script name from 'alldiff2' to 'alldiff3'.


 #5. alldiff3 jcl3.old jcl3  <-- run alldiff3 to create tmp1/jcl3.dif
     ======================

You will get more console messages that for alldiff2, such as the prompt for other keywords to cause difference eliminations (not shown here). Note that alldiff3 creates the same report as alldiff2 in tmp/... And then it creates a 2nd report in tmp1/... with the differences eliminated.


 #6. vi tmp1/jcl3.dif   <-- investigate the alldiff3 report
     ================     - with insignificant differences eliminated

alldiff3 report - tmp1/jcl3.dif

56c56 < exportfile SALELST data1/ar.sales.list #(,CATLG,DELETE) > exportfile SALELST data1/ar.sales.list200 #(,CATLG,DELETE) diff file# 2 - jcl3.old/... vs jcl3/jar200.ksh 1 significant difference in 7 files compared jcl3.old to jcl3

alldiff3 calls diffdrop2 to drop insigificants

Script 'alldiff3' calls uvcopy job 'diffdrop2' to drop insigificant differences caused by date/time stamps, versions, etc, depending on a table of patterns in $UV/ctl/diffdroptable2. You can add your own patterns to ctl/diffdroptable2, Or temporarily add 1 or 2 patterns at the arg1/arg2 prompts.

We will list 'alldiff3' on the following page & you can see the uvcopy job 'diffdrop2' at $UV/pf/util/diffdrop2.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5L4. alldiff2 & alldiff3 for all files in 2 directories

alldiff3 script listing

 #!/bin/ksh
 # alldiff3 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # alldiff3 - script to compare all text files in 1 directory to a 2nd directory
 #          - redirects output to tmp/dir2.dif & prompts to view/print/etc
 #Jul11/17 - add diffdrop2 script to drop insignificant lines on JCL/script diffs
 #         - ver:, UVSW, diff file# line, empty diffs after pattern drops
 #
 d1="$1"; d2="$2";
 echo "alldiff3 - compare all files in 2 subdirs & write all diffs to tmp/dir2.dif"
 echo " - then drop diffs due to time stamps & table of patterns (prompt to add)"
 if [[ -d "$1" && -d $2 ]]; then :
 else echo "usage: alldiff3 dir1 dir2"
      echo "       =================="
      echo "example: alldiff3 cbls.old cbls"
      echo "         ======================"
      exit 1; fi
 #
 d2b=$(basename $d2)   # get basename of dir2 (drop any preceding /path/...)
 log=tmp/$d2b.dif      # make name for output log file
 log2=tmp1/$d2b.dif    # make name for diffdrop2 outputfile
 >$log                 #init logfile in tmp subdir w same name as dir2 + .dif
 x=0; y=0;
 for i in $d1/*
 do
   let x=x+1
   typ=$(file $i)
   if [[  $typ == *text* || $typ == *script* || $typ == *data* ]]
      then f=${i##*/}
           diff -a -bBw $d1/$f $d2/$f >>$log
           #=================================
           if [[ $? -gt 0 ]]; then
              echo "diff file# $x - $d1/... vs $d2/$f" >>$log
              echo " " >>$log
              let y=y+1
           fi
      else echo " file# $x $typ - NOT text/script file" >>$log
   fi
 done
 lines=$(wc -l $log)      # capture line count
 echo "$y different of $x files compared $d1 to $d2" >>$log
 echo "$y diff $x files in $d1 & $d2, report: $log $lines lines"
 echo "--> enter additional drop patterns 'arg1=xxx,arg2=yyy' or null"
 read args
 uvcopy diffdrop2,fili1=$log,filo1=$log2,$args
 #============================================
 echo "--> enter command to view (vi,cat,more,uvlp12,etc,or null)"
 read cmd
 if [[ ! "$cmd" = "" ]]; then $cmd $log2; fi
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5M1. uvcmpFA1 - compare packed/binary files

'uvcmpFA1' will compare files with packed/binary fields &/or files without the LineFeeds required by unix 'diff' & the 'alldiff2/3' scripts. There are several versions of uvcmp... for ASCII, EBCDIC, fixed & variable length files. See uvsoftware.ca/cmpjobs.htm for full documentation.

Here is a brief demo you can run using dat2/custmas1 & dat2/custmas2, which have 32 records of 256 bytes with differences in only 1 record.

The demo will ask you to try unix tools (vi,diff,etc), but they are ineffective for files with packed fields & unix 'cmp' displays only in octal (vs hexadecimal) and shows only bytes that are different (vs showing complete record pairs with differences marked with '*'s).


 #0a. Login userxx --> /home/userxx/
 #0b. cd demo -------> /home/userxx/demo/

 #1. vi dat2/custmas1   <-- try to display packed file with unix 'vi'
     ================     - display not shown, mostly garbage

 #2. diff dat2/custmas1 dat2/custmas2  <-- try unix diff
     ================================

     --> "Binary files dat2/custmas1 and dat2/custmas2 differ" <-- not helpful
         =====================================================

 #3. uvhd dat2/custmas1 r256   <-- display packed file with 'uvhd'
     =======================
     version=20190321 records=32  rsize=256 fsize=8192
                      10        20        30        40        50        60
     r#    1 0123456789012345678901234567890123456789012345678901234567890123
           0 130140    EVERGREEN MOTORS LTD.    1815 BOWEN ROAD          NANA
             3333332222454545444244545524542222233332445442544422222222224444
             130140000056527255E0DF4F230C44E0000181502F75E02F140000000000E1E1
          64 IMO          BC V9S1H1    250-754-5531 LARRY WRENCH     ..4V|...
             4442222222222442535343222233323332333324455525544442222201357000
             9DF00000000002306931810000250D754D55310C12290725E38000000246C000
         128 .........W0....`........)X}..f3.....\.................4V}...f...
             0000000005300016000000002570063100095000000000000000013570016000
             0C0000C0270D0540C0000C0098D0263C0444C0000C0000C0000C0246D0056C00
         192 .E|...V}.......................f.....<........f.C 20190320     .
             0470005700000000880000000018000680001300000000694233333333222220
             35C0046D0000C0023C0000C0083C0056D0012C0000C0016D302019032000000A

 #3a. uvhd dat2/custmas2 r256  <-- could display 2nd file & look for diffs
      =======================    - but unlikely you could ever find them

 #4. cmp -l dat2/custmas1 dat2/custmas2  <-- try unix cmp
     ==================================
      2430   0   2       <-- displays 4 lines for 4 bytes different
      2431   2   3         - in octal (hexadecimal option not available)
      2435   0   3         - see uvcmpFA1 on next page, which shows
      2436   3   4           record pairs with '*'s flags on unmatched bytes

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5M2. uvcmpFA1 - compare packed/binary files

uvcmpFA1 - compare packed/binary files


 #5. uvcmpFA1 dat2/custmas1 dat2/custmas2 r256
     =========================================
Note
  • uvcmpFA1 creates reports in subdir ./rptcmp (made if not present)
  • and names the report from the 1st filename specified

 #6. vi rptcmp/custmas1   <-- view report from #4. uvcmpFA1 ...
     ==================
 uvcmp1 - compare 2 files, print mismatched records, '*' flag diffs
 2019/08/23_12:09:44 uop=q1p30r256s6t1000000u3x2y0q1r256
 recsize  reccount   file-size  typ Report=rptcmp/custmas1
 1:  256        32       8,192  RSF  File1=dat2/custmas1
 2:  256        32       8,192  RSF  File2=dat2/custmas2
                       1         2         3         4         5         6
 f#record#byte# 0123456789012345678901234567890123456789012345678901234567890123
 ===============================================================================
 1      10    0 154688    TAURUS RESOURCES         1110 - 625 HOWE STREET   VANC
                3333332222545555254545544522222222233332223332445425554452225444
                15468800004152530253F5235300000000011100D062508F75034255400061E3
             64 OUVER        BC V6C2T6    604 685 8565MEL RAHAL         ........
                4554522222222442534353222233323332333344425444422222222200000001
                F5652000000002306632460000604068508565D5C02181C0000000000000C021
                                                                             **
            128 (...U'...f`]..8d........g ..............)l......................
                2100528006650036900000016280000000000000260000000000000000000000
                8C0357C0060D0084D0000C0070C0000C0000C0359C0000C0000C0000C0000C00
                  **
 2      10      154688    TAURUS RESOURCES         1110 - 625 HOWE STREET   VANC
                3333332222545555254545544522222222233332223332445425554452225444
                15468800004152530253F5235300000000011100D062508F75034255400061E3
             64 OUVER        BC V6C2T6    604 685 8565MEL RAHAL         ........
                4554522222222442534353222233323332333344425444422222222200000001
                F5652000000002306632460000604068508565D5C02181C0000000000000C231
                                                                             **
            128 (...U'...f`]..8d........g ..............)l......................
                2100528006650036900000016280000000000000260000000000000000000000
                8C3457C0060D0084D0000C0070C0000C0000C0359C0000C0000C0000C0000C00
                  **
 ==================== EOF or StopPrint/StopRead count reached  ==============
 F1Count=32, F2Count=32, StopPrint=6, StopRead=1000000
 F1Reads=32, MisMatches=1, MisMatsPrinted=1, Recsize=256
Note
  • record pair diffs flagged with '*'s below vertical hexadecimal bytes
  • bytes 192-255 omitted since no diffs (compare to uvhd on prior page)
  • omitting matching segments minimizes diff reports for large records

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5M3. uvcmpFA1 - compare packed/binary files

uvhd to update packed/binary files

Here is how I created custmas2 with 4 bytes different in only 1 record of 32, for the uvcmpFA1 compare demo on the previous page.


 #1. cp dat2/custmas1 dat2/custmas2  <-- copy custmas1 to custmas2
     ==============================

 #2. uvhd dat2/custmas1 r256u  <-- update custmas2 different than custmas1
     ========================    - note option 'u' to allow update
                - 1st record display omitted to save space

 #2a. --> 10  <-- goto record 10, displayed as follows:
          ===
 rec#=10 rsize=256 fptr=2304 fsize=8192 records=32
                      10        20        30        40        50        60
 r#       10 0123456789012345678901234567890123456789012345678901234567890123
        2304 154688    TAURUS RESOURCES         1110 - 625 HOWE STREET   VANC
             3333332222545555254545544522222222233332223332445425554452225444
             15468800004152530253F5235300000000011100D062508F75034255400061E3
          64 OUVER        BC V6C2T6    604 685 8565MEL RAHAL         ........
             4554522222222442534353222233323332333344425444422222222200000001
             F5652000000002306632460000604068508565D5C02181C0000000000000C021
         128 (...U'...f`]..8d........g ..............)l......................
             2100528006650036900000016280000000000000260000000000000000000000
             8C0357C0060D0084D0000C0070C0000C0000C0359C0000C0000C0000C0000C00
         192 ................Q...A.\........(<...............C 19980331     .
             0000000000000001580048500000001230000000000000004233333333222220
             00C0000C0000C0081C0019C0000C0068C0000C0000C0000C301998033100000A

 #2b. --> u 125(2),x'0203'    <-- update x'0002' to x'0203'
          ================
 #2c. --> u 130(2),x'0304'    <-- update x'0003' to x'0304'
          ================
 rec#=10 rsize=256 fptr=2304 fsize=8192 records=32
                      10        20        30        40        50        60
 r#       10 0123456789012345678901234567890123456789012345678901234567890123
        2304 154688    TAURUS RESOURCES         1110 - 625 HOWE STREET   VANC
             3333332222545555254545544522222222233332223332445425554452225444
             15468800004152530253F5235300000000011100D062508F75034255400061E3
          64 OUVER        BC V6C2T6    604 685 8565MEL RAHAL         ........
             4554522222222442534353222233323332333344425444422222222200000001
             F5652000000002306632460000604068508565D5C02181C0000000000000C231
         128 (...U'...f`]..8d........g ..............)l......................
             2100528006650036900000016280000000000000260000000000000000000000
             8C3457C0060D0084D0000C0070C0000C0000C0359C0000C0000C0000C0000C00
         192 ................Q...A.\........(<...............C 19980331     .
             0000000000000001580048500000001230000000000000004233333333222220
             00C0000C0000C0081C0019C0000C0068C0000C0000C0000C301998033100000A
       record updated - u 125(2),x'0203'
       record updated - u 130(2),x'0304'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5N1. system/application administration scripts

chmod1 - set perms on all levels - using 'find'

 #!/bin/ksh
 # chmod1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/...
 # chmod1 - change permissions on subdirs & files under a specified superdir
 #        - using 'find' to process all levels of directories & files
 #        - need to manually add 'x' perms on any bin/* & script/* dirs
 #        - by Owen Townsend, April 26/2006
 #
 # chmod1 directory dir-perms file-perms   <-- command format
 # ======================================
 # chmod1 directory 775 664                <-- recommended permissions
 # =========================
 #
 # - also see alternatives chmod2 & chmod3 (to this *chmod1)
 #*chmod1 - change perms on dirs & files, using 'find'
 # chmod2 - change perms on dirs & files, using 'recursion'
 # chmod3 - change perms on dirs & files, using 'recursion'
 #        - sets 'x' perm by testing for known names of bin/ & script/ subdirs
 #
 # After running chmod1 or chmod2, you must manually fix permissions on
 # executable programs & scripts via -->chmod 775 bin/*; chmod 775 scripts/*
 #
 # capture arguments & force perms integers
 dir="$1";
 typeset i dperm="$2"; typeset i fperm="$3";
 #
 # ensure arg1 is directory & length of perms are 3 digits
 dpl=${#dperm}; fpl=${#fperm};
 #
 if [[ -d "$dir" ]] && ((dpl==3 && fpl==3)); then :
    else echo "usage: chmod1 directory dir-perms file-perms"
         echo "       ====================================="
         echo "example: chmod1 dirxx 775 664"
         echo "         ===================="
         echo " - arg1 must be dir, args 2 & 3 must be 3 digits"
         exit 90; fi
 #
 echo -n "chmod1: set perms $dperm/$fperm from: $dir - enter to continue"
 read reply
 #
 find $dir -type d -exec chmod $dperm {} \;
 #=========================================
 #
 find $dir -type f -exec chmod $fperm {} \;
 #=========================================
 #
 echo "chmod1: perms set $dperm/$fperm for all subdirs & files within: $dir"
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5O1. lsold/mvold/rmold scripts

      lsold/mvold/rmold scripts to list/move/remove files older than x days.
      lsnew/mvnew/rmnew scripts to list/move/remove files newer than x days.
      - can find files anywhere in your directories & list/move/remove as desired

lsold1 - list files older than x days

 #!/bin/ksh
 # lsold1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # lsold1 - list files older than a specified number of days
 #
 if [[ -d "$1" && -n "$2" ]]; then :
    else echo "usage: lsold1 directory days"
         echo "       ====================="
         echo "example: lsold1 /u2/apps/data 30"
         echo "         ======================="
         echo "- list files in directory tree older than 30 days"
         echo " "
         echo "example2: lsold1 . 30  (use '.' if in directory)"
         echo "          ==========="
         echo "- Also see 'lsold1p' that allows a filename matching pattern"
         exit 1; fi
 days="+$2"
 #
 find $1 -mtime $days -type f -exec ls -l {} \;
 #=============================================
 # - list files older than a specified number of days
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5O2. lsold/mvold/rmold scripts

rmold1 - remove files older than x days

 #!/bin/ksh
 # rmold1 - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # rmold1 - remove files from a specified directory (& all subdirs),
 #          that are older(+) than a specified number of days
 #        - option to prompt y/n for each file to be removed
 #
 # 3 variations of this script as follows:
 #*rmold1 (this) - remove old files, option to prompt for each file to be removed
 # rmold2 - lists all old files, then prompts for each old file to be removed
 # rmold3 - lists all old files, then prompts once to remove all old files
 #
 # Also see:
 # lsold1 - list files older than specified number of days
 #        - might run this script before 'rmold1' (removes files)
 #          to prove files to be removed are as intended
 # mvold1 - might run this script instead of 'rmold1' (removes files)
 #          so you could recover files from the move to dir if required
 #
 if [[ -d "$1" && "$2" == +* && ("$3" == "-i" || "$3" == "-f") ]]; then :
    else echo "usage: rmold1 directory +days option"
         echo "       ============================="
         echo "example rmold1 history +730 -i"
         echo "        ======================"
         echo " - remove files over 2 yrs old, prompting for confirmation"
         echo " - arg1 must be a directory"
         echo " - arg2 must start with + (older than)"
         echo " - arg3 must be '-i' for confirmation, '-f' for none"
         exit 1; fi
 #
 find $1 -mtime $2 -exec rm $3 {} \;
 #==================================
 exit 0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5P1. 'dtree' script to draw directory trees

 'dtree' script to draw directory tree illustrations.
 - very useful for documentation (note usage in this documentation).

dtree - create directory tree illustration

 #!/bin/ksh
 # dtree - Korn shell script from UVSI stored in: /home/uvadm/sf/util/
 # dtree - list a directory tree
 #       - contributed by Howard Lobsinger (Peacock Engineering, Montreal)
 #usage: dtree directory
 #       ===============
 D=${1:-`pwd`}
 (cd $D; pwd)
 find $D -type d -print | sort |
 sed -e "s,^$D,,"\
     -e "/^$/d"\
     -e "s,[^/]*/\([^/]*\)$,\:-----\1,"\
     -e "s,[^/]*/,:     ,g" | more
 exit 0

dtree example


 #1. dtree /home/uvadm/demo   <-- demo with small directory
     ======================
 /home/uvadm/demo
 :-----archive
 :-----backup
 :     :-----dat1
 :     :-----dat2
 :     :-----sf
 :-----dat1
 :-----dat2
 :-----ftps
 :-----jcl2
 :-----jcl3
 :-----pf
 :-----rpts
 :-----selects
 :-----sf
 :-----sql0
 :-----sqls
 :-----stats
 :-----tf
 :-----tmp

 #2. dtree /home/uvadm    <-- demo with large directory
     =================      - would create about 600 lines
                            - not shown here, but you can run & observe

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_6 UVdemos - Vancouver Utilities for Unix/Linux users

Part 6 Help Menus - Contents

The uvhelp menus are handy reminders of the most useful Vancouver Utilities available - just enter 'uvhelp' at your shell prompt & scroll thru the list. Use whenever you have a problem that you think may have a Vancouver Utility solution, but can't remember the exact command.

using uvhelp & sub-menus v11-v92

Assuming you are 'userxx', have Vancouver Utilities installed,& have modified your profile to call the $UV/env/common_profile (documented in install.htm).


 #0. Login userxx --> /home/userxx
 #1. uvhelp  <-- display main-menu (this page)
 #2. scroll down to find the sub-menu (v11,v12,etc) for command desired
 #3. q       <-- quit main-menu
 #4. v##     <-- enter code for sub-menu (v11,v12,etc, lower-case)
 #5. q       <-- quit sub-menu
 #6. xxxxx   <-- enter command desired

menus are scripts with 'more' + documentation + EOF

The menus are simply shell scripts with a 'more' command at the begining & an EOF marker at the end. For example:

 # v11 - help sub-menu for Vancouver Utilities, stored at $UV/help/v11
 #     - called by main-menu $UV/help/uvhelp
 more <<'EOF'
                 ** v11 - Listing files & directories **
 l    - 'l' (alias l='ls -l') saves keystrokes for this commonly used command
 llm  - List Long files in Directory & pipe to more (sorted by filename)
      - like 'ls -l | more', but allows argument directory (default current)
      - saves keystrokes & adds functionality to this often repeated command
 EOF           --- omitting remainder of help/v11 ---

uvhelp menus part of Vancouver Utilities

The uvhelp menus are installed as part of Vancouver Utilities. You can see separate install instructions at uvhelp.htm#6V91 & how to create your own menus at uvhelp.htm#6V92.

 /home/uvadm           <-- Vancouver utilities homedir
 :     :                   (not showing other subdirs)
 :-----help              - subdir for help menus
 :     :-----uvhelp    <-- main-menu for Vancouver Utility helps
 :     :-----v11         - 1st sub-menu for VU helps
 :     :-----...               --- etc ---
 :     :-----v99         - last sub-menu for VU helps

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

uvhelp Reminders for script names & arguments

For this example, let's suppose we know there is a script to count files & lines in a directory, but we have forgotten its name & operating instructions. We can easily find it by entering 'uvhelp' for the main menu, scroll down to find the sub-menu code, then quit the main-menu, enter the sub-menu code to see the detailed operating instructions.


 #0. Login userxx --> /home/userxx

 #1. uvhelp   <-- display uvhelp main-menu
     ======
               ** uvhelp - main-menu of sub-menus **
 v11. Listing files & directories sorted by name,date,size etc
      llm  - sorted by filename, same as 'ls -l | more', saves keystrokes
      llt  - sorted by creation date, latest first
      lls  - sorted by file size, biggest first
      ...  - several more v11 scripts (omitted here)
 v12. counting files in directories & lines in files, some matching patterns
      (wc, cfl, cfd, cfdt, cfdpl, cfdd, cfddt, cfdpf, cfddf, cfdmm)
      wc    - unix Word Count (wc used in some of the following scripts)
      cfl   - Count Lines in 1 File
 -->  cfd   - Count Files,Lines,KB in 1 Directory    <-- we want this script
      cfdt  - Totals-Only version of Count Files,Lines,& KiloBytes in a Directory
                  --- omitting other scripts ---
        ----- Now we see the script we want 'cfd', can Quit the main menu -----
               & goto the sub-menus for script details & examples

 #3. q       <-- QUIT main-menu
     ====

 #4. v12     <-- enter code for sub-menu 'v12' to see details for
     ====
 #4a. ...    <-- scroll down to cfd instruction & sample report
               - a few lines shown below

#2. cfd - Count Files,lines,KB in 1 directory


 cfd dat1        <-- Command Format & Sample Report
 ========
 cfd - Count Files,Lines,& KiloBytes in a Directory
 File#    Lines      KB  Directory/Filename 20190722:0819
      1     335      32  dat1/CanadaMPs
                           --- 4 lines omitted ---
      6      50       4  dat1/USstates
 ******    1250     112  *Totals* in Directory /home/uvadm/demo/dat1
                           --- remainder v12 SUB-menu omitted ---
 #5. q      <-- QUIT sub-menu
     ===

 #6. cfd /home/demo/dat1   <-- enter desired command for desired directory
     ====================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

reading documentation vs running help menus

If you don't already have the Utiliites installed or just want to read more about the uvhelp menus, you can click on the following links. Note that the special format "+" links will take you to separate document uvhelp.htm.

To avoid losing your place in this UVdemos.doc, it is best to 'right-click' on the desired links & open in 'new-tab'. Then you will not get lost in uvhelp.doc without an easy way back.

Do not confuse the HTML-Links (6V11-6V99 UPPER-case 'V') in website documentation with the sub-menu codes (v11-v99 lower-case 'v') required at your shell prompt.

uvhelp - main-menu of sub-menus

Note
  • You must enter the sub-menus in lower-case ('v11' vs '6V11')
  • the UPPER-case 'V' is required for the HTML links
  • but the menu scripts are coded with lower-case.

6V11+ Listing files & directories sorted by name,date,size etc
llm - sorted by filename, same as 'ls -l | more', saves keystrokes
llt - sorted by creation date, latest first
llta - sorted by last access time, latest first
lls - sorted by file size, biggest first
llc - similar to 'llm', but with file seq# & line counts/file
llc2 - 'llm' + seq# + line count + Min/Max Record-Size & line# of Min/Max
... - several more, all scripts pipe to more, enter for next screen

6V12+ counting files in directories & lines in files, some matching patterns
wc - unix Word Count (wc used in some of the following scripts)
cfl - Count Lines in 1 File
cfd - Count Files,Lines,KB in 1 Directory
cfdt - Totals-Only version of Count Files,Lines,& KiloBytes in a Directory
cfdpl - Count Files in a Directory with a Pattern [or not] on any line in any file
cfdd - Count Files,Lines,& KB in a ALL Sub-Dirs in a Super-Directory
cfddt - Count Files,Lines,& KB in a ALL Sub-Dirs in a Super-Directory Totals-Only
cfdpf - Count Files in a Directory with a Pattern [or not] in filenames
cfddf - Count Files,Lines,KB in ALL Sub-Dirs in a Super-Dir + 1st few files
cfdmm - List Directory: File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#, Dir/Filename

6V13+ Renaming files in a dirctory (all or matching a pattern)
renameL - translate filenames to lower case, renameU to UPPER case
renameX - replace eXtension with alternate, renameP Prefix to alternate
renameAA - replace Any pattern with any Alternate anywhere in filename
renameB2_ - change blanks to underscores, renameD2_ '$' signs to '_'s
rename1R - rename 1 file Recursively, searching directory tree
......... - several more, all scripts display old/new filenames with seq#

6V14+ finding differences between 2 files or 2 directories of files
diff - show differences in 2 text files, most impressive unix command
alldiff2 - diff all files in 2 sub-dirs, great for mass changes
- save old, run new, alldiff prove intended changes, no unwanted
uvcmp1 - for fixed-length files (diff works only for text files)
- great for files with packed/binary, shows diffs in hexadecimal

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

uvhelp - unix & VU most useful command summaries


6V15+ list/remove/move files older than or newer than specified days
lsold1 - list files older than a specified number of days
lsnew1 - list files newer than a specified number of days
rmold1 - remove files older than the specified number of days
...... - several more, all recursive (descending directory tree)
- can list old files before removing (or moving to archive dir)

6V21+ find - finding files whose filenames match a pattern
- examples using unix 'find' (recursive, descends directory tree)

6V22+ grep - finding records (within files) matching a data pattern
- examples using unix 'grep', alias with -n to show line# of matches

6V23+ sed - unix utility to search/modify text files
- examples using 'sed' for search/replace
runsed1 - script to repeat sed for all files in directory

6V24+ removeCR - copy a file to tmp/ removing CRs & nulls, then prompt to copy back"
insertCR - copy a file to tmp/ inserting CRs, then prompt to copy back"
removeCRall indir outdir - remove all CR's & nulls from ALL files in a directory
insertCRall indir outdir - insert CR's on all lines of ALL files in a directory
cat12all - concatenate 1 file to each of all files in a directory
- use to append a LineFeed (missing only on last line)
of all files in a direcotory of files FTP'd from a mainframe

6V25+ dropbl0 - drop all blank lines using 'sed'
dropbl1 - drop all blank lines using 'uvcp' option typ=LSTb2
dropbl2 - reduce multi blank lines to 1 using 'uvcp' option typ=LSTb1
dropbl3 - drop blank lines & COBOL blank comments (only '*'s & blanks)
- all scripts copy to tmp/ for inspection & copy back if OK

6V26+ tr - unix command to translate any specified characters to alternates
- folowing scripts use 'tr' to translate all files in directory
 all2lower - translate to lower-case, copying all files to 2nd subdir
 all2upper - translate to UPPER-case, copying all files to 2nd subdir
  - mainframe conversions need to inhibit translates within quotes
tolower - uvcopy job translate to lower with options inhibit within quotes
toupper - uvcopy job translate to UPPER with options inhibit within quotes
cleanup - uvcopy job with many options for lower/UPPER, clear 1-6, 73-80,etc

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

uvhelp - unix & VU most useful command summaries


6V31+ Printing scripts, print text files at various lpi, cpi, margins, etc
 uvlp__ scripts - for listing scripts, programs, parameter files, etc
  - page headings with filename,date,user,page#,line#
uvlp12 - list at 12 cpi (90 chars on 8", leaving margin for 3 holes)
uvlp13 - list at 13 cpi (100 chars on 8", leaving margin for 3 holes)
 uvlp14,16,18 - 14 cpi=110 chars, 16 cpi=120 chars, 18 cpi=135
 uvlp13D - Duplex, 13 cpi, 6 lpi, 60 lines/page, for UV documentation
 uvlp13LD - Landscape & Duplex, 13 cpi, 8 lpi, 66 lines of 132
 uvlpd13 - list all files in directory with file list on 1st page
 ....... - about 50 more with various combinations cpi, lpi, etc
 listall1 - script to print multiple small files, combines as many
  small files as fits on a page & knows when to start newpage
 lpstat - unix command to display status of printers & queues
 allcancel - cancel all outstanding print requests for logged in user

6V32+ uvlp__A4 scripts print text files for 'A4' paper 8x12 European/Asian
- page headings with filename,date,user,page#,line#
- perfect for listing scripts, programs, parameter files, etc
Note - 'uvlp' script names for A4 paper are similar to the 8 1/2 x 11 names
- just append 'A4' onto the names for 8 1/2 x 11 paper
uvlp13A4 - list at 13 cpi (100 chars on 8", leaving margin for 3 holes)
 uvlp14A4,16A4,18A4 - list at 14,16,18 cpi (18 for 132 chars on 8")
 uvlp13DA4 - Duplex, 13 cpi, 6 lpi, 60 lines/paage, for VU documentation
 uvlp13LDA4 - Landscape & Duplex, 13 cpi, 8 lpi, 66 lines of 132
 uvlpd13A4 - list all files in directory with file list on 1st page
 ....... - about 50 more with various combinations cpi, lpi, etc
 lpstat - unix command to display status of printers & queues
 allcancel - cancel all outstanding print requests for logged in user

6V33+ uvhd - display any file in vertical hexadecimal
- browse,search,select,update,print,acum,count,
- a free download from www.uvsoftware.ca/libuvhd.htm

6V34+ uvhdcob - similar to uvhd, but displays cobol copybook fieldnames
beside data field contents

6V35+ uvcp - command line utility for record selection, reformatting,
translation, numeric field conversions among packed/zoned/binary, etc.

6V36+ uvsort - provides mainframe sorting capability to UNIX systems.
- replaces mainframe SORT/SYNCSORT on unix/linux systems
 UNIX system sort cannot process fixed record sizes without LineFeeds,
 packed decimal fields,& Indexed files.

6V37+ uvcpfix1 - copy Text files changing file-type, record-size,& reformatting
uvcpfix2 - copy Fixed-Length records changing type, size,& reformatting
These scripts call the 'uvcp' utility & the commands are much easier to
enter than calling uvcp directly. The scripts copy the data file to tmp/...
You can then inspect & if OK, copy to desired location or overwrite input

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

uvhelp - unix & VU most useful command summaries


6V41+ uvcopy powerful (can do anything) data file manipulation utility.
the power of assembler without the complexity (no compile)
just edit a parameter/instruction file & run with uvcopy interpreter
essential for mainframe applications on Unix/Linux systems,
because it can do many things not possible with the unix utilities
such as: processing 'packed decimal' fields, handling 'Indexed files',
& addressing by column# (common on mainframe, but not on unix.

6V42+ uvfix1,uvfixA,uvfix2,uvfixB modify 1 file or all files in directory
applying uvcopy instructions without writing file I/O framework
These scripts make it EASY to use the POWER of the uvcopy utility.
       You can do useful work with only 1 instruction such as 'fix' to convert
       delimited files to fixed-field formats. Or 'rep' to search for a pattern
       & REPlace all occurrences with an alternate pattern.
       Or even No instructions - just enter '.' period (indicates end of
       instructions) to copy 1 or ALL files in a directory, automatically
       removing trailing blanks & removing CR's x'0D's that can cause problems
       on unix systems (CR's can cause scripts to fail).

6V43+ list long records, 100 bytes/line with scale
- great for mainframe type records when you need to determine column#s
 listrec1/listhex1 - for Text records with LF terminators
 listrec2/listhex2 - for Fixed-Length records without LFs

6V44+ tabfix1,2,3 - converting tabs to blanks
option for how many, or depending on column, or tab-rack

6V45+ table summaries, count records & acum amounts by any argument
 table2 script makes it easy, example: count sales by product#
 table2 dat1/sales3 a30b6r64 - options identify argument & recsize
 ===========================
- a30=displacement(to product#), b6=length, r64=record-size

6V46+ addup1 - accumulate any field by options for displacement & length
- addup1 for Text files terminated by Line-Feeds
- addup2 for Fixed-Length record files (max recsize 4096)
- Example, to acum $sales start col=54,length=9,decimals=2
 uvcopy addup1,fili1=dat1/sales2,uop=b53c9e2
 ===========================================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

uvhelp - unix & VU most useful command summaries


6V81+ vi Command Summary - minimum essentials
+ links to advanced tutorials

6V82+ Unix/Linux Tips & Directory Housekeeping

6V83+ Vancouver Utilities Directories
- links to installation & profiles required

6V84+ Mainframe Conversion Super-dirs & sub-dirs
- links to more information

6V85+ cross-references - COBOL & JCL/scripts
 sample reports - xcobcopy1/2, xkshprog1/2, cobfiles report, etc
- links to more samples & operating instructions

6V91+ Download & Install uvhelp & uvscripts
- available to anyone regardless of customer or not

6V92+ Creating your own help menus
- for your own applications, or for any commands that you would like
  to have an easy way to be reminded of the arguments required.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_7. Vancouver Utility Programming Aids - uvcopy & uvfix

Part 7 uvcopy & uvfix - Contents

'uvcopy' is the most powerful Vancouver Utility. It is a 'general purpose' (can do anything) data manipulation utility. It gives you the power of assembler without the complexity. It interprets the instruction files for pre-programmed jobs such as SelectJobs & TableJObs. You do not need to know the uvcopy instruction set to run the pre-programmed jobs.

But if you want to use it for your own customized applications, you can see the over 100 instructions at uvcopy3.htm. And be sure to see scripts uvfix1 & uvfixA in UVdemos.htm#Part_7 - the easy way to use the power of uvcopy instructions without writing uvcopy jobs.


7A1. Sample Problem for demo - Convert .csv file to fixed-fields - 2 solutions
1st uvcopy - write complete uvcopy job
2nd uvfix1 - prompts to enter only a few instructions

7A2. Directories to demo uvcopy & uvfix1
Tips & Recommednations for writing uvcopy jobs

7A3. uvcopy job to convert USstates.csv to fixed fields

7A4. Executing uvcopy job & console log

7B1. uvfix1 - easy way to modify files with uvcopy powerful instructions
- alternate solution converting .csv file to fixed field format

7C1. uvfixA - modify ALL files in directory
- copy all files from 1 directory to a 2nd directory, modifying records
  in all files via instruction keyins (vs uvfix1 1 file at a time)
- example translates directory of SQL scripts to lowercase content & filenames
  & automatically shortens maninframe FTP 80 column records to last non-blank

7D1. uvfix1 - creating scripts with filenames in fixed columns
- useful making scripts to copy all files from 1 directory to a 2nd directory,
  changing filenames (UPPERcase,lowercase,drop extensions,add extensions,etc).
 Example given to change filenames in directory of SQL scripts from mainframe:
- from --> SELECT.CUSTOMERS.ALL
- to ---> select_customers_all.sql

7E1. MakeTelList - uvcopy job to create a Telephone List
- see this uvcopy job at /home/uvadm/demo/pf/MakeTelList.
- suggested exercise to create the same list using uvfix1 vs uvcopy
- similar to those shown on pages '7D1' to create a script to copy SQL scripts
  changing filenames from mainframe conventions to Linux conventions.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part 7 uvcopy - Contents (continued)


7F1. Expanding $VARIABLES using the uvcopy 'evs' instruction
- demo jobs to illustrate the 'evs' instruction
testevs1 - hard-coded variables expanded ($HOME,$LOGNAME,etc)

7F2. testevs2 - prompts you to enter your strings with $Variables

7F3. expandevs1 - copy a file expanding $VARIABLEs
- testevs1 & testevs2 above were demos only
 expandevs1 can be used to do real work
- critical converting mainframe JCL unloading DB2 tables
  for operation on unix/linux systems. (see next page '7F4')
 First a demo that you can run

7F4. expandevs1 used for mainframe DB2 conversions to unix
- used to expand $variables in DB2 scripts to load/unload tables
  We added code with $Variables to the SYSIN files for the db2 utility
  & used 'expandevs1' to copy the file before submitting to db2 utility.
- required because the 'db2' utility does NOT EXPAND $VARIABLEs.

7F5. JCL conversion to unload DB2 tables on unix/linux
- illustrate the 3 components involved (essential lines only).
 #1. parms/db2sysinxx
 #2. DB2 step in JCL/script
 #3. dsntiaul - script to unload DB2 tables

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

uvcopy - power of assembler without the complexity

The uvcopy language is based on the IBM 360/ZOS assembler instruction set, which means many mainframers will be familiar with it. Since it is interpretive, it gives you the power of assembly without the complexity. No compile required, just edit/modify the parameter file and run/rerun.

'uvcopy' provides the capabilities on unix/linux to replace various mainframe utilities such as IDCAMS, IEBGENER, EASYTRIEVE, QUIKJOB, etc.

'uvcopy' allows you to do anything on unix/linux systems that you could do with mainframe utilities - process packed decimal fields, Indexed files, variable length files, fixed length records without linefeeds, and column addressing. Column addressing is heavily used by mainframe utiltiies but not provided by the most Unix/Linux system utilties.

See over 100 uvcopy instructions documented at www.uvsoftware.ca/uvcopy3.htm and see sample problems & solutions at www.uvsoftware.ca/uvdemos.htm#Part_7

The 1st example at uvsoftware.ca/uvdemos.htm#7A1 illustrates the 'fix' instruction to convert a .csv (comma separated variables) file to a customized fixed field format which might be required for input to your existing applications.

Another example of uvcopy's powerful instructions is 'evs' which expands $VARIABLEs. See demo job testevs1 at uvsoftware.ca/uvcopy3.htm#evs which copies a demo file expanding $HOME, $LOGNAME, $HOSTNAME, $datetime, etc.

A practical example of 'evs' was preprocessing DB2 SYSIN files expanding $VARIABLEs such as $SYSREC00 for input to db2 which does not expand $VARIABLEs. See the demo version at uvsoftware.ca/uvcopy3.htm#expandevs1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7A1. Vancouver Utility Programming Aids - uvcopy & uvfix

uvcopy/uvfix1 - Convert .csv file to fixed-fields

We wanted to create a table of US state names & abbreviations. We downloaded the following .csv file. Here are a few sample records of Input/Output records:

                ** Input: $UV/demo/dat1/USstates.csv
      ALABAMA,Alabama,AL,Ala.
      ALASKA,Alaska,AK,Alaska
      ARIZONA,Arizona,AZ,Ariz.
   -------- omitting 44 records --------
      WEST VIRGINIA,West Virginia,WV,W.Va.
      WISCONSIN,Wisconsin,WI,Wis.
      WYOMING,Wyoming,WY,Wyo.

desired Output: $UV/demo/dat1/USstates

               1         2         3         4         5         6
      123456789012345678901234567890123456789012345678901234567890
      01  AL  Alabama
      02  AK  Alaska
      03  AZ  Arizona
        -------- omitting 44 records --------
      48  WV  West Virginia
      49  WI  Wisconsin
      50  WY  Wyoming

We will show you 2 solutions illustrated on the following pages.

  1. Writing a 'uvcopy job' - Parameter File of uvcopy instructions to be interpreted by the uvcopy interpreter.

  2. Using script 'uvfix1' & pre-programmed job 'uvfix1' to supply the uvcopy framework, file-handling, prompting for input & output viewing. This means we only have to code a few instructions.

First we will show you the uvcopy job followed by the uvfix1 solution.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7A2. Converting .csv file to fixed format to demo uvcopy/uvfix1

Directories to demo uvcopy/uvfix1

Here are the most relevant subdirs/files required to demo the uvcopy & uvfix1 methods of converting dat1/USstates.csv to fixed field dat1/USstates.

 /home/userxx/demo/              - copy to your homedir from /home/uvadm/demo/*
 :-----dat1                    <-- testdata files for demos
 :     :-----USstates.csv      <-- US state names & 2 letter code abbreviations
 :     :-----USstates            - desired fields extracted to fixed positions
 :     :
 :-----pf                      <-- uvcopy job Parameter Files
 :     :-----USstatescsv2txt     - suggested name for job to convert csv to fixed
 :     :-----                    - you could create your own uvcopy jobs here
 :     :
 :-----tmp                     <-- tmp dirs for temp files, sorts, etc
 :     :                         - uvfix1 output for inspection & copyback
 :-----tmp1                    <-- suggest using tmp1/... for uvfixA outputs

Tips for writing uvcopy jobs

We have already written this uvcopy job. See the result listed on the next page, but first, here are a few tips to help you develop uvcopy jobs for your purposes.

  1. When you need to write a new program (in any language), you usually search for an already written program that does a similar function & copy/rename/modify.

  2. You can search the $UV/pf/.../... directories for similar jobs. Note that /home/uvadm/pf/* has 4 subdirs pf/adm/..., pf/demo/..., pf/util/..., pf/IBM/...

  3. I suggest you grep the $UV/pf/demo directory for words that would occur in a similar job. In this case you would grep for '.csv'.

  4. Now copy/rename/modify to solve your current problem.

  5. See all instructions documented at uvsoftware.ca/uvcopy3.htm.

    Recommendations for writing uvcopy jobs

Use lots of #comments to explain your job to other programmers & for yourself 6 months later. Here are 3 important #comment groups at the begining of the job:

  1. uvcopy jobname, purpose, where stored, author,& date

  2. Execution instructions

  3. Sample I/O records

  4. Logic description if complex (can omit for short simple jobs)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7A3. uvcopy job to convert USstates.csv to Fixed-Fields

writing uvcopy job to convert USstates.csv to fixed fields


 #0a. Login userxx --> /home/userxx/
 #0b. cd demo -------> /home/userxx/demo/

 #1. vi pf/USstatescsv2txt   <-- write the uvcopy job instructions
     =====================
 # USstatescsv2txt - convert US States & Abbrevs from comma delimited to fixed columns
 #                 - stored in $UV/pf/demo/... & in $UV/demo/pf/...
 #                 - by Owen Townsend, UV Software, Jun30/2019
 #
 # uvcopy USstatescsv2txt,fili1=dat1/USstates.csv,filo1=dat1/USstates
 # ==================================================================
 # uvcopy USstatescsv2txt   <-- same as above, file defaults on fili1/filo1 below
 #
 #   ** sample I/O 1st & last records **
 #
 # ALABAMA,Alabama,AL,Ala.
 # WYOMING,Wyoming,WY,Wyo.
 #         10        20        30
 # 123456789012345678901234567890
 # 01  AL  Alabama
 # 50  WY  Wyoming
 #
 opr='$jobname - convert comma delimited US City populations to fixed columns'
 rop=r1  # EOF option prompt to view output (reply: vi,cat,more,etc)
 fili1=?dat1/USstates.csv,typ=LST,rcs=512
 filo1=?dat1/USstates,typ=LST,rcs=100
 @run
         opn    all
 man20   get    fili1,a0                 get next record
         skp>   man90                    (cc set > at EOF)
 # fix fields 100 bytes apart & reformat comma separated fields into desired format
         fix    c0(100),a0(256),4,','    separate fields 100 apart in area 'c'
         clr    b0(100)                  clear output record area 'b'
         add    $ca1,1                   increment sequence#
         mvn    b0(2),$ca1               sequence#
         mvc    b4(2),c200               state abbreviation
         mvc    b8(16),c100              state name
         put    filo1,b0                 write output from area 'b'
         skp    man20                    return to get next record
 # EOF - close files & end job
 man90   cls    all                      close files
         eoj                             end job

Area addresses must be zero relative displacements (vs 1 relative columns) and data movement is from operand 2 (sending) to operand 1 (receiving). See all instructions documented at uvsoftware.ca/uvcopy3.htm.

The most vital instruction is 'fix' that separates the ',' comma delimited input fields to fixed fields 100 bytes apart. We can then easily reformat the fields to the desired output locations.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7A4. uvcopy job to convert USstates.csv to Fixed-Fields

Executing uvcopy job to convert USstates.csv to fixed fields


 #2. uvcopy pf/USstatescsv2txt  <-- execute uvcopy on Param File USstatescsv2txt
     =========================

console log

 USstatescsv2txt - convert comma delimited US City populations to fixed columns
 190711:174047:USstatescsv2tx: uvcopy ver=20190704 pf=pf/USstatescsv2txt
 uvcopy LNX H64 license=20190704_99V_930630 site=UV_Software
 dat1/USstates.csv = default fili01 - null accept or reenter ?
 dat1/USstates = default filo01 - null accept or reenter ?
 190711:174051:USstatescsv2tx: EOF fili01 rds=50 size=1357: dat1/USstates.csv
 190711:174051:USstatescsv2tx: EOF filo01 wrts=50 size=872: dat1/USstates
 EOJ, Output File written to: dat1/USstates
 default command = null, OR enter: vi,cat,more,lp,uvlp12,etc -->
 --------> head <-- enter head to display 1st 10 output records
 01  AL  Alabama
 02  AK  Alaska
 03  AZ  Arizona
 04  AR  Arkansas
 05  CA  California
 06  CO  Colorado
 07  CT  Connecticut
 08  DE  Delaware
 09  FL  Florida
 10  GA  Georgia

Notes

  1. In this case we coded the 'pf/' directory on the uvcopy command - because we are operating within the $HOME/demo/... directory.

  2. uvcopy can find most uvcopy jobs automatically because the common_profile includes 'export PFPATH=...' (similar to export PATH=...)

      export PFPATH=$HOME/pf,$UV/pf/adm,$UV/pf/demo,$UV/pf/util,$UV/pf/IBM
      ====================================================================
  1. $HOME/pf is coded prior to $UV/pf/... so your same-named uvcopy jobs would be found before Vancouver Utility jobs.

  2. We used ',' separators vs the usual ':' separators to allow the same PFPATH definition on Windows systems, but either works on Unix/Linux.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7B1. uvfix1 - easy way to use uvcopy for simple jobs

'uvfix1' is the easy way to use the power of uvcopy when only a few instructions are required. You execute script $UV/sf/util/uvfix1 which calls uvcopy job $UV/pf/util/uvfix1. You only need to specify the input file.

The output is automatically written to the tmp/ directory in your current directory. You can inspect the output & if OK, you can copyback replacing the original if your intention was to fix a problem in the file (often the case).

uvfix1 alternate to convert .csv file to fixed field


 #0a. Login userxx --> /home/userxx/
 #0b. cd demo -------> /home/userxx/demo/

 #1. uvcopy uvfix1,fili1=dat1/USstates.csv  <-- run uvcopy uvfix1 directly
     =====================================

 #1a. uvfix1 dat1/USstates.csv  <-- run script uvfix1 to call uvcopy uvfix1
      ========================    - easier Alternative (fewer keystrokes)

uvfix1 log

 uvfix1 - copy a text file & modify via uvcopy instruction keyins
        - input rec in area a has already been copied to output rec in b
 uop=q1c0n99r8192 - user option defaults
       c1         - pass thru any "#" col1 #comment records
         n99      - output record count limit (high# copies to EOF)
            r8192 - recsize ONLY for Fixed-Length OUTPUT records if desired
                  - for Fixed-Length, reply RSF/RST at prompt(default text typ=LSTt)
                  - for Text files: accept default recsize & default typ=LSTt
 User OPtion (uop) defaults  = q1c0n999999999r8192
 -->null to accept or override -->   <-- null to accept default options
 -->LSTt = typ default filo01 --->   <-- null to accept default file typ
   clr  b0(80)      <-- example: clr - CLeaR output area b
   mvc  b6(80),a0   <-- example: mvc - MoVe Char a to b offset 6 right
   add  $ca1,1      <-- example: add - count records
   mvn  b0(5),$ca1  <-- example: mvn - MoVe Numeric seq# to 1st 5 bytes
 enter uvcopy instructions (period "." terminates entries)
 --> fix c0(100),a0(256),4,','     <-- Note 6 Instructions entered
 --> clr b0(100),' '
 --> add $ca1,1
 --> mvn b0(2),$ca1
 --> mvc b4(2),c200
 --> mvc b8(16),c100
 --> .
 190712:121236:uvfix1: EOF fili01 rds=50 size=1357: dat1/USstates.csv
 190712:121236:uvfix1: EOF filo01 wrts=50 size=872: tmp/USstates.csv
 default command = null,vi,cat,more,lp,uvlp12,etc --> head <-- show 1st 10 records
 01  AL  Alabama
       ------          <-- showing 1st & last records only
 10  GA  Georgia
 check output in tmp/USstates.csv, copyback over-writing original ???
 ---- OR, may need to copy/rename (preserving original) ???

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7C1. uvfixA - modify ALL files in directory

'uvfixA' will copy all files from 1 directory to a 2nd directory, modifying records in all files via instruction keyins (vs 'uvfix1' modify only 1 file at a time).

uvfixA Example - modify directory of SQL scripts

Assume that we have FTP'd multiple SQL scripts from a mainframe to Linux & we wish to convert them to Linux standards vs mainframe conventions. Output files from the mainframe FTP ASCII are usually fixed length 80 bytes with CR/LF in columns 81-82, filenames are UPPER-case,& the data contents are UPPER-case.

We will copy all SQL scripts from the FTP output subdir sql0/... to tmp2/... while translating both filenames & data contents from UPPER-case to lower-case & automatically shortening mainframe FTP 80 column records to last non-blank.

relevant subdirs/files to demo uvfixA SQL scripts

 /home/userxx/demo/
 :-----sql0                          <-- SQL scripts FTP'd from mainframe
 :     :-----SELECT_CUSTOMERS_ALL
 :     :   --- 1st & last files ---
 :     :-----SELECT_STUDENTS_FAILING
 :-----sqls                          <-- SQL scripts copied from tmp2
 :     :                                - after checking OK
 :-----tmp2                          <-- tmp2/ output from uvfixA
 :     :-----select_Customers_all
 :     :   --- 1st & last files ---  <-- inspect output, if OK copy to sqls/...
 :     :-----select_students_failing

 #0a. Login userxx --> /home/userxx/
 #0b. cd demo -------> /home/userxx/demo/

 #1. l sql0      <-- list files FTP'd from mainframe
     ======
 -rw-rw-r-- 1 userxx apps  55 Jul 14 15:15 SELECT_CUSTOMERS_ALL
 -rw-rw-r-- 1 userxx apps  97 Jul 14 15:15 SELECT_CUSTOMERS_BASIC
 -rw-rw-r-- 1 userxx apps 146 Jul 14 15:15 SELECT_CUSTOMERS_SALES10000
 -rw-rw-r-- 1 userxx apps 149 Jul 14 15:15 SELECT_CUSTOMERS_SALES20000
 -rw-rw-r-- 1 userxx apps 136 Jul 14 15:15 SELECT_EMPLOYEE_DEPT123
 -rw-rw-r-- 1 userxx apps 115 Jul 14 15:15 SELECT_EMPLOYEE_SEATTLE
 -rw-rw-r-- 1 userxx apps  76 Jul 14 15:15 SELECT_ORDERS_BIG
 -rw-rw-r-- 1 userxx apps 109 Jul 14 15:15 SELECT_PATIENTS_OVER70
 -rw-rw-r-- 1 userxx apps 109 Jul 14 15:15 SELECT_PATIENTS_OVER80
 -rw-rw-r-- 1 userxx apps 109 Jul 14 15:15 SELECT_PATIENTS_OVER90
 -rw-rw-r-- 1 userxx apps 126 Jul 14 15:15 SELECT_STUDENTS_FAILING

 #2. cat sql0/SELECT_CUSTOMERS_ALL  <-- inspect 1st file, UPPER case ?
     =============================
      /* SELECT_CUSTOMERS_ALL */
         SELECT * FROM CUSTOMERS;

 #3. rm -f tmp2/*             <-- remove all files from intended output subdir
     ============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7C2. uvfixA - modify ALL files in directory

Execute uvfixA to modify All files in directory of SQL scripts


 #4. uvfixA sql0 tmp2 LSTt2 t1  <-- arg1=indir,arg2=outdir,arg3=outfiletype,arg4=options
     =========================    - log as follows:
uop=q1c0n99r8192t0
  • user option defaults
  • pass thru any "#" col1 #comment records
  • output record limit (n99 special meaning ALL records)
  • records counted in $ca10 & compared to option n
  • recsize ONLY for Fixed-Length OUTPUT records if desired
  • for Fixed-Length, reply RSF/RST at prompt (default text typ=LSTt)
  • for Text files: accept default recsize & default typ=LSTt
  • translate output file-names to lower-case
  • translate output file-NAMES to UPPER-case

User OPtion (uop) defaults = q1c0n999999999r8192t0 #4a. -->null accept or override --> t1 <-- could enter options here (if not on cmdline)

enter instruction: #4b. --> trl b0(100) <-- enter 'trl' (TRanslate to Lower) area 'b' 1st 100 bytes

 enter instruction:
 #4c. --> .             <-- enter '.' period to end instruction keyins
 190714:154752:uvfixA: EOF fili01 rds=2 size=55: sql0/SELECT_CUSTOMERS_ALL
 190714:154752:uvfixA: EOF filo02 wrts=2 size=55: tmp2/select_customers_all
         --- showing I/O stats for 1st & last files only ---
 190714:154752:uvfixA: EOF fili01 rds=4 size=126: sql0/SELECT_STUDENTS_FAILING
 190714:154752:uvfixA: EOF filo02 wrts=4 size=126: tmp2/select_students_failing

 #5. l tmp2             <-- list output files (checck lower-case filenames)
     ======
      -rw-rw-r-- 1 userxx apps  55 Jul 14 15:47 select_customers_all
      -rw-rw-r-- 1 userxx apps  97 Jul 14 15:47 select_customers_basic
      -rw-rw-r-- 1 userxx apps 146 Jul 14 15:47 select_customers_sales10000
      -rw-rw-r-- 1 userxx apps 149 Jul 14 15:47 select_customers_sales20000
      -rw-rw-r-- 1 userxx apps 136 Jul 14 15:47 select_employee_dept123
      -rw-rw-r-- 1 userxx apps 115 Jul 14 15:47 select_employee_seattle
      -rw-rw-r-- 1 userxx apps  76 Jul 14 15:47 select_orders_big
      -rw-rw-r-- 1 userxx apps 109 Jul 14 15:47 select_patients_over70
      -rw-rw-r-- 1 userxx apps 109 Jul 14 15:47 select_patients_over80
      -rw-rw-r-- 1 userxx apps 109 Jul 14 15:47 select_patients_over90
      -rw-rw-r-- 1 userxx apps 126 Jul 14 15:47 select_students_failing

 #6. cat tmp2/*         <-- display 1st file contents (check lower-case data)
     ==========
      /* select_customers_all */
         select * from customers;

 #7. cp tmp2/* sqls     <-- if OK - copy files back to desired directory
     ==============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7D1. uvfix1 - easy way to use uvcopy for simple jobs

uvfix1 creating scripts with filenames in fixed columns

'uvfix1' helps make scripts with filenames in desired columns for neater appearance.

Useful when making scripts to copy all files from 1 directory to a 2nd directory, changing the filenames in various ways (UPPERcase, lowercase, dropping extensions, adding extensions, etc).

For example, suppose we have FTP'd SQL scripts from a mainframe to Linux & we wish to modify the filenames. Here are samples of the input & output filenames desired.

SQL scripts from mainframe in sql0/...

                        'SELECT.CUSTOMERS.ALL'
                        'SELECT.CUSTOMERS.BASIC'
                        'SELECT.CUSTOMERS.SALES10000'

SQL scripts on Linux renamed in sqls/...

                         select_customers_all.sql
                         select_customers_basic.sql
                         select_customers_sales10000.sql

filename rename changes desired

  1. convert UPPER-case filenames to lower-case
  2. remove the single quotes enclosing the filenames
  3. convert '.' period separators to '_' underscore separators
  4. add extension 'sql'
  5. align all output filenames the same at column 41 (neater apearance)

    sample output

 cp sql0/SELECT.CUSTOMERS.ALL            sqls/select_customers_all.sql
 cp sql0/SELECT.CUSTOMERS.BASIC          sqls/select_customers_basic.sql
 cp sql0/SELECT.CUSTOMERS.SALES10000     sqls/select_customers_sales10000.sql

Operating Procedures


 #0a. Login userxx --> /home/userxx/
 #0b. cd demo -------> /home/userxx/demo/

 #1. ls sql0 > sf/copy_rename_sql0
     =============================
     - capture mainframe filenames assigning appropriate name for intended script
       to be created by manipulations with uvfix1

 #2. cat sf/copy_rename_sql0   <-- confirm input as expected
     =======================     - showing 1st 3 of 11 lines
       SELECT.CUSTOMERS.ALL
       SELECT.CUSTOMERS.BASIC
       SELECT.CUSTOMERS.SALES10000

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7D2. uvfix1 creating scripts with filenames in fixed columns

uvfix1 - Execution & Console Log


 #3. uvfix1 sf/copy_rename_sql0
     ==========================
     - execute uvfix1, input filenames, named for intended script
     - console log as follows:
uop=q1c0n99r8192
  • user option defaults
  • pass thru any "#" col1 #comment records
  • output record limit (n99 special meaning ALL records)
  • records counted in $ca10 & compared to option n
  • recsize ONLY for Fixed-Length OUTPUT records if desired
  • for Fixed-Length, reply RSF/RST at prompt(default text typ=LSTt)
  • for Text files: accept default recsize & default typ=LSTt

User OPtion (uop) defaults = q1c0n999999999r8192 -->null to accept or override --> <-- null to accept default options -->LSTt = typ default filo01 ---> <-- null to accept default file typ

   clr  b0(80)      <-- example: clr - CLeaR output area b
   mvc  b6(80),a0   <-- example: mvc - MoVe Char a to b offset 6 right
   add  $ca1,1      <-- example: add - count records
   mvn  b0(5),$ca1  <-- example: mvn - MoVe Numeric seq# to 1st 5 bytes
 enter a uvcopy instruction (period "." terminates entries)
 #3a. --> mvf b0(100),'cp sql0/'
 #3b. --> cat b0(100),a0(30)
 #3c. --> mvf b40(60),'sqls/'     <-- see explanations on next page
 #3d. --> cat b45(50),a0(30)
 #3e. --> trl b45(50)
 #3f. --> rep b45(50),x'27',''    <-- remove the single quotes
 #3g. --> rep b45(50),'.','_'
 #3h. --> cat b45(50),'.sql'
 #3h. .
 190715:113936:uvfix1: EOF fili01 rds=11 size=259: sf/copy_rename_sql0
 190715:113936:uvfix1: EOF filo01 wrts=11 size=798: tmp/copy_rename_sql0
 EOJ, Output File written to: tmp/copy_rename_sql0

 #4. cat tmp/copy_rename_sql0       <-- confirm output as expected
     ========================         - showing 1st 3 of 11 lines
       cp sql0/SELECT.CUSTOMERS.ALL            sqls/select_customers_all.sql
       cp sql0/SELECT.CUSTOMERS.BASIC          sqls/select_customers_basic.sql
       cp sql0/SELECT.CUSTOMERS.SALES10000     sqls/select_customers_sales10000.sql

 #5. cp tmp/copy_rename_sql0 sf/    <-- copy tmp/ output to intended script
     ===========================

 #6. chmod 775 sf/copy_rename_sql0  <-- add execute permissions
     =============================

 #7. [ rm sqls/* ]         <-- may need to remove existing files in sqls/... ???
     =============

 #8. sf/copy_rename_sql0   <-- Execute script to copy sql0/* to sqls/...
     ===================       renaming files as desired

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7D3. uvfix1 creating scripts with filenames in fixed columns

uvfix1 - Explanations of Instructions entered

 We will explain the intructions entered to uvfix1 on lines #3a to #3h above
 - illustrating with the 1st record step by step manipulations
 SELECT.CUSTOMERS.ALL                   <-- 1st record input
       --- outputs after each instruction ---

 cp sql0/
 ========================================================
                                           ^-- after #3a. mvf b0(100),'cp sql0/'

 cp sql0/SELECT.CUSTOMERS.ALL
 ========================================================
                                           ^-- after #3b. cat b0(100),a0(30)

 cp sql0/SELECT.CUSTOMERS.ALL    sqls/
 =========================================================
                                           ^-- after #3c. mvf b40(60),'sqls/'

 cp sql0/SELECT.CUSTOMERS.ALL    sqls/'SELECT.CUSTOMERS.ALL'
 ===========================================================
                                           ^-- after #3d. cat b45(50),a0(30)

 cp sql0/SELECT.CUSTOMERS.ALL    sqls/'select.customers.all'
 ===========================================================
                                           ^-- after #3e. trl b45(50)

 cp sql0/SELECT.CUSTOMERS.ALL    sqls/select.customers.all
 =========================================================
                                           ^-- after #3f. rep b45(50),x'27',''

 cp sql0/SELECT.CUSTOMERS.ALL    sqls/select_customers_all
 =========================================================
                                           ^-- after #3f. rep b45(50),'.','_'

 cp sql0/SELECT.CUSTOMERS.ALL    sqls/select_customers_all.sql
 =============================================================
                                           ^-- after #3g. cat b45(50),'.sql'

Notes

  1. Could do some of these steps using an editor (such as 'vi')

  2. But hard to copy data to desired output columns for neat appearance

  3. And hard to restrict operations to desired column ranges

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7E1. Vancouver Utility Programming Aids - uvcopy & uvfix

MakeTelList - uvcopy job to create a Telephone List

See this uvcopy job at /home/uvadm/demo/pf/MakeTelList. You can execute as documented on line 4 or 6 of the listing below. See sample Input records on lines 8-10 & sample Output records on lines 14-16.

 # MakeTelList - reformat Customer Master to a Telephone List
 #             - see www.uvsoftware.ca/uvdemos.htm#7E1
 #
 # uvcopy MakeTelList,fili1=dat2/ar.custmas,filo1=tmp1/ar.custmas.tellist
 # ======================================================================
 # uvcopy MakeTelList  <-- same as above, files default below as shown above
 # ==================
 #                     ----- sample Input -----
 # 139923    JOHNSTONE BOILER & TANKS 1250 EAST PENDER STREET  VANCOUVER        BC V5L1W1    604-320-1845 GEORGE BROWN
 # 147615  X O'CONNER R.V. CENTRE     44430 YALE ROAD WEST     CHILLIWACK       BC V2P6J1    604-858-4161 BONNIE
 # 152355    SHAW, JOHN               477 CARIBOO CRES.        COQUITLAM        BC V3M1X1    604-291-6261 JOHN
 #         10        20        30        40        50        60        70        80        90       100       110       120
 # 123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 #                     ----- sample Output -----
 # 139923   JOHNSTONE BOILER & TANKS   604-320-1845  GEORGE BROWN
 # 147615   O'CONNER R.V. CENTRE       604-858-4161  BONNIE
 # 152355   SHAW, JOHN                 604-291-6261  JOHN
 #
 fili1=?dat2/ar.custmas,typ=LST,rcs=256
 filo1=?tmp1/ar.custmas.tellist,typ=LSTt,rcs=80
 @run
       opn   all                    open files
 # begin loop to get, reformat,& put records until EOF
 loop  get   fili1,a0               get record into area 'a'
       skp>  eof                    (condition code set > at EOF)
 #-----------------------------
       clr   b0(256),' '            clear output area
       mvc   b0(6),a0               cust#
       mvc   b9(25),a10             customer name
       mvc   b36(12),a90            telephone#
       mvc   b50(18),a103           contact name
 #-----------------------------
       put   filo1,b0               write record to output file
       skp   loop                   return to get next record
 eof   cls   all                    close files
       eoj                          end job

Suggested Exercise for you

Create the same Telephone list using 'uvfix1' (vs uvcopy job above). Procedures would be similar to those shown on pages '7D1' - 7D2 to create a script to copy SQL scripts changing filenames from mainframe conventions to Linux conventions.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7F1. Vancouver Utility Programming Aids - uvcopy

Expanding $VARIABLES

The uvcopy 'evs' instruction copies text strings expanding environmental $VARIABLEs. Here is a simple demo job you can run:


 #0a. Login userxx --> /home/userxx/
 #0b. cd demo -------> /home/usrxx/demo/

 #1. uvcopy testevs1      <-- demo 'evs' expand Environmental Variables
     ===============        - output shown below:
      homedir=$HOME, user=$LOGNAME, datetime=$datetime              <-- input
      ------------------------------------------------
      homedir=/home/uvadm, user=uvadm, datetime=2020/03/28_08:00:29 <-- output
      -------------------------------------------------------------

 #2. cat $UV/pf/adm/testevs1   <-- display the uvcopy job (Parameter File)
     =======================     - code listed below:
 # testevs1 - test uvcopy instruction 'evs' (test job in $UV/pf/adm/testevs1)
 #          - copy a string expanding environmental-variables
 #          - this job includes a hard-coded string with $HOME,$LOGNAME,etc
 #
 # uvcopy testevs1    <-- execute this job to test the "evs" instruction
 # ===============      - with hard-coded $Variables (see testevs2 prompts for entry)
 @run
         msg     '  --- test uvcopy instruction "evs" copy text expanding $VARIABLES ---'
         mvf     a0(100),'homedir=$HOME, user=$LOGNAME, machine=$HOSTNAME, datetime=$datetime'
         msg     a0(100)                show string BEFORE expanding variables
         evs     b0(100),a0(100)        copy from area 'a' to area 'b' expanding $variables
         msg     b0(100)                show string AFTER expanding variables
         eoj

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7F2. Demo uvcopy 'evs' instruction to expand $VARIABLES

testevs2 - 'evs' demo#2 - enter your strings with $Variables


 #3. uvcopy testevs2   <-- run 2nd 'evs' demo (prompts for your text with $Variables)
     ===============     - sample input & output shown below:
      Enter text containing $VARIABLEs ($HOME,$LOGNAME,$PWD,$datetime,etc)
      --> I am $LOGNAME, homedir=$HOME, HOST=$HOSTNAME, date/time=$datetime
      --> I am userxx, homedir=/home/userxx, HOST=uvsoft5, date/time=2020/03/30_10:14:24
      Enter another string OR null entry to end job

 #4. cat $UV/pf/adm/testevs2   <-- display the uvcopy job (Parameter File)
     =======================     - code listed below:
 # testevs2 - test uvcopy instruction 'evs' (test job in $UV/pf/adm/testevs2)
 #          - copy a string expanding environmental-variables
 #          - this job prompts you to enter a string containing any environment variables
 #
 # uvcopy testevs2    <-- execute this job to test the "evs" instruction
 # ===============      - prompting to enter string with $VARIABLEs
 @run
         msg     '          --- test "evs" - copy YOUR text expanding $VARIABLES ---'
 man10   msga1w  '     Enter text containing $VARIABLEs (ex: $HOME, $PWD, $datetime, etc)'
         skp<    man99                  goto end job if null entry
         mvfv3   a0(100),$arg1
         evs     b0(100),a0(100)        copy from area 'a' to area 'b' expanding $variables
         msg     b0(100)                show string AFTER expanding variables
         msg     '     Enter another string OR null entry to end job'
         skp     man10
 man99   eoj

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7F3. uvcopy 'evs' instruction to expand $VARIABLES

expandevs1 - copy a file expanding $VARIABLEs

The above (testevs1 & testevs2) were only demos, whereas 'expandevs1' can be used to do some real work - see the next page '7F4' where expandevs1 plays a critical role in converting mainframe JCL unloading DB2 tables to unix scripts for operation on unix/linux systems.

First, we show you the uvcopy job listing for expandevs1 & a short demo showing the test input file & output file with $variables expanded.

 # expandevs1 - copy a text file expanding unix ENVIRONMENTAL $VARIABLEs
 #            - by Owen Townsend, UV Software, Jan 2018
 # Example below used by db2 script dsntiaul to expand $SYSRTMP
 #  - JCL converter uses this technique to load/unload DB2 tables
 #
 # uvcopy expandevs1,fili1=$SYSIN,filo1=$SYSINTMP
 # ==============================================
 # - copy $SYSIN to $SYSINTMP, expanding $VARIABLEs ($SYSRTMP, possible others)
 # - used by db2 script dsntiaul to expand variables such as $SYSRTMP
 #   since input on db2 -txf $SYSIN does not expand
 # - may run this demo job with supplied test file $UV/tf/expandevs1_data
 #
 rop=r1      # prompt for output file disposition at EOJ
 fili1=?tf/expandevs1_data,rcs=256,typ=LST
 filo1=?tmp/$fili1,rcs=256,typ=LSTt
 @run
         opn    all
 # begin loop to copy expanding $variables
 man20   get    fili1,a0                get next line
         skp>   man90                   (cc set > at EOF)
         mvc    b0(200),a0              copy to output in case # col1
         cmc    a0(1),'#'               inhibit expansion ? (# col 1)
         skp=   man30                   ifso - skip evs, go output
         evs    b0(200),a0(100)         copy expanding any $variables
 man30   put    filo1,b0                write to output file
         skp    man20                   repeat loop until EOF
 # EOF - close files & end job
 man90   cls    all
         eoj

run expandevs1 with demo file


 #5. uvcopy expandevs1,fili1=tf/expandevs1_data,filo1=tmp/expandevs1_data
     ====================================================================
 #5a. uvcopy expandevs1       <-- same as above, I/O files default as shown
      =================
 #6. cat tmp/expandevs1_data  <-- display output file
     =======================
      # HOME=$HOME HOSTNAME=${HOSTNAME} LOGNAME=$LOGNAME   <-- input
      #-------------------------------------------------
      HOME=/home/uvadm HOSTNAME=uvsoft5 LOGNAME=uvadm      <-- output
      #-------------------------------------------------

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7F4. Demo uvcopy 'evs' instruction to expand $VARIABLES

expandevs1 used for mainframe DB2 conversions to unix

'expandevs1' can do some serious valuable work at unix/linux sites. It has been used to expand $variables in DB2 scripts to load/unload tables at JCL conversion sites. We added code with $Variables to the SYSIN files for the db2 utility & then used 'expandevs1' to copy the file before submitting to the db2 utility.

To understand this explanation, please refer to the samples on the following page:


 #1. parms/db2sysinxx - SYSIN SQL commands to db2 utility
 #2. DB2 unload table step - from unix script (converted from mainframe JCL)
 #3. dsntiaul - script to unload DB2 tables (called by JCL/script)

Why expandevs1 needed for JCL conversions

  1. We needed 'expandevs1' because unix versions of DB2 (or SQL Server) can NOT unload tables to 'Fixed-Field' copybook layouts expected by following steps in the JCL. Unix versions of DB2 or SQL server can unload only to 'Delimited-Formats'. So, we follow the DB2 script with a uvcopy job which uses the copybook, to convert the unloaded delimited format (in $SYSRTMP) to the fixed-field layout expected by following COBOL programs.

  2. DB2 UDB unloads tables in delimited format to a file defined by $SYSREC00. On the mainframe SYSREC00 would define a file in fixed field format expected by some following step in the JCL/script.

  3. We solved the problem by writing unix scripts (named the same as the mainframe utiities (DSNTIAUL,DSNUTILB,etc) in lower case (dsntiaul,dsnutilb,etc). These scripts execute the command line utility (db2 or sqlcmd) with SQL instructions from the SYSIN parm files.

  4. We redefine the output file 'SYSREC00' (fixed-field format on the mainframe) as $SYSRTMP (within & only for the dsntiaul script). 'db2' will then output to $SYSRTMP (in delimited format), which will be converted to fixed-field back in the JCL/script where SYSRTMP defines the delimited format & SYSREC00 defines the fixed-field format --> exportfile SYSREC00 $SYSRTMP <--

  5. uvcopy input 'fili1=SYSRTMP' will be delimited format created by unix DB2. uvcopy output 'filo1=$SYSREC00' wil be fixed field copybook layout, expected by the following step.

  6. This logic required parms with SELECTs to be modified by inserting --> export to $SYSRTMP of del modified by coldel| nochardel <-- This modification was performed by uvcopy job $UV/pf/db2delimfix1 while copying all original parm1/... to parms/...

  7. And since '$SYSRTMP' is not expanded by 'db2 -txf $SYSIN', we changed it to 'db2 -txf $SYSINTMP' & preceded it with 'uvcopy expandevs1', to expand $VARIABLEs ($SYSRTMP) in $SYSIN while copying to $SYSINTMP.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7F5. Demo uvcopy 'evs' instruction to expand $VARIABLES

JCL conversion to unload DB2 tables on unix/linux

Here are the 3 components involved reduced to essential lines only.

#1. parms/db2sysinxx - SQL input to db2 utility

 /* db2sysinxx - 'SYSIN' file to DB2 command line utility 'db2' */
 /*            - to demo JCL conversion to unix/linux scripts   */
 /*            - this demo file stored at $UV/tf/db2sysinxx     */
 export to $SYSRTMP of del modified by coldel| nochardel
 SELECT * FROM tablexx;

#2. DB2 unload table step - from JCL/script


 #1========== begin step#S0040 DSNTIAUL UNLOADSEL ============
 exportfile SYSREC00 data1/fixed.field.datafilexx
 exportfile SYSIN $RUNLIBS/parms/db2sysinxx
 exportfile SYSRTMP $JTMP/${JSTEP}_SYSREC00
 exportfile SYSINTMP $JTMP/db2sysinxx
 #3---------------------------------------------------------------
 dsntiaul <$SYSIN   #<-- call script to execute db2 utility to unload
 uvcopy $RUNLIBS/pfq1/delim2fixedxx,fili1=$SYSRTMP,filo1=$SYSREC00
 #4---------------------------------------------------------------
 # - uvcopy job converts unloaded delimited to fixed format for collowing COBOLs

#3. dsntiaul - unload table script - called by JCL/script

 # dsntiaul - UNLOAD DB tables - SELECT with delimiter via export chardel
 #          - minimal comments - see complete version at $UV/sf/IBM/dsntiaul
 #
 rm -f $SYSRTMP            # remove any existing $SYSRTMP
 touch $SYSRTMP            # recreate empty file
 export SYSREC00=$SYSRTMP  #<-- redef SYSREC00 as $SYSRTMP (only in this script)
 #
 uvcopy expandevs1,fili1=$SYSIN,filo1=$SYSINTMP
 #=============================================
 # - copy $SYSIN to $SYSINTMP, expanding $VARIABLEs ($SYSRTMP, possible others)
 #   since input by 'db2 -txf $SYSIN' does not expand
 #-------------------------------------------------------------------------------
 export DB2CLP=**$$**
 echo "db2 script begin: dsntiaul - UNLOAD SELECT with delimiter via export chardel"
 db2 "CONNECT TO $DBNAME USER $DBUSR USING $DBPWD"
 db2 "SET SCHEMA = $DBUSR"
 db2 -txf $SYSINTMP       #<-- Note $SYSINTMP $variables expanded by expandevs1 above
 db2 "connect reset"

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_8. UVdemos Vancouver Utility Programming Aids

Part 8 - Search/Replace - Contents


8A1. scan1d pre-programmed job to scan all files in a directory
- searching for 1 pattern & creating a report matching lines

8B1. rep1d pre-programmed job to search for 1 pattern & replace with an alternate
- while copying all files from 1 directory to a 2nd (empty) directory

8B2. rep2d pre-programmed job to search for multiple patterns & replace with alternates
- uses a pre-edited table of search/replace patterns
- while copying all files from 1 directory to a 2nd (empty) directory

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

8A1. UVdemos Vancouver Utility Programming Aids

scan1d - scan all files in directory reporting matches to 1 pattern

'scan1d' is a pre-programmed uvcopy job to scan all files in a directory searching for 1 pattern & reporting matches. Of course you would usually do this with 'grep' but you could use scan1d when you want a nicer report or you want to qualify the matches by 1 or 2 other patterns that must be present or absent on the same line.

We will demo scan1d using the FTP demo files in demo/ftps/... All 4 files are

listed on page '1C3', but here is the 1st 1 as a reminder.

      # ftps/ftpdemo1a
      open 192.168.0.4
      user user01 user01pw
      lcd data1
      get /home/userxx/ar/sales.items

 #0a. login userxx --> /home/userxx
 #0b. cd demo      --> /home/usrxx/demo

 #1. uvcopy scan1d,fild1=ftps,arg1=open,filo1=tmp/ftps_open
     ======================================================
      - scan all files in ftps/... reporting matches to 'open'
      - output report as follows:
      Job: scan1d  Dir: ftps  Search: open  Qual1:   Qual2:
       SearchBgn:    SearchEnd:    MatchOps:   UserOps: q1a1b99999c0d256e0p1
      ====================================================== 2019/02/15_11:17:53
      00002 open 192.168.0.4
      00007                   1 hits @EOF: ftps/ftpdemo1a
      00002 open 192.168.0.5
      00008                   1 hits @EOF: ftps/ftpdemo1b
      00002 open 192.168.0.6
      00009                   1 hits @EOF: ftps/ftpget1
      00002 open 192.168.0.6
      00009                   1 hits @EOF: ftps/ftpput1
      ====================================================== 2019/02/15_11:17:53
       EOD: 00004 hits in 0004 files of 0004 total (00033 lines)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

8B1. UVdemos Vancouver Utility Programming Aids

rep1d - search/replace 1 pattern copying all files to 2nd directory

'rep1d' is a pre-programmed uvcopy job to scan all files in a directory searching for 1 pattern & replacing with an alternate. You would usually use 'sed' for this, but rep1d could be used when you need to qualify the replacements by 1 or 2 other patterns that must be present or absent on the same line.


 #1. uvcopy rep1d,fild1=ftps,fild2=tmp1,arg1=192.168.0.6,arg2=\$IP6
                             ,filo1=tmp/ftps_192.168.0.6
     ==============================================================
      - copy files from ftps/* to tmp1/... replacing IP '192.168.0.6' with '$IP6'
      - output dir defaults to tmp1/... & audit file defaults to tmp/${fild1}_$arg1
      - the audit file shows the line after the replacement has been made

output audit file

      Job: rep1d InDir: ftps OutDir: tmp1 Search: 192.168.0.6 Replace: $IP6
      QualPresent:   Absent:   SearchBgn:    SearchEnd:
      MatchOptions:   UserOptions: q1a1b99999c0d256e0p1
      ====================================================== 2019/02/15_11:21:34
      00002 open $IP6
      00009                   1 reps @EOF: tmp1/ftpget1
      00002 open $IP6
      00009                   1 reps @EOF: tmp1/ftpput1
      ====================================================== 2019/02/15_11:21:34
      EOD: 2 reps in 2 files of 4 total (33 lines)
      Audit filename: tmp/ftps_192.168.0

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

8B2. UVdemos Vancouver Utility Programming Aids

rep2d search/replace multiple patterns copying all files to 2nd dir

'rep2d' is a pre-programmed uvcopy job to copy all files in a directory to a 2nd directory, searching/replacing via a table of multiple search/replace patterns, optionally qualified by patterns that must be present or absent on the same line.

You would pre-edit the search/replace table, as shown in the demo below: I suggest you use the demo/ctl/ subdir to store your search/replace tables.


 #1. vi ctl/IPreplace.tbl   <-- edit search/replace table for multiple IP#s
     ====================
      # IPreplace.tbl - table of search/replace IP#s stored in /home/uvadm/demo/ctl/IPreplace.tbl
      #               - to demo uvcopy 'rep2d' search/replace by table of old/new patterns
      #               - by Owen Townsend, UV Software, Feb15/2019
      #               - see www.uvsoftware.ca/uvdemos.htm
      # 01-30=search, 31-60=replace,61-80=qualifier_present,81-100=qualifier_absent
      #
      192.168.0.4~~~~~~~~~~~~~~~~~~~$IP4~~~~~~~~~~~~~~~~~~~~~~~~~~open~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      192.168.0.5~~~~~~~~~~~~~~~~~~~$IP5~~~~~~~~~~~~~~~~~~~~~~~~~~open~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      192.168.0.6~~~~~~~~~~~~~~~~~~~$IP6~~~~~~~~~~~~~~~~~~~~~~~~~~open~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Note
  • the # comment lines will be ignored by the 'rep2d' job.
  • the non-comment search/replace patterns will be shown in the audit file

 #2. uvcopy rep2d,fild1=ftps,fild2=tmp1,fili3=ctl/IPreplace.tbl
     ==========================================================
      - copy all files from ftps/* to tmp1/... searching/replacing via the table
        of multiple patterns stored in ctl/IPreplace.tbl (specify with fili3=...)
      - here is the audit file, showing the search/replace table used & only
        the modified lines after replacements made
      Job: rep2d  InDir: ftps  OutDir: tmp1  Table: ctl/IPreplace.tbl
      SearchBgn:    SearchEnd:    MatchOptns: df2g2  UserOptns: q1a1b99999c0d256e0p1
      ====================================================== 2019/02/15_11:41:25
      192.168.0.4~~~~~~~~~~~~~~~~~~~$IP4~~~~~~~~~~~~~~~~~~~~~~~~~~open~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      192.168.0.5~~~~~~~~~~~~~~~~~~~$IP5~~~~~~~~~~~~~~~~~~~~~~~~~~open~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      192.168.0.6~~~~~~~~~~~~~~~~~~~$IP6~~~~~~~~~~~~~~~~~~~~~~~~~~open~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      ====================================================== 2019/02/15_11:41:25
      00002 open $IP4
      00007                   1 reps @EOF: tmp1/ftpdemo1a
      00002 open $IP5
      00008                   1 reps @EOF: tmp1/ftpdemo1b
      00002 open $IP6
      00009                   1 reps @EOF: tmp1/ftpget1
      00002 open $IP6
      00009                   1 reps @EOF: tmp1/ftpput1
      ====================================================== 2019/02/15_11:41:25
       EOD: 00004 reps in 0004 files of 0004 total (00033 lines)
      Audit filename: tmp/ftps_IPreplace.tbl

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_9. UVdemos - Vancouver Utilities for Unix/Linux

Part 9 Contents


9A1. Introduction & Objectives of Part 9 demos
Part 9 ---> 3 demos: Election1, Election2, Election3
 Election1 - uvcp/uvsort directly in data subdir (not recommended)
 Election2 - Working-Directory & subdirs for data,scripts,uvcopy
 Election3 - using Advanced Features from JCL Conversions in your scripts

9A2. Directories for Election1,2,3 demos

Part 9 Contents (Election1 demos)


9B1. Election1 demos - datafiles directly in working directory
Pre-requisites to run Election1 demos
- modify your profile: export RUNDATA=$HOME/demo/election1

9B2. Login to run Election1 demos

9B3. Inspect Election1/... data file contents

9C1. Run 'uvcp' to load/unload candidates master file to Indexed file

9C2. Run 'uvsort' to load/unload candidates master file to Indexed file
- same effect as 'uvcp' since records already in sequence

9D1. Recommended Command Line Working Habits
Working-Directory Recommendations
- datafiles directly in working directory (Not recommended)
- better to have subdirs for data, scripts, etc (as in Election2)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part 9 Contents (Election2 demos)


9F1. Election2 demos - Working-Directories & subdirs for data,scripts,uvcopys

9F2. Election2 demos - uvcp/uvsort subdirs for data,scripts,uvcopys
Pre-Requisites/Assumptions to run election1 demos
 Objectives of election2 demo/tutorials

9G1. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc
Login to Run Election2 demos
Inspect Election2 Demo Files - with 'llr'

9G2. Inspect Election2 Demo Files - with 'tree' (vs llr above)

9G3. Inspect Election2/data1/... file CONTENTS

9H1. EXECUTE scripts load/unload candidates Master

9H2. EXECUTE uvcopy jobs to update Master file with campaign info

9I1. LISTINGS scripts to SORT/LOAD/UNLOAD candidates Master

9I2. LISTING uvcopy job to convert updates Delimited to Fixed Fields

9I3. LISTING uvcopy jobs to UPDATE candidates Master

9J1. Recommended Command Line Working Habits
Working-Directory Recommendations/Tips

9K1. Working-Directory - Best Practices

9K2. tmp/ subdir in Working-Directory

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part 9 Contents (Election3 demos)


9M1. Election3 demos - using JCL conversion Power Features in your scripts
Pre-Requisites/Assumptions to run Election3 demos
 Objectives of Election3 Demos/Tutrorials

9N1. Benefits of using JCL conversion Features in your scripts
- GDG files, testcc, joblogs, jobmsgs, Restart, Keywords=Values
- autoload functions in separate library
- $RUNLIBS/#RUNDATA allow different test/production scripts & programs
  for different sets of test/rpoduction data
- different sets of testdata for different programmers
- never need to modify scripts for test or production

9N2. Profiles are critical to benefiting from the JCL conversion features
- See stub & common profiles listed on pages '1B2' & '1B3'.
- Programmers can setup their own set of testdatafiles by changing their
  .bash_profile to --> export RUNDATA=$HOME/testdata <--
  and logoff/logon to update the search PATHs when their .bash_profile
  calls the common profile ($APPSADM/env/common_profile_uv).
- must use the Korn shell for scripts that use JCL conversion features
  since ksh has 'FPATH' to define a common directory to store the functions
  supporting JCL conversion features (exportgen,jobset51,jobend51,testcc,etc)
see --> export FPATH=$APPSADM/sfun/ <-- in the common_profile.

9N3. $RUNDATA/$RUNLIBS allow different sets of Libraries & Data
for different programmers or production operators.
Requirements to provide this powerful feature.

9N4. Directories for Election3 demos

9O1. Inspect election3 data FILES (optional separate dir for GDG files)
9O2. Inspect election3 data file CONTENTS

9P1. EXECUTE scripts to Load & update GDG Master candidatesMG_
9P2. EXECUTE uvcopy jobs to update Master file with campaign info
9P3. JOBLOG from upcandimg1 script to update Master file with campaign info

9Q1. Election3 demos - using JCL conversion Features in your scripts
LISTINGS of scripts to sort/load/unload candidates Master
Benefits of JCL Conversion Features
9Q2. JCL example to illustrate Benefits of JCL converion features
Original mainframe JCL 'UPCANDIMG0' BEFORE conversion to script
9Q3. Unix/Linux script 'upcandimg0' AFTER conversion from JCL
9Q4. Edited version 'upcandimg1' with Power Feature lines marked
9Q5. upcandimg2 - alternate version with fewer #comments

9R1. upcandimg3 - 2nd step added to demo RESTART feature
9R2. demo RESTART at any step - showing joblog from restart

9S1. Election3 demos - using JCL conversion Features in your scripts
'loadcandidatesMG1' script BEFORE adding Advanced Features
'loadcandidatesMG2' script AFTER adding Advanced Features

9T1. Benefits of JCL/script conversion power features
9T2. Requirements to use JCL/script conversion power features

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9A1. Introduction & Objectives of Part 9 demos

election1, election2, election3 demo progression

Election1
  • uvcp/uvsort directly in data subdir (not recommended)
Election2
  • Working-Directory & subdirs for data,scripts,uvcopy
Election3
  • using Advanced Features from JCL Conversions in your scripts

The primary objective of Part_9 is to demonstrate the powerful features that were developed for mainframe JCL conversions & how you can include them in your scripts. This is also a great place to start if you are intending to do mainframe JCL conversion & want to understand how the generated scripts provide the advanced features.

We split Part 9 into 3 parts (3 working directories). You will modify your .bash_profile to define RUNDATA & RUNLIBS to $HOME/demo/election1,2,or 3 to perform the demos progressively from simple to most advanced. You logoff & back on to make changes to RUNDATA & RUNLIBS effective. This is not too important for the election1 & election2 demos, but is vital for the election3 demos.

election1, election2, election3 demo progression

election1
  • simplest setup to test Vancouver Utilities 'uvcp' & 'uvsort' ***
  • datafiles directly within the election1 working-directory
  • no subdirs, no scripts, not recommended
election2
  • illustrates recommended command line working habits
  • all files in subdirs (data1, sf (scripts), pf (uvcopy jobs)
  • no files in the working-direcctory
  • tmp/ dir for sort merge files & any misc temp outputs
election3
  • illustrates the advanced features used in mainframe JCL conversions
    that you can add to your scripts (at non conversion sites).

Part 9 will also demonstrate good command line working habits, Best-Practices working-directory standards - all files in subdirs, using a tmp subdir for miscellaneous temp outputs, etc.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9A2. Directories for election1,2,3 demos

 /home/userxx/demo/
 |   election1                     <-- election1 demos 9A1 - 9D3
 |   |-- 1README_election1           - no subdirs, no scripts, not recommended
 |   |-- candidatesM
 |   |-- candidatesMI.dat
 |   |-- candidatesMI.idx
 |   |-- candidatesMun
 |-- election2                     <-- election2 demos 9F1 - 9K2
 |   |-- 1README_election2           - subdirs for data, scripts,& uvcopy jobs
 |   |-- data1
 |   |   |-- candidatesM
 |   |   |-- candidatesMI.dat
 |   |   |-- candidatesMI.idx
 |   |   |-- candidatesMun
 |   |   |-- candiupsD20200229
 |   |   `-- candiupsF20200229
 |   |-- pf
 |   |   |-- fixcandiupsD2F
 |   |   `-- upcandidatesMI
 |   |-- sf
 |   |   |-- loadcandidatesMI
 |   |   `-- unloadcandidatesMI
 |   `-- tmp
 |-- election3                     <-- election3 demos 9M1 - 9T2
 |   |-- 1README_election3           - advanced features from JCL conversions
 |   |-- ctl                         - GDG files, joblogs,jobmsgs,jobtimes
 |   |   |-- gdgctl51                - demo Restart, $RUNDATA/$RUNLIBS, etc
 |   |-- data1
 |   |   |-- candidatesM
 |   |   |-- candiupsD20200229
 |   |   `-- candiupsF20200229
 |   |-- datag1
 |   |   |-- candidatesMG_000001
 |   |   `-- candidatesMG_000002
 |   |-- jcls
 |   |   |-- loadcandidatesMG2
 |   |   |-- upcandimg0
 |   |   `-- upcandimg1
 |   |-- joblog
 |   |   `-- upcandimg1.log
 |   |-- jobmsgs
 |   |   `-- 200315
 |   |       `-- 180004_upcandimg1
 |   |-- jobtimes
 |   |   |-- _UPCANDIMG1_jobtimes
 |   |-- jobtmp
 |   |   `-- upcandimg1
 |   `       |-- GDG
 |   |-- pf
 |   |   |-- fixcandiupsD2F
 |   |   `-- upcandidatesMG
 |   |-- sf
 |   |   `-- loadcandidatesMG1
 |   `-- tmp

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9B1. Election1 demos - datafiles directly in working directory

Pre-Requisites/Assumptions to run election1 demos

  1. Installed Vancouver Utilities as per uvsoftware.ca/install.htm

  2. You have a user logon or setup as per uvsoftware.ca/uvdemos.htm#1C1 (we will call it 'userxx')

  3. You have setup subdir demo & copied $UV/demo/* to your $HOME/demo/

  4. You have copied/renamed $APPSADM/env/bash_profile_uv to your $HOME/.bash_profile

  5. Now modify your $HOME/.bash_profile, to run the election1 demo/tutorials:

    export RUNLIBS=$HOME/demo RUNDATA=$HOME/demo    <-- change this
    ============================================
    export RUNLIBS=$HOME/demo/election1 RUNDATA=$HOME/demo/election1  <-- to this
    ================================================================
  1. Log off & back on to make the new RUNLIBS/RUNDATA definitions effective
Note
  • Above 5 & 6 not essential to run election1 demos, but will be
    critical when we get to the election3 demos.
  • Only required for aliases cdd/cdl to work for election1.

Objectives of election1 demo/tutorials

The election1 demos were inspired by a client who wanted to use the Vancouver Utilities to load/unload Indexed files compatible with Micro Focus COBOL. uvcopy,uvsort,uvcp,etc use the D-ISAM file handler (from www.isamcentral.com) which is compatible with Micro Focus COBOL & C-ISAM.

AS well as demonstrate using the utilities to load/unload Indexed files, these election1 demos illustrate command line working habits (good & bad).

See the summary at the end of these election1 demos to see the bad habits that will be improved in the election2 demos.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9B2. Election1 demos - datafiles directly in working directory

login to Run election1 demos


 #1.  Logon userxx --> /home/userxx
      ============
 #2.  cdd  (alias 'cd $RUNDATA) --> /home/userxx/demo/election1/
      ===
 OR-> cdl  (alias 'cd $RUNLIBS) --> /home/userxx/demo/election1/
      ===    - same effect since RUNLIBS/RUNDATA defined the same

 #3.  l    <-- alias='ls -l' list long (alias saves keystrokes)
      ===    - all dirs/files in current directory (election1)
 election1/
      -rw-rw-r-- 1 userxx apps 1186 Feb 24 13:28 1README
      -rw-rw-r-- 1 userxx apps  754 Feb 21 11:32 candidatesM
      -rw-rw-r-- 1 userxx apps  891 Feb 23 18:06 candidatesMI.dat
      -rw-rw-r-- 1 userxx apps 3072 Feb 23 18:06 candidatesMI.idx
      -rw-rw-r-- 1 userxx apps  754 Feb 24 12:17 candidatesMun

 #4. tree .  <-- alternate way to list all files (Recursively)
     ======      (gives us space on the right to describe files)
      election1
      |-- candidatesM       <-- Master file (before loading to Indexed file)
      |-- candidatesMI.dat  <-- data partition of Master Indexed file
      |-- candidatesMI.idx  <-- index partition of Master Indexed file
      |-- candidatesMun     <-- Master file UNloaded from Indexed file

Notes

  1. 'candidatesM' is the only file you need to run the demos, since it will be used to create the Indexed file (2 parts .dat & .idx) which is then unloaded to create 'candidatesMun'.

  2. The next page will display the data file contents using 'cat'. We prefer the unix 'cat' command to display short files vs more or less which prompt for browsing at EOF (vs just quitting).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9B3. Election1 demos - datafiles directly in working directory

INSPECT election1/... data file contents


 #0a. Logon userxx --> /home/userxx
 #0b. cdd  (alias 'cd $RUNDATA) --> /home/userxx/demo/election1/

 #0c. l data1/   <-- list all data filenames in data1/...
      ========     - before listing each data file content
      -rw-rw-r-- 1 userxx apps  754 Feb 21 11:32 candidatesM
      -rw-rw-r-- 1 userxx apps  891 Feb 23 18:06 candidatesMI.dat
      -rw-rw-r-- 1 userxx apps 3072 Feb 23 18:06 candidatesMI.idx
      -rw-rw-r-- 1 userxx apps  754 Feb 24 12:17 candidatesMun

 #1. cat data1/candidatesM   <-- list candidates master file
     =====================     - sequential before loading to Indexed file
      #1 candidatesM - master file 2020 election (uvsoftware.ca/uvdemos.htm#Part_9)
      #2 Name   Age  City     State  Month   Poll% Delegs Month$ Total$ (millions)
      #3       1         2         3         4         5         6         7         8
      #4345678901234567890123456789012345678901234567890123456789012345678901234567890
      Biden      77  Wilmington  DE  20200131   27      0      0     61
      Bloomberg  77  New York    NY  20200131    7      0      0    200
      Buttigieg  38  South Bend  IN  20200131    8      0      0     76
      Klobuchar  59  Minneapolis MN  20200131    4      0      0     29
      Sanders    78  Burlington  VT  20200131   22      0      0    108
      Trump      74  New York    NY  20200131   44  2,472      0    150
      Warren     70  Cambridge   MA  20200131   15      0      0     82

 #2a. cat data1/candidatesMI.dat  <-- list candidates master Indexed data
      ==========================    - loaded from sequential file
       - will be same as #1. data1/candidatesM shown above

 #2b. cat data1/candidatesMI.idx  <-- list candidates master Index (vs .dat)
      ==========================    - could try but would get garbage display
      uvhd data1/candidatesMI.idx  <-- use 'uvhd' to see hex display of binary file
      ==========================

 #3. cat data1/candidatesMun   <-- list candidates Master file
     =======================     - unloaded from Indexed file
      - will be same as #1. data1/candidatesM shown above

We have shown you the original data file contents from UV Software, and you should get the same results after you run the load/unloads on the next page. Before we run the loads/unloads we will remove all files except 'candidadtesM' so you can know your loads/unloads were successful. But 1st you might save in subdir archive/ for later restore if needed.


 #4a. mkdir archive     <-- make subdir to save files before recreate
      =============
 #4b. cp * archive      <-- copy original demo files to archive/...
      ============

 #5. rm candidatesMI.dat candidatesMI.idx candidatesMun
     ==================================================
     - remove all files except original candidatesM Master file

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9C1. Election1 demos - datafiles directly in working directory

EXECUTE uvcp to load/unload candidates Master file


 #0a. Logon userxx --> /home/userxx
 #0b. cdd  (alias 'cd $RUNDATA) --> /home/userxx/demo/election1/

The demo Op. Instrns. below do not show the data file contents, but you can run cat/more/vi/etc after each command to confirm you get same results as shown above.


 #1. uvcp "fili1=candidatesM,rcs=80,typ=LST,filo1=candidatesMI,rcs=80,typ=ISF,isk1=0(10)"
     ====================================================================================
      - load an Indexed file (2 parts, condidatesMI.dat & candidatesMI.idx)
      --> displays log as shown below
      200225:130834:uvcp: uvcp fili1=data1/candidatesM,filo1=data1/candidatesMI
      200225:130834:uvcp: EOF fili01 11 rds, 755 size; data1/candidatesM
      200225:130834:uvcp: EOF filo01 11 wrts, 0 size; data1/candidatesMI
      200225:130834:uvcp: DISAM.dat: recs=11,rcsz=80,keys=1; data1/candidatesMI
      200225:130834:uvcp: KEY1: start=0,length=10

 #2. uvcp "fili1=candidatesMI,typ=ISF,rcs=80,filo1=candidatesMun,typ=LST,rcs=80"
     ===========================================================================
      - unloads data1/condidatesMI.dat/idx to data1/candidatesMun
      --> displays log as shown below
      200225:131528:uvcp: uvcp fili1=data1/candidatesMI,filo1=data1/candidatesMun
      200225:131528:uvcp: EOF fili01 11 rds, 891 size; data1/candidatesMI
      200225:131528:uvcp: DISAM.dat: recs=11,rcsz=80,keys=1; data1/candidatesMI
      200225:131528:uvcp: EOF filo01 11 wrts, 755 size; data1/candidatesMun

 #3. diff -b candidatesM candidatesMun
     =================================
      - diff original file (before loading Indexed) to the unloaded file
      - no output shown if same (as expected)
      - option '-b' ignores any trailing blanks

 #4. diff candidatesM candidatesMI.dat     <-- run diff without '-b' option
     =================================
      - diff original file to the data partition of the Indexed file without '-b' option
      - will show diffs because Indexed .dat records are blank filled to rcs=80 specified
      - here is just 1st record in each file
      < #1 candidatesM - master file 2020 election (uvsoftware.ca/uvdemos.htm#Part_9)
      ---
      > #1 candidatesM - master file 2020 election (uvsoftware.ca/uvdemos.htm#Part_9)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9C2. Election1 demos - datafiles directly in working directory

investigate results loading Indexed file


 #5. diff -b candidatesM candidatesMI.dat  <-- repeat diff WITH '-b' option
     ====================================
      - will show no output since '-b' ignores trailing blanks

 #6. uvhd candidatesMI.dat   <-- can use 'uvhd' to see trailing blanks
     =====================     - output as follows (1st record only)
      rec#=1 rsize=81 fptr=0 fsize=891 records=11
                    10        20        30        40        50        60        70        80
      r# 1 012345678901234567890123456789012345678901234567890123456789012345678901234567890
           #1 candidatesM - master file 2020 election (uvsoftware.ca/uvdemos.htm#Part_9)   .
           232666666676742226677672666623333266667666227776677676266277666672676256775322220
           31031E4941453D0D0D13452069C50202005C5349FE08563F647125E31F5645DF3E84D30124F99000A
      Now you can see the 3 trailing blanks prior to the x'0A' LineFeed that terminates records
      The LineFeed is in byte 81 after 80 bytes of data & will be changed to x'00' if deleted
      by the Indexed file handler.

'uvsort' Alternative to 'uvcp' (#1 above) to load Indexed file


 #7. uvsort "fili1=candidatesM,rcs=80,typ=LST,filo1=candidatesMI,rcs=80,typ=ISF,isk1=0(10),key1=0(10)"
     =================================================================================================
      - NOT required since candidatesM file was already in sequence on keyfield 1st 10 bytes
      - only difference is adding the sort 'key1=0(10)' (& changing 'uvcp' to 'uvsort')

separate 'uvsort' + 'uvcp' to load Indexed file


 #8a. uvsort "fili1=candidatesM,rcs=80,typ=LST,filo1=candidatesM,rcs=80,typ=LST,key1=0(10)"
      =====================================================================================
      - could 1st sort only to a sequential file, Before loading the Indexed file
      - OK to sort back to the same filename (data1/candidatesM), since uvsort allows.

 #8b. uvcp "fili1=candidatesM,rcs=80,typ=LST,filo1=candidatesMI,rcs=80,typ=ISF,isk1=0(10)"
      ====================================================================================
      - now load the Indexed file from the sorted file

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9D1. Election1 demos - datafiles directly in working directory

Recommended Command Line Working Habits

AS well as demonstrating using the utilities to load/unload Indexed files, these election1 demos illustrate command line working habits (good & bad). Here are several habits that will be improved in the election2 demos.

Working-Directory Recommendations/Tips

  1. We recommend defining your working directory for your current project using 'RUNLIBS' & 'RUNDATA' in your .bash_profile (listed on page '1B2'). For small projects, RUNLIBS & RUNDATA can define the same directory, but for large projects, they should be different & in large file systems rather than in /home/... Here is line# 38 in our .bash_profile.

    export RUNLIBS=$HOME/demo/election1 RUNDATA=$HOME/demo/election1
    ================================================================
  1. Defining RUNLIBS/RUNDATA in your .bash_profile has several advantages which will be illustrated later in election2 & election3. For example you can use the 'cdl' (or 'cdd') to change to your libs/data directory with 3 keystrokes from anywhere else you might be.

  2. The election1 demos stored the data files directly in the working directory, and did NOT use sub-directories to store different types of files which we strongly recommend.

  3. It might be OK to have files directly in the working directory for very small projects with short life-span, but usually better to take time to setup different subdirs for different file types that may be added to the project.

  4. For example, the next project (election2) will have a subdir data1/... for data files + sf/... for Script Files + pf/... for Parameter Files + tmp/... for temporary files.

  5. We will use 'data1' (not just 'data'), better to assign unique names to directories so your can search & find all instances in multiple files. Searching for just 'data' would get a lot of matches you don't want. So the data files in election2 will be:
      election2
      |-- data1/
      |   |-- candidatesM
      |   |-- candidatesMI.dat
      |   |-- candidatesMI.idx
      |   |-- candidatesMun

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9D2. Election1 demos - datafiles directly in working directory

  1. You should stay in the working directory, do not change into the subdirs, address your files thru 1 level of subdir. Don't waste time by changing into a subdir, looking at 1 file,& changing back out. Of course OK, if you intend to inspect/edit multiple files.

  2. The election1 demos executed the utilities (uvcp,uvsort) directly, which resulted in long command lines, for example:

     uvcp "fili1=candidatesM,rcs=80,typ=LST,filo1=candidatesMI,rcs=80,typ=ISF,isk1=0(10)"
     ====================================================================================
  1. Better to create scripts that call the utilities with desired parameters, especially if you may need to repeat the commands multiple times in future. Election2 will create the 'loadcandidatesM' script as follows:
      #!/bin/bash
      # loadcandidatesM - script calling uvcp to load indexed file
      #  - demo by Owen Townsend, UVsoftware, Feb/2020
      #
      uvcp "fili1=data1/candidatesM,rcs=80,typ=LST,filo1=data1/candidatesMI,rcs=80,typ=ISFo4,isk1=0(10)"
      #=================================================================================================
      exit
      #Note: 'filo1=data1/candidatesMI,typ=ISF' & .dat/.idx extensions on Indexed files
      # - Indexed file handler will output 2 files candidatesM.dat & candidatesM.idx
  1. Note that we inserted 'data1/' on the file definitions, since we know we will be in the working directory & need to address files thru 1 level of subdir of subdir Later in election3, we will show you how this can work no matter where you are.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9D3. Election1 demos - datafiles directly in working directory

  1. Election2 will of course setup a separate directory for the project's Script Files (called 'sf' for brevity). There will also be a 'pf' directory for uvcopy's Parameter Files (called 'pf'). So now the directory tree for election2 will be:
      election2
      |-- backup/
      |-- data1/                   <-- data files
      |   |-- candidatesM
      |   |-- candidatesMI.dat
      |   |-- candidatesMI.idx
      |   |-- candidatesMun
      |   |-- candidateupsD
      |   |-- candidateupsF
      |-- pf/                       <-- Parameter Files
      |   |-- fixcandidateupsD2F
      |   |-- upcandidatesMI
      |-- sf/                       <-- Script Files
      |   |-- loadcandidatesM*
      |   |-- sortcandidatesM*
      |   |-- sortloadcandidatesM*
      |   |-- unloadcandidatesMI*
      |-- tmp/
      Note - 'tree' has option -F to append '/' on directories & '*' on excutable files
  1. To run the 'loadcandidatesM' script, from the working directory, you could enter:

      sf/loadcandidatesM
      ==================
  1. But, you do not need the 'sf/' prefix, since the supplied profiles include '$RUNLIBS/sf' in $PATH. (profiles on pages '1B2' & '1B3').

      loadcandidatesM
      ===============

For the election2 demos, you do need to be in the working directory, since the script addresses the data files relative to the working directory, but later in election3, we will show you how this can work no matter where you are.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9F1. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

Pre-Requisites/Assumptions to run election2 demos

  1. Installed Vancouver Utitiliites as per uvsoftware.ca/install.htm
  2. You have a user logon or setup as per uvsoftware.ca/uvdemos.htm#1C1 (we will call it 'userxx')
  3. You have setup subdir demo & copied $UV/demo/* to your $HOME/demo/
  4. You have copied/renamed $APPSADM/env/bash_profile_uv to your $HOME/.bash_profile
Note
  • 1,2,3,4 already performed on for Election1 demos on page '9B1'
  1. Now modify your $HOME/.bash_profile, to run the election2 demo/tutorials:

    export RUNLIBS=$HOME/demo RUNDATA=$HOME/demo    <-- change this
    ============================================
    export RUNLIBS=$HOME/demo/election1 RUNDATA=$HOME/demo/election1 <-- OR this
    ================================================================

    export RUNLIBS=$HOME/demo/election2 RUNDATA=$HOME/demo/election2  <-- to this
    ================================================================
  1. Log off & back on to make the new RUNLIBS/RUNDATA definitions effective

    Objectives of election2 demo/tutorials

There are 3 levels of these demo/tutorials (election1,election2,election3) in separate directories $HOME/demo/election1,2,3 or $RUNDATA/demo/election1,2,3

Election1 demos were very simple - just 4 data files in the working directory with no subdirs. We entered uvcp/uvsort commands to load/unload Indexed files.

Election2 demos illustrate better working habits - setting up separate subdirs for various file types: ./data1/... for DataFiles, ./sf/... for ScriptFiles, ./pf/... for ParameterFiles,& ./tmp/... for temporary files.

Note
  • See the "Recommended Command Line Working Habits" at the end of these
    election2 demo/tutorials.

Election3 demos will illustrate how any site can use the many powerful features used in the JCL/scripts at mainframe migration sites.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9G1. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

Login to Run election2 demos


 #1.  Logon userxx --> /home/userxx
      ============
 #2.  cdd  (alias 'cd $RUNDATA) --> /home/userxx/demo/election2/
      ===
 OR-> cdl  (alias 'cd $RUNLIBS) --> /home/userxx/demo/election2/
      ===    - same effect since RUNLIBS/RUNDATA defined the same

Inspect Demo Files - with 'llr'


 #3.  llr .  <-- llr (alias ls -lR) list Recursively
      =====    - all dirs/files in election2/...
 election2/
      -rw-rw-r-- 1 userxx apps 1186 Feb 24 13:28 1README
      drwxrwxr-x 3 userxx apps 4096 Feb 24 15:28 backup
      drwxrwxr-x 2 userxx apps 4096 Feb 24 11:56 data1
      drwxrwxr-x 2 userxx apps 4096 Feb 24 12:19 pf
      drwxrwxr-x 2 userxx apps 4096 Feb 24 12:18 sf
      drwxrwxr-x 2 userxx apps 4096 Feb 24 15:29 tmp
 election2/backup:
      drwxrwxr-x 2 userxx apps 4096 Feb 24 15:28 data1_Jan2020
       - backup/data1_Jan2020 files same as data1/. files below
 election2/data1:
      -rw-rw-r-- 1 userxx apps  754 Feb 21 11:32 candidatesM
      -rw-rw-r-- 1 userxx apps  891 Feb 23 18:06 candidatesMI.dat
      -rw-rw-r-- 1 userxx apps 3072 Feb 23 18:06 candidatesMI.idx
      -rw-rw-r-- 1 userxx apps  754 Feb 24 12:17 candidatesMun
      -rw-rw-r-- 1 userxx apps  322 Feb 21 11:32 candiupsD20200229
      -rw-rw-r-- 1 userxx apps  385 Feb 21 13:03 candiupsF20200229
 election2/pf:
      -rw-rw-r-- 1 userxx apps 1851 Feb 24 11:48 fixcandiupsD2F
      -rw-rw-r-- 1 userxx apps 2320 Feb 23 18:06 upcandidatesMI
 election2/sf:
      -rwxrwxr-x 1 userxx apps 1517 Feb 24 12:05 loadcandidatesM
      -rwxrwxr-x 1 userxx apps  960 Feb 24 12:09 sortcandidatesM
      -rwxrwxr-x 1 userxx apps 1181 Feb 24 12:16 sortloadcandidatesM
      -rwxrwxr-x 1 userxx apps 1149 Feb 24 12:17 unloadcandidatesM

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9G2. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

Inspect Demo Files - with 'tree' (vs llr above)

The 'tree' utility is a good way to understand the directory structure 'tree' is open software created by Steve Baker (ice@mama.indstate.edu) You can install with apt-get, yum, etc, But it is included with the Vancouver Utilities in $UV/bin/tree or if not in $UV/bin, copy from $UV/lib/tree to $UV/bin/tree

 /home/userxx/demo/election2/
 |-- 1README_election2
 |-- backup/                          <-- directory for backups
 |   |-- data1_Jan2020/               <-- subdirs with date stamps
 |       |-- candidatesM
 |       |-- candidatesMI.dat
 |       |-- candidatesMI.idx
 |       |-- candidatesMun
 |       |-- candidateupsD
 |       |-- candidateupsF
 |   |
 |-- data1/                           <-- data files (except for GDGs)
 |   |-- candidatesM                    - candidates Sequential Master file
 |   |-- candidatesMI.dat               - candidates Indexed Master file (Data)
 |   |-- candidatesMI.idx               - candidates Indexed Master file (Index)
 |   |-- candidatesMun                  - candidates Master UNloaded to Seqntl
 |   |-- candiupsD20200229              - campaign updates pipe|Delimited format
 |   `-- candiupsF20200229              - campaign updates Fixed field format
 |-- pf/                             <-- uvcopy jobs (Parameter Files)
 |   |-- fixcandiupsD2F                - convert pipe|delimited to fixed fields
 |   |-- upcandidatesMI                - update caandidates Indexed Master file
 |   `-- upcandidatesMG                - update candidates GDG Master file (alternative)
 |   |
 |-- sf/                             <-- bash shell scripts
 |   |-- loadcandidatesMI              - load candidates Master Sequential file
 |   |-- loadcandidatesMG1             - load candidates Master Indexed file
 |   |-- sortcandidatesM               - sort candidates Master Sequential file
 |   |-- sortloadcandidatesMI          - load & load candidates Master Sequential file
 |   `-- unloadcandidatesMI            - unload candidates Master Indexed file
 `-- tmp/

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9G3. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

Inspect election2/data1/... file CONTENTS


 #0a. Logon userxx --> /home/userxx
 #0b. cdd  (alias 'cd $RUNDATA) --> /home/userxx/demo/election2/

 #0c. l data1/   <-- list all data filenames in data1/...
      ========     - bfore listing each data file content
      -rw-rw-r-- 1 userxx apps  754 Feb 21 11:32 candidatesM
      -rw-rw-r-- 1 userxx apps  891 Feb 23 18:06 candidatesMI.dat
      -rw-rw-r-- 1 userxx apps 3072 Feb 23 18:06 candidatesMI.idx
      -rw-rw-r-- 1 userxx apps  754 Feb 24 12:17 candidatesMun
      -rw-rw-r-- 1 userxx apps  322 Feb 21 11:32 candiupsD20200229
      -rw-rw-r-- 1 userxx apps  385 Feb 21 13:03 candiupsF20200229

 #1. cat data1/candidatesM   <-- display candidates master file
     =====================     - sequential before loading to Indexed file
      #1 candidatesM - master file 2020 election (uvsoftware.ca/uvdemos.htm#Part_9)
      #2 Name   Age  City     State  Month    Poll% Delegs Month$ Total$ (millions)
      #3       1         2         3         4         5         6         7         8
      #4345678901234567890123456789012345678901234567890123456789012345678901234567890
      Biden      77  Wilmington  DE  20200131   27      0      0     61
      Bloomberg  77  New York    NY  20200131    7      0      0    200
      Buttigieg  38  South Bend  IN  20200131    8      0      0     76
      Klobuchar  59  Minneapolis MN  20200131    4      0      0     29
      Sanders    78  Burlington  VT  20200131   22      0      0    108
      Trump      74  New York    NY  20200131   44  2,472      0    150
      Warren     70  Cambridge   MA  20200131   15      0      0     82

 #2a. cat data1/candidatesMI.dat  <-- display candidates master Indexed data
      ==========================    - loaded from sequential file
      - will be same as #1. data1/candidatesM shown above

 #2b. cat data1/candidatesMI.idx   <-- display candidates master Index (vs .dat)
      ==========================     - could try but would get garbage display
      uvhd data1/candidatesMI.idx  <-- use 'uvhd' to see hex display of binary file
      ===========================

 #3. cat data1/candidatesMun   <-- display candidates Master file
     =======================     - unloaded from Indexed file
     - will be same as #1. data1/candidatesM shown above

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9G4. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

Inspect election2/data1/files (continued)


 #4. cat data1/candiupsD20200229   <-- display candidates update records
     ===========================     - pipe delimited fields from spreadsheet
      #1 candiupsD20200229 - update file 2020 election (uvsoftware.ca/uvdemos.htm#Part_9)
      #2 Name | month | poll% | Delegs | month$  <-- pipe delimited from spreadsheet
      Biden|20200229|23|6|9|
      Bloomberg|20200229|10|0|264|
      Buttigieg|20200229|9|22|6|
      Klobuchar|20200229|5|7|6|
      Sanders|20200229|25|21|25|
      Trump|20200229|44|2,472|15|
      Warren|20200229|14|8|11|

 #5. cat data1/candiupsF20200229   <-- display candidates update records
     ===========================
     - Fixed field records extracted from Delimited by uvcopy job pf/fixcandiupsD2F
      Biden                          20200229  23      6      9
      Bloomberg                      20200229  10      0    264
      Buttigieg                      20200229   9     22      6
      Klobuchar                      20200229   5      7      6
      Sanders                        20200229  25     21     25
      Trump                          20200229  44   2472     15
      Warren                         20200229  14      8     11

 #6. cat data1/candidatesMI.dat   <-- display candidates master file AFTER update
     ==========================       with current months donations
      #1 candidatesM - master file 2020 election (uvsoftware.ca/uvdemos.htm#Part_9)
      #2 Name   Age  City     State  Month   Poll% Delegs Month$ Total$ (millions)
      #3       1         2         3         4         5         6         7         8
      #4345678901234567890123456789012345678901234567890123456789012345678901234567890
      Biden      77  Wilmington  DE  20200229   23      6      9     70
      Bloomberg  77  New York    NY  20200229   10      0    264    464
      Buttigieg  38  South Bend  IN  20200229    9     22      6     82
      Klobuchar  59  Minneapolis MN  20200229    5      7      6     35
      Sanders    78  Burlington  VT  20200229   25     21     25    133
      Trump      74  New York    NY  20200229   44   2472     15    165
      Warren     70  Cambridge   MA  20200229   14      8     11     93

Notes

  1. We have shown you the orignal data file contents from UV Software, and you should get the same results after you run the scripts below.

  2. The demo Op. Instrns. below do not show the data file contents, but you can run 'cat/more/less' commands after each script to confirm you get same results as shown above.

  3. We prefer the unix 'cat' command to display short files vs more or less which prompt for browsing at EOF (vs just quitting).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9H1. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

EXECUTE scripts load/unload candidates Master Jan2020

Note
  • We are showing Operating Instructions & console logs BEFORE we show
    the coding of the scripts. See code LISTINGS 2 pages ahead --->

 #0a. Logon userxx --> /home/userxx
 #0b. cdl  (alias 'cd $RUNLIBS) --> /home/userxx/demo/election2/

 #0c. vi sf/*    <-- inspect scripts to load/unload Master file (optional)
     =======      - see Script Files listed further below

 #1. loadcandidatesM   <-- load data1/candidatesM to Indexed file
     ===============    --> data1/condidatesMI.dat & data1/candidatesMI.idx
                        --> displays log as shown below
      200225:130834:uvcp: uvcp fili1=data1/candidatesM,filo1=data1/candidatesMI
      200225:130834:uvcp: EOF fili01 11 rds, 755 size; data1/candidatesM
      200225:130834:uvcp: EOF filo01 11 wrts, 0 size; data1/candidatesMI
      200225:130834:uvcp: DISAM.dat: recs=11,rcsz=80,keys=1; data1/candidatesMI
      200225:130834:uvcp: KEY1: start=0,length=10

      sf/loadcandidatesM  <-- Alternate, same as #1 above
      ==================
      - do not need 'sf/...' since :$RUNLIBS/sf: is in $PATH
        defined in your profile $HOME/.bash_profile

 #2. sortloadcandidatesMI  <-- SORT & LOAD Indexed file
      ===================    - alternate to #1. loadcandidatesM
      - NOT required since candidatesM file is already in sequence
        on the keyfield 1st 10 bytes (candidate last name)

 #3. unloadcandidatesMI <-- unload Indexed file to sequential file
     ==================
      - unloads data1/condidatesMI.dat/idx to data1/candidatesM
      --> displays log as shown below
      200225:131528:uvcp: uvcp fili1=data1/candidatesMI,filo1=data1/candidatesMun
      200225:131528:uvcp: EOF fili01 11 rds, 891 size; data1/candidatesMI
      200225:131528:uvcp: DISAM.dat: recs=11,rcsz=80,keys=1; data1/candidatesMI
      200225:131528:uvcp: EOF filo01 11 wrts, 755 size; data1/candidatesMun

 #4. sortcandidatesM    <-- SORT data1/candidatesM back to same filename
     ===============      - uvsort allows this

 #5. loadcandidatesMI   <-- LOAD sorted file to data1/candidatesMI.dat & .idx
     ================
Note
  • #4 & #5 NOT required since #2 sortloadcandidatesMI does same in 1 script

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9H2. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

EXECUTE uvcopy jobs to update Master file with campaign info

Note
  • We are showing Operating Instructions & console logs BEFORE we show
    the code LISTINGS of the uvcopy jobs. See those 2 pages ahead --->

 #0a. Logon userxx --> /home/userxx
 #0b. cdl  (alias 'cd $RUNLIBS) --> /home/userxx/demo/election2/

 #0c. vi pf/*    <-- inspect uvcopy jobs (Parameter Files) to update Master file
      =======      - see uvcopy Parameter Files listed further below

 #1. Convert Monthly campaign results from Delimited to Fixed field format
     - prior to Master file update job

 #1a. uvcopy fixcandiupsD2F,fili1=data1/candiupsD20200229,filo1=data1/candiupsF20200229
      =================================================================================

 #1b. uvcopy fixcandiupsD2F  <-- same/easier, filenames default as shown above
      =====================   --> displays log as shown below
       fixcandiupsD2F - convert pipe Delimited to Fixed field format
       200225:132300:fixcandiup: uvcopy ver=20190714 pf=./pf/fixcandiupsD2F
       uvcopy LNX H64 license=20190714_99V_930630 site=UV_Software
       200225:132301:fixcandiup: EOF fili01 rds=9 size=323: data1/candiupsD20200229
       200225:132301:fixcandiup: EOF filo01 wrts=7 size=385: data1/candiupsF20200229

 #1c. cat data1/candiupsD20200229  <-- could display INPUT Delimited file
      ===========================    - already shown above

 #1d. cat data1/candiupsF20200229  <-- could display OUTPUT Fixed field file
      ===========================    - already shown above

 #2. Update Master Indexed file with monthly campaign info (poll%,delegates,donations)

 #2a. uvcopy upcandidatesMI,fili1=data1/candiupsF20200229,filr2=data1/candidatesMI
      ============================================================================

 #2b. uvcopy upcandidatesMI   <-- same/easier, filenames default as shown above
      =====================    --> displays log as shown below
       updtcandidatesMI - update candidates master file with current month campaign info
       200225:132407:updtcandidates: uvcopy ver=20190714 pf=./pf/updtcandidatesMI
       uvcopy LNX H64 license=20190714_99V_930630 site=UV_Software
       200225:132407:updtcandidates: EOF fili01 rds=7 size=385: data1/candiupsF20200229
       200225:132407:updtcandidates: EOF filr02 rds=7 upds=7 size=891: data1/candidatesMI
       200225:132407:updtcandidates: DISAM.dat: recs=11,rcsz=80,keys=1; data1/candidatesMI

 #2c. cat data1/candidatesM  <-- could display Master file Before load Indexed
      =====================      & Before updating candidatesMI.dat
                                - already shown above

 #2d. cat data1/candidatesM.dat  <-- could display Master file AFTER updating
      =========================    - with current month's campaign info
                                    - already shown above

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9I1. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

LISTINGS of scripts to sort/load/unload candidates Master

Note
  • Here are the code listings for the scripts, which we have already run.
    <--- See Execute Instructions & console logs 2 pages back.

Script Files are stored in $RUNLIBS/sf/... subdir. RUNLIBS is defined by your .bash_profile & the result is appended to your $PATH by the common_profile. You can inspect the scripts as follows:


 #0a. Logon userxx --> /home/userxx
 #0b. cdl  (alias 'cd $RUNLIBS) --> /home/userxx/demo/election2/

 #1. cat sf/sortloadcandidatesM
     ==========================
 #!/bin/bash
 # sortloadcandidatesMI - script calling uvsort to sort & load indexed file
 #  - demo by Owen Townsend, UVsoftware, Feb/2020
 #  - also see script unloadcandidatesMI to unload Indexed file to sequential file
 #  - alternative to this sortloadcandidates would be sortcandidatesM + loadcandidatesM
 #    would make no diff for the candidates demo file (already in sequence on key 1st 10 bytes)
 #  - uvsort,uvcopy,uvcp use D-ISAM Indexed file handler from Byte Designs www.isamcentral.com
 #  - compatible with C-ISAM & Micro Focus COBOL IDXFORMAT1 (fixed record lengths)
 #
 uvsort "fili1=data1/candidatesM,rcs=80,typ=LST,filo1=data1/candidatesMI,rcs=80,typ=ISF,isk1=0(10),key1=0(10)"
 #============================================================================================================
 exit
 #Note: 'filo1=data1/candidatesMI,typ=ISF' & .dat/.idx extensions on Indexed files
 #     - Indexed file handler in uvsort,uvcopy,uvcp,etc will output 2 files __________.dat & __________.idx
 #     - Input & Output filenames could be coded the same since D-ISAM appends .dat & .idx on output filenames
 #       BUT, we prefer to append 'I' (candidatesM --> candidatesMI) to clarify

 #2. cat sf/unloadcandidatesMI
     =========================
 #!/bin/bash
 # unloadcandidatesMI - script calling uvcp to unload candidates Master Indexed file
 #  - demo by Owen Townsend, UVsoftware, Feb/2020
 #  - also see sortloadcandidatesM to sort & load Indexed file & loadcandidatesM load w/o sort
 #
 uvcp "fili1=data1/candidatesMI,typ=ISF,rcs=80,filo1=data1/candidatesMun,typ=LST,rcs=80"
 #======================================================================================
 exit
 #Note: 'fili1=data1/candidatesMI,typ=ISF' & .dat/.idx extensions on Indexed files
 # - Indexed file handler assumes input as 2 files candidatesMI.dat & candidatesMI.idx
 # - MicroFocus default omits the .dat extension, but there is an option to change default to .dat
 #   see "IDXNAMETYPE=2" in $UV/ctl/extfh.cfg control file listed at www.uvsoftware.ca/mvscobol.htm#5D2
 #   I have my Indexed file handler defaulted to .dat for the data partition of an Indexed file
 #   absence of '.dat' extension identifies the sequential file before loading to Indexed file
 #   the 'un' suffix on output file 'candidatesMun' identifies the UNloaded file
 #   would look same as the Indexed ".dat" file, but trailing blanks would be truncated

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9I2. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

LISTINGS of uvcopy jobs to update candidates Master

Note
  • Here are the code listings for the uvcopy jobs, which we have already run.
    <--- See Execute Instructions & console logs 2 pages back.

uvcopy jobs are stored in $RUNLIBS/pf/... subdir. RUNLIBS is defined by your .bash_profile & the result is appended to your $PFPATH by the common_profile. You can inspect the uvcopy jobs as follows:


 #0a. Logon userxx --> /home/userxx
 #0b. cdl  (alias 'cd $RUNLIBS) --> /home/userxx/demo/election2/

 #1. cat pf/fixcandiupsD2F
     =====================
 # fixcandiupsD2F - convert pipe Delimited to Fixed fields format
 #                    - demo by Owen Townsend, UVsoftware, Feb/2020
 # - see UVprojects at uvsoftware.ca/uvdemos.htm#Part_9
 # - run this uvcopy job as follows:
 #
 # uvcopy fixcandiupsD2F,fili1=data1/candiupsD20200229,filo1=data1/candiupsF20200229
 # =================================================================================
 # uvcopy fixcandiupsD2F <-- same/easier, filenames default as shown above
 # =====================
 #
 opr='$jobname - convert pipe Delimited to Fixed field format'
 rop=r1  # option to prompt for output file display (vi,cat,more,uvlp13)
 fili1=?data1/candiupsD20200229,typ=LST,rcs=80
 filo1=?data1/candiupsF20200229,typ=LST,rcs=80
 @run
         opn     all                      open files
 #
 # begin loop to get/process/put records until EOF bypassing any #comments
 man20   get     fili1,a0                 get next record
         skp>    man90                    (cc set > at EOF)
         cmc     a0(1),'#'                #comment record ?
         skp=    man20                    ifso - bypass, return to get next
 #
 # convert delimted to fixed fields 100 apart at b100,b200,b300,b400,b500
         fix     b100(100),a0(80),5,'|'   convert pipe delimited to fixed
 #
 # clear output record area & format in same columns as the candidates master record
         clr     c0(80),' '               clear output record area
         mvc     c0(10),b100              candidate name
         mvc     c31(8),b200              month
         edt     c41(3),b300(3),'zz9'     poll%
         edt     c46(5),b400(5),'z,zzz9'  delegates
         edt     c53(5),b500(5),'z,zzz9'  month$ donations
         put     filo1,c0                 write to output file
         skp     man20                    return to get next
 #
 # EOF - close files & end job
 man90   cls     all
         eoj

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9I3. LISTINGs of Election2 scripts & uvcopy jobs

uvcopy job to update candidates Master file


 #2. cat pf/upcandidatesMI
     =====================
 # upcandidatesMI - uvcopy job to update candidates Master Indexed file with current month
 #                - demo by Owen Townsend, UVsoftware, Feb/2020
 # - see UVprojects at uvsoftware.ca/uvdemos.htm#Part_9
 # - run this uvcopy job as follows:
 #
 # uvcopy upcandidatesMI,fili1=data1/candiupsF20200229,filr2=data1/candidatesMI
 # ============================================================================
 # uvcopy upcandidatesMI <-- same, filenames in job default as shown above
 # =====================
 #
 opr='$jobname - update candidates master file with current month campaign info'
 fili1=?data1/candiupsF20200229,typ=LST,rcs=80
 filr2=?data1/candidatesMI,typ=ISFs4,rcs=80
 @run
         opn     all                      open files
 #
 # begin loop to get updates & apply to master until EOF bypassing any #comments
 man20   get     fili1,a0                 get next update record
         skp>    man90                    (cc set > at EOF)
         cmc     a0(1),'#'                #comment record ?
         skp=    man20                    ifso - bypass, return to get next
 #
 # get master record by matching key (lastname)
 man30   clr     d0(80),' '               clear read area
         mvc     d0(10),a0                store key
         redm5   filr2,d0(80)             read on key= (option 'm5')
         skp=    man50                    (cc set = on match)
 #
 # nomatch - errmsg, bypass & return to get next detail
 man40   msg     a0(80)                   disaplay update record
         msgw    'nomatch for update record (above), enter to bypass to next'
         skp     man20
 #
 # match found - update Month, poll%, delegates, month$ (combined fields 23 bytes)
 man50   mvc     d31(27),a31              store new mth,poll%,dels,mth$ in master
         add     d58(7),a51(7)            add cur mth$ to total$
         edt     d58(7),d58(7),'zzz,zz9'  edit zero suppress
         upd     filr2,d0(80)             upate master record
         skp=    man20                    (cc = if OK)
 #
 # update I/O ERR - show errmsg & bypass to next detail
 man60   msg     a0(80)                   disaplay update record
         msgw    'master update I/O ERR for update record (above) - enter to bypass to next'
         skp     man20                    return to get next
 #
 # EOF - close files & end job
 man90   cls     all
         eoj

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9J1. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

Recommended Command Line Working Habits

AS well as demonstrating the Vancouver Utilities (uvcp,uvsort,uvcopy,etc), these election1/election2/election3 demos illustrate command line working habits that have been improved in the election2 demos, compared to the election1 habits, and will be further improved in the following election3 demos.

Working-Directory Recommendations/Tips

  1. We recommend defining your working directory for your current project using 'RUNLIBS' & 'RUNDATA' in your .bash_profile (listed on page '1B2'). For small projects, RUNLIBS & RUNDATA can define the same directory, but for large projects, they should be different & in large file systems rather than in /home/... Here is line# 38 in our .bash_profile.

    export RUNLIBS=$HOME/demo/election2 RUNDATA=$HOME/demo/election2
    ================================================================
  1. Defining RUNLIBS/RUNDATA in your .bash_profile has several advantages which will be illustrated here & later in election3. For example you can use the 'cdl' (or 'cdd') to change to your libs/data directory with 3 keystrokes from anywhere else you might be. This is made possible by defining RUNLIBS/RUNDATA in your .bash_profile as your working-directory & defining aliases cdl & cdd in the common_profile (cdl='cd $RUNLIBS' & cdd='cd $RUNDATA').

  2. The election1 demos stored the data files directly in the working directory, and did NOT use sub-directories to store different types of files. We strongly recommend setting up separate subdirs to store different file types.

  3. It might be OK to have files directly in the working directory for very small projects with short life-span, but usually better to take time to setup different subdirs for different file types that may be added to the project.

  4. For example, the election2 demos have subdir data1/... for data files, sf/... for Script Files, pf/... for Parameter Files,& tmp/... for temp files.

  5. We use 'data1' (not just 'data'), better to assign unique names to directories so your can search & find all instances in multiple directories. Searching for just 'data' would get a lot of matches you don't want. So the data files in election2 will be:
      election2
      |-- data1/
      |   |-- candidatesM
      |   |-- candidatesMI.dat
      |   |-- candidatesMI.idx
      |   |-- candidatesMun

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9J2. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

  1. The election1 demos executed the utilities (uvcp,uvsort) directly, which resulted in long command lines, for example:

     uvcp "fili1=candidatesM,rcs=80,typ=LST,filo1=candidatesMI,rcs=80,typ=ISF,isk1=0(10)"
     ====================================================================================
  1. For election2, we created scripts that call the utilities with desired parameters, especially advantageous if you need to repeat the commands multiple times in future. Election2 created the 'loadcandidatesM' script as follows:
      #!/bin/bash
      # loadcandidatesMI - script calling uvcp to load indexed file
      #  - demo by Owen Townsend, UVsoftware, Feb/2020
      #
      uvcp "fili1=data1/candidatesM,rcs=80,typ=LST,filo1=data1/candidatesMI,rcs=80,typ=ISFo4,isk1=0(10)"
      #=================================================================================================
      exit
      #Note: 'filo1=data1/candidatesMI,typ=ISF' & .dat/.idx extensions on Indexed files
      # - Indexed file handler will output 2 files candidatesM.dat & candidatesM.idx
  1. Note that we inserted 'data1/' on the file definitions, since we know we will be in the working directory & need to address files thru 1 level of subdir of subdir Later in election3, we will show you how this can work no matter where you are.

  2. Here is the 'tree' diagram for the subdirs in election2. Since we recommend staying in the working-directory & addressing files thru 1 level of subdir, we use short subdir names: 'sf/' for Script Files, 'pf/' for Parameter Files, & 'tmp/' for temporary files.
      election2
      |-- backup/
      |-- data1/                   <-- data files
      |   |-- candidatesM
      |   |-- candidatesMI.dat
      |   |-- candidatesMI.idx
      |   |-- candidatesMun
      |   |-- candiupsD20200229
      |   |-- candiupsF20200229
      |-- pf/                       <-- Parameter Files
      |   |-- fixcandiupsD2F
      |   |-- upcandidatesMI
      |-- sf/                       <-- Script Files
      |   |-- loadcandidatesM*
      |   |-- sortcandidatesM*
      |   |-- sortloadcandidatesM*
      |   |-- unloadcandidatesMI*
      |-- tmp/
      Note - 'tree' has a option -F to append '/' on directories & '*' on excutable files

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9J3. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

  1. To run the 'loadcandidatesM' script, from the working directory, you could enter:

      sf/loadcandidatesM
      ==================
  1. But, you do not need the 'sf/' prefix, since the supplied profiles include '$RUNLIBS/sf' in $PATH. (see profiles on pages '1B2' & '1B3').

      loadcandidatesM
      ===============

For the election2 demos, you do need to be in the working directory, since the script addresses the data files relative to the working directory, but later in election3, we will show you how this can work no matter where you are.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9K1. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

Working-Directory - Best Practices

For UVdemos, you copied $UV/demo/* to your $HOME/demo/... For Election2 demos you changed into the election3/ which is your 'working-directory' for the demos. Here are a few subdirs/files to illustrate Working-Directory Best-Practices.

 /home/uvadm/demo/...         <-- demo files supplied
 /home/userxx/demo/           <-- demo files copied to your homedir/demo/...
 election2/                   <-- demo files for the election2 demos
 |-- data1/                   <-- data files
 |   |-- candidatesM
 |   |-- candidatesMI.dat
 |   |-- candidatesMI.idx
 |   |-- candiupsD20200229
 |-- pf/                       <-- Parameter Files
 |   |-- fixcandiupsD2F
 |   |-- upcandidatesMI
 |-- sf/                       <-- Script Files
 |   |-- loadcandidatesM*
 |   |-- unloadcandidatesMI*
 |-- tmp/
 |   |-- SORTtmptest
  1. A working-directory with all files in subdirs keeps your working sapce clean & well organized - your subdirs will probably fit on 1 screen, which makes it easy for you to review your project.

  2. You should stay in the working directory - do not change into the subdirs to investigate/edit your files.

  3. Don't waste time by changing into a subdir, looking at 1 file,& changing back out. Of course OK, if you intend to inspect/edit multiple files, but not for 1 or 2.

  4. Staying in your working-directory protects your files, even the infamous 'rm *' will do no harm when all files are in subdirs.

  5. Working-Directory with subdirs allows you to use the same filename for applications that process the same datafile thru multiple stages. For example JCL & COBOL conversions require multi-step processing.

  6. Many utilities automatically write their output to the tmp/... directory & will make ./tmp/ if it does not exist.

  7. uvsort defaults to using ./tmp for merge files & tests presence & write access by writing tmp/SORTtmptest before attempting the sort (you can export SORTDIR=... in the common_profile to change the default).

  8. Garbage files (accidentally created) in the working-direcotry, rather then the intended subdir will stand out, so you will notice them, investigate the reason & correct the problem.

  9. Files created in the working directory can occur when there is a mistake in assigning a logical $VARIABLE filename to the physical subdir/filename. You would not notice if you were operating in the subdir with hundreds of files.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9K2. Election2 demos - subdirs for data,scripts,uvcopys,tmp,etc

tmp/ subdir in Working-Directory

A 'tmp/' subdir in in our working-directory has many uses & benefits. Do not confuse our ./tmp/ subdir in our working-directory with the unix system '/tmp/' at the top of the root file system.

  1. Several Vancouver Utility programs & scripts expect a tmp subdir and some will make it if not present.

  2. 'uvsort' expects a ./tmp/ subdir for merge files unless you define an alternate in the common_profile via 'export SORTDIR=...'

  3. 'uvhd' will make a tmp/ dir (if not present) to store an index for variable length & text files, so you can directly access any record by its record#. uvhd also uses the tmp/ dir for outputs from record write or print selections.

  4. Several Vancouver utility scripts output to the tmp/ subdir. For example 'alldiff2' runs the unix 'diff' command for all files in 2 directories, and writes the output to tmp/... using the 2nd directory-name.dif. Then you can use the editor to investigate the diff report (more flexible than cat/more).

  5. Redirecting large outputs of unix utilities such as 'grep' & then investigating with 'vi' is much more flexible than piping to more or less.

  6. You can use the tmp/ dir for cut & paste between files - edit the 1st file, write out the desired lines to tmp/xxx. Then edit the 2nd file & read the tmp/xxx file into the desired location.

  7. You can use the tmp/... directory like the unix system '|' command that pipes the output of 1 comamnd directly into a 2nd command, when you want to inspect or modify the 1st output before inputting to the 2nd command.

  8. Using a tmp/ subdir for any misc file outputs keeps your working-directory clean & well organized - no garbage file clutter, your subdirs can probably fit on 1 screen.

  9. Any files showing up on your working-directory list will signal an error to be investigated & removed. For example an incorrect variable assignment (export LOGICALNAME=subdir/filename) will cause a program to write the file into the working-directory rather than the intended subdir.

  10. We suggest your clean out your tmp/* dir frequently (at least daily) so you can easily see your current tmp/files vs yesterdays.

  11. These concepts protect your files, even the infamous 'rm *' will do no harm - because you stay in the working directory which has no files.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9M1. Election3 demos - using JCL conversion Features in your scripts

Pre-Requisites/Assumptions to run election3 demos

  1. Installed Vancouver Utitiliites as per uvsoftware.ca/install.htm
  2. You have a user logon or setup as per uvsoftware.ca/uvdemos.htm#1C1 (we will call it 'userxx')
  3. You have setup subdir demo & copied $UV/demo/* to your $HOME/demo/
  4. You have copied/renamed $APPSADM/env/bash_profile_uv to your $HOME/.bash_profile
Note
  • 1,2,3,4 already performed on for Election1 demos on page '9B1'
  1. Now modify your $HOME/.bash_profile, to run the election3 demo/tutorials:

    export RUNLIBS=$HOME/demo RUNDATA=$HOME/demo    <-- change this
    ============================================
    export RUNLIBS=$HOME/demo/election2 RUNDATA=$HOME/demo/election2 <-- OR this
    ================================================================

    export RUNLIBS=$HOME/demo/election3 RUNDATA=$HOME/demo/election3  <-- to this
    ================================================================
  1. Log off & back on to make the new RUNLIBS/RUNDATA definitions effective

    Objectives of ELECTION3 demo/tutorials

There are 3 levels of these demo/tutorials (election1,election2,election3) in separate directories $HOME/demo/election1,2,3 or $RUNDATA/demo/election1,2,3

Election1 demos were very simple - just 4 data files in the working directory with no subdirs. We entered uvcp/uvsort commands to load/unload Indexed files.

Election2 demos illustrated better working habits - setting up separate subdirs for various file types: ./data1/... for DataFiles, ./sf/... for ScriptFiles, ./pf/... for ParameterFiles,& ./tmp/... for temporary files.

Election3 demos will illustrate how any site can use the many powerful features used in the JCL/scripts at mainframe migration sites.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9N1. Election3 demos - using JCL conversion Features in your scripts

Benefits of JCL conversion scripts

  1. GDG files provided by functions exportgen0 & exportgen1

  2. 'testcc' function to test the status of each step with 1 line of code. Failure will display the status code & error messages & abort the job. 'testcc' failure ensures that any new generation GDG files are not committed.

  3. 'joblog' script captures console logs, for Example: --> joblog jobname <-- captures the log into $RUNDATA/joblog/jobname.log

  4. 'jobset51' function allows you to have multiple sets of Data because it changes directory to $RUNDATA (superdir for data subdirs/files). Your scripts can then be written to address files relative to the subdirs. Different programmers can have different sets of test-date, since they can define RUNDATA in their .bash_profile, for example: --> export RUNDATA=$HOME/testdata/

  5. 'jobset51' allows you to run scripts from anywhere, because it changes directory to $RUNDATA, so it does not matter where you are when you run a script that is written to the JCL conversion standards.

  6. Restart at any step in your script: To use this feature you would insert a label at the begining of each step, for example 'S0010=A' for step1, 'S0020=A' for step2, etc. To restart at step2, you run the job as follows: --> joblog jobname STEP=S0020 <--

  7. KEYWORDS=VALUES on the command line are automatically available as $KEYWORD anywhere in your scripts, for example: --> joblog jobname DATE=20200229 <-- makes $DATE available in the script

  8. 'logmsg2' function (vs 'echo') captures critical messages into --> $HOME/$RUNLIBS/jobmsgs/yymmdd/yymmdd_jobname <-- subdirectoried by date & messages are time-stamped

  9. Korn shell 'autoload' typeset allows storing functions in a separate library using $FPATH. Here is the FPATH in the supplied common_profile_uv. --> export FPATH=$APPSADM/sfun/ <--

  10. $RUNLIBS/#RUNDATA allow multiple sets of test & production LIBRARIES (scripts & programs) for multiple sets of test & production DATA. - different sets of testdata for different programmers - never need to modify scripts for test or production
Note
  • A call to the 'jobset51' function must be inserted at the begining
    of scripts in which you use these powerful features.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9N2. Election3 demos - using JCL conversion Features in your scripts

profiles - stub & common profiles

The stub & common profiles (listed on pages '1B2' & '1B3') are essential to providing the benefits described above. The supplied stub profile is to be copied to user homedirs & renamed as '.profile' or '.bash_profile'. The stub_profile calls the common_profile to setup directory search PATHs, etc.

Each programmer can modify his stub_profile (.bash_profile) to define his RUNDATA & RUNLIBS as desired. He could define RUNDATA as a directory within his homedir to have his own set of testfiles for no conflict with other programmers.

Programmers could also define RUNLIBS as a directory in thier homedir, but usually their would be a common RUNLIBS directory for all programmers since they would be working on different scripts/programmers.

RUNLIBS/RUNDATA could be defined as PRODLIBS/PRODDATA vs TESTLIBS/TESTDATA on the same machine, but usually production would be on a different machine.

Profiles are critical to benefiting from the JCL conversion features See stub & common profiles listed on pages '1B2' & '1B3'.

Programmers can setup their own set of test data files by changing their .bash_profile to 'export RUNDATA=$HOME/testdata', then logoff/logon to update the search PATHs when their .bash_profile calls the common profile ($APPSADM/env/common_profile_uv).

You must use the Korn shell (1st line must be '!/bin/ksh') for scripts that use JCL conversion features, since ksh has 'FPATH' to define a common directory to store the functions supporting JCL conversion features (exportgen,jobset51,jobend51,testcc,etc). See 'export FPATH=$APPSADM/sfun' in the common_profile.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9N3. Election3 demos - using JCL conversion Features in your scripts

$RUNDATA/$RUNLIBS & multiple sets LIBrarieS/DATA

$RUNDATA & $RUNLIBS defined in user profiles allow different sets of Libraries & DATA for different users. Here are the requirements that make this possible:

  1. All file definitions in the scripts must be RELATIVE to the current directory since the 'jobset51' function (inserted at begining of scripts) performs 'cd $RUNDATA'. IE - no absolute pathnames to datafiles.

  2. Any calls to user scripts & programs that are not in the $PATH defined in the common_profile (modified by the users .bash_profile) must call using $RUNLIBS explicitly.

2a. Example to call a Micro Focus COBOL program:


      cobrun $ANIM $CBLX/car100  <-- execute MF COBOL program car100.int
      =========================
      export CBLX=$RUNLIBS/cblx  <-- definition of CBLX in the common_profile

2b. Example to input a SYSIN file (from the 'parms' subdir in $RUNLIBS) Must insert $RUNLIBS prefix to parms/modulename since we know all scripts are executing within $RUNDATA (not $RUNLIBS).


       exportfile SYSIN $RUNLIBS/parms/modulexx
       =========================================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9N4. Election3 demos - using JCL conversion Features in your scripts

directories for election3 demos

 /home/userxx/demo/election3/
 |-- 1README_election3
 |-- backup/
 |-- ctl/                             <-- control files
 |   |-- gdgctl51                       - text file generations for each GDG file
 |   |-- gdgctl51I.dat                  - edit gdgctl51 & load to gdgctl51I.dat/.idx
 |   `-- gdgctl51I.idx                    with --> uvcopy gdgload51 <--
 |   |
 |-- data1/                           <-- data files (except for GDGs)
 |   |-- candidatesM                    - candidates Sequential Master file
 |   |-- candidatesMI.dat               - candidates Indexed Master file (Data)
 |   |-- candidatesMI.idx               - candidates Indexed Master file (Index)
 |   |-- candidatesMun                  - candidates Master UNloaded to Seqntl
 |   |-- candiupsD20200229              - campaign updates pipe|Delimited format
 |   `-- candiupsF20200229              - campaign updates Fixed field format
 |   |
 |-- datag1/                          <-- GDG files (separate subdir optional)
 |   |-- candidatesMG_000001            - 1st generation
 |   `-- candidatesMG_000002            - 2nd generation
 |   |
 |-- jcls/                            <-- Korn shell scripts with advanced features
 |   |-- loadcandidatesMG2              - load 1st GDG generation from sequential file
 |   |-- upcandimg0                     - original JCL converted to script from JCL
 |   |-- upcandimg1                     - 1 line modified to execute 'uvcopy' (vs COBOL)
 |   |-- upcandimg2                     - same job with fewer comments/warnings
 |   `-- upcandimg3                     - same job with 2nd step to demo RESTART
 |   |
 |-- joblog/                         <-- joblog directory
 |   |-- loadcandidatesMG2.log         - log from jcls/loadcandidatesMG2
 |   `-- upcandimg1.log                - log from jcls/upcandimg1
 |-- jobmsgs/
 |   `-- 20200314                    <-- jobmsgs for 20200314
 |       `-- 080140_loadcandidatesMG2  - jobmsgs from jcls/loadcandidatesMG2 at 08:01:40
 |       `-- 080255_upcandimg1         - jobmsgs from jcls/upcandimg1 at 08:02:55
 |-- jobtimes/
 |-- jobtmp/                         <-- jobtmp/ setup by jobset51 at begin scripts
 |   |-- loadcandidatesMG2             - jobtmp files for jcls/loadcandidatesMG2
 |   |   |-- GDG/...                   - GDG files restored to datag1/... by jobend51
 |   |-- upcandimg1                    - jobtmp files for jcls/upcandiup1
 |   `   |-- GDG/...                   - GDG files restored to datag1/... by jobend51
 |   |
 |-- pf/                             <-- uvcopy jobs (Parameter Files)
 |   |-- fixcandiupsD2F                - convert pipe|delimited to fixed fields
 |   |-- upcandidatesMI                - update candidates Indexed Master file
 |   `-- upcandidatesMG                - update candidates GDG Master file (alternative)
 |   |
 |-- sf/                             <-- bash shell scripts (most for Election2 demos)
 |   |-- loadcandidatesMI              - load candidates Master Sequential file
 |   |-- loadcandidatesMG1   <-- load GDG master file ** ONLY 1 used in Election3 demos **
 |   |-- sortcandidatesM               - sort candidates Master Sequential file
 |   |-- sortloadcandidatesMI          - sort & load candidates Master Sequential file
 |   `-- unloadcandidatesMI            - unload candidates Master Indexed file
 `-- tmp/

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9O1. Election3 demos - using JCL conversion Features in your scripts

INSPECT Election3 data FILES


 #1.  Logon userxx --> /home/userxx
      ============
 #2.  cdd  (alias 'cd $RUNDATA) --> /home/userxx/demo/election3/
      ===
 OR-> cdl  (alias 'cd $RUNLIBS) --> /home/userxx/demo/election3/
      ===    - same effect since RUNLIBS/RUNDATA defined the same

 #3a. llr .    <-- llr (alias ls -lR) list Recursively
      =====      - all dirs/files in election3/...

 #3b. tree .   <-- alternative to illustrate the directory tree structure
      ======     - see output on previous page

 #4a. l data1/   <-- list filenames in data1/...
      =======      - will show file contents later
      -rw-rw-r-- 1 userxx apps  754 Feb 21 11:32 candidatesM
      -rw-rw-r-- 1 userxx apps  891 Feb 23 18:06 candidatesMI.dat
      -rw-rw-r-- 1 userxx apps 3072 Feb 23 18:06 candidatesMI.idx
      -rw-rw-r-- 1 userxx apps  754 Feb 24 12:17 candidatesMun
      -rw-rw-r-- 1 userxx apps  322 Feb 21 11:32 candiupsD20200229
      -rw-rw-r-- 1 userxx apps  385 Feb 21 13:03 candiupsF20200229

 #4b. l data1g/   <-- list filenames in datag1/... (GDG files)
      ========
      -rw-rw-r-- 1 userxx apps 779 Mar  2 12:42 candidatesMG_000001
      -rw-rw-r-- 1 userxx apps 462 Mar 13 15:52 candidatesMG_000002

 #5. cat datag1/candidatesMG_000001   <-- display candidatesMG GDG master file
     ==============================     - loaded from data1/candidatesM
      #1 candidatesM - master file 2020 election (uvsoftware.ca/uvdemos.htm#Part_9)
      #2 Name   Age  City     State  Month    Poll% Delegs Month$ Total$ (millions)
      #3       1         2         3         4         5         6         7         8
      #4345678901234567890123456789012345678901234567890123456789012345678901234567890
      Biden      77  Wilmington  DE  20200131   27      0      0     61
      Bloomberg  77  New York    NY  20200131    7      0      0    200
      Buttigieg  38  South Bend  IN  20200131    8      0      0     76
      Klobuchar  59  Minneapolis MN  20200131    4      0      0     29
      Sanders    78  Burlington  VT  20200131   22      0      0    108
      Trump      74  New York    NY  20200131   44  2,472      0    150
      Warren     70  Cambridge   MA  20200131   15      0      0     82

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9O2. Election3 demos - using JCL conversion Features in your scripts

INSPECT Election3 data file CONTENTs


 #6. cat data1/candiupsF20200229   <-- display candidates update records
     ===========================
     - Fixed field records extracted from Delimited by uvcopy job pf/fixcandiupsD2F
     - see Delimited file data1/candiupsD20200229 in previous Election2 demos
      Biden                          20200229  23      6      9
      Bloomberg                      20200229  10      0    264
      Buttigieg                      20200229   9     22      6
      Klobuchar                      20200229   5      7      6
      Sanders                        20200229  25     21     25
      Trump                          20200229  44   2472     15
      Warren                         20200229  14      8     11

 #7. cat datag1/candidatesMG_000002  <-- display GDG master file AFTER update
     ==============================      with current months donations
      #1 candidatesM - master file 2020 election (uvsoftware.ca/uvdemos.htm#Part_9)
      #2 Name   Age  City     State  Month   Poll% Delegs Month$ Total$ (millions)
      #3       1         2         3         4         5         6         7         8
      #4345678901234567890123456789012345678901234567890123456789012345678901234567890
      Biden      77  Wilmington  DE  20200229   23      6      9     70
      Bloomberg  77  New York    NY  20200229   10      0    264    464
      Buttigieg  38  South Bend  IN  20200229    9     22      6     82
      Klobuchar  59  Minneapolis MN  20200229    5      7      6     35
      Sanders    78  Burlington  VT  20200229   25     21     25    133
      Trump      74  New York    NY  20200229   44   2472     15    165
      Warren     70  Cambridge   MA  20200229   14      8     11     93

Notes

  1. We have shown you the orignal data file contents from UV Software, and you should get the same results after you run the scripts below.

  2. The demo Op. Instrns. below do not show the data file contents, but you can run 'cat/more/less' commands after each script to confirm you get same results as shown above.

  3. We prefer the unix 'cat' command to display short files vs more or less which prompt for browsing at EOF (vs just quitting).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9P1. Election3 demos - using JCL conversion Features in your scripts

EXECUTE scripts to Load & update GDG Master candidatesMG_

Note
  • We are showing Operating Instructions & console logs BEFORE we show
    the coding of the scripts. See code LISTINGS 2 pages ahead --->

 #0a. Logon userxx --> /home/userxx
 #0b. cdd  (alias 'cd $RUNDATA) --> /home/userxx/demo/election3/
 OR-> cdl  (alias 'cd $RUNLIBS) --> /home/userxx/demo/election3/
      ===    - same effect since RUNLIBS/RUNDATA defined the same

 #1.  Load candidates GDG Master file - 2 'uvcp' scripts provided:
      ============================================================
      #1a - hard-codes 1st generation filename
      #1b - using GDG functions

 #1a. loadcandidatesMG1 - hardcodes the output filename as datag1/candidatesMG_000001
      ================= - requires only 1 line of code (vs #comment lines)
                        - displays console log as shown below:
      200314:150629:uvcp: uvcp fili1=data1/candidatesM,filo1=datag1/candidatesMG_000001
      200314:150629:uvcp: EOF fili01 11 rds, 779 size; data1/candidatesM
      200314:150629:uvcp: EOF filo01 11 wrts, 779 size; datag1/candidatesMG_000001
      uvcp Start=2020/03/14_15:06:29 End=15:06:29 Elapsed=0.125

 #1b. loadcandidatesMG2 - load Sequential data1/candidatesM to GDG file
      ================= - uses the GDG function exportgen1 to determine output name
                        - requires 15 lines of code (vs #comments/explanations)
                        - extra code to support use of GDG files
                        - see script listed later on page '9S1'
                        - joblog below (4 long lines split to fit documentation page width)

console log from 'loadcandidatesMG2'

      200314:074738:loadcandidatesmg2: Begin Job=loadcandidatesmg2
      200314:074738:loadcandidatesmg2: HOSTNAME=uvsoft5 LOGNAME=userxx
      200314:074738:loadcandidatesmg2: ProcessID=5784 RUNDATE=20200314
      200314:074738:loadcandidatesmg2: RUNDATA=/home/userxx/demo/election3
      200314:074738:loadcandidatesmg2: file: CANDIDATESM=data1/candidatesM fsize=779
      200314:074738:loadcandidatesmg2: gen1+1 CANDIDATESMG=jobtmp/loadcandidatesmg2/GDG/@home@userxx\
                                              @demo@election3@datag1@candidatesMG_000001 gens=15
      200314:074738:loadcandidatesmg2: Executing--> uvcp to load 1st gen GDG candidates Master file
      200314:074738:LOADCANDIDATESMG2: uvcp fili1=data1/candidatesM,filo1=jobtmp/loadcandidatesmg2/GDG\
                                            /@home@userxx@demo@election3@datag1@candidatesMG_000001
      200314:074738:LOADCANDIDATESMG2: EOF fili01 11 rds, 779 size; data1/candidatesM
      200314:074738:LOADCANDIDATESMG2: EOF filo01 11 wrts, 779 size; jobtmp/loadcandidatesmg2/GDG\
                                            /@home@userxx@demo@election3@datag1@candidatesMG_000001
       uvcp start 2020/03/14_07:47:38 end 07:47:38 elapsed 26403287:38
      200314:074738:loadcandidatesmg2: CC = 0
      200314:074738:loadcandidatesmg2: moving /home/userxx/demo/election3/jobtmp/loadcandidatesmg2/GDG\
                                            /files back to intended directories
      /home/userxx/demo/election3/datag1/candidatesMG_000001
      200314:074738:LOADCANDIDATESMG2: Job Times: Begun=07:47:38 NormalEnd=07:47:38 Elapsed=00:00:00
      200314:074738:loadcandidatesmg2: JobEnd=Normal, JCC=0, StepsExecuted=0, LastStep=S0000

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9P2. Election3 demos - using JCL conversion Features in your scripts

EXECUTE uvcopy jobs to update Master file with campaign info

Note
  • We are showing Operating Instructions & console logs BEFORE we show
    the code LISTINGS of the uvcopy jobs. See those 2 pages ahead --->

 #2. Convert Monthly campaign results from Delimited to Fixed field format
     - prior to Master file update job

 #2a. uvcopy fixcandiupsD2F,fili1=data1/candiupsD20200229,filo1=data1/candiupsF20200229
      =================================================================================

 #2b. uvcopy fixcandiupsD2F  <-- same/easier, filenames default as shown above
      =====================   --> displays log as shown below
       fixcandiupsD2F - convert pipe Delimited to Fixed field format
       200225:132300:fixcandiup: uvcopy ver=20190714 pf=./pf/fixcandiupsD2F
       uvcopy LNX H64 license=20190714_99V_930630 site=UV_Software
       200225:132301:fixcandiup: EOF fili01 rds=9 size=323: data1/candiupsD20200229
       200225:132301:fixcandiup: EOF filo01 wrts=7 size=385: data1/candiupsF20200229

 #2c. cat data1/candiupsD20200229  <-- could display INPUT Delimited file
      ===========================    - already shown above

 #2d. cat data1/candiupsF20200229  <-- could display OUTPUT Fixed field file
      ===========================    - already shown above

 #3. Update GDG Master file with monthly campaign info - 3 ways to Run:
     =============================================================================
      #3a - Run script without joblog
      #3b - Run script with joblog
      #3c - Run uvcopy job directly without script (N/R - need to know generation#s)
          - script GDG functions exportgen0/exportgen1 auto determine current & next gen#s

 #3a. upcandimg1 date=20200229   <-- run script to update candidatesMG_ GDG master file
      ========================     - console display shown on next page below
       - 'without joblog' - console display same as with 'joblog' (#3a & #3b same display)

 #3b. joblog upcandimg1 date=20200229 <-- run script update GDG master file with joblog
      ===============================   - with joblog shown below
       - 'with joblog' - also captures the display in a file ($RUNDATA/joblog/upcandimg1.log)
Note
  • script 'upcandimg1' is written to the JCL conversion standards & benefits
  • see the benefits explained on page '9T1'
  • one benefit provides for 'joblog's (console displays also captured in a file)
  • the joblog file is named the same as the job/script name with extension '.log'
  • running 'joblog upcandimg1' will create '$RUNDATA/joblog/upcandimg1.log'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9P3. EXECUTE Election3 demos - Advanced JCL Conversion Features

JOBLOG from upcandimg1 - upate Master with campaign details

Note
  • We are repeating #3b from the previous page, so we can show the comamnd
    & resulting joblog on this page.

 #3b. joblog upcandimg1 date=20200229 <-- run script update GDG master file with joblog
      ===============================   - with joblog shown below
      200314:085115:upcandimg1: Begin Job=upcandimg1
      200314:085115:upcandimg1: HOSTNAME=uvsoft5 LOGNAME=userxx
      200314:085115:upcandimg1: ProcessID=6458 RUNDATE=20200314
      200314:085115:upcandimg1: RUNLIBS=/home/userxx/demo/election3
      200314:085115:upcandimg1: RUNDATA=/home/userxx/demo/election3
      200314:085115:upcandimg1: Arguments: upcandimg1 date=20200229
      200314:085115:upcandimg1:     --- Executing Step S0010/1 upcandidatesMG ---
      200314:085115:upcandimg1: gen0 CMASOLD=datag1/candidatesMG_000001 insize=4K
      200314:085115:upcandimg1: gen1+1 CMASNEW=jobtmp/upcandimg1/GDG/@home@userxx@demo@election3\
                                                             @datag1@candidatesMG_000002 gens=15
      200314:085115:upcandimg1: file: CMUPDTS=data1/candiupsF20200229 fsize=413
      200314:085115:upcandimg1: Executing--> uvcopy upcandidatesMG update candidates Master GDG file
      upcandidatesMG - update candidates master file with current month campaign info
      200314:085115:UPCANDIMG1: uvcopy ver=20200313 pf=./pf/upcandidatesMG
      uvcopy LNX H64 license=20200313_99V_930630 site=UV_Software
      200314:085115:UPCANDIMG1: EOF fili01 rds=11 size=779: datag1/candidatesMG_000001
      200314:085115:UPCANDIMG1: EOF fili02 rds=7 size=413: data1/candiupsF20200229
      200314:085115:UPCANDIMG1: EOF filo01 wrts=7 size=462: jobtmp/upcandimg1/GDG/@home@userxx@demo\
                                                            @election3@datag1@candidatesMG_000002
      200314:085115:upcandimg1: CC = 0
      200314:085115:upcandimg1: moving /home/userxx/demo/election3/jobtmp/upcandimg1/GDG/files back to\
                                           /home/userxx/demo/election3/datag1/candidatesMG_000002
      200314:085115:UPCANDIMG1: Job Times: Begun=08:51:15 NormalEnd=08:51:15 Elapsed=00:00:00
      200314:085115:upcandimg1: JobEnd=Normal, JCC=0, StepsExecuted=1, LastStep=S0010

 #3b2. vi joblog/upcandimg1.log  - can review the joblog file at any time
       ========================  - until replaced by the next run of 'joblog upcandimg1'
        - could use 'joblog1' (vs 'joblog') to append a date/time stamp on joblog/filenames

 #3c. uvcopy upcandidatesMG,fili1=datag1/candidatesMG_000001,filo1=datag1/candidatesMG_000002\
                           ,fili2=data/candiupsF20200229
      ========================================================================================
       - could run uvcopy directly if you know generation#s required (above 1st generation update)
       - better to use the script

 #4c. cat data1/candidatesMG_000001  <-- could display GDG Master file Before 1st update
      =============================      which copies/updates to datag1/candidatesMG_000002
                                       - already shown 2 pages above

 #4d. cat data1/candidatesMG_000002  <-- could display GDG generation #2 AFTER updating
      =============================      with current 1st month's campaign info
                                       - also shown 2 pages above

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9Q1. Election3 demos - using JCL conversion Features in your scripts

LISTINGS of scripts to sort/load/unload candidates Master

Starting on the next page we will list the scripts & uvcopy jobs, we have already run (see Execute Instructions & console logs 2 pages back).

Our main intention with these listings is to illustrate the BENEFITs of the JCL conversion scripts previously explained on page '9N1' & repeated on page '9T1' followed by REQUIREMENTs on '9T2'.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9Q2. LISTINGs of JCL/scripts used in Election3 demos

JCL conversion scripts to illustrate Benefits

Our main intention with these listings is to illustrate the benefits of writing your scripts to the standards used for our JCL conversions. (If interested see uvsoftware.ca/jclcnv1demo.htm).

Here is a sample of mainframe JCL & the automatic conversion to script.

Original mainframe JCL

 //UPCANDIMG0 JOB  'update candidatesmg election GDG master file'
 //* see www.uvsoftware.ca/uvdemos.htm#Part9 (election3 demos)
 //* mainframe GDG files converted to functions exportgen0 & exportgen1
 //STEP010   EXEC PGM=upcandidatesmg
 //CMUPDTS   DD   DSN=candiupsF&date,DISP=OLD
 //CMASOLD   DD   DSN=candidatesmg(0),DISP=OLD
 //CMASNEW   DD   DSN=candidatesmg(+1),DISP=(,CATLG,DELETE),
 //               UNIT=DISK,SPACE=(TRK,(50,50),RLSE),
 //               DCB=(MODEL.DSCB,LRECL=128,BLKSIZE=6118,RECFM=FBA)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9Q3. LISTINGs of JCL/scripts, Election3 demos, Advanced Features

JCL conversion to script for unix/Linux

 #!/bin/ksh
 ##UPCANDIMG0 JOB  'update candidatesmg election GDG master file'
 export   JOBIDX=$(basename $0) JOBID=${JOBIDX%.*} scriptpath="$0" args="$*"
 integer  JCC=0 SCC=0 LCC=0  # init step status return codes
 autoload jobset51 jobend51 jobabend51 logmsg1 logmsg2 stepctl51 testcc evalcp1
 autoload exportfile exportgen0 exportgen1 exportgen2 exportgenall exportgenx
 jobset51 "$args"  # call function for JCL/script initialization
 goto     #optional restart at any step, goto step default S0000=A in jobset51
 S0000=A
 # * see www.uvsoftware.ca/uvdemos.htm#Part9 (election3 demos)
 # * mainframe GDG files converted to functions exportgen0 $ exportgen1
 #1==================== upcandimg0 begin step#S0010 upcandidatesmg COBOL ================
 S0010=A
 alias goto="";
 stepctl51;
 goto
 export JSTEP=S0010; ((XSTEP+=1)); SCC=0; SMAX=4;
 logmsg2 "********** Begin Step $JSTEP/$XSTEP upcandidatesmg COBOL **********"
 ##S001010   EXEC PGM=upcandidatesmg
 export PROGID=upcandida  # XREFID=COBOL:upcandidatesmg,upcandimg0
 export PARM=""
 exportfile  CMUPDTS data1/candiupsf${date} #OLD
 exportgen0 0 CMASOLD data1/candidatesmg_ #OLD
 exportgen1 +1 CMASNEW data1/candidatesmg_ #(,CATLG,DELETE)_128_FBA
 #exportgen1 GDGs written to jobtmp/$jobid2 restored to outdir at Normal EOJ
 logmsg2 "Executing--> cobrun \$ANIM \$CBLX/upcandidatesmg"
 #3----------------------------------------------------------------------
 ## cobrun $ANIM $CBLX/upcandidatesmg  #<-- COBOL not available, changed to uvcopy
 uvcopy upcandidatesMG,fili1=$CMASOLD,filo1=$CMASNEW,fili2=$CMUPDTS #<-- only change
 #4----------------------------------------------------------------------
 testcc $? $SMAX
 # * jclproc51 version 20200313, executed on 20200313:1503050312
 #8======================================================================
 S9000=A
 jobend51 #move any new GDG files from jobtmp to intended outdirs
 logmsg2 "JobEnd=Normal, JCC=$JCC, StepsExecuted=$XSTEP, LastStep=$JSTEP"
 # jclunix51 options=a2b0c8d0e2f0g1h0i0j0k15l1999m1n3o8p0q0r0s0t15u0v0w0x0y6z0
 exit $JCC #jclunix51 version=20200313, converted 20200313:1503050316 sitename=UV Software
 Note - Only 1 line changed from JCL conversion (since COBOL was not available)
      --> ## cobrun $ANIM $CBLX/upcandiatesmg  <-- ##commented out,
      --> uvcopy upcandidatesMG,fili1=...      <-- replaced by uvcopy job
      - see 4 versions of this demo job:
upcandimg0
  • original converted from mainframe JCL BEFORE enhancements (above)
upcandimg1
  • enhanced model for users to copy marked lines into their scripts
upcandimg2
  • alternate version, same with fewer #comments & wearnings
upcandimg3
  • same step1 with 2nd step added to demo RESTART feature

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9Q4. LISTINGs of JCL/scripts, Election3 demos, Advanced Features

upcandimg1 - enhanced version marking lines for user scripts

'upcandimg1' is the same functionality as 'upcandimg0' above, with #comments to highlight the common-code lines that you need to insert into your scripts to take advantage of the many benefits available in JCL conversion scripts.

 #!/bin/ksh
 # upcandimg1 - update candidates election GDG master file
 #            - by Owen Townsend, UV Software, Mar 2020
 # This script intended to highlight the common-code lines that you can
 # --> INSERT into your scripts to get the benefits available in JCL conversion scripts.
 #           - GDG files, testcc step status (avoids GDG commit on error)
 #           - run from anywhere, step restart, keyword=values on cmdline,
 #           - logmsg2 stores critical msgs in $RUNDATA/jobmsgs/datetime stamped
 #           - run with joblog script to save console log in $RUNDATA/joblog/jobname.log
 #
 #--> INSERT next 7 lines at begining of your new scripts
 export   JOBIDX=$(basename $0) JOBID=${JOBIDX%.*} scriptpath="$0" args="$*"
 integer  JCC=0 SCC=0 LCC=0 SMAX=0  # init step status return codes
 autoload jobset51 jobend51 jobabend51 logmsg1 logmsg2 stepctl51 testcc evalcp1
 autoload exportfile exportgen0 exportgen1 exportgen2 exportgenall exportgenx
 jobset51 "$args"  # call function for JCL/script initialization
 goto     #optional restart at any step, goto step default S0000=A in jobset51
 S0000=A
 #S0000=A #***Note step labels S0010=A,etc 7 chars only (no comments on right)
 #1============== upcandimg1 begin step#S0010 upcandidatesMG ===============
 #--> INSERT next 4 lines at begining of each step (optional)
 S0010=A
 export JSTEP=S0010; ((XSTEP+=1)); SCC=0; SMAX=4;
 export PROGID=upcandidatesMG
 logmsg2 "    --- Executing Step $JSTEP/$XSTEP $PROGID ---"
 #2-------------------------------------------------------------------------
 #--> Optional Insert 4 lines to ensure operator entered date=... on command line
 if [[ "$date" == 2020* ]]; then :
 else echo "*** Must specify date on command line matching suffix on candiupsFyyyymmdd file ***"
      echo "upcandimg1 date=20200229  #<-- example 1st update with demo data"; exit 9; fi
 exportgen0  0 CMASOLD datag1/candidatesMG_
 exportgen1 +1 CMASNEW datag1/candidatesMG_
 exportfile    CMUPDTS data1/candiupsF$date
 #exportgen1 GDGs written to jobtmp/$jobid2 restored to outdir at Normal EOJ
 logmsg2 "Executing--> uvcopy upcandidatesMG update candidates Master GDG file"
 #3----------------------------------------------------------------------
 uvcopy upcandidatesMG,fili1=$CMASOLD,filo1=$CMASNEW,fili2=$CMUPDTS
 #4----------------------------------------------------------------------
 #--> INSERT following 1 line after each step program execution
 testcc $? $SMAX
 #--> INSERT next 4 lines at the end of your scripts
 #8======================================================================
 S9000=A
 jobend51 #move any new GDG files from jobtmp to intended outdirs
 logmsg2 "JobEnd=Normal, JCC=$JCC, StepsExecuted=$XSTEP, LastStep=$JSTEP"
 exit $JCC

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9Q5. LISTINGs of JCL/scripts, Election3 demos, Advanced Features

upcandimg2 - alternate version with fewer #comments

 #!/bin/ksh
 # upcandimg2 - update candidates election GDG master file
 #            - by Owen Townsend, UV Software, Feb 2020
 #
 # This script is the same as 'upcandimg1', with fewer #comment lines that
 # highlighted the common-code lines that you can INSERT into your scripts
 # to get the benefits available in JCL conversion scripts.
 #
 # upcandimg2 date=20200229  <-- execute this script
 # ========================    - arg1 date for suffix on candiupsF$date
 #
 #--> insert next 7 lines at begining of your new scripts
 export   JOBIDX=$(basename $0) JOBID=${JOBIDX%.*} scriptpath="$0" args="$*"
 integer  JCC=0 SCC=0 LCC=0  # init step status return codes
 autoload jobset51 jobend51 jobabend51 logmsg1 logmsg2 stepctl51 testcc
 autoload exportfile exportgen0 exportgen1 exportgen2 exportgenall exportgenx
 jobset51 "$args"  # call function for JCL/script initialization
 goto     #optional restart at any step, goto step default S0000=A in jobset51
 S0000=A
 #1============== upcandimg2 begin step#S0010 upcandidatesMG ===============
 #--> insert next 4 lines at begining of each step (optional)
 S0010=A
 export JSTEP=S0010; ((XSTEP+=1)); SCC=0; SMAX=4;
 export PROGID=upcandidatesMG
 logmsg2 "      ----- Executing Step $JSTEP/$XSTEP $PROGID -----"
 #2-------------------------------------------------------------------------
 exportgen0  0 CMASOLD datag1/candidatesMG_
 exportgen1 +1 CMASNEW datag1/candidatesMG_
 exportfile    CMUPDTS data1/candiupsF$date
 logmsg2 "Executing--> uvcopy upcandidatesMG update candidates Master GDG file"
 #3----------------------------------------------------------------------
 uvcopy upcandidatesMG,fili1=$CMASOLD,filo1=$CMASNEW,fili2=$CMUPDTS
 #4----------------------------------------------------------------------
 #--> INSERT following 1 line after each step program execution
 testcc $? $SMAX
 #--> INSERT next 4 lines at the end of your scripts
 #8======================================================================
 S9000=A
 jobend51 #move any new GDG files from jobtmp to intended outdirs
 logmsg2 "JobEnd=Normal, JCC=$JCC, StepsExecuted=$XSTEP, LastStep=$JSTEP"
 exit $JCC

4 versions of this demo job

upcandimg0
  • original converted from mainframe JCL BEFORE enhancements (above)
upcandimg1
  • enhanced model for users to copy marked lines into their scripts
upcandimg2
  • alternate version, same with fewer #comments & wearnings
upcandimg3
  • same step1 with 2nd step added to demo RESTART feature

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9R1. LISTINGs of JCL/scripts, Election3 demos, Advanced Features

upcandimg3 - 2nd step added to demo RESTART feature

 #!/bin/ksh
 # upcandimg3 - update candidates election GDG master file
 #            - by Owen Townsend, UV Software, Feb 2020
 #
 # upcandimg3 date=20200229  <-- normal execution
 # ========================
 #
 # upcandimg3 date=20200229 STEP=S0020  <-- rerun starting at step2
 # ===================================
 #
 export   JOBIDX=$(basename $0) JOBID=${JOBIDX%.*} scriptpath="$0" args="$*"
 integer  JCC=0 SCC=0 LCC=0  # init step status return codes
 autoload jobset51 jobend51 jobabend51 logmsg1 logmsg2 stepctl51 testcc
 autoload exportfile exportgen0 exportgen1 exportgen2 exportgenall exportgenx
 jobset51 "$args"  # call function for JCL/script initialization
 goto     #optional restart at any step, goto step default S0000=A in jobset51
 S0000=A
 #1============== upcandimg3 begin step#S0010 upcandidatesMG ===============
 S0010=A
 export JSTEP=S0010; ((XSTEP+=1)); SCC=0; SMAX=4;
 export PROGID=upcandidatesMG
 logmsg2 "      ----- Executing Step $JSTEP/$XSTEP $PROGID -----"
 #2-------------------------------------------------------------------------
 exportgen0  0 CMASOLD datag1/candidatesMG_
 exportgen1 +1 CMASNEW datag1/candidatesMG_
 exportfile    CMUPDTS data1/candiupsF$date
 logmsg2 "Executing--> uvcopy upcandidatesMG update candidates Master GDG file"
 #3----------------------------------------------------------------------
 uvcopy upcandidatesMG,fili1=$CMASOLD,filo1=$CMASNEW,fili2=$CMUPDTS
 #4----------------------------------------------------------------------
 testcc $? $SMAX
 #1============== upcandimg3 begin step#S0020 upcandidatesMG ===============
 S0010=A
 export JSTEP=S0020; ((XSTEP+=1)); SCC=0; SMAX=4;
 export PROGID=cat
 exportgen0  0 CMASNEW datag1/candidatesMG_
 logmsg2 "      ----- Executing Step $JSTEP/$XSTEP $PROGID -----"
 #3----------------------------------------------------------------------
 cat $CMASNEW     #<-- display candidates updated Master file
 #4----------------------------------------------------------------------
 testcc $? $SMAX
 #8======================================================================
 S9000=A
 jobend51 #move any new GDG files from jobtmp to intended outdirs
 logmsg2 "JobEnd=Normal, JCC=$JCC, StepsExecuted=$XSTEP, LastStep=$JSTEP"
 exit $JCC

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9R2. Election3 demos - using JCL conversion Features in your scripts

demo RESTART at any step


 #0a. Logon userxx --> /home/userxx
 #0b. cdd  (alias 'cd $RUNDATA) --> /home/userxx/demo/election3/
      ===

 #1. joblog upcandimg1 date=20200229 <-- normal run (starts at step1)
     ===============================
      - console log already shown for normal run (page '9P3')

 #2. joblog upcandimg1 date=20200229 step=S0020  <-- to RESTART at step2
     ==========================================

joblog RESTARTing at step2

      200315:153627:upcandimg3: WARN: **START** at start=S0020
      200315:153627:upcandimg3: Begin Job=upcandimg3
      200315:153627:upcandimg3: HOSTNAME=uvsoft5 LOGNAME=userxx
      200315:153627:upcandimg3: ProcessID=7287 RUNDATE=20200315
      200315:153627:upcandimg3: RUNLIBS=/home/userxx/demo/election3
      200315:153627:upcandimg3: RUNDATA=/home/userxx/demo/election3
      200315:153627:upcandimg3: Arguments: upcandimg3 step=S0020
      200315:153627:upcandimg3: gen0 CMASNEW=datag1/candidatesMG_000001 insize=4K
      200315:153627:upcandimg3:       ----- Executing Step S0020/1 cat -----
      #1 candidatesM - master file 2020 election (uvsoftware.ca/uvdemos.htm#Part_9)
      #2 Name   Age  City     State  Month   Poll% Delegs Month$ Total$ (millions)
      #3       1         2         3         4         5         6         7         8
      #4345678901234567890123456789012345678901234567890123456789012345678901234567890
      Biden      77  Wilmington  DE  20200131   27      0      0     61
      Bloomberg  77  New York    NY  20200131    7      0      0    200
      Buttigieg  38  South Bend  IN  20200131    8      0      0     76
      Klobuchar  59  Minneapolis MN  20200131    4      0      0     29
      Sanders    78  Burlington  VT  20200131   22      0      0    108
      Trump      74  New York    NY  20200131   44  2,472      0    150
      Warren     70  Cambridge   MA  20200131   15      0      0     82
      200315:153627:upcandimg3: CC = 0
      200315:153627:UPCANDIMG3: Job Times: Begun=15:36:27 NormalEnd=15:36:27 Elapsed=00:00:00
      200315:153627:upcandimg3: JobEnd=Normal, JCC=0, StepsExecuted=1, LastStep=S0020

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9S1. LISTINGs of JCL/scripts, Election3 demos, Advanced Features

script BEFORE adding Advanced Features

 #!/bin/bash
 # loadcandidatesMG1 - script calling uvcp to load GDG Master (1st generation)
 #                   - demo by Owen Townsend, UVsoftware, Mar/2020
 #*loadcandidatesMG1 - simple script (not using exportfile & exportgen1)
 # loadcandidatesMG2 - alternate script using exportfile & exportgen1 to define the files
 #                   - prior to uvcp referencing via $SYMBOLS
 #
 uvcp "fili1=data1/candidatesM,rcs=80,typ=LST,filo1=datag1/candidatesMG_000001,rcs=80,typ=LST"
 #============================================================================================
 exit
 #Note - 1st generation of the GDG file is explicitly defined as 'datag1/candidatesMG_000001'
 # - alternate script loadcandidatesMG2 defines as just 'datag1/candidatesMG_'
 # - omiting generation# since function 'exportgen1' will determine based on existing files
 # - would default to 000001 if no matching filename exists

Script ABOVE is much simpler, but you have to specify exact generation#s

Script BELOW requires more code, but you do not need to specify exact genertion#s since the exportgen1 determines the gen# based on existing generations. If none existing it will use '_000001'.

script AFTER adding Advanced Features

 #!/bin/ksh
 # loadcandidatesMG2 - script calling uvcp to load GDG Master (1st generation)
 #                   - demo by Owen Townsend, UVsoftware, Mar/2020
 export   JOBIDX=$(basename $0) JOBID=${JOBIDX%.*} scriptpath="$0" args="$*"
 integer  JCC=0 SCC=0 LCC=0 SMAX=0 # init step status return codes
 autoload jobset51 jobend51 jobabend51 logmsg1 logmsg2 stepctl51 testcc evalcp1
 autoload exportfile exportgen0 exportgen1 exportgen2 exportgenall exportgenx
 jobset51 "$args"  # call function for JCL/script initialization
 goto     #optional restart at any step, goto step default S0000=A in jobset51
 S0000=A
 #================== Load 1st generation GDG Master file ================
 S0010=A
 export JSTEP=S0010; ((XSTEP+=1)); SCC=0; SMAX=4;
 exportfile    CANDIDATESM  data1/candidatesM
 exportgen1 +1 CANDIDATESMG datag1/candidatesMG_
 logmsg2 "Executing--> uvcp to load 1st gen GDG candidates Master file"
 #3-------------------------------------------------------------------------
 uvcp "fili1=$CANDIDATESM,rcs=80,typ=LST,filo1=$CANDIDATESMG,rcs=80,typ=LST"
 #4-------------------------------------------------------------------------
 testcc $? $SMAX
 #8======================================================================
 S9000=A
 jobend51 #move any new GDG files from jobtmp to intended outdirs
 logmsg2 "JobEnd=Normal, JCC=$JCC, StepsExecuted=$XSTEP, LastStep=$JSTEP"
 exit $JCC

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9T1. Election3 demos - using JCL conversion Features in your scripts

Benefits of JCL conversion scripts

  1. GDG files provided by functions exportgen0 & exportgen1

  2. 'testcc' function to test the status of each step with 1 line of code. Failure will display the status code & error messages & abort the job. 'testcc' failure ensures that any new generation GDG files are not committed.

  3. 'joblog' script captures console logs, for Example: --> joblog jobname <-- captures the log into $RUNDATA/joblog/jobname.log

  4. 'jobset51' function allows you to have multiple sets of Data because it changes directory to $RUNDATA (superdir for data subdirs/files). Your scripts can then be written to address files relative to the subdirs. Different programmers can have different sets of test-date, since they can define RUNDATA in their .bash_profile, for example: --> export RUNDATA=$HOME/testdata/

  5. 'jobset51' allows you to run scripts from anywhere, because it changes directory to $RUNDATA, so it does not matter where you are when you run a script that is written to the JCL conversion standards.

  6. Restart at any step in your script: To use this feature you would insert a label at the begining of each step, for example 'S0010=A' for step1, 'S0020=A' for step2, etc. To restart at step2, you run the job as follows: --> joblog jobname STEP=S0020 <--

  7. KEYWORDS=VALUES on the command line are automatically available as $KEYWORD anywhere in your scripts, for example: --> joblog jobname DATE=20200229 <-- makes $DATE available in the script

  8. 'logmsg2' function (vs 'echo') captures critical messages into --> $HOME/$RUNLIBS/jobmsgs/yymmdd/yymmdd_jobname <-- subdirectoried by date & messages are time-stamped

  9. Korn shell 'autoload' typeset allows storing functions in a separate library using $FPATH. Here is the FPATH in the supplied common_profile_uv. --> export FPATH=$APPSADM/sfun/ <--

  10. $RUNLIBS/#RUNDATA allow multiple sets of test & production LIBRARIES (scripts & programs) for multiple sets of test & production DATA. - different sets of testdata for different programmers - never need to modify scripts for test or production
Note
  • A call to the 'jobset51' function must be inserted at the begining
    of scripts in which you use these powerful features.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

9T2. Election3 demos - using JCL conversion Features in your scripts

REQUIREMENTS to use Advanced Features

  1. Must use 'ksh' vs 'bash' for 'autoload' functions
      #!/bin/ksh   <-- 1st line of script for Korn shell if autoload functions used
      #!/bin/bash  <-- Bash shell OK if no autoload function required
  1. Insert 7 lines at begining of the script
       export   JOBIDX=$(basename $0) JOBID=${JOBIDX%.*} scriptpath="$0" args="$*"
       integer  JCC=0 SCC=0 LCC=0 SMAX=0 # init step status return codes
       autoload jobset51 jobend51 jobabend51 logmsg1 logmsg2 stepctl51 testcc evalcp1
       autoload exportfile exportgen0 exportgen1 exportgen2 exportgenall exportgenx
       jobset51 "$args"  # call function for JCL/script initialization
       goto     #optional restart at any step, goto step default S0000=A in jobset51
       S0000=A
  1. Insert 3 lines at begining of each step
      S0010=A
      export JSTEP=S0010; ((XSTEP+=1)); SCC=0; SMAX=4;
      export PROGID=jobnamexxx
  1. Assign $SYMBOLs for GDG files using functions exportgen0 & exportgen1
      exportgen0 - defines next generation for reading
      exportgen1 - defines next generation for writing
      exportfile - defines non-GDG files, could use just 'export'
                 - these 3 fucntions display filename on console log (& joblog)
  1. Use the $SYMBOLs on utilities (uvcp,uvsort,etc) or in programs (COBOL,uvcopy,JAVA) for the GDG files defined previously with exportgen0, exportgen1 (& exportfile)

  2. Insert --> testcc $? $SMAX <-- after each step execution (utility,program,shell command)
  1. Insert end of script lines --> S9000=A, jobend51, logms2, exit $JCC
      S9000=A
      jobend51 #move new GDG files from jobtmp/GDG/... back to intended outdir (datag1/...)
      logmsg2 "JobEnd=Normal, JCC=$JCC, StepsExecuted=$XSTEP, LastStep=$JSTEP"
      exit $JCC
  1. Must use the VU stub & common profiles (see '1B2'), most vital definitions:
      $UV - Vancouver Utilities homedir, $APPSADM homedir,
      $RUNLIBS & $RUNDATA - defined in stub profile for use in common_profile
      $PATH - unix search path for scripts & programs
      FPATH - directory for Korn shell functions (exportgen,jobset51,testcc,etc)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_11 UVdemos.doc - Vancouver Utilities for Unix/Linux users

Part 11 Games & Education - C programming


11A1. Introduction & Overview
- Games written in the "C" program language
- fun way to learn C programming
- binary integers, BigEnd/LittleEnd binary
- max values stored in binary shorts, ints, longs,& int128
- simple C programs to read/write data files

11B1. insult2 - insult generator
- generates insults using, using random numbers to select insult words
  from 3 columns of old English Shakespearean insult adjectives & nouns.
- original words from Karma&Dogma, The Vancouver Sun. Sat, May 31,1997 G12
- original program amuse.c by Larry Manulak
- contributed to the Vancouver Utilities, Owen Townsend, UV Software, 1997
- Now (April 2020) modified/renamed as insult2.c

11B2. Data files for the insult generator
- dat0/insults_Shakespeare - original insults Vancouver Sun 1997
- dat0/insults_Owen - alternate insults from Owen & Google
- dat1/insults - active default file used by insult2 program

11B2. Executing insult2
- copy/rename desired file to the active file
- insult2 10 - generate 10 insults (min 1, max 500)
- insult2 list - list the insult words file on the screen

11B3. Creating your own insults file
- edit your own insults file, save in dat0/... named appropriately
- copy/rename your dat0/insults_XX to dat1/insults (active file)
- insult2 10 - test your insults
- insult2 list - list your insults

11B4. insult2.c - source program for the insult generator
- an interesting program for young programmers learning C
- intersting example of generating random numbers

11C1. lotto1 - generate random numbers for lottery tickets
- generate specified number of random number in a range
  of any 2 numbers (lower & upper range).
- could be useful generating random numbers for real work applications

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part 11 Education - C programming


11D1. Educational C programs
- binary integers, BigEnd/LittleEnd binary
- max values stored in binary shorts, ints, longs,& int128
- simple C programs to convert numerics to integers & read/write data files
- prints binary progressions 1,2,4,8,etc to max possible in a long

11E1. test64a - Little-End binary increasing right to left (Intel)

11E2. test64b - Big-End binary increasing left to right (mainframe & COBOL comp4)

11E3. test64c - Little-End with hard-coded: cycles=26, initvalue=4096, multiplier=4

11G1. testmaxints - shows 'sizeof' integers: short, int, long,& int128

11H1. testio1 - simple C program to illustrate read/write I/O

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

11A1. Games & Education - Overview

Games written in C

Part11 is a fun way to help you learn to program in the C language. I was inspired to add this for my grandson - perhaps you have grandchildren who would be interested.

I think they would enjoy the 'insult generator' that uses random numbers to generate insults using, using random numbers to select insult words from 3 columns of old English Shakespearean insult adjectives & nouns.

'lotto1.c' is another C program generating random numbers for lotteries, but could also be sued to generate random numbers for useful work.

Education - Integers, Hexadecimal, conversions

We present several more C programs to illustrate the binary coding (in hexadecimal) & maximum capacities for integers (short, int, long,& int128) for both Big-End & Little-End computers.

Note that the compile & operating instrucitons for these demo C programs are included as /*---comments---*/ at the begining of the program, so we do not need separate documentation for the simpler programs.

Books to learn C programming

C Programming Language (2nd edition) - Kernighan & Ritchie

C Proramming for Absolute Beginners (3rd edition) - Perry & Miller

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

11B1. Games & Education - C programming

insult2 - Insult Generator

'insult2 is a C program to generate insults using, using random numbers to select insult words from 3 columns of old English Shakespearean insult adjectives & nouns.

sample insults file

 001 # dat0/insults_Shakespeare - data file for $UV/src/games/insult2.c
 002 # dat1/insults <-- must copy/rename from dat0/insults_Shakepeare
 003 #                - see www.uvsoftware.ca/uvdemos.htm#Part_11
 004 # - insult2 reads insults from subdir/file = dat1/insults
 005 # - allows multiple files, in dat0/insultsA,B,C,etc, copy 1 to dat1/insults
 006 # - seq# cols 1-3, 3 insult words, 25 bytes each, at columns 6,31,56
 007 # - insult words must start on line 11, preceded by 10 lines of #comments
 008 #
 009 #    1         2         3         4         5         6         7         8
 010 #678901234567890123456789012345678901234567890123456789012345678901234567890
 011  artless                  base-court               apple-john
 012  bawdy                    bat-fowling              baggage
 013  beslubbering             beef-witted              barnacle
 014  bootless                 beetle-headed            bladder
 015  churlish                 boil-brained             boar-pig
 016  cockered                 clapper-clawed           bugbear
 017  clouted                  clay-brained             bum-bailey
 018  craven                   common-kissing           canker-blossom
 019  currish                  crook-pated              clack-dish
 020  dissembling              dizzy-eyed               coxcomb
          -------------- 39 lines not shown ----------------

          insult2 10    <-- sample command & output generated
          ==========
                   Thou  fawning,  half-faced,  measle!
                   Thou  puking,  pox-marked,  wagtail!
                   Thou  jarring,  onion-eyed,  clack-dish!
                   Thou  dissembling,  plume-plucked,  baggage!
                   Thou  spongy,  shard-borne,  bugbear!
                   Thou  paunchy,  fat-kidneyed,  mumble-news!
                   Thou  roguish,  dizzy-eyed,  bugbear!
                   Thou  mangled,  hasty-witted,  pignut!
                   Thou  goatish,  fen-sucked,  varlet!
                   Thou  fobbing,  pottle-deep,  baggage!

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

11B2. insult2 - Insult Generator

Investigate Relevant files


 #1. Login userxx --> /home/userxx
     ============

 #2. l dat0       <-- list permanent files stored in dat0/...
     ======
     --> dat0/insults_Shakespeare - original insults Vancouver Sun 1997
     --> dat0/insults_Owen        - alternate insults from Owen & Google

 #3. l dat1       <-- list active files stored in dat1/...
     ======
     --> dat1/insults   <-- active default file used by insult2 program
Note
  • dat0 saves multiple versions of insult files named approriately
  • insults_Shakespeare & insults_Owen distributed with VU
  • You can make up additional insult files & save in dat0/...
  • can copy your file from dat0/insults_XX to dat1/insults
    to make it the default insults file used by the insult2 program

Executing insult2


 #1. cp dat0/insults_Shakespeare dat1/insults
     ========================================
      - copy/rename desired file to the active file

 #2. insult2 5    <-- generate 5 insults (arg1 min 1, max 500)
     =========      - must specify number to generate as arg1
      Thou cockered, pottle-deep, dewberry!
      Thou ruttish, onion-eyed, skainsmate!
      Thou rank, beetle-headed, mumble-news!
      Thou gleeking, base-court, skainsmate!
      Thou venomed, boil-brained, barnacle!

 #3. insult2 list  <-- list the insult words file on the screen
     ============    - must code arg1 as "list"
      001 # dat0/insults_Shakespeare - data file for $UV/src/games/insult2.c
      002 # dat1/insults <-- must copy/rename from dat0/insults_Shakepeare
      003 #                - see www.uvsoftware.ca/uvdemos.htm#Part_11
      004 # - insult2 reads insults from subdir/file = dat1/insults
      005 # - allows multiple files, in dat0/insultsA,B,C,etc, copy 1 to dat1/insults
      006 # - seq# cols 1-3, 3 insult words, 25 bytes each, at columns 6,31,56
      007 # - insult words must start on line 11, preceded by 10 lines of #comments
      008 #
      009 #   1         2         3         4         5         6         7         8
      010 #678901234567890123456789012345678901234567890123456789012345678901234567890
      011  artless                  base-court               apple-john
      012  bawdy                    bat-fowling              baggage
      013  beslubbering             beef-witted              barnacle
                  ----------- lines 014-058 not shown ------------
      059  yeasty                   weather-bitten           wagtail

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

11B3. insult2 - Insult Generator

Creating your own insults file

Note
  • The 1st 10 lines must be the same for all inserts files
  • see sample listed on page above
  • create new file using insult2 "list" option redirect to dat0/...
  • then edit, removing all except 1st 10 lines
  • add your own insult words from line 11
  • be sure to code in same columns as the demo files
  • seq# cols 1-3, 3 insult words, 25 bytes each, at columns 6,31,56

 #1. insult2 list >dat0/insults_XX  <-- list to a file to create new
     =============================    - makeup new name approriate

 #2. vi dat0/insults_XX   <-- edit as desired (must retain 1st 10 lines)
     ==================     - but change name & #comment as desired
      - makeup your insult words (using language of your peers, etc)

 #3. cp dat0/insults_XX dat1/insults  <-- copy/rename to active file
     ===============================

 #4. insult2 10    <-- test your insults
     ==========

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

11B4. insult2 - Insult Generator

insult2 - C Source program listing

 /* insult2.c  <-- modified version of original amuse.c from Larry Manulak  */
 /*              - original by Larry Manulak, lmanulak@shaw.ca, Vancouver   */
 /*              - contributed to Owen Townsend, UV Software, 1997          */
 /*              - Now (April 2020) modified/renamed as insult2.c           */
 /*                                                                         */
 /* Construct a Shakespearean insult by combining one word from each column */
 /* source: Karma & Dogma, The Vancouver Sun. Saturday, May 31, 1997. p G12 */
 /*                                                                         */
 /* See documentation at www.uvsoftware.ca/uvdemos.htm#Part_11 (games)      */
 /* - page '11B1' documents generating insults using insult2                */
 /*   from 3 columns of Shakespearean insults using random numbers          */
 /*                                                                         */
 /* Also see '11C1' documents using random2.c to generate lotto649 numbers  */
 /* - similar to the random number genration in this 'insult2.c' program    */
 /*                                                                         */
 /*                  ** update history April 2020 **                        */
 /*                                                                         */
 /* insult1 - original amuse.c copied/renamed as insult1.c & modified to:   */
 /*         - use timeofday microseconds as seed req'd to gen multi insults */
 /*         - accept arg1 no of insults to generate                         */
 /*         - or accept arg1 "list" to list the 3 columns of insult words   */
 /*         - combined the 3 columns of words into 1 3 column table         */
 /*         - more compact listing, easier to add new insult words          */
 /*           (but would need to be somewhat familiar with C programming)   */
 /*                                                                         */
 /**insult2 - This version reads table of insult words from a file          */
 /*         - insults file must be stored in $UV/demo/dat1/insults          */
 /*         - allows users to make-up their own insult files                */
 /*         - suggest storing alternates with appropriate alternate names   */
 /*           & copy/rename desired alternate to dat1/insults               */
 /*                                                                         */

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 /*                        ** sample insults file **                                 */
 /*                                                                                  */
 /* 001 # dat0/insults_Shakespeare - data file for $UV/src/games/insult2.c           */
 /* 002 # dat1/insults <-- must copy/rename from dat0/insults_Shakepeare             */
 /* 003 #                - see www.uvsoftware.ca/uvdemos.htm#Part_11                 */
 /* 004 # - insult2 reads insults from subdir/file = dat1/insults                    */
 /* 005 # - allows multiple files, in dat0/insultsA,B,C,etc, copy 1 to dat1/insults  */
 /* 006 # - seq# cols 1-3, 3 insult words, 25 bytes each, at columns 6,31,56         */
 /* 007 # - insult words must start on line 11, preceded by 10 lines of #comments    */
 /* 008 #                                                                            */
 /* 009 #    1         2         3         4         5         6         7         8 */
 /* 010 #678901234567890123456789012345678901234567890123456789012345678901234567890 */
 /* 011  artless                  base-court               apple-john                */
 /* 012  bawdy                    bat-fowling              baggage                   */
 /* 013  beslubbering             beef-witted              barnacle                  */
 /* 014  bootless                 beetle-headed            bladder                   */
 /* 015  churlish                 boil-brained             boar-pig                  */
 /* 016  cockered                 clapper-clawed           bugbear                   */
 /* 017  clouted                  clay-brained             bum-bailey                */
 /* 018  craven                   common-kissing           canker-blossom            */
 /* 019  currish                  crook-pated              clack-dish                */
 /* 020  dissembling              dizzy-eyed               coxcomb                   */
 /* 021  droning                  doghearted               codpiece                  */
 /* 022  errant                   dread-bolted             death-token               */
 /* 023  fawning                  earth-vexing             dewberry                  */
 /* 024  fobbing                  elf-skinned              lap-dragon                */
 /* 025  froward                  fat-kidneyed             flax-wench                */
 /* 026  frothy                   fen-sucked               flirt-gill                */
 /* 027  gleeking                 flap-mouthed             foot-licker               */
 /* 028  goatish                  fly-bitten               fustilarian               */
 /* 029  gorbellied               folly-fallen             giglet                    */
 /* 030  impertinent              fool-born                gudgeon                   */
 /*           -------------- 29 lines not shown ----------------                     */
 /*                                                                                  */
 /*         insult2 10    <-- sample command & output generated                      */
 /*         ==========                                                               */
 /*                                                                                  */
 /*                  Thou  fawning,  half-faced,  measle!                            */
 /*                  Thou  puking,  pox-marked,  wagtail!                            */
 /*                  Thou  jarring,  onion-eyed,  clack-dish!                        */
 /*                  Thou  dissembling,  plume-plucked,  baggage!                    */
 /*                  Thou  spongy,  shard-borne,  bugbear!                           */
 /*                  Thou  paunchy,  fat-kidneyed,  mumble-news!                     */
 /*                  Thou  roguish,  dizzy-eyed,  bugbear!                           */
 /*                  Thou  mangled,  hasty-witted,  pignut!                          */
 /*                  Thou  goatish,  fen-sucked,  varlet!                            */
 /*                  Thou  fobbing,  pottle-deep,  baggage!                          */
 /*                                                                                  */

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 /*                     ** Operating Instructions **                        */
 /*                                                                         */
 /* Assumptions:                                                            */
 /* - logged in as uvadm in homedir /home/uvadm/...                         */
 /* - relevant subdirs/files:                                               */
 /*   tf/insults_Shakespeare <-- original insults from Vancouver Sun 1997   */
 /*   tf/insults_Owen        <-- alternate insults from Owen & Google       */
 /*   dat1/insults      <-- active default file used by insult2 program     */
 /*                       - copied/renamed from insult files in tf/...      */
 /*                       - you can make up additional insult files         */
 /*                                                                         */
 /* #1. cp tf/insults_Shakespeare dat1/insults                              */
 /*     ======================================                              */
 /*      - copy/rename desired file to the active file                      */
 /*                                                                         */
 /* #2. insult2 10    <-- generate 10 insults (arg1 min 1, max 500)         */
 /*     ==========                                                          */
 /*                                                                         */
 /* #3. insult2 list  <-- list the insult words on the screen               */
 /*     ============                                                        */
 /*                                                                         */
 /* #4. insult2 list >tf/insults_XX  <-- list to a file to create new       */
 /*     ===========================    - makeup new name approriate         */
 /*                                                                         */
 /* #5. vi tf/insults_XX     <-- edit as desired (must retain 1st 10 lines) */
 /*     ================       - but change name & #comment as desired      */
 /*      - makeup your insult words (using language of your peers, etc)     */
 /*                                                                         */
 /* #6. cp tf/insults_XX dat1/insults    <-- copy/rename to active file     */
 /*     =============================                                       */
 /*                                                                         */
 /* #7. insult2 10    <-- test your insults                                 */
 /*     ==========                                                          */
 /*                                                                         */
 /*Note - see doc at www.uvsoftware.ca/uvdemos.htm#Part_11                  */
 /*     - runs the demo in /home/userxx (vs /home/uvadm above)              */
 /*     - master insults files stored in dat0/... vs tf/... shown above     */
 /*   dat0/insults_Shakespeare <-- original insults Vancouver Sun 1997      */
 /*   dat0/insults_Owen        <-- alternate insults from Owen & Google     */
 /*   dat1/insults             <-- copy desired file to dat1/...            */
 /*                                                                         */
 /*                     ** compile program **                               */
 /*                                                                         */
 /* Assumptions:                                                            */
 /* - logged in as uvadm --> homedir /home/uvadm                            */
 /* - src/games/insult2.c   <-- source code to be compiled                  */
 /* - bin/insult            <-- compiler output                             */
 /*                                                                         */
 /* cc src/games/insult2.c -o bin/insult2   <-- compile source to binary    */
 /* =====================================                                   */

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

#include <stdio.h> #include <stdlib.h> #include <string.h> #include <unistd.h> #include <time.h> #include <sys/time.h>

/* insults table - 500 max entries of 100 bytes each max */ char insults[500][100]; /* - will be loaded from file ./dat1/insults */

 /* file controls to read insult words from a file            */
 /* - must be dat1/insults (subdir dat1/ in current dirctory) */
 char fil1rec[256];       /* input record area                */
 char fil1name[64] = "dat1/insults"; /* input filename        */
 FILE *fil1ptr;           /* file pointer from fopen          */
 int fil1recs;            /* record count                     */
 int cc;                  /* current fgetc return char/err/EOF*/
 int arg1i;               /* number of insults to generate    */
 char arg1c[16];          /* "list" to list 3 columns of words*/
 char first[8] = "Thou";
 int tt,rn1,rn2,rn3;
 int words;               /* number of insults in insult table*/
 char word1[40];
 char word2[40];
 char word3[40];
 struct timeval tv;       /* for gettimeofday(&tv,0); below  */
 /* showing timeval here as comments (actual in sys/time.h)  */
 /* struct timeval                                           */
 /* { time_t tv_sec;                                         */
 /*  suseconds_t tv_usec;                                    */
 /* };                                                       */
 suseconds_t micros1;    /* from struct timeval microseconds */

/* subfunction prototypes */ int getrec1(char *fil1rec,FILE *fil1ptr); int storeinsults(void); void shorten(char *word); void listinsults(void);

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

/* mainline */ int main (int argc, char **argv) { /* verify 1 argument specified */ if (argc != 2) { printf("One of two arguments must be specified, either of:\n"); printf("- number of insults to generate (1 to 500 max)\n"); printf("- or \"list\" to list the 3 columns of insult words\n"); exit(91); }

/* open the insults file & store words on table */ if ((fil1ptr = fopen(fil1name,"rb")) == (FILE*)0) { printf("cant open input file %s \n",fil1name); exit(92); } /* call subrtn to get records & store in insults table */ storeinsults();

 /* calc no of 3 word groups in the insults table */
 for (tt=0; tt < 500; tt++)
   { if (!(insults[tt][0]))
       { break;
       }
   }
 words = (tt - 10);
 /* Error check user file */
 if ((tt < 20) || (memcmp(insults[9],"010 #",5) != 0) || (memcmp(insults[10],"011  ",5) != 0)
      || (insults[10][4] != ' ') || (insults[10][29] != ' ') || (insults[10][54] != ' ')
      || (insults[10][5] <= ' ') || (insults[10][30] <= ' ') || (insults[10][55] <= ' '))
   { printf("dat1/insults file format invalid, must be > 20 lines, seq# cols 1-3\n");
     printf("line 10 must be '#' in col 5, line 11 must be 1st insult (col 5 not '#'\n");
     printf("insult words must start in columns 6, 31,& 56\n");
     exit(93);
   }

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

/* get command line arguments */ /* "list" or number of insults to generate */ strcpy(arg1c,argv[1]); if (strcmp(arg1c,"list") == 0) { listinsults(); exit(0); }

/* get no of insults to generate (1 to 500 max) */ arg1i = atoi(argv[1]); if (arg1i < 1) { arg1i = 1; } else if (arg1i > 500) { arg1i = 500; }

 /* get seed for rand() function */
 /* - use gettimeofday microseconds for srand seed*/
 gettimeofday(&tv,0);
 micros1 = tv.tv_usec;          /* store local for debug */
 srand((unsigned) tv.tv_usec);  /* get seed using TOD microsecs*/
 /* begin loop to generate arg1i number of insults */
 for (tt=0; tt< arg1i; tt++)
   { /* random gives an integer between 0 and 2^31 - 1       */
     /* scale this to no of entries in each of 3 word tables */
     rn1 = ((random() % words) + 10);
     rn2 = ((random() % words) + 10);
     rn3 = ((random() % words) + 10);
     strncpy(word1,insults[rn1]+05,25);
     strncpy(word2,insults[rn2]+30,25);
     strncpy(word3,insults[rn3]+55,25);
     shorten(word1);
     shorten(word2);
     shorten(word3);
     printf ("%s  %s,  %s,  %s!\n", first, word1, word2, word3);
   }
 }   /* end of main */

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 /* subfunctions */
 /*---------------------- storeinsults ------------------------*/
 /* storeinsults() - read insults file, store records in table */
 int storeinsults(void)
 { int gg=0;
   tt=0;
 while(1)
   { gg = getrec1(fil1rec,fil1ptr);
     if (gg <= 0)
       { break;
       }
     strncpy(&insults[tt][0],fil1rec,100);
     tt++;
   }
 memset(insults[tt],'\0',200); /* mark EOT with nulls */
 }
 /*----------------------- getrec1 -------------------------*/
 /* getrec1() - get record from insults input file          */
 int getrec1(char *fil1rec, FILE *fil1ptr)
 {
 int jj;
 memset(fil1rec,'\0',256);
 /* begin loop get each byte until LF EOR or EOF or ERROR   */
 for (jj=0; jj < 256; jj++)
   { cc = fgetc(fil1ptr);             /* get next character */
     if ((cc == '\n') || (cc < '\n')) /* EOR or EOF ? */
       { break;
       }
     fil1rec[jj] = cc;          /* store current char */
   }
 fil1recs++;                    /* count records      */
 /* reached LF or EOF (jj <= 0 when no more data)     */
 return(jj);
 }
 /*--------------------- listinsults -----------------------*/
 /* listinsults() - list the 3 columns of insult words      */
 void listinsults(void)
 { tt = 0;
   for(tt=0; insults[tt][0] && tt< 500; tt++)
     { printf ("%03d %s\n",tt+1,insults[tt]+4);
     }
 }
 /*------------------------ shorten ---------------------------*/
 /* shorten() - insert null after righthand nonblank character */
 void shorten(char *word)
 { int ii;
   for ( ii = 19; (ii > 0 && word[ii] <= ' '); ii-- )
     { ;
     }
   word[ii+1] = '\0';
 }
 /*------------------- end of insult2.c ----------------------*/

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

11C1. Games & Education - C programming

lotto1 - generate random#s for lotteries, etc

 /* lotto1.c - C program to generate random numbers         */
 /*          - by Owen Townsend, UV Software, March 2020    */
 /*          - source code stored in $UV/src/lotto1.c       */
 /*                                                         */
 /* lotto1.c - copied/renamed from $UV/testC/random2.c      */
 /*          - generate numbers for lottery tickets         */
 /*          - example below for Lotto 649                  */
 /*          - could use for many lotteries                 */
 /*          - could use for non-frivolous applications     */
 /*                                                         */
 /* cc src/testC/lotto1.c -obin/lotto1  <-- compile         */
 /* ====================================                    */
 /*                                                         */
 /* Execution example:                                      */
 /* ==================                                      */
 /*  - generate 6 random numbers between 1 & 49 (inclusive) */
 /*                                                         */
 /* lotto1 6 1 49    <-- example (for Lotto 649)            */
 /* =============      - output shown below:                */
 /* 44                                                      */
 /* 10                                                      */
 /* 42                                                      */
 /* 33                                                      */
 /* 30                                                      */
 /* 35                                                      */
 /*                                                         */
 /* See doc at www.uvsoftware.ca/uvdemos.htm#Part_11 (games)*/
 /* - page '11C1' documents using lotto1.c to generate      */
 /*   lotto649 numbers (same as you see above)              */

#include <stdio.h> #include <stdlib.h> #include <time.h> #include <sys/time.h>

 int cycles;       /* arg#1 - random numbers desired         */
 long low;         /* arg#2 - low raange                     */
 long high;        /* arg#3 - high range                     */
 int count = 0;    /* counter up to cycles random#s desired  */
 time_t timeseed;  /* seed for srand()                       */
 long rand1;       /* current random# generated              */
 struct timeval tv;       /* for gettimeofday(&tv,0); below  */
 /* showing timeval here as comments (actual in sys/time.h)  */
 /* struct timeval         */
 /* { time_t tv_sec;       */
 /*  suseconds_t tv_usec;  */
 /* };                     */
 suseconds_t micros1;  /* from struct timeval suseconds_t tv_usec */
 long        micros2;  /* copied to long check suseconds_t 8 bytes*/

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 /* mainline for lotto1 */
 int main(int argc, char *argv[])
 {
 /* verify operator entered correct number of arguments   */
 if (argc != 4)
   { printf("--> lotto1 cycles low high <-- ***ERROR***, 3 numeric arguments expected \n");
     printf("--> lotto1   6     1   49  <-- example for Lotto 649\n");
     exit(91);
   }
 /* capture command line arguments */
 cycles = atoi(argv[1]);   /* arg1 random#s desired */
 low    = atol(argv[2]);   /* arg2 low range        */
 high   = atol(argv[3]);   /* arg3 high range       */
 /* get seed for rand() function - using current time */
 /* srand((unsigned) time(&timeseed));                */
 /* time_t seconds gets same seed unless wait 1+ secs */
 /* - use gettimeofday get microseconds for srand seed*/
 gettimeofday(&tv,0);
 micros1 = tv.tv_usec;          /* store local for debug */
 micros2 = tv.tv_usec;          /* also as long for debug*/
 srand((unsigned) tv.tv_usec);  /* get seed using TOD microsecs*/

/* verify low < high */ if (low >= high) { printf("ERROR: low range > highrange\n"); exit(92); }

 /* begin loop to calc & print results */
 while (count < cycles)
   {  rand1 = (rand() % high);
      if (rand1 < low)
        { continue;
        }
      count++;       /* count up to cycles */
      printf("%ld\n",rand1);
   }
 exit(0);
 }
 /*--------------------- end of lotto1.c ------------------*/

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

11D1. Games & Education - C programming

Educational C programs


11E1. test64a - Little-End binary increasing right to left (Intel)

11E2. test64b - Big-End binary increasing left to right (mainframe & COBOL comp4)

11E3. test64c - Little-End with hard-coded: cycles=26, initvalue=4096, multiplier=4

11G1. testmaxints - shows 'sizeof' integers: short, int, long,& int128

11H1. testio1 - simple C program to illustrate read/write I/O

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

11E1. Educational C programs - test64a.c

 /* test64a.c - test long integer format & max for unix/linux/windows C programs  */
 /*           - by Owen Townsend, UV Software, original 2010, updated 2020        */
 /* convert arg1 numstring to long, multiply by arg2, repeat arg3 times,& print   */
 /*                                                                               */
 /*    -------------------- 3 versions of this program ----------------------     */
 /* test64a * Little-End binary increasing right to left (Intel)                  */
 /* test64b - Big-End    binary increasing left to right (mainframe & COBOL comp4)*/
 /* test64c - Little-End with hard-coded: cycles=26, initvalue=4096, multiplier=4 */
 /*                                                                               */
 /* gcc src/testC/test64a.c -obin/test64a  <-- compile                            */
 /* =====================================                                         */
 /*                                                                               */
 /* test64a cycles initvalue multiplier    <-- command format                     */
 /* ===================================                                           */
 /*                                                                               */
 /* test64a 65 1 2     <-- example #1                                             */
 /* ==============       - expected result below          ----> Little End -----> */
 /*                                                                               */
 /* 00 00000000000000000001 * 02 = 00000000000000000002 = 02 00 00 00 00 00 00 00 */
 /* 01 00000000000000000002 * 02 = 00000000000000000004 = 04 00 00 00 00 00 00 00 */
 /* 02 00000000000000000004 * 02 = 00000000000000000008 = 08 00 00 00 00 00 00 00 */
 /* 03 00000000000000000008 * 02 = 00000000000000000016 = 10 00 00 00 00 00 00 00 */
 /* 04 00000000000000000016 * 02 = 00000000000000000032 = 20 00 00 00 00 00 00 00 */
 /* 05 00000000000000000032 * 02 = 00000000000000000064 = 40 00 00 00 00 00 00 00 */
 /* 06 00000000000000000064 * 02 = 00000000000000000128 = 80 00 00 00 00 00 00 00 */
 /* 07 00000000000000000128 * 02 = 00000000000000000256 = 00 01 00 00 00 00 00 00 */
 /* 08 00000000000000000256 * 02 = 00000000000000000512 = 00 02 00 00 00 00 00 00 */
 /* 09 00000000000000000512 * 02 = 00000000000000001024 = 00 04 00 00 00 00 00 00 */
 /* 10 00000000000000001024 * 02 = 00000000000000002048 = 00 08 00 00 00 00 00 00 */
 /* 11 00000000000000002048 * 02 = 00000000000000004096 = 00 10 00 00 00 00 00 00 */
 /* 12 00000000000000004096 * 02 = 00000000000000008192 = 00 20 00 00 00 00 00 00 */
 /*       ----------------------- 50 lines omitted --------------------------     */
 /* 61 02305843009213693952 * 02 = 04611686018427387904 = 00 00 00 00 00 00 00 40 */
 /* 62 04611686018427387904 * 02 = -9223372036854775808 = 00 00 00 00 00 00 00 80 */
 /* 63 -9223372036854775808 * 02 = 00000000000000000000 = 00 00 00 00 00 00 00 00 */
 /*                                                                               */
 /* test64a 17 1 16      <-- example #2                                           */
 /* ===============        - expected result below        ----> Little End -----> */
 /*                                                                               */
 /* 00 00000000000000000001 * 16 = 00000000000000000016 = 10 00 00 00 00 00 00 00 */
 /* 01 00000000000000000016 * 16 = 00000000000000000256 = 00 01 00 00 00 00 00 00 */
 /* 02 00000000000000000256 * 16 = 00000000000000004096 = 00 10 00 00 00 00 00 00 */
 /* 03 00000000000000004096 * 16 = 00000000000000065536 = 00 00 01 00 00 00 00 00 */
 /* 04 00000000000000065536 * 16 = 00000000000001048576 = 00 00 10 00 00 00 00 00 */
 /* 05 00000000000001048576 * 16 = 00000000000016777216 = 00 00 00 01 00 00 00 00 */
 /* 06 00000000000016777216 * 16 = 00000000000268435456 = 00 00 00 10 00 00 00 00 */
 /* 07 00000000000268435456 * 16 = 00000000004294967296 = 00 00 00 00 01 00 00 00 */
 /* 08 00000000004294967296 * 16 = 00000000068719476736 = 00 00 00 00 10 00 00 00 */
 /* 09 00000000068719476736 * 16 = 00000001099511627776 = 00 00 00 00 00 01 00 00 */
 /* 10 00000001099511627776 * 16 = 00000017592186044416 = 00 00 00 00 00 10 00 00 */
 /* 11 00000017592186044416 * 16 = 00000281474976710656 = 00 00 00 00 00 00 01 00 */
 /* 12 00000281474976710656 * 16 = 00004503599627370496 = 00 00 00 00 00 00 10 00 */
 /* 13 00004503599627370496 * 16 = 00072057594037927936 = 00 00 00 00 00 00 00 01 */
 /* 14 00072057594037927936 * 16 = 01152921504606846976 = 00 00 00 00 00 00 00 10 */
 /* 15 01152921504606846976 * 16 = 00000000000000000000 = 00 00 00 00 00 00 00 00 */

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

#include <stdio.h> #include <stdlib.h>

 int cycles;                  /* arg#1 - no of calc cycles ex: 15   */
 long value1;                 /* arg#2 - initial value     ex: 4096 */
 int mult;                    /* arg#3 - multiplier        ex: 4    */
 long value2;                 /* save current value for printing    */
 int ii;                      /* loop index                         */

/* union to hold long & 8 byte char array in same memory positions */ /* - 8 byte character array (for hex representation conversion) */ union LCH { long l; unsigned char ch[8]; } lch;

 /* mainline for test64a */
 int main(int argc, char *argv[])
 {
 /* verify operator entered correct number of arguments */
 if (argc != 4)
   { printf("usage: test64a cycles init-value multiplier \n");
     printf("example: test64a  26     4096       4  \n");
     exit(9);
   }
 cycles = atoi(argv[1]);         /* arg1 cycles to int                 */
 /* value1 = strtol(argv[2],(char**)0,10); <-- strtol/strtoll obsolete */
 value1 = atol(argv[2]);         /* use atol convert initvalue to long */
 mult   = atoi(argv[3]);         /* arg3 multiplier to int             */
 /* begin loop to calc & print results */
 for (ii=0; ii < cycles; ii++)
   { value2 = value1;           /* save current value for printing     */
     value1 *= mult;            /* multiply value by mplr              */
     lch.l = value1;            /* store value in union for hex cnvrt  */
 printf("%02d %020ld * %02d = %020ld = %02X %02X %02X %02X %02X %02X %02X %02X\n"
     ,ii,value2,mult,value1
     ,lch.ch[0],lch.ch[1],lch.ch[2],lch.ch[3]
     ,lch.ch[4],lch.ch[5],lch.ch[6],lch.ch[7]);
   }
 return(0);
 }
 /*Note - type long (64 bits) holds 19 digit values, but not 20 digits            */
 /*     - there is type '__int128', but no conversion format for __int128         */
 /*     - '%ld' converts longs to strings, BUT NO '%__int128' to convert 128 bits */

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

11E2. Educational C programs - test64b.c

 /* test64b.c - test long integer format & max for unix/linux/windows C programs  */
 /*           - by Owen Townsend, UV Software, original 2010, updated 2020        */
 /* convert arg1 numstring to long, multiply by arg2, repeat arg3 times,& print   */
 /*                                                                               */
 /*    -------------------- 3 versions of this program ----------------------     */
 /* test64a - Little-End binary increasing right to left (Intel)                  */
 /* test64b * Big-End    binary increasing left to right (mainframe & COBOL comp4)*/
 /* test64c - Little-End with hard-coded: cycles=26, initvalue=4096, multiplier=4 */
 /*                                                                               */
 /* gcc src/testC/test64b.c -obin/test64b  <-- compile                            */
 /* =====================================                                         */
 /*                                                                               */
 /* test64b cycles initvalue multiplier    <-- command format                     */
 /* ===================================                                           */
 /*                                                                               */
 /* test64a 65 1 2     <-- example #1                                             */
 /* ==============       - expected result below          <---- Big-End <-------- */
 /*                                                                               */
 /* 00 00000000000000000001 * 02 = 00000000000000000002 = 00 00 00 00 00 00 00 02 */
 /* 01 00000000000000000002 * 02 = 00000000000000000004 = 00 00 00 00 00 00 00 04 */
 /* 02 00000000000000000004 * 02 = 00000000000000000008 = 00 00 00 00 00 00 00 08 */
 /* 03 00000000000000000008 * 02 = 00000000000000000016 = 00 00 00 00 00 00 00 10 */
 /* 04 00000000000000000016 * 02 = 00000000000000000032 = 00 00 00 00 00 00 00 20 */
 /* 05 00000000000000000032 * 02 = 00000000000000000064 = 00 00 00 00 00 00 00 40 */
 /* 06 00000000000000000064 * 02 = 00000000000000000128 = 00 00 00 00 00 00 00 80 */
 /* 07 00000000000000000128 * 02 = 00000000000000000256 = 00 00 00 00 00 00 01 00 */
 /* 08 00000000000000000256 * 02 = 00000000000000000512 = 00 00 00 00 00 00 02 00 */
 /* 09 00000000000000000512 * 02 = 00000000000000001024 = 00 00 00 00 00 00 04 00 */
 /* 10 00000000000000001024 * 02 = 00000000000000002048 = 00 00 00 00 00 00 08 00 */
 /* 11 00000000000000002048 * 02 = 00000000000000004096 = 00 00 00 00 00 00 10 00 */
 /* 12 00000000000000004096 * 02 = 00000000000000008192 = 00 00 00 00 00 00 20 00 */
 /*       ----------------------- 50 lines omitted --------------------------     */
 /* 61 02305843009213693952 * 02 = 04611686018427387904 = 40 00 00 00 00 00 00 00 */
 /* 62 04611686018427387904 * 02 = -9223372036854775808 = 80 00 00 00 00 00 00 00 */
 /* 63 -9223372036854775808 * 02 = 00000000000000000000 = 00 00 00 00 00 00 00 00 */
 /*                                                                               */
 /* test64a 17 1 16      <-- example #2                                           */
 /* ===============        - expected result below        <---- Big-End <-------- */
 /*                                                                               */
 /* 00 00000000000000000001 * 16 = 00000000000000000016 = 00 00 00 00 00 00 00 10 */
 /* 01 00000000000000000016 * 16 = 00000000000000000256 = 00 00 00 00 00 00 01 00 */
 /* 02 00000000000000000256 * 16 = 00000000000000004096 = 00 00 00 00 00 00 10 00 */
 /* 03 00000000000000004096 * 16 = 00000000000000065536 = 00 00 00 00 00 01 00 00 */
 /* 04 00000000000000065536 * 16 = 00000000000001048576 = 00 00 00 00 00 10 00 00 */
 /* 05 00000000000001048576 * 16 = 00000000000016777216 = 00 00 00 00 01 00 00 00 */
 /* 06 00000000000016777216 * 16 = 00000000000268435456 = 00 00 00 00 10 00 00 00 */
 /* 07 00000000000268435456 * 16 = 00000000004294967296 = 00 00 00 01 00 00 00 00 */
 /* 08 00000000004294967296 * 16 = 00000000068719476736 = 00 00 00 10 00 00 00 00 */
 /* 09 00000000068719476736 * 16 = 00000001099511627776 = 00 00 01 00 00 00 00 00 */
 /* 10 00000001099511627776 * 16 = 00000017592186044416 = 00 00 10 00 00 00 00 00 */
 /* 11 00000017592186044416 * 16 = 00000281474976710656 = 00 01 00 00 00 00 00 00 */
 /* 12 00000281474976710656 * 16 = 00004503599627370496 = 00 10 00 00 00 00 00 00 */
 /* 13 00004503599627370496 * 16 = 00072057594037927936 = 01 00 00 00 00 00 00 00 */
 /* 14 00072057594037927936 * 16 = 01152921504606846976 = 10 00 00 00 00 00 00 00 */
 /* 15 01152921504606846976 * 16 = 00000000000000000000 = 00 00 00 00 00 00 00 00 */

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

#include <stdio.h> #include <stdlib.h>

 int cycles;                  /* arg#1 - no of calc cycles ex: 15   */
 long value1;                 /* arg#2 - initial value     ex: 4096 */
 int mult;                    /* arg#3 - multiplier        ex: 4    */
 long value2;                 /* save current value for printing    */
 int ii;                      /* loop index                         */
 /* union to hold long & 8 byte char array in same memory positions */
 /* - to switch ends, on Intel Little-End to Big-End                */
 /* - then print 8 character in hex representation                  */
 union ULC
 { long l;
   unsigned char c[8];
 } ;
 union ULC u1;
 union ULC u2;
 /* mainline for test64b */
 int main(int argc, char *argv[])
 {
 /* verify operator entered correct number of arguments */
 if (argc != 4)
   { printf("usage: test64b cycles init-value multiplier \n");
     printf("example: test64b  26     4096       4  \n");
     exit(9);
   }
 cycles = atoi(argv[1]);         /* arg1 cycles to int                 */
 /* value1 = strtol(argv[2],(char**)0,10); <-- strtol/strtoll obsolete */
 value1 = atol(argv[2]);         /* use atol convert initvalue to long */
 mult   = atoi(argv[3]);         /* arg3 multiplier to int             */
 /* begin loop to calc & print results */
 for (ii=0; ii < cycles; ii++)
   { value2 = value1;           /* save current value for printing     */
     value1 *= mult;            /* multiply value by mplr              */
     /* switch ends, on Intel Little_End to Big_End */
     u1.l = value1;
     u2.c[0] = u1.c[7]; u2.c[1] = u1.c[6]; u2.c[2] = u1.c[5]; u2.c[3] = u1.c[4];
     u2.c[4] = u1.c[3]; u2.c[5] = u1.c[2]; u2.c[6] = u1.c[1]; u2.c[7] = u1.c[0];
 printf("%02d %020ld * %02d = %020ld = %02X %02X %02X %02X %02X %02X %02X %02X\n"
     ,ii,value2,mult,value1
     ,u2.c[0],u2.c[1],u2.c[2],u2.c[3],u2.c[4],u2.c[5],u2.c[6],u2.c[7]);
   }
 return(0);
 }
 /*Note - type long (64 bits) holds 19 digit values, but not 20 digits            */
 /*     - there is type '__int128', but no conversion format for __int128         */
 /*     - '%ld' converts longs to strings, BUT NO '%__int128' to print 128 bits   */

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

11F1. Educational C programs - testmaxints.c

testmaxints - show sizeof integers & max values

'testmaxints.c' will show sizeof & maximum capacities of binary integers (short, int, long, long long,& int128)

Assumptions to run testmaxints

  1. Logged in as uvadm --> $UV (usually /home/uvadm)

  2. $UV/src/testC/testmaxints.c - C source program

  3. $UV/bin/testmaxints.c - compiled executable program

    Operating Instructions


 #1. gcc src/testC/testmaxints.c -obin/testmaxints <-- compile
     =============================================

 #2. testmaxints       <-- execute
     ===========         - expected output below:

 #2a--> sizeof_short=2, sizeof_int=4, sizeof_long=8 sizeof_longlong=8 sizeof_int128=16
        ------------------------------------------------------------------------------

 #2b--> maxshort=32767,  maxint=2147483647,    maxlong=9223372036854775807
        ------------------------------------------------------------------

 #2c--> maxshort=32,767, maxint=2,147,483,647, maxlong=9,223,372,036,854,775,807
        ------------------------------------------------------------------------

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

11F2. testmaxints.c - C source code listing

 /* testmaxints.c - show sizeof C integers                      */
 /*               - short, int, long, long long, __int128       */
 /*               - stored in $UV/src/testC/testmaxints.c       */
 /* Problem 64 bit long integers max value 19 digits            */
 /* - testmaxints.c written to test max int sizes               */
 /*                                                             */
 /* gcc src/testC/testmaxints.c -obin/testmaxints <-- compile   */
 /* =============================================               */

 /* testmaxints   <-- execute                                                      */
 /* ==============     - expected output below:                                    */
 /*                                                                                */
 /* sizeof_short=2, sizeof_int=4, sizeof_long=8 sizeof_longlong=8 sizeof_int128=16 */
 /* ------------------------------------------------------------------------------ */
 /* maxshort=32767,  maxint=2147483647,    maxlong=9223372036854775807             */
 /* ------------------------------------------------------------------------------ */
 /* maxshort=32,767, maxint=2,147,483,647, maxlong=9,223,372,036,854,775,807       */
 /*                                                                                */
 /* - type __int128 available for arithmetic operations, BUT no print format       */
 /* - print format types: short=%d, int=%d, long=%ld, BUT NO %__int128 format      */
 /* - C compiler recognizes type __int128, proven by sizeof(__int128)=16           */

#include <stdio.h> #include <stdlib.h> int main(int argc, char **argv) { short s; int i; long l; long long ll; __int128 int128; int ii; short maxshort=0, maxshort1=1, maxshort2=0; int maxint=0, maxint1=1, maxint2=0; long maxlong=0, maxlong1=1, maxlong2=0;

 /* get sizes of short, int, long, long long, __int128 & print */
 int sos = sizeof(short);
 int soi = sizeof(int);
 int sol = sizeof(long);
 int soll = sizeof(long long);
 int soint128 = sizeof(int128);
 printf("\nsizeof_short=%d, sizeof_int=%d, sizeof_long=%d sizeof_longlong=%d sizeof_int128=%d \n"
        ,sos,soi,sol,soll,soint128);

/* calc max ints & print */ for (ii=0; (maxshort1 > 0 && ii < 64); ii++) { maxshort2 = maxshort1; maxshort1 *= 2; } for (ii=0; (maxint1 > 0 && ii < 64); ii++) { maxint2 = maxint1; maxint1 *= 2; } for (ii=0; (maxlong1 > 0 && ii < 64); ii++) { maxlong2 = maxlong1; maxlong1 *= 2; } maxshort = ((maxshort2 -1) + maxshort2); maxint = ((maxint2 -1) + maxint2); maxlong = ((maxlong2 -1) + maxlong2); printf("maxshort=%d, maxint=%d, maxlong=%ld\n\n",maxshort,maxint,maxlong); return 0; }

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

11G1. Educational C programs - testio.c

 /* testio1.c - test program to investigate various problems           */
 /*           - by Owen Townsend, www.uvsoftware.ca                    */
 /* - copies a file of fixed length 256 byte records                   */
 /* - records could contain binary characters, LineFeeds not required  */
 /* - see testio2.c for sample program to copy text file with LineFeeds*/
 /* - using only native C functions (no UV subfunctns as testio5.c)    */
 /*                                                                    */
 /* cc src/testio1.c -o bin/testio1    <-- compile                     */
 /* ===============================                                    */
 /* bin/testio1 dat1/custmas1 tmp/cm1  <-- execute                     */
 /* =================================                                  */
 #include <stdio.h>
 #include <string.h>
 char f1n[64];            /* input filename                    */
 FILE *f1p;               /* file pointer from fopen           */
 char f1rec[256];         /* input record area                 */
 int f1recs;              /* record count                      */
 int f1rstat;             /* current fread return status       */
 char f2n[64];            /* output filename                   */
 FILE *f2p;               /* file pointer from fopen           */
 int f2wstat;             /* current fwrite return status      */
 int main(int argc, char **argv)
 {
 if (argc != 3)
   { printf("usage: testio1 infilename outfilename \n");
     return(1);
   }
 strcpy(f1n,argv[1]);     /* store fieldnames from cmnd line  */
 strcpy(f2n,argv[2]);
 if ((f1p = fopen(f1n,"rb")) == (FILE*)0)  /* <-- NOTE "rb" Binary spcfd */
   { printf("cant open input file %s \n",f1n); return(3); }
 if ((f2p = fopen(f2n,"wb")) == (FILE*)0)
   { printf("cant open output file %s \n",f2n); return(4); }
 while (1)
   { f1rstat = fread(f1rec,1,256,f1p);      /* get a record             */
     if (f1rstat <= 0)                     /* EOF ?                    */
       { break; }
     f1recs++;                             /* inc count for line seq#  */
     f2wstat = fwrite(f1rec,1,256,f2p);
     if (f2wstat < 0)
       { printf("ERROR, writing output file %s \n",f2n); return(5); }
   }

/* close files, display EOF msg with record count & exit */ fclose(f1p); fclose(f2p); printf("EOF: copied file %s to %s, records=%d \n",f1n,f2n,f1recs); return(0); }

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_12 Games & Education - uvcopy jobs

Part 12 - Contents


12A1. Introduction & Overview

12B1. lotto1uv - generate random numbers for various uses
- example for lottery tickets
- uvcopy equivalent of C program lotto1.c (see '11C1')

12C1. testint1 - test binary long integer maximums
- show hex values for binary progression 1,16,256,4096,etc
- uvcopy equivalent of C programs test64a.c & test64b.c (see '11E1' & 11E2)

12D1. hexcalc1 - convert values form decimal to binary hexadecimal & the reverse

12E1. datedemo1 - convert dates between calendar, julian,& days-since-1900
- solicits dates & input formats,& display all output formats
- allows add or sub to days-since-1900 & display all 3 outputs
- great for calculating how many days between 2 calendar dates
  or the calendar date prior to or after a given calendar date

12F1. calendar1

12G1. expandvars1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

12A1. Games & Education - uvcopy jobs - Overview

Games written in uvcopy

Part11 is a fun way to help you learn to program in the C language. I was inspired to add this for my grandson - perhaps you have grandchildren who would be interested.

I think they would enjoy the 'insult generator' that uses random numbers to generate insults using, using random numbers to select insult words from 3 columns of old English Shakespearean insult adjectives & nouns.

'lotto1.c' is another C program generating random numbers for lotteries, but could also be sued to generate random numbers for useful work.

Education - Integers, Hexadecimal, conversions

We present several more C programs to illustrate the binary coding (in hexadecimal) & maximum capacities for integers (short, int, long,& int128) for both Big-End & Little-End computers.

Note that the compile & operating instrucitons for these demo C programs are included as /*---comments---*/ at the begining of the program, so we do not need separate documentation for the simpler programs.

programming in uvcopy

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

12B1. Games & Education - uvcopy jobs

lotto1uv - generate random numbers with uvcopy

 # lotto1uv - generate random numbers with uvcopy
 #          - stored at $UV/pf/demo/lotto1uv
 #          - by Owen Townsend, UV Software, April/2020
 #  - same functionality as C program $UV/src/games/lotto1.c
 #  - generate any quantity of random#s between any 2 limits
 #  - default for Lotto649, execute as shown below:
 #
 # uvcopy lotto1uv  <-- start random# generator
 # ================    - will prompt for number,low,high ranges
 # enter: number of random#s desired, low range, high range
 # --> 6,1,49       <-- default, gen 6 random#s between 1 & 49
 #     ======         - for the 6/49 lottery
 # random# 1 = 20
 # random# 2 = 25
 # random# 3 = 8
 # random# 4 = 23
 # random# 5 = 37
 # random# 6 = 19
 @run
 man10   msg     'enter number,low,high Not entered, enter 3 arguments now'
         msgwa1  ' - OR reply null to default 1,6,49 - OR enter q to quit'
         cmc     $arg1(1),'q'            quit ?
         skp=    man90                   yes - goto EOJ
         fix     a0(20),$arg1,3,','      separate to a0(20),a20(20),a40(20)
         mvnx3   $ca1,a0(20)             convert to binary in $ca1,$ca2,$ca3
         cmn     $ca3,0                  3 arguments entered ?
         skp>    man20
         mvn     $ca1,6
         mvn     $ca2,1
         mvn     $ca3,49
 #
 # begin loop to generate $ca1 random#s between $ca2 & $ca3
 man20   add     $ca5,1                  count loops
 #       =====================
         ran     $ca4,$ca2,$ca3          gen current random# in $ca4
 #       =====================
         msgv1   'random# $ca5 = $ca4'   show current loop count & random#
         cmn     $ca5,$ca1               loop ctr = number requested ?
         skp<    man20                   no - repeat loop
         mvn     $ca5,0                  clear loop ctr for next set
         skp     man10                   return to prompt
 man90   eoj                             End job

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

12C1. Games & Education - uvcopy jobs

testint1

 # testint1 - test conversion from decimal to binary & back
 #          - by Owen Townsend, UV Software, April 2020
 #          - uvcopy Parameter File stored in: $UV/pf/adm/testint1
 #          - similar to C programs test64a.c & test64b.c
 #
 # uvcopy testint1,fili1=tf/testint1_data,filo1=tmp/testint1_data
 # ==============================================================
 # uvcopy testint1     <-- same as above, files default as shown above
 # ===============
 #
 # Output display will be different depending on machine architecture
 # - BigEndMachines (AIX,SUN,MVS,ZOS) vs LittlEndMachines (INTEL,AMD,MAC)
 rop=r1x4    # r1 = prompt for report disposition, x4 = default 'more'
 fili1=?tf/testint1_data,rcs=128,typ=LST
 filo1=?tmp/$fili1,rcs=128,typ=LSTt
 @run
        opn    all
        mvfv3  b0(80),'# Date=$date, Machine=$machine, Bits=$longbits'
        putb   filo1,b0(80)       write info to 1st line of report
 #
 man20  get    fili1,a0
        skp>   eof
        mvc    b0(80),a0           move input to output
        cmc    a0(1),'#'           comment ?
        skp=   man40
 man30  mvn    $ca1,b3(20)         cnvt to int (4/8 via H32 vs H64/S64)
        mvn    c0(8b),$ca1         move to 8 bytes in area c
        hxc    b43(16),c0(8)       convert to hex for display
        mvn    d0(8bs),$ca1        move to area d with big/little end switch
        hxc    b61(16),d0(8)       convert to hex
 man40  put    filo1,b0            write output
        skp    man20
 #
 eof    cls    all
        eoj

Execute testint1


 uvcopy testint1,fili1=tf/testint1_data,filo1=tmp/testint1_data
 ==============================================================

 uvcopy testint1   - same as above, files default as shown
 ===============   - see Input & Output listed below

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

12C2. Games & Education - uvcopy jobs

tf/testint1_data - INPUT file for uvcopy testint1

 # testint1_data - test data for testint1 (test decimal to binary & back)
 #               - uvadm/tf/testint1_data for uvcopy job uvadm/pf/testint1
 # BigEndMachines = AIX,SUN,MVS/ZOS, LittleEndMachines = INTEL,AMD,MAC,etc
 #        1         2         3         4         5         6         7         8
 #2345678901234567890123456789012345678901234567890123456789012345678901234567890
 #                        expected output      --- actual output in hex ---
 # decimal-input             binary hex       byte-native       byte-swapped
 #===============================================================================
 00 00000000000000000001  0100000000000000
 01 00000000000000000016  1000000000000000
 02 00000000000000000256  0001000000000000
 03 00000000000000004096  0010000000000000
 04 00000000000000065536  0000010000000000
 05 00000000000001048576  0000100000000000
 06 00000000000016777216  0000000100000000
 07 00000000000268435456  0000001000000000
 #  --- 32 bit limit ---
 08 00000000004294967296  0000000001000000
 09 00000000068719476736  0000000010000000
 10 00000001099511627776  0000000000010000
 11 00000017592186044416  0000000000100000
 12 00000281474976710656  0000000000000100
 13 00004503599627370496  0000000000001000
 14 00072057594037927936  0000000000000000
 #  --- 64 bit limit ---
 15 01152921504606846976  0000000000000010
 16 18446744073709551616  FFFFFFFFFFFFFF7F

tmp/testint1_data - OUTPUT file from uvcopy testint1

 # testint1_data - test data for testint1 (test decimal to binary & back)
 #                        expected output      --- actual output in hex ---
 # decimal-input             binary hex       byte-native       byte-swapped
 #===============================================================================
 00 00000000000000000001  0100000000000000  0100000000000000  0000000000000001
 01 00000000000000000016  1000000000000000  1000000000000000  0000000000000010
 02 00000000000000000256  0001000000000000  0001000000000000  0000000000000100
 03 00000000000000004096  0010000000000000  0010000000000000  0000000000001000
 04 00000000000000065536  0000010000000000  0000010000000000  0000000000010000
 05 00000000000001048576  0000100000000000  0000100000000000  0000000000100000
 06 00000000000016777216  0000000100000000  0000000100000000  0000000001000000
 07 00000000000268435456  0000001000000000  0000001000000000  0000000010000000
 #  --- 32 bit limit ---
 08 00000000004294967296  0000000001000000  0000000001000000  0000000100000000
 09 00000000068719476736  0000000010000000  0000000010000000  0000001000000000
 10 00000001099511627776  0000000000010000  0000000000010000  0000010000000000
 11 00000017592186044416  0000000000100000  0000000000100000  0000100000000000
 12 00000281474976710656  0000000000000100  0000000000000100  0001000000000000
 13 00004503599627370496  0000000000001000  0000000000001000  0010000000000000
 14 00072057594037927936  0000000000000000  0000000000000001  0100000000000000
 #  --- 64 bit limit ---
 15 01152921504606846976  0000000000000010  0000000000000010  1000000000000000
 16 18446744073709551616  FFFFFFFFFFFFFF7F  FFFFFFFFFFFFFF7F  7FFFFFFFFFFFFFFF

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

12D1. Games & Education - uvcopy jobs

hexcalc1

 # hexcalc1 - uvcopy Parameter File from UVSI stored in: /home/uvadm/pf/util/
 # hexcalc1 - hexadecimal calculator, hex to decimal, or decimal to hex
 #          - hexcalc1 for Intel machines, use hexcalc2 for RISC machines
 @run
        msg     'hexcalc1 for Intel machines, use hexcalc2 for RISC machines'
 # begin loop to solicit numbers for conversion
 man20  nop
        msg     'decimal->hex,hex->decimal,ex: d4096->x1000, x1000->d4096'
        msgwa1  'enter number with prefix "d" decimal, "x" hex rep, "q" quit'
        mvc     a0(80),$arg1          store entry & blank extra w/s
        mvc     a20(16),a1            drop prefix
        sqzr    a20(16),' '           right justify
        rep     a20(16),' ','0'       left zero fill
        cmc     a0(1),'d'             dcml to hex ?
        skp=    dtox
        cmc     a0(1),'x'             hex to dcml ?
        skp=    xtod
        cmc     a0(1),'q'             quit ?
        skp=    quit
        msg     'invalid entry, 1st char not "d" or "x"'
        skp     man20
 #
 # convert decimal to hex
 dtox   mvn     b0(8bs),a20(16)       cnvrt dcml to binary (switch Intel)
        hxc     b10(16),b0(8b)        cnvrt binary to hex rep
        mvf     b100(60),'decimal ddd = xxx in hex representation'
        rep     b100(60),'ddd',a20(16)
        rep     b100(60),'xxx',b10(16)
        msg     b100(60)              display result
        msg     ' '                   space between repeat calcs
        skp     man20
 #
 # convert hex to decimal
 xtod   chx     c0(8),a20(16)         cnvrt hex to binary
        mvn     c10(16),c0(8bs)       cnvrt binary to decimal (switch Intel)
        mvf     c200(60),'hex representation xxx = ddd in decimal'
        rep     c200(60),'xxx',a20(16)
        rep     c200(60),'ddd',c10(16)
        msg     c200(60)              display result
        msg     ' '                   space between repeat calcs
        skp     man20
 quit   eoj

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_13 Vancouver Utility Big Time Savers

Part13 - Big_Time_Savers - Contents


13A1. Overview - Big_Time_Savers
- things that are hard to do with unix system editors & utilities
- But easy to do with Vancouver Utilities (uvcopy,uvsort,etc).

13B0. uvfix1 - 'uvfix1' has to be your 1st thought when you want to perform
editing operations on text files that are hard to do with the
usual unix system editors & utilities.

13C0. catfiles2 - a uvcopy utility that allows you to insert or replace columns
of data in text files, required for some control files & looks
neater in documentation tables. Example to create the 3 column
table of insult words for the insult generator (see '11B1')

13D0. splitfiles3 - extract up to 3 columns of data from text files. For example
- to sort a column of data in a table files, use splitfiles3 to extract
  the desired column to a separate file, sort the file, & then use
  catfiles2 to replace the unsorted column with the sorted version.
- Demo uses splitfiles3 to extracts the 3 columns of insult words
  from '11B1' insult generator, for updating & re-sorting separately,
  then using catfiles2 to recombine into the 3 column inserts file.

13E0. catfiles2 - 2nd demo using AutoSales, inserting current year sales into a
history file of last 5 years history side by side comparisons.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13A1. Vancouver Utility Big Time Savers

Big_Time_Savers - Overview

The Vancouver Utilities & in particular uvcopy can save you a lot of time when you need to create & modify your text files of scripts, programs, documentation, control files, etc.

Some operations are difficult to do with the unix system editors & utilities. For example - addressing fields by column# hard to do with vi, etc, but easy to do with uvcopy & the many pre-programmed jobs using uvcopy.

catfiles2 - insert or replace columns of data in table files

We will use 'catfiles2' for our 1st Big_Time_Saver demo. 'catfiles2 is a uvcopy utility that allows you to insert or replace columns of data in text files - may be required for some control files & looks neater in documentation tables.

We will show how 'catfiles2' was used to create the data files for the insults geenrator which was documented in '11B1'. The C program insult2 generates insults by randomly selecting words from 3 columns of old English Shakespearean insult words.

splitfiles3 - extract columns of data to separate files

'splitfiles3' will extract up to 3 columns of data from text files. For example you might want to sort a column of data in 1 of your table files. You could use splitfiles3 to extract the desired column to a separate file, sort the file, and then use 'catfiles2' to replace the unsorted column with the sorted version.

Run tthe Demos

Run the demos to get familiar with using these tools & to recognize situations in your editing tasks where these tools can save you a lot of time.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13B0. Vancouver Utility Big Time Savers

uvfix1 - #1 Big Time saver

'uvfix1' has to be your 1st thought when you want to perform editing operations on text files that are hard to do with the usual unix system editors & utilities.

'uvfix1' is the easy way to use the power of uvcopy when only a few instructions are required. You execute script 'uvfix1' ($UV/sf/util/uvfix1) which calls uvcopy job $UV/pf/util/uvfix1. You only need to specify the input file.

The output is automatically written to the tmp/ directory in your current directory. You can inspect the output & if OK, you can copyback replacing the original if your intention was to fix a problem in the file (often the case).

You can do useful work with only 1 instruction such as 'fix' to convert delimited files to fixed-field formats, or the reverse 'var' to convert fixed to delimited.

You can do useful work with NO instructions to copy 1 or ALL files in a directory, automatically removing trailing blanks & Carriage Returns (x'0D's) that can cause problems on unix systems (CR's can cause scripts to fail).

uvcopy instructions

'uvcopy' is the most powerful Vancouver Utility. It gives you the power of assembler without the complexity & without compilation - just edit source type instructions & execute with the uvcopy interpreter. See over 100 instructions at uvsoftware.ca/uvcopy3.htm

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13B1. test data files to demo uvfix1

First we will show you the test data files used for our 1st 3 uvfix1 demos. We are showing the instructions used, and on the next page we will give you the step by step operating instructions to sequence# tf/nameadrs3b to td/nameadrs3d.

tf/nameadrs3a

         Justin Trudeau|24 Sussex Drive|Ottawa ON|K1Y2L6 Canada|
         John Horgan|501 Belleville St.|Victoria BC|V2V2L8 Canada|
         Doug Ford|6 Tettenhall Rd.|Toronto ON|M9A2C3 Canada|
         Donald Trump|1600 Pennsylvannia|Washington DC|20500 USA|
         Andrew Cuomo|138 Eagle St.|Albany NY|12242 USA|
         Bill Gates|500 5th Ave. N.|Seattle WA|98109 USA|

tf/nameadrs3b


      fix b0(20),a0(100),4,'|'    <-- 1 instruction to convert above tf/nameadrs3a
      ========================      - to following tf/nameadrs3b
         Justin Trudeau      24 Sussex Drive     Ottawa ON           K1Y2L6 Canada
         John Horgan         501 Belleville St.  Victoria BC         V2V2L8 Canada
         Doug Ford           6 Tettenhall Rd.    Toronto ON          M9A2C3 Canada
         Donald Trump        1600 Pennsylvannia  Washington DC       20500 USA
         Andrew Cuomo        138 Eagle St.       Albany NY           12242 USA
         Bill Gates          500 5th Ave. N.     Seattle WA          98109 USA

tf/nameadrs3c


      var b0(100),a0(20),4,';'    <-- 1 instruction to convert above tf/nameadrs3b
      ========================      - to following tf/nameadrs3c
         Justin Trudeau;24 Sussex Drive;Ottawa ON;K1Y2L6 Canada;
         John Horgan;501 Belleville St.;Victoria BC;V2V2L8 Canada;
         Doug Ford;6 Tettenhall Rd.;Toronto ON;M9A2C3 Canada;
         Donald Trump;1600 Pennsylvannia;Washington DC;20500 USA;
         Andrew Cuomo;138 Eagle St.;Albany NY;12242 USA;
         Bill Gates;500 5th Ave. N.;Seattle WA;98109 USA;

tf/nameadrs3d

      add $ca1,10       <-- 4 instructions to sequence tf/nameadrs3b to tf/nameadrs3d
      clr b0(100)         - clear output area 'b'
      mvn b0(5),$ca1      - insert sequence# in 1st 5 bytes of output area
      mvc b6(100),a0      - copy input area 'a' shifting right 6 bytes for sequence#
      ==============
         00010 Justin Trudeau      24 Sussex Drive     Ottawa ON           K1Y2L6 Canada
         00020 John Horgan         501 Belleville St.  Victoria BC         V2V2L8 Canada
         00030 Doug Ford           6 Tettenhall Rd.    Toronto ON          M9A2C3 Canada
         00040 Donald Trump        1600 Pennsylvannia  Washington DC       20500 USA
         00050 Andrew Cuomo        138 Eagle St.       Albany NY           12242 USA
         00060 Bill Gates          500 5th Ave. N.     Seattle WA          98109 USA

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13B2. uvfix1 - #1 Big Time saver

uvfix1 demo - Insert Sequence#s


 #0a. Login as userxx       <-- Login as userxx
      ===============
 #0b. Login as uvadm          - OR uvadm
      ==============
 #0c. cd demo               <-- change to demo dir
      =======
 #0d. rm -f tmp/* tmp1/*    <-- clear tmp dirs to receive output files
      ==================

 #1. uvfix1 tf/nameadrs3b
     ====================
        - copy a text file & modify via uvcopy instruction keyins
        - input rec in area a has already been copied to output rec in b
        uop=q1c0n99r8192 - user option defaults
              c1         - pass thru any "#" col1 #comment records
                n99      - output record limit (n99 special meaning **ALL** records)
                         - records counted in $ca10 & compared to option n
                   r8192 - recsize ONLY for Fixed-Length OUTPUT records if desired
                         - for Fixed-Length, reply RSF/RST at prompt(default text typ=LSTt)
                         - for Text files: accept default recsize & default typ=LSTt
        OPtion defaults  = q1c0n99r8192 -->   <-- null to accept or enter/override
        LSTt = typ default filo01 - null accept or enter new typ ?
    clr  b0(80)      <-- example: insert sequence#s in 1st 5 columns
    mvc  b6(80),a0   <-- example: mvc - MoVe Char a to b offset 6 right
    add  $ca1,1      <-- example: add - count records
    mvn  b0(5),$ca1  <-- example: mvn - MoVe Numeric seq# to 1st 5 bytes
                       - may use "skp?? getrec" inhibit output current record
                       - may use "skp?? EOF"    end job now before normal EOF
                    <-- enter instruction (or period "." end entries & execute)
 clr b0(100),' '
                    <-- enter instruction (or period "." end entries & execute)
 add $ca1,100
                    <-- enter instruction (or period "." end entries & execute)
 mvn b0(5),$ca1
                    <-- enter instruction (or period "." end entries & execute)
 mvc b6(100),a0
                    <-- enter instruction (or period "." end entries & execute)
 .
 200425:164808:uvfix1: EOF fili01 rds=6 size=432: tmp1/nameadrs3b
 200425:164808:uvfix1: EOF filo01 wrts=6 size=468: tmp/nameadrs3b
 ----- enter: vi,cat,more,lp,uvlp13,etc --> cat
 00100 Justin Trudeau      24 Sussex Drive     Ottawa ON     K1Y2L6 Canada
 00200 John Horgan         501 Belleville St.  Victoria BC   V2V2L8 Canada
 00300 Doug Ford           6 Tettenhall Rd.    Toronto ON    M9A2C3 Canada
 00400 Donald Trump        1600 Pennsylvannia  Washington DC 20500 USA
 00500 Andrew Cuomo        138 Eagle St.       Albany NY     12242 USA
 00600 Bill Gates          500 5th Ave. N.     Seattle WA    98109 USA
 ----- output in tmp/nameadrs3b, copy to wherever required
 ----- OR copy/rename as appropriate

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13B3. uvfix1 - #1 Big Time saver

uvfix Tips & Diffs vs uvcopy instructions

  1. You do not need to code the uvcopy framework instructions. The 'get', test EOF, 'mvc' inarea 'a' to outarea 'b', & 'put' are automatic

  2. You cannot define labels (optional in cols 1-8 of uvcopy instrns) So, you can code the instruction operations starting in column 1

  3. The only goto labels allowed are 'getrec', 'putrec',& 'EOF'. - 'skp?? getrec' would inhibit output for the current record. - 'skp?? putrec' would inhibit further instructions for current record. - 'skp?? EOF' would inhibit processing any remaining records in the file.

  4. You would code 'skp??' to test the results of many instructions such as: cmc, cmn, scn, rep, lok, tst, tsb (& many more, see uvcopy3.htm).

  5. Condition codes shown as '??' on 'skp??'s above can be any 1 of '=', '!', '<', '>', '<=', '>='

  6. You do not need to code 'get' instructions, use the special 'skp?? getrec which bypasses the automatic 'put' for the current record.

  7. You can code 'put' instructions if desired to output multiple records on 1 input record cycle. You might then use special 'skp?? getrec' to inhibit the automatic 'put' for the current record.

  8. No point in coding comments since the entries are not permanent (as for uvcopy instructions, usually stored in a pf/... directory)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13B4. uvfix1 - #1 Big Time saver

Easy way to drop CR's & shorten to last non-blank

Carriage Returns x'0D' can cause problems on unix systems (scripts can fail) & the trailing blanks are annoying. An easy way to remove the CR's & trailing blanks is to use 'uvfix1' for 1 file at a time or 'uvfixA' for all files in a directory.

Parm files FTP'd ASCII mode from the mainframe are blank filled to 80 bytes + CR x'0D' + LF x'0A' = 82 bytes. You can't see them with vi,cat,etc so use uvhd:

For example, we will use $UV/tf/prodmas0 (8 records of 82 bytes each). Here is the 1st record displayed by uvhd. Note the trailing blanks & the ending Carriage Return x'0D' + Line Feed x'0A':


 uvhd $UV/tf/prodmas0 t   <-- use option 't' for Text files (end record on LF x'0A')
 ======================
 rec#=1 rsize=82 fptr=0 fsize=656 records=8
                 10        20        30        40        50        60        70        80
   r# 1 0123456789012345678901234567890123456789012345678901234567890123456789012345678901
      0 BBQ010 BAR-B-Q         0019500                                                  ..
        4453332445242522222222233333332222222222222222222222222222222222222222222222222200
        2210100212D2D1000000000001950000000000000000000000000000000000000000000000000000DA

 #0. Login --> /home/userxx

 #1. rm -f tmp/* tmp1/* tmp2/*   <-- clear tmp files
     =========================

 #2. cp $UV/tf/prodmas0 tmp1/    <-- copy demo file to your $HOME/tmp1/...
     ========================

 #3. uvfix1 tmp1/prodmas0        <-- run uvfix1 to remove trailing blanks & CR's
     ====================
               - omitting several display lines for this documentation
               - until the prompt to enter instructions
      --> .  <-- No instructions required, just enter the '.' period
               - to signal end of instructions & proceed to processing
               - copies the file to tmp/prodmas0 (removing trailing blanks & CRs)

 #4. uvhd tmp/prodmas0 t     <-- investigate the output file
     ===================
 rec#=1 rsize=31 fptr=0 fsize=248 records=8
                 10        20        30
   r# 1 0123456789012345678901234567890
      0 BBQ010 BAR-B-Q         0019500.   <-- confirming trailing blanks removed
        4453332445242522222222233333330     - and CR's x'0D' removed
        2210100212D2D10000000000019500A     - terminated by LF's only x'0A's

 #5. cp tmp/prodmas0 tmp1/   <-- could copy back overwriting original file
     =====================     - but dont do this in $UV (would destroy $UV testfile)
Note
  • could use 'uvfixA' to process ALL files in the directory
  • while copying to a 2nd directory, uvfixA does not output to tmp (like uvfix1)

 #3a. uvfixA tmp1 tmp2 LST   <-- process ALL files in tf/* copying to tmp1/...
      ====================     - arg3 'LST' is the file type (Line Seqntl Terminated)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13B5. uvfix1 - #1 Big Time saver

uvfixA - modify ALL Text files in subdir

Given
  • directory of COBOL programs /home/uvadm/mf/cbls

 Required - copy/modify 'ALL' COBOL programs in directory
 ======== - clear cols 1-6 & 73-80, sequence# in cols 1-4
          - translate to lower-case except in quotes

 #1. Login yourself --> /home/userxx
 #2. mkdir tmp1 tmp2      <-- make subdirs in your homedir
 #2a. rm -f tmp1/* tmp2/*   - or clear subdirs if existing

 #3. cp $UV/mf/cbls/car* tmp1  <-- copy demo progams to your subdir
     ========================    - 3 programs car100, car120, car200

 #4. uvfixA tmp1 tmp2 LSTt     <-- run uvfixA specifying I/O subdirs
     =====================       - reply to prompts as follows:
      uop=q1r8192n99999 - user option defaults
            r8192       - output reccsize for fixed length records typ=RSF/RST
                        - for text files: accept default recsize & default typ=LSTt
                n999999 - output record count limit (high# copies to EOF)
      User OPtion (uop) defaults  = q1r8192n999999

 #4a. -->null to accept or enter/override -->   <-- null reply, no options required
 #4b. -->did you create outdir (dflt tmp) --> y <-- reply 'y' to continue

 #5.      reply to instruction prompts as follows:
          ========================================
 #5a. --> clr   b72(8)      <-- clear cols 73-80
 #5b. --> clr   b0(6)       <-- clear cols 1-6
 #5c. --> add   $ca1,1      <-- increment line counter
 #5d. --> mvn   b0(4),$ca1  <-- insert seuw=ence# in cols 1-4
 #5e. --> trlq3 b0(80)      <-- translate to lowercase except in quotes
 #5f. --> .                <-- enter '.' to end keyins & execute

 #6. vi tmp2/*    <-- check results
     =========

sample INPUT - 1st 5 lines of 1st program

      000010 IDENTIFICATION DIVISION.                                    CAR100
      000020 PROGRAM-ID. CAR100.                                         CAR100
      000030 ENVIRONMENT DIVISION.                                       CAR100
      000040 INPUT-OUTPUT SECTION.                                       CAR100
      000050 FILE-CONTROL.                                               CAR100

sample OUTPUT - 1st 5 lines of 1st program

      0001   identification division.
      0002   program-id. car100.
      0003   environment division.
      0004   input-output section.
      0005   file-control.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13C0. Vancouver Utility Big Time Savers

catfiles2 - concatenate 2 files

'catfiles2' makes it easy to insert a column of data into an existing file from a 2nd file, using optoins to specify the displacements & lengths of the data columns in the receiving & sending files (hard to do with unix editors).

catfiles2 Example - create 3 column file for insults generator

Insults Generator ($UV/src/games/insult2.c) requires 3 columns of data See the insults generator documented at www.uvsoftware.ca/uvdemos.htm#11A1 (in Part_11 Games & Education). See the existing file at $UV/tf/insults_Owens

 001 # tf/insults_Owens - data file for $UV/src/games/insult2.c
 002 # dat1/insults <-- must copy/rename from tf/insults_Owens
 003 #                - see www.uvsoftware.ca/uvdemos.htm#Part_11
 004 # - insult2 reads insults from subdir/file = dat1/insults
 005 # - allows multiple files, in tf/insultsA,B,C,etc, copy 1 to dat1/insults
 006 # - seq# cols 1-3, 3 insult words, 25 bytes each, at columns 6,31,56
 007 # - insult words must start on line 11, preceded by 10 lines of #comments
 008 #
 009 #    1         2         3         4         5         6         7         8
 010 #678901234567890123456789012345678901234567890123456789012345678901234567890
 011  arrogant                 asinine                  beetlehead
 012  banal                    boorish                  birdbrain
 013  brazen                   cantankerous             bootlicker
 014  catty                    churlish                 cockroach
 015  clammy                   contrary                 degenerate
 016  cunning                  cynical                  doofus
 017  daft                     damned                   dyslexic
 018  deceitful                decrepit                 earthworm
 019  decrepit                 deficient                flake
 020  degrading                deleterious              fopdoodle
         --- actual file has 35 insults, we use only 1st 10 for demo ---
 045  vulgar                   vapid                    zounderkite

catfiles2 demo - Create 3 column file from 3 separate files

The demo illustrates how the 3 column insults file was created from 3 separate files supplied in $UV/tf/f1,f2,f3 (or copied to your $HOME/demo/tf/f1,f2,f3)

 $UV/demo
 $HOME/demo/   <-- showing only files relevant to catfiles2 demo
 |-- tf            --- separate files ---
 |   |-- f1    <-- 1st column of insult words
 |   |-- f2    <-- 2nd column of insult words
 |   |-- f3    <-- 3rd column of insult words
 |-- tmp
 |-- tmp1
 |   `-- f123  <-- output file (3 columns combined)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13C1. Big_Time_Savers - catfiles2 - concatenate 2 files

Operating Instructions


 #0a. login as userxx   login as userxx
      ===============
 #0b. login as uvadm    OR uvadm & cd demo/
      ==============
 #0c. cd demo
      =======

 #1. rm -f tmp/* tmp1/* tmp2/*  <-- clear tmp dirs to receive output files
     =========================

options to define column locations & lengths

The easy way to see the options for defining the column locations in the receiving & sending files is to run 'uvcopy catfiles2' with no files defined.


 #2. uvcopy catfiles2    <-- run with no files defined
     ================      - will display options as follows:
 uop=a30b25c00d25e0f0g0h1r0x0 - default options
     a##                      - displacement in file1 records to the insert point
        b##                   - length of field where data inserted in file1 records
           c##                - displacement in file2 to data to be inserted
              d##             - length of data to be inserted from file2
                 e#           - spaces to insert BEFORE inserted data from file2
                   f#         - spaces to insert AFTER  inserted data from file2
                     g0       - bypass records with "#" column 1 in either file
                     g1       - allow #records in file1 (always bypassed in file2)
                       h0     - bypass records with "*" column 1 in either file
                       h1     - allow *records in file1 (always bypassed in file2)
                       h1     * default h1 intended for column scale header records
                         r0   - Insert (vs Replace) data from file2
                         r1   - Replace data from file2 (default)
                           x0 - no special meaning for file1 records with ">" col 1
                           x1 - only file1 records with ">" col 1 will be matched to file2
                              - all file2 records matched (file2 col 1 #/* not tested)
 uop=a30b25c00d25e0f0g0h1r0x0 - defaults demo creating 1st 2 columns of 3 col insults file
                              - see doc at www.uvsoftware.ca/uvdemos.htm#13C1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13C2. Big_Time_Savers - catfiles2 - concatenate 2 files

run catfiles2 with files & options defined on command line


 #3. uvcopy catfiles2,fili1=tf/f1,fili2=tf/f2,filo1=tmp1/f12,uop=a30b25c00d25h1
     ==========================================================================
      - concat 1st & 2nd adjective (tf/f1 & tf/f2 defaults)
Note
  • you could omit the files & options (shown above) but only for the 1st run
  • because the job was coded with the 1st pair of files tf/f1 & tf/f2
    & the column locations coded as the defaults.

 #3a. uvcopy catfiles2  <-- same as above (files & options default as shown above)
      ================    - result as follows:
     *        1         2         3         4         5         6         7         8
     *2345678901234567890123456789012345678901234567890123456789012345678901234567890
     001  arrogant                 asinine
     002  banal                    boorish
     003  brazen                   cantankerous
     004  catty                    churlish
     005  clammy                   contrary
     006  cunning                  cynical
     007  daft                     damned
     008  deceitful                decrepit
     009  decrepit                 deficient
     010  degrading                deleterious

 #4. uvcopy catfiles2,fili1=tmp1/f12,fili2=tf/f3,filo1=tmp1/f123,uop=a55b25c00d25h1
     ==============================================================================
      - concat tmp1/f12 + tf/f3 = tmp1/f123, option a changed from a30 to a45 for 3rd columnmn
      *        1         2         3         4         5         6         7         8
      *2345678901234567890123456789012345678901234567890123456789012345678901234567890
      001  arrogant                 asinine                  beetlehead
      002  banal                    boorish                  birdbrain
      003  brazen                   cantankerous             bootlicker
      004  catty                    churlish                 cockroach
      005  clammy                   contrary                 degenerate
      006  cunning                  cynical                  doofus
      007  daft                     damned                   dyslexic
      008  deceitful                decrepit                 earthworm
      009  decrepit                 deficient                flake
      010  degrading                deleterious              fopdoodle

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13C3. Big_Time_Savers - catfiles2 - concatenate 2 files

demo Insert anywhere desired

Above examples are inserting on the right side, but you can insert anywhere. For example insert 4th column in column 3 position & shift existing column 3 over to column 4


 #5. uvcopy catfiles2,fili1=tmp1/f123,fili2=tf/f4,filo1=tmp1/f1234,uop=a55b25c00d25h1
     ================================================================================
      - concat tmp1/f123 + tf/f4 = tmp1/f1234, option a55 for existing 3rd column
      - results as follows:
      *        1         2         3         4         5         6         7         8         9       100
      *234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
      001  arrogant                 asinine                  moribund                 beetlehead
      002  banal                    boorish                  obsessive                birdbrain
      003  brazen                   cantankerous             petulant                 bootlicker
      004  catty                    churlish                 procrastinating          cockroach
      005  clammy                   contrary                 rambunctious             degenerate
      006  cunning                  cynical                  shiftless                doofus
      007  daft                     damned                   surly                    dyslexic
      008  deceitful                decrepit                 truculent                earthworm
      009  decrepit                 deficient                vainglorious             flake
      010  degrading                deleterious              vulgar                   fopdoodle

demo Replace option 'r1' (vs Insert default)

 You can replace existing columns vs Insert which shifts exisitng columns over
 - by specifying option 'r1' replace vs option 'r0' (insert default)

 #6. uvcopy catfiles2,fili1=tmp1/f123,fili2=tf/f4,filo1=tmp1/f123c2r1,uop=a30b25c00d25h1r1
     =====================================================================================
      - replace 2nd column in tmp1/f123 (output of #4 on page above)
      - note option 'r1' appended on option string uop=...r1 (results shown below)
      - note changed output filename to f123c2r1 to indicate column2 Replaced
     *        1         2         3         4         5         6         7         8           9       100
     *23456789012345678901234567890123456789012345678901234567890123456789012345678901234567678901234567890
     001  arrogant                 moribund                 beetlehead
     002  banal                    obsessive                birdbrain
     003  brazen                   petulant                 bootlicker
     004  catty                    procrastinating          cockroach
     005  clammy                   rambunctious             degenerate
     006  cunning                  shiftless                doofus
     007  daft                     surly                    dyslexic
     008  deceitful                truculent                earthworm
     009  decrepit                 vainglorious             flake
     010  degrading                vulgar                   fopdoodle

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13D0. Vancouver Utility Big Time Savers

splitfiles3 - extract columns of data to separate files

'splitfiles3' will extract up to 3 columns of data from text files. For example you might want to update &/or sort 1 or more columns of data in multi column table files. You can use splitfiles3 to extract the desired column(s) to separate files, then update &/or sort the separate files,& then use 'catfiles2' to replace the original columns with the updated/sorted versions.

splitfiles3 demo

Input file $UV/tf/f123 was created by 'catfiles2' (see '13C1' above), from 3 separate files of Shakespearean insult words, for the insults generator (see '11B1' above).

Suppose we want to make a lot of upates to existing words, add a lot of new words, and keep them in alphabetic sequence. You can see that would be an awkward editing task. It would be much easier to use this 'splitfiles3' utility to extract the 3 columns to 3 separate files, make updates, add new words anywhere, sort the 3 files & then use 'catfiles2' to recreate the 3 column file in the required layout.

splitfiles3 demo - input $UV/tf/insults_Owens

     001 # tf/insults_Owens - data file for $UV/src/games/insult2.c
     002 # dat1/insults <-- must copy/rename from tf/insults_Owens
     003 #                - see www.uvsoftware.ca/uvdemos.htm#Part_11
     004 # - insult2 reads insults from subdir/file = dat1/insults
     005 # - allows multiple files, in dat0/insultsA,B,C,etc, copy 1 to dat1/insults
     006 # - seq# cols 1-3, 3 insult words, 25 bytes each, at columns 6,31,56
     007 # - insult words must start on line 11, preceded by 10 lines of #comments
     008 #
     009 #    1         2         3         4         5         6         7         8
     010 #678901234567890123456789012345678901234567890123456789012345678901234567890
     011  arrogant                 asinine                  beetlehead
     012  banal                    boorish                  birdbrain
     013  brazen                   cantankerous             bootlicker
     014  catty                    churlish                 cockroach
     015  clammy                   contrary                 degenerate
     016  cunning                  cynical                  doofus
     017  daft                     damned                   dyslexic
     018  deceitful                decrepit                 earthworm
     019  decrepit                 deficient                flake
     020  degrading                deleterious              fopdoodle
             ---------- 24 lines removed -----------
     045  vulgar                   vapid                    zounderkite
Note
  • for our demo, we will remove 1st 10 lines & use only lines 011-020

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13D1. Vancouver Utility Big_Time_Savers

splitfiles3 Demo Op. Instrns.


 #0a. login as uvadm --> $UV (usually /home/uvadm/)
      ==============
 #0b. rm -f tmp/* tmp1/* tmp2/*    <-- clear tmp files
      =========================      - will use tmp1 to split files & tmp2 to recombine

 #1a. cp tf/insults_Owens tmp1/f0  <-- copy insults file & shorten name for demo
      ===========================

 #1b. vi tmp1/f0   <-- edit file to remove 1st 10 #comment lines & last 39 data lines
      ==========     - keeping only lines 011-020 of data
                     - see listing on the page above
 Note - to understand the options to define the 3 columns of data, we can run
       the job with no I/O files or options defined to get the following display:

  uvcopy splitfiles3  <-- run splitfiles3 with no files or options
  =================    - to display help screen with available options
  --> uop=a00b30c30d25e55d25     - default options
  -->     a##                    - displacement to 1st data column
  -->        b##                 - length of 1st data column
  -->           c##              - displacement to 2nd data column
  -->              d##           - length of 2nd data column
  -->                 e##        - displacement to 3rd data column
  -->                    f##     - length of 3rd data column
  --> uop=a00b30c30d25e55f25     - defaults extract 3 cols of insult words to 3 files
 Note - using the input file listing with scale (see prior page)
     - we can determine the required options & format the required command:

 #2. uvcopy splitfiles3,fili1=tmp1/f0,filo1=tmp1/f1,filo2=tmp1/f2,filo3=tmp1/f3,uop=a05b25c30d25e55d25
     =================================================================================================
 Note - actually, we coded the default options for this particular task
     - so we in this case, we could run without specifying filenames & options

 #2a. uvcopy splitfiles3  <-- same as #1. above, files & options default as above
      =================    - now we have 3 files as shown below:
                           - shown horizontally to save space
      <-------- tmp1/f1 ------>  <-------- tmp1/f2 ------>  <------- tmp1/f3 ------->
      arrogant                   asinine                    beetlehead
      banal                      boorish                    birdbrain
      brazen                     cantankerous               bootlicker
      catty                      churlish                   cockroach
      clammy                     contrary                   degenerate
      cunning                    cynical                    doofus
      daft                       damned                     dyslexic
      deceitful                  decrepit                   earthworm
      decrepit                   deficient                  flake
      degrading                  deleterious                fopdoodle

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13D2. Big_Time_Savers - splitfiles3

update separate files, sort, & recreate 3 column file


 #3a. vi tmp1/f1     <-- update separate files (add/modify insult words)
      ==========       - can add new words anywhere, since we will sort
 #3b. vi tmp1/f2
      ==========
 #3c. vi tmp1/f3
      ==========

 #4. cat tmp1/f1 tmp1/f2 tmp1/f3    <-- show results
     ===========================
      - separate files, but listed side by side to save space
     <-------- tmp2/f1 ------>  <-------- tmp2/f2 ------>  <------- tmp2/f3 ------->
     arrogant                   boorish                    beetlehead
     brazen                     cantankerous               birdbrain
     deceitful                  churlish                   bootlicker
     decrepit                   tickle-brained             dyslexic
     villainous                 unemployed                 earthworm
     loggerheaded               lilly-livered              fopdoodle
     gorbellied                 half-witted                zoomer
     dissembling                flap-mouthed               maggot-pie
     beslubbering               binge-watching             hugger-mugger
     fobbing                    clapper-clawed             baby-boomer

 #5a. sort tmp1/f1 -o tmp2/f1up    <-- sort alphabetically
      =========================
 #5b. sort tmp1/f1 -o tmp2/f2up
      =========================
 #5c. sort tmp1/f1 -o tmp2/f3up
      =========================

recreate 3 column file with catfiles2


 #6a. cp tmp1/f1 tmp2/f1  <-- copy 1st file from tmp1/... to tmp2/...
      ==================

 #6b. uvcopy catfiles2,fili1=tmp2/f1up,fili2=tmp2/f2up,filo1=tmp2/f12,uop=a25b25c00d25
      ================================================================================
       - concat 1st & 2nd adjective words

 #6c. uvcopy catfiles2,fili1=tmp2/f12,fili2=tmp2/f3up,filo1=tmp2/f123,uop=a50b25c00d25
      ================================================================================
       - concat 3rd column on to 1st 2 to create 3 column file
     arrogant                 binge-watching           baby-boomer
     beslubbering             boorish                  beetlehead
     brazen                   cantankerous             birdbrain
     deceitful                churlish                 bootlicker
     decrepit                 clapper-clawed           dyslexic
     dissembling              flap-mouthed             earthworm
     fobbing                  half-witted              fopdoodle
     gorbellied               lilly-livered            hugger-mugger
     loggerheaded             tickle-brained           maggot-pie
     villainous               unemployed               zoomer

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13D3. Big_Time_Savers - splitfiles3

recreated file must satisfy insults generator rules

Note
  • insults generator requires 1st 10 lines #comments as shown on next page
  • all records must be sequenced in 1st 3 columns
  • line 11 must be 1st non #comment & insult words must start in cols 6,31,& 56

 #7a.  vi tmp2/f123   <-- edit recombined columns over 5 spaces to allow for sequence#s
       ============

 #7b.  cat tmp2/f123  <-- check result
       =============    - shifted 5 spaces to the right
          arrogant                 binge-watching           baby-boomer
          beslubbering             boorish                  beetlehead
          brazen                   cantankerous             birdbrain
          deceitful                churlish                 bootlicker
          decrepit                 clapper-clawed           dyslexic
          dissembling              flap-mouthed             earthworm
          fobbing                  half-witted              fopdoodle
          gorbellied               lilly-livered            hugger-mugger
          loggerheaded             tickle-brained           maggot-pie
          villainous               unemployed               zoomer

Insert 1st 10 lines & re-sequence


 #8a. vi tf/insults_Owens  <-- to get 1st lines
      ===================
      --> "a10yy           <-- yank 1st 10 lines into vi buffer "a

 #8b. vi tmp2/f123         <-- edit recreated file
      ============
      --> "aP              <-- Put buffer "a into 1st 10 lines of recreated file

 #8c. cat tmp2/f123        <-- check result
      =============          - 10 insults header records inserted
     001 # tf/insults_Owens   <-- data file for $UV/src/games/insult2.c
     002 # dat1/insults       <-- must copy/rename from dat0/insults_Owens
     003 #                - see www.uvsoftware.ca/uvdemos.htm#Part_11
     004 # - insult2 reads insults from subdir/file = dat1/insults
     005 # - allows multiple files, in dat0/insultsA,B,C,etc, copy 1 to dat1/insults
     006 # - seq# cols 1-3, 3 insult words, 25 bytes each, at columns 6,31,56
     007 # - insult words must start on line 11, preceded by 10 lines of #comments
     008 #
     009 #    1         2         3         4         5         6         7         8
     010 #678901234567890123456789012345678901234567890123456789012345678901234567890
          arrogant                 binge-watching           baby-boomer
          beslubbering             boorish                  beetlehead
          brazen                   cantankerous             birdbrain
          deceitful                churlish                 bootlicker
          decrepit                 clapper-clawed           dyslexic
          dissembling              flap-mouthed             earthworm
          fobbing                  half-witted              fopdoodle
          gorbellied               lilly-livered            hugger-mugger
          loggerheaded             tickle-brained           maggot-pie
          villainous               unemployed               zoomer

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13D4. Big_Time_Savers - splitfiles3

Note
  • now OK, except we need to re-sequence in 1st 3 columns

Re-Sequence in 1st columns


 #9. uvfix1 tmp2/f123    <-- start uvfix1, will prompt for instructions
     ================
     --> add $ca1,1      <-- ADD to a counter (initally zero)
     --> mvn b0(3),$ca1  <-- MoVe Numeric to 1st 3 output columns
     --> .               <-- enter '.' to end isntructions

 #10. cat tmp2/f123   <-- check result, should be as follows:
      =============
     001 # tf/insults_Owens   <-- data file for $UV/src/games/insult2.c
     002 # dat1/insults       <-- must copy/rename from dat0/insults_Owens
     003 #                - see www.uvsoftware.ca/uvdemos.htm#Part_11
     004 # - insult2 reads insults from subdir/file = dat1/insults
     005 # - allows multiple files, in dat0/insultsA,B,C,etc, copy 1 to dat1/insults
     006 # - seq# cols 1-3, 3 insult words, 25 bytes each, at columns 6,31,56
     007 # - insult words must start on line 11, preceded by 10 lines of #comments
     008 #
     009 #    1         2         3         4         5         6         7         8
     010 #678901234567890123456789012345678901234567890123456789012345678901234567890
     011  arrogant                 binge-watching           baby-boomer
     012  beslubbering             boorish                  beetlehead
     013  brazen                   cantankerous             birdbrain
     014  deceitful                churlish                 bootlicker
     015  decrepit                 clapper-clawed           dyslexic
     016  dissembling              flap-mouthed             earthworm
     017  fobbing                  half-witted              fopdoodle
     018  gorbellied               lilly-livered            hugger-mugger
     019  loggerheaded             tickle-brained           maggot-pie
     020  villainous               unemployed               zoomer

 #11. cp tmp2/f123 tf/insults_userxx     <-- copy to Test Files directory
      ==============================

 #12. cp tf/insults_userxx dat1/insults  <-- copy to dat1/... for insults generator
      =================================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13E0. Vancouver Utility Big Time Savers

catfiles2 - 2nd demo using Auto Sales History file

These demos are intended for Unix/Linux users who prefer to work on the command line using text editors vs GUI spread sheets.

These techniques could apply to any document with multiple side by side columns of data that might need to be extracted, updated, re-sorted,& re-inserted back into the document.

This demo illustrates how you could use catfiles2 to insert the most recent year of Auto Sales into a sales history file containing multi columns of side by side sales for the last 5 years.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13E1. Vancouver Utility Big Time Savers

DemoFile#1 - AutoSalesUS2018History

 This file is 'AutoSalesUS2018History' - demo file#1 to illustrate inserting
 new columns of data into documents with multi side by side columns of data
 - using uvcopy catfiles2 documented at uvsoftware.ca/uvdemos.htm#13E0

 >Manufacturer        Sales_2018          Sales_2017
 >===========         ==========          ==========
 >GM                  2,934,742           3,002,241
 >Ford                2,490,175           2,575,200
 >Toyota              2,522,458           2,434,515
 >Chrysler            2,238,307           2,073,073
 >Honda               1,604,561           1,641,429
 >Nissan              1,611,951           1,697,150
 >Hyundai-Kia         1,278,740           1,275,223
 >Subaru              680,135             647,956
 >Volkswagen          636,392             625,068
 >BMW                 354,698             354,110
 >Daimler             350,360             375,311
 >Mazda               300,325             289,470
 >Tesla               197,517             55,120
 >JaguarLandRover     122,397             114,333
 >Volvo                98,537             81,507
                 ---------- Notes ----------
 - file1 Lines with '>' col 1 will be matched to 'non-comment' lines in file2
   & data columns inserted from file2 to file1 depending on options that define
   the column positions of the data in the sending & receiving files.
 Note - Please understand that DemoFile#1 ($UV/tf/AutoSalesUS2018History) includes
        all the above lines on this page except for the headings at the top & this
       'Note' at the bottom.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13E2. Vancouver Utility Big Time Savers

DemoFile#2 - AutoSalesUS2019

 # This file is 'AutoSalesUS2019' - demo file to illustrate inserting
 # new columns of data into documents with multi side by side columns of data
 # - using uvcopy catfiles2 documented at uvsoftware.ca/uvdemos.htm#13E0
 #
 Manufacturer        Sales_2019
 ============        ==========
 GM                  2,870,165
 Ford                2,396,532
 Toyota              2,383,318
 Chrysler            2,203,673
 Honda               1,608,370
 Nissan              1,466,730
 Hyundai-Kia         1,324,621
 Subaru              700,117
 Volkswagen          649,001
 BMW                 360,990
 Daimler             352,734
 Mazda               278,550
 Tesla               195,125
 JaguarLandRover     125,741
 Volvo               108,227
 #
 #                   ---------- Notes ----------
 # - file2 lines with '#' column 1 will be bypassed/ignored by catfiles2
 #
 # - file2 Lines WITHOUT '#' col 1 will be matched to corresponding lines in file2
 #   with '>' in column 1 & data columns inserted from file2 to file1 depending on
 #   options that define the column positions of data in the sending & receiving files.
 #
 # - For this example the 2019 sales from file2 will be inserted into cols 20-40
 #   of file1, shifting existing columns 20-100 over to 40-120.
 #
 # - file1 lines without '>' in column 1 are passed thru as is so file1 could be
 #   a long descriptive document with only the '>' sales lines to be updated from file2
 Note - Please understand that DemoFile#2 ($UV/tf/AutoSalesUS2019) includes
        all the above lines on this page except for the headings at the top & this
       'Note' at the bottom.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13E2. Update AutoSalesUS2018History with AutoSalesUS2019

Pre-Requisites to run AutoSales update demo

  1. You have installed Vancovuer Utilities (see summary on page '1A2')

  2. You have setup a login account for yourself (we will call it 'userxx'). Then login & copy $UV/demo/* to your homedir (see setups on page '1C1').

2a. Or use your existing login account by making a $HOME/demo/ directory & copying $UV/demo/* to your $HOME/demo/... & cd demo to run demos. See this alternative setup on page '1C2'.

Relevant files for AutoSales update demo

 /home/uvadm     <-- could login as uvadm
 /home/userxx    <-- better to setup your own login for demos
 /home/existing  <-- or use your exisitng account & setup demo/ directory
 ./demo
 |-- |-- tf
 |   |   |-- AutoSalesUS2018History   <-- AutoSales History file
 |   |   |-- AutoSalesUS2019          <-- Autosales current year update file
 |   |-- tmp
 |   |-- tmp1
 |   |   `-- AutoSalesUS2019History   <-- new/updated AutoSales History file
 |   `-- tmp2

Operating Instructions


 #1a. login as uvadm   <-- could login as uvadm
      ==============

 #1b. login as userxx  <-- better to use your own account
      ===============

 #2. cd demo   <-- change into demo/ directory
     =======

 #3. rm -f tmp/* tmp1/* tmp2/*  <-- clear tmp dirs to receive output files
     =========================

 #4. uvcopy catfiles2,fili1=tf/AutoSalesUS2018History,fili2=tf/AutoSalesUS2019
                 ,filo1=tmp1/AutoSalesUS2019History,uop=a20b20c20d20x1
     =========================================================================
      - copies tf/AutosalesUS2018History to tmp1/AutoSalesUS2019History
      - inserting columns 21-40 from cols 21-40 of tf/AutoSalesUS2019
        shifting existing 21-100 of ...2018History over to 41-120
      - see output file below

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

13E3. Update AutoSalesUS2018History with AutoSalesUS2019

Output file - AutoSalesUS2018History

 This file is 'AutoSalesUS2018History' - demo file#1 to illustrate inserting
 new columns of data into documents with multi side by side columns of data
 - using uvcopy catfiles2 documented at uvsoftware.ca/uvdemos.htm#13E0

 >Manufacturer       Sales_2019           Sales_2018          Sales_2017
 >===========        ==========           ==========          ==========
 >GM                 2,870,165            2,934,742           3,002,241
 >Ford               2,396,532            2,490,175           2,575,200
 >Toyota             2,383,318            2,522,458           2,434,515
 >Chrysler           2,203,673            2,238,307           2,073,073
 >Honda              1,608,370            1,604,561           1,641,429
 >Nissan             1,466,730            1,611,951           1,697,150
 >Hyundai-Kia        1,324,621            1,278,740           1,275,223
 >Subaru             700,117              680,135             647,956
 >Volkswagen         649,001              636,392             625,068
 >BMW                360,990              354,698             354,110
 >Daimler            352,734              350,360             375,311
 >Mazda              278,550              300,325             289,470
 >Tesla              195,125              197,517             55,120
 >JaguarLandRover    125,741              122,397             114,333
 >Volvo              108,227               98,537             81,507
                 ---------- Notes ----------
 - file1 Lines with '>' col 1 will be matched to 'non-comment' lines in file2
   & data columns inserted from file2 to file1 depending on options that define
   the column positions of the data in the sending & receiving files.
Note
  • Please understand that the output file ($UV/tf/AutoSalesUS2019History)
    includes all the above lines on this page except for the headings at
    the top & this 'Note' at the bottom.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_20 UVdemos - Vancouver Utilities for Unix/Linux users

Part 10 Contents

Part_20 documents items that were considered too long & complex for presentation earlier in the UVdemos documentation. We may have made brief references to these with links to see the full presentations here in Part_20.


10B1. Removing Carriage Returns from scripts. CR's (x'0D's) cause script failures.
10B1-10B11 gives the full explanations (vs '2B1'-2B2 brief introduction).
 Here we present multiple methods of fixing the problem (uvhd,uvcp,uvcopy,uvfix)
 for 1 file or all files in a directory.
10B2.  Run uvhd to see problems not shown with 'vi'
10B3.  Notes re uvhd displays, default option r256, option 't' for text files
10B4.  Fixing the problem - Replacing Carriage Returns (CRs)

10B5. Alternate solutions (other than uvhd) to remove CR's from text files
- using script 'removeCR' (see /home/uvadm/sf/util/removeCR)
10B6. - using 'uvcp' (Vancouver command line Utility)
10B7. - using 'uvcopy' (Vancouver data manipulation Utility)
10B9. - using 'uvfix1' pre-programmed job prompts for instructions
- saves writing a uvcopy job, uvfix1 fixes 1 file at a time
10B10. - using 'uvfixA' pre-programmed job prompts for instructions
- to fix all files in directory while copying to 2nd directory

10C1. Converting UTF-8 French Accents to unaccented equivalents for old applications
Files with high-bit characters may cause problems for older applications.
For Example - files with UTF-8 characters input to legacy appliciations,
not designed to handle them. This demo will show you how to identify them
with uvhd & convert them to unaccented characters with uvhd or uvcp.

10D1. CanadaMPscsv2txt - uvcopy job to convert Canadian MPs from csv to fixed field text

10E1. UScongresscsv2txt - convert US Gov Senators/Representatives from csv to fixed field

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10B1. Basic Vancouver Utilities

uvhd demo#1 - Removing CRs x'0D's from scripts

'scripts' received via Email might have problems not visible with 'vi' When you try to execute an emailed script, you might get errors such as:


 bash: sf/llsx: /bin/bash^M: bad interpreter
 ===========================================

We are supplying the bad script in demo/sf/llsx & good version in demo/sf/llsy Try the following tutorial to illustrate the problem with sf/llsx & correct it:


 #0a. login userxx --> /home/userxx
 #0b. cd demo      --> /home/userxx/demo

 #1. l sf       <-- list the sf (Script Files) directory
     ====
      -rwxrwxr-x 1 uvadm apps 566 Feb 25 12:23 fixaccents2
      -rwxrwxr-x 1 uvadm apps 244 Feb 23 17:43 llsx
      -rwxrwxr-x 1 uvadm apps 296 Feb 24 13:25 llsy

 #2a. sf/llsx sf   <-- execute the BAD script (to list files in sf/...)
      ==========     - get following error:
     "bash: sf/llsx: /bin/bash^M: bad interpreter: No such file"

 #2b. sf/llsy sf   <-- execute the GOOD script to compare results:
      ==========
      llsy - list directory sorted by file-size (biggest files 1st)
       -rwxrwxr-x 1 uvadm apps 566 Feb 25 12:23 fixaccents2
       -rwxrwxr-x 1 uvadm apps 296 Feb 24 13:25 llsy
       -rwxrwxr-x 1 uvadm apps 244 Feb 23 17:43 llsx

 #3a. cat sf/llsx   <-- display the script, to see what's wrong
      ===========       (or could use 'more', 'vi', etc)
      #!/bin/bash
      # llsx - List files in directory, in Size order, Biggest 1st
      echo "llsx - list directory sorted by file-size (biggest files 1st)"
      if  [[ -z "$1" ]] ; then dir="."; else dir=$1; fi
      ls -lS $dir | more
      #=================

We do not see anything wrong with BAD script (sf/llsx), so let's compare to the GOOD script (sf/llsy).


 #3b. cat sf/llsy   <-- display the script, to see what's wrong
      ===========       (or could use 'more', 'vi', etc)
      #!/bin/bash
      # llsy - List files in directory, in Size order, Biggest 1st
      echo "llsy - list directory sorted by file-size (biggest files 1st)"
      if  [[ -z "$1" ]] ; then dir="."; else dir=$1; fi
      ls -lS $dir | more
      #=================

We see NO DIFFERENCE between the BAD & GOOD scripts, using unix system utilities (cat, more, vi, lp, etc). We will try 'uvhd' below:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10B2. using 'uvhd' to find problems in files

run uvhd to see problems not shown with 'vi'

'uvhd' will display any file in "Vertical Hexadecimal". uvhd expects the filename as arg1 & filetype options as arg2. If you do not know the filetype options, you can first try uvhd without options & hopefully see what the options should be.


 #4. uvhd sf/llsx       <-- NOTE - arg2 filetype options omitted
     ============
     rec#=1 rsize=244 fptr=0 fsize=244 records=0
                  10        20        30        40        50        60
 r#    1 0123456789012345678901234567890123456789012345678901234567890123
       0 #!/bin/bash..# llsx - List files in directory, in Size order, Bi
         2226662667600226677222467726666726626676676772266256762676672246
         31F29EF2138DA30CC380D0C934069C5309E0492534F29C09E039A50F2452C029
      64 ggest 1st..echo "llsx - list directory sorted by file-size (bigg
         6667723770066662266772226677266766767727677662672666627676226666
         775340134DA538F02CC380D0C9340492534F2903F2454029069C5D39A5082977
     128 est files 1st)"..if  [[ -z "$1" ]] ; then dir="."; else dir=$1;
         6772666672377220066225522722232255232766626673222326676266732332
         534069C53013492DA9600BB0DA024120DD0B0485E0492D2E2B05C350492D41B0
     192 fi..ls -lS $dir | more..#=================..exit 0..
         6600672265226672726676002333333333333333330067672300
         69DAC30DC3044920C0DF25DA3DDDDDDDDDDDDDDDDDDA589400DA

When arg2 options are omitted, they default to: 'r256l64'. 'r256' is the default Record-size & 'l64' is default Line display size.

In this case (since the entire script file is less than 256, it displays as 1 block of 4 64 byte segments of 3 vertical hexadecimal line groups.

In the 3 line groups of Vertical Hexadecimal the ASCII characters are displayed on the 1st line, the zones on the 2nd line,& the digits on the 3rd line.

For example, the 1st byte of the file is a '#' & directly below you see the (horizontal) hexadecimal value is x'23' (zone 2 & digit 3). You can easily find an ASCII code chart (on the internet) to confirm this.

For a 2nd example, the 4th byte is 'b', which is x'62' in horizontal hex.

In the display above, the 7 records in the file are all run together in a block of 244 bytes, because we did not give uvhd any filetype options that would separate the records.

But we know that script files are TEXT files, so we need to specify the "text file option" to uvhd, which is simply 't'.

See our next attempt with option 't' on the page below:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10B3. using 'uvhd' to find problems in files

uvhd default option r256, option 't' for text files

Since we know script files are 'text' files, we will use the 't' option.


 #5. uvhd sf/llsx t  <-- rerun with option 't' separate lines on LineFeeds x'0A'
     ==============
     rec#=1 rsize=13 fptr=0 fsize=244 records=7
                      10
 r#        1 0123456789012
           0 #!/bin/bash..
             2226662667600
             31F29EF2138DA
     rec#=2 rsize=62 fptr=13 fsize=244 records=7
                      10        20        30        40        50        60
 r#        2 01234567890123456789012345678901234567890123456789012345678901
          13 # llsx - List files in directory, in Size order, Biggest 1st..
             22667722246772666672662667667677226625676267667224666677237700
             30CC380D0C934069C5309E0492534F29C09E039A50F2452C029775340134DA
                 --- showing 1st 2 lines, omitting last 5 lines ---

Can you see the problem using uvhd, that we could not see using cat,vi,etc ? We could display the good script demo/sf/llsy & compare:


 #6. uvhd sf/llsy t    <-- display good script & compare to bad
     ==============
     rec#=1 rsize=12 fptr=0 fsize=237 records=7
                      10
 r#        1 012345678901
           0 #!/bin/bash.
             222666266760
             31F29EF2138A
     rec#=2 rsize=61 fptr=12 fsize=237 records=7
                      10        20        30        40        50        60
 r#        2 0123456789012345678901234567890123456789012345678901234567890
          12 # llsy - List files in directory, in Size order, Biggest 1st.
             2266772224677266667266266766767722662567626766722466667723770
             30CC390D0C934069C5309E0492534F29C09E039A50F2452C029775340134A
                 --- showing 1st 2 lines, omitting last 5 lines ---
 You can see the difference:
 - the bad script (sf/llsx) lines end with x'0D' & x'0A'.
 - the good script (sf/llsy) lines end with just x'0A'.
 - it is the x'0D' Carriage returns (CRs) that cause the problem
 - the shell was searching for the script interpreter /bin/bash+x'CR'
 - and gives the errmsg "bash: sf/llsx: /bin/bash^M: bad interpreter"

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10B4. using 'uvhd' to find problems in scripts

Fixing the problem - Replacing Carriage Returns (CRs)


 #7. cp backup/sf/llsx sf/  <-- restore bad file (in case fixed by prior run)
     =====================

 #8. uvhd sf/llsx tu  <-- run with option 't' & 'u' to allow Update
     ===============
 r#    1 0123456789012
       0 #!/bin/bash..
         2226662667600
         31F29EF2138DA
 rec#=1 rsize=13 fptr=0 fsize=244 records=7
 Enter command, or ?=Help, q=quit, null=NextRecord -->

 #8a. --> v99 x'0D',x'20'  <-- search/replace x'0D' with x'20' (space)
          ===============
          v99 x'0D',' '    <-- could code space as ' ' vs x'20'

 #8b. --> 1  <-- enter 1 to return to 1st record in file (to compare)
          ===
 r#    1 012345678901
       0 #!/bin/bash.
         2226662667620     <-- Note CR x'0D' now  replaced with x'20' (space)
         31F29EF21380A

 #8c. --> q       <-- reply 'q' to quit
          ===

REMOVING Carriage Returns (CRs) vs REPLACING (above)


 #9. cp backup/sf/llsx sf/  <-- restore bad file (in case fixed by prior run)
     =====================

 #10. uvhd sf/llsx tu  <-- run with option 't' & 'u' to allow Update
      ===============
 r#    1 0123456789012
       0 #!/bin/bash..
         2226662667600
         31F29EF2138DA
 rec#=1 rsize=13 fptr=0 fsize=244 records=7
 Enter command, or ?=Help, q=quit, null=NextRecord -->

 #10a. --> v99 x'0D',''   <-- search/replace x'0D' with '' (null)
           ============

 #10b. --> 1  <-- enter 1 to return to 1st record in file (to compare)
          ===
 r#    1 012345678901
       0 #!/bin/bash.
         222666266760   <-- Note CR x'0D' now removed
         31F29EF2138A

 #10c. --> q       <-- reply 'q' to quit
           ===

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10B5. using 'uvhd' to find problems in scripts

Notes re Updating with uvhd

  1. Must enter option 'u' on the command line if intent to update

  2. 'v' is command to search/replace, 99 of 'v99' is the number of records to process, but '99' has special meaning to process the entire file, resets file pointer to 1st record & processes all records until EOF.

  3. You could remove the CR by replacing with null as follows:

 #7a. --> v99 x'0D',''   <-- search/replace x'0D' with null ''
          ============
  1. Replacing with null will work in this case, but should normally be avoided. Normally, you should replace with an equal length string, because uvhd updates "in place", so the record rewritten will be shifted left at the replace point, inserting a blank at the end of the record, following the LineFeed x'0A', so we will get a blank ' ' at the begining of all records after 1st update, which will not be a problem for script execution, because scripts bypass blanks. But this would likely be a problem for most data files.

  2. See next page for an alternate solution using 'uvcp' which can remove the CR x'0D' & shift the LF x'0A' left 1 byte, shortening the records. (vs uvhd which always rewrites same size records). uvcp can do this because it copies the file record by record (vs uvhd update in place).

  3. See all uvhd documentation at uvsoftware.ca/uvhd.htm, OR enter '?' at the uvhd prompt for 25 HELP screens.

  4. You can change the default options by adding export UVHDROP=... to your profile (or the common_profile), for example, I often use 'export UVHDROP=l100' for Line length 100 bytes (vs default 64).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10B6. using 'uvhd' to find problems in scripts

Alternate solutions (vs uvhd) to Remove CRs from files

There are several alternate ways (vs uvhd shown above) to remove CR's from files: You may need solution #5 to fix all files in a directory.

  1. 'removeCR' script to copy file using --> tr -d '\r' <filein >fileout <-- - you can see this script at /home/uvadm/sf/util/removeCR

  2. 'uvcp' to copy the file using instruction --> rep=0(256):0x0D:: <--

  3. 'uvcopy', copies file same as uvcp, instruction --> rep 0(256),x'0D','' <--

  4. 'uvfix1 - pre-programmed uvcopy job prompts for instructions (rep above) - saves you from having to write the uvcopy file I/O framework - you can see this script at /home/uvadm/sf/util/uvfix1

  5. 'uvfixA - same as uvfix1, but copies All files from 1 directory to a 2nd directory applying the instructions entered at the prompt. - you can see this script at /home/uvadm/sf/util/uvfix1A.

    1. Remove CRs using script 'removeCR'

We will use 'removeCR' to copy the bad script from demo/sf/llsx to tmp/llsx while removing the CR's & then prompt to copy back overwriting original. You could inspect in tmp/llsx before copying back (but you can be confident).


 #0a. login userxx --> /home/userxx
 #0b. cd demo      --> /home/userxx/demo

 #1a. cp backup/sf/llsx sf/  <-- restore bad file (in case fixed by prior run)
      =====================

 #1b. removeCR sf/llsx    <-- call removeCR to remove CR's
      ================      - log listed below:
      removeCR  - copy a file to tmp/... removing CRs & nulls, prompt to copy back
      input  lines & bytes:           7 sf/llsx 244	sf/llsx
      output lines & bytes:           7 tmp/llsx 237	tmp/llsx
      copy back overwriting original y/n ? --> y <-- reply 'y'
      copied back to sf/llsx
      -rwxrwxr-x 1 uvadm apps 244 Feb 21 17:28 sf/llsx

 #1c. uvhd sf/llsx t      <-- check results (uvhd display not shown)
      ==============        - same as #10b on page '10B4'
Note
  • see the 'removeCR script at /home/uvadm/sf/util/removeCR
  • Or see at uvsoftware.ca/scripts1.htm#6A1
  • also see removeCRall to process all files in directory.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10B7. Alternate solutions (vs uvhd) to Remove CRs from files

2. Remove CRs using 'uvcp'

We will use 'uvcp' to copy the bad script from demo/sf/llsx to tmp/llsx, while removing CR's. You could inspect with 'uvhd tmp/llsx t' before copying back with 'cp'.


 #2a. cp backup/sf/llsx sf/  <-- restore bad file (in case fixed by prior run)
      =====================

 #2b. uvcp "fili1=sf/llsx,rcs=256,typ=LSTt,filo1=tmp/llsx,rep=0(256),x'0D'::"
      =======================================================================

 #2c. uvhd tmp/llsx t      <-- inspect with uvhd, confirm CR's removed
      ===============
      r#    1 012345678901
            0 #!/bin/bash.
              222666266760      <-- Note CR x'0D' now removed
              31F29EF2138A

 #2d. cp tmp/llsx sf/      <-- copy back to sf/...
      ===============

 #2e. chmod 775 sf/llsx    <-- restore execute bits on script file
      =================

We need to restore the execute bits on all 5 solutions, except #3, since uvcopy allows file typ p775 to set output permissions.

See 'uvcp' documentation at uvsoftware.ca/uvcp.htm

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10B8. Alternate solutions (vs uvhd) to Remove CRs from files

3. Remove CRs using 'uvcopy'

'uvcopy' is similar to 'uvcp', much more powerful, but does require a separate "Parameter File" (or "uvcopy job") vs uvcp which accepts all instructions on the command line. We will 1st edit a "uvcopy job" & then execute with uvcopy.


 #3a. cp backup/sf/llsx sf/  <-- restore bad file (in case fixed by prior run)
      =====================

 #3b. vi pf/removeCR_uvcopy   <-- create uvcopy job to remove CR's
      =====================     - listed below:
      # removeCR_uvcopy - copy a file removing any CR's
      #                 - by Owen Townsend, UV Software, Feb 21/2019
      # - documented at uvsoftware.ca/uvdemos.htm#10B8
      # - demo Vancouver Utilities usefulness to any unix/linux user
      #
      # uvcopy pf/removeCR_uvcopy    <-- execution command
      # =========================
      opr='$jobname - copy a file removing any Carriage Returns'
      fili1=?sf/llsx,rcs=256,typ=LST        #<-- declare input file
      filo1=?tmp/llsx,rcs=256,typ=LSTtp775  #<-- declare output file
      @run
              opn     all                    open files
      # begin loop to read/process/write records until EOF
      man20   get     fili1,a0               get next record into area 'a'
              skp>    man90                  (cc set > at EOF)
              mvc     b0(256),a0             copy input area 'a' to output area 'b'
              rep     b0(256),x'0D',''       remove CRs
              put     filo1,b0               write record to output file
              skp     man20                  return to get next record
      # EOF - close files & end job
      man90   cls     all
              eoj

 #3c. uvcopy pf/removeCR_uvcopy     <-- execute uvcopy job
      =========================
      - no need to define files on command line, since coded in job as the defaults

      uvcopy pf/removeCR_uvcopy,fili1=sf/llsx,filo1=tmp/llsx
      ======================================================
      - but could define on command line when different than the defaults
Note
  • the '?'s on the file declarations (fili1=?sf/llsx,... & filo1=?tmp1/llsx).
    cause prompts for alternate filename keyin, null entry for default coded.

 #3d. uvhd tmp/llsx t    <-- inspect with uvhd, confirm CR's removed
      ===============      - display same as 2c on page '10B7'

 #3e. cp tmp/llsx sf/    <-- copy back to sf/...
      ===============

 #3f. chmod 775 sf/llsx  <-- restore execute bits <-- do NOT need
      =================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10B9. Alternate solutions (vs uvhd) to Remove CRs from files

Notes re uvcopy jobs

  1. Here is the one vital instruction in the uvcopy job listed above.

         rep     b0(256),x'0D',''       remove CRs
    ==============================================
  1. The other instructions are the framework to define the files, and to read & write the records.

  2. Solution #4 will use script 'uvfix1' to supply the framework & the only instruction you have to enter is the 'rep' (as shown above).

  3. Note that we stored our uvcopy job in subdir 'pf' (Parameter Files) & the execution command was:

      uvcopy pf/removeCR_uvcopy     <-- execute uvcopy job
      =========================
  1. We had to code the subdir because /home/userxx/demo/pf/ is not defined in the common_profile as part of export PFPATH=...

  2. You do not need to code the directories on the many pre-programmed uvcopy jobs mentioned in this documentation, because they are in PFPATH. Here is the definition of PFPATH in $APPSADM/env/common_profile_uv:

    export PFPATH=$UV/pf/adm:$UV/pf/demo:$UV/pf/util:$UV/pf/IBM:$HOME/pf
    ====================================================================
  1. You can see that this is similar to export PATH=... used in profiles to define the directories for system & user programs.

  2. PFPATH includes $HOME/pf/, but not $HOME/demo/pf/, so that is why we needed to code pf/... on our execution --> uvcopy pf/removeCR_uvcopy

  3. Alternatively, we could add $HOME/demo/pf to PFPATH in the common_profile.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10B10. Alternate solutions (vs uvhd) to Remove CRs from files

4. Remove CRs with 'uvfix1' pre-programmed uvcopy job

'uvfix1' is a pre-programmed uvcopy job that prompts for instructions & saves you from having to write the uvcopy file I/O framework.

'uvfix1' works the same way as above solutions, it will copy the bad script from sf/llsx to tmp/llsx (while applying the 'rep' instruction). You can then inspect the result in tmp/llsx & if satisfied, copy back from tmp/llsx to sf/llsx.


 #4a. cp backup/sf/llsx sf/  <-- restore bad file (in case fixed by prior run)
      =====================

 #4b. uvfix1 sf/llsx    <-- call uvfix1, arg1 specifies the file to be modified
      ==============      - console log with prompts shown below:
      uvfix1 - copy a text file & modify via uvcopy instruction keyins
             - input rec in area a has already been copied to output rec in b
      uop=q1c0n99r8192 - user option defaults
            c1         - pass thru any "#" col1 #comment records
              n99      - output record count limit (high# copies to EOF)
                 r8192 - recsize ONLY for Fixed-Length OUTPUT records if desired
                       - for Text files: accept default recsize & default typ=LSTt
      User OPtion (uop) defaults  = q1c0n999999999r8192
 #4c. -->null to accept or enter/override -->  <-- enter null to accept default options
      typ=LSTt default, null accept or enter new typ -->   <-- null accept default typ
       mvc  b27(25),a0  <-- sample instrns, dup cols 1-25 to 27-51
                          - moves from op2 area a to op1 area b
      enter a uvcopy instruction (period "." terminates entries)

 #4d. ---> rep b0(256),x'0D',''   <--- ENTER rep instrn
           ====================
 #4e. ---> .                      <--- ENTER '.' to continue
      190222:092255:uvfix1: EOF fili01 rds=7 size=244: sf/llsx
      190222:092255:uvfix1: EOF filo01 wrts=7 size=237: tmp/llsx
      check output in tmp/llsx, then move back to sf/llsx ?

 #4f. uvhd tmp/llsx t        <-- check output in tmp/llsx, see if OK ?
      ===============
      r#    1 012345678901
            0 #!/bin/bash.
              222666266760   <-- Yes, OK CR x'0D' removed
              31F29EF2138A

 #4g. cp tmp/llsx sf/        <-- copy back overwriting bad script
      ================
Note
  • see 'uvfix1' documented at uvsoftware.ca/uvhelp.htm#V42
  • see 'uvfix1 script' at /home/uvadm/sf/util/uvfix1, which calls
    uvcopy job 'uvfix1' stored at /home/uvadm/pf/util/uvfix1.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10B11. Alternate solutions (vs uvhd) to Remove CRs from files

5. Remove CRs with 'uvfixA' pre-programmed uvcopy job

'uvfixA' is similar to 'uvfix1', But it processes ALL files in a directory. You must specify arg1=InDir, arg2=OutDir,& arg3=filetype (LSTt in this case).

You could run the same fix as above (removing CRs from scripts) & it would not change anything in scripts that did not have CRs. We will use tmp1 as output directory.


 #5a. cp backup/sf/llsx sf/  <-- restore bad file (in case fixed by prior run)
      =====================

 #5b. mkdir tmp1        <-- make output directory if not already present
      ==========
      rm -f tmp1/*      <-- OR, remove all files if tmp1 already present
      ============

 #5c. uvfixA sf tmp1 LSTt   <-- call uvfixA arg1/arg2 I/O directories & arg3 file type
      ===================     - console log with prompts shown below:
      uvfixA - copy/modify all text files from 1 directory to a 2nd directory
             - modifying the files via uvcopy instruction keyins
             - input rec in area a has already been copied to output rec in b
      uop=q1c0n99r8192 - user option defaults
            c1         - pass thru any "#" col1 #comment records
              n99      - output record count limit (high# copies to EOF)
                 r8192 - recsize ONLY for Fixed-Length OUTPUT records if desired
                       - for Text files: accept default recsize & default typ=LSTt
      User OPtion (uop) defaults  = q1c0n999999999r8192
 #5d. -->null accept or enter/override -->    <-- enter null (accept default options)
 #5e. did you create outdir or remove files ? --> y  <--ENTER "y"
          clr  b0(6)," "     <-- sample, clear 1-6 (use single quotesnot doubles)
          add  $ca1,1        <-- sample, increment counter #1
          mvn  b0(4),$ca1    <-- sample, insert sequence# in 1-4
      enter a uvcopy instruction (period "." terminates entries)

 #5f. ---> rep b0(256),x'0D',''   <--- ENTER rep instrn
           ====================
 #5g. ---> .                      <--- ENTER '.' to continue
      190222:101902:uvfixA: EOF fili01 rds=7 size=244: sf/llsx
      190222:101902:uvfixA: EOF filo02 wrts=7 size=237: tmp1/llsx
      190222:101902:uvfixA: EOF fili01 rds=7 size=244: sf/llsy
      190222:101902:uvfixA: EOF filo02 wrts=7 size=237: tmp1/llsy
      190222:101902:uvfixA: EOF fild02 size=4096: tmp1

 #5h. uvhd tmp1/llsx t   <-- check 1 or more in tmp1/llsx, see if OK ?
      ================

 #5i. cp tmp1/* sf/      <-- copy All scripts back overwriting originals
      =============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10C1. UVdemos - Vancouver Utilities for Unix/Linux users

high-bit characters may cause problems for old applications

Files with high-bit characters may cause problems for older applications. For Example - files with UTF-8 characters input to legacy appliciations, not designed to handle them. This demo will show you how to identify them with uvhd & convert them to unaccented characters with uvhd or uvcp.

Let's suppose I get an email from Therese asking me how to convert accented French characters to unaccented equivalents. She knows the problem is that the new files are using UTF-8 character encoding & has identified the French accented characters & their UTF-8 codes in hexadecimal.

dat2/accents1 - French UTF-8 accented characters


 #0a. login userxx --> /home/userxx
 #0b. cd demo      --> /home/usrxx/demo

 #1. cat dat2/accents1   <-- display sample file with UTF-8 chars
     =================
      Dear Owen: Re: French Accents (dat2/accents1)
      11. cedilla      c ---> Ç ---> x'C387' ---> x'63' c
      21. accent acute e ---> é ---> x'C3A9' ---> x'65' e
      31. accent grave a ---> à ---> x'C3A0' ---> x'61' a
      32.              e ---> è ---> x'C3A8' ---> x'65' e
      33.              u ---> ù ---> x'C3B9' ---> x'75' u
      41. dieresis     e ---> ë ---> x'C3AB' ---> x'65' e
      42.              i ---> ï ---> x'C3AF' ---> x'69' i
      43.              u ---> ü ---> x'C3BC' ---> x'75' u
      51. circumflex   a ---> â ---> x'C3A2' ---> x'61' a
      52.              e ---> ê ---> x'C3AA' ---> x'65' e
      53.              i ---> î ---> x'C3AE' ---> x'69' i
      54.              o ---> ô ---> x'C3B4' ---> x'6F' o
      55.              u ---> û ---> x'C3BB' ---> x'75' u
      UTF-8 2 byte codes (1st byte x'C3')
      How can we convert to unaccented characters ?
      Thanks Thérèse
Note
  • If you have a printed copy of this documentation, you will probably see
    that the UTF-8 accented chars print differently than the screen display,
    probably because the printer is not setup for UTF-8 characters.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10C2. High-Bit characters may cause problems for Legacy Applications

Investigating Accented Characters with 'uvhd'

Note
  • The file shown above would take too many lines in this documentation
  • so we will use a shorter version (dat2/accents2 vs dat2/accents1).

 #2. cat dat2/accents2   <-- display short version of UTF-8 sample file
     =================
      Dear Owen: Re: French Accents (dat2/accents2)
      1. accent grave a ---> à ---> x'C3A0' ---> x'61' a
      2. accent acute e ---> é ---> x'C3A9' ---> x'65' e
      3. accent grave e ---> è ---> x'C3A8' ---> x'65' e
      How can we convert to unaccented characters ?
      Thanks Thérèse

 #3. uvhd dat2/accents2 t  <-- investigate with 'uvhd'
     ====================    - using option 't' for Text files
 rec#=1 rsize=46 fptr=0 fsize=265 records=6
                   10        20        30        40
 r#     1 0123456789012345678901234567890123456789012345
        0 Dear Owen: Re: French Accents (dat2/accents2).
          4667247663256324766662466667722667326666677320
          45120F75EA025A0625E3801335E43084141F1335E4329A
 rec#=2 rsize=52 fptr=46 fsize=265 records=6
                   10        20        30        40        50
 r#     2 0123456789012345678901234567890123456789012345678901
       46 1. accent grave a ---> .. ---> x'C3A0' ---> x'61' a.
          32266666726767626222232CA222232724343222223272332260
          1E01335E4072165010DDDE0300DDDE087331070DDDE08761701A
 rec#=3 rsize=52 fptr=98 fsize=265 records=6
                   10        20        30        40        50
 r#     3 0123456789012345678901234567890123456789012345678901
       98 2. accent acute e ---> .. ---> x'C3A9' ---> x'65' e.
          32266666726677626222232CA222232724343222223272332260
          2E01335E4013545050DDDE0390DDDE087331970DDDE08765705A
 rec#=4 rsize=52 fptr=150 fsize=265 records=6
                   10        20        30        40        50
 r#     4 0123456789012345678901234567890123456789012345678901
      150 3. accent grave e ---> .. ---> x'C3A8' ---> x'65' e.
          32266666726767626222232CA222232724343222223272332260
          3E01335E4072165050DDDE0380DDDE087331870DDDE08765705A
 rec#=5 rsize=46 fptr=202 fsize=265 records=6
                   10        20        30        40
 r#     5 0123456789012345678901234567890123456789012345
      202 How can we convert to unaccented characters ?.
          4672666276266676772762766666676626667667677230
          8F7031E07503FE652404F05E1335E454038121345230FA
 rec#=6 rsize=17 fptr=248 fsize=265 records=6
                   10
 r#     6 01234567890123456
      248 Thanks Th..r..se.
          566667256CA7CA760
          481EB30483923835A

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10C3. Investigating Accented Characters with 'uvhd'

Notes re UTF-8 French Accented Characters

We will use record #2 (isolated from 6 uvhd records above), as the basis for the notes following.

                   10        20        30        40        50
 r#     2 0123456789012345678901234567890123456789012345678901
       46 1. accent grave a ---> .. ---> x'C3A0' ---> x'61' a.
          32266666726767626222232CA222232724343222223272332260
          1E01335E4072165010DDDE0300DDDE087331070DDDE08761701A
  1. The vertical hex for grave 'a' under scale bytes 23 & 24 are shown again in horizontal hex under bytes 33-36 as x'C3A0', followed by the unaccented equivalent 'a' under scale 46-47 as x'61'.

  2. x'C3A0' is UTF-8 coding for grave 'a'. UTF-8 uses just 1 byte for the 1st 128 bytes of the ASCII code table. The next 128 (hi-bit) ASCII codes use 2 bytes & the 1st byte is x'C2' for 128-191 & x'C3' for 192-255. The French accents all have 1st byte x'C3'.

  3. So, to convert the French accented characters to unaccented equivalents, we could remove the 1st byte x'C3' & convert the 2nd byte to the unaccented equivalent. For this example grave a 2nd byte x'A0' to x'61' unaccented 'a'.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10C4. Updating files with uvhd

converting accented characters to unaccented equivalents

For this demo, we will convert only the subset of 3 French characters. We will use uvhd to convert the x'C3' to space x'20' & the 3 French accents as shown below:

      1. accent grave a ---> à ---> x'C3A0' ---> x'61' a
      2. accent acute e ---> é ---> x'C3A9' ---> x'65' e
      3. accent grave e ---> è ---> x'C3A8' ---> x'65' e

 #0a. login userxx --> /home/userxx
 #0b. cd demo      --> /home/usrxx/demo
 #0c. cp dat2/accents2.bak dat2/accents2  <-- restore file if re-testing
      =================================

 #1. uvhd dat2/accents2 tu
     =====================
                  10        20        30        40
      r#    1 0123456789012345678901234567890123456789012345
            0 Dear Owen: Re: French Accents (dat2/accents2).
              4667247663256324766662466667722667326666677320
              45120F75EA025A0625E3801335E43084141F1335E4329A
              --- uvhd shows 1st record & prompts for your command:

 #1a. v99 x'C3',x'20'    <-- search/replace 1st byte UTF-8 to space
      ===============
      v99 x'C3',' '      <-- can code the space as character vs hex
      r#    6 01234567890123456
          248 Thanks Th .r .se.
              5666672562A72A760
              481EB30480920835A
      EOF, 6 read, 5 reps in 4 records, v99 x'C3',x'20'
      --- v99 searches/replaces entire file, shows last record updated,
          reports results as shown above, & prompts for next command

 #1b. v99 x'A0','a'  <-- search/replace grave 'a' with unaccented 'a'
      =============
      r#    2 0123456789012345678901234567890123456789012345678901
           46 1. accent grave a --->  a ---> x'C3A0' ---> x'61' a.
              3226666672676762622223226222232724343222223272332260
              1E01335E4072165010DDDE0010DDDE087331070DDDE08761701A
      EOF, 6 read, 1 reps in 1 records, v99 x'A0','a'

 #1c. v99 x'A9','e'  <-- search/replace acute 'e' with unaccented 'e'
      =============

 #1d. v99 x'A8','e'  <-- search/replace grave 'e' with unaccented 'e'
      =============

 #1e. q              <-- quit uvhd
      ===
Note
  • we have omitted showing results of #1c & #1d updates to save space
  • see the updated file on the page below:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10C5. converting accented characters to unaccented equivalents

Results Before & After conversion


 #2. cat dat2/accents2.bak   <-- show demo file Before updates
     =====================
      Dear Owen: Re: French Accents (dat2/accents2)
      1. accent grave a ---> à ---> x'C3A0' ---> x'61' a
      2. accent acute e ---> é ---> x'C3A9' ---> x'65' e
      3. accent grave e ---> è ---> x'C3A8' ---> x'65' e
      How can we convert to unaccented characters ?
      Thanks Thérèse

 #3. cat dat2/accents2        <-- show demo file After updates
     =================
      Dear Owen: Re: French Accents (dat2/accents2)
      1. accent grave a --->  a ---> x'C3A0' ---> x'61' a
      2. accent acute e --->  e ---> x'C3A9' ---> x'65' e
      3. accent grave e --->  e ---> x'C3A8' ---> x'65' e
      How can we convert to unaccented characters ?
      Thanks Th er ese

Results of updates with uvhd are not perfect, because we converted the 1st byte of UTF-8 codes to space (as we see in 'Th er ese').

We could have removed 1st byte by converting to null (v99 x'C3','') but should not do with uvhd, because it updates in place & rewriting would shift the record terminating LineFeed x'0A' left 1, thus creating a space prior to the next record.

But, we can remove the 1st byte x'C3's by using uvcp because we will copy the file while removing x'C3's & converting the accents to equivalents.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10C6. converting accented characters to unaccented equivalents

converting accented characters with 'uvcp'


 #0a. login userxx --> /home/userxx
 #0b. cd demo      --> /home/usrxx/demo

 #0c. cp dat2/accents2.bak dat2/accents2  <-- restore file if re-testing
      =================================

 #1. rm -f tmp/*     <-- remove any old files from tmp/...
     ===========         (uvcp will write output file to tmp/...)

 #2. uvcp "fili1=dat2/accents2,rcs=256,typ=LSTt,filo1=tmp/accents2,rep=0(256):0xC3:\
                        ,rep=0(256):0xA0:0x61,rep=0(256):0xA9:0x65,rep=0(256):0xA8:0x65"
     ===================================================================================
      alternative --->  ,rep=0(256):0xA0:a,rep=0(256):0xA9:e,rep=0(256):0xA8:e"
      NOTE - see script equivalent of this long command listed at #4. below
           - much easier to run with the script

 #3. cat tmp/accents2   <-- confirm output results as desired
     ================
      Dear Owen: Re: French Accents (dat2/accents2)
      1. accent grave a ---> a ---> x'C3A0' ---> x'61' a
      2. accent acute e ---> e ---> x'C3A9' ---> x'65' e
      3. accent grave e ---> e ---> x'C3A8' ---> x'65' e
      How can we convert to unaccented characters ?
      Thanks Therese
Note
  • We have provided the uvcp command as a script in demo/sf/fixaccents2
  • you can inspect the script & execute as follows:

 #4. cat sf/fixaccents2     <-- inspect uvcp script to remove accents
     ==================       - script same as manually entered uvcp command above
                                but with #comment explanations added
      #!/bin/ksh
      # fixaccents2 - uvcp script to convert French accents to unaccented equivalents
      #             - see demo in www.uvsoftware.ca/uvdemos.htm#10C6
      uvcp "fili1=?dat2/accents2,rcs=256,typ=LSTt,filo1=?tmp/accents2,rep=0(256):0xC3:\
                           ,rep=0(256):0xA0:0x61,rep=0(256):0xA9:0x65,rep=0(256):0xA8:0x65"
      #====================================================================================
      #   alternative ---> ,rep=0(256):0xA0:a,rep=0(256):0xA9:e,rep=0(256):0xA8:e"
      #
      # - could have coded the replacements as characters vs hex codes as shown above

 #5. sf/fixaccents2     <-- execute uvcp script to remove accents
     ==============

The easiest way to convert accented characters to unaccented equivalents would be to use 'uvfix1' the pre-programmed uvcopy job, that prompts for instructions (& saves writing the uvcopy job framework). We will run uvfix1 for dat2/accents1 which has all 13 French accents (vs accents2 demo with only 3).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10C7. converting accented characters to unaccented equivalents

converting accented characters with 'uvfix1'


 #0a. login userxx --> /home/userxx
 #0b. cd demo      --> /home/usrxx/demo

 #1. uvfix1 dat2/accents1    <-- call uvfix1, arg1 specifies the file to be modified
     ====================      - console log with prompts shown below:
      uvfix1 - copy a text file & modify via uvcopy instruction keyins
             - input rec in area a has already been copied to output rec in b
      uop=q1c0n99r8192 - user option defaults
            c1         - pass thru any "#" col1 #comment records
              n99      - output record count limit (high# copies to EOF)
                 r8192 - recsize ONLY for Fixed-Length OUTPUT records if desired
                       - for Text files: accept default recsize & default typ=LSTt
      User OPtion (uop) defaults  = q1c0n999999999r8192
 #1a. -->null to accept or enter/override -->  <-- enter null to accept default options
      typ=LSTt default, null accept or enter new typ -->   <-- null accept default typ
      mvc  b27(25),a0  <-- sample instrns, dup cols 1-25 to 27-51
                         - moves from op2 area a to op1 area b
      enter a uvcopy instruction (period "." terminates entries)

 #1b. ---> rep b0(256),x'C3',''   <--- ENTER rep to remove 1st byte of UTF-8 pairs
           ====================
 #1c. ---> repm b0(256),x'87A9A0A8B9ABAFBCA2AAAEB4BB','ceaeueiuaeiou'
           =========================================================
            - ENTER repm to REPlace Multiple bytes with alternates
            - specifying searches in hexadecimal & replacements as characters
 #1d. ---> .                      <--- ENTER '.' to continue
           190225:140942:uvfix1: EOF fili01 rds=17 size=834: dat2/accents1
           190225:140942:uvfix1: EOF filo01 wrts=17 size=819: tmp/accents1
           check output in tmp/llsx, then move back to sf/llsx ?

 #2. cat tmp/accents1      <-- inspect output results
     ================
      Dear Owen: Re: French Accents (dat2/accents1)
      11. cedilla      c ---> c ---> x'C387' ---> x'63' c
      21. accent acute e ---> e ---> x'C3A9' ---> x'65' e
      31. accent grave a ---> a ---> x'C3A0' ---> x'61' a
      32.              e ---> e ---> x'C3A8' ---> x'65' e
      33.              u ---> u ---> x'C3B9' ---> x'75' u
      41. dieresis     e ---> e ---> x'C3AB' ---> x'65' e
      42.              i ---> i ---> x'C3AF' ---> x'69' i
      43.              u ---> u ---> x'C3BC' ---> x'75' u
      51. circumflex   a ---> a ---> x'C3A2' ---> x'61' a
      52.              e ---> e ---> x'C3AA' ---> x'65' e
      53.              i ---> i ---> x'C3AE' ---> x'69' i
      54.              o ---> o ---> x'C3B4' ---> x'6F' o
      55.              u ---> u ---> x'C3BB' ---> x'75' u
      UTF-8 2 byte codes (1st byte x'C3')
      How can we convert to unaccented characters ?
      Thanks Therese

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10D1. UVdemos - Vancouver Utilities for Unix/Linux users

convert Canadian MPs from csv to fixed field text

 # CanadaMPscsv2txt - convert Canadian MPs from csv to fixed field text
 #                  - by Owen Townsend, UV Software, May12/2019, updated Jul06/2019
 #                  - for various UVdemos, see www.uvcoftware.ca/uvdemos.htm
 # CanadaMPscsv2txt     - for download with Windows 3.1 Latin 1 accented chars
 # CanadaMPscsv2txtUTF8 - for original download with UTF8 chars for accented French chars
 #
 # uvcopy CanadaMPscsv2txt,fili1=dat0/CanadaMPs.csv,filo1=tmp1/CanadaMPs  <-- Note tmp1/ output
 # ======================================================================
 # uvcopy CanadaMPscsv2txt  <-- same, filenames hardcoded defaults below
 # ========================
 #
 #                 ------- sample INPUT dat0/CanadaMPs.csv -------
 #Title,First Name,Last Name,Constituency,Province / Territory,Political Affiliation,Start Date,End Date
 #,Charlie,Angus,Timmins/James Bay,Ontario,NDP,10/19/2015 12:00:00 AM,
 #,Xavier,Barsalou-Duval,Pierre-Boucher/Les Patriotes/Verchères,Quebec,Bloc Québécois,10/19/2015 12:00:00 AM,
 #,Elizabeth,May,Saanich/Gulf Islands,British Columbia,Green Party,10/19/2015 12:00:00 AM,
 #,Jody,Wilson-Raybould,Vancouver Granville,British Columbia,Independent,10/19/2015 12:00:00 AM,
 #
 #                  ------- sample OUTPUT tmp1/CanadaMPs -------
 #seq# LastName           FirstName       Constituency            Province Party      YearElected
 #         1         2         3         4         5         6         7         8         9       100
 #1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 #                ------------ samples of all parties in 335 Canadaian MPs ------------
 #010  Anderson           David           Cypress Hills/Grasslands      SK Conservative     2015
 #011  Angus              Charlie         Timmins/James Bay             ON NDP              2015
 #023  Barsalou-Duval     Xavier          Pierre-Boucher/Les Patriotes/ QC Bloc Quebecois   2015
 #202  May                Elizabeth       Saanich/Gulf Islands          BC Green Party      2015
 #328  Wilson-Raybould    Jody            Vancouver Granville           BC Independent      2015
 #
 opr='$jobname - convert Canadaian MPs from csv to fixed field text'
 rop=r1   # option to prompt for output file view at EOF
 fili1=?dat0/CanadaMPs.csv,typ=LST,rcs=200
 filo1=?tmp1/CanadaMPs,typ=LST,rcs=200
 # load table of province names & 2 letter abbreviations
 lod=p0(100)
 01  AB  Alberta
 02  BC  British Columbia
 03  MB  Manitoba
 04  NB  New Brunswick
 05  NL  Newfoundland and Labrador
 06  NT  Northwest Territories
 07  NS  Nova Scotia
 08  NU  Nunavut
 09  ON  Ontario
 10  PE  Prince Edward Island
 11  QC  Quebec
 12  SK  Saskatchewan
 13  YT  Yukon
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 # See lok instrn to lookup 8(30) cols 9-38 to get 4(2) cols 5-6
 # - could have loaded table from a file as follows:
 # fili2=dat1/CanadaProvinces,typ=LST,rcs=100
 #    rtb   fili1,d0(100),d0(100)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

 @run
         opn    all                      open files
         get    fili1,a0                 get header record & don't output
 # begin loop to get records, reformat, write output, until EOF
 man20   get    fili1,a0                 get next record
         skp>   man90                    (cc set > at EOF)
         rep    a0(200),x'97','/'        convert constituency separator to '/'
 #
 # convert Windows Latin1 accented chars to unaccented equivalents
         repm   a0(200),x'C9CEE2E7E8E9EBEEF4','EIaceeeio'
 # C9=E acute, CE=I circumflex, E2=a circumflex, E7=c cedilla,
 # E8=e grave, E9=e acute, EB=e diaresis, EE=i circumflex,F4=o circumflex
 #
 # fix csv fields 100 apart, then reformat into desired fixed positions
         fix    b0(100),a0(200),10,','   separate fields 100 apart
         clr    c0(100),' '              clear output record area
         add    $ca1,1                   count records
         mvn    c0(3),$ca1               insert seq#
         mvc    c5(18),b200              last name
         mvc    c24(15),b100             first name
         mvc    c40(29),b300             consituency
         lok    p0(100),p8(30),b400(30)  lookup Prov Abbrev table
         mvc    c70(2),px4               get abbrev code from table lookup
         mvc    c73(15),b500             political party
         scnn2  b600(10),'/'             scan to 2nd '/' of MM/DD/YYYY
         mvc    c90(4),bx601             store year by register 'x' +1
         put    filo1,c0                 write record to output file
         skp    man20                    return to get next record
 #
 # EOF - close files & end job
 man90   cls    all
         eoj

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

10E1. UVdemos - Vancouver Utilities for Unix/Linux users

convert US Senators/Representatives csv to fixed

 # UScongresscsv2txt - convert US Gov Senators/Representatives from csv to fixed field text
 #              - for various UVdemos, see www.uvcoftware.ca/uvdemos.htm
 #              - by Owen Townsend, UV Software, Jul04/2019
 #
 # uvcopy UScongresscsv2txt,fili1=dat0/UScongress.csv,filo1=tmp1/UScongress
 # =================================================================
 # uvcopy UScongresscsv2txt   <-- same, filenames hardcoded defaults in job
 # ========================
 #       -------- sample Input/Output records ----------
 # Abraham,Ralph,,,,Ralph Lee Abraham,1954-09-16,M,rep,LA,5,,Republican,
 # Adams,Alma,,,,Alma S. Adams,1946-05-27,F,rep,NC,12,,Democrat,
 # #        1         2         3         4         5         6         7         8
 # 12345678901234567890123456789012345678901234567890123456789012345678901234567890
 # 001  Abraham             Ralph          19540916  M  LA  rep  Republican
 # 002  Adams               Alma           19460527  F  NC  rep  Democrat
 #
 opr='$jobname - convert US Senators/Representatives from csv to fixed field text'
 rop=r1   # option to prompt for output file view at EOF
 fili1=?dat0/UScongress.csv,typ=LST,rcs=800
 filo1=?tmp1/UScongress,typ=LST,rcs=100
 @run
         opn    all                         open files
         get    fili1,a0                    get header record & don't output
 # begin loop to get records, reformat, write output, until EOF
 man20   get    fili1,a0                    get next record
         skp>   man90                       (cc set > at EOF)
 # remove accents - convert UTF8 double bytes to single byte equivalents
         rep    a0(800),x'C3',''               remove UTF8 lead character
         repm   a0(800),x'A1A9ADB3BA','aeiou'  convert to unaccented equivalent
         rep    a0(800),x'E28099',x'27'        convert backtick to single quote
 # separate csv delimited fields 100 bytes apart for easy reformatting
         fixr2  b100(100),a0(800),13,'","'  separate fields 100 apart
 # remove any double quotes from name-fields & hyphens from date-field
         sqz    b100(40),'"'                remove double quotes from last name
         sqz    b200(40),'"'                remove double quotes from first name
         sqz    b700(10),'-'                remove '-' separators from date-field
 # now reformat input from separated fields into desired column layout
         clr    c0(100),' '                 clear output record area
         add    $ca1,1                      count records
         mvn    c00(3),$ca1                 insert seq#
         mvc    c05(19),b100                last-name
         mvc    c25(14),b200                first-name
         mvc    c40(9),b700                 birthdate
         mvc    c50(1),b800                 M/F
         mvc    c53(2),b1000                state
         mvc    c57(3),b900                 sen/rep
         mvc    c62(20),b1300               party
         put    filo1,c0                    write record to output file
         skp    man20                       return to get next record
 # EOF - close files & end job
 man90   cls    all                         close files
         eoj                                end job

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Permuted Index of Keywords from ** Headings **

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1C2 $home: copy demo files to your $HOME/demo
3B2 $home: copy $UV/demo to your $HOME/demo/...
4B3 $home: copy $UV/demo to your $HOME/demo/...
5C1 $home: copy $UV/demo to your $HOME/demo/...
9N3 $rundata: $RUNDATA/$RUNLIBS & multiple sets LIBrarieS/DATA
9N3 $runlibs: $RUNDATA/$RUNLIBS & multiple sets LIBrarieS/DATA
13D0 $uv: splitfiles3 demo - input $UV/tf/insults_Owens
3B2 $uv: copy $UV/demo to your $HOME/demo/...
4B3 $uv: copy $UV/demo to your $HOME/demo/...
5C1 $uv: copy $UV/demo to your $HOME/demo/...
7A1 $uv: desired Output $UV/demo/dat1/USstates
7F1 $variables: Expanding $VARIABLES
7F2 $variables: testevs2 - 'evs' demo#2 - enter your strings with $Variables
7F3 $variables: expandevs1 - copy a file expanding $VARIABLEs

13B4 &: Easy way to drop CR's & shorten to last non-blank
3E3 >: searching with > & < conditions
3E3 <: searching with > & < conditions

3I1 *arg: Notes Re selectfdd1 selections by *arg1 or arg2
3I2 *arg2: Notes selections by arg1 or *arg2 & options 's1t1'
3G1 *selectlfgfd1: Notes Re *selectlfgfd1 (& selectlgfd2) vs prior selectlf1/selectl
3G2 *selectlfgfd2: Notes Re *selectlfgfd2 (& selectlgfd1) vs prior selectlf1/selectl

3C2 +2@canada: selects/dat1_nameadrs1_+1@Townsend+2@Canada-3@Owen

3B2 ...: copy $UV/demo to your $HOME/demo/...
3D4 ...: reports written to selects/... subdir
3F1 ...: 4 SQL "customer" scripts in sqls/...
3I1 ...: stats Report written to selects/...
3I2 ...: stats Report written to selects/...
4A1 ...: table3d report created in stats/...
4B3 ...: copy $UV/demo to your $HOME/demo/...
5C1 ...: copy $UV/demo to your $HOME/demo/...
5J2 ...: RenameL - 1 example of 37 in sf/util/...
7D1 ...: SQL scripts from mainframe in sql0/...
7D1 ...: SQL scripts on Linux renamed in sqls/...
9B3 ...: INSPECT election1/... data file contents
9G3 ...: Inspect election2/data1/... file CONTENTS
3E1 .csv: samples of Canadian MPs .csv converted to fixed fields
3J1 .csv: samples US Reps & Senators .csv converted to fixed field
4H1 .csv: samples US Reps & Senators .csv converted to fixed field
4K1 .csv: samples of Canadian MPs .csv converted to fixed fields
7A1 .csv: uvcopy/uvfix1 - Convert .csv file to fixed-fields
7B1 .csv: uvfix1 alternate to convert .csv file to fixed field
1B4 .vimrc: - set options for vim
1B4 .vimrc: file contents

5D2 A: 6. cfdd - count Files,Lines,& KB in all subdirs of a superdir
2D3 A=ebcdic->ascii: uvhd options a=EBCDIC->ASCII, r=RecordLth, l=LineLth
10C1 Accented: dat2/accents1 - French UTF-8 accented characters
10C2 Accented: Investigating Accented Characters with 'uvhd'
10C3 Accented: Notes re UTF-8 French Accented Characters
10C4 Accented: converting accented characters to unaccented equivalents
10C6 Accented: converting accented characters with 'uvcp'
10C7 Accented: converting accented characters with 'uvfix1'
2C1 Accents: dat2/accents3 - UTF-8 accents, short test file
10C1 Accents1: dat2/accents1 - French UTF-8 accented characters
2C1 Accents3: dat2/accents3 - UTF-8 accents, short test file
1C1 Account: setup user account to run demos
9R1 Added: upcandimg3 - 2nd step added to demo RESTART feature
3D3 Adm: executing sf/adm/select_nameadrs1
9S1 Advanced: script AFTER adding Advanced Features
9S1 Advanced: script BEFORE adding Advanced Features
9T2 Advanced: REQUIREMENTS to use Advanced Features
1B5 Advantages: Advantages of RUNLIBS/RUNDATA & stub_profile/common_profile
2F2 Advantages: uvsort advantages vs unix system sort
3A2 Advantages: SelectJob Advantages
5L1 Alldiff2: Alternative #2 - copy jcl3.old from archive for alldiff2 with jcl3
5L1 Alldiff2: & alldiff3 diff between all files in 2 directories
5L1 Alldiff2: run alldiff2 & investigate diff report
5L2 Alldiff2: report - tmp/jcl3.dif
5L1 Alldiff3: alldiff2 & alldiff3 diff between all files in 2 directories
5L3 Alldiff3: calls diffdrop2 to drop insigificants
5L3 Alldiff3: report - tmp1/jcl3.dif
5L3 Alldiff3: run alldiff3 to drop insignicant differences
5L4 Alldiff3: script listing
5A1 Analysts: UVscripts Benefits to Programmers & Analysts
3H1 And: selectlgfd1 - select JCL SYSINs with AND & OR conditions
3H2 And: selectlgfd2 - select JCL SYSINs with AND & OR conditions (BETTER)
13C3 Anywhere: demo Insert anywhere desired
10C1 Applications: high-bit characters may cause problems for old applications
2C1 Applications: high-bit characters may cause problems for old applications
5L1 Archive: Alternative #2 - copy jcl3.old from archive for alldiff2 with jcl3
2G3 Archive_uvbriefs_uv_output report - in tmp1/archive_uvbriefs_uv_d1f3l3
5J2 Are: How to know there are 37 rename... scripts
3I2 Arg: Notes selections by arg1 or *arg2 & options 's1t1'
3D4 Arg1: Special Characters in arg1 patterns
3I1 Arg1: selectfdd1 Example#1 - multi-patterns on 1 line (arg1)
3I1 Arg1=pattern: select ENTIRE FILE to a 2nd directory - arg1=patterns on 1 line
3I1 Arg2: Notes Re selectfdd1 selections by *arg1 or arg2
3I2 Arg2: selectfdd1 Example#2 - multi-patterns on ANY line (arg2)
3I2 Arg2=pattern: select ENTIRE FILE to a 2nd directory - arg2=patterns on ANY line
4E2 Arguments: Notes re arguments & options
2D2 Ascii: uvhd option 'a' translate EBCDIC to ASCII
2E1 Ascii: Notes re translate to ASCII & insert LineFeeds
2E1 Ascii: uvcp - translate to ASCII & insert LineFeeds at same time
2E1 Ascii: uvcp demo - translate to ASCII (& insert LineFeeds)
11F1 Assumptions: Assumptions to run testmaxints
9B1 Assumptions: Pre-Requisites/Assumptions to run election1 demos
9F1 Assumptions: Pre-Requisites/Assumptions to run election2 demos
9M1 Assumptions: Pre-Requisites/Assumptions to run election3 demos
4G3 Attempt: table3d summarize SQL TABLEs - 1st attempt
8B1 Audit: output audit file
13E2 Autosales: Pre-Requisites to run AutoSales update demo
13E2 Autosales: Relevant files for AutoSales update demo
13E1 Autosalesus2018histoDemoFile#1 - AutoSalesUS2018History
13E3 Autosalesus2018histoOutput file - AutoSalesUS2018History
13E2 Autosalesus2019: DemoFile#2 - AutoSalesUS2019
2G2 Available: 'uvlist' scripts available

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1B2 Bash_profile_uv: stub_profile_uv OR bash_profile_uv
5F1 Batch: cftest - run all 9 cfd scripts in batch mode
4H1 Below: table2 1st Example for discussion below
4A1 Benefits: Notes, Benefits of table3d reports
4H2 Benefits: table2 - Notes & Benefits
5A1 Benefits: UVscripts Benefits to Programmers & Analysts
9N1 Benefits: Benefits of JCL conversion scripts
9Q2 Benefits: JCL conversion scripts to illustrate Benefits
9T1 Benefits: Benefits of JCL conversion scripts
1B6 Best: Working-Directory - Best Practices
9K1 Best: Working-Directory - Best Practices
3H2 Better: selectlgfd2 - select JCL SYSINs with AND & OR conditions (BETTER)
13B0 Big: uvfix1 - #1 Big Time saver
13A1 Big_time_savers: Big_Time_Savers - Overview
2D3 Binary: updating packed/binary files with uvhd
5M2 Binary: uvcmpFA1 - compare packed/binary files
5M3 Binary: uvhd to update packed/binary files
11A1 Books: Books to learn C programming
3J3 Born: search UScongress file for Female Democrats born after 1983
3J3 Born: search UScongress file for Male Republicans born after 1983
2C1 Brief: Brief Intro here, More demos later

3J2 California: search UScongress file for Male Replublicans from CAlifornia
7F5 Called: #3. dsntiaul - unload table script - called by JCL/script
5L3 Calls: alldiff3 calls diffdrop2 to drop insigificants
9H2 Campaign: EXECUTE uvcopy jobs to update Master file with campaign info
9P2 Campaign: EXECUTE uvcopy jobs to update Master file with campaign info
9P3 Campaign: JOBLOG from upcandimg1 - upate Master with campaign details
1D1 Canadamps: dat1/CanadaMPs - 334 members - samples of all parties
3E1 Canadamps: dat1/CanadaMPs - larger test file to demo Select Jobs
3E2 Canadamps: search CanadaMPs MP file for Liberals in Vancouver
3E2 Canadamps: suggested patterns for searching dat1/CanadaMPs
4K2 Canadamps: 3 Table Summaries from dat1/CanadaMPs
4K4 Canadamps: script to run multiple stats on dat1/CanadaMPs
5C2 Canadamps: dat1/CanadaMPs - 334 members - samples of all parties
1D1 Canadaprovinces: dat1/CanadaProvinces
5C2 Canadaprovinces: dat1/CanadaProvinces
10D1 Canadian: convert Canadian MPs from csv to fixed field text
3E1 Canadian: sample selection from CAnadian Members of Parliament
3E1 Canadian: samples of Canadian MPs .csv converted to fixed fields
4K1 Canadian: Table2 Summaries of Canadian Members of Parliament
4K1 Canadian: samples of Canadian MPs .csv converted to fixed fields
9C1 Candidates: EXECUTE uvcp to load/unload candidates Master file
9H1 Candidates: EXECUTE scripts load/unload candidates Master Jan2020
9I1 Candidates: LISTINGS of scripts to sort/load/unload candidates Master
9I2 Candidates: LISTINGS of uvcopy jobs to update candidates Master
9I3 Candidates: uvcopy job to update candidates Master file
9Q1 Candidates: LISTINGS of scripts to sort/load/unload candidates Master
9P1 Candidatesmg_: EXECUTE scripts to Load & update GDG Master candidatesMG_
3A2 Capabilitie: Example to Illustrate SelectJob Capabilities vs grep
10B4 Carriage: Fixing the problem - Replacing Carriage Returns (CRs)
10B4 Carriage: REMOVING Carriage Returns (CRs) vs REPLACING (above)
2B2 Carriage: Fix problem - Remove Carriage Returns from scripts
5K1 Carriage: removeCR - remove Carriage Returns from text files
5K1 Carriage: removeCR - remove Carriage Returns from text files
13A1 Catfiles2: - insert or replace columns of data in table files
13C0 Catfiles2: - concatenate 2 files
13C0 Catfiles2: Example - create 3 column file for insults generator
13C0 Catfiles2: demo - Create 3 column file from 3 separate files
13C2 Catfiles2: run catfiles2 with files & options defined on command line
13D2 Catfiles2: recreate 3 column file with catfiles2
13E0 Catfiles2: - 2nd demo using Auto Sales History file
10C1 Cause: high-bit characters may cause problems for old applications
2C1 Cause: high-bit characters may cause problems for old applications
5D1 Cfd: 2. cfd - Count Files,lines,KB in 1 directory
5F1 Cfd: cftest - run all 9 cfd scripts in batch mode
5G1 Cfd: Recommendations using cfd scripts on your own directories
5H2 Cfd: - Count Files,lines,KB in 1 directory
5D1 Cfd*: Run test/demos for 9 cfd* scripts
5E1 Cfd...: cfhelp - Help screen for cfd... scripts
5H1 Cfd...: Listings of cfd... scripts
5D2 Cfdd: 6. cfdd - count Files,Lines,& KB in all subdirs of a superdir
5D2 Cfddf: 8. cfddf - Count Files,Lines,KB in SubDirs of Super-Dir + 1st few files
5D2 Cfddt: 7. cfddt - count Files,Lines,& KB all subdirs of superdir (Totals-On
5D3 Cfdmm: 9. cfdmm - list directory File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#
5D2 Cfdpf: 4. cfdpf - Count Files,Lines,KB in Dir with/without Pattern in filename
5D1 Cfdt: 3. cfdt - Count Files,lines,KB in 1 directory (Totals-Only)
5E1 Cfhelp: - Help screen for cfd... scripts
5D1 Cfl: 1. cfl - Count Lines & KB in 1 File
5F1 Cftest: - run all 9 cfd scripts in batch mode
10C1 Characters: dat2/accents1 - French UTF-8 accented characters
10C1 Characters: high-bit characters may cause problems for old applications
10C2 Characters: Investigating Accented Characters with 'uvhd'
10C3 Characters: Notes re UTF-8 French Accented Characters
10C4 Characters: converting accented characters to unaccented equivalents
10C6 Characters: converting accented characters with 'uvcp'
10C7 Characters: converting accented characters with 'uvfix1'
2C1 Characters: high-bit characters may cause problems for old applications
3D4 Characters: Special Characters in arg1 patterns
3B3 Check-list: Setup check-list - Ready to run test/demos ?
4B1 Check-list: Setup check-list - Ready to run TableJob test/demos ?
5C1 Check-list: Setup check-list - Ready to run UVscripts test/demos ?
5N1 Chmod1: - set perms on all levels - using 'find'
13A1 Column: catfiles2 - insert or replace columns of data in table files
13C0 Column: catfiles2 Example - create 3 column file for insults generator
13C0 Column: catfiles2 demo - Create 3 column file from 3 separate files
13C1 Column: options to define column locations & lengths
13D2 Column: recreate 3 column file with catfiles2
13D2 Column: update separate files, sort, & recreate 3 column file
13A1 Columns: splitfiles3 - extract columns of data to separate files
13D0 Columns: splitfiles3 - extract columns of data to separate files
13D4 Columns: Re-Sequence in 1st columns
7D1 Columns: uvfix1 creating scripts with filenames in fixed columns
4G4 Combine: Solution - combine SQL statements onto 1 line
4G5 Combine: Problem Solution - Combine SELECT stmnts onto 1 line
4G5 Combined: rerun table3d inputting SELECT stmnts combined onto 1 line
13C2 Command: run catfiles2 with files & options defined on command line
4E2 Command: now execute the table3d command
9D1 Command: Recommended Command Line Working Habits
9J1 Command: Recommended Command Line Working Habits
3E3 Commands: Try some other commands such as
9Q5 Comments: upcandimg2 - alternate version with fewer #comments
9N2 Common: profiles - stub & common profiles
1B5 Common_profile: Advantages of RUNLIBS/RUNDATA & stub_profile/common_profile
1B3 Common_profile_uv:
3D1 Compare: dat1/nameadrs1 - compare demo outputs to these 8 records
5M2 Compare: uvcmpFA1 - compare packed/binary files
13C0 Concatenate: catfiles2 - concatenate 2 files
3E3 Conditions: searching with > & < conditions
3H1 Conditions: selectlgfd1 - select JCL SYSINs with AND & OR conditions
3H2 Conditions: selectlgfd2 - select JCL SYSINs with AND & OR conditions (BETTER)
4H3 Console: Console Log - options & prompts
7A4 Console: log
7D2 Console: uvfix1 - Execution & Console Log
9P1 Console: log from 'loadcandidatesMG2'
4H3 Console-log: table2 Execution & Console-Log
3E1 Converte: samples of Canadian MPs .csv converted to fixed fields
3J1 Converte: samples US Reps & Senators .csv converted to fixed field
4H1 Converte: samples US Reps & Senators .csv converted to fixed field
4K1 Converte: samples of Canadian MPs .csv converted to fixed fields
5L1 Converter: Alternative #1 - if you have JCL converter
3B2 Copied: SelectJobs demo files copied to your homedir
4B3 Copied: TableJobs demo files copied to your homedir
5B2 Copied: UVscripts demo files copied to your homedir
1C2 Copy: demo files to your $HOME/demo
3B2 Copy: $UV/demo to your $HOME/demo/...
4B3 Copy: $UV/demo to your $HOME/demo/...
5C1 Copy: $UV/demo to your $HOME/demo/...
5L1 Copy: Alternative #2 - copy jcl3.old from archive for alldiff2 with jcl3
7F3 Copy: expandevs1 - copy a file expanding $VARIABLEs
8B1 Copying: rep1d - search/replace 1 pattern copying all files to 2nd directory
8B2 Copying: rep2d search/replace multiple patterns copying all files to 2nd dir
4G6 Correct: rerun table3d to create correct summary of SQL TABLEs
5D1 Count: 1. cfl - Count Lines & KB in 1 File
5D1 Count: 2. cfd - Count Files,lines,KB in 1 directory
5D1 Count: 3. cfdt - Count Files,lines,KB in 1 directory (Totals-Only)
5D2 Count: 4. cfdpf - Count Files,Lines,KB in Dir with/without Pattern in filename
5D2 Count: 6. cfdd - count Files,Lines,& KB in all subdirs of a superdir
5D2 Count: 7. cfddt - count Files,Lines,& KB all subdirs of superdir (Totals-On
5D2 Count: 8. cfddf - Count Files,Lines,KB in SubDirs of Super-Dir + 1st few files
5H2 Count: cfd - Count Files,lines,KB in 1 directory
5E2 Counting: v12 - counting Lines,Files,& KB in Directories
5E2 Counting: v12 - help menu for counting scripts
10B1 Crs: uvhd demo#1 - Removing CRs x'0D's from scripts
10B10 Crs: 4. Remove CRs with 'uvfix1' pre-programmed uvcopy job
10B11 Crs: 5. Remove CRs with 'uvfixA' pre-programmed uvcopy job
10B4 Crs: Fixing the problem - Replacing Carriage Returns (CRs)
10B4 Crs: REMOVING Carriage Returns (CRs) vs REPLACING (above)
10B6 Crs: 1. Remove CRs using script 'removeCR'
10B6 Crs: Alternate solutions (vs uvhd) to Remove CRs from files
10B7 Crs: 2. Remove CRs using 'uvcp'
10B8 Crs: 3. Remove CRs using 'uvcopy'
2H1 Crs: uvcopy demo - removing CRs from scripts
10E1 Cs: convert US Senators/Representatives csv to fixed
10D1 Csv: convert Canadian MPs from csv to fixed field text
3F1 Customer: 4 SQL "customer" scripts in sqls/...
3F1 Customer: select "customer" references from all scripts in directory

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

13A1 Dat: catfiles2 - insert or replace columns of data in table files
13A1 Dat: splitfiles3 - extract columns of data to separate files
13D0 Dat: splitfiles3 - extract columns of data to separate files
1D1 Dat1: dat1/CanadaMPs - 334 members - samples of all parties
1D1 Dat1: dat1/CanadaProvinces
1D1 Dat1: dat1/UScongress - 1st & last 3 of 539 members
1D1 Dat1: dat1/USstates - 1st 3 & last 3 of 50 states
1D1 Dat1: dat1/nameadrs1 - test/demo file for SelectJobs
3C1 Dat1: dat1/nameadrs1 - testdata file for 1st demo
3D1 Dat1: dat1/nameadrs1 - compare demo outputs to these 8 records
3E1 Dat1: dat1/CanadaMPs - larger test file to demo Select Jobs
3E2 Dat1: suggested patterns for searching dat1/CanadaMPs
3J1 Dat1: Select Demos using dat1/UScongress
3J2 Dat1: suggested patterns for searching dat1/UScongress
3J3 Dat1: more examples searching dat1/UScongress
4I1 Dat1: 4 Table Summaries from dat1/UScongress
4J1 Dat1: script to run 13 table summaries on dat1/UScongress
4K2 Dat1: 3 Table Summaries from dat1/CanadaMPs
4K4 Dat1: script to run multiple stats on dat1/CanadaMPs
5C2 Dat1: dat1/CanadaMPs - 334 members - samples of all parties
5C2 Dat1: dat1/CanadaProvinces
5C2 Dat1: dat1/UScongress - 1st 5 & last of 539 members
5C2 Dat1: dat1/USstates - 1st 5 & last of 50 states
7A1 Dat1: desired Output $UV/demo/dat1/USstates
3C2 Dat1_nameadrs1_+1@toselects/dat1_nameadrs1_+1@Townsend+2@Canada-3@Owen
10C1 Dat2: dat2/accents1 - French UTF-8 accented characters
1D2 Dat2: dat2/sales2 - sales file with zone signs
1D2 Dat2: dat2/sales3 - sales file with separate signs
2C1 Dat2: dat2/accents3 - UTF-8 accents, short test file
2F1 Dat2: dat2/sales3 - input file
1B5 Data: RUNDATA determine PATHs to DATA files
3D4 Data: Try selectlf1 with YOUR DATA files & PATTERNS
9B3 Data: INSPECT election1/... data file contents
9N3 Data: $RUNDATA/$RUNLIBS & multiple sets LIBrarieS/DATA
9O1 Data: INSPECT Election3 data FILES
9O2 Data: INSPECT Election3 data file CONTENTs
9G3 Data1: Inspect election2/data1/... file CONTENTS
9G4 Data1: Inspect election2/data1/files (continued)
5O1 Days: lsold1 - list files older than x days
5O2 Days: rmold1 - remove files older than x days
7F4 Db2: expandevs1 used for mainframe DB2 conversions to unix
7F5 Db2: #1. parms/db2sysinxx - SQL input to db2 utility
7F5 Db2: #2. DB2 unload table step - from JCL/script
7F5 Db2: JCL conversion to unload DB2 tables on unix/linux
7F5 Db2sysinxx: #1. parms/db2sysinxx - SQL input to db2 utility
10B3 Default: uvhd default option r256, option 't' for text files
13C3 Default: demo Replace option 'r1' (vs Insert default)
2D1 Defaults: uvhd display record & line length defaults
13C1 Define: options to define column locations & lengths
13C2 Defined: run catfiles2 with files & options defined on command line
10B1 Demo: uvhd demo#1 - Removing CRs x'0D's from scripts
13B2 Demo: uvfix1 demo - Insert Sequence#s
13C0 Demo: catfiles2 demo - Create 3 column file from 3 separate files
13C3 Demo: Insert anywhere desired
13C3 Demo: Replace option 'r1' (vs Insert default)
13D0 Demo: splitfiles3 demo
13D0 Demo: splitfiles3 demo - input $UV/tf/insults_Owens
13D1 Demo: splitfiles3 Demo Op. Instrns.
13E0 Demo: catfiles2 - 2nd demo using Auto Sales History file
13E2 Demo: Pre-Requisites to run AutoSales update demo
13E2 Demo: Relevant files for AutoSales update demo
1C2 Demo: Alternative - setup demo/ dir in existing home dir
1C2 Demo: copy demo files to your $HOME/demo
1C2 Demo: copy demo files to your $HOME/demo
1C4 Demo: directories/files illustrated by 'tree2'
1D1 Demo: dat1/nameadrs1 - test/demo file for SelectJobs
1D4 Demo: input files - SQL scripts
1D5 Demo: input files - pf
1D5 Demo: input files - sf
1D6 Demo: input files - JCL
2B1 Demo: uvhd demo#1 - find/fix problem CR's in scripts
2D1 Demo: uvhd demo#3 - investigating unknown files
2E1 Demo: uvcp demo - translate to ASCII (& insert LineFeeds)
2G3 Demo: listall1 - print multiple small files - demo
2H1 Demo: uvcopy demo - removing CRs from scripts
3B2 Demo: SelectJobs demo files copied to your homedir
3B2 Demo: copy $UV/demo to your $HOME/demo/...
3B2 Demo: copy $UV/demo to your $HOME/demo/...
3C1 Demo: dat1/nameadrs1 - testdata file for 1st demo
3D1 Demo: dat1/nameadrs1 - compare demo outputs to these 8 records
3E1 Demo: dat1/CanadaMPs - larger test file to demo Select Jobs
3H1 Demo: Notes re tts & selectlgfd1 demo
4B3 Demo: TableJobs demo files copied to your homedir
4B3 Demo: copy $UV/demo to your $HOME/demo/...
4B3 Demo: copy $UV/demo to your $HOME/demo/...
4E3 Demo: table3d output report for 3rd demo JCL programs
4F2 Demo: tableIP - simplified table summary demo
4F3 Demo: Notes re uvcopy demo job 'tableIP'
5B2 Demo: UVscripts demo files copied to your homedir
5C1 Demo: copy $UV/demo to your $HOME/demo/...
5C1 Demo: copy $UV/demo to your $HOME/demo/...
7A1 Demo: desired Output $UV/demo/dat1/USstates
7A2 Demo: Directories to demo uvcopy/uvfix1
7C1 Demo: relevant subdirs/files to demo uvfixA SQL scripts
7F2 Demo: testevs2 - 'evs' demo#2 - enter your strings with $Variables
7F3 Demo: run expandevs1 with demo file
9A1 Demo: election1, election2, election3 demo progression
9A1 Demo: election1, election2, election3 demo progression
9B1 Demo: Objectives of election1 demo/tutorials
9F1 Demo: Objectives of election2 demo/tutorials
9G1 Demo: Inspect Demo Files - with 'llr'
9G2 Demo: Inspect Demo Files - with 'tree' (vs llr above)
9M1 Demo: Objectives of ELECTION3 demo/tutorials
9Q5 Demo: 4 versions of this demo job
9R1 Demo: upcandimg3 - 2nd step added to demo RESTART feature
9R2 Demo: RESTART at any step
3J2 Democrats: search UScongress file for Female Democrats from teXas
3J3 Democrats: search UScongress file for Female Democrats born after 1983
13E1 Demofile: DemoFile#1 - AutoSalesUS2018History
13E2 Demofile: DemoFile#2 - AutoSalesUS2019
13A1 Demos: Run tthe Demos
1C1 Demos: setup user account to run demos
2C1 Demos: Brief Intro here, More demos later
3B3 Demos: Setup check-list - Ready to run test/demos ?
3D1 Demos: Test/Demos to illustrate Pattern Rules
3D3 Demos: script to run 11 selectlf1 test/demos
3J1 Demos: Select Demos using dat1/UScongress
4B1 Demos: Setup check-list - Ready to run TableJob test/demos ?
5C1 Demos: Setup check-list - Ready to run UVscripts test/demos ?
5D1 Demos: Run test/demos for 9 cfd* scripts
9B1 Demos: Pre-Requisites/Assumptions to run election1 demos
9B2 Demos: login to Run election1 demos
9F1 Demos: Pre-Requisites/Assumptions to run election2 demos
9G1 Demos: Login to Run election2 demos
9M1 Demos: Pre-Requisites/Assumptions to run election3 demos
9N4 Demos: directories for election3 demos
13C3 Desired: demo Insert anywhere desired
7A1 Desired: Output $UV/demo/dat1/USstates
7D1 Desired: filename rename changes desired
9P3 Details: JOBLOG from upcandimg1 - upate Master with campaign details
5L1 Diff: alldiff2 & alldiff3 diff between all files in 2 directories
5L1 Diff: run alldiff2 & investigate diff report
5L3 Diffdrop2: alldiff3 calls diffdrop2 to drop insigificants
5L2 Differences: Significant Differences only in file#2
5L3 Differences: run alldiff3 to drop insignicant differences
13B3 Diffs: uvfix Tips & Diffs vs uvcopy instructions
1C2 Dir: Alternative - setup demo/ dir in existing home dir
1C2 Dir: Alternative - setup demo/ dir in existing home dir
5D2 Dir: 4. cfdpf - Count Files,Lines,KB in Dir with/without Pattern in filename
8B2 Dir: rep2d search/replace multiple patterns copying all files to 2nd dir
7C2 Director: Execute uvfixA to modify All files in directory of SQL scripts
1C4 Directories: demo directories/files illustrated by 'tree2'
5E2 Directories: v12 - counting Lines,Files,& KB in Directories
5G1 Directories: Recommendations using cfd scripts on your own directories
5I3 Directories: All scripts to list files in directories
5L1 Directories: alldiff2 & alldiff3 diff between all files in 2 directories
7A2 Directories: Directories to demo uvcopy/uvfix1
9N4 Directories: for election3 demos
3F1 Directory: select "customer" references from all scripts in directory
3F1 Directory: selectlfd1 - SELECT from all files in a directory
3I1 Directory: select ENTIRE FILE to a 2nd directory - arg1=patterns on 1 line
3I2 Directory: select ENTIRE FILE to a 2nd directory - arg2=patterns on ANY line
5D1 Directory: 2. cfd - Count Files,lines,KB in 1 directory
5D1 Directory: 3. cfdt - Count Files,lines,KB in 1 directory (Totals-Only)
5D3 Directory: 9. cfdmm - list directory File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#
5H2 Directory: cfd - Count Files,lines,KB in 1 directory
5P1 Directory: dtree - create directory tree illustration
7C1 Directory: uvfixA Example - modify directory of SQL scripts
8A1 Directory: scan1d - scan all files in directory reporting matches to 1 pattern
8B1 Directory: rep1d - search/replace 1 pattern copying all files to 2nd directory
4H1 Discussion: table2 1st Example for discussion below
4H1 Displacement: table2 - fixed position fields by displacement(length)
2D1 Display: uvhd display record & line length defaults
13B4 Dro: Easy way to drop CR's & shorten to last non-blank
5L3 Drop: alldiff3 calls diffdrop2 to drop insigificants
5L3 Drop: run alldiff3 to drop insignicant differences
7F5 Dsntiaul: #3. dsntiaul - unload table script - called by JCL/script
5P1 Dtree: - create directory tree illustration
5P1 Dtree: example

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

13B4 Easy: Easy way to drop CR's & shorten to last non-blank
2D2 Ebcdic: uvhd option 'a' translate EBCDIC to ASCII
11A1 Education: Education - Integers, Hexadecimal, conversions
12A1 Education: Education - Integers, Hexadecimal, conversions
11D1 Educational: Educational C programs
9A1 Election1: election1, election2, election3 demo progression
9A1 Election1: election1, election2, election3 demo progression
9B1 Election1: Objectives of election1 demo/tutorials
9B1 Election1: Pre-Requisites/Assumptions to run election1 demos
9B2 Election1: login to Run election1 demos
9B3 Election1: INSPECT election1/... data file contents
9A1 Election2: election1, election2, election3 demo progression
9A1 Election2: election1, election2, election3 demo progression
9F1 Election2: Objectives of election2 demo/tutorials
9F1 Election2: Pre-Requisites/Assumptions to run election2 demos
9G1 Election2: Login to Run election2 demos
9G3 Election2: Inspect election2/data1/... file CONTENTS
9G4 Election2: Inspect election2/data1/files (continued)
9A1 Election3: election1, election2, election3 demo progression
9A1 Election3: election1, election2, election3 demo progression
9M1 Election3: Objectives of ELECTION3 demo/tutorials
9M1 Election3: Pre-Requisites/Assumptions to run election3 demos
9N4 Election3: directories for election3 demos
9O1 Election3: INSPECT Election3 data FILES
9O2 Election3: INSPECT Election3 data file CONTENTs
9Q4 Enhanced: upcandimg1 - enhanced version marking lines for user scripts
7F2 Enter: testevs2 - 'evs' demo#2 - enter your strings with $Variables
7D3 Entered: uvfix1 - Explanations of Instructions entered
3I1 Entire: select ENTIRE FILE to a 2nd directory - arg1=patterns on 1 line
3I2 Entire: select ENTIRE FILE to a 2nd directory - arg2=patterns on ANY line
1B1 Env: 'profiles' provided in /home/uvadm/env
10C4 Equivalents: converting accented characters to unaccented equivalents
7F2 Evs: testevs2 - 'evs' demo#2 - enter your strings with $Variables
13C0 Example: catfiles2 Example - create 3 column file for insults generator
3A2 Example: Example to Illustrate SelectJob Capabilities vs grep
3C1 Example: selectlf1 - 1st Example for Pattern Rules following
3I1 Example: selectfdd1 Example#1 - multi-patterns on 1 line (arg1)
3I2 Example: selectfdd1 Example#2 - multi-patterns on ANY line (arg2)
4A1 Example: Example#1 - table summarize all IP#s in FTP scripts
4D1 Example: Example#2 - option 'w#' TargetWord offset from KeyWord
4D2 Example: Example#3 - option 'y' no of words to table
4H1 Example: table2 1st Example for discussion below
5A1 Example: Initial Example to get you interested
5J2 Example: RenameL - 1 example of 37 in sf/util/...
5P1 Example: dtree example
7C1 Example: uvfixA Example - modify directory of SQL scripts
3J3 Examples: more examples searching dat1/UScongress
12C1 Execute: Execute testint1
4E2 Execute: now execute the table3d command
7C2 Execute: Execute uvfixA to modify All files in directory of SQL scripts
9C1 Execute: EXECUTE uvcp to load/unload candidates Master file
9H1 Execute: EXECUTE scripts load/unload candidates Master Jan2020
9H2 Execute: EXECUTE uvcopy jobs to update Master file with campaign info
9P1 Execute: EXECUTE scripts to Load & update GDG Master candidatesMG_
9P2 Execute: EXECUTE uvcopy jobs to update Master file with campaign info
11B2 Executing: Executing insult2
3D3 Executing: sf/adm/select_nameadrs1
7A4 Executing: Executing uvcopy job to convert USstates.csv to fixed fields
4H3 Execution: table2 Execution & Console-Log
7D2 Execution: uvfix1 - Execution & Console Log
7E1 Exercise: Suggested Exercise for you
1C2 Existing: Alternative - setup demo/ dir in existing home dir
7F3 Expandevs1: - copy a file expanding $VARIABLEs
7F3 Expandevs1: run expandevs1 with demo file
7F4 Expandevs1: Why expandevs1 needed for JCL conversions
7F4 Expandevs1: used for mainframe DB2 conversions to unix
7F1 Expanding: Expanding $VARIABLES
7F3 Expanding: expandevs1 - copy a file expanding $VARIABLEs
3E2 Expected: Other searches showing only number of matches expected
7D3 Explanations: uvfix1 - Explanations of Instructions entered
13A1 Extract: splitfiles3 - extract columns of data to separate files
13D0 Extract: splitfiles3 - extract columns of data to separate files

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

9R1 Feature: upcandimg3 - 2nd step added to demo RESTART feature
9S1 Features: script AFTER adding Advanced Features
9S1 Features: script BEFORE adding Advanced Features
9T2 Features: REQUIREMENTS to use Advanced Features
3J2 Female: search UScongress file for Female Democrats from teXas
3J3 Female: search UScongress file for Female Democrats born after 1983
5D2 Few: 8. cfddf - Count Files,Lines,KB in SubDirs of Super-Dir + 1st few files
9Q5 Fewer: upcandimg2 - alternate version with fewer #comments
10D1 Field: convert Canadian MPs from csv to fixed field text
3J1 Field: samples US Reps & Senators .csv converted to fixed field
4H1 Field: samples US Reps & Senators .csv converted to fixed field
7B1 Field: uvfix1 alternate to convert .csv file to fixed field
3E1 Fields: samples of Canadian MPs .csv converted to fixed fields
4H1 Fields: table2 - fixed position fields by displacement(length)
4K1 Fields: samples of Canadian MPs .csv converted to fixed fields
7A3 Fields: writing uvcopy job to convert USstates.csv to fixed fields
7A4 Fields: Executing uvcopy job to convert USstates.csv to fixed fields
7B1 Fil: uvfix1 alternate to convert .csv file to fixed field
9C2 Fil: 'uvsort' Alternative to 'uvcp' (#1 above) to load Indexed fil
5D2 Filename: 4. cfdpf - Count Files,Lines,KB in Dir with/without Pattern in filename
7D1 Filename: rename changes desired
7D1 Filenames: uvfix1 creating scripts with filenames in fixed columns
2B1 Find: uvhd demo#1 - find/fix problem CR's in scripts
5N1 Find: chmod1 - set perms on all levels - using 'find'
2B1 Fix: uvhd demo#1 - find/fix problem CR's in scripts
2B2 Fix: Fix problem - Remove Carriage Returns from scripts
10D1 Fixed: convert Canadian MPs from csv to fixed field text
10E1 Fixed: convert US Senators/Representatives csv to fixed
3E1 Fixed: samples of Canadian MPs .csv converted to fixed fields
3J1 Fixed: samples US Reps & Senators .csv converted to fixed field
4H1 Fixed: samples US Reps & Senators .csv converted to fixed field
4H1 Fixed: table2 - fixed position fields by displacement(length)
4K1 Fixed: samples of Canadian MPs .csv converted to fixed fields
7A3 Fixed: writing uvcopy job to convert USstates.csv to fixed fields
7A4 Fixed: Executing uvcopy job to convert USstates.csv to fixed fields
7B1 Fixed: uvfix1 alternate to convert .csv file to fixed field
7D1 Fixed: uvfix1 creating scripts with filenames in fixed columns
7A1 Fixed-fields: uvcopy/uvfix1 - Convert .csv file to fixed-fields
10B4 Fixing: Fixing the problem - Replacing Carriage Returns (CRs)
3C1 Following: selectlf1 - 1st Example for Pattern Rules following
3C2 Format: Report Format for SelectJobs
10C1 French: dat2/accents1 - French UTF-8 accented characters
10C3 French: Notes re UTF-8 French Accented Characters
4A1 Ftp: Example#1 - table summarize all IP#s in FTP scripts

11A1 Games: Games written in C
12A1 Games: Games written in uvcopy
9P1 Gdg: EXECUTE scripts to Load & update GDG Master candidatesMG_
4E3 General: General Purpose Utilities
11C1 Generate: lotto1 - generate random#s for lotteries, etc
12B1 Generate: lotto1uv - generate random numbers with uvcopy
11B1 Generator: insult2 - Insult Generator
13C0 Generator: catfiles2 Example - create 3 column file for insults generator
13D3 Generator: recreated file must satisfy insults generator rules
5A1 Get: Initial Example to get you interested
3A2 Grep: Example to Illustrate SelectJob Capabilities vs grep
1E1 Guide: Hard-copy guide to work thru tutorials

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

9D1 Habits: Recommended Command Line Working Habits
9J1 Habits: Recommended Command Line Working Habits
1E1 Hard-copy: Hard-copy guide to work thru tutorials
5L1 Have: Alternative #1 - if you have JCL converter
5E1 Help: Help screens for each script (vs all 9 above)
5E1 Help: cfhelp - Help screen for cfd... scripts
5E2 Help: v12 - help menu for counting scripts
2C1 Here: Brief Intro here, More demos later
11A1 Hexadecimal: Education - Integers, Hexadecimal, conversions
12A1 Hexadecimal: Education - Integers, Hexadecimal, conversions
12D1 Hexcalc1:
10C1 High-bit: characters may cause problems for old applications
2C1 High-bit: characters may cause problems for old applications
13E0 History: catfiles2 - 2nd demo using Auto Sales History file
1B1 Home: 'profiles' provided in /home/uvadm/env
1C2 Home: Alternative - setup demo/ dir in existing home dir
1A1 Homedir: Vancouver Utilities homedir contents
3B2 Homedir: SelectJobs demo files copied to your homedir
4B3 Homedir: TableJobs demo files copied to your homedir
5B2 Homedir: UVscripts demo files copied to your homedir
4I2 House: table US Represenatives/Senators by House & Party
4K3 House: table CA MPs by House & Party
5J2 How: How to know there are 37 rename... scripts

3A2 Illustrate: Example to Illustrate SelectJob Capabilities vs grep
3D1 Illustrate: Test/Demos to illustrate Pattern Rules
9Q2 Illustrate: JCL conversion scripts to illustrate Benefits
1C4 Illustrated: demo directories/files illustrated by 'tree2'
5P1 Illustration: dtree - create directory tree illustration
5D1 In: 1. cfl - Count Lines & KB in 1 File
5D1 In: 2. cfd - Count Files,lines,KB in 1 directory
5D1 In: 3. cfdt - Count Files,lines,KB in 1 directory (Totals-Only)
5D2 In: 4. cfdpf - Count Files,Lines,KB in Dir with/without Pattern in filename
5D2 In: 6. cfdd - count Files,Lines,& KB in all subdirs of a superdir
5D2 In: 8. cfddf - Count Files,Lines,KB in SubDirs of Super-Dir + 1st few files
5E2 In: v12 - counting Lines,Files,& KB in Directories
5H2 In: cfd - Count Files,lines,KB in 1 directory
9C2 Indexed: 'uvsort' Alternative to 'uvcp' (#1 above) to load Indexed fil
9C2 Indexed: investigate results loading Indexed file
9C2 Indexed: separate 'uvsort' + 'uvcp' to load Indexed file
5I1 Info: more info on UV scripts
9H2 Info: EXECUTE uvcopy jobs to update Master file with campaign info
9P2 Info: EXECUTE uvcopy jobs to update Master file with campaign info
4G5 Inputting: rerun table3d inputting SELECT stmnts combined onto 1 line
13A1 Insert: catfiles2 - insert or replace columns of data in table files
13B2 Insert: uvfix1 demo - Insert Sequence#s
13C3 Insert: demo Insert anywhere desired
13C3 Insert: demo Replace option 'r1' (vs Insert default)
13D3 Insert: Insert 1st 10 lines & re-sequence
2E1 Insert: Notes re translate to ASCII & insert LineFeeds
2E1 Insert: uvcp - translate to ASCII & insert LineFeeds at same time
2E1 Insert: uvcp demo - translate to ASCII (& insert LineFeeds)
5L3 Insigificants: alldiff3 calls diffdrop2 to drop insigificants
5L3 Insignicant: run alldiff3 to drop insignicant differences
9B3 Inspect: INSPECT election1/... data file contents
9G1 Inspect: Inspect Demo Files - with 'llr'
9G2 Inspect: Inspect Demo Files - with 'tree' (vs llr above)
9G3 Inspect: Inspect election2/data1/... file CONTENTS
9G4 Inspect: Inspect election2/data1/files (continued)
9O1 Inspect: INSPECT Election3 data FILES
9O2 Inspect: INSPECT Election3 data file CONTENTs
1A2 Installing: Installing Vancouver Utilities - Summary
13D1 Instrns.: splitfiles3 Demo Op. Instrns.
11F1 Instructions: Operating Instructions
13B0 Instructions: uvcopy instructions
13B3 Instructions: uvfix Tips & Diffs vs uvcopy instructions
13C1 Instructions: Operating Instructions
13E2 Instructions: Operating Instructions
7D3 Instructions: uvfix1 - Explanations of Instructions entered
11B1 Insult: insult2 - Insult Generator
11B1 Insult2: - Insult Generator
11B2 Insult2: Executing insult2
11B4 Insult2: - C Source program listing
11B1 Insults: sample insults file
11B3 Insults: Creating your own insults file
13C0 Insults: catfiles2 Example - create 3 column file for insults generator
13D3 Insults: recreated file must satisfy insults generator rules
13D0 Insults_owens: splitfiles3 demo - input $UV/tf/insults_Owens
11A1 Integers: Education - Integers, Hexadecimal, conversions
11F1 Integers: testmaxints - show sizeof integers & max values
12A1 Integers: Education - Integers, Hexadecimal, conversions
5A1 Interested: Initial Example to get you interested
2C1 Intro: Brief Intro here, More demos later
3A1 Introduction: SelectJobs - Introduction & Overview
11B2 Investigate: Investigate Relevant files
5L1 Investigate: run alldiff2 & investigate diff report
9C2 Investigate: results loading Indexed file
10C2 Investigating: Investigating Accented Characters with 'uvhd'
2D1 Investigating: uvhd demo#3 - investigating unknown files
2B1 Investigation: uvhd - file investigation utility

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

9H1 Jan2020: EXECUTE scripts load/unload candidates Master Jan2020
5L2 Jcl3.dif: alldiff2 report - tmp/jcl3.dif
5L3 Jcl3.dif: alldiff3 report - tmp1/jcl3.dif
5L1 Jcl3.old: Alternative #2 - copy jcl3.old from archive for alldiff2 with jcl3
9P3 Joblog: JOBLOG from upcandimg1 - upate Master with campaign details
9R2 Joblog: RESTARTing at step2

4D1 Keyword: Example#2 - option 'w#' TargetWord offset from KeyWord
5J2 Know: How to know there are 37 rename... scripts

2D3 L=linelth: uvhd options a=EBCDIC->ASCII, r=RecordLth, l=LineLth
3E1 Larger: dat1/CanadaMPs - larger test file to demo Select Jobs
2C1 Later: Brief Intro here, More demos later
11A1 Learn: Books to learn C programming
13C1 Lengths: options to define column locations & lengths
5N1 Levels: chmod1 - set perms on all levels - using 'find'
3E2 Liberal: search CanadaMPs MP file for Liberals in Vancouver
9N3 Libraries: $RUNDATA/$RUNLIBS & multiple sets LIBrarieS/DATA
13C2 Line: run catfiles2 with files & options defined on command line
2D1 Line: uvhd display record & line length defaults
3G1 Line: selectlgfd1 - Multi-Condition-Series on any ONE line
3G2 Line: selectlgfd2 - Multi-Condition-Series on MULTI lines (vs 1 line)
3I1 Line: select ENTIRE FILE to a 2nd directory - arg1=patterns on 1 line
3I1 Line: selectfdd1 Example#1 - multi-patterns on 1 line (arg1)
3I2 Line: select ENTIRE FILE to a 2nd directory - arg2=patterns on ANY line
3I2 Line: selectfdd1 Example#2 - multi-patterns on ANY line (arg2)
4G4 Line: Solution - combine SQL statements onto 1 line
4G5 Line: Problem Solution - Combine SELECT stmnts onto 1 line
4G5 Line: rerun table3d inputting SELECT stmnts combined onto 1 line
9D1 Line: Recommended Command Line Working Habits
9J1 Line: Recommended Command Line Working Habits
2E1 Linefeed: uvcp - translate to ASCII & insert LineFeeds at same time
2E1 Linefeeds: Notes re translate to ASCII & insert LineFeeds
2E1 Linefeeds: uvcp demo - translate to ASCII (& insert LineFeeds)
7D1 Linux: SQL scripts on Linux renamed in sqls/...
7F5 Linux: JCL conversion to unload DB2 tables on unix/linux
9Q3 Linux: JCL conversion to script for unix/Linux
5D3 List: 9. cfdmm - list directory File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#
5I1 List: List files using unix 'ls' options & more
5I2 List: llt - sample of 20 scripts to list files
5I3 List: All scripts to list files in directories
5O1 List: lsold1 - list files older than x days
7E1 List: MakeTelList - uvcopy job to create a Telephone List
2G3 Listall1: - print multiple small files - demo
4G2 Listed: SQL scripts listed - test files for table3d
11B4 Listing: insult2 - C Source program listing
5L4 Listing: alldiff3 script listing
5H1 Listings: Listings of cfd... scripts
5J3 Listings: Rename Script Listings - ALL
9I1 Listings: LISTINGS of scripts to sort/load/unload candidates Master
9I2 Listings: LISTINGS of uvcopy jobs to update candidates Master
9Q1 Listings: LISTINGS of scripts to sort/load/unload candidates Master
5I1 Llc: sample output for 'llc'
9G1 Llr: Inspect Demo Files - with 'llr'
9G2 Llr: Inspect Demo Files - with 'tree' (vs llr above)
5I2 Llt: - sample of 20 scripts to list files
9C1 Load: EXECUTE uvcp to load/unload candidates Master file
9C2 Load: 'uvsort' Alternative to 'uvcp' (#1 above) to load Indexed fil
9C2 Load: separate 'uvsort' + 'uvcp' to load Indexed file
9H1 Load: EXECUTE scripts load/unload candidates Master Jan2020
9I1 Load: LISTINGS of scripts to sort/load/unload candidates Master
9P1 Load: EXECUTE scripts to Load & update GDG Master candidatesMG_
9Q1 Load: LISTINGS of scripts to sort/load/unload candidates Master
9P1 Loadcandidatesmg2: console log from 'loadcandidatesMG2'
9C2 Loading: investigate results loading Indexed file
13C1 Locations: options to define column locations & lengths
3B2 Locations: uvcopy job Locations
4H3 Log: Console Log - options & prompts
7A4 Log: console log
7B1 Log: uvfix1 log
7D2 Log: uvfix1 - Execution & Console Log
9P1 Log: console log from 'loadcandidatesMG2'
9B2 Login: to Run election1 demos
9G1 Login: Login to Run election2 demos
11C1 Lotteries: lotto1 - generate random#s for lotteries, etc
11C1 Lotto1: - generate random#s for lotteries, etc
12B1 Lotto1uv: - generate random numbers with uvcopy
5O1 Lsold1: - list files older than x days

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

7D1 Mainframe: SQL scripts from mainframe in sql0/...
7F4 Mainframe: expandevs1 used for mainframe DB2 conversions to unix
9Q2 Mainframe: Original mainframe JCL
7E1 Maketellist: MakeTelList - uvcopy job to create a Telephone List
3J2 Male: search UScongress file for Male Replublicans from CAlifornia
3J3 Male: search UScongress file for Male Republicans born after 1983
9Q4 Marking: upcandimg1 - enhanced version marking lines for user scripts
9C1 Master: EXECUTE uvcp to load/unload candidates Master file
9H1 Master: EXECUTE scripts load/unload candidates Master Jan2020
9H2 Master: EXECUTE uvcopy jobs to update Master file with campaign info
9I1 Master: LISTINGS of scripts to sort/load/unload candidates Master
9I2 Master: LISTINGS of uvcopy jobs to update candidates Master
9I3 Master: uvcopy job to update candidates Master file
9P1 Master: EXECUTE scripts to Load & update GDG Master candidatesMG_
9P2 Master: EXECUTE uvcopy jobs to update Master file with campaign info
9P3 Master: JOBLOG from upcandimg1 - upate Master with campaign details
9Q1 Master: LISTINGS of scripts to sort/load/unload candidates Master
8A1 Matche: scan1d - scan all files in directory reporting matches to 1 pattern
3E2 Matches: Other searches showing only number of matches expected
11F1 Max: testmaxints - show sizeof integers & max values
5D3 Maxrec: 9. cfdmm - list directory File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#
5D3 Maxsize: 9. cfdmm - list directory File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#
10C1 May: high-bit characters may cause problems for old applications
2C1 May: high-bit characters may cause problems for old applications
4K1 Member: Table2 Summaries of Canadian Members of Parliament
1D1 Members: dat1/CanadaMPs - 334 members - samples of all parties
1D1 Members: dat1/UScongress - 1st & last 3 of 539 members
3E1 Members: sample selection from CAnadian Members of Parliament
5C2 Members: dat1/CanadaMPs - 334 members - samples of all parties
5C2 Members: dat1/UScongress - 1st 5 & last of 539 members
5E2 Menu: v12 - help menu for counting scripts
5J1 Method: alternate method to rename files
5D3 Minrec: 9. cfdmm - list directory File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#
5D3 Minsize: 9. cfdmm - list directory File#,Lines,Minsize,Maxsize,Minrec#,Maxrec#
4G4 Missing: PROBLEM - some TABLE names missing ?
5F1 Mode: cftest - run all 9 cfd scripts in batch mode
13B5 Modify: uvfixA - modify ALL Text files in subdir
7C1 Modify: uvfixA Example - modify directory of SQL scripts
7C2 Modify: Execute uvfixA to modify All files in directory of SQL scripts
4D1 Most: useful options - option 'w#'
4D2 Most: useful options - option 'y#'
4K2 Mp: table CA MPs by Province
4K3 Mp: table CA MPs by House & Party
4K3 Mp: table CA MPs by Year-Elected
10D1 Mps: convert Canadian MPs from csv to fixed field text
3E1 Mps: samples of Canadian MPs .csv converted to fixed fields
4K1 Mps: samples of Canadian MPs .csv converted to fixed fields
3G2 Multi: selectlgfd2 - Multi-Condition-Series on MULTI lines (vs 1 line)
3G1 Multi-condition-seriselectlgfd1 - Multi-Condition-Series on any ONE line
3G2 Multi-condition-seriselectlgfd2 - Multi-Condition-Series on MULTI lines (vs 1 line)
3I1 Multi-patterns: selectfdd1 Example#1 - multi-patterns on 1 line (arg1)
3I2 Multi-patterns: selectfdd1 Example#2 - multi-patterns on ANY line (arg2)
13D3 Must: recreated file must satisfy insults generator rules

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

1D1 Nameadrs1: dat1/nameadrs1 - test/demo file for SelectJobs
3C1 Nameadrs1: dat1/nameadrs1 - testdata file for 1st demo
3D1 Nameadrs1: dat1/nameadrs1 - compare demo outputs to these 8 records
13B1 Nameadrs3a: tf/nameadrs3a
13B1 Nameadrs3b: tf/nameadrs3b
13B1 Nameadrs3c: tf/nameadrs3c
13B1 Nameadrs3d: tf/nameadrs3d
7F4 Needed: Why expandevs1 needed for JCL conversions
13B4 Non-blank: Easy way to drop CR's & shorten to last non-blank
4E2 Now: execute the table3d command
3E2 Number: Other searches showing only number of matches expected
12B1 Numbers: lotto1uv - generate random numbers with uvcopy

9B1 Objectives: Objectives of election1 demo/tutorials
9F1 Objectives: Objectives of election2 demo/tutorials
9M1 Objectives: Objectives of ELECTION3 demo/tutorials
4D2 Of: Example#3 - option 'y' no of words to table
4D1 Offset: Example#2 - option 'w#' TargetWord offset from KeyWord
10C1 Old: high-bit characters may cause problems for old applications
2C1 Old: high-bit characters may cause problems for old applications
5O1 Older: lsold1 - list files older than x days
5O2 Older: rmold1 - remove files older than x days
3G1 One: selectlgfd1 - Multi-Condition-Series on any ONE line
4G4 Onto: Solution - combine SQL statements onto 1 line
4G5 Onto: Problem Solution - Combine SELECT stmnts onto 1 line
4G5 Onto: rerun table3d inputting SELECT stmnts combined onto 1 line
13D1 Op.: splitfiles3 Demo Op. Instrns.
11F1 Operating: Operating Instructions
13C1 Operating: Operating Instructions
13E2 Operating: Operating Instructions
5J1 Operating: Operating Procedures
7D1 Operating: Operating Procedures
10B3 Option: uvhd default option r256, option 't' for text files
10B3 Option: uvhd default option r256, option 't' for text files
13C3 Option: demo Replace option 'r1' (vs Insert default)
2D2 Option: uvhd option 'a' translate EBCDIC to ASCII
4D1 Option: Example#2 - option 'w#' TargetWord offset from KeyWord
4D1 Option: most useful options - option 'w#'
4D2 Option: Example#3 - option 'y' no of words to table
4D2 Option: most useful options - option 'y#'
13C1 Options: to define column locations & lengths
13C2 Options: run catfiles2 with files & options defined on command line
1B4 Options: .vimrc - set options for vim
2D3 Options: uvhd options a=EBCDIC->ASCII, r=RecordLth, l=LineLth
3I2 Options: Notes selections by arg1 or *arg2 & options 's1t1'
4C1 Options: table3d options
4D1 Options: most useful options - option 'w#'
4D2 Options: most useful options - option 'y#'
4E2 Options: Notes re arguments & options
4H3 Options: Console Log - options & prompts
5I1 Options: List files using unix 'ls' options & more
3D1 Outputs: dat1/nameadrs1 - compare demo outputs to these 8 records
13A1 Overview: Big_Time_Savers - Overview
3A1 Overview: SelectJobs - Introduction & Overview
11B3 Own: Creating your own insults file
5G1 Own: Recommendations using cfd scripts on your own directories

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

2D3 Packed: updating packed/binary files with uvhd
5M2 Packed: uvcmpFA1 - compare packed/binary files
5M3 Packed: uvhd to update packed/binary files
3E1 Parliament: sample selection from CAnadian Members of Parliament
4K1 Parliament: Table2 Summaries of Canadian Members of Parliament
7F5 Parms: #1. parms/db2sysinxx - SQL input to db2 utility
5A1 Part: UVscripts same as UVdemos Part 5
1D1 Parties: dat1/CanadaMPs - 334 members - samples of all parties
5C2 Parties: dat1/CanadaMPs - 334 members - samples of all parties
4I1 Party: table US Represenatives/Senators by Party
4I2 Party: table US Represenatives/Senators by House & Party
4I3 Party: table US Represenatives/Senators by Party & Sex
4I4 Party: table US Represenatives/Senators by State & Party
4K3 Party: table CA MPs by House & Party
1B5 Paths: RUNDATA determine PATHs to DATA files
1B5 Paths: RUNLIBS determine PATHs to programs,scripts,jcls,etc
5D2 Patter: 4. cfdpf - Count Files,Lines,KB in Dir with/without Pattern in filename
3C1 Pattern: selectlf1 - 1st Example for Pattern Rules following
3D1 Pattern: Test/Demos to illustrate Pattern Rules
8A1 Pattern: scan1d - scan all files in directory reporting matches to 1 pattern
8B1 Pattern: rep1d - search/replace 1 pattern copying all files to 2nd directory
3D4 Patterns: Special Characters in arg1 patterns
3D4 Patterns: Try selectlf1 with YOUR DATA files & PATTERNS
3D4 Patterns: Try selectlf1 with YOUR PATTERNS
3E2 Patterns: suggested patterns for searching dat1/CanadaMPs
3J2 Patterns: suggested patterns for searching dat1/UScongress
8B2 Patterns: rep2d search/replace multiple patterns copying all files to 2nd dir
5N1 Perms: chmod1 - set perms on all levels - using 'find'
4E2 Pgm: table summarize PGM names in JCL
4H1 Position: table2 - fixed position fields by displacement(length)
1B6 Practices: Working-Directory - Best Practices
9K1 Practices: Working-Directory - Best Practices
10B10 Pre-programmed: 4. Remove CRs with 'uvfix1' pre-programmed uvcopy job
10B11 Pre-programmed: 5. Remove CRs with 'uvfixA' pre-programmed uvcopy job
4F1 Pre-programmed: Writing specific uvcopy jobs vs Pre-Programmed TableJobs
13E2 Pre-requisites: Pre-Requisites to run AutoSales update demo
9B1 Pre-requisites: Pre-Requisites/Assumptions to run election1 demos
9F1 Pre-requisites: Pre-Requisites/Assumptions to run election2 demos
9M1 Pre-requisites: Pre-Requisites/Assumptions to run election3 demos
2G3 Print: listall1 - print multiple small files - demo
3G1 Prior: Notes Re *selectlfgfd1 (& selectlgfd2) vs prior selectlf1/selectl
3G2 Prior: Notes Re *selectlfgfd2 (& selectlgfd1) vs prior selectlf1/selectl
10B4 Problem: Fixing the problem - Replacing Carriage Returns (CRs)
2B1 Problem: uvhd demo#1 - find/fix problem CR's in scripts
2B2 Problem: Fix problem - Remove Carriage Returns from scripts
3F1 Problem: PROBLEM Notes
4G1 Problem: PROBLEM in SQL scripts for table3d
4G4 Problem: PROBLEM - some TABLE names missing ?
4G5 Problem: Problem Solution - Combine SELECT stmnts onto 1 line
10B2 Problems: run uvhd to see problems not shown with 'vi'
10C1 Problems: high-bit characters may cause problems for old applications
2C1 Problems: high-bit characters may cause problems for old applications
5J1 Procedures: Operating Procedures
7D1 Procedures: Operating Procedures
1B1 Profiles: 'profiles' provided in /home/uvadm/env
9N2 Profiles: - stub & common profiles
9N2 Profiles: - stub & common profiles
11B4 Program: insult2 - C Source program listing
13B5 Program: sample INPUT - 1st 5 lines of 1st program
13B5 Program: sample OUTPUT - 1st 5 lines of 1st program
4E1 Program: table3d - create summary table of programs in JCL
5A1 Programmers: UVscripts Benefits to Programmers & Analysts
11A1 Programming: Books to learn C programming
12A1 Programming: in uvcopy
11D1 Programs: Educational C programs
1B5 Programs: RUNLIBS determine PATHs to programs,scripts,jcls,etc
4E3 Programs: table3d output report for 3rd demo JCL programs
9A1 Progression: election1, election2, election3 demo progression
9A1 Progression: election1, election2, election3 demo progression
4H3 Prompts: Console Log - options & prompts
1B1 Provided: 'profiles' provided in /home/uvadm/env
4K2 Province: table CA MPs by Province

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

10B3 R256: uvhd default option r256, option 't' for text files
2D3 R=recordlth: uvhd options a=EBCDIC->ASCII, r=RecordLth, l=LineLth
11C1 Random: lotto1 - generate random#s for lotteries, etc
12B1 Random: lotto1uv - generate random numbers with uvcopy
13D3 Re-sequence: Insert 1st 10 lines & re-sequence
13D4 Re-sequence: Re-Sequence in 1st columns
5G1 Recommendations: Recommendations using cfd scripts on your own directories
7A2 Recommendations: Recommendations for writing uvcopy jobs
9D1 Recommendations: Working-Directory Recommendations/Tips
9J1 Recommendations: Working-Directory Recommendations/Tips
9D1 Recommended: Recommended Command Line Working Habits
9J1 Recommended: Recommended Command Line Working Habits
2D1 Record: uvhd display record & line length defaults
3D1 Records: dat1/nameadrs1 - compare demo outputs to these 8 records
13D2 Recreate: 3 column file with catfiles2
13D2 Recreate: update separate files, sort, & recreate 3 column file
13D3 Recreated: file must satisfy insults generator rules
3F1 References: select "customer" references from all scripts in directory
11B2 Relevant: Investigate Relevant files
13E2 Relevant: Relevant files for AutoSales update demo
3B1 Relevant: Vancouver Utilities - subdirs relevant to SelectJobs
4B2 Relevant: Vancouver Utilities - subdirs relevant to TableJobs
5B1 Relevant: Vancouver Utilities - subdirs relevant to UVscripts
7C1 Relevant: subdirs/files to demo uvfixA SQL scripts
10B10 Remove: 4. Remove CRs with 'uvfix1' pre-programmed uvcopy job
10B11 Remove: 5. Remove CRs with 'uvfixA' pre-programmed uvcopy job
10B6 Remove: 1. Remove CRs using script 'removeCR'
10B6 Remove: Alternate solutions (vs uvhd) to Remove CRs from files
10B7 Remove: 2. Remove CRs using 'uvcp'
10B8 Remove: 3. Remove CRs using 'uvcopy'
2B2 Remove: Fix problem - Remove Carriage Returns from scripts
5K1 Remove: removeCR - remove Carriage Returns from text files
5K1 Remove: removeCR - remove Carriage Returns from text files
5O2 Remove: rmold1 - remove files older than x days
10B6 Removecr: 1. Remove CRs using script 'removeCR'
5K1 Removecr: removeCR - remove Carriage Returns from text files
5K1 Removecr: removeCR - remove Carriage Returns from text files
10B1 Removing: uvhd demo#1 - Removing CRs x'0D's from scripts
10B4 Removing: REMOVING Carriage Returns (CRs) vs REPLACING (above)
2H1 Removing: uvcopy demo - removing CRs from scripts
5J1 Rename: Rename scripts
5J1 Rename: alternate method to rename files
5J3 Rename: Rename Script Listings - ALL
7D1 Rename: SQL scripts on Linux renamed in sqls/...
7D1 Rename: filename rename changes desired
5J2 Rename...: How to know there are 37 rename... scripts
5J2 Renamel: RenameL - 1 example of 37 in sf/util/...
8B1 Rep1d: - search/replace 1 pattern copying all files to 2nd directory
8B2 Rep2d: search/replace multiple patterns copying all files to 2nd dir
13A1 Replace: catfiles2 - insert or replace columns of data in table files
13C3 Replace: demo Replace option 'r1' (vs Insert default)
8B1 Replace: rep1d - search/replace 1 pattern copying all files to 2nd directory
8B2 Replace: rep2d search/replace multiple patterns copying all files to 2nd dir
10B4 Replacing: Fixing the problem - Replacing Carriage Returns (CRs)
10B4 Replacing: REMOVING Carriage Returns (CRs) vs REPLACING (above)
3J2 Replublicans: search UScongress file for Male Replublicans from CAlifornia
8A1 Reporting: scan1d - scan all files in directory reporting matches to 1 pattern
4I1 Represenatives: table US Represenatives/Senators by Party
4I2 Represenatives: table US Represenatives/Senators by House & Party
4I3 Represenatives: table US Represenatives/Senators by Party & Sex
4I4 Represenatives: table US Represenatives/Senators by State & Party
10E1 Representatives: convert US Senators/Representatives csv to fixed
3J1 Reps: samples US Reps & Senators .csv converted to fixed field
4H1 Reps: samples US Reps & Senators .csv converted to fixed field
3J3 Republicans: search UScongress file for Male Republicans born after 1983
9T2 Requirements: REQUIREMENTS to use Advanced Features
4G5 Rerun: table3d inputting SELECT stmnts combined onto 1 line
4G6 Rerun: table3d to create correct summary of SQL TABLEs
9R1 Restart: upcandimg3 - 2nd step added to demo RESTART feature
9R2 Restart: demo RESTART at any step
9R2 Restarting: joblog RESTARTing at step2
10C5 Results: Results Before & After conversion
9C2 Results: investigate results loading Indexed file
10B4 Returns: Fixing the problem - Replacing Carriage Returns (CRs)
10B4 Returns: REMOVING Carriage Returns (CRs) vs REPLACING (above)
2B2 Returns: Fix problem - Remove Carriage Returns from scripts
5K1 Returns: removeCR - remove Carriage Returns from text files
5K1 Returns: removeCR - remove Carriage Returns from text files
5O2 Rmold1: - remove files older than x days
13D3 Rules: recreated file must satisfy insults generator rules
3C1 Rules: selectlf1 - 1st Example for Pattern Rules following
3D1 Rules: Test/Demos to illustrate Pattern Rules
10B2 Run: uvhd to see problems not shown with 'vi'
11F1 Run: Assumptions to run testmaxints
13A1 Run: Run tthe Demos
13C2 Run: catfiles2 with files & options defined on command line
13E2 Run: Pre-Requisites to run AutoSales update demo
1C1 Run: setup user account to run demos
3B3 Run: Setup check-list - Ready to run test/demos ?
3D3 Run: script to run 11 selectlf1 test/demos
4B1 Run: Setup check-list - Ready to run TableJob test/demos ?
4J1 Run: script to run 13 table summaries on dat1/UScongress
4K4 Run: script to run multiple stats on dat1/CanadaMPs
5C1 Run: Setup check-list - Ready to run UVscripts test/demos ?
5D1 Run: Run test/demos for 9 cfd* scripts
5F1 Run: cftest - run all 9 cfd scripts in batch mode
5L1 Run: alldiff2 & investigate diff report
5L3 Run: alldiff3 to drop insignicant differences
7F3 Run: expandevs1 with demo file
9B1 Run: Pre-Requisites/Assumptions to run election1 demos
9B2 Run: login to Run election1 demos
9F1 Run: Pre-Requisites/Assumptions to run election2 demos
9G1 Run: Login to Run election2 demos
9M1 Run: Pre-Requisites/Assumptions to run election3 demos
1B5 Rundata: Advantages of RUNLIBS/RUNDATA & stub_profile/common_profile
1B5 Rundata: RUNDATA determine PATHs to DATA files
1B5 Runlibs: Advantages of RUNLIBS/RUNDATA & stub_profile/common_profile
1B5 Runlibs: RUNLIBS determine PATHs to programs,scripts,jcls,etc

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

10B1 S: uvhd demo#1 - Removing CRs x'0D's from scripts
3I2 S1t1: Notes selections by arg1 or *arg2 & options 's1t1'
13E0 Sales: catfiles2 - 2nd demo using Auto Sales History file
1D2 Sales: dat2/sales2 - sales file with zone signs
1D2 Sales: dat2/sales3 - sales file with separate signs
1D2 Sales2: dat2/sales2 - sales file with zone signs
1D2 Sales3: dat2/sales3 - sales file with separate signs
2F1 Sales3: dat2/sales3 - input file
2E1 Same: uvcp - translate to ASCII & insert LineFeeds at same time
5A1 Same: UVscripts same as UVdemos Part 5
11B1 Sample: insults file
13B5 Sample: INPUT - 1st 5 lines of 1st program
13B5 Sample: OUTPUT - 1st 5 lines of 1st program
3E1 Sample: selection from CAnadian Members of Parliament
3J1 Sample: selection from UScongress testfile
5I1 Sample: output for 'llc'
5I2 Sample: llt - sample of 20 scripts to list files
7D1 Sample: output
1D1 Samples: dat1/CanadaMPs - 334 members - samples of all parties
3E1 Samples: of Canadian MPs .csv converted to fixed fields
3J1 Samples: US Reps & Senators .csv converted to fixed field
4H1 Samples: US Reps & Senators .csv converted to fixed field
4K1 Samples: of Canadian MPs .csv converted to fixed fields
5C2 Samples: dat1/CanadaMPs - 334 members - samples of all parties
13D3 Satisfy: recreated file must satisfy insults generator rules
13B0 Saver: uvfix1 - #1 Big Time saver
8A1 Scan: scan1d - scan all files in directory reporting matches to 1 pattern
8A1 Scan1d: - scan all files in directory reporting matches to 1 pattern
5E1 Screen: cfhelp - Help screen for cfd... scripts
5E1 Screens: Help screens for each script (vs all 9 above)
3E2 Search: CanadaMPs MP file for Liberals in Vancouver
3J2 Search: UScongress file for Female Democrats from teXas
3J2 Search: UScongress file for Male Replublicans from CAlifornia
3J3 Search: UScongress file for Female Democrats born after 1983
3J3 Search: UScongress file for Male Republicans born after 1983
8B1 Search: rep1d - search/replace 1 pattern copying all files to 2nd directory
8B2 Search: rep2d search/replace multiple patterns copying all files to 2nd dir
3E2 Searches: Other searches showing only number of matches expected
3E2 Searching: suggested patterns for searching dat1/CanadaMPs
3E3 Searching: with > & < conditions
3J2 Searching: suggested patterns for searching dat1/UScongress
3J3 Searching: more examples searching dat1/UScongress
10B2 See: run uvhd to see problems not shown with 'vi'
3E1 Select: dat1/CanadaMPs - larger test file to demo Select Jobs
3F1 Select: "customer" references from all scripts in directory
3F1 Select: selectlfd1 - SELECT from all files in a directory
3H1 Select: selectlgfd1 - select JCL SYSINs with AND & OR conditions
3H2 Select: selectlgfd2 - select JCL SYSINs with AND & OR conditions (BETTER)
3I1 Select: ENTIRE FILE to a 2nd directory - arg1=patterns on 1 line
3I2 Select: ENTIRE FILE to a 2nd directory - arg2=patterns on ANY line
3J1 Select: Select Demos using dat1/UScongress
4G5 Select: Problem Solution - Combine SELECT stmnts onto 1 line
4G5 Select: rerun table3d inputting SELECT stmnts combined onto 1 line
3D3 Select_nameadrs1: executing sf/adm/select_nameadrs1
3I1 Selectfdd1: Notes Re selectfdd1 selections by *arg1 or arg2
3I1 Selectfdd1: Example#1 - multi-patterns on 1 line (arg1)
3I2 Selectfdd1: Example#2 - multi-patterns on ANY line (arg2)
3E1 Selection: sample selection from CAnadian Members of Parliament
3I1 Selection: Notes Re selectfdd1 selections by *arg1 or arg2
3J1 Selection: sample selection from UScongress testfile
3I2 Selections: Notes selections by arg1 or *arg2 & options 's1t1'
3A2 Selectjob: Example to Illustrate SelectJob Capabilities vs grep
3A2 Selectjob: SelectJob Advantages
1D1 Selectjobs: dat1/nameadrs1 - test/demo file for SelectJobs
3A1 Selectjobs: SelectJobs - Introduction & Overview
3B1 Selectjobs: Vancouver Utilities - subdirs relevant to SelectJobs
3B2 Selectjobs: SelectJobs demo files copied to your homedir
3C2 Selectjobs: Report Format for SelectJobs
3G1 Selectl: Notes Re *selectlfgfd1 (& selectlgfd2) vs prior selectlf1/selectl
3G2 Selectl: Notes Re *selectlfgfd2 (& selectlgfd1) vs prior selectlf1/selectl
3C1 Selectlf1: - 1st Example for Pattern Rules following
3C2 Selectlf1: Output Report from selectlf1
3D3 Selectlf1: script to run 11 selectlf1 test/demos
3D4 Selectlf1: Try selectlf1 with YOUR DATA files & PATTERNS
3D4 Selectlf1: Try selectlf1 with YOUR PATTERNS
3G1 Selectlf1: Notes Re *selectlfgfd1 (& selectlgfd2) vs prior selectlf1/selectl
3G2 Selectlf1: Notes Re *selectlfgfd2 (& selectlgfd1) vs prior selectlf1/selectl
3F1 Selectlfd1: - SELECT from all files in a directory
3H2 Selectlgfd: Notes re ttx vs tts & selectlgfd2 vs selectlgfd1
3G1 Selectlgfd1: - Multi-Condition-Series on any ONE line
3G2 Selectlgfd1: Notes Re *selectlfgfd2 (& selectlgfd1) vs prior selectlf1/selectl
3H1 Selectlgfd1: Notes re tts & selectlgfd1 demo
3H1 Selectlgfd1: - select JCL SYSINs with AND & OR conditions
3H2 Selectlgfd1: Notes re ttx vs tts & selectlgfd2 vs selectlgfd1
3G1 Selectlgfd2: Notes Re *selectlfgfd1 (& selectlgfd2) vs prior selectlf1/selectl
3G2 Selectlgfd2: - Multi-Condition-Series on MULTI lines (vs 1 line)
3H2 Selectlgfd2: - select JCL SYSINs with AND & OR conditions (BETTER)
3C2 Selects: selects/dat1_nameadrs1_+1@Townsend+2@Canada-3@Owen
3D4 Selects: reports written to selects/... subdir
3I1 Selects: stats Report written to selects/...
3I2 Selects: stats Report written to selects/...
4I1 Senator: table US Represenatives/Senators by Party
4I2 Senator: table US Represenatives/Senators by House & Party
4I3 Senator: table US Represenatives/Senators by Party & Sex
4I4 Senator: table US Represenatives/Senators by State & Party
10E1 Senators: convert US Senators/Representatives csv to fixed
3J1 Senators: samples US Reps & Senators .csv converted to fixed field
4H1 Senators: samples US Reps & Senators .csv converted to fixed field
13A1 Separate: splitfiles3 - extract columns of data to separate files
13C0 Separate: catfiles2 demo - Create 3 column file from 3 separate files
13D0 Separate: splitfiles3 - extract columns of data to separate files
13D2 Separate: update separate files, sort, & recreate 3 column file
1D2 Separate: dat2/sales3 - sales file with separate signs
9C2 Separate: 'uvsort' + 'uvcp' to load Indexed file
13B2 Sequence: uvfix1 demo - Insert Sequence#s
1B4 Set: .vimrc - set options for vim
5N1 Set: chmod1 - set perms on all levels - using 'find'
9N3 Sets: $RUNDATA/$RUNLIBS & multiple sets LIBrarieS/DATA
1C1 Setup: user account to run demos
1C2 Setup: Alternative - setup demo/ dir in existing home dir
3B3 Setup: Setup check-list - Ready to run test/demos ?
4B1 Setup: Setup check-list - Ready to run TableJob test/demos ?
5C1 Setup: Setup check-list - Ready to run UVscripts test/demos ?
4I3 Sex: table US Represenatives/Senators by Party & Sex
2C1 Short: dat2/accents3 - UTF-8 accents, short test file
13B4 Shorte: Easy way to drop CR's & shorten to last non-blank
11F1 Show: testmaxints - show sizeof integers & max values
3E2 Showing: Other searches showing only number of matches expected
10B2 Shown: run uvhd to see problems not shown with 'vi'
5L2 Significant: Significant Differences only in file#2
1D2 Signs: dat2/sales2 - sales file with zone signs
1D2 Signs: dat2/sales3 - sales file with separate signs
4F2 Simplified: tableIP - simplified table summary demo
11F1 Sizeof: testmaxints - show sizeof integers & max values
2G3 Small: listall1 - print multiple small files - demo
4G4 Solution: Solution - combine SQL statements onto 1 line
4G5 Solution: Problem Solution - Combine SELECT stmnts onto 1 line
10B6 Solutions: Alternate solutions (vs uvhd) to Remove CRs from files
3E3 Some: Try some other commands such as
4G4 Some: PROBLEM - some TABLE names missing ?
13D2 Sort: update separate files, sort, & recreate 3 column file
2F2 Sort: uvsort advantages vs unix system sort
9I1 Sort: LISTINGS of scripts to sort/load/unload candidates Master
9Q1 Sort: LISTINGS of scripts to sort/load/unload candidates Master
11B4 Source: insult2 - C Source program listing
3D4 Special: Special Characters in arg1 patterns
4F1 Specific: Writing specific uvcopy jobs vs Pre-Programmed TableJobs
13A1 Splitfiles3: - extract columns of data to separate files
13D0 Splitfiles3: - extract columns of data to separate files
13D0 Splitfiles3: demo
13D0 Splitfiles3: demo - input $UV/tf/insults_Owens
13D1 Splitfiles3: Demo Op. Instrns.
1D4 Sql: demo input files - SQL scripts
3F1 Sql: 4 SQL "customer" scripts in sqls/...
4G1 Sql: PROBLEM in SQL scripts for table3d
4G1 Sql: table3d - create summaries of TABLEs in SQL scripts
4G2 Sql: SQL scripts listed - test files for table3d
4G3 Sql: table3d summarize SQL TABLEs - 1st attempt
4G4 Sql: Solution - combine SQL statements onto 1 line
4G6 Sql: rerun table3d to create correct summary of SQL TABLEs
7C1 Sql: relevant subdirs/files to demo uvfixA SQL scripts
7C1 Sql: uvfixA Example - modify directory of SQL scripts
7C2 Sql: Execute uvfixA to modify All files in directory of SQL scripts
7D1 Sql: SQL scripts from mainframe in sql0/...
7D1 Sql: SQL scripts on Linux renamed in sqls/...
7F5 Sql: #1. parms/db2sysinxx - SQL input to db2 utility
7D1 Sql0: SQL scripts from mainframe in sql0/...
3F1 Sqls: 4 SQL "customer" scripts in sqls/...
7D1 Sqls: SQL scripts on Linux renamed in sqls/...
4K4 Stat: script to run multiple stats on dat1/CanadaMPs
4I4 State: table US Represenatives/Senators by State & Party
4G4 Statements: Solution - combine SQL statements onto 1 line
1D1 States: dat1/USstates - 1st 3 & last 3 of 50 states
5C2 States: dat1/USstates - 1st 5 & last of 50 states
3I1 Stats: Report written to selects/...
3I2 Stats: Report written to selects/...
4A1 Stats: table3d report created in stats/...
7F5 Step: #2. DB2 unload table step - from JCL/script
9R1 Step: upcandimg3 - 2nd step added to demo RESTART feature
9R2 Step: demo RESTART at any step
9R2 Step2: joblog RESTARTing at step2
4G5 Stmnts: Problem Solution - Combine SELECT stmnts onto 1 line
4G5 Stmnts: rerun table3d inputting SELECT stmnts combined onto 1 line
7F2 Strings: testevs2 - 'evs' demo#2 - enter your strings with $Variables
9N2 Stub: profiles - stub & common profiles
1B5 Stub_profile: Advantages of RUNLIBS/RUNDATA & stub_profile/common_profile
1B2 Stub_profile_uv: OR bash_profile_uv
13B5 Subdir: uvfixA - modify ALL Text files in subdir
1B7 Subdir: tmp/ subdir in Working-Directory
3D4 Subdir: reports written to selects/... subdir
5D2 Subdir: 6. cfdd - count Files,Lines,& KB in all subdirs of a superdir
5D2 Subdir: 7. cfddt - count Files,Lines,& KB all subdirs of superdir (Totals-On
5D2 Subdir: 8. cfddf - Count Files,Lines,KB in SubDirs of Super-Dir + 1st few files
9K2 Subdir: tmp/ subdir in Working-Directory
3B1 Subdirs: Vancouver Utilities - subdirs relevant to SelectJobs
4B2 Subdirs: Vancouver Utilities - subdirs relevant to TableJobs
5B1 Subdirs: Vancouver Utilities - subdirs relevant to UVscripts
7C1 Subdirs: relevant subdirs/files to demo uvfixA SQL scripts
3E3 Such: Try some other commands such as
3E2 Suggested: patterns for searching dat1/CanadaMPs
3J2 Suggested: patterns for searching dat1/UScongress
7E1 Suggested: Suggested Exercise for you
4G6 Summar: rerun table3d to create correct summary of SQL TABLEs
4J1 Summarie: script to run 13 table summaries on dat1/UScongress
4G1 Summaries: table3d - create summaries of TABLEs in SQL scripts
4I1 Summaries: 4 Table Summaries from dat1/UScongress
4K1 Summaries: Table2 Summaries of Canadian Members of Parliament
4K2 Summaries: 3 Table Summaries from dat1/CanadaMPs
4A1 Summarize: Example#1 - table summarize all IP#s in FTP scripts
4E2 Summarize: table summarize PGM names in JCL
4G3 Summarize: table3d summarize SQL TABLEs - 1st attempt
1A2 Summary: Installing Vancouver Utilities - Summary
4E1 Summary: table3d - create summary table of programs in JCL
4F2 Summary: tableIP - simplified table summary demo
5D2 Super-dir: 8. cfddf - Count Files,Lines,KB in SubDirs of Super-Dir + 1st few files
5D2 Superdir: 6. cfdd - count Files,Lines,& KB in all subdirs of a superdir
5D2 Superdir: 7. cfddt - count Files,Lines,& KB all subdirs of superdir (Totals-On
3H1 Sysins: selectlgfd1 - select JCL SYSINs with AND & OR conditions
3H2 Sysins: selectlgfd2 - select JCL SYSINs with AND & OR conditions (BETTER)
2F2 System: uvsort advantages vs unix system sort

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

13A1 Table: catfiles2 - insert or replace columns of data in table files
4A1 Table: Example#1 - table summarize all IP#s in FTP scripts
4D2 Table: Example#3 - option 'y' no of words to table
4E1 Table: table3d - create summary table of programs in JCL
4E2 Table: summarize PGM names in JCL
4F2 Table: tableIP - simplified table summary demo
4G1 Table: table3d - create summaries of TABLEs in SQL scripts
4G4 Table: PROBLEM - some TABLE names missing ?
4I1 Table: 4 Table Summaries from dat1/UScongress
4I1 Table: US Represenatives/Senators by Party
4I2 Table: US Represenatives/Senators by House & Party
4I3 Table: US Represenatives/Senators by Party & Sex
4I4 Table: US Represenatives/Senators by State & Party
4J1 Table: script to run 13 table summaries on dat1/UScongress
4K2 Table: 3 Table Summaries from dat1/CanadaMPs
4K2 Table: CA MPs by Province
4K3 Table: CA MPs by House & Party
4K3 Table: CA MPs by Year-Elected
7F5 Table: #2. DB2 unload table step - from JCL/script
7F5 Table: #3. dsntiaul - unload table script - called by JCL/script
7F5 Table: JCL conversion to unload DB2 tables on unix/linux
4H1 Table2: - fixed position fields by displacement(length)
4H1 Table2: 1st Example for discussion below
4H2 Table2: - Notes & Benefits
4H3 Table2: Execution & Console-Log
4K1 Table2: Table2 Summaries of Canadian Members of Parliament
4A1 Table3d: Notes, Benefits of table3d reports
4A1 Table3d: report created in stats/...
4C1 Table3d: options
4E1 Table3d: - create summary table of programs in JCL
4E2 Table3d: now execute the table3d command
4E3 Table3d: output report for 3rd demo JCL programs
4G1 Table3d: PROBLEM in SQL scripts for table3d
4G1 Table3d: - create summaries of TABLEs in SQL scripts
4G2 Table3d: SQL scripts listed - test files for table3d
4G3 Table3d: summarize SQL TABLEs - 1st attempt
4G5 Table3d: rerun table3d inputting SELECT stmnts combined onto 1 line
4G6 Table3d: rerun table3d to create correct summary of SQL TABLEs
4F2 Tableip: tableIP - simplified table summary demo
4F3 Tableip: Notes re uvcopy demo job 'tableIP'
4B1 Tablejob: Setup check-list - Ready to run TableJob test/demos ?
4B2 Tablejobs: Vancouver Utilities - subdirs relevant to TableJobs
4B3 Tablejobs: TableJobs demo files copied to your homedir
4F1 Tablejobs: Writing specific uvcopy jobs vs Pre-Programmed TableJobs
4G3 Tables: table3d summarize SQL TABLEs - 1st attempt
4G6 Tables: rerun table3d to create correct summary of SQL TABLEs
4D1 Targetword: Example#2 - option 'w#' TargetWord offset from KeyWord
7E1 Telephone: MakeTelList - uvcopy job to create a Telephone List
1D1 Test: dat1/nameadrs1 - test/demo file for SelectJobs
2C1 Test: dat2/accents3 - UTF-8 accents, short test file
3B3 Test: Setup check-list - Ready to run test/demos ?
3D1 Test: Test/Demos to illustrate Pattern Rules
3D3 Test: script to run 11 selectlf1 test/demos
3E1 Test: dat1/CanadaMPs - larger test file to demo Select Jobs
4B1 Test: Setup check-list - Ready to run TableJob test/demos ?
4G2 Test: SQL scripts listed - test files for table3d
5C1 Test: Setup check-list - Ready to run UVscripts test/demos ?
5D1 Test: Run test/demos for 9 cfd* scripts
3B3 Testdata: file contents
3C1 Testdata: dat1/nameadrs1 - testdata file for 1st demo
4B1 Testdata: file contents
7F2 Testevs2: - 'evs' demo#2 - enter your strings with $Variables
3J1 Testfile: sample selection from UScongress testfile
12C1 Testint1: Execute testint1
12C1 Testint1:
12C2 Testint1: tf/testint1_data - INPUT file for uvcopy testint1
12C2 Testint1: tmp/testint1_data - OUTPUT file from uvcopy testint1
12C2 Testint1_data: tf/testint1_data - INPUT file for uvcopy testint1
12C2 Testint1_data: tmp/testint1_data - OUTPUT file from uvcopy testint1
11F1 Testmaxints: Assumptions to run testmaxints
11F1 Testmaxints: - show sizeof integers & max values
3J2 Texas: search UScongress file for Female Democrats from teXas
10B3 Text: uvhd default option r256, option 't' for text files
10D1 Text: convert Canadian MPs from csv to fixed field text
13B5 Text: uvfixA - modify ALL Text files in subdir
5K1 Text: removeCR - remove Carriage Returns from text files
5K1 Text: removeCR - remove Carriage Returns from text files
5J2 There: How to know there are 37 rename... scripts
1E1 Thru: Hard-copy guide to work thru tutorials
13B3 Tips: uvfix Tips & Diffs vs uvcopy instructions
7A2 Tips: Tips for writing uvcopy jobs
9D1 Tips: Working-Directory Recommendations/Tips
9J1 Tips: Working-Directory Recommendations/Tips
12C2 Tmp: tmp/testint1_data - OUTPUT file from uvcopy testint1
1B7 Tmp: tmp/ subdir in Working-Directory
2F1 Tmp: tmp/s3 - output file
5L2 Tmp: alldiff2 report - tmp/jcl3.dif
9K2 Tmp: tmp/ subdir in Working-Directory
2G3 Tmp1: output report - in tmp1/archive_uvbriefs_uv_d1f3l3
5L3 Tmp1: alldiff3 report - tmp1/jcl3.dif
10B6 To: Alternate solutions (vs uvhd) to Remove CRs from files
9C2 To: 'uvsort' Alternative to 'uvcp' (#1 above) to load Indexed fil
5D2 Totals-on: 7. cfddt - count Files,Lines,& KB all subdirs of superdir (Totals-On
5D1 Totals-only: 3. cfdt - Count Files,lines,KB in 1 directory (Totals-Only)
2E1 Translat: Notes re translate to ASCII & insert LineFeeds
2D2 Translate: uvhd option 'a' translate EBCDIC to ASCII
2E1 Translate: uvcp - translate to ASCII & insert LineFeeds at same time
2E1 Translate: uvcp demo - translate to ASCII (& insert LineFeeds)
5P1 Tree: dtree - create directory tree illustration
9G2 Tree: Inspect Demo Files - with 'tree' (vs llr above)
1C4 Tree2: demo directories/files illustrated by 'tree2'
3D4 Try: Try selectlf1 with YOUR DATA files & PATTERNS
3D4 Try: Try selectlf1 with YOUR PATTERNS
3E3 Try: Try some other commands such as
3H2 Tt: Notes re ttx vs tts & selectlgfd2 vs selectlgfd1
13A1 Tthe: Run tthe Demos
3H1 Tts: Notes re tts & selectlgfd1 demo
3H2 Tts: Notes re ttx vs tts & selectlgfd2 vs selectlgfd1
1E1 Tutorials: Hard-copy guide to work thru tutorials
9B1 Tutorials: Objectives of election1 demo/tutorials
9F1 Tutorials: Objectives of election2 demo/tutorials
9M1 Tutorials: Objectives of ELECTION3 demo/tutorials

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

A  B  C  D  E  F  G  H  I  J  K  L  M  N  O  P  Q  R  S  T  U  V  W  X  Y  Z 

10C4 Unaccented: converting accented characters to unaccented equivalents
2F2 Unix: uvsort advantages vs unix system sort
5I1 Unix: List files using unix 'ls' options & more
7F4 Unix: expandevs1 used for mainframe DB2 conversions to unix
7F5 Unix: JCL conversion to unload DB2 tables on unix/linux
9Q3 Unix: JCL conversion to script for unix/Linux
2D1 Unknown: uvhd demo#3 - investigating unknown files
7F5 Unload: #2. DB2 unload table step - from JCL/script
7F5 Unload: #3. dsntiaul - unload table script - called by JCL/script
7F5 Unload: JCL conversion to unload DB2 tables on unix/linux
9C1 Unload: EXECUTE uvcp to load/unload candidates Master file
9H1 Unload: EXECUTE scripts load/unload candidates Master Jan2020
9I1 Unload: LISTINGS of scripts to sort/load/unload candidates Master
9Q1 Unload: LISTINGS of scripts to sort/load/unload candidates Master
9P3 Upate: JOBLOG from upcandimg1 - upate Master with campaign details
9P3 Upcandimg1: JOBLOG from upcandimg1 - upate Master with campaign details
9Q4 Upcandimg1: - enhanced versio