UVdemos1 - Contents


Part_0 Introduction & Overview of UVdemos1
- Preliminary to UVdemos2 & all Vancouver Utilities
- Prospects may be given a userid/password to evaluate online

Part_1 Login procedure to run demos, exercises, & tutorials
- preparation to run OnLine OR OffLine
- profile changes to run Demos or JCL conversions
- tree diagrams of demo directories & files

Part_2 Compiling & Executing C programs & Equivalent uvcopy jobs
- some games & educational programs for beginners
- should be fun learning to work with unix/linux tools on the command line
- random number generation for lottery numbers & Shakespearean insults
- programs to test limits for 16, 32,& 64 bit integers (19 decimal digits)

Part_3 uvcopy jobs most interesting for beginners
hexcalc1 - convert numbers from decimal to binary or binary to decimal
- displayed in hexadecimal
datedemo1 - convert dates between Calendar, Julian,& days-since-1900
- fun way to calculate your age in days or weekday you were born on

Part_4 More serious uvcopy jobs & tutorials to do real work at unix/linux sites
- converting delimited files to fixed-fields & reverse with 1 instruction
- using demo test-files of Canadian Provinces & US Congress members
- insert 1 or a few instructions into 'copy1' framework job for file I/O
- create table summaries of provinces & populations with 2 instructions
  or summarize US Congress members by party, house/senate, sex, etc

Part_5 uvfix1 - easy way to run uvcopy for whateve data manipulation is required
- solicits instructions interactively, no need to edit into a framework
- examples that are impossible or hard to do with editors
  such as creating scripts with fields in desired column locations
- uvfix1 automatically saves the created job in case you want to rerun
uvfixA - alternate job to process all files in a directory (vs 1 at a time)

Part_6 Pre-Programmed utility jobs for data search/replace beyond unix tools
- mass changes to all files in directories of programs, scripts,& documentation
- search/replace depending on multiple patterns present &/or not present
- table summaries based on keywords & offsets to target words
- these tools can save many hours of manual editing

Part_7 Demonstrates some of the many powerful features of uvcopy that might
provide solutions to various problems that come up at any computer site.
We illustrate uvcopy capabilities that go beyond Unix/Linux utilities.

20A1. Links to other Vancouver Utilities Documentation
- UVdemos2 uvhd uvsort uvcp uvcopy uvtrain uvhelp etc

20B1. uvcopy Instruction summary
- over 100 instructions documented at uvsoftware.ca/uvcopy3.htm

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_0. Introduction to the Vancouver Utilities

0A1. UVdemos1 - Preliminary to UVdemos2

'UVdemos1' (subset of the larger 'UVdemos2') is an easy way to become familiar with the Vancouver Utilities and using unix/linux on the command line. UVdemos1 was inspired by my grandchildren. I wanted to allow them to login to my website with a userid/password & run the C compiler & simpler UV demos without having to download & install the Vancouver Utilities.

I have selected some demos that should interest any unix/linux user:

 $HOME/demo holds a subset of Vancouver Utilities for easier demos.
 $HOME/demo/src - contains a few C programs you can compile & execute
 - helloworld.c lotto1.c insult1.c insult2.c test64a.c test64b.c test64c.c

We will introduce you to 'uvcopy' a Vancouver Utility program (written in C) that makes it easy to perform whatever data manipulations you might require much simpler than writing C programs. You can use any unix editor to pepare an instruction file to be interpreted by uvcopy (no compile required).

See over 100 powerful instructions at uvsoftware.ca/uvcopy3.htm. For example 'uvcopy uvlotto1' creates the same output as 'lotto1.c' with only 16 instructions (vs 49 for lotto1.c). The uvcopy instructions are much easier to learn & understand than C programming. You get the power of mainframe assembly without the complexity.

More importantly, uvcopy can do things that unix system utilities can not do, since it was developed to replace mainframe computer utilities. For example, uvcopy understands packed decimal fields, Indexed files, & column addressing which was heavily used by mainframe utilities, but almost non-existent on unix.

'right-click' on links to other documents

Be sure to 'right-click' when you want to follow HTML links to other documents (such as the uvsoftware.ca/uvcopy3.htm link to uvcopy instructions).

To avoid losing your place in this UVdemos2.doc, it is best to 'right-click' on the desired links & open in a 'new-tab'. This allows you to leave the other document open if desired & switch easily between multiple documents.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

0A2. UVdemos1 - Starting Point Preliminary to UVdemos2

UVdemos1 - On-Line Availability

After an evaluation or partnership agreement has been signed, UV Software can provide you with a userid & password, so you can login & run the demos before you download, install,& compile the Utilities on your machine. This saves you having to setup the directories & profiles on your machine.

For users with an online userid/password, your userid will be in the range of 'uvsoft01' to 'uvsoft99'. In this documentation we will refer to your userid as 'uvsoftxx' for OnLine logins or 'userxx' for OffLine users on their machines. Passwords will be 'passxx'. You will use 'ssh' (Secure SHell) for OnLin logins to the UV Software website & the usual login procedures for OffLine logins.

Exercises & Tutorials

If you perform the step by step tutorials & compare your outputs to the expected outputs (usually listed following the executions), I am pretty sure that you will soon understand these utilities & be able to use them for your own purposes.

Useful Tools for IT Professionals

Are you are an IT professional, working with programs, scripts,& documentation ? If yes, we think you will find uvfix1 & uvfixA the most useful tools in this collection.

Have you sometimes wished you had an easier more powerful way to make mass changes in a file or in All files in a directory ?

If so, then see 'Part_5' uvfix1 & uvfixA to modify 1 or all files in a directory. uvfix1 & uvfixA are interactive utilities that solicit your instructions and automatically write outputs to a temp directory for inspection & copy back if OK.

You can do complex data manipulation with only 1 instruction - such as converting delimited files to fixed fields or vice versa. You can apply more qualifiers to mass changes than your can with unix system utilities.

Your instructions & the uvcopy framework with I/O directory/file name are saved in case you later need to repeat or modify & repeat.

Feedback Invited

Please let me know if you find any bugs, errors in the documentation, or instructions that are not clear & need more explanation. Please send email to owen@uvsoftware.ca

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_1. UVdemos1 - Starting Point Preliminary to UVdemos2

UVdemos1 - Part 1 - Contents


1A1. Preparation to Run UVdemos1 Demos & Tutorials
- run OnLine with userid/password form UV Software
- OR Offline if you have downloaded,installed,compiled on your machine

1A2. login & list the directories in your homedir/...
- appsadm, archive, demo, doc, testdata1, testlibs1, upload, uvadm
- you will change into demo/... to run the demos & tutorials

1A3. 'tree' utility to diagram directory & subdirs

1A4. Update 'RUNDATA/RUNLIBS' in .bash_profile for UVdemos vs JCL conversions.
Links to listings of bash_profile & common_profile in other documents.

1A5. Logout & Login again if RUNDATA/RUNLIBS modified
cdd/cdl - aliases for 'cd $RUNDATA' & 'cd $RUNLIBS'

1A6. Directories & Files relevant to UVdemos1 (vs UVdemos2)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A1. Preparation to Run UVdemos1 Demos & Tutorials

Run UVdemos1 OnLine - with userid/password from UV Software

No preparation is required if you are evaluating the Vancouver utilities with a userid/password assigned to your from UV Software.

Note that the documentation in UVdemos1 is written giving priority to the OnLine option (vs UVdemos2 which assumes that you have already downloaded, installed, compiled, & setup a user account/login on your machine).

But since we recommend all users should start their learning experience with UVdemos1, we have tried to write the UVdemos1 documentation for both OnLine and OffLine users.

Run UVdemos1 OffLine - Alternative to OnLine Login

Here is the Alternative to 'Running UVdemos1 OnLine', for users that have installed the Vancouver Utilities on their own machines.

As mentioned above, the priority in the UVdemos1 documentation was given to OnLine users evaluating the Utilities the easy way without having to first download, install, compile,& setup users on their own machines.

Whereas the UVdemos2 documentation was written assuming that users would do the download,install,compile,setups on their own machines.

But we recommend all users should start their learning experience with UVdemos1, so here are the procedures that users operating on their own machines must have performed before they can run the various demos & tutorials in UVdemos1.

Pre-Requisites to Run UVdemos1 on your machine vs OnLine

  1. Download, Install,& compile the Vancouver Utilities. See the install summary at UVdemos2.htm#1A2 & complete documentation at install.htm

  2. Setup your user login account if not already existing. See UVdemos2.htm#1C1.

  3. If you already have a login on your machine, you can setup a 'demo' subdir in your homedir and copy the demo files from $UV/demo/* as documented at UVdemos2.htm#1C2.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A2. Beginner's Introduction to the Vancouver Utilities

Login to run UVdemos1 On-Line


 #1. ssh uvsoftxx@uvsoftware.ca  <-- Online userid in range uvsoft00-uvsoft99
     ==========================
     --> passxx                  <-- enter your password at the prompt

 #2. ls -l    <-- List top-level directories in OnLine homedir
     =====
      -rw-rw-r--  1 uvsoftxx uvsoftxx 6900 Jun 15 10:20 .bash_profile
      -rw-rw-r--  1 uvsoftxx uvsoftxx 3149 Sep 11 15:50 1README_env_profiles
      -rw-rw-r--  1 uvsoftxx uvsoftxx 3399 Aug  5 17:22 1README_uvsoftware_demos
      -rw-rw-r--  1 uvsoftxx uvsoftxx 3696 Aug  5 17:15 1README_uvsoftware_download
                     ----- Directories Required for UVdemos -----
      drwxrwxr-x 12 uvsoftxx uvsoftxx 4096 Jun 15 16:58 appsadm
      drwxrwxr-x 30 uvsoftxx uvsoftxx 4096 Jun 15 00:32 demo/... <-- UVdemos files
      drwxrwxr-x  2 uvsoftxx uvsoftxx  305 Jun 15 16:55 upload
      drwxrwxr-x 28 uvsoftxx uvsoftxx 4096 Jun 14 19:54 uvadm
                     ----- additional for JCL conversions ------
      drwxrwxr-x 13 uvsoftxx uvsoftxx  247 Jun 14 19:59 testdata1
      drwxrwxr-x 31 uvsoftxx uvsoftxx 4096 Jun 14 19:55 testlibs1

Alternative Login Off-Line on your machine


 #1a. Login userxx   <-- Alternative Login on your own machine
      ============       if you have downloaded & installed Vancouver Utilities
     --> passxx      <-- enter your password at the prompt

 #2a. ls -l    <-- List top-level directories in your homedir (shown below)
      =====     - see Note2 below replacing 'ls -l' with alias 'l'
      -rw-rw-r--  1 userxx apps 6900 Jun 15 10:20 .bash_profile
      drwxrwxr-x 30 userxx apps 4096 Jun 15 00:32 ........   <-- your other subdirs
      drwxrwxr-x 30 userxx apps 4096 Jun 15 00:32 demo/...   <-- UVdemos files Note1
Note1
  • OffLine users need to copy demo/* files from $UV/demo/* to $HOME/demo/...
Note2
  • 'ls -l' unix command to List dirs/files with Long Option '-l'
  • will be replaced with just 'l' (alias l='ls -l') from now on
Note3
  • From now on we will usually use just 'userxx' (vs 'uvsoftxx') for your
    userid regardless of OnLine Login 'uvsoftxx' or OffLine 'userxx'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A3. Investigate Demo Directories & Files Available

'tree' utility to diagram directory & subdirs

'tree' is great utility to display directory structures. 'tree' is open software created by Steve Baker (ice@mama.indstate.edu).


 #3. tree /home/userxx -L 1   <-- display "tree" diagram of dirs/files
     ======================     - Option "-L 1" for top level only
      /home/userxx
      |-- .bash_profile                <-- active profile (Hidden file)
      |-- 1README_env_profiles
      |-- 1README_uvsoftware_demos     <-- README files
      |-- 1README_uvsoftware_download
      |-- appsadm     <-- administration files
      |   |-- env     <-- subdir holding profiles
      |   |   |-- bash_profile_uv
      |   |   |-- bash_profile_web1
      |   |   |-- bash_profile_web2
      |   |   |-- common_profile_uv
      |   |   |-- common_profile_web1
      |   |   |-- common_profile_web2
      |   `
      |-- demo         <-- superdir for UVdemos (copied from uvadm/demo)
      |   |--          <-- see relevant subdirs further below (bin,dat1,pf,src,tf)
      |-- testdata1    <-- superdir for JCL conversion DATA files
      |-- testlibs1    <-- superdir for JCL conversion LIBrarieS (jcls,procs,parms,etc)
      |-- tmp
      |-- upload       <-- zip files uploaded by UV Software for user downloads
      |   |-- 20200911___uvadm.zip
      |   |-- 20200911_appsadm.zip
      |   |-- 20200911_uvdemos.zip
      |   |-- UVdemos1.doc
      |   |-- UVdemos1.pdf
      |   `-- uvdemos1.htm

0 directories, 6 files

      `-- uvadm        <-- Vancouver Utilities
Note
  • tree /home/userxx (without option "-L 1") would be 9000 lines long
  • we are showing ".bash_profile", a HIDDEN file, normally not shown
  • we have added a few relevant files from subdir env/... on the 2nd Level
  • "bash_profiles_uv,web1,web2" ONE of which has been copied to .bash_profile
  • "common_profiles_uv,web1,web2" ONE of which is called by .bash_profile
         ** Updating .bash_profile to run UVdemos1 (vs JCL Conversions) *8

You may need to update the '.bash_profile' in your login homedir to run the demos (vs JCL Conversions that some other logins might desire).

The bash_profile stored on UV Software computers defaults for JCL conversions. UV Software should have already changed the default from JCL conversions to the demos. We mention it here in case we forgot, Or if you later wish to demo the JCL Conversions.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A4. Check login .bash_profile setup for UVdemos

update 'RUNDATA/RUNLIBS' for UVdemos vs JCL conversions

RUNLIBS & RUNDATA are environmental variables defined in the .bash_profile that must be changed to switch from JCL Conversions to the demos. The usual default is for the JCL Conversions, but we change this for the Demos when we know the recipients are more interested in the Demos than the JCL conversions.

If you need to switch, you can edit .bash_profile to activate the desired "export" & #comment the other "export" - either line 30 or line 45. Here are the relevant lines.


 #4. vi .bash_profile    <-- edit the .bash_profile
     ================      - showing lines 29 - 43 (line#s inserted)
 029 # 1. Mainframe JCLConversions - library/data super-dirs in $HOME
 030 export RUNLIBS=$HOME/testlibs1 RUNDATA=$HOME/testdata1 CNVDATA=$HOME/cnvdata1 <-- DEFAULTs
 031 #============================================================================
 032 #
 033 # 2. Mainframe JCL/COBOL/DATA Migrations - conversion/production, separate file systems
 034 #    export RUNLIBS=/p1/apps/ABClibs1 RUNDATA=/p2/apps/ABCdata1 CNVDATA=/p3/apps/ABCcnvdata1
 035 #    ======================================================================================
 036 #    - better to use larger file systems for production (p1,p2,p3/apps/ABC... vs $HOME)
 037 #    - appended digit '1' for future possible alternates ...libs2/...data2/...cnvdata2,etc
 038 #    - could have multi-companies/systems on same machine, example for ABC company
 039 #    - RUNDATA could be defined differently for different programmers for testing
 040 #    - see uvsoftware.ca/jclcnv1demo.htm#1B3 RUNLIBS/RUNDATA defines for migrations
 041 #
 042 # 3. Vancouver Utilities Demos/Tutorials - libs/data in $HOME/demo
 043 #    export RUNLIBS=$HOME/demo RUNDATA=$HOME/demo CNVDATA=$HOME/demo  <-- Activate for demos
 044 #    ===============================================================
 045 #    - may define RUNLIBS/RUNDATA/CNVDATA the same for small projects (not major migrations)
 046 #    - see uvsoftware.ca/uvdemos2.htm#1B5 more about RUNLIBS/RUNDATA stub/common profiles

 030 #    export RUNLIBS=$HOME/testlibs1 RUNDATA=$HOME/testdata1 CNVDATA=$HOME/cnvdata1
 031 #    =============================================================================

 043 export RUNLIBS=$HOME/demo RUNDATA=$HOME/demo CNVDATA=$HOME/demo
 044 #===============================================================

Listings of .bash_profile & common_profile

OR see your actual profiles using the editor as follows:


 #4a. vi $HOME/.bash_profile  - copied/renamed from $APPSADM/env/bash_profile_uv
      ======================               - calls 1 of following common_profiles
 #4b. vi $APPSADM/env/common_profile_uv    - download/install on user machines
 #4c. vi $APPSADM/env/common_profile_web1  - logins to UV Software uvadm in $HOME
 #4d. vi $APPSADM/env/common_profile_web2  - logins to UV Software uvadm elsewhere

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A5. Login to UV Software website to run UVdemos1

Logout & Login again if RUNDATA/RUNLIBS modified


 #0.  exit    <-- logoff & backon if you changed RUNDATA/RUNLIBS definitions
      ====        (to change $PATH & $PFPATH definitions in the common_profile)

 #1. ssh uvsoftxx@uvsoftware.ca  <-- Online userid in range uvsoft00-uvsoft99
     ==========================
     --> passxx                  <-- enter your password at the prompt

 #1a. Login userxx   <-- Alternative Login on your own machine
      ============       if you have downloaded & installed Vancouver Utilities
     --> passxx      <-- enter your password at the prompt

 #2.  cdd  --> /home/uvsoftxx/demo (alias cdd='cd $RUNDATA')

 #2a. cdd  --> /home/userxx/demo OR 'userxx' if OffLine account

 #3. tree . | more  <-- list directory TREE diagram (for '.' current directory)
     ===i=========     - Here are relevant files for the 1st few demos
                       - compiling/executing some C program games
                       - executing equivalent uvcopy jobs

cdd/cdl - aliases cd to $RUNDATA/$RUNLIBS

'cdd/cdl' are aliases that give you quick access to $RUNDATA & $RUNLIBS Aliases 'cdd/cdl' are defined in the comon_profile (about line 140) as follows:

      alias cdd="cd $RUNDATA"
      alias cdl="cd $RUNLIBS"

For JCL conversions, RUNDATA & RUNLIBS are defined as:


 030 export RUNLIBS=$HOME/testlibs1 RUNDATA=$HOME/testdata1 CNVDATA=$HOME/cnvdata1
 031 #============================================================================

For these UVdemos, we define RUNDATA & RUNLIBS as:


 043 export RUNLIBS=$HOME/demo RUNDATA=$HOME/demo CNVDATA=$HOME/demo
 044 #===============================================================

As you can see RUNDATA & RUNLIBS are difined differently for JCL conversions, but are defined the same '$HOME/demo' for the UVdemos2. So 'cdd' & 'cdl' have the same effect for the UVdemos.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A6. UVdemos1 - Introduction to the Vancouver Utilities

subdirs/files in demo/... relevant to UVdemos1 (vs UVdemos2)

 /home/uvadm/demo
 |-- bin/                 <---- compiled C program binaries
 |   |-- helloworld*
 |   |-- insult1*             - see descriptions below under src/...
 |   |-- insult2*
 |   |-- lotto1*
 |   |-- test64a*
 |   |-- test64b*
 |   `-- test64c*
 |-- dat0/                <---- .csv files to be converted to fixed layouts
 |   |-- Provinces.csv      <-- will be converted to dat1/Provinces1 & Provinces2
 |   |-- UScongress.csv       - Congress members to demo table summary by party,sex,etc
 |   |-- USstates.csv         - US states, population, capital, capital pop
 |   |-- ...                  - several files not shown (not relevant to UVdemos1)
 |   `
 |-- dat1/                <---- fixed-field layout files converted from dat0/...csv files
 |   |-- ...                  - several files not shown (not relevant to UVdemos1)
 |   |-- Provinces1         <-- converted from Provinces.csv, all fields 20 bytes apart
 |   |-- Provinces2           - fields reformatted appropriate to content
 |   |-- UScongress           - used to demo table summary by party, sex, etc
 |   |-- insults            <-- data file for insult2
 |   `--
 |-- dat2/                 <--- subdir/files used in UVdemos2 but not in UVdemos1
 |   |--...
 |   `
 |-- dat3/                    <--- subdir/files to demo uvfix1 creating scripts
 |   |-- about.dat3.files
 |   |-- gl.account.trans
 |   |-- product.master1
 |   |-- uvfix.promo
 |   |-- warranty_master
 |   `
 |-- dat9/                <---- subdir/files created by YOUR tutorials (vs our demos)
 |   |-- provinces2         <-- created by pfxx/provincefix2 (user tutorial)
 |   |--                      - or by pfyy/provincefix2 (UV Software's solution)
 |   |-- provinces2a        <-- created by pfxx/provincefix2a (or pfyy/provincefix2a)
 |   |--                      - same as provinces2 with Total population
 |   |-- provinces2.tbl     <-- created by pfxx/provincefixtbl2 (or pfyy/provincetbl2)
 |   `--
 |-- pf/                  <---- uvcopy demo jobs (written by UV Software)
 |   |-- provinces2           - same as Provinces2 above, but created by your job
 |   |-- ...
 |   `
 |-- pf/                  <---- uvcopy demo jobs (written by UV Software)
 |   |-- Provincefix1       <-- convert Provinces.csv to Provinces1 fixed 20 apart
 |   |-- Provincefix2       <-- convert Provinces.csv to Provinces2 reformatted
 |   |-- USstatesfix1         - see usage in UVdemos2 (vs this UVdemos1)
 |   |-- USstatesfix2
 |   |-- testint1           <-- uvcopy equivalent of C program test64a.c
 |   |-- uvlotto1           <-- uvcopy equivalent of C program lotto1.c
 |   `--

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

subdirs/files in $HOME/demo/... (continued) ***

Note
  • we are showing ONLY the directories RELEVANT to running the demos OnLine
 |-- pfxx/                <---- user written uvcopy job tutorials
 |   |-- copy1                - skeleton (framework) for creating new uvcopy jobs
 |   |-- .........          <-- user to create uvcopy jobs by copy/rename/modify copy1
 |   |-- .........              (provincefix1,provincefix2,provincefix2a,provincetbl2)
 |   |-- .........            - may compare to pfyy/uvcopy jobs (by UV Software)
 |   |--
 |-- pfyy/                <---- answers to user tutorials
 |   |-- provincefix1         - compare your solutions to UV Software solutions
 |   |-- provincefix2         - minimal version pf/Provincefix2, created from copy1
 |   |-- provincefix2a        - same with total population line added
 |   |-- provincetbl2         - table summary of province counts & populations by Gov Party
 |   |-- UScongresstbl1       - table summaries of congress members by party, sex, etc
 |   |-- UScongresstbl2
 |   |-- UScongresstbl3
 |   `--
 |-- src/                 <---- C source programs
 |   |-- helloworld.c       <-- traditional C learner's 1st C program
 |   |-- insult1.c          <-- generate Shakespearean insults
 |   |-- insult2.c            - same, using a data file that you can update
 |   |-- lotto1.c           <-- random number generator
 |   |-- test64a.c          <-- test 64 bit limits, multiply 1 by 16, repeat to limit
 |   |-- test64b.c            - same as test64a & accepts initial value & multiplier
 |   `-- test64c.c            - solicits 1 value & shows short,int,long conversions
 |   `--
 |-- sf                   <---- Script Files for UVdemos2 (vs UVdemos1)
 |   `--
 |-- sfxx                 <---- Script Files - exercises for UVdemos1 Tutorials
 |   |-- rename_dat3          - enhance messy scripts by column aligning
 |   `--
 |-- sfyy                 <---- Script Files - answers to Tutorials
 |   |-- rename_neat1         - create scripts with column alignment
 |   |-- rename_dat3          - enhance messy scripts - BEFORE alignment
 |   |-- rename_dat3_neat2      enhance messy scripts - AFTER alignment
 |   `--
 |-- tf/                  <---- Test Files (similar to dat1/...files)
 |   |-- ...                  - several files not shown (not relevant to UVdemos1)
 |   |-- insults_Owens      <-- alternate files of insults
 |   `-- insults_Shakespeare  - can copy/rename to dat1/insults
 |-- tf/                  <---- Test Files (similar to dat1/...files)
 |   |-- ...                  - several files not shown (not relevant to UVdemos1)
 |   |-- insults_Owens      <-- alternate files of insults
 |   `-- insults_Shakespeare  - can copy/rename to dat1/insults
 |   `--
 |-- tmp/                 <---- uvfix jobs preliminary output to tmp/...
 |   |--                      - for inspection before copy back to desired directory
 |-- tmp1/                    - tmp dirs keep demo/... working directory clean
 |-- tmp2/                    - suggest --> rm -f tmp*/* <-- before each session

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

1A1. Preparation to Run UVdemos1 Demos & Tutorials

Printing UVdemos1.pdf Guide for Tutorials

We recommend printing UVdemos1.pdf as a guide when you are executing the various demo tutorials. You can highlight the commands to be run & make notes for any errors & unclear explanations you find in the documentation that you could send to UV Software.

Of course if you have 2 terminals or a big screen with 2 big windows, you could have www.uvsoftware.ca/uvdemos.htm on 1 window & run the demos on a 2nd window.

It is good to have a screen/window for the documentation on the wedbsite to follow the many links to other parts of UVdemos1 & other documents such as the uvcopy instruction documentaation at www.uvsoftware.ca/uvcopy3.htm

And also good to have the hard-copy for making notes & highlights.

After you ssh/login you will see UVdemos1.pdf in subdir upload/... For example:


 #1. l upload
     ========
      -rw-rw-r-- 1 uvbak apps 29950592 Sep 11 10:46 20200911_uvadm.zip
      -rw-rw-r-- 1 uvbak apps   546966 Sep 11 10:50 20200911_appsadm.zip
      -rw-rw-r-- 1 uvbak apps   572158 Sep 11 10:53 20200911_uvdemos.zip
      -rw-rw-r-- 1 uvbak apps   312008 Sep 11 10:36 UVdemos1.doc
      -rw-rw-r-- 1 uvbak apps   364894 Sep 11 10:37 UVdemos1.pdf         <--Note pdf
      -rw-rw-r-- 1 uvbak apps   390081 Sep 11 10:39 uvdemos1.htm

 #2. You could sftp/login & download UVdemos1.pdf to print on your printer.
     It is about 65 pages Duplexed.

UVdemos1.pdf email attachment

Alternatively, we usually attach UVdemos.pdf to the email when we send you your userid/password to ssh/login.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_2. Running C programs & Equivalent uvcopy jobs

UVdemos1 - Part 2 - Contents


2A1. lotto1.c - C program to generate Random Numbers

2A2. lotto1.c - source listing of C program

2A3. uvlotto1 - uvcopy job to generate Random Numbers
- uvcopy equivalent of lotto1.c program

2A4. countcode1 - utility to Count Lines of code in programming languages
such as C & uvcopy jobs

2B1. insult2.c - C program to generate Random Insults

2C1. test64a.c - 1st of 4 programs to test integer limits
#1. test64a - test 64 bit limits - start# 1 & multiply by 16

2C2. test64b.c - 2nd of 4 programs to test integer limits
#2. test64b - like test64a, but accepts start# & multiplier

2C3. uvcopy testint1 - 3rd of 4 programs to test integer limits
#3. testint1 - uvcopy job to test 64 bit integer limits

2C4. source listing of uvcopy job to test 64 bit integer limits

2C5. #4. test64c.c - 4th of 4 programs to test integer limits

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2A1. Running C programs & Equivalent uvcopy jobs

As mentioned previously, UVdemos1 was inspired by my grandchildren. I wanted to allow them to login to my website with a userid/password & run the C compiler & simpler UV demos without having to download & install the Vancouver Utilities.

I have selected some demos that should interest any unix/linux user:

lotto1.c - C program to generate Random Numbers


 #1. ssh uvsoftxx@uvsoftware.ca  <-- Online userid in range uvsoft00-uvsoft99
     ==========================
     --> passxx                  <-- enter your password at the prompt

 #1a. Login userxx   <-- Alternative Login on your own machine
      ============       if you have downloaded & installed Vancouver Utilities
     --> passxx      <-- enter your password at the prompt

 #2.  cdd  --> /home/uvsoftxx/demo (alias cdd='cd $RUNDATA')
      ===
 #2a. cdd  --> /home/userxx/demo OR 'userxx' if OffLine account

 #3. cc src/lotto1.c -o bin/lotto1 <-- compile source to executable
     ============================

 #4. lotto1 6 1 49       <-- generate 6 random numbers from 1 to 49 (Canada's Lotto649)
     =============
      39
      14                 <-- sample output
      15
      25
      15
      47

 #5. vi src/lotto1.c   <-- inspect source code if desired
     ===============

The lotto1.c C source program is 95 lines (49 code & 46 /*comments*/) We list it on the next page with minimal /*comments*/

Note re $PATH

If you are new to Unix/Linux, you might wonder how the system finds executable programs (& scripts) when you enter only the program/script name (as in #4 above) ? Answer - by searching '$PATH' setup by the common_profile when you login.


 #6. echo $PATH  --> results very long, but contains ...:$HOME/demo/bin:...
     ==========

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2A2. Running C programs & Equivalent uvcopy jobs

Listing of lotto1.c program to generate random numbers

 #include <stdio.h>
 #include <stdlib.h>
 #include <time.h>
 #include <sys/time.h>
 int cycles;           /* arg#1 - random numbers desired         */
 long low;             /* arg#2 - low raange                     */
 long high;            /* arg#3 - high range                     */
 int count = 0;        /* counter up to cycles random#s desired  */
 time_t timeseed;      /* seed for srand()                       */
 long rand1;           /* current random# generated              */
 struct timeval tv;    /* for gettimeofday(&tv,0);               */
 suseconds_t micros1;  /* from struct timeval suseconds_t tv_usec*/
 /* mainline for lotto1 */
 int main(int argc, char *argv[])
 {
 /* verify operator entered correct number of arguments   */
 if (argc != 4)
   { printf("--> lotto1 cycles low high <-- ***ERROR***, 3 numeric arguments expected \n");
     printf("--> lotto1   6     1   49  <-- example for Lotto 649\n");
     exit(91);
   }
 /* capture command line arguments */
 cycles = atoi(argv[1]);   /* arg1 random#s desired */
 low    = atol(argv[2]);   /* arg2 low range        */
 high   = atol(argv[3]);   /* arg3 high range       */
 /* get seed for rand() function - using current time */
 /* srand((unsigned) time(&timeseed));                */
 /* time_t seconds gets same seed unless wait 1+ secs */
 /* - use gettimeofday get microseconds for srand seed*/
 gettimeofday(&tv,0);
 micros1 = tv.tv_usec;          /* store local for debug */
 srand((unsigned) tv.tv_usec);  /* get seed using TOD microsecs*/

/* verify low < high */ if (low >= high) { printf("ERROR: low range > highrange\n"); exit(92); }

 /* begin loop to calc & print results */
 while (count < cycles)
   {  rand1 = (rand() % high);
      if (rand1 < low)
        { continue;
        }
      count++;       /* count up to cycles */
      printf("%ld\n",rand1);
   }
 exit(0);
 }

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2A3. Running C programs & Equivalent uvcopy jobs

uvlotto1 - uvcopy job to gnerate Random Numbers

'uvlotto1' is a uvcopy job that does the same thing as the 'lotto1' C program, but with only 20 lines of code (& 22 lines of #comments).


 #1. uvcopy uvlotto1   <-- run uvcopy job pf/uvlotto1
     ===============     - same as lotto1.c program
      enter: number of random#s desired, low range, high range
      --> 6,1,49 <-- Example: 6 numbers from 1 to 49 (6/49 lottery)
      random# 1 = 12
      random# 2 = 7
      random# 3 = 3
      random# 4 = 30
      random# 5 = 12
      random# 6 = 47

 #2. vi $UV/pf/uvlotto1    <-- inspect source code for uvcopy job 'uvlotto1'
     ==================      - listed below:
 # uvlotto1 - generate random numbers with uvcopy
 @run
 man10   msg     'enter number,low,high Not entered, enter 3 arguments now'
         msgwa1  ' - OR reply null to default 6,1,49 - OR enter q to quit'
         cmc     $arg1(1),'q'            quit ?
         skp=    man90                   yes - goto EOJ
         fix     a0(20),$arg1,3,','      separate to a0(20),a20(20),a40(20)
         mvnx3   $ca1,a0(20)             convert to binary in $ca1,$ca2,$ca3
         cmn     $ca3,0                  3 arguments entered ?
         skp>    man20
         mvn     $ca1,6
         mvn     $ca2,1
         mvn     $ca3,49
 #
 # begin loop to generate $ca1 random#s between $ca2 & $ca3
 man20   add     $ca5,1                  count loops
 #       =====================
         ran     $ca4,$ca2,$ca3          gen current random# in $ca4
 #       =====================
         msgv1   'random# $ca5 = $ca4'   show current loop count & random#
         cmn     $ca5,$ca1               loop ctr = number requested ?
         skp<    man20                   no - repeat loop
         mvn     $ca5,0                  clear loop ctr for next set
         skp     man10                   return to prompt
 man90   eoj                             End job
Note

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2A4. Running C programs & Equivalent uvcopy jobs

Counting & Comparing Lines of code for C & uvcopy

We can run a pre-programmed uvcopy utility job 'countcode1' to count Lines of Code for various programming & script languages, for example:


 #1. uvcopy countcode1,fili1=src/lotto1.c   <-- count lines of code & /* comments
     ====================================       in C program src/lotto1.c

 #2. cat tmp/lotto1.counts
     =====================
      countcode1 - count lines of code & comments for scripts,programs,uvcopy jobs,etc
      Host=uvsoft5, User=uvadm, Date=2020/09/03_17:03:30, File=src/games/lotto1.c
      Lines with '#'  col 1   =        4   without =       91
      Lines with '/*' col 1-2 =       46   without =       49
      Lines with all spaces -->        8
      Total Lines ------------>       95

 #3. uvcopy countcode1,fili1=pf/uvlotto1   <-- count lines of code & /* comments
     ===================================       in uvcopy job pf/uvlotto1

 #4. cat tmp/uvlotto1.counts
     =======================
 countcode1 - count lines of code & comments for scripts,programs,uvcopy jobs,etc
 Host=uvsoft5, User=uvadm, Date=2020/09/03_17:02:27, File=pf/demo/uvlotto1
 Lines with '#'  col 1   =       23   without =       20
 Lines with '/*' col 1-2 =        0   without =       43
 Lines with all spaces -->        0
 Total Lines ------------>       43
Note
  • the 4 lines with '#' col 1 of the C program are code (not comments as in uvlotto1)
  • the C program has 49+4 = 53 lines of code & 46 lines of comments
  • the uvcopy job has 20 lines of code & 23 lines of comments

You can run 'uvcopy countcode1' for any other C programs & uvcopy jobs you see

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2B1. Running C programs & Equivalent uvcopy jobs

insult2.c - C program to generate Random Insults


 #1. cc src/insult2.c -o bin/insult2 <-- compile source to executable
     ===============================

 #2. insult2 5      <-- Execute bin/insult2 to generate 5 insults
     =========
      Thou  jarring,  boil-brained,  hugger-mugger!
      Thou  errant,  idle-headed,  lap-dragon!
      Thou  qualling,  dizzy-eyed,  harpy!
      Thou  impertinent,  fool-born,  fustilarian!
      Thou  wayward,  beef-witted,  baggage!

 #3. vi src/insult2.c   <-- inspect source code if desired (310 lines)
     ================

 #4. vi dat1/insults    <-- inspect the file of insult words
     ===============
 001 # tf/insults_Shakespeare - data file for $UV/src/games/insult2.c
 002 # dat1/insults <-- must copy/rename from tf/insults_Shakepeare
 003 #                - see www.uvsoftware.ca/uvdemos2.htm#Part_11
 004 # - insult2 reads insults from subdir/file = dat1/insults
 005 # - allows multiple files, in dat0/insultsA,B,C,etc, copy 1 to dat1/insults
 006 # - seq# cols 1-3, 3 insult words, 25 bytes each, at columns 6,31,56
 007 # - insult words must start on line 11, preceded by 10 lines of #comments
 008 #
 009 #    1         2         3         4         5         6         7         8
 010 #678901234567890123456789012345678901234567890123456789012345678901234567890
 011  artless                  base-court               apple-john
 012  bawdy                    bat-fowling              baggage
 013  beslubbering             beef-witted              barnacle
 014  bootless                 beetle-headed            bladder
 015  churlish                 boil-brained             boar-pig
 016  cockered                 clapper-clawed           bugbear
 017  clouted                  clay-brained             bum-bailey
 018  craven                   common-kissing           canker-blossom
 019  currish                  crook-pated              clack-dish
 020  dissembling              dizzy-eyed               coxcomb
                  ------- 37 lines omitted -------
 057  wayward                  toad-spotted             vassal
 058  weedy                    unchin-snouted           whey-face
 059  yeasty                   weather-bitten           wagtail

Two versions of the insult program

insult1
  • same as insult2 (demo above), but with the 3 columns of insult words
    hard-coded within the C program
insult2
  • same as insult1, but reads the 3 columns of insult words from a data-file
    which makes it easier for you to change the insults if desired
  • or make up an entirely new insults file.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2C1. Running C programs & Equivalent uvcopy jobs

test64a.c - 1st of 4 programs to test integer limits

  1. * $UV/src/testC/test64a.c - test 64 bit limits, hard-coded init 1, multiplier 16
  2. $UV/src/testC/test64b.c - same as test64a & accepts initial value & multiplier
  3. $UV/pf/adm/testint1 - uvcopy job that creates same report as C program test64a.c
  4. $UV/src/testC/test64c.c - solicits 1 value & shows short,int.long conversions

    #1. test64a - test 64 bit limits - start# 1 & mpy 16


 #1. cc src/test64a.c -o bin/test64a  <-- compile source to executable
      ===============================

 #2. test64a   <-- repeat loop (1*16*) until 0 or negative (17 times)
     =======
 test64a.c - test 64 bit long integer limits (19 digits)
 cycle   decimal-value        hex native       hex swapped   back to decimal ,edited
 00 00000000000000000001 0100000000000000 0000000000000001                          1
 01 00000000000000000016 1000000000000000 0000000000000010                         16
 02 00000000000000000256 0001000000000000 0000000000000100                        256
 03 00000000000000004096 0010000000000000 0000000000001000                      4,096
 04 00000000000000065536 0000010000000000 0000000000010000                     65,536
 05 00000000000001048576 0000100000000000 0000000000100000                  1,048,576
 06 00000000000016777216 0000000100000000 0000000001000000                 16,777,216
 07 00000000000268435456 0000001000000000 0000000010000000                268,435,456
 08 00000000004294967296 0000000001000000 0000000100000000              4,294,967,296
 09 00000000068719476736 0000000010000000 0000001000000000             68,719,476,736
 10 00000001099511627776 0000000000010000 0000010000000000          1,099,511,627,776
 11 00000017592186044416 0000000000100000 0000100000000000         17,592,186,044,416
 12 00000281474976710656 0000000000000100 0001000000000000        281,474,976,710,656
 13 00004503599627370496 0000000000001000 0010000000000000      4,503,599,627,370,496
 14 00072057594037927936 0000000000000001 0100000000000000     72,057,594,037,927,936
 15 01152921504606846976 0000000000000010 1000000000000000  1,152,921,504,606,846,976
 16 00000000000000000000 0000000000000000 0000000000000000                          0

 #3. vi $UV/src/testC/test64a.c  <-- inspect source code of C program
     ==========================
     - 160 lines (100 lines of code & 60 lines of /*comments*/)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2C2. Running C programs & Equivalent uvcopy jobs

test64b.c - 2nd of 4 programs to test integer limits

  1. $UV/src/testC/test64a.c - test 64 bit limits, hard-coded init 1, multiplier 16
  2. * $UV/src/testC/test64b.c - same as test64a & accepts initial value & multiplier
  3. $UV/pf/adm/testint1 - uvcopy job that creates same report as C program test64a.c
  4. $UV/src/testC/test64c.c - solicits 1 value & shows short,int.long conversions

    #2. test64b - like test64a, but accepts start# & multiplier


 #8a. cc src/test64b.c -o bin/test64b  <-- compile source to executable
      ===============================

 #8b. test64b 1 2   <-- starting from 1, multiply by 2,& repeat until 0 or negative
      ===========     - cycles 64 times as shown below
 test64b.c - test 64 bit limits (like test64a, but enter your start# & multiplier)
 cycle   decimal-value        hex native       hex swapped   back to decimal ,edited
 00 00000000000000000001 0100000000000000 0000000000000001                          1
 01 00000000000000000002 0200000000000000 0000000000000002                          2
 02 00000000000000000004 0400000000000000 0000000000000004                          4
 03 00000000000000000008 0800000000000000 0000000000000008                          8
 04 00000000000000000016 1000000000000000 0000000000000010                         16
 05 00000000000000000032 2000000000000000 0000000000000020                         32
 06 00000000000000000064 4000000000000000 0000000000000040                         64
 07 00000000000000000128 8000000000000000 0000000000000080                        128
 08 00000000000000000256 0001000000000000 0000000000000100                        256
 09 00000000000000000512 0002000000000000 0000000000000200                        512
 10 00000000000000001024 0004000000000000 0000000000000400                      1,024
 11 00000000000000002048 0008000000000000 0000000000000800                      2,048
 12 00000000000000004096 0010000000000000 0000000000001000                      4,096
 13 00000000000000008192 0020000000000000 0000000000002000                      8,192
 14 00000000000000016384 0040000000000000 0000000000004000                     16,384
 15 00000000000000032768 0080000000000000 0000000000008000                     32,768
                ------- 16 bit short limit -------
 16 00000000000000065536 0000010000000000 0000000000010000                     65,536
 17 00000000000000131072 0000020000000000 0000000000020000                    131,072
 18 00000000000000262144 0000040000000000 0000000000040000                    262,144
                         12 lines omitted
 30 00000000001073741824 0000004000000000 0000000040000000              1,073,741,824
 31 00000000002147483648 0000008000000000 0000000080000000              2,147,483,648
                ------- 32 bit int limit -------
 32 00000000004294967296 0000000001000000 0000000100000000              4,294,967,296
 33 00000000008589934592 0000000002000000 0000000200000000              8,589,934,592
                         28 lines omitted
 61 02305843009213693952 0000000000000020 2000000000000000  2,305,843,009,213,693,952
 62 04611686018427387904 0000000000000040 4000000000000000  4,611,686,018,427,387,904
                ------- 64 bit long limit -------
 63 -9223372036854775808 0000000000000080 8000000000000000 -9,223,372,036,854,775,808

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2C3. Running C programs & Equivalent uvcopy jobs

uvcopy testint1 - 3rd of 4 programs to test integer limits

  1. $UV/src/testC/test64a.c - test 64 bit limits, hard-coded init 1, multiplier 16
  2. $UV/src/testC/test64b.c - same as test64a & accepts initial value & multiplier
  3. * $UV/pf/adm/testint1 - uvcopy job that creates same report as C program test64a.c
  4. $UV/src/testC/test64c.c - solicits 1 value & shows short,int.long conversions

    testint1 - uvcopy job to test 64 bit integer limits

'testint1' is a uvcopy job that creates the saame output as the 'test64b.c' C program. You can run it as follows:


 #1. uvcopy testint1       <-- Execute job
     ===============         - writes report to tmp/testint1.H64
      uop=m16   - default multiplier of 16 - 17 cycles to reach limit
      uop=m2    - change multiplier to 2 - 65 cycles to reach limit
      null to accept or enter/override -->   <-- null to accept default m16
      20200621:482406:testint1: EOF filo01 wrts=19 size=1662: tmp/testint1.H64
      EOJ, Output File written to: tmp/testint1.H64
      enter: vi,cat,more,lp,uvlp12,etc (default more) -->   <--null to default to 'more'
      --- OR could inspect later using 'vi' as follows:

 #2. cat tmp/testint1.H64  <-- inspect output file
     ====================
                             ** sample report **
 testint1 - Site=UV_Software, Host=uvsoft5, Login=uvadm, Date=2020/06/19_21:06:36
 cycle   decimal-value        hex native       hex swapped     back to decimal ,edited
 01  00000000000000000001  0100000000000000  0000000000000001                          1
 02  00000000000000000016  1000000000000000  0000000000000010                         16
 03  00000000000000000256  0001000000000000  0000000000000100                        256
 04  00000000000000004096  0010000000000000  0000000000001000                      4,096
 05  00000000000000065536  0000010000000000  0000000000010000                     65,536
 06  00000000000001048576  0000100000000000  0000000000100000                  1,048,576
 07  00000000000016777216  0000000100000000  0000000001000000                 16,777,216
 08  00000000000268435456  0000001000000000  0000000010000000                268,435,456
 09  00000000004294967296  0000000001000000  0000000100000000              4,294,967,296
 10  00000000068719476736  0000000010000000  0000001000000000             68,719,476,736
 11  00000001099511627776  0000000000010000  0000010000000000          1,099,511,627,776
 12  00000017592186044416  0000000000100000  0000100000000000         17,592,186,044,416
 13  00000281474976710656  0000000000000100  0001000000000000        281,474,976,710,656
 14  00004503599627370496  0000000000001000  0010000000000000      4,503,599,627,370,496
 15  00072057594037927936  0000000000000001  0100000000000000     72,057,594,037,927,936
 16  01152921504606846976  0000000000000010  1000000000000000  1,152,921,504,606,846,976
 17  00000000000000000000  0000000000000000  0000000000000000                          0

 #3. vi $UV/pf/adm/testint1  <-- inspect uvcopy job
     ======================    - see listing below:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2C4. Running C programs & Equivalent uvcopy jobs

source listing of uvcopy job to test 64 bit integer limits

 opr='$jobname - uvcopy job to test 64 bit integer limits'
 opr='uop=m16 - default options'
 opr='    m16 - default multiplier of 16 - 17 cycles to reach limit'
 opr='    m2  - changing multiplier to 2 - 65 cycles to reach limit'
 uop=q1m16      # default options
 rop=r1x4       # r1 = prompt for report disposition, x4 = default 'more'
 filo1=?tmp/$jobname.$bits,rcs=128,typ=LSTt
 lodv1=h0(100)
 $jobname - Site=$sitename, Host=$hostname, Login=$logname, Date=$datetime
 cycle   decimal-value        hex native       hex swapped     back to decimal ,edited
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 @run
        opn    all
        wtb    filo1,h0(100),h0(100)    write page headings
        mvn    $ca1,1                   init accumulator
 #
 # begin loop to convert, print, double,& repeat
 man20  clr    b0(80),' '               clear output area
        add    $ca4,1                   count cycles sequence#
        cmn    $ca4,66                  limit safeguard ?
        skp>   man20                    repeat loop until err or 67 cycles
 # convert binary to numeric & back to binary
        mvn    c0(20),$ca1              convert binary to numeric
        mvn    $ca2,c0(20)              convert numeric back to binary
 # edit fields for print output
        mvn    b0(2),$ca4               cycle#
        mvn    b4(20),$ca1              decimal value in 20 bytes
        hxc    b26(16),$ca1             convert to hex display
        mvn    c20(8bs),$ca1            swap Big/Little ends for display
        hxc    b44(16),c20(8)           convert to hex display
        edt    b62(26),$ca2,'z,zzz,zzz,zzz,zzz,zzz,zz9-'
        put    filo1,b0                 write output
 # multiply current value by multplier & test limit reached
        mpy    $ca1,$uopbm              current * multiplier for next cycle
        skp>   man20                    quit on error reaching 20 digits
        add    $ca5,1                   switch for 1 more cycle to show err
        cmn    $ca5,2                   reached 2nd err ?
        skp<   man20                    return until 2nd err
 #
 man90  cls    all                      close files
        eoj                             end job

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

2C5. Running C programs & Equivalent uvcopy jobs

test64c.c - 4th of 4 programs to test integer limits

  1. $UV/src/testC/test64a.c - test 64 bit limits, hard-coded init 1, multiplier 16
  2. $UV/src/testC/test64b.c - same as test64a & accepts initial value & multiplier
  3. $UV/pf/adm/testint1 - uvcopy job that creates same report as C program test64a.c
  4. * $UV/src/testC/test64c.c - solicits 1 value & shows short,int.long conversions

 #1. cc src/test64c.c -o bin/test64c  <-- compile source to executable
     ===============================

 #2a. test64c 32767   <-- test conversion to binary & back to numeric
      =============
      short: 32767  int: 32767  long: 32767  <-- conversion 32767 max for short
      =====================================

 #2b. test64c 1234567890 <-- 10 digits max for 32 bit int
      ==================
      short: 722  int: 1234567890  long: 1234567890  <-- short conversion garbage
      =============================================

 #2c. test64c 1234567890123456789   <-- 19 digits max for 64 bit long
      ===========================
      short: -32491  int: 2112454933  long: 1234567890123456789
      =========================================================
       - short & int conversions garbage, but long OK for 19 digits

 3. vi $UV/src/testC/test64c.c  <-- inspect source code of C program
     ==========================
     - shortened here to 21 lines of code by omiting 30 lines of front end /*comments*/
       (explanations & examples similar to the above)
      #include <stdio.h>
      #include <stdlib.h>
      int main(int argc, char **argv)
      {
      short s;
      int i;
      long l;
      __int128 ll;  /* no printf format for __int128 */
      printf("test64c.c - test printf max digits for short(16), int(32),& long(64) \n");
      printf(" - short(16) max=32768, int(32) max=...  ,long(64) max=... \n");
      printf(" - enter your number & increase digits until failure for short,int,long\n");
      if (argc != 2)
        { fprintf(stderr, "\n ERROR (no number entered), Usage--> %s number \n", argv[0]);
          return 1;
        }
      s = atoi(argv[1]);
      i = atoi(argv[1]);
      l = atol(argv[1]);
      printf("\nshort: %d  int: %d  long: %ld \n",s,i,l);
      return(0);
      }

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_3. uvcopy jobs most interesting for beginners

UVdemos1 - Part 3 - Contents


3A1. hexcalc1 - convert numbers from decimal to binary or binary to decimal
- displayed in hexadecimal

3A2. hexcalc1 - uvcopy job listing

3B1. datedemo1 - convert dates between Calendar, Julian,& days-since-1900
- handy tool to calculate days between 2 dates, etc
- fun way to calculate your age in days or weekday you were born on

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3A1. Handy Tools for Unix/Linux

hexcalc1 - convert hex to decimal & decimal to hex

There are 2 versions, hexcalc1 for Little-End machines (Intel) & hexcalc2 for Big-End machines (mainframes). Look at my samples, then try your own conversions.


 uvcopy hexcalc1       <-- run hexcalc1 for Intel (use hexcalc2 for RISC)
 ===============
 enter number with prefix "d" decimal, "x" hex rep, "q" quit
 entry-----> x100
 response--> hex 0000000000000100 = 256 in decimal
 enter number with prefix "d" decimal, "x" hex rep, "q" quit
 entry-----> d256
 response--> decimal 256 = 0000000000000100 in hex
 enter number with prefix "d" decimal, "x" hex rep, "q" quit
 entry-----> d1,000,000
 response--> decimal 1,000,000 = 00000000000F4240 in hex
 enter number with prefix "d" decimal, "x" hex rep, "q" quit
 entry-----> d1,000,000,000
 response--> decimal 1,000,000,000 = 000000003B9ACA00 in hex
 enter number with prefix "d" decimal, "x" hex rep, "q" quit
 entry-----> d1,000,000,000,000
 response--> decimal 1,000,000,000,000 = 000000E8D4A51000 in hex
 enter number with prefix "d" decimal, "x" hex rep, "q" quit
 entry-----> d1,000,000,000,000,000
 response--> decimal 1,000,000,000,000,000 = 00038D7EA4C68000 in hex
 enter number with prefix "d" decimal, "x" hex rep, "q" quit
 entry-----> d1,000,000,000,000,000,000
 response--> decimal 1,000,000,000,000,000,000 = 0DE0B6B3A7640000 in hex
 enter number with prefix "d" decimal, "x" hex rep, "q" quit
 entry-----> x0DE0B6B3A7640000
 response--> hex 0DE0B6B3A7640000 = 1,000,000,000,000,000,000 in decimal
 enter number with prefix "d" decimal, "x" hex rep, "q" quit
 entry-----> q
 uvcopy hexcalc1 start 2020/06/22_21:30:17 end 21:33:16 elapsed 002_59_185

 uvcopy hexcalc1    <-- now try your own conversions
 ===============
 enter number with prefix "d" decimal, "x" hex rep, "q" quit
 entry-----> ...    <-- your entries

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3A2. hexcalc1 - convert hex to decimal & decimal to hex

hexcalc1 - uvcopy job listing

 # hexcalc1 - uvcopy Parameter File from UVSI stored in: /home/uvadm/pf/util/
 #          - hexadecimal calculator, hex to decimal, or decimal to hex
 # hexcalc1 - Little-End machines (Intel) & hexcalc2 for Big-End (mainframes)
 @run
        msg     'hexcalc1 for Little-End (Intel), hexcalc2 for Big-End (mainframes)'
        msg     'decimal->hex,hex->decimal,ex: d4096->x1000, x1000->d4096'
 #
 # begin loop to solicit numbers for conversion
 man20  msgwa1  'enter number with prefix "d" decimal, "x" hex rep, or "q" quit'
        mvc     a0(80),$arg1            store entry & blank extra w/s
        cmc     a0(1),'d'               dcml to hex ?
        skp=    dtox
        cmc     a0(1),'x'               hex to dcml ?
        skp=    xtod
        cmc     a0(1),'q'               quit ?
        skp=    quit
        msg     'invalid entry, 1st char not "d" or "x"'
        skp     man20
 #
 # convert decimal to hex
 # - max 20 numerics, allow 26 characters in case ','s present
 dtox   mvn     a40(20),a1(26)          drop the 'd' & any ',' commas
        mvn     b00(8bs),a40(20)        cnvrt dcml to binary ('s' switch Intel)
        hxc     b40(16),b00(8b)         cnvrt binary to hex rep
        mvf     b100(80),'decimal ddd = xxx in hex '
        rep     b100(80),'ddd',a1(26)   replace pattern with valaue
        rep     b100(80),'xxx',b40(16)  always show hex result in 16 bytes
        sqzc1   b100(80),' '            squeeze to 1 blank between words
        msgl2   b100(80)                display result
        skp     man20
 #
 # convert hex to decimal
 xtod   mvc     a40(16),a1              drop prefix
        sqzr    a40(16),' '             right justify
        rep     a40(16),' ','0'         left zero fill
        chx     c00(8),a40(16)          cnvrt hex to binary
        mvn     c40(20),c0(8bs)         cnvrt binary to decimal (switch Intel)
        edt     c60(26),c0(8bs),'z,zzz,zzz,zzz,zzz,zzz,zzz-'
        mvf     c100(80),'hex xxx = ddd in decimal'
        rep     c100(80),'xxx',a40(16)
        rep     c100(80),'ddd',c60(26)
        sqzc1   c100(80),' '            squeeze to 1 blank between words
        msgl2   c100(80)                display result
        skp     man20
 quit   eoj

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B2. uvcopy utility jobs you might find useful

datedemo1 - convert between calendar, julian,& days-since-1900

'datedemo1' is an interactive demo job to convert dates between 3 formats, and to add/subtract a number of days from previous days-since-1900, using commands:

  1. 'c' & ccyymmdd - calendar date to be converted to all 3 formats
  2. 'j' & ccyyddd - julian date to be convedrted to all 3 formats
  3. 'n' & number of days since 1900 - to be converted to all 3 formats
  4. '+' or '-' & number of days to add to or subtract from current date

Since UVdemos1 was originally intended to assist my grandchildren to learn about programming & working with unix/linux command line utilities, I tailored the examples to answer questions about his next birthday, such as:

How many days to next birthday & how many total days since Birth Sep 20/2006 ?


 #0. uvcopy datedemo1    <-- initiate interactive date conversion
     ================      - shows Today's dates before prompting for user input
            calendar  julian   days1900  -priordays  # day mth dd yyyy
 Today--->  20200702  2020184   44013       44013    4 Thu Jul 02 2020
 ================================================================================
 c=calendar, j=julian, n=days-since-1900, +/- days, q=quit <-- COMMANDs c,j,n,+,-
 c20200101,  j2020001,     n43830,        +180, -90        <-- EXAMPLEs

 #1. --> c20200920    <-- input 'c' + Calendar date for 14th Birthday Sep 20/2020
         =========      - see below '80' days from Today to 14th birthdaay

 c20200920  20200920  2020264   44093          80    0 Sun Sep 20 2020
 ================================================================================
 c=calendar, j=julian, n=days-since-1900, +/- days, q=quit <-- COMMANDs c,j,n,+,-
 c20200101,  j2020001,     n43830,        +180, -90        <-- EXAMPLEs

 #2. --> c20060920    <-- input 'c' + Calendar date for Birth on Sep 20/2006
         =========      - see below '5,114' total days old on 14th birthday

 c20060920  20060920  2006263   38979        5114-   3 Wed Sep 20 2006
 ================================================================================
 c=calendar, j=julian, n=days-since-1900, +/- days, q=quit <-- COMMANDs c,j,n,+,-
 c20200101,  j2020001,     n43830,        +180, -90        <-- EXAMPLEs

 #3. --> +5114       <-- might test to see if conversions agree (c,+,-,n,j)
         =====         - +5114 gets us back to 2020 14th birthday from Birth 2006

 +5114      20200920  2020264   44093        5114    0 Sun Sep 20 2020
 ================================================================================
 c=calendar, j=julian, n=days-since-1900, +/- days, q=quit <-- COMMANDs c,j,n,+,-
 c20200101,  j2020001,     n43830,        +180, -90        <-- EXAMPLEs

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

3B3. datedemo1 - convert between calendar, julian,& days-since-1900

Questions you can answer with datedemo1

  1. How many days old are you ?

  2. On which day of the week were you born ?

  3. How many days to your next birthday ?

  4. What other questions can you think of ?

See the uvcopy job stored at $UV/pf/demo/datedemo1. The all-important instruction is 'dat' documented at uvsoftware.ca/uvcopy.htm#dat.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_4. uvcopy demos & tutorials

UVdemos1 - Part 4 - Contents


4A1. uvcopy Demos & Tutorials

4A2. Provincefix1 - sample uvcopy job to convert .csv file to fixed-fields
 Input $UV/demo/dat0/Provinces.csv, Output $UV/demo/dat1/Provinces1
- sample/demo explained before we ask you to write a similar job

4A3. Execute 'uvcopy Provincefix1' & to convert .csv file to fixed fields.
- showing Input & Output files

4A4. Directories/Files relevant to uvcopy tutorials

4A5. Notes Re pf,pfxx,pfyy directories for uvcopy jobs
Create your jobs in pfxx & see UV Softwares solutions in pfyy

4B1. 'copy1' - uvcopy skeleton framework for your instructions
- Copy, Rename,& edit to Insert your instructions

4B2. pfxx/provincefix1 - copied/renamed/modified from copy1
- Notes re documenting uvcopy jobs (& scripts)

4B3. Execute your uvcopy job & check Input/Output files

4B4. Adding sequence#s & reformatting as desired

4B5. AFTER Adding seq#s, reformatting,& editing population
- accumulate Total population & print at end of report

4C1. Create Table Summary of Provinces & Populations by Party

4C2. Execute job to summarize provinces & populations
Input dat9/provinces2, output tmp/provinces2.tbl

4D1. Create job to summarize US Congress members by Party
Execute job to summarize provinces & populations
Input dat1/UScongress, Output tmp/UScongress.tbl

4D2. Documenting your new job

4D3. Table Summarize US Congress Members by Party & Sex
- concatenating 2 fields for the table argument

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4A1. uvcopy demos & tutorials

sample job to convert .csv file to fixed-fields

The previous pages have demonstrated several 'pre-programmed' uvcopy jobs (uvlotto1, testint1, hexcalc1, datedemo1), and now we will illustrate how you can write your own uvcopy jobs for whatever data file manipulations you might need.

'uvcopy' has over 100 instructions documented at uvsoftware.ca/uvcopy3.htm. In addition to the basic instructions (move,add,etc), there are many calls that perform hundreds of C instructions to perform commonly required data manipulations, such as delimiting/undeliminating, searching/replacing, table analysis, etc.

For our 1st example, we will illustrate the 'fix' instruction which separates '.csv' files to separate fixed-length fields depending on the longest field. Then you can reformat to a desired layout, assigning field lengths appropriate to the contents. Here are the 1st 3 records of the test data file used to illustrate the 'fix' instruction.

Input - $UV/demo/dat0/Provinces.csv

            ABR,Province,Capital,Premier,Government,Population
            AB,Alberta,Edmonton,Jason Kenney,UCP,4429000
            BC,British Columbia,Victoria,John Horgan,NDP,5121000

Output - $UV/demo/dat1/Provinces1

         10        20        30        40        50        60        70        80        90       100       110
 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 ABR                 Province            Capital             Premier             Government          Population
 AB                  Alberta             Edmonton            Jason Kenney        UCP                 4429000
 BC                  British Columbia    Victoria            John Horgan         NDP                 5121000

      fix    b0(20),a0(200),6,','     <-- fix (separate fields) as shown above
   ================================     - 4 operands described below
op1
  • defines 1st output field, in area 'b' & 20 bytes long
op2
  • defines input record, in area 'a', 200 bytes allowed
op3
  • number of fields
op4
  • separator, could be multi chars (example '","')

Since uvcopy was designed to replace IBM mainframe utilities, the operand usage is similar to that of the original IBM 360 assembler langauge - data movement is always from the 2nd operand to the 1st operand, which makes sense because some instructions have only 1 operand & op2 could be a long constant.

See complete description of 'fix' at uvsoftware.ca/uvcopy3.htm#fix

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4A2. uvcopy tutorials for creating uvcopy jobs

$UV/demo/pf/Provincefix1 demonstrates that only 1 uvcopy instruction can do complex data manipulations that could take many lines of code in other languages.

Note that the uvcopy job illustrates the fix Input/Output conversions as #comments at the begining of the coding & highlights the 'fix' instruction with '==='s

Convert .csv to fixed fields for Cdn Provinces

 # Provincefix1 - convert Canadian Provinces file from comma delimited to fixed fields
 #              - uvcopy job stored in $UV/pf/demo/... & in $UV/demo/pf/...
 #              - by Owen Townsend, UV Software, July 2020
 #
 # uvcopy Provincefix1,fili1=dat0/Provinces.csv,filo1=dat1/Provinces1
 # ==================================================================
 # uvcopy Provincefix1      <-- same as above, see defaults on fili1/filo1 below
 # ===================
 #
 #                ** sample 1st 3 records Input & Output **
 #
 # ABR,Province,Capital,Premier,Government,Population
 # AB,Alberta,Edmonton,Jason Kenney,UCP,4429000                                     <-- Input area 'a'
 # BC,British Columbia,Victoria,John Horgan,NDP,5121000
 #
 #          1         2         3         4         5         6         7         8
 # 12345678901234567890123456789012345678901234567890123456789012345678901234567890 <-- Output area 'b'
 # ABR                 Province            Capital             Premier             Government          Population
 # AB                  Alberta             Edmonton            Jason Kenney        UCP                 4429000
 # BC                  British Columbia    Victoria            John Horgan         NDP                 5121000
 #
 opr='$jobname - convert Cdn Provinces file from comma delimited to fixed fields'
 rop=r1  # option to view output at EOF (vi,cat,more,etc)
 fili1=?dat0/Provinces.csv,typ=LST,rcs=256
 filo1=?dat1/Provinces1,typ=LST,rcs=256
 @run
         opn    all
 # begin loop to get into area 'a', fix fields 20 apart in area 'b', & output
 loop    get    fili1,a0                 get next record
         skp>   eof                      (cc set > at EOF)
 #==================================
         fix    b0(20),a0(256),6,','     separate fields 20 apart in area 'b'
 #==================================
         put    filo1,b0                 write output from area 'b'
         skp    loop                     return to get next record
 eof     cls    all                      close files
         eoj                             end job
Note
  • all the work is performed by only 1 instruction, the rest is #comments
    & uvcopy framework of file Input/Output.
  • We will provide you with a framework job $UV/pf/demo/copy1 that you
    can copy/rename/modify, so you only need to code 1 or a few instructions
    and declare your files vs demo files.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4A3. uvcopy tutorials for creating uvcopy jobs

Execute Provincefix1


 #0a. ssh uvsoftxx@uvsoftware.ca  <-- login if not already logged in
      ==========================      (online userids uvsoft00-uvsoft99)
      --> passxx                  <-- enter your password at the prompt

 #0b. Login userxx          <-- Alternative Login on your own machine
      ============              if you have downloaded & installed Vancouver Utilities

 #0c. cdd  ------>              /home/uvsoftxx/demo (alias cdd='cd $RUNDATA')
      ===  --OR-->              /home/userxx/demo

 #1. cat dat0/Provinces.csv  <-- show Input file (.csv)
     ======================
      ABR,Province,Capital,Premier,Government,Population
      AB,Alberta,Edmonton,Jason Kenney,UCP,4429000
      BC,British Columbia,Victoria,John Horgan,NDP,5121000
      MB,Manitoba,Winnipeg,Brian Pallister,PC,1380000
      NB,New Brunswick,Fredericton,Blaine Higgs,PC,781000
      NL,Newfoundland,St. John's,Dwight Ball,Liberal,521000
      NS,Nova Scotia,Halifax,Stephen McNeil,Liberal,979000
      ON,Ontario,Doug Ford,Toronto,PC,14746000
      PE,PrinceEdwardIsland,Charlottetown,Dennis King,PC,159000
      QC,Quebec,Quebec City,Francois Legault,CAQ,8553000
      SK,Saskatchewan,Regina,Scott Moe,SaskParty,1182000
      NT,NW Territories,Yellowknife,Caroline Cochrane,Consensus,45000
      NU,Nunavut,Joe Savikataaq,Iqualuit,Consensus,40000
      YK,Yukon,Sandy Silver,Yellowknife,Liberal,42000

 #2. uvcopy Provincefix1  <-- execute uvcopy demo job to convert .csv to fixed fields
     ===================

 #3. cat dat1/Provinces1  <-- show Output file (Fixed-Fields)
     ===================
         10        20        30        40        50        60        70        80        90       100       110
 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 ABR                 Province            Capital             Premier             Government          Population
 AB                  Alberta             Edmonton            Jason Kenney        UCP                 4429000
 BC                  British Columbia    Victoria            John Horgan         NDP                 5121000
 MB                  Manitoba            Winnipeg            Brian Pallister     PC                  1380000
 NB                  New Brunswick       Fredericton         Blaine Higgs        PC                  781000
 NL                  Newfoundland        St. John's          Dwight Ball         Liberal             521000
 NS                  Nova Scotia         Halifax             Stephen McNeil      Liberal             979000
 ON                  Ontario             Toronto             Doug Ford           PC                  14746000
 PE                  PrinceEdwardIsland  Charlottetown       Dennis King         PC                  159000
 QC                  Quebec              Quebec City         Francois Legault    CAQ                 8553000
 SK                  Saskatchewan        Regina              Scott Moe           SaskParty           1182000
 NT                  NW Territories      Yellowknife         Caroline Cochrane   Consensus           45000
 NU                  Nunavut             Iqualuit            Joe Savikataaq      Consensus           40000
 YK                  Yukon               Yellowknife         Sandy Silver        Liberal             42000

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4A4. uvcopy tutorials for creating uvcopy jobs

Directories/Files relevant to uvcopy tutorials

 /home/uvadm/demo
 |-- dat0/                <---- .csv files to be converted to fixed layouts
 |   |-- ...                  - several files not shown (not relevant to UVdemos1)
 |   |-- Provinces.csv      <-- used to demo converting to fixed-field format
 |   |-- UScongress.csv       - used to demo table summary by party, sex, etc
 |   |-- USstates.csv         - used to demo accumulating state pop & calc % total pop
 |   `
 |-- dat1/                <---- fixed-field layout (converted from dat0/...csv files)
 |   |-- ...                  - several files not shown (not relevant uvcopy tutorials)
 |   |-- Provinces1         <-- converted from Provinces.csv, all fields 20 bytes apart
 |   |-- Provinces2           - outputs of UVSW jobs in pf/... vs USER jobs in pfxx...
 |   |-- UScongress           - used to demo table summary by party, sex, etc
 |   |-- insults            <-- data file for insult2
 |   `--
 |-- dat9/                <---- subdir/files created by YOUR exercises (vs our demos)
 |   |-- provinces2         <-- created by pfxx/provincefix2 (yours vs our Provinces2)
 |   |--                      - or by pfyy/provincefix2 (UV Software's solution)
 |   |-- provinces2a        <-- created by pfxx/provincefix2a (or pfyy/provincefix2a)
 |   |--                      - same as provinces2 with Total population
 |   |-- provinces2.tbl     <-- created by pfxx/provincefixtbl2 (or pfyy/provincetbl2)
 |   `--
 |-- pf/                  <---- uvcopy demo jobs (written by UV Software)
 |   |-- Provincefix1       <-- convert Provinces.csv to Provinces1 fixed 20 apart
 |   |-- Provincefix2       <-- convert Provinces.csv to Provinces2 reformatted
 |   `--
 |-- pfxx/                <---- user written uvcopy job tutorials
 |   |-- copy1                - skeleton (framework) for creating new uvcopy jobs
 |   |-- .........          <-- user to create uvcopy jobs by copy/rename/modify copy1
 |   |-- .........              (provincefix1,provincefix2,provincefix2a,provincetbl2)
 |   |-- .........            - may compare to pfyy/uvcopy jobs (by UV Software)
 |   |--
 |-- pfyy/                <---- answers to user tutorials
 |   |-- provincefix1         - compare your solutions to UV Software solutions
 |   |-- provincefix2         - minimal version pf/Provincefix2, created from copy1
 |   |-- provincefix2a        - same with total population line added
 |   |-- provincetbl2         - table summary of province counts & populations by Gov Party
 |   |-- UScongresstbl1       - table summary of US Congress Members by Party
 |   `-- UScongresstbl2       - table summary of US Congress Members by Party & Sex
 |-- sf/                  <---- Script Files for UVdemos2 vs this UVdemos1
 |   |-- ...                  - UV Software demo scripts
 |-- sfxx/                <---- user written scripts
 |   |-- ...                  - see tutorials ahead (might create with uvfix1)
 |-- sfyy/                <---- answers to user tutorials
 |   |-- rename_dat3_neat1    - create scripts with column alignment
 |   |-- rename_dat3          - enhance existing scripts with column alignment
 |   |-- rename_dat3_neat2    - scripts created by $UV/pf/util/uvfix1
 |-- tmp/                 <---- uvfix jobs preliminary output to tmp/...
 |   |--                      - for inspection before copy back to desired directory

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4A5. uvcopy tutorials for creating uvcopy jobs

Notes Re pf,pfxx,pfyy directories for uvcopy jobs

pf
  • is the traditional directory for uvcopy jobs (in $PFPATH)
  • contains UV Software demo jobs Provincefix1, Provincefix2, etc
pfxx
  • intended for user written jobs for tutorials (not in $PFPATH)
  • user to write provincefix1,provincefix2,provincefix2a,provincetbl2,etc
  • starting from the 'copy1' skeleton framework uvcopy job (for file I/O)
  • inserting your instructions where indicated
pfyy
  • UV Software's solutions for user comparisons (not in $PFPATH)
  • for comparison to user written jobs (provincefix1,provincefix2,etc)
  • can execute these 1st & then write your own version
  • try to write your version in pfxx/... before you look at pfyy/...

pfxx & pfyy are NOT in $PFPATH, you need to specify on the uvcopy comamnd, for example:


 uvcopy pfxx/provincefix1   <-- need to specify directory for jobs in pfxx/ & pfyy/
 ========================

As opposed to uvcopy jobs whose directories are in $PATH, foe example:


 uvcopy Provincefix1        <-- do not need to specify directory is in $PFPATH
 ===================

PFPATH is defined in $APPSADM/env/comon_profile_uv, for example:


 export PFPATH=$RUNLIBS/pf,$HOME/pf,$UV/pf/adm,$UV/pf/demo,$UV/pf/util,$UV/pf/IBM
 ================================================================================

Notes Re sf,sfxx,sfyy directories Script Files

sf
  • Script Files (demo scripts written by UV Software)
sfxx
  • user written scripts - see tutorials ahead (might create with uvfix1)
sfyy
  • answers to user tutorials, for comparison to your sripts
  • but try to write your version in sfxx/... before you look at sfyy/...

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B1. uvcopy tutorials for creating uvcopy jobs

'copy1' - uvcopy framework for your instructions

 # copy1 - uvcopy skeleton job - to be copied,renamed,& modified as required
 #       - this job is for 'text' files (variable length records ended by 'LF's)
 #       - see 'copyf1' fixed length seqntl files, 'copyI1' for Indexed files
 # 1. cp $UV/pf/demo/copy1 $HOME/pf/myfilefix - copy/rename to your homedir/pf/yourname
 # 2. vi pf/myfilefix                   - modify for your purposes
 # 3. uvcopy pf/myfilefix               - execute modified job
 rop=r1    #<-- EOF option to view output file (reply vi,cat,more,etc)
 fili1=?tf/test100,typ=LST,rcs=512    # '?' prompts for change, code your input file
 filo1=?tmp/$fili1,typ=LSTt,rcs=512   # this would write same filename in tmp/..
 @run
         opn     all                      open files
 # begin loop to get & put records until EOF
 getrec  get     fili1,a0                 get record into area 'a'
         skp>    EOF                      (cc set > at EOF)
         mvc     b0(512),a0               move input area 'a' to output area 'b'
 #---------------------------------
 #       ---     -----,-----          <-- add your instructions here **
 #---------------------------------
 putrec  put     filo1,b0                 write to output file from area 'b'
         skp     getrec                   return to get next record
 #
 EOF     cls     all                      close files
         eoj                              end job

Copy/Rename/Insert the 'fix' instruction


 #0a. ssh userxx@uvsoftware.ca     <-- login to your account
      ========================
  --> passxx   <-- enter password

 #0b. cdd  --> /home/userxx/demo   (alias cdd='cd $RUNDATA')
      ===

 #1. cp pf/copy1 pfxx/provincefix1
     =============================
      - copy/renaming to 'pfxx/provincefix1' or whatever makes sense to you
      - UV Software's version was called 'pf/Provincefix1'

 #2. vi pfxx/provincefix1     <-- edit your version to insert 'fix' instruction
     ====================         (replacing the existing 'mvc' instruction)
      - see results listed below:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B2. uvcopy Tutorials & Exercises

pfxx/provincefix1 - copied/renamed/modified from copy1

 # provincefix1 - convert Provinces.csv to fixed-field layout
 #              - created by copying/renaming/modifying copy1 to provincefix1
 #              - replacing 'mvc' with 'fix' instruction
 #              - by userxx, XX company, July 04/2020
 #
 # uvcopy provincefix1,fili1=dat0/Provinces.csv,filo1=dat9/provinces1
 # ==================================================================
 # uvcopy provincefix1  <-- same as above, I/O files default as shown
 # ===================
 #
 #                 ** Sample Record Input & Output **
 #
 # AB,Alberta,Edmonton,Jason Kenney,UCP,4429000
 #         10        20        30        40        50        60        70        80        90       100       110
 # 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 # AB                  Alberta             Edmonton            Jason Kenney        UCP                 4429000
 #
 rop=r1    #<-- option for EOF prompt to view output file (reply vi,cat,more,etc)
 fili1=?dat0/Provinces.csv,typ=LST,rcs=512   # Input filename changed
 filo1=?dat9/provinces1,typ=LSTt,rcs=512     # Output filename changed
 @run
        opn    all                       open files
 # begin loop to get & put records until EOF
 getrec get    fili1,a0                  get record into area 'a'
        skp>   EOF                       (cc set > at EOF)
 #----------------------------------
        fix    b0(20),a0(200),6,','  <-- fix 6 fields 20 apart in 'b' from 'a' by ','s
 #----------------------------------
 putrec put    filo1,b0                  write to output file from area 'b'
        skp    getrec                    return to get next record
 EOF    cls    all                       close files
        eoj                              end job

Notes re documenting uvcopy jobs (& scripts)

  1. Add #comments as shown above - 1st line should specify jobname & what it does. Document author name, date written or modified, company name if applicable

  2. Show the uvcopy command to run the job & default I/O filenames

  3. Nice to show 1 or a few sample records Input/Output

  4. Code the fili1/filo1 definitions with '?' prefix to prompt for possible change. Reply null to accept default or enter new filenames if desired. Prompts will be inhibited if job called from comamnd line or script with filenames coded.

  5. Add #comments as required for other programmers to understand your job

  6. Comments on separate #lines or coded on right side of instructions, but must be separated by at least 2 spaces from the last operand.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B3. uvcopy Tutorials & Exercises

Execute your uvcopy job & check Input/Output files


 #3. uvcopy pfxx/provincefix1   <-- execute your new uvcopy job
     ========================

 #4. cat dat0/Provinces.csv
     ======================
      ABR,Province,Capital,Premier,Government,Population
      AB,Alberta,Edmonton,Jason Kenney,UCP,4429000
      BC,British Columbia,Victoria,John Horgan,NDP,5121000
      MB,Manitoba,Winnipeg,Brian Pallister,PC,1380000
      NB,New Brunswick,Fredericton,Blaine Higgs,PC,781000
      NL,Newfoundland,St. John's,Dwight Ball,Liberal,521000
      NS,Nova Scotia,Halifax,Stephen McNeil,Liberal,979000
      ON,Ontario,Toronto,Doug Ford,PC,14746000
      PE,PrinceEdwardIsland,Charlottetown,Dennis King,PC,159000
      QC,Quebec,Quebec City,Francois Legault,CAQ,8553000
      SK,Saskatchewan,Regina,Scott Moe,SaskParty,1182000
      NT,NW Territories,Yellowknife,Caroline Cochrane,Consensus,45000
      NU,Nunavut,Iqualuit,Joe Savikataaq,Consensus,40000
      YK,Yukon,Yellowknife,Sandy Silver,Liberal,42000

 #5. cat dat9/provinces1
     ===================
         10        20        30        40        50        60        70        80        90       100       110       120
 123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 ABR                 Province            Capital             Premier             Government          Population
 AB                  Alberta             Edmonton            Jason Kenney        UCP                 4429000
 BC                  British Columbia    Victoria            John Horgan         NDP                 5121000
 MB                  Manitoba            Winnipeg            Brian Pallister     PC                  1380000
 NB                  New Brunswick       Fredericton         Blaine Higgs        PC                  781000
 NL                  Newfoundland        St. John's          Dwight Ball         Liberal             521000
 NS                  Nova Scotia         Halifax             Stephen McNeil      Liberal             979000
 ON                  Ontario             Toronto             Doug Ford           PC                  14746000
 PE                  PrinceEdwardIsland  Charlottetown       Dennis King         PC                  159000
 QC                  Quebec              Quebec City         Francois Legault    CAQ                 8553000
 SK                  Saskatchewan        Regina              Scott Moe           SaskParty           1182000
 NT                  NW Territories      Yellowknife         Caroline Cochrane   Consensus           45000
 NU                  Nunavut             Iqualuit            Joe Savikataaq      Consensus           40000
 YK                  Yukon               Yellowknife         Sandy Silver        Liberal             42000

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B4. Improving provincefix1 by copy/modify to provincefix2

Adding sequence#s & reformatting as desired

Let's assume you want to add sequence#s, reformat the spacing,& edit the population field as shown below by 1st 3 records (before & after):

 ABR                 Province            Capital             Premier             Government          Population
 AB                  Alberta             Edmonton            Jason Kenney        UCP                 4429000
 BC                  British Columbia    Victoria            John Horgan         NDP                 5121000
         10        20        30        40        50        60        70        80        90       100       110       120
 123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 00  ABR   Province            Capital             Premier             Government
 01  AB    Alberta             Edmonton            Jason Kenney        UCP                  4,429,000
 02  BC    British Columbia    Victoria            John Horgan         NDP                  5,121,000

 #6. cp pfxx/provincefix1 pfxx/provincefix2  <-- copy/rename existing job
     ======================================

 #7. vi pfxx/provincefix2     <-- edit as shown by the 'AFTER' job shown below
     ====================         vs your 'BEFORE' job shown above.
 # provincefix2 - improved version of 'provincefix1'
 #              - created by copying/renaming/modifying provincefix1 to provincefix2
 #              - adding code to sequence number
 #
 rop=r1    #<-- option for EOF prompt to view output file (reply vi,cat,more,etc)
 fili1=?dat0/Provinces.csv,typ=LST,rcs=512   # Input filename changed
 filo1=?dat9/provinces2,typ=LSTt,rcs=512     # Output filename changed
 @run
        opn    all                       open files
 # begin loop to get & put records until EOF
 getrec get    fili1,a0                  get record into area 'a'
        skp>   eof                       (cc set > at EOF)
 #------------------------------------------------------------------------
 # fix fields 20 bytes apart from area 'a' to area 'b', then reformat to area 'c'
        fix    b0(20),a0(200),6,','      fix fields 20 bytes apart
 #------------------------------------------------------------------------
 #Note - 5 sequence# instructions added to provincefix2 (vs provincefix1)
        mvn    c0(2),$ca1                sequence# in cols 01-02
        add    $ca1,1                    increment sequence#
        mvc    c4(3),b0                  province abrev
        mvc    c10(80),b20               shift province,city,premier left by 10
        edt    c90(10),b100(10),'zz,zzz,zzz'  edit population
 #------------------------------------------------------------------------
        put    filo1,c0                  write to output file from area 'c'
        skp    getrec                    return to get next record
 eof    cls    all                       close files
        eoj                              end job

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4B5. Improving provincefix1 by copy/modify to provincefix2


 #8. uvcopy pfxx/provincefix2    <-- Execute new enhanced job
     ========================      - see output below:

AFTER Adding seq#s, reformatting,& editing population


 #9. cat dat9/provinces2    <-- display new output file
     ===================      - compare to previous version above
 00  ABR   Province            Capital             Premier             Government
 01  AB    Alberta             Edmonton            Jason Kenney        UCP                  4,429,000
 02  BC    British Columbia    Victoria            John Horgan         NDP                  5,121,000
 03  MB    Manitoba            Winnipeg            Brian Pallister     PC                   1,380,000
 04  NB    New Brunswick       Fredericton         Blaine Higgs        PC                     781,000
 05  NL    Newfoundland        St. John's          Dwight Ball         Liberal                521,000
 06  NS    Nova Scotia         Halifax             Stephen McNeil      Liberal                979,000
 07  ON    Ontario             Toronto             Doug Ford           PC                  14,746,000
 08  PE    PrinceEdwardIsland  Charlottetown       Dennis King         PC                     159,000
 09  QC    Quebec              Quebec City         Francois Legault    CAQ                  8,553,000
 10  SK    Saskatchewan        Regina              Scott Moe           SaskParty            1,182,000
 11  NT    NW Territories      Yellowknife         Caroline Cochrane   Consensus               45,000
 12  NU    Nunavut             Iqualuit            Joe Savikataaq      Consensus               40,000
 13  YK    Yukon               Yellowknife         Sandy Silver        Liberal                 42,000
 Total Population of Canada --->                                                           37,978,000

Adding Total population

In fact, the above (dat9/provinces2a vs dat9/provinces2) shows the extra total line.

Can you add more instructions to accumulate the total population & print at EOF ?


 #10. cp pfxx/provincefix2 pfxx/provincefix2a  <-- copy/rename for the new version
      ===================================

 #11. vi pfxx/provincefix2a   <-- modify
      ===================

 #12. uvcopy pfxx/province2a  <-- execute new versions
      ====================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4C1. uvcopy Table Summary Instructions 'tbl' & 'tbp'

Create job to summarize Provinces & Populations by Party

Let's create another job to demo the uvcopy 'tbl' & 'tbp' instructions. We can copy/rename/modify the copy1 skeleton job similar to prior creation of provincefix1


 #1. cp pfxx/copy1 pfxx/provincetbl2
     ===============================
      - copy/renaming to 'provincefix1' or whatever makes sense to you
      - existing version was called 'Provincefix1'

 #2. vi pfxx/provincetbl2     <-- edit to insert 'tbl' & 'tbp' instructions
     ====================       - see results listed below:
 # provincetbl2 - table summarize province counts & populations by political party
 #              - created by copying/renaming/modifying copy1 to province2tbl
 #              - inserting 'tbl' & 'tbp' instructions
 #              - by userxx, XX company, July 04/2020
 #
 rop=r1    #<-- option for EOF prompt to view output file (reply vi,cat,more,etc)
 fili1=?dat9/provinces2,typ=LST,rcs=512        # Input filename changed
 filo1=?dat9/provinces2.tbl,typ=LSTt,rcs=512   # Output filename changed
 @run
        opn     all                       open files
 # begin loop to get & put records until EOF
 getrec get     fili1,a0                  get record into area 'a'
        skp>    EOF                       (cc set > at EOF)
 #------------------------------------------------------------------------
        tblt1f1 a70(10),'Gov Party; population',a90(10)
 #------------------------------------------------------------------------
        skp     getrec                    return to get next record
 EOF    tbpt1   filo1,'province counts & populations by Government Party'
        cls     all                       close files
        eoj                               end job

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4C2. uvcopy Table Summary Instructions tbl & tbp

Execute job to summarize provinces & populations


 #3. uvcopy pfxx/provincetbl2   <-- Execute
     ========================

 #4. cat dat9/provinces2        <-- display input file
     ===================

Input File - dat9/provinces2

         10        20        30        40        50        60        70        80        90       100       110
 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 00  ABR   Province            Capital             Premier             Government
 01  AB    Alberta             Edmonton            Jason Kenney        UCP                  4,429,000
 02  BC    British Columbia    Victoria            John Horgan         NDP                  5,121,000
 03  MB    Manitoba            Winnipeg            Brian Pallister     PC                   1,380,000
 04  NB    New Brunswick       Fredericton         Blaine Higgs        PC                     781,000
 05  NL    Newfoundland        St. John's          Dwight Ball         Liberal                521,000
 06  NS    Nova Scotia         Halifax             Stephen McNeil      Liberal                979,000
 07  ON    Ontario             Toronto             Doug Ford           PC                  14,746,000
 08  PE    PrinceEdwardIsland  Charlottetown       Dennis King         PC                     159,000
 09  QC    Quebec              Quebec City         Francois Legault    CAQ                  8,553,000
 10  SK    Saskatchewan        Regina              Scott Moe           SaskParty            1,182,000
 11  NT    NW Territories      Yellowknife         Caroline Cochrane   Consensus               45,000
 12  NU    Nunavut             Iqualuit            Joe Savikataaq      Consensus               40,000
 13  YK    Yukon               Yellowknife         Sandy Silver        Liberal                 42,000

 #5. cat dat9/provinces2.tbl   <-- display table summary report
     =======================
 provincetbl2  2020/07/05_17:41:36_086  province counts & populations by Government Party
 tbl#0001  tblt1f1 a70(10)      <----- argument ----->                -acum#1-    %
 line#   1strec#  %      count  Gov Party                           population
     1        10   7         1  CAQ                                  8,553,000   22
     2        12  14         2  Consensus                               85,000    0
     3         1   7         1  Government                                   0    0
     4         6  21         3  Liberal                              1,542,000    4
     5         3   7         1  NDP                                  5,121,000   13
     6         4  28         4  PC                                  17,066,000   44
     7        11   7         1  SaskParty                            1,182,000    3
     8         2   7         1  UCP                                  4,429,000   11
                 100        14*  *TOTAL*                            37,978,000 *100

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4D1. uvcopy Table Summary Instructions tbl & tbp

Create job to summarize US Congress members by Party

              ----------- 1st 3 records of 539 -----------
         10        20        30        40        50        60        70        80
 12345678901234567890123456789012345678901234567890123456789012345678901234567890
 001  Abraham             Ralph          19540916  M  LA  rep  Republican
 002  Adams               Alma           19460527  F  NC  rep  Democrat
 003  Aderholt            Robert         19650722  M  AL  rep  Republican

 #1. cp pfxx/copy1 pfxx/UScongresstbl1
     =================================
      - copy/rename template job 'copy1' to 'UScongresstbl1' (or whatever you like)
      - solution in pfyy is 'UScongresstbl1' but try making your version in ppfxx/...

 #2. vi pfxx/UScongresstbl1     <-- edit to insert 'tbl' & 'tbp' instructions
     ======================       - see results listed below:
 # UScongresstbl1 - table summarize US Congress members by party
 #                - created by copy/rename/modify 'copy1' uvcopy skeleton/framework job
 #                - by Owen Townsend, UV Software, Juky09/2020
 #
 rop=r1    #<-- option for EOF prompt to view output file (reply vi,cat,more,etc)
 fili1=?dat1/UScongress,typ=LST,rcs=512       # '?' prompts for change
 filo1=?tmp/UScongress.tbl,typ=LSTt,rcs=512   # change provinces to UScongress <--M1
 @run
         opn     all                       open files
 # begin loop to get & put records until EOF
 getrec  get     fili1,a0                  get record into area 'a'
         skp>    eof                       (cc set > at EOF)
 #-----------------------------
         tblt1f1 a62(15),'Members by Party'   #<-- insert tbl... instrn  <--------M2
 #-----------------------------
 ####### put     filo1,b0  <-- ##disable detail output, dump table at EOF <-------M3
         skp     getrec                    return to get next record
 #
 # EOF - dump table, close files,& end job
 eof     tbpt1   filo1,'Congress Members summarized by Party'             <-------M4
         cls     all                       close files
         eoj                               end job

Execute job to summarize UScongress members by Party


 #3. uvcopy pfxx/UScongresstbl1   <-- Execute
     ==========================

 #4. cat tmp/UScongress.tbl1      <-- display output file
     =======================
 UScongresstbl1  2020/07/09_15:42:12_143  US Congress Members summarized by Party
 tbl#0001  tblt1f1 a62(15)      <----- argument ----->
 line#   1strec#  %      count  Members by Party
     1         2  52       284  Democrat
     2       265   0         2  Independent
     3         1  46       253  Republican
                 100       539*  *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4D2. uvcopy job to summarize US Congress members by Party

Add documentation to your new job


 #5a. vi dat1/UScongress       <-- read input file into the editor
      ==================
  --> "a3yy     <-- yank 1st 3 lines into buffer "a"

 #5b. vi tmp/UScongress.tbl1   <-- read output file into the editor
      ======================
  --> "b7yy     <-- yank all lines (7) into buffer "b"

 #5c. vi pfxx/UScongresstbl1   <-- edit your job
      ======================
  --> add front end lines with jobname,purpose,author,date (as shown below)
  --> "ap       <--Put buffer "a" in desired location & insert '#' column 1
  --> "bp       <--Put buffer "b" in desired location & insert '#' column 1
 # UScongresstbl1 - table summarize US Congress members by party
 #                - created by copy/rename/modify 'copy1' uvcopy skeleton/framework job
 #                - by Owen Townsend, UV Software, July09/2020
 #
 #              ----------- 1st 3 records of 539 -----------
 #         10        20        30        40        50        60        70        80
 # 12345678901234567890123456789012345678901234567890123456789012345678901234567890
 # 001  Abraham             Ralph          19540916  M  LA  rep  Republican
 # 002  Adams               Alma           19460527  F  NC  rep  Democrat
 # 003  Aderholt            Robert         19650722  M  AL  rep  Republican
 #
 #            ---------- summary table dumped at EOF ----------
 # UScongresstbl1  2020/07/09_15:42:12_143  US Congress Members summarized by Party
 # tbl#0001  tblt1f1 a62(15)      <----- argument ----->
 # line#   1strec#  %      count  Members by Party
 #     1         2  52       284  Democrat
 #     2       265   0         2  Independent
 #     3         1  46       253  Republican
 #                 100       539*  *TOTAL*
 #
 rop=r1    #<-- option for EOF prompt to view output file (reply vi,cat,more,etc)
 fili1=?dat1/UScongress,typ=LST,rcs=512       # '?' prompts for change
 filo1=?tmp/UScongress.tbl,typ=LSTt,rcs=512   # change provinces to UScongress <--M1
 @run
         opn     all                       open files
 # begin loop to get & put records until EOF
 getrec  get     fili1,a0                  get record into area 'a'
         skp>    eof                       (cc set > at EOF)
 #-----------------------------
         tblt1f1 a62(15),'Members by Party'   #<-- insert tbl... instrn  <--------M2
 #-----------------------------
 ####### put     filo1,b0  <-- ##disable detail output, dump table at EOF <-------M3
         skp     getrec                    return to get next record
 #
 # EOF - dump table, close files,& end job
 eof     tbpt1   filo1,'Congress Members summarized by Party'             <-------M4
         cls     all                       close files
         eoj                               end job

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

4D3. Table Summaries by Multiple Arguments

Summarize US Congress Members by Party & Sex


 #1. cp pfxx/UScongresstbl1 pfxx/UScongresstbl2
     ==========================================
      - copy/rename your prior job to a new jobname

 #2. vi pfxx/UScongresstbl2   <-- add instruction to concatenate Sex onto Party
     ======================     - change argument heading & desriptions appropriately

 #3. uvcopy UScongresstbl2    <-- Execute
     =====================      - see output below (inserted into job as documentation)
 # UScongresstbl2 - table summarize US Congress members by party & sex
 #                - created by copy/rename/modify 'copy1' uvcopy skeleton/framework job
 #                - by Owen Townsend, UV Software, Juky09/2020
 #                - see uvsoftware.ca/uvdemos1.htm#___
 #
 #              ----------- 1st 3 records of 539 -----------
 #         10        20        30        40        50        60        70        80
 # 12345678901234567890123456789012345678901234567890123456789012345678901234567890
 # 001  Abraham             Ralph          19540916  M  LA  rep  Republican
 # 002  Adams               Alma           19460527  F  NC  rep  Democrat
 # 003  Aderholt            Robert         19650722  M  AL  rep  Republican
 #
 #            ---------- summary table dumped at EOF ----------
 # UScongresstbl2  2020/07/09_16:05:41_381  US Congress Members by Party & Sex
 # tbl#0001  tblt1f1 a62(15)      <----- argument ----->
 # line#   1strec#  %      count  Members by Party & Sex
 #     1         2  20       108  Democrat     F
 #     2         4  32       176  Democrat     M
 #     3       265   0         2  Independent  M
 #     4         9   4        23  Republican   F
 #     5         1  42       230  Republican   M
 #                 100       539*  *TOTAL*
 #
 rop=r1    #<-- option for EOF prompt to view output file (reply vi,cat,more,etc)
 fili1=?dat1/UScongress,typ=LST,rcs=512        # '?' prompts for change
 filo1=?tmp/UScongress.tbl2,typ=LSTt,rcs=512   # change provinces to UScongress <--M1
 @run
         opn     all                       open files
 # begin loop to get & put records until EOF
 getrec  get     fili1,a0                  get record into area 'a'
         skp>    eof                       (cc set > at EOF)
 #-----------------------------
         mvc     a75(1),a50                   #<-- move sex after party  <--------M2a
         tblt1f1 a62(15),'Members by Party'   #<-- insert tbl... instrn  <--------M2b
 #-----------------------------
 ####### put     filo1,b0  <-- ##disable detail output, dump table at EOF <-------M3
         skp     getrec                    return to get next record
 #
 # EOF - dump table, close files,& end job
 eof     tbpt1   filo1,'Congress Members summarized by Party'             <-------M4
         cls     all                       close files
         eoj                               end job

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_5. uvcopy Tutorials & Exercises

UVdemos1 Part 5 - Contents


5A1. uvfix1 - EASY way to execute uvcopy instructions
- interactive utility that solicits your instructions & inserts them
  into a complete uvcopy job including the file I/O logic
 Tips & Differences from uvcopy direct use, which requires editing or copying
 the file I/O framework,as well as your instructions).

5B1. uvfix1 first demo - convert .csv to fixed-field 20 bytes apart
- province abr, province-name, capital, premier, government-party
- same as the previous pfxx/provincefix1 job which was created by editing
  instructions into a supplied framework job ($UV/pf/demo/copy1).

5B2. uvfix1 options available (help screen)
- options b1/b2 to drop blank lines or reduce multi blank lines to 1
- option 'p1' to translate unprintable characters to '.' periods
- options m & n to sequence# Input &/or Output records
- option 'e#' to select Every nth record ('e5' select every 5th record)
- option 's#' to Stop copy after 's#' records

5C1. demo uvfix1 OPTIONS to sequence# & drop blank lines
5C2. option 'n4' 4 digit sequence#, 'b1' reduce multi blanks lines to 1
5C3. option 'n6' (seq# 6 digits) & 'b2' (drop All blank lines)

5D1. uvfix1 demo - convert dat0/USstates.csv to fixed-fields 20 bytes apart
- state abrev, state-name, state-population, captial, capital-population
5D2.  option 's#' Stop copy after # records
 option 'e5' select Every 5th record (or as desired)

5E1. Run uvfix1 with No Instructions to automatically:
- remove trailing blanks (mainframe files blank filled to col 80)
- remove any CR's x'0D' from mainframes or Windows files

5F1. 'uvfixA' script to process All files in a directory copying to a 2nd directory
5F2. - also demos option 'p1' to translate unprintable characters to '.'s

5G1. uvfix1 to SELECT records
- demo selects records with 'New' from dat1/USstates1

5G2. uvfix1 to REPLACE patterns with alternates
- demo selects records with 'New' from dat1/USstates1

5G3. uvfix1 translate to lowercase, to UPPERcase, EBCDIC to ASCII, ASCII to EBCDIC

5G4. UVFIX1 left adjust & squeeze multi-spaces to 1 between words

5H1. uvfixA - process All files in the parms/* directory
- clear columns 73-80 (seq#s from mainframe, N/A on unix)

5H2. Insert $date8 & $filenames in All files in parms/...
- uvcopy provides access to 85 $variables (stored at uvcopy startup)
- use option v1 on mvfv1, msgv1, lodv1, sysv1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

UVdemos1 Part 5 - Contents (continued)


5H3. External ${VARIABLE} values provided by uvcopy option v2.
- accessed when you execute mvfv2, msgv2, lodv2, sysv2
- access both internal & external with 'v3' (mvfv3,msgv3,lodv3, sysv3)

5I1. uvfix1 demo - create 'neat' column-aligned scripts
- testfiles provided in demo/dat3
- sf/rename_neat1 - copy/rename script 'TO BE CREATED'
- 1st Try to create column aligned script with 'vi'

5I2. Attempt to create column aligned script with 'vi'

5I3. Creating column aligned scripts with 'uvfix1'

5I4. step by step procedure using 'uvfix1'

5I5. Explanations of Instructions to create sf/rename_neat1

5J1. using 'uvfix1' to column align an EXISTING script
sf/rename_dat3 - script BEFORE column-alignment

5J2. sf/rename_dat3_neat2 - script AFTER column-alignment

5J3. Explanations of Instructions to create sf/rename_dat3_neat2

5K1. Instruction entry error messages & prompt to re-enter
- illustrate by re-running '5B1' fix dat1/Provinces.csv with errors
- Instruction Error msgs - op1 area code missing, op2 length missing, etc

5L1. Complete job (with file I/O framework) saved in tmp/filename.pf
- Can save for later re-runs without having to re-enter instructions
- Can easily modify as desired, inserting additional instructions
- Can use many uvcopy features not available in interactive modes
  (add more files, accumulate & print total lines, etc)

uvfix1 - EASY way to execute uvcopy instructions

Previously (on page '4B1'), we showed you how to create new uvcopy jobs by copying/renaming/modifing the 'copy1' skeleton job (framework to handle FILE I/O), and then you only had to write the instructions to do whatever you wanted to do. To convert a .csv file to fixed-fields we only had to write ONE 'fix' instruction.

Now, we will show you how to eliminate the need to insert your desired instructions into the 'copy1' framework job - by running interactive job 'uvfix1' that solicits your instructions & then executes the job when you enter a '.' period.

uvfix1 also saves the complete job in tmp/filename.pf, which you could save to a permanent pf directory for susbequent reruns (or edit & rerun).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5A1. uvfix1 - EASY way to execute uvcopy instructions

uvfix Tips & Differences from uvcopy direct use

  1. You do not need to code the uvcopy framework instructions. The 'get', test EOF, 'mvc' inarea 'a' to outarea 'b', & 'put' are automatic

  2. You would code 'skp??' to test the results of many instructions such as: cmc, cmn, scn, rep, lok, tst, tsb (& many more, see uvcopy3.htm).

  3. Condition codes on 'skp??'s can be any of '=', '!', '<', '>', '<=', '>='

  4. Normally, you would not code the 'get' instruction since it is autoamtic, but you could code 'skp?? getrec if you wanted to bypass the automatic 'put' for the current record & get the next record.

  5. You can code 'put' instructions if desired to output multiple records on 1 input record cycle. You might then use special 'skp?? getrec' to inhibit the automatic 'put' for the current record.

  6. You cannot define labels (optional in cols 1-8 of uvcopy instructions) So, you can code the instruction operations starting in column 1

  7. The only goto labels allowed are 'getrec', 'putrec',& 'EOF'. - 'skp?? getrec' would inhibit output for the current record. - 'skp?? putrec' would inhibit further instructions for current record. - 'skp?? EOF' would inhibit processing any remaining records in the file.

  8. For most jobs you will address only area 'a' for Input records & area 'b' for Output records, but areas 'c', 'd', 'e' are reserved for your use. You could manipulate data in c,d,e,& move results to the output area 'b'. You could save data in c,d,e & address later when you reach desired data.

  9. No point in coding comments since the entries are not permanent - as for complete uvcopy jobs built with the editor & saved in a pf/directory.

  10. The output file is automatically wrritten to tmp/inputfilename in your current working directory & given the same name as the input file. You could however specify any dir/name on filo1=outdir?/yourname?

  11. Also note that the complete uvcopy job (framework I/O + your instructions) is saved in tmp/inputfilename.pf - so you could copy the complete job to a permanent pf/... directory & re-run it as is, or edit it & rerun it.

  12. Best to remove all files from the tmp/... subdir in your current working directory (rm -f tmp/*) before running uvfix1 jobs that are unrelated.

  13. For uvfixA (process all files in a directory vs uvfix1 for 1 file), you the output directory is automatically assumed to be tmp2/... in your current working directory. You could specify diffrently on fild2=... but the complete uvcopy job is always written to tmp/indirfilename.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5B1. uvcopy Tutorials & Exercises

uvfix1 - EASY way to execute uvcopy instructions

First, we will demo uvfix1 to do the same thing as we did with the complete uvcopy job previously illustrated on pages '4B1' & '4B2' which was constructed by editing your desired instructions into a supplied job ($UV/pf/demo/copy1)

Now, we will show you how to eliminate the need to insert your desired instructions into the 'copy1' framework job - by running interactive job 'uvfix1' that solicits your instructions & then executes the job when you enter a '.' period.

uvfix1 demo#1 - convert .csv to fixed-field


 #1. head -5 dat0/Provinces.csv   <-- show 1st 5 input records
     ==========================
      ABR,Province,Capital,Premier,Government,Population
      AB,Alberta,Edmonton,Jason Kenney,UCP,4429000
      BC,British Columbia,Victoria,John Horgan,NDP,5121000
      MB,Manitoba,Winnipeg,Brian Pallister,PC,1380000
      NB,New Brunswick,Fredericton,Blaine Higgs,PC,781000

 #2. uvcopy uvfix1,fili1=dat1/Provinces.csv
     ======================================
      - execute uvfix1 to prompt for instruction keyins
Note
  • We are OMITTING the options help screen & various other file info msgs
  • Please see the complete 'uvfix1 OPTIONS' shown on the next page ------------->
  • just enter nulls to accept the defaults, until you see "enter instruction"
  • enter instruction (or period "." end entries & execute)
      ========================
 #2a. fix b0(20),a0(100),6,','   <-- enter 'fix' instrn (convert csv to fixed-field)
      ========================
 #2b. .                          <-- enter '.' period to execute
      ====
        20200707:551000:uvfix1: EOF fili01 rds=14 size=720: dat1/Provinces.csv
        20200707:551000:uvfix1: EOF filo01 wrts=14 size=1506: tmp/Provinces
        EOJ, Output File written to: tmp/Provinces

 #3.  head -5 tmp/Provices       <-- inspect output & if OK, copy to wherever you want it
     =====================         - in case of a fixup job, you may want to overwrite the original
                                   - or you may want to rename the output file
      ABR                 Province            Capital             Premier             Government          Population
      AB                  Alberta             Edmonton            Jason Kenney        UCP                 4429000
      BC                  British Columbia    Victoria            John Horgan         NDP                 5121000
      MB                  Manitoba            Winnipeg            Brian Pallister     PC                  1380000
      NB                  New Brunswick       Fredericton         Blaine Higgs        PC                  781000

 #4. cp tmp/Provinces dat1/provinces1  <-- copy/rename the tmp/output to a permanent directory
     ================================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5B2. uvfix1 - EASY way to execute uvcopy instructions

uvfix1 options


 uvcopy uvfix1,fili1=dat1/Provinces.csv,uop=n4 <-- Re-Run with option n4 sequence#
 =============================================   - could enter options at prompt below
 uvfix1 - copy a text file & modify via uvcopy instruction keyins
        - enter your input-file on command line (else defaults to demo file)
        - output-file will be written ti tmp/... with same name as input-file
 uop=q1b0c0h0m0n0p0r4096s99 - user option defaults
       b1              - reduce multiple consecutive blank records to 1
       b2              - drop all blank records (based on 1st100 bytes)
         c1            - pass thru any "#" col1 #comment records
         c2            - drop any # col1 records
           e#          - select Every e# input record, Example: e10 Every 10th
             h1        - display uvfix help screen with more (quits for rerun)
               m#      - show INput seq# in  # bytes of output record m0 inhibits
                       - m# unusual, could use m# & n# with e# (Every e# records)
                 n#    - show OUTput seq# in # bytes of output record n0 inhibits
                       - output records counted in $ca12 & compared to option n
                   p1  - translate any unprintable charactersto "." periods
              r4096    - recsize: max for text-records, exactfor Fixed-Length-records
                       - for Fixed-Length, reply RSF/RST at prompt (default text typ=LSTt)
                       - for Text files: accept default recsize & default typ=LSTt
           s99         - Stop count (s99 special meaning **ALL** records)
 User OPtion (uop) defaults  = q1b0c0h0m0n0p0r4096s99
 -->null to accept or enter/override -->    <-------- enter null accept default options #1 *
 LSTt = typ default filo01 - null accept or enter new typ ? <-- null accept default typ #2 *
 Enter Instructions: inarea "a" copied to outarea "b"
 Example: seq# in 1st 5 cols, shifting data 6 bytes right
   add  $ca1,10      <-- increment ctr#1 ($ca1) by 10
   clr  b0(100)      <-- clear output record area "b"
   mvn  b0(5),$ca1   <-- mvn - MoVe Numeric seq# to 1st 5 bytes
   mvc  b6(100),a0   <-- mvc - MoVe Char a to b offset 6 right
                     <-- option "uop=n5" does same as above 4 instructions
                     <-- enter instruction (or "." period to end entries & Execute)
 fix b0(20),a0(100),6,','                   <-------- enter insrtructions (only 1 here) #3 *
                     <-- enter "." period to end entries & Execute
 .                                          <----------------- "." entered to Execute   #4 *
 20200820:171400:uvfix1: EOF fili01 rds=14 size=720: dat1/Provinces.csv
 20200820:171400:uvfix1: EOF filo01 wrts=14 size=1506: tmp/Provinces.csv
 20200820:171400:uvfix1: EOF filo02 wrts=33 size=1737: tmp/Provinces.csv.pf
 EOJ, Output File written to: tmp/Provinces.csv
 ----- enter: vi,cat,more,etc (default null) --> more <--- enter "more" to show output  #5 *
 AB                  Alberta             Edmonton            Jason Kenney UCP 4429000
 BC                  British Columbia    Victoria            John Horgan NDP 5121000
       --- showing only 1st 2 records of demo file above ---
Note
  • Entry #1 null acceepts default options, #2 accepts default output file type
  • Entry #3 is our 1 instruction (fix) to convert csv delimited to fixed fields
  • Entry #4 "." to execute the instructions while copying input to output
  • Entry #5 "more" displays the output file (in tmp/... with same name as infile)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5C1. uvfix1 - EASY way to execute uvcopy instructions

uvfix1 OPTIONS demo - sequence# & drop blank lines

You can do useful work with NO options & NO instructions to copy 1 or ALL files in a directory, automatically removing trailing blanks & Carriage Returns (x'0D's) that can cause problems on unix systems (CR's can cause scripts to fail).

This is useful to clean up files FTP'd from mainframes which are usually blank filled to 80 bytes appended with a LineFeed & a Carriage-Return (to be dropped). We will demo that process on page '5E1' ahead, but 1st we will demo the options.

Please see all the options on the previous page (b1/b2,c1,d1,h1,n99,p1,r4096,s#). Here we will demo options b1/b2 (drop blank lines) & n4 sequence# (in 1st 4 columns), using $UV/demo/dat3/uvfix.promo.


 #1. rm -f tmp*/*           <-- clear All tmp files
     ============             - tmp*/* same as tmp/* tmp1/* tmp2/*

 #2. cat dat3/uvfix.promo    <-- inspect input file to demo uvfix1 options
     ====================
 0001                    ** uvfix1 & uvfixA - Most Usefull Tools **
 0002
 0003 If you are an IT professional, working with programs, scripts,& documentation, etc
 0004 We think you will find uvfix1 & uvfixA the most useful tools in this collection.
 0005
 0006 Remember times when unix tools alone did not do the selections or changes you needed ?
 0007
 0008
 0009                    ** Advantages of uvfix1 & uvfixA **
 0010
 0011 Interactive utility to fix problems in text files with 1 or a few instructions.
 0012 Over 100 powerful instructions (search,replace,concatenate,delimit,translate,etc).
 0013
 0014 Can apply multiple conditions for selections & mass changes.
 0015
 0016 Copies the file or all files to a tmp/... dir for inspection before copy back.
Note
  • The actual input file (dat3/uvfix.promo) does not have the sequence numbers
  • we show them here, so you can compare to the output file on the page below
  • we sequenced number with uvfix1 & option n4 to insert 4 digit sequence#s
  • see all options listed on the previous page
  • here are just the options used for this demo to drop blank lines & sequence#
 uop=q1b0c0h0m0n0p0r4096s99 - user option defaults
 --->  b1              - reduce multiple consecutive blank records to 1        <---
 --->  b2              - drop all blank records (based on 1st100 bytes)        <---
         c1            - pass thru any "#" col1 #comment records
         c2            - drop any # col1 records
           e#          - select Every e# input record, Example: e10 Every 10th
             h1        - display uvfix help screen with more (quits for rerun)
               m#      - show INput seq# in 1st # bytes of Input record
 --->            n#    - show OUTput seq# in 1st # bytes of Output record      <---
                   p1  - translate any unprintable charactersto "." periods
              r4096    - recsize: max for text-records, exact for Fixed-Length
           s99         - Stop count (s99 special meaning **ALL** records)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5C2. uvfix1 - EASY way to execute uvcopy instructions

uvfix1 OPTIONS demo - sequence# & drop blank lines


 #3. uvcopy uvfix1,fili1=dat3/uvfix.promo,uop=n4b1  <-- note options 'n4' & 'b1'
     =============================================
      - option 'n4' sequence#s in 1st 4 bytes
      - option 'b1' reduces multiple blank lines to 1 blank line
      --- OR, for simpler syntax run the 'script' uvfix1 that calls 'uvcopy uvfix1'

 #3a. uvfix1 dat3/uvfix.promo n4b1  <-- run script 'uvfix1' (calls 'uvcopy uvfix1')
      ============================
      Note - We have OMITTED the options help screen & various other prompts
           - Please see the 'uvfix1 OPTIONS' listed on page '5B2'
           - just enter nulls to accept the defaults, until you see "enter instruction"
           - enter instruction (or period "." end entries & execute)

 #4. cat tmp/uvfix.promo     <-- display the output file
     ===================       - note effects of options n4 & b1
 0001                    ** uvfix1 & uvfixA - Most Usefull Tools **
 0002
 0003 If you are an IT professional, working with programs, scripts,& documentation, etc
 0004 We think you will find uvfix1 & uvfixA the most useful tools in this collection.
 0005
 0006 Remember times when unix tools alone did not do the selections or changes you needed ?
 0007
 0008                    ** Advantages of uvfix1 & uvfixA **
 0009
 0010 Interactive utility to fix problems in text files with 1 or a few instructions.
 0011 Over 100 powerful instructions (search,replace,concatenate,delimit,translate,etc).
 0012
 0013 Can apply multiple conditions for selections & mass changes.
 0014
 0015 Copies the file or all files to a tmp/... dir for inspection before copy back.
Note
  • Compare the output file (15 lines) to the input file (16 lines) on the previous page
  • 1 blank line deleted by option 'b1' which reduces multi-blank lines to 1
    (input file had 2 blank lines after line# 0006)
Note
  • We coded the options "uop=b1n4" on the command line, to make it easier to repeat
    the command by searching Linux command line history with up arrow (or alt p).
  • But we could have wait for the options prompt (see options screen on page '5B2')

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5C3. uvfix1 OPTIONS demo - sequence# & drop blank lines

options "n6" (seq# 6 digits) & "b2" (drop All blank lines)


 #1. uvcopy uvfix1,fili1=dat3/uvfix.promo,uop=n6b2
     =============================================
      - OR, use script 'uvfix1' below (easier syntax than 'uvcopy uvfix1')

 #1a. uvfix1 dat3/uvfix1.promo n4b2  <-- option n6 seq# 6 digits, b2 drop all blank lines
      =============================
Note
  • we will omit the options screen (shown on page '5B2')
  • and just show the output file as follows:

 #2. cat tmp/uvfix.promo     <-- display the output file
     ===================       - note effects of options n6 & b2
 000001                    ** uvfix1 & uvfixA - Most Usefull Tools **
 000002 If you are an IT professional, working with programs, scripts,& documentation, etc
 000003 We think you will find uvfix1 & uvfixA the most useful tools in this collection.
 000004 Remember times when unix tools alone did not do the selections or changes you needed ?
 000005                    ** Advantages of uvfix1 & uvfixA **
 000006 Interactive utility to fix problems in text files with 1 or a few instructions.
 000007 Over 100 powerful instructions (search,replace,concatenate,delimit,translate,etc).
 000008 Can apply multiple conditions for selections & mass changes.
 000009 Copies the file or all files to a tmp/... dir for inspection before copy back.

Comoare this output file (options n6b2) to prior page options n4b1

Note
  • Above we were showing both the 'uvcopy' & 'script' versions of uvfix1
  • From now on we will show just the 'script' version (easier syntax)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D1. uvfix1 - EASY way to execute uvcopy instructions

convert dat0/USstates.csv to Fixed-Fields


 #1. cdd  ------>  /home/uvsoftxx/demo (alias cdd='cd $RUNDATA')
     ===  --OR-->  /home/userxx/demo

 #2. rm -f tmp*/*             <-- clear All tmp files
     ============               - tmp*/* same as tmp/* tmp1/* tmp2/*

 #3. head dat0/USstates.csv   <-- show 1st 10 records of Input file
     ======================
      abr,state,state-pop,capital,capital-pop
      AL,Alabama,4908620,Montgomery,205764,
      AK,Alaska,734002,Juneau,31275,
      AZ,Arizona,7378490,Phoenix,1445632,
      AR,Arkansas,3039000,Little Rock,193524,
      CA,California,39937500,Sacramento,466488,
      CO,Colorado,5845530,Denver,600158,
      CT,Connecticut,3563080,Hartford,124775,
      DE,Delaware,982895,Dover,36047,
      FL,Florida,21993000,Tallahassee,181376,

 #4. uvfix1 dat0/USstates.csv   <-- run uvfix1 script (to solicit instructions)
     ========================     - OMITTING options screen & filename prompts
                                  - until the '.' prompt for instructions
                                  - see uvfix1 OPTIONS listed on page '5B2'
     #4a.  ========================
           fix b0(20),a0(100),5,','   <-- enter "fix" instruction
           ========================
     #4b.    .                        <-- enter '.' period to execute
           ====
      20200902:241806:uvfix1: EOF fili01 rds=51 size=1916: dat0/USstates.csv
      20200902:241806:uvfix1: EOF filo01 wrts=51 size=4423: tmp/USstates.csv
      20200902:241806:uvfix1: EOF filo02 wrts=26 size=1432: tmp/USstates.csv.pf
      uvcopy uvfix1 start 2020/09/02_16:24:01 end 16:24:18 elapsed000_16_634
      EOJ, Output File written to: tmp/USstates.csv

 #5. cp tmp/USstates.csv dat1/USstates1  <-- save tmp/output to a permanent directory
     ==================================    - renaming (replacing '.csv' with '1'

 #6. more dat1/USstates1     <-- show all 50 states on the next page -------->
     ===================

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D2. convert dat0/USstates.csv to Fixed-Fields dat1/USstates1

                 ** dat1/USstates1 - with state & capital populations *8
 abr                 state               state-pop           capital             capital-pop
 AL                  Alabama             4908620             Montgomery          205764
 AK                  Alaska              734002              Juneau              31275
 AZ                  Arizona             7378490             Phoenix             1445632
 AR                  Arkansas            3039000             Little Rock         193524
 CA                  California          39937500            Sacramento          466488
 CO                  Colorado            5845530             Denver              600158
 CT                  Connecticut         3563080             Hartford            124775
 DE                  Delaware            982895              Dover               36047
 FL                  Florida             21993000            Tallahassee         181376
 GA                  Georgia             10736100            Atlanta             420003
 HI                  Hawaii              1412690             Honolulu            337256
 ID                  Idaho               1826160             Boise               205671
 IL                  Illinois            12659700            Springfield         116250
 IN                  Indiana             6745350             Indianapolis        820445
 IA                  Iowa                3179850             Des Moines          203433
 KS                  Kansas              2910360             Topeka              127473
 KY                  Kentucky            4499690             Frankfort           25527
 LA                  Louisiana           4645180             Baton Rouge         229493
 ME                  Maine               1345790             Augusta             19136
 MD                  Maryland            6083120             Annapolis           38394
 MA                  Massachusetts       6976600             Boston              617594
 MI                  Michigan            10045000            Lansing             114297
 MN                  Minnesota           5700670             Saint Paul          285068
 MS                  Mississippi         2989260             Jackson             173514
 MO                  Missouri            6169270             Jefferson City      43079
 MT                  Montana             1086760             Helana              28190
 NE                  Nebraska            1952570             Lincoln             258379
 NV                  Nevada              3139660             Carson City         55274
 NH                  New Hampshire       1371250             Concord             42695
 NJ                  New Jersey          8936570             Trenton             84913
 NM                  New Mexico          2096640             Santa Fe            67947
 NY                  New York            19440500            Albany              97856
 NC                  North Carolina      10611900            Raleigh             403892
 ND                  North Dakota        761723              Bismarck            61272
 OH                  Ohio                11747700            Columbus            787033
 OK                  Oklahoma            3954820             Oklahoma City       579999
 OR                  Oregon              4301090             Salem               154637
 PA                  Pennsylvania        12820900            Harrisburg          49528
 RI                  Rhode Island        1056160             Providence          178042
 SC                  South Carolina      5210100             Columbia            129272
 SD                  South Dakota        903027              Pierre              13646
 TN                  Tennessee           6897580             Nashville           601222
 TX                  Texas               29472300            Austin              790390
 UT                  Utah                3282120             Salt Lake City      186440
 VT                  Vermont             628061              Montpelier          7855
 VA                  Virginia            8626210             Richmond            204214
 WA                  Washington          7797100             Olympia             46478
 WV                  West Virginia       1778070             Charleston          51400
 WI                  Wisconsin           5851750             Madison             233209
 WY                  Wyoming             567025              Cheyenne            59466

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5D3. uvfix1 OPTIONS - Stop count & select Every 5th (or as desired)

We will demo selecting every 5th record using demo file dat1/USstates1. You can see all 50 records (+ header) in $UV/demo/dat1/USstates1, but here we will showw only the 1st 10 records using option 's10'. Easier to use 'head 10', but we wanted to demo the STOP COUNT option.


 #1. rm -f tmp*/*           <-- clear All tmp files
     ============             - tmp*/* same as tmp/* tmp1/* tmp2/*

 #2. uvfix1 dat1/USstates1 s10n3  <-- copy dat1/USstates1 to tmp/USstates1
     ===========================    - option s10 Stop at record #10
                                    - option n3 3 digit sequence Number

 #3. cat tmp/USstates1_s10n3  <-- display result (1st 10 records)
     =======================    - show sample records before demo option 'e5'
     001 abr                 state               state-pop           capital             capital-pop
     002 AL                  Alabama             4908620             Montgomery          205764
     003 AK                  Alaska              734002              Juneau              31275
     004 AZ                  Arizona             7378490             Phoenix             1445632
     005 AR                  Arkansas            3039000             Little Rock         193524
     006 CA                  California          39937500            Sacramento          466488
     007 CO                  Colorado            5845530             Denver              600158
     008 CT                  Connecticut         3563080             Hartford            124775
     009 DE                  Delaware            982895              Dover               36047
     010 FL                  Florida             21993000            Tallahassee         181376

select Every 5th record demo


 #4. uvfix1 dat1/USstates1 e5m3n3  <-- copy dat1/USstates1 to tmp/USstates1
     ============================
      - option 'e5' select every 5th record (of 50)
      - option 'm3' sequence Number Input records & 'n3' seq# Output records

 #5. cat tmp/USstates1_e5m3n3  <-- display result (Every 5th record)
     ========================
     005 001 AR                  Arkansas            3039000             Little Rock         193524
     010 002 FL                  Florida             21993000            Tallahassee         181376
     015 003 IN                  Indiana             6745350             Indianapolis        820445
     020 004 ME                  Maine               1345790             Augusta             19136
     025 005 MS                  Mississippi         2989260             Jackson             173514
     030 006 NH                  New Hampshire       1371250             Concord             42695
     035 007 ND                  North Dakota        761723              Bismarck            61272
     040 008 RI                  Rhode Island        1056160             Providence          178042
     045 009 UT                  Utah                3282120             Salt Lake City      186440
     050 010 WI                  Wisconsin           5851750             Madison             233209
Note
  • As well as option 'e5' (select Every 5th record), we coded options
    'm3' (seq# input records) & 'n3' (seq# output records).
  • 'm3' allows you to see the original seq# in the input file.
Note
  • uvfix1 automatically appends the options on the output filenames
    so we can run multiple uvfix1 jobs & not overwrite previous output files
  • Be sure to remove all tmp/* files before starting a series of related jobs

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5E1. uvfix1 - EASY way to execute uvcopy instructions

Drop CR's & Trailing Blanks

Carriage Returns x'0D' can cause problems on unix systems (scripts can fail) & the trailing blanks are annoying. An easy way to remove the CR's & trailing blanks is to use 'uvfix1' for 1 file at a time or 'uvfixA' for all files in a directory.

Files from outside sources (Windows or mainframe systems) often have CR's & trailing spaces, which you cannot see, but are annoying when you edit the files. Files from m ainframes are usually space filled to 80 bytes +CR +LF = 82 bytes. You can't see them with vi,cat,etc so use uvhd:

For example, we will use $UV/tf/product.master1 (8 records of 82 bytes each). Here is the 1st record displayed by uvhd. Note the trailing blanks & the ending Carriage Return x'0D' + Line Feed x'0A':


 uvhd $UV/dat3/product.master1 t   <-- use option 't' for Text files (end record on LF x'0A')
 ===============================
 rec#=1 rsize=82 fptr=0 fsize=656 records=8
                 10        20        30        40        50        60        70        80
   r# 1 0123456789012345678901234567890123456789012345678901234567890123456789012345678901
      0 BBQ010 BAR-B-Q         0019500                                                  ..
        4453332445242522222222233333332222222222222222222222222222222222222222222222222200
        2210100212D2D1000000000001950000000000000000000000000000000000000000000000000000DA

 #1. rm -f tmp*/*                      <-- clear tmp files
     ============                        - clears tmp/* + tmp1/* + tmp2/*

 #2. cp $UV/tf/product.master1 tmp1/   <-- copy demo file to your $HOME/tmp1/...
     ===============================

 #3. uvfix1 dat3/product.master1       <-- run uvfix1 to remove trailing blanks & CR's
     ===========================
               - omitting the option & filename prompts, until the prompt for instructions
      --> .  <-- No instructions required, just enter the '.' period
               - to signal end of instructions & proceed to processing
               - copies the file to tmp/product.master1 (removing trailing blanks & CRs)

 #4. uvhd tmp/product.master1 t     <-- investigate the output file
     ==========================
 rec#=1 rsize=31 fptr=0 fsize=248 records=8
                 10        20        30
   r# 1 0123456789012345678901234567890
      0 BBQ010 BAR-B-Q         0019500.   <-- confirming trailing blanks removed
        4453332445242522222222233333330     - and CR's x'0D' removed
        2210100212D2D10000000000019500A     - terminated by LF's only x'0A's

 #5. cp tmp/product.master1 dat3/   <-- could copy back overwriting original file
     ============================     - but dont do this in $UV (would destroy $UV testfile)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5F1. Easy way to shorten to last non-blank & drop CR's

uvfixA - process All files in directory

Use 'uvfixA' to process ALL files in the directory while copying to a 2nd directory. uvfixA does not output to tmp like uvfix1, You must specify the I/O directories. For example, remove CR's & trailing spaces from all files in demo/dat3/* as follows:


 #1. rm -f tmp*/*             <-- clear tmp files
     ============               - clears tmp/* + tmp1/* + tmp2/*

 #2. uvfixA dat3 tmp2 LST p1  <-- process ALL files in dat3/* while copying to tmp2/...
     =======================    - arg3 'LST' is the file type (Line Seqntl Terminated)
                                - arg4 'p1' option converts unprintable chars to "."s
 uvfixA - copy/modify all text files from 1 directory to a 2nd directory
        - modifying the files via uvcopy instruction keyins
 uop=q1c0n99r4096t0 - user option defaults
       c1           - pass thru any "#" col1 #comment records
         n99        - output record limit (n99 special meaning **ALL** records)
            p1      - translate any unprintable characters to "." periods'
              r4096 - recsize ONLY for Fixed-Length OUTPUT records if desired
                    - for Text files: accept default recsize & default typ=LSTt
                 t1 - translate output file-names to lower-case, t2 for UPPER-case
 User OPtion (uop) defaults  = q1c0n99r4096t0
 -->null to accept or enter/override --------> p1 <--- convert unprintables to "."s
 did you create outdir ? (or remove files) --> y  <--- Yes, we cleared outdir tmp2/*
 Example: sequence# in 1st 5 cols, shifting record 6 bytes right
  ... example omitted, not relevant, since we need no Instructions for this job
                    <-- enter instruction (or period "." end entries & execute)
 .                  <-- enter "." to Execute
 20200821:374704:uvfixA: EOF filo02 wrts=12 size=805: tmp2/about.dat3.files
 20200821:374704:uvfixA: EOF filo02 wrts=20 size=1400: tmp2/gl.account.trans
 20200821:374704:uvfixA: EOF filo02 wrts=8 size=248: tmp2/product.master1
 20200821:374704:uvfixA: EOF filo02 wrts=16 size=678: tmp2/uvfix.promo
 20200821:374704:uvfixA: EOF filo02 wrts=9 size=576: tmp2/warranty.master
   ----- showing only output file stats, omitting several other lines

Notes re uvfixA

  1. We are using the script $UV/sf/util/uvfixA vs uvcopy ($UV/pf/util/uvfixA) since the command line is shorter for uvfixA, here are both for comparison:


     uvcopy uvfixA,fild1=dat3,fild2=tmp2,arg1=LST,uop=p1  <-- uvcopy command LONGER
     ===================================================

     uvfixA dat3 tmp2 LST p1  <-- script version SHORTER & easier to enter
     =======================
  1. We specified the 'p1' option on both the comamnd line & the prompt (REDUNDANT). We recommend specifying all args on teh comamnd line, so you can easly repeat the command from the Linux history (if required).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5F2. Easy way to shorten to last non-blank & drop CR's

Option 'p1' convert unprintables to '.'s

  1. Option "p1" will convert any unprintable characters to '.'s. The warranty.master file has packed fields. You can use uvhd to see the difference BEFORE & AFTER:


      uvhd dat3/warranty.master t   <-- see packed field chars in dat3/ BEFORE conversion
      ===========================
      rec#=1 rsize=64 fptr=0 fsize=576 records=9
                      10        20        30        40        50        60
      r#   1 0123456789012345678901234567890123456789012345678901234567890123
           0 12345 12345 sony television   950101 1-year .`.. 00000291q..P...
             3333323333327667276667676662223333332327667206112333333337095110
             1234501234503FE9045C56939FE00095010101D95120900C00000029111900CA
                                                         ^ ^^          ^^ ^^
      Note - packed/binary characters marked with '^'

      uvhd tmp2/warranty.master t   <-- see packed field chars substituted in tmp2 OUTPUT
      ===========================
      rec#=1 rsize=64 fptr=0 fsize=576 records=9
                      10        20        30        40        50        60
      r#   1 0123456789012345678901234567890123456789012345678901234567890123
           0 12345 12345 sony television   950101 1-year .`.. 00000291q..P...
             3333323333327667276667676662223333332327667226222333333337225220
             1234501234503FE9045C56939FE00095010101D95120E0EE0000002911EE0EEA
                                                         ^ ^^          ^^ ^^
      Note - packed/binary characters now REPLACED with REAL "."s
           - vs the "."s that uvhd substitutes for unprintable characters

Do not confuse the '.'s that uvhd substitutes for unprintable packed/binary characters in the input file, with the '.'s for real '.'s in the output file.

For example, byte 44 of input file is x'09' in horizontal hex (vs vertical hex above), But byte 44 of the output file is x'2E' which is a real '.' period in the ASCII set.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5G1. uvfix1 - demo Select records by scanning for patterns

SELECT records with 'New' from dat1/USstates1


 #1. rm -f tmp*/*            <-- clear All tmp files
     ============              - tmp*/* same as tmp/* tmp1/* tmp2/*

 #2. uvfix1 dat1/USstates2   <-- run uvfix1 script with input file dat1/USstates1
     =====================     - omitting option & filename prompts
                               - until the '.' prompt for instructions

     #2a. scn b0(100),'New'  <-- scan for 'New' anywhere in the current record
          ================

     #2b. skp! getrec        <-- skip on condition code '!' (not found)
          ===========            to 'getrec', which bypasses output of current record
                               - so output file will contain only 'New' founds

     #2c. .                  <-- enter '.' to signal end of instructions
          ===========            & Execute the get/put cycle until EOF

 #3. cat tmp/USstates2       <-- display the output file
     =================
     030 NH                  New Hampshire       1371250             Concord             42695
     031 NJ                  New Jersey          8936570             Trenton             84913
     032 NM                  New Mexico          2096640             Santa Fe            67947
     033 NY                  New York            19440500            Albany              97856

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5G2. uvfix1 - demo REPLACE patterns with alternates

Replace 'New' with 'Old' in dat1/USstates1


 #1. rm -f tmp/*            <-- clear tmp/* files
     ===========

 #2. uvfix1 dat1/USstates2         <-- run uvfix1 script with input file dat1/USstates1
     =====================           - omitting option & filename prompts
                                     - until the '.' prompt for instructions

     #2a. rep b0(100),'New','Old'  <-- scan for 'New' anywhere in the current record
          ======================       & replace with 'Old'

     #2d. skp! getrec              <-- skip on condition code '!' (not found)
          ===========                  to 'getrec', which bypasses output of current record
                                     - so output file will contain only New/Old replacements

#2e. edt


     #2e. .                 <-- enter '.' to signal end of instructions
          ===========           & Execute the get/put cycle until EOF

 #3. cat tmp/USstates2      <-- display the output file (with New/Old replacements)
     =================
     030 NH                  Old Hampshire       1371250             Concord             42695
     031 NJ                  Old Jersey          8936570             Trenton             84913
     032 NM                  Old Mexico          2096640             Santa Fe            67947
     033 NY                  Old York            19440500            Albany              97856
Note
  • Normally you would omit the 'skp! getrec' when searching/replacing
  • But we did not want to show all 50 output records

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5G3. uvfix1 - demo Edit & Translate

Edit populations & Translate UPPER/lower case

We will demo both editing & translate UPPER/lower in same job. The translations are impractical but we wanted to demo them without taking up a separate page.


 #1. rm -f tmp/*              <-- clear tmp/* files
     ===========

 #2. head -5 dat1/USstates1   <-- show 1st 5 records of Input file
     ======================
 #       10        20        30        40        50        60        70        80        90       100
 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
 abr                 state               state-pop           capital             capital-pop
 AL                  Alabama             4908620             Montgomery          205764
 AK                  Alaska              734002              Juneau              31275
 AZ                  Arizona             7378490             Phoenix             1445632
 AR                  Arkansas            3039000             Little Rock         193524

 #3. uvfix1 dat1/USstates1   <-- run uvfix1 script with input file dat1/USstates1
     =====================     - omitting options screen & filename prompts
                               - see uvfix1 OPTIONS listed on page '5B2'
                               - until the '.' prompt for instructions

     #3a. edt b40(12),a40(20),'zzz,zzz,zz9'  <-- edit state population
          =================================

     #3b. edt b80(12),a80(20),'zzz,zzz,zz9'  <-- edit capital population
          =================================

     #3c. tru b20(20)           <-- translate cols 21-40 to UPPER-case
          ===========

     #3d. trl b60(20)           <-- translate cols 61-80 to lower-case
          ===========

     #3e. .                     <-- enter '.' to signal end of instructions
          ===========              & Execute the get/put cycle until EOF

 #4. head -5 tmp/USstates1      <-- display 1st 10 records of output file
     =====================
     abr                 STATE                          0        capital                        0
     AL                  ALABAMA                4,908,620        montgomery               205,764
     AK                  ALASKA                   734,002        juneau                    31,275
     AZ                  ARIZONA                7,378,490        phoenix                1,445,632
     AR                  ARKANSAS               3,039,000        little rock              193,524

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5G4. uvfix1 - Left adjust & squeeze multi-spaces to 1 between words

Left adjust & squeeze spaces in dat1/USstates1

In order to demo the 'sqz' squeeze instruction, we will show you the hard way to create a .csv file - by inserting ',' commas bewteen fields & squeezing out spaces. This will be incorrect when fields have embedded spaces. The proper way to create csv files is with the 'var' instruction - see uvcopy3.htm#var.


 #1. rm -f tmp/*             <-- clear tmp/* files
     ===========

 #2. head -5 dat1/USstates1   <-- show 1st 5 records of INPUT file
     ======================
 #       10        20        30        40        50        60        70        80
 12345678901234567890123456789012345678901234567890123456789012345678901234567890
 abr                 state               state-pop           capital             capital-pop
 AL                  Alabama             4908620             Montgomery          205764
 AK                  Alaska              734002              Juneau              31275
 AZ                  Arizona             7378490             Phoenix             1445632
 AR                  Arkansas            3039000             Little Rock         193524

 #3. uvfix1 dat1/USstates1   <-- run uvfix1 with input file dat1/USstates1
     =====================     - omitting option & filename prompts
                               - see the OPTIONS listed on page '5B2'
                               - until the '.' prompt for instructions

     #3a. mvc b19(1),','     <-- insert ',' commas between fields
          ==============
     #3b. mvc b39(1),','
          ==============
     #3c. mvc b59(1),','
          ==============
     #3d. mvc b59(1),','
          ==============
     #3e. sqz b0(100),' '   <-- squeeze out all spaces
          ===============
     #3f. .                 <-- enter '.' to signal end of instructions
          ===========           & Execute the get/put cycle until EOF

 #4. head -5 tmp/USstates1      <-- display 1st 10 records of OUTPUT file
     =====================
      abr,state,state-pop,capital,capital-pop
      AL,Alabama,4908620,Montgomery,205764
      AK,Alaska,734002,Juneau,31275
      AZ,Arizona,7378490,Phoenix,1445632
      AR,Arkansas,3039000,LittleRock,193524

     #3e. sqzc1 b0(100),' '     <-- TRY repeating with option 'c1' on 'sqz'
          ================        - to leave 1 space between fields
                                  - see doc at uvcopy3.htm#sqz

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5H1. uvfixA - process All files in directory

uvfixA - process All files in the parms/* directory

This example is applicable to mainframe JCL conversions to unix/linux systems. The mainframe parameters (parms) for SORTs, SQL & FTP scripts are often in a separate directory & often have sequence numbers in columns 73-80 which are not required & we want to clear them to make them easier to edit. Clearing 73-80 will also clear back to the last non-blank since that is automatic when wrriting to text files (typ=LST).


 #1. rm -f tmp*/*      <-- clear tmp files
     ============        - clears tmp/* + tmp1/* + tmp2/*

 #2. cat parms/*       <-- inspect all 4 files in the parms/* directory
     ===========         - note sequence#s in 73-80 (to be cleared)
 # ftps/ftpdemo1
   open 192.168.0.5                                                      00000010
   user user02 user02pw                                                  00000020
   put ar.sales.items C:\AR\SALES.ITEMS                                  00000030
 * sort100
   SORT FIELDS=(78,2,CH,A,61,15,CH,A,11,25,CH,A)                         00000010
 * sort200
   SORT FIELDS=(31,6,CH,A,1,6,CH,A),                                     00000010
   OMIT COND=((11,2,CH,EQ,C'65'),OR,(COND=(11,2,CH,EQ,C'85'))            00000020
 /* select_customers.sql */
    SELECT * FROM CUSTOMERS;                                             00000010

 #3. uvfixA parms tmp2 LST  <-- process ALL files in parms/* copying to tmp2/...
     ====================     - omitting options screen & filename prompts
                              - until the '.' prompt for instructions

     #3a. clr b72(8),' '    <-- clear columns 73-80
          ==============
     #3b. .                 <-- enter '.' to execute
          ==============

 #4. vi tmp2/*              <-- inspect a few of the output files
     =========                - if not OK, repeat #2 with corrections
                              - if OK, proceed to copy to new parms/... dir
Note
  • You might wonder why we edited into the 1st 12 bytes of the field (vs all 20) ?
  • Because 'edt' right adjusts into the output field (vs input left adjusted from fix)
  • edt b40(20),a40(20),'zzz,zzz,zz9' <-- would leave no space between edt result & capital
Note
  • In real life, you would now copy tmp2/* back to parms/... as below #5a,b,c,d
  • but please omit this for these demos

 #5a. mv parms parms.old    <-- save parms as parms.old until new parms proven OK
 #5b. mkdir parms           <-- make new parms dir for uvfixA output
 #5c. cp tmp2/* parms       <-- copy uvfixA outputs back to the permanent parms/ dir
 #5d. rm -r parms.old       <-- remove the old parms when new versions proven OK

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5H2. uvfixA - Insert $VARIABLE data into all records

Insert $date8 & $filenames in All files in parms/...


 #1. rm -f tmp*/*      <-- clear tmp files
     ============        - clears tmp/* + tmp1/* + tmp2/*

 #2. cat parms/*       <-- inspect all 4 files in the parms/* directory
     ===========         - any sequence#s will be replaced by $date8

 #3. uvfixA parms tmp2 LST     <-- process ALL files in parms/* copying to tmp2/...
     =====================       - omitting options screen & filename prompts
                                 - until the '.' prompt for instructions

     #3a. mvfv1 b72(8),'$date8'      <-- insert yyyymmdd into cols 73-80
          =====================       - option 'v1' expands $date8 variable

     #3b. mvfv1 b81(19),'$fili1+6'   <-- insert current input filename in cols 82-100
          ========================     - '+6' to drop 'parms/' of parms/current-filename

     #3b. .                 <-- enter '.' to execute
          ==============

 #4. cat tmp2/*             <-- display all output files in tmp2/...
     ==========
 # ftps/ftpdemo1                                                         20200828 ftpdemo1
   open 192.168.0.5                                                      20200828 ftpdemo1
   user user02 user02pw                                                  20200828 ftpdemo1
   put ar.sales.items C:\AR\SALES.ITEMS                                  20200828 ftpdemo1
 * sort100                                                               20200828 pap100s1
   SORT FIELDS=(78,2,CH,A,61,15,CH,A,11,25,CH,A)                         20200828 pap100s1
 * sort200                                                               20200828 parm200s
   SORT FIELDS=(31,6,CH,A,1,6,CH,A),                                     20200828 parm200s
   OMIT COND=((11,2,CH,EQ,C'65'),OR,(COND=(11,2,CH,EQ,C'85'))            20200828 parm200s
 /* select_customers.sql */                                              20200828 selectcustomers
    SELECT * FROM CUSTOMERS;                                             20200828 selectcustomers
Note
  • uvcopy provides access to the values of many $variables
  • lower case variables are stored by uvcopy startup
  • UPPER case variables are usual for external environmental variables
  • see details on the next page

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5H3. $VARIABLE values provided by uvcopy

  1. See uvcopy2.htm#B7 for the many $variable values provided by uvcopy. Here are 20 more useful ones (of 85 available) & values on Aug28/2020

      uvcopyvars2 - show values of uvcopy $symbols  uvsoft5 uvadm 2020/08/28_11:41:55
      $symbol             definition          character value OR binary below
                                      binary* hexadecimal value   numeric value
      $date8              =y000(08)           20200828
      $datetime           =y020(019)          2020/08/28_11:41:55
      $date               =y020(10)           2020/08/28
      $time               =y031(8)            11:41:55
      $version            =y100(8)            20200825
      $license            =y110(19)           20200825_99V_930630
      $sitename           =y130(030)          UV_Software
      $regname            =y160(030)          Owen_Townsend
      $jobname            =y220(030)          uvcopyvars2
      $hostname           =y280(020)          uvsoft5
      $cwd                =y500(050)          /home/uvadm
      $logname            =y550(050)          uvadm
      $home               =y600(050)          /home/uvadm
      $hscale             =y2300(100)         1.......10........20........30 etc to 100
      $uscale             =y2600(200)         123456789012345678901234567890 etc to 200
      $fili1              =z00000(180)        /home/uvadm/src/uvcopy.c
      $filo1              =z24000(180)        tmp/uvcopy_symbols_values
  1. You can access the values using 'mvfv1' (mvf instruction with option 'v1') Here is an example from the demo above


      mvfv1 b72(8),'$date8'      <-- insert yyyymmdd into cols 73-80
      =====================        - option 'v1' expands $date8 variable
  1. You can access values of external environmental variables using option 'v3'. For example to get your login-name (& store it in cols 100+ of current record)


      mvfv3 b100(20),'${LOGNAME}'   <-- insert login-name in cols 101+ (120 max)
      ===========================     - option 'v3' expands external ${VARIABLE}s
  1. You need to surround external variables with braces '{' & '}'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5I1. uvfix1 - EASY way to execute uvcopy instructions

uvfix1 demo - create 'neat' column-aligned scripts

'uvfix1' can help you to create good-looking scripts with columns aligned. uvfix1 can do things that are hard to do with 'vi' (or 'vim'), such as copying data from 1 position in a line to a 2nd position at a fixed column.

Example
  • create a script to copy all files from 1 directory to a 2nd directory
  • BUT, we might need to do this for directories with HUNDREDs of files

testfiles provided in demo/dat3

      about.dat3.files
      gl.account.trans
      product.master1
      uvfix.promo
      warranty_master

script created by unix tools - messy, NOT "column-aligned"

cp dat3/about.dat3.files tmp2/about_dat3_files cp dat3/gl.account.trans tmp2/gl_account_trans cp dat3/product.master1 tmp2/product_master1 cp dat3/uvfix.promo tmp2/uvfix_promo cp dat3/warranty_master tmp2/warranty_master

desired script created by uvfix1 - Column-Aligned

 cp dat3/about.dat3.files                tmp2/about_dat3_files
 cp dat3/gl.account.trans                tmp2/gl_account_trans
 cp dat3/product.master1                 tmp2/product_master1
 cp dat3/uvfix.promo                     tmp2/uvfix_promo
 cp dat3/warranty_master                 tmp2/warranty_master

Notes Re sf,sfxx,sfyy directories Script Files

sf
  • Script Files (demo scripts written by UV Software)
sfxx
  • user written scripts - see tutorials ahead (might create with uvfix1)
sfyy
  • answers to user tutorials, for comparison to your sripts
  • but try to write your version in sfxx/... before you look at sfyy/...

FIRST "TRY" to create column aligned script with 'vi'


 #1. rm -f tmp/* tmp2/*          <-- clear tmp* subdirs in the demo/ directory
     ==================

 #2. ls dat3 >sfxx/rename_dat3   <-- capture filenames into the 'sfxx' Script Files subdir
     =========================      assigning an appropriate name

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5I2. uvfix1 demo - create 'neat' Column-Aligned scripts


 #3. vi sfxx/rename_dat3              <-- edit file list & try to convert into script
     ===================
      about.dat3.files                <-- filenames of testfiles provided
      gl.account.trans
      product.master1
      uvfix.promo
      warranty_master

 #3a. --> :%s/^/cp dat3\//              <-- vi Substitute command to insert 'cp/dat3/'
          =================                - '^' means at begining of line
                                           - '\' to  insert actual '/' (vs end pattern '/')
      cp dat3/about.dat3.files          <-- 'cp dat3/' inserted in front of filenames
      cp dat3/gl.account.trans
      cp dat3/product.master1
      cp dat3/uvfix.promo
      cp dat3/warranty_master

 #3b. --> :%s/$/               tmp2\//  <-- Substitute at '$' (End of Line)
          =============================   - 15 spaces preceding 'tmp2/'
                                          - attempt to separate 2nd column of filenames
      cp dat3/about.dat3.files                tmp2/... vi cannot copy within line ...
      cp dat3/gl.account.trans                tmp2/
      cp dat3/product.master1                tmp2/
      cp dat3/uvfix.promo                tmp2/
      cp dat3/warranty_master                tmp2/
Problem#1
  • inserting 15 spaces (attempt to space 2nd column away from filenames)
    might be nicer than inserting just 1 space, but not aligned as desired.
Problem#2
  • 'vi' has no way to copy the filename to a 2nd position in the line
problem#3
  • you can NOT substitute '.'s with '_'s for only PART of a line

DESIRED script - CAN be created with 'uvfix1' (see below)

      cp dat3/about.dat3.files                tmp2/about_dat3_files
      cp dat3/gl.account.trans                tmp2/gl_account_trans
      cp dat3/product.master1                 tmp2/product_master1
      cp dat3/uvfix.promo                     tmp2/uvfix_promo
      cp dat3/warranty_master                 tmp2/warranty_master

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5I3. uvfix1 demo - create 'neat' Column-Aligned scripts

create "Column-Aligned" script with 'uvfix1'


 #1. rm -f tmp/* tmp2/*          <-- clear tmp* subdirs in the demo/ directory
     ==================
 #1a. rm -f tmp*/*               <-- same as above, but fewer keystrokes
      ============

 #2. ls dat3 >sfxx/rename_dat3_neat1  <-- capture filenames into the Script Files subdir
     ===============================      assigning an appropriate name

 #3. cat sfxx/rename_dat3_neat1       <-- inspect filenames to be converted into a script
     ==========================
      about.dat3.files             `    <-- filenames of testfiles provided
      gl.account.trans
      product.master1
      uvfix.promo
      warranty_master

 #4. uvcopy uvfix1,fili1=sfxx/rename_dat3_neat1
     ==========================================
        - copy a text file & modify via uvcopy instruction keyins
        - input rec in area a has already been copied to output rec in b
      Note - We have OMITTED several prompts here to change options & show an example
           - enter null to accept the defaults, until you see "enter instruction"
                               <-- enter instruction (or period "." end entries & execute)

 #4a. mvf b0(100),'cp dat3/'   <-- MoVe cp cmd & 1st dir to output area 'b'
      ======================       & Fill remainder of output area with spaces

 #4b. cat b0(100),a0(50)       <-- concatenate the filename onto cp cmd & dir1
      ==================

 #4c. mvc b40(6),','tmp2/'     <-- store 2nd dir at dsplcmnt 40 (column 41)
      ====================

 #4d. cat b40(50),a0(50)       <-- concatenate filename onto destination directory
      ==================

 #4e. rep b40(50),'.','_'      <-- REPlace all '.' periods in 2nd filename with '_' underscores
      ===================

 #4f.  .                       <-- enter '.' to End Instrutions & Execute job
      ===
      20200707:414201:uvfix1: EOF fili01 rds=5 size=132: tf/rename_dat3_neat1
      20200707:414201:uvfix1: EOF filo01 wrts=5 size=317: tmp/rename_dat3_neat1
      EOJ, Output File written to: tmp/rename_dat3_neat1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5I4. uvfix1 demo - create 'neat' column-aligned scripts


 #6. cat tmp/rename_dat3_neat1           <-- inspect script created in the tmp/... dir
     =========================             - if problems, repeat #2 thru #2f
      cp dat3/about.dat3.files                tmp2/about_dat3_files
      cp dat3/gl.account.trans                tmp2/gl_account_trans
      cp dat3/product.master1                 tmp2/product_master1
      cp dat3/uvfix.promo                     tmp2/uvfix_promo
      cp dat3/warranty_master                 tmp2/warranty_master

 #7. cp tmp/rename_dat3_neat1 sfxx/      <-- save script in user tutorial scripts directory
     ==============================

 #8. chmod 775 sfxx/rename_dat3_neat1    <-- add execute permissions
     ================================

 #9. sfxx/rename_dat3_neat1              <-- execute the script
     ======================

 #9a. rename_dat3_neat1        <-- will not work because sfxx is not in $PATH
      =================

 #10. sfyy/rename_dat3_neat1   <-- could execute the UV Software answer version
      ======================     - but we hope you will create your own version in sfxx/...

Many rename... scripts provided in $UV/sf/util/...

UV Software already proviides script 'renameP2U' - just ONE of over 40, Here are a few examples, see iall in $UV/sf/util/...

renameL
  • translate All filenames in directory to lower case
renameU
  • translate All filenames in directory to UPPER case
renameX
  • replace eXtension with alternate, renameP Prefix to alternate
renameAA
  • replace Any pattern with any Alternate anywhere in filename
renameB2U
  • change blanks to underscores, renameD2_ '$' signs to '_'s
renameD2U
  • change '$' signs to '_' Underscores
renameP2U
  • change '.' Periods to '_' Underscores
  • many more, all scripts display old/new filenames with seq#

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5I5. uvfix1 creating scripts with filenames in fixed columns

Explanations of Instructions to create sfxx/rename_dat3_neat1

 We will explain the intructions entered to uvfix1 on lines #5a to #5f above
 - illustrating with the 1st record & showing results of each instruction in area 'b'
 about.dat3.files
 about.dat3.files         <-- area 'a' with 1st record
 ================================================================================
         10        20        30        40        50        60        70        80
 12345678901234567890123456789012345678901234567890123456789012345678901234567890

 about.dat3.files         <-- area 'b' 1st record copied before user instructions
 ================================================================================
                                                      #5a. mvf b0(100),'cp dat3/'
 cp dat3/                                                      <-- results
 ================================================================================
                                                      #5b. cat b0(100),a0
 cp dat3/about.dat3.files
 ================================================================================
                                                      #5c. mvc b40(6),','tmp2/'

 cp dat3/about.dat3.files                tmp2/
 ================================================================================
                                                      #5d. cat b0(100),ax1(50)

 cp dat3/about.dat3.files                tmp2/about.dat3.files
 ================================================================================
                                                      #5e. rep b40(30),'.','_'

 cp dat3/about.dat3.files                tmp2/about_dat3_files
 ================================================================================
                                                      #5f.  .
 ================================================================================
                                                      #6. cat tmp/rename_dat3_neat1
 cp dat3/about.dat3.files                tmp2/about_dat3_files
 cp dat3/gl.account.trans                tmp2/gl_account_trans
 cp dat3/product.master1                 tmp2/product_master1
 cp dat3/uvfix.promo                     tmp2/uvfix_promo
 cp dat3/warranty_master                 tmp2/warranty_master

Notes

  1. Could do some of these steps using an editor (such as 'vi')
  2. But hard to copy data to desired output columns for neat appearance
  3. And hard to restrict operations to desired column ranges

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5J1. uvfix1 demo - column-aligned EXISTING scripts

Suppose we just want to make an existing script neater by aligning output files at as specified column. As an example we will use the script created in demo but with only 1 space between input tmp1/filenames & output tmp2/filenames:

sfxx/rename_dat3 - script BEFORE column-alignment

      cp dat3/about.dat3.files tmp2/about_dat3_files
      cp dat3/gl.account.trans tmp2/gl_account_trans
      cp dat3/product.master1 tmp2/product_master1
      cp dat3/uvfix.promo tmp2/uvfix_promo
      cp dat3/warranty_master tmp2/warranty_master

sfxx/rename_dat3_neat2 - script AFTER column-alignment

      cp dat3/about.dat3.files                tmp2/about_dat3_files
      cp dat3/gl.account.trans                tmp2/gl_account_trans
      cp dat3/product.master1                 tmp2/product_master1
      cp dat3/uvfix.promo                     tmp2/uvfix_promo
      cp dat3/warranty_master                 tmp2/warranty_master

 #1. rm -f tmp/* tmp2/*   <-- clear tmp* subdirs in the demo/ directory
     ==================

 #2. uvcopy uvfix1,fili1=sfxx/rename_dat3,uop=c1
     ============================================
        - copy a text file & modify via uvcopy instruction keyins
        - use ',uop=c1' option to pass thru '#' Comment records unchanged
        - use ',uop=d1' option to Delete '#' comment records
        - input rec in area a has already been copied to output rec in b
        - We have OMITTED several prompts here to change options & show an example
        - enter null to accept the defaults, until you see "enter instruction"
                              <-- enter instruction (or period "." end entries & execute)

      #2a. scnn2p a0(100),' !'     <-- scan to 2nd occurrence of space + non-space
           ===================       - option 'n2' for 2nd occurrence
                                     - option 'p' for pattern scan ('!' any non-space)
                                     - scan sets register 'x' to dsplcmnt of found pattern
      #2b. clr bx0(100),' '        <-- clear output dir/file from area 'b'
           ================
      #2c. mvc b40(50),ax1         <-- move ouput dir/file from area 'a' to area 'b' column 40+
           ===============
      #2d. .                       <-- enter '.' period to end entries & execute
           ===
           20200708:153902:uvfix1: EOF fili01 rds=8 size=473: sfxx/rename_dat3
           20200708:153902:uvfix1: EOF filo01 wrts=5 size=381: tmp/rename_dat3

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5J2. uvfix1 demo - create 'neat' column-aligned scripts


 #3. cat tmp/rename_dat3                   <-- inspect script created in the tmp/... dir
     ===================                     - if problems, repeat #2 thru #2d
      cp dat3/about.dat3.files                tmp2/about_dat3_files
      cp dat3/gl.account.trans                tmp2/gl_account_trans
      cp dat3/product.master1                 tmp2/product_master1
      cp dat3/uvfix.promo                     tmp2/uvfix_promo
      cp dat3/warranty_master                 tmp2/warranty_master

 #4. copy the new version from tmp/rename_dat3 to permanent Scripts directory
     - renaming to avoid overwriting the original
     - in case you want to rerun this exercise

 #4a. cp tmp/rename_dat3 sfxx/rename_dat3_neat2  <-- copy/renaming to avoid overwrite
      =========================================

 #5. chmod 775 sfxx/rename_dat3_neat2             <-- add execute permissions
     ================================

 #6. sfxx/rename_dat3_neat2                       <-- execute the script
     ======================

 #7. rename_dat3_neat2        <-- will not work because sfxx is not in $PATH
     =================

 #8. sfyy/rename_dat3_neat2   <-- could execute the UV Software answer version
     ======================     - but we hope you will create your own version in sfxx/...

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5J3. uvfix1 creating scripts with filenames in fixed columns

Explanations of Instructions to create sf/rename_dat3_neat2

 We will explain the intructions entered to uvfix1 on lines #2a to #2d above
 - illustrating with the 1st record & showing results of each instruction in area 'b'

 # rename_dat3 - script to copy a script column aligning output filenames
 ================================================================================
                              #2a. scnn2p a0(100),' !'   <-- scan to 2nd dir/file
         10        20        30        40        50        60        70        80
 12345678901234567890123456789012345678901234567890123456789012345678901234567890
 cp tmp1/about.dat3.files     tmp2/about.dat3.files     <-- area 'a' 1st non-cmt record
 ================================================================================
                             ^---- register 'x' set to 2nd occurrence of space+non-space
                             #2b. clr bx0(100),' '  <-- clear area 'b' from $rx col 29 on

 cp tmp1/about.dat3.files                               <-- area 'b' cleared from col 29+
 ================================================================================
                             #2c. mvc b40(50),ax1   <-- move out dir/file to b+$rx(28)+1

 cp tmp1/about.dat3.files                tmp2/about.dat3.files
 ================================================================================
                             #2d. .                 <-- enter '.' end entries & execute

 #2. cat tmp/rename_dat3_neat2         <-- inspect script created in the tmp/... dir
     ====================
      cp dat3/about.dat3.files                tmp2/about_dat3_files
      cp dat3/gl.account.trans                tmp2/gl_account_trans
      cp dat3/product.master1                 tmp2/product_master1
      cp dat3/uvfix.promo                     tmp2/uvfix_promo
      cp dat3/warranty_master                 tmp2/warranty_master

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5K1. uvfix1 Tutorials & Exercises

Instruction ERROR msgs & prompt to re-enter

When uvcopy is executed in batch mode (with a pre-edited parameter file of instructions), any error aborts the job & you would correct the parameter file & re-execute.

Since uvfix1 operates in interactive mode, we would not want to abort, so we have an option (rop=e2) to cause prompts for instruction re-entry.

Here is a demo with intentional errors to show you some of the diagnostic error msgs:


 #1. rm -f tmp/*            <-- clear tmp/* files
     ===========

 #2. uvfix1 dat1/Provinces.csv    <-- run uvfix1 script with input file dat1/USstates1
     =========================      - omitting option & filename prompts
                                    - enter nulls until the '.' prompt for instructions

      #2a. fix 0(20),a0(100),',',6
           =======================
           uvcopy uvfix1 ERR - area code a-z missing for op# 1
                                      <-- Re-enter instruction (or "." to Execute)
      #2b. fix b0(20),a0(000),',',6
           =======================
           uvcopy uvfix1 ERR - 0 lth, missing op# 2, invalid tag(3 alpha b4 numeric)
                                      <-- Re-enter instruction (or "." to Execute)
      #2c. fix b0(20),a0(100),',',6
           =======================
           uvcopy uvfix1 ERR - invalid format for operand # 3
                                      <-- Re-enter instruction (or "." to Execute)
      #2d. fix b0(20),a0(100),6,','
           =======================
                                      <-- enter instruction (or "." Execute)
 .
 20200829:513208:uvfix1: EOF fili01 rds=14 size=720: dat1/Provinces.csv
 20200829:513208:uvfix1: EOF filo01 wrts=14 size=1506: tmp/Provinces.csv
 20200829:513208:uvfix1: EOF filo02 wrts=33 size=1737: tmp/Provinces.csv.pf

Notes re Instruction Error msgs

  1. We are not showing the output file since the point of this demo was to show error-messages & prompts to re-enter.

  2. There are many more errmsgs that would terminate the job without offering a re-enter option, but we think we have allowed for all the errors that you might make during interactive instruction entry.

  3. You might experiment with intentional errors & let us know if you find an error which does not allow a re-entry.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5L1. uvfix1 Tutorials & Exercises

complete uvcopy job saved in tmp/filename.pf

'uvfix1' is quick & easy, when all you need to do is copy 1 input file to 1 output file & apply a few instructions. This page shows you what you can do when you need more files or need to print total lines, etc.

Actually, uvfix1 saves the created job, as well as executing immediately. The complete job (including file definitions & I/O commands to get & put records) is saved in tmp/... with the same filename as the output file + extension '.pf'.

For example, suppose you want to read dat1/USstates1, accumulating the state populations & printing the total at EOF. You can run uvfix1 with no instructions, save the framework job to a permanent pf/... directory, modify to print totals,& execute the modified job.


 #1. rm -f tmp/*                <-- clear tmp/* files
     ===========

 #2. uvfix1 dat1/USstates1      <-- run uvfix1 script with input file dat1/USstates1
     =====================        - omitting option & filename prompts
                                  - enter nulls until the '.' prompt for instructions
      #2a. .                    <-- enter '.' to end job & write tmp/USstates1.pf
      ========                    - no instrucitons entered (will modify saved job)

 #3. l tmp/                     <-- list outputs in tmp
     =======
     --> tmp/USstates1          <-- output file (will not be used)
     --> tmp/USstates1.pf       <-- complete uvcopy job with file I/O

 #4. cp tmp/USstates1.pf pfxx   <-- save RE-RUN job in a permanent pf directory
     ========================

 #5. cat pfxx/USstates.pf        <-- display re-run job BEFORE copy/modify
     ====================
 # USstates1.pf - uvcopy job created by --> uvcopy uvfix1,fili1=dat1/USstates1
 #             - stored in tmp/USstates1.pf in case required for modify/rerun
 #             - named same as uvfix1 data output in tmp/... but with '.pf' appended
 rop=r1                #<-- EOF option to view output file (reply vi,cat,more,etc)
 fili1=?dat1/USstates1,typ=LST,rcs=4096       # '?' prompts for change, code your input file
 filo1=?tmp/USstates1,typ=LSTt,rcs=4096   # this would write same filename in tmp/..
 @run
         opn     all                     open files
 # begin loop to get & put records until EOF
 getrec  get     fili1,a0                get record into area 'a'
         skp>    EOF                     (cc set > at EOF)
         mvc     b0(4096),a0             move input area 'a' to output area 'b'
 #-------------------------------------
 #-------------------------------------
 putrec  put     filo1,b0                write to output file from area 'b'
         skp     getrec                  return to get next record
 EOF     cls     all                     close files
         eoj                             end job

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5L2. modifying uvcopy job saved in tmp/filename.pf

Accumulate total state population to print at EOF


 #6. vi pfxx/USstates1.pf       <-- edit adding instructions to accumulate
     ====================           populations & print totals at End Of Job
 # USstates1.pf - uvcopy job created by --> uvcopy uvfix1,fili1=dat1/USstates1
 #             - saved in tmp/USstates1.pf in case required for modify/rerun
 #             - named same as uvfix1 data output in tmp/... but with '.pf' appended
 #
 # 1. cp tmp/USstates1.pf pfxx    <-- copy to a permanent pf directory
 # 2. vi pfxx/USstates1.pf        <-- edit to modify/add desired instructions
 # 3. uvcopy pfxx/USstates1.pf    <-- rerun modified job
 #
 #Note - This is the modified result of tutorial at www.uvsoftware.ca/uvdemos1.htm#5L1 & 5L2
 #     - marking Additions/Modifications marked on right with <--A--
 #
 uop=q0c0n99r4096      # user option defaults ('n99' means **ALL** records)
 rop=r1                #<-- EOF option to view output file (reply vi,cat,more,etc)
 fili1=?dat1/USstates1,typ=LST,rcs=4096    # '?' prompts for change, code your input file
 filo1=?tmp/USstates1A,typ=LSTt,rcs=4096   # modified outfilename (+A)    <--A#1
 @run
         opn     all                     open files
 # begin loop to get & put records until EOF
 getrec  get     fili1,a0                get record into area 'a'
         skp>    EOF                     (cc set > at EOF)
         mvc     b0(4096),a0             move in-area 'a' to out-area 'b'
 #-------------------------------------
         add     $ca1,a40(20)            Accumulate state-population      <--A#2
         edt     b40(12),a40(20),'zzz,zzz,zz9'   edit state-pop           <--A#3
         edt     b80(12),a80(20),'zzz,zzz,zz9'   edit capital-pop         <--A#4
 #-------------------------------------
 putrec  put     filo1,b0                write to out-file from area 'b'
         skp     getrec                  return to get next record
 #
 EOF     mvf     c0(100),'** Total US states Population'                  <--A#5
         edt     c40(12),$ca1,'zzz,zzz,zz9'                               <--A#6
         put     filo1,c0(100)                                            <--A#7
         cls     all                     close files                      <--A#8
         eoj                             end job

Notes re Added/modified Instructions


 #7. uvcopy pfxx/USstates1.pf      <-- execute the saved & Modified re-run job
     ========================        - see report on the page below:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5L3. Accumulate total state population to print at EOF


 #8. more tmp/USstates1    <-- display report from modified re-run job
     ==================
 abr                 state                          0        capital                        0
 AL                  Alabama                4,908,620        Montgomery               205,764
 AK                  Alaska                   734,002        Juneau                    31,275
 AZ                  Arizona                7,378,490        Phoenix                1,445,632
 AR                  Arkansas               3,039,000        Little Rock              193,524
 CA                  California            39,937,500        Sacramento               466,488
 CO                  Colorado               5,845,530        Denver                   600,158
 CT                  Connecticut            3,563,080        Hartford                 124,775
 DE                  Delaware                 982,895        Dover                     36,047
 FL                  Florida               21,993,000        Tallahassee              181,376
 GA                  Georgia               10,736,100        Atlanta                  420,003
 HI                  Hawaii                 1,412,690        Honolulu                 337,256
 ID                  Idaho                  1,826,160        Boise                    205,671
 IL                  Illinois              12,659,700        Springfield              116,250
 IN                  Indiana                6,745,350        Indianapolis             820,445
 IA                  Iowa                   3,179,850        Des Moines               203,433
 KS                  Kansas                 2,910,360        Topeka                   127,473
 KY                  Kentucky               4,499,690        Frankfort                 25,527
 LA                  Louisiana              4,645,180        Baton Rouge              229,493
 ME                  Maine                  1,345,790        Augusta                   19,136
 MD                  Maryland               6,083,120        Annapolis                 38,394
 MA                  Massachusetts          6,976,600        Boston                   617,594
 MI                  Michigan              10,045,000        Lansing                  114,297
 MN                  Minnesota              5,700,670        Saint Paul               285,068
 MS                  Mississippi            2,989,260        Jackson                  173,514
 MO                  Missouri               6,169,270        Jefferson City            43,079
 MT                  Montana                1,086,760        Helana                    28,190
 NE                  Nebraska               1,952,570        Lincoln                  258,379
 NV                  Nevada                 3,139,660        Carson City               55,274
 NH                  New Hampshire          1,371,250        Concord                   42,695
 NJ                  New Jersey             8,936,570        Trenton                   84,913
 NM                  New Mexico             2,096,640        Santa Fe                  67,947
 NY                  New York              19,440,500        Albany                    97,856
 NC                  North Carolina        10,611,900        Raleigh                  403,892
 ND                  North Dakota             761,723        Bismarck                  61,272
 OH                  Ohio                  11,747,700        Columbus                 787,033
 OK                  Oklahoma               3,954,820        Oklahoma City            579,999
 OR                  Oregon                 4,301,090        Salem                    154,637
 PA                  Pennsylvania          12,820,900        Harrisburg                49,528
 RI                  Rhode Island           1,056,160        Providence               178,042
 SC                  South Carolina         5,210,100        Columbia                 129,272
 SD                  South Dakota             903,027        Pierre                    13,646
 TN                  Tennessee              6,897,580        Nashville                601,222
 TX                  Texas                 29,472,300        Austin                   790,390
 UT                  Utah                   3,282,120        Salt Lake City           186,440
 VT                  Vermont                  628,061        Montpelier                 7,855
 VA                  Virginia               8,626,210        Richmond                 204,214
 WA                  Washington             7,797,100        Olympia                   46,478
 WV                  West Virginia          1,778,070        Charleston                51,400
 WI                  Wisconsin              5,851,750        Madison                  233,209
 WY                  Wyoming                  567,025        Cheyenne                  59,466
 ** Total US states Population            330,598,493

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5L4. Further Enhance to calc state pop % of total USA pop

Further Enhancements

  1. Add page headings with jobname, filname,& $date

  2. Calculate state pop as percentage of total pop - add user option 't' for total pop accumulated by 1st modified report above - declare at begin job on uop=t330698493 to be addressed as $uopbt - insert edited percentage between state-pop & capital city name


 #1. cp pf/USstates1.pf pf/USstates1.pf2  <-- copy 1st job above (USstates1.pf)
     ===================================    - renaming as USstates1.pf2

 #2. vi pf/USstates1.pf2   <-- further enhance as described above
     ===================     - changes marked below with <--EA#_ on right
 # USstates1.pf2 - Further Enhanced version of USstates.pf
 # USstates1.pf  - original uvcopy job created by --> uvcopy uvfix1,fili1=dat1/USstates1
 #               - original saved/rerun job was modified to accumulate state-pop & print total
 # USstates1.pf2 - Further Enhanced to print page headings with jobname,filename,date
 #               - And calculate percentage state-pop of total pop
 #
 #Note - This is the ENHANCED result of tutorial at www.uvsoftware.ca/uvdemos1.htm#5Lr13 - 5L5
 #     - ORIGINAL Additions/Modifications were marked on right with <--A--
 #     - Further ENHANCED additions/modifications marked with <--EA--
 #
 uop=q0c0n99r4096t330598493  # option defaults + Total pop option t=330,598,493 <--EA#1
 #Note - uop t330698493 added from original USstates.pf accumulation total pop
 rop=r1                      # EOF option to view output (reply vi,cat,more,etc)
 fili1=?dat1/USstates1,typ=LST,rcs=4096    # '?' prompts for change, code your input file
 filo1=?tmp/USstates1E,typ=LSTt,rcs=4096   # modiifed outfilename (+E)    <--EA#2
 @run
         opn     all                     open files
         mvfv1   c0(100),'$jobname - US state populations & percentage of total - $date'
         put     filo1,c0(100)            print page heading              <--EA#3
 # begin loop to get & put records until EOF
 getrec  get     fili1,a0                get record into area 'a'
         skp>    EOF                     (cc set > at EOF)
         mvc     b0(4096),a0             move in-area 'a' to out-area 'b'
 #-------------------------------------
         add     $ca1,a40(20)            Accumulate state-population      <--A#1
         edt     b40(12),a40(20),'zzz,zzz,zz9'   edit state-pop           <--A#2
         edt     b80(12),a80(20),'zzz,zzz,zz9'   edit capital-pop         <--A#3
         mvn     $ca2,a40(20)            move current state pop to ctr#2  <--EA#4
         mpy     $ca2,1000               multiply 1000 percent 1 decimal  <--EA#5
         div     $ca2,$uopbt             divide by US total pop           <--EA#6
         edt     b53(5),$ca2,'zz9.9'     edit state pop percentage        <--EA#7
 #-------------------------------------
 putrec  put     filo1,b0                write to out-file from area 'b'
         skp     getrec                  return to get next record
 #
 EOF     mvf     c0(100),'** Total US states Population'                  <--A#4
         edt     c40(12),$ca1,'zzz,zzz,zz9'                               <--A#5
         put     filo1,c0(100)                                            <--A#6
         cls     all                     close files                      <--A#7
         eoj                             end job

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5L5. MODIFY USstates1.pf to calc state pop % of total USA pop

Notes re Calculating state pop % of USA Total pop

Problem
  • We need to have the total USA population to calc state pop,
    But how can we know total pop BEFORE we get total accumulation at EOF ???
Solution
  • Get the total pop from the previous report shown on page '5L3'
    And add it as option 't' on the end of the uop=... options line
 uop=q0c0n99r4096      # options line from pfxx/USstates1.pf job on page '5L2'
                       # - append option 't' with USA total pop in modified job page '5L4'

 uop=q0c0n99r4096t330598493  # option with Total pop option t=330,598,493 <--EA#1
                 ==========  # options line with total pop appended
Note
  • see the modifed report on the page below, with state pop as % of total USA pop
    on all 50 state lines & total pop at end of report

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

5L6. MODIFY USstates1.pf to calc state pop % of total USA pop


 #3. uvcopy pf/USstates1.pf2  <-- Execute further enhanced job
     =======================    - see report below
 USstates1E - US state populations & percentage of total - 2020/09/03
 abr                 state                          0   0.0  capital                        0
 AL                  Alabama                4,908,620   1.4  Montgomery               205,764
 AK                  Alaska                   734,002   0.2  Juneau                    31,275
 AZ                  Arizona                7,378,490   2.2  Phoenix                1,445,632
 AR                  Arkansas               3,039,000   0.9  Little Rock              193,524
 CA                  California            39,937,500  12.0  Sacramento               466,488
 CO                  Colorado               5,845,530   1.7  Denver                   600,158
 CT                  Connecticut            3,563,080   1.0  Hartford                 124,775
 DE                  Delaware                 982,895   0.2  Dover                     36,047
 FL                  Florida               21,993,000   6.6  Tallahassee              181,376
 GA                  Georgia               10,736,100   3.2  Atlanta                  420,003
 HI                  Hawaii                 1,412,690   0.4  Honolulu                 337,256
 ID                  Idaho                  1,826,160   0.5  Boise                    205,671
 IL                  Illinois              12,659,700   3.8  Springfield              116,250
 IN                  Indiana                6,745,350   2.0  Indianapolis             820,445
 IA                  Iowa                   3,179,850   0.9  Des Moines               203,433
 KS                  Kansas                 2,910,360   0.8  Topeka                   127,473
 KY                  Kentucky               4,499,690   1.3  Frankfort                 25,527
 LA                  Louisiana              4,645,180   1.4  Baton Rouge              229,493
 ME                  Maine                  1,345,790   0.4  Augusta                   19,136
 MD                  Maryland               6,083,120   1.8  Annapolis                 38,394
 MA                  Massachusetts          6,976,600   2.1  Boston                   617,594
 MI                  Michigan              10,045,000   3.0  Lansing                  114,297
 MN                  Minnesota              5,700,670   1.7  Saint Paul               285,068
 MS                  Mississippi            2,989,260   0.9  Jackson                  173,514
 MO                  Missouri               6,169,270   1.8  Jefferson City            43,079
 MT                  Montana                1,086,760   0.3  Helana                    28,190
 NE                  Nebraska               1,952,570   0.5  Lincoln                  258,379
 NV                  Nevada                 3,139,660   0.9  Carson City               55,274
 NH                  New Hampshire          1,371,250   0.4  Concord                   42,695
 NJ                  New Jersey             8,936,570   2.7  Trenton                   84,913
 NM                  New Mexico             2,096,640   0.6  Santa Fe                  67,947
 NY                  New York              19,440,500   5.8  Albany                    97,856
 NC                  North Carolina        10,611,900   3.2  Raleigh                  403,892
 ND                  North Dakota             761,723   0.2  Bismarck                  61,272
 OH                  Ohio                  11,747,700   3.5  Columbus                 787,033
 OK                  Oklahoma               3,954,820   1.1  Oklahoma City            579,999
 OR                  Oregon                 4,301,090   1.3  Salem                    154,637
 PA                  Pennsylvania          12,820,900   3.8  Harrisburg                49,528
 RI                  Rhode Island           1,056,160   0.3  Providence               178,042
 SC                  South Carolina         5,210,100   1.5  Columbia                 129,272
 SD                  South Dakota             903,027   0.2  Pierre                    13,646
 TN                  Tennessee              6,897,580   2.0  Nashville                601,222
 TX                  Texas                 29,472,300   8.9  Austin                   790,390
 UT                  Utah                   3,282,120   0.9  Salt Lake City           186,440
 VT                  Vermont                  628,061   0.1  Montpelier                 7,855
 VA                  Virginia               8,626,210   2.6  Richmond                 204,214
 WA                  Washington             7,797,100   2.3  Olympia                   46,478
 WV                  West Virginia          1,778,070   0.5  Charleston                51,400
 WI                  Wisconsin              5,851,750   1.7  Madison                  233,209
 WY                  Wyoming                  567,025   0.1  Cheyenne                  59,466
 ** Total US states Population            330,598,493

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_6. Mass Change Search/Replace Beyond Unix Tools

UVdemos1 - Part 6 - Contents


6A1. Overview of Vancouver Utility Mass Change Utilities
- search/replace depending on multiple patterns present &/or not present
- table summaries based on keywords & offsets to target words
- these tools can save days of manual editing when you need to make mass
  changes to large directories of programs & documentation
- parameter driven tools can do in minutes what otherwise would require
  hours of manual editing or writing a complex C program to accomplish

6B1. scan1d - scan a directory for 1 pattern qualified by 1 or 2 other patterns
that must be or must not be present

6C1. scan2d - scan a directory for matches to multiple patterns in a tablefile
- each entry qualified by 1 or 2 patterns present or not present

6D1. rep1d - scan a directory replacing 1 pattern with an alternate
- qualified by 1 or 2 other patterns present or not present

6E1. rep2d - search/replace by table for all files in a directory
- each entry qualified by 1 or 2 patterns present or not present

6F0. TableJobs - Powerful tool for programmers to analyze problems & plan changes
- build tables of counts & values to be dumped to a report file at EOF.
- specify Key-Words preceding the Target-Words to be table summarized.

6F1. table3d Example - For FTP scripts, table the IP#s following the KeyWord "OPEN",
or table the userids & passwords following the KeyWord "USER".

6G1. Mass Change Project - change all occurences of UVdemos.htm to UVdemos2.htm
- required when we added UVdemos1.htm (preliminary to existing UVdemos.doc).
- Vancouver Utility documentation is 325,518 lines in 152 text files
- UVdemos.doc was just 1 of the 152 files, but there were hundreds of references
  to various pages from many other documents.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6A1. Search/Replace Beyond Unix System Tools

Overview of Vancouver Utility Mass Change Utilities

The Vancouver Utilities are invaluable to IT professionals at Unix/Linux sites.

  1. They are much more powerful than grep because they can search for multiple patterns with 1 pass, using AND & OR relationships.

  2. You can specify that the multiple patterns must be found or not found on the same line or on any line in the file, or on 1 or any line between a begining & ending line, that are themselves identified by 1 or multiple patterns.

  3. You can use these tools to investigate problems in text files of any kind: data files, scripts (SQLs, JCLs, FTPs, etc),& any programming language.

  4. You can use these tools to research the impacts of planned changes to your scripts, programs, documentation, etc.

  5. These tools create reports in subdirs (rpts,stats,selects) of the current working directory, automatically named from the file or directories being searched & the report names are concatenated with the patterns used. This allows you to review past searches & easily modify & repeat the search.

  6. The search reports include all relevant info (files,dirs,search patterns,date/time, user,host,etc) to help you document problems & plan proposed changes.

  7. uvcopy makes it easy to do mass changes that would otherwise require many hours of editing or writing specific programs in other languages.

  8. uvcopy has all the power required to deliver solutions much faster than other languages. uvcopy is parameter driven & requires no compiles - the power of assembly without the complexity.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B1. Search/Replace Beyond Unix System Tools

scan1d - scan directory for 1 pattern qualified by other patterns

'scan1d' is a uvcopy job to scan all files in a directory searching for any 1 pattern, qualified by 1 or 2 other patterns (present or not present), creating a report with matching lines, line#s,& total matches by file.


 uvcopy scan1d   - would prompt for directory,arguments,& options
 =============   - better to code directory & arguments on command line
                 - to show on audit report for easier re-runs

command format & arguments


 uvcopy scan1d,fild1=dir,arg1=search-pat,arg2=qual-present,arg3=qual-absent
 ==========================================================================
  ,arg6=searchbgn,arg7=searchend,arg8=match-optns,arg9=viewer,uop=user-optns
  ==========================================================================
   - command format above, see example/demo on page below
 arg1 - search pattern (prefix 0x indicates hex, 0x2C for a comma)
 arg2 - qualifier present (optional)
 arg3 - qualifier absent, or 2nd present if option p2
 arg6 - pattern to begin scanning for search pattern (optional)
      - also see alternative uop=a1 to begin scan by data file line
 arg7 - pattern to end scanning for search pattern (optional)
      - also see alternative uop=b99999 to end scan by data file line
 arg8 (match options): q3 inhibits scanning within quotes
        i = case insensetive (patterns must be entered in lower case)
        p = pattern match chars (@=any alpha,=any numeric,etc)
 uop (user options): q1 causes prompt for change, q0 inhibits prompt
        a1=line(within file) to begin scan,b99999=lineto end scan
        c0=column(within lines, zero rel) begin scan, d256=length of scan
        p1 - qualifier1 Present & qualifier2 Absent'
        p2 - qualifier1 Present & qualifier2 Present'
        p4 - qualifier1 Absent  & qualifier2 Absent'

July 2020 updates

 scan1d,scan2d,rep1d,rep2d updated to create audit reports in rpts/...
 - named as jobname_indir_oudir_searchpattern_replace_pattern_optins
 - showing command used to create report, making it easy to rerun
 New doc at uvdemos1.htmPart_6 (vs old doc at scanjobs.htm)
 - using new demo files in $UV/demo/dat1/... (vs $UV/tf/...)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B2. scan1d - scan directory for 1 pattern qualified by other patterns

scan1d demo


 #1. cdd   <-- alias to change to the demo/... subdir
     ===     - /home/uvsoftxx/demo/... for online to UVsoftware website
             - /home/userxx/demo/..... for offline on user machines

 #2. uvcopy scan1d,fild1=demo/dat1,arg1=Idaho
     ========================================
      - scan all files in demo/dat1/... searching for "Idaho"
      - output report will default to rpts/demo_dat1_Idaho
      - First --> mkdir rpts <-- if not already present
      - See demo output Report listed below

 #3. cat rpts/scan1d_dat1_Idaho
     ==========================
 Report: rpts/scan1d_dat1_Idaho
 -----> uvcopy scan1d,fild1=dat1,arg1=Idaho
        ============================================================================
 Format: uvcopy scan1d,fild1=indir,arg1=search,arg2=qual1,arg3=qual2,uop=UserOptions
                     ,arg6=searchbgn,arg7=searchend,arg8=MatchOptions
 MatchOptnsArg8: i=Insensetive, q3=notinquotes, p=patternchars(@=alpha,=num,etc)
 UserOptions: a0=BgnScan at bgnfile or arg6 if spcfd, a=BgnScanLine
             b0=EndScan at endfile or arg7 if spcfd, b=EndSscanLine
             c0=BgnScan at dsplcmnt 0 or cdsp, d0=EndScan at 256 or ddsplcmnt
             p1=qual1Present & qual2Absent, p2=bothPresent, p4=bothAbsent
 Date=2020/07/20_12:58:00, Host=uvsoft5, User=uvadm, Option-Overrides=
 ====================================================================================
 00101 097  ID  Idaho                Boise City city                            205,671
 00291 287  ID  Idaho                Meridian city                               75,092
 00305rt: rpts/scan1d    2 hits @EOF: dat1/UScities
 00013 ID                  Idaho               Boise               205671
 00051                   1 hits @EOF: dat1/USstates1
 00013 12  ID    Idaho               Boise                  205,671
 00051                   1 hits @EOF: dat1/USstates2
 =======================================================2020/07/20_12:58:03======
 EOD: 4 hits in 3 files of 16 total (1460 lines)
 Note - Audit report saved in rpts/... for later reviews, reruns, changes & reruns
     - includes all info you might need directory & arguments
     - Audit report-name changes to reflect changes you make to indir & arguments

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6B3. scan1d - scan directory for 1 pattern qualified by other patterns

Alternate Demos Suggested

 Here are some more suggested demos you can run. We will not show the reports
 - but will tell you how many hits to expect (see if your results agree ?).

 #0. rm -f rpts/*   <-- remove any previous audit reports
     ============

 #1. uvcopy scan1d,fild1=dat1,arg1=Vancouver,arg2=BC  <-- search 'Vancouver' And 'BC'
     ===============================================      (arg2 Qualifier-Present)
      - expect 10 hits in 2 files of 13 total

 #2. uvcopy scan1d,fild1=dat1,arg1=Vancouver,arg3=BC  <-- search 'Vancouver' and NOT 'BC'
     ===============================================      (arg3 Qualifier-Absent)
      - expect 1 hit in 1 file of 13 total

 #3. uvcopy scan1d,fild1=dat1,arg1=Richmond,arg2=VA,arg3=BC  <-- 'Richmond' & 'VA', NOT 'BC'
     ======================================================      (arg2 Present, arg3 Absent)
      - expect 3 hits in 3 files of 13 total

 #4. l rpts    <-- list Audit files from alternate demos
     ======
      -rw-rw-r-- 1 uvadm apps 1405 Jul 22 12:45 scan1d_dat1_Richmond_VA_not_BC
      -rw-rw-r-- 1 uvadm apps 2082 Jul 22 12:22 scan1d_dat1_Vancouver_BC
      -rw-rw-r-- 1 uvadm apps 1150 Jul 22 12:23 scan1d_dat1_Vancouver_not_BC
Note
  • Appending search args on audit filenames avoids overwriting previous reports

Run your own scan1d's


 #1. ls dat1      <-- list test file-names available
     =======
      CanadaMPs CanadaProvinces Provinces1 Provinces2
      UScandidates UScandidateups UScities UScongress USstates1 USstates2
      insults nameadrs1 payroll1.dlm

 #2. more dat1    <-- list test-file contents
     =========      - not shown here

 #3. uvcopy scan1d,fild1=dat1,arg1=SearchPattern,arg2=QualifierPresent
                             ,arg3=QualifierAbsent,uop=options
     =================================================================
      - make up your own scan1d's

 #4. l rpts/      <-- list Audit reports, see most recent
     =======        - ReportName includes your filename & arguments

 #5. more rpts/*  <-- inspect your audit reports
     ===========

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6C1. Search/Replace Beyond Unix System Tools

scan2d - scan directory for matches to patterns in a tablefile

'scan2d' is a uvcopy job to scan all files in a directory searching for any match to a table file of search patterns & optional qualifying patterns, matching lines are written to rpts/... with a filename that identifies the jobname (scan2d), the direcrory, the table-filename, & any options.


 uvcopy scan2d   <-- would solicit files & arguments
 =============     - better to code directory & arguments on command line
                   - to show on audit report for easier re-runs
 uvcopy scan2d,fild1=indir,fili2=tablefile,arg6=SearchBegin,arg7=SearchEnd
                          ,arg8=MatchOptions,uop=UserOptions
 ==========================================================================
  - command format above, see example/demo on page below
arg6
  • pattern to begin scanning for search pattern
  • also see alternative uop=a1 to begin scan by data file line
arg7
  • pattern to end scanning for search pattern
  • also see alternative uop=b99999 to end scan by data file line
arg8
  • match options:
    i = case insensetive (table & data trsltd to lower)
    p = pattern match chars (@=any alpha,=any numeric,etc)
    q3 = inhibit scanning within quotes
uop
  • user options:
    a1 = line(within file) to begin scan,b99999=lineto end scan
    c0 = column(within lines, zero rel) begin scan, d256=length of scan
    p1 = qual1 present & qual2 absent, p2 = both qualifiers present
    t1 = qualifiers current table-line, t2 = qualifiers apply to any line

scan2d demo - July 2020


 #1. cdd   <-- alias to change to the demo/... subdir
     ===     - /home/uvsoftxx/demo/... for online to UVsoftware website
             - /home/userxx/demo/..... for offline on user machines

 #2. cat tf/scan2d.tbl   <-- inspect table file to be used for scan2d demo
     =================     - you could prepare alternates with 'vi'
 # scan2d.tbl - table file to demo --> uvcopy scan2d,fild1=directory,fili2=patterntable
 # scan2d - scan all files in directory for any match in a table of patterns
 # 01-30 - search pattern (ended by 1st tilde)
 # 31-60 - qualifier1, must ALSO be present, if coded (disabled by tildes)
 # 61-80 - qualifier2, must  NOT be present, if coded (disabled by tildes)
 #       - option p1 (default above), option p2 both present
 North Vancouver~~~~~~~~~~~~~~~BC~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Vancouver~~~~~~~~~~~~~~~~~~~~~WA~~~~~~~~~~~~~~~~~~~~~~~~~~~~BC~~~~~~~~~~~~~~~~~
 Olympia~~~~~~~~~~~~~~~~~~~~~~~WA~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 ~~~~~~~~~~~~~~~~~~~~~~~~(table end marker)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6C2. scan2d - scan directory for matches to patterns in a tablefile

scan2d demo


 #3. uvcopy scan2d,fild1=dat1,fili2=tf/scan2.tbl  <-- execute scan2d on demo dir
    ==========================================       with demo table-file patterns
      - best to specify indir, tablefile,& any arguments on comamnd line
        (will show on audit report, making it easier for any later reruns)
      - could specify audit report via filo1=..., but it defaults as shown below

 #4. cat rpts/scan2d_dat1_scan2d.tbl   <-- inspect output report
    ===============================
 Report: rpts/scan2d_dat1_scan2d.tbl
 -----> uvcopy scan2d,fild1=dat1,fili2=tf/scan2d.tbl
        ============================================================================
 Format: uvcopy scan2d,fild1=indir,fili2=pattern-table,uop=UserOptions
                     ,arg6=searchbgn,arg7=searchend,arg8=MatchOptions
 MatchOptnsArg8: i=Insensetive, q3=notinquotes, p=patternchars(@=alpha,=num,etc)
 UserOptions: a0=BgnScan at bgnfile or arg6 if spcfd, a=BgnScanLine
             b0=EndScan at endfile or arg7 if spcfd, b=EndSscanLine
             c0=BgnScan at dsplcmnt 0 or cdsp, d0=EndScan at 256 or ddsplcmnt
             p1=qualifier1 Present & qualifier2 Absent, p2=both Present
             t1=type 1 table - qualifiers apply only to current line (default)
             t2=type 2 table - qualifiers independent of search patterns
 Date=2020/07/20_12:27:43, Host=uvsoft5, User=uvadm, OptionOverrides=
 ====================================================================================
                     ** Table-File = tf/scan2d.tbl **
 North Vancouver~~~~~~~~~~~~~~~BC~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Vancouver~~~~~~~~~~~~~~~~~~~~~WA~~~~~~~~~~~~~~~~~~~~~~~~~~~~BC~~~~~~~~~~~~~~~~~
 Olympia~~~~~~~~~~~~~~~~~~~~~~~WA~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 ================================================================================
 00048 WA                  Washington          Olympia             46478
 00051                   1 hits @EOF: dat1/USstates1
 00048 47  WA    Washington          Olympia                 46,478
 00051                   1 hits @EOF: dat1/USstates2
 00001 Owen Townsend       4667 Hoskins Rd     North Vancouver, BC Canada V7K2R3
 00008                   1 hits @EOF: dat1/nameadrs1
 00327 327  Wilkinson          Jonathan        North Vancouver               BC Liberal          2015
 00335                   1 hits @EOF: dat1/CanadaMPs
 00142 138  WA  Washington           Vancouver city                             161,791
 00305                   1 hits @EOF: dat1/UScities
 =======================================================2020/07/20_12:27:44======
 EOD: 00005 hits in 0005 files of 0016 total (01460 lines)
 Note - Audit report saved in rpts/... for later reviews, reruns, changes & reruns
     - includes all info you might need directory, table-file, arguments
     - Audit report-name includes any changes you make indir,tablefile,arguments

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6D1. Search/Replace Beyond Unix System Tools

rep1d - scan a directory replacing 1 pattern with an alternate

'rep1d' is a uvcopy job to scan all files in a directory searching for any 1 pattern & replacing with an alternate pattern. Replacement may be qualified by 1 or 2 other patterns that must be or must not be present.

Note
  • you must create the output directory or remove all files if already existing
  • the output directory defaults to tmp2/ (no need to specify on command line)
  • or you may specify on the command line via ',fild2=...'

 uvcopy rep1d   - directories & arguments will be solicited if omitted
 ============   - could specify on comamnd line as follows:
 uvcopy rep1d,fild1=indir,fild2=outdir,arg1=search,arg2=replace
             ,arg3=qual-present,arg4=qual-absent,arg6=repbgn,arg7=repend
             ,arg8=match-optns,uop=user-optns
 arg1 - search pattern
 arg2 - replacement
 arg3 - qualifier present (optional)
 arg4 - qualifier absent or 2nd qual present if option p2
 arg6 - pattern to begin scanning for search pattern (optional)
      - also see alternative uop=a1 to begin scan by data file line
 arg7 - pattern to end scanning for search pattern (optional)
      - also see alternative uop=b99999 to end scan by data file line
 arg8 (match options): q3 inhibits scanning within quotes
        p - pattern match chars (@=any alpha,=any numeric,etc)
        s - allows match chars in replacement patterns
 uop  - user options (see options on the next page)

rep1d OLD demo


 uvcopy rep1d,fild1=tf,fild2=tmp2,arg1=Canada,arg2=Iceland,arg3=~,arg4=BC,arg8=n
 ==============================================================================
  - output directory will default to tmp2/ if unspecified
  - output audit report will default to 'tmp/indir_searchpattern' if unspcfd
  - demo1 at 'uvsoftware.ca/repjobs.htmE1', listing I/O demo files
  - original demo from around Year 2000

July 2020 updates

 scan1d,scan2d,rep1d,rep2d updated to create audit reports in rpts/...
 - named as jobname_indir_oudir_searchpattern_replace_pattern_optins
 - showing command used to create report, making it easy to rerun
 New documentation at 'uvdemos1.htmPart_6'
 - vs old doc at scanjobs.htm & repjobs.htm
 - see July2020 new rep1d demo below

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6D2. rep1d - scan a directory replacing 1 pattern with an alternate

rep1d demo - July 2020


 #1. cdd   <-- alias to change to the demo/... subdir
     ===     - /home/uvsoftxx/demo/... for online to UVsoftware website
             - /home/userxx/demo/..... for offline on user machines

 #2. uvcopy rep1d,fild1=dat1,arg1=Vancouver,arg2=CoastCity,arg4=Liberal
     ==================================================================
      - search/replace 'Vancouver' with 'CoastCity' in all files of $UV/demo/dat1/...
        while copying to $UV/demo/tmp2/...

audit report created in rpts/rep1d_indir_arg1_arg2

 Report: rpts/rep1d_dat1_Vancouver_CoastCity
 -----> uvcopy rep1d,fild1=dat1,arg1=Vancouver,arg2=CoastCity,arg4=Liberal
        ============================================================================
 Format: uvcopy rep1d,fild1=indir,fild2=outdir,arg1=search,arg2=replace
              ,arg3=qual-present,arg3=qual-absent,arg6=repbgn,arg7=repend
              ,arg8=Match-Options,uop=User-Options
 MatchOptions: i=caseInsensetive, q3=NoMatchInQuotes, p=patternchars(@=alpha,=num,etc)
 UserOptions:  a0=BgnScan at bgnfile or arg6 if spcfd, a=BgnScanLine
              b0=EndScan at endfile or arg7 if spcfd, b=EndSscanLine
              c0=BgnScan at dsplcmnt 0 or cdsp, d0=EndScan at 256 or ddsplcmnt
              p1=qual1Present & qual2Absent, p2=bothPresent, p4=bothAbsent
 Date=$datetime, Host=$hostname, User=$logname, Option-Overrides=UOPS2
 ====================================================================================
 00075 075  Davies             Don             CoastCity Kingsway            BC NDP              2015
 00165 165  Kwan               Jenny           CoastCity East                BC NDP              2015
 00328 328  Wilson-Raybould    Jody            CoastCity Granville           BC Independent      2015
 00335                   3 reps @EOF: tmp2/CanadaMPs
 00142 138  WA  Washington           CoastCity city                             161,791
 00305                   1 reps @EOF: tmp2/UScities
 00001 Owen Townsend       4667 Hoskins Rd     North CoastCity, BC Canada V7K2R3
 00002 Jenny Townsend      21 Canada Place     CoastCity, BC       Canada V5P3V8
 00008                   2 reps @EOF: tmp2/nameadrs1
 ====================================================== 2020/07/21_10:05:00
 EOD: 6 reps in 3 files of 16 total (1460 lines)
 Audit filename: rpts/rep1d_dat1_Vancouver

'alldiff2' to verify search/replacements


 #3. alldiff2 dat1 tmp2   <-- verify changes as intended & nothing unintended
     ==================     - will match the audit file shown above

 #4. cp tmp2/* dat1       <-- copy back overwriting original files
     ==============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6E1. Search/Replace Beyond Unix System Tools

rep2d - search/replace by table for all files in a directory

'rep2d' is a uvcopy job to search all files in a directory, using a search/replace table file of search-patterns & replacement-patterns & optionally qualified by a pattern that must be present & a 2nd pattern that must not be present.

sample search/replace table

 # rep2d.tbl - table file to demo uvcopy rep2d mass change via search/replace table
 # uvcopy rep2d,fild1=indir,fild2=outdir,fili2=patterntable
 # 001-030 - search pattern (ended by 1st tilde)
 # 031-060 - replace pattern (ended by 1st tilde)
 # 061-080 - qualifier#1, must ALSO be present, if coded (disabled by tildes)
 # 081-100 - qualifier#2, must  NOT be present, if coded (disabled by tildes)
 #         - option p1 (default above), option p2 both present
 North Vancouver~~~~~~~~~~~~~~~South Vancouver~~~~~~~~~~~~~~~BC~~~~~~~~~~~~~~~~~~Liberal~~~~~~~~~~~~~
 New Mexico~~~~~~~~~~~~~~~~~~~~OLD Mexico~~~~~~~~~~~~~~~~~~~~NM~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Richmond~~~~~~~~~~~~~~~~~~~~~~Poormond~~~~~~~~~~~~~~~~~~~~~~VA~~~~~~~~~~~~~~~~~~BC~~~~~~~~~~~~~~~~~~
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~(table end marker)~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 uvcopy rep2d   - directories & arguments will be solicited
 ============   - But, better to code on command line
                - to appear on the audit report, to make reruns easier
 uvcopy rep2d,fild1=indir,fild2=outdir,fili3=table,filo4=auditreport
             ,arg6=RepBgn,arg7=RepEnd,arg8=match-optns,uop=user-optns
 ====================================================================
 arg6 - pattern to begin scanning for search pattern (optional)
      - also see alternative uop=a1 to begin scan by data file line
 arg7 - pattern to end scanning for search pattern (optional)
      - also see alternative uop=b99999 to end scan by data file line
 arg8 (match options): q3 inhibits scanning within quotes
        p - pattern match chars (@=any alpha,=any numeric,etc)
        s - allows match chars in replacement patterns
 uop  - user options (see listed in the audit report below)

July 2020 updates

 scan1d,scan2d,rep1d,rep2d updated to create audit reports in rpts/...
 - named as jobname_indir_oudir_searchpattern_replace_pattern_optins
 - showing command used to create report, making it easy to rerun
 New documentation at 'uvdemos1.htmPart_6'
 - vs old doc at scanjobs.htm & repjobs.htm
 - see July2020 new rep2d demo below

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6E2. rep2d - search/replace by table for all files in a directory

rep2d demo - July 2020


 #1. cdd   <-- alias to change to the demo/... subdir
     ===     - /home/uvsoftxx/demo/... for online to UVsoftware website
             - /home/userxx/demo/..... for offline on user machines

 #2. uvcopy rep2d,fild1=dat1,fild2=tmp2,fili3=tf/rep2d.tbl [,filo4=audit-file]
     ============================================================================
     - copy files from dat1/* to tmp2/... using search/replace table tf/rep2d.tbl
     - audit file defaults to rpts/rep2d_dat1_tmp2_rep2d.tbl

audit report created in rpts/rep2d_indir_outdir_tablefile

 Report: rpts/rep2d_dat1_rep2d.tbl
 -----> uvcopy rep2d,fild1=dat1,fild2=tmp2,fili3=tf/rep2d.tbl
        ============================================================================
 Format: uvcopy rep2d,fild1=indir,fild2=outdir,fili3=SearchReplaceTableFile
              ,arg6=searchbgn,arg7=searchend,arg8=MatchOptions,uop=UserOptions
 MatchOptions: i=Insensetive, q3=notinquotes, p=patternchars(@=alpha,=num,etc)
 UserOptions:  a0=BgnScan at bgnfile or arg6 if spcfd, a=BgnScanLine
              b0=EndScan at endfile or arg7 if spcfd, b=EndSscanLine
              c0=BgnScan at dsplcmnt 0 or cdsp, d0=EndScan at 256 or ddsplcmnt
              p1=qualifier1 Present & qualifier2 Absent
 Date=2020/07/21_18:16:10, Host=uvsoft5, User=uvadm, OptionOverrides=
 ====================================================================================
                     ** Table-File = tf/rep2d.tbl **
 North Vancouver~~~~~~~~~~~~~~~South Vancouver~~~~~~~~~~~~~~~BC~~~~~~~~~~~~~~~~~~Liberal~~~~~~~~~~~~~
 New Mexico~~~~~~~~~~~~~~~~~~~~OLD Mexico~~~~~~~~~~~~~~~~~~~~NM~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Richmond~~~~~~~~~~~~~~~~~~~~~~Poormond~~~~~~~~~~~~~~~~~~~~~~VA~~~~~~~~~~~~~~~~~~BC~~~~~~~~~~~~~~~~~~
 ====================================================================================================
 00036=032  NM  OLD Mexico           Albuquerque city                           545,852
 00102 098  VA  Virginia             Poormond city                              204,214
 00305                   2 reps @EOF: tmp2/UScities
 00032 NM                  OLD Mexico          Santa Fe            67947
 00047 VA                  Virginia            Poormond            204214
 00051                   2 reps @EOF: tmp2/USstates1
 00001 Owen Townsend       4667 Hoskins Rd     South Vancouver, BC Canada V7K2R3
 00008                   1 reps @EOF: tmp2/nameadrs1
 ========================================================== 2020/07/21_18:16:14
  EOD: 00007 reps in 0004 files of 0016 total (01460 lines)
 Audit filename: rpts/rep2d_dat1_rep2d

'alldiff2' to verify search/replacements


 #3. alldiff2 dat1 tmp2   <-- verify changes as intended & nothing unintended
     ==================     - will match the audit file shown above

 #4. cp tmp2/* dat1       <-- copy back overwriting original files
     ==============

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6F0. Vancouver Utility Programming Aids - TableJobs

TableJobs Benefits to Programmers & Analysts

  1. TableJobs build summary tables of counts & values to be dumped to a report file at EOF. There are various TableJobs for various file-types (text,fixed,variable), but we think you will find 'table3d' for text files most useful to supply you with statistics from directories of programs, scripts, parameter files, etc.

  2. You specify a Key-Word preceding the Target-Word to be table summarized. For example, given a directory of JCLs, you could specify KeyWord "PGM" preceding the program name in statements such as "// EXEC PGM=programxx".

  3. For FTP scripts, you could table the IP#s following the KeyWord "OPEN", or table the userids & passwords following the KeyWord "USER". For SQL scripts you could table the table-name following the KeyWord "FROM".

  4. You can also create table summary reports from fixed or variable length data-files which may have packed/binary fields.

  5. You can use Tablejobs to research the impacts of planned changes to your scripts & programs.

  6. TableJobs create reports in subdir stats/... of the current working directory, automatically named from the file or directory concatenated with Keywords used. This allows you to review table history & easily modify & repeat.

  7. The table reports include all relevant info (files,dirs,Keywords,date/time, user,host,etc) to help you document problems & plan proposed changes.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6F1. TableJobs - create Table Summaries via Keywords/Targetwords

Example#1 - table summarize all IP#s in FTP scripts

For example, 'table3d' can read all files in the ftps/ directory, searching for KeyWord "open" & count occurrences of the following TargetWord (the IP#).


 #1. cdd   <-- alias to change to the demo/... subdir
     ===     - /home/uvsoftxx/demo/... for online to UVsoftware website
             - /home/userxx/demo/..... for offline on user machines

 #2. mkdir stats  <-- make subdir for table3d output reports
     ===========      OR --> rm stats/*  <-- clear reports from old projects

 #3. uvcopy table3d,fild1=ftps,arg1=open
     ===================================
      - read ftps/* search for "open" & count occurrences of following IP#
      - output report created in subdir stats/... & automatically named by concatenating
        directory + keywords separated by underscores
      #Report: /home/uvadm/demo/stats/table3d_ftps_open
      #------> uvcopy table3d,fild1=ftps,arg1=open
      # ==============================================================================
      # - scan all files in directory for a keyword & table counts of following word
      # InDir=ftps  Keyword=open  Qual1=   Qual2=
      # WordSepsBlank=  Options=q1a0b3c0d0f0j0l0m0p1s0t0x0w1
      # - default selects following word, use option w2 to table 2nd word, etc
      # - specify arg1=Keyword,arg2=Qualifier-present,arg2=Qualifier-absent
      # &\,$=/>-<:(.');_*[|]"<--- arg4=punctuation to blank for word separation
      # abcdefghjklpqrsuALPRQ<--- equivalents, easier to enter, may enter "all"
      # Date=2020/07/24_15:43:23, Site=UV_Software, Host=uvsoft5, User=uvadm
      #===============================================================================
      #---> uvcopy table3d,fild1=ftps,arg1=open
      table3d  2020/07/24_15:43:27_209  Counts by Targetword following specified Keyword
      tbl#0001  tblt1f7 e0(48)       <----- argument ----->
      line#   1strec#  %      count  target-word
          1         2  25         1   192.168.0.4
          2         2  25         1   192.168.0.5
          3         2  50         2   192.168.0.6
                      100         4*  *TOTAL*

Notes, Benefits of table3d reports

  1. Output reports are automatically created in subdir stats/... named by concatenating the input directory & keywords separated by '_' underscores. The Report filename is coded on the 1st line of the report.

  2. The uvcopy command used to create the report is coded on the 2nd line, so you can look back at previous runs & repeat them (with modifications as desired).

  3. The reports are date/time stamped with all relevent info (site,host,user,options).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G1. Search/Replace Beyond Unix System Tools

Mass Change Project - all "UVdemos.doc" to "UVdemos2.doc"

Here is a real 'Mass Change Project' that was required at UV Software, when we added this document UVdemos1.htm as a preliminary to the existing UVdemos.htm. It then made sense to change the existing document name to UVdemos2.htm.

The Vancouver Utilities documentation is 325,518 lines in 152 text files. UVdemos.htm was just 1 of the 152 files, but there were hundreds of references to various pages from many other documents.

Note that the .htm files are just text files created with the 'vi' editor and converted to HTML .htm files for the UV Software website (automatically by the 'uvcopy' utility).

Also note that the change was somewhat complicated by the convention of allowing the .htm text files to contain some UPPER case letters (UV), but the HTML files are always all lower case. So the filename changes desired were:

Most of the references would contain a page code, for example:

Such references need to be changed to:

Before attempting mass changes, we should determine the number of changes required, and we can do this with the table analysis & scan tools.

Do not Execute - but Do understand this Example

This documents how I made a mass change to my Vancovuer Utilities, but do not attempt to re-execute. The documentation has already been updated & the $UV directory may not be available to the online demos.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G2. Mass Change all UVdemos.xxx to UVdemos2.xxx

Table Analysis to count changes rquired

'uvcopy table3d' can search a directory & count all occurrences of specified patterns.


 #1. cd $UV       <-- change to Vancouver Utilities homedir with subdir doc/...
     ======         - /home/uvadm/ offline OR /home/uvsoft__/uvadm/ online

 #2. mkdir stats  <-- make subdir for table3d output reports
     ===========      OR --> rm stats/*  <-- clear reports from old projects

 #3. uvcopy table3d,fild1=doc,arg1=UVdemos:uvdemos,arg4=all,uop=b1w0
     ===============================================================
      - search all files in doc/* & count occurrences of 'UVdemos' & 'uvdemos'
      - creates report in subdir stats/... as shown below:
 #Report: /home/uvadm/stats/table3d_doc1_UVdemos:uvdemos_all_b1w0
 #------> uvcopy table3d,fild1=doc1,arg1=UVdemos:uvdemos,arg4=all,uop=b1w0
 # ==============================================================================
 # - scan all files in directory for a keyword & table counts of following word
 # InDir=doc1  Keyword=UVdemos:uvdemos  Qual1=   Qual2=
 # WordSepsBlank=&\,$=/>-<:(.');_*[|]"  Options=q1a0b3c0d0f0j0l0m0p1s0t0x0w1b1w0
 # - default selects following word, use option w2 to table 2nd word, etc
 # - specify arg1=Keyword,arg2=Qualifier-present,arg2=Qualifier-absent
 # &\,$=/>-<:(.');_*[|]"<--- arg4=punctuation to blank for word separation
 # abcdefghjklpqrsuALPRQ<--- equivalents, easier to enter, may enter "all"
 # Date=2020/07/24_09:58:50, Site=UV_Software, Host=uvsoft5, User=uvadm
 #===============================================================================
 #---> uvcopy table3d,fild1=doc1,arg1=UVdemos:uvdemos,arg4=all,uop=b1w0
 table3d  2020/07/24_09:58:58_503  Counts by Targetword following specified Keyword
 tbl#0001  tblt1f7 e0(48)       <----- argument ----->
 line#   1strec#  %      count  target-word
     1       133  41       174   UVdemos      <-- to be changed to --> UVdemos2
     2       164  18        77   UVdemos1
     3         2  17        73   UVdemos2
     4       738  19        81   uvdemos      <-- to be changed to --> uvdemos2
     5       166   1         8   uvdemos1
     6       193   2        10   uvdemos2
                 100       423*  *TOTAL*

Note the '#Report: ...' name on the 1st line includes the search args & options, And the 2nd line '#----> ...' shows the uvcopy command used, making it easy to later repeat the command (same or modified) if required.

'arg4=all' causes all(most) punctuation to be blanked, important because words are separated bases on spaces between, so we want to blank any preceding quote, and the '.' period following - uvdemos.htm#1A1 for example.

Options 'uop=b1w0' are critical, default b3 assumes blank Before & After words, 'b1' assumes Blank Before only, lack of 'b2' means do not assume Blank After so search for 'uvdemos' picks up 'uvdemos1' & 'uvdemos2' as well.

option 'w0' tables the search word itself (vs default 'w1' tables following word).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G3. Mass Change all UVdemos.xxx to UVdemos2.xxx

Find all occurrences of patterns to be changed

'uvcopy scan1d' will find all occurrences of the patterns to be changed:


 #1. cd $UV       <-- change to Vancouver Utilities homedir with subdir doc/...
     ======         - /home/uvadm/ offline OR /home/uvsoftxx/uvadm/ online

 #2. mkdir rpts  <-- make subdir for scan1d output reports
     ==========      OR --> rm rpts/*  <-- clear reports from old projects

 #3. uvcopy scan1d,fild1=doc,arg1=uvdemos,arg8=i
     ===========================================
      - search all files in doc/*, selecting lines with 'uvdemos' OR 'UVdemos'
        ('arg8=i' is the case-insensetive option)
      - creates report in subdir rpts/... as shown below:
 #Report: rpts/scan1d_doc_uvdemos_i
 # -----> uvcopy scan1d,fild1=doc,arg1=uvdemos,arg8=i
 #        ============================================================================
 #Format: uvcopy scan1d,fild1=indir,arg1=search,arg2=qual1,arg3=qual2,uop=UserOptions
 #                     ,arg6=searchbgn,arg7=searchend,arg8=MatchOptions
 #MatchOptnsArg8: i=Insensetive, q3=notinquotes, p=patternchars(@=alpha,#=num,etc)
 #UserOptions: a0=BgnScan at bgnfile or arg6 if spcfd, a#=BgnScanLine#
 #             b0=EndScan at endfile or arg7 if spcfd, b#=EndSscanLine#
 #             c0=BgnScan at dsplcmnt 0 or c# dsp#, d0=EndScan at 256 or d# dsplcmnt
 #             p1=qual1Present & qual2Absent, p2=bothPresent, p4=bothAbsent
 # Date=2020/07/23_08:45:58, Host=uvsoft5, User=uvadm, Option-Overrides=
 #====================================================================================
 00671 # - see uvsoftware.ca/uvdemos.htm#1B5 more about RUNLIBS/RUNDATA stub/common profiles
 00673 # 4. Programmer testing/development projects, Example uvsoftware.ca/uvdemos.htm#Part_9
 11398rt: rpts/scan1d    2 hits @EOF: doc/ADMjobs.doc
 00091 #04. https://uvsoftware.ca/uvdemos.htm
 00099      - 'UVdemos' is a great place to start exploring Vancouver Utilities
 01186                   2 hits @EOF: doc/JCLcnv0brief.doc
              ----------- 450 lines omitted --------------
 00302 https://uvsoftware.ca/uvdemos.htm
 00310  - 'UVdemos' is a great place to start exploring Vancouver Utilities
 02295                   2 hits @EOF: doc/jclcnv_uvdemo.doc
 00345 See https://uvsoftware.ca#uvdemos.htm for many powerful uvcopy utilities to select
 04086                   1 hits @EOF: doc/uvhelp.doc
 =======================================================2020/07/23_08:46:04======
 EOD: 424 hits in 19 files of 152 total (325518 lines)

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G4. Mass Change all UVdemos.xxx to UVdemos2.xxx

use 'rep2d' to make the mass changes

'uvcopy rep2d' applies a table-file with multiple search/replace patterns while copying all files from 1 directory to a 2nd directory.


 #1. cd $UV       <-- change to Vancouver Utilities homedir with subdir doc/...
     ======         - /home/uvadm/ offline OR /home/uvsoftxx/uvadm/ online

 #2. mkdir rpts  <-- make subdir for scan1d output reports
     ==========      OR --> rm rpts/*  <-- clear reports from old projects

 #3. vi tf/rep2d.tbl_UVdemos2   <-- create table of search/replace patterns
     ========================     - as shown below:
      # rep2d.tbl_UVdemos - table file for uvcopy rep2d mass change to VU documentation
      # 001-030 - search pattern (ended by 1st tilde)
      # 031-060 - replacement pattern (ended by 1st tilde)
      # 061-080 - optional PRESENT qualifier
      # 081-100 - optional NOT-PRESENT qualifier
      UVdemos.~~~~~~~~~~~~~~~~~~~~~~UVdemos2.~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      uvdemos.~~~~~~~~~~~~~~~~~~~~~~uvdemos2.~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      ~~~~~~~~~~~~~~~~~~~~~~~~~ (end of table marker) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 #4. mkdir doc2      <-- make output directory
     ==========

 #5. uvcopy rep2d,fild1=doc,fild2=doc2,fili3=tf/rep2d.tbl_UVdemos2
     =============================================================
      - copy all files from doc/* to doc2/... while searching/replacing
        with table-file tf/rep2d.tbl_UVdemos2
      - creates report in subdir rpts/... as shown below:

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

6G5. Mass Change all UVdemos.xxx to UVdemos2.xxx

use 'rep2d' to make the mass changes


 #6. vi rpts/rep2d_doc_rep2d.tbl_UVdemos2
     ====================================
      - check the search/replace audit report
 #Report: rpts/rep2d_doc_rep2d.tbl_UVdemos2
 # -----> uvcopy rep2d,fild1=doc,fild2=doc2,fili3=tf/rep2d.tbl_UVdemos2
 #===================================================================================================
 #Format: uvcopy rep2d,fild1=indir,fild2=outdir,fili3=SearchReplaceTableFile
 #              ,arg6=searchbgn,arg7=searchend,arg8=MatchOptions,uop=UserOptions
 #MatchOptions: i=Insensetive, q3=notinquotes, p=patternchars(@=alpha,#=num,etc)
 #UserOptions:  a0=BgnScan at bgnfile or arg6 if spcfd, a#=BgnScanLine#
 #              b0=EndScan at endfile or arg7 if spcfd, b#=EndSscanLine#
 #              c0=BgnScan at dsplcmnt 0 or c# dsp#, d0=EndScan at 256 or d# dsplcmnt
 #              p1=qualifier1 Present & qualifier2 Absent
 # Date=2020/07/23_09:02:49, Host=uvsoft5, User=uvadm, OptionOverrides=
 #===================================================================================================
                     ** Table-File = tf/rep2d.tbl_UVdemos2 **
 UVdemos.~~~~~~~~~~~~~~~~~~~~~~UVdemos2.~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 uvdemos.~~~~~~~~~~~~~~~~~~~~~~uvdemos2.~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 ====================================================================================================
 00671=# - see uvsoftware.ca/uvdemos2.htm#1B5 more about RUNLIBS/RUNDATA stub/common profiles
 00673 # 4. Programmer testing/development projects, Example uvsoftware.ca/uvdemos2.htm#Part_9
 11398                   2 reps @EOF: doc2/ADMjobs.doc
 00091 #04. https://uvsoftware.ca/uvdemos2.htm
 01186                   1 reps @EOF: doc2/JCLcnv0brief.doc
                 ------------- 175 lines omitted ---------------
 01180 For example see uvsoftware.ca/uvdemos2.htm#1A1. You could also use for
 02386 UVdemos2.doc - Brief descriptions of uvhd,uvsort,uvcp,uvlist,& uvcopy + tutorials
 02399              - addition to UVdemos2.doc
 03174                   3 reps @EOF: doc2/install.doc
 00345 See https://uvsoftware.ca#uvdemos2.htm for many powerful uvcopy utilities to select
 04086                   1 reps @EOF: doc2/uvhelp.doc
 ============================================================ 2020/07/23_09:03:01
  EOD: 157 reps in 18 files of 152 total (325518 lines)
 Audit filename: rpts/rep2d_doc_rep2d

Notes

You might ask why the 'total hits' from previous 'scan1d' audit report do not match the 'total reps' shown here for 'rep2d' ?

      EOD: 424 hits in 19 files of 152 total (325518 lines)  <-- previous 'scan1d'
      EOD: 157 reps in 18 files of 152 total (325518 lines)  <-- above 'rep2d'

Because we intentionally only inserted the '2' in 'UVdemos.xxx' or 'uvdemos.xxx' on the HTML hyperlink references, but not on general references to just 'UVdemos' which is OK. Main intention was to avoid any hyperlink "file not found"s.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Part_7. uvcopy solutions to problems at Unix/Linux sites

Part 7 will demonstrate some of the many powerful features of uvcopy that might provide solutions to the various problems that come up at any computer site. Our intention is to illustrate uvcopy capabilities that go beyond Unix/Linux utilities.

Please study & run the demos provided. You might see similarities to problems you have experienced in the past & we hope uvcopy can help you with solutions in the future.

UVdemos1 - Part 7 - Contents


7A1. expandvars1 - copy text files expanding unix ENVIRONMENTAL $VARIABLEs
- used to expand $Variables in DB2 SQL scripts
- you might find usefull to front-end some of your scripts

7A2. testevs1 - uvcopy job that to test the 'evs' instruction
- displays common variables $HOME, $LOGNAME, etc & their values
- then prompts you to enter any other $Variables you can think of

7B1. UScongresstbl3 - uvcopy job with 2 tables (could have max 255 in 1 job).
- Congress members by Party & by Party & Sex

7C1. Translate tables - powerful tool that can be used for a lot more than
translating code sets (Ascii to Ebcdic, Ebcdic to Ascii, etc).
We will illustrate how you can use them to solve problems in your data files.

7C2. Translate tables to print files with binary characters
- illustrated with uvcopy job 'printable1' & testdatafile dat3/warranty_master

7C3. Listing of the uvcopy job to translate unprintables to '.' periods
- $HOME/demo/pf/printable1

7C4. Listing of the translate table to translate unprintables to '.' periods
- $UV/srcf/trtprint.c

7C5. Fix problems such as bad data in incoming files to your applications
For example, datafiles with "`" back-quotes instead of "'" single-quotes
Modify a neutral translate table, replacing bad data bytes with desired data bytes

7C6. Listing of uvcopy copy job to translate back-quotes to single-quotes
- $HOME/demo/pf/fixquote1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

UVdemos1 Part 7 - Contents (continued)


7D1. Indexed Files - typical example is a product master file where the indexed
key-field is the product# - used to lookup the product description & price
 using product# from a sales_item file.

7D2. extendsales1 - uvcopy job to copy the sales_item file, extending with the
product price & description from the product master INDEXED file

7D3. Execute extendsales1 (Indexed file method) showing the input & output files.

7E1. extendsales2 - Table LOAD & Table LOOKUP - alternate method of extending
the sales_item file with product description & price.

7E2. Execute extendsales2 (Table Load/Lookup method) showing the I/O files.

7E3. uvcopy debugger - sample session stepping thru execution of extendsales2,
displaying data areas & registers.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7A1. uvcopy solutions to problems at Unix/Linux sites

expandvars1 - copy files expanding $VARIABLEs

 # expandvars1 - copy a text file expanding unix ENVIRONMENTAL $VARIABLEs
 #             - copy any file expanding $SYMBOLs (in unix scripts, etc)
 #             - by Owen Townsend, UV Software, March 2018
 #
 # uvcopy expandvars1,fili1=$UV/tf/testexpandvars1,filo1=tmp/testexpandvars1x
 # ==========================================================================
 #  - demo with test file provided, INPUT/OUTPUT shown below:
 #
 # HOME=$HOME HOSTNAME=${HOSTNAME} LOGNAME=$LOGNAME SHELL=${SHELL}  <-- INPUT
 # HOME=/home/uvadm HOSTNAME=uvsoft5 LOGNAME=uvadm SHELL=/bin/bash  <-- OUTPUT
 #
 rop=r1    # option prompt to view output file at EOF
 fili1=?${UV}/tf/testexpandvars1,rcs=256,typ=LST
 filo1=?tmp/testexpandvars1x,rcs=256,typ=LSTt
 @run
         opn    all
 # begin loop to copy expanding $variables
 man20   get    fili1,a0                 get next line
         skp>   man90                    (cc set > at EOF)
         mvc    c0(200),a0               copy in case expand inhibited by '#'
         cmc    a0(1),'#'                #cmts pass thru w/o expanding $VARs ?
         skp=   man40
         evs    b0(200),a0(200)      <-- copy expanding EXTERNAl $VARIABLEs ($HOME,$PWD,etc)
         mvfv1  c0(200),b0(200)        - also expand uvcopy INTERNAL $variables ($date,$time,etc)
 man40   put    filo1,c0                 write to output file
         skp    man20                    repeat loop until EOF
 # EOF - close files & end job
 man90   cls    all
         eoj

Execute expandvars1 with demo file default


 #1. cdd   <-- alias to change to the demo/... subdir
     ===     - /home/uvsoftxx/demo/... for ssh to UVsoftware website
             - /home/userxx/demo/..... for logins on user machines

 #2. uvcopy expandvars1  <-- easy demo, files default as shown above on fili1/filo1
     ==================

 #3. cat tmp/testexpandvars1x  <-- display output, just 1 line I/O shown below
     ========================    - may also be some other #comment lines
     # HOME=$HOME HOSTNAME=${HOSTNAME} LOGNAME=$LOGNAME SHELL=${SHELL}  <-- INPUT
       HOME=/home/uvadm HOSTNAME=uvsoft5 LOGNAME=uvadm SHELL=/bin/bash  <-- OUTPUT

expandvars1 used in DB2 SQL files

See uvdemos.htm#7F4 - expandvars1 was used to expand $Variables in SQL file input for table unloads in JCL conversions. We needed to redefine the $SYSREC00 output file defined in the JCL/script & the 'db2' utility does not do that.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7A2. uvcopy solutions to problems at Unix/Linux sites

testevs1 - uvcopy job to test 'evs' instruction

Note that 'evs' is the critical instruction in the expandvars1 job listed above. 'evs' is just 1 of the 100+ instructions in uvcopy - see uvcopy3.htm#evs.

'$UV/pf/adm/testevs1' is a uvcopy job that was written to test the 'evs' instruction when it was added to the uvcopy utility. Here is the listing with run instructions:

 # testevs1 - test uvcopy instruction 'evs' (test job in $UV/pf/adm/testevs1)
 #          - copy a string expanding environmental-variables
 #          - also see $UV/pf/util/expandvars1 expand $VARIABLEs in any file
 #
 # uvcopy testevs1    <-- execute this job to test the "evs" instruction
 # ===============      - shows a BEFORE & AFTER expansion demo
 #                      - then prompts you to enter your own $Variables
 @run
         msg     '  --- test uvcopy instruction "evs" copy text expanding $VARIABLES ---'
         mvf     a0(200),'homedir=$HOME, user=$LOGNAME, shell=$SHELL, date=$date       <-- demo INPUT'
         msg     a0(200)                show string BEFORE expanding variables
         evs     b0(200),a0(200)    <-- copy from 'a' to 'b' expanding external $VARIABLEs
         mvfv1   c0(200),b0(200)      - also expand uvcopy internal $variables ($date,$time,etc)
         rep     c0(200),'      <-- demo INPUT','<-- demo OUTPUT'  # correct for OUTPUT
         msg     c0(200)                show string AFTER expanding variables
 # Now prompt user to enter any text with $Variables to be expanded
 loop    msga1l1w  ' --- Enter your text containing $VARIABLEs (ex: $HOME, $PWD, $date, etc)'
         skp<    EOF                    goto end job if null entry
         mvf     a0(200),$arg1
 #       ===
         evs     b0(200),a0(200)    <-- copy from 'a' to 'b' expanding $variables
 #       ===
         mvfv1   c0(200),b0(200)      - also expand uvcopy internal $variables ($date,$time,etc)
         msg     c0(200)                show string AFTER expanding variables
         msg     '     Enter another string OR null entry to end job'
         skp     loop
 EOF     eoj

 uvcopy testevs1   <-- Execute 'testevs1'
 ===============     - sample output shown below
   --- test uvcopy instruction "evs" copy text expanding $VARIABLES ---
 homedir=$HOME, user=$LOGNAME, shell=$SHELL           <-- demo INPUT
 homedir=/home/uvtest, user=uvtest, shell=/bin/bash   <-- demo OUTPUT
  --- Enter your text containing $VARIABLEs (ex: $HOME, $PWD,$PS1, etc)
 \$PWD=$PWD  $\HOSTNAME=$HOSTNAME  \$EDITOR=$EDITOR   <-- YOUR INPUT
 \$PWD=/home/uvtest  $\HOSTNAME=uvsoft5  \$EDITOR=vi  <-- YOUR OUTPUT
      Enter another string OR null entry to end job
  --- Enter your text containing $VARIABLEs (ex: $HOME, $PWD,$PS1, etc)

The test run above demonstrates that you can inhibit the $EXPANSION by \escaping the $VARIABLE with the '\' escape character (same as unix).

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7B1. Table Summary Instructions tbl & tbp

We have already given examples of 'tbl' & 'tbp' on pages '4D1' - '4D3', but we thought it might be useful to give an example of multiple tables in 1 uvcopy job.

demo/pfyy/UScongresstbl3 - sample job with 2 'tbl's

 # UScongresstbl3 - table summarize US Congress members by party & party + sex
 #                - by Owen Townsend, UV Software, July 2020
 rop=r1    #<-- EOF option prompt to view output file (reply vi,cat,more,etc)
 fili1=?dat1/UScongress,typ=LST,rcs=512    # '?' prompts for change, code your input file
 filo1=?tmp/$jobname,typ=LSTt,rcs=512      # write to tmp/$jobname = tmp/UScongresstbl3
 @run
         opn     all                       open files
 # begin loop to get & put records until EOF
 getrec  get     fili1,a0                  get record into area 'a'
         skp>    eof                       (cc set > at EOF)
 #-----------------------------
         tblt1f1 a62(15),'Members by Party'
         mvc     a75(1),a50                move F/M to end of party field
         tblt2f1 a62(15),'Members by Party & Sex'  table by Party & Sex
 #-----------------------------
         skp     getrec                    return to get next record
 # EOF - dump table, close files,& end job
 eof     tbpt1   filo1,'US Congress Members summarized by Party'
         tbpt2   filo1,'US Congress Members summarized by Party & Sex'
         cls     all                       close files
         eoj                               end job

Execute job to summarize provinces & populations


 #1. uvcopy pfyy/UScongresstbl3   <-- Execute
     ==========================

 #2. cat tmp/UScongress.tbl3      <-- display output file
     =======================
 UScongresstbl3  2020/07/31_10:23:23_065  US Congress Members summarized by Party
 tbl#0001  tblt1f1 a62(15)      <----- argument ----->
 line#   1strec#  %      count  Members by Party
     1         2  52       284  Democrat
     2       265   0         2  Independent
     3         1  46       253  Republican
                 100       539*  *TOTAL*
 UScongresstbl3  2020/07/31_10:23:23_065  US Congress Members summarized by Party & Sex
 tbl#0002  tblt2f1 a62(15)      <----- argument ----->
 line#   1strec#  %      count  Members by Party & Sex
     1         2  20       108  Democrat     F
     2         4  32       176  Democrat     M
     3       265   0         2  Independent  M
     4         9   4        23  Republican   F
     5         1  42       230  Republican   M
                 100       539*  *TOTAL*

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7B2. Table Summary Instructions tbl & tbp

Input file for UScongresstbl3 table job


 #3. more dat1/UScongress   <-- display input data file for UScongresstbl3
     ====================
              ----------- 1st 3 records of 539 -----------
         10        20        30        40        50        60        70        80
 12345678901234567890123456789012345678901234567890123456789012345678901234567890
 001  Abraham             Ralph          19540916  M  LA  rep  Republican
 002  Adams               Alma           19460527  F  NC  rep  Democrat
 003  Aderholt            Robert         19650722  M  AL  rep  Republican

Notes Re: tbl & tbp

  1. The example above shows a uvcopy job with 2 tables, but you could have up to 255 tables in 1 job. Default allows 10 tables, increase via rop=u99. Default allows 4000 table entries for all tables, increase via rop=v9999.

  2. This 'tbl' example is for a custom written uvcopy job, but you do not have to write any uvcopy code to use this powerful feature. 'You can use 'table3d', a pre-programmed utility to search for a keyword & table the following word. Here is the example from page '6F1' which searches FTP scripts for 'open' & tables the IP# following, But adding option 'f1' to prepend the target-word with the current filename. (could use 'uop=f2' to APPEND vs PREPEND.


 #2a. uvcopy table3d,fild1=ftps,arg1=open,uop=f1
      ==========================================
     #Report: /home/uvadm/demo/stats/table3d_ftps_open_f1
     #------> uvcopy table3d,fild1=ftps,arg1=open,uop=f1
     table3d  2020/07/31_14:43:56_434  Counts by Targetword following specified Keyword
     tbl#0001  tblt1f7 e0(48)       <----- argument ----->
     line#   1strec#  %      count  target-word
         1         2  25         1  ftpdemo1a 192.168.0.4
         2         2  25         1  ftpdemo1b 192.168.0.5
         3         2  25         1  ftpget1 192.168.0.6
         4         2  25         1  ftpput1 192.168.0.6
                     100         4*  *TOTAL*
  1. See many more examples of table3d in uvsoftware.ca/uvdemos2.htm#Part_4

  2. See detailed documentation for 'tbp' & 'tbp' at uvsoftware.ca/uvcopy3.htm#tbl

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7C1. uvcopy solutions to problems at Unix/Linux sites

translate tables

Translate tables are a very powerful tool that can be used for a lot more than translateing code sets (Ascii to Ebcdic, Ebcdic to Ascii, etc). We will illustrate how you can use them to solve problems in your data files.

The Vancouver Utilities include several translate tables in $UV/srcf/... such as:


 source     uvcopy-symbol   Description
 =============================================
 asc2ebc.c    $trta2e       ASCII to EBCDIC
 ebc2asc.c    $trte2a       EBCDIC to ASCII
 trtprint.c   $trtper       translate all unprintable characters to '.' periods
 trtunder.c   $trtunder     translate all unprintable characters to '_' underscores
 trtneut.c    $trtneut      Neutral translate table

These tranlate tables are compiled into uvcopy for use by several instructions

tra
tre
trl
tru
trt
  • translate op1 area using translate table specified by op2 uvcopy3.htm#trt
  • op2 translate table could be $trta2e,$trte2a,$trtper,$rtrunder,$trtneut(modified)
  • can copy $trtneut & modify for customized translations to scrub your data

 l $UV/srcf   <-- list translate table names in $UV/srcf/...
 ==========

 vi $UV/srcf/trtprint.c      <-- investigate translate table source code
 ======================        - trtprint.c $trtrper in uvcopy job below

    trt  b0(4096),$trtper    <-- translate instruction from uvcopy job
 =========================     - see uvcopy job 'printable1' on page '7C3'

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7C2. uvcopy solutions to problems at Unix/Linux sites

translate tables to print files with binary characters

Here is an example of trying to print a file with packed/binary characters:


 #1. cat dat3/warranty_master   <-- try 'cat' or other unix system utilities
     ========================       to print/display files with binary/paccked fields
 12345 12345 sony television   950101 1-year 	` 00000291qP
 22222 11111 refrigerator      960101 2-year 	 000004922`
 33333 22222 dish washer       961231 3-year 	# 00000673sa#
 33333 11111 refrigerator      970101 3-year  000006744p
 44444 33333 freezer           970101 4-year  00000995up
 55555 55555 clothes dryer     991231 5-year !# 000055566#
 66666 66666 blank dates              6-year
                                             000066600 <-- x'0C' Breaks Line
 77777 77777 zero/blank dates  000000 7-year
                                             00007770p
 99999 99999 nines dates       999999 7-year	 00009999y
Note
  • packed field signs x'0C' are the worst problem, since that is a FormFeed
  • causes line break on screen & worse if sent to printer
  • causes skip to next page, often causing 1 line per page

 #2. uvcopy printable1,fili1=dat3/warranty_master  <-- print with uvcopy job
     ============================================
Note
  • $UV/pf/util/printable1 is a uvcopy job using a translate table to translate
    packed/binary charcters to '.' PERIODs, output file in tmp/...

 #3. cat tmp/warranty_master   <-- display 'cleaned-up' copy of dat3/warranty_master
     =======================     - with any unprintable chars converted to '.' periods
 12345 12345 sony television   950101 1-year .`.. 00000291q..P..
 22222 11111 refrigerator      960101 2-year .... 000004922..`..
 33333 22222 dish washer       961231 3-year ..#. 00000673s..a#.
 33333 11111 refrigerator      970101 3-year .... 000006744..p..
 44444 33333 freezer           970101 4-year .... 00000995u..p..
 55555 55555 clothes dryer     991231 5-year .!#. 000055566...#.
 66666 66666 blank dates              6-year .... 000066600.....
 77777 77777 zero/blank dates  000000 7-year .... 00007770p.....
 99999 99999 nines dates       999999 7-year .... 00009999y.....

 #4. uvhd /home/uvadm/demo/tmp1/wm t   <-- uvhd to show rec#7 with 1st Line Break
     ===============================
      rec#=7 rsize=64 fptr=384 fsize=576 records=9
                        10        20        30        40        50        60
      r#     7 0123456789012345678901234567890123456789012345678901234567890123
           384 66666 66666 blank dates              6-year .... 000066600......
               3333323333326666626676722222222222222327667200002333333333000090
               6666606666602C1EB041453000000000000006D95120000C00000666000001CA
Note
  • byte# 47 x'0C' packed zero + sign same as FormFeed caauses line/page breaks

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7C3. translate tables to print files with binary characters

printable1 - uvcopy job to print files with binary chars


 #5. cat $UV/pf/util/printable1   <-- inspect uvcopy job to copy files translating
     ==========================       any unprintable chars to '.' periods
 # printable1 - uvcopy Parameter File from UVSI stored in: /home/uvadm/pf/util/...
 # printable1 - copy a file, translating any packed/binary chars to '.' periods
 #            - by Owen Townsend, UV Software, Jan2014, update Jul2020
 #
 # pf/util/printable1 - for text files with LineFeeds (see printable2 if no LineFeeds)
 # pf/util/listrec1   - alternate job for text files that lists in 100 byte segments
 # sf/util/printable1 - script using 'tr', vs uvcopy job, same name in sf/util vs pf/util
 #
 opr='$jobname - copy a file, translating any unpritable characters to periods'
 opr='         - default output file to tmp/... with same name as input file'
 rop=r1   # EOJ prompt to view output file (vi,cat,more,lp,uvlp13,etc)
 fili1=?input,typ=LST,rcs=4096
 filo1=?tmp/$fili1,typ=LSTt,rcs=4096
 @run
        opn    all                      open files
 # begin loop to get & put records until EOF
 getrec get    fili1,a0                 get record into area 'a'
        skp>   eof                      (condition code set > at EOF)
        mvc    b0(4096),a0              move input record to output area 'b'
 #-------------------------------
        trt    b0(4096),$trtper         translate any unprintables to '.' periods
 #-------------------------------
        put    filo1,b0                 write record to output file
        skp    getrec                   return to get next record
 eof    cls    all                      close files
        eoj                             end job

Notes re the 'trt' instruction

  1. See the translate table listed on the next page. Source code ($UV/srcf/trtprint.c) is compiled into uvcopy & can be addressed as '$trtper' (shown above on trt instruction).

  2. 'trt' replaces each byte of op1 with the contents of a byte in the translate table whose address is calculated by adding the binary value of the contents to the base address of op2 which must be a 256 byte area (translate table).

  3. In the ASCII character set, the 1st 32 bytes with values x'00' - x'1F' are unprintable control codes & $trtper will be replace them with '.' periods

  4. The next 96 bytes with values x'20' - x'7F' are the printable ASCII characters & $trtper wuill replace them with contents from corresponding locations, BUT, these have the same values as the printable chars (so they will be unchanged).

  5. The Last 128 bytes are extended Ascii (foreign languages & grpahic characters). Op1 bytes with values x'80' - x'FF' will be replaced with the '.' periods stored at these locations.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7C4. uvcopy solutions to problems at Unix/Linux sites

$UV/srcf/trtprint.c - translate unprintables to periods

 /* trtprint.c - translate table - convert unprintable chars to periods */
 unsigned char trtprint[256] = {
 0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,
 0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,
 0x20,0x21,0x22,0x23,0x24,0x25,0x26,0x27,0x28,0x29,0x2A,0x2B,0x2C,0x2D,0x2E,0x2F,
 /*     !    "    #    $    %    &    '    (    )    *    +    ,    -    .    / */
 0x30,0x31,0x32,0x33,0x34,0x35,0x36,0x37,0x38,0x39,0x3A,0x3B,0x3C,0x3D,0x3E,0x3F,
 /*0    1    2    3    4    5    6    7    8    9    :    ;    <    =    >    ? */
 0x40,0x41,0x42,0x43,0x44,0x45,0x46,0x47,0x48,0x49,0x4A,0x4B,0x4C,0x4D,0x4E,0x4F,
 /*@    A    B    C    D    E    F    G    H    I    J    K    L    M    N    O */
 0x50,0x51,0x52,0x53,0x54,0x55,0x56,0x57,0x58,0x59,0x5A,0x5B,0x5C,0x5D,0x5E,0x5F,
 /*P    Q    R    S    T    U    V    W    X    Y    Z    [    \    ]    ^    _ */
 0x60,0x61,0x62,0x63,0x64,0x65,0x66,0x67,0x68,0x69,0x6A,0x6B,0x6C,0x6D,0x6E,0x6F,
 /*`    a    b    c    d    e    f    g    h    i    j    k    l    m    n    o */
 0x70,0x71,0x72,0x73,0x74,0x75,0x76,0x77,0x78,0x79,0x7A,0x7B,0x7C,0x7D,0x7E,0x2E,
 /*p    q    r    s    t    u    v    w    x    y    z    {    |    }    ~      */
 0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,
 0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,
 0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,
 0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,
 0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,
 0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,
 0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,
 0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E,0x2E
 };

How Translate Tables work

Input data bytes are replaced with the contents of a byte in the translate table whose address is calculated by adding the binary value of the input data byte to the base address of the translate table which is a 256 bytes (all possible values of an 8 bit byte - x'00' - x'FF', decimal 0 - 255).

So, for the trtprint.c traanslate table shown above:

  1. The ASCII control characters in the 1st 32 of the 256 byte translate table will be translated to PERIODs (coded as x'2E' to keep the table neat).

  2. The printable ASCII characters x'20' to x'7F' will be unchanged, since the table replacement is the same as the input data bytes.

  3. The extended ASCII characters x'80' to x'FF' will be translated to PERIODs

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7C5. uvcopy solutions to problems at Unix/Linux sites

fix problems by modifying a neutral translate table

You can use the neutral translate table to fix unique problems in the data inputs to your applications. For example, some incoming data might use "`" back-quotes instead of "'" single-quotes.

You can fix this problem with the uvcopy job listed on teh next page, but 1st we will show the neutral translate table & explain how you can modify it for the fix.

$UV/srcf/trtneut.c - Neutral Translate Table

 /* trtneut.c - translate table - neutral, use as basis to create others */
 unsigned char trtneut[256] = {
 0x00,0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09,0x0A,0x0B,0x0C,0x0D,0x0E,0x0F,
 0x10,0x11,0x12,0x13,0x14,0x15,0x16,0x17,0x18,0x19,0x1A,0x1B,0x1C,0x1D,0x1E,0x1F,
 0x20,0x21,0x22,0x23,0x24,0x25,0x26,0x27,0x28,0x29,0x2A,0x2B,0x2C,0x2D,0x2E,0x2F,
 /*     !    "    #    $    %    &    '    (    )    *    +    ,    -    .    / */
 0x30,0x31,0x32,0x33,0x34,0x35,0x36,0x37,0x38,0x39,0x3A,0x3B,0x3C,0x3D,0x3E,0x3F,
 /*0    1    2    3    4    5    6    7    8    9    :    ;    <    =    >    ? */
 0x40,0x41,0x42,0x43,0x44,0x45,0x46,0x47,0x48,0x49,0x4A,0x4B,0x4C,0x4D,0x4E,0x4F,
 /*@    A    B    C    D    E    F    G    H    I    J    K    L    M    N    O */
 0x50,0x51,0x52,0x53,0x54,0x55,0x56,0x57,0x58,0x59,0x5A,0x5B,0x5C,0x5D,0x5E,0x5F,
 /*P    Q    R    S    T    U    V    W    X    Y    Z    [    \    ]    ^    _ */
 0x60,0x61,0x62,0x63,0x64,0x65,0x66,0x67,0x68,0x69,0x6A,0x6B,0x6C,0x6D,0x6E,0x6F,
 /*`    a    b    c    d    e    f    g    h    i    j    k    l    m    n    o */
 0x70,0x71,0x72,0x73,0x74,0x75,0x76,0x77,0x78,0x79,0x7A,0x7B,0x7C,0x7D,0x7E,0x7F,
 /*p    q    r    s    t    u    v    w    x    y    z    {    |    }    ~      */
 0x80,0x81,0x82,0x83,0x84,0x85,0x86,0x87,0x88,0x89,0x8A,0x8B,0x8C,0x8D,0x8E,0x8F,
 0x90,0x91,0x92,0x93,0x94,0x95,0x96,0x97,0x98,0x99,0x9A,0x9B,0x9C,0x9D,0x9E,0x9F,
 0xA0,0xA1,0xA2,0xA3,0xA4,0xA5,0xA6,0xA7,0xA8,0xA9,0xAA,0xAB,0xAC,0xAD,0xAE,0xAF,
 0xB0,0xB1,0xB2,0xB3,0xB4,0xB5,0xB6,0xB7,0xB8,0xB9,0xBA,0xBB,0xBC,0xBD,0xBE,0xBF,
 0xC0,0xC1,0xC2,0xC3,0xC4,0xC5,0xC6,0xC7,0xC8,0xC9,0xCA,0xCB,0xCC,0xCD,0xCE,0xCF,
 0xD0,0xD1,0xD2,0xD3,0xD4,0xD5,0xD6,0xD7,0xD8,0xD9,0xDA,0xDB,0xDC,0xDD,0xDE,0xDF,
 0xE0,0xE1,0xE2,0xE3,0xE4,0xE5,0xE6,0xE7,0xE8,0xE9,0xEA,0xEB,0xEC,0xED,0xEE,0xEF,
 0xF0,0xF1,0xF2,0xF3,0xF4,0xF5,0xF6,0xF7,0xF8,0xF9,0xFA,0xFB,0xFC,0xFD,0xFE,0xFF
 };

Note that the "`" backquote Ascii value is x'60' & we can simply replace the character at this location with a single-quote - using the following instruction (extracted from the job listed on the next page).

 #-------------------------------
        mvc    c96(1),x'27'             replace back-quote with single-quote
 #-------------------------------

We use decimal displacement 96 since hex x'60' is (6*16) = 96.

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7C6. uvcopy solutions to problems at Unix/Linux sites

fixquote1 - copy translating back-quotes to single-quotes

 # fixquote1 - uvcopy Parameter File stored in: /home/uvadm/pf/demo/...
 # fixquote1 - copy a file translating any "`" back-quotes to "'" single-quotes
 #           - by Owen Townsend, UV Software, July 2020
 # demo/example - for users to copy/rename/modify to fix similar problems
 #
 opr='$jobname - copy a file translating any back-quotes to single-quotes'
 rop=r1   # EOJ prompt to view output file (vi,cat,more,lp,uvlp13,etc)
 fili1=?input,typ=LST,rcs=4096
 filo1=?tmp/$fili1,typ=LSTt,rcs=4096
 @run
        opn    all                      open files
        mvc    c0(256),$trt             copy neutral translate table to area 'c'
 #-------------------------------
        mvc    c96(1),x'27'             replace back-quote with single-quote
 #-------------------------------
 # begin loop to get & put records until EOF
 getrec get    fili1,a0                 get record into area 'a'
        skp>   eof                      (condition code set > at EOF)
        mvc    b0(4096),a0              move input record to output area 'b'
 #-------------------------------
        trt    b0(4096),c0(4096)        translate changing any back-quotes to single-quotes
 #-------------------------------
        put    filo1,b0                 write record to output file
        skp    getrec                   return to get next record
 eof    cls    all                      close files
        eoj                             end job

test trtfixquote1


 #1. cat testfixquote1    <-- investigate INPUT test-data-file (listed below)
     =================      - Note '`' back-quotes
      # testfixquote1 - test data file for uvcopy job `$UV/pf/demo/fixquote1'
      #    `fixquote1` copies a file translating any `back-quotes` to "single-quotes"
      # - demos modifying a `neutral` translate table to clean-up inputs to your applications

 #2. uvcopy fixquote1,fili1=tf/testfixquote1  <-- execute fixquote1 with testfile
     =======================================
      - output file will be in tmp/... with same filename as input by default
      - could append ",filo1=..." to specify outdir/outfile as desired

 #3. cat tmp/testfixquote1    <-- investigate OUTPUT test-data-file (listed below)
     =====================      - Note '`' back-quotes REPLACED by single-quotes
      # testfixquote1 - test data file for uvcopy job '$UV/pf/demo/fixquote1'
      #    'fixquote1' copies a file translating any 'back-quotes' to "single-quotes"
      # - demos modifying a 'neutral' translate table to clean-up inputs to your applications

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7D1. Indexed Files

A typical example of an Indexed file is a product master file where the indexed key-field is the product#. It can be used to lookup the product description & price using the product# from a sales_item file. Here are our demo files:

dat2/product_master - Indexed by product#

 # product_master - Indexed on product# looked-up for description & price
 #        1         2         3         4         5
 #2345678901234567890123456789012345678901234567890
 #prod#====Description===========price=============
 BBQ010    BAR-B-Q             0019500
 CHR015    LAWN CHAIR          0004500
 HAM010    CLAW HAMMER         0000950
 HAM035    JACK HAMMER         0029500
 SAW011    HAND SAW            0001975
 SAW012    RIP SAW             0002500
 SAW051    POWER SAW           0008500
 TAB012    LAWN TABLE          0008500

dat2/sales_items - sequential file

 # sales_items - to demo extendsales1 table load & table lookup
 #        1         2         3         4         5         6         7         8
 #2345678901234567890123456789012345678901234567890123456789012345678901234567890
 #cust#=slm===date====invoice==product====qty===price==amount===description======
 130140  44 20200801 INV11001  HAM010  000010
 130140  44 20200801 INV11001  CHR015  000020
 139923  21 20200802 INV11002  BBQ010  000010
 139923  21 20200802 CR5421    SAW051  000005
 250825  44 20200803 INV11003  HAM099  000004
 401210  21 20200804 INV11004  SAW011  000025

tmp/sales_itemsx1 - Extended with product price & description

 # sales_itemsx1 - to demo extendsales1 table load & table lookup
 #        1         2         3         4         5         6         7         8
 #2345678901234567890123456789012345678901234567890123456789012345678901234567890
 #cust#=slm===date====invoice==product===qty===price==ExtAmount=description======
 130140  44 20200801 INV11001  HAM010  000010 0000950 000009500 CLAW HAMMER
 130140  44 20200801 INV11001  CHR015  000020 0004500 000090000 LAWN CHAIR
 139923  21 20200802 INV11002  BBQ010  000010 0019500 000195000 BAR-B-Q
 139923  21 20200802 CR5421    SAW051  000005 0008500 000042500 POWER SAW
 250825  44 20200803 INV11003  HAM099  000004                    ** ERR product# INVALID **
 401210  21 20200804 INV11004  SAW011  000025 0001975 000049375 HAND SAW

We will use uvcopy job demo/pf/extendsales1 as follows:


 uvcopy extendsales1,fili1=dat2/sales_items,filr2=dat2/product_master,filo1=tmp/sales_itemsx1
 ============================================================================================
 - fili1=dat2/sales_items    - read sequentially
 - filr2=dat2/product_master - read Randomly using product# from sales_items records
 - filo1=tmp/sales_itemsx1   - written sequentially with price,amount,dscription added

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7D2. Indexed Files

extendsales1 - uvcopy using Indexed file

 # extendsales1 - copy sales items adding prices & descriptions from Indexed product master
 #              - to demo Indexed files
 #              - must 1st load product_master into an Indexed file as follows:
 #
 # uvcp "fili1=dat2/product_master,typ=LST,rcs=100,filo1=dat2/product_masterI,typ=ISF,rcs=99,isk1=0(6)"
 # ====================================================================================================
 #
 # uvcopy extendsales1,fili1=dat2/sales_items,filr2=dat2/product_masterI,filo1=tmp/sales_itemsx1
 # =============================================================================================
 # uvcopy extendsales1   <-*- same as above, but easier, files default as shown above
 # ===================
 #
 rop=r1     # EOF option prompt to viiew output file (vi,cat,lp,etc)
 fili1=?dat2/sales_items,rcs=100,typ=LST                # sales item file
 filr2=?dat2/product_masterI,rcs=99,typ=ISF,isk1=0(6)   # product master Indexed file *Note1*
 filo1=?tmp/sales_itemsx1,rcs=100,typ=LST               # output items extended w price/dscrptn
 @run
        opn    all                      open all files
 # begin loop: read sales items, extending with price & dscrptn from Indexed file
 man20  get    fili1,a0                 read next sales item record
        skp>   man90                    (cc set > at EOF)
        cmc    a0(1),'#'                #comment to bypass ?
        skp=   man20
        mvc    b0(100),a0               copy input to output area
        mvf    c0(100),a30(6)           store product# in key position & clear remainder
        redm5  filr2,c0(100)            read key equal (option m5) *Note2*
        skp=   man30                    (cc equal if match)
 man28  mvf    b63(30),' ** ERR product# INVALID **'
        skp    man40
 man30  mvn    b45(7),c30(7)            price from product_master Indexed file
        mvn    b53(9),c30(7)            price to amount for extension
        mpy    b53(9),b38(6)            master price * item qty = extended amount
        mvc    b63(20),c10              product dscrptn from Indexed file
 man40  put    filo1,b0                 write output
        skp    man20                    return to get next record
 #
 # EOF - close files & end job (rop=r1 prompts to show output file)
 man90  cls    all
        eoj
 #
 #                         ** Notes Re: Indexed file **
 #
 #Note1 - Indexed files have 2 partitions: product_masterI.dat & product_masterI.idx (in this case)
 #      - the ".idx" is binary, but the ".dat" data partition can be viewed & printed with unix tools
 #        because Indexed file records have a status byte appended which is normally x'0A' (a LineFeed)
 #      - we specified "rcs=99" on filr2=... because we wanted the result to be 100 bytes
 #        (we could have specified 38 bytes for this demo, but we might later add more product info)
 #Note2 - 'redm5' random read option 'm5' means key= to value stored in key position (1st 6 bytes)
 #      - see uvcopy3.htm#red for documentation on reading Indexed files
 #      - see uvcp.htm#D2 or uvsort.htm#A2 for more examples of loading Indexed files

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7D3. Indexed Files

Execute uvcopy extendsales1 Indexed file method


 #1. cdd   <-- ensure you are in the demo/... subdir
     ===     - /home/uvsoftxx/demo/... for ssh to UVsoftware website
             - /home/userxx/demo/..... for logins on user machines

Before we can run our demo, we must 1st load the Indexed file with uvcp or uvsort:


 #2. uvcp "fili1=dat2/product_master,typ=LST,rcs=100,filo1=dat2/product_masterI,typ=ISF,rcs=99,isk1=0(6)"
     ===================================================================================================

 #2a. uvsort "fili1=dat2/product_master,typ=LST,rcs=100,filo1=dat2/product_masterI,typ=ISF,rcs=99,key1=0(6)"
      =====================================================================================================
       - see uvcp.htm#D2 or uvsort.htm#A2 for more examples of loading Indexed files

 #3. cat dat2/product_masterI.dat  <-- display the Product Master INDEXED file
     ============================    - only 1st 3 records, see all 8 on page '7D1' above
      # product_master - Indexed on product# looked-up for description & price
      #        1         2         3         4         5
      #2345678901234567890123456789012345678901234567890
      #prod#====Description===========price=============
      BBQ010    BAR-B-Q             0019500
      CHR015    LAWN CHAIR          0004500
      HAM010    CLAW HAMMER         0000950

 #4. cat dat2/sales_items     <-- display the sales_items SEQUENTIAL INPUT file
     ====================       - only 1st 3 records, see all 6 on page '7D1' above
      # sales_items - to demo extendsales1 table load & table lookup
      #        1         2         3         4         5         6         7         8
      #2345678901234567890123456789012345678901234567890123456789012345678901234567890
      #cust#=slm===date====invoice==product====qty===price==amount===description======
      130140  44 20200801 INV11001  HAM010  000010
      130140  44 20200801 INV11001  CHR015  000020
      139923  21 20200802 INV11002  BBQ010  000010

 #5. uvcopy extendsales1,fili1=dat2/sales_items,filr2=dat2/product_masterI,filo1=tmp/sales_itemsx1
     =============================================================================================
      - copy sales_items, looking-up product_master Indexed file, to get price & description

 #5a. uvcopy extendsales1   <-*- same as above but easier, filenames default as shown above
      ===================      - see filenames coded within the job listed on page above

 #6. cat tmp/sales_itemsx1  <-- investigate the OUTPUT file
     =====================    - Note addition of--> price  ExTAmount Description
      130140  44 20200801 INV11001  HAM010  000010 0000950 000009500 CLAW HAMMER
      130140  44 20200801 INV11001  CHR015  000020 0004500 000090000 LAWN CHAIR
      139923  21 20200802 INV11002  BBQ010  000010 0019500 000195000 BAR-B-Q
      139923  21 20200802 CR5421    SAW051  000005 0008500 000042500 POWER SAW
      250825  44 20200803 INV11003  HAM099  000004                    ** ERR product# INVALID **
      401210  21 20200804 INV11004  SAW011  000025 0001975 000049375 HAND SAW

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7E1. Table Load & Table Lookup

extendsales2 - table load/lookup (vs Indexed file)

 # extendsales2 - copy sales items adding prices & descriptions from product master
 #              - demo Table Load & Table Lookup Aug2020
 # - using uvcopy instructions 'rtb' Read TaBle & 'lok' Look up table
 # - load product master file into memory table at begin job
 # - extend sales items by looking up product master for price & description
 #
 # uvcopy extendsales2,fili1=dat2/sales_items,fili2=dat2/product_master,filo1=tmp/sales_itemsx2
 # ============================================================================================
 # uvcopy extendsales2   <-*- same as above, but easier, filenames default as shown above
 # ===================
 #
 rop=r1     # EOF option prompt to viiew output file (vi,cat,lp,etc)
 fili1=?dat2/sales_items,rcs=100,typ=LST      # sales item file
 fili2=?dat2/product_master,rcs=100,typ=LST   # product master file
 filo1=?tmp/sales_itemsx2,rcs=100,typ=LST     # output items extended w price/dscrptn
 @run
        opn    all                      open all files
        rtbc1  fili2,c0(100),c0(100)    read product master into memory area 'c'
 #                                      option 'c1' bypasses #comment lines
 # begin loop: read sales items, extending with price & dscrptn from table lookup
 man20  get    fili1,a0                 read next sales item record
        skp>   man90                    (cc set > at EOF)
        cmc    a0(1),'#'                #comment to bypass ?
        skp=   man20
        mvc    b0(100),a0               copy input to output area
        lokz1  cc0(100),cc0(6),a30(6)   lookup table by product code
        skp=   man30                    (skp cc equal if match)
 man28  mvf    b63(30),' ** ERR product# INVALID **'
        skp    man40
 man30  mvn    b45(7),cc30(7)           price from table
        mvn    b53(9),cc30(7)           price to amount for extension
        mpy    b53(9),b38(6)            master price * item qty = amount
        mvc    b63(20),cc10             product dscrptn from table
 man40  put    filo1,b0                 write output
        skp    man20                    return to get next record
 # EOF - close files & end job (rop=r1 prompts to show output file)
 man90  cls    all
        eoj
 #
 #    ** Re: 'lokz1 cc0(100),cc0(6),a30(6)' Area 'c' Register 'c' **
 #
 # - 1st 'c' of cc0 is area 'c', 2nd 'c' of cc0 is Index Register 'c'
 # - option 'z1' ensures Register 'c' reset to 0
 # - lok instruction searches the 100 byte table entries loaded by 'rtb'
 #   for a match on 1st 6 bytes to product# from columns 31-36 of sales_items
 # if Match - cc = (if Nomatch cc unequal causing ERR msg & skp man40 output)
 # if Match - table lookup register 'c' will hold displacement to the matching entry
 # if match on 3rd table entry Register 'c' will be 200 & cc30 will be 230 in area c
 # - see more about Index Registers at uvcopy2.htm#H0 - H7.
 # - see more about 'rtb' Read files & TaBle in memory at uvcopy3.htm#rtb
 # - see more about 'lok' LOoK up tables in memory at uvcopy3.htm#lok

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7E2. Table Load & Table Lookup

Execute uvcopy extendsales2 - Table Load/Lookup method


 #1. cdd   <-- ensure you are in the demo/... subdir
     ===     - /home/uvsoftxx/demo/... for ssh to UVsoftware website
             - /home/userxx/demo/..... for logins on user machines

 #2. cat dat2/sales_items     <-- display the sales_items INPUT file
     ====================       - only 1st 3 records, see all 6 on page '7D1' above
      # sales_items - to demo extendsales1 table load & table lookup
      #        1         2         3         4         5         6         7         8
      #2345678901234567890123456789012345678901234567890123456789012345678901234567890
      #cust#=slm===date====invoice==product====qty===price==amount===description======
      130140  44 20200801 INV11001  HAM010  000010
      130140  44 20200801 INV11001  CHR015  000020
      139923  21 20200802 INV11002  BBQ010  000010

 #3. cat dat2/product_master  <-- display the Product Master 2nd INPUT to be TABLED
     =======================    - only 1st 3 records, see all 8 on page '7D1' above
      # product_master - Tabled in memory, looked-up for description & price
      #        1         2         3         4         5
      #2345678901234567890123456789012345678901234567890
      #prod#====Description===========price=============
      BBQ010    BAR-B-Q             0019500
      CHR015    LAWN CHAIR          0004500
      HAM010    CLAW HAMMER         0000950

 #4. uvcopy extendsales2,fili1=dat2/sales_items,fili2=dat2/product_master,filo1=tmp/sales_itemsx2
     ============================================================================================
      - copy sales_items, load/lookup product_master TABLE to get price & description

 #4a. uvcopy extendsales2   <-*- same as above but easier, filenames default as shown above
      ===================      - see filenames coded within the job listed on page above

 #5. cat tmp/sales_itemsx2  <-- investigate the OUTPUT file
     =====================    - Note addition of--> price  ExTAmount Description
      130140  44 20200801 INV11001  HAM010  000010 0000950 000009500 CLAW HAMMER
      130140  44 20200801 INV11001  CHR015  000020 0004500 000090000 LAWN CHAIR
      139923  21 20200802 INV11002  BBQ010  000010 0019500 000195000 BAR-B-Q
      139923  21 20200802 CR5421    SAW051  000005 0008500 000042500 POWER SAW
      250825  44 20200803 INV11003  HAM099  000004                    ** ERR product# INVALID **
      401210  21 20200804 INV11004  SAW011  000025 0001975 000049375 HAND SAW

 #6. l tmp   <-- list output files in tmp/...
     =====     - 'sales_itemsx1' from extendsales1 & 'sales_itemsx2' from extendsales2
      -rw-rw-r-- 1 uvadm apps 456 Aug  3 12:14 sales_itemsx1
      -rw-rw-r-- 1 uvadm apps 456 Aug  3 13:55 sales_itemsx2

 #6. diff tmp/sales_itemsx1 tmp/sales_itemsx2   <-- prove 2 output files same ?
     ========================================
      - diff shows NO output if files are identical

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

7E3. uvcopy Debugger

See uvcopy2.htm#J1 for documentation on the uvcopy debugger. You activate with Run OPtion 'd' - append ',rop=d' on the uvcopy command. Here is a debug session for extendsales2. Null entries execute 1 instruction & prompt for a command. I have high-lighted the commands by repeating on the right with short description.


 uvcopy extendsales2,rop=d   <-- Execute with DEBUG option ',rop=d'
 =========================
      0         opn .. all                      open all f
 debug>
    136         rtb ..c1 fili2,c0(100),c0(100)    read pro
 debug>
    344 man20   get .. fili1,a0                 read next
 debug>
    528         skp >. man90                    (cc set >
 debug> p12 c0(100)                                                <-- p12 c0(100)
        0000-->#2345678901234567890123456789012345678901234567890    - Print 12 lines area c (table)
        0100-->#prod#====Description==========price==============
        0200-->BBQ010    BAR-B-Q             0019500
        0300-->CHR015    LAWN CHAIR          0004500
        0400-->HAM010    CLAW HAMMER         0000950               <-- 1st table lookup below
        0500-->HAM035    JACK HAMMER         0029500
        0600-->SAW011    HAND SAW            0001975
        0700-->SAW012    RIP SAW             0002500
        0800-->SAW051    POWER SAW           0008500
        0900-->TAB012    LAWN TABLE          0008500
        1000-->~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 debug>
    664         cmc .. a0(1),'#'                #comment t
 debug> p a0                                                       <-- p a0
         -----># sales_items - extendsales1 table load & lookup      - Print area a (input)
 debug> t1 man22                                                   <-- t1 man22
    936 man22   mvc .. b0(100),a0               copy input           - goto Tag 'man22'
   1072         lok ..z1 cc0(100),cc0(6),a30(6)   lookup t
   1232         skp =. man30                    (skp cc eq
 debug> r                                                          <-- r (display Registers)
 registers: c=400,u=100,v=44,x=400,z=44,                             - c=400 points to HAM010
                                                                     - cc30 = c430 = price
   1688 man30   mvn .. b45(7),cc30(7)           price from
   1824         mvn .. b53(9),cc30(7)           price to a
   1960         mpy .. b53(9),b38(6)            master pri
   2096         mvc .. b63(20),cc10             product ds
   2232 man40   put .. filo1,b0                 write outp
   2416         skp .. man20                    return to
 debug> p a0                                                       <-- p a0 (Print input record)
         ----->130140  44 20200801 INV11001  HAM010  000010
 debug> p b0                                                       <-- p b0 (Print output record)
         ----->130140  44 20200801 INV11001  HAM010  000010 0000950 000009500 CLAW HAMMER
 debug> g                                                          <-- g (Go to EOJ)
 20200804:330006:extendsales2: EOF fili01 rds=10 size=576: dat2/sales_items
 20200804:330006:extendsales2: EOF fili02 rds=12 size=518: dat2/product_master
 20200804:330006:extendsales2: EOF filo01 wrts=6 size=456: tmp/sales_itemsx2

See all debugger commands at uvcopy2.htm#J1

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

20A1. Relevant Vancouver Utilities Documentation

Here are links to other relevant Vancouver Utilities Documentation.


 https://uvsoftware.ca/uvdemos2.htm
  ==================================
  - Vancouver Utilities for any Unix/Linux site
  - minimizing JCL conversion & maximizing General Purpose use
  - many useful command line utilities & scripts
  - Selectjobs, Tablejobs, UVscripts
  - presented in a demo/tutorial manner
  - run the comamnds & see if your outputs matches expected results

 https://uvsoftware.ca/uvhd.htm
  ===============================
  - interactive data file investigation utiltiy
  - displays records in 'vertical hexadecimal'
  - great for mainframe type files with packed/binary data
  - browse,search,update,select,print,etc
  - most customer's favorite utility

 https://uvsoftware.ca/uvsort.htm
  =================================
  - Vancouver Utility replacement for mainframe SORT
  - JCL converter automatically converts SORT steps to uvsort syntax
  - handles all but a few complex DFSORT functions such as JOIN & PARSE
    which are handled by uvcopy with some manual coding required

 https://uvsoftware.ca/uvcp.htm
  ===============================
  - Vancouver Utility replacement for IDCAMS, IEBGENER, etc
  - JCL converter converts IDCAMS,IEBGENER,etc to uvcp
  - can select/delete records, reformat records, etc

 https://uvsoftware.ca/uvcopy1.htm
  ==================================
  - most powerful Vancouver Utility
  - replaces mainframe utilities
    (IEBGENER,SORT JOINKEYS,PARSE,ICETOOL,QUIKJOB,EASYTRIEVE,etc)
  - documentation in 3 parts

 https://uvsoftware.ca/uvcopy2.htm
  ==================================
  - uvcopy functions (vs instructions in uvcopy3)
  - file declarations, file types,
  - options, work areas, variable names

 https://uvsoftware.ca/uvcopy3.htm
  ==================================
  - uvcopy instructions (over 100 instructions)
  - uvcopy an extremely powerful data manipulation utility
  - the power of assembler language without the complexity
  - interpretive, just create/modify a parameter file & execute it

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page


 'https://uvsoftware.ca/install.htm'
  ==================================
  Install Vancouver Utilities

 https://uvsoftware.ca/uvtrain.htm
  ==================================
  Vanvouver Utilities Training Guide
  - for uvsort, uvcp, uvcopy, uvhd, uvlist
  - examples & exercises

 https://uvsoftware.ca/uvhelp.htm
  =================================
  - command line menus to most useful utilities
  - both unix system utilities & Vancouver Utilities
  - handy way to find the syntax of many useful commands
  - just enter --> uvhelp <-- at your shell prompt

 https://uvsoftware.ca/scripts1.htm
  ===================================
  many usefull scripts calling both unix utilities & Vancover Utiliites
  - just a few examples are: renaming files,
  - counting lines in all files in directory
  - counting files in all subdirs of a superdir
  - running unix 'diff' for all files in 2 directories
  - listing, moving, removing files older than a number of days, etc

 https://uvsoftware.ca/jclcnvbrief.htm
  ======================================
  JCL Conversion - a 'SHORTER' more concise version
  (not yet complete as of May 2020)

 https://uvsoftware.ca/jclcnv1demo.htm
  ======================================
  JCL Conversion - the 'LONG' version

 https://uvsoftware.ca/jclcnvbrief.htm
  ======================================
  JCL Conversion - a 'SHORTER' more concise version
  (not yet complete as of May 2020)

 https://uvsoftware.ca/jclcnv3aids.htm
  ======================================
  Testing aids & utilties that may assist JCL conversions

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

20B1. Links to uvcopy Instructions documeentation

Here are links to uvcopy instructions documented in uvsoftware.ca/uvcopy3.htm

Clicking on these '+' links take you into that separate document, so you should "right-click" on these links to open in a new tab, so you will not lose your place in this uvdemos1.doc.

 +`add` - add op2 data to op1 data & store result in op1                    - 29
 +`anc` - ANd Character (erase bits in op1 with corresponding 0 bits in op2 - 17
 +`bal` - branch & link to a subroutine                                     - 84
 +`bcr` - Branch on Condition & Return                                      - 87
 +`bcr example` - testbcr1 test/demo job for bcr instruction                - 88
 +`cat` - concatenate op2 data onto op1 data (within op1 limits)            - 13
 +`can` - cancel the uvcopy job                                             - 82
 +`chx` - convert character representation of hex to true hex data          - 93
 +`clr` - clear the op1 area to the op2 byte value                          - 14
 +`cls` - close a file                                                      - 45
 +`cmc` - compare and set internal condition code for following skp         - 36
 +`cmn` - compare numeric & set condition code for following skp            - 38
 +`cnt` - count the number of op2 patterns in the op1 area                  -142
 +`ctr` - center data within the op1 area                                   -145
 +`data type & sign codes`                                                  - 22
 +`dat` - convert dates between: calendar,julian,& days-since-1900          -155
 +`del` - delete a record in an Indexed file                                - 77
 +`div` - divide the op1 data by the op2 data & store result in op1         - 32
 +`dlm` - convert Fixed-Field to "delimited","format"                       -152
 +`dtf` - Delimited To Fixed-field                                          -201
 +`edt` - edit op2 data into op1 area controlled by op3 mask                - 15
 +`env` - get value for any specified environmental variable                -167
 +`eoj` - end the job                                                       - 82
 +`evt` - get & table the values for all environmental variables            -168
 +`evs` - copy text, expanding any environmental variables                  -169
 +`expandevs1` - using "evs" to expand $VARIABLEs in DB2 scripts            -170
 +`fix` - convert variable field format to fixed length fields              -146
 +`ftd` - Fixed-field To Delimited                                          -200
 +`fxt` - add a uvcopy instruction to the execution table                   -166
 +`get` - next record from a sequential file & store in the op2 area        - 46
 +`getrecsizes1` - demo recsizes stored in register 'v' by 'get'            - 50
 +`getcopydir1` - demo 'get' filenames from directory & copy all files      - 51
 +`hxc` - convert hex data to hex character representation                  - 93
 +`ins` - insert op2 data prior to op1 data                                 -144
 +`lck` - lock a D-ISAM Indexed file                                        - 78
 +`lok` - look-up a table of data (op1) for match to argument (op2)         -162
 +`mpy` - multiply op1 data by op2 data & store result in op1               - 31
 +`msg` - display a msg (1 or 2 lines) on stdout                            - 79
 +`msgw` - display message & wait for reply (option "w" of msg)             - 80
 +`mvc` - move character data from op2 to op1 for op1 length                -  8
 +`mvf` - move op2 to op1 for lth op2 & fill excess op1 with op3 char       -  9
 +`mvr` - move data from op2 to op1 right adjusted, left filled with op3    - 11
 +`mvp` - move op2 to op1, if all EBCDIC blanks, convert to ASCII blanks    - 12
 +`mvn` - move numeric data depending on op1/op2 types & lengths            - 21
 +`mvu` - move op2 data to op1 area until the op3 pattern detected          -106

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

INDEX to uvcopy INSTRUCTIONS (continued)

 +`opn` - open a file - or all files declared if op1 = 'all'                - 44
 +`orc` OR Character - set 1 bits in op1 with corresponding 1 bits in op2   - 18
 +`pac` - pack the op2 data into the op1 area                               - 33
 +`pop` - process option string into process/user option storage            -165
 +`put` - a record to a sequential file                                     - 52
 +`ran` - generate random numbers                                           -202
 +`rel` - set file pointer (relative record# or relative byte#)             - 58
 +`red` - read a record from an indexed file                                - 68
 +`ret` - return from a subroutine (to the instruction following the bal)   - 86
 +`rep` - scan op1 replacing all op2 patterns with the op3 data             -103
 +`rpt` - replace by table                                                  -121
 +`rtb` - read a table of data into the op1 area (at run time)              - 60
 +`rts` - scan a table for a pattern & replace with an alternate            -133
 +`rtt` - replace patterns in data table via search/replace table           -136
 +`scn` - scan op1 area for op2 pattern & set cc equal if found             - 96
 +`sct` - scan by table                                                     -109
 +`set` - set key of reference & file pointer in an Indexed file            - 67
 +`shf` - shift data a specified number of bits left or right               - 20
 +`skeleton1` - template job to copy, rename,& modify as required           -  5
 +`skp` - skip to a tag (or number of instrns) if condition code matches    - 39
 +`skptable` - demo job lookup table of Record-Types & Processing-Labels    - 40
 +`sort` (external) - sxo, sxp, sxs, sxg, sxc                               -173
 +`sqz` - squeeze out occurrences of the op2 character from op1             -139
 +`sqf` - squeeze multiple contiguous fields & blank fill at the end        -141
 +`srt` - internal sort (tables, record arrays, etc)                        -171
 +`sst` - scan using table patterns to match or not-match                   -111
 +`sta` - search data tables & count matches in a pattern table             -125
 +`sts` - scan a table for a character string                               -127
 +`stt` - search data tables for any match to a pattern table               -130
 +`sub` - subtract the op2 data from the op1 data & store result in op1     - 30
 +`subrtn example` - cymd compare 6 digit dates for Year 2000               - 85
 +`swp` - swap left & right sides of data in op1 based on op2 separator     -154
 +`sys` - issue system commands - call the shell to execute op1 string      -159
 +`table overview` - build tables in memory to dump at EOJ                  -179
 +`table formats` - supplied formats available (f1-f12)                     -191
 +`table entry layout` - dumped by tbd (unedited)                           -194
 +`tbl` - build tables in memory to be dumped at end of job                 -180
 +`tbf` - declare formats to be used when editing tables to output files    -182
 +`tbh` - declare table column (field) headings                             -183
 +`tbp` - print (edit) the table entries into a file (usually at eoj)       -184
 +`tbd` - dump (unedited) the table entries into a file (usually at eoj)    -186
 +`tim` - get current date & time & store in area y for user access         - 83
 +`tra` - translate to ASCII (from EBCDIC)                                  - 89
 +`tre` - translate to EBCDIC (from ASCII)                                  - 90
 +`trl` - translate to lower case (any UPPER case present)                  - 90
 +`tru` - translate to UPPER case (any lower case present)                  - 90
 +`trt` - translate via user specified (or modified) translate table        - 91
 +`trs` - translate/search command                                          - 92
 +`tst` - test each op1 sub-field for match to any op2 sub-field            - 41
 +`tsb` - test 1 byte of data for any match to the bits in the 1 byte op2   - 43
 +`tts` - scan data table lines for matches to pattern table                -116
 +`ttx` - scan data table groups for matches to pattern table               -118

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

INDEX to uvcopy INSTRUCTIONS (continued)

 +`ulk` - unlock a D-ISAM Indexed file                                      - 78
 +`unp` - unpack the op2 data into the op1 area                             - 34
 +`und` - convert "delimited","format" to Fixed-Field                       -153
 +`update_rel` reldemo1 - update in place                                   - 58
 +`upd` - update a record in an Indexed file                                - 74
 +`upw` - update/write record depending on key present/absent               - 76
 +`var` - convert multi fixed length fields to a variable length string     -150
 +`vhx` - convert data to vertical hexadecimal representation               - 94
 +`vnf` - Verify Numeric Field                                              -203
 +`wat` - wait a specified number of seconds                                - 83
 +`wrt` - write a new record into an Indexed file                           - 73
 +`wtb` - write a table of data from the op1 area (at run time)             - 65
 +`xft` - crossfoot multiple op2 fields & store the result in op1           - 35
 +`xor` - eXclusive OR (bit manipulation)                                   - 19
 +`xxa` - call user subfunction (written in C & linked to uvcopy)           -206

Goto:   Begin this doc End this doc Index this doc Contents this library UVSI Home-Page

Visitor Counters for ThisYear and LastYear