Most Useful Command Line Scripts

Here are a few command line Korn (or Bash) shell scripts that I have found most useful when working with files & directories on Unix & Linux systems. And now you can use them on Windows with Windows 10 which runs Bash shell scripts as of Summer 2016.

You can download them individually from directory www.uvsoftware.ca/FreeScripts/ or all combined from www.uvsoftware.ca/FreeScripts/FreeScripts.zip

These are some of the most useful scripts, and you can see many more listed at www.uvsoftware.ca/scripts1.htm. You could download that entire document and cut out individual scripts that you desire.

As well as listing the scripts, www.uvsoftware.ca/scripts1.htm gives many illustrations of using the scripts & displays sample outputs. See just one such example near the end of this newsletter for script 'cfdd' count files in all subdirs of a parent directory.

Note that these scripts have built-in 'help' - just enter the script name only (without arguments) - to show the arguments expected and an example. You might like to copy this feature in the scripts that you write.

Goto:   Begin this document End this document UVSI Home-Page

Rename scripts

These scripts rename all files in a directory saving hours of keyboard drudgery. Renaming hundreds of files is often required after FTP'ing files from mainframes.

renameL
  • translate filenames to lower case
renameU
  • translate filenames to UPPER case
renameLsd
  • translate filenames to lower case in all subdirs of a superdir
renameX
  • change any specified extension to any specified alternate
rename-X
  • remove any specified extension
rename+X
  • add any specified extension
rename-AA
  • remove Any pattern Anywhere in filename
renameAA
  • replace Any pattern with any Alternate
renameCC
  • replace Any 1 character with an alternate character
  • for example: replace all '.' periods with '_' underscores
renameB2U
  • change any embedded blanks to underscores
renameD2U
  • change any '$' dollar signs to '_' underscores
rename-QQ
  • remove any 'Quotes'
rename-PP
  • remove any (Parenthesis)
rename2PP
  • rename to whatever is in (Parenthesis)
rename-TS
  • remove any timestamp _HHMMSS ('_' + 6 digits)
  • preserves anything following _###### (.dat, .txt, etc)

Listing Files in various sequences

llm
  • list long & pipe to more (in the usual filename sequence)
  • saves keystrokes for the often repeated 'ls -l | more'
llc
  • list files with File & Line Counts (in filename sequence)
llt
  • list files, with the latest First
lls
  • list files in Size order
lld
  • list directories only (omit files)
llr
  • list files & subdirs Recursively (all the way down)

Counting Files & Lines in Directories

cfd
  • Count Files in a Directory listing filenames
cfdt
  • Count Files in a Directory, Total only (1 line answer)
cfdd
  • Count Files in sub-Directories within a parent Directory
cfdp
  • Count Files in a Directory with a Pattern on any line in any file
clf
  • Count Lines in 1 File
clfd
  • Count Lines & Files in a Directory
clfp
  • Count Lines in a File with a specified Pattern
clfdp
  • Count Lines in all Files of a Directory with a Pattern

Goto:   Begin this document End this document UVSI Home-Page

List, Remove, Copy files older/newer than specified days

lsold1
  • list files older than a specified number of days
  • recursive, descending all levels of a directory tree,
    looking for files older than a specified no of days.
lsnew1
  • list files newer than a specified number of days
lsnew2
  • list files created/modified within 2 specified days
rmold1
  • remove files older than the specified number of days
  • recursive, descends directory tree searching for all old files
  • can 1st run lsold1/cpold1 to verify/save files to be deleted
  • alrternates rmold2,rmold3 to prompt before removal
rmold2
  • lists all old files, then prompts for each old file to be removed
rmold3
  • lists all old files, then prompts once to remove all old files
cpold1
  • copy files older than the x days to a specified directory
mvold1
  • move files older than the x days to a specified directory

mass changes to files in directories

removeCR
  • remove Carriage Returns from text files (Windows --> Unix/Linux)
  • copies to tmp/... removing CRs & prompts to copy back
insertCR
  • insert Carriage Returns in text files (Unix/Linux --> Windows)
  • copies to tmp/... removing CRs & prompts to copy back
removeCRall
  • remove CRs from all files in directory, copying to 2nd dir
insertCRall
  • insert CRs into all files in directory, copying to 2nd dir
runsed1
  • apply sed script to process all files from 1 dir to a 2nd dir
  • great for mass changes to scripts & programs
runsed2
  • follow-on script to use sed command saved by runsed1 in tmp/runsed1
  • in case you wish to rerun same script on other dirs
dropbl0
  • drop all blank lines, using 'grep', copies to tmp/...
  • for inspection & copy back
uvfix1
  • copy 1 Text file to tmp/... applying uvcopy instructions
  • may then inspect (with vi tmp/...) & copy back if desired
uvfixA
  • copy ALL Text files from 1 subdir to a 2nd subdir
    while applying uvcopy instructions
Note
  • uvcopy is a powerful interpretive data manipulation utility
  • the power of assembler without the complexity of compiles
  • see all 100 uvcopy instructions at www.uvsoftware.ca/uvcopy3.htm
  • mvc,clr,edt,ins,rep,scn,add,sub,mpy,div,tbl,tra,tre,trl,tru,etc

Goto:   Begin this document End this document UVSI Home-Page

Miscellaneous scripts

allcancel
  • cancel all printer requests (spooled by 'lp') for current user
  • does an lpstat to get request-ids & cancels them
alldiff
  • run 'diff' for all file pairs in 2 directories
  • great for verifying mass changes after modifying conversion scripts
  • to confirm desired changes & no unintended changes
printable1
  • ensure printable characters in a text file
  • copies to tmp/... converting any unprintables to '.' periods
longest1
  • find longest line in a text file

Aliases to prompt before remove/overwrite

 alias rm='rm -i'              # confirm removes
 alias mv='mv -i'              # confirm renames
 alias cp='cp -i'              # confirm copy overwrites
 alias l='ls -l'               # save keystrokes

Goto:   Begin this document End this document UVSI Home-Page

Sample script 'cfdd' - Listing & example of use

 #!/bin/ksh
 # cfdd - Count Files in a all sub-Directories within a Directory
 echo "cfdd - Count Files in all sub-Directories of a super-Directory"
 if [[ ! -d "$1" ]]; then
    echo "usage: cfdd parentdirectory"
    echo "       ===================="
    echo "ex#1:  cfdd /home/uvadm/sf  <-- count Script Files in sf/..."
    echo "       ==================="
    exit 99; fi
 typeset -RZ5 cnt=0 fcnt=0 dcnt=0
 for dir in $1/*
 do if [[ ! -d $dir ]]; then continue; fi
    ((dcnt+=1))
    cnt=$(ls $dir | wc -w)
    #=====================
    ((fcnt+=cnt))
    echo "$cnt files in subdir #$dcnt $dir"
 done
 echo "$fcnt Total files in $dcnt subdirs of $1 $(date +%Y%d%m:%H%M)"
 exit 0

sample usage of 'cfdd'


 cfdd /home/uvadm/sf   <-- count files in all subdirs of directory sf
 ===================     - assuming we are in /home/uvadm
                         - output displayed as follows:
 cfdd - Count Files in all sub-Directories of a super-Directory
 00156 files in subdir #00001 sf/adm
 00079 files in subdir #00002 sf/demo
 00247 files in subdir #00003 sf/IBM
 00478 files in subdir #00004 sf/util
 00960 Total files in 00004 subdirs of sf 20160907:1803

You can see a help screen for most scripts, by entering the script-name only, without its required arguments (omit the directory).


 cfdd       <-- enter script name only, omitting argument (directory)
 ====         - will display usage & help as follows:

 usage: cfdd parentdirectory
        ====================
 ex#1:  cfdd /home/uvadm/sf  <-- count Script Files in sf/...
        ===================

Goto:   Begin this document End this document UVSI Home-Page

links to more information

 http://www.uvsoftware.ca                 - UV Software Home Page
 http://www.uvsoftware.ca/scripts1.htm    - most useful script listings
 http://www.uvsoftware.ca/jclcnv1demo.htm - JCL Conversion to scripts
 http://www.uvsoftware.ca/uvsort.htm      - uvsort documentation
 http://www.uvsoftware.ca/uvprices.htm    - prices & partner agreements
 http://www.uvsoftware.ca/uvintro.htm#G1  - customer list & reference letters
 http://www.uvsoftware.ca/libuvhd.htm     - download uvhd FREE utility
 http://www.uvsoftware.ca/news/           - all newsletters past & present

I invite your feedback. Let me know if these news letters are useful to you, and any suggestions you may have for improvements.

Thank You, Owen Townsend, UV Software Inc.

Goto:   Begin this document End this document UVSI Home-Page