UNIX/Linux Programming Chapter 5 - PP

Putting it All Together to Produce the Report

Combine small scripts into a larger script file convenient complete a large task by dividing it into a series of smaller ones test each small script independently

Advancing your File-Processing Techniques

Commands for file processing are in two categories: Selection Commands: focus on extracting specific information from files Manipulation and transformation Commands: alter and transform extracted information into useful and appealing formats

Designing Records

First task in record design phase: Defining fields in the records These definitions take the form of a Record Layout Record layout identifies each field by name and data type Design records to store only fields relevant to the record's primary purpose Short records are preferable Easier to understand Brief and to the point Must include a field that uniquely identifies each record in the file Key So you can join them to other files

Linking Files with Keys

Key: common field shared by each of the linked files Multiple files can be joined by a key Another important task in the design phase it to plan a way to join files in design phase, IF necessary.

printf function within awk

Syntax: awk -F: '{printf "%d %-12.12s %-10.10s\n", $1, $2, $3}' programmer Each % symbol in the format string corresponds with a $ field, as follows: %d indicates that field $1 (programmer number) is to appear in decimal digits. %-12.12s indicates that field $2 (programmer name) is to appear as a string. The hyphen (-) specifies the string is to be left-justified. The 12.12 indicates the string should appear in a field padded to 12 spaces, with a maximum size of 12 spaces. %-10.10s indicates that field $3 (programmer salary) is to appear as a string. The hyphen (-) specifies the string is to be left-justified. The 10.10 indicates the string should appear in a field padded to 10 spaces, with a maximum size of 10 spaces.

Using a Shell Script to Implement the Application

Tip: test and debug each command before you place it in your script file Use vi or Emacs to create script files Commenting shell scripts is crucial Helps creator and other programmers Use pound (#) character

Running a Shell Script

You can run a shell script in virtually any shell(default is Bash) Two Ways to Run Scripts: call the interpreter: sh testscript can accompany it with several debugging options type ./ in front of name: ./testscript must make script executable first use chmod to add x permission ADVICE: specify w/ what shell your script is intended to be used EXAMPLE: #!/bin/bash

More Options

d for a decimal integer - to left-justify; s for an array of characters (string) width to set size; .prec to set maximum string width or digits to the right of the decimal point; f for a floating-point decimal number Enclosed in double quotation marks (" "), format is often referred to as a mask that overlays the data fields going into it.

Using the Selection Commands

grep Syntax: command [-options] pattern [filename] Find and displays lines containing a particular search pattern, Can be used on text and binary regular files, Can search multiple files in one command EXAMPLE: grep -r Computer /documentation diff Syntax: diff[-options] file1 file2 used to determine the minimal set of changes needed to convert file1 to file2 differing text preceded by < or > EXAMPLE: diff zoo1 zoo2 uniq Syntax: uniq [-options] [file1>file2] Removes duplicate lines from a file Compares only consecutive lines Requires sorted input EXAMPLE: uniq -u parts > single_items comm Syntax: comm [-options] file1 file2 identifies duplicate lines but it doesn't delete them works with two files compares two sorted files for common lines and generates 3 columns of output to show which lines are unique to each file and which are common to both files EXAMPLE: wc Syntax: wc[-options] [files] output displays Lines, Words, Bytes can specify all three options in the command line(-lwc) if entered without options, you see counts of lines, words, and characters in that order EXAMPLE: wc /etc/printcap

Using the Pipe Operator

pipe(|)operator: another redirection operator < and > can be used, redirects the output of one command into the input of another command, it can connect several commands EXAMPLE: ls /etc | sort -r | less

Formatting Output

printf function within awk is used to format output

Using Manipulation and Transformation Commands

sed Syntax: sed[-options] [command] [file(s)] sed[-options][-f scriptfile][file(s)] a stream editor that can be used on one or more files, and is particularly useful for making global changes on large files first form specifies editing command on command line second form specifies script file containing sed commands tr Syntax: tr[-options]["string1""string2"] translates each character in string1 into the character in the corresponding position in string2 popularly converts lowercase to uppercase EXAMPLE: tr "c" " " < constants pr Syntax: pr [-options] [file...] formats one or more files by providing pagination, columns, and column heads if no file is specific or "-" is specified as the file, reads the standard input default output: single-column pages of 66 lines, each page has a five-line header

Why do we need to learn all these commands?

~To develop an application ~important phase is DESIGN phase ~the design of records in a file influences what you can do with an app(can be inefficient/difficult or of limited value)