BASH 322

What does ls do?

'ls' lists all the files or directories in the current directory that you're residing in

When I login to a system, what directory am I popped into

'~' :the The home directory. If I say pwd when I first login, I'll get /home/ece322

Bash integer comparatos

-ne, gt, ge, lt

How is the $ sign special and different from bash

-variables in awk not prepended with $ to get the value $0- replaced with the entire line $1 $2 $3 - replaced with each first, second, and third word from each line • SO that middle bracket is an automatic loop that we run once for every line of the file ◦ and then for every line of the file, you might want to do something ◦ If you wanna do something like parse the line, you can automatically grab the words of a line by using these variables $1, $2 $3 EX: cat file.txt | awk '{print $3 " " $2 " " $1}' You can take a text file that has 3 columns and you can reverse those columns

How do I execute a binary/executable/program on the command that I've compiled, and thus is not on the path?

./test.sh, its telling it look here. That thing won't be on your path

Why does typing just cat work? If I compile a binary executable file will typing its name work? Why or why not

/bin/cat is the full path, typing just cat works. Unix defines an enviornment variable called the path. The path contains many direcotiries. The system will search for the first program that matches name cat. /bin is located on this path. So the system will find cat in the /bin directory and it'll run that cat. If you're inputting and running a program by just referencing the name of the program and that program is not on this path then it won't execute and the system will say "Can't find your program". That's because the program is in your current working directory. If you type something in it'll look on the path. This path, does not include the current working directory that you're located in. So If you compile some program and it has an executable binary with a name and you just type that name in the directory it will not run because the system can't find it on the path. You must use ./name because the '.' sais look here in the current working directory.

String comparators

==, !=, <,>

Environment variables

A Unix system concept associated with a process. They help DEFINE a process. When a shell forks() off a new process, the child inherits a clone of the parent (the shell), so the child inherits all enviornment variables of the shell. One way of passing data to programs. Every process has a set of enviornment variables that get passed to it

What are signals

A signal is a message to a process. The kernel sends a signals to the destination process when an external event has occurred. The kernel kind of forces that destination process to react to that external event. The destination process can install signal handlers to deal with these signals that get received. We can also block certain signals and unblock signals depending on the operation we're doing. This is important because sometimes you dont want to be interrupted by a signal when you're adding a certain operation. For example when the parent is adding jobs to a list, the parent should block SIGChild signals because the child has to be added to the job list before you reap them. Or else you may get a SIGCHILD signal and then reap that child in the signal handler, and then after that, add dead child to the active job list, and thats a problem because the child has already died.

Pipes Explain how processes in a pipeline run?

All processes in a pipeline run concurrentlyy. The order in which they run is not defined. The last one may run first. They do not execute in sequence necessarily. Can have multiple pipes on a single command line du | sort -n | sort If Im getting my input from the pipe, I will try and read from the pipe and if theres nothing there, I will wait until there is something there If Im at the start of the pipe and im going to write to it, and if the pipe array gets filled up, I will wait until theres room in it

Process

An instance of a running program. A set of instructions and associated memory. Processes provide each program with two key abstractions: Logical Control Flow: (Each program has the ILLUSION of having full exclusive use and access of the cpu), provided by kernel mechanism called context switching. Private address space: (Each program has the illusion of exclusive use of main memory)-provided by kernel mechanism called virtual memory.

Why do we need jumps

Another form of exceptional control flow. When I setjmp, im storing all the stack pointer, all that info. I make a bunch of calls, and error happens, and instead of returning many times, I can longjmp back to where I originally setjmp. The one catch is that say within one stackframe, I call a function p1(), p1() calls p2(), p2(_) setjmp, and then p2() returns to p1(), p1() calls p3(), p3() longjmps. PROBLEM! When p2 returned, that stack frame got deleted, and so wherever you setjmp is no longer there. In fact, when you called p3(), you overwrote the stack frame, and so you would basically jumping to you are at p3(). So you can only longjmp within the stack that setjmp() was called in

Regular Expression: *

Any preceding character any number of times

Associative Arrays in Awk

Arrays in awk can be indexed using info/text rather than numbers Array lookups like a database search.

AWK Scripts have three segments enclosed in braces. list explain

BEGIN segment - executed once before reading any input Middle segment (unlabeled) -executed for every line read by awk END Segment - executed once after line processed by the middle segments auto looping feature

Explain the uses for each AWK Segment

BEGIN: declare initialize variables MIDDLE segment: Automatic while loop that automatically runs once for each line of input data. - by default, delimited by new lines END segment: processing/writing output

How do I write an arithmetic operation in bash

Bash has no datatypes, assumes every command is text so you have to LET it know that you're doing arithmetic let variable=5+3 let variable=2**3 Equivelenttly variable = $((5+4))

Why Bash and awk?

Bash: default shell for linux Awk: Good for text processing parsing, handles input from pipes

By default processes will run where?

By default processes run in the foreground. When a process is running in the foreground, it prevents us from running other programs, thus we can't input new commands to the command line. This is because foreground jobs, we wait for them in the main function before we proceed with another job (reaping them in the signal handler). So if we're waiting for a foreground job to finish, we can't add a new job. So if I start off a process in the command line, again, its automatically put in the foreground, and so while that job is running, if I try execute ls, it wont work, and in part because input from my keyboard is not. going to the shell, it's going to the process. SO thats why we can still send signals to pause a foreground process. But we can't start new processes or execute new commands

How do I kill a process running in the foreground

CTRL-C sends a signal to kill a process running in the foreground

How do I pause a process running in the foreground and have it run in the background?

CTRL-Z pauses a foregeround job. We can run programs in the background by inputting: bg

How do I pause a process that's running in the foreground

CTRL-Z will send a signal to the process to pause. This gives us control back so we can run other processes or commands while that process is stopped

Loops and piping

Can pipe output of commands to loops. Loops run for every line of input.txt cat input.txt | while read lineVal do echo $lineVal done Can also pipe output of a loop to some command cat input.txt | while read lineVal do echo $lineVal done | grep "boring" Notice here we dont have to specify a file for grep, because its STDIN is pulling from the pipe. So the pipe redirects the STDOUT of the echos in the loop, to the STDIN of grep.

How do I run multiple commands in parallel?

Commands can only be run in parallel if they're put in the background (command1;command2;command3) & () executes every command in parentheses (in sequence) in parallel to the main script & - puts the command in the background

Explain how ctrl-c works

Ctrl-c kills the current running program. It does that by sending a SIGINT signal (kernel sends the Signal to the running process thats an interrupt that stops execution of the program

Shell - what's the default shell, what is it used for

Default is bash. Bash is just another program. Shell is used for executing programs on the command line and manipulating their input and output. Every command is a program with its own executable or binary

List the 3 default file descriptors (data streams)

Every Unix process starts off with 3 default files: stdin - input stream from keyboard the input to your program stdout - output stream to terminal stderr - output stream to terminal

~ is what directory ? What's the shorthand for it

Home directory. $Home

What can I do in the Shell

Inside the shell itself on the command line you can enter in programming statements: If statements, loops.

What is a script what goes in a script?

List of commands connected by Unix PIPES and redirector operators

Enviornment variables

Named variable, value pairs associated with the enviornment of a process. child Inherits from a parent process, another way to pass data to process

Disadvantages of scripts

Numeric processing Slower than a compiled program - because parsing every line of code and executing it -cant optimize code like compilers, as compilers have this global view of the code No datatypes in bash scripts Not good for arithmetic Hard to debug large programs infeasible

Pipes - What are they and how do they work. Describe it as best possible

Pipes are applying two forms of re-direciton Redirecting the STDOUT of one process and redirecting the STDIN of another process Control data streams between files/programs Has one read end and one write end Pipes take the STDOUT from one program/process and pipe it to the STDIN of another program/process Gives the ability to control and redirect data streams, STDOUT,STDIn, STDERROR Unix programs usually will take input from STDINPUT (keyboard) but they are designed to check for a pipe EX:

the dot '.' is an alias for what directory

Present working directory

Shell

Primary interface to Unix. Powerful user interface. It is a programming environment for composing and COORDINATING programs (PROCESSES) together and interactively (Can execute programs in the shell! (Shell forks off processes and execs a new program within them). It is both an interactive programming environment and a programmable user interface

AWK

Programming language for text processing -Not a shell enviornment like bash shell -can write full programs -AUTOMATIC LINE PARSER

What is a script?

Programs written in a Unix Shell language. Bash scripting: Code written for an interpreter. Default interpreter for Unix. The shell is an interpreter, it takes the line you wrote, parses the line and looks to what is in the line, executing any code, one line at a time

Unix redirectors explain

Redirectors allow us to redirect standard output to a file echo "hello world" > helloworld.txt -echo program just prints text to STDOUT >- redirects standard out to a new file echo "hello word" >> helloworld.txt >> means append STDOUT from echo to a file rather than overwrite Redirecting standard error to file blah 2> helloworld.txt ->redirects file descriptor 2 to a file -file descriptor2 is a the STDERROR -blah is not a program, so this error message telling you that gets sent/written to STDERROR, but instead of going to the terminal, we redirect to a file Redirecting standard input from a file Here rather than listing a file on the command line, you specify it should be read from standard in. STDIN redirects to read not from keyboard bu rather come from a file cat <hello.txt EX: bash.halscript < answer.txt this script designed to.pause and take user input from the keyboard. Instead we redirect halscripts STDIN to come from a file instead of the default STDIN (keyboard)

Shell glowing vs regular expressions

Shell glowing: The pattern matching done on the command line regular expressions: Pattern matching supported by awk, grep

Why are Singlas important?

Signals are a core OS abstraction/communciation tool, its one way that an OS can exert control over a program after it's started. A signal is software interrupt. Signals are a form of interprocess communication Enables one process to send a notification to another process Signals are numbered 1-30

When to use scripting.

Simple OS-related functionality like text processing and reading and writing to lots of files

What are directories

So Unix is a uniform file system that starts at the root "/" or the root directory and so we can get to any other directory from the root, and directories hold other directories or files, and within a directory I can execute a command to operate on a file within that directory. I can also execute programs within a directory or if I have an executable file within the directory I can execute that using ./

Describe a background job

So background jobs can still pollute our screen, just like foreground jobs. The difference with background jobs, is that when we put a job in the background, we're not waiting on it to finish. We loop back to the top of the loop, and we're ready to execute another command, or start a new process. And so that means multiple background jobs can be executed once. So if I type a command while a background job is running, it'll still execute the command, my terminal is still reading from my keyboard because the other processs is still running in the background Use the & at the end of a program execution to execute a background job

What is the root user

Special user that has access to everything on the system and can do whatever they want

$1 $2 $3

Special variables. First, second, third command line arguments when you call a program #!/bin/bash echo $1 echo$2 ./test.sh hello goodbye hello goodbye

Why are datatypes important?

The datatype determines how we operate on the bits within an operation. You know so you have an int And you're adding it to a float, well the int gets cast to a float, and we do float addition. The important thing here is that int with int addition is a different operation than adding two floats. So the datatypes that you choose to operate on matter, and it dictates how the bits get operated on

What is the path?

The path is an enviornment variable in Unx. The path is compromised of many directories. The shell has many enviornment variables that make up what the shell does. The home directory is also an enviornment variable. If you type a command into command line, the shell is going to look on the path to see if the command is located in one of these directories. For example the cat command is located in /bin/cat on the path, and so if you type just cat and the shell well execute the first instance of cat that it finds along the path. This can lead to problems say if you have many versions of a certain command because the shell will take the first version that it finds, and so you may be running the wrong version of your program. If the command is not found within one of these directories along the path, an error will print to screen

Binary/Executable

The program that you execute in the shell. 'ls' is a program.

What is the Kernel

The resident-memory OS. The operating system code, the colloction of things that compromise it. It deals with exceptions. So if an exception happens, we transition to kernel code, and the kernel is going to send the process a signal saying hey "An external has occurred, deal with it" and so that process can catch a signal with a signal handler or potentially ignore it.

What type of file system is Unix

UNIFORM file system starting at the root. "ls /" shows all the root direcotires

Unix has a Uniform file system. What does that mean?

Unix has a uniform file system starting at the root. It hides from you the devices that the file system is composed of or the nature of those devices.

The root of the filesystem is what?

Unix has uniform file system that starts at the root. Root of the file system is at the slash "/". The root is the top directory and thus you can get to every other file using this slash "/". "ls /" lists the root directory at the top of the file system tree.

File Permissions - what are they?

Unix supports multiple users and groups. Every file is owned by some user and some group. ls -la for permissions. The three permission types: read: 'r' - can read the file write: 'w' - can only write the file (modify it ) execute: 'x' - can execute the file

How do I kill a background job? Why is this command able to kill background jobs

Use the KILL command. The KILL command is the only signal that's not catchable. The program cant see this signal being sent. All other signals, their default behavior may be to kill (like CTRL-C), but they are catchable/ignorable Every program that gets started has a PID KILL -9 <process ID>

How do I set a variable in bash

VARNAME=value (no datatypes)

Scripting

When you write interpreted programs in the shell, those are called scripts

Bash has no floating point, use

bc or awk (pipe to bc)

What does cat do? show how its used. Why might it be bad?

cat <filename1> <filename2> ... <filenameN> lets you display the contents of a concatination of the files in the terminal. Give it a bunch of files as inout and it dumps them out to the screen one after the other. Good for seeing what's in small files, It's bad for viewing large files. Used to move the contents of a file into standard output ( What you see on the screen). It's also useful for redirecting STDOUT of cat to a text file, or maybe piping STDOUT of cat to STDIN of another command.

How do I move up a directory

cd ..

How do I change or move into a directory

cd <directoryName>

How do I navigate to my home directory

cd ~

How do I change permissions for the three file permission roles

chmod - changes permissions for the three roles chmod u+r <filename> chmod g+x chmod o-w +: adds permissions for that entity -: Takes permissions away

How do I change permissions in bulk?

chmod 777 <filename> rwxrwxrwx == (111,111,111) = 777 everyone can read-write-execute rw-r--r-- == (110,100,100) = 644 the user can read and write, anyone else can only read rw------- == (110,000,000) = 600 -only the user can read and write

How do I put a process into the foreground?

fg

Lists don't have to be integers

for i in `ls`; do sleep 1; echo $I; done prints out every file in directory one a new line . Shell is mainly text processing. so useful

grep

globally search regex pattern and print used for pattern matching in files. -Applies <regex> to each line of a file by default -Prints to STDOUT all of the lines that match grep <regex> <fileName>

How do I search a file for a certain pattern?

grep <pattern> <file> globally search regex pattern and print. Searching utility, does pattern matching. Will output any lines of file that have pattern as substring

How do I get a scrollable interface for viewing large files

less <fileName> will give you a scrollable interface for viewing large text files.

List the files in the present working directory, or if specified, dir

ls <dir>. Or instead of specifying the <dir> just input ls in the directory that you're in, and that'll print out all the directories/files that reside within your working directory

Problem: I don't know how a command works, or how a certain option on a command works. How do I solve this problem?

man <thing> - tells you what is that command? What is this C standard library function? Check to see if the command has man page! EX: man grep man cat

How to see what the command does? What is this C standard Library Function?

man <thing> it's a manual page.

what does sleep(1)

pause for 1 second

How do I print a special character without it being interpreted in Bash?

pre-pend with the \ echo \$100 prints $100 use apostrophes echo '$100' prints $100 enclosing strings in quotes " " prevents most interpretations but not the big ones like $

echo

print arguments to screen (STDOUT) EX: echo $hello

echo

prints arguments to screen (STDOUT)

echo $?

prints exit code of the last process that ran successful-0 non-success-1 ‣ The shell is the parent of all processes that it spawns. Every command that you execute is a process that runs. ‣ All of those processes with some exit code ‣ that exit code tells you if that process ran successfully or not • If You want to check that you can look at $?

echo $$

prints process ID of running script

How do I print my working directory

pwd - print working directory

How do I remove files

rm <file1> <file2> <filen>

How to loop over many iterations in bash?

seq <start> <increment> <end> use outputs to replace with the output of seq. Can write on command line separated by semicolons for i in `seq 1 1 1000`; do sleep 1; echo $i; done prints 1 2 3 4 each new line

What happens when I have an unsigned and a signed variable in the same expression

signed gets implicitly cast to unsigned

Advantages of scripts

text processing Good OS integration: -Coordinating processes -manipulating files -Good for rapid prototyping in a few lines of code

Function arguments are passed through

the command line function ThisIsAFunctionName { //echo the first two arguments echo $1 echo $2 } thisIsAFunctionName arg1 arg2 echo the return value is $? -Use $1 and $2 to reference function arguments -Use $? to return the return value of the function -All vars in a function global by default, prepend w/local to localize it

How to set a variable as output of a command

use backticks around the command. This sets the variable equal to the STDOUT of some command. Here instead of echo printing to terminal, STDOUT is redirected to the variable VARNAME = `echo $VARNAME`

Regular Expressions

used for generalized pattern matching. Especially used by grep to match and print certain patterns

What are the three roles in file permissions?

user - "u" group "g" anyone "o" • First set of three: ◦ drwxrwxrwx ◦ the user permissions, can the user that owns the file, can they read the file, can they write to the file, can they execute the file • The second set of three ◦ drwxrwxrwx ◦ The group (a collection of user), if someone is in that group, can they read the file, can they write to the file, can they execute to the file • The third set of three ◦ drwxrwxrwx ◦ Final set of three: thats anyone, can anyone read the file, can they write to the file, can they execute the file

What do I do if I want to find which specific binary the path points to.

which cat : /bin/cat

Summarize Awk

• Basically: You've manipulated the text in the file • Ran the awk command, and the awk runs automatically, once for every line ◦ right so the awk autoloop will run once for every line, and so for every line we print $3 "," $2 "," $1 "," • This code just prints out the third word , second word , first word • Reverses each line's words and puts commas between them • AWK gives you auto ability to parse and manipulate text thats very easy • Its a full on programming language so you can do a lot of stuff in awk • or you can use it like a command that you run as a part of bash *awk is good for text processing but unlike bash not good for coordinating programs, redirecting, piping all that

More pipes

• Pipes are a re-direction of stdin and stdoutput • If we start up a program here, it gets some stdinput (coming traditionally from a keyboard) and traditionally its stdoutput goes to the screen • But what that pipe does is it takes the stdoutput and redirects it to be the stdinput of another process • It changes the STDoutput of the process on the left side of the pipe (wont go to screen, will go to the other process) and it changes the STDInput of the process on the right side of the pipe to pull from the STDOutput of the other process.

Without examples do the redirector operators

• Pipes | (redirect STDOUT of one program to go to STDIN of another program) • Redirectors STDOUT > (redirect STDOUT of a command/program to go to a file instead of screen) • STDERROR 2> (redirect STDERROR from one program to go to a file instead of the screen) • STDIN < (redirect STDIN of a program to come from a file instead of the keyboard/user input

* vs the '..'

• [a-z].ot ◦ Match any three character string ending in ot (that starts with lowercase character. and it can only be one letter on the left. ◦ BECAUSE the thing that preceeds the dot '.' sais match a single character in the set a-z ‣ anything that starts with a lowercase character: cot, pot, lot • [a-z]*ot ◦ Not only will it match cot, it will match ccccccccccccot or would match just ot because its zero or more times ◦ If the string contained 'ot', it will match because the * matches zero or more times, so the regex doesn't even have to apear • the dot '.' - the preceeding character has to apear ATLEAST once and only once so it would not match ot with the dot '.' because it has to have a letter first