Shell Scripts Saving Your Work ██ Why Write Scripts? 2 / 57 ██ The Case for Automation Top three reasons to write shell scripts: • Automation • Automation • Automation Other reasons: • Speed - computers are faster than humans at repetitive tasks • Accuracy - humans make mistakes, especially when bored • Reproducibility - run the exact same commands again later • Documentation - a script shows exactly what you did Anything you can type at the command line can be saved in a script and run later. 3 / 57 ██ Building Your Personal Toolkit As you learn new commands, save useful combinations in scripts. Over time, you'll build a library of tools customized to your work: • Scripts to set up your environment • Scripts to process data files • Scripts to automate tedious tasks • Scripts to avoid remembering complex command sequences This is one of the most valuable skills you'll learn in this class. 4 / 57 ██ Our First Shell Script 5 / 57 ██ What is a Shell Script? A shell script is just a text file containing commands. Instead of typing commands one at a time, you: 1. Write them in a file 2. Run the file The shell reads the file and executes each command in order. 6 / 57 ██ So we need a text editor Let's install nano (if it isn't already installed).
7 / 57 ██ Creating a Shell Script Three things you need to do: 1. Add the shebang - First line must be #!/bin/bash
2. Make it executable - Run chmod +x script.sh
3. Run it - Use ./script.sh (note the ./)
8 / 57 ██ The Shebang Line The first line of a script tells the system which program should run it: #!/bin/bash
This is called the "shebang" (or "hashbang"): • #! - the shebang characters
• /bin/bash - path to the bash shell
Without this line, the system doesn't know how to interpret your script. 9 / 57 ██ Making Scripts Executable By default, new files are not executable. You need to give yourself permission: $ chmod +x myscript.sh
chmod = "change mode"
+x = "add execute permission"
You only need to do this once per script. 10 / 57 ██ Running Your Script To run a script in the current directory, use ./:
$ ./myscript.sh
Why ./? The shell only looks for commands in certain directories. The current directory is usually not one of them (for security reasons).
./ tells the shell "look right here in the current directory."
11 / 57 ██ Demo - First Script 12 / 57 ██ The Two-Terminal Workflow A useful workflow for developing scripts: Terminal 1: Text editor with your script open
Terminal 2: Command line to run and test your script
This way you can: • Edit your script • Save it • Switch to the other terminal • Run it immediately • See the results • Switch back and make changes Pro Tip: use entr to run the script automatically!
13 / 57 ██ Demo - Two-Terminal Workflow 14 / 57 ██ Useful Tools We can use tools to help: • entr : automatically run the script when it changes.
• shellcheck : analyze our script for potential problems.
██ First Useful Script Let's write a script to download and untar the module sandboxes. What we will need: • wget : command line tool to download files from the internet.
██ Variables 15 / 57 ██ Why Use Variables? Variables let you: • Store values you want to use multiple times • Make scripts easier to modify • Make scripts more readable • Avoid repetition (and typos) 16 / 57 ██ Setting Variables Syntax for setting a variable: x=value
msg="text with spaces"
Important: No spaces around the = sign!
x=10 # correct
x = 10 # WRONG - shell thinks "x" is a command
x= 10 # WRONG
x =10 # WRONG
17 / 57 ██ Using Variables To use a variable's value, put it inside ${}:
msg="Hello World"
echo ${msg}
Output: Hello World
The ${} tells the shell "replace this with the variable's value."
18 / 57 ██ Variables in Text Variables can be expanded inside quoted text: name="Alice"
echo "Hello, ${name}!"
Output: Hello, Alice!
count=42
echo "There are ${count} files."
Output: There are 42 files.
19 / 57 ██ Building Up Variables Variables can be set using other variables: first="Hello"
second="World"
message="${first} ${second}"
echo ${message}
Output: Hello World
You can also append to a variable: msg="Step 1 done."
msg="${msg} Step 2 done."
echo ${msg}
Output: Step 1 done. Step 2 done.
20 / 57 ██ Demo - Variables 21 / 57 ██ Environment Variables The shell provides some built-in variables called "environment variables": • ${HOME} - your home directory
• ${USER} - your username
• ${PWD} - current working directory
echo "You are ${USER}"
echo "Your home is ${HOME}"
echo "You are in ${PWD}"
These are useful for writing scripts that work for any user. 22 / 57 ██ Script Arguments 23 / 57 ██ Making Scripts Flexible So far, our scripts do the exact same thing every time. What if we want a script that can work on different files or values? We can pass arguments to scripts, just like we pass arguments to commands:
$ ./process.sh data.txt
$ ./process.sh results.csv
24 / 57 ██ Positional Parameters When you pass arguments to a script, the shell stores them in special variables: • $1 - first argument
• $2 - second argument
• $3 - third argument
• ... and so on #!/bin/bash
echo "First argument: $1"
echo "Second argument: $2"
$ ./myscript.sh hello world
First argument: hello
Second argument: world
25 / 57 ██ Using Positional Parameters A script to back up a file: #!/bin/bash
cp $1 $1.bak
echo "Backed up $1 to $1.bak"
$ ./backup.sh important.txt
Backed up important.txt to important.txt.bak
Now the same script works for any file! 26 / 57 ██ Combining Variables and Arguments You can use positional parameters with other variables: #!/bin/bash
filename=$1
backup_dir="${HOME}/backups"
cp ${filename} ${backup_dir}/${filename}.bak
echo "Backed up ${filename} to ${backup_dir}"
This makes the script more readable and easier to modify. 27 / 57 ██ Demo - Script Arguments 28 / 57 ██ Conditionals 29 / 57 ██ Making Decisions in Scripts Sometimes you want your script to do different things depending on the situation: • Does the file exist before we try to read it? • Did the user provide the right number of arguments? • Did the previous command succeed? The if statement lets your script make decisions.
30 / 57 ██ The if Statement if [ condition ]; then
# commands to run if condition is true
fi
The fi ("if" backwards) marks the end of the block.
Important: The spaces inside [ ] are required!
if [ -f data.txt ]; then
echo "data.txt exists"
fi
31 / 57 ██ Common Test Conditions File tests: • [ -f file ] — file exists and is a regular file
• [ -d dir ] — directory exists
String tests: • [ -z "$var" ] — string is empty
• [ "$a" = "$b" ] — strings are equal
Number tests: • [ "$a" -eq "$b" ] — numbers are equal
• [ "$a" -lt "$b" ] — a is less than b
• [ "$a" -gt "$b" ] — a is greater than b
32 / 57 ██ if/else You can provide an alternative when the condition is false: if [ -f "$1" ]; then
echo "Processing $1..."
wc -l "$1"
else
echo "Error: $1 not found"
fi
This pattern is great for checking that a file exists before working with it. 33 / 57 ██ Checking Script Arguments A practical use — make sure the user provides the required arguments: #!/bin/bash
set -e
if [ -z "$1" ]; then
echo "Usage: ./backup.sh filename"
exit 1
fi
cp "$1" "$1.bak"
echo "Backed up $1 to $1.bak"
exit 1 stops the script and signals that something went wrong.
34 / 57 ██ Demo - Conditionals 35 / 57 ██ Loops 36 / 57 ██ Repeating Commands Loops let you run the same commands multiple times with different values. This is where scripts really shine — instead of typing the same command 100 times, write it once in a loop. 37 / 57 ██ The for Loop for variable in list; do
# commands using $variable
done
A simple example: for name in Alice Bob Carol; do
echo "Hello, ${name}!"
done
Output: Hello, Alice!
Hello, Bob!
Hello, Carol!
38 / 57 ██ Looping Over Files One of the most common uses — process every file matching a pattern: for file in *.txt; do
echo "Processing ${file}..."
wc -l "${file}"
done
This runs wc -l on every .txt file in the current directory.
39 / 57 ██ Looping with a Sequence Use seq to generate a range of numbers:
for i in $(seq 1 5); do
echo "Run number ${i}"
done
Output: Run number 1
Run number 2
Run number 3
Run number 4
Run number 5
This is useful for repeating experiments or creating numbered files. 40 / 57 ██ Combining Loops with Variables and Arguments A script that backs up multiple files: #!/bin/bash
set -e
backup_dir="${HOME}/backups"
mkdir -p "${backup_dir}"
for file in "$@"; do
cp "${file}" "${backup_dir}/${file}.bak"
echo "Backed up ${file}"
done
"$@" expands to all arguments passed to the script:
$ ./backup_all.sh notes.txt data.csv report.md
Backed up notes.txt
Backed up data.csv
Backed up report.md
41 / 57 ██ Demo - Loops 42 / 57 ██ Error Handling Basics 43 / 57 ██ When Things Go Wrong What happens if a command in your script fails? By default, the script keeps running the next commands anyway. This can be very bad...
44 / 57 ██ A Dangerous Example Consider this script: #!/bin/bash
cd
cd tmp
rm -rf *
What happens if ~/tmp does not exist?
45 / 57 ██ A Dangerous Example #!/bin/bash
cd # go to home directory
cd tmp # ERROR: tmp doesn't exist, but script continues!
rm -rf * # deletes everything in HOME directory!
The cd tmp fails, but we're still in the home directory.
Then rm -rf * deletes everything there.
This is why error handling matters. 46 / 57 ██ The Solution: set -e Add set -e near the top of your script:
#!/bin/bash
set -e
cd
cd tmp
rm -rf *
Now if cd tmp fails, the script stops immediately.
The -e flag means "exit on error."
47 / 57 ██ Always Use set -e Make it a habit to include set -e in your scripts:
#!/bin/bash
set -e
# rest of your script...
This is a simple safety measure that can prevent disasters. 48 / 57 ██ Demo - Error Handling 49 / 57 ██ Putting It All Together 50 / 57 ██ A Complete Script Template Here's a good starting template for any script: #!/bin/bash
set -e
# Your commands go here
As you learn more, you'll add to this template. 51 / 57 ██ Script Development Workflow 1. Start with a working command at the command line 2. Create a new script file with the shebang and set -e
3. Add your command to the script 4. Make it executable with chmod +x
5. Test it 6. Add variables to make it flexible 7. Add arguments to make it reusable 8. Test again 52 / 57 ██ Practice 1. Write a script that prints a greeting with your name using a variable. 2. Modify the script to take a name as an argument instead. 3. Write a script that creates a backup of a file by copying it to filename.bak.
4. Write a script that downloads and extracts a sandbox tarball (combine what you learned in Module 01 with scripting). 53 / 57 ██ Summary We learned: • Why scripts are valuable (automation, reproducibility, building your toolkit) • How to create a script (shebang, chmod +x, ./) • The two-terminal workflow for development • Variables (setting with =, using with ${})
• Environment variables (${HOME}, ${USER})
• Positional parameters ($1, $2, $3)
• Conditionals (if/else, file tests, argument checking)
• Loops (for loops over lists, files, and sequences)
• Error handling with set -e
Key habit: When you figure out how to do something useful, save it in a script!
54 / 57 ██ Next Time In Module 03, we'll learn: • Standard input, output, and error streams • Redirecting output to files (>, >>)
• Pipes — connecting commands together (|)
• Text processing commands (grep, wc, sort, uniq, cut)
• Command substitution ($(command))
55 / 57 ██ Last Slide This space intentionally left blank 56 / 57 57 / 57