Part 0 : A word of caution

Using globs at the command line is very powerful…

Part 1 : Plain Text Files

Plain text files

• A lot of work we are going to do will be centered around creating, reading, and modifying plain text files.
• What is a plain text file?
  • Open up “LibreOffice” and create a new document.
  • Open up Text Editor, and create a new file.
  • Now open a terminal and cd to the directory that you just saved them in.
  • Run the cat command on each file.
• Plain text files contain just text.

Looking at file contents

There are several commands for looking at the content of files.

• cat : print contents of all arguments to screen
• head : print first 10 lines of all arguments to screen
• tail : print last 10 lines of all arguments to screen
• grep : look for a pattern in all arguments and print all matches to screen
• wc : print the number of lines, words, and characters in the file
• less : show the contents of a file in a “pager”

WARNING

If you enter any of the commands without a filename, they will go into “stdin mode”

$ cat

You have to press Ctr-C

Demo - file contents

demos/01-fine_contents.sh

The find command

• The find command is a tool for searching for files.
• It is very useful when you are in a new directory structure, or your trying to find some configuration file on your system.

find takes a directory named as an argument followed by options

Demo - find

demos/02-find.sh

Part 2 : Sending Output to a File

File redirection

• Anything that gets printed to the screen can be saved to a file.
  • cmd > file : whatever the command cmd outputs will be saved in a file named file. If file exists, it will be overwritten.
  • cmd >> file : whatever the command cmd outputs will be appended to the file named file. If file does not exists, it is created.
• Example:
  • tail -n 1 *.txt > last_lines.txt will create a file named last_lines.txt that contains the last line of all files in the current directory that end with .txt
  • tail -n 1 *.bin >> last_lines.txt will add the last lines of all files in the current directory that end with .bin to the file last_lines.txt.
  • tail -n 1 *.log > last_lines.txt will overwrite the file named last_lines.txt with the last line of all files in the current directory that end with .log

Demo - File Redirection

demos/03-file_redirection.sh

Part 3 : Text Editors

Text Editors

So far we have just been looking at and creating plain text files. What if we want to edit them?

Demo - Text Editors

demos/04-text_editors.sh

Part 4 : Inserting Output into the Command Line

Command Substitution

• The shell will run any commands in $(), and insert the output of the command into the command line as if you typed it in.
• If a file named files.txt contains a list of file names, then head $(cat files.txt) would run head on all of the files in the list.

Demo - Command Substitution

demos/05-command_substitution.sh

One last thing…

• So far, we have learned to use several different commands

  • cd
  • rm
  • mv
  • head
  • tail
  • cat
  • grep
  • etc., etc., etc.

• All of these commands take arguments, and most of them take options.

• It can be difficult to remember all of them

Manual pages

• All of these commands come with an instruction manual.
• We can read the manual using the man command.

$ man grep

Practice Problems

1. The sandbox directory contains a directory named simulations/, which contains a directory name backups.201601/. Go into this directory
   1. What is the 5th number in the second column of the file named data-01.txt?
   2. What is the second to last number in the second column of the file named data-02.txt?
   3. Which file in this directory has the most lines?
   4. Create a file named data-XX.txt that contains the lines from all the data files in this directory
   5. How many lines does the file you just created contain?
   6. Create a file named data-last_lines.txt that contains just the last line from all the data files in this directory
2. Go into the simulations/ directory of the sandbox.
   1. The directory contains several files that are named run-XXX.out. These files contain the output of some program that was used to processed data. Some of the runs “failed”, i.e. there was an error during the processing. When this happened, the processor printed the word “ERROR”, which is contained in the file. How many of the run-XXX.out files contain error messages?
   2. Create a list of the files that have error messages in them and how many error messages they have. Save this list to a file named errors.txt.
   3. The data processor also printed a warning if only a few data points were found during processing. In this case, the processor printed “WARNING” to the file. Create a list of all the files that contain warnings and/or errors named warnings_and_errors.txt