Chapter 5. I/O redirection

Table of Contents

Simple redirections
What are standard input and standard output?
The redirection operators
Advanced redirection features
Use of file descriptors
Examples
Filters
More about grep
Filtering output
Summary
Exercises

Abstract

This chapter describes more about the powerful UNIX mechanism of redirecting input, output and errors. Topics include:

  • Standard input, output and errors

  • Redirection operators

  • How to use output of one command as input for another

  • How to put output of a command in a file for later referrence

  • How to append output of multiple commands to a file

  • Input redirection

  • Handling standard error messages

  • Combining redirection of input, output and error streams

  • Output filters

Simple redirections

What are standard input and standard output?

Most Linux commands read input, such as a file or another attribute for the command, and write output. By default, input is being given with the keyboard, and output is displayed on your screen. Your keyboard is your standard input (stdin) device, and the screen or a particular terminal window is the standard output (stdout) device.

However, since Linux is a flexible system, these default settings don't necessarily have to be applied. The standard output, for example, on a heavily monitored server in a large environment may be a printer.

The redirection operators

Output redirection with > and |

Sometimes you will want to put output of a command in a file, or you may want to issue another command on the output of one command. This is known as redirecting output. Redirection is done using either the > (greater-than symbol), or using the | (pipe) operator which sends the standard output of one command to another command as standard input.

As we saw before, the cat command concatenates files and puts them all together to the standard output. By redirecting this output to a file, this file name will be created - or overwritten if it already exists, so take care.

nancy:~> cat test1
some words

nancy:~> cat test2
some other words

nancy:~> cat test1 test2 > test3

nancy:~> cat test3
some words
some other words
[Warning]Don't overwrite!

Be careful not to overwrite existing (important) files when redirecting output. Many shells, including Bash, have a built-in feature to protect you from that risk: noclobber. See the Info pages for more information. In Bash, you would want to add the set -o noclobber command to your .bashrc configuration file in order to prevent accidental overwriting of files.

Redirecting nothing to an existing file is equal to emptying the file:

nancy:~> ls -l list
-rw-rw-r--    1 nancy   nancy     117 Apr  2 18:09 list

nancy:~> > list

nancy:~> ls -l list
-rw-rw-r--    1 nancy   nancy       0 Apr  4 12:01 list

This process is called truncating.

The same redirection to an nonexistent file will create a new empty file with the given name:

nancy:~> ls -l newlist
ls: newlist: No such file or directory

nancy:~> > newlist

nancy:~> ls -l newlist
-rw-rw-r--  1 nancy   nancy	    0 Apr  4 12:05 newlist

Chapter 7, Home sweet /home gives some more examples on the use of this sort of redirection.

Some examples using piping of commands:

To find a word within some text, display all lines matching pattern1, and exclude lines also matching pattern2 from being displayed:

grep pattern1 file | grep -v pattern2

To display output of a directory listing one page at a time:

ls -la | less

To find a file in a directory:

ls -l | grep part_of_file_name

Input redirection

In another case, you may want a file to be the input for a command that normally wouldn't accept a file as an option. This redirecting of input is done using the < (less-than symbol) operator.

Below is an example of sending a file to somebody, using input redirection.

andy:~> mail mike@somewhere.org < to_do

If the user mike exists on the system, you don't need to type the full address. If you want to reach somebody on the Internet, enter the fully qualified address as an argument to mail.

This reads a bit more difficult than the beginner's cat file | mail someone, but it is of course a much more elegant way of using the available tools.

Combining redirections

The following example combines input and output redirection. The file text.txt is first checked for spelling mistakes, and the output is redirected to an error log file:

spell < text.txt > error.log

The following command lists all commands that you can issue to examine another file when using less:

mike:~> less --help | grep -i examine
  :e [file]      Examine a new file.
  :n          *  Examine the (N-th) next file from the command line.
  :p          *  Examine the (N-th) previous file from the command line.
  :x          *  Examine the first (or N-th) file from the command line.

The -i option is used for case-insensitive searches - remember that UNIX systems are very case-sensitive.

If you want to save output of this command for future reference, redirect the output to a file:

mike:~> less --help | grep -i examine > examine-files-in-less

mike:~> cat examine-files-in-less
  :e [file]      Examine a new file.
  :n          *  Examine the (N-th) next file from the command line.
  :p          *  Examine the (N-th) previous file from the command line.
  :x          *  Examine the first (or N-th) file from the command line.

Output of one command can be piped into another command virtually as many times as you want, just as long as these commands would normally read input from standard input and write output to the standard output. Sometimes they don't, but then there may be special options that instruct these commands to behave according to the standard definitions; so read the documentation (man and Info pages) of the commands you use if you should encounter errors.

Again, make sure you don't use names of existing files that you still need. Redirecting output to existing files will replace the content of those files.

The >> operator

Instead of overwriting file data, you can also append text to an existing file using two subsequent greater-than signs:

Example:

mike:~> cat wishlist
more money
less work

mike:~> date >> wishlist

mike:~> cat wishlist
more money
less work
Thu Feb 28 20:23:07 CET 2002

The date command would normally put the last line on the screen; now it is appended to the file wishlist.