Linux I/O Redirection

Linux I/O Redirection

 

This post was last updated on May 26th, 2020 at 03:07 pm

Cool commands

  • The following command saves stdout and stderr to the files “out” and “err”, respectively.
  • [[email protected] /root]# ./cmd 1>out 2>err
    

  • The following command appends stdout and stderr to the files “out” and “err”, respectively.
  • [[email protected] /root]# ./cmd 1>>out 2>>err
    

  • The following command functions similar to the above two commands, but also copies stdout and stderr to the files “stdout” and “stderr”, respectively.
  • [[email protected] /root]# (((./cmd | tee stdout) 3>&1 1>&2 2>&3\
    |tee stderr) 3>&1 1>&2 2>&3) 1>out 2>err
    

    Note: Lines that end in a backslash are continued on the next line. Any such lines should be keyed in as one complete line. The lines are too long to fit on the web page without formatting them this way.

  • Cool Tar Command
  • [[email protected] /root]# (cd /var/ftp/pub/rh71prof/disk1.iso.dir \
    && tar -cvf - .) | (cd /var/ftp/pub/rh71prof/i386 && tar -xvf -)
    

    Note: Lines that end in a backslash are continued on the next line. Any such lines should be keyed in as one complete line. The lines are too long to fit on the web page without formatting them this way.

  • Cool Find Command – looks in each file for searchstring
  • [[email protected] /root]# find . -type f -exec grep -i \
    searchstring \{\} --with-filename \;
    

    Note: Lines that end in a backslash are continued on the next line. Any such lines should be keyed in as one complete line. The lines are too long to fit on the web page without formatting them this way.

  • Sample Test Script
  • #!/bin/sh
    #You can use this sample script for testing.  The echo
    #    statements explain how this script works.
    echo "This is Standard Out" >&1
    echo "This is Standard Error" >&2
    

  • For loop demonstrating stderr and stdout
  • echo Standard Out >stdout.out
    echo Standard Error >stderr.out
    for X in bzImage modules modules_install; do
         make $X;
    done 1>>stdout.out 2>>stderr.out
    

  • Redirection in Linux
  • 0 = stdin
    1 = stdout
    2 = stderr

  • Using the tee command to save stdout to tee.out. stdout is still displayed on the screen.
  • [[email protected] /root]# cmd | tee tee.out
    

  • Using the tee command to append stdout to tee.out. stdout is still displayed on the screen.
  • [[email protected] /root]# cmd | tee -a tee.out
    

  • Using the script command to capture both stderr and stdout
  • [[email protected] stdout]# script
    Script started, file is typescript
    [[email protected] stdout]# ./cmd
    This is Standard Out
    This is Standard Error
    [[email protected] stdout]# exit
    exit
    Script done, file is typescript
    [[email protected] stdout]# cat typescript 
    Script started on Thu Oct 11 11:47:36 2001
    [[email protected] stdout]# ./cmd
    This is Standard Out
    This is Standard Error
    [[email protected] stdout]# exit
    exit
    
    Script done on Thu Oct 11 11:47:39 2001
    [[email protected] stdout]# 
    

  • Notes about pipe “|”
    • Lets consider a channel one of stdout or stderr.
    • The pipe can only carry one channel, namely stdout.
    • When you have a command line which utilizes the pipe command, stderr gets sent to the display while stdout gets sent through the pipe on to the other awaiting commands.
    • Swapping stdout and stderr will allow stdout to be sent to the display while stderr can be sent through the pipe.
    • Pipe by itself will only take stdout. If you swap stdout and stderr before you get to the pipe then the pipe will take stdout (which is now stderr).
    • stdout and stderr can be combined and sent through the pipe; however, you have now combined both stderr and stdout and there is no way to separate them.
    • using a pipe in a subshell “()” causes stdout to go through the pipe and causes stderr to pass through the subshell back to the display. This provides a way to grab stdout and stderr. Basically you could get both stdout and stderr out of a subshell.
  • Order of redirecting
    • Method 1
    • [[email protected] /root]# cmd 2>&1 1>outfile
      

      #The above command causes the original stderr to go to stdout,
      # and causes the original stdout to go to outfile
      #Notice that 1=outfile when 2 is redefined, so 2 goes to outfile

    • Method 2
    • #The following command causes both the original stdout and stderr to go to outfile.

      [[email protected] /root]# cmd 1>outfile 2>&1
      

      #Notice that 1=outfile when 2 is redefined, so 2 goes to outfile (the two
      # channels are combined).


Capturing stderr with tee
Swapping stderr and stdout

  • The full command
[[email protected]rver /root]# (((./cmd | tee stdout) 3>&1 1>&2 2>&3 \
| tee stderr) 3>&1 1>&2 2>&3) 1>out 2>err

  • The walk through

The following section will walk through the theory behind this command. Once you have a general understanding of this theory, you should be able to regenerate this entire command without notes.

  1. Contents of the ./cmd script
  2. #!/bin/sh
    #You can use this sample script for testing.  The echo
    #    statements explain how this script works.
    echo "This is Standard Out" >&1
    echo "This is Standard Error" >&2
    

  3. Results from running the ./cmd script
  4. [[email protected] /root]# ./cmd
    This is Standard Out
    This is Standard Error
    [[email protected] /root]# 
    

    Although you see both lines printed on the screen, behind the scenes one actually went to stdout and the other went to stderr. If you were to do a pipe, only stdout goes through the pipe. Normally this is the desired effect.

  5. Capturing stdout
  6. The following will capture a copy of stdout and save it to a file called “stdout”

    [[email protected] /root]# ./cmd | tee stdout
    

    stdout goes through the pipe and tee is able to save a copy of it to the file “stdout”; however, we just lost control of stderr. stderr will not go through the pipe, instead it goes directly to our display.

  7. Gaining control of stderr and stdout.
  8. Lets gain control again of stderr and stdout. We do this by surrounding our command with a set of parenthesis.

    [[email protected] /root]# (./cmd | tee stdout)
    

  9. Swapping stdout and stderr.
  10. Now that we have captured stdout, we wish to capture stderr using tee as well. The pipe will only accept stdout, so we must swap stderr and stdout to do this.

    Note: The switch is using the standard variable switch (you have 2 variables and you need to switch the contents – you must bring in a 3rd temporary variable to hold the contents of one value so you can properly swap them).

    [[email protected] /root]# (./cmd | tee stdout) 3>&1 1>&2 2>&3
    

  11. Capturing stderr
  12. Now that we have swapped our stdout and stderr, lets hook up tee once again. tee will now capture stderr (tee believes that it is really stdout because stdout is the only thing that can come through the pipe).

    [[email protected] /root]# (./cmd | tee stdout) 3>&1 1>&2 2>&3 \
    | tee stderr
    

  13. Gaining control of stderr and stdout, for the 2nd time.
  14. Tee grabs stderr, but once again the channel that doesn’t go through the pipe gets sent to the display and we loose it. Lets capture both our stderr and stdout, once again, by using parenthesis.

    [[email protected] /root]# ((./cmd | tee stdout) 3>&1 1>&2 2>&3 \
    | tee stderr)
    

  15. Swapping stdout and stderr back to their normal state.
  16. Now we have, once again, captured both stderr and stdout for our use. Currently they are reversed. Lets switch them back for proper use in our pipeline. Again, lets use the standard variable switch to swap them around.

    [[email protected] /root]# ((./cmd | tee stdout) 3>&1 1>&2 2>&3 \
    | tee stderr) 3>&1 1>&2 2>&3
    

  17. Gaining control of stderr and stdout, for the 3rd time.
  18. At this point we have swapped stdout and stderr back to their normal positions; however, if we will be manipulating stdout and stderr any further, we should complete this command with either a pipe or another set of parenthesis.

    Since we want to be as complete as possible in this example we will use parenthesis. Using parenthesis will gain control over both stdout and stderr. Using a pipe will only gain control over stdout.

    Note: If we use a pipe or parenthesis the next process that hooks up to this command will see stderr and stdout in their proper place. If we don’t add the last set of parenthesis, or go through a pipe, the order will remain messed up.

    [[email protected] /root]# (((./cmd | tee stdout) 3>&1 1>&2 2>&3 \
    | tee stderr) 3>&1 1>&2 2>&3)
    

  19. Redirecting stdout and stderr to separate files
  20. Now lets do something productive with stdout and stderr so that we can really prove that everything went back to their proper place. Lets tell our command to redirect stdout to “out” and stderr to “err”.

    [[email protected] /root]# (((./cmd | tee stdout) 3>&1 1>&2 2>&3 \
    | tee stderr) 3>&1 1>&2 2>&3) 1>out 2>err
    

    Note: This last step is optional, normally you would insert your other required commands here, commands that would more than likely operate on stdout.

    Please note that the results for “out” and “err” are the same as when you run the following command. This proves that we restored stdout and stderr back to their normal usable posisitions. The above command; however, gives us the capability to copy out stdout and stderr using tee and still be able to use stdout and stderr like we always have.

    [[email protected] /root]# ./cmd 1>out 2>err
    

Other Good Reading:

Job Control in Linux
man bash

Previous Controlling Jobs in Linux
Next File Permissions in Linux
 

About author

Sibananda Sahu
Sibananda Sahu 207 posts

A Linux Kernel Developer and a Firmware Developer by profession. Have worked with few big companies: BROADCOM Corporation, Cypress Semiconductor, LSI Corporation, TOSHIBA Corporation, Western Digital; on various cutting edge technologies and product lines, such as: RAID storage Driver, SSD Firmware, WLAN Firmware etc. Having more than 9 years of experience in Software Engineering domain. Now, took a pledge to educate all aspirant students to teach about Linux Kernel Development.

View all posts by this author →

You might also like

How To 0 Comments

How to PDF Documents in Linux

This post was last updated on June 3rd, 2020 at 04:23 pmContents1 Using ps2pdf in Linux2 Using Windows and Linux to Generate PDF Documents3 Using Linux Only to Generate PDF

Linux Topics 0 Comments

How To Kill A Running Process In Linux/Unix?

This post was last updated on August 1st, 2020 at 12:44 pmIn Linux, a process is an instance of a program that is running on a computer. When a program

Linux Topics 0 Comments

How To Create A Password Protected Zip Archive In Linux?

This post was last updated on August 1st, 2020 at 12:25 pmFiles inside a zip archive can have important data. So you may want to protect it from the unusual

0 Comments

No Comments Yet!

You can be first to comment this post!

Leave a Reply