Working with Python subprocess – Shells, Processes, Streams, Pipes, Redirects

Note

Much of the “What Happens When you Execute a Command?” is based on information in http://en.wikipedia.org/wiki/Redirection_(computing) so go there for the latest version. This post is released under the GFDL.

Contents

• What Happens When you Execute a Command?
  • Streams
• Working with the Shell
  • Redirecting Standard input and Standard Output to Files
  • Piping
  • Redirecting Standard Input and Standard Output to and from the Standard File Handles
  • Chained pipelines
  • Redirect to multiple outputs
  • Here Documents
• Introducing subprocess
• Using the Shell
• Strings or Argument Lists
• Without the Shell
  • Checking a Program is on the PATH
• Reading from Standard Output and Standard Error
  • Redirecting stderr to stdout
• Writing to Standard Input
  • The File-like Attributes
  • Reading and Writing to the Same Process
• Accessing Return Values, poll() and wait()
• Convenience Functions
• Understanding sys.argv
  • Shell Expansion
• Further Reading

In my last post I wrote about how to build a command line interface with sub-commands in Python. In this post I’m going to look at how you can interact with other command line programs using Python’s subprocess module.

What I want to be able to do is:

• Find out exactly what happens when you run commands on a command line
• Find out if a command exists and where it actually is
• Execute a command a command from Python either directly or via a shell
• Read from STDOUT and write to STDIN of a running process
• Check the exit status of a process
• Understand the role of Bash in interpreting patterns and sending them to command line programs