Executing commands in ruby

Ruby allows many different ways to execute a command or a sub-process. In this article we are going to see some of them.

backtick

1. Returns standard output

backtick returns the standard output of the operation.

output = `ls`
puts "output is #{output}"

Result of above code is

$ ruby main.rb
output is lab.rb
main.rb

2. Exception is passed on to the master program

Backtick operation forks the master process and the operation is executed in a new process. If there is an exception in the sub-process then that exception is given to the main process and the main process might terminate if exception is not handled.

In the following case I am executing xxxxx which is not a valid executable name.

output = `xxxxxxx`
puts "output is #{output}"

Result of above code is given below. Notice that puts was never executed because the backtick operation raised exception.

$ ruby main.rb
main.rb:1:in ``': No such file or directory - xxxxxxx (Errno::ENOENT)
	from main.rb:1:in `<main>'

3. Blocking operation

Backtick is a blocking operation. The main application waits until the result of backtick operation completes.

4. Checking the status of the operation

To check the status of the backtick operation you can execute $?.success?

output = `ls`
puts "output is #{output}"
puts $?.success?

Notice that the last line of the result contains true because the backtick operation was a success.

$ ruby main.rb
output is lab.rb
main.rb
true

backtick returns STDOUT. backtick does not capture STDERR . If you want to learn about STDERR then checkout this excellent article .

You can redirect STDERR to STDOUT if you want to capture STDERR using backtick.

output = `grep hosts /private/etc/* 2>&1`

5. String interpolation is allowed within the ticks

cmd = 'ls'
`#{cmd}`

6. Different delimiter and string interpolation

%x does the same thing as backtick. It allows you to have different delimiter.

output = %x[ ls ]
output = %x{ ls }

backtick runs the command via shell. So shell features like string interpolation and wild card can be used. Here is an example.

$ irb
> dir = '/etc'
> %x<ls -al #{dir}>
=> "lrwxr-xr-x@ 1 root  wheel  11 Jan  5 21:10 /etc -> private/etc"

system

system behaves a bit like backtick operation. However there are some differences.

First let's look at similarities.

1. Blocking operation

Just like backtick, system is a blocking operation.

2. Eats up all exceptions

system eats up all the exceptions. So the main operation never needs to worry about capturing an exception raised from the child process.

output = system('xxxxxxx')
puts "output is #{output}"

Result of the above operation is given below. Notice that even when exception is raised the main program completes and the output is printed. The value of output is nil because the child process raised an exception.

$ ruby main.rb
output is

3. Checking the status of the operation

system returns true if the command was successfully performed ( exit status zero ) . It returns false for non zero exit status. It returns nil if command execution fails.

system("command that does not exist")  #=> nil
system("ls")                           #=> true
system("ls | grep foo")                #=> false

exec

exec replaces the current process by running the external command. Let's see an example.

Here I am in irb and I am going to execute exec('ls').

$ irb
e1.9.3-p194 :001 > exec('ls')
lab.rb  main.rb

nsingh ~/dev/lab 1.9.3
$

I see the result but since the irb process was replaced by the exec process I am no longer in irb .

Behind the scene both system and backtick operations use fork to fork the current process and then they execute the given operation using exec .

Since exec replaces the current process it does not return anything if the operation is a success. If the operation fails then `SystemCallError is raised.

sh

sh actually calls system under the hood. However it is worth a mention here. This method is added by FileUtils in rake. It allows an easy way to check the exit status of the command.

require 'rake'
sh %w(xxxxx) do |ok, res|
   if !ok
     abort 'the operation failed'
   end
end

popen3

If you are going to capture stdout and stderr then you should use popen3 since this method allows you to interact with stdin, stdout and stderr .

I want to execute git push heroku master programmatically and I want to capture the output. Here is my code.

require 'open3'
cmd = 'git push heroku master'
Open3.popen3(cmd) do |stdin, stdout, stderr, wait_thr|
  puts "stdout is:" + stdout.read
  puts "stderr is:" + stderr.read
end

And here is the output. It has been truncated since rest of output is not relevant to this discussion.

stdout is:
stderr is:
-----> Heroku receiving push
-----> Ruby/Rails app detected
-----> Installing dependencies using Bundler version 1.2.1

The important thing to note here is that when I execute the program ruby lab.rb I do not see any output on my terminal for first 10 seconds. Then I see the whole output as one single dump.

The other thing to note is that heroku is writing all this output to stderr and not to stdout .

Above solution works but it has one major drawback. The push to heroku might take 10 to 20 seconds and for this period we do not get any feedback on the terminal. In reality when we execute git push heroku master we start seeing result on our terminal one by one as heroku is processing things.

So we should capture the output from heroku as it is being streamed rather than dumping the whole output as one single chunk of string at the end of processing.

Here is the modified code.

require 'open3'
cmd = 'git push heroku master'
Open3.popen3(cmd) do |stdin, stdout, stderr, wait_thr|
  while line = stderr.gets
    puts line
  end
end

Now when I execute above command using ruby lab.rb I get the output on my terminal incrementally as if I had typed git push heroku master .

Here is another example of capturing streaming output.

require 'open3'
cmd = 'ping www.google.com'
Open3.popen3(cmd) do |stdin, stdout, stderr, wait_thr|
  while line = stdout.gets
    puts line
  end
end

In the above case you will get the output of ping on your terminal as if you had typed ping www.google.com on your terminal .

Now let's see how to check if command succeeded or not.

require 'open3'
cmd = 'ping www.google.com'
Open3.popen3(cmd) do |stdin, stdout, stderr, wait_thr|
  exit_status = wait_thr.value
  unless exit_status.success?
    abort "FAILED !!! #{cmd}"
  end
end

popen2e

popen2e is similar to popen3 but merges the standard output and standard error .

require 'open3'
cmd = 'ping www.google.com'
Open3.popen2e(cmd) do |stdin, stdout_err, wait_thr|
  while line = stdout_err.gets
    puts line
  end

  exit_status = wait_thr.value
  unless exit_status.success?
    abort "FAILED !!! #{cmd}"
  end
end

In all other areas this method works similar to popen3 .

Process.spawn

Kernel.spawn executes the given command in a subshell. It returns immediately with the process id.

irb(main)> pid = Process.spawn("ls -al")
=> 81001