Test runner in Rails 5

If you run bin/rails -h in a Rails 5 app, you will see a new command for running tests.

1$ bin/rails -h
2Usage: rails COMMAND [ARGS]
3
4The most common rails commands are:
5 generate    Generate new code (short-cut alias: "g")
6 console     Start the Rails console (short-cut alias: "c")
7 server      Start the Rails server (short-cut alias: "s")
8 test        Run tests (short-cut alias: "t")
9
Copy

Before Rails 5, we had to use bin/rake test to run tests. But in Rails 5, we can use bin/rails test. It is not just replacement of old rake task but it is backed by a test runner inspired from RSpec, minitest-reporters, maxitest and others.

Let's see what bin/rails test can do.

Running a single test

Now it is possible to run a single test out of the box using the line number of the test.

1$ bin/rails test test/models/user_test.rb:27
Copy

Rails will intelligently run the test from user_test.rb having line number 27. Note that the line number 27 does not need to be first line of the test. Below is an example.

122: def test_valid_user
223:   hash = { email: '[email protected]',
324:            first_name: 'John',
425:            last_name: 'Smith' }
526:
627:   user = User.new hash
728:
829:   assert user.valid?
930: end
Copy

In the above case, test_valid_user can be run as long as the line number provided is between 22 and 30.

Running multiple tests

You can also pass multiple test paths and Rails will run all of those tests.

1$ bin/rails test test/models/user_test.rb:27 test/models/post_test.rb:42
Copy

It is also possible to run all tests from a directory.

1$ bin/rails test test/controllers test/integration
Copy

Improved failure messages

When a test fails, Rails displays a command which can be used to just run the failed test.

1$ bin/rails t
2Run options: --seed 51858
3
4# Running:
5
6.F
7
8Failure:
9PostsControllerTest#test_should_get_new:
10Expected response to be a <success>, but was a <302> redirect to <http://test.host/posts>
11
12
13bin/rails test test/controllers/posts_controller_test.rb:15
Copy

I can simply copy bin/rails test test/controllers/posts_controller_test.rb:15 and rerun the failing test.

Failing fast

By default when a test fails then rails reports about the test failures and then moves on to the next test. If you want to stop running the test when a test fails then use option -f.

1$ bin/rails t -f
2Run options: -f --seed 59599
3
4# Running:
5
6..F
7
8Failure:
9PostsControllerTest#test_should_get_new:
10Expected response to be a <success>, but was a <302> redirect to <http://test.host/posts>
11
12
13bin/rails test test/controllers/posts_controller_test.rb:15
14
15Interrupted. Exiting...
16
17
18Finished in 0.179125s, 16.7481 runs/s, 22.3308 assertions/s.
19
203 runs, 4 assertions, 1 failures, 0 errors, 0 skips
Copy

Defer test output until the end of the full test run

By default when a test fails then rails prints F and then details about the failure like what assertion failed and how to re run the test etc.

If you want to have a clean output of . and F and would like all the test failures report to come at the every end then use option -d.

1$ bin/rails t -d
2Run options: -d --seed 29906
3
4# Running:
5
6..F...F
7
8Finished in 0.201320s, 34.7704 runs/s, 49.6721 assertions/s.
9
10  1) Failure:
11PostsControllerTest#test_should_create_post [/Users/prathamesh/Projects/fun/rails-5-test-runner-app/test/controllers/posts_controller_test.rb:19]:
12"Post.count" didn't change by 1.
13Expected: 3
14  Actual: 2
15
16
17  2) Failure:
18PostsControllerTest#test_should_get_new [/Users/prathamesh/Projects/fun/rails-5-test-runner-app/test/controllers/posts_controller_test.rb:15]:
19Expected response to be a <success>, but was a <302> redirect to <http://test.host/posts>
20
217 runs, 10 assertions, 2 failures, 0 errors, 0 skips
22
23Failed tests:
24
25bin/rails test test/controllers/posts_controller_test.rb:19
26bin/rails test test/controllers/posts_controller_test.rb:15
Copy

Better backtrace output

By default when an error is encountered while running the test then the output does not contain full stacktrace. This makes debugging little bit difficult.

1Error:
2PostsControllerTest#test_should_create_post:
3NameError: undefined local variable or method `boom' for #<PostsController:0x007f86bc62b728>
4    app/controllers/posts_controller.rb:29:in `create'
5    test/controllers/posts_controller_test.rb:20:in `block (2 levels) in <class:PostsControllerTest>'
6    test/controllers/posts_controller_test.rb:19:in `block in <class:PostsControllerTest
Copy

Now we can use -b switch, which will display complete backtrace of error message.

1$ bin/rails t -b
2
3Error:
4PostsControllerTest#test_should_create_post:
5NameError: undefined local variable or method `boom' for #<PostsController:0x007fc53c4eb868>
6    /rails-5-test-runner-app/app/controllers/posts_controller.rb:29:in `create'
7    /sources/rails/actionpack/lib/action_controller/metal/basic_implicit_render.rb:4:in `send_action'
8    /sources/rails/actionpack/lib/abstract_controller/base.rb:183:in `process_action'
9    /sources/rails/actionpack/lib/action_controller/metal/rendering.rb:30:in `process_action'
10    /sources/rails/actionpack/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
11    /sources/rails/activesupport/lib/active_support/callbacks.rb:126:in `call'
12.....
13    /sources/rails/activesupport/lib/active_support/testing/assertions.rb:71:in `assert_difference'
14    /rails-5-test-runner-app/test/controllers/posts_controller_test.rb:19:in `block in <class:PostsControllerTest>'
Copy

Leveraging power of Minitest

The test runner also leverages power of minitest by providing some handy options.

Switch -s to provide your own seed

Now we can also provide our own seed using -s switch.

1$ bin/rails t --s 42000
Copy

Switch -n to run matching tests

Switch -n will run tests matching the given string or regular expression pattern.

1$ bin/rails t -n "/create/"
2Run options: -n /create/ --seed 24558
3
4# Running:
5
6E
7
8Error:
9PostsControllerTest#test_should_create_post:
10NameError: undefined local variable or method `boom' for #<PostsController:0x007faa39c2df90>
11    app/controllers/posts_controller.rb:29:in `create'
12    test/controllers/posts_controller_test.rb:20:in `block (2 levels) in <class:PostsControllerTest>'
13    test/controllers/posts_controller_test.rb:19:in `block in <class:PostsControllerTest>'
14
15
16bin/rails test test/controllers/posts_controller_test.rb:18
17
18Finished in 0.073857s, 13.5396 runs/s, 0.0000 assertions/s.
19
201 runs, 0 assertions, 0 failures, 1 errors, 0 skips
Copy

Verbose output

It is also possible to see the verbose output using -v switch. It shows time required to run each test. This would help in detecting slow running tests.

1$ bin/rails t -v
2Run options: -v --seed 30118
3
4# Running:
5
6PostsControllerTest#test_should_destroy_post = 0.07 s = .
7PostsControllerTest#test_should_update_post = 0.01 s = .
8PostsControllerTest#test_should_show_post = 0.10 s = .
9PostsControllerTest#test_should_create_post = 0.00 s = F
10
11Failure:
12PostsControllerTest#test_should_create_post:
13"Post.count" didn't change by 1.
14Expected: 3
15  Actual: 2
16
17bin/rails test test/controllers/posts_controller_test.rb:19
18
19PostsControllerTest#test_should_get_new = 0.02 s = .
20PostsControllerTest#test_should_get_index = 0.01 s = .
21PostsControllerTest#test_should_get_edit = 0.00 s = .
22
23Finished in 0.210071s, 33.3220 runs/s, 47.6028 assertions/s.
24
257 runs, 10 assertions, 1 failures, 0 errors, 0 skips
Copy

Colored output

Now by default we will get colored output. No need to add additional gem to colored output.

With all these awesome features, testing Rails 5 apps has definitely become a better experience. Rails has shipped all these features within the framework itself so you don't have to use multiple gems and libraries to achieve all of these things.