Today I Learned

14 posts about #testing

Regexes are supported by Capybara has_content?

Problem

I am trying to assert the presence of a button on the page by checking the button text:

expect(page).to have_content("Close")

Unfortunately, the page also has a dropdown that contains the option Closed.

Solution

Regex to the rescue!

expect(page).to have_content(/Close\b/)

Add have_selector() matcher to RSpec Request Specs

If you are used to writing controller specs, you are probably comfortable with the `have_selector matcher. However, in request specs this matcher is not available. By default, you can only do text search inside the request body which leads to brittle assertions.

You can add the have_selector matcher by updating your RSpec config to include the Capybara matchers on request specs as well.

RSpec.configure do |config|

config.include Capybara::RSpecMatchers, type: :request

end

Then you can write more confident Request specs by using assertions like expect(response.body).to have_selector('ul li', text: 'List content here!')

Creating multiple objects with Factory Girl

If you need to create more than one object from a factory, you can use the create_list method to accomplish this.

FactoryGirl.create_list(:user, 10, :supervisor)

Will create 10 supervisor users with the supervisor trait.

Capybara will skip invisible elements by default

While working on an acceptance test for a date range filter in GO, we were having an issue where Capybara couldn't find an element on the page, even though we could verify it was there. Eventually we realized that the element had an opacity of 0, and that Capybara was passing it over. To illustrate, imagine you have an element with the id #myElement.

CSS:

#myElement { opacity: 0; }

And in your Rails spec:

page.find("#myElement");

The spec will fail, because #myElement can't be found.

Fortunately, there is a visible option that can be set to false so that Capybara doesn't skip the element. So now, changing the line in the spec to:

page.find("#myElement", visible: false);

will cause it to pass.

Creating More than One Instance with FactoryGirl

In some test fixtures I need to create an array of instances . FactoryGirl provides the create_list method for exactly this purpose.

To create four shipments on an outbound trailer:

FactoryGirl.create_list(:shipment, 4, outbound_trailer: trailer)

In the example above, create_list returns an array containing the newly created shipments.

Compounding expectations in Rspec and Chai

When I had multiple expectations on the same object in rspec, I would write the code like so:

expect(page).to have_content("Foo")
expect(page).to have_content("Bar")
expect(page).to have_content("Other Stuff")

You can save yourself some typing if you instead use compound expectations, which is basically the usage of the and function after the previous expectation. Doing so will allow the previous code to be writted as such:

expect(page).to have_content("Foo")
  .and have_content("Bar")
  .and have_content("Other Stuff")

The same concept also exists in the Chai JavaScript testing library (documentation):

expect(page).to.contain("Foo")
  .and.contain("Bar")
  .and.contain("Other Stuff");

RSpec Matchers for Array Comparisons

Whenever you are matching arrays ask yourself two questions:

  • Is order important?
  • Am I matching a subset of the elements or all of the elements?

How I decide on a matcher:

  1. Choose between the eq and be matcher if order is important.
  2. Choose the include matcher if you want to match on a subset of the elements.
  3. Choose between the match_array and contain_exactly matcher if you want to match all elements (and order doesn't matter).

Below is an example of an improvement to a previously intermittent test. I replaced the eq matcher with the match_array matcher because I wanted to match all location_ids and order doesn't matter.

expect(location_ids).to eq([location_2.id, location_3.id])
expect(location_ids).to match_array([location_2.id, location_3.id])

The root cause of the intermittent test was that the locations were being retrieved from the database with no order specified. From the PostreSQL documentation: If sorting is not chosen, the rows will be returned in an unspecified order. The actual order in that case will depend on the scan and join plan types and the order on disk, but it must not be relied on.

Testing an Independent Mixin With RSpec

Objective: write a spec for the Inventory::Query mixin.

Note: the mixin is independent of the including class as it does not depend on any instance variables or instance methods.

Original Approach

class InventoryQueryTest
  include Inventory::Query
end
subject(:inventory_query) { InventoryQueryTest.new }

Preferred Approach

subject(:inventory_query) { (Class.new { include Inventory::Query }).new }

Advantage

Simpler and avoids polluting the global namespace with a test class.

Using WIP acceptance specs

Context

I usually follow the following approaching when working on a story:

  1. Write a failing acceptance spec.
  2. Do a spike to validate the proposed solution. Get the spike to pass.
  3. Capture learnings, and blow away the spike changes.
  4. Properly TDD away at the solution.

One annoyance with this approach was:

What do I do with the failing acceptance spec?

I usually try not to commit failing specs, since that makes git bisect less useful when I'm trying to see what broke it.

Solution

RSpec tags to the rescue.

Configure your specs to ignore wip specs by default:

RSpec.configure do |c|
  c.filter_run_excluding wip: true
end

Write a WIP spec:

it 'tests my yet-to-be-added feature', :wip do
  "my test"
end

Run the spec:

rspec my_acceptance_spec.rb --tag=wip

The acceptance spec can be committed, because it won't run as part of your regular test suite.

Once the story is done, make sure you remove the :wip flag!

Temporarily skip an RSpec example group

I knew about prefixing an RSpec example with x to skip it. I just found out that a describe or context example group can also be temporarily skipped using xdescribe and xcontext.

How did I find out? RTSL

Special bonus: the focus effect works similarly: fit, fdescribe, and fcontext.

Re-run only failed tests through Rspec

In cases where large refactoring is taking place and there are multiple tests across multiple files failing, there's an easy shortcut built into Rspec that allows you to re-run your specs but only the ones that failed.

The command

rspec --only-failures



This allows for a tighter feedback loop to get failing tests green.

Setup

This functionality doesn't come for free and some simple, but required, setup is necessary. Details about what's required can be found here.

The quick rundown is your Rspec configuration needs some extra flags set:

RSpec.configure do |c|
  c.example_status_persistence_file_path = "failing_specs.txt" 
  c.run_all_when_everything_filtered = true
end

This is required so Rspec will output any failing specs to a file and then read from it when --only-failures is specified.

Permanently using this, it's also a great idea to add it to your .gitignore file.

Capybara !has_content? vs has_no_content?

Let's say you're using RSpec and Capybara.

What's the difference between the following:

  expect(page).to_not have_content("not on page")

and

  expect(page).to have_no_content("not on page")

Assuming that the content not on page is expected to disappear from the page:

The first one will wait until the Capybara default wait time is over (2 seconds), then pass the assertion. The second one will pass as soon as not on page disappears from the page.

The reason the first one waits is because has_content? waits for the content to appear. When the timeout expires, it returns false, which passes the to_not assertion.

Never use to_not have_content or !has_content in Capybara.

Specify Multiple Examples by Line Number to RSpec

I can specify multiple examples as a colon-delimited list of line numbers to RSpec:

ryandv $ rspec my_spec.rb:2:8
Run options: include {:locations=>{"./my_spec.rb"=>[2, 8]}}
..

Finished in 0.00052 seconds (files took 0.08873 seconds to load)
2 examples, 0 failures

RSpec option to quickly fix transient failures

Problem:

How do I quickly find the smallest set of tests to reproduce a transient RSpec failure?

Context:

Sometimes, we find non-deterministic RSpec failures in our test suite that we often call transient failures. These tests only fail when they are run in a specific order (aka, using the same RSpec order seed), and they always pass when run in isolation.

Solution:

RSpec provides an option to find the minimum number of tests to run to reproduce the failure by doing bisection.

To use it, run RSpec with the order seed of one of the fail runs, and the --bisect option. For example,

rspec spec/cool_feature_spec.rb --seed 21952 --bisect

and RSpec will find the minimum reproduction command.

....
The minimal reproduction command is:
  rspec './spec/cool_feature_spec.rb[1:1:1,1:1:2]' --seed 21952