ruby/spec

Guards

Overview

The ruby/spec project intends to provide a complete specification of the Ruby language and its libraries. There is a single standard implementation of Ruby. The standard includes the stable, released versions available from http://ruby-lang.org. At present, the standard is 1.8.6, 1.8.7, and 1.9.2. Collectively, the standard is often referred to as MRI.

The challenge for ruby/spec is to correctly spec the different behaviors across versions, platforms, and implementations. To do this, ruby/spec depends on guards provided by MSpec. Guards are methods that may or may not take arguments and operate by yielding to a block if certain conditions are true. If the conditions for the guard are not true, the guard method does not yield to the block and the specs contained in the block are not run.

The guards serve two functions: 1) controlling which specs are run; 2) documenting the specs. The documentation function of the guards is as important for ruby/spec as controlling which specs are run. Additionally, the guard structure itself was chosen to be visually and conceptually similar to the describe/it blocks in the specs.

The guard blocks should be placed around describe or it blocks. Guards should not be placed inside it blocks. The it description string should concisely describe the facet of behavior illustrated by the example code. If a guard is present, it necessarily means the guarded code behaves differently. That difference should be described in the it string. In the case of before or after actions, use judgment to keep the specs concise but clear. If clarity would be enhanced by a few more lines of code, put the guards around the before or after action itself. Clarity always trumps counts of lines of code.

There are five categories of guards:

1. versions
2. platforms
3. bugs
4. implementations
5. environments

The specific guards in each of these categories are explained below.

1. Versions

Different versions of Ruby have some methods that behave differently. That sounds circular, but it is the essence of software versions. To handle different version behaviors, MSpec provides the ruby_version_is guard.

The ruby_version_is guard has two forms. One form gives a single version. Any implementation whose version number is greater than or equal to the version in the guard will run the guarded specs. The second form takes a range of versions. Any implementation whose version number is in the range will run the guarded specs. See the examples below and note the range form of the guard may include or exclude the ending version.

# Yields block if version >= 1.8.6-p114.
ruby_version_is "1.8.6.114" do
  it "returns true" do
  end
end

# Yields block if version is in the 1.8 series, except 1.8.7 and beyond.
ruby_version_is "1.8" .. "1.8.6" do
  it "returns nil" do
  end
end

# Yields block if version < 1.9.
ruby_version_is "" ... "1.9" do
  it "returns false" do
  end
end

The ruby_version_is guard takes one argument that may be a string or a range of two strings. The format of the string is A.B.C.D, where:

• A is the major version
• B is the minor version
• C is the tiny (teeny) version
• D is the patchlevel

The string is converted to a number on which comparisons between versions can be made so that, for instance, “1.8.6.37” is less than “1.8.6.112”.

The range behaves as expected, respecting #exclude_end?. In the example above, “” … “1.9” means every version before 1.9.

2. Platforms

A single version of Ruby may have different behaviors depending on the platform on which it runs. MSpec provides several guards for these situations:

• big_endian
• little_endian
• platform_is
• platform_is_not

The big_endian guard yields to the block if the platform is big endian. Likewise for the little_endian guard.

The platform_is and platform_is_not guards are more complex. As their names suggest, they are inverses.

platform_is :linux, :bsd do
  it "opens the file" do
  end
end

The guard above will yield if RUBY_PLATFORM matches either “linux” OR “bsd”.

platform_is :linux, :wordsize => 32 do
  it "opens the file" do
  end
end

The guard above will yield if RUBY_PLATFORM matches “linux” AND the processor word size is 32-bit.

platform_is_not :windows, :wordsize => 32 do
  it "opens the file" do
  end
end

The guard above will yield if RUBY_PLATFORM does not matches “windows” AND the processor word size is not 32-bit.

Special functionality exists for matching :windows and :java as platforms. For details, refer to the MSpec source.

platform_is :os => [:darwin, :bsd] do
  it "opens the file" do
  end
end

The guard above will yield if Config::CONFIG['host_os'] matches either “darwin” OR “BSD”. If Config::CONFIG['host_os'] is not set in rbconfig, the :os parameters will be matched against RUBY_PLATFORM instead.

3. Bugs

Sometimes a bug is discovered in the standard. In this case, we do two things:

1. File a ticket on the “bug tracker”:http://bugs.ruby-lang.org/ to find out if the suspected behavior is actually considered a bug.
2. Add a ruby_bug guard that wraps the spec showing what is considered to be the correct behavior.

ruby_bug "#5555", "1.8.6.114" do
  it "returns the sum" do
    (1 + 1).should == 2
  end
end

The above guard will NOT yield to the block on any version of the standard less than or equal to 1.8.6 patchlevel 114.

The ruby_bug guard serves three purposes:

1. It documents that there is a bug in a particular version of the standard and refers to the ticket for that bug.
2. It provides the correct behavior description for all implementations
3. It prevents the spec for the correct behavior from running (and failing) on versions of the standard implementation that have the bug and have already been released.

If it is determined that the behavior is not a bug but rather a version difference, the ruby_bug guard should be replaced by an appropriate ruby_version_is guard.

4. Implementations

Ideally, all Ruby implementations would have exactly the same behaviors. This is not realistically possible given differences in the underlying technology, for instance, whether the process fork facility is available.

It should be obvious, but it bears repeating, that in a specification for a standard, there should be an absolute minimum of incompatible behavior. These guards are not provided to make it easy to be inconsistent. Rather, they are provided to make specifying a single standard as simple as possible, recognizing that 100% conformity is an impossible ideal.

Recall that the purpose of the guards are both to control which specs are run AND document the specs. There are four distinct situations covered by the five guards below. Each of the guards documents this difference.

• compliant_on
• not_compliant_on
• not_supported_on
• deviates_on
• extended_on

Keep in mind that the arguments to these guards are communal property (as are all the specs) and respect them as you would want to be respected. There is no concept of opt-in or opt-out here. Every implementation is responsible for ensuring that their implementation’s behavior is accurately represented in one of these compliance scenarios. If an implementation has an excessive number of non-compliant behaviors, this will be clearly visible in the specs.

4.1 compliant_on / not_compliant_on

The compliant_on and not_compliant_on guards are inverses. They document that the enclosed spec passes or does not pass on the listed implementations or platforms.

compliant_on :jruby, :rubinius do
  it "returns true" do
  end
end

The spec above will run ONLY on the standard, JRuby, or Rubinius. The compliant_on guard will not run on any other implementation or platform except the ones listed.

Note that this guard could prevent the guarded spec from running on an implementation that passes the spec. It is important to only use it in cases where the behavior is clearly supported only on the listed implementations. Generally, it is better to use not_compliant_on to explicitly blacklist implementations.

There is no need to list :ruby for this guard. There is only one standard and specs are always expected to run correctly on the standard. If there is version-specific behavior or a bug in the standard, use the ruby_version_is or ruby_bug guard instead.

not_compliant_on :rubinius do
  it "returns false" do
  end
end

The spec above will run on EVERY implementation except Rubinius. The not_compliant_on guard will always run on the standard.

It should be obvious that the compliant_on guard is more convenient when the number of implementations that conform to the standard is small compared to the total number of implementations. The not_compliant_on guard is useful when the number of non-conforming implementations is small. Another way to look at this, compliant_on essentially whitelists the compliant implementations, while not_compliant_on blacklists the non-comforming implementations.

It is reasonable to assume that if there is a not_compliant_on guard for a particular implementation, then there is also a deviates_on or extended_on guarded spec for the same facet of behavior. This is not enforced in any way by the system of guards. It makes sense that if a facet of behavior is not consistent with the standard, then that can be illustrated by another spec. This is not always the case. For instance, the standard may call a particular method on an instance that an alternative implementation has no need to call. The alternative implementation may behave the same in every aspect except calling the method. In such case, the not_compliant_on guard on that one facet of behavior is sufficient. It does not enhance the specs to add a deviates_on spec that merely creates a mock should_not_receive expectation. This obviously is in the gray line between implementation and interface.

4.2 not_supported_on

The not_supported_on guard documents that there is no behavior like that described in the guarded spec for the listed implementations.

not_supported_on :jruby do
  it "forks the process" do
  end
end

The above spec will run on EVERY implementation except JRuby. In particular, it will always run an the standard. Further, it will raise an exception if passed :ruby.

The key difference between not_supported_on and the compliance guards above is that not_supported_on offers no alternative to the standard behavior.

4.3 deviates_on

If an implementation does not conform to the standard behavior but instead offers an alternative behavior, the spec illustrating that is wrapped in a deviates_on guard.

deviates_on :rubinius do
  it "coerces to two Bignums" do
  end
end

The above spec will run ONLY on Rubinius. More than one implementation can be listed. This guard will NEVER run on the standard and will raise an exception if passed :ruby.

4.4 extended_on

If an implementation offers a behavior that does not exist at all in the standard, the spec illustrating that behavior is wrapped in an extended_on guard.

extended_on :rubinius do
  it "returns an immutable vector" do
  end
end

The above spec will run ONLY on Rubinius. More than one implementation can be listed. This guard will NEVER run on the standard and will raise an exception if passed :ruby.

5. Environments

The following guards are broadly grouped by their relation to how the specs are run. This is referred to as the spec environment. It includes guards for which runner (e.g. RSpec or MSpec) is executing the specs, which classes are loaded when the specs run, and whether the process running the specs has superuser privileges.

• runner_is
• runner_is_not
• conflicts_with
• as_superuser
• quarantine!

5.1 runner_is / runner_is_not

These two guards were initially added to protect specs whose behavior would change in the presence of certain standard library classes like Rational. This situation has been taken over by the conflicts_with guard below. These guards should now only be used if the particular runner framework itself is an issue.

runner_is :rspec do
  it "does something that only runs under RSpec" do
  end
end

runner_is :mspec do
  it "does something that only runs under MSpec" do
  end
end

5.2 conflicts_with

This guard wraps specs for methods whose behavior may be changed incompatibly by certain other classes.

conflicts_with :SomeClass do
  it "returns an Integer" do
  end
end

The above guard will NOT yield if SomeClass is defined.

5.3 as_superuser

Some Ruby methods will only behave as expected if the process running the code example has superuser privileges.

as_superuser do
  describe "File.chown" do
  end
end

The guard above will only yield if Process.euid == 0.

5.4 quarantine!

The quarantine! guard will never yield to the block, so the specs inside the guard will not run. This guard is only used to temporarily disable a guard that is causing the standard to fail severely (for example, by causing a segfault) AND the correctness of the spec is suspect. The guard allows the spec to be investigated but not cause any failures.

If a spec exposes a bug that is causing a segfault, the ruby_bug guard should be used.

quarantine! do
  it "does something that causes a segfault" do
  end
end