Meszaros G. xUnit Test Patterns Refactoring Test Code

Подождите немного. Документ загружается.

Four-Phase Test

How do we structure our test logic to make what we are testing obvious?

We structure each test with four distinct parts executed in sequence.

How It Works

We design each test to have four distinct phases that are executed in sequence:

ﬁ xture setup, exercise SUT, result veriﬁ cation, and ﬁ xture teardown.

• In the ﬁ rst phase, we set up the test ﬁ xture (the “before” picture) that

is required for the SUT to exhibit the expected behavior as well as any-

thing you need to put in place to be able to observe the actual outcome

(such as using a Test Double; see page 522).

• In the second phase, we interact with the SUT.

• In the third phase, we do whatever is necessary to determine whether

the expected outcome has been obtained.

• In the fourth phase, we tear down the test ﬁ xture to put the world back

into the state in which we found it.

Why We Do This

The test reader must be able to quickly determine what behavior the test is

verifying. It can be very confusing when various behaviors of the SUT are being

invoked—some to set up the pre-test state (ﬁ xture) of the SUT, others to exercise

Create

Testcase

Object

testMethod_n

Testcase

Object

testMethod_1

testMethod_n

Test

Suite

Object

Exercise

Create

Exercise

Create

Fixture

SUT

Run

Suite

Test Runner

Setup

Exercise

Verify

Teardown

Testcase

Class

Create

Testcase

Object

testMethod_n

Testcase

Object

testMethod_1

testMethod_n

Test

Suite

Object

Exercise

Create

Exercise

Create

Fixture

SUT

Run

Suite

Test Runner

Setup

Exercise

Verify

Teardown

Testcase

Class

Four-Phase

Test

Chapter 19 xUnit Basics Patterns

358

the SUT, and yet others to verify the post-test state of the SUT. Clearly identifying

the four phases makes the intent of the test much easier to see.

The ﬁ xture setup phase of the test establishes the SUT’s state prior to the test,

which is an important input to the test. The exercise SUT phase is where we actu-

ally run the software we are testing. When reading the test, we need to see which

software is being run. The result veriﬁ cation phase of the test is where we specify

the expected outcome. The ﬁ nal phase, ﬁ xture teardown, is all about housekeeping.

We wouldn’t want to obscure the important test logic with it because it is com-

pletely irrelevant from the perspective of Tests as Documentation (see page 23).

We should avoid the temptation to test as much functionality as pos-

sible in a single Test Method (page 348) because that can result in Obscure

Tests (page 186). In fact, it is preferable to have many small Single-Condition

Tests (see page 45). Using comments to mark the phases of a Four-Phase Test

is a good source of self-discipline, in that it makes it very obvious when our

tests are not Single-Condition Tests. It will be self-evident if we have multiple

exercise SUT phases separated by result veriﬁ cation phases or if we have inter-

spersed ﬁ xture setup and exercise SUT phases. Sure, the tests may work—but

they will provide less Defect Localization (see page 22) than if we have a bunch

of independent Single-Condition Tests.

Implementation Notes

We have several options for implementing the Four-Phase Test. In the simplest

case, each test is completely free-standing. All four phases of the test are con-

tained within the body of the Test Method. This structure implies we are using

In-line Setup (page 408) and either Garbage-Collected Teardown (page 500) or

In-line Teardown (page 509). It is the most appropriate choice when we are using

Testcase Class per Class (page 617) or Testcase Class per Feature (page 624) to

organize our Test Methods.

The other choice is to take advantage of the Test Automation Framework’s

(page 298) support for Implicit Setup (page 424) and Implicit Teardown (page 516).

We factor out the common ﬁ xture setup and ﬁ xture teardown logic into setUp and

tearDown methods on the Testcase Class (page 373). This leaves only the exercise SUT

and result veriﬁ cation phases in the Test Method. This approach is an appropriate

choice when we are using Testcase Class per Fixture (page 631). We can also use

this approach to set up common parts of the ﬁ xture when using Testcase Class per

Class (page 617) or Testcase Class per Feature or to tear down the ﬁ xture when

using Automated Teardown (page 503).

Four-Phase

Test

Four-Phase Test

359

Example: Four-Phase Test (In-line)

Here is an example of a test that is clearly a Four-Phase Test:

public void testGetFlightsByOriginAirport_NoFlights_inline()

throws Exception {

// Fixture setup

NonTxFlightMngtFacade facade =new NonTxFlightMngtFacade();

BigDecimal airportId = facade.createTestAirport("1OF");

try {

// Exercise system

List ﬂightsAtDestination1 =

facade.getFlightsByOriginAirport(airportId);

// Verify outcome

assertEquals( 0, ﬂightsAtDestination1.size() );

} ﬁnally {

// Fixture teardown

facade.removeAirport( airportId );

}

All four phases of the Four-Phase Test are included as in-line code. Because the

calls to Assertion Methods (page 362) raise exceptions, we need to surround the

ﬁ xture teardown part of the Test Method with a try/ﬁ nally construct to ensure that

it is run in all cases.

Example: Four-Phase Test (Implicit Setup/Teardown)

Here is the same Four-Phase Test with the ﬁ xture setup and ﬁ xture teardown

logic moved out of the Test Method:

NonTxFlightMngtFacade facade = new NonTxFlightMngtFacade();

private BigDecimal airportId;

protected void setUp() throws Exception {

// Fixture setup

super.setUp();

airportId = facade.createTestAirport("1OF");

}

public void testGetFlightsByOriginAirport_NoFlights_implicit()

throws Exception {

// Exercise SUT

List ﬂightsAtDestination1 =

facade.getFlightsByOriginAirport(airportId);

// Verify outcome

assertEquals( 0, ﬂightsAtDestination1.size() );

}

protected void tearDown() throws Exception {

Four-Phase

Test

Chapter 19 xUnit Basics Patterns

360

// Fixture teardown

facade.removeAirport(airportId);

super.tearDown();

}

Because the tearDown method is called automatically even after test failures,

we don’t need the try/ﬁ nally construct inside the Test Method. The downside,

however, is that references to our ﬁ xture must be held in instance variables

rather than local variables.

Four-Phase

Test

Four-Phase Test

361

Assertion Method

How do we make tests self-checking?

We call a utility method to evaluate whether an expected

outcome has been achieved.

A key part of writing Fully Automated Tests (see page 26) is to make them Self-

Checking Tests (see page 26) to avoid having to inspect the outcome of each

test for correctness each time it is run. This strategy involves ﬁ nding a way to

express the expected outcome so that it can be veriﬁ ed automatically by the

test itself.

Assertion Methods give us a way to express the expected outcome in a way

that is both executable by the computer and useful to the human reader, who can

then use the Tests as Documentation (see page 23).

How It Works

We encode the expected outcome of the test as a series of assertions that state what

should be true for the test to pass. The assertions are realized as calls to Assertion

Methods that encapsulate the mechanism that causes the test to fail. The Assertion

Methods may be provided by the Test Automation Framework (page 298) or by

the test automater as Custom Assertions (page 474).

Testcase

Object

testMethod_n

Testcase

Object

testMethod_1

Testcase

Class

testMethod_1

testMethod_n

Test

Suite

Object

Exercise

Create

Exercise

Create

Fixture

SUT

Run

Suite

Setup

Exercise

Verify

Teardown

Assertion

Method

Test Runner

Test failed

Testcase

Object

testMethod_n

Testcase

Object

testMethod_1

Testcase

Class

testMethod_1

testMethod_n

Test

Suite

Object

Exercise

Create

Exercise

Create

Fixture

SUT

Run

Suite

Setup

Exercise

Verify

Teardown

Assertion

Method

Test Runner

Test failed

Assertion

Method

Chapter 19 xUnit Basics Patterns

362

Why We Do This

Encoding the expected outcome using Conditional Test Logic (page 200) is very

verbose and makes tests hard to read and understand. It is also much more likely

to lead to Test Code Duplication (page 213) and Buggy Tests (page 260). Asser-

tion Methods help us avoid these issues by moving that complexity into reusable

Test Utility Methods (page 599); these methods can then be veriﬁ ed as working

correctly using Test Utility Tests (see Test Utility Method).

Implementation Notes

Although all members of the xUnit family provide Assertion Methods, they do

so with a fair degree of variability. The key implementation considerations are

as follows:

• How to call the Assertion Methods

• How to choose the best Assertion Method to call

• What information to include in the Assertion Message (page 370)

Calling Built-in Assertion Methods

The way the Assertion Methods are called from within the Test Method (page 348)

varies from language to language and from framework to framework. The lan-

guage features determine what is possible and preferable, while the framework

builders chose which options to use. The names these developers chose for the

Assertion Methods were inﬂ uenced by how they chose to access them. Here are

the most common options for accessing the Assertion Methods:

• The Assertion Methods are inherited from a Testcase Superclass (page 638)

provided by the framework. Such methods may be invoked as though

they were provided locally on the Testcase Class (page 373). The original

version of Java’s JUnit, for example, used this approach by providing a

Testcase Superclass that inherits from the class Assert, which contains the

actual Assertion Methods.

• The Assertion Methods are provided via a globally accessible class or

module. They are invoked using the class or module name to fully qualify

the Assertion Method name. NUnit, for example, uses this approach

[e.g., Assert.isTrue(x);]. JUnit does allow assertions to be invoked as static

Assertion

Method

Assertion Method

363

methods on the Assert class [e.g., Assert.assertTrue(x)] but this is not usually

necessary because they are inherited via the Testcase Superclass.

• The Assertion Methods are provided as “mix-ins” or macros. Ruby’s Test::

Unit, for example, provides the Assertion Methods in a module called

Assert that can be included in any class,

thereby allowing the Assertion

Methods to be used as though deﬁ ned within the Testcase Class [e.g.,

assert_equal(a,b)]. CppUnit, by contrast, deﬁ nes the Assertion Methods as

macros, which are expanded before the compiler sees the code.

Assertion Messages

Assertion Methods typically take an optional Assertion Message as a text param-

eter that is included in the output when the assertion fails. This structure allows the

test automater to explain to the test maintainer exactly which Assertion Method

failed and to better explain what should have occurred. The error detected by the

test will be much easier to debug if the Assertion Method provides more informa-

tion about why it failed. Choosing the right Assertion Method goes a long way

toward achieving this goal because many of the built-in Assertion Methods provide

useful diagnostic information about the values of the arguments. This is especially

true for Equality Assertions.

One of the biggest differences between members of the xUnit family is where

the optional Assertion Message appears in the argument list. Most members

tack it on to the end as an optional argument. JUnit, however, makes the Asser-

tion Message the ﬁ rst argument when it is present.

Choosing the Right Assertion

We have two goals for the calls to Assertion Methods in our Test Methods:

• Fail the test when something other than the expected outcome occurs

• Document how the SUT is supposed to behave (i.e., Tests as Documen-

tation)

To achieve these goals we must strive to use the most appropriate Assertion

Method. While the syntax and naming conventions vary from one member of

the xUnit family to the next, most provide a basic set of assertions that fall into

the following categories:

This approach is particularly useful when we are building Mock Objects (page 544) because

these objects are outside the Testcase Class but need to invoke Assertion Methods

Assertion

Method

Chapter 19 xUnit Basics Patterns

364

• Single-Outcome Assertions such as fail; these take no arguments because

they always behave the same way.

• Stated Outcome Assertions such as assertNotNull(anObjectReference) and

assertTrue(aBooleanExpression); these compare a single argument to an

outcome implied by the method name.

• Expected Exception Assertions such as assert_raises(expectedError)

{ codeToExecute }

; these evaluate a block of code and a single expected

exception argument.

• Equality Assertions such as assertEqual(expected, actual); these compare

two objects or values for equality.

• Fuzzy Equality Assertions such as assertEqual(expected, actual, tolerance);

these determine whether two values are “close enough” to each other

by using a “tolerance” or “comparison mask.”

Variation: Equality Assertion

Equality Assertions are the most common examples of Assertion Methods.

They are used to compare the actual outcome with an expected outcome that is

expressed in the form of a constant Literal Value (page 714) or an Expected Object

(see State Veriﬁ cation on page 462). By convention, the expected value is speci-

ﬁ ed ﬁ rst and the actual value follows it. The diagnostic message that is generated

by the Test Automation Framework makes sense only when they are provided in

this order. The equality of the two objects is usually determined by invoking the

equals method on the expected object. If the SUT’s deﬁ nition of equals is not what

we want to use in our tests, either we can make Equality Assertions on individual

ﬁ elds of the object or we can implement our test-speciﬁ c equality on a Test-Speciﬁ c

Subclass (page 579) of the Expected Object.

Variation: Fuzzy Equality Assertion

When we cannot guarantee an exact match due to variations in precision or

expected variations in value, it may be appropriate to use a Fuzzy Equality

Assertion. Typically, these assertions look just like Equality Assertions with the

addition of an extra “tolerance” or “comparison map” parameter that speciﬁ es

how close the actual argument must be to the expected one. The most common

example of a Fuzzy Equality Assertion is the comparison of ﬂ oating-point num-

bers where the limitations of arithmetic precision need to be accounted for

by providing a tolerance (the maximum acceptable distance between the two

values).

Assertion

Method

Assertion Method

365

We use the same approach when comparing XML documents where direct

string comparisons may result in failure owing to certain ﬁ elds having unpre-

dictable content. In this case, the “fuzz” speciﬁ cation is a “comparison schema”

that speciﬁ es which ﬁ elds need to match or which ﬁ elds should be ignored. This

kind of Equality Assertion is very similar to asserting that a string conforms to

a regular expression or other form of pattern matching.

Variation: Stated Outcome Assertion

Stated Outcome Assertions are a way of saying exactly what the outcome should

be without passing an expected value as an argument. The outcome must be

common enough to warrant a special Assertion Method. The most common

examples are as follows:

• assertTrue(aBooleanExpression), which fails if the expression evaluates to

FALSE

• assertNotNull(anObjectReference), which fails if the objectReference doesn’t

refer to a valid object

Stated Outcome Assertions are often used as Guard Assertions (page 490) to

avoid Conditional Test Logic.

Variation: Expected Exception Assertion

In languages that support block closures, we can use a variation of a Stated

Outcome Assertion that takes an additional parameter specifying the kind of

exception we expect. We can use this Expected Exception Assertion to say, “Run

this block and verify that the following exception is thrown.” This format is more

compact than using a try/catch construct. Some typical examples follow:

• should: [aBlockToExecute] raise: expectedException in Smalltalk’s SUnit

• assert_raises( expectedError) { codeToExecute } in Ruby’s Test::Unit

Variation: Single-Outcome Assertion

A Single-Outcome Assertion always behaves the same way. The most commonly

used Single-Outcome Assertion is fail, which causes a test to be treated as a

failure. It is typically used in two circumstances:

• As an Unﬁ nished Test Assertion (page 494) when a test is ﬁ rst identiﬁ ed

and implemented as a nearly empty Test Method. By including a call to

fail, we can have the Test Runner (page 377) remind us that we still have

a test to ﬁ nish writing.

Assertion

Method

366

Chapter 19 xUnit Basics Patterns

• As part of a try/catch (or equivalent) block in an Expected Exception Test

(see Test Method) by including a call to fail in the try block immediately

after the call that is expected to throw an exception. If we don’t want

to assert something about the exception that was caught, we can avoid

an empty catch block by using the Single-Outcome Assertion success to

document that this is the expected outcome.

One circumstance in which we really should not use Single-Outcome Assertions

is in Conditional Test Logic. There is almost never a good reason to include

conditional logic in a Test Method, as there is usually a more declarative way

to handle this situation using other styles of Assertion Methods. For example,

use of Guard Assertions results in tests that are more easily understood and less

likely to yield incorrect results.

Motivating Example

The following example illustrates the kind of code that would be required for

each item we wanted to verify if we did not have Assertion Methods. All we

really want to do is this:

if (x.equals(y)) {

throw new AssertionFailedError(

"expected: <" + x.toString() +

"> but found: <" + y.toString() + ">");

} else { // Okay, continue

// ...

}

Unfortunately, this code will cause a NullPointerException if x is null, and it would

be hard to distinguish this exception from an error in the SUT. Thus we need to

put some guard clauses around this functionality so that we always throw an

AssertionFailedException:

if (x == null) { // cannot do null.equals(null)

if (y == null ) { // they are both null so equal

return;

} else {

throw new AssertionFailedError(

"expected null but found: <" + y.toString() +">");

}

} else if (!x.equals(y)) { // comparable but not equal!

throw new AssertionFailedError(

"expected: <" + x.toString() +

"> but found: <" + y.toString() + ">");

} // equal

Assertion

Method

Assertion Method

367