Meszaros G. xUnit Test Patterns Refactoring Test Code

Подождите немного. Документ загружается.

468

Chapter 21 Result Verification Patterns

Behavior Veriﬁ cation

How do we make tests self-checking when there is no state to verify?

We capture the indirect outputs of the SUT as they occur and compare

them to the expected behavior.

A Self-Checking Test (see page 26) must verify that the expected outcome has

occurred without manual intervention by whoever is running the test. But what

do we mean by “expected outcome”? The SUT may or may not be “stateful”; if

it is stateful, it may or may not be expected to end up in a different state after it

has been exercised. The SUT may also be expected to invoke methods on other

objects or components.

Behavior Veriﬁ cation involves verifying the indirect outputs of the SUT as it

is being exercised.

How It Works

Each test speciﬁ es not only how the client of the SUT interacts with it during the

exercise SUT phase of the test, but also how the SUT interacts with the compo-

nents on which it should depend. This ensures that the SUT really is behaving as

speciﬁ ed rather than just ending up in the correct post-exercise state.

Fixture

DOC

Exercise

Setup

Exercise

Verify

Teardown

SUT

Behavior

(Indirect

Outputs)

Verify

Fixture

DOC

Exercise

Setup

Exercise

Verify

Teardown

SUT

Behavior

(Indirect

Outputs)

Verify

Also known as:

Interaction

Testing

Behavior

Veriﬁ cation

469

Behavior Veriﬁ cation almost always involves interacting with or replacing

a depended-on component (DOC) with which the SUT interacts at runtime.

The line between Behavior Veriﬁ cation and State Veriﬁ cation (page 462) can

get a bit blurry when the SUT stores its state in the DOC because both forms

of veriﬁ cation involve layer-crossing tests. We can distinguish between the two

cases based on whether we are verifying the post-test state in the DOC (State

Veriﬁ cation) or whether we are verifying the method calls made by the SUT on

the DOC (Behavior Veriﬁ cation).

When to Use It

Behavior Veriﬁ cation is primarily a technique for unit tests and component tests. We

can use Behavior Veriﬁ cation whenever the SUT calls methods on other objects or

components. We must use Behavior Veriﬁ cation whenever the expected outputs of

the SUT are transient and cannot be determined simply by looking at the post-exercise

state of the SUT or the DOC. This forces us to monitor these indirect outputs as

they occur.

A common application of Behavior Veriﬁ cation is when we are writing our

code in an “outside-in” manner. This approach, which is often called need-driven

development, involves writing the client code before we write the DOC. It is

a good way to ﬁ nd out exactly what the interface provided by the DOC needs

to be based on real, concrete examples rather than on speculation. The main

objection to this approach is that we need to use a lot of Test Doubles (page 522)

to write these tests. That could result in Fragile Tests (page 239) because each

test knows so much about how the SUT is implemented. Because the tests

specify the behavior of the SUT in terms of its interactions with the DOC, a

change in the implementation of the SUT could break a lot of tests. This kind of

Overspeciﬁ ed Software (see Fragile Test) could lead to High Test Maintenance

Cost (page 265).

The jury is still out on whether Behavior Veriﬁ cation is a better approach

than State Veriﬁ cation. In most cases, State Veriﬁ cation is clearly necessary; in

some cases, Behavior Veriﬁ cation is clearly necessary. What has yet to be deter-

mined is whether Behavior Veriﬁ cation should be used in all cases or whether

we should use State Veriﬁ cation most of the time and resort to

Behavior Veriﬁ -

cation only when State Veriﬁ cation falls short of full test coverage.

Implementation Notes

Before we exercise the SUT by invoking the methods of interest, we must ensure

that we have a way of observing its behavior. Sometimes the mechanisms that the

Behavior Verification

Behavior

Veriﬁ cation

470

Chapter 21 Result Verification Patterns

SUT uses to interact with the components surrounding it make such observation

possible; when this is not the case, we must install some sort of Test Double to

monitor the SUT’s indirect outputs. We can use a Test Double as long as we have

a way to replace the DOC with the Test Double. This could be via Dependency

Injection (page 678) or by Dependency Lookup (page 686).

There are two fundamentally different ways to implement Behavior Veriﬁ ca-

tion, each with its own proponents. The Mock Object (page 544) community has

been very vocal about the use of “mocks” as an Expected Behavior Speciﬁ cation,

so it is the more commonly used approach. Nevertheless, Mock Objects are not

the only way of doing Behavior Veriﬁ cation.

Variation: Procedural Behavior Veriﬁ cation

In Procedural Behavior Veriﬁ cation, we capture the method calls made by the SUT

as it executes and later get access to them from within the Test Method (page 348).

Then we use Equality Assertions (see Assertion Method on page 362) to compare

them with the expected results.

The most common way of trapping the indirect outputs of the SUT is to

install a Test Spy (page 538) in place of the DOC during the ﬁ xture setup phase

of the Four-Phase Test (page 358). During the result veriﬁ cation phase of the

test, we ask the Test Spy how it was used by the SUT during the exercise SUT

phase. Use of a Test Spy does not require any advance knowledge of how the

methods of the DOC will be called.

The alternative is to ask the real DOC how it was used. Although this scheme

is not always feasible, when it is, it avoids the need to use a Test Double and

minimizes the degree to which we have Overspeciﬁ ed Software.

We can reduce the amount of code in the Test Method (and avoid Test Code

Duplication; see page 213) by deﬁ ning Expected Objects (see State Veriﬁ cation)

for the arguments of method calls or by delegating the veriﬁ cation of them to

Custom Assertions (page 474).

Variation: Expected Behavior Speciﬁ cation

Expected Behavior Speciﬁ cation is a different way of doing Behavior Veriﬁ cation.

Instead of waiting until after the fact to verify the indirect outputs of the SUT

by using a sequence of assertions, we load the Expected Behavior Speciﬁ cation

into a Mock Object and let it verify that the method calls are correct as they are

received.

We can use an Expected Behavior Speciﬁ cation when we know exactly what

should happen ahead of time and we want to remove all Procedural Behavior

Veriﬁ cation from the Test Method. This pattern variation tends to make the

Also known as:

Expected

Behavior

Veriﬁ cation

471

test shorter (assuming we are using a compact representation of the expected

behavior) and can be used to cause the test to fail on the ﬁ rst deviation from the

expected behavior if we so choose.

One distinct advantage of using Mock Objects is that Test Double generation

tools are available for many members of the xUnit family. They make imple-

menting Expected Behavior Speciﬁ cation very easy because we don’t need to

manually build a Test Double for each set of tests. One drawback of using a

Mock Object is that it requires that we can predict how the methods of the DOC

will be called and what arguments will be passed to it in the method calls.

Motivating Example

The following test is not a Self-Checking Test because it does not verify that

the expected outcome has actually occurred; it contains no calls to Assertion

Methods, nor does it set up any expectations on a Mock Object. Because we are

testing the logging functionality of the SUT, the state that interests us is actu-

ally stored in the logger rather than within the SUT itself. The writer of this test

hasn’t found a way to access the state we are trying to verify.

public void testRemoveFlightLogging_NSC() throws Exception {

// setup

FlightDto expectedFlightDto = createARegisteredFlight();

FlightManagementFacade facade =

new FlightManagementFacadeImpl();

// exercise

facade.removeFlight(expectedFlightDto.getFlightNumber());

// verify

// have not found a way to verify the outcome yet

// Log contains record of Flight removal

}

To verify the outcome, whoever is running the tests must access the database

and the log console and compare what was actually output to what should have

been output.

One way to make the test Self-Checking is to enhance the test with Expected

State Speciﬁ cation (see State Veriﬁ cation) of the SUT as follows:

public void testRemoveFlightLogging_ESS() throws Exception {

// ﬁxture setup

FlightDto expectedFlightDto = createAnUnregFlight();

FlightManagementFacadeImplTI facade =

new FlightManagementFacadeImplTI();

// exercise

facade.removeFlight(expectedFlightDto.getFlightNumber());

// verify

Behavior Verification

Behavior

Veriﬁ cation

472

Chapter 21 Result Verification Patterns

assertFalse("ﬂight still exists after being removed",

facade.ﬂightExists( expectedFlightDto.

getFlightNumber()));

}

Unfortunately, this test does not verify the logging function of the SUT in any

way. It also illustrates one reason why Behavior Veriﬁ cation came about: Some

functionality of the SUT is not visible within the end state of the SUT itself, but

can be seen only if we intercept the behavior at an internal observation point

between the SUT and the DOC or if we express the behavior in terms of state

changes for the objects with which the SUT interacts.

Refactoring Notes

When we made the changes in the second code sample in the “Motivating

Example,” we weren’t really refactoring; instead, we added veriﬁ cation logic to

make the tests behave differently. There are, however, several refactoring cases

that are worth discussing.

To refactor from State Veriﬁ cation to Behavior Veriﬁ cation, we must do a

Replace Dependency with Test Double (page 522) refactoring to gain visibility

of the indirect outputs of the SUT via a Test Spy or Mock Object.

To refactor from an Expected Behavior Speciﬁ cation to Procedural Behavior

Veriﬁ cation, we install a Test Spy instead of the Mock Object. After exercising the

SUT, we make assertions on values returned by the Test Spy and compare them

with the expected values that were originally used as arguments when we initially

conﬁ gured the Mock Object (the one that we just converted into a Test Spy).

To refactor from Procedural Behavior Veriﬁ cation to an Expected Behavior

Speciﬁ cation, we conﬁ gure a Mock Object with the expected values from the

assertions made on values returned by the Test Spy and install the Mock Object

instead of the Test Spy.

Example: Procedural Behavior Veriﬁ cation

The following test veriﬁ es the basic functionality of creating a ﬂ ight but uses

Procedural Behavior Veriﬁ cation to verify the indirect outputs of the SUT. That

is, it uses a Test Spy to capture the indirect outputs and then veriﬁ es those out-

puts are correct by making in-line calls to the Assertion Methods.

public void testRemoveFlightLogging_recordingTestStub()

throws Exception {

// ﬁxture setup

FlightDto expectedFlightDto = createAnUnregFlight();

FlightManagementFacade facade =

new FlightManagementFacadeImpl();

Behavior

Veriﬁ cation

473

// Test Double setup

AuditLogSpy logSpy = new AuditLogSpy();

facade.setAuditLog(logSpy);

// exercise

facade.removeFlight(expectedFlightDto.getFlightNumber());

// verify

assertEquals("number of calls", 1,

logSpy.getNumberOfCalls());

assertEquals("action code",

Helper.REMOVE_FLIGHT_ACTION_CODE,

logSpy.getActionCode());

assertEquals("date", helper.getTodaysDateWithoutTime(),

logSpy.getDate());

assertEquals("user", Helper.TEST_USER_NAME,

logSpy.getUser());

assertEquals("detail",

expectedFlightDto.getFlightNumber(),

logSpy.getDetail());

}

Example: Expected Behavior Speciﬁ cation

In this version of the test, we use the JMock framework to deﬁ ne the expected

behavior of the SUT. The method expects on mockLog conﬁ gures the Mock Object

with the Expected Behavior Speciﬁ cation (speciﬁ cally, the expected log message).

public void testRemoveFlight_JMock() throws Exception {

// ﬁxture setup

FlightDto expectedFlightDto = createAnonRegFlight();

FlightManagementFacade facade =

new FlightManagementFacadeImpl();

// mock conﬁguration

Mock mockLog = mock(AuditLog.class);

mockLog.expects(once()).method("logMessage")

.with(eq(helper.getTodaysDateWithoutTime()),

eq(Helper.TEST_USER_NAME),

eq(Helper.REMOVE_FLIGHT_ACTION_CODE),

eq(expectedFlightDto.getFlightNumber()));

// mock installation

facade.setAuditLog((AuditLog) mockLog.proxy());

// exercise

facade.removeFlight(expectedFlightDto.getFlightNumber());

// verify

// verify() method called automatically by JMock

}

Behavior Verification

Behavior

Veriﬁ cation

474

Chapter 21 Result Verification Patterns

Custom Assertion

How do we make tests self-checking when we have test-speciﬁ c equality logic?

How do we reduce Test Code Duplication when the same assertion

logic appears in many tests?

How do we avoid Conditional Test Logic?

We create a purpose-built Assertion Method that compares only those

attributes of the object that deﬁ ne test-speciﬁ c equality.

Most members of the xUnit family provide a reasonably rich set of Assertion

Methods (page 362). But sooner or later, we inevitably ﬁ nd ourselves saying,

“This test would be so much easier to write if I just had an assertion that did . . . .”

So why not write it ourselves?

The reasons for writing a Custom Assertion are many, but the technique is

pretty much the same regardless of our goal. We hide the complexity of whatever

it takes to prove the system is behaving correctly behind an Assertion Method

with an Intent-Revealing Name [SBPP].

How It Works

We encapsulate the mechanics of verifying that something is true (an assertion)

behind an Intent-Revealing Name. To do so, we factor out all the common

assertion code within the tests into a Custom Assertion that implements the

Fixture

Setup

Exercise

Verify

Teardown

SUT

Custom

Assertion

Method

Assertion

Method

Fixture

Setup

Exercise

Verify

Teardown

SUT

Custom

Assertion

Method

Assertion

Method

Also known as:

Bespoke

Assertion

Custom

Assertion

475

veriﬁ cation logic. A Custom Equality Assertion takes two parameters: an

Expected Object (see State Veriﬁ cation on page 462) and the actual object.

A key characteristic of Custom Assertions is that they receive everything they

need to pass or fail the test as parameters. Other than causing the test to fail,

they have no side effects.

Typically, we create Custom Assertions through refactoring by identifying

common patterns of assertions in our tests. When test driving, we might just

go ahead and call a nonexistent Custom Assertion because it makes writing our

test easier; this tactic lets us focus on the part of the SUT that needs to be tested

rather than the mechanics of how the test would be carried out.

When to Use It

We should consider creating a Custom Assertion whenever any of the following

statements are true:

• We ﬁ nd ourselves writing (or cloning) the same assertion logic in test

after test (Test Code Duplication; see page 213).

•We ﬁ nd ourselves writing Conditional Test Logic (page 200) in the result

veriﬁ cation part of our tests. That is, our calls to Assertion Methods are

embedded in if statements or loops.

• The result veriﬁ cation parts of our tests suffer from Obscure Test

(page 186) because we use procedural rather than declarative result

veriﬁ cation in the tests.

• We ﬁ nd ourselves doing Frequent Debugging (page 248) whenever

assertions fail because they do not provide enough information.

A key reason for moving the assertion logic out of the tests and into Custom

Assertions is to Minimize Untestable Code (see page 44). Once the veriﬁ cation

logic has been moved into a Custom Assertion, we can write Custom Assertion

Tests (see Custom Assertion on page 474) to prove the veriﬁ cation logic is work-

ing properly. Another important beneﬁ t of using Custom Assertions is that they

help avoid Obscure Tests and make tests Communicate Intent (see page 41).

That, in turn, will help produce robust, easily maintained tests.

If the veriﬁ cation logic must interact with the SUT to determine the actual

outcome, we can use a Veriﬁ cation Method (see Custom Assertion) instead of a

Custom Assertion. If the setup and exercise parts of the tests are also the same

except for the values of the actual/expected objects, we should consider using

a Parameterized Test (page 607). The primary advantage of Custom Assertions

over both of these techniques is reusability; the same Custom Assertion can be

CUSTOM ASSERTION

Custom

Assertion

476

Chapter 21 Result Verification Patterns

reused in many different circumstances because it is independent of its context

(its only contact with the outside world occurs through its parameter list).

We most commonly write Custom Assertions that are Equality Assertions

(see Assertion Method), but there is no reason why we cannot write other kinds

as well.

Variation: Custom Equality Assertion

For custom equality assertions, the Custom Assertion must be passed an Expected

Object and the actual object to be veriﬁ ed. It should also take an Assertion Mes-

sage (page 370) to avoid playing Assertion Roulette (page 224). Such an assertion

is essentially an equals method implemented as a Foreign Method [Fowler].

Variation: Object Attribute Equality Assertion

We often run across Custom Assertions that take one actual object and several

different Expected Objects that need to be compared with speciﬁ c attributes of

the actual object. (The set of attributes to be compared is implied by the name of

the Custom Assertion.) The key difference between these Custom Assertions and

a Veriﬁ cation Method is that the latter interacts with the SUT while the Object

Attribute Equality Assertion looks only at the objects passed in as parameters.

Variation: Domain Assertion

All of the built-in Assertion Methods are domain independent. Custom Equal-

ity Assertions implement test-speciﬁ c equality but still compare only two

objects. Another style of Custom Assertion helps contribute to the deﬁ nition

of a “domain-speciﬁ c” Higher-Level Language (see page 41)—namely, the

Domain Assertion.

A Domain Assertion is a Stated Outcome Assertion (see Assertion Method)

that states something that should be true in domain-speciﬁ c terms. It helps elevate

the test into “business-speak.”

Variation: Diagnostic Assertion

Sometimes we ﬁ nd ourselves doing Frequent Debugging whenever a test fails

because the assertions tell us only that something is wrong but do not identify

the speciﬁ c problem (e.g., the assertions indicate these two objects are not equal

but it isn’t clear what isn’t equal about the object). We can write a special kind of

Custom Assertion that may look just like one of the built-in assertions but pro-

vide more information about what is different between the expected and actual

values than a built-in assertion because it is speciﬁ c to our types. (For example,

it might tell us which attributes are different or where long strings differ.)

Custom

Assertion

477

On one project, we were comparing string variables containing XML.

Whenever a test failed, we had to bring up two string inspectors and scroll

through them looking for the difference. Finally, we got smart and included

the logic in a Custom Assertion that told us where the ﬁ rst difference between

the two XML strings occurred. The small amount of time we spent writing

the diagnostic custom assertion was paid back many times over as we ran

our tests.

Variation: Veriﬁ cation Method

In customer tests, a lot of the complexity of verifying the outcome is related

to interacting with the SUT. Veriﬁ cation Methods are a form of Custom Asser-

tions that interact directly with the SUT, thereby relieving their callers from this

task. This simpliﬁ es the tests signiﬁ cantly and leads to a more “declarative”

style of outcome speciﬁ cation. After the Custom Assertion has been written, we

can write subsequent tests that result in the same outcome much more quickly.

In some cases, it may be advantageous to incorporate even the exercise SUT

phase of the test into the Veriﬁ cation Method. This is one step short of a full

Parameterized Test that incorporates all the test logic in a reusable Test Utility

Method (page 599).

Implementation Notes

The Custom Assertion is typically implemented as a set of calls to the various

built-in Assertion Methods. Depending on how we plan to use it in our tests,

we may also want to include the standard Equality Assertion template to ensure

correct behavior with null parameters. Because the Custom Assertion is itself an

Assertion Method, it should not have any side effects, nor should it call the SUT.

(If it needs to do so, it would be a Veriﬁ cation Method.)

Variation: Custom Assertion Test

Testing zealots would also write a Custom Assertion Test (a Self-Checking Test—

see page 26—for Custom Assertions) to verify the Custom Assertion. The beneﬁ t

from doing so is obvious: increased conﬁ dence in our tests. In most cases, writing

Custom Assertion Tests isn’t particularly difﬁ cult because Assertion Methods take

all their arguments as parameters.

We can treat the Custom Assertion as the SUT simply by calling it with various

arguments and verifying that it fails in the right cases. Single-Outcome Assertions

(see Assertion Method) need only a single test because they don’t take any parameters

(other than possibly an Assertion Message). Stated Outcome Assertions need one

Custom Assertion

Custom

Assertion