Meszaros G. xUnit Test Patterns Refactoring Test Code

Подождите немного. Документ загружается.

108

Chapter 10 Result Verification

assertions after the ﬁ xture setup phase of the test to ensure that the ﬁ xture is set

up correctly. This practice almost always contributes to Obscure Tests (page 186),

so I would rather write unit tests for the Test Utility Methods (page 599).

Some

styles of testing do require us to set up our expectations before we exercise the SUT;

this topic is discussed in more detail in Chapter 11, Using Test Doubles. We’ll see

several examples of calling Assertion Methods from within Test Utility Methods

in this chapter.

One possible—though rarely used—place to put calls to Assertion Methods is

in the tearDown method used in Implicit Teardown (page 516). Because this method

is run for every test, whether that test passed or failed (as long as the setUp method

succeeded), one can put assertions here. This scheme involves the same trade-off as

using Implicit Setup (page 424) for building our test ﬁ xture; it’s less visible but done

automatically. See the sidebar “Using Delta Assertions to Detect Data Leakage”

(page 487) for an example of putting assertions in the tearDown method used by

Implicit Teardown of a superclass to detect when tests leave leftover test objects

in the database.

Verify State or Behavior?

Ultimately, test automation is about verifying the behavior of the SUT. Some

aspects of the SUT’s behavior can be veriﬁ ed directly; the value returned by a

function is a good example. Other aspects of the behavior are more easily veri-

ﬁ ed indirectly by looking at the state of some object. We can verify the actual

behavior of the SUT in our tests in two ways:

1. We can verify the states of various objects affected by the SUT by

extracting each state using an observation point and using assertions

to compare it to the expected state.

2. We can verify the behavior of the SUT directly by using observation

points inserted between the SUT and its depended-on component

(DOC) to monitor its interactions (in the form of the method calls it

makes) and comparing those method calls with what we expected.

State Veriﬁ cation (page 462) is done using assertions and is the simpler of the

two approaches. Behavior Veriﬁ cation (page 468) is more complicated and

builds on the assertion techniques we use for verifying state.

The one exception is when we must use a Shared Fixture (page 317); it may be worth-

while to use a Guard Assertion (page 490) to document what the test requires from it and

to produce a test failure if the ﬁ xture is corrupted. We could also do so from within the

Finder Methods (see Test Utility Method) that we use to retrieve the objects in the Shared

Fixture (page 317) we will use in our tests.

109

State Veriﬁ cation

The “normal” way to verify the expected outcome has occurred is called State

Veriﬁ cation (Figure 10.1). First we exercise the SUT; then we examine the post-

exercise state of the SUT using assertions. We may also examine anything returned

by the SUT as a result of the method call we made to exercise it. What is most

notable is what we do not do: We do not instrument the SUT in any way to detect

how it interacts with other components of the system. That is, we inspect only

direct outputs and we use only direct method calls as our observation points.

Figure 10.1 State Veriﬁ cation. In State Veriﬁ cation, we assert that the SUT and

any objects it returns are in the expected state after we have exercised the SUT.

We “pay no attention to the man behind the curtain.”

State Veriﬁ cation can be done in two slightly different ways. Procedural State

Veriﬁ cation (see State Veriﬁ cation) involves writing a sequence of assertions that

pick apart the end state of the SUT and verify that it is as expected. Expected

Object (see State Veriﬁ cation) is a way of describing the expected state in such a

way that it can be compared with a single Assertion Method call; this approach

minimizes Test Code Duplication and increases test clarity (more on this later

in this chapter). With both strategies, we can use either “built-in” assertions or

Custom Assertions (page 474).

Fixture

SUT

DOC

Exercise

Setup

Exercise

Verify

Teardown

Get State

Behavior

(Indirect

Outputs)

Fixture

SUT

DOC

Exercise

Setup

Exercise

Verify

Teardown

Get State

Behavior

(Indirect

Outputs)

State Verification

110

Chapter 10 Result Verification

Using Built-in Assertions

We use the assertions provided by our testing framework to specify what should

be and depend on them to tell us when it isn’t so! But simply using the built-in

assertions is only a small part of the story.

The simplest form of result veriﬁ cation is the assertion in which we specify

what should be true. Most members of the xUnit family support a range of dif-

ferent Assertion Methods, including the following:

• Stated Outcome Assertions (see Assertion Method) such as assertTrue

(aBooleanExpression)

• Simple Equality Assertions such as assertEquals(expected, actual)

• Fuzzy Equality Assertions such as assertEquals(expected, actual, tolerance),

which are used for comparing ﬂ oats

Of course, the test programming language has some inﬂ uence on the nature of

the assertions. In JUnit, SUnit, CppUnit, NUnit, and CsUnit, most of the Equal-

ity Assertions take a pair of Objects as their parameters. Some languages support

“overloading” of method parameter types so we can have different implemen-

tations of an assertion for different types of objects. Some languages—C, for

example—don’t support objects, so we cannot compare objects, only values.

There are several issues to consider when using Assertion Methods. Naturally,

the ﬁ rst priority is the veriﬁ cation of all things that should be true. The better

our assertions, the ﬁ ner our Safety Net (see page 24) and the higher our conﬁ -

dence in our code. The second priority is the documentation value of the asser-

tions. Each test should make it very clear that “When the system is in state S1

and I do X, the result should be R and the system should be in state S2.” We

put the system into state S1 in our ﬁ xture setup logic. “I do X” corresponds to

the exercise SUT phase of the test. “The result is R” and “the system is in state

S2” are implemented using assertions. Thus we want to write our assertions in

such a way that they succinctly describe “R” and “S2.”

Another thing to consider is that when the test fails, we want the failure

message to tell us enough to enable us to identify the problem.

Therefore, we

should almost always include an Assertion Message (page 370) as the optional

message parameter (assuming our xUnit family member has one!). This tactic

avoids the possibility of us playing Assertion Roulette (page 224), in which we

cannot even tell which assertion is failing without running the test interactively;

In his book [TDD-APG], Dave Astels claims he never/rarely used the Eclipse Debugger

while writing the code samples because the assertions always told him enough about

what was wrong. This is what we strive for!

111

it makes Integration Build [SCM] failures much easier to reproduce and ﬁ x. It

also makes troubleshooting broken tests easier by telling us what should have

happened; the actual outcome tells us what did happen!

When we use a Stated Outcome Assertion (such as JUnit’s assertTrue), the

failure messages tend to be unhelpful (e.g., “Assertion failed”). We can make

the assertion output much more speciﬁ c by using an Argument-Describing Mes-

sage (see Assertion Message) constructed by incorporating useful bits of data

into the message. A good start is to include each of the values in the expression

passed as the Assertion Method’s arguments.

Delta Assertions

When using a Shared Fixture (page 317), we may ﬁ nd that we have Interacting

Tests (see Erratic Test on page 228) because each test adds more objects/rows

into the database and we can never be certain exactly what should be there af-

ter the SUT has been exercised. One way to deal with this uncertainty is to use

Delta Assertions (page 485) to verify only the newly added objects/rows. In this

approach, we take some sort of “snapshot” of the relevant tables/classes at the

beginning of the test; we then remove these tables/classes from the collection

of actual objects/rows produced at the end of the test before comparing them

to the Expected Objects. Although this tactic can introduce signiﬁ cant extra

complexity into the tests, the added complexity can be refactored into Custom

Assertions and/or Veriﬁ cation Methods (see Custom Assertion). The “before”

snapshot may be taken on an in-line basis within the Test Method or in the

setUp method if all setup occurs before the Test Method is invoked [e.g., Implicit

Setup, a Shared Fixture, or a Prebuilt Fixture (page 429)].

External Result Veriﬁ cation

Thus far we have described only conventional “in-memory” veriﬁ cation of the

expected results. In fact, another approach is possible—one that involves storing

the expected and actual results in ﬁ les and using an external comparison pro-

gram to report on any differences. This is, in effect, a form of Custom Assertion

that uses a “deep compare” on two ﬁ le references. The comparison program

often needs to be told which parts of the ﬁ les to ignore (or these parts need to be

stripped out ﬁ rst), effectively making this a Fuzzy Equality Assertion.

External result veriﬁ cation is particularly appropriate for automating accep-

tance tests for regression-testing an application that hasn’t changed very much.

The major disadvantage of this approach is that we almost always end up with a

Mystery Guest (see Obscure Test) from the test reader’s perspective because the

State Verification

112

Chapter 10 Result Verification

expected results are not visible inside the test. One way to avoid this problem is

to have the test write the contents of the expected ﬁ le, thereby making the con-

tents visible to the test reader. This step is practical only if the amount of data is

quite small—another argument in favor of a Minimal Fixture (page 302).

Verifying Behavior

Verifying behavior is more complicated than verifying state because behavior is

dynamic. We have to catch the SUT “in the act” as it generates indirect outputs

to the objects it depends on (Figure 10.2). Two basic styles of behavior veriﬁ ca-

tion are worth discussing: Procedural Behavior Veriﬁ cation and Expected

Behavior. Both require a mechanism to access the outgoing method calls of the

SUT (its indirect outputs). This and other uses of Test Doubles (page 522) are

described in more detail in Chapter 11, Using Test Doubles.

Figure 10.2 Behavior Veriﬁ cation. In Behavior Veriﬁ cation, we focus our

assertions on the indirect outputs (outgoing interfaces) of the SUT. This typically

involves replacing the DOC with something that facilitates observing and

verifying the outgoing calls.

Fixture

DOC

Exercise

Setup

Exercise

Verify

Teardown

SUT

Behavior

(Indirect

Outputs)

Verify

Fixture

DOC

Exercise

Setup

Exercise

Verify

Teardown

SUT

Behavior

(Indirect

Outputs)

Verify

113

Procedural Behavior Veriﬁ cation

In Procedural Behavior Veriﬁ cation, we capture the behavior of the SUT as it

executes and save that data for later retrieval. The test then compares each out-

put of the SUT (one by one) with the corresponding expected output. Thus, in

Procedural Behavior Veriﬁ cation, the test executes a procedure (a set of steps)

to verify the behavior.

public void testRemoveFlightLogging_recordingTestStub()

throws Exception {

// ﬁxture setup

FlightDto expectedFlightDto = createAnUnregFlight();

FlightManagementFacade facade =

new FlightManagementFacadeImpl();

// Test Double setup

AuditLogSpy logSpy = new AuditLogSpy();

facade.setAuditLog(logSpy);

// exercise

facade.removeFlight(expectedFlightDto.getFlightNumber());

// verify

assertEquals("number of calls", 1,

logSpy.getNumberOfCalls());

assertEquals("action code",

Helper.REMOVE_FLIGHT_ACTION_CODE,

logSpy.getActionCode());

assertEquals("date", helper.getTodaysDateWithoutTime(),

logSpy.getDate());

assertEquals("user", Helper.TEST_USER_NAME,

logSpy.getUser());

assertEquals("detail",

expectedFlightDto.getFlightNumber(),

logSpy.getDetail());

}

The key challenge in Procedural Behavior Veriﬁ cation is capturing the behavior

as it occurs and saving it until the test is ready to use this information. This task

is accomplished by conﬁ guring the SUT to use a Test Spy (page 538) or a Self

Shunt (see Hard-Coded Test Double on page 568)

instead of the depended-on

class. After the SUT has been exercised, the test retrieves the recording of the

behavior and veriﬁ es it using assertions.

Expected Behavior Speciﬁ cation

If we can build an Expected Object and compare it with the actual object

returned by the SUT for verifying state, can we do something similar for verifying

A Test Spy built into the Testcase Class (page 373).

Verifying Behavior

114

Chapter 10 Result Verification

behavior? Yes, we can and do. Expected Behavior is often used in conjunction

with layer-crossing tests to verify the indirect outputs of an object or compo-

nent. We conﬁ gure a Mock Object (page 544) with the method calls we expect

the SUT to make to it and install this object before exercising the SUT.

public void testRemoveFlight_JMock() throws Exception {

// ﬁxture setup

FlightDto expectedFlightDto = createAnonRegFlight();

FlightManagementFacade facade =

new FlightManagementFacadeImpl();

// mock conﬁguration

Mock mockLog = mock(AuditLog.class);

mockLog.expects(once()).method("logMessage")

.with(eq(helper.getTodaysDateWithoutTime()),

eq(Helper.TEST_USER_NAME),

eq(Helper.REMOVE_FLIGHT_ACTION_CODE),

eq(expectedFlightDto.getFlightNumber()));

// mock installation

facade.setAuditLog((AuditLog) mockLog.proxy());

// exercise

facade.removeFlight(expectedFlightDto.getFlightNumber());

// verify

// verify() method called automatically by JMock

}

Reducing Test Code Duplication

One of the most common test smells is Test Code Duplication. With every test

we write, there is a good chance we have introduced some duplication, but

especially if we used “cut and paste” to create a new test from an existing test.

Some will argue that duplication in test code is not nearly as bad as duplication

in production code. Test Code Duplication is bad if it leads to some other smell

such as Fragile Test (page 239), Fragile Fixture (see Fragile Test), or High Test

Maintenance Cost (page 265) because too many tests are too closely coupled to

the Standard Fixture (page 305) or the API of the SUT. In addition, Test Code

Duplication may sometimes be a symptom of another problem—namely, the

intent of the tests being obscured by too much code (i.e., an Obscure Test).

In result veriﬁ cation logic, Test Code Duplication usually shows up as a set

of repeated assertions. Several techniques are available to reduce the number of

assertions in such cases:

115

• Expected Objects

• Custom Assertions

• Veriﬁ cation Methods

Expected Objects

Often, we will ﬁ nd ourselves doing a series of assertions on different ﬁ elds of

the same object. If we begin repeating this group of assertions (whether multiple

times in a single test or in multiple tests), we should look for a way to reduce the

Test Code Duplication. The next listing shows one Test Method that compares

several attributes of a single object. Many other Test Methods probably require

the same sequence of assertions.

public void testInvoice_addLineItem7() {

LineItem expItem = new LineItem(inv, product, QUANTITY);

// Exercise

inv.addItemQuantity(product, QUANTITY);

// Verify

List lineItems = inv.getLineItems();

LineItem actual = (LineItem)lineItems.get(0);

assertEquals(expItem.getInv(), actual.getInv());

assertEquals(expItem.getProd(), actual.getProd());

assertEquals(expItem.getQuantity(), actual.getQuantity());

}

The most obvious alternative is to use a single Equality Assertion to compare

two whole objects to each other rather than using many Equality Assertion calls

to compare them ﬁ eld by ﬁ eld. If the values are stored in individual variables, we

may need to create a new object of the appropriate class and initialize its ﬁ elds

with those values. This technique works as long as we have an equals method

that compares only those ﬁ elds and we have the ability to create the Expected

Object at will.

public void testInvoice_addLineItem8() {

LineItem expItem = new LineItem(inv, product, QUANTITY);

// Exercise

inv.addItemQuantity(product, QUANTITY);

// Verify

List lineItems = inv.getLineItems();

LineItem actual = (LineItem)lineItems.get(0);

assertEquals("Item", expItem, actual);

}

But what if we don’t want to compare all the ﬁ elds in an object or the equals

method looks for identity rather than equality? What if we want test-speciﬁ c

Reducing Test Code Duplication

116

Chapter 10 Result Verification

equality? What if we cannot create an instance of the Expected Object because

no constructor exists? In this scenario, we have two options: We can implement

a Custom Assertion that deﬁ nes equality the way we want it or we can imple-

ment our test-speciﬁ c equality in the equals method of the class of the Expected

Object we pass to the Assertion Method. This class doesn’t need to be the same

class as that of the actual object; it just needs to implement equals to compare

itself with an instance of the actual object’s class. Therefore, it can be a simple

Data Transfer Object [CJ2EEP] or it can be a Test-Speciﬁ c Subclass (page 579)

of the real (production) class with just the equals method overridden.

Some test automaters don’t think we should ever rely on the equals method

of the SUT when making assertions because it could change, thereby causing

tests that depend on this method to fail (or to miss important differences). I pre-

fer to be pragmatic about this decision. If it seems reasonable to use the equals

deﬁ nition supplied by the SUT, then I do so. If I need something else, I deﬁ ne

a Custom Assertion or a test-speciﬁ c Expected Object class. I also ask myself

how hard it would be to change my strategy if the equals method should later

change. For example, in statically typed languages that support parameter type

overloading (such as Java), we can add a Custom Assertion that uses different

parameter types to override the default implementation when speciﬁ c types are

used. This code can often be retroﬁ tted quite easily if a change to equals causes

problems at a later date.

Custom Assertions

A Custom Assertion is a domain-speciﬁ c assertion we write ourselves. Custom

Assertions hide the procedure for verifying the results behind a declarative name,

making our result veriﬁ cation logic more intent-revealing. They also prevent

Obscure Tests by eliminating of a lot of potentially distracting code. Another

beneﬁ t of moving the code into a Custom Assertion is that the assertion logic

can now be unit-tested by writing Custom Assertion Tests (see Custom Asser-

tion). The assertions are no longer Untestable Test Code (see Hard-to-Test Code

on page 209)!

static void assertLineItemsEqual(

String msg, LineItem exp, LineItem act) {

assertEquals (msg+" Inv", exp.getInv(), act.getInv());

assertEquals (msg+" Prod", exp.getProd(), act.getProd());

assertEquals (msg+" Quan", exp.getQuantity(), act.getQuantity());

}

There are two ways to create Custom Assertions: (1) by refactoring existing

complex test code to reduce Test Code Duplication and (2) by coding calls to

117

nonexistent Assertion Methods as we write tests and then ﬁ lling in the method

bodies with the appropriate logic once we land on the suite of Custom Assertions

needed by a set of Test Methods. The latter technique is a good way of reminding

ourselves what we expect the outcome of exercising the SUT to be, even though

we haven’t yet written the code to verify it. Either way, the deﬁ nition of a set

of Custom Assertions is the ﬁ rst step toward creating a Higher-Level Language

(see page 41) for specifying our tests.

When refactoring to Custom Assertions, we simply use Extract Method

[Fowler] on the repeated assertions and give the new method an Intent-Revealing

Name [SBPP]. We pass in the objects used by the existing veriﬁ cation logic as

arguments and include an Assertion Message to differentiate between calls to

the same assertion method.

Outcome-Describing Veriﬁ cation Method

Another technique that is born from ruthless refactoring of test code is the “out-

come-describing” Veriﬁ cation Method. Suppose we ﬁ nd that a group of tests all

have identical exercise SUT and verify outcome sections. Only the setup portion

is different for each test. If we do an Extract Method refactoring on the common

code and give it a meaningful name, we need less code, achieve more understand-

able tests, and produce testable veriﬁ cation logic all at the same time! If this isn’t

a worthwhile reason for refactoring code, then I don’t know what else could be.

void assertInvoiceContainsOnlyThisLineItem(

Invoice inv,

LineItem expItem) {

List lineItems = inv.getLineItems();

assertEquals("number of items", lineItems.size(), 1);

LineItem actual = (LineItem)lineItems.get(0);

assertLineItemsEqual("",expItem, actual);

}

The major difference between a Veriﬁ cation Method and a Custom Assertion

is that the latter only makes assertions, while the former also interacts with

the SUT (typically for the purpose of exercising it). Another difference is that

Custom Assertions typically have a standard Equality Assertion signature:

assertSomething(message, expected, actual)

. In contrast, Veriﬁ cation Methods may

have completely arbitrary parameters because they require additional param-

eters to pass into the SUT. They are, in essence, halfway between a Custom

Assertion and a Parameterized Test (page 607).

Reducing Test Code Duplication