Meszaros G. xUnit Test Patterns Refactoring Test Code

Подождите немного. Документ загружается.

478

Chapter 21 Result Verification Patterns

test for each possible value (or boundary value). Equality Assertions need one test

that compares two objects deemed to be equivalent, one test that compares an

object with itself, and one test for each attribute whose inequality should cause

the assertion to fail. Attributes that don’t affect equality can be veriﬁ ed in one

additional test because the Equality Assertion should not raise an error for any of

them.

The Custom Assertions follow the normal Simple Success Test (see Test

Method on page 348) and Expected Exception Test (see Test Method) tem-

plates with one minor difference: Because the Assertion Method is the SUT, the

exercise SUT and verify outcome phases of the Four-Phase Test (page 358) are

combined into a single phase.

Each test consists of setting up the Expected Object and the actual object and

then calling the Custom Assertion. If the objects should be equivalent, that’s all

there is to it. (The Test Automation Framework described on page 298 would

catch any assertion failures and fail the test.) For the tests where we expect the

Custom Assertion to fail, we can write the test as an Expected Exception Test

(except that the exercise SUT and verify outcome phases of the Four-Phase Test

are combined into the single call to the Custom Assertion).

The simplest way to build the objects to be compared for a speciﬁ c test is to

do something similar to One Bad Attribute (see Derived Value on page 718)—

that is, build the ﬁ rst object and make a deep copy of it. For successful tests,

modify any of the attributes that should not be compared. For each test failure,

modify one attribute that should be grounds for failing the assertion.

A brief warning about a possible complication in a few members of the

xUnit family: If all of the test failure handling does not occur in the Test Runner

(page 377), calls to

fail or built-in assertions may add messages to the failure

log even if we catch the error or exception in our Custom Assertion Test. The

only way to circumvent this behavior is to use an “Encapsulated Test Runner”

to run each test by itself and verify that the one test failed with the expected

error message.

Motivating Example

In the following example, several test methods repeat the same series of

assertions:

public void testInvoice_addOneLineItem_quantity1_b() {

// Exercise

inv.addItemQuantity(product, QUANTITY);

// Verify

List lineItems = inv.getLineItems();

assertEquals("number of items", lineItems.size(), 1);

Custom

Assertion

479

// Verify only item

LineItem expItem = new LineItem(inv, product, QUANTITY);

LineItem actual = (LineItem)lineItems.get(0);

assertEquals(expItem.getInv(), actual.getInv());

assertEquals(expItem.getProd(), actual.getProd());

assertEquals(expItem.getQuantity(), actual.getQuantity());

}

public void testRemoveLineItemsForProduct_oneOfTwo() {

// Setup

Invoice inv = createAnonInvoice();

inv.addItemQuantity(product, QUANTITY);

inv.addItemQuantity(anotherProduct, QUANTITY);

LineItem expItem = new LineItem(inv, product, QUANTITY);

// Exercise

inv.removeLineItemForProduct(anotherProduct);

// Verify

List lineItems = inv.getLineItems();

assertEquals("number of items", lineItems.size(), 1);

LineItem actual = (LineItem)lineItems.get(0);

assertEquals(expItem.getInv(), actual.getInv());

assertEquals(expItem.getProd(), actual.getProd());

assertEquals(expItem.getQuantity(), actual.getQuantity());

}

// Adding TWO line items

public void testInvoice_addTwoLineItems_sameProduct() {

Invoice inv = createAnonInvoice();

LineItem expItem1 = new LineItem(inv, product, QUANTITY1);

LineItem expItem2 = new LineItem(inv, product, QUANTITY2);

// Exercise

inv.addItemQuantity(product, QUANTITY1);

inv.addItemQuantity(product, QUANTITY2);

// Verify

List lineItems = inv.getLineItems();

assertEquals("number of items", lineItems.size(), 2);

// Verify ﬁrst item

LineItem actual = (LineItem)lineItems.get(0);

assertEquals(expItem1.getInv(), actual.getInv());

assertEquals(expItem1.getProd(), actual.getProd());

assertEquals(expItem1.getQuantity(), actual.getQuantity());

// Verify second item

actual = (LineItem)lineItems.get(1);

assertEquals(expItem2.getInv(), actual.getInv());

assertEquals(expItem2.getProd(), actual.getProd());

assertEquals(expItem2.getQuantity(), actual.getQuantity());

}

Custom Assertion

Custom

Assertion

480

Chapter 21 Result Verification Patterns

Note that the ﬁ rst test ends with a series of three assertions and the second test

repeats the series of three assertions twice, once for each line item. This is clearly

a bad case of Test Code Duplication.

Refactoring Notes

Refactoring zealots can probably see that the solution is to do an Extract Meth-

od [Fowler] refactoring on these tests. If we pull out all the common calls to

Assertion Methods, we will be left with only the differences in each test. The

extracted method is our Custom Assertion. We may also need to introduce an

Expected Object to hold all the values that were being passed to the individual

Assertion Methods on a single object to be passed to the Custom Assertion.

Example: Custom Assertion

In this test, we use a Custom Assertion to verify that LineItem matches the expected

LineItem(s). For one reason or another, we have chosen to implement a test-speciﬁ c

equality rather than using a standard Equality Assertion.

public void testInvoice_addOneLineItem_quantity1_() {

Invoice inv = createAnonInvoice();

LineItem expItem = new LineItem(inv, product, QUANTITY);

// Exercise

inv.addItemQuantity(product, QUANTITY);

// Verify

List lineItems = inv.getLineItems();

assertEquals("number of items", lineItems.size(), 1);

// Verify only item

LineItem actual = (LineItem)lineItems.get(0);

assertLineItemsEqual("LineItem", expItem, actual);

}

public void testAddItemQuantity_sameProduct_() {

Invoice inv = createAnonInvoice();

LineItem expItem1 = new LineItem(inv, product, QUANTITY1);

LineItem expItem2 = new LineItem(inv, product, QUANTITY2);

// Exercise

inv.addItemQuantity(product, QUANTITY1);

inv.addItemQuantity(product, QUANTITY2);

// Verify

List lineItems = inv.getLineItems();

assertEquals("number of items", lineItems.size(), 2);

// Verify ﬁrst item

LineItem actual = (LineItem)lineItems.get(0);

assertLineItemsEqual("Item 1",expItem1,actual);

// Verify second item

Custom

Assertion

481

actual = (LineItem)lineItems.get(1);

assertLineItemsEqual("Item 2",expItem2, actual);

}

The tests have become signiﬁ cantly smaller and more intent-revealing. We have

also chosen to pass a string indicating which item we are examining as an argu-

ment to the Custom Assertion to avoid playing Assertion Roulette when a test

fails.

This simpliﬁ ed test was made possible by having the following Custom

Assertion available to us:

static void assertLineItemsEqual(

String msg, LineItem exp, LineItem act) {

assertEquals(msg+" Inv", exp.getInv(),act.getInv());

assertEquals(msg+" Prod", exp.getProd(), act.getProd());

assertEquals(msg+" Quan", exp.getQuantity(), act.getQuantity());

}

This Custom Assertion compares the same attributes of the object as we were

comparing on an in-line basis in the previous version of the test; thus the seman-

tics of the test haven’t changed. We also concatenate the name of the attribute

being compared with the message parameter to get a unique failure message,

which allows us to avoid playing Assertion Roulette when a test fails.

Example: Domain Assertion

In this next version of the test, we have further elevated the level of the asser-

tions to better communicate the expected outcome of the test scenarios:

public void testAddOneLineItem_quantity1() {

Invoice inv = createAnonInvoice();

LineItem expItem = new LineItem(inv, product, QUANTITY);

// Exercise

inv.addItemQuantity( product, QUANTITY);

// Verify

assertInvoiceContainsOnlyThisLineItem( inv, expItem);

}

public void testRemoveLineItemsForProduct_oneOfTwo_() {

Invoice inv = createAnonInvoice();

inv.addItemQuantity( product, QUANTITY);

inv.addItemQuantity( anotherProduct, QUANTITY);

LineItem expItem = new LineItem( inv, product, QUANTITY);

// Exercise

inv.removeLineItemForProduct( anotherProduct );

// Verify

assertInvoiceContainsOnlyThisLineItem( inv, expItem);

}

Custom Assertion

Custom

Assertion

482

Chapter 21 Result Verification Patterns

This simpliﬁ ed version of the test was made possible by extracting the following

Domain Assertion method:

void assertInvoiceContainsOnlyThisLineItem(

Invoice inv,

LineItem expItem) {

List lineItems = inv.getLineItems();

assertEquals("number of items", lineItems.size(), 1);

LineItem actual = (LineItem)lineItems.get(0);

assertLineItemsEqual("item",expItem, actual);

}

This example chose to forgo passing a message to the Domain Assertion to save

a bit of space. In real life, we would typically include a message string in the

parameter list and concatenate the messages of the individual assertions to one

passed in. See Assertion Message (page 370) for more details.

Example: Veriﬁ cation Method

If the exercise SUT and result veriﬁ cation phases of several tests are pretty much

identical, we can incorporate both phases into our reusable Custom Assertion.

Because this approach changes the semantics of the Custom Assertion from being

just a function free of side effects to an operation that changes the state of the

SUT, we usually give it a more distinctive name starting with “verify”.

This version of the test merely sets up the test ﬁ xture before calling a Veriﬁ ca-

tion Method that incorporates both the exercise SUT and verify outcome phases

of the test. It is most easily recognized by the lack of a distinct “exercise” phase

in the calling test and the presence of calls to methods that modify the state of

one of the objects passed as a parameter of the Veriﬁ cation Method.

public void testAddOneLineItem_quantity2() {

Invoice inv = createAnonInvoice();

LineItem expItem = new LineItem(inv, product, QUANTITY);

// Exercise & Verify

verifyOneLineItemCanBeAdded(inv, product, QUANTITY, expItem);

}

The Veriﬁ cation Method for this example looks like this:

public void verifyOneLineItemCanBeAdded(

Invoice inv, Product product,

int QUANTITY, LineItem expItem) {

// Exercise

inv.addItemQuantity(product, QUANTITY);

// Verify

assertInvoiceContainsOnlyThisLineItem(inv, expItem);

}

Custom

Assertion

483

This Veriﬁ cation Method calls the “pure” Custom Assertion, although it could

just as easily have included all the assertion logic if we didn’t have the other Cus-

tom Assertion to call. Note the call to addItemQuantity on the parameter inv; this is

what changes if from a Custom Assertion to a Veriﬁ cation Method.

Example: Custom Assertion Test

This Custom Assertion isn’t particularly complicated, so we may feel comfort-

able without having any automated tests for it. If there is anything complex

about it, however, we may ﬁ nd it worthwhile to write tests like these:

public void testassertLineItemsEqual_equivalent() {

Invoice inv = createAnonInvoice();

LineItem item1 = new LineItem(inv, product, QUANTITY1);

LineItem item2 = new LineItem(inv, product, QUANTITY1);

// exercise/verify

assertLineItemsEqual("This should not fail",item1, item2);

}

public void testassertLineItemsEqual_differentInvoice() {

Invoice inv1 = createAnonInvoice();

Invoice inv2 = createAnonInvoice();

LineItem item1 = new LineItem(inv1, product, QUANTITY1);

LineItem item2 = new LineItem(inv2, product, QUANTITY1);

// exercise/verify

try {

assertLineItemsEqual("Msg",item1, item2);

} catch (AssertionFailedError e) {

assertEquals("e.getMsg",

"Invoice-expected: <123> but was <124>",

e.getMessage());

return;

}

fail("Should have thrown exception");

}

public void testassertLineItemsEqual_differentQuantity() {

Invoice inv = createAnonInvoice();

LineItem item1 = new LineItem(inv, product, QUANTITY1);

LineItem item2 = new LineItem(inv, product, QUANTITY2);

// exercise/verify

try {

assertLineItemsEqual("Msg",item1, item2);

} catch (AssertionFailedError e) {

pass(); // to indicate that no assertion is needed

return;

}

fail("Should have thrown exception");

}

Custom Assertion

Custom

Assertion

484

Chapter 21 Result Verification Patterns

This example includes a few of the Custom Assertion Tests needed for this Custom

Assertion. Note that the code includes one “equivalent” and several “different”

tests (one for each attribute whose difference should cause the test to fail). We have

to use the second form of the Expected Exception Test template in those cases

where the assertion was expected to fail, because fail throws the same exception as

our assertion method. In one of the “different” tests, we have included sample logic

for asserting on the exception message. (Although I’ve abridged it to save space, the

example here should give you an idea of where to assert on the message.)

Custom

Assertion

485

Delta Assertion

How do we make tests self-checking when we cannot control

the initial contents of the ﬁ xture?

We specify assertions based on differences between the pre- and

post-exercise state of the SUT.

When we are using a Shared Fixture (page 317) such as a test database, it can

be challenging to code the assertions that state what the content of the ﬁ xture

should be after the SUT has been exercised. This is because other tests may have

created objects in the ﬁ xture that our assertions may detect and that may cause

our assertions to fail. One solution is to isolate the current test from all other

tests by using a Database Partitioning Scheme (see Database Sandbox on page

650). But what can we do if this option is not available to us?

Using Delta Assertions allows us to be less dependent on which data already

exist in the Shared Fixture.

How It Works

Before exercising the SUT, we take a snapshot of relevant parts of the Shared

Fixture. After exercising the SUT, we specify our assertions relative to the saved

snapshot. The Delta Assertions typically verify that the number of objects has

changed by the right number and that the contents of collections of objects

returned by the SUT in response to our queries have been augmented by the

expected objects.

Teardown

3. Compare with

Expected Difference

Data

Fixture

Setup

Exercise

Verify

After

Before

1. Get Pre-test State

2. Get Post-test State

SUT

Teardown

3. Compare with

Expected Difference

Data

Fixture

Setup

Exercise

Verify

After

Before

1. Get Pre-test State

2. Get Post-test State

SUT

Delta Assertion

Delta

Assertion

486

Chapter 21 Result Verification Patterns

When to Use It

We can use a Delta Assertion whenever we don’t have full control over the test

ﬁ xture and we want to avoid Interacting Tests (see Erratic Test on page 228).

Using Delta Assertions will help make our tests more resilient to changes in the

ﬁ xture. We can also use Delta Assertions in concert with Implicit Teardown

(page 516) to detect memory or data leaks in the code that we are testing. See

the sidebar “Using Delta Assertions to Detect Data Leakage” on page 487 for a

more detailed description.

Delta Assertions work well when tests are run one after another from the

same Test Runner (page 377). Unfortunately, they cannot prevent a Test Run

War (see Erratic Test) because such a problem arises when tests are run at the

same time from different processes. Delta Assertions work whenever the state

of the SUT and the ﬁ xture are modiﬁ ed only by our own test. If other tests are

running in parallel (not before or after the current test, but at the same time), a

Delta Assertion won’t be sufﬁ cient to avoid the Test Run War problem.

Implementation Notes

When saving the pre-test state of the Shared Fixture or SUT, we must make sure

that the SUT cannot change our snapshot. For example, if our snapshot consists

of a collection of objects returned by the SUT in response to a query, we must

perform a deep copy; a shallow copy copies only the Collection object and not the

objects to which it refers. Shallow copying would allow the SUT to modify the

very objects it returned to us as we exercise it; as a consequence, we would lose

the reference snapshot with which we are comparing the post-test state.

We can ensure that we have the correct post-test state in several different ways.

Assuming that our test adds any new objects it plans to modify, one approach is

to ﬁ rst check that the result collection (1) has the right number of items, (2) con-

tains all the pre-test items, and (3) contains the new Expected Objects (see State

Veriﬁ cation on page 462). Another approach is to remove all the saved items

from the result collection and then compare what remains with the collection of

new expected objects. Both of these approaches can be hidden behind a Custom

Assertion (page 474) or a Veriﬁ cation Method (see Custom Assertion).

Delta

Assertion

487

Using Delta Assertions to Detect Data Leakage

A long time ago, on a project far away, we were experimenting with

different ways to clean up our test ﬁ xtures after the customer tests. Our

tests were accessing a database and leaving objects behind. This behavior

caused all sorts of problems with Unrepeatable Tests (see Erratic Test on

page 228) and Interacting Tests (see Erratic Test). We were also suffering

from Slow Tests (page 253).

Eventually we hit upon the idea of keeping track of all the objects we

were creating in our tests by registering them with an Automated Tear-

down (page 503) mechanism. Then we found a way to stub out the data-

base with a Fake Database (see Fake Object on page 551). Next we made

it possible to run the same test against either the fake database or the real

one. This solved many of the interaction problems when running against

the fake database, although those problems still occurred when we ran

the tests against the real database—tests still left objects behind, and we

wanted to know why. But ﬁ rst we had to determine precisely which tests

were at fault.

The solution turned out to be pretty simple. In our Fake Database—which

was implemented using simple hash tables—we added a method to count

the total number of objects. We simply saved this value in an instance

variable in the

setUp method and used it as the expected value passed to an

Equality Assertion (see Assertion Method on page 362) called in the tear-

Down

method to verify that we had cleaned up all objects properly. [This is

an example of using Delta Assertions (page 485).] Once we implemented

this little trick, we quickly found out which tests were suffering from the

Data Leak (see Erratic Test). We could then focus our efforts on a much

smaller number of tests.

Even today, we ﬁ nd it useful to be able to run the same test against the

database and in memory. Similarly, we still occasionally see a test fail

when the tearDown method inherited from our company-speciﬁ c Testcase

Superclass (page 638) has a Delta Assertion failure. Perhaps the same

idea could be applied to checking for memory leaks in programming lan-

guages with manual memory management (such as C++).

Delta Assertion

Delta

Assertion