Meszaros G. xUnit Test Patterns Refactoring Test Code

Подождите немного. Документ загружается.

Chapter 9 Persistent Fixture Management

// Verify Outcome

assertEquals(0,ﬂightsAtDestination1.size());

} ﬁnally {

facade.removeAirport(outboundAirport);

}

Unfortunately, the mechanism to ensure that the teardown code always runs in-

troduces a fair bit of complication into the Test Method. Matters become even

more complicated when we must tear down several resources: Even if our attempt

to clean up one resource fails, we want to ensure that the other resources are still

cleaned up. We can address part of this problem by using Extract Method [Fowler]

refactoring to move the teardown code into a Test Utility Method (page 599)

that we call from inside the error-handling construct. Although this Delegated

Teardown (see In-line Teardown) hides the complexity of dealing with teardown

errors, we still need to ensure that the method gets called even when test errors

or test failures occur.

Most members of the xUnit family solve this problem by supporting Implicit

Teardown (page 516). The Test Automation Framework (page 298) calls a spe-

cial tearDown method after each Test Method regardless of whether the test passed

or failed. This approach avoids placing the error-handling code within the Test

Method but imposes two requirements on our tests. First, the ﬁ xture must be

accessible from the tearDown method, so we must use instance variables (pre-

ferred), class variables, or global variables to hold the ﬁ xture. Second, we must

ensure that the tearDown method works properly with each of the Test Methods

regardless of which ﬁ xture it sets up.

Matching Setup with Teardown Code Organization

Given the three ways of organizing our setup code—In-line Setup (page 408),

Delegated Setup (page 411), and Implicit Setup (page 424)—and the three ways

of organizing our teardown code—In-line Teardown, Delegated Teardown, and

Implicit Teardown—nine different combinations are available to us. Choosing

the right one turns out to be an easy decision because it is not important for

the teardown code to be visible to the test reader. We simply choose the most

appropriate setup code organization and either the equivalent or more hidden

version of teardown (Table 9.1). For example, it is appropriate to use Implicit

Teardown even with In-line Setup or Delegated Setup; it is almost never a good

This is less of an issue with Testcase Class per Fixture (page 631) because the ﬁ xture

should always be the same. With other Testcase Class organizations, we may need to

include Teardown Guard Clauses (see In-line Teardown) within the

tearDown method to

ensure that it doesn’t produce errors when it runs.

idea to use In-line Teardown with anything other than In-line Setup, and even

then it should probably be avoided!

Table 9.1 The Compatibility of Various Fixture Setup and Teardown Strate-

gies for Persistent Test Fixtures

Teardown Mechanism

Setup Mechanism In-line Teardown Delegated Teardown Implicit Teardown

In-line Setup Not recommended Acceptable Recommended

Delegated Setup Not recommended Acceptable Recommended

Implicit Setup Not recommended Not recommended Recommended

Automated Teardown

Hand-coded teardown is associated with two problems: Extra work is required

to write the tests, and the teardown code is hard to get right and even harder

to test. When the teardown goes wrong, it may lead to Erratic Tests caused by

Resource Leakage because the test that fails as a result is often different from the

one that didn’t clean up properly.

In languages that support garbage collection, tearing down a Transient Fresh

Fixture should be pretty much automatic. As long as our ﬁ xtures are referenced only

by instance variables that go out of scope when our Testcase Object (page 382) is

destroyed, garbage collection will clean them up. Garbage collection won’t work,

however, if we use class variables or if our ﬁ xtures include persistent objects such

as ﬁ les or database rows. In those cases, we need to perform our own cleanup.

Not surprisingly, this situation may inspire the lazy but creative programmer

to come up with a way to automate the teardown logic. The important thing to

note is that teardown code doesn’t help us understand the test so it is better for

it to remain hidden.

We can eliminate the need to write hand-crafted teardown

code for each Test Method or Testcase Class by building an Automated Tear-

down (page 503) mechanism. It consists of three parts:

1. A well-tested mechanism to iterate over a list of objects that need to be

deleted and catch/report any errors it encounters while ensuring that

all the deletions are attempted.

2. A dispatching mechanism that invokes the deletion code appropriate

to the kind of object to be deleted. This mechanism is often imple-

mented as a Command [GOF] object that wraps each object to be

Unlike setup code, which is often very important for understanding the test.

Managing Persistent Fresh Fixtures

100

Chapter 9 Persistent Fixture Management

deleted, but could be as simple as calling a delete method on the object

itself or using a switch statement based on the object’s class.

3. A registration mechanism to add newly created objects (suitably

wrapped if necessary) to the list of objects to be deleted.

Once we have built our Automated Teardown mechanism, we can simply in-

voke the registration method from our Creation Methods (page 415) and the

cleanup method from the tearDown method. The latter operation can be speciﬁ ed

in a Testcase Superclass (page 638) that all of our Testcase Classes inherit from.

We can even extend this mechanism to delete objects created by the SUT as it

is exercised. To do so, we use an observable Object Factory (see Dependency

Lookup on page 686) inside the SUT and have our Testcase Superclass register

itself as an Observer [GOF] of object creation.

Database Teardown

When our persistent Fresh Fixture has been built entirely in a relational database,

we can take advantage of certain features of the database to implement its tear-

down. Table Truncation Teardown (page 661) is a brute-force way to blow away

the entire contents of a table with a single database command. Of course, it is

appropriate only when each Test Runner (page 377) has its own Database Sand-

box (page 650). A somewhat less drastic approach is to use Transaction Rollback

Teardown (page 668) to undo all changes made within the context of the current

test. This mechanism relies on the SUT having been designed using the Humble

Transaction Controller pattern (see Humble Object on page 695) so that we can

invoke the business logic from the test without having the SUT commit the trans-

action automatically. Both of these database-speciﬁ c teardown patterns are most

commonly implemented using Implicit Teardown to keep the teardown logic out

of the Test Methods.

Avoiding the Need for Teardown

So far, we have looked at ways to do ﬁ xture teardown. Now, let us look at ways

to avoid ﬁ xture teardown.

Avoiding Fixture Collisions

We need to do ﬁ xture teardown for three reasons:

1. The accumulation of leftover ﬁ xture objects can cause tests to run

slowly.

101

2. The leftover ﬁ xture objects can cause the SUT to behave differently or

our assertions to report incorrect results.

3. The leftover ﬁ xture objects can prevent us from creating the Fresh Fix-

ture required by our test.

The issue that is easiest to address is the ﬁ rst one: We can schedule a periodic

cleansing of the persistence mechanism back to a known, minimalist state. Un-

fortunately, this tactic is useful only if we can get the tests to run correctly in the

presence of accumulated test detritus.

The second issue can be addressed by using Delta Assertions (page 485)

rather than “absolute” assertions. Delta Assertions work by taking a snapshot

of the ﬁ xture before the test is run and verifying that the expected differences

have appeared after we exercise the SUT.

The third issue can be addressed by ensuring that each test generates a differ-

ent set of ﬁ xture objects each time it is run. Thus any objects that the test needs

to create must be given totally unique identiﬁ ers—that is, unique ﬁ lenames,

unique keys, and so on. To do so, we can build a simple unique ID generator

and create a new ID at the beginning of each test. We can then use that unique

ID as part of the identity of each newly created ﬁ xture object. If the ﬁ xture is

shared beyond a single Test Runner, we may need to include something about

the user in the unique identiﬁ ers we create; the currently logged-in user ID is

usually sufﬁ cient. Using Distinct Generated Values as keys offers another ben-

eﬁ t: It allows us to implement a Database Partitioning Scheme (see Database

Sandbox) in which we can use absolute assertions despite the presence of left-

over ﬁ xture objects.

Avoiding Fixture Persistence

We seem to be going to a lot of trouble to undo the side effects caused by a

persistent Fresh Fixture. Wouldn’t it be nice if we could avoid all of this work?

The good news is that we can. The bad news is that we need to make our Fresh

Fixture nonpersistent to do so. When the SUT is to blame for the persistence of

the ﬁ xture, one possibility is to replace the persistence mechanism with a Test

Double (page 522) that the test can wipe out at will. A good example of this ap-

proach is the use of a Fake Database (see Fake Object on page 551). When the

test is to blame for ﬁ xture persistence, the solution is even easier: Just use a less

persistent ﬁ xture reference mechanism.

Managing Persistent Fresh Fixtures

102

Chapter 9 Persistent Fixture Management

Dealing with Slow Tests

A major drawback of using a Persistent Fresh Fixture is speed or, more precisely,

the lack thereof. File systems and databases are much slower than the processors

used in modern computers. As a consequence, tests that interact with databases

tend to run much more slowly than tests that run entirely in memory. Part of

this difference arises because the SUT is accessing the ﬁ xture from disk—but this

issue turns out to be only a small part of the reason for the slowdown. Setting

up the Fresh Fixture at the beginning of each test and tearing it down at the end

of each test typically takes many more disk accesses than those used by the SUT

to access the ﬁ xture. As a result, tests that access the database often take 50 to

100 times

longer to run than tests that run entirely in memory, all other things

being equal.

The typical reaction to slow tests caused by Persistent Fresh Fixtures is to

eliminate the ﬁ xture setup and teardown overhead by reusing the ﬁ xture across

many tests. Assuming we have ﬁ ve disk accesses to set up and tear down the

ﬁ xture for every disk access performed by the SUT, the absolute best

we can

do by switching to a Shared Fixture is somewhere around ten times as slow.

Of course, this outcome is still too slow in most situations and it comes with a

hefty price: The tests are no longer independent. That means we will likely have

Interacting Tests (see Erratic Test), Lonely Tests (see Erratic Test), and Unre-

peatable Tests on top of our Slow Tests!

A much better solution is to eliminate the need to have a disk-based database

under the application. With a small amount of effort, we should be able to re-

place the disk-based database with an In-Memory Database (see Fake Object)

or a Fake Database. This decision is best made early in the project while the

effort is still low. Yes, there are some challenges, such as dealing with stored

procedures, but they are all surmountable.

This tactic isn’t the only way to deal with Slow Tests, of course. The side-

bar “Faster Tests Without Shared Fixtures” (page 319) explores some other

strategies.

This is two orders of magnitude!

Your mileage may vary.

103

Managing Shared Fixtures

Managing Shared Fixtures has a lot in common with managing Persistent Fresh

Fixtures, except that we deliberately choose not to tear the ﬁ xture down after every

test so that we can reuse it in subsequent tests (Figure 9.2). This implies two things.

First, we must be able to access the ﬁ xture in the other tests. Second, we must have

a way of triggering both the construction and the teardown of the ﬁ xture.

Figure 9.2 A Shared Fixture with two Test Methods that share it. A Shared

Fixture is set up once and used by two or more tests that may interact, either

deliberately or accidentally, as a result. Note the lack of a ﬁ xture setup phase for

the second test.

Accessing Shared Fixtures

Regardless of how and when we choose to build the Shared Fixture, the tests

need a way to ﬁ nd the test ﬁ xture they are to reuse. The choices available to

us depend on the nature of the ﬁ xture. When the ﬁ xture is stored in a database

(the most common usage of a Shared Fixture), tests may access it directly with-

out making direct references to the ﬁ xture objects as long as they know about

the database. There may be a temptation to use Hard-Coded Values (see Literal

Value on page 714) in database lookups to access the ﬁ xture objects. This is al-

most always a bad idea, however, because it leads to a close coupling between

tests and the ﬁ xture implementation and because it has poor documentation value

(Obscure Test; page 186). To avoid these potential problems, we can use Finder

Methods (see Test Utility Method) with Intent-Revealing Names [SBPP] to access

Setup

Exercise

Verify

Teardown

Fixture

Exercise

Verify

Teardown

SUT

Setup

Exercise

Verify

Teardown

Fixture

Exercise

Verify

Teardown

SUT

Managing Shared Fixtures

104

Chapter 9 Persistent Fixture Management

the ﬁ xture. These Finder Methods may have names that are very similar to those

of Creation Methods, but they return references to existing ﬁ xture objects rather

than building brand new ones.

We have a range of possible solutions when the ﬁ xture is stored in memory. If

all tests that need to share the ﬁ xture are in the same Testcase Class, we can use a

ﬁ xture holding class variable to hold the reference to the ﬁ xture. As long as we give

the variable an Intent-Revealing Name, the test reader should be able to understand

the pre-conditions of the test. Another alternative is to use a Finder Method.

If we need to share the ﬁ xture across many Testcase Classes, we must use a

more sophisticated technique. We could, of course, let one class declare the ﬁ xture

holding class variable and have the other tests access the ﬁ xture via that variable.

Unfortunately, this approach may create unnecessary coupling between the tests.

Another alternative is to move the declaration to a well-known object—namely, a

Test Fixture Registry (see Test Helper on page 643). This Registry [PEAA] object

could be something like a test database or it could merely be a class. It can expose

various parts of a ﬁ xture via discrete ﬁ xture holding class variables or via Finder

Methods. When the Test Fixture Registry has only Finder Methods that know how

to access the objects but don’t hold references to them, we call it a Test Helper.

Triggering Shared Fixture Construction

For a test ﬁ xture to be shared, it must be built before any Test Method needs it.

This construction could take place as late as right before the Test Method’s logic

is run, just before the entire test suite is run, or at some earlier time (Figure 9.3).

This leads us to the basic patterns of Shared Fixture creation.

Figure 9.3 The plethora of ways to manage a Shared Fixture. A Shared Fixture

can be set up at a variety of times; the decision is based on how many tests need

to reuse the ﬁ xture and how many times they need to do so.

Shared Fixture Setup

Shared

Fixture

Setup

Decorator

Suite

Fixture

Setup

Lazy

Setup

Prebuilt

Fixture

Chained

Tests

Shared Fixture Setup

Shared

Fixture

Setup

Decorator

Suite

Fixture

Setup

Lazy

Setup

Prebuilt

Fixture

Chained

Tests

105

If we are happy with the idea of creating the test ﬁ xture the ﬁ rst time any test

needs it, we can use Lazy Setup (page 435) in the setUp method of the corre-

sponding Testcase Class to create it as part of running the ﬁ rst test. Subsequent

tests will then see that the ﬁ xture already exists and reuse it. Because there is no

obvious signal that the last test in a test suite (or Suite of Suites; see Test Suite

Object on page 387) has been run, we won’t know when to tear down the ﬁ x-

ture after each test run. This can lead to Unrepeatable Tests because the ﬁ xture

may survive across test runs (depending on how the various tests access it).

If we need to share the ﬁ xture more broadly, we could include a Fixture Set-

up Testcase at the beginning of the test suite. This is a special case of Chained

Tests and suffers from the same problem as Lazy Setup—speciﬁ cally, we don’t

know when it is time to tear down the ﬁ xture. It also depends on the ordering

of tests within a suite, so it works best with Test Enumeration (page 399).

If we need to be able to tear down the test ﬁ xture after running a test suite,

we must use a ﬁ xture management mechanism that tells us when the last test

has been run. Several members of the xUnit family support the concept of a

setUp method that runs just once for the test suite created from a single Testcase

Class. This Suite Fixture Setup (page 441) method has a corresponding tearDown

method that is called when the last Test Method has ﬁ nished running.

We can

then guarantee that a new ﬁ xture is built for each test run. The ﬁ xture is not left

over to cause problems with subsequent test runs, which prevents Unrepeatable

Tests; it does not prevent Interacting Tests within the test run, however. This

capability could be added as an extension to any member of the xUnit family.

When it isn’t supported or when we need to share the ﬁ xture beyond a single

Testcase Class, we can resort to using a Setup Decorator (page 447) to bracket

the running of a test suite with the execution of the ﬁ xture setUp and tearDown

logic. The biggest drawback of Setup Decorator is that tests that depend on the

decorator cannot be run by themselves; they are Lonely Tests.

The ﬁ nal option is to build the ﬁ xture well before the tests are run—that is,

to employ a Prebuilt Fixture (page 429). This approach offers the most options

regarding how the test ﬁ xture is actually constructed because the ﬁ xture setup

need not be executable from within xUnit. For example, it could be set up manu-

ally, by using database scripts, by copying a “golden” database, or by running

a data generation program. The major disadvantage with a Prebuilt Fixture

is that if any tests are Unrepeatable Tests, we will need to perform a Manual

Intervention (page 250) before each test run. As a result, a Prebuilt Fixture is of-

ten used in combination with a Fresh Fixture to construct an Immutable Shared

Fixture (see Shared Fixture).

Think of it as a built-in decorator for a single Testcase Class.

Managing Shared Fixtures

106

Chapter 9 Persistent Fixture Management

What’s Next?

Now that we’ve determined how we will set up and tear down our ﬁ xtures, we are

ready to turn our attention to exercising the SUT and verifying that the expected

outcome has occurred using calls to Assertion Methods. This process is described

in more detail in Chapter 10, Result Veriﬁ cation.

107

Chapter 10

Result Veriﬁ cation

About This Chapter

Chapter 8, Transient Fixture Management, and Chapter 9, Persistent Fixture

Management, described how to set up the test ﬁ xture and how to tear it down

after exercising the SUT. This chapter introduces a variety of options for verify-

ing that the SUT has behaved correctly, including exercising the SUT and com-

paring the actual outcome with the expected outcome.

Making Tests Self-Checking

One of the key characteristics of tests automated using xUnit is that they can be

(and should be) Self-Checking Tests (see Goals of Test Automation on page 21).

This characteristic makes them cost-effective enough to be run very frequently.

Most members of the xUnit family come with a collection of built-in Assertion

Methods (page 362) and some documentation that tells us which one to use

when. On the surface this sounds pretty simple—but there’s a lot more to writ-

ing good tests than just calling the built-in Assertion Methods. We also need to

learn key techniques for making tests easy to understand and for avoiding and

removing Test Code Duplication (page 213).

A key challenge in coding the assertions is getting access to the information

we want to compare with the expected results. This is where observation points

come into play; they provide a window into the state or behavior of the SUT

so that we can pass it to the Assertion Methods. Observation points for infor-

mation accessible via synchronous method calls are relatively straightforward;

observation points for other kinds of information can be quite challenging,

which is precisely what makes automated unit testing so interesting.

Assertions are usually—but not always—called from within the Test Method

(page 348) body right after the SUT has been exercised. Some test automaters put