Meszaros G. xUnit Test Patterns Refactoring Test Code

Подождите немного. Документ загружается.

218

Chapter 15 Code Smells

Ariane

The maiden ﬂ ight of the Ariane 5 rocket was a complete disaster: The

rocket blew up only 37 seconds after takeoff. The culprit was a seem-

ingly innocuous bit of code that was used only while the rocket was on

the ground but unfortunately was left running for the ﬁ rst 40 seconds of

ﬂ ight. When it tried to assign a 64-bit number representing the sideways

velocity of the rocket to a 16-bit ﬁ eld, the navigation computer decided

that the rocket was going the wrong way! It tried to correct the course,

but the sudden change in direction tore the booster rocket apart. While

this is not quite an example of Test Logic in Production (page 217), it

certainly does illustrate the risks associated with this type of error.

Could this disaster have been prevented by use of automated tests? While

it is difﬁ cult to say with certainty, and one could certainly claim that any

number of process changes could have detected this problem before it

occurred, it is conceivable that automated tests could have averted this

catastrophe.

In particular, a test should have addressed the boundary condition—

namely, what happens when a number exceeds the maximum value stor-

able. Such a test would have prevented an exception from occurring for

the ﬁ rst time ever in production.

In addition, the presence of the tests from the Ariane 4 version of the

rocket would have documented the maximum down-range velocity. It is

quite possible that these tests would have been updated when the Ariane

5 software was being developed and that the new tests would have failed

because of the new rocket’s higher speed.

For a slightly more detailed (and very interesting) description of “the

little bug that could,” visit http://www.around.com/ariane.html.

Impact

Code that was not designed to work in production and that has not been veri-

ﬁ ed to work properly in the production environment could accidentally be run

in production and create serious problems.

The Ariane 5 rocket blew up 37 seconds after takeoff on its maiden ﬂ ight

because a piece of code that was used only while the rocket was on the ground

was left running for the ﬁ rst 40 seconds of ﬂ ight. This code tried to assign

a 64-bit number representing the sideways velocity of the rocket to a 16-bit

Test

Logic in

Production

219

ﬁ eld—an operation that convinced the rocket’s navigation computer that it

was going the wrong way. (See the sidebar on Ariane on page 218 for more

details.) While we believe the Test Hook would never be exercised in produc-

tion, do we really want to take this kind of chance?

Root Cause

In some cases, the Test Logic in Production is introduced to make the behavior

of the SUT more deterministic by returning known (hard-coded) values. In other

cases, the Test Logic in Production may have been introduced to avoid execut-

ing code that cannot be run in a test environment. Unfortunately, this approach

can result in failure to execute that code in the production environment if some-

thing is misconﬁ gured.

In some cases, tests may require that the SUT execute additional code that

would otherwise be executed by a depended-on component. For example, code

run from a trigger in a database will not run if the database is replaced by a

Fake Database (see Fake Object on page 551); thus the test needs to ensure that

the equivalent logic is executed from somewhere within the SUT.

Possible Solution

Instead of adding test logic into the production code directly, we can move logic

into a substitutable dependency. We can put code that should be run in only pro-

duction into a Strategy [GOF] object that is installed by default and replaced by a

Null Object [PLOPD3] when running our tests. In contrast, code that should be

run only during tests can be put into a Strategy [GOF] object that is conﬁ gured

as a Null Object by default. Then, when we want the SUT to execute extra code

during testing, we can replace this Strategy object with a test-speciﬁ c version. To

ensure this mechanism is conﬁ gured properly, we should have a Constructor Test

(see Test Method on page 348) to verify that any variables holding references to

Strategy objects are initialized correctly when they are not overridden by the test.

It may also be possible to override speciﬁ c methods of the SUT in a Test-

Speciﬁ c Subclass (page 579) if the production logic we want to circumvent is

localized in overridable methods. This ability is enabled by Self-Calls [WWW].

Cause: For Tests Only

Code exists in the SUT strictly for use by tests.

Symptoms

Some of the methods of the SUT are used only by tests. Some of the attributes

are public when they really should be private.

Test Logic in Production

Test

Logic in

Production

220

Chapter 15 Code Smells

Impact

Software that is added to the SUT For Tests Only makes the SUT more complex.

It can confuse potential clients of the software’s interface by introducing addi-

tional methods that should not be used by any code other than the tests. These

methods may have been tested only in very speciﬁ c circumstances, so they might

not work in the typical usage patterns used by real client software.

Root Cause

The test automater may need to add methods to a class that expose information

needed by the test or methods that provide greater control over initialization

(such as for the installation of a Test Double; see page 522). Test-driven devel-

opment will lead to the creation of these additional methods even though they

aren’t really needed by clients. When retroﬁ tting tests onto legacy code, the test

automater may need access to information or functionality that is not already

exposed.

For Tests Only can also result when a SUT is used asymmetrically in real life.

Automated tests (especially round-trip tests) typically use software in a more

symmetric fashion and hence may need methods that the real software clients

do not need.

Possible Solution

We can assure that tests have access to private information by creating a Test-

Speciﬁ c Subclass of the SUT, which then provides methods to expose the needed

attributes or initialization logic. A test needs to be able to create instances of the

subclass instead of the SUT class for this approach to work.

If for some reason the extra methods cannot be moved to a Test-Speciﬁ c Sub-

class, they should be clearly labeled For Tests Only. This can be done by adopt-

ing a naming convention such as starting the names with “FTO_”.

Cause: Test Dependency in Production

Production executables depend on test executables.

Symptoms

We cannot build only the production code; some test code must be included

in the build to allow the production code to compile. Alternatively, we might

notice that we cannot run the production code if the test executables are not

present.

Test

Logic in

Production

221

Impact

Even if the production modules do not contain any test code, problems can arise

if any of these modules depends on a test module. At minimum, this dependency

increases the size of the executable even if none of the test code is actually used

in production scenarios. It also opens the door to accidental execution of test

code during production.

Root Cause

Test Dependency in Production is usually caused by a lack of attention to

inter-module dependencies. It may also arise when a built-in self-test requires

access to parts of the test automation infrastructure, such as Test Utility

Methods (page 599) or the Test Automation Framework (page 298), to report

test results.

Possible Solution

We must manage our dependencies carefully to ensure that no production code

depends on test code even for innocuous things such as type deﬁ nitions.

Anything required by both test and production code should live in a production

module or class that is accessible to both.

Cause: Equality Pollution

Another cause of Test Logic in Production is the implementation of test-speciﬁ c

equality in the equals method of the SUT.

Symptoms

Equality Pollution can be difﬁ cult to spot once it has occurred—what is notable

is that the SUT doesn’t actually need the equals method to be implemented. In

other cases, behavioral symptoms may appear, such as test failure when the

equals method is modiﬁ ed to support the speciﬁ c needs of a test or when the deﬁ -

nition of equals changes within the SUT as part of a new feature or user story.

Impact

We may write unnecessary equals methods simply to satisfy tests. We may

also change the deﬁ nition of equals so that it no longer satisﬁ es the business

requirements.

Equality Pollution may make it difﬁ cult to introduce the equals logic pre-

scribed by some new requirement if it already exists to support test-speciﬁ c

equality for another test.

Test Logic in Production

Test

Logic in

Production

222

Chapter 15 Code Smells

Root Cause

Equality Pollution is caused by a lack of awareness of the concept of test-speciﬁ c

equality. Some early versions of dynamic Mock Object (page 544) generation tools

forced us to use the SUT’s deﬁ nition of equals, which led to Equality Pollution.

Possible Solution

When a test requires test-speciﬁ c equality, we should use a Custom Asser-

tion (page 474) instead of modifying the equals method just so that we can use a

built-in Equality Assertion (see Assertion Method on page 362).

When using dynamic Mock Object generation tools, we should use a Com-

parator [WWW] rather than relying on the equals method supplied by the SUT.

We can also implement the equals method on a Test-Speciﬁ c Subclass of an

Expected Object (see State Veriﬁ cation on page 462) to avoid adding it to a

production class directly.

Further Reading

For Tests Only and Equality Pollution were ﬁ rst introduced in a paper at XP2001

called “Refactoring Test Code” [RTC].

Test

Logic in

Production

223

Chapter 16

Behavior Smells

Smells in This Chapter

Assertion Roulette . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

Erratic Test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

Fragile Test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

Frequent Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

Manual Intervention. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

Slow Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

Behavior

Smells

224

Assertion Roulette

It is hard to tell which of several assertions within the same

test method caused a test failure.

Symptoms

A test fails. Upon examining the output of the Test Runner (page 377), we cannot

determine exactly which assertion failed.

Impact

When a test fails during an automated Integration Build [SCM], it may be hard

to tell exactly which assertion failed. If the problem cannot be reproduced on

a developer’s machine (as may be the case if the problem is caused by environ-

mental issues or Resource Optimism; see Erratic Test on page 228) ﬁ xing the

problem may be difﬁ cult and time-consuming.

Causes

Cause: Eager Test

A single test veriﬁ es too much functionality.

Symptoms

A test exercises several methods of the SUT or calls the same method several

times interspersed with ﬁ xture setup logic and assertions.

public void testFlightMileage_asKm2() throws Exception {

// set up ﬁxture

// exercise constructor

Flight newFlight = new Flight(validFlightNumber);

// verify constructed object

assertEquals(validFlightNumber, newFlight.number);

assertEquals("", newFlight.airlineCode);

assertNull(newFlight.airline);

// set up mileage

newFlight.setMileage(1122);

// exercise mileage translator

int actualKilometres = newFlight.getMileageAsKm();

// verify results

int expectedKilometres = 1810;

assertEquals( expectedKilometres, actualKilometres);

Chapter 16 Behavior Smells

Assertion

Roulette

225

// now try it with a canceled ﬂight

newFlight.cancel();

try {

newFlight.getMileageAsKm();

fail("Expected exception");

} catch (InvalidRequestException e) {

assertEquals( "Cannot get cancelled ﬂight mileage",

e.getMessage());

}

Another possible symptom is that the test automater wants to modify the Test

Automation Framework (page 298) to keep going after an assertion has failed

so that the rest of the assertions can be executed.

Root Cause

An Eager Test is often caused by trying to minimize the number of unit tests

(whether consciously or unconsciously) by verifying many test conditions

in a single Test Method (page 348). While this is a good practice for manu-

ally executed tests that have “liveware” interpreting the results and adjusting

the tests in real time, it just doesn’t work very well for Fully Automated Tests

(see page 26).

Another common cause of Eager Tests is using xUnit to automate customer

tests that require many steps, thereby verifying many aspects of the SUT in

each test. These tests are necessarily longer than unit tests but care should be

taken to keep them as short as possible (but no shorter!).

Possible Solution

For unit tests, we break up the test into a suite of Single-Condition Tests (see

page 45) by teasing apart the Eager Test. It may be possible to do so by using

one or more Extract Method [Fowler] refactorings to pull out independent

pieces into their own Test Methods. Sometimes it is easier to clone the test once

for each test condition and then clean up each Test Method by removing any

code that is not required for that particular test conditions. Any code required

to set up the ﬁ xture or put the SUT into the correct starting state can be ex-

tracted into a Creation Method (page 415). A good IDE or compiler will then

help us determine which variables are no longer being used.

If we are automating customer tests using xUnit, and this effort has resulted

in many steps in each test because the work ﬂ ows require complex ﬁ xture setup,

we could consider using some other way to set up the ﬁ xture for the latter parts

of the test. If we can use Back Door Setup (see Back Door Manipulation on

page 327) to create the ﬁ xture for the last part of the test independently of the

Assertion Roulette

Assertion

Roulette

226

Chapter 16 Behavior Smells

ﬁ rst part, we can break one test into two, thereby improving our Defect Local-

ization (see Goals of Test Automation). We should repeat this process as many

times as it takes to make the tests short enough to be readable at a single glance

and to Communicate Intent (see page 41) clearly.

Cause: Missing Assertion Message

Symptoms

A test fails. Upon examining the output of the Test Runner, we cannot deter-

mine exactly which assertion failed.

Root Cause

This problem is caused by the use of Assertion Method (page 362) calls with

identical or missing Assertion Messages (page 370). It is most commonly

encountered when running tests using a Command-Line Test Runner (see Test

Runner) or a Test Runner that is not integrated with the program text editor or

development environment.

In the following test, we have a number of Equality Assertions (see Assertion

Method):

public void testInvoice_addLineItem7() {

LineItem expItem = new LineItem(inv, product, QUANTITY);

// Exercise

inv.addItemQuantity(product, QUANTITY);

// Verify

List lineItems = inv.getLineItems();

LineItem actual = (LineItem)lineItems.get(0);

assertEquals(expItem.getInv(), actual.getInv());

assertEquals(expItem.getProd(), actual.getProd());

assertEquals(expItem.getQuantity(), actual.getQuantity());

}

When an assertion fails, will we know which one it was? An Equality Assertion

typically prints out both the expected and the actual values—but it may prove

difﬁ cult to tell which assertion failed if the expected values are similar or print

out cryptically. A good rule of thumb is to include at least a minimal Assertion

Message whenever we have more than one call to the same kind of Assertion

Method.

Possible Solution

If the problem occurred while we were running a test using a Graphical Test

Runner (see Test Runner) with IDE integration, we should be able to click on

the appropriate line in the stack traceback to have the IDE highlight the failed

Assertion

Roulette

227

assertion. Failing this, we can turn on the debugger and single-step through the

test to see which assertion statement fails.

If the problem occurred while we were running a test using a Command-

Line Test Runner, we can try running the test from a Graphical Test Runner

with IDE integration to determine the offending assertion. If that doesn’t work,

we may have to resort to using line numbers (if available) or apply a process of

elimination to deduce which of the assertions it couldn’t be to narrow down the

possibilities. Of course, we could just bite the bullet and add a unique Assertion

Message (even just a number!) to each call to an Assertion Method.