Meszaros G. xUnit Test Patterns Refactoring Test Code

Подождите немного. Документ загружается.

118

Chapter 10 Result Verification

Parameterized and Data-Driven Tests

We can go even further in factoring out the commonality between tests. If the

logic to set up the test ﬁ xture is the same but uses different data, we can extract

the common ﬁ xture setup, exercise SUT, and verify outcome phases of the test

into a new Parameterized Test method. This Parameterized Test is not called

automatically by the Test Automation Framework (page 298) because it requires

arguments; instead, we deﬁ ne very simple Test Methods for each test, which

then call the Parameterized Test and pass in the data required to make this test

unique. This data may include that required for ﬁ xture setup, exercising the

SUT, and the corresponding expected result. In the following tests, the method

generateAndVerifyHtml is the Parameterized Test.

def test_extref

sourceXml = "<extref id='abc' />"

expectedHtml = "<a href='abc.html'>abc</a>"

generateAndVerifyHtml(sourceXml,expectedHtml,"<extref>")

end

def test_testterm_normal

sourceXml = "<testterm id='abc'/>"

expectedHtml = "<a href='abc.html'>abc</a>"

generateAndVerifyHtml(sourceXml,expectedHtml,"<testterm>")

end

def test_testterm_plural

sourceXml = "<testterms id='abc'/>"

expectedHtml = "<a href='abc.html'>abcs</a>"

generateAndVerifyHtml(sourceXml,expectedHtml,"<plural>")

end

In a Data-Driven Test (page 288), the test case is completely generic and directly

executable by the framework; it reads the arguments from a test data ﬁ le as it

executes. Think of a Data-Driven Test as a Parameterized Test turned inside out:

A Test Method passes test-speciﬁ c data to a Parameterized Test; a Data-Driven

Test is the Test Method and reads the test-speciﬁ c data from a ﬁ le. The contents

of the ﬁ le are a Higher-Level Language for testing; the Data-Driven Test method

is the Interpreter [GOF] of that language. This scheme is the xUnit equivalent of

a Fit test. A simple example of a Data-Driven Test method is shown in this code

sample written in Ruby:

def test_crossref

executeDataDrivenTest "CrossrefHandlerTest.txt"

end

def executeDataDrivenTest ﬁlename

dataFile = File.open(ﬁlename)

119

dataFile.each_line do | line |

desc, action, part2 = line.split(",")

sourceXml, expectedHtml, leftOver = part2.split(",")

if "crossref"==action.strip

generateAndVerifyHtml sourceXml, expectedHtml, desc

else # new "verbs" go before here as elsif's

report_error( "unknown action" + action.strip )

end

Here is the comma-delimited data ﬁ le that the Data-Driven Test method reads:

ID, Action, SourceXml, ExpectedHtml

Extref,crossref,<extref id='abc'/>,<a href='abc.html'>abc</a>

TTerm,crossref,<testterm id='abc'/>,<a href='abc.html'>abc</a>

TTerms,crossref,<testterms id='abc'/>,<a href='abc.html'>abcs</a>

Avoiding Conditional Test Logic

Another thing we want to avoid in our tests is conditional logic. Conditional

Test Logic (page 200) is bad because the same test may execute differently in

different circumstances. Conditional Test Logic reduces our trust in the tests

because the code in our Test Methods is Untestable Test Code. Why is this

important? Because the only way we can verify our Test Method is to manually

edit the SUT so that it produces the error we want to be detected. If the Test

Method has many paths through it, we need to make sure each path is coded

correctly. Isn’t it so much simpler just to have only one possible execution path

through the test? Let us look at some reasons why we might include conditional

logic in our tests:

• We don’t want to execute certain assertions because their execution

doesn’t make sense given what we have already discovered at this point

in the test (typically a failure condition).

• We have to allow for various situations in the actual results that we are

comparing to the expected results.

• We are trying to reuse a Test Method in several different circumstances

(essentially merging several tests into a single Test Method).

The problem with using Conditional Test Logic in the ﬁ rst two cases is that it

makes the code hard to read and may mask cases of reusing test methods via

Flexible Tests (see Conditional Test Logic). The last “reason” is just a bad idea,

Avoiding Conditional Test Logic

120

Chapter 10 Result Verification

plain and simple. There are much better ways of reusing test logic than trying

to reuse the Test Method itself. We have already seen some of these reuse tech-

niques elsewhere in this chapter (in Reducing Test Code Duplication), and we

will see other ways elsewhere in this book. Just say “no”!

The good news is that it is relatively straightforward to remove all legitimate

uses of Conditional Test Logic from our tests.

Eliminating “if” Statements

What should we do when we don’t want to execute an assertion because we

know it will result in a test error and we would prefer to have a more meaning-

ful test failure message? The normal reaction is to place the assertion inside an

“if” statement, as shown in the following listing. Unfortunately, this approach

results in Conditional Test Logic, which we would dearly like to avoid because

we want exactly the same code to run each time we run the test.

List lineItems = invoice.getLineItems();

if (lineItems.size() == 1) {

LineItem expected =

new LineItem(invoice, product,5,

new BigDecimal("30"),

new BigDecimal("69.96"));

LineItem actItem = (LineItem) lineItems.get(0);

assertEquals("invoice", expected, actItem);

} else {

fail("Invoice should have exactly one line item");

}

The preferred solution is to use a Guard Assertion (page 490) as shown in this

revised version of the test code:

List lineItems = invoice.getLineItems();

assertEquals("number of items", lineItems.size(), 1);

LineItem expected =

new LineItem(invoice, product, 5,

new BigDecimal("30"),

new BigDecimal("69.96"));

LineItem actItem = (LineItem) lineItems.get(0);

assertEquals("invoice", expected, actItem);

The nice thing about Guard Assertions is that they keep us from hitting the as-

sertion that would cause a test error but without introducing Conditional Test

Logic. Once we get used to them, these assertions are fairly obvious and intuitive

to read. We may even ﬁ nd ourselves wanting to assert the pre-conditions of our

methods in our production code!

121

Eliminating Loops

Conditional Test Logic may also appear as loops that verify the content of a col-

lection returned by the SUT matches what we expected. Putting loops directly

into the Test Method creates three problems:

• It introduces Untestable Test Code because the looping code, which is part

of the test, cannot be tested with Fully Automated Tests (see page 26).

• It leads to Obscure Tests because all that looping code obscures the real

intent: Does or doesn’t the collection match?

• It can lead to the project-level smell Developers Not Writing

Tests (page 263) because the complexity of writing the loops may dis-

courage the developer from writing the Self-Checking Test.

A better solution is to delegate this logic to a Test Utility Method with an Intent-

Revealing Name, which can be both tested and reused.

Other Techniques

This section outlines some other techniques for writing easy-to-understand tests.

Working Backward, Outside-In

A useful little trick for writing very intent-revealing code is to work backward.

This is an application of Stephen Covey’s idea, “Start with the end in mind.” To

do so, we write the last line of the function or test ﬁ rst. For a function, its whole

reason for existence is to return a value; for a procedure, it is to produce one or

more side effects by modifying something. For a test, the raison d’ tre is to verify

that the expected outcome has occurred (by making assertions).

Working backward means we write these assertions ﬁ rst. We assert on the values

of suitably named local variables to ensure that the assertion is intent-revealing.

The rest of writing the test simply consists of ﬁ lling in whatever is needed to

execute those assertions: We declare variables to hold the assertion arguments

and initialize them with the appropriate content. Because at least one argument

should have been retrieved from the SUT, we must, of course, invoke the SUT.

To do so, we may need some variables to use as SUT arguments. Declaring and

initializing a variable after it has been used forces us to understand the variable

better when we introduce it. This scheme also results in better variable names

and avoids meaningless names like invoice1 and invoice2.

Other Techniques

122

Chapter 10 Result Verification

Working “outside-in” (or “top-down” as it is sometimes called) means staying

at a consistent level of abstraction. The Test Method should focus on what we

need to have in place to induce the relevant behavior in the SUT. The mechanics

of how we reach that place should be delegated to a “lower layer” of test soft-

ware. In practice, we code this behavior as calls to Test Utility Methods, which

allows us to stay focused on the requirements of the SUT as we write each Test

Method. We don’t need to worry about how we will create that object or verify

that outcome; we merely need to describe what that object or outcome should

be. The utility method we just used but haven’t yet deﬁ ned acts as a placeholder

for the unﬁ nished test automation logic.

We can move on to writing the other

tests we need for this SUT while they are still fresh in our minds. Later, we can

switch to our “toolsmith” hat and implement the Test Utility Methods.

Using Test-Driven Development to Write Test Utility

Methods

Once we are ﬁ nished writing the Test Method(s) that used the Test Utility Method,

we can start the process of writing the Test Utility Method itself. Along the way,

we can take advantage of test-driven development by writing Test Utility Tests

(see Test Utility Method). It doesn’t take very long to write these unit tests that

verify the behavior of our Test Utility Methods and we will have much more

conﬁ dence in them.

We start with the simple case (say, asserting the equality of two identical

collections that hold the same item) and work up to the most complicated case

that the Test Methods actually require (say, two collections that contain the

same two items but in different order). TDD helps us ﬁ nd the minimal implemen-

tation of the Test Utility Method, which may be much simpler than a complete

generic solution. There is no point in writing generic logic that handles cases that

aren’t actually needed but it may be worthwhile to include a Guard Assertion or

two inside the Custom Assertion to fail tests in cases it doesn’t support.

Where to Put Reusable Veriﬁ cation Logic?

Suppose we have decided to use Extract Method refactorings to create some reus-

able Custom Assertions or we have decided to write our tests in an intent-revealing

way using Veriﬁ cation Methods. Where should we put these bits of reusable test

We should always give this method an Intent-Revealing Name and stub it out with a call

to the

fail assertion to remind ourselves that we still need to write the method’s body.

123

logic? The most obvious place is in the Testcase Class (page 373) itself. We can

allow this logic to be reused more broadly by using a Pull-Up Method [Fowler]

refactoring to move them up to a Testcase Superclass (page 638) or a Move Method

[Fowler] refactoring to move them into a Test Helper (page 643). This issue is dis-

cussed in more detail in Chapter 12, Organizing Our Tests.

What’s Next?

This discussion of techniques for verifying the expected outcome concludes our

introduction to the basic techniques of automating tests using xUnit. Chapter 11,

Using Test Doubles, introduces some advanced techniques involving the use of

Test Doubles.

What’s Next?

This page intentionally left blank

125

Chapter 11

Using Test Doubles

About This Chapter

The last few chapters concluding with Chapter 10, Result Veriﬁ cation, intro-

duced the basic mechanisms of running tests using the xUnit family of Test

Automation Frameworks (page 298). For the most part we assumed that the SUT

was designed such that it could be tested easily in isolation of other pieces of soft-

ware. When a class does not depend on any other classes, testing it is relatively

straightforward and the techniques described in this chapter are unnecessary.

When a class does depend on other classes, we have two choices: We can test it

together with all the other classes it depends on or we can try to isolate it from the

other classes so that we can test it by itself. This chapter introduces techniques for

isolating the SUT from the other software components on which it depends.

What Are Indirect Inputs and Outputs?

The problem with testing classes in groups or clusters is that it becomes very hard

to cover all the paths through the code. The depended-on component (DOC) may

return values or throw exceptions that affect the behavior of the SUT, but it may

prove difﬁ cult or impossible to cause certain cases to occur. The indirect inputs

received from the DOC may be unpredictable (such as the system clock or cal-

endar). In other cases, the DOC may not be available in the test environment or

may not even exist. How can we test dependent classes in these circumstances?

In other cases, we need to verify that certain side effects of executing the

SUT have, indeed, occurred. If it is too difﬁ cult to monitor these indirect

outputs of the SUT (or if it is too expensive to retrieve them), the effectiveness of

our automated testing may be compromised.

As you will no doubt have guessed from the title of this chapter, the solution

to these problems is often the use of a Test Double (page 522). We will start by

126

Chapter 11 Using Test Doubles

looking at how we can use Test Doubles to test indirect inputs and outputs. We

will then describe a few other uses of these helpful mechanisms.

Why Do We Care about Indirect Inputs?

Calls to DOCs often return objects or values, update their arguments or even throw

exceptions. Many of the execution paths within the SUT are intended to deal with

these return values and to handle the possible exceptions. Leaving these paths un-

tested leads to Untested Code (see Production Bugs on page 268). These paths can

be the most challenging to test effectively but are also among the most likely to lead

to catastrophic failures if exercised for the very ﬁ rst time in production.

We certainly would rather not have the exception-handling code execute for the

ﬁ rst time in production. What if it was coded incorrectly? Clearly, it would be high-

ly desirable to have automated tests for such code. The testing challenge is to some-

how cause the DOC to throw an exception so that the error path can be tested. The

exception we expect the DOC to throw is a good example of an indirect input test

condition (Figure 11.1). Our means of injecting this input is a control point.

Figure 11.1 An indirect input being received by the SUT from a DOC. Not

all inputs of the SUT come from the test. Some indirect inputs come from other

components called by the SUT in the form of return values, updated parameters,

or exceptions thrown.

Why Do We Care about Indirect Outputs?

The concept of encapsulation often directs us to not care about how some-

thing is implemented. After all, that is the whole purpose of encapsulation—to

alleviate the need for clients of our interface to care about our implementation.

Fixture

Setup

Exercise

Verify

Teardown

SUT

DOC

Indirect

Input

Fixture

Setup

Exercise

Verify

Teardown

SUT

DOC

Indirect

Input

127

When testing, we try to verify the implementation precisely so our clients do not

have to care about it.

Consider for a moment a component that has a method in its API that

returns nothing—or at least nothing that can be used to determine whether it

has performed its function correctly. In this situation, we have no choice but to

test through the back door. A good example of this is a message logging system.

Calls to the API of a logger rarely return anything that indicates it did its job

correctly. The only way to determine whether the message logging system is

working as expected is to interact with it through some other interface—one

that allows us to retrieve the logged messages.

A client of the logger may specify that the logger be called when certain con-

ditions are met. These calls will not be visible on the client’s interface but would

typically be a requirement that the client needs to satisfy and, therefore, would

be something we want to test. The circumstances that should result in a messag-

ing being logged are indirect output test conditions (Figure 11.2) for which we

need to write tests so that we can avoid having Untested Requirements (see Pro-

duction Bugs). Our means of seeing this output is an observation point.

Figure 11.2 An indirect output being received by the SUT. Not all outputs of

the SUT are directly visible to the test. Some indirect outputs are sent to other

components in the form of method calls or messages.

In other cases, the SUT does produce visible behavior that can be veriﬁ ed

through the front door but also has some expected side effects. Both outputs

need to be veriﬁ ed in our tests. Sometimes this testing is simply a matter of

adding assertions for the indirect outputs to the existing tests to verify the

Untested Requirement.

Fixture

Setup

Exercise

Verify

Teardown

SUT

DOC

Indirect

Output

Fixture

Setup

Exercise

Verify

Teardown

SUT

DOC

Indirect

Output

What Are Indirect Inputs and Outputs?