Meszaros G. xUnit Test Patterns Refactoring Test Code

Подождите немного. Документ загружается.

Why We Do This

We wouldn’t want each test automater to have to provide a special means of

running his or her own test suites. That requirement would just get in the way

of our ability to simultaneously run all the tests automated by different people.

By providing a standard Test Runner, we encourage developers to make it easy

to run tests written by different people. We can also provide different ways of

running the same tests.

Implementation Notes

Several styles of Test Runners are available. The most common variations are

running tests from within an IDE and running tests from the command line.

All of these schemes depend on the fact that all Testcase Objects implement a

standard interface.

Standard Test Interface

Statically typed languages (such as Java and C#) typically include an interface

type (fully abstract class) that deﬁ nes the interface that all Testcase Objects and

Test Suite Objects must implement. Some languages (such as C# and Java 5.0)

“mix” in the implementation by using class attributes or annotations on the

Testcase Class. In dynamically typed languages, this interface may not exist

explicitly. Instead, each implementation class simply implements the standard

interface methods. Typically, the standard test interface includes methods on it

to count the available tests and to run the tests. Where the framework supports

Test Enumeration, each Testcase Class and test suite class must also implement

the Test Suite Factory method (see Test Enumeration on page 399), which is

typically called suite.

Variation: Graphical Test Runner

A Graphical Test Runner is typically a desktop application or part of an IDE

(either built-in or a plug-in) for running tests. At least one, IeUnit, runs inside a

Web browser rather than an IDE. The most common feature of the Graphical

Test Runner is some sort of real-time progress indicator. This monitor typically

includes a running count of test failures and errors and often includes a colored

progress bar that starts off green and turns red as soon as an error or failure is

encountered. Some members of the xUnit family include a graphical Test Tree

Explorer as a means to drill down and run a single test from within a Suite of

Suites.

Test Runner

378

Chapter 19 xUnit Basics Patterns

Here is the Graphical Test Runner from the JUnit plug-in for Eclipse:

The red bar near the top indicates that at least one test has failed. The upper

text pane shows a list of test failures and test errors. The lower pane shows the

traceback from the failed test selected in the upper pane.

Variation: Command-Line Test Runner

Command-Line Test Runners are designed to be used from an operating system

command line or from batch ﬁ les or shell scripts. They are very handy when

working remotely via remote shells or when running the tests from a build script

such as “make,” Ant, or a continuous integration tool such as “Cruise Control.”

The following example shows how to run an runit (one of the xUnit imple-

mentations for the Ruby programming language) test from the command line:

Test Runner

379

Test Runner

380

Chapter 19 xUnit Basics Patterns

>ruby testrunner.rb c:/examples/tests/SmellHandlerTest.rb

Loaded suite SmellHandlerTest

Started

.....

Finished in 0.016 seconds.

5 tests, 6 assertions, 0 failures, 0 errors

>Exit code: 0

The ﬁ rst line is the invocation at the command prompt. In this example we are

running the tests deﬁ ned in a single Testcase Class, SmellHandlerTest. The next

two lines are the initial feedback as the tests begin. The series of dots indicates

the tests’ progress, one per test completed. This particular Command-Line Test

Runner replaces the dot with an “E” or an “F” if the test produces an error or

fails. The last three lines are summary statistics that provide an overview of

what happened. Typically, the exit code is set to the total number of failed/error

tests so that a non-zero exit code can be interpreted easily as a build failure when

run from an automated build tool.

Variation: File System Test Runner

Some Command-Line Test Runners provide the option of searching a speciﬁ ed

directory for all ﬁ les that are tests and running them all at once. This automated

Testcase Class Discovery (see Test Discovery) avoids the need to build the Suite

of Suites in code (Test Enumeration) and helps avoid Lost Tests (see Production

Bugs on page 268).

In addition, some external tools will search the ﬁ le system for ﬁ les matching

speciﬁ c patterns and then invoke an arbitrary command against the matched

ﬁ les. These ﬁ les can be passed to the Test Runner from a build tool.

Variation: Test Tree Explorer

Members of the xUnit family that turn each Test Method into a Testcase Object

can manipulate the tests easily. Many of them provide a graphical representation

of the Suite of Suites and allow the user to select an entire Test Suite Object or

a single Testcase Object to run. This eliminates the need to create a Single Test

Suite (see Named Test Suite on page 592) class to run a single test.

Here is the Test Tree Explorer of JUnit plug-in for Eclipse shown “popped

out” over other Eclipse views:

Test Runner

381

The left pane of the IDE is the JUnit view within Eclipse. The progress bar ap-

pears at the top of the view, the upper pane is the Test Tree Explorer, and the

lower pane is the traceback for the currently selected test failure. Note that some

Test Suite Objects in the Test Tree Explorer are “open,” revealing their contents;

others are closed down. The colored annotation next to each Testcase Object

shows its status; the annotations for each Test Suite Object indicate whether any

contained Testcase Objects failed or produced an error. The Test Suite Object

called “Test for com.clrstream.ex8.test” is a Suite of Suites for the package “com.

clrstream.ex8.test

”; “Test for allJUnitTests” is the topmost Suite of Suites for run-

ning all the tests.

Test Runner

Testcase Object

How do we run the tests?

We create a Command object for each test and call the run method when we

wish to execute it.

The Test Runner (page 377) needs a way to ﬁ nd and invoke the appropriate Test

Methods (page 348) and to present the results to the user. Many Graphical Test

Runners (see Test Runner) let the user drill down into the tree of tests and pick

individual tests to run. This capability requires that the Test Runner be able to

inspect and manipulate the tests at runtime.

How It Works

We instantiate a Command [GOF] object to represent each Test Method that

should execute. We use the Testcase Class (page 373) as a Test Suite Factory

to create a Test Suite Object (page 387) to hold all the Testcase Objects for a

particular Testcase Class. We can use either Test Discovery (page 393) or Test

Enumeration to create the Testcase Objects.

Testcase

Object

testMethod_n

Testcase

Object

testMethod_1

Testcase

Class

testMethod_1

testMethod_n

Test

Suite

Object

Exercise

Create

Exercise

Create

Fixture

SUT

Run

Suite

Test Runner

Testcase

Object

testMethod_n

Testcase

Object

testMethod_1

Testcase

Class

testMethod_1

testMethod_n

Test

Suite

Object

Exercise

Create

Exercise

Create

Fixture

SUT

Run

Suite

Test Runner

Testcase

Object

Chapter 19 xUnit Basics Patterns

382

Why We Do This

Treating tests as ﬁ rst-class objects opens up many new possibilities that are not

available to us if we treat the tests as simple procedures. It is a lot easier for the

Test Runner of the Test Automation Framework (page 298) to manipulate tests

when they are objects. We can hold them in collections (Test Suite Objects),

iterate over them, invoke them, and so on.

Most members of the xUnit family create a separate Testcase Object for

each test to isolate the tests from one another as prescribed by Independent

Test (see page 42). Unfortunately, there is always an exception (see the sidebar

“There’s Always an Exception” on page 384), and users of the affected Test

Automation Frameworks need to be a bit more cautious.

Implementation Notes

Each Testcase Object implements a standard test interface so that the Test

Runner does not need to know the speciﬁ c interface for each test. This scheme

allows each Testcase Object to act as a Command object [GOF]. This allows

us to build collections of these Testcase Objects, which we can then iterate

across to do counting, running, displaying, and other operations.

In most programming languages, we need to create a class to deﬁ ne the

behavior of the Testcase Objects. We could create a separate Testcase Class

for each Testcase Object. It is more convenient to host many Test Methods

on a single Testcase Class, however, as this strategy results in fewer classes to

manage and facilitates reuse of Test Utility Methods (page 599). This approach

requires that each Testcase Object of the Testcase Class have a way to deter-

mine which Test Method it should invoke. Pluggable Behavior [SBPP] is the

most common way to do this. The constructor of the Testcase Class takes the

name of the method to be invoked as a parameter and stores this name in an

instance variable. When the Test Runner invokes the run method on the Test-

case Object, it uses reﬂ ection to ﬁ nd and invoke the method whose name is in

the instance variable.

Testcase

Object

Testcase Object

383

384

Chapter 19 xUnit Basics Patterns

There’s Always an Exception

Whether we are learning to conjugate verbs in a new language or looking

for patterns in how software is built, there’s always an exception!

One of the most notable exceptions in the xUnit family relates to

the use of a Testcase Object (page 382) to represent each Test Meth-

od (page 348) at runtime. This key design feature of xUnit offers a

way to achieve an Independent Test (see page 42). The only members

of the xUnit family that don’t follow this scheme are TestNG and

NUnit (version 2.x). For the reasons described below, the builders

of NUnit 2.0 chose to stray from the well-worn path of one Testcase

Object per Test Method and create only a single instance of the Test-

case Class (page 373). This instance, which they call the test ﬁ xture, is

then reused for each Test Method. One of the authors of NUnit 2.0,

James Newkirk, writes:

I think one of the biggest screw-ups that was made when we wrote

NUnit V2.0 was to not create a new instance of the test ﬁ xture class

for each contained test method. I say “we” but I think this one was

my fault. I did not quite understand the reasoning in JUnit for cre-

ating a new instance of the test ﬁ xture for each test method. I look

back now and see that reusing the instance for each test method

allows someone to store a member variable from one test and use

it in another. This can introduce execution-order dependencies,

which for this type of testing is an anti-pattern. It is much better to

fully isolate each test method from the others. This requires that a

new object be created for each test method.

Unfortunately, this has some very interesting—and undesirable—

consequences when one is familiar with the “JUnit New Instance Behav-

ior” of a separate Testcase Object per method. Because the object is reused,

any objects it refers to via an instance variable are available to all subse-

quent tests. This results in an implicit Shared Fixture (page 317) along

with all the forms of Erratic Tests (page 228) that go with it. James goes

on to say:

Since it would be difﬁ cult to change the way that NUnit works now,

and too many people would complain, I now make all of the mem-

ber variables in test ﬁ xture classes static. It’s almost like truth in

advertising. The result is that there is only one instance of this

variable, no matter how many test ﬁ xture objects are created. If

the variable is static, then someone who may not be familiar with

Testcase

Object

how NUnit executes would not assume that a new one is created

before each test is executed. This is the closest I can get to how

JUnit works without changing the way that NUnit executes test

methods.

Martin Fowler felt this exception was important enough that he wrote

an article about why JUnit’s approach is correct. See http://martinfowler.

com/bliki/JunitNewInstance.html.

Example: Testcase Object

The main evidence of the existence of Testcase Objects appears in the Test Tree

Explorer (see Test Runner) when we “drill down” into the Test Suite Object to

expose the Testcase Objects it contains. Let’s look at an example from the JUnit

Graphical Test Runner that is built into Eclipse. Here’s the list of objects created

from the sample code from the write-up of Testcase Class:

TestSuite("...ﬂightstate.featuretests.AllTests")

TestSuite("...ﬂightstate.featuretests.TestApproveFlight")

TestApproveFlight("testScheduledState_shouldThrowIn..ReEx")

TestApproveFlight("testUnsheduled_shouldEndUpInAwai..oval")

TestApproveFlight("testAwaitingApproval_shouldThrow..stEx")

TestApproveFlight("testWithNullArgument_shouldThrow..ntEx")

TestApproveFlight("testWithInvalidApprover_shouldTh..ntEx")

TestSuite("...ﬂightstate.featuretests.TestDescheduleFlight")

TestDescheduleFlight("testScheduled_shouldEndUpInSc..tate")

TestDescheduleFlight("testUnscheduled_shouldThrowIn..stEx")

TestDescheduleFlight("testAwaitingApproval_shouldTh..stEx")

TestSuite("...ﬂightstate.featuretests.TestRequestApproval")

TestRequestApproval("testScheduledState_shouldThrow..stEx")

TestRequestApproval("testUnsheduledState_shouldEndU..oval")

TestRequestApproval("testAwaitingApprovalState_shou..stEx")

TestSuite("...ﬂightstate.featuretests.TestScheduleFlight")

TestScheduleFlight("testUnscheduled_shouldEndUpInSc..uled")

TestScheduleFlight("testScheduledState_shouldThrowI..stEx")

TestScheduleFlight("testAwaitingApproval_shouldThro..stEx")

The name outside the parentheses is the name of the class; the string inside the

parentheses is the name of the object created from that class. By convention, the

name of the Test Method

to be run is used as the name of the Testcase Object,

and the name of a Test Suite Object is whatever string was passed to the Test

Suite Object constructor. In this example we’ve used the full package and class-

name of the Testcase Class.

I replaced part of the name with “..” to keep each line within the page width limit.

Testcase

Object

Testcase Object

385

This is what this scheme might look like when viewed in a Test Tree Explorer:

Testcase

Object

386

Chapter 19 xUnit Basics Patterns

Test Suite Object

How do we run the tests when we have many tests to run?

We deﬁ ne a collection class that implements the standard test interface and

use it to run a set of related Testcase Objects.

Given that we have created Test Methods (page 348) containing our test logic

and placed them on a Testcase Class (page 373) so we can construct a Testcase

Object (page 382) for each test, it would be nice to be able to run these tests as

a single user operation.

How It Works

We deﬁ ne a Composite [GOF] Testcase Object called a Test Suite Object to hold

the collection of individual Testcase Objects to execute. When we want to run

all tests in the test suite at once, the Test Runner (page 377) asks the Test Suite

Object to run all its tests.

Why We Do This

Treating test suites as ﬁ rst-class objects makes it easier for the Test Runner of

the Test Automation Framework (page 298) to manipulate tests in the test suite.

With or without a Test Suite Object, the Test Runner would have to hold some

kind of collection of Testcase Objects (so that we could iterate over them, count

them, and so on). When we make the collection “smart,” it becomes a simple

matter to add other uses such as the Suite of Suites.

Testcase

Class

Run

Testcase

Object

testMethod_n

Testcase

Object

testMethod_1

testMethod_n

Test

Suite

Object

Exercise

Create

Exercise

Create

Fixture

SUT

Suite

Test Runner

Run

Testcase

Class

Run

Testcase

Object

testMethod_n

Testcase

Object

testMethod_1

testMethod_n

Test

Suite

Object

Exercise

Create

Exercise

Create

Fixture

SUT

Suite

Test Runner

Run

Test Suite

Object

Test Suite Object

387