Meszaros G. xUnit Test Patterns Refactoring Test Code

Подождите немного. Документ загружается.

Variation: Testcase Class Suite

To run all the Test Methods in a single Testcase Class, we simply build a Test

Suite Object for the Testcase Class and add one Testcase Object for each Test

Method. This allows us to run all the Test Methods in the Testcase Class simply

by passing the name of the Testcase Class to the Test Runner.

Variation: Suite of Suites

We can build up larger Named Test Suites (page 592) by organizing smaller

test suites into a tree structure. The Composite pattern makes this organization

invisible to the Test Runner, allowing it to treat a Suite of Suites exactly the

same way it treats a simple Testcase Class Suite or a single Testcase Object.

Implementation Notes

As a Composite object, each Test Suite Object implements the same interface as

a simple Testcase Object. Thus neither the Test Runner nor the Test Suite Object

needs to be aware of whether it is holding a reference to a single test or an entire

suite. This makes it easier to implement any operations that involve iterating

across all the tests such as counting, running, and displaying.

Before we can do anything with our Test Suite Object, we must construct it.

We can choose from several options to do so:

• Test Discovery (page 393): We can let the Test Automation Framework

discover our Testcase Classes and Test Methods for us.

• Test Enumeration (page 399): We can write code that enumerates

which Test Methods we want to include in a Test Suite Object. This

usually involves creating a Test Suite Factory (see Test Enumeration).

• Test Selection (page 403): We can specify which subset of the Testcase

Objects we want to include from an existing Test Suite Object.

Variation: Test Suite Procedure

Sometimes we have to write code in programming or scripting languages that do

not support objects. Given that we have written a number of Test Methods, we

need to give the Test Runner some way to ﬁ nd the tests. A Test Suite Procedure

allows us to enumerate all the tests we want to run by invoking each test in turn.

The calls to each test are hard-coded within the body of the Test Suite Object.

Of course, a Test Suite Procedure may call several other Test Suite Procedures

to realize a Suite of Suites.

Test Suite

Object

Chapter 19 xUnit Basics Patterns

388

The major disadvantage of this approach is that it forces us into Test

Enumeration, which increases both the effort required to write tests and the

likelihood of Lost Tests (see Production Bugs on page 268). Because we do not

treat our code as “data,” we lose the ability to manipulate the code at runtime.

As a consequence, it is more difﬁ cult to build a Graphical Test Runner (see Test

Runner) with a hierarchy (tree) view of our Suite of Suites.

Example: Test Suite Object

Most members of the xUnit family implement Test Discovery, so there isn’t much

of an example of Test Suite Object to see. The main evidence of the existence of

Test Suite Objects appears in the Test Tree Explorer (see Test Runner) when we

“drill down” into the Test Suite Object to expose the Testcase Objects it contains.

Here’s an example from the JUnit Graphical Test Runner built into Eclipse:

Example: Suite of Suites Built Using Test Enumeration

Here is an example of using Test Enumeration to construct a Suite of Suites:

public class AllTests {

public static Test suite() {

Test Suite

Object

Test Suite Object

389

TestSuite suite = new TestSuite("Test for allJunitTests");

suite.addTestSuite(

com.clrstream.camug.example.test.InvoiceTest.class);

suite.addTest(com.clrstream.ex7.test.AllTests.suite());

suite.addTest(com.clrstream.ex8.test.AllTests.suite());

suite.addTestSuite(com.xunitpatterns.guardassertion.Example.class);

return suite;

}

The ﬁ rst and last lines add the Test Suite Objects created from a single Testcase

Class. Each of the middle two lines calls the Test Suite Factory for another Suite

of Suites. The Test Suite Object we return is likely at least three levels deep:

1. The Test Suite Object we instantiated and populated before returning

2. The AllTests Test Suite Objects returned by the two calls to factory

methods

3. The Test Suite Objects for each of the Testcase Classes aggregated into

those Test Suite Objects

This is illustrated in the following tree of objects:

TestSuite("Test for allJunitTests");

TestSuite("com.clrstream.camug.example.test.InvoiceTest")

TestCase("testInvoice_addLineItem")

...

TestCase("testRemoveLineItemsForProduct_oneOfTwo")

TestSuite("com.clrstream.ex7.test.AllTests")

TestSuite("com.clrstream.ex7.test.TimeDisplayTest")

TestCase("testDisplayCurrentTime_AtMidnight")

TestCase("testDisplayCurrentTime_AtOneMinAfterMidnight")

TestCase("testDisplayCurrentTime_AtOneMinuteBeforeNoon")

TestCase("testDisplayCurrentTime_AtNoon")

...

TestSuite("com.clrstream.ex7.test.TimeDisplaySolutionTest")

TestCase("testDisplayCurrentTime_AtMidnight")

TestCase("testDisplayCurrentTime_AtOneMinAfterMidnight")

TestCase("testDisplayCurrentTime_AtOneMinuteBeforeNoon")

TestCase("testDisplayCurrentTime_AtNoon")

...

TestSuite("com.clrstream.ex8.test.AllTests")

TestSuite("com.clrstream.ex8.FlightMgntFacadeTest")

TestCase("testAddFlight")

TestCase("testAddFlightLogging")

TestCase("testRemoveFlight")

TestCase("testRemoveFlightLogging")

...

TestSuite("com.xunitpatterns.guardassertion.Example")

TestCase("testWithConditionals")

TestCase("testWithoutConditionals")

...

Test Suite

Object

390

Chapter 19 xUnit Basics Patterns

Note that this class doesn’t subclass any other class. It does need to import

TestSuite and the classes it is using as Test Suite Factories.

Example: Test Suite Procedure

In the early days of agile software development, before any agile project manage-

ment tools were available, I built a set of Excel spreadsheets for managing tasks

and user stories. To make life simpler, I automated frequently performed tasks

such as sorting all stories by release and iteration, sorting tasks by iteration and

status, and so on. Eventually, I got bold enough to write a macro (a program,

really) that would sum up the estimated and actual effort of all tasks for each

story. At this point, the code was becoming somewhat complex and was more

challenging to maintain. In particular, if one of the named ranges used by the

sorting macros was accidentally deleted, the macro would produce an error.

Unfortunately, there was no xUnit framework for VBA at the time, so all of

this work was done without Tests as Safety Net (see page 24). Here is the main

program of the reporting macro. All output was written to a new sheet in the

workbook.

'Main Macro

Sub summarizeActivities()

Call VerifyVersionCompatability

Call initialize

Call SortByActivity

For row = ﬁrstTaskDataRow To lastTaskDataRow

If numberOfNumberlessTasks < MaxNumberlessTasks Then

thisActivity =

ActiveSheet.Cells(row, TaskActivityColumn).Value

If thisActivity <> currentActivity Then

Call ﬁnalizeCurrentActivityTotals

currentActivity = thisActivity

Call initializeCurrentActivityTotals

End If

Call accumulateActivityTotals(row)

Else

lastTaskDataRow = row ' end the For loop right away

End If

Next row

Call cleanUp

End Sub

Test Suite

Object

Test Suite Object

391

Without any tests or Test Automation Framework, I had to do what I could to

introduce some kind of regression testing. In this case, it was enough of a challenge

(and a win) just to be able to exercise all the macros. If they ran to completion, it

was a much better indication that I hadn’t broken anything major than not running

the macros at all. Because VBA is based on Visual Basic 5, it has no classes. Thus

we have no Testcase Class and no runtime Testcase Objects. The following is an

example of the various Test Suite Procedures and the Test Methods my tests called:

Sub TestAll()

Call TestAllStoryMacros

Call TestAllTaskMacros

Call TestReportingMacros

Call TestToolbarMenus 'All The Same

End Sub

Sub TestAllStoryMacros()

Call TestActivitySorting

Call TestStoryHiding

Call ReportSuccess("All Story Macros")

End Sub

Sub TestActivitySorting()

Call SortStoriesbyAreaAndNumber

Call SortActivitiesByIteration

Call SortActivitiesByIterationAndOrder

Call SortActivitiesByNumber

Call SortActivitiesByPercentDone

End Sub

Sub TestReportingMacros()

Call summarizeActivities

End Sub

The ﬁ rst Test Suite Procedure is a Suite of Suites; the second Test Suite Procedure

is the equivalent of a single Test Suite Object. The third Sub is the Test Method for

exercising all of the sorting macros. The last Sub exercises the summarizeActivities

macro using a Prebuilt Fixture (page 429).

For those who might be wondering what happened to the verify outcome phase of the

test, there isn’t one in this test. It is neither a Self-Checking Test nor a Single-Condition

Test. Shame on me!

Test Suite

Object

Chapter 19 xUnit Basics Patterns

392

Test Discovery

How does the Test Runner know which tests to run?

The Test Automation Framework discovers all tests that belong to the test

suite automatically.

Given that we have written a number of Test Methods (page 348) on one or more

Testcase Classes (page 373), we need to give the Test Runner (page 377) some

way to ﬁ nd the tests. Test Discovery eliminates most of the hassles associated with

Test Enumeration (page 399).

How It Works

The Test Automation Framework (page 298) uses runtime reﬂ ection (or com-

pile-time knowledge) to discover all Test Methods that belong to the test suite

and/or all Test Suite Objects (page 387) that belong to a Suite of Suites (see

Test Suite Object). It then builds up the Test Suite Objects containing the

corresponding Testcase Objects (page 382) and other Test Suite Objects in

preparation for running all the tests.

When to Use It

We should use Test Discovery whenever our Test Automation Framework

supports it. This pattern reduces the effort required to automate tests and greatly

reduces the possibility of Lost Tests (see Production Bugs on page 268). The

Testcase

Object

testMethod_n

Testcase

Object

testMethod_1

Testcase

Class

testMethod_1

testMethod_n

Test

Suite

Object

Exercise

Create

Fixture

SUT

Run

Test Runner

Test

Discovery

Mechanism

Create

Testcase

Object

testMethod_n

Testcase

Object

testMethod_1

Testcase

Class

testMethod_1

testMethod_n

Test

Suite

Object

Exercise

Create

Fixture

SUT

Run

Test Runner

Test

Discovery

Mechanism

Create

Test

Discovery

Test Discovery

393

only times to consider using Test Enumeration are (1) when our framework

does not support Test Discovery and (2) when we wish to deﬁ ne a Named Test

Suite (page 592) that consists of a subset of tests

chosen from other test suites

and the Test Automation Framework does not support Test Selection (page 403).

It is not uncommon to combine Test Suite Enumeration (see Test Enumeration)

with Test Method Discovery; the reverse is less common.

Implementation Notes

Building the Suite of Suites to be executed by the Test Runner involves two steps.

First, we must ﬁ nd all Test Methods to be included in each Test Suite Object.

Second, we must ﬁ nd all Test Suite Objects to be included in the test run, albeit

not necessarily in this order. Each of these steps may be done manually via Test

Method Enumeration (see Test Enumeration) and Test Suite Enumeration or

automatically via Test Method Discovery and Testcase Class Discovery.

Variation: Testcase Class Discovery

Testcase Class Discovery is the process by which the Test Automation Frame-

work discovers the Testcase Classes on which it should do Test Method Dis-

covery. One solution involves tagging each Testcase Class by subclassing a

Testcase Superclass (page 638) or implementing a Marker Interface [PJV1].

Another alternative, used in the .NET languages and newer versions of JUnit,

is to use a class attribute (e.g., "[Test Fixture]") or annotation (e.g., "@Testcase")

to identify each Testcase Class. Yet another solution is to put all Testcase

Classes into a common directory and point the Test Runner or some other

program at this directory. A fourth solution is to follow a Testcase Class

naming convention and use an external program to ﬁ nd all ﬁ les matching

this naming pattern. Whichever way we choose to perform this task, once

a Testcase Class has been discovered we can proceed to either Test Method

Discovery or Test Method Enumeration.

Variation: Test Method Discovery

Test Method Discovery involves providing a way for the Test Automation Frame-

work to discover the Test Methods in our Testcase Classes. There are two basic

ways to indicate that a method of a Testcase Class is a Test Method. The more

traditional approach is to use a Test Method naming convention such as “starts

with ‘test’.” The Test Automation Framework then iterates over all methods of

the Testcase Class, selects those that start with the string “test” (e.g., testCounters),

A Smoke Test [SCM] suite is a good example.

Test

Discovery

Chapter 19 xUnit Basics Patterns

394

and calls the one-argument constructor to create the Testcase Object for that Test

Method. The other alternative, which is used in the .NET languages and newer

versions of JUnit, is to use a method attribute (e.g., “[Test]”) or annotation (e.g.,

“@Test”) to identify each Test Method.

Motivating Example

The following example illustrates the kind of code that would be required for

each Test Method to do Test Method Enumeration if we did not have Test

Discovery available:

public:

static CppUnit::Test *suite()

{

CppUnit::TestSuite *suite =

new CppUnit::TestSuite( "ComplexNumberTest" );

suite>addTest(

new CppUnit::TestCaller<ComplexNumberTest>(

"testEquality",

&ComplexNumberTest::testEquality ) );

suite>addTest(

new CppUnit::TestCaller<ComplexNumberTest>(

"testAddition",

&ComplexNumberTest::testAddition ) );

return suite;

}

This example is from the tutorial for an earlier version of CppUnit. Newer

versions no longer require this approach.

Refactoring Notes

Luckily for the users of existing xUnit family members, the inventors of xUnit

realized the importance of Test Discovery. Therefore all we have to do is follow

their advice on how to identify our test methods. If the developers of our xUnit

version used a naming convention, we may have to do a Rename Method [Fowler]

refactoring to get xUnit to discover our Test Method. If they implemented method

attributes, we just add the appropriate attribute to our Test Methods.

Example: Test Method Discovery (Using Method Naming

and Compiler Macro)

When the programming language is capable of managing the tests as objects

and invoking the methods but cannot easily ﬁ nd all methods to use as tests, we

Test

Discovery

Test Discovery

395

396

Chapter 19 xUnit Basics Patterns

may need to give it a small push as encouragement to do so. Newer versions

of CppUnit provide a macro that ﬁ nds all Test Methods at compile time and

generates the code to build the test suite as illustrated in the previous example.

The following code snippet triggers the Test Method Discovery:

CPPUNIT_TEST_SUITE_REGISTRATION( FlightManagementFacadeTest );

This macro uses a method naming convention to determine which methods

(“member functions”) it should turn into Testcase Objects by wrapping each

with a TestCaller, much like in the manual example we saw earlier.

Example: Test Method Discovery (Using Method Naming)

The following examples are more notable for the code that is missing than for

the code that is present. Note that there is no code to add the Test Methods to

the Test Suite Object.

In this Java example, the framework automatically runs all test methods that

start with “test” and have no arguments (a total of two):

public class TimeDisplayTest extends TestCase {

public void testDisplayCurrentTime_AtMidnight()

throws Exception {

// Set up SUT

TimeDisplay theTimeDisplay = new TimeDisplay();

// Exercise SUT

String actualTimeString =

theTimeDisplay.getCurrentTimeAsHtmlFragment();

// Verify outcome

String expectedTimeString =

"<span class=\"tinyBoldText\">Midnight</span>";

assertEquals( "Midnight",

expectedTimeString,

actualTimeString);

}

public void testDisplayCurrentTime_AtOneMinuteAfterMidnight()

throws Exception {

// Set up SUT

TimeDisplay actualTimeDisplay = new TimeDisplay();

// Exercise SUT

String actualTimeString =

actualTimeDisplay.getCurrentTimeAsHtmlFragment();

// Verify outcome

String expectedTimeString =

"<span class=\"tinyBoldText\">12:01 AM</span>";

assertEquals( "12:01 AM",

expectedTimeString,

actualTimeString);

}

Test

Discovery

Example: Test Method Discovery (Using Method Attributes)

In this C# example, the tests are labeled with the method attribute [Test]. Both

CsUnit and NUnit use this way of identifying Test Methods.

[Test]

public void testFlightMileage_asKm()

{

// set up ﬁxture

Flight newFlight = new Flight(validFlightNumber);

newFlight.setMileage(1122);

// exercise mileage translator

int actualKilometres = newFlight.getMileageAsKm();

int expectedKilometres = 1810;

// verify results

Assert.AreEqual( expectedKilometres, actualKilometres);

}

[Test]

[ExpectedException(typeof(InvalidArgumentException))]

public void testSetMileage_invalidInput_attribute()

{

// set up ﬁxture

Flight newFlight = new Flight(validFlightNumber);

// exercise SUT

newFlight.setMileage(-1122);

}

Example: Testcase Class Discovery (Using Class Attributes)

Here is an example of using a class attribute to identify a Testcase Class (called

a “Test Fixture” in NUnit) to the Test Runner:

[TestFixture]

public class SampleTestcase

{

}

Example: Testcase Class Discovery (Using Common

Location and Testcase Superclass)

The following Ruby example ﬁ nds all ﬁ les with the .rb extension in the “tests”

directory and requires them from this ﬁ le. This causes Test::Unit to look for all tests

in each ﬁ le because the Testcase Class in each ﬁ le extends Test::Unit::TestCase.

Dir['tests/*.rb'].each do |each|

require each

end

Test

Discovery

Test Discovery

397