Meszaros G. xUnit Test Patterns Refactoring Test Code
Подождите немного. Документ загружается.
Test Method
Where do we put our test code?
We encode each test as a single Test Method on some class.
Fully Automated Tests (see page 26) consist of test logic. That logic has to live
somewhere before we can compile and execute it.
How It Works
We defi ne each test as a method, procedure, or function that implements the four
phases (see Four-Phase Test on page 358) necessary to realize a Fully Automated
Test. Most notably, the Test Method must include assertions if it is to be a Self-
Checking Test (see page 26).
We organize the test logic following one of the standard Test Method templates
to make the type of test easily recognizable by test readers. In a Simple Success
Test, we have a purely linear fl ow of control from fi xture setup through exercis-
ing the SUT to result verifi cation. In an Expected Exception Test, language-based
structures direct us to error-handling code. If we reach that code, we pass the test;
if we don’t, we fail it. In a Constructor Test, we simply instantiate an object and
make assertions against its attributes.
Testcase
Object
testMethod_n
Testcase
Object
testMethod_1
Testcase
Class
testMethod_1
testMethod_n
Test
Suite
Object
Exercise
Create
Exercise
Create
Create
Fixture
SUT
Run
Test Runner
Suite
Testcase
Object
testMethod_n
Testcase
Object
testMethod_1
Testcase
Class
testMethod_1
testMethod_n
Test
Suite
Object
Exercise
Create
Exercise
Create
Create
Fixture
SUT
Run
Test Runner
Suite
Test Method
Chapter 19 xUnit Basics Patterns
348
Why We Do This
We have to encode the test logic somewhere. In the procedural world, we would
encode each test as a test case procedure located in a fi le or module. In object -
oriented programming languages, the preferred option is to encode them as
methods on a suitable Testcase Class (page 373) and then to turn these Test
Methods into Testcase Objects (page 382) at runtime using either Test Discovery
(page 393) or Test Enumeration (page 399).
We follow the standard test templates to keep our Test Methods as simple
as possible. This greatly increases their utility as system documentation
(see page 23) by making it easier to fi nd the description of the basic behavior of
the SUT. It is a lot easier to recognize which tests describe this basic behavior if
only Expected Exception Tests contain error-handling language constructs such
as try/catch.
Implementation Notes
We still need a way to run all the Test Methods tests on the Testcase Class. One
solution is to defi ne a static method on the Testcase Class that calls each of the
test methods. Of course, we would also have to deal with counting the tests and
determining how many passed and how many failed. Because this functionality
is needed for a test suite anyway, a simple solution is to instantiate a Test Suite
Object (page 387) to hold each Test Method.
1
This approach is easy to imple-
ment if we create an instance of the Testcase Class for each Test Method using
either Test Discovery or Test Enumeration.
In statically typed languages such as Java and C#, we may have to include
a throws clause as part of the Test Method declaration so the compiler won’t
complain about the fact that we are not handling the checked exceptions that
the SUT has declared it may throw. In effect, we tell the compiler that “The Test
Runner (page 377) will deal with the exceptions.”
Of course, different kinds of functionality need different kinds of Test
Methods. Nevertheless, almost all tests can be boiled down to one of three
basic types.
Variation: Simple Success Test
Most software has an obvious success scenario (or “happy path”). A Simple
Success Test verifi es the success scenario in a simple and easily recognized way.
1
See the sidebar “There’s Always an Exception” (page 384) for an explanation of when
this isn’t the case.
Test Method
Test Method
349
We create an instance of the SUT and call the method(s) that we want to test. We
then assert that the expected outcome has occurred. In other words, we follow
the normal steps of a Four-Phase Test. What we don’t do is catch any exceptions
that could happen. Instead, we let the Test Automation Framework (page 298)
catch and report them. Doing otherwise would result in Obscure Tests (page 186)
and would mislead the test reader by making it appear as if exceptions were
expected. See Tests as Documentation for the rationale behind this approach.
Another benefi t of avoiding try/catch-style code is that when errors do occur,
it is a lot easier to track them down because the Test Automation Framework
reports the location where the actual error occurred deep in the SUT rather
than the place in our test where we called an Assertion Method (page 362) such
as fail or assertTrue. These kinds of errors turn out to be much easier to trouble-
shoot than assertion failures.
Variation: Expected Exception Test
Writing software that passes the Simple Success Test is pretty straightforward.
Most of the defects in software appear in the various alternative paths—especially
the ones that relate to error scenarios, because these scenarios are often Untested
Requirements (see Production Bugs on page 268) or Untested Code (see Produc-
tion Bugs). An Expected Exception Test helps us verify that the error scenarios
have been coded correctly. We set up the test fi xture and exercise the SUT in
each way that should result in an error. We ensure that the expected error has
occurred by using whatever language construct we have available to catch the
error. If the error is raised, fl ow will pass to the error-handling block. This diver-
sion may be enough to let the test pass, but if the type or message contents of the
exception or error is important (such as when the error message will be shown
to a user), we can use an Equality Assertion (see Assertion Method) to verify it.
If the error is not raised, we call fail to report that the SUT failed to raise an
error as expected.
We should write an Expected Exception Test for each kind of exception that
the SUT is expected to raise. It may raise the error because the client (i.e., our
test) has asked it to do something invalid, or it may translate or pass through an
error raised by some other component it uses. We should not write an Expected
Exception Test for exceptions that the SUT might raise but that we cannot force
to occur on cue, because these kinds of errors should show up as test failures
in the Simple Success Tests. If we want to verify that these kinds of errors are
handled properly, we must fi nd a way to force them to occur. The most common
way to do so is to use a Test Stub (page 529) to control the indirect input of the
SUT and raise the appropriate errors in the Test Stub.
Test Method
Chapter 19 xUnit Basics Patterns
350
Exception tests are very interesting to write about because of the different
ways the xUnit frameworks express them. JUnit 3.x provides a special Expected-
Exception
class to inherit from. This class forces us to create a Testcase Class
for each Test Method (page 348), however, so it really doesn’t save any effort
over coding a try/catch block and does result in a large number of very small
Testcase Classes. Later versions of JUnit and NUnit (for .NET) provide a special
ExpectedException method attribute (called an annotation in Java) to tell the Test
Automation Framework to fail the test if that exception isn’t raised. This method
attribute allows us to include message text if we want to specify exactly which
text to expect in addition to the type of the exception.
Languages that support blocks, such as Smalltalk and Ruby, can provide
special assertions to which we pass the block of code to be executed as well
as the expected exception/error object. The Assertion Method implements
the error-handling logic required to determine whether the error has, in fact,
occurred. This makes our Test Methods much simpler, even though we may
need to examine the names of the assertions more closely to see which type of
test we have.
Variation: Constructor Test
We would have a lot of Test Code Duplication (page 213) if every test we wrote
had to verify that the objects it creates in its fi xture setup phase are correctly
instantiated. We avoid this step by testing the constructor(s) separately from
other Test Methods whenever the constructor contains anything more com-
plex than a simple fi eld assignment from the constructor parameters. These
Constructor Tests provide better Defect Localization (see page 22) than includ-
ing constructor logic verifi cation in other tests. We may need to write one or
more tests for each constructor signature. Most Constructor Tests will follow a
Simple Success Test template; however, we can use an Expected Exception Test
to verify that the constructor correctly reports invalid arguments by raising
an exception.
We should verify each attribute of the object or data structure regardless of
whether we expect it to be initialized. For attributes that should be initialized,
we can use an Equality Assertion to specify the correct value. For attributes that
should not be initialized, we can use a Stated Outcome Assertion (see Assertion
Method) appropriate to the type of the attribute [e.g., assertNull(anObjectReference)
for object variables or pointers]. Note that if we are organizing our tests with
one Testcase Class per Fixture (page 631), we can put each assertion into a sepa-
rate Test Method to give optimal Defect Localization.
Test Method
Test Method
351
Variation: Dependency Initialization Test
When we have an object with a substitutable dependency, we need to make sure
that the attribute that holds the reference to the depended-on component (DOC)
is initialized to the real DOC when the software is run in production. A Depen-
dency Initialization Test is a Constructor Test that asserts that this attribute is
initialized correctly. It is often done in a different Test Method from the normal
Constructor Tests to improve its visibility.
Example: Simple Success Test
The following example illustrates a test where the novice test automater has
included code to catch exceptions that he or she knows might occur (or that the
test automater might have encountered while debugging the code).
public void testFlightMileage_asKm() throws Exception {
// set up fixture
Flight newFlight = new Flight(validFlightNumber);
try {
// exercise SUT
newFlight.setMileage(1122);
// verify results
int actualKilometres = newFlight.getMileageAsKm();
int expectedKilometres = 1810;
// verify results
assertEquals( expectedKilometres, actualKilometres);
} catch (InvalidArgumentException e) {
fail(e.getMessage());
} catch (ArrayStoreException e) {
fail(e.getMessage());
}
}
The majority of the code is unnecessary and just obscures the intent of the test.
Luckily for us, all of this exception handling can be avoided. xUnit has built-in
support for catching unexpected exceptions. We can rip out all the exception-
handling code and let the Test Automation Framework catch any unexpected
exception that might be thrown. Unexpected exceptions are counted as test
errors because the test terminates in a way we didn’t anticipate. This is useful
information and is not considered to be any more severe than a test failure.
public void testFlightMileage_asKm() throws Exception {
// set up fixture
Flight newFlight = new Flight(validFlightNumber);
newFlight.setMileage(1122);
// exercise mileage translator
int actualKilometres = newFlight.getMileageAsKm();
Test Method
Chapter 19 xUnit Basics Patterns
352
353
// verify results
int expectedKilometres = 1810;
assertEquals( expectedKilometres, actualKilometres);
}
This example is in Java (a statically typed language), so we had to declare that
the SUT may throw an exception as part of the Test Method signature.
Example: Expected Exception Test Using try/catch
The following example is a partially complete test to verify an exception case.
The novice test automater has set up the right test condition to cause the SUT
to raise an error.
public void testSetMileage_invalidInput() throws Exception {
// set up fixture
Flight newFlight = new Flight(validFlightNumber);
// exercise SUT
newFlight.setMileage(-1122); // invalid
// how do we verify an exception was thrown?
}
Because the Test Automation Framework will catch the exception and fail the test,
the Test Runner will not exhibit the green bar even though the SUT’s behavior
is correct. We can introduce an error-handling block around the exercise phase
of the test and use it to invert the pass/fail criteria (pass when the exception is
thrown; fail when it is not). Here’s how to verify that the SUT fails as expected in
JUnit 3.x:
public void testSetMileage_invalidInput() throws Exception {
// set up fixture
Flight newFlight = new Flight(validFlightNumber);
try {
// exercise SUT
newFlight.setMileage(-1122);
fail("Should have thrown InvalidInputException");
} catch( InvalidArgumentException e) {
// verify results
assertEquals( "Flight mileage must be positive",
e.getMessage());
}
}
This style of try/catch can be used only in languages that allow us to specify exactly
which exception to catch. It won’t work if we want to catch a generic exception or
the same exception that the Assertion Method fail throws, because these excep-
tions will send us into the catch clause. In these cases we need to use the same style
of Expected Exception Test as used in tests of Custom Assertions (page 474).
Test Method
Test Method
public void testSetMileage_invalidInput2() throws Exception {
// set up fixture
Flight newFlight = new Flight(validFlightNumber);
try {
// exercise SUT
newFlight.setMileage(-1122);
// cannot fail() here if SUT throws same kind of exception
} catch( AssertionFailedError e) {
// verify results
assertEquals( "Flight mileage must be positive",
e.getMessage());
return;
}
fail("Should have thrown InvalidInputException");
}
Example: Expected Exception Test Using Method Attributes
NUnit provides a method attribute that lets us write an Expected Exception Test
without forcing us to code a try/catch block explicitly.
[Test]
[ExpectedException(typeof( InvalidArgumentException),
"Flight mileage must be > zero")]
public void testSetMileage_invalidInput_AttributeWithMessage()
{
// set up fixture
Flight newFlight = new Flight(validFlightNumber);
// exercise SUT
newFlight.setMileage(-1122);
}
This approach does make the test much more compact but doesn’t provide a
way to specify anything but the type of the exception or the message it contains.
If we want to make any assertions on other contents of the exception (to avoid
Sensitive Equality; see Fragile Test on page 239), we’ll need to use try/catch.
Example: Expected Exception Test Using Block Closure
Smalltalk’s SUnit provides another mechanism to achieve the same thing:
testSetMileageWithInvalidInput
self
should: [Flight new mileage: -1122]
raise: RuntimeError new 'Should have raised error'
Because Smalltalk supports block closures, we pass the block of code to be
executed to the method should:raise: along with the expected Exception object.
Ruby’s Test::Unit uses the same approach:
Test Method
Chapter 19 xUnit Basics Patterns
354
def testSetMileage_invalidInput
flight = Flight.new();
assert_raises( RuntimeError, "Should have raised error") do
flight.setMileage(-1122)
end
end
The code between the do/end pair is a closure that is executed by the assert_raises
method. If it doesn’t raise an instance of the fi rst argument (the class RuntimeError),
the test fails and presents the error message supplied.
Example: Constructor Test
In this example, we need to build a fl ight to test the conversion of the fl ight
distance from miles to kilometers. First, we’ll make sure the fl ight is constructed
properly.
public void testFlightMileage_asKm2() throws Exception {
// set up fixture
// exercise constructor
Flight newFlight = new Flight(validFlightNumber);
// verify constructed object
assertEquals(validFlightNumber, newFlight.number);
assertEquals("", newFlight.airlineCode);
assertNull(newFlight.airline);
// set up mileage
newFlight.setMileage(1122);
// exercise mileage translator
int actualKilometres = newFlight.getMileageAsKm();
// verify results
int expectedKilometres = 1810;
assertEquals( expectedKilometres, actualKilometres);
// now try it with a canceled flight
newFlight.cancel();
try {
newFlight.getMileageAsKm();
fail("Expected exception");
} catch (InvalidRequestException e) {
assertEquals( "Cannot get cancelled flight mileage",
e.getMessage());
}
}
This test is not a Single-Condition Test (see page 45) because it examines both
object construction and distance conversion behavior. If object construction
fails, we won’t know which issue was the cause of the failure until we start
debugging the test.
Test Method
Test Method
355
It would be better to separate this Eager Test (see Assertion Roulette on page
224) into two tests, each of which is a Single-Condition Test. This is most easily
done by cloning the Test Method, renaming each copy to refl ect what it would
do if it were a Single-Condition Test, and then removing any code that doesn’t
satisfy that goal.
Here’s an example of a simple Constructor Test:
public void testFlightConstructor_OK() throws Exception {
// set up fixture
// exercise SUT
Flight newFlight = new Flight(validFlightNumber);
// verify results
assertEquals( validFlightNumber, newFlight.number );
assertEquals( "", newFlight.airlineCode );
assertNull( newFlight.airline );
}
While we are at it, we might as well specify what should occur if an invalid
argument is passed to the constructor by using the Expected Exception Test
template for our Constructor Test:
public void testFlightConstructor_badInput() {
// set up fixture
BigDecimal invalidFlightNumber = new BigDecimal(-1023);
// exercise SUT
try {
Flight newFlight = new Flight(invalidFlightNumber);
fail("Didn't catch negative flight number!");
} catch (InvalidArgumentException e) {
// verify results
assertEquals( "Flight numbers must be positive",
e.getMessage());
}
}
Now that we know that our constructor logic is well tested, we are ready to
write our Simple Success Test for our mileage translation functionality. Note
how much simpler it has become because we can focus on verifying the business
logic:
public void testFlightMileage_asKm() throws Exception {
// set up fixture
Flight newFlight = new Flight(validFlightNumber);
newFlight.setMileage(1122);
// exercise mileage translator
int actualKilometres = newFlight.getMileageAsKm();
// verify results
int expectedKilometres = 1810;
assertEquals( expectedKilometres, actualKilometres);
}
Test Method
Chapter 19 xUnit Basics Patterns
356
So what happens if the constructor logic is defective? This test will likely fail
because its output depends on the value passed to the constructor. The con-
structor test will also fail. That failure will tell us to look at the constructor
logic fi rst. Once that problem is fi xed, this test will likely pass. If it doesn’t, then
we can focus on fi xing the getMileageAsKm method logic. This is a good example
of Defect Localization.
Test Method
Test Method
357