JUnit Reloaded

Let's start with the most obvious: JUnit 4 is completely based on annotations. Ah, now I know--and what exactly does that mean? First of all, you no longer need to extend class TestCase. Also, your test-method names do not have to start with the prefix test. All you have to do is to mark your test method with a @Test annotation. Take a look at this example:

import junit.framework.TestCase;
import org.junit.Test;

public class CalculatorTest extends TestCase {

    @Test
    public void testadd() { 
        ....
    } 
} 

Okay, not having to extend TestCase might be helpful in some environments. But what about the assert... methods I used to inherit? They come to you using another Java5 feature: static imports.

import org.junit.Test;
import static org.junit.Assert.*;

public class CalculatorTest {

    @Test
    public void add() { 
        ...
        assertEquals( 4, calculator.add( 1, 3 ) );
    } 
}

Setting Up the Test Environment

So writing a test isn't that different with JUnit 4. Wait, but what happened to setUp() and tearDown()? Here we go: You may now decorate any method with the annotations @Before and @After:

public class CalculatorTest {

    @Before
    public void prepareTestData() { ... }

    @After
    public void cleanupTestData() { ... }
}

You may even have multiple @Before and @After methods, but be aware that nothing is said about the order in which they are executed:

public class CalculatorTest {

    @Before
    public void prepareTestData() { ... }

    @Before
    public void setupMocks() { ... }

    @After
    public void cleanupTestData() { ... }
}

One thing worth noticing is that inherited @Before and @After methods are executed symmetrically. This means if we have a test class ScientificCalculatorTest that extends CalculatorTest, they are executed as follows:

CalculatorTest#Before
ScientificCalculatorTest#Before
ScientificCalculatorTest#After
CalculatorTest#After

One feature a lot of people had been missing is the ability to define a setup/teardown for a whole set of tests. This is useful if you have a very expensive setup that does not need to be run for every test, such as setting up a database connection. This can now be done using the annotations @BeforeClass and @AfterClass:

public class CalculatorTest {

    @BeforeClass
    public static void setupDatabaseConnection() { ... }

    @AfterClass
    public static void teardownDatabaseConnection() { ... }
}

The same rules apply as for @Before and @After [15], meaning you can have multiple @BeforeClass and @AfterClass methods and also inherit from super classes. Be aware that these methods must be static.

Testing for Exceptions

Checking that our code produces correct results is one thing, but what about error handling? When things go wrong, usually an exception is thrown. Testing that your code behaves correctly in an exceptional case is as important (or even more important) than the real functionality. And that's where one of the great benefits of unit tests pays off: you can drive the code into a failure situation in a controlled manner (maybe with the help of some mocks [16]) and check if it throws the expected exception. In the JUnit 3.8.x world, the pattern for testing exceptions was:

public class CalculatorTest {

    public void testDivisionByZero() {
        try {
            new Calculator().divide( 4, 0 );
        } catch (ArithmeticException e) {}
    }
}

With JUnit 4, you declare an expected exception in the @Test annotation:

public class CalculatorTest {

    @Test(expected=ArithmeticException.class)
    public void testDivisionByZero() {
        new Calculator().divide( 4, 0 );
    }
}

Test with Timeout

Unit tests should be short-running so you're willing to run them frequently. But sometimes you have some tests that take their time, particularly if network connectivity is involved. So whenever you have reasonable doubts that your test will finish in time, you should make sure that it will be canceled after a specified amount of time. With JUnit 3.8.x, you had to use additional libraries for that or, even worse, do it on your own by utilizing new Threads. Now this task has been eased; all you need to do is to specify a timeout as a parameter of the @Test annotation:

    @Test(timeout=5000)
    public void testLengthyOperation() {
        ...
    }

If the timeout occurs before the test is finished, you'll get an appropriate failure message:

java.lang.Exception: test timed out after 5000 milliseconds

Omitting Tests

There might be the case that you want some tests to be ignored by the test runner. Maybe the current release of the third-party library you're using has a bug, or the malicious infrastructure prevents your test from being run successfully. Whatever your (more or less) good reason is, in JUnit 3.8.x you had to comment out your tests or exclude them from the test suite. With JUnit 4 you may decorate your method with the @Ignore annotation.

public class CalculatorTest {

    @Ignore("Not running because <fill in a good reason here>")
    @Test
    public void testTheWhatSoEverSpecialFunctionality() {
    }
}

The text handed to the @Ignore annotation will be reported by the test runners. Even if it is optional, you should always provide a comment about why the test is ignored so you won't forget about it. The TestRunner built into Eclipse marks ignored tests visually by rendering it strikethrough. Alas, the comment is not provided (yet).

Test Suites

The good old suite() method also found its annotation-based replacement in JUnit 4. Instead of

public class AllTests extends TestCase {

    public static Test suite() {
        TestSuite suite = new TestSuite();
        suite.addTestSuite(CalculatorTest.class);
        suite.addTestSuite(AnotherTest.class);
        return suite;
    }

you write:

@RunWith(value=Suite.class)
@SuiteClasses(value={CalculatorTest.class, AnotherTest.class})
public class AllTests {
...
}

Alright, what's that? Well, JUnit 4 introduces a hook to specify different test runners. In this case, we told JUnit to RunWith the Suite runner to execute this test. The @SuiteClasses annotation is evaluated by the Suite runner, it specifies the tests that belong to this suite. Boy, I sure liked that old suite-style better. Maybe so, but this new way to define a suite also has an advantage: You can specify @BeforeClass and @AfterClass methods that will be executed before the first of the suite and after the last test, respectively. With this, you can define a suite-wide test setup.

Parameterized Tests

Besides the Suite [17] runner, JUnit 4 comes with another special runner: Parameterized. It allows you to run the same test(s) with different datasets. Let's do this by example: we will write a test for a method that calculates the factorial [18] for a given number n:

@RunWith(value=Parameterized.class)
public class FactorialTest {

    private long expected;
    private int value;

    @Parameters
    public static Collection data() {
        return Arrays.asList( new Object[][] {
                             { 1, 0 },   // expected, value
                             { 1, 1 }, 
                             { 2, 2 },
                             { 24, 4 },
                             { 5040, 7 },
                             });
    }

    public FactorialTest(long expected, int value) {
        this.expected = expected;
        this.value = value;
    }

    @Test
    public void factorial() {
        Calculator calculator = new Calculator();
        assertEquals(expected, calculator.factorial(value));
    }
}

What the Parameterized runner is doing is to run all tests in the FactorialTest (we have just one here ;-) with the data provided by the method decorated with @Parameters. In this case, we have five items in our List. Each item consists of an array that will be used as constructor arguments for the FactorialTest. Our factorial() test then uses this data in the assertEquals(). After all, this means our test is run five times with the following data:

factorial#0:  assertEquals( 1, calculator.factorial( 0 ) );
factorial#1:  assertEquals( 1, calculator.factorial( 1 ) );
factorial#2:  assertEquals( 2, calculator.factorial( 2 ) );
factorial#3:  assertEquals( 24, calculator.factorial( 4 ) );
factorial#4:  assertEquals( 5040, calculator.factorial( 7 ) );

Miscellaneous Debris

The new JUnit classes come within a new package: org.junit. The old test framework is still contained in the package junit.framework for compatibility reasons. Another subtle little change is that instead of junit.framework.AssertionFailedError, the (java.lang.)AssertionError is now thrown.

Something really useful is the new assert for checking the equality of arrays, which first checks if the arrays have an equal length and then compares the array's items using equals().

assertEquals(Object[] expected, Object[] actual)

JUnit 4 no longer supports a UI-based TestRunner; this is left to the IDE developers. But there is still a command-line tool you can use to manually run tests. Just call the class org.junit.runner.JUnitCore and pass the (fully qualified) names of your test classes:

java -cp ... org.junit.runner.JUnitCore CalculatorTest AnotherTest

Hidden Surprise

Finally, here's some practical joking. When I was fooling around with JUnit 4, one of my first tests was for the trivial Calculator.add() method:

public class Calculator {

    public long add(long number1, long number2) {
        return number1 + number2;
    }
}

public class CalculatorTest {

    @Test
    public void add() throws Exception {
        Calculator calculator = new Calculator();
        assertEquals(4, calculator.add(1, 3));
    }
}

Easy going, eh? But when I was running the test, I got:

java.lang.AssertionError: expected:<4> but was:<4>

What?! Why is that? That's autoboxing, my friend. In JUnit 4, there is no more need for special assertEquals() for primitive types; there is just assertEquals(Object expected, Object actual). If you pass two ints, they are autoboxed to Integer. And now our problem enters the catwalk: Calculator.add() returns a long, but by default Java uses int for number. So together with autoboxing we get something like this:

        assertEquals(new Integer(4), new Long(calculator.add(1, 3)));

OK, that's not what I've been expecting, but is that a problem? Yep, have a look at the equals() implementation of Integer:

    public boolean equals(Object obj) {
        if (obj instanceof Integer) {
            return value == ((Integer)obj).intValue();
        }
        return false;
    }

Unfortunately, we don't compare with another Integer but a Long, so equals() always returns false. This could be an issue when you transfer your old tests to JUnit 4. In JUnit3.8.x, this test would have run, since there are assertEquals() for all kinds of primitive types, so in our case the compiler would have chosen assertEquals(long expected, long actual). To get around this, you have to make sure that long is used in this case:

assertEquals( (long)4, calculator.add(1, 3) );

So What?

JUnit 4 brings some fancy new stuff to us: it is annotation-based, test setup has been significantly improved, there are extension hooks for new runners, and it even adds the much-wanted assertEquals() for comparing arrays. Other frameworks like TestNG introduced this kind of stuff long ago! You're right, but JUnit is still the most widely used (Java) testing framework out there, so it makes sense to introduce a more recent technical base. JUnit's advance is that every major development tool comes with out-of-the-box support: no need to install a plugin, no need to pimp-up my build process. And due to its open annotation-based architecture, extensions [19] are already appearing. Enough of the big words, just give it a try.

There is a difference between knowing the path and walking the path
-- Morpheus

Resources

• JUnit.org [20]
• JUnit 4 extensions [19]
• TestNG [21]

Ralf Stuckert [1] is an IT consultant for compeople AG [22], a European IT-Service company located in Frankfurt, Germany.