Confessions of an Agile Coach: Exceptional Exception Testing with JUnit 4.11 Rules and Java 8/JUnit5

Zipporah taking exception to poor 'ol Babbage

You've got something that takes exception to something else and now you want to automate a test to confirm it correctly throws per your specification:

public class ProblemCauser {
    public void throwsException() {
        throw new IllegalArgumentException("Negative");
    }
}

One way is the below. But can you see a problem?

import static org.junit.Assert.*;
import org.junit.Test;
public class RulesAndExceptionTest {

    @Test(expected=IllegalArgumentException.class)
    public void canOnlyCheckTypeButNotMessage()
    {
        ProblemCauser problem = new ProblemCauser();
        problem.throwsException();
    }
}

How do you know the exception message has been set to "Negative?" In the past, that left you with the following strategy:

    @Test
    public void theOldWayToTestExceptionMessageAndType() {
        try {
            ProblemCauser problem = new ProblemCauser();
            problem.throwsException();
            fail("didn't get expected exception");
        } catch (IllegalArgumentException expected) {
            assertEquals("Negative", expected.getMessage());
        }
    }

This does the job. Good work! You can actually fully test drive exception handling with this strategy. The drawback is this is a lot of code to express a simple check. Even worse, many people forget to include the fail() call, which means if an exception doesn't happen, the test would pass. This would be an incorrect test case. A better strategy is to use Junit 4.11's Rules package:

import static org.junit.Assert.*;
import org.junit.Ignore;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;

public class RulesAndExceptionTest {
@Rule
public ExpectedException exceptionPolicy = ExpectedException.none(); // Set the default expectation.

    @Test
    public void theNewWayToTestExceptionAndType() {
        exceptionPolicy.expect(IllegalArgumentException.class);
        exceptionPolicy.expectMessage("Negative");

        ProblemCauser problem = new ProblemCauser();
        problem.throwsException();
    }
}

This is equivalent but uses less code. How does it work?

Exceptional with @Rules

org.junit.Rule is an annotation that can be applied to a field or method. When JUnit executes the test class, it hands control to the Rule and decides how to execute Before, Test, and After, allowing the it to manage whatever it is supposed to manage. In the case of ExpectedException, allowing it to execute the @Test method and handle any exception thrown out to see if it's expected.

You can have multiple Rules in a test class doing various things. Here are other org.junit.rules.*(http://junit.org/junit4/javadoc/latest/org/junit/rules/package-summary.html):

Click image to zoom.

Using JUnit's Rules, you can clean up exception handling code to one to three lines of code (one for the test class field, then one where you expect to get an exception) making it more readable and less error prone than using a try{...fail();}catch(Exception ex) { assertEquals(...);} pattern. More functionality with less code. That's exceptional!

Even MORE Exceptional with Java 8 and JUnit 5

JUnit 5 alpha was released in 2016 and it was designed to use Java 8 language features. assertThrows() tests if the exception type was correct. expectThrows() does the same AND returns the exception type so you can examine it further.

import static org.hamcrest.CoreMatchers.startsWith;
import static org.hamcrest.MatcherAssert.assertThat;
import org.junit.gen5.api.Assertions;
import org.junit.gen5.api.Test;
import org.junit.gen5.junit4.runner.JUnit5;
import org.junit.runner.RunWith;

@RunWith(JUnit5.class)
public class ExceptionTestingTest {
   @Test
   public void throwsExceptionWhenPopped() {
        ProblemCauser problem = new ProblemCauser();
        Throwable exception = Assertions.expectThrows(IllegalAccessException.class, () -> problem.throwsException());
        assertThat(exception.getMessage(), startsWith("Error:"));
   }
}

Java 8 and JUnit5 make exception testing even more exceptional. Java 8 makes it possible to pass function pointers. So now the function pointer to throwsException is passed into expectThrows. And because expect throws returns the Throwable, it's possible to check it's state. Notice the use of hamcrest's assertThat as JUnit5 doesn't have one.
To get the Alpha 5.0 release to work with Eclipse's JUnit 4 plugin per: http://junit.org/junit5/docs/current/user-guide/#using-junit-4-to-run-junit-5-tests, I had to add the following dependencies to my build path (see Referenced Libraries).

Notice JUnit 4 on the end. This is for the @RunWith. If you have problems, scroll down to this article's Troubleshooting section. JUnit 5 and Open Test are available at Maven.org.

References

http://junit.org/junit4/javadoc/latest/org/junit/Rule.html
http://junit.org/junit5/docs/current/user-guide/#migrating-from-junit4
http://junit.org/junit5/docs/current/user-guide/#using-junit-4-to-run-junit-5-tests
JUnit 5 API docs on assertThrows and expectThrows:

Troubleshooting

JUnit5 with Eclipse JUnit 4 plugin test runner:

ClassNotFound issue with DiscoveryFilter

Solution: add junit-engine-api-5.0.0-ALPHA.jar to the build path.

TestRunner executes but doesn't show any test being run.

In the console, you see the following: org.junit.gen5.launcher.main.ServiceLoaderTestEngineRegistry loadTestEngines
INFO: Discovered TestEngines with IDs [junit5]
java.lang.NoClassDefFoundError: org/opentest4j/TestAbortedException
at org.junit.gen5.engine.support.hierarchical.HierarchicalTestExecutor.(HierarchicalTestExecutor.java:37)
Solution: go to Maven repo and add OpenTest4J jar to the build path.