Tuesday, May 24, 2016

Exceptional Exception Testing with JUnit 4.11 Rules and Java 8/JUnit5

Zipporah taking exception to poor 'ol Babbage


You've got something that takes exception to something else and now you want to automate a test to confirm it correctly throws per your specification:




 public class ProblemCauser {
    public void throwsException() {
        throw new IllegalArgumentException("Negative");
    }
}  
One way is the below. But can you see a problem?
import static org.junit.Assert.*;
import org.junit.Test;
public class RulesAndExceptionTest { 
 
    @Test(expected=IllegalArgumentException.class)
    public void canOnlyCheckTypeButNotMessage()
    {
        ProblemCauser problem = new ProblemCauser();
        problem.throwsException();       
    }
}
How do you know the exception message has been set to "Negative?" In the past, that left you with the following strategy:
    @Test
    public void theOldWayToTestExceptionMessageAndType() {
        try {
            ProblemCauser problem = new ProblemCauser();
            problem.throwsException();
            fail("didn't get expected exception");
        } catch (IllegalArgumentException expected) {
            assertEquals("Negative", expected.getMessage());
        }

    }
This does the job. Good work! You can actually fully test drive exception handling with this strategy.  The drawback is this is a lot of code to express a simple check. Even worse, many people forget to include the fail() call, which means if an exception doesn't happen, the test would pass.  This would be an incorrect test case. A better strategy is to use Junit 4.11's Rules package:
import static org.junit.Assert.*;
import org.junit.Ignore;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;

public class RulesAndExceptionTest {
    @Rule
    public ExpectedException
exceptionPolicy = ExpectedException.none();  // Set the default expectation.
    @Test
    public void theNewWayToTestExceptionAndType() {
        exceptionPolicy.expect(IllegalArgumentException.class);
        exceptionPolicy.expectMessage("Negative");


        ProblemCauser problem = new ProblemCauser();
        problem.throwsException();
    }
}
 This is equivalent but uses less code. How does it work?

Exceptional with @Rules

 org.junit.Rule is an annotation that can be applied to a field or method. When JUnit executes the test class, it hands control to the Rule and decides how to execute Before, Test, and After, allowing the it to manage whatever it is supposed to manage. In the case of ExpectedException, allowing it to execute the @Test method and handle any exception thrown out to see if it's expected.

You can have multiple Rules in a test class doing various things. Here are other org.junit.rules.*(http://junit.org/junit4/javadoc/latest/org/junit/rules/package-summary.html):
Click image to zoom.
 Using JUnit's Rules, you can clean up exception handling code to one to three lines of code (one for the test class field, then one where you expect to get an exception) making it more readable and less error prone than using a try{...fail();}catch(Exception ex) { assertEquals(...);} pattern. More functionality with less code. That's exceptional!

Even MORE Exceptional with Java 8 and JUnit 5

JUnit 5 alpha was released in 2016 and it was designed to use Java 8 language features. assertThrows() tests if the exception type was correct. expectThrows() does the same AND returns the exception type so you can examine it further.
import static org.hamcrest.CoreMatchers.startsWith;
import static org.hamcrest.MatcherAssert.assertThat;
import org.junit.gen5.api.Assertions;
import org.junit.gen5.api.Test;
import org.junit.gen5.junit4.runner.JUnit5;
import org.junit.runner.RunWith;

@RunWith(JUnit5.class)
public class ExceptionTestingTest {
    @Test
    public void throwsExceptionWhenPopped() {
        ProblemCauser problem = new ProblemCauser();
        Throwable exception = Assertions.expectThrows(IllegalAccessException.class, () -> problem.throwsException());
        assertThat(exception.getMessage(), startsWith("Error:"));
    }
}
Java 8 and JUnit5 make exception testing even more exceptional. Java 8 makes it possible to pass function pointers. So now the function pointer to throwsException is passed into expectThrows. And because expect throws returns the Throwable, it's possible to check it's state. Notice the use of hamcrest's assertThat as JUnit5 doesn't have one. 
To get the Alpha 5.0 release to work with Eclipse's JUnit 4 plugin per: http://junit.org/junit5/docs/current/user-guide/#using-junit-4-to-run-junit-5-tests, I had to add the following dependencies to my build path (see Referenced Libraries). 
Notice JUnit 4 on the end.  This is for the @RunWith. If you have problems, scroll down to this article's Troubleshooting section. JUnit 5 and Open Test are available at Maven.org.

Other Solutions

Outside of JUnit 5, here are some other solutions but they seem "exceptional" as they require more code and dependencies than using JUnit4 Rules and the code doesn't seem more readable. Here are some articles:
http://blog.codeleak.pl/2014/07/junit-testing-exception-with-java-8-and-lambda-expressions.html 

For background on why you'd want to write micro tests, use your commute time to listen to the Agile Thoughts podcast series about the Test Automation Pyramid, episodes 1 through 8.  Each episode is short, to the point, and entertaining.
Episodes 1 through 8

References

http://junit.org/junit4/javadoc/latest/org/junit/Rule.html
http://junit.org/junit5/docs/current/user-guide/#migrating-from-junit4
http://junit.org/junit5/docs/current/user-guide/#using-junit-4-to-run-junit-5-tests
JUnit 5 API docs on assertThrows and expectThrows:  

Troubleshooting

JUnit5 with Eclipse JUnit 4 plugin test runner:
  •  ClassNotFound issue with DiscoveryFilter 
    • Solution: add junit-engine-api-5.0.0-ALPHA.jar to the build path.
  •  TestRunner executes but doesn't show any test being run. 
    • In the console, you see the following: org.junit.gen5.launcher.main.ServiceLoaderTestEngineRegistry loadTestEngines
      INFO: Discovered TestEngines with IDs [junit5]
      java.lang.NoClassDefFoundError: org/opentest4j/TestAbortedException
          at org.junit.gen5.engine.support.hierarchical.HierarchicalTestExecutor.(HierarchicalTestExecutor.java:37)
    • Solution: go to Maven repo and add OpenTest4J jar to the build path.
  

No comments:

Post a Comment