Confessions of an Agile Coach: JUnit

Showing posts with label JUnit. Show all posts

Tuesday, May 24, 2016

Exceptional Exception Testing with JUnit 4.11 Rules and Java 8/JUnit5

Zipporah taking exception to poor 'ol Babbage

You've got something that takes exception to something else and now you want to automate a test to confirm it correctly throws per your specification:

public class ProblemCauser {
    public void throwsException() {
        throw new IllegalArgumentException("Negative");
    }
}

One way is the below. But can you see a problem?

import static org.junit.Assert.*;
import org.junit.Test;
public class RulesAndExceptionTest {

    @Test(expected=IllegalArgumentException.class)
    public void canOnlyCheckTypeButNotMessage()
    {
        ProblemCauser problem = new ProblemCauser();
        problem.throwsException();
    }
}

How do you know the exception message has been set to "Negative?" In the past, that left you with the following strategy:

    @Test
    public void theOldWayToTestExceptionMessageAndType() {
        try {
            ProblemCauser problem = new ProblemCauser();
            problem.throwsException();
            fail("didn't get expected exception");
        } catch (IllegalArgumentException expected) {
            assertEquals("Negative", expected.getMessage());
        }
    }

This does the job. Good work! You can actually fully test drive exception handling with this strategy. The drawback is this is a lot of code to express a simple check. Even worse, many people forget to include the fail() call, which means if an exception doesn't happen, the test would pass. This would be an incorrect test case. A better strategy is to use Junit 4.11's Rules package:

import static org.junit.Assert.*;
import org.junit.Ignore;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;

public class RulesAndExceptionTest {
@Rule
public ExpectedException exceptionPolicy = ExpectedException.none(); // Set the default expectation.

    @Test
    public void theNewWayToTestExceptionAndType() {
        exceptionPolicy.expect(IllegalArgumentException.class);
        exceptionPolicy.expectMessage("Negative");

        ProblemCauser problem = new ProblemCauser();
        problem.throwsException();
    }
}

This is equivalent but uses less code. How does it work?

Exceptional with @Rules

org.junit.Rule is an annotation that can be applied to a field or method. When JUnit executes the test class, it hands control to the Rule and decides how to execute Before, Test, and After, allowing the it to manage whatever it is supposed to manage. In the case of ExpectedException, allowing it to execute the @Test method and handle any exception thrown out to see if it's expected.

You can have multiple Rules in a test class doing various things. Here are other org.junit.rules.*(http://junit.org/junit4/javadoc/latest/org/junit/rules/package-summary.html):

Click image to zoom.

Using JUnit's Rules, you can clean up exception handling code to one to three lines of code (one for the test class field, then one where you expect to get an exception) making it more readable and less error prone than using a try{...fail();}catch(Exception ex) { assertEquals(...);} pattern. More functionality with less code. That's exceptional!

Even MORE Exceptional with Java 8 and JUnit 5

JUnit 5 alpha was released in 2016 and it was designed to use Java 8 language features. assertThrows() tests if the exception type was correct. expectThrows() does the same AND returns the exception type so you can examine it further.

import static org.hamcrest.CoreMatchers.startsWith;
import static org.hamcrest.MatcherAssert.assertThat;
import org.junit.gen5.api.Assertions;
import org.junit.gen5.api.Test;
import org.junit.gen5.junit4.runner.JUnit5;
import org.junit.runner.RunWith;

@RunWith(JUnit5.class)
public class ExceptionTestingTest {
   @Test
   public void throwsExceptionWhenPopped() {
        ProblemCauser problem = new ProblemCauser();
        Throwable exception = Assertions.expectThrows(IllegalAccessException.class, () -> problem.throwsException());
        assertThat(exception.getMessage(), startsWith("Error:"));
   }
}

Java 8 and JUnit5 make exception testing even more exceptional. Java 8 makes it possible to pass function pointers. So now the function pointer to throwsException is passed into expectThrows. And because expect throws returns the Throwable, it's possible to check it's state. Notice the use of hamcrest's assertThat as JUnit5 doesn't have one.
To get the Alpha 5.0 release to work with Eclipse's JUnit 4 plugin per: http://junit.org/junit5/docs/current/user-guide/#using-junit-4-to-run-junit-5-tests, I had to add the following dependencies to my build path (see Referenced Libraries).

Notice JUnit 4 on the end. This is for the @RunWith. If you have problems, scroll down to this article's Troubleshooting section. JUnit 5 and Open Test are available at Maven.org.

References

http://junit.org/junit4/javadoc/latest/org/junit/Rule.html
http://junit.org/junit5/docs/current/user-guide/#migrating-from-junit4
http://junit.org/junit5/docs/current/user-guide/#using-junit-4-to-run-junit-5-tests
JUnit 5 API docs on assertThrows and expectThrows:

Troubleshooting

JUnit5 with Eclipse JUnit 4 plugin test runner:

ClassNotFound issue with DiscoveryFilter

Solution: add junit-engine-api-5.0.0-ALPHA.jar to the build path.

TestRunner executes but doesn't show any test being run.

In the console, you see the following: org.junit.gen5.launcher.main.ServiceLoaderTestEngineRegistry loadTestEngines
INFO: Discovered TestEngines with IDs [junit5]
java.lang.NoClassDefFoundError: org/opentest4j/TestAbortedException
at org.junit.gen5.engine.support.hierarchical.HierarchicalTestExecutor.(HierarchicalTestExecutor.java:37)
Solution: go to Maven repo and add OpenTest4J jar to the build path.

Thursday, May 5, 2016

Asserting like a Hipster

You know the type. For the past year they've been using the latest new programming language that you just discovered yesterday. They have the latest mobile device with the beta OS, and their laptop is loaded out with more experimental tools than a mad scientist's closet. Hipsters! Every team should have one because they keep us pushing for the next "something better." (Image source: Damn it's hard (and expensive) to be an Alabama hipster.)

Years ago new Java language features along with Hamcrest's matcher library created some changes in JUnit 4.4. Although experienced hands are used to reading the assertEquals(expected, butGot) pattern, if you free yourself from tradition, I think you'll agree that combining Hamcrest with Junit makes tests more readable. Click on the below thumbnail and compare the readability of the test on your left versus right. Although you may be used to reading assertEquals(...), which one would your mother find most understandable?

Tap to zoom

Not only does assertThat(...) read better, it has more advanced matching capabilities so you can do in one line what takes several lines of assertEquals(...) or assertTrue(...). (Source: https://github.com/junit-team/junit4/wiki/Assertions)

tap to zoom

And the error messages are readable without having to write your own (See assertTrue versus assertThat, for example.):

tap to zoom

Setting it up

The setup is difficult which is probably why not many use the more expressive assertThat:

The eclipse JUnit plugin ships with a paired down version of Hamcrest which means although you get good API help via examples on the internet, the version with Eclipse sometimes can't do it or the packages are different (for example, can't do assertThat on a double because the paired down Hamcrest lacks the Matchers for doubles.
"Out of the box" Eclipse's content assist (aka: command complete, intelli-sense) feature isn't any help for discovering the Hamcrest Matchers.
assertThat isn't necessary for doing TDD. TDD adopted alone is hard enough so mixing the "doing TDD" with using assertThat means adding yet another barrier to coaching developers to do TDD, so it's skipped as a future refinement to learn.

To use assertThat, the "hipster tax" must be paid. Let's get started.

Install latest Hamcrest library

Go get java-hamcrest jar and add it to your project as an external jar library, and add it ahead of JUnit 4 in the build path (otherwise it'll use the "partial" hamcrest library that was added into the JUnit 4 plugin.

The dependencies for using Junit assertThat with Hamcrest Matchers is satisfied. Create a test class and copy paste the below code into your project, and run the test to see if your environment is OK. The second test won't even compile if you're trying get by with the Hamcrest that came with Eclipse's JUnit plugin.

package foo;
import static org.hamcrest.CoreMatchers.is;
//import static org.hamcrest.Matchers.closeTo;
import static org.junit.Assert.*;
import org.junit.Test;

public class AssertThatTest {
    @Test
    public void assertThat_worksWithHamcrest() {
        int five = 5;
        assertThat(five, is(5));
    }
/*    @Test
    public void assertThat_accessToModernHamcrest() {
        Double five = 5.005d;
        assertThat(five, is(closeTo(5d, .005d)));
    }
    */
}

You can get started testing, but to make working with Hamcrest enjoyable we need to configure Eclipse.

Configuring Content Assist

Without Content Assist support, using Hamcrest Matchers becomes a productivity suck. Until fixing this, the only recourse is Google or hunting via typing full package names to figuring out what matcher to use.

Out of the box Ecilpse is zero help finding matchers to use with assertThat

Go into Eclipse's Preferences and configure "Favorites" to clue in Content Assist for org.hamcrest.CoreMatchers and org.hamcrest.Matchers.

Afterwards, Eclipse will give you a helping hand, and at this point the hipster tax has largely been paid (anyone know anyone at Eclipse Foundation who is positioned to fix this so it works out of the box?):

Think "is" for comparing a single something. Think "has" when comparing an array or collection to a single or collection of somethings.

You'll be able to do more with less code, and that's a beautiful thing. The following is a taste of the Matcher menu: collections, string comparisons, iteratable objects, time comparison.

Done

After this, you can assertThat(you, is(hipster_cool)). Better get started though. Because soon everyone will be doing it and assertThat won't be hip any longer 'cause it'll be mainstream. For background on why you'd want to write micro tests, use your commute time to listen to the Agile Thoughts podcast series about the Test Automation Pyramid, episodes 1 through 8. Each episode is short, to the point, and entertaining.

Episodes 1 through 8

Now get 'er done hipster!

Troubleshooting

If you don't change the project dependency order so that java-hamcrest is before JUnit 4, you'll get the following exception during test execution because of a compatibility issue when trying to use Eclipse's JUnit plugin's older/partial Hamcrest: