Confessions of an Agile Coach: Unit Testing Spring Applications

Abstract: there are at least four ways to handle dependency injection for unit testing a Spring application: configuration by Spring XML configuration file, DIY injection, Spring's Java API, and Spring's injection via Reflection utility. The later three are suitable for unit tests. At the end are benchmarks for each strategy and a discussion on using Mockito with Spring.

You're a hot shot Java developer so of course you're doing the XP practice of TDD. You're working with Spring, then SHAZAM! You need to inject a mock dependency and are wondering how to do it. To make matters worse, you notice every time you add a new test class with @RunWith(SpringJunitTestRunner), those tests take two orders of magnitude longer to execute. (The other
tests run in milliseconds, SpringJunitTestRunner tests take hundreds of milliseconds.

What's happening?

Spring isn't a lightweight framework. It does a lot of work in the background, some of which includes looking around for spring configuration XML files on the file system.

Although you wanted to write fast feedback *unit* tests, now you're dependent on file system resources! Here is a simple example to illustrate the strategies to unit testing a Spring application.

MainApp.java:

package foo;

import org.springframework.beans.factory.annotation.Autowired;

@Component()
public class MainApp {

// NullPointerException occurs if no injection

@Autowired()
private MessageBean helloWorld;

    // A command line app needs to setup ApplicationContext.
    public static void main(String[] args) {
        ClassPathXmlApplicationContext spring = new ClassPathXmlApplicationContext("beans.xml");
        springInjectAndDoWork(spring);
        spring.close();
    }

    // separated the concern of deciding what kind of ApplicationContext from the work.
    public static void springInjectAndDoWork(ApplicationContext spring) {
        MainApp app = spring.getBean(MainApp.class);
        MessageBean message = (MessageBean) spring.getBean("helloWorld");
        app.callGreeter();
    }

    public void callGreeter() {
        helloWorld.getMessage();
    }
}

MessageBean.java:

package foo;
import org.springframework.stereotype.Component;

@Component
public class MessageBean {
   public void getMessage(){
      System.out.println("My Message ");
   }
}

beans.xml (tap to read):

And unfortunately, this often done trap of building a charlatan unit test is:

package foo;
import static org.junit.Assert.*;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("beansTest.xml")
public class SpringTestRunnerAndXMLFileTest {
    @Test
    public void construct_callToMainThrowsNoException() {
        MainApp main = new MainApp();
        main.main(null);
    }
}

This will allow you to build test automation but is this strategy appropriate for *unit* test automation?

Using the SpringJUnit4ClassRunner creates a Spring ApplicationContext and enables some hooks such as specifying the spring configuration file to use.

Problem 1: slow

Tests that require filesystem access can never be a unit test. Tests like the above will load the spring configuration .xml file and this will turn a test that takes tens of milliseconds into something that takes hundreds.
Problem 2: poor isolation
Each @Test method often needs to insert different fake data via Spring’s dependency injection. Having a different config file for each situation means moving valuable information from a single source (the test class or test method) into a separate file at the expense of making it harder to maintain. Also, these config files tend to be used like shotguns, loaded with a bunch of beans that aren't specific to supporting any specific test's injection. This makes maintaining the test harder. Most developers create a single XML file to be used for a lot of test classes becoming a "global" shared by all the project's tests. When the XML file becomes flawed, then many unit tests are affected because the file becomes a dumping ground for injection for many tests rather than a single test. Like anything "global" the file can't scale to many tests and becomes an "untouchable" that no one wants to fiddle with because no one can predict the impact. This situation can be improved by creating an XML file customized for each test so the XML file becomes documentation for each specific test class.

Alternatives

Here are some strategies that scale to supporting hundreds to thousands of tests and give fast feedback.

DIY Injection

When unit testing, dump Spring framework and inject the beans yourself. When your application runs in production, it still uses Spring.

Drawbacks:

adding to the class under test's API for injection via: constructor injection or setters. If you've got a lot of collaborators, the API reflects the dependencies (for good or bad).

Positives:

Fast. In fact, so fast my laptop reports 0ms.
Doing say Constructor Injection, leaves a nice clear design artifact for future test creators to use because it documents in one location all the dependencies.
Test isolation is maintained as each test is injecting only what it needs. And if you do this well, your test documents what collaborators it depends on for the test.
Configuration is located in the test class making maintainability easier.

package foo;

import org.junit.Test;
public class DIYInjectionTest {
    @Test
    public void construct_callToMainThrowsNoException() {
        MainApp main = new MainApp(new MessageBean()); // using DIY constructor injection
        main.callGreeter();
    }
}

package foo;
import org.springframework.beans.factory.annotation.Autowired;
@Component()
public class MainApp {
    @Autowired()
    private MessageBean helloWorld;

    MainApp(MessageBean messageSender) { helloWorld = messageSender;    } // added for DIY injection test

    // A command line app needs to setup ApplicationContext.
    public static void main(String[] args) { // this main commits you to XML, so put very little code in here or make your caller give you an ApplicationContext.
        ClassPathXmlApplicationContext spring = new ClassPathXmlApplicationContext("beans.xml");
        springInjectAndDoWork(spring);
        spring.close();
    }

    // separated the concern of deciding what kind of ApplicationContext from the rest of the work.
    public static void springInjectAndDoWork(ApplicationContext spring) {
        MainApp app = spring.getBean(MainApp.class);
        MessageBean message = (MessageBean) spring.getBean("helloWorld");
        app.callGreeter();
    }

    public void callGreeter() {
       helloWorld.getMessage();
    }
}

Spring's Configuration Java API

Use Spring’s java classes to configure how to perform the Spring injection by creating a class that models what's in the configuration XML file using annotations such as @Configuration, @Bean, ... etc.
Drawbacks:

If you've a ton of injection to do for the unit under test, you'll feel the pain that was hidden within the XML configuration file by needing to express the configuration again as Java code. This pain can be made easier by pushing that code into a super class if a lot of test classes require it

Positives:

Fast.
Keeps injection happening via Spring so no class's interface needs to be changed.
Test isolation is maintained.
Configuration is collocated with its test class (assuming you're using a static inner class like the below example)

package foo;
import org.junit.Before;
public class SpringConfigurationAPITest {
    // The below inner class strategy doesn't work unless the class is static.
    @Configuration
    static public class ContextConfigurationAsInnerClass {
        @Bean
        public MessageBean helloWorld()    {return new MessageBean();}
        @Bean
        public MainApp mainApp() {return new MainApp();}
    }

    @Test
    public void construct_callToMainThrowsNoException() {
        MainApp main = new MainApp();
        ApplicationContext spring = new AnnotationConfigApplicationContext(ContextConfigurationAsInnerClass.class);

        main.springInjectAndDoWork(spring); // using Spring injection but without an XML file
    }
}

SpringTestUtil

Spring provides a simple API that lets you inject the collaborators your test needs and does it with less effort than using its Java Configuration API. Simply use ReflectionTestUtils.setField(...) to inject the collaborators. It will do this regardless if the field you're injecting to is private.

Drawbacks:

if you've got a lot of injection to do, you need to make a lot of calls to setField.

Positives:

fast
test isolation is maintained
no changes to the class under test's API.
it doesn't require Spring framework to be active (no need for an ApplicationContext).
this api is simpler than configuring Spring using its Java Configuration API.
Configuration is collocated with its test class. This strategy documents the dependencies that are necessary to unit test the class under test, keeping maintainability easy and what to do when creating more new tests

package foo;

import org.junit.Before;
public class SpringReflectionInjectionTest {
    private MainApp main;

    @Before
    public void injections()
    {
        main = new MainApp();
        ReflectionTestUtils.setField(main, "helloWorld", new MessageBean());
    }

    @Test
    public void mainThrowsNoException() {
        main.callGreeter();
    }
}

Benchmarks

Here are the benchmarks for the four strategies.
Run 1:

Run 2:

Notice the test that depends on the configuration file has a large variance in run time. I've seen it take as long as 700ms on my SSD laptop. Remember that this is only one test. Real applications that are under development for twelve months (five developers) typically need about three thousand tests and easily get them to execute in 3 minutes, or 60 milliseconds per test method. If your tests require configuration files to work then you're looking at 9 to double digit minutes. A lot of human factors on feedback will work against you because few people are going to sit at their workstation to wait for those tests. When the tests take a long time to give feedback, people are already checking email or taking a break, losing context of what they did. The wait state is too long.
And if you keep this application around for the industry average of 10 years, it's not uncommon you'll have ten thousand tests. Beyond the cost of waiting for feedback, it's harder to maintain tests that depend on external XML files. The test triangle refers to the geometric concept that unit tests should be most numerous just as the area of the unit test section is much greater than the UI area of the triangle.

Now you've got three strategies for making fast feedback unit tests for a Spring application (bottom of the triangle) and XML files for slower feedback system tests (the middle tier of the triangle). Like a golfer who selects the "best club for the situation" you want to use the best tool for the job.

Spring and Mockito

Many people on the Java stack are using Mockito for mocking their objects. Not only can Mockito be used to mock objects, it can inject those objects similar to Spring. Out of the many ways to inject Mockito mocked beans, here are the most promising ones: use SpringTestUtil.setField(...), use Spring to inject mocks, or use Mockito injection.

SpringTestUtil.setField(...)

One common configuration is to use Mockito to build the mocks then use SpringTestUtil.setField(...) for injection. Just change the example code for SpringTestUtil.setField(...) to:

@Before
    public void injections()
    {
       main = new MainApp();
       ReflectionTestUtils.setField(main, "helloWorld", Mockito.mock(MessageBean.class));
    }

Configure Spring to inject Mockito Mocks

Using Sprig's Configuration Java API to return Mockito mocks:

package foo;
import org.junit.Before;
public class MockitoMockInjectionViaSpringConfigurationAPITest {
    static private MessageBean messageBeanMock;
    // I'd rather not use static, but Inner class @Configuration doesn't work unless the class is static.
    @Configuration
    static public class ContextConfigurationAsInnerClass {
       @Bean
       public MessageBean helloWorld()   {return messageBeanMock; } //alternatively, put Mockito.mock(...) here

       @Bean
       public MainApp mainApp() {return new MainApp();}
    }

    @Before
    public void initStaticsToMaintainTestIsolation(){messageBeanMock = Mockito.mock(MessageBean.class);   }

    @Test
    public void construct_callToMainThrowsNoException() {
       MainApp main = new MainApp();
       ApplicationContext spring = new AnnotationConfigApplicationContext(ContextConfigurationAsInnerClass.class);

       main.springInjectAndDoWork(spring); // using Spring injection but without an XML file
       Mockito.verify(messageBeanMock).getMessage();
    }
}

Mixing Mockito injection with Spring injection

Mixing Mockito mock injection with Spring mock injection is tricky. Mockito does injection upon calling MocktioAnnotations.initMocks(...) or when JUnit sets up the test class via @RunWith(MockitoJUnitRunner). Spring does it's injection when you call ApplicationContext.getBean(...) or when JUnit tests up the test class via @RunWith(SpringJUnit4ClassRunner). (Hopefully you won't use SpringJUnit4ClassRunner as you'll not have a unit test when you're finished.) You'll need to understand what's happening and when so you don't have your mockito injected objects later overwritten by Spring's injection. Although getting this working is doable I try to avoid un-necessary complications so I'll move on to using Mocktio only injection.

Mockito injection

Mockito injection is configured with @Mock and @InjectMocks. Mark beans, etc, to be injected with @Mock. Mark the class to inject the @Mock(s) into with @InjectMocks. Injection is triggered by using @RunWith(MockitoJUnitRunner) on the Test class or explictly calling MockitoAnnotations.initMocks(...).

package foo;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.runners.MockitoJUnitRunner;

@RunWith(MockitoJUnitRunner.class)
public class MockitoRunWithInjectionTest {
    @Mock
    private MessageBean messageBeanMock;
    @InjectMocks
    private MainApp main;

    @Test
    public void construct_callToMainThrowsNoException() {
        main.callGreeter();
        Mockito.verify(messageBeanMock).getMessage();
    }
}

Or by using initMocks(...):

package foo;
import org.junit.Before;
public class MockitoInjectionViaExplicitCallTest {
    @Mock
    private MessageBean messageBeanMock;
    @InjectMocks
    private MainApp main;

    @Before
    public void doInjections()    {MockitoAnnotations.initMocks(this);}

    @Test
    public void construct_callToMainThrowsNoException() {
        main.callGreeter();
        Mockito.verify(messageBeanMock).getMessage();
    }
}

Mockito uses Reflection and runtime bytecode generation to mock objects. Because Mockito is slower than simply using POJOs (plain old java objects) it's better to use your product's objects if they are suitable for the task. Since Mockito doesn't use files for configuration, its impact isn't the two orders of magnitude of using Spring and XML configuration files.

For some great background on WHY writing FAST and Furious micro tests is so important, listen to the short, to the point, and entertaining Testing Pyramid series (episodes 1 through 8) on the Agile Thoughts podcast.

Episodes 1 through 8

References

Common gotcha's:

Most of the examples on the internet have people making "top level" java classes annotated with @Configuration. This is not a good model for unit tests where the unit test should (as much as is practical) be a simple record for setting up a unit test. It's really better to use an inner class. If you see the below exception regarding your inner @Configuration class, declare it "static" as I did in the code example.

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'springConfigurationAPITest.ContextConfigurationAsInnerClass': Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [foo.SpringConfigurationAPITest$ContextConfigurationAsInnerClass$$EnhancerBySpringCGLIB$$a473a0f4]: No default constructor found; nested exception is java.lang.NoSuchMethodException: foo.SpringConfigurationAPITest$ContextConfigurationAsInnerClass$$EnhancerBySpringCGLIB$$a473a0f4.