Tuesday, May 24, 2016

Exceptional Exception Testing with JUnit 4.11 Rules and Java 8/JUnit5

Zipporah taking exception to poor 'ol Babbage


You've got something that takes exception to something else and now you want to automate a test to confirm it correctly throws per your specification:




 public class ProblemCauser {
    public void throwsException() {
        throw new IllegalArgumentException("Negative");
    }
}  
One way is the below. But can you see a problem?
import static org.junit.Assert.*;
import org.junit.Test;
public class RulesAndExceptionTest { 
 
    @Test(expected=IllegalArgumentException.class)
    public void canOnlyCheckTypeButNotMessage()
    {
        ProblemCauser problem = new ProblemCauser();
        problem.throwsException();       
    }
}
How do you know the exception message has been set to "Negative?" In the past, that left you with the following strategy:
    @Test
    public void theOldWayToTestExceptionMessageAndType() {
        try {
            ProblemCauser problem = new ProblemCauser();
            problem.throwsException();
            fail("didn't get expected exception");
        } catch (IllegalArgumentException expected) {
            assertEquals("Negative", expected.getMessage());
        }

    }
This does the job. Good work! You can actually fully test drive exception handling with this strategy.  The drawback is this is a lot of code to express a simple check. Even worse, many people forget to include the fail() call, which means if an exception doesn't happen, the test would pass.  This would be an incorrect test case. A better strategy is to use Junit 4.11's Rules package:
import static org.junit.Assert.*;
import org.junit.Ignore;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;

public class RulesAndExceptionTest {
    @Rule
    public ExpectedException
exceptionPolicy = ExpectedException.none();  // Set the default expectation.
    @Test
    public void theNewWayToTestExceptionAndType() {
        exceptionPolicy.expect(IllegalArgumentException.class);
        exceptionPolicy.expectMessage("Negative");


        ProblemCauser problem = new ProblemCauser();
        problem.throwsException();
    }
}
 This is equivalent but uses less code. How does it work?

Exceptional with @Rules

 org.junit.Rule is an annotation that can be applied to a field or method. When JUnit executes the test class, it hands control to the Rule and decides how to execute Before, Test, and After, allowing the it to manage whatever it is supposed to manage. In the case of ExpectedException, allowing it to execute the @Test method and handle any exception thrown out to see if it's expected.

You can have multiple Rules in a test class doing various things. Here are other org.junit.rules.*(http://junit.org/junit4/javadoc/latest/org/junit/rules/package-summary.html):
Click image to zoom.
 Using JUnit's Rules, you can clean up exception handling code to one to three lines of code (one for the test class field, then one where you expect to get an exception) making it more readable and less error prone than using a try{...fail();}catch(Exception ex) { assertEquals(...);} pattern. More functionality with less code. That's exceptional!

Even MORE Exceptional with Java 8 and JUnit 5

JUnit 5 alpha was released in 2016 and it was designed to use Java 8 language features. assertThrows() tests if the exception type was correct. expectThrows() does the same AND returns the exception type so you can examine it further.
import static org.hamcrest.CoreMatchers.startsWith;
import static org.hamcrest.MatcherAssert.assertThat;
import org.junit.gen5.api.Assertions;
import org.junit.gen5.api.Test;
import org.junit.gen5.junit4.runner.JUnit5;
import org.junit.runner.RunWith;

@RunWith(JUnit5.class)
public class ExceptionTestingTest {
    @Test
    public void throwsExceptionWhenPopped() {
        ProblemCauser problem = new ProblemCauser();
        Throwable exception = Assertions.expectThrows(IllegalAccessException.class, () -> problem.throwsException());
        assertThat(exception.getMessage(), startsWith("Error:"));
    }
}
Java 8 and JUnit5 make exception testing even more exceptional. Java 8 makes it possible to pass function pointers. So now the function pointer to throwsException is passed into expectThrows. And because expect throws returns the Throwable, it's possible to check it's state. Notice the use of hamcrest's assertThat as JUnit5 doesn't have one. 
To get the Alpha 5.0 release to work with Eclipse's JUnit 4 plugin per: http://junit.org/junit5/docs/current/user-guide/#using-junit-4-to-run-junit-5-tests, I had to add the following dependencies to my build path (see Referenced Libraries). 
Notice JUnit 4 on the end.  This is for the @RunWith. If you have problems, scroll down to this article's Troubleshooting section. JUnit 5 and Open Test are available at Maven.org.

Other Solutions

Outside of JUnit 5, here are some other solutions but they seem "exceptional" as they require more code and dependencies than using JUnit4 Rules and the code doesn't seem more readable. Here are some articles:
http://blog.codeleak.pl/2014/07/junit-testing-exception-with-java-8-and-lambda-expressions.html 

For background on why you'd want to write micro tests, use your commute time to listen to the Agile Thoughts podcast series about the Test Automation Pyramid, episodes 1 through 8.  Each episode is short, to the point, and entertaining.
Episodes 1 through 8

References

http://junit.org/junit4/javadoc/latest/org/junit/Rule.html
http://junit.org/junit5/docs/current/user-guide/#migrating-from-junit4
http://junit.org/junit5/docs/current/user-guide/#using-junit-4-to-run-junit-5-tests
JUnit 5 API docs on assertThrows and expectThrows:  

Troubleshooting

JUnit5 with Eclipse JUnit 4 plugin test runner:
  •  ClassNotFound issue with DiscoveryFilter 
    • Solution: add junit-engine-api-5.0.0-ALPHA.jar to the build path.
  •  TestRunner executes but doesn't show any test being run. 
    • In the console, you see the following: org.junit.gen5.launcher.main.ServiceLoaderTestEngineRegistry loadTestEngines
      INFO: Discovered TestEngines with IDs [junit5]
      java.lang.NoClassDefFoundError: org/opentest4j/TestAbortedException
          at org.junit.gen5.engine.support.hierarchical.HierarchicalTestExecutor.(HierarchicalTestExecutor.java:37)
    • Solution: go to Maven repo and add OpenTest4J jar to the build path.
  

Thursday, May 5, 2016

Asserting like a Hipster

You know the type. For the past year they've been using the latest new programming language that you just discovered yesterday. They have the latest mobile device with the beta OS, and their laptop is loaded out with more experimental tools than a mad scientist's closet. Hipsters! Every team should have one because they keep us pushing for the next "something better." (Image source: Damn it's hard (and expensive) to be an Alabama hipster.)

Years ago new Java language features along with Hamcrest's matcher library created some changes in JUnit 4.4. Although experienced hands are used to reading the assertEquals(expected, butGot) pattern, if you free yourself from tradition, I think you'll agree that combining Hamcrest with Junit makes tests more readable. Click on the below thumbnail and compare the readability of the test on your left versus right. Although you may be used to reading assertEquals(...), which one would your mother find most understandable?
Tap to zoom
Not only does assertThat(...) read better, it has more advanced matching capabilities so you can do in one line what takes several lines of assertEquals(...) or assertTrue(...). (Source: https://github.com/junit-team/junit4/wiki/Assertions)
tap to zoom
And the error messages are readable without having to write your own (See assertTrue versus assertThat, for example.):

tap to zoom

Setting it up

The setup is difficult which is probably why not many use the more expressive assertThat:
  1. The eclipse JUnit plugin ships with a paired down version of Hamcrest which means although you get good API help via examples on the internet, the version with Eclipse sometimes can't do it or the packages are different (for example, can't do assertThat on a double because the paired down Hamcrest lacks the Matchers for doubles.
  2. "Out of the box" Eclipse's content assist (aka: command complete, intelli-sense) feature isn't any help for discovering the Hamcrest Matchers.
  3. assertThat isn't necessary for doing TDD. TDD adopted alone is hard enough so mixing the "doing TDD" with using assertThat means adding yet another barrier to coaching developers to do TDD, so it's skipped as a future refinement to learn.
 To use assertThat, the "hipster tax" must be paid. Let's get started.

Install latest Hamcrest library

 Go get java-hamcrest jar and add it to your project as an external jar library, and add it ahead of JUnit 4 in the build path (otherwise it'll use the "partial" hamcrest library that was added into the JUnit 4 plugin.
The dependencies for using Junit assertThat with Hamcrest Matchers is satisfied. Create a test class and copy paste the below code into your project, and run the test to see if your environment is OK.  The second test won't even compile if you're trying get by with the Hamcrest that came with Eclipse's JUnit plugin.
package foo;
import static org.hamcrest.CoreMatchers.is;
//import static org.hamcrest.Matchers.closeTo;
import static org.junit.Assert.*;
import org.junit.Test;

public class AssertThatTest {
    @Test
    public void assertThat_worksWithHamcrest() {
        int five = 5;
        assertThat(five, is(5));
    }
/*    @Test
    public void assertThat_accessToModernHamcrest() {
        Double five = 5.005d;
        assertThat(five, is(closeTo(5d, .005d)));
    }
    */   
}
You can get started testing, but to make working with Hamcrest enjoyable we need to configure Eclipse.

Configuring Content Assist

Without Content Assist support, using Hamcrest Matchers becomes a productivity suck. Until fixing this, the only recourse is Google or hunting via typing full package names to figuring out what matcher to use.
Out of the box Ecilpse is zero help finding matchers to use with assertThat
Go into Eclipse's Preferences and configure "Favorites" to clue in Content Assist for org.hamcrest.CoreMatchers and org.hamcrest.Matchers.





 Afterwards, Eclipse will give you a helping hand, and at this point the hipster tax has largely been paid (anyone know anyone at Eclipse Foundation who is positioned to fix this so it works out of the box?):
Think "is" for comparing a single something. Think "has" when comparing an array or collection to a single or collection of somethings

You'll be able to do more with less code, and that's a beautiful thing. The following is a taste of the Matcher menu: collections, string comparisons, iteratable objects, time comparison.




Done

After this, you can assertThat(you, is(hipster_cool)). Better get started though. Because soon everyone will be doing it and assertThat won't be hip any longer 'cause it'll be mainstream. For background on why you'd want to write micro tests, use your commute time to listen to the Agile Thoughts podcast series about the Test Automation Pyramid, episodes 1 through 8.  Each episode is short, to the point, and entertaining.
Episodes 1 through 8

Now get 'er done hipster!

Troubleshooting

If you don't change the project dependency order so that java-hamcrest is before JUnit 4, you'll get the following exception during test execution because of a compatibility issue when trying to use Eclipse's JUnit plugin's older/partial Hamcrest:
Select to zoom.

References

http://edgibbs.com/junit-4-with-hamcrest/

http://stackoverflow.com/questions/2922879/best-way-to-unit-test-collection 

http://stackoverflow.com/questions/21624592/hamcrest-compare-collections


Sunday, May 1, 2016

Unit Testing Spring Applications

Abstract: there are at least four ways to handle dependency injection for unit testing a Spring application: configuration by Spring XML configuration file, DIY injection, Spring's Java API, and Spring's injection via Reflection utility. The later three are suitable for unit tests. At the end are benchmarks for each strategy and a discussion on using Mockito with Spring.

You're a hot shot Java developer so of course you're doing the XP practice of TDD. You're working with Spring, then SHAZAM! You need to inject a mock dependency and are wondering how to do it. To make matters worse, you notice every time you add a new test class with @RunWith(SpringJunitTestRunner), those tests take two orders of magnitude longer to execute. (The other
tests run in milliseconds, SpringJunitTestRunner tests take hundreds of milliseconds.

What's happening?

Spring isn't a lightweight framework. It does a lot of work in the background, some of which includes looking around for spring configuration XML files on the file system.
Although you wanted to write fast feedback *unit* tests, now you're dependent on file system resources! Here is a simple example to illustrate the strategies to unit testing a Spring application.
MainApp.java:
package foo;

import org.springframework.beans.factory.annotation.Autowired;

@Component()
public class MainApp {
 // NullPointerException occurs if no injection
    @Autowired()
    private MessageBean helloWorld;
    // A command line app needs to setup ApplicationContext.
    public static void main(String[] args) { 
        ClassPathXmlApplicationContext spring = new ClassPathXmlApplicationContext("beans.xml");
        springInjectAndDoWork(spring);
        spring.close();
    }
    // separated the concern of deciding what kind of ApplicationContext from the work.
    public static void springInjectAndDoWork(ApplicationContext spring) {
        MainApp app = spring.getBean(MainApp.class);
        MessageBean message = (MessageBean) spring.getBean("helloWorld");
        app.callGreeter();
    }

    public void callGreeter() {
        helloWorld.getMessage();
    }
}
MessageBean.java:
package foo;
import org.springframework.stereotype.Component;

@Component
public class MessageBean {
   public void getMessage(){
      System.out.println("My Message  ");
   }
}
beans.xml (tap to read):
And unfortunately, this often done trap of building a charlatan unit test is:
package foo;
import static org.junit.Assert.*;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("beansTest.xml") 
public class SpringTestRunnerAndXMLFileTest {
    @Test
    public void construct_callToMainThrowsNoException() {
        MainApp main = new MainApp();
        main.main(null);
    }
}
This will allow you to build test automation but is this strategy appropriate for *unit* test automation?
Using the SpringJUnit4ClassRunner creates a Spring ApplicationContext and enables some hooks such as specifying the spring configuration file to use.
Problem 1: slow
Tests that require filesystem access can never be a unit test. Tests like the above will load the spring configuration .xml file and this will turn a test that takes tens of milliseconds into something that takes hundreds.
Problem 2: poor isolation
Each @Test method often needs to insert different fake data via Spring’s dependency injection. Having a different config file for each situation means moving valuable information from a single source (the test class or test method) into a separate file at the expense of making it harder to maintain.  Also, these config files tend to be used like shotguns, loaded with a bunch of beans that aren't specific to supporting any specific test's injection. This makes maintaining the test harder. Most developers create a single XML file to be used for a lot of test classes becoming a "global" shared by all the project's tests. When the XML file becomes flawed, then many unit tests are affected because the file becomes a dumping ground for injection for many tests rather than a single test.  Like anything "global" the file can't scale to many tests and becomes an "untouchable" that no one wants to fiddle with because no one can predict the impact. This situation can be improved by creating an XML file customized for each test so the XML file becomes documentation for each specific test class.

Alternatives

Here are some strategies that scale to supporting hundreds to thousands of tests and give fast feedback.

DIY Injection

When unit testing, dump Spring framework and inject the beans yourself. When your application runs in production, it still uses Spring.
Drawbacks: 
  • adding to the class under test's API for injection via: constructor injection or setters. If you've got a lot of collaborators, the API reflects the dependencies (for good or bad).
Positives: 
  • Fast. In fact, so fast my laptop reports 0ms.
  • Doing say Constructor Injection, leaves a nice clear design artifact for future test creators to use because it documents in one location all the dependencies.
  • Test isolation is maintained as each test is injecting only what it needs. And if you do this well, your test documents what collaborators it depends on for the test.
  • Configuration is located in the test class making maintainability easier.
package foo;
import org.junit.Test;
public class DIYInjectionTest { 
    @Test
    public void construct_callToMainThrowsNoException() {
        MainApp main = new MainApp(new MessageBean());  // using DIY constructor injection
        main.callGreeter();
    }
}
package foo;
import org.springframework.beans.factory.annotation.Autowired;
@Component()
public class MainApp {
    @Autowired()
    private MessageBean helloWorld;

    MainApp(MessageBean messageSender) { helloWorld = messageSender;    } // added for DIY injection test

    // A command line app needs to setup ApplicationContext.
    public static void main(String[] args) {  // this main commits you to XML, so put very little code in here or make your caller give you an ApplicationContext.
        ClassPathXmlApplicationContext spring = new ClassPathXmlApplicationContext("beans.xml");
        springInjectAndDoWork(spring);
        spring.close();
    }

    // separated the concern of deciding what kind of ApplicationContext from the rest of the work.
    public static void springInjectAndDoWork(ApplicationContext spring) {
        MainApp app = spring.getBean(MainApp.class);
        MessageBean message = (MessageBean) spring.getBean("helloWorld");
        app.callGreeter();
    }

    public void callGreeter() {
       helloWorld.getMessage();
    }
}

Spring's Configuration Java API

Use Spring’s java classes to configure how to perform the Spring injection by creating a class that models what's in the configuration XML file using annotations such as @Configuration, @Bean, ... etc.
Drawbacks:
  • If you've a ton of injection to do for the unit under test, you'll feel the pain that was hidden within the XML configuration file by needing to express the configuration again as Java code. This pain can be made easier by pushing that code into a super class if a lot of test classes require it
Positives:
  • Fast. 
  • Keeps injection happening via Spring so no class's interface needs to be changed.
  • Test isolation is maintained.
  • Configuration is collocated with its test class (assuming you're using a static inner class like the below example)
package foo;
import org.junit.Before;
public class SpringConfigurationAPITest {
    // The below inner class strategy doesn't work unless the class is static.
    @Configuration
    static public class ContextConfigurationAsInnerClass {
        @Bean
        public MessageBean helloWorld()    {return new MessageBean();}
        @Bean
        public MainApp mainApp() {return new MainApp();}
    }

   
    @Test
    public void construct_callToMainThrowsNoException() {
        MainApp main = new MainApp();
        ApplicationContext spring = new AnnotationConfigApplicationContext(ContextConfigurationAsInnerClass.class);
       
        main.springInjectAndDoWork(spring);  // using Spring injection but without an XML file
    }
}

SpringTestUtil

Spring provides a simple API that lets you inject the collaborators your test needs and does it with less effort than using its Java Configuration API. Simply use ReflectionTestUtils.setField(...) to inject the collaborators. It will do this regardless if the field you're injecting to is private.

Drawbacks:
  • if you've got a lot of injection to do, you need to make a lot of calls to setField.
Positives:
  • fast
  • test isolation is maintained
  • no changes to the class under test's API.
  • it doesn't require Spring framework to be active (no need for an ApplicationContext).
  • this api is simpler than configuring Spring using its Java Configuration API.
  • Configuration is collocated with its test class. This strategy documents the dependencies that are necessary to unit test the class under test, keeping maintainability easy and what to do when creating more new tests
package foo;
import org.junit.Before;
public class SpringReflectionInjectionTest {
    private MainApp main;
   
    @Before
    public void injections()
    {
        main = new MainApp();
        ReflectionTestUtils.setField(main, "helloWorld", new MessageBean());
    }
   
    @Test
    public void mainThrowsNoException() {       
        main.callGreeter();
    }
}

Benchmarks

Here are the benchmarks for the four strategies.
Run 1:
Run 2:
Notice the test that depends on the configuration file has a large variance in run time. I've seen it take as long as 700ms on my SSD laptop. Remember that this is only one test. Real applications that are under development for twelve months (five developers) typically need about three thousand tests and easily get them to execute in 3 minutes, or 60 milliseconds per test method. If your tests require configuration files to work then you're looking at 9 to double digit minutes. A lot of human factors on feedback will work against you because few people are going to sit at their workstation to wait for those tests. When the tests take a long time to give feedback, people are already checking email or taking a break, losing context of what they did. The wait state is too long.
And if you keep this application around for the industry average of 10 years, it's not uncommon you'll have ten thousand tests. Beyond the cost of waiting for feedback, it's harder to maintain tests that depend on external XML files. The test triangle refers to the geometric concept that unit tests should be most numerous just as the area of the unit test section is much greater than the UI area of the triangle.
Now you've got three strategies for making fast feedback unit tests for a Spring application (bottom of the triangle) and XML files for slower feedback system tests (the middle tier of the triangle). Like a golfer who selects the "best club for the situation" you want to use the best tool for the job.

Spring and Mockito

Many people on the Java stack are using Mockito for mocking their objects. Not only can Mockito be used to mock objects, it can inject those objects similar to Spring. Out of the many ways to inject Mockito mocked beans, here are the most promising ones: use SpringTestUtil.setField(...), use Spring to inject mocks, or use Mockito injection. 

SpringTestUtil.setField(...)

One common configuration is to use Mockito to build the mocks then use SpringTestUtil.setField(...) for injection. Just change the example code for SpringTestUtil.setField(...) to:
 @Before
    public void injections()
    {
        main = new MainApp();
        ReflectionTestUtils.setField(main, "helloWorld", Mockito.mock(MessageBean.class));
    }

Configure Spring to inject Mockito Mocks

Using Sprig's Configuration Java API to return Mockito mocks:
package foo;
import org.junit.Before;
public class MockitoMockInjectionViaSpringConfigurationAPITest {
    static private MessageBean messageBeanMock;
    // I'd rather not use static, but Inner class @Configuration doesn't work unless the class is static.
    @Configuration
    static public class ContextConfigurationAsInnerClass {  
        @Bean
        public MessageBean helloWorld()    {return messageBeanMock; }
//alternatively, put Mockito.mock(...) here
      
        @Bean
        public MainApp mainApp() {return new MainApp();}
    }
   
    @Before
    public void initStaticsToMaintainTestIsolation(){messageBeanMock = Mockito.mock(MessageBean.class);    }
   
    @Test
    public void construct_callToMainThrowsNoException() {
        MainApp main = new MainApp();
        ApplicationContext spring = new AnnotationConfigApplicationContext(ContextConfigurationAsInnerClass.class);
      
        main.springInjectAndDoWork(spring);  // using Spring injection but without an XML file
        Mockito.verify(messageBeanMock).getMessage();
    }
}

Mixing Mockito injection with Spring injection

Mixing Mockito mock injection with Spring mock injection is tricky. Mockito does injection upon calling MocktioAnnotations.initMocks(...) or when JUnit sets up the test class via @RunWith(MockitoJUnitRunner). Spring does it's injection when you call ApplicationContext.getBean(...) or when JUnit tests up the test class via @RunWith(SpringJUnit4ClassRunner). (Hopefully you won't use SpringJUnit4ClassRunner as you'll not have a unit test when you're finished.) You'll need to understand what's happening and when so you don't have your mockito injected objects later overwritten by Spring's injection. Although getting this working is doable I try to avoid un-necessary complications so I'll move on to using Mocktio only injection.

Mockito injection

Mockito injection is configured with @Mock and @InjectMocks. Mark beans, etc, to be injected with @Mock.  Mark the class to inject the @Mock(s) into with @InjectMocks. Injection is triggered by using @RunWith(MockitoJUnitRunner) on the Test class or explictly calling MockitoAnnotations.initMocks(...).
package foo;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.runners.MockitoJUnitRunner;

@RunWith(MockitoJUnitRunner.class)
public class MockitoRunWithInjectionTest {
    @Mock
    private MessageBean messageBeanMock;
    @InjectMocks
    private MainApp main;


    @Test
    public void construct_callToMainThrowsNoException() {       
        main.callGreeter();
        Mockito.verify(messageBeanMock).getMessage();
    }
}
Or by using initMocks(...):
package foo;
import org.junit.Before;
public class MockitoInjectionViaExplicitCallTest {
    @Mock
    private MessageBean messageBeanMock;
    @InjectMocks
    private MainApp main;

   
    @Before
    public void doInjections()    {MockitoAnnotations.initMocks(this);}

    @Test
    public void construct_callToMainThrowsNoException() {       
        main.callGreeter();
        Mockito.verify(messageBeanMock).getMessage();
    }
}
Mockito uses Reflection and runtime bytecode generation to mock objects. Because Mockito is slower than simply using POJOs (plain old java objects) it's better to use your product's objects if they are suitable for the task. Since Mockito doesn't use files for configuration, its impact isn't the two orders of magnitude of using Spring and XML configuration files.
For some great background on WHY writing FAST and Furious micro tests is so important, listen to the short, to the point, and entertaining Testing Pyramid series (episodes 1 through 8) on the Agile Thoughts podcast.
Episodes 1 through 8

References

How Spring Inversion of Control works

http://docs.spring.io/autorepo/docs/spring/3.2.x/spring-framework-reference/html/beans.html
Using Spring 4 for Dependency Injection and why you'd want a dependency injection framework.
https://www.youtube.com/watch?v=6F3Cv1a7G0w

Using Spring in a standalone app

http://stackoverflow.com/questions/3659720/using-spring-3-autowire-in-a-standalone-java-application?lq=1

Spring's JUnitTestRunner

http://docs.spring.io/autorepo/docs/spring-framework/3.2.6.RELEASE/javadoc-api/org/springframework/test/context/junit4/SpringJUnit4ClassRunner.html

http://lstierneyltd.com/blog/development/examples/unit-testing-spring-apps-with-runwithspringjunit4classrunner-class/

SpringTestUtil

http://mrhaki.blogspot.com/2010/09/private-spring-dependency-injections.html
http://stackoverflow.com/questions/6840939/what-is-the-best-way-to-inject-mocked-spring-autowired-dependencies-from-a-unit

Spring Configuration Java API

http://www.nurkiewicz.com/2011/01/spring-framework-without-xml-at-all.html
Application Contexts to use with the above:
http://docs.spring.io/autorepo/docs/spring/4.1.1.RELEASE/javadoc-api/org/springframework/context/annotation/Configuration.html

Common gotcha's:

Most of the examples on the internet have people making "top level" java classes annotated with @Configuration. This is not a good model for unit tests where the unit test should (as much as is practical) be a simple record for setting up a unit test. It's really better to use an inner class. If you see the below exception regarding your inner @Configuration class, declare it "static" as I did in the code example.
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'springConfigurationAPITest.ContextConfigurationAsInnerClass': Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [foo.SpringConfigurationAPITest$ContextConfigurationAsInnerClass$$EnhancerBySpringCGLIB$$a473a0f4]: No default constructor found; nested exception is java.lang.NoSuchMethodException: foo.SpringConfigurationAPITest$ContextConfigurationAsInnerClass$$EnhancerBySpringCGLIB$$a473a0f4.

About static inner classes:

http://www.geeksforgeeks.org/static-class-in-java/

Testing Triangle (or incorrectly referred to as Testing Pyramid :-)

Friday, October 16, 2015

Test Driven Development Environment for Javascript

Episodes 1-8

Even though JS frequents the GUI slice of an architecture diagram, there is ample functionality that can be unit tested. (For an overview of the testing pyramid, Agile Thoughts podcast has a nice overview on this topic.) The Javascript environment has a a rich history of unit testing tools.  JSUnit is the earliest that I know and was part of the initial wave of xUnit test frameworks in early 2000.  Due to the explosion of xJS frameworks in the last three years, it's time to update knowledge of what tools to use for doing TDD in Javascript.

The tool chains I evaluated were: NodeJS + Karma + Jasmine versus NodeJS + Karma + Mocha + Chai + Sinon.

Here is what they do:
NodeJS is a javascript runtime environment which will run our test tools.
Karma enables pushing our tests into different browsers and automated test launching.
Jasmine versus Mocha + Chai are two choices for test libraries for organizing our tests and give us ways to build assertions.
Jasmin versus Sinon are choices for Mocking

Take a look at the picture below and you'll see the same unit test expressed in three different was.
Jasmine
The above is a nice incremental improvement on typical xUnit with the "toBe."

Mocha and using Chai's "expect"

Using Chai's expect is a bit better than Jasmine's as it allows building of "chains of purpose."

Mocha and using Chai "Should"
I wanted to use Jasmine since it included a lot of functionality as opposed to installing Mocha + Chai + Sinon.  But Chai's "should" is really superior as it prominently shows what is being tested (translate in this case) and gets to the point about what's expected.  Notice how you're less likely to develop "parenthesis blindness."  Here is a good overview of mocha, chai, sinon.  Let's talk about how to put these tools pulled together into an environment.

Install and setup NodeJS, Mocha, Chai, Sinon

1) https://nodejs.org/en/

Install a javascript runtime and package manager.  We'll use NodeJs's package manager to install the remaining tools.  NodeJs's runtime will be used to operate our tools, which are also written in javascript, on our workstation.  Make a work directory to install your javascript test tools.  From this location, you'll configure the test tools to find your source code.

2) Karma and Friends

I opted for Karma to Javascript code in assortment of browsers in order to execute the tests in the browser environments.  Karma will do all this automatically by running a server to controll those browser environments in NodeJs.

Install the Karma cross browser execution framework:
npm install karma --save-dev
npm -g install karma-cli
"-g" is used to do a "global" install, meaning get class paths setup so you can conveniently execute it.

The steps at http://attackofzach.com/setting-up-a-project-using-karma-with-mocha-and-chai/ are pretty close but miss on the dependencies as they've changed since authored, and "npm init" is unnecessary.  So here is what to do:
npm install X --save-dev, where X =>{mocha, karma-mocha, chai, karma-chai, sinon, karma-sinon, karma-chrome-launcher}
Said another way:
npm install mocha  --save-dev
npm install karma-mocha  --save-dev
npm install chai --save-dev 
npm install karma-chai --save-dev
npm install sinon --save-dev   
npm install karma-sinon --save-dev
npm install karma-chrome-launcher --save-dev

(If after doing a "npm install... "and there's a warning about not installing dependencies, respond by installing those dependencies explicitly as ordered to by the computer.)

3) Initialize karma

karma init
"Karma init" will interrogate about the below to generate a boilerplate karma.conf.js.  You'll want to tell it the following:

  • select mocha test framework
  • Add Require.js which we'll use for loading dependencies.
  • Select what browser(s) you want to test against.
  • For location of source files, I used the below. After you're up and running and have written a few tests, you'll want to change the source code settings to point at your code under source control:
    • js/*.js
    • test/*.js
    • lib/*.js 
    • Exclude js/main.js if you have a main.js so that your application under tests doesn't get control of the javascript boot loader.  You want your tests to be loaded and executed rather than your application, right?
  • Accept the defaults for the rest.

4) example karma.conf.js

My file looks like the below.  Note especially the frameworks setting as you'll need to add chai.  If you messed up during the init interrogation, then you can correct it by hand.
Check karma.conf.js to see that you've got the correct file filtering setup.  As of today, the Windows install of Karma does the wrong thing and filters out all your tests.  You want included:true.
 // list of files / patterns to load in the browser
    files: [
      {pattern: 'test/*.js', included: true},
      {pattern: 'lib/*.js', included: true},
      {pattern: 'js/*.js', included: true}
    ],
Here is my entire config file on Windows.
module.exports = function(config) {
  config.set({
    // base path that will be used to resolve all patterns (eg. files, exclude)
    basePath: '',
    // frameworks to use
    // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
    frameworks: ['mocha', 'sinon-chai'],
    // list of files / patterns to load in the browser
    files: [
      {pattern: 'test/*.js', included: true},
      {pattern: 'lib/*.js', included: true},
      {pattern: 'js/*.js', included: true}
    ],
    // list of files to exclude
    exclude: [
      'js/main.js'
    ],
    // preprocess matching files before serving them to the browser
    // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
    preprocessors: {
    },
    // test results reporter to use
    // possible values: 'dots', 'progress'
    // available reporters: https://npmjs.org/browse/keyword/karma-reporter
    reporters: ['progress'],
    // web server port
    port: 9876,
    // enable / disable colors in the output (reporters and logs)
    colors: true,
    // level of logging
    // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
    logLevel: config.LOG_INFO,
    // enable / disable watching file and executing tests whenever any file changes
    autoWatch: true,
    // start these browsers
    // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
    browsers: ['Chrome'],
    // Continuous Integration mode
    // if true, Karma captures browsers, runs the tests and exits
    singleRun: false
  })
}

Test the environment

Type: $karma start
Hopefully you see this:
If you read the messages carefully, you'll see that it didn't find any tests to executed.

If "karma start" fails, then the karma.conf.js likely has an error.  Read the message and see if you can figure out what it's asking for, then use "npm install" to install what's missing or fix the problem in karma.conf.js.  If you get no error, it just runs the karma process.

If Karma is running, you'll see a browser window launched. This is what Karmar does: it uses a NodJS server client and server (one that can access you tests in the configured test directory, and one in the browser(s) in which you want to execute the unit test.


Lets write a test:

describe("This test suite will fail", function() {
   it('should fail', function() {
         expect(false).to.be.false;
    expect(true).to.not.be.true;
   });
});
As soon as the test is saved, the part of Karma that is watching your test directory files for changes, will grab that test and ship it to the browser(s) it's controlling.  After running the test on the browser(s), it returns a report to the Karma controlling running in your DOS prompt and will write out the results.  In the DOS prompt where you launched Karma, you should see something like this:

What's Karma, What's Mocha, What's Chai?

Karma runs in Nod.js and looks for and executes test ".js" files.  It loads the .js file and executes any "describe(...)."  Mocha, the "test runner" handles reporting results.  In the test above, Mocha is called via the "it(...)."  Chai is the library for investigating test results using "expect(...)" among many other library calls.

Mocking

Karma-chai-Sinon plugin is a good way to get everything you needed installed in fewer step. I'll update the above with this improvement. Then I'll put a nice mocking tutorial here. 

Tips

Karma tips: http://www.methodsandtools.com/tools/karma.php

Trouble shooting

When karma executes against a changed test, it returns Executed 0 of 0 ERROR

This happen because my karma.conf.js had, in the FILES section, had "exclude=true." This problem is also documented at: https://github.com/karma-runner/karma/issues/713

After "karma start" returns: Delaying execution, these browsers are not ready:

Recently, this happened when I tried to use RequireJS as a framework in my karma.conf.  When I removed this, it worked.  I don't need RequireJS.  Lots of people do.  I'm sure there is a way to get this to work.  Please comment if you know how to fix that problem.

Thursday, April 9, 2015

Know thy Most Productive Mode (part 2 of Having Great Standups)

When working with new Scrum teams, team members often don't recognize what is impeding them because even though something is obvious to me, to them it's business as usual.

Working with these new teams, I've stumbled upon a way to structure their thinking around this retrospection: Identifying the Most Productive Mode. Once the team knows what their most productive mode is, they can see what is impeding their ability to reach that state. Once they "see it" then they can do something about it such as bringing it up at standup.


MOST PRODUCTIVE MODE

Teams under stress will feel that cutting out some of their new development practices will somehow make them go faster. This is a natural reaction to fear--going back to the old way. This is why it's so important for each team member to intellectualize (know) why they are doing the practices and how *not* doing them will slow them down.

It's better to deliver as much code as you can with your new good practices and communicate what is keeping you from reaching your most productive mode:

Characteristics of productivity
Focus-- focus on one task at a time and getting the artifacts finished and checked in before dealing with distractions (email, phone, meetings, lunch break). Do this work with another (pair programming) who can bring immediate feedback, mentorship, and different perspective

Quick Feedback-- create feedback loops so you know ASAP that something is right or wrong:
writing tests and code in baby-steps (Test Driven Development);
executing the unit tests at least once every fifteen minutes and recognizing you've bitten off too much to do in one step when you can't (a coverage report is a medium-slow feedback mechanism, but will be a constant one once we get it fixed so it gives you feedback every time you run the tests in JUnit); and
getting feedback on the product owner as soon as a story is done.

When something gets in the way of you reaching your most productive modes, something is impeding you. (Examples: the unit tests take more than 30 seconds to run, the IDE is difficult to setup into a productive state, it's hard to setup a pairing station, people/email/IM keep taking you out of your most productive state, you've been pairing too long and you want to disagree with everything your partner is saying so you need a break, you come into work too tired to interact with people because you were up late developing code at home/work.) We're human, sometimes these can't be avoided. If most of the time you can't reach your most productive mode, then something is wrong.

PREPARATION FOR YOUR MOST PRODUCTIVE MODE:
Clarity--
* You should know what is on the sprint backlog. If you're confused about this, then something is dearly wrong! During standup, if you can't see how what someone is saying maps back to the backlog, then ask. If they don't know, then solve the problem: stop working on stuff that's not on the backlog.
* Get those definitions of done for a Sprint and a Release in a clear, easy to understand state. If it has 20 separate items and you need a lawyer, it doesn't help with clarity.
* You should know your velocity by sprint planning for the next sprint.
* At standup, the team members should all understand at some level what each person is saying. If you don't have a clue, say "what's that about?" so you know. Sometimes you'll realize it isn't helpful for person A to recount a blow-by-blow of a meeting about some other concern that has little to no impact to your project. In that case, ask them to parking lot it or follow up after standup with the one other person they are talking to.
* Standups > 15 minutes should be an anomaly. If they aren't then something is wrong. Keep standup focused on the team talking to each other about the three questions. Make it clear when standup is over and your going to shift into something else so people can leave or understand what you are about to go into is a hallway meeting. Otherwise, standups turn into 45 minute hallway meetings every day. Immediately raise impediments during standup about why standup is going too long (trying to do release planning, trying to solve a problem by falling into a 5 minute discussion).
* Use the fist of five. It's simple and effective.
* Come to standup prepared. You may need to make some notes. You may need to meet with a subset of the team *before* standup (the ones which could be a 1-2 on a fist-of-five) and have those long discussions so you have some alignment so standup goes smoother.
* Use big visible charts. If you're team has a problem, then make a chart. If your standups run too long, make a chart of the time. If the team sees the chart, but refuses to adjust despite that, then you need another approach. (Setting an egg timer for two minutes and passing it around to each speaker really works because it provides a quick feedback mechanism). By day three most of them will have figured out how to communicate effectively in that constraint.)