Wednesday, December 3, 2014

Recommended Behavior Driven Development (BDD) Toolsets

As a consultant who works with many different teams on mostly .Net,  Java and JavaScript projects, let me recommend my favorites.

Cuccumber versus JBehave
I don't have a clear favorite between these two.  The tools between them seems to be about at the same level of maturity (or immaturity).  Both allow you to use JUnit as a test runner.  Both allow Gherkin syntax.  I give an edge to Cucumber in that it needs a little less hand-holding configuration files to get things up and running.

Here is my favorite Cucumber setup which I'm using (as of Mar 2017):

If you just want to download cucumber jars the traditional way:

The above will allow you to develop and execute feature files and BDD automation.  Great for executing using maven and add these external jars to your build so you can build step definitions in Java.  But no sane person would do this without a nice syntax highlighting editor and "open on declaration" goodies plugged into Eclipse.


Natural is a feature file editor that uses code-assist to fill in Given, When, Then, and allows you to jump to the step definition.  Install Natural from the Eclipse Marketplace found in Eclipse at help->Eclipse Marketplace. Natural itself wasn't tagged in the depot correctly so you'll find it by searching for "cucumber," "jbehave," or several other BDD related tags.  Install it.

Test it out by creating a plain text file with a .feature extension.  I find when creating a new file with a ".feature" extension, I need to close the tab and re-open it so Eclipse can hand off editing the .feature file to Natural.

What Natural does for you when editing a feature file:
  • content assistance in choosing Steps, 
  • F3 (open declaration) which shows the Java definition for the step (assuming one exists), 
  • Outline view, 
  • syntax highlighting, and
  • caution marks for Gherkin steps that don't have a Java definition.

Manual installation
Sometime back, when traveling Asia, I needed to do manual installs as the Eclipse Marketplace and Natural repository were timing out.  In those cases I did manual installs.
  1. Using Eclipse Install XText from this location:
  2. Download an Eclipse archive (.zip) of Natural from this location:
  3. In the Eclipse->Install New Software, click "Add..." and this time select "Archive" and select the path to the zip file downloaded in the above step.

But Natural alone won't run your feature tests.  You need to install Cucumber-Eclipse, or as I prefer, run them via "cucumberized" JUnit test cases via @RunWith(Cucumber.class).

I don't recommend this tool as I've never gotten it to work for the last 12 month (2016/2017).  But If you want to give it a go....  As it's not in the Eclipse Marketplace for whatever reason, use help->install new software.  Click Add and add this url:
Select Cucumber-Eclipse and install.

What Cucumber-Ecilpse should do you for you
The context menu will have "Run feature tests" and "Debug feature tests" which should use the Cucumber CLI to execute the selected feature file, showing the test results in the Eclipse Console.  As of 2016/2017, I've not got this feature to work.  I think it can be made to work via fiddling with Runtime configurations.  I've given up on it and execute BDD tests via cucumberized JUnit test cases and JUnit suits.  (Naturally you can do the same with TestNG.) 

Good Organization and Configuration
Ask yourself how many features you'll build this year.  Then ask yourself what those categories look like.  Now go into eclipse and make a heirarchy, something like this: features->, features->.

So sprint by sprint, add your feature files into to that hierarchy and grow the hierarchy as needed.  I suggest representing the hierarchy as a package in the source code alongside the application it's testing.  If your Steps definitions have a strong relationship with their features, then put them alongside the feature files.  If there isn't such a relationship then don't do that.  (People have different feelings about 'global' namespaces of the BDD Steps.)


Most teams hook their JUnit runner to their BDD tests (using JUnit's @RunWith(...).  Put that test class in the features directory and make it responsible for running all your features.

Using the above organization, follow the principle of "keep things that relate, together" and place the step definitions along side the feature files:

If you have global steps definitions (don't worry about this when starting out), then put that library in the features directory.  If you're driving a UI, you'll need a place to put your page objects too.  (Please avoid testing the UI unless you have to.  Also use the emergent design principle and develop your page objects as you need to.)

For .Net, all I ever seem to use is Spec Flow and it works good enough.  Here are some pretty good directions.  (FYI, don't believe him that this will work for Express versions of VS.Net as Microsoft turns off useful things like plug-in installation and debugger .)  The main thing is you need to install two things: SpecFlow libraries, and SpecFlow templates (for editing .feature files).

1 comment:

  1. Intellij has Cucumber integration by default.. much better than Eclipse!