Test Driven Development Environment for Javascript

Episodes 1-8

Even though JS frequents the GUI slice of an architecture diagram, there is ample functionality that can be unit tested. (For an overview of the testing pyramid, Agile Thoughts podcast has a nice overview on this topic.) The Javascript environment has a a rich history of unit testing tools.  JSUnit is the earliest that I know and was part of the initial wave of xUnit test frameworks in early 2000.  Due to the explosion of xJS frameworks in the last three years, it's time to update knowledge of what tools to use for doing TDD in Javascript.

The tool chains I evaluated were: NodeJS + Karma + Jasmine versus NodeJS + Karma + Mocha + Chai + Sinon.

Here is what they do:
NodeJS is a javascript runtime environment which will run our test tools.
Karma enables pushing our tests into different browsers and automated test launching.
Jasmine versus Mocha + Chai are two choices for test libraries for organizing our tests and give us ways to build assertions.
Jasmin versus Sinon are choices for Mocking

Take a look at the picture below and you'll see the same unit test expressed in three different was.

Jasmine

The above is a nice incremental improvement on typical xUnit with the "toBe."

Mocha and using Chai's "expect"

Using Chai's expect is a bit better than Jasmine's as it allows building of "chains of purpose."

Mocha and using Chai "Should"

I wanted to use Jasmine since it included a lot of functionality as opposed to installing Mocha + Chai + Sinon.  But Chai's "should" is really superior as it prominently shows what is being tested (translate in this case) and gets to the point about what's expected.  Notice how you're less likely to develop "parenthesis blindness."  Here is a good overview of mocha, chai, sinon.  Let's talk about how to put these tools pulled together into an environment.

Install and setup NodeJS, Mocha, Chai, Sinon

1) https://nodejs.org/en/

Install a javascript runtime and package manager.  We'll use NodeJs's package manager to install the remaining tools.  NodeJs's runtime will be used to operate our tools, which are also written in javascript, on our workstation.  Make a work directory to install your javascript test tools.  From this location, you'll configure the test tools to find your source code.

2) Karma and Friends

I opted for Karma to Javascript code in assortment of browsers in order to execute the tests in the browser environments.  Karma will do all this automatically by running a server to controll those browser environments in NodeJs.

Install the Karma cross browser execution framework:

npm install karma --save-dev

npm -g install karma-cli

"-g" is used to do a "global" install, meaning get class paths setup so you can conveniently execute it.

The steps at http://attackofzach.com/setting-up-a-project-using-karma-with-mocha-and-chai/ are pretty close but miss on the dependencies as they've changed since authored, and "npm init" is unnecessary.  So here is what to do:

npm install X --save-dev, where X =>{mocha, karma-mocha, chai, karma-chai, sinon, karma-sinon, karma-chrome-launcher}

Said another way:
npm install mocha  --save-dev
npm install karma-mocha  --save-dev
npm install chai --save-dev 
npm install karma-chai --save-dev
npm install sinon --save-dev   
npm install karma-sinon --save-dev
npm install karma-chrome-launcher --save-dev

(If after doing a "npm install... "and there's a warning about not installing dependencies, respond by installing those dependencies explicitly as ordered to by the computer.)

3) Initialize karma

karma init

"Karma init" will interrogate about the below to generate a boilerplate karma.conf.js.  You'll want to tell it the following:

• select mocha test framework
• Add Require.js which we'll use for loading dependencies.
• Select what browser(s) you want to test against.
• For location of source files, I used the below. After you're up and running and have written a few tests, you'll want to change the source code settings to point at your code under source control:
• js/*.js
• test/*.js
• lib/*.js 
• Exclude js/main.js if you have a main.js so that your application under tests doesn't get control of the javascript boot loader.  You want your tests to be loaded and executed rather than your application, right?
• Accept the defaults for the rest.

4) example karma.conf.js

My file looks like the below.  Note especially the frameworks setting as you'll need to add chai.  If you messed up during the init interrogation, then you can correct it by hand.
Check karma.conf.js to see that you've got the correct file filtering setup.  As of today, the Windows install of Karma does the wrong thing and filters out all your tests.  You want included:true.

 // list of files / patterns to load in the browser
    files: [
      {pattern: 'test/*.js', included: true},
      {pattern: 'lib/*.js', included: true},
      {pattern: 'js/*.js', included: true}
    ],

Here is my entire config file on Windows.

module.exports = function(config) {
  config.set({
    // base path that will be used to resolve all patterns (eg. files, exclude)
    basePath: '',
    // frameworks to use
    // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
    frameworks: ['mocha', 'sinon-chai'],
    // list of files / patterns to load in the browser
    files: [
      {pattern: 'test/*.js', included: true},
      {pattern: 'lib/*.js', included: true},
      {pattern: 'js/*.js', included: true}
    ],
    // list of files to exclude
    exclude: [
      'js/main.js'
    ],
    // preprocess matching files before serving them to the browser
    // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
    preprocessors: {
    },
    // test results reporter to use
    // possible values: 'dots', 'progress'
    // available reporters: https://npmjs.org/browse/keyword/karma-reporter
    reporters: ['progress'],
    // web server port
    port: 9876,
    // enable / disable colors in the output (reporters and logs)
    colors: true,
    // level of logging
    // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
    logLevel: config.LOG_INFO,
    // enable / disable watching file and executing tests whenever any file changes
    autoWatch: true,
    // start these browsers
    // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
    browsers: ['Chrome'],
    // Continuous Integration mode
    // if true, Karma captures browsers, runs the tests and exits
    singleRun: false
  })
}

Test the environment

Type: $karma start

Hopefully you see this:

If you read the messages carefully, you'll see that it didn't find any tests to executed.

If "karma start" fails, then the karma.conf.js likely has an error.  Read the message and see if you can figure out what it's asking for, then use "npm install" to install what's missing or fix the problem in karma.conf.js.  If you get no error, it just runs the karma process.

If Karma is running, you'll see a browser window launched. This is what Karmar does: it uses a NodJS server client and server (one that can access you tests in the configured test directory, and one in the browser(s) in which you want to execute the unit test.

Lets write a test:

describe("This test suite will fail", function() {
   it('should fail', function() {
         expect(false).to.be.false;
    expect(true).to.not.be.true;
   });
});

As soon as the test is saved, the part of Karma that is watching your test directory files for changes, will grab that test and ship it to the browser(s) it's controlling.  After running the test on the browser(s), it returns a report to the Karma controlling running in your DOS prompt and will write out the results.  In the DOS prompt where you launched Karma, you should see something like this:

What's Karma, What's Mocha, What's Chai?

Karma runs in Nod.js and looks for and executes test ".js" files.  It loads the .js file and executes any "describe(...)."  Mocha, the "test runner" handles reporting results.  In the test above, Mocha is called via the "it(...)."  Chai is the library for investigating test results using "expect(...)" among many other library calls.

Mocking

Karma-chai-Sinon plugin is a good way to get everything you needed installed in fewer step. I'll update the above with this improvement. Then I'll put a nice mocking tutorial here. 

Tips

Karma tips: http://www.methodsandtools.com/tools/karma.php

Trouble shooting

When karma executes against a changed test, it returns Executed 0 of 0 ERROR

This happen because my karma.conf.js had, in the FILES section, had "exclude=true." This problem is also documented at: https://github.com/karma-runner/karma/issues/713

After "karma start" returns: Delaying execution, these browsers are not ready:

Recently, this happened when I tried to use RequireJS as a framework in my karma.conf.  When I removed this, it worked.  I don't need RequireJS.  Lots of people do.  I'm sure there is a way to get this to work.  Please comment if you know how to fix that problem.