Teaching Cucumber about Boundaries

Developers are trained to avoid using global anything in programs because you're making big commitments to the uncertain future.  Behavior Driven Development (BDD) demands teams and their source of requirements to decide what words mean, and then write test automation code that enforces the meaning.  In this way BDD encourages the definition of words to be used globally in a specific way in requirements.  The problem with Cumber (and all? implementations of Gherkin) is that the only information passed from the feature file to the automation framework is the the usual Gherkin keywords: Given, When, and Then, which isn't the whole picture.  Developers, since they can read the whole feature file, can teach the automation to do the right thing by linking to specific steps definitions designed with the whole situation in mind.  The BDD framework Cucumber, in particular, doesn't make it simple to control linking feature files to specific test automation (known as step definitions) because it's optimized for a flat global namespace between a feature file and the many step definitions, but there are some boundary controls.  Let's look at different levels of boundaries that can be created between your features files with Cucumber-JVM.

The Problem

Take a look at the below two feature files (click for high-res viewing):

Feature File

To implement test automation for these two feature files, a Java programmer (in the case of Cucumber-JVM) must write java code that drives their product to prove it works as the customer expects.  The problem comes up with "Then signoff" because a single Java method cannot figure out how to handle this case.  Although "signoff" is used in both features, it means two different things.  It's a pity that although the feature files have quite a lot of context (Feature title, user story, scenario title, directory location) which actually makes it clear what is meant by "signoff," none of that is available to communicate context when doing test automation in any of the Gherkin BDD frameworks.  The above two features are in two domains different domains: the domain of package delivery and the domain of multi-user systems.  One strategy is to recast the sentence "Then signoff" in one or both of those feature files so this overlap doesn't exist, and this is the first thing to try when faced with this.  In cases where such a collision is using words that are a "fixed part of the domain language" then we may want to *not* have a global namespaces and instead create a boundary (AKA a bounded context in Domain Driven Design terms).

Conceptually global step functions look like this (click pic for high resolution image):

Global Scope Step Definitions in a global 'dictionary'

Where you'll have a lot of Java steps definitions designed around each gherkin statement.  There isn't a direct relationship between a specific feature file and a specific steps definition.  Here the idea is to have many re-usable steps definitions (and an automation library that's used across all the steps definitions).

But perhaps, you want this for fine grain control and let the developer control all aspects of how each step in the feature file is automated (click pic for high resolution image):

Each feature file has it's own namespace for steps definitions

Here, each feature file gets one (or more) java classes to handle the scenarios within the feature file.  The relationship between the steps and the feature file are intimate.  The idea here is to have a very simple steps definition for a specific .feature file, and to push as much code as possible into the automation library so it can be re-used across all the steps definitions.

You can get to this extreme with some easy to do but additional steps.

Global Scope

This one is easiest to get started with as Cucumber is geared to make this easy.  Your test automation project should have package arranged like:

test.pageobjects
test.features
test.features.major_feature_one
test.features.major_feature_one.subfeature
test.features.major_feature_two
...

Create a JUnit execution class at the top of your test automation project like shown.

How everything works is that when launching the JUnit test case, it hands off to Cucumber via the @RunWith, which associates all the subpackage from that point as the namespace to use for matching step definitions.  So by putting the JUnit Test class at test.features makes all of our feature files from test.features and downward, match to any of the step definitions declared from this point on down.  (BTW, there are tools such as Natural that help in editing feature files and finding which step definition is active.)

Here's the big news: Nothing is stopping us from having multiple JUnit Test classes in different peer packages, effectively creating boundaries for how step functions are picked up by feature files.

Package Scope

By putting a Cucumberized JUnit Test case at the top of specific packages, we create separate bounded contexts, a term used by the practice Domain Driven Design, which basically sums up to: put the features and steps for the package delivery in one Java namespace and the features and steps for multiuser in another Java namespace, and then in each package, setup a Cucumberized JUnit Test case.  In this way, boundaries between these two different domains are made.

It looks like this:

Package organization

test.features.packagedeliver contains one Cucumberized JUnit Test case, many steps definitions, and many feature files.  test.features.processcontrol contains one Cucumberized JUnit Test case, many steps definitions, and many feature files.

Thusly its possible to have a "Then signoff" defined in a steps definition in the processcontrol context and a different definition for "Then signoff" by a steps definition in the packagedelivery context.  If another "Then signoff" steps definition is added to say, test.features.processcontrol.increase_system_load, Cucumber will put the brakes on and complain that "Then signoff" is ambiguous as there are multiple definitions in its bounded context.

But what if that isn't enough?  Can we make every feature file have its own context?  You bet.

Feature File Scope

If you need to do this, it's a bit of drudge work but it's not complex: for every feature file you'll need to add a Cucumberized JUnit test case, and due to a lovely quirk of Cucumber, drop that feature file's step definitions in their own package. #annoying

Conceptually that looks like this:

Literally, it looks like this (click image to see what's going on):

Two feature files in the test.features.processcontrol package want different step definitions for "Then signoff"

Now each Cucumberized JUnit test case defines the mapping from a specific feature file and a package to where to load its step definitions.  Because this strategy will end up producing a bunch of different JUnit test cases, you'll want to add a JUnit test suite so you can easily execute all your feature tests with one click.

Conclusions

You've got the tools to do whatever you want with Cucumber.  Treating every feature file as having its own scope is the ultimate of control but do you really need that?  The OCD control freak inside does want that and if so, better to find another framework to support that then Cucumber.  Defining boundaries using packages will keep the DDD people happy and would work in a pinch when you need it.  Cucumber is courageous in pushing people to build global dictionaries.  I've only heard of great drama coming out of such striving though people have successfully done it.  It seems unfair to having our language in feature files be global context (easy to do as we are humans) and then forcing the automation (a computer program) to discover the context when Gherkin doesn't pass along all the context from the feature files.

I'm happy to hear from you if you've a story to share: Twitter:@LancerKind, LancerKind@gmail.com.

References

• Learn style tips for writing your Gherkin with Behavior Driven Development Elements of Style.
• CucumberOptions
• CucumberOptions Javadoc
• JUnit 4 Test Suites

Recommended Behavior Driven Development (BDD) Toolsets

As a consultant who works with many different teams on mostly .Net,  Java and JavaScript projects, let me recommend my favorites.

Cuccumber versus JBehave
I don't have a clear favorite between these two.  The tools between them seems to be about at the same level of maturity (or immaturity).  Both allow you to use JUnit as a test runner.  Both allow Gherkin syntax.  I give an edge to Cucumber in that it needs a little less hand-holding configuration files to get things up and running.

Cucumber
Here is my favorite Cucumber setup which I'm using (as of Mar 2017):

If you just want to download cucumber jars the traditional way:

http://repo1.maven.org/maven2/info/cukes/cucumber-java/1.2.5/
http://repo1.maven.org/maven2/info/cukes/cucumber-junit/1.2.5/
http://repo1.maven.org/maven2/info/cukes/cucumber-core/1.2.5/
http://repo1.maven.org/maven2/info/cukes/gherkin/2.12.2/
http://repo1.maven.org/maven2/info/cukes/cucumber-jvm-deps/1.0.3/
http://repo1.maven.org/maven2/info/cukes/gherkin-jvm-deps/1.0.2/ http://repo1.maven.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar  http://repo1.maven.org/maven2/junit/junit/4.12/junit-4.12.jar

The above will allow you to develop and execute feature files and BDD automation.  Great for executing using maven and add these external jars to your build so you can build step definitions in Java.  But no sane person would do this without a nice syntax highlighting editor and "open on declaration" goodies plugged into Eclipse.

Natural

Natural is a feature file editor that uses code-assist to fill in Given, When, Then, and allows you to jump to the step definition.  Install Natural from the Eclipse Marketplace found in Eclipse at help->Eclipse Marketplace. Natural itself wasn't tagged in the depot correctly so you'll find it by searching for "cucumber," "jbehave," or several other BDD related tags.  Install it.

Test it out by creating a plain text file with a .feature extension.  I find when creating a new file with a ".feature" extension, I need to close the tab and re-open it so Eclipse can hand off editing the .feature file to Natural.

What Natural does for you when editing a feature file:

• content assistance in choosing Steps, 
• F3 (open declaration) which shows the Java definition for the step (assuming one exists), 
• Outline view, 
• syntax highlighting, and
• caution marks for Gherkin steps that don't have a Java definition.

Manual installation

Sometime back, when traveling Asia, I needed to do manual installs as the Eclipse Marketplace and Natural repository were timing out.  In those cases I did manual installs.

https://github.com/rlogiacco/Natural/wiki/Installation-Guide

1. Using Eclipse Install XText from this location: http://download.eclipse.org/modeling/tmf/xtext/updates/composite/releases/
2. Download an Eclipse archive (.zip) of Natural from this location: https://github.com/rlogiacco/Natural/releases
3. In the Eclipse->Install New Software, click "Add..." and this time select "Archive" and select the path to the zip file downloaded in the above step.

But Natural alone won't run your feature tests.  You need to install Cucumber-Eclipse, or as I prefer, run them via "cucumberized" JUnit test cases via @RunWith(Cucumber.class).

Cucumber-Eclipse
I don't recommend this tool as I've never gotten it to work for the last 12 month (2016/2017).  But If you want to give it a go....  As it's not in the Eclipse Marketplace for whatever reason, use help->install new software.  Click Add and add this url:
Select Cucumber-Eclipse and install.

What Cucumber-Ecilpse should do you for you
The context menu will have "Run feature tests" and "Debug feature tests" which should use the Cucumber CLI to execute the selected feature file, showing the test results in the Eclipse Console.  As of 2016/2017, I've not got this feature to work.  I think it can be made to work via fiddling with Runtime configurations.  I've given up on it and execute BDD tests via cucumberized JUnit test cases and JUnit suits.  (Naturally you can do the same with TestNG.) 

Good Organization and Configuration
Ask yourself how many features you'll build this year.  Then ask yourself what those categories look like.  Now go into eclipse and make a heirarchy, something like this: features->, features->.

So sprint by sprint, add your feature files into to that hierarchy and grow the hierarchy as needed.  I suggest representing the hierarchy as a package in the source code alongside the application it's testing.  If your Steps definitions have a strong relationship with their features, then put them alongside the feature files.  If there isn't such a relationship then don't do that.  (People have different feelings about 'global' namespaces of the BDD Steps.)

source/java/com/my/awesome/app
source/java/com/my/features

Most teams hook their JUnit runner to their BDD tests (using JUnit's @RunWith(...).  Put that test class in the features directory and make it responsible for running all your features.
source/java/com/my/features/RunFeatureTests.java

Using the above organization, follow the principle of "keep things that relate, together" and place the step definitions along side the feature files:
source/java/com/my/features/Foo/Foo.feature
source/java/com/my/features/Foo/FooSteps.java
source/java/com/my/features/ShoppingCart/Purchase.feature
source/java/com/my/features/ShoppingCart/PurchaseSteps.java
source/java/com/my/features/ShoppingCart/TakePayPal.feature
source/java/com/my/features/ShoppingCart/TakePayPalSteps.java
source/java/com/my/features/ShoppingCart/TakeVisa.feature
source/java/com/my/features/ShoppingCart/TakeVisaSteps.java

If you have global steps definitions (don't worry about this when starting out), then put that library in the features directory.  If you're driving a UI, you'll need a place to put your page objects too.  (Please avoid testing the UI unless you have to.  Also use the emergent design principle and develop your page objects as you need to.)
source/java/com/my/features/global_steps/*Steps.java
source/java/com/my/features/page_objects/Login.java
source/java/com/my/features/page_objects/ProductPage.java

SpecFlow
For .Net, all I ever seem to use is Spec Flow and it works good enough.  Here are some pretty good directions.  (FYI, don't believe him that this will work for Express versions of VS.Net as Microsoft turns off useful things like plug-in installation and debugger .)  The main thing is you need to install two things: SpecFlow libraries, and SpecFlow templates (for editing .feature files).

Don't forget to AGILE your Test Plan when transitioning from Waterfall to Scrum

Goal: Repeatable quality through automated tests!

But we need people to develop them. Traditional organizations call these people testers.


Tester-> Automated Tests!

In trad. organizations, testers rarely do this because of the traditions of Waterfall.

Tester-> Test Plan -> Automated Tests!

And this is where the trouble starts. Due to the divide-and-conquer and handoff approach of Waterfall, it made sense to split the role of software development into programmer and tester (I'll not debate the pros and cons of doing this, Agile comic SCRUM NOIR—A Silo to Hell! does a nice job.) but in the Agile context, these testers are challenged in integrating their work with a Scrum team: they can't write automated tests until late in the Sprint. This is a big problem when moving to a practice such as Acceptance Test Driven Development where automated test development starts on the first day of the Sprint.

Iterative Testing Problems:

1. Testers don't plan together with the developers during Sprint planning.
2. Testers never check-in an automated test on the first day of the Sprint.
3. Test Planning isn't done continuously (a little bit here, a little bit there, and then executed, then repeat) but instead is an upfront event that takes up 50% or more of the Sprint.
4. Testers complain they don't have enough information to get started.

The above happen in organizations in transition to Agile where people carry old habits and behaviors to their new roles, even if the behaviors aren't optimal. The Waterfall process was planning heavy and everyone (testers, developers, ...) were encouraged to spend a few months writing documents and reviewing them, creating a false sense that we "planned well." (The security was demonstratively false since the plans between waterfall phases always changed.) Since test and development are often in separate organizations, they coordinate around events set on calendars to deliver artifacts: Test Plans (among other things). The tester is responsible for the Test Plan and had dependencies on pretty near everything in order to produce it (development, architecture, business requirements, ...).

This habit is antithetical to the Scrum process since a Scrum team is doing incremental delivery of product and doesn't know until Sprint Planning what will be done during the next Sprint. This forces the Testers to produce Test Plans under immense stress if they preserve old habits.

This results in:

1. Testers hating Agile.
2. The Test part of the organization hates Agile and starts working against change initiatives.
3. Testers complain that they have to drive everything because the tester's need becomes the event rather than a scheduled milestone.
4. Many preconditions to creating a Test Plan: Testers demanding a lot of input documentation from other roles (design specs, arch specs, use cases, business case studies, hardware diagrams, call-flow, process flow,...) before they feel they can complete their Test Plan.
5. Equating Incremental testing with incomplete testing: Tester's consider their Test Plans incomplete because they are only supposed to think of testing the functionality they are delivering that Sprint rather than for the entire release.

Points 1 and 2 are effects which are mitigated by showing them how to be successful. Point 3 is discomfort about the culture change from Waterfall, where the process drives the events, into an Agile process where individuals drive the process. Testers and those who support Test Plan creation get used to it after a few Sprints. Point 4 is a "process over working software" habit from Waterfall where if testers (or their management) feels rushed, they can buy more time by demanding more process and documentation from others as dependencies. Point 5 requires another change in thinking which will happen after doing a few Sprints, witnessing that small high quality subfeatures will sum up to a large high quality feature.

If you change your process without changing how you do things then nothing's going to change (except for the labels). So points 3 and 5 are natural to "storming" (Tuckman model) and must be allowed to happen. Once we get to later steps, "norming" and "performing," and do it say in 3 Sprints, the dangers of the change rejection (points 1 and 2) will go away. Point 4 is a deep culture change which takes time until people to understand the Agile Manifesto Values and Principles, namely, Working Software is the primary measure of progress (automated tests are working software) rather than process checkpoints, milestones, and documentation completion.

No matter what, an organization in transition will have storming and that is healthy if conflicts are allowed to be exposed AND resolved (healthy stress/conflict drives change). To help "storm well," here are some activities.

How to Agile your Test Plan

The Scrum team containing programmers and testers must re-invent how they work together and be open to new ideas and ditch some bad ones (the heavy weight, comprehensive test plan). Each story needs it's own Test Plan. Look at solving this problem at two levels: Test Plan format and timing.

Making the format you already have, lightweight (format)

Rather than introduce any additional process/methodologies, make what you already do lightweight. If your usual Test Plans are 10-20 pages for features completed during a multi-month release, try and do the same plan but on one sticky for one User Story.

How can one stick get results as good as multipage test plans? We're leveraging the fact (and would be foolish not to) that within a few days we'll be building small features, so we don't need to document so heavily.

Agile Context (and why Agile development really is different than Waterfall):

• a lot of information is successfully retained in our tacit knowledge since we are working on a small team that interacts daily,
• we are dedicated to one sprint backlog and become experts in its execution,
• we'll start acting on our plan within days,
• we're working on sub features and only need to test the sub features we are doing this Sprint, and
• each User Story is independent of the others so each must be tested independently
• the majority (if not all) of the tests will be automated and checked in and will be our best documentation and reporting system

If you argue that your automated tests cannot act as your documentation, then you'd better work on your test design because you've got a problem that must be resolved.

Limber the mind

To change the results of your work you need to change yourself. A lot of what stops us are habits learned during the Waterfall context which need to be shed so we can develop sensible ones for the Agile context. This problem affects anyone when changing work contexts: I'm a science fiction author who spends hours getting my words right who sometimes spends hours getting an email right. This is complete waste for a one-off email. I had to learn that when I write technical planning documents or emails to reel myself back since prose isn't necessary. It took conscious effort to shift gears and doing pair work with another helped.

To do so I had to:

• decide it was important to change,
• be open to new ideas, in fact, be open to trying something so crazy it couldn't possibly work and then go for it.

James Whittaker of Google has a great facilitation style called The 10 Minute Test Plan which breaks down mental barriers that prevent writing a good and quick Test Plan.

Acceptance Criteria (format)

Acceptance Criteria are the most lightweight and commonly used of all Test Plan formats: a simple bullet list of what the application should do before the PO will accept the User Story.

Acceptance Criteria can also take the format of wire frames, call flows, non functional requirements, ....

Each User Story should have acceptance criteria before going into Sprint Planning. More can be added at any time, but it's important to have a rough list before Sprint Planning or the meeting will become overloaded and slow.

4D Analysis (format)

4D Analysis was introduced to one of my teams by friend and fellow coach XuYi. 4D adds three dimensions of additional analysis to simply using Acceptance Criteria. Adding more analysis isn't necessarily a good thing because that's what got us the 20 page Test Plan. However 4D is still one page and maybe your team'll find working on the first 3Ds helpful before getting to the Acceptance Criteria. The 4D analysis is attached behind each user story.

The Process dimension is further broken down into 3 types:

• User workflow (how it works from a User's perspective)
• Business process (how it works from a Business Analysis perspective)
• Technical flow (system engineer or architecture view)

The idea is that for an individual User Story you'll need one of the above process types and rarely, you'll need more than one.

Strangely enough, the feedback I've received from teams doing 4D Analysis is that despite the single sheet of paper, the amount of analysis is greater than they were used to from their traditional multipage test plan. Customize the 4D analysis to fit your team's needs.

BDD (process, format, and technology)

Get your PO adding Behavior Driven Development scenarios to your User Stories. Or at a minimum work with your PO and get them written down. Good BDD tests make wonderful test plans AND automated Tests AND traceability documents!

Given there exists bad entries in [playlist]
When trying to play this playlist
Then remove invalid playlist entries
examples:
|playlist|
|Never Played|
|My Top Rated|
|Whole Library|

More BDD articles:Well written BEHAVIOR Driven Development Scenarios, BDD Practices that Maximize Team Collaboration and Reduce Risk.

Timing

When should Test Plans be created? The two strategies are:

• Create Test plans during the Sprint
• Create Test plans before Sprint Planning.

The testers and developers include in their estimates made during Sprint planning the effort to create Test Plans and implement tests. But they need to learn to move fast and test like a jazz band rather than a symphony which has a conductor that has organized a lot of planning and process.

Lightweight test plans can be created before the Sprint by leveraging Backlog Grooming. Here's an agenda that broke backlog grooming up into the following: 30 minute kickoff, offline (outside of meeting) collaborative work, 1 hour grooming meeting.

Here's an example calendar for a three week Sprint:

MTWTF MTWTF MTWTF
         ^- Kickoff Preperation for Grooming
            ^- Groom results for 1 hour

Constraints:

• a team spends no more than 10% of a Sprint preparing for the next Sprint
• other than Sprint Demo and Retro, avoid meetings on last 2 days during Sprint crunch time
• have Grooming far enough in advance of Sprint Planning to fix problems uncovered in Grooming

Kickoff Agenda (team, PO, SMEs in attendance) < 30 minutes

• During meeting
• PO brings proposed Sprint Backlog
• Team members select stories to prep for grooming and are encouraged to collaborate with other team members
• Decide what stories need SME deliverables/support and make that status visible
• Offline, to be finished by Grooming Meeting start (Monday 2PM)
• Fill out prep document
• User Scenarios {Functional flow, User Experience behavior} (done by: programmer or tester)
• Y/N needs SME deliverables (architecture view, etc.)  (roles: tester, SME)
• Refine User Story Acceptance Criteria (roles: everyone)
• Update Assumptions (roles: everyone)
• PO and SME will visit each team member (individuals and interactions per the Agile Manifesto) before Grooming Meeting and see how they can help.
• ScrumMaster will make sure the process is successful and that everyone comes to the Backlog Grooming meeting prepared.

Grooming Meeting 1hour, review the results of the prep work as a group.

A team reviewing User Stories and grooming prep doc. They are standing up during the action moments for maximum collaboration. Stand up meetings are 30% faster than their "sit down" counterparts.

During Sprint Planning, the team will adjust further and are expected to refine further even during the Sprint.

Summary

Goal: Automated Tests!

But we need people to develop them.

Ideal:
Tester->Automated Tests!

Usually, teams do at least a lightweight Test Plan: 
Tester-> Acceptance Criteria-> Automated Tests
Tester-> BDD-> Automated Tests
Tester-> 4D Analysis -> Acceptance Criteria -> Automated Tests 

Some teams need more analysis. Find a way to do it using only 10% of your current Sprint to plan for you next Sprint:
Tester->Test Plan->{Acceptance Criteria, BDD, 4D Analysis}-> Automated Tests

Remember you're doing Test Plans to have automated test cases to defend your product from regression. Your Test Plans should be designed to serve this purpose. Having large Test Plan documents was never the goal. If you can produce 2-6 automated test cases from a one page Test Plan, and spend no more than 10% of your Sprint doing Test Plans, then you're on the right track.

Well written BEHAVIOR Driven Development Scenarios

In my work with coaching teams in doing Behavior Driven Development (BDD), it's been non-trivial (doable, but takes time) to teach what is a GOOD BDD example.  I'm finding that teams that "self start" in BDD are creating UI driven scenarios rather than behavior.  They end up creating scenarios such as the following.

(for an iTunes plugin that culls out invalid song entries in a playlist)
(THESE ARE EXAMPLES OF BAD BDD SO DON'T DO THIS!)
Given iTunes is launched and there exists bad entries in playlist Never Played
When user selects Music and playlist Never Played
Then show dialog listing invalid songs

Given showing user listing invalid songs
When user clicks OK
Then remove invalid playlist entries


There are bad smells in these scenarios:

1. the end user is mentioned
2. contains User Interface (UI) language (dialogs, selecting, clicking)
3. the scenario language contains details that only an experienced user of iTunes can follow it

While the nature of smells isn't that these things are forbidden, but that it's best to marshal your energy in expunging them as much as possible, especially if you're new to BDD.

Although UI scenarios will make most engineers happy because it's clear what the UI is supposed to do and will provide usable system tests, you will create the following problems:

1. High maintenance--Too much presentation language means the tests need to be updated when the presentation changes even though the behavior hasn't changed. 
2. Implies the UI design is finished--By virtue of writing the scenario in presentation language (rather than behavior), the presentation design is now fixed in the reader's mind.  The engineer may not wish to challenge this even if they see a better way.
3. The BDD test will be a UI test--If all the BDD scenarios are written in UI language, then all the testing will be through the UI.  Remember the Test pyramid!  You want as few UI tests as possible because UI tests are inflexible, slow, the most expensive to maintain, and prone to false positives.  In this case, the team is building the plugin, not iTunes so why should they have a Given/When/Then that also describes code delivered by the iTunes team?
4. The BDD tests will always be slow and slow is less valuable--If all the tests are UI tests, they will run slower than those designed to only test behavior.  A test failure that tells the team that something happened in the last fifteen minutes that caused a regression will be trivial to correct (say less than 30 minutes).  A test falure that tells the team that something happened in the last 24 hours is going to require using a debugger to figure out where the bug is and then figure out which change caused the bug (likely will be an hour or several hours).
5. Discussions are anchored to UI--Teams that can understand their work at the behavior level will be able to discover problems easier without having their thinking distracted by what's happening in the UI domain and clutter communication with others.

The above scenarios should be rewritten by focusing on the behavior:
(for an iTunes plugin that culls out invalid song entries in a playlist)

Given there exists bad entries in playlist Never Played
When trying to play this playlist
Then remove invalid playlist entries

This is something the user will pay for!  They don't care about the clicks or if the pointer is used at all!  They want the bad entries in their playlist removed!  If you have POs create such scenarios, they can easily communicate and debate them to product marketing about what behaviors the user wants without needing someone to translate what is the result of UI manipulations.  Give a Scrum team this scenario during sprint planning, they're going to need to discusses possible UIs to allow this behavior to happen.  But they don't need to choose right now anymore than they need to decide how many Classes they need to design and what are the public methods/attributes.  They just need to know if they can do any of the options during the Sprint.  When sprinting, they *will* need to decide on a UI implementation and they will need to implement the test driver which *may* need to drive the UI, or even better, trust that iTunes does it's job to hand off to their plugin and the team only tests that their plugin causes the behavior to happen.  (They likely will need at least *one* end-to-end UI test that ensures iTunes does hand-off to their plugin.)

By the way, for the BDD experienced in the crowd, it would be a great idea to parametrize the scenario to handle many/all playlists when the team is needing to add features for many play lists:

Given there exists bad entries in [playlist]
When trying to play this playlist
Then remove invalid playlist entries
examples:
|playlist|
|Never Played|
|My Top Rated|
|Whole Library|
...

Check out the other BDD resources at this blog such as:

• the section on GWT smells (GWT = Given When Then) in: BDD Practices that Maximize Team Collaboration and Reduce Risk

Other Resources:

I highly recommend reading the following which will NOT teach you how to write the code but teaches you how to think behavior and why it's important to do so (available in paper and Kindle):



Here is a great BDD tutorial if you are into music (guitar in particular) http://www.ryangreenhall.com/articles/bdd-by-example.html

BDD Practices that Maximize Team Collaboration and Reduce Risk

Maximizing team collaboration and reducing risk

Guiding Principles: KISS – Keep it simple silly & Incremental delivery

I’ll think about the future but only implement what I need this sprint, this day, this hour. I do that so I can start my implementation at the simplest level (but not too simple) and grow the feature minute by minute, hour by hour.

Because I work this way, I can check-in often. This enables other team members to have visibility on what I’m doing. This allows for cheaper integration (merges) for the team and myself. It allows me to get the latest changes from the source tree so I can receive new work from the team (and get visibility into what they’re doing). Checking in often reduces risk and increases code reuse. I can’t reuse my team member’s latest new shared libraries if it isn’t checked in. If I get pulled away from the work (personal or fire drill), another team member (who has some insight into my work because they’ve been seeing my many changes coming into their development environment, who has heard about what I’m doing at a high level during standup, who is collocated and has heard me working with my pair, who maybe has paired with me) can finish the task or story so the team is successful.

Because the team is checking in often, we are releasing our best design efforts and reducing latency for others to review and use our work.

Test Strategies

Guiding Principles: Tests run independent of each other, give timely feedback, are maintainable, deterministic, and test a feature once.

It’s best that EACH test run in a clean environment: clean data, clean server state, clean client state. This may not be practical if the tests take too long to give timely feedback.

Common strategies are any combination of:

• Test login only once, the other tests skip login: share selenium driver across tests, or have a way to generate a session token so your tests don’t need to login.
• Don’t commit data destructive tests to the DB or find a way to get the data "reset."
• If you don’t mind your tests running slow in different contexts, then make the above "workarounds" configurable so you get fast feedback in dev branch, but the "slow tests" that login and clean all the state/data in int branch.

Managing Data:

• Maintain a data dictionary—This is a single point where we declare what data in the DB we are dependent on. Three columns: short hand description that’s used in the code, database id, notes. The data dictionary reserves data and is a signal to the viewer that they shouldn’t write to the data or change it as there are tests relying in the data.

BDD in your Sprint

Behavior Driven Development implies that something is driving development. This something is failing or pending BDD tests. Immediately after Sprint planning, get those failing tests checked in and executing so that failures/pending are visible on your continuous build monitor. Why? So everyone can casually see the status of your project, just as you do when your commute takes you by a construction project.

Three levels of being driven:

1. Immediately get the entire sprint backlog features checked in as pending tests. Then before each feature is finished, implement the Steps to prove the feature works.

Risk: Too much automation work is left for the end of the sprint and we deliver features without automation. We don't discover requirements gaps early in the sprint because automation is postponed.

Value: Make visible what work is accomplished versus pending. Doing automation before implementation tests your understanding before you do it all wrong. :-)

2. Immediately get the entire sprint backlog features checked in as pending tests. Next, implement the Steps to prove each isn’t working (imagining the UI elements are built, coding to IDs or button names that you make up in your head). As each feature is implemented, tests should pass.

Value: if you can automate the tests, you've proven a deep understanding of the requirements or have revealed problems early in the sprint when there is time to fix that problem through collaboration.

Also, you've made visible on your build monitor a lot of status: Pending automation versus automation completed versus features completed.

3. At project start, enter all the pending features in the product backlog, and then sprint by sprint stakeholders see progress toward features as the teams use the above level 2, and as time passes, those features are split or remove or new ones added.
Value: Makes visible how the project is doing at the release level.

What happens if I get stuck automating?

1. Get help from a team member because pair working results in 80% effort toward correctness and we come up with more ideas to try and solve the problem.
2. Talk to the whole team about the issue as no single individual owns quality.
3. Escalate issue to organization
4. By Sprint end, maybe the team decides it’s not automatable and that they’ll test it manually.



Given When Then Design

These are BAD smells:
• Too many GIVEN, GIVEN, AND, AND,WHEN, AND, AND, THEN.
• Need an engineer or power user to understand the GWT. (An end user or end user’s boss whose never used the product can understand the GWT.)
• UI language mentioned in the GWT
• feature file is many pages long
• GWT language is not reusable. Lack of consistency of language.
What if my "feature" has no behavior:
• Go find it!!!
  • Talk to your PO and find out "why" the customer wants this.
• If you’re delivering a component that is *part* of a feature, then
  • Go look at the feature’s GWT and understand how your component supports that feature.
  • Implement Steps for that feature to test that your component delivers that part of the feature.

Test Steps

These are NICE smells:
• Step methods are about three lines long.
• No @Alias(es).
• Steps call into a shared library (rather than call interfaces on other Step classes).

Use Continuous Integration

It's hard to get started using CI if you aren't already. Here's how to get started.


Guiding Principles: you can’t automated what you can’t do manually and you'll always find a better way to do it next week (but don't wait).

If you tackle all your technical difficulties from head on, it's hard to make progress. So deliver your way to a CI by using evolving coding and learning, and incremental delivery. Because each CI server out there can work with any possible script, script it using methodologies that get the job done (for you). Start getting the value from your integration efforts as soon as possible.

1. Manual integration: integrate on a separate machine (using scripts that later will be executed continuously.)Scripts should be able to find new tests at run time rather than rely on static lists. A script should be able to build, install/deploy. If you can't yet create scripts, do it manually until you learn how.
2. Continuous Integration: Select CI software, install it on a server, and then automate the execution of scripts ina CI.

If you have feedback, agree, disagree, feel free to comment as you'll be adding value!

Posted with Blogsy