StoryQ: BDD / Acceptance Testing with a little help from LINQ

By Rob on Thursday, July 24, 2008

16 Comments

Filed Under: StoryQ

My first ever “write” operation to the Open Source community just happened! Todd (who I am working with at Datacom) was really keen to start using NBehave. He wanted to provide our clients with more visibility of what features were implemented / broken in each release. Because we were releasing every time someone checked in some code, it was pretty important to automate this.

NBehave, much like ruby’s rspec, is a great framework for making user stories executable within your IDE. Outside of your IDE, you’ve got things like Fitnesse, ZiBreve, and Twist - which are better if you want non technical people to be able to interactively define storys. On our project, however, we had technical people putting the acceptance tests into our Nunit project. Now, NBehave is written as an extension to NUnit, so i guess you could say they are compatible, but NBehave wasn’t compatible with the two NUnit runners that we were using all the time. Our developers used the superb Resharper test unit runner as they coded, and our build server (TeamCity, damn you JetBrains!) both wanted to run plain old NUnit.Framework.TestAttribute Tests.

So we set out to create a simple framework for running things that looked like story tests from within any test framework. StoryQ is what Todd & I came up with. The key points of difference from NBehave are:

  • No dependency on any particular unit testing framework (although StoryQ does like to know what kind of Inconclusive / Pending exception your framework uses)
  • StoryQ can use Lambda Expressions to make a sentence-like string out of a method call. This might not sound like much, but it actually takes away a lot of duplication pain. Thanks, LINQ!
  • When you run a brand new story test, it will be marked as pending (not failed).
  • By default, StoryQ output goes to Console.Out. You can also make it write a coloured html report to a file.
  • There is a GUI in the project that can convert from plain text into test case code.

We’ve tried to keep it all very simple and lightweight. If you feel like your current acceptance testing framework is too clunky, or if any of the benefits above sound good to you, then please grab the code and check out the samples. There’s an explanation up on the project homepage. We are very keen for feedback: issues, comments, feature requests, contributors – please don’t hold back.

16 Comments for this post

StoryQ sounds really good, nice work on pushing it out under the MIT licence. I think I need to read up on some of the awesomeness that LINQ provides.
Still yet to use BDD in practice, so I’m really keen on hearing the easyb.org talk at the Java Emerging Technologies conference (http://java.meetup.com/162/calendar/8313099/).

Posted onAugust 8th, 2008 at 7:39 am
Richard Paul

Have you read ‘xUnit Test Patterns’ by Gerard Meszaros?
I’ve got it sitting next to me right now, really want to dedicate some time to read it in full.

Posted onAugust 8th, 2008 at 4:08 pm
Rob

I’ve heard of it, haven’t read it though. Give me a good summary when you are done ;)

Posted onAugust 8th, 2008 at 5:17 pm
Hugh

Hey Rob. Was gonna take a look at your silverlight slides and came across this. BDD sounds very interesting indeed, I think I will read more about it. :P

Posted onAugust 9th, 2008 at 7:23 pm

Interesting – the Eleutian guys have been doing something similar with MSpec:

http://blog.eleutian.com/SearchView.aspx?q=mspec

I still only dabble with BDD – but haven’t become a complete convert yet – I’m thinking MSPec or StoryQ could push me over the edge :)

Posted onSeptember 3rd, 2008 at 6:45 pm

Hi, I cant seem to access the code/sample page. Can someone please provide a link?

Posted onDecember 10th, 2008 at 2:43 am
Rob

Hi Mark

The wiki links on the storyQ home page seem broken. I totally blame Codeplex. Todd & I will look into that and send you an email when its fixed. Thanks for checking out our project – we’re actually slowly doing some awesome background work on it now.

Cheers – Rob

Posted onDecember 10th, 2008 at 5:43 am
Roger Martin

Looks really great, really love the way it integrates so easily.

Any plans to support Actions with parameters and a catalog of actions like NBehave?

Posted onMarch 14th, 2009 at 7:22 am
Rob

Hi Roger

Thanks!

Could you please give me some examples of what a catalog of actions is? Parameterised actions do work, as long as your parameters have reasonable ToString() implementations

Posted onMarch 14th, 2009 at 4:35 pm
Roger Martin

Hi Rob,

What I had in mind for the catalog of actions was something like the example in http://www.codeplex.com/NBehave/Wiki/View.aspx?title=Examples&referringTitle=Home

where the first usage of a “given, when or then” stores the implementation (the delegate) and it can be used again, with different parameter values, specifying just the name (description).

However, I see now your usage pattern is a little different. I guess your use of an expression tree means that simple expressions (and not statement lambdas) are required – and so you probably don’t have a strong need to remember the implementation (as it is usually a one-liner).

Given all that, maybe it would be useful to combine the functionality of your two buttons in your UI tool to offer a third button that generates code that uses the Expression overloads of the Given,When,Then,And methods. Could have private stub implementations of the expressions.

Roger

Posted onMarch 17th, 2009 at 6:15 am
Rob

Hi Roger

I was reading your comment and thinking “yes that’s brilliant”, when I realised that that feature’s already there :) .

If you tick the “Declare Method” checkbox, then the GUI will generate expression calls instead of strings. You can then use Visual Studio’s smart tags or Resharper to generate the method stub.

Cheers – Rob

Posted onMarch 17th, 2009 at 7:42 am
Roger Martin

Hi Rob,

trying again, coz the formatting went a bit bad with my angle brackets…

didn’t work for me… checking “Declare Method” resulted in this:

– start of code snippet –
[Test]
public void ()
{

}
– end of code snippet —

but I still had this:

– start of code snippet –
.WithScenario(”s”)
.Given(”That a“)
.When(”b“)
.Then(”c”)
– end of code snippet –

when what I really wanted was this:

– start of code snippet –
.WithScenario(”s”)
.Given(() => ThatA())
.When(() => b())
.Then(() => c())
– end of code snippet –

I do like your idea of relying on Resharper or VS to help out instead of having the StoryQ UI tool generate stubs, as generating stubs suggests a pattern of usage that you probably don’t want to recommend.

Roger

Posted onMarch 19th, 2009 at 6:29 am

I’m new to both StoryQ and TeamCity.

How do you configure TeamCity to display the html reports from StoryQ nicely?

Posted onOctober 13th, 2009 at 9:29 am

@Martin

Integrating reports into TeamCity is a two step process:
1. Make sure that you are creating the html report for each method. This means using the story.AssertAndReportToFile(MethodBase.GetCurrentMethod());

2. In team city, add them project > General Settings > Artifact Path > **/*.html => storyq (something like that)

I hope that helps.

Posted onOctober 13th, 2009 at 10:28 pm

[...] think this is a good opportunity to talk about some of the new features in StoryQ. My first post on StoryQ came when we’d just committed it to codeplex, and a lot has changed since [...]

Posted onJune 22nd, 2010 at 11:59 am

Leave a comment

Name (required) Comment
Mail (required)
Website