Update: Jetbrains have just released Resharper 4.5, which contains native MSTest support. No need to read the rest of this article, then!
Sweet. Now I can use the Resharper test runner (superb) on MsUnit tests (mandated).
Here in .Net land, we've got an awful lot of choice if we want to write unit tests. First we had NUnit and MbUnit, then Microsoft came to the party and cloned NUnit. These day's there's also xUnit and the exotic NBehave.
Back when I was learning Java, there was pretty much just JUnit. You could write your tests in that one framework, and then they could be run pretty much anywhere (just like java ;)). You could use the JUnit console runner, the GUI runner, eclipse, intelliJ, NAnt, CruiseControl - they all knew how to run JUnit tests, because JUnit was the defacto.
These days, in .Net, we get to choose between lots of different testing frameworks, each with their own strengths and weaknesses. Each of these frameworks comes with their own test runner, for example, visual studio is (arguably) capable of executing MsUnit (AKA MsTest AKA TfsUnit) tests. TestDriven.Net came along and made the visual studio integration much nicer. You'd also typically be executing these tests from MSBuild if you were trying to do Continuous Integration. NUnit has its own GUI for running NUnit tests, and there are a lot of third party NUnit test runners.
Personally, I'm a big fan of the Resharper test runner, because you just click on the little green icon by your test and it's started testing. The test results window is what cinches the deal, it provides you with a hierarchical view of your test results, and makes it really easy to re-run whatever you like.
But it only works on Nunit:
James Kovacs had written a MsUnit plugin that let Resharper run MsUnit tests - but those crazy jetbrainers have been upgrading things again and now the plugin doesn't work.
Enter Gallio:
The Gallio Automation Platform is an open, extensible, and neutral system for .NET that provides a common object model, runtime services and tools (such as test runners) that may be leveraged by any number of test frameworks.
Essentially, Gallio is a bridge between ALL of the testing frameworks I've just mentioned and a huge number of test runners. The number one thing for me is that now I can use the Resharper runner on my MsUnit tests. All I had to do was grab the installer off the Gallio website, run it, and the next time I started Visual Studio, Resharper was ready to run my MsUnit tests:
I'm impressed that the people behind Gallio have been able to define a common object model for all these different testing frameworks.
I'd be really keen to see a Gallio plugin for TeamCity. Browsing through the Gallio discussions, it seems like it's definitely on its way. But seriously, I think Jetbrains should be lending a hand here: both with the TeamCity plugin and with Resharper. They'd only be helping themselves.
NB: I just tried this with Resharper 4.1 RC2, and it seems like those rascally jetbrainers have broken things again. I'm sure the Gallio team will get right on it as soon as Resharper 4.1 is stable...
If you're looking to get Gallio working with Reshaper 4.1+, check out Jeremy's comment on this post...
Jeremy Gray
26 August 2008 at 5:57 am
Easy workaround – Dig into the JetBrains\Resharper Program Files folder. Notice how there are folders for multiple versions. Open the v4.0 folder. Notice how there is next to nothing in that folder after upgrading to the latest R# bits (which go into the v4.1 folder). There is a bin folder left over, and a plugins folder within. Notice how within the plugins folder there is a Gallio sub-folder. Move or copy the Gallio sub-folder into the v4.1\Bin\plugins folder. Voila! Gallio is working again under R#!
Rob
26 August 2008 at 6:35 am
Jeremy you rock! Thanks for taking the time to explain that…
Korayem
7 September 2008 at 4:19 pm
@Jeremy: Awesome, thanks for the tip. I got Gallio running successfully with Resharper 4.1
Charlie
8 September 2008 at 9:29 pm
A small clarification regarding Resharper’s emulation of NUnit: It supports only a subset of NUnit attributes and other features. It’s a bit of an annoyance, especially for those who discover their tests aren’t actually running!
I haven’t had a chance to test it under Gallio yet.
Rob
9 September 2008 at 5:58 am
@Charlie oh really? What’s Resharper missing? Maybe we could start a petition on jetbrains’ bug tracker.
I wonder if NUnit to Gallio to Resharper does (or even could) fix that?
You’ve got me worried about the TeamCity unit test runner now too
Jeff Brown
9 September 2008 at 8:50 am
ReSharper’s support of NUnit is somewhat old. It can’t handle newer extensions at all, as far as I know.
Also, at the moment the NUnit adapter for Gallio does not support the pure reflection-based interfaces needed to work inside of ReSharper. So NUnit to Gallio to ReSharper is not possible just now.
The trick is that when exploring tests inside ReSharper, we don’t have a compiled test assembly to work from. So we can’t do reflection the usual way with Assembly, Type, MemberInfo, etc.
Instead, we have to do reflection using ReSharper code model interfaces. Gallio wraps these up and presents them as IAssemblyInfo, ITypeInfo, IMemberInfo, etc.
Unfortunately the NUnit extensions only support standard reflection protocols. So we would still have to emulate their behavior for the test exploration phase so that these tests will show up in the UI.
At runtime, however, we natively run NUnit so there is no problem.