A short while ago I wanted to get VSTS code coverage reports from NUnit tests ... and after some asking around found that using the Profiler I could get what I needed. I just released a CodePlex project to wrap this functionality. Hope it helps somebody else. Project Description
This project is an attempt to provide a wrapper around the VSTS instrumentation and code coverage tools. VSTS provides very powerful code coverage tools with Visual Studio, but as packaged, you need to be running MSTest unit tests to obtain this coverage data. There is a way of getting this data without using MSTest though, bu using another test framework such as NUnit or MbUnit for example. This wrapper encapsulates the generation of the .coverage file from a run of NUnit or MbUnit (or similar), but in addition it provides an automated generation of the .xml version of the .coverage file (both can be read by Visual Studio directly), and then uses an XSLT template to transform the XML output into a final human readabale page (by default a .html file) Further extension will pull the actual parameters from the hardcoded values in the console application into a wrapper class, and will wrap both with a command line tool and with an MSBuild task - however the code is functional enough, and is transparent enough, that both of these tasks should be easy to implement for anyone wishing to do so for themselves if they require this immediately.
OK, so I figure it's 6am - in the interests of personal education, and possible improvement of my framework for my client, I decide I best get down and dirty with Unity and see how simple it is to replace Windsor in our current application. If it is simple, and it makes the application simpler, then I'll go the Unity route (as a MS fanboi I really do prefer the MS option where all other factors are equal). Some debate on altdotnet made me consider my position on it. Unity is roughly comparable to IoC containers like Windsor and StructureMap, the CodePlex site description says: The Unity Application Block (Unity) is a lightweight extensible dependency injection container with support for constructor, property, and method call injection. So ... I download the project from CodePlex ... and double click the solution file ... I only have VS2008 Development Edition on this laptop, no VS2005, so a conversion is required. OK - I have done this before - it should work right, after all CodePlex says: You can modify or extend the Unity Application Block using Visual Studio 2008. When you open a solution, Visual Studio 2008 will upgrade the projects to its format and you can edit and compile the code to create assemblies targeted at version 3.0 of the .NET Framework. However, you will not be able to convert the projects back into Visual Studio 2005 format. Therefore, it is a good idea to work with a copy of the original solutions and projects. Guess what ... the conversion fails. Well mostly it works, but all of the Unit Test projects fail to convert. Come on MS ... these are your things ... your IDE, your conversion wizard, your CodePlex project, and your unit testing framework - this is not rocket science. OK - I'll give you a little leeway as Unity is a CTP ... So I close it down after struggling a little with the sample applications when I don't have unit tests to read (more on the sample apps shortly). Go and read CodePlex for 20 mins to try and see if anyone has these problems, but with no luck. I brace myself to figure this stuff out without unit tests, and double click the solution again. It now tells me it needs to do a conversion. Deja Vu moment ... surely I just converted it already ... maybe I was dreaming. I tell it to do the conversion, and bingo it converts successfully, including all the unit test projects it failed to convert 20 minutes ago. Inconsistent behaviour annoys me ... either fail to convert every damn time, or work every damn time, but don't pick and choose based on some ethereal criteria I am unaware of! Right ... I have a full solution loaded ... lets compile this and see it fly ... Nice idea ... except VS thinks otherwise ... all references to using Microsoft.VisualStudio.TestTools.UnitTesting are showing red in the test projects. All references are broken ... they point to Microsoft.VisualStudio.QualityTools.UnitTestFramework correctly, but they show pointing to version 0.0.0.0 which I guess is the VS2005 reference misbehaving in VS2008 ... so I have to manually update each reference, either by changing the "Specific Version" property to false in which case it auto picks the version 9.0 assembly, or by removing the reference and re-adding it (and we all know how tedious that is to do). Why????? Why make me go through all this to see Unity, when every tool I was using was from MS ??? I understand it isn't entirely the fault of the Unity team here, but surely you guys are in the best possible position to know about the way each others products work. I choose MS products because I expect them to *just work* out of the box ... I choose certain OSS products expecting them to not be packaged as nicely, or as well documented, and expect a few niggles. So I have wasted at least 30 minutes on Unity already, and haven't read any code. Please Microsoft, make your things play nicely with each other ... is that too much to ask? Right ... off to actually see how Unity works, and how it can make my life easier now ...
Today I wanted to setup JetBrains TeamCity as a pilot to run continuous integration for us.
With their new licencing model, TeamCity is free for up to 20 build configurations, 3 build agents and 20 users, which is more than enough for us to get going and see if it improves our process.
Other options were Team Foundation Build or CC.Net
Personally I discounted Team Build early, various comments on mailing lists and forums made it very clear that it was far from being a CI server out of the box, and although after some apparently extensive configuration it could be made to do something similar, it is far from a friction free piece of software.
CC.Net I have used a number of times before, but the XML configuration it requires is less than joyous, and no matter how often I set it up, I never seem to get quicker at it. Good it may be, easy to configure it isn't.
So we decided to start off with a clean Windows 2003 Server installation, as it comes out of the box. That meant that we didn't even install IIS, as TeamCity includes Tomcat to serve up its dashboard.
We had a few teething problems, but the installation was up and running inside a few short minutes ... albeit not building.
The first problem was rather major - under the VCS settings in TeamCity we had opted for Team Foundation Server, and given it all the settings we knew to be correct - but it repeatedly gave us an obscure error - one with no information other than a java exception:
jetbrains.buildServer.vcs.VcsException: TFS execution error
Stdout:
TFS Native Verifier v1.0 Copyright (C) 2006-2007 JetBrains s.r.o. by Mikhail Pilin and Eugene Petrenko
Stderr:
Exception:
at jetbrains.buildServer.buildTriggers.vcs.tfs.TfsNativeExeRunner.start(TfsNativeExeRunner.java:20)
We spent a fair while on this, eventually giving in and dropping an email to JetBrains - all credit to Serge there for a quick response. After a few confusing emails backwards and forwards, the first solution they could suggest was that I needed a "full TFS" install on the CI server.
This wasn't a great solution - we were deliberately trying to keep the server clean, the last thing I wanted was Visual Studio or Team Explorer installed.
After a bit of searching around, and some helpful hints on the altdotnet mailing list, I found that MS now package the Team Explorer as a download on its own. This looked after unpacking to install some kind of Visual Studio "light" which I still didn't want on the server - but after drilling down I found the Microsoft.TeamFoundation.xxxx assemblies were included in the package. I added these into the GAC, and the problem was instantly resolved without an install of Team Explorer or Visual Studio. A very short while afterwards a clarification came from JetBrains:
To be more specific, the following components must be in PATH:
Microsoft.TeamFoundation.Client.dll
Microsoft.TeamFoundation.Common.dll
Microsoft.TeamFoundation.Common.Library.dll
Microsoft.TeamFoundation.dll
Microsoft.TeamFoundation.VersionControl.Client.dll
Microsoft.TeamFoundation.VersionControl.Common.dll
Microsoft.TeamFoundation.VersionControl.Common.Integration.dll
Microsoft.TeamFoundation.WorkItemTracking.Client.Cache.dll
Microsoft.TeamFoundation.WorkItemTracking.Client.DataStore.dll
Microsoft.TeamFoundation.WorkItemTracking.Client.dll
Microsoft.TeamFoundation.WorkItemTracking.Client.Provision.dll
Microsoft.TeamFoundation.WorkItemTracking.Client.QueryLanguage.dll
Microsoft.TeamFoundation.WorkItemTracking.Client.RuleEngine.dll
Microsoft.TeamFoundation.WorkItemTracking.Proxy.dll
The GAC solution was therefore on the mark, but we may have got away with dropping the assemblies in the path too. Either way - this was a much cleaner install.
Our second problem was a silly mistake in configuring the project. We are using MSBuild to do the building, and we kept getting a "MSBuild Schema could not be found" error. After a lot of head bashing it turned out I had managed to click on the Validate option in TeamCity, and it was TeamCity rather than MSBuild that was throwing the error - TC could certainly do with better logging. The missing schema file could probably have been dropped alongside the .build file, but as we didn't need it to be validated, we unticked the option, and voila ... it all worked ... almost ...
One last little niggle caught us, NCover was timing out while running, and giving us an error regarding the profiler. We had included NCover in our TFS tree, along with NUnit, Rhino, and all the other tools we need, this was to avoid having to install anything on a workstation to be able to build the projects.
After a hint from Ayende, it turned out that NCover requires an assembly to be registered with regsvr32, something I haven't done in so long, I had forgotten the command even existed! So to solve this problem, the answer is:
regsvr32 coverlib.dll
It may not be ideal, but it does the job, and I think I can live with one assembly needing to be registered, at least in the short term.
Update: Grant Drake posted about this a while ago, and has 4 possible solutions here ... using //reg on the command line is a much better option in a CI environment, so I'm switching our build to do that this morning.
And with all that done, we now have a *very* clean CI server, running a very basic Windows Server install, with a very neat TeamCity install.
Now to show the business benefit!
Surely some mistake? I write an Event Handler for SharePoint 2007, deriving from SPItemEventReceiver, and all is well. Until I try to put in some basic IoC using the Windsor container. My thinking is that Windsor will allow me to separate the mechanism and plumbing of the Event Handler from the implementation of my nitty gritty stuff - and indeed it does - all is well and good. Until, that is, I decide to put the Windsor container where it really belongs, in the HttpApplication global scope. My first try for testing this was : if (HttpContext.Current.Application["WindsorContainer"] == null)
HttpContext.Current.Application["WindsorContainer"] =
new WindsorContainer(new XmlInterpreter());
IWindsorContainer container =
(IWindsorContainer)HttpContext.Current.Application["WindsorContainer"];
Which was all well and good until my Event Handler just fell over ... so I stepped into debug mode ... and was beffudled ... there is no HttPContext.Current ... Nor is there any SPContext.Current either ...
This is confusing, and annoying ... my event is clearly working from an event on a list, which is clearly running within a web application, and within a SharePoint application. Apparently I'm not the first to discover this, but I haven't found a solution yet ... AAARRRRGGGHHHHHH!!!!!!!
Update: Oren and Ken both came up with similar suggestions on how to improve this situation specifically for Windsor, by putting a static class within the assembly that has the container within it. Certainly makes this a little more palatable from the Windsor point of view, but this isn't just a Windsor issue ... SharePoint really does enjoy making my life hell! :)
I have my MVP project coming along nicely, unit tests all running, presenters and interfaces doing their basic stuff. The assemblies are all being stongly named for SP deployment (into the GAC). Then I add my WebPart and implement IView on it. Problem Numer 1 : Implementing an Interface on a WebPart using VSeWSS Causes Deploy to Fail Oh yeah, that was fun. For speed and as I am working on a VM, I used the VSeWSS extensions to quickly create and deploy the web part. An hour later wasted, I pin down a problem with the Deploy command the Visual Studio extension provide. Just implementing an interface on the WebPart class causes the Deploy to fail - with a cryptic message about a reflection error. Well done guys, I guess you are trying to find something that implements WebPart and nothing else? Whatever the reason, VSeWSS gone for good from my machine, I'm going back to doing this the hard way, where at least I have some control. Problem Number 2 : The GAC ... "I'm not lying. Take my pen, write this down. Do not trust her." So, my WebPart is now working, and all seems happy. With the assemblies deployed, I can now view my WebPart in all it's glory displaying "Hello World" with data coming from my repository via the presenter. All is good. All is level. All is pretty. Then I do a bit of basic refactoring. Following some suggestions from ALT.NET people, I play around with my event signatures, remove out 'object sender' as this will be provided to the presenter on construction, and define an EmptyEventHandlerArg. All pretty simple stuff. Hit build, all compiles just fine first time. Run all unit tests ... whoa there donkey... System.InvalidOperationException: You have called the event raiser with the wrong number of parameters. Expected 1 but was 0
at Rhino.Mocks.Impl.EventRaiser.AssertMatchingParameters(MethodInfo method, Object[] args)
at Rhino.Mocks.Impl.EventRaiser.Raise(Object[] args)
Surely some mistake. Go over all code. Try to figure out if my usage of Rhino Mocks is wrong. Nope - all is correct, the IEventRaiser is being called correctly with the right number of parameters. The correct number of parameters in the view and the presenter. Try this a dozen time more. Can't figure out why this worked just fine in my first prototype project where I had damn near the same code, so go back to that project. It all works just fine there. Change the signature there. It works just fine. Back to my WebPart project. Stare blankly. Drink more coffee. Spark of inspiration ... Delete the previous built assembly from the GAC. Now it all runs just fine. It seems that the unit tests/Rhino are getting their references from the GAC, where as my projects are getting their reference from the build directory. On a machine where the assembly is built, but may also be in the GAC this is an issue for unit tests. Off to try and find an explanation or solution. Of course I can copy to the GAC every build ... but I'm not sure I want to ... and I don't think it is healthy to have to ...
Jean-Paul has updated his test naming macro to deal with class names as well as method names, and while I like naming my tests in the form of Test_That_Data_Has_Been_Loaded, I find that JP's macro doesn't work well for me. The reason is that as I type in the form JP does ("Test That Data Has Been Loaded") I find ReSharper kicks in and start inserting class names for me. I have also got in the habit of typing "TestThatDataHasBeenLoaded" which is a lot less readable than JP's convention. So I took JP's macro and modified it for my own use ... it will now turn my style of typing into JP's style of naming... voila ... readable test names! TestThatDataHasBeenLoaded ==>> Test_That_Data_Has_Been_Loaded Code below - all credit to JP for the bulk of this (and I am no VB programmer so my changes are pretty poorly coded, no critique needed pls): Imports System
Imports System.Windows.Forms
Imports EnvDTE
Imports EnvDTE80
Imports System.Diagnostics
Imports System.Text.RegularExpressions
Public Module CodeEditor
Sub ReplaceSpacesInTestNameWithUnderscores()
If DTE.ActiveDocument Is Nothing Then Return
Dim wrCS As Boolean = DTE.Properties("TextEditor", "CSharp").Item("WordWrap").Value
Try
DTE.Properties("TextEditor", "CSharp").Item("WordWrap").Value = False
Dim selection As TextSelection = CType(DTE.ActiveDocument.Selection(), EnvDTE.TextSelection)
Dim index As Integer
selection.SelectLine()
If selection.Text = "" Then Return
Dim methodIndex As Integer = selection.Text.IndexOf("public void ")
Dim classIndex As Integer = selection.Text.IndexOf("public class ")
index = CType(IIf(methodIndex >= 0, methodIndex, classIndex), Integer)
Dim prefix As String = CType(IIf(methodIndex >= 0, "public void ", "public class "), String)
prefix = selection.Text.Substring(0, index) + prefix
Dim description As String = selection.Text.Replace(prefix, String.Empty).Trim
Dim r = New Regex("[A-Z][^A-Z]*")
Dim myMatches As MatchCollection
Dim reformedDescription As String
myMatches = r.Matches(description)
Dim successfulMatch As Match
For Each successfulMatch In myMatches
reformedDescription = reformedDescription + successfulMatch.Value + "_"
Next
reformedDescription = reformedDescription.Trim("_")
selection.Text = prefix + reformedDescription + vbCrLf
selection.LineDown()
selection.EndOfLine()
Catch ex As Exception
MsgBox(ex.Message)
Finally
DTE.Properties("TextEditor", "CSharp").Item("WordWrap").Value = wrCS
End Try
End Sub
End Module
Previously I blogged about finding out ther hard way about the limits on database size for MOSS and SQL Express. At the time we took the 'reinstall the whole damn show' approach - after all it is a dev server! However it struck me there must be a better way, and indeed there is as Todd Klindt has recently posted: Microsoft so very graciously provides a free version of SQL 2005, SQL Express, with MOSS. If you install MOSS using the Basic option or Single Server under Advanced you get SQL Express automatically. So what if as a budding newbie SharePoint admin you chose the Basic option, but now as a wise aged SharePoint admin you've seen the error of your ways and want to use a more respectable version of SQL for your SharePoint backend? You're in luck. In this blog post I'll walk you through upgrading SQL Express to SQL Standard or Enterprise. http://www.toddklindt.com/blog/Lists/Posts/Post.aspx?ID=55
The interfaces we have created in Abstracting SharePoint allow us to unit test our code, but only if we mock the objects. We can do this in 2 ways, we either write an implementation of our own that mimics the SharePoint model (by storing data in in-memory objects for example), or we can use a mocking framework like Rhino Mocks or TypeMock.NET
My first approach was to write my own mock of SharePoint. This, I reasoned to myself, would allow me to actually setup test data, and to use the mocked layer to operate as a way of demonstrating the system in addition to testing.
This approach soon became a nightmare, the SharePoint object model is convoluted and confused. The interactions between objects are unclear, and often unexpected. I gave up on this one after a few hours of work that convinced me I needed to go down the other route.
For mocking I had a range of mocking frameworks to choose from, but Rhino Mocks (which I have used before with great success) was the obvious choice. Rhino has one limitation though which made me consider an alternative, it cannot mock Sealed classes. The bright sparks that wrote Microsoft.SharePoint decided the classes would be sealed, so although I can use Rhino to mock my newly formed interfaces, I would not be able to use it to mock Microsoft.SharePoint classes should the need arise (for example if it becomes too complex to complete the interface abstractions).
TypeMock.NET can, I am told, mock sealed classes. This would make it an ideal candidate for dealing with SharePoint. It is however a commercial product, and a pricey one at that. Cost isn't really a major issue here, but still it is a factor. It is also somewhat trickier than Rhino to get to grips with, at least for someone that has only just got to grips with the simpler aspects of Rhino.
For the present I am going to use Rhino, and I am going to hope I do not have to mock any sealed classes. If I do then TypeMock.NET may save me, but I suspect at that point I am in deeper trouble anyway. The two syntaxes are different enough that the learning curve is probably not worth it until I hit those 'sealed class' cases.
The Rhino Mocks version: [TestMethod]
public void TestLookupUniqueItemReturnsSingle()
{
MockRepository mocks = new MockRepository();
ICustomSPQuery query = mocks.CreateMock<ICustomSPQuery>();
ICustomSPList list = mocks.CreateMock<ICustomSPList>();
ICustomSPListItemCollection expectedResults = mocks.CreateMock<ICustomSPListItemCollection>();
ICustomSPListItem listItem = mocks.CreateMock<ICustomSPListItem>();
int numberOfResultsToReport = 1;
With.Mocks(mocks).Expecting(
delegate
{
using (mocks.Unordered())
{
Expect.Call(query.Query).PropertyBehavior().IgnoreArguments();
Expect.Call(list.GetItems(query)).IgnoreArguments().Return(
expectedResults);
Expect.Call(expectedResults.Count).Return(numberOfResultsToReport).
Repeat.Any();
Expect.Call(expectedResults[0]).Return(listItem);
}
})
.Verify(
delegate
{
ListLookup lookup = new ListLookup(list, query);
ICustomSPListItem lookupResult = lookup.LookupUniqueItem("test", "test");
Assert.AreEqual(listItem, lookupResult);
}
);
}
And the TypeMock.NET version: [TestMethod]
public void TestSingleItemFindsMatch()
{
MockManager.Init();
ICustomSPList list = (ICustomSPList)RecorderManager.CreateMockedObject(typeof(ICustomSPList));
ICustomSPQuery query =
(ICustomSPQuery)RecorderManager.CreateMockedObject(typeof(ICustomSPQuery));
ICustomSPListItemCollection expectedResults = (ICustomSPListItemCollection)RecorderManager
.CreateMockedObject(typeof(ICustomSPListItemCollection));
ICustomSPListItem item =
(ICustomSPListItem)RecorderManager.CreateMockedObject(typeof(ICustomSPListItem));
using (RecordExpectations recorder = new RecordExpectations())
{
query.Query = "";
recorder.ExpectAndReturn(list.GetItems(null), expectedResults);
recorder.ExpectAndReturn(expectedResults.Count, 1).RepeatAlways();
recorder.ExpectAndReturn(expectedResults[0], item);
}
ListLookup listToLookup = new ListLookup(list, query);
ICustomSPListItem result = listToLookup.LookupUniqueItem("FieldName", "MyValue");
Assert.AreEqual(item, result); MockManager.Verify();
}
There is one more point in favour of Rhino Mocks ... its author Oren Eini (known as Ayende) is possibly one of the most talented developers I know, has a real passion for good quality code, and a deep commitment to open source and providing great tools to other developers. That alone makes Rhino an excellent choice.
Fingers crossed!
I've been trying to work out in my head how I can deal with SharePoint in an abstracted a way as possible on my current projects. Frankly the interfaces MS provided were less than useful, or to be precise, not only did they forget to include any interfaces, but they also decided to make all their classes sealed too (*). I'm sure there are more unfriendly things they could do to developers writing SharePoint code, but I'm struggling to think of anything right now.
* Edit 29/11/2007 - just pointed out to me, that 'all their classes sealed' was a bit of a generalisation (and me ranting and fuming). Actually only a few key ones are sealed, but they do have a habit of having no default constructors, have no interfaces, and frankly are damn impossible to mock properly (TypeMock.NET excepted) ... hence the need for all this stuff.
So my thinking at present is along the lines of "I'm gonna write me a whole damn abstraction layer to SharePoint, because it has to be faster than coding against SharePoint directly and dealing with lack of unit testing and any kind of decoupling"
The Interfaces
The first step was to start creating interfaces that mimicked the Microsoft.SharePoint API:
This was pretty much the minimum number of interfaces I had to create, and I expect many more. This set will let you do basic list operations, but for anything beyond that, expect a lot more typing (or cheat and use ReSharper to extract the interfaces from your decorated SP classes)
A typical interface looks like: public interface ICustomSPList
{
ICustomSPListItemCollection Items { get; }
ICustomSPListItemCollection GetItems(ICustomSPQuery query);
}
This is a BIG task. A REALLY BIG TASK! So my first steps were to only abstract the classes, properties and methods I was using in the first few tasks of the overall project. Obviously this reduces the up-front work considerably, but I am crossing my fingers that it doesn't hide a nasty problem down the line. I'm pretty sure MS has done something really dumb somewhere to prevent this working, but so far so good.
The Decorators
Then there was a need to create decorators for each of the SharePoint objects we are abstracting, and delegating all calls through to the Microsoft.SharePoint classes:
The decorator class for the ICustomSPList looks like this: public class CustomSPList : ICustomSPList
{
private readonly Microsoft.SharePoint.SPList _list;
public ICustomSPListItemCollection Items
{
get { return new CustomSPListItemCollection(_list.Items);}
}
public CustomSPList(Microsoft.SharePoint.SPList list)
{
_list = list;
}
public ICustomSPListItemCollection GetItems(ICustomSPQuery query)
{
Microsoft.SharePoint.SPQuery tempQuery = new Microsoft.SharePoint.SPQuery();
tempQuery.Query = query.Query;
return new CustomSPListItemCollection(_list.GetItems(tempQuery));
}
}
Basic Usage
We now have a basic way to perform SharePoint operations without being tied into the Microsoft.SharePoint dependency: ICustomSPWeb web = properties.OpenWeb();
ICustomSPList registryList = web.Lists["Registry"];
ICustomSPQuery query = new SPQuery();
query.Query = "<Where><Eq><FieldRef Name='Title' /><Value Type='Text'>My Title</Value>
</Eq></Where>";
ICustomSPListItemCollection regList = registryList.GetItems(query);
The Next Step
We have our basic abstraction in place now, and with a little magic from an IoC container like Windsor we can write the core of our application safe in the knowledge that we aren't heading down a dead end of a coupled system. We also have a system that, although it is using the SharePoint API, could be switched to a different persistence medium should the need arise.
But that is only half the story, and in fact the really important part ... we can now properly mock and unit test our code.
As a contractor I move between clients on a regular basis, and there are certain things I miss when I arrive ... the things I miss the most though, are:
ReSharper
Seriously, how can you develop without it? I guesstimate I lose 2-3 *hours* a day ithout it just typing redundant code or manually refactoring code. The auto rename of files to match the containing class alone saves massive amounts of time. Not to mention it just makes my code 100% better than if I had coded it by hand.
TestDriven.NET
Testing without it is just ... tedious. Right click a method to run the unit test, right click a class to run all the tests in the class, right click a solution ... well you get the idea. There is nothing faster or easier.
Now that Jamie has sorted his differences with MS, we can all get on and enjoy using it.
Almost everything else I can live without ... well except caffiene!
|