Prompted by a discussion on the altdotnet yahoo group ... why I like broken builds ...
Some people try desperately to avoid breaking the build, by doing very infrequenet check ins (the number of broken builds per day drops dramatically, the time to fix each goes up though), or by trying to ensure their development environment is as close to the build environment as possible.
But a broken build is not a problem - it is an opportunity. A broken build allows you to identify a weakness, and to resolve it early.
The weakness might be in your architecture, it may be in your dependencies, or in your assumptions, or in your developer skill levels - but the whole point of having a continuous integration server is to fail fast and let you deal with the underlying problems rather than the specifics of the code you checked in.
A short while ago I wanted to get VSTS code coverage reports from NUnit tests ... and after some asking around found that using the Profiler I could get what I needed. I just released a CodePlex project to wrap this functionality. Hope it helps somebody else. Project Description This project is an attempt to provide a wrapper around the VSTS instrumentation and code coverage tools. VSTS provides very powerful code coverage tools with Visual Studio, but as packaged, you need to be running MSTest unit tests to obtain this coverage data. There is a way of getting this data without using MSTest though, bu using another test framework such as NUnit or MbUnit for example. This wrapper encapsulates the generation of the .coverage file from a run of NUnit or MbUnit (or similar), but in addition it provides an automated generation of the .xml version of the .coverage file (both can be read by Visual Studio directly), and then uses an XSLT template to transform the XML output into a final human readabale page (by default a .html file) Further extension will pull the actual parameters from the hardcoded values in the console application into a wrapper class, and will wrap both with a command line tool and with an MSBuild task - however the code is functional enough, and is transparent enough, that both of these tasks should be easy to implement for anyone wishing to do so for themselves if they require this immediately.
OK, so I figure it's 6am - in the interests of personal education, and possible improvement of my framework for my client, I decide I best get down and dirty with Unity and see how simple it is to replace Windsor in our current application. If it is simple, and it makes the application simpler, then I'll go the Unity route (as a MS fanboi I really do prefer the MS option where all other factors are equal). Some debate on altdotnet made me consider my position on it. Unity is roughly comparable to IoC containers like Windsor and StructureMap, the CodePlex site description says: The Unity Application Block (Unity) is a lightweight extensible dependency injection container with support for constructor, property, and method call injection. So ... I download the project from CodePlex ... and double click the solution file ... I only have VS2008 Development Edition on this laptop, no VS2005, so a conversion is required. OK - I have done this before - it should work right, after all CodePlex says: You can modify or extend the Unity Application Block using Visual Studio 2008. When you open a solution, Visual Studio 2008 will upgrade the projects to its format and you can edit and compile the code to create assemblies targeted at version 3.0 of the .NET Framework. However, you will not be able to convert the projects back into Visual Studio 2005 format. Therefore, it is a good idea to work with a copy of the original solutions and projects. Guess what ... the conversion fails. Well mostly it works, but all of the Unit Test projects fail to convert. Come on MS ... these are your things ... your IDE, your conversion wizard, your CodePlex project, and your unit testing framework - this is not rocket science. OK - I'll give you a little leeway as Unity is a CTP ... So I close it down after struggling a little with the sample applications when I don't have unit tests to read (more on the sample apps shortly). Go and read CodePlex for 20 mins to try and see if anyone has these problems, but with no luck. I brace myself to figure this stuff out without unit tests, and double click the solution again. It now tells me it needs to do a conversion. Deja Vu moment ... surely I just converted it already ... maybe I was dreaming. I tell it to do the conversion, and bingo it converts successfully, including all the unit test projects it failed to convert 20 minutes ago. Inconsistent behaviour annoys me ... either fail to convert every damn time, or work every damn time, but don't pick and choose based on some ethereal criteria I am unaware of! Right ... I have a full solution loaded ... lets compile this and see it fly ... Nice idea ... except VS thinks otherwise ... all references to using Microsoft.VisualStudio.TestTools.UnitTesting are showing red in the test projects. All references are broken ... they point to Microsoft.VisualStudio.QualityTools.UnitTestFramework correctly, but they show pointing to version 0.0.0.0 which I guess is the VS2005 reference misbehaving in VS2008 ... so I have to manually update each reference, either by changing the "Specific Version" property to false in which case it auto picks the version 9.0 assembly, or by removing the reference and re-adding it (and we all know how tedious that is to do). Why????? Why make me go through all this to see Unity, when every tool I was using was from MS ??? I understand it isn't entirely the fault of the Unity team here, but surely you guys are in the best possible position to know about the way each others products work. I choose MS products because I expect them to *just work* out of the box ... I choose certain OSS products expecting them to not be packaged as nicely, or as well documented, and expect a few niggles. So I have wasted at least 30 minutes on Unity already, and haven't read any code. Please Microsoft, make your things play nicely with each other ... is that too much to ask? Right ... off to actually see how Unity works, and how it can make my life easier now ...
Today I wanted to setup JetBrains TeamCity as a pilot to run continuous integration for us.
With their new licencing model, TeamCity is free for up to 20 build configurations, 3 build agents and 20 users, which is more than enough for us to get going and see if it improves our process.
Other options were Team Foundation Build or CC.Net
Personally I discounted Team Build early, various comments on mailing lists and forums made it very clear that it was far from being a CI server out of the box, and although after some apparently extensive configuration it could be made to do something similar, it is far from a friction free piece of software.
CC.Net I have used a number of times before, but the XML configuration it requires is less than joyous, and no matter how often I set it up, I never seem to get quicker at it. Good it may be, easy to configure it isn't.
So we decided to start off with a clean Windows 2003 Server installation, as it comes out of the box. That meant that we didn't even install IIS, as TeamCity includes Tomcat to serve up its dashboard.
We had a few teething problems, but the installation was up and running inside a few short minutes ... albeit not building.
The first problem was rather major - under the VCS settings in TeamCity we had opted for Team Foundation Server, and given it all the settings we knew to be correct - but it repeatedly gave us an obscure error - one with no information other than a java exception:
jetbrains.buildServer.vcs.VcsException: TFS execution error Stdout: TFS Native Verifier v1.0 Copyright (C) 2006-2007 JetBrains s.r.o. by Mikhail Pilin and Eugene Petrenko Stderr: Exception: at jetbrains.buildServer.buildTriggers.vcs.tfs.TfsNativeExeRunner.start(TfsNativeExeRunner.java:20)
We spent a fair while on this, eventually giving in and dropping an email to JetBrains - all credit to Serge there for a quick response. After a few confusing emails backwards and forwards, the first solution they could suggest was that I needed a "full TFS" install on the CI server.
This wasn't a great solution - we were deliberately trying to keep the server clean, the last thing I wanted was Visual Studio or Team Explorer installed.
After a bit of searching around, and some helpful hints on the altdotnet mailing list, I found that MS now package the Team Explorer as a download on its own. This looked after unpacking to install some kind of Visual Studio "light" which I still didn't want on the server - but after drilling down I found the Microsoft.TeamFoundation.xxxx assemblies were included in the package. I added these into the GAC, and the problem was instantly resolved without an install of Team Explorer or Visual Studio. A very short while afterwards a clarification came from JetBrains:
To be more specific, the following components must be in PATH:
Microsoft.TeamFoundation.Client.dll Microsoft.TeamFoundation.Common.dll Microsoft.TeamFoundation.Common.Library.dll Microsoft.TeamFoundation.dll Microsoft.TeamFoundation.VersionControl.Client.dll Microsoft.TeamFoundation.VersionControl.Common.dll Microsoft.TeamFoundation.VersionControl.Common.Integration.dll Microsoft.TeamFoundation.WorkItemTracking.Client.Cache.dll Microsoft.TeamFoundation.WorkItemTracking.Client.DataStore.dll Microsoft.TeamFoundation.WorkItemTracking.Client.dll Microsoft.TeamFoundation.WorkItemTracking.Client.Provision.dll Microsoft.TeamFoundation.WorkItemTracking.Client.QueryLanguage.dll Microsoft.TeamFoundation.WorkItemTracking.Client.RuleEngine.dll Microsoft.TeamFoundation.WorkItemTracking.Proxy.dll
The GAC solution was therefore on the mark, but we may have got away with dropping the assemblies in the path too. Either way - this was a much cleaner install.
Our second problem was a silly mistake in configuring the project. We are using MSBuild to do the building, and we kept getting a "MSBuild Schema could not be found" error. After a lot of head bashing it turned out I had managed to click on the Validate option in TeamCity, and it was TeamCity rather than MSBuild that was throwing the error - TC could certainly do with better logging. The missing schema file could probably have been dropped alongside the .build file, but as we didn't need it to be validated, we unticked the option, and voila ... it all worked ... almost ...
One last little niggle caught us, NCover was timing out while running, and giving us an error regarding the profiler. We had included NCover in our TFS tree, along with NUnit, Rhino, and all the other tools we need, this was to avoid having to install anything on a workstation to be able to build the projects.
After a hint from Ayende, it turned out that NCover requires an assembly to be registered with regsvr32, something I haven't done in so long, I had forgotten the command even existed! So to solve this problem, the answer is:
regsvr32 coverlib.dll
It may not be ideal, but it does the job, and I think I can live with one assembly needing to be registered, at least in the short term.
Update: Grant Drake posted about this a while ago, and has 4 possible solutions here ... using //reg on the command line is a much better option in a CI environment, so I'm switching our build to do that this morning.
And with all that done, we now have a *very* clean CI server, running a very basic Windows Server install, with a very neat TeamCity install.
Now to show the business benefit!
Previously I got my basic interception going, to allow me to strongly type resource requests.
The next step was to allow me to inject that capability via Windsor. The test for this was a bit tricky, due to me not understanding the Windsor container syntax in C#, but after a bit of playing around, we end up with the following code to create my IResource via the container. [TestFixture]
public class SpikeProxyResources
{
[Test]
public void TestWithFactoryCreation()
{
IWindsorContainer container = new WindsorContainer();
FactorySupportFacility facility = new FactorySupportFacility();
container.AddFacility("factory.support", facility);
facility.AddFactory<IResource, ResourceFactory>("ResourceFactory", "CreateResource");
IResource resource = container.Resolve<IResource>();
Assert.AreEqual("GetVal", resource.GetVal);
Assert.AreEqual("NewValue", resource.NewValue);
}
}
public class ResourceFactory : IInterceptor
{
public IResource CreateResource()
{
ProxyGenerator generator = new ProxyGenerator();
IResource proxy = generator.CreateInterfaceProxyWithoutTarget<IResource>(this);
return proxy;
}
public void Intercept(IInvocation invocation)
{
if (invocation.Method.Name.StartsWith("get_"))
{
//TODO: Lookup resource here....
invocation.ReturnValue = invocation.Method.Name.Substring(4);
}
else
invocation.ReturnValue = null;
}
}
The next step will be to pull this configuration out into Xml ... or if I can convince the people that matter, into Binsor and Boo!
Whilst discussing Globalization and Localization on the current SharePoint stuff I am working on, the issue of resource files, their location, their abstraction, and their general use came up.
One option was to use resgen.exe to create strongly typed classes for the resources, but I felt that this lost us the flexibility that dropping the .resx files into app_globalresources would provide us with. Although accessing the compiled resources from the AppDomain is easy enough (look for an assembly called "app_GlobalResources"), we would then lose the strong typing, and have to resort to: ResourceManager rm = new ResourceManager(... ,...);
rm.GetString("MyNonTypedName");
So to work around this I spent a bit of time looking into DynamicProxy2 from the Castle stack. After a few interesting diversions into the world of DictionaryAdapter (thanks to Ken Egozii and Josh Robb. Sadly poorly documented, but I'm sure it will prove invaluable soon), I finally came up with the simple bit of code I was trying to achieve: public class SpikeProxyResources
{
[Test]
public void Test()
{
ProxyGenerator pg = new ProxyGenerator();
IResource proxy = pg.CreateInterfaceProxyWithoutTarget<IResource>(new MyInterceptor());
string s = proxy.GetVal;
Assert.AreEqual("GetVal", s);
Assert.AreEqual("NewValue", proxy.NewValue);
}
}
public interface IResource
{
string GetVal { get;}
string NewValue { get; }
}
public class MyInterceptor :IInterceptor
{
public void Intercept(IInvocation invocation)
{
if (invocation.Method.Name.StartsWith("get_"))
{
//TODO: Lookup resource here....
invocation.ReturnValue = invocation.Method.Name.Substring(4);
}
else
invocation.ReturnValue = null;
}
}
Fundamentally, this allows us to intercept the calls to the getter on IResource, and look up the resource we are after within the compiled resources. For the moment it may only return the called Method Name, but only because I saved you the boring bit that goes off and gets back the real value.
IResource can now be instantiated by Windsor and adding more strongly typed resources is just a matter of adding a get property onto the interface.
Whilst reading a great article on getter and setter methods, I came across a small quotation that perfectly encapsulated the problems I see with a lot of code at my current client: Developers by their nature have a desire to hang on to global data, and to create global methods.
In 1989, Kent Beck and Ward Cunningham taught classes on OO design, and they had problems getting people to abandon the get/set mentality. They characterized the problem as follows:
The most difficult problem in teaching object-oriented programming is getting the learner to give up the global knowledge of control that is possible with procedural programs, and rely on the local knowledge of objects to accomplish their tasks. Novice designs are littered with regressions to global thinking: gratuitous global variables, unnecessary pointers, and inappropriate reliance on the implementation of other objects.
This problem is particularly common where I am at present as a large part of the development work is in MOSS 2007, which encourages global data and methods in a large number of insidious ways.
I'm now off to re-read Beck and Cunningham in more detail ... but in the meantime, pretty please MS, could you stop encouraging bad practices!
I have little idea why, but I have now encountered this a fair few times, so much so that I thought I might as well blog it, others must be getting it too. If you are missing your 'Step Into' option from the Debug menu and toolbar in VS, there is a simple fix: Select Tools / Import and Export Settings from the menu, then select Reset All Settings and follow through the next parts of the wizard. This should have your Debug menu items back, though you may have to fix some other things back to how they were (like fonts and colours). Also if you have ReSharper installed, you need to go to the ReSharper / Options menu and under the General section reset your ReSharper shortcuts there.
Simple ... and SO MUCH FASTER than iisreset ... cscript "c:\windows\system32\iisapp.vbs" /a "AppPoolName" /r ... where "AppPoolName" is replaced by the pool you want to recycle. Damn useful for MOSS development, where the environment dictates a lot of resetting of IIS to see changes made. Technorati Tags: IIS, SharePoint
Whilst looking for something entirely different, I came across a post by Eric Kraus on putting options on the explorer context menu for .DLL files to allow them to be installed in the GAC and to open them in Reflector. As posted his code didn't quite work, so I modified it a little and it is below. Create a dll.reg file in notepad, copy and paste the following in, then double click it to install the shortcuts. Make sure you change the paths if needed, but be careful to keep the escape sequences intact: Windows Registry Editor Version 5.00 [HKEY_CLASSES_ROOT\dllfile\Shell] @="\"c:\\Program Files\\Reflector\\reflector.exe\" \"%1\"" [HKEY_CLASSES_ROOT\dllfile\Shell\Reflector] [HKEY_CLASSES_ROOT\dllfile\Shell\Reflector\command] @="\"c:\\Program Files\\Reflector\\reflector.exe\" \"%1\"" [HKEY_CLASSES_ROOT\dllfile\Shell\InstallGAC] [HKEY_CLASSES_ROOT\dllfile\Shell\InstallGAC\command] @="\"c:\\Program Files\\Microsoft Visual Studio 8\\SDK\\v2.0\\Bin\\gacutil.exe\" /i \"%1\"" Thanks Eric Technorati Tags: Reflector, GacUtil
Jean-Paul has updated his test naming macro to deal with class names as well as method names, and while I like naming my tests in the form of Test_That_Data_Has_Been_Loaded, I find that JP's macro doesn't work well for me. The reason is that as I type in the form JP does ("Test That Data Has Been Loaded") I find ReSharper kicks in and start inserting class names for me. I have also got in the habit of typing "TestThatDataHasBeenLoaded" which is a lot less readable than JP's convention. So I took JP's macro and modified it for my own use ... it will now turn my style of typing into JP's style of naming... voila ... readable test names! TestThatDataHasBeenLoaded ==>> Test_That_Data_Has_Been_Loaded Code below - all credit to JP for the bulk of this (and I am no VB programmer so my changes are pretty poorly coded, no critique needed pls): Imports System
Imports System.Windows.Forms
Imports EnvDTE
Imports EnvDTE80
Imports System.Diagnostics
Imports System.Text.RegularExpressions
Public Module CodeEditor
Sub ReplaceSpacesInTestNameWithUnderscores()
If DTE.ActiveDocument Is Nothing Then Return
Dim wrCS As Boolean = DTE.Properties("TextEditor", "CSharp").Item("WordWrap").Value
Try
DTE.Properties("TextEditor", "CSharp").Item("WordWrap").Value = False
Dim selection As TextSelection = CType(DTE.ActiveDocument.Selection(), EnvDTE.TextSelection)
Dim index As Integer
selection.SelectLine()
If selection.Text = "" Then Return
Dim methodIndex As Integer = selection.Text.IndexOf("public void ")
Dim classIndex As Integer = selection.Text.IndexOf("public class ")
index = CType(IIf(methodIndex >= 0, methodIndex, classIndex), Integer)
Dim prefix As String = CType(IIf(methodIndex >= 0, "public void ", "public class "), String)
prefix = selection.Text.Substring(0, index) + prefix
Dim description As String = selection.Text.Replace(prefix, String.Empty).Trim
Dim r = New Regex("[A-Z][^A-Z]*")
Dim myMatches As MatchCollection
Dim reformedDescription As String
myMatches = r.Matches(description)
Dim successfulMatch As Match
For Each successfulMatch In myMatches
reformedDescription = reformedDescription + successfulMatch.Value + "_"
Next
reformedDescription = reformedDescription.Trim("_")
selection.Text = prefix + reformedDescription + vbCrLf
selection.LineDown()
selection.EndOfLine()
Catch ex As Exception
MsgBox(ex.Message)
Finally
DTE.Properties("TextEditor", "CSharp").Item("WordWrap").Value = wrCS
End Try
End Sub
End Module
OK, I am slightly exaggerating ... but honestly, without ReSharper, Visual Studio 2008 is painful to use.
Why on earth couldn't they have improved refactoring and editor support as part of the upgrade. Sure the .NET 3.5 stuff is fun, multi targetting is cute, and those yummy visual designers ... but I spend 90% of my VS time actually typing code, and for that VS2008 still sucks almost as badly as VS2005 did. Have MS conceeded that Jetbrains can do a better job of the editor than they can?
ReSharper works in VS2008 - but not quite...

...and that 'not quite' bit is very irritating. It is better than nothing, but roll on ReSharper for VS2008 proper.
edit: Jeffrey Palermo has a couple of good tips ... but I still feel the pain!
I've been trying to work out in my head how I can deal with SharePoint in an abstracted a way as possible on my current projects. Frankly the interfaces MS provided were less than useful, or to be precise, not only did they forget to include any interfaces, but they also decided to make all their classes sealed too (*). I'm sure there are more unfriendly things they could do to developers writing SharePoint code, but I'm struggling to think of anything right now.
* Edit 29/11/2007 - just pointed out to me, that 'all their classes sealed' was a bit of a generalisation (and me ranting and fuming). Actually only a few key ones are sealed, but they do have a habit of having no default constructors, have no interfaces, and frankly are damn impossible to mock properly (TypeMock.NET excepted) ... hence the need for all this stuff.
So my thinking at present is along the lines of "I'm gonna write me a whole damn abstraction layer to SharePoint, because it has to be faster than coding against SharePoint directly and dealing with lack of unit testing and any kind of decoupling"
The Interfaces
The first step was to start creating interfaces that mimicked the Microsoft.SharePoint API:
This was pretty much the minimum number of interfaces I had to create, and I expect many more. This set will let you do basic list operations, but for anything beyond that, expect a lot more typing (or cheat and use ReSharper to extract the interfaces from your decorated SP classes)
A typical interface looks like: public interface ICustomSPList
{
ICustomSPListItemCollection Items { get; }
ICustomSPListItemCollection GetItems(ICustomSPQuery query);
}
This is a BIG task. A REALLY BIG TASK! So my first steps were to only abstract the classes, properties and methods I was using in the first few tasks of the overall project. Obviously this reduces the up-front work considerably, but I am crossing my fingers that it doesn't hide a nasty problem down the line. I'm pretty sure MS has done something really dumb somewhere to prevent this working, but so far so good.
The Decorators
Then there was a need to create decorators for each of the SharePoint objects we are abstracting, and delegating all calls through to the Microsoft.SharePoint classes:

The decorator class for the ICustomSPList looks like this: public class CustomSPList : ICustomSPList
{
private readonly Microsoft.SharePoint.SPList _list;
public ICustomSPListItemCollection Items
{
get { return new CustomSPListItemCollection(_list.Items);}
}
public CustomSPList(Microsoft.SharePoint.SPList list)
{
_list = list;
}
public ICustomSPListItemCollection GetItems(ICustomSPQuery query)
{
Microsoft.SharePoint.SPQuery tempQuery = new Microsoft.SharePoint.SPQuery();
tempQuery.Query = query.Query;
return new CustomSPListItemCollection(_list.GetItems(tempQuery));
}
}
Basic Usage
We now have a basic way to perform SharePoint operations without being tied into the Microsoft.SharePoint dependency: ICustomSPWeb web = properties.OpenWeb();
ICustomSPList registryList = web.Lists["Registry"];
ICustomSPQuery query = new SPQuery();
query.Query = "<Where><Eq><FieldRef Name='Title' /><Value Type='Text'>My Title</Value>
</Eq></Where>";
ICustomSPListItemCollection regList = registryList.GetItems(query);
The Next Step
We have our basic abstraction in place now, and with a little magic from an IoC container like Windsor we can write the core of our application safe in the knowledge that we aren't heading down a dead end of a coupled system. We also have a system that, although it is using the SharePoint API, could be switched to a different persistence medium should the need arise.
But that is only half the story, and in fact the really important part ... we can now properly mock and unit test our code.
This is a repost from my old blog, as it is code I'm bound to need again sometime...
More than once I have needed to sort items according to some arbitrary sort criteria, for example to sort a list of status codes. They can't be logically sorted alphabetically, so I wrote these routines.
In the example, it is a string list, but it could be any object with a bit of modification. Essentially this uses a Comparer, which uses the position within the SortString to decide on the order of the items. Nice and simple.
List<string> MyList = new List<string>();
MyList.Add("OK");
MyList.Add("Failed");
MyList.Add("Stopped");
MyList.Add("Beginning");
SortStringList(MyList, "Beginning;OK;Failed;Stopped;");
private void SortStringList(List<string> list, string sortString)
{
if (string.IsNullOrEmpty(SortDirection))
list.Sort(new StringListComparer(sortString));
else
{
StringListComparer.SortDirection direction;
try
{
direction =
(StringListComparer.SortDirection)
Enum.Parse((typeof(StringListComparer.SortDirection)), SortDirection);
list.Sort(new StringListComparer(sortString, direction));
}
catch(ArgumentException ex)
{
throw new ArgumentException(
string.Format("An attempt was made to parse the
SortDirection key ('{0}') which did not evaluate
to a known value", SortDirection),
ex);
}
}
}
public class StringListComparer : IComparer<string>
{
public enum SortDirection
{
Ascending,
Descending
}
private SortDirection _sortDirection;
private string _sortString;
public StringListComparer(string sortString)
: this(sortString, SortDirection.Ascending)
{
}
public StringListComparer(string sortString, SortDirection direction)
{
if (sortString == null)
_sortString = string.Empty;
else
_sortString = sortString;
_sortDirection = direction;
}
public int Compare(string x, string y)
{
if (x == null || y == null)
return EvaluateForNullStrings(x, y);
else
return EvaluateForNonNullStrings(x, y);
}
private int EvaluateForNonNullStrings(string x, string y)
{
int indexOfX = _sortString.IndexOf(x) + 1;
int indexOfY = _sortString.IndexOf(y) + 1;
if (indexOfX == 0 && indexOfY != 0)
return (_sortDirection == SortDirection.Ascending) ? -1 : 1;
else if (indexOfX != 0 && indexOfY == 0)
return (_sortDirection == SortDirection.Ascending) ? 1 : -1;
else
return (_sortDirection == SortDirection.Ascending)
?
indexOfX.CompareTo(indexOfY)
:
indexOfY.CompareTo(indexOfX);
}
private int EvaluateForNullStrings(string x, string y)
{
if (x == null && y != null)
{
return (_sortDirection == SortDirection.Ascending) ? -1 : 1;
}
else if (x != null && y == null)
{
return (_sortDirection == SortDirection.Ascending) ? 1 : -1;
}
return 0;
}
}
|