Long term readers will know that theJoyOfCode.com
is a bespoke blogging application. Not because I don't think any existing blogging engines are good enough. It's purely an exercise in developer masturbation. As a consultant it's easy to forget the day to day pain of real-life development and updating the blog (and my several other projects) are a great way to keep one small toe in the world of a real developer.
Last week I had one of those days that reminded me just how painful development can
be. I started to author a suite of Visual Studio Team System (VSTS) Web Tests that I could use for regression testing as I moved the blog to use a host of new technologies (including ADO.NET Entity Framework) and add a bunch of other features.
And everything was fine until I deployed the application to my local IIS and executed the web tests; I had been working in Cassini (ASP.NET Development Web Server) until then and everything worked fine. At first, everything was fine in IIS too; I could browse my site and all my manual tests worked just fine through both Internet Explorer and Firefox.
However, when I ran my WebTests ASP.NET borked and threw an assembly load exception. Que? So my website works fine if I access it through a browser but if I access it via a WebTest it throws a BadImageFormatException? and gives me a big red "Could not load file or assembly 'YourAssembly' or one of its dependencies. An attempt was made to load a program with an incorrect format". Huh.
Some experimenting later and it turns out the application worked fine from both
browser and webtest client if I set the IIS Applicaiton pool to run in 32-bit mode (my dev laptop is 64-bit).
Huh. Clearly something very strange was going on here - probably my x64 process loading some x86 assemblies but I checked and all builds were set to 'Any CPU' so that couldn't be the problem. None of the dependencies of the rogue dll were x86 only either - and to be 100% sure of this I stubbed them all out (which took longer than I would have liked).
Still no worky. Huh.
It was only after much hair pulling (who am I kidding?), fusion logging and filemon watching that I spotted a bunch of additional dlls inside my bin folder. They were inside funny looking folders called EQ_<GUID here> and accompanied by bunch of bat files with comments about 'deploy code coverage'. Ah.
Much earlier on I had turned on VSTS code coverage for some of the assemblies in my website. I assumed that assemblies were instrumented at runtime or at JIT but it turns out not. The assemblies have to instrumented as on disk and new assemblies are generated. You guessed it 32-bit only assemblies too! The code coverage engine automatically deploys and undeploys these new assemblies during the execution of any tests. This explains the crazy behavior of the site working via the browser but not from the unit tests. It wasn't a matter of the client invoking the different behavior but the running of the tests which automatically deployed the code coverage assemblies. If I'd have hit IIS with a browser during test execution I'd have seen the difference.
Big thanks to Neil Kidd
for helping resolve this one.
Next, how my WebTests accidentally highlighted a security vulnerability on this very blog!