Fit acceptance testing and OpenWiki

NUnit now includes the Framework for Integrated Test (Fit) engine. I'm not entirely sure why (there seems to be some support for using it to run NUnit tests) but my curiosity upon seeing the new DLL led me to read more about Fit. And it couldn't have been more timely - the project I was working on had something like 47 test cases taken directly from sales orders, complete with supplier-provided "known good" outputs - a perfect ahem fit for Fit testing.

And what a beautiful thing it is. A few minutes work, and the Excel spreadsheet of test cases was a HTML document suitable for feeding into the .NET version of Fit - much preferable to the manual translation of each scenario into a NUnit test, or (shudder) manually entering each case into my interactive testing app. I'd been planning on creating a bulk-testing interface anyway, and this just seemed too good to be true - it was almost exactly what I'd been planning, and saved me the time and effort of writing and debugging a custom CSV parser + test harness.

The only thing lacking was integration with our wiki, which was the other part of my idle plan... Sure, there's FitNesse - but that just solves one problem with yet another wiki to maintain.

So I hacked together an ASP.NET runner for Fit, threw it on our build machine, and then built in a new processing instruction to let OpenWiki forward on a page to this service for processing.

Satisfaction!

In case anyone else has a use for it, here's the code required:

OpenWiki #FITTESTS processing instruction (modify mywiki.asp)

Function MyWikifyProcessingInstructions(pText)
       
   ' other custom processing instructions...
   
   ' Fit integration: detect Fit processing instruction, flag for later processing
   dim FitPI, FitPILen
   FitPI = "#FITTESTS"
   FitPILen = Len(FitPI)
   if m(pText, "\n" & FitPI & "\s+", False, False) Then
      dim lineStart, lineEnd
      lineStart = InStr(pText, FitPI)
      lineEnd = InStr(lineStart, pText, vbCR)
      If lineEnd > lineStart+FitPILen Then
         gFitTestPath = Trim(Mid(pText, lineStart+FitPILen, lineEnd - lineStart - FitPILen))
      End If
      pText = s(pText, "\n" & FitPI & "[^\r\n]*\r*\n", "", False, False)
   End If

   ' other custom processing instructions...
   
    MyWikifyProcessingInstructions = pText
End Function

Additional modifications to mywiki.asp - these actually call the testing web service after the page has been mostly built.

Function MyLastMinuteChanges(pText)

   ' other custom last-minute changes...

   ' Fit integration
   If (gAction = "view" and Len(gFitTestPath) > 0 ) Then
      dim TestResults
      TestResults = RunFitTests(pText, gFitTestPath)
      If ( err.number <> 0 ) then
         pText = "<i>Error running tests: " & err.description & "</i><br />" & pText
      Elseif ( Len(TestResults) = 0 ) then
         pText = "Unknown error running tests" & pText
      Else
         pText = TestResults
      End If
   End If

   MyLastMinuteChanges = pText
End Function

Function RunFitTests(pText, gFitTestPath)
   dim url
   dim verb
   dim form

   ' location of FIT service. Need not be local.
   url = "http://localhost:8088/FitService.ashx"
   verb = "POST"
   form = "ClassPath=" & Server.URLEncode(gFitTestPath) _
      & "&TestData=" & Server.URLEncode(pText)
      
   on error resume next  
   
   dim xmlhttp
   set xmlhttp = Server.CreateObject("Msxml2.ServerXMLHTTP.4.0")
   if ( xmlhttp is nothing or err.number <> 0 ) then
      err.clear
      set xmlhttp = Server.CreateObject("Msxml2.ServerXMLHTTP")
   end if
   
   dim xmlDoc
   if ( err.number = 0 ) then
      xmlhttp.open verb, url, false ' synchronous
   end if
   
   if ( err.number = 0 ) then
      xmlhttp.setRequestHeader "Content-Type", "application/x-www-form-urlencoded" 
      xmlhttp.send form
   end if
   
   ' try to figure out if it worked or not. And if not, why. VBS error handling... yuck.
   dim status
   if ( err.number = 0 ) then
      status = xmlhttp.status
   end if
   
   if (err.number <> 0) then
   elseif (status <> 200) then 
      Err.Raise 65000, "", xmlhttp.statusText & "(" & status & ")", "", 0
   else
      ' clean up Fit html
      RunFitTests = Replace(xmlhttp.ResponseText, "&nbsp;", " ")
      RunFitTests = Replace(RunFitTests, "<br>", "<br/>")
      RunFitTests = Replace(RunFitTests, "<hr>", "<hr/>")
      RunFitTests = Replace(RunFitTests, "<font size=-1", "<font size='-1'")
   end if  
   
   set xmlhttp = nothing  
End Function

FitService.ashx - quick'n'dirty Fit runner. You might want to add some error handling...

<%@ WebHandler Language="C#" Class="FitService" %>

using System;
using System.Web;
using fit;

public class FitService : IHttpHandler
{

   public void ProcessRequest(HttpContext context)
   {
      context.Response.ContentType = "text/html";
      context.Response.ContentEncoding = System.Text.Encoding.UTF8;

      Fixture.assemblyDirs = context.Request.Form["ClassPath"].Split(new char[]{';'});
      
      Fixture fixture = new Fixture();
      Parse tables = new Parse(context.Request.Form["TestData"]);
      fixture.doTables(tables);
      tables.print(context.Response.Output);
   }

   public bool IsReusable
   {
      get
      {
         return false;
      }
   }
}

With these in place, i can add a line to the beginning of any wiki entry:

#FITTESTS c:\MfgEngineInterface\Current build\

...followed by a table containing the tests, and the tests will be automatically run and their results provided upon page generation.