Unobtrusive continuous testing with FAKE

Previously I have tried different tools for continuous testing. Probably most advanced one is NCrunch, but it is also quite expensive. I also tried ContinuousTests, but it had several bugs running my builds and tests. There is also Giles I haven't tried.

For long period I have used Visual Studio Ultimate Test Explorer's feature running tests on build (see image below), but unfortunately it is not available in Visual Studio Professional I have now.

So as I am learning F# I decided to try FAKE.

I did not want to install FAKE in my team's project to not bother other team members who might not want to use FAKE to run tests. One way is to add FAKE project's folder into ignore list of your repository, but it will require to change .gitignore file for the project, so it is not a solution. Another way is to put your FAKE project beside team's project.

The easiest way to start with FAKE is just creating new F# library or console application project and install FAKE NuGet package.

Install-Package FAKE

I am using xUnit 2 and I need test runner for it, so install console runner.

Install-Package xunit.runner.console -Version 2.0.0

Following Getting started tutorial create build.bat file for running FAKE script. Just use proper FAKE version in the path.

@echo off
cls
"..\packages\FAKE.3.34.7\tools\Fake.exe" build.fsx
pause

And then create build.fsx script which references FAKE library and opens FAKE namespaces. For xUnit 2 I had to include Fake.Testing namespace.

#r @"../packages/FAKE.3.34.7/tools/FakeLib.dll"
open Fake
open Fake.Testing

Then create function which will run tests.

let testDir  = "../../TeamProject/TeamProject.Tests/bin/Debug/"

let runTests () =
    tracefn "Running tests..."
    !! (testDir @@ "*.Tests.dll")
    |> xUnit2 (fun p -> {
                        p with HtmlOutputPath = Some(testDir @@ "xunit.html");
                               ToolPath = @"../packages/xunit.runner.console.2.0.0/tools/xunit.console.exe"
                        })

By convention it will run all tests in .Tests.dll libraries. testDir is relative directory to the test project's build output. I also had to provide test runner path in XUnit2Params.

Next I had to define Watch task which will look for changes in test project's build output and run tests. Unfortunately there is an issue with FAKE's Watch that it works only with absolute paths. To get the full path from relative one I had to cuse System.IO.Path.GetFullPath function.

let fullDir = System.IO.Path.GetFullPath testDir
Target "Watch" (fun _ ->
    use watcher = !! (fullDir @@ "*.*") |> WatchChanges (fun changes ->
        runTests()
    )
    System.Console.ReadLine() |> ignore
    watcher.Dispose()
)

It is also good to run tests in the beginning, so create separate task for it.

Target "Test" (fun _ ->
    runTests()
)

And the last step - create default target, configure order of targets and run.

// Default target
Target "Default" (fun _ ->
    trace "The end."
)

"Test"
==> "Watch"
==> "Default"

RunTargetOrDefault "Default"

Script should be run from directory it relies. After running script first it will run tests and then it will start watching for test dll changes. After you change your tests and build the project, FAKE will run tests again.