Installation

The first step starting with canopy is creating a F# project - F# console application. And then installing canopy:

Install-Package canopy

For more installation instructions check the documentation.

After canopy installation, make sure that Selenium.WebDriver and FSharp.Core packages are updated to the latest version. This will help you avoid unexpected behavior.

Configuration

With canopy, you can use different browsers for testing. I prefer Firefox which is the default choice, but you can use Chrome or even headless browser - PhantomJS.

When you choose the Firefox, you have to add geckodriver.exe to the project like in the canopy starter pack. Or download it into some folder on your system and add it to the Path environment variable. You can download latest geckodriver.exe here: https://github.com/mozilla/geckodriver/releases.

Now, you can start with a basic UI testing program. Open Program.fs and replace the code there with this:

open canopy
open runner

start firefox

// Write yout UI tests here

run()

printfn "press [enter] to exit"
System.Console.ReadLine() |> ignore

quit()

This code starts a new Firefox instance and runs tests you defined between start firefox and run(). You can try to run this application and will see that a new Firefox instance gets opened.

There is one more thing I would make configurable - the root URL of your website. For this purpose, I have created a module in a new file - Common.fs. Add this file before Program.fs.

module Common

open canopy

let mutable rootUrl = ""

let goto path = url (rootUrl + path)

Here I have created a mutable variable which I will set in the Program.fs. I also added a helper method - goto, which I will use later to navigate to a specific relative URL in a website. The method is calling canopy's url method and combines root UR with a relative path.

Now you can set the root URL in the Program.fs before the start firefox call.

open canopy
open runner
open Common

rootUrl <- "http://localhost:50356"

start firefox

run()

printfn "press [enter] to exit"
System.Console.ReadLine() |> ignore

quit()

Test structuring

While I could write my tests directly in the Program.fs, it is not practical. Instead, I am organizing tests in different scenarios files. For this purpose, I have created a folder - Scenarios directly above the Program.fs file. In this folder, I am adding separate scenarios I want to test. Mostly those scenarios match a single page or a feature. For example, I might have home page scenarios or search scenarios - here the home page scenarios match tests against the home page, but the search scenarios might execute a search query in the page header and then assert on results.

As an example, I will test an Alloy site.

Scenarios

Let's look at the home page scenarios. The home page looks like this:

The page has header and footer which are common functionality for the whole site. So those are not a part of our home page scenarios. We care only about specific home page functionality - a jumbotron block on the top and three teaser blocks below.

For the home page, we can verify that all blocks are in place. But we should not test the content which can change often. In our case, the jumbotron texts might often change while we know that our teaser block titles will likely not change. An editor might also add/remove blocks. In this case, we will have to modify our tests when it happens.

So let's create a scenario.

module HomePageScenarios

open canopy
open Common

let positive _ =
    context "Positive home page tests"

    "When on home page" &&& fun _ ->
        goto "/"

    "it contains jumbotron" &&& fun _ ->
        displayed ".jumbotronblock"

    "it contains Alloy Plan" &&& fun _ ->
        "h2" *~ "Alloy Plan"

    "it contains Alloy Track" &&& fun _ ->
        "h2" *~ "Alloy Track"

    "it contains Alloy Meet" &&& fun _ ->
        "h2" *~ "Alloy Meet"

let all _ =
    positive()

Here I have created a module - HomePageScenarios. I am splitting tests into separate contexts (groups). For the home page, I have only one - positive. And then at the end, I have defined a function which executes all test contexts I have in this module.

Then I am opening the home page using our goto function from the Common module. The first assertion checks if jumbotron is displayed. I am not asserting the content as it might change quite often. The last three tests check for the teaser block titles.

"*~" is a special operator in the canopy which executes a regex against the content. In our example, it just checks if the h2 content is same as on the right, but it allows much more complex comparisons. One case which might be useful is a case-insensitive comparison.

"h2" *~ "(?i)Alloy Meet"

But knowing how to do with regex and repeating it for all assertions you need it is quite time-consuming. So we can create our assertion operator.

module Assertions

open canopy

let ( *=~ ) cssSelector value = cssSelector *~ ("(?i)" + value)

I have put it in a separate module - Assertions. My operator is "*=~" and the resulting assert looks like this:

"h2" *=~ "Alloy Meet"

One disadvantage is that you have to remember all these operators. Instead, you can use normal functions with a descriptive name.

Pages

There is one thing bothering me with navigation to the pages - I am hardcoding page URLs into tests. If these URLs change, then I have to change those in all tests. Same applies for selectors of elements. For example, ".jumbotronblock" selector might change in the future, but we are hardcoding it in the test.

For this purpose, I have created a separate module where I am defining page data.

module Pages

type CommonData = { url: string; heading: string }
type BasicPage = { common: CommonData }

let articlePage = {
    common = { url = "/article"; heading = "Article" }
    }

type HomePage = { common: CommonData; jumbotron: string }
let homePage = {
    common = { url = "/"; heading = "Home" };
    jumbotron = ".jumbotronblock"
    }

Here I am defining several types for our page definition. All pages will share some common data, so I have created CommondData type to hold the URL of the page and the heading. Then I have created a BasicPage type which just uses common data. This is useful for simple pages where you are not using any element selectors. You can see how I defined a BasicPage value for article page. For the home page, I have created a separate type which includes common data and a jumbotron selector.

Now we can use Pages module in our tests.

module HomePageScenarios

open canopy
open Common
open Pages

let positive _ =
    context "Positive home page tests"

    "When on home page" &&& fun _ ->
        goto homePage.common.url

    "it contains jumbotron" &&& fun _ ->
        displayed homePage.jumbotron

Now, tests are easier to maintain. It is also better for readability.

Navigation

When testing a CMS website, it is important to navigate to the pages through the navigation. In most cases, you should start with the home page, then navigate to the page you want to test, and only then perform the tests on that page.

You can write navigation manipulation directly in your tests, but it will be hard to maintain as you have to repeat this from test case to test case. So it is good to create some abstraction over navigation.

module Navigate

open canopy

let toAlloyPlan () =
    element ".nav"
    |> elementWithin "Alloy Plan"
    |> click

let toNth index =
    nth index ".nav > li a"
    |> click

Here I am showing two different approaches. The first function - toAlloyPlan uses a certain navigation element. This approach works when navigation doesn't change often.

The second method - toNth uses another approach. It navigates to the element by its index. This approach is useful when your navigation is dynamic.

You can combine these two approaches for your website. For example, you could have a top navigation with an element Categories which has a sub-menu. The sub-menu can contain a list of categories which are dynamic. In this case, you can explicitly click on the Categories navigation item and then randomly click on the category.

Summary

canopy gives you a great lightweight API over Selenium which together with F# syntax makes simple and readable tests. But you can also write your UI tests successfully in C# and Selenium without any framework. The main idea is writing your tests keeping in mind that CMS content might change and structure tests accordingly.

You can find a test project on GitHub.