notes from a passionate developer

Developer that lives by the mantra "code is meant to be shared".




This is a personal blog. The opinions expressed here represent my own and not those of my employer, nor current or previous. All content is published "as is", without warranty of any kind and I don't take any responsibility and can't be liable for any claims, damages or other liabilities that might be caused by the content.

xUnit - dynamically skipping tests for different test-environments

Daniel WertheimDaniel Wertheim

While developing MyCouch, I need support for multiple test-environments targetting local CouchDb installs as well as for cloud services such as Cloudant. Each different test-environment supports different features/scenarios. E.g. When I execute my integration tests against Cloudant, I don't want to support creation and deletion of databases. This post will show you how I'm using xUnit's Skip property of the FactAttribute, to skip the execution of tests, dynamically determined by configuration.

Since I do target Windows store apps as well as .Net4.0 and .Net4.5, I don't have this different connection settings for each environment in an app.config (not supported by WinRT). Also, I don't want to check in Cloudant configuration as I then would be putting my credentials out in the open for a cloud service. Instead, I have the configuration being delivered via simple HTTP-request to a local Nancy hosted webapp, that just reads and returns JSON lying in configuration files on disk. The configuration looks like this:

    "Supports": [
        "attachmentscontext", "changescontext",
        "databasecontext", "databasescontext",
        "documentscontext", "entitiescontext",
        "replicationcontext", "viewscontext",
        "mycouchstore", "createdb", "deletedb"],

This JSON is simply deserialized to a simple entity:

public class TestEnvironment  
    public ServerClientConfig ServerClient { get; set; }
    public DbClientConfig DbClient { get; set; }
    public string TempDbName { get; set; }
    public string[] Supports { get; set; }

    public bool SupportsEverything
        get { return Supports.Contains("*"); }

    public TestEnvironment()
        ServerClient = new ServerClientConfig();
        DbClient = new DbClientConfig();
        Supports = new[] { "*" };
        TempDbName = "mycouchtests-temp";

    public virtual bool HasSupportFor(string requirement)
            SupportsEverything ||
            Supports.Contains(requirement, StringComparer.OrdinalIgnoreCase);

The important configuration here isSupports. It defines what the test-environment supports. A test then uses a custom FactAttribute called MyFactAttribute which allows me to specify "requirements/scenarios" for a test:

public void When_GET_of_document_with_id_and_rev_It_returns_the_document()  

I can also specify more than one requirement/scenario that has to be supported:

public void Can_create_and_drop_db()  

The implementation of the MyFactAttribute is quite simple. The "ugliest" part of it is that it has a dependency on a static IntegrationTestsRuntime to look at the current test-environment and what it supports:

public class MyFactAttribute : FactAttribute  
  public MyFactAttribute(params string[] scenarios)
    var environment = IntegrationTestsRuntime.Environment;

    if(!scenarios.Any() || environment.SupportsEverything)

    if (!scenarios.All(r => environment.HasSupportFor(r)))
      Skip = string.Format(
             "TestEnvironment does not support ALL test scenario(s): '{0}'.",
             string.Join("|", scenarios));

If I now execute the Can_create_and_drop_db test, against a test-environment that has no support for either: DatabasesContext, CreateDb or DeleteDb, the test-runner will skip the test and output something like:

Can_Create_and_drop_db ignored: TestEnvironment does not support test scenario(s): 'databasescontext|createdb|deletedb'.  

Not sure if it's the correct solution, but it's easy and solves my problem. Perhaps it can help you to.



Developer that lives by the mantra "code is meant to be shared".