GrailForge.TestCore 1.1.0

GrailForge.TestCore

Intended as a common core for test projects.

Assumes .Net 5 and using xUnit.

Usage

Writing integration tests

Write tests using the [IntegrationTest] attribute as a substitute for [Fact].

[IntegrationTest]
public void Rigged_deal_always_returns_blackjack()
{
    var sut = BlackjackClient();
    
    var result = sut.DealRigged();
    
    Assert.Equal(21, result.HandValue);
}

Data-driven tests can be written using [IntegrationTestTheory] as a substitute for xUnit's [Theory] attribute.

Running integration tests

Tests marked as [IntegrationTest] will be skipped by default. To ensure these are run instead of skipped, set the INTEGRATIONTEST environment variable to something. This can be done as a build-specific setting for UI-based test runners. If using the .Net CLI you can enable integration tests temporarily as follows:

• Windows: set INTEGRATIONTEST 1; dotnet test or $Env:INTEGRATIONTEST = 1; dotnet test
• Linux: INTEGRATIONTEST=1 dotnet test

For CI pipelines, integration tests can be enabled/disabled in specific builds with the same environment variable. For example you may not want integration tests to be run in a votatile dev pipeline but want them to run on staging builds.

Writing bug tests

The [Bug] attribute is for marking tests which reproduce a specific known bug which is for whatever reason not prioritised or practical to fix immediately. If using a bug tracking system like Bugzilla or Jira you can provide an optional ticket number to the attribute, e.g. [Bug("123")]. This opens the door to things like automatically generating release notes which include a change log on newly introduced/identified and newly fixed bugs, both cases which use the actual code as the source of truth instead of the issue tracker tickets.

// with issue tracking reference
[Bug("GFX-1077")]
public void Doubling_down_always_throws_exception()
{
    var result = Record.Exception(() => DoTheThing());

    Assert.NotNull(result);
}

// without issue tracking
[Bug]
public void Doubling_down_always_throws_exception()
{
    var result = Record.Exception(() => DoTheThing());

    Assert.NotNull(result);
}

The guidance here is to write code which asserts a reproduction of the bug without actually fixing it or working around it. If it is testing code which fixes the bug, it should The release notes can then include any new [Bug] tests under new/known bugs. Likewise, any removed [Bug] tests can be assumed to be fixed and included in release notes accordingly. This of course is dependent on a compatible work flow and a sufficiently large project to be of much value.

Miscellaneous

there is also a [Category] attribute which is a way of categorising xUnit tests in a more terse fashion than xUnit's Trait.

Changelog

• v1.1 (2021-06-17)

  • Add IntegrationTestTheory for more DRY data-driven integration tests

• v1.0 (2021-05-28)

  • Initial public release

Roadmap

• Introduce Fixed property of BugAttribute to enable identifying fixed bugs as distinct from unresolved bugs without losing the issue tracker reference.

License

This is free software, distributed under the terms of the GNU AFFERO GENERAL PUBLIC LICENSE as published by the Free Software Foundation. For more information, see the file LICENSE.

Contributing

Pull requests are welcome.

Contact

https://gitlab.com/grailforge/grailforge.testcore