SlimCAT 1.1.2

Install-Package SlimCAT -Version 1.1.2
dotnet add package SlimCAT --version 1.1.2
<PackageReference Include="SlimCAT" Version="1.1.2" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add SlimCAT --version 1.1.2
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
#r "nuget: SlimCAT, 1.1.2"
#r directive can be used in F# Interactive, C# scripting and .NET Interactive. Copy this into the interactive tool or source code of the script to reference the package.
// Install SlimCAT as a Cake Addin
#addin nuget:?package=SlimCAT&version=1.1.2

// Install SlimCAT as a Cake Tool
#tool nuget:?package=SlimCAT&version=1.1.2
The NuGet Team does not provide support for this client. Please contact its maintainers for support.

SlimCAT - A DevOps CI/CD Performance Test Tool

<img src="https://github.com/charlesdwmorrison/SlimCAT/blob/master/SlimCAT.jpg?raw=true" align="left" alt="drawing" width="250" height="300"/>

  • SlimCAT is a C# (.Net 5.0 /.Net Core) class library implementing features typically found in load tools:
          - launch varying amounts of multiple threads (users) and run them for a specific duration or a specified number of requests.
          - correlate data from one request to another so user "flows" or scenarios can be contructed from multiple requests (such as: logon, do transaction, log out).
          - measurement of response time, throughput
  • SlimCAT can be used in CI/CD pipelines or as a standalone desktop tool.
  • SlimCAT makes writing multi-threaded tests as easy as writing any functional, MSTest or NUnit test.

SlimCAT SignalR Chart.js

  • SlimCAT uses NUnit to assert against response time metrics; pipelines can then make decsions based on this pass/fail result.
  • SlimCAT has a simple design. If you can write some C# code, you can follow the examples and load test your own applications.
  • SlimCAT follows principles and architecture of well-known load tools, with scripts, correlations, and data sources.
  • Tests can be as short as 10-13 seconds (ideal for pipeline use), or as long as several hours.

Features/Components

Advancements over vernerable tools such as Netling and K6 include:

  • .Net 5.0 (.Net Core). SlimCAT can be executed on Linux or a Mac.
  • Correlation so that you can add data and vary the body or URI.
  • Ability to test more than one URL or endpoint per test test.
  • Test user scenarios,not just one URI; just like LoadRunner or Visual Studio load tests.
  • User flows ("scripts") composed in familiar C# code, not JavaScript.
  • Use NUnit or MSTest asserts to generate pass/fail results for any URL or for the test as a whole.
  • Ramp up users gradually, not just "slam testing."
  • Logging class logs response time, throughput and other metrics to a CSV file.
  • Perf Metrics class calculates response time average, percentiles and throughput.
  • Stay in the Visual Studio IDE to create scripts.
  • Easier to understand script syntax compared to K6 or some other tools.
  • Open Source and easily customizable.
  • Use standard DevOps functional test agent machines instead of dedicated load generators.

SlimCAT takes its name from the old Microsoft WCAT - Web Capacity Analysis Tool which Fiddler can still export. See also here

Usage

Scripts (User Flows)

Scripts consist of a collection of requests. E.g.,

List<Req>

Scripts in SlimCAT are classes with one method, BuildRequest(). BuildRequest() returns a list of the requests you want to execute.
The following example should look somewhat familiar if you have done any LoadRunner scripting:

public class S02_OnlineRestExampleScript : Script
    {
        public  List<Req> BuildRequestList()
        {
            requestList = new List<Req>()
              {
                new Req()
                {
                    uri = urlPrefix + "/employees",
                    method = Method.GET,
                    extractText = true,
                    nameForCorrelatedVariable = "empId",           
                    regExPattern = "(?<={\"id\":\")(.*?)(?=\",\"employee_name)"
                },
                new Req()
                {
                    method = Method.GET,
                    // Example of using correlated value from above
                    useExtractedText = true, // instructs SendRequest() to use a correlated value
                    nameForCorrelatedVariable = "empId",
                    uri = urlPrefix + "/employee/" + correlationsDict["empId"]
                },
                new Req
                {
                    uri = urlPrefix + "/create",
                    method = Method.POST,
                    body = "{\"name\":\"test\",\"salary\":\"123\",\"age\":\"23\"}"
                },
                 new Req
                {
                    uri = urlPrefix + "/update/1",
                    method = Method.PUT,
                    body = "{\"name\":\"test\",\"salary\":\"123\",\"age\":\"23\"}"
                },
                new Req
                {
                    uri = urlPrefix + "/delete/2",
                    method = Method.DELETE,
                }
            };

            return requestList;
        }
    }

Correlation (using the response of one request as data for the next) is accomplished by means of regular expressions:

Correlation RegEx Syntax:

(?<=  <left boundary> )(.*?)(?=   < right boundary> )

Users (Threads)

To create multiple users, a script object is passed to a user controller class, which launches threads (in the form of C# tasks). The threads execute the requests defined in the List<Req> collection.
The method "AddUsersByRampUp()" takes any collection of requests, creates a new instance of it every X number of seconds, and passes it to a SendRequest() method:

public async Task AddUsersByRampUp(Script script = null, int newUserEvery = 2000, int maxUsers = 2, long testDurationSecs = 360)
{
    while ((testStopWatch.ElapsedMilliseconds < testDurationSecs * 1000) & (Interlocked.Increment(ref numThreads) <= maxUsers)) // loop as long as load test lasts. 
    {
       var t = Task.Run(() =>
       {
         LaunchClientAndHandleStop(script, numThreads, testDurationSecs);
       });
       tasksInProgressLst.Add(t);
       Thread.Sleep(newUserEvery);
    }
     await Task.WhenAll(tasksInProgressLst);
}

Test Execution

To make a multi-user test, you just put the above two components together:

  • call the BuildRequestList() method of the script
  • pass the script object to the user controller
  • finally, add an assertion:
[Test]
public async Task S02_OnlineRest_3_Users()
{
    // arrange
    S02_OnlineRestExampleScript onlineRestExampleScript = new S02_OnlineRestExampleScript();
    UserController uc = new UserController();
    PerfMetrics pm = new PerfMetrics();

    // act
    List<Req> requestList = onlineRestExampleScript.BuildRequestList();    
    await Task.Run(() => uc.AddUsersByRampUp(script: onlineRestExampleScript, newUserEvery: 10000, maxUsers: 3, testDurationSecs: 30));
    Dictionary<string,double> perfMetrics = pm.CalcualteAllMetrics(ResponseDb.conCurResponseDict);

    // assert
    Assert.IsTrue(perfMetrics["avgResponseTime"] < 3, "Expected:Avg. Response time < 3 seconds");
}

As you can see from the above, SlimCAT also has a PerfMetrics class which performs calculations on the results and which you can assert against to determine if the test passed or failed.

The SendRequest() class currently consists of a static .Net HttpClient which sends the requests. The class also performs the correlations defined in the script, and starts and stops a timer measuring response time.

Planned Enhancements

  • GUI with a chart to show response time and throughput. This will be done as Blazor WebAssembly Progressive Web App.
  • Examples showing how to use a CSV file as data input.
  • Examples of how to assert against a particular URL within a script.
  • Create scripts programmatically by importing .har files.
  • Run multiple scripts at the same to create a load scenario.
  • Create a console version in order to execute from the command line. (I'm not 100% sure this is a goal.)
  • Test Agents to use more than one machine (distant goal).

Suggestions/FAQ

  • Leverage Application Insights, AppDynamics, Splunk or other monitoring tool to visualize the load test.
  • If you do that, you do not need a GUI.
  • A SignlR GUI is provided for cases when a monitoring tool is not available.

Illustration 112514351 � Martin Malchev | Dreamstime.com

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
1.1.2 118 10/29/2021
1.1.0 105 10/27/2021
1.0.0 100 8/24/2021

In this update the REST client has been replaced with HTTPClient; so there is no longer a dependency on RESTSharp.
Correlation now actually works.
appSettings.json has been added to read configurations.
SlimCatChart.html has been improved. There is now a throughput line, and line colors are not just randomly chosen.
v 1.1.2 fixes some math rounding errors.