ThrottlingTroll 1.0.0
See the version list below for details.
dotnet add package ThrottlingTroll --version 1.0.0
NuGet\Install-Package ThrottlingTroll -Version 1.0.0
<PackageReference Include="ThrottlingTroll" Version="1.0.0" />
paket add ThrottlingTroll --version 1.0.0
#r "nuget: ThrottlingTroll, 1.0.0"
// Install ThrottlingTroll as a Cake Addin #addin nuget:?package=ThrottlingTroll&version=1.0.0 // Install ThrottlingTroll as a Cake Tool #tool nuget:?package=ThrottlingTroll&version=1.0.0
ThrottlingTroll
Yet another take on rate limiting/throttling in ASP.NET.
Features
- Ingress throttling, aka let your service automatically respond with
429 TooManyRequests
to some obtrusive clients. Implemented as an ASP.NET Core Middleware. - Egress throttling, aka limit the number of calls your code is making against some external endpoint. Implemented as an HttpClient DelegatingHandler, which produces
429 TooManyRequests
response (without making the actual call) when a limit is exceeded. - Storing rate counters in a distributed cache, making your throttling policy consistent across all your computing instances. Both Microsoft.Extensions.Caching.Distributed.IDistributedCache and StackExchange.Redis are supported.
- Propagating
429 TooManyRequests
from egress to ingress, aka when your service internally makes an HTTP request which results in429 TooManyRequests
, your service can automatically respond with same429 TooManyRequests
to its calling client. - Dynamically configuring rate limits, so that those limits can be adjusted on-the-go, without restarting the service.
How to configure
Quick example of an ingress config setting:
"ThrottlingTrollIngress": {
"Rules": [
{
"UriPattern": "/api/values",
"RateLimit": {
"Algorithm": "FixedWindow",
"PermitLimit": 5,
"IntervalInSeconds": 10
}
}
]
}
ThrottlingTroll's configuration (both for ingress and egress) is represented by ThrottlingTrollConfig class. It contains a list of rate limiting Rules and some other settings.
Each Rule defines a pattern that HTTP requests should match. A pattern can include the following properties (all are optional):
- UriPattern - a Regex pattern to match request URI against. Empty string or null means any URI. Note that this value is treated as a Regex, so symbols that have special meaning in Regex language must be escaped (e.g. to match a query string specify
\\?abc=123
instead of?abc=123
). - Method - request's HTTP method. E.g.
POST
. Empty string or null means any method. - HeaderName - request's HTTP header to check. If specified, the rule will only apply to requests with this header set to HeaderValue.
- HeaderValue - value for HTTP header identified by HeaderName. The rule will only apply to requests with that header set to this value. If HeaderName is specified and HeaderValue is not - that matches requests with any value in that header.
- IdentityId - request's custom Identity ID. If specified, the rule will only apply to requests with this Identity ID. When specifying IdentityId you will also need to provide a custom identity extraction routine via IdentityIdExtractor setting.
If any of the above properties is empty or not specified, this means matching any request.
Then each Rule must specify the rate limiting algorithm to be applied for matched requests and its parameters.
The following algorithms are currently supported:
- FixedWindow. No more than PermitLimit requests are allowed in IntervalInSeconds. Example:
"ThrottlingTrollIngress": {
"Rules": [
{
"RateLimit": {
"Algorithm": "FixedWindow",
"PermitLimit": 5,
"IntervalInSeconds": 10
}
}
]
}
- SlidingWindow. No more than PermitLimit requests are allowed in IntervalInSeconds, but that interval is split into NumOfBuckets. The main benefit of this algorithm over FixedWindow is that if a client constantly exceedes PermitLimit, it will never get any valid response and will always get
429 TooManyRequests
. Example:
"ThrottlingTrollIngress": {
"Rules": [
{
"RateLimit": {
"Algorithm": "SlidingWindow",
"PermitLimit": 5,
"IntervalInSeconds": 15,
"NumOfBuckets": 3
}
}
]
}
Requests that should be whitelisted (exempt from the above Rules) can be specified via WhiteList property:
"ThrottlingTrollIngress": {
"WhiteList": [
"/api/healthcheck",
"api-key=my-unlimited-api-key"
]
},
When using the same instance of a distributed cache for multiple different services you might also need to specify a value for UniqueName property. That value will be used as a prefix for cache keys, so that those multiple services do not corrupt each other's rate counters.
Rate limits and other settings can be configured:
- Statically, via appsettings.json.
- Programmatically, at service startup.
- Dynamically, by providing a callback, which ThrottlingTroll will periodically call to get updated values.
See more examples for all of these options below.
How to use for Ingress Throttling
To configure via appsettings.json
- Add the following section to your config file:
"ThrottlingTrollIngress": {
"Rules": [
... here go rate limiting rules and limits...
],
"WhiteList": [
... here go whitelisted URIs...
]
},
- Add the following call to your startup code:
app.UseThrottlingTroll();
To configure programmatically
Use the following configuration method at startup:
app.UseThrottlingTroll(options =>
{
options.Config = new ThrottlingTrollConfig
{
Rules = new[]
{
new ThrottlingTrollRule
{
UriPattern = "/api/values",
LimitMethod = new SlidingWindowRateLimitMethod
{
PermitLimit = 5,
IntervalInSeconds = 10,
NumOfBuckets = 5
}
},
// add more rules here...
}
};
});
To configure dynamically
Specify a callback that loads rate limits from some shared persistent storage and a time interval to periodically call it:
app.UseThrottlingTroll(options =>
{
options.GetConfigFunc = async () =>
{
var myThrottlingRules = await LoadThrottlingRulesFromDatabase();
return new ThrottlingTrollConfig
{
Rules = myThrottlingRules
};
};
options.IntervalToReloadConfigInSeconds = 10;
});
NOTE: if your callback throws an exception, ThrottlingTroll will get suspended (will not apply any rules) until the callback succeeds again.
How to use for Egress Throttling
To configure via appsettings.json
- Add the following section to your config file:
"ThrottlingTrollEgress": {
"Rules": [
... here go rate limiting rules and limits...
],
"WhiteList": [
... here go whitelisted URIs...
],
"PropagateToIngress": true
},
- Use the following code to configure a named HttpClient at startup:
builder.Services.AddHttpClient("my-throttled-httpclient").AddThrottlingTrollMessageHandler();
- Get an instance of that HttpClient via IHttpClientFactory:
var throttledHttpClient = this._httpClientFactory.CreateClient("my-throttled-httpclient");
To configure programmatically
Create an HttpClient instance like this:
var myThrottledHttpClient = new HttpClient
(
new ThrottlingTrollHandler
(
new ThrottlingTrollEgressConfig
{
Rules = new[]
{
new ThrottlingTrollRule
{
UriPattern = "/some/external/url",
LimitMethod = new SlidingWindowRateLimitMethod
{
PermitLimit = 5,
IntervalInSeconds = 10,
NumOfBuckets = 5
}
},
},
PropagateToIngress = true
}
)
);
NOTE: normally HttpClient instances should be created once and reused.
To configure dynamically
- Use the following code to configure a named HttpClient at startup:
builder.Services.AddHttpClient("my-throttled-httpclient").AddThrottlingTrollMessageHandler(options =>
{
options.GetConfigFunc = async () =>
{
var myThrottlingRules = await LoadThrottlingRulesFromDatabase();
return new ThrottlingTrollConfig
{
Rules = myThrottlingRules
};
};
options.IntervalToReloadConfigInSeconds = 10;
});
- Get an instance of that HttpClient via IHttpClientFactory:
var throttledHttpClient = this._httpClientFactory.CreateClient("my-throttled-httpclient");
Supported Rate Counter Stores
By default ThrottlingTroll will store rate counters in memory, using MemoryCacheCounterStore (which internally uses System.Runtime.Caching.MemoryCache).
Other supported options are:
- DistributedCacheCounterStore. Uses IDistributedCache instance taken from DI container.
- RedisCounterStore. Specifically designed to work with Redis. Prefer this one over DistributedCacheCounterStore + Distributed Redis Cache.
You can also create your custom Counter Store by implementing the ICounterStore interface.
How to specify a Rate Counter Store to be used
Either put a desired ICounterStore implementation into DI container:
builder.Services.AddSingleton<ICounterStore>(
provider => new DistributedCacheCounterStore(provider.GetRequiredService<IDistributedCache>())
);
Or provide it via UseThrottlingTroll() method:
app.UseThrottlingTroll(options =>
{
options.CounterStore = new RedisCounterStore(app.Services.GetRequiredService<IConnectionMultiplexer>());
});
Samples
Here is a sample project, that demonstrates all the above concepts.
Contributing
Is very much welcomed.
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net5.0 is compatible. net5.0-windows was computed. net6.0 was computed. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
-
net5.0
- Microsoft.AspNetCore.Http (>= 2.2.2)
- Microsoft.Extensions.Caching.StackExchangeRedis (>= 6.0.9)
- Microsoft.Extensions.Configuration.Binder (>= 7.0.2)
- Microsoft.Extensions.Http (>= 7.0.0)
- StackExchange.Redis (>= 2.6.90)
- System.Runtime.Caching (>= 7.0.0)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
Version | Downloads | Last updated |
---|---|---|
7.3.0 | 278 | 10/14/2024 |
7.2.0 | 597 | 8/31/2024 |
7.2.0-beta1 | 106 | 8/31/2024 |
7.1.2 | 508 | 6/12/2024 |
7.1.1 | 143 | 6/3/2024 |
7.1.0 | 127 | 6/1/2024 |
7.1.0-beta1 | 104 | 6/1/2024 |
7.0.0 | 185 | 5/15/2024 |
7.0.0-beta2 | 94 | 5/10/2024 |
7.0.0-beta1 | 106 | 5/5/2024 |
6.1.2 | 668 | 4/5/2024 |
6.1.0 | 500 | 1/28/2024 |
6.1.0-beta1 | 190 | 1/28/2024 |
6.0.0 | 581 | 12/4/2023 |
6.0.0-beta6 | 336 | 11/26/2023 |
6.0.0-beta5 | 298 | 11/26/2023 |
6.0.0-beta4 | 294 | 11/26/2023 |
6.0.0-beta3 | 288 | 11/26/2023 |
6.0.0-beta2 | 301 | 11/26/2023 |
6.0.0-beta1 | 306 | 11/25/2023 |
5.0.0 | 1,313 | 11/2/2023 |
4.0.5 | 868 | 8/27/2023 |
4.0.4 | 599 | 7/22/2023 |
4.0.2 | 517 | 7/22/2023 |
4.0.1 | 522 | 7/22/2023 |
4.0.0 | 507 | 7/22/2023 |
3.0.4 | 1,004 | 5/17/2023 |
3.0.3 | 493 | 5/15/2023 |
3.0.2 | 508 | 5/14/2023 |
3.0.1 | 516 | 5/14/2023 |
3.0.0 | 502 | 5/13/2023 |
2.0.0 | 1,050 | 4/1/2023 |
1.2.0 | 640 | 2/21/2023 |
1.1.0 | 5,903 | 1/22/2023 |
1.0.0 | 623 | 1/18/2023 |