NServiceBus.Compression 6.0.0

NServiceBus.Compression

Adds message body compression to the NServiceBus pipeline.

When to use

Compressing message bodies can be useful when the message body exceeds the transports its maximum message size or just in general to reduce network IO.

Note: In the past the maximum message size for Azure Service Bus was much smaller.

Alternatives

Compression only works well on text-based payloads like XML and Json any payload (text or binary) that contains repetitive data.

1. Use a binary serializer
   • In the past some ready to use packages were available via https://github.com/NServiceBusExtensions but none of the binary serializers are available for NServiceBus 8+.
   • ramonsmits/NServiceBus.ProtoBufNet (original archived at NServiceBusExtensions/NServiceBus.ProtoBufNet
   • Implementation a binary serializer is simple and just requires a few lines of code
2. Use the claim check pattern for binary message attachments via https://github.com/NServiceBusExtensions/NServiceBus.Attachments
3. Use any of the above in combination with compression

Version compatibility

Please note that there might be versions targeting other NServiceBus versions. Please check the Releases for all versions. or check the root of the master branch of the repository.

Introduction

This package is based on the mutator example from the NServiceBus documentation website but is has these additional features:

• Drop in auto enable compresssion, no need to recompile! Can be saviour if affected by production incidents due to unable to send too large messages.
• Requires messages to be of a minimal size but this thresshold is configurable.
• Compression level can be configured to have more flexibility between CPU cycles and message size.
• Multiple compression algorithms: GZip, Brotli, Deflate, and ZLib.
• Uses a header similar as the http specification Content-Encoding.

Supported algorithms

Decompression supports all algorithms regardless of the configured compression algorithm, enabling rolling upgrades where endpoints can switch algorithms independently. When switching to a non-GZip algorithm, ensure all receivers are upgraded to v6.x first. GZip is the only algorithm supported by all versions.

Configuration

The defaults are:

• Algorithm: GZip
• Size treshold: 1,000 bytes
• Compression level: Fastest

This is to not spend CPU cycles on messages that are small. Compression not only costs CPU cycles but also introduces latency both during outgoing (compression) and incoming (decompression) messages.

The following only compresses messages over 16KB in size and uses the highest compression level.

endpointConfiguration.CompressMessageBody(System.IO.Compression.CompressionLevel.Optimal, 16 * 1024);

To use a specific algorithm:

endpointConfiguration.CompressMessageBody(CompressionAlgorithm.Brotli, System.IO.Compression.CompressionLevel.Optimal, 16 * 1024);

Benchmarks

Compression and decompression performance using CompressionLevel.Fastest with a repeating text payload.

Key observations:

• Brotli compression is consistently faster than GZip (0.30x-0.56x) with dramatically lower allocations (0.1%-26% of GZip).
• Brotli decompression is slower than GZip/Deflate/ZLib. This is inherent to the Brotli algorithm which trades decompression speed for better compression ratios. Brotli uses a span-based decompression path that halves memory allocations at larger sizes compared to the stream-based approach used by other algorithms.
• GZip, Deflate, and ZLib have nearly identical performance as they all use the DEFLATE codec internally.
• Brotli is best suited when compression ratio and bandwidth savings outweigh decompression latency, such as over slow or metered network links.