StreamStore.S3.AWS 0.4.0

StreamStore.S3.AWS

Amazon S3 backend database for StreamStore event stream library.

Installation

To install the package, you can use the following command from the command line:

  dotnet add package StreamStore

  dotnet add package StreamStore.S3.AWS

or from Nuget Package Manager Console:

  Install-Package StreamStore

  Install-Package StreamStore.S3.AWS

Usage

Storage

To be able to use library you need to:

• Create an S3 bucket, by default named streamstore and have read, write, list and delete permissions to it.
• Enable bucket versioning.

Configuration

Library is using an official Amazon AWSSDK.S3 nuget package, so you must follow the AWSSDK.S3 configuration steps.

or you can provide the configuration in code, see section below.

Register in DI container

   // Adding StreamStore
   services.ConfigureStreamStore();

   // Adding AWS S3 database 
   services.ConfigureS3AmazonStreamStoreDatabase();

  // Or configuring it manually
  services
    .UseS3AmazonStreamStoreDatabase()
      .WithBucketName("your-bucket-name")
    .Configure();

If you want to use advanced scenarios for configuring the S3 client, you can create and register your own IAmazonS3ClientFactory:

   services.AddSingleton<IAmazonS3ClientFactory, YourOwnAmazonClientFactory>();

• For usage of StreamStore, please refer to the StreamStore documentation.

Good to know

• The library implements two-phase locking mechanism for pessimistic concurrency control on stream level.
  First, it trying to lock in memory and if it is successful, it tries to exclusively lock stream in the storage for the duration of the transaction. If it fails to lock in storage, it will release the lock in memory.

• Since S3 storage does not provide locking mechanism for files, lock in storage implemented by creating a file with the same name as the stream id and trying to lock it by creating a file with the same name and checking if it is already exists.

• Committed and uncommitted events are stored in separate root directories.

• Each event is stored in a separate file with the name of the event id.

• Each stream is stored in a separate directory with the name of the stream id.

Storage structure

• persistent-streams - committed streams
  • [stream-id] - directory with stream data
    • events - directory with events
      • [event-id] - file with event data
    • __metadata - file with stream metadata
• transient-streams - uncommitted streams
  • [stream-id] - directory with stream transactions
    • [transaction-id] - directory with transaction data
      • events - directory with events
        • [event-id] - file with event data
      • __metadata - file with transaction metadata
• locks - directory with locks
  • [stream-id] - file with lock data

Example

You can find an example of usage in the StreamStore.S3.Example project.

Testing

To be able to run tests from StreamStore.S3.Tests project, you need to create empty appsettings.Development.json file.

License

MIT License