CloudStorageORM 1.0.16

CloudStorageORM

Simplify persistence. Embrace scalability. Build the future.

CloudStorageORM is an Entity Framework-style provider that persists entities into cloud object storage. The current main branch targets .NET 10, uses EF Core 9, and currently ships with Azure Blob Storage and * AWS S3* providers. Support for Google Cloud Storage remains on the roadmap.

👉 See the roadmap

✨ Current status

• ✅ Current release line: v1.0.16
• ✅ Targets net10.0
• ✅ Azure Blob Storage provider is implemented
• ✅ AWS S3 provider is implemented
• ✅ EF-style DbContext integration via UseCloudStorageOrm(...)
• ✅ Sample app runs the same CRUD flow against EF InMemory, Azure, and AWS
• ✅ Unit + integration tests run locally with Azurite and LocalStack
• ✅ Coverage collection is wired with Coverlet + ReportGenerator
• ✅ v1.0.16 adds configurable transient-fault retries (bounded exponential backoff + jitter) at shared persistence/query execution boundaries
• ✅ v1.0.16 completes runtime logging/tracing wiring across save/query/transaction/recovery paths
• ✅ v1.0.16 Azure and AWS integration suites cover rollback, commit, crash-recovery, and stale-ETag conflict windows
• ✅ v1.0.15 implements crash-safe, idempotent transaction replay with operation-level progress tracking
• ✅ v1.0.14 preserves If-Match ETag preconditions for staged transaction save/delete replay and surfaces conflicts as DbUpdateConcurrencyException
• ✅ v1.0.14 enables Dependabot for NuGet and GitHub Actions updates via .github/dependabot.yml
• ✅ v1.0.13 adds server-side Skip/Take pushdown for supported query shapes
• ✅ v1.0.13 refreshes observability guidance for logging, tracing, and diagnostics options
• 🚧 Google Cloud Storage provider is planned

📦 Installation

From NuGet

dotnet add package CloudStorageORM

From source (main branch)

The repository currently targets .NET 10 SDK.

git clone https://github.com/rzavalik/CloudStorageORM.git
cd CloudStorageORM
dotnet restore CloudStorageORM.sln

🚀 Getting started

The current recommended integration pattern is to configure a regular EF Core DbContext with UseCloudStorageOrm(...).

using CloudStorageORM.Contexts;
using CloudStorageORM.Enums;
using CloudStorageORM.Extensions;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;

public sealed class AppDbContext(DbContextOptions<AppDbContext> options)
    : CloudStorageDbContext(options)
{
    public DbSet<User> Users => Set<User>();

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<User>().HasKey(x => x.Id);
        base.OnModelCreating(modelBuilder);
    }
}

var services = new ServiceCollection();

services.AddDbContext<AppDbContext>(options =>
{
    options.UseCloudStorageOrm(storage =>
    {
        storage.Provider = CloudProvider.Azure;
        storage.ContainerName = "sampleapp-container";
        storage.Azure.ConnectionString = "UseDevelopmentStorage=true";
    });
});

Then use the context with familiar EF operations and LINQ:

await using var provider = services.BuildServiceProvider();
await using var scope = provider.CreateAsyncScope();
var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();

var user = new User
{
    Id = Guid.NewGuid().ToString(),
    Name = "John Doe",
    Email = "john.doe@example.com"
};

db.Add(user);
await db.SaveChangesAsync();

var users = await db.Set<User>().ToListAsync();
var found = db.Set<User>().FirstOrDefault(x => x.Id == user.Id);

db.Remove(found!);
await db.SaveChangesAsync();

Current provider support on main: Azure Blob Storage and AWS S3.

🧭 Important notes for the current branch

• The base context namespace is now CloudStorageORM.Contexts.
• Configuration uses composition on CloudStorageOptions: common fields stay on the root, while provider-specific fields are under storage.Azure and storage.Aws.
• Observability is optional and configured under storage.Observability (logging, tracing, and diagnostics toggles).
• CloudStorageOptions.ConnectionString was removed; use storage.Azure.ConnectionString for Azure configuration.
• Primary-key query predicates now support direct range-aware loading for >, >=, <, and <= in addition to equality-based lookups.
• Coding style is enforced with file-scoped namespaces (namespace X;).
• The sample app is covered by integration tests that publish the sample (dotnet publish) and verify the published app exits successfully.
• Integration fixtures can skip Azure/AWS scenarios when Azurite/LocalStack are unavailable.
• CloudStorage transaction support now uses a durable transaction journal under __cloudstorageorm/tx/<transactionId>/manifest.json.
• SaveChanges during an active transaction stages durable operations in the manifest; Commit marks the manifest as committed and replays operations; Rollback marks the transaction as aborted.
• Each transaction has a unique TransactionId (Guid) and only one active transaction is allowed per DbContext instance.
• On startup of a new transaction manager instance, committed manifests are replayed and finalized (Completed), while pre-commit manifests are marked as aborted (Aborted).
• Opt-in optimistic concurrency is available through object-store ETags. Configure entities with modelBuilder.Entity<TEntity>().UseObjectETagConcurrency() (shadow ETag) or UseObjectETagConcurrency(e => e.ETag) (mapped property).
• Entities can optionally implement IETag to expose the current ETag value after materialization and successful saves; this interface is not required.
• When ETag concurrency is enabled, updates/deletes use provider-native If-Match conditions (Azure Blob and AWS S3) and conflicts are raised as DbUpdateConcurrencyException.
• Transient retries are optional and configured under storage.Retry (Enabled, MaxRetries, BaseDelay, MaxDelay, JitterFactor).
• Future versions may add provider-native temporary locking (for example, Azure blob leases and AWS conditional/object-lock strategies) to improve concurrent-writer coordination.
• IDatabaseCreator behavior is still minimal right now: schema-style database lifecycle methods are not fully implemented because object storage does not map 1:1 to relational database creation semantics.

Optional observability configuration

services.AddDbContext<AppDbContext>(options =>
{
    options.UseCloudStorageOrm(storage =>
    {
        storage.Provider = CloudProvider.Azure;
        storage.ContainerName = "sampleapp-container";
        storage.Azure.ConnectionString = "UseDevelopmentStorage=true";

        // Keep defaults (all true) or disable selectively.
        storage.Observability.EnableLogging = true;
        storage.Observability.EnableTracing = true;
        storage.Observability.EnableDiagnostics = true;
    });
});

What each toggle does on the current branch:

• EnableLogging: enables CloudStorageORM log events through ILogger.
• EnableTracing: enables CloudStorageORM ActivitySource spans (CloudStorageORM).
• EnableDiagnostics: currently represented in options/debug info; custom DiagnosticListener events are not yet emitted.

If you only need tracing:

storage.Observability.EnableLogging = false;
storage.Observability.EnableTracing = true;
storage.Observability.EnableDiagnostics = false;

For full details and consumption examples, see docs/observability.md.

🧪 Running tests locally

CI-parity local run (recommended)

./scripts/run-local-ci-tests.sh

This script mirrors CI by starting Azurite and LocalStack, then running restore, build, and tests with TRX and coverage collection.

Start Azurite (Azure integration)

docker rm -f azurite || true
docker run -d \
  -p 10000:10000 \
  -p 10001:10001 \
  -p 10002:10002 \
  --name azurite \
  mcr.microsoft.com/azure-storage/azurite:latest \
  azurite --blobHost 0.0.0.0 --queueHost 0.0.0.0 --tableHost 0.0.0.0 --skipApiVersionCheck

Run the solution tests

dotnet test CloudStorageORM.sln --nologo -v minimal

Start LocalStack (AWS integration)

docker rm -f localstack || true
docker run -d \
  -p 4566:4566 \
  --name localstack \
  -e SERVICES=s3 \
  -e AWS_DEFAULT_REGION=us-east-1 \
  localstack/localstack:3

Optional AWS environment overrides

The integration fixture uses defaults, but you can override them explicitly:

export CLOUDSTORAGEORM_AWS_SERVICE_URL=http://127.0.0.1:4566
export CLOUDSTORAGEORM_AWS_ACCESS_KEY_ID=test
export CLOUDSTORAGEORM_AWS_SECRET_ACCESS_KEY=test
export CLOUDSTORAGEORM_AWS_REGION=us-east-1
export CLOUDSTORAGEORM_AWS_BUCKET=cloudstorageorm-integration-tests

Collect coverage

dotnet test CloudStorageORM.sln --nologo --settings coverlet.runsettings --collect:"XPlat Code Coverage" -v minimal
dotnet tool restore
dotnet tool run reportgenerator \
  -reports:"tests/**/TestResults/*/coverage.cobertura.xml" \
  -targetdir:"coverage/report" \
  -reporttypes:"Html"

The HTML report is generated at coverage/report/index.html.

For CI-equivalent behavior (per-test-project TRX and coverage artifacts), see docs/ci.md.

🧪 Running the sample app

dotnet run --project samples/CloudStorageORM.SampleApp/SampleApp.csproj

The app runs the same CRUD flow three times:

1. Once against EF Core InMemory
2. Once against CloudStorageORM configured for Azure Blob Storage / Azurite
3. Once against CloudStorageORM configured for AWS S3 / LocalStack

For CloudStorageORM runs, the sample also executes a transaction scenario:

• add entity inside a transaction and Rollback (entity should not persist)
• add entity inside a transaction and Commit (entity should persist)

See docs/sampleapp.md for details.

📚 Documentation

• Library documentation
• Observability guide
• API reference (DocFX)
• API reference landing page
• Sample app guide
• Testing with Azurite
• Testing with LocalStack
• CI workflow and artifacts
• Contributing
• Roadmap

🛡️ License

This project is licensed under the GNU General Public License v3.0 (GPL-3.0-or-later). See LICENSE for details.

🤝 Contributing

Contributions are welcome. Please read CONTRIBUTING.md before opening a PR.

CloudStorageORM aims to make cloud object storage feel familiar to EF-oriented .NET applications, while staying explicit about the current provider and platform limits.