EightBot.Nebula.DocumentDb 8.0.0-preview.2

EightBot.Nebula

A collection of .NET libraries providing common utilities, Azure service integrations, data access patterns, and REST client functionality. All libraries target .NET 8.0.

A nebula is an interstellar cloud of dust, hydrogen, helium and other ionized gases. Originally, nebula was a name for any diffuse astronomical object, including galaxies beyond the Milky Way.

📦 Packages

🏗️ Solution Architecture

graph TB
    subgraph "Azure Storage"
        AzureBlob[EightBot.Nebula.AzureBlob]
        AzureQueue[EightBot.Nebula.AzureQueue]
    end

    subgraph "Data Access"
        DocumentDb[EightBot.Nebula.DocumentDb]
        EF[EightBot.Nebula.EntityFramework]
        EFCore[EightBot.Nebula.EntityFrameworkCore]
        EFIdentity[EightBot.Nebula.EntityFramework.Identity]
    end

    subgraph "Infrastructure"
        Cache[EightBot.Nebula.Cache]
        Config[Eightbot.Nebula.Configuration]
        RestClient[EightBot.Nebula.RestClient]
    end

    subgraph "Utilities"
        Extensions[EightBot.Nebula.Extensions]
    end

    EFIdentity --> EF

    classDef azure fill:#0078d4,stroke:#005a9e,color:#fff
    classDef data fill:#68217a,stroke:#4a1561,color:#fff
    classDef infra fill:#107c10,stroke:#0b5c0b,color:#fff
    classDef util fill:#ff8c00,stroke:#cc7000,color:#fff

    class AzureBlob,AzureQueue azure
    class DocumentDb,EF,EFCore,EFIdentity data
    class Cache,Config,RestClient infra
    class Extensions util

📋 Package Details

Azure Storage Libraries

EightBot.Nebula.AzureBlob

Typed Azure Blob Storage client with support for:

• Container management with typed base classes
• Upload/download with streaming support
• Blob leasing for concurrency control
• SAS token generation for secure access
• Dependency injection extensions

// Define a container
public class MyBlobContainer : BlobContainerBase
{
    public override string ContainerName => "my-container";
}

// Register and use
services.AddBlobClient<MyBlobContainer>(connectionString, BlobAccessType.Private);

// Upload/download
await blobClient.UploadAsync("path/file.json", data, overwrite: true);
var content = await blobClient.DownloadAsync<MyModel>("path/file.json");

EightBot.Nebula.AzureQueue

Typed Azure Queue Storage client with:

• Typed queue definitions
• Message enqueue/dequeue with visibility timeout
• Queue length monitoring
• Automatic message serialization

// Define a queue and message
public class MyQueue : QueueBase { public override string QueueName => "my-queue"; }
public class MyMessage { public string Data { get; set; } }

// Register and use
services.AddQueueClient<MyQueue, MyMessage>(connectionString);

await queueClient.EnqueueAsync(new MyMessage { Data = "Hello" });
await queueClient.DequeueAsync(async msg => await ProcessAsync(msg));

Data Access Libraries

EightBot.Nebula.DocumentDb

Azure Cosmos DB client with:

• Repository pattern implementation
• CRUD operations with strong typing
• Query support with LINQ
• Patch operations for partial updates

public class MyDocument : DocumentBase
{
    public string Name { get; set; }
}

// Use the data client
var doc = await dataClient.CreateAsync(new MyDocument { Name = "Test" });
var results = await dataClient.QueryAsync<MyDocument>(q => q.Where(x => x.Name == "Test"));

EightBot.Nebula.EntityFramework

Entity Framework 6 utilities:

• EntityBase - Base class with Id and timestamps
• AuditBase - Automatic audit trail tracking
• DataService<T> - Generic CRUD operations
• RepositoryService - Unit of work pattern

EightBot.Nebula.EntityFrameworkCore

Entity Framework Core utilities:

• EntityBase / EntityBaseAudit - Base classes with auditing
• Soft delete support with EntityBaseAuditDelete
• Generic data and repository services
• Automatic timestamp management

public class MyEntity : EntityBaseAudit
{
    public string Name { get; set; }
}

// Automatic CreatedDate, ModifiedDate tracking

EightBot.Nebula.EntityFramework.Identity

ASP.NET Identity integration:

• NebulaUserManager - Extended user management
• NebulaRoleManager - Role management
• NebulaIdentityManager - Combined identity operations

Infrastructure Libraries

EightBot.Nebula.Cache

Caching abstractions with multiple implementations:

• ICacheClient - Unified caching interface
• InMemoryCacheClient - Local memory cache
• RedisCacheClient - Distributed Redis cache

// Use either implementation interchangeably
await cacheClient.SetAsync("key", myObject, TimeSpan.FromMinutes(30));
var cached = await cacheClient.GetAsync<MyObject>("key");

Eightbot.Nebula.Configuration

Azure App Configuration integration:

• Simplified connection to Azure App Configuration
• Feature flag support
• Key Vault reference resolution

builder.Configuration.AddAzureAppConfiguration(options =>
{
    options.UseNebulaDefaults(connectionString);
});

EightBot.Nebula.RestClient

HTTP REST client with authentication support and optimized JSON serialization:

• RestClient - Base HTTP client with full request/response control
• RestClientBasic - Basic authentication (username/password)
• RestClientToken<TToken> - OAuth2/OpenID token authentication with automatic refresh
• High-performance JSON serialization using System.Text.Json with cached options
• Streaming response support for reduced memory usage
• GZip compression for uploads
• Source generator support for AOT compilation

Installation:

dotnet add package EightBot.Nebula.RestClient

Dependency Injection Setup:

// Register HttpClient with GZip decompression
services.AddDefaultHttpClient("api", DecompressionMethods.GZip);

// Register RestClient with optional custom JSON options
services.AddNebulaRestClient("api", jsonSerializerOptions);

Basic Usage:

// Simple GET request
var response = await restClient.GetJsonAsync<MyResponse>("/api/resource");

if (response.IsSuccessStatusCode)
{
    var data = response.Result;
}

// POST with JSON body
var createResponse = await restClient.PostJsonAsync<CreatedResource>(
    new { Name = "Test", Value = 42 }, 
    "/api/resources"
);

// With custom headers
var headers = new List<KeyValuePair<string, string>>
{
    new("X-Custom-Header", "value")
};
var response = await restClient.GetJsonAsync<MyResponse>("/api/resource", headers);

Basic Authentication:

var basicClient = new RestClientBasic(httpClient, logger);
basicClient.SetAuthorization("username", "password");

var response = await basicClient.GetJsonAsync<MyData>("/api/protected");

Token-Based Authentication:

var tokenClient = new RestClientToken<OAuth2Token>(httpClient, logger);

// Set the token (typically obtained from OAuth2 flow)
tokenClient.SetAuthorization(myOAuth2Token);

// Optional: Configure automatic token refresh
tokenClient.CheckToken = async (cancellationToken) =>
{
    if (tokenClient.Token.IsExpired)
    {
        // Refresh the token
        var newToken = await RefreshTokenAsync(tokenClient.Token.RefreshToken);
        tokenClient.SetAuthorization(newToken);
    }
};

var response = await tokenClient.GetJsonAsync<MyData>("/api/protected");

Configuration Options:

var restClient = new RestClient(httpClient, logger)
{
    // Enable streaming responses (default: true)
    // Allows deserialization before full response is received
    UseStreamingResponse = true,
    
    // Include full request/response in logs for debugging
    IncludeFullRequest = true,
    IncludeFullResponse = true,
    
    // Configure upload compression
    UploadType = UploadType.GZip,
    UploadBufferSize = 512,
    
    // Custom log name for tracing
    LogName = "MyApp.ApiClient",
    
    // Mask sensitive data in logs
    SensitiveTokens = new Dictionary<string, int>
    {
        { "password", 3 },  // Show first 3 chars only
        { "apiKey", 0 }     // Completely hide
    }
};

JSON Serialization Options:

The RestClient uses JsonSerializerOptionsCache for optimal performance:

// Use preset options
restClient.SerializerOptions = JsonSerializerOptionsCache.Default;  // camelCase, case-insensitive
restClient.SerializerOptions = JsonSerializerOptionsCache.Web;      // Web API defaults
restClient.SerializerOptions = JsonSerializerOptionsCache.Fast;     // Minimal features, max speed

// Custom options with caching
restClient.SerializerOptions = JsonSerializerOptionsCache.GetOrCreate(
    namingPolicy: JsonNamingPolicy.SnakeCaseLower,
    caseInsensitive: true
);

Source Generator Support (AOT-compatible):

For optimal performance and AOT compilation support, use source-generated JSON contexts:

// Define a source generator context
[JsonSerializable(typeof(MyDto))]
[JsonSerializable(typeof(List<MyDto>))]
[JsonSerializable(typeof(ApiErrorResponse))]
public partial class MyJsonContext : JsonSerializerContext { }

// Use with RestClient
var options = JsonSerializerOptionsCache.CreateWithContext(MyJsonContext.Default);
restClient.SerializerOptions = options;

// Or combine multiple contexts with fallback
var options = JsonSerializerOptionsCache.CreateWithResolvers(
    MyContext1.Default,
    MyContext2.Default,
    new DefaultJsonTypeInfoResolver()  // fallback for unknown types
);

Warm-up for Performance:

Pre-JIT frequently used types at startup to avoid first-request overhead:

// In Program.cs or startup
JsonSerializerOptionsCache.WarmUp<MyDto>();
JsonSerializerOptionsCache.WarmUp<List<MyDto>>();

// Or warm up multiple types at once
JsonSerializerOptionsCache.WarmUp(
    JsonSerializerOptionsCache.Default,
    typeof(MyDto), typeof(OtherDto), typeof(ApiResponse<>)
);

Error Handling:

// Configure global error processing
restClient.ProcessError = async (errorResponse) =>
{
    // Log, notify, or handle errors globally
    await LogErrorAsync(errorResponse);
};

// Check response status
var response = await restClient.GetJsonAsync<MyData>("/api/resource");

if (response.IsSuccessStatusCode)
{
    var data = response.Result;
}
else
{
    // Access error details
    var statusCode = response.StatusCode;
    var errorContent = response.Error;
}

File Downloads:

var downloadResponse = await restClient.DownloadAsync("/api/files/document.pdf");

if (downloadResponse.IsSuccessStatusCode)
{
    byte[] fileBytes = downloadResponse.Result;
    await File.WriteAllBytesAsync("document.pdf", fileBytes);
}

Utility Libraries

EightBot.Nebula.Extensions

Common extension methods:

• String: IsNullOrEmpty(), ToTitleCase(), Truncate(), ToSlug()
• DateTime: StartOfDay(), EndOfDay(), ToUnixTimestamp()
• Enumerable: ForEach(), Batch(), DistinctBy()
• Object: ToJson(), FromJson<T>()
• Stream: ToByteArray(), ToMemoryStream()

🔄 CI/CD Workflows

flowchart LR
    subgraph "Continuous Integration"
        Push[Push/PR] --> Build[Build All Solutions]
        Build --> Test[Run Tests]
        Test --> Coverage[Upload Coverage]
    end

    subgraph "Release Pipeline"
        Release[GitHub Release] --> BuildR[Build Release]
        BuildR --> Pack[Pack NuGet]
        Pack --> Publish[Publish to NuGet.org]
    end

    Push -.-> |feature branches| Build
    Release -.-> |tags| BuildR

Build Workflow

Triggers on push and pull requests to master, main, and feature/** branches:

• Matrix build for all 9 solutions
• Parallel execution on ubuntu-latest
• Test results and coverage artifacts uploaded

Release Workflow

Triggers on GitHub releases:

• Builds all solutions in Release configuration
• Packs all library projects
• Publishes to NuGet.org (requires NUGET_API_KEY secret)

🚀 Getting Started

Prerequisites

• .NET 8.0 SDK or later
• For Azure libraries: Azure Storage/Cosmos DB account
• For Redis cache: Redis server instance

Building from Source

# Clone the repository
git clone https://github.com/TheEightBot/Nebula.git
cd Nebula

# Build all solutions
dotnet build EightBot.Nebula.AzureBlob/EightBot.Nebula.AzureBlob.sln
dotnet build EightBot.Nebula.AzureQueue/EightBot.Nebula.AzureQueue.sln
dotnet build EightBot.Nebula.Cache/EightBot.Nebula.Cache.sln
dotnet build Eightbot.Nebula.Configuration/Eightbot.Nebula.Configuration.sln
dotnet build EightBot.Nebula.DocumentDb/EightBot.Nebula.DocumentDb.sln
dotnet build EightBot.Nebula.EntityFramework/EightBot.Nebula.EntityFramework.sln
dotnet build EightBot.Nebula.EntityFrameworkCore/EightBot.Nebula.EntityFrameworkCore.sln
dotnet build EightBot.Nebula.Extensions/EightBot.Nebula.Extensions.sln
dotnet build EightBot.Nebula.RestClient/EightBot.Nebula.RestClient.sln

# Run tests
dotnet test EightBot.Nebula.Extensions/EightBot.Nebula.Extensions.sln

Installation via NuGet

dotnet add package EightBot.Nebula.Extensions
dotnet add package EightBot.Nebula.AzureBlob
dotnet add package EightBot.Nebula.Cache
# ... etc

📁 Repository Structure

EightBot.Nebula/
├── .github/
│   └── workflows/
│       ├── build.yml          # CI build workflow
│       └── release.yml        # NuGet release workflow
├── EightBot.Nebula.AzureBlob/
│   ├── EightBot.Nebula.AzureBlob/
│   └── EightBot.Nebula.AzureBlob.Tests/
├── EightBot.Nebula.AzureQueue/
│   ├── EightBot.Nebula.AzureQueue/
│   └── EightBot.Nebula.AzureQueue.Tests/
├── EightBot.Nebula.Cache/
│   ├── EightBot.Nebula.Cache/
│   └── EightBot.Nebula.Cache.Tests/
├── Eightbot.Nebula.Configuration/
│   ├── Eightbot.Nebula.Configuration/
│   ├── Eightbot.Nebula.Configuration.Examples.Api/
│   ├── Eightbot.Nebula.Configuration.Examples.Function/
│   └── Eightbot.Nebula.Configuration.Tests/
├── EightBot.Nebula.DocumentDb/
│   ├── EightBot.Nebula.DocumentDb/
│   └── EightBot.Nebula.DocumentDb.Tests/
├── EightBot.Nebula.EntityFramework/
│   ├── EightBot.Nebula.EntityFramework/
│   ├── EightBot.Nebula.EntityFramework.Identity/
│   └── EightBot.Nebula.EntityFramework.Tests/
├── EightBot.Nebula.EntityFrameworkCore/
│   ├── EightBot.Nebula.EntityFrameworkCore/
│   └── EightBot.Nebula.EntityFrameworkCore.Tests/
├── EightBot.Nebula.Extensions/
│   ├── EightBot.Nebula.Extensions/
│   └── EightBot.Nebula.Extensions.Tests/
├── EightBot.Nebula.RestClient/
│   ├── EightBot.Nebula.RestClient/
│   ├── EightBot.Nebula.RestClient.ApiBasic/
│   ├── EightBot.Nebula.RestClient.ApiOAuth2/
│   └── EightBot.Nebula.RestClient.Testing/
├── nuget.config
└── README.md

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request