Regira.IO.Storage 6.6.1

Regira IO.Storage

Regira IO.Storage provides a unified abstraction for file storage operations across multiple backends. All implementations share the same IFileService interface, making storage backends interchangeable in consuming code.

Projects

Installation

<PackageReference Include="Regira.IO.Storage" Version="6.*" />


<PackageReference Include="Regira.IO.Storage.Azure" Version="6.*" />


<PackageReference Include="Regira.IO.Storage.SSH" Version="6.*" />


<PackageReference Include="Regira.IO.Storage.GitHub" Version="6.*" />

Quick Start

services.AddSingleton<IFileService>(_ =>
    new BinaryFileService(new FileSystemOptions { RootFolder = "/var/app/uploads" }));

// In a service
var bytes = await storage.GetBytes("invoices/2024/inv-001.pdf");
await storage.Save("exports/report.pdf", pdfBytes);

IFileService

All backends implement this interface. Identifiers are relative paths within the storage root (e.g. "folder/file.pdf"). URIs are the absolute addresses returned by GetAbsoluteUri.

Read

Task<bool>                Exists(string identifier)
Task<byte[]?>             GetBytes(string identifier)
Task<Stream?>             GetStream(string identifier)
Task<StoredFileInfo?>     GetInfo(string identifier)
Task<IEnumerable<string>> List(FileSearchObject? so = null)

GetInfo answers a file's size and last-modified time (UTC) from the store's metadata, without reading the content, and null for a missing file or a folder. GitHub leaves LastModified null. A zip entry's size is the one the archive records, and its time is its DOS time, kept to two seconds and read in the server's time zone, so it is exact only for an archive written in that zone. An IFileService of your own that does not implement GetInfo opens the content with GetStream to measure it, leaves LastModified null, and answers null when GetStream does: a store whose GetStream throws for a missing file or a folder implements GetInfo itself.

Write

Task<string> Save(string identifier, byte[] bytes,  string? contentType = null)
Task<string> Save(string identifier, Stream stream, string? contentType = null)
Task         Move(string sourceIdentifier, string targetIdentifier)
Task         Delete(string identifier)

Save returns the final identifier used — it may differ if the backend renames on conflict.

Passing an IMemoryFile to Save? Read it with GetBytes(), not .Bytes. IMemoryFile extends both IMemoryBytesFile (Bytes) and IMemoryStreamFile (Stream), and a producer fills exactly one — which one varies per method, not per implementation. .Bytes is therefore null for anything built from a stream (an Excel export, a PDF merge, a database backup), and Save(identifier, null!) writes an empty file with no exception. GetBytes()/GetStream() (MemoryFileExtensions, Regira.IO.Extensions in Regira.Common) normalise both halves.

URI helpers

string  Root { get; }                        // storage root URI / path
string  GetAbsoluteUri(string identifier)    // relative → absolute
string  GetIdentifier(string uri)            // absolute → relative
string? GetRelativeFolder(string identifier) // extract parent folder

File Identification

Files are addressed uniformly across all backends using three coordinated concepts.

IFileService.Root

Every IFileService implementation exposes a Root — the backend-specific base address for that storage scope:

All IFileService methods (GetBytes, Save, List, …) accept and return identifiers — paths relative to this root — keeping consuming code independent of the backend.

IBinaryFile.Identifier and IBinaryFile.Prefix

BinaryFileItem (and anything implementing IBinaryFile / IStorageFile) carries two address parts:

Root        →  /var/app/storage/
Prefix      →                   invoices/2024/
FileName    →                                 inv-001.pdf
Identifier  →                   invoices/2024/inv-001.pdf
Path        →  /var/app/storage/invoices/2024/inv-001.pdf

The local, SFTP, blob and GitHub backends resolve every identifier against the root and throw UnauthorizedAccessException when it escapes it (through ../, say). The blob and GitHub backends check the address as a URL reads it, %2e as a dot, and as a server that decodes %2f into a / would; they and the SFTP backend also refuse a null character, where a server may end the path. Zip extraction to a folder (the targetDirectory overloads of ZipUtility.Unzip) checks every entry the same way, while ZipUtility.Unzip(IBinaryFile) returns each entry name as stored, ../x included — check one before using it as a path. Folder names compare as the file system does: regardless of case on Windows and macOS, exactly elsewhere. The check is on by default (Contained = true on FileSystemOptions, SftpConfig, AzureOptions and GitHubOptions); turn it off only for trusted input. An empty RootFolder turns it off too: the file-system identifiers are then paths, relative to the working directory or absolute, so set one whenever an identifier comes from a user.

Converting between identifier and absolute URI

IFileService provides helpers to move between the two representations:

IFileService service = new BinaryFileService(new FileSystemOptions { RootFolder = "/var/app/storage" });

string absolute = service.GetAbsoluteUri("invoices/2024/inv-001.pdf");
// → /var/app/storage/invoices/2024/inv-001.pdf  (or the equivalent Azure/SFTP URL)

string identifier = service.GetIdentifier("/var/app/storage/invoices/2024/inv-001.pdf");
// → invoices/2024/inv-001.pdf

string? folder = service.GetRelativeFolder("invoices/2024/inv-001.pdf");
// → invoices/2024

Use Identifier as the portable key that survives a backend swap; only resolve to Path when you need the actual physical/network address.

FileSearchObject

Filter parameter for List().

IFileService storage = new BinaryFileService(new FileSystemOptions { RootFolder = "/var/app/storage" });

var images = await storage.List(new FileSearchObject
{
    FolderUri  = "products/",
    Extensions = [".jpg", ".webp"],
    Recursive  = true,
    Type       = FileEntryTypes.Files
});

Implementations

File System (BinaryFileService)

Stores files on the local disk.

Package: Regira.IO.Storage

var service = new BinaryFileService(new FileSystemOptions { RootFolder = "/var/app/storage" });

Network shares — for a UNC path protected by a username & password, use NetworkFileService with a NetworkShareCommunicator. The communicator authenticates against the share lazily on the first file operation (or eagerly via await communicator.Open()); dispose it on application shutdown to release the connection.

services.AddSingleton(new NetworkFileSystemOptions
{
    RootFolder = @"\\fileserver\share\uploads",
    UserName   = configuration["Storage:Share:UserName"]!,
    Password   = configuration["Storage:Share:Password"],
    Domain     = configuration["Storage:Share:Domain"]   // optional — or "DOMAIN\user" as UserName
});
services.AddSingleton<NetworkShareCommunicator>();
services.AddSingleton<IFileService, NetworkFileService>();

Windows only (uses the WNet API). On Linux/macOS, mount the share at OS level (e.g. mount.cifs) and use plain FileSystemOptions. Connections are ref-counted per share + user across the process — the share connection is only released when the last communicator on it is disposed. If the connection is cancelled outside the process (e.g. net use /delete), call await communicator.Reconnect() to re-establish it; Close() only drops this communicator's reference.

Text files — use TextFileService directly, or wrap any IFileService with the DefaultTextFileService decorator:

var text = new DefaultTextFileService(anyFileService, Encoding.UTF8);
string? content = await text.GetContents("config/app.json");
await text.Save("config/app.json", jsonString);

Azure Blob Storage (BinaryBlobService)

Package: Regira.IO.Storage.Azure — NuGet dependency: Azure.Storage.Blobs

var communicator = new AzureCommunicator(new AzureOptions
{
    ConnectionString = "DefaultEndpointsProtocol=https;AccountName=…",
    ContainerName    = "my-container"
});
await communicator.Open();   // idempotent — safe to call multiple times

var service = new BinaryBlobService(communicator);

Save stores its contentType as the blob's Content-Type, which decides whether a browser following a SAS or CDN link shows the file or downloads it. Without one — null, empty or blank — the type comes from the identifier's extension. Never pass an upload's IFormFile.ContentType: the client chose it, so an avatar.png declared text/html would be served as a web page. Leave the argument out, or derive it from the name with ContentTypeUtility.GetContentType(fileName). Save sets no Content-Encoding; a blob that already carries one keeps it until it is saved again.

SSH / SFTP (SftpService)

Package: Regira.IO.Storage.SSH — NuGet dependency: SSH.NET

var communicator = new SftpCommunicator(new SftpConfig
{
    Host               = "sftp.example.com",
    Port               = 22,
    UserName           = "deploy",
    Password           = configuration["Sftp:Password"],
    ContainerName      = "/home/deploy/files",
    HostKeyFingerprint = "SHA256:ohD8VZEXGWo6Ez8GSEJQ9WpafgLFsOfLOtGGQCQo6Og"   // the key type the server negotiates: ssh-keyscan sftp.example.com | ssh-keygen -lf - lists one per type
});

var service = new SftpService(communicator);

SftpCommunicator holds a single persistent connection. Dispose it on application shutdown.

GitHub (GitHubService)

Package: Regira.IO.Storage.GitHub — NuGet dependency: none (uses HttpClient)

ISerializer jsonSerializer = new JsonSerializer();   // e.g. Regira.Serializing.Newtonsoft

var service = new GitHubService(
    new GitHubCommunicator(new GitHubOptions
    {
        Uri       = "https://api.github.com/repos/owner/repo",
        Key       = configuration["GitHub:Token"],   // PAT — optional for public-repo reads; keep it out of source
        UserAgent = "MyApp/1.0"
    }),
    jsonSerializer
);

Every communicator sends through a connection pool the package shares, and every request carries its own headers, the token included.

Fully implements IFileService — Save/Move/Delete create commits on Branch, so it's not suited for high-frequency writes. A file over 1 MB, whose content the contents API leaves out, is read from its git blob in a second request (GitHub stores files up to 100 MB). Such a read is held in memory whole, more than once over while it reads, and throws TaskCanceledException when it takes longer than HttpClient's 100-second timeout, as every request does: a 100 MB file needs about 8 Mbit/s. An identifier ending in ?ref={branch} reads and writes that branch instead, so a file name cannot hold a ?; a # is part of the name.

ZIP / Compression

ZipFileService — browse an archive via IFileService

ZipFileService implements IFileService and IDisposable. Construct it with a ZipFileCommunicator that points to an existing archive or starts a fresh one:

// Open an existing zip file
using var zipService = new ZipFileService(new ZipFileCommunicator { SourceFile = existingZip });
var entries = await zipService.List(new FileSearchObject { Recursive = true, Type = FileEntryTypes.Files }); // every file, not only the top level
var bytes   = await zipService.GetBytes("report.pdf");

// Start a new empty archive
using var newZip = new ZipFileService(new ZipFileCommunicator());
await newZip.Save("data.csv", csvBytes);

List and Save answer identifiers with / between folders, and a FolderUri — written with either separator — matches that folder as a whole: dir2/dir2.1 does not include dir2/dir2.10. List reads a FileSearchObject as BinaryFileService does: the folder's own entries unless Recursive, and files, folders or both as Type says. The folders are those the entries lie in, so an archive lists them whether or not it holds an entry of their own.

ZipFileServiceFactory is a convenience wrapper: new ZipFileServiceFactory().Create(sourceFile, password) is equivalent to constructing ZipFileService directly.

ZipBuilder — create archives

byte[] pdfBytes = [/* … */], csvBytes = [/* … */];

IMemoryFile zip = await new ZipBuilder()
    .For([new BinaryFileItem { FileName = "report.pdf", Bytes = pdfBytes },
          new BinaryFileItem { FileName = "data.csv",   Bytes = csvBytes }])
    .Build();

ZipUtility — zip/unzip helpers

Zip is an extension method; Unzip is a static method on ZipUtility.

IMemoryFile zipFromFiles    = files.Zip();                             // collection → zip
IMemoryFile zipFromPaths    = paths.Zip(baseFolder: "/var/exports");   // paths → zip
BinaryFileCollection items  = ZipUtility.Unzip(existingZip);           // zip → collection
string[] extracted          = ZipUtility.Unzip(existingZip, targetDirectory: "/tmp/out");

An entry is named after the file's Identifier (its FileName when there is none) with / between folders, as the ZIP format requires, so an archive made on Windows unzips into the same folders on Linux and macOS. Unzip reads a \ in an entry name as a separator too.

Neither ZipUtility.Unzip nor ZipFileService caps what an archive unpacks to. For one from an untrusted source, such as an upload, use ZipManager with MaxUnzippedSize (Compression).

Helpers

FileProcessor — recursive processing

IFileService fileService = new BinaryFileService(new FileSystemOptions { RootFolder = "/var/app/storage" });

await new FileProcessor(fileService).ProcessFiles(
    new FileSearchObject { FolderUri = "exports/", Recursive = true },
    async (uri, svc) => { /* process each file */ }
);

The handler runs for one file at a time and receives the file's absolute URI. Recursive = true lists the files of every subfolder in one listing; processRecursively: true lists folder by folder instead. A handler that wants parallel work adds it itself, with a bound.

FileNameHelper — unique filenames

IFileService fileService = new BinaryFileService(new FileSystemOptions { RootFolder = "/var/app/storage" });

var helper = new FileNameHelper(fileService);
string safe = await helper.NextAvailableFileName("invoices/report.pdf");
// → "invoices/report-(1).pdf" when "invoices/report.pdf" already exists

Customise the pattern: new FileNameHelper.Options { NumberPattern = " ({0})" }

ExportHelper — copy between services

IFileService source = new BinaryFileService(new FileSystemOptions { RootFolder = "/var/app/storage" });
IFileService target = new BinaryFileService(new FileSystemOptions { RootFolder = "/mnt/backup" });

await new ExportHelper(source, target)
    .Export(new FileSearchObject { FolderUri = "backups/", Recursive = true });

FileNameUtility (Regira.IO.Storage.FileSystem) — path helpers

FileNameUtility.GetAbsoluteUri("folder/file.txt", root)
FileNameUtility.GetRelativeUri(absolutePath, root)
FileNameUtility.GetCleanFileName("folder/sub/file.txt")  // → "file.txt"
FileNameUtility.Combine("folder", "sub", "file.txt")
FileNameUtility.SanitizeFilename(@"CON\report:v2.txt")   // → "_XXX_/report_v2.txt" ('\' on Windows) — replaces the characters Windows rejects and any path segment that exactly matches a Windows reserved name ("con.txt" is left as-is), on every platform
FileNameUtility.GetUncShareRoot(@"\\server\share\sub")   // → @"\\server\share" (null for non-UNC)

SanitizeFilename makes a path valid, not safe: .. segments and a leading separator pass through. The file services refuse an identifier that leaves their root while Contained is on (the default); for a path you build yourself from untrusted input, FileNameUtility.EnsureContained(Path.Combine(root, path), root) returns the full path or throws UnauthorizedAccessException.

Overview

1. Index — Overview, interface, and implementation reference
2. Examples — Backend swap, transform & re-upload, GitHub→Azure mirror, ZIP export, safe upload
3. Compression — SharpZipLib's ZipManager: password-protected ZIP, and a size cap for an untrusted archive

License

Apache License 2.0 — this package contains no license validation and no runtime limits. See LICENSE. A few companion packages are commercially licensed with a free tier; see the licensing overview.