Mozo.Fwob 1.1.0

There is a newer version of this package available.
See the version list below for details.
dotnet add package Mozo.Fwob --version 1.1.0
NuGet\Install-Package Mozo.Fwob -Version 1.1.0
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Mozo.Fwob" Version="1.1.0" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add Mozo.Fwob --version 1.1.0
#r "nuget: Mozo.Fwob, 1.1.0"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install Mozo.Fwob as a Cake Addin
#addin nuget:?package=Mozo.Fwob&version=1.1.0

// Install Mozo.Fwob as a Cake Tool
#tool nuget:?package=Mozo.Fwob&version=1.1.0

Fixed-Width Ordered Binary (FWOB) Format Library

NuGet version (Mozo.Fwob) Build workflow Release workflow

This repository contains the library implementation of the FWOB file format.

Features

  • Enfores increasing order of data frames by key
  • Binary format storage
  • User-defined flat data structure
  • High-efficient (de)serializer (40% performance vs the best C++ implementation)
  • Built-in support for a string table
  • Supports on-disk file and in-memory storage

Capacity

  • File length: up to 2^63 - 1 bytes (8,192 PiB)
  • String table length: up to 2^31 - 1 bytes (2 GiB)
  • Fwob title: up to 16 bytes
  • Frame name: up to 16 bytes
  • Frame type: up to 16 fields
  • Field name: up to 8 bytes
  • Field type: of any primitive types (non-nullable) and string type
  • String length: up to 255 bytes

Build from source

# Restore any necessary imported packages
dotnet restore

# Build the entire solution
dotnet build --no-restore

# Run the test cases in the test directory and generate coverage report
dotnet test --no-build --verbosity normal /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura

# Visualize coverage report in HTML (requires the dotnet tool dotnet-reportgenerator-globaltool to be installed)
reportgenerator -reports:test/coverage.cobertura.xml -targetdir:CoverageReport -reporttypes:Html -historydir:CoverageHistory

# Publish a portable .dll for any supported system that have the .NET Runtime/SDK installed
dotnet publish -c Release --no-self-contained

# Pack the built manifests into a .nupkg package
dotnet pack -c Release

How to Use

Define your frame data structure

The data structure can be any class or record class that implements Mozo.Fwob.Models.IFrame<TKey>, where TKey is a struct type and implements IComparable<TKey>.

The data structure can have fields, methods, indexers, properties, custom parameterized constructors and non-public members. But only public fields and the parameterless constructor will be accessed by the FWOB library.

Since the frame must be fixed-width, a field of string type must define a fixed length. This can be achieved by annotating with the Mozo.Fwob.LengthAttribute.

Here is an example,

public class StockTick : IFrame<uint>
{
    public uint Price;

    [Key]
    public uint Time;

    public double RealPrice
    {
        get => Price / 10000.0;
        set => Price = (uint)Math.Round(value * 10000);
    }

    public int Size;

    [Ignore]
    public int Extra;

    [Length(4)]
    public string SpecCond;
}

Only the public fields Price, Time, Size and SpecCond will hold data in a FWOB file. The RealPrice property, which is not a field, and the Extra field, which is ignored with the Mozo.Fwob.IgnoreAttribute annotation, won't hold any data in the file. By convention, the library use the first field defined of TKey type as the key. To specify a custom key, use the Mozo.Fwob.KeyAttribute to annotate a field of TKey type. The key field will be used by the library to compare and order the frames in the file.

The library checks the schema in the initialization of either an FwobFile<TFrame, TKey> or an InMemoryFwobFile<TFrame, TKey> file object. Exceptions will be thrown if conflicts is detected or rules are violated.

Create an on-disk FWOB file

var fwobFile = FwobFile<StockTick, uint>.CreateNew(fileName, "FileTitle");

Open an existing on-disk FWOB file

var fwobFile = new FwobFile<StockTick, uint>(fileName);

Create an in-memory FWOB file

var fwobFile = new InMemoryFwobFile<StockTick, uint>(title);

Read frames

// Get the first frame in the file
var firstFrame = fwobFile.FirstFrame;

// Get the last frame in the file
var lastFrame = fwobFile.LastFrame;

// Get the first frame of a given key in the file
var frame = fwobFile.GetFrame(key);

// Get an enumerator for iterating the frames of a given key in the file, in case the key is not unique
var frames = fwobFile.GetFrames(key);

// Get an enumerator for iterating the frames of a given key range [firstKey, lastKey) in the file
var frames = fwobFile.GetFrames(firstKey, lastKey);

// Get an enumerator for iterating the frames of a given lower bound (inclusive) in the file
var frames = fwobFile.GetFramesAfter(firstKey);

// Get an enumerator for iterating the frames of a given higher bound (inclusive) in the file
var frames = fwobFile.GetFramesBefore(lastKey);

Write frames

// Append frames to the end of the file, only the ascending prefixing frames will be taken
long appendedCount = fwobFile.AppendFrames(frames);

// Append frames to the end of the file while enforcing the ascending order by key and no data will be appended if the ordering rule is violated
long appendedCount = fwobFile.AppendFramesTx(frames);

Delete frames

// Deletes all frames whose key is greater than or equal to the given key
long deletedCount = fwobFile.DeleteFramesAfter(firstKey);

// Deletes all frames whose key is less than or equal to the given key
long deletedCount = fwobFile.DeleteFramesBefore(lastKey);

// Deletes all frames from the storage
fwobFile.ClearFrames();

String table

// Load and unload the string table into memory, not needed for the in-memory storage
fwobFile.LoadStringTable();
fwobFile.UnloadStringTable();

// The string table is accessible via a list
var list = fwobFile.Strings;

// Get a specific string at a given index
var str = fwobFile.GetString(index);

// Get a specific index of a given string
int index = fwobFile.GetString(str);

// Append a string to the string table if not exist
int index = fwobFile.AppendString(str);

// Check if a string is in the string table
bool exist = fwobFile.ContainsString(str);

// Remove all strings from the string table
fwobFile.ClearStrings();
Product Compatible and additional computed target framework versions.
.NET net5.0 was computed.  net5.0-windows was computed.  net6.0 was computed.  net6.0-android was computed.  net6.0-ios was computed.  net6.0-maccatalyst was computed.  net6.0-macos was computed.  net6.0-tvos was computed.  net6.0-windows was computed.  net7.0 is compatible.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 was computed.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
.NET Core netcoreapp2.0 was computed.  netcoreapp2.1 was computed.  netcoreapp2.2 was computed.  netcoreapp3.0 was computed.  netcoreapp3.1 was computed. 
.NET Standard netstandard2.0 is compatible.  netstandard2.1 was computed. 
.NET Framework net461 was computed.  net462 was computed.  net463 was computed.  net47 was computed.  net471 was computed.  net472 was computed.  net48 is compatible.  net481 was computed. 
MonoAndroid monoandroid was computed. 
MonoMac monomac was computed. 
MonoTouch monotouch was computed. 
Tizen tizen40 was computed.  tizen60 was computed. 
Xamarin.iOS xamarinios was computed. 
Xamarin.Mac xamarinmac was computed. 
Xamarin.TVOS xamarintvos was computed. 
Xamarin.WatchOS xamarinwatchos was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.
  • .NETFramework 4.8

    • No dependencies.
  • .NETStandard 2.0

    • No dependencies.
  • net7.0

    • No dependencies.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
1.5.0 213 4/8/2023
1.4.3 163 4/8/2023
1.4.2 170 4/7/2023
1.4.1 187 4/1/2023
1.4.0 185 4/1/2023
1.3.0 220 3/8/2023
1.2.0 226 2/27/2023
1.1.0 239 2/26/2023
1.0.0 299 2/19/2023