DeltaLake 0.1.0-rc.3

This is a prerelease version of DeltaLake.
There is a newer prerelease version of this package available.
See the version list below for details.
dotnet add package DeltaLake --version 0.1.0-rc.3                
NuGet\Install-Package DeltaLake -Version 0.1.0-rc.3                
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="DeltaLake" Version="0.1.0-rc.3" />                
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add DeltaLake --version 0.1.0-rc.3                
#r "nuget: DeltaLake, 0.1.0-rc.3"                
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install DeltaLake as a Cake Addin
#addin nuget:?package=DeltaLake&version=0.1.0-rc.3&prerelease

// Install DeltaLake as a Cake Tool
#tool nuget:?package=DeltaLake&version=0.1.0-rc.3&prerelease                

<p align="center"> <img src="https://raw.githubusercontent.com/johnsusi/delta-net/main/docs/logo.svg" alt="delta-net logo" height="200"> </p> <p align="center"> A dotnet library for Delta Lake. </p>

Introduction

This project uses Delta Lake, an open-source storage layer that brings ACID (Atomicity, Consistency, Isolation, Durability) transactions to big data workloads.

Delta Lake provides the ability to perform batch and streaming workloads on a single platform with high reliability and performance. It offers schema enforcement and evolution, ensuring data integrity. It also provides a full historical audit trail of all the changes made to the data.

Getting Started

Install the package

dotnet add package DeltaLake

Usage

Reading a table

import DeltaLake;

var table = new DeltaTable.Builder()
    .WithFileSystem("file:///path/to/table")
    .Build();

await foreach (var batch in table.GetRecordBatches())
{

}

Reading a typed table

record FooTable : ITable<Foo>
{

    public static Schema Schema { get; } = new([
        new("id", Int32Type.Default, false, []),
        new("foo", StringType.Default, true, [])
    ], []);

    public static IEnumerable<TestTable> Enumerate(RecordBatch batch)
    {
        for (int i = 0; i < batch.Length; i++)
        {
            var idArray = batch.Column(0) as IReadOnlyList<int?> ?? throw new Exception("Expected non-null array");
            var fooArray = batch.Column(1) as IReadOnlyList<string?> ?? throw new Exception("Expected non-null array");
            yield return new TestTable()
            {
                Id = idArray[i] ?? throw new Exception("Cannot be null"),
                Foo = fooArray[i]
            };
        }
    }
}

var table = new DeltaTable<FooTable>.Builder()
    .WithFileSystem("file:///path/to/table")
    .Build();

await foreach (var row in table.ReadAll())
{
    Console.WriteLine("{0}", row);
}

Create a table

import DeltaLake;

var table = new DeltaTable.Builder()
    .WithFileSystem("file:///path/to/table")
    .WithSchema(schema)
    .EnsureCreated()
    .Build();

Update a table

import DeltaLake;

table = ...;

using var data = new RecordBatch(table.Schema, [
    new Int32Array
        .Builder()
        .Append(1)
        .Append(2)
        .Append(3)
        .Build(),
    new StringArray
        .Builder()
        .Append("one")
        .AppendNull()
        .Append("two")
        .Build(),
], 3);

table = new DeltaTable.Builder()
    .FromTable(table)
    .Add(data)
    .Build();

Product Compatible and additional computed target framework versions.
.NET net8.0 is compatible.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (1)

Showing the top 1 NuGet packages that depend on DeltaLake:

Package Downloads
DeltaLake.Azure

A native .NET library for Delta Lake in Azure Data Lake

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
0.1.0-rc.8 54 7/3/2024
0.1.0-rc.7 199 4/25/2024
0.1.0-rc.6 80 4/24/2024
0.1.0-rc.5 142 4/21/2024
0.1.0-rc.4 46 4/21/2024
0.1.0-rc.3 50 4/21/2024
0.1.0-rc.2 83 4/11/2024
0.1.0-rc.1 59 4/11/2024
0.1.0-alpha.2 58 3/23/2024