LocalAI.NET.Oobabooga 1.0.2

LocalAI.NET.Oobabooga

A .NET client library for interacting with Oobabooga's text-generation-webui through its OpenAI-compatible API endpoints. This library provides a simple, efficient way to use local LLMs in your .NET applications.

This package serves as the Oobabooga integration layer for the LocalAI.NET ecosystem.

Features

• OpenAI-compatible API support
• Text completion and chat completion
• Streaming responses support
• Character templates and instruction formats
• Comprehensive configuration options
• Built-in error handling and logging
• Cross-platform compatibility
• Full async/await support

📦 View Package on NuGet

Installation

Install via NuGet:

dotnet add package LocalAI.NET.Oobabooga

Quick Start

using LocalAI.NET.Oobabooga.Client;
using LocalAI.NET.Oobabooga.Models.Common;
using LocalAI.NET.Oobabooga.Models.Chat;

// Configure the client
var options = new OobaboogaOptions
{
    BaseUrl = "http://localhost:5000",  // Default port for text-generation-webui
    TimeoutSeconds = 120
};

// Create client instance
using var client = new OobaboogaClient(options);

// Simple completion
var response = await client.CompleteAsync(
    "Write a short story about a robot:",
    new OobaboogaCompletionOptions
    {
        MaxTokens = 200,
        Temperature = 0.7f,
        TopP = 0.9f
    });

Console.WriteLine(response);

// Chat completion
var messages = new List<OobaboogaChatMessage>
{
    new() { Role = "user", Content = "Write a poem about coding" }
};

var chatResponse = await client.ChatCompleteAsync(
    messages,
    new OobaboogaChatCompletionOptions
    {
        Mode = "instruct",
        InstructionTemplate = "Alpaca",
        MaxTokens = 200
    });

Console.WriteLine(chatResponse.Choices[0].Message.Content);

// Stream chat completion
await foreach (var message in client.StreamChatCompletionAsync(messages))
{
    Console.Write(message.Content);
}

Configuration Options

Basic Options

var options = new OobaboogaOptions
{
    BaseUrl = "http://localhost:5000",    // text-generation-webui server URL
    ApiKey = "optional_api_key",          // Optional API key for authentication
    TimeoutSeconds = 120                  // Request timeout
};

Chat Completion Options

var options = new OobaboogaChatCompletionOptions
{
    ModelName = "optional_model_name",    // Specific model to use
    MaxTokens = 200,                      // Maximum tokens to generate
    Temperature = 0.7f,                   // Randomness (0.0-1.0)
    TopP = 0.9f,                         // Nucleus sampling threshold
    StopSequences = new[] { "\n" },      // Stop sequences
    Mode = "chat",                       // "chat" or "instruct"
    InstructionTemplate = "Alpaca",      // Template for instruction format
    Character = "Assistant"              // Character template to use
};

Text Completion Options

var options = new OobaboogaCompletionOptions
{
    ModelName = "optional_model_name",
    MaxTokens = 200,
    Temperature = 0.7f,
    TopP = 0.9f,
    StopSequences = new[] { "\n" }
};

Error Handling

try
{
    var response = await client.ChatCompleteAsync(messages, options);
}
catch (OobaboogaException ex)
{
    Console.WriteLine($"Oobabooga error: {ex.Message}");
    Console.WriteLine($"Provider: {ex.Provider}");
    Console.WriteLine($"Status code: {ex.StatusCode}");
    Console.WriteLine($"Response content: {ex.ResponseContent}");
}
catch (Exception ex)
{
    Console.WriteLine($"General error: {ex.Message}");
}

Logging

The client supports Microsoft.Extensions.Logging:

ILogger logger = LoggerFactory
    .Create(builder => builder
        .AddConsole()
        .SetMinimumLevel(LogLevel.Debug))
    .CreateLogger<OobaboogaClient>();

var client = new OobaboogaClient(options, logger);

JSON Serialization

Custom JSON settings can be provided:

var jsonSettings = new JsonSerializerSettings
{
    NullValueHandling = NullValueHandling.Ignore
};

var client = new OobaboogaClient(options, logger, jsonSettings);

Testing

The library includes both unit and integration tests. Integration tests require a running text-generation-webui server.

To run the tests:

dotnet test

Configure test environment:

Environment.SetEnvironmentVariable("OOBABOOGA_BASE_URL", "http://localhost:5000");

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Support

For issues and feature requests, please use the GitHub issues page.