Whisper.net.Runtime.Cuda
1.7.3
dotnet add package Whisper.net.Runtime.Cuda --version 1.7.3
NuGet\Install-Package Whisper.net.Runtime.Cuda -Version 1.7.3
<PackageReference Include="Whisper.net.Runtime.Cuda" Version="1.7.3" />
paket add Whisper.net.Runtime.Cuda --version 1.7.3
#r "nuget: Whisper.net.Runtime.Cuda, 1.7.3"
// Install Whisper.net.Runtime.Cuda as a Cake Addin #addin nuget:?package=Whisper.net.Runtime.Cuda&version=1.7.3 // Install Whisper.net.Runtime.Cuda as a Cake Tool #tool nuget:?package=Whisper.net.Runtime.Cuda&version=1.7.3
Whisper.net
Open-Source Whisper.net
Dotnet bindings for OpenAI Whisper made possible by whisper.cpp
Build Status
Build type | Build Status |
---|---|
CI Status (Native + dotnet) |
Getting Started
To install Whisper.net with all the available runtimes, run the following command in the Package Manager Console:
PM> Install-Package Whisper.net.AllRuntimes
Or add a package reference in your .csproj
file:
<PackageReference Include="Whisper.net.AllRuntimes" Version="1.7.2" />
Whisper.net
is the main package that contains the core functionality but does not include any runtimes. Whisper.net.AllRuntimes
includes all available runtimes for Whisper.net.
Installing Specific Runtimes
To install a specific runtime, you can install them individually and combine them as needed. For example, to install the CPU runtime, add the following package references:
<PackageReference Include="Whisper.net" Version="1.7.2" />
<PackageReference Include="Whisper.net.Runtime" Version="1.7.2" />
GPT for Whisper
We also have a custom-built GPT inside ChatGPT, which can help you with information based on this code, previous issues, and releases. Available here.
Please try to ask it before publishing a new question here, as it can help you a lot faster.
Runtimes Description
Whisper.net comes with multiple runtimes to support different platforms and hardware acceleration. Below are the available runtimes:
Whisper.net.Runtime
The default runtime that uses the CPU for inference. It is available on all platforms and does not require any additional dependencies.
Examples:
- Simple usage example
- Simple usage example (without Async processing)
- NAudio integration for mp3
- NAudio integration for resampled wav
- Simple channel diarization
- Blazor example
Pre-requisites
- Windows: Microsoft Visual C++ Redistributable for at least Visual Studio 2019 (x64) Download Link
- Linux:
libstdc++6
,glibc 2.31
- macOS: TBD
- For x86/x64 platforms, the CPU must support AVX, AVX2, FMA and F16C instructions. If your CPU does not support these instructions, you'll need to use the
Whisper.net.Runtime.NoAvx
runtime instead.
Supported Platforms
- Windows x86, x64, ARM64
- Linux x64, ARM64, ARM
- macOS x64, ARM64 (Apple Silicon)
- Android
- iOS
- MacCatalyst
- tvOS
- WebAssembly
Whisper.net.Runtime.NoAvx
For CPUs that do not support AVX instructions.
Pre-requisites
- Windows: Microsoft Visual C++ Redistributable for at least Visual Studio 2019 (x64) Download Link
- Linux:
libstdc++6
,glibc 2.31
- macOS: TBD
Supported Platforms
- Windows x86, x64, ARM64
- Linux x64, ARM64, ARM
Whisper.net.Runtime.Cuda
Contains the native whisper.cpp library with NVidia CUDA support enabled.
Examples
Pre-requisites
- Everything from Whisper.net.Runtime pre-requisites
- NVidia GPU with CUDA support
- CUDA Toolkit (>= 12.1)
Supported Platforms
- Windows x64
- Linux x64
Whisper.net.Runtime.CoreML
Contains the native whisper.cpp library with Apple CoreML support enabled.
Examples:
Supported Platforms
- macOS x64, ARM64 (Apple Silicon)
- iOS
- MacCatalyst
- tvOS
Whisper.net.Runtime.OpenVino
Contains the native whisper.cpp library with Intel OpenVino support enabled.
Examples
Pre-requisites
- Everything from Whisper.net.Runtime pre-requisites
- OpenVino Toolkit (>= 2024.4)
Supported Platforms
- Windows x64
- Linux x64
Whisper.net.Runtime.Vulkan
Contains the native whisper.cpp library with Vulkan support enabled.
Examples
Pre-requisites
- Everything from Whisper.net.Runtime pre-requisites
- Vulkan Toolkit (>= 1.3.290.0)]
Supported Platforms
- Windows x64
Multiple Runtimes Support
You can install and use multiple runtimes in the same project. The runtime will be automatically selected based on the platform you are running the application on and the availability of the native runtime.
The following order of priority will be used by default:
Whisper.net.Runtime.Cuda
(NVidia devices with all drivers installed)Whisper.net.Runtime.Vulkan
(Windows x64 with Vulkan installed)Whisper.net.Runtime.CoreML
(Apple devices)Whisper.net.Runtime.OpenVino
(Intel devices)Whisper.net.Runtime
(CPU inference)Whisper.net.Runtime.NoAvx
(CPU inference without AVX support)
To change the order or force a specific runtime, set the RuntimeOrder
on the RuntimeOptions
:
RuntimeOptions.Instance.SetRuntimeLibraryOrder(new List<RuntimeLibrary>()
{
RuntimeLibrary.CoreML,
RuntimeLibrary.OpenVino,
RuntimeLibrary.Cuda,
RuntimeLibrary.Cpu
});
Versioning
Each version of Whisper.net is tied to a specific version of Whisper.cpp. The version of Whisper.net is the same as the version of Whisper it is based on. For example, Whisper.net 1.2.0 is based on Whisper.cpp 1.2.0.
However, the patch version is not tied to Whisper.cpp. For example, Whisper.net 1.2.1 is based on Whisper.cpp 1.2.0 and Whisper.net 1.7.0 is based on Whisper.cpp 1.7.1.
Ggml Models
Whisper.net uses Ggml models to perform speech recognition and translation. You can find more about Ggml models here.
For easier integration, Whisper.net provides a Downloader using Hugging Face.
var modelName = "ggml-base.bin";
if (!File.Exists(modelName))
{
using var modelStream = await WhisperGgmlDownloader.GetGgmlModelAsync(GgmlType.Base);
using var fileWriter = File.OpenWrite(modelName);
await modelStream.CopyToAsync(fileWriter);
}
Usage
using var whisperFactory = WhisperFactory.FromPath("ggml-base.bin");
using var processor = whisperFactory.CreateBuilder()
.WithLanguage("auto")
.Build();
using var fileStream = File.OpenRead(wavFileName);
await foreach (var result in processor.ProcessAsync(fileStream))
{
Console.WriteLine($"{result.Start}->{result.End}: {result.Text}");
}
Documentation
You can find the documentation and code samples here.
Building The Runtime
This section describes how to build the native runtime libraries for Whisper.net. Normally, you would not need to build the runtime libraries yourself, as they are available as NuGet packages.
The build scripts are a combination of PowerShell scripts and a Makefile. You can read each of them for the underlying cmake
commands being used, or run them directly from the scripts.
You can also check the github actions available here
Android
make android
Before running, create an environment variable for NDK_PATH
with the path to your Android NDK. For example:
NDK_PATH=/Users/UserName/Library/Developer/Xamarin/android-sdk-macosx/ndk-bundle
Apple
make apple
Compiling the Apple libraries requires a Mac with Xcode installed.
Apple CoreML
make apple_coreml
Compiling the Apple libraries requires a Mac with Xcode installed.
Linux
make linux
Windows
Import the PowerShell module:
Import-Module ./windows-scripts.ps1
Run BuildWindowsAll
to build all Windows libraries.
Alternatively, you can run BuildWindows
with the desired parameters.
BuildWindows -Arch "x64" -Configuration "Release" -NoAvx $true
License
MIT License. See LICENSE for details.
Learn more about Target Frameworks and .NET Standard.
-
- Whisper.net.Runtime.Cuda.Linux (>= 1.7.3)
- Whisper.net.Runtime.Cuda.Windows (>= 1.7.3)
NuGet packages (1)
Showing the top 1 NuGet packages that depend on Whisper.net.Runtime.Cuda:
Package | Downloads |
---|---|
Whisper.net.AllRuntimes
Whisper.net.Runtime.Cuda contains the native runtime libraries to enable Whisper on .NET with Whisper.net and GPU support (Cuda) |
GitHub repositories (1)
Showing the top 1 popular GitHub repositories that depend on Whisper.net.Runtime.Cuda:
Repository | Stars |
---|---|
sandrohanea/whisper.net
Whisper.net. Speech to text made simple using Whisper Models
|
Version | Downloads | Last updated |
---|---|---|
1.7.3 | 296 | 12/1/2024 |
1.7.3-preview2 | 109 | 11/29/2024 |
1.7.3-preview1 | 103 | 11/28/2024 |
1.7.2 | 656 | 11/10/2024 |
1.7.2-preview3 | 143 | 11/5/2024 |
1.7.2-preview2 | 112 | 10/31/2024 |
1.7.2-preview1 | 104 | 10/27/2024 |
1.7.1 | 425 | 10/17/2024 |
1.7.1-preview1 | 102 | 10/14/2024 |
1.7.0 | 226 | 10/8/2024 |