Microsoft.ML.Tokenizers
0.21.0-preview.22621.2
Prefix Reserved
This is a prerelease version of Microsoft.ML.Tokenizers.
There is a newer version of this package available.
See the version list below for details.
See the version list below for details.
dotnet add package Microsoft.ML.Tokenizers --version 0.21.0-preview.22621.2
NuGet\Install-Package Microsoft.ML.Tokenizers -Version 0.21.0-preview.22621.2
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="Microsoft.ML.Tokenizers" Version="0.21.0-preview.22621.2" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add Microsoft.ML.Tokenizers --version 0.21.0-preview.22621.2
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
#r "nuget: Microsoft.ML.Tokenizers, 0.21.0-preview.22621.2"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install Microsoft.ML.Tokenizers as a Cake Addin #addin nuget:?package=Microsoft.ML.Tokenizers&version=0.21.0-preview.22621.2&prerelease // Install Microsoft.ML.Tokenizers as a Cake Tool #tool nuget:?package=Microsoft.ML.Tokenizers&version=0.21.0-preview.22621.2&prerelease
The NuGet Team does not provide support for this client. Please contact its maintainers for support.
Microsoft.ML.Tokenizers contains the implmentation of the tokenization used in the NLP transforms.
| Product | Versions Compatible and additional computed target framework versions. |
|---|---|
| .NET | net5.0 was computed. net5.0-windows was computed. net6.0 was computed. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
| .NET Core | netcoreapp2.0 was computed. netcoreapp2.1 was computed. netcoreapp2.2 was computed. netcoreapp3.0 was computed. netcoreapp3.1 was computed. |
| .NET Standard | netstandard2.0 is compatible. netstandard2.1 was computed. |
| .NET Framework | net461 was computed. net462 was computed. net463 was computed. net47 was computed. net471 was computed. net472 was computed. net48 was computed. net481 was computed. |
| MonoAndroid | monoandroid was computed. |
| MonoMac | monomac was computed. |
| MonoTouch | monotouch was computed. |
| Tizen | tizen40 was computed. tizen60 was computed. |
| Xamarin.iOS | xamarinios was computed. |
| Xamarin.Mac | xamarinmac was computed. |
| Xamarin.TVOS | xamarintvos was computed. |
| Xamarin.WatchOS | xamarinwatchos was computed. |
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.
-
.NETStandard 2.0
- System.Text.Json (>= 6.0.1)
NuGet packages (18)
Showing the top 5 NuGet packages that depend on Microsoft.ML.Tokenizers:
| Package | Downloads |
|---|---|
|
Microsoft.KernelMemory.Core
The package contains the the core logic and abstractions of Kernel Memory, not including extensions. |
|
|
Microsoft.KernelMemory.AI.OpenAI
Provide access to OpenAI LLM models in Kernel Memory to generate embeddings and text |
|
|
Microsoft.ML.TorchSharp
Microsoft.ML.TorchSharp contains ML.NET integration of TorchSharp. |
|
|
Microsoft.KernelMemory.AI.TikToken
Provide TikToken tokenizers in Kernel Memory |
|
|
Microsoft.Teams.AI
SDK focused on building AI based applications for Microsoft Teams. |
GitHub repositories (10)
Showing the top 5 popular GitHub repositories that depend on Microsoft.ML.Tokenizers:
| Repository | Stars |
|---|---|
|
microsoft/semantic-kernel
Integrate cutting-edge LLM technology quickly and easily into your apps
|
|
|
microsoft/kernel-memory
RAG architecture: index and query any data using LLM and natural language, track sources, show citations, asynchronous memory patterns.
|
|
|
microsoft/teams-ai
SDK focused on building AI based applications and extensions for Microsoft Teams and other Bot Framework channels
|
|
|
dotnet/ai-samples
|
|
|
axzxs2001/Asp.NetCoreExperiment
原来所有项目都移动到**OleVersion**目录下进行保留。新的案例装以.net 5.0为主，一部分对以前案例进行升级，一部分将以前的工作经验总结出来，以供大家参考！
|
| Version | Downloads | Last updated |
|---|---|---|
| 1.0.0 | 1,665 | 11/14/2024 |
| 0.22.0 | 1,113 | 11/13/2024 |
| 0.22.0-preview.24526.1 | 1,632 | 10/27/2024 |
| 0.22.0-preview.24522.7 | 1,187 | 10/23/2024 |
| 0.22.0-preview.24378.1 | 92,369 | 7/29/2024 |
| 0.22.0-preview.24271.1 | 144,411 | 5/21/2024 |
| 0.22.0-preview.24179.1 | 141,442 | 4/2/2024 |
| 0.22.0-preview.24162.2 | 19,997 | 3/13/2024 |
| 0.21.1 | 92,829 | 1/18/2024 |
| 0.21.0 | 51,446 | 11/27/2023 |
| 0.21.0-preview.23511.1 | 51,610 | 10/13/2023 |
| 0.21.0-preview.23266.6 | 51,200 | 5/17/2023 |
| 0.21.0-preview.22621.2 | 2,106 | 12/22/2022 |
| 0.20.1 | 86,553 | 2/1/2023 |
| 0.20.1-preview.22573.9 | 2,299 | 11/24/2022 |
| 0.20.0 | 30,651 | 11/8/2022 |
| 0.20.0-preview.22551.1 | 233 | 11/1/2022 |