NipahTokenizer 3.1.9

.NET 6.0 This package targets .NET 6.0. The package is compatible with this framework or higher. 
dotnet add package NipahTokenizer --version 3.1.9                

                    
                

            
NuGet\Install-Package NipahTokenizer -Version 3.1.9
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package. 
<PackageReference Include="NipahTokenizer" Version="3.1.9" />
For projects that support PackageReference, copy this XML node into the project file to reference the package. 
paket add NipahTokenizer --version 3.1.9                

                    
                

            
#r "nuget: NipahTokenizer, 3.1.9"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package. 
// Install NipahTokenizer as a Cake Addin
#addin nuget:?package=NipahTokenizer&version=3.1.9

// Install NipahTokenizer as a Cake Tool
#tool nuget:?package=NipahTokenizer&version=3.1.9

Nipah Tokenizer

NuGet version (NipahTokenizer)

Installing

dotnet add package NipahTokenizer

Using

First setup the tokenizer options as needed, you can use the default as follows:

var tokenizerOptions = TokenizerOptions.Default;

Then just create a new instance of tokenizer and call "Tokenize" passing tokenizer options as arg:

var tokenizer = new Tokenizer();
var tokens = tokenizer.Tokenize("any-text-here", tokenizerOptions);

You can now just iterate over tokens as needed.

Product Compatible and additional computed target framework versions.
.NET net6.0 is compatible.  net6.0-android was computed.  net6.0-ios was computed.  net6.0-maccatalyst was computed.  net6.0-macos was computed.  net6.0-tvos was computed.  net6.0-windows was computed.  net7.0 was computed.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 was computed.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

  • net6.0

    • No dependencies.

NuGet packages (1)

Showing the top 1 NuGet packages that depend on NipahTokenizer:

Package Downloads
Nipah.Markdown

A library to make simple parses of markdown

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
3.1.9 185 9/9/2023
3.1.8 156 9/5/2023
3.1.7 136 9/3/2023
3.1.6 511 12/18/2022
3.1.5 416 10/22/2022
3.1.3 424 10/1/2022
3.1.1 424 10/1/2022
3.1.0 421 10/1/2022
3.0.0 457 9/24/2022
1.0.1 415 10/9/2021
1.0.0 358 10/9/2021

Now '' can also be used for making string literals as well as ""