NipahTokenizer 3.1.9

dotnet add package NipahTokenizer --version 3.1.9
NuGet\Install-Package NipahTokenizer -Version 3.1.9
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="NipahTokenizer" Version="3.1.9" />
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add NipahTokenizer --version 3.1.9
#r "nuget: NipahTokenizer, 3.1.9"
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install NipahTokenizer as a Cake Addin
#addin nuget:?package=NipahTokenizer&version=3.1.9

// Install NipahTokenizer as a Cake Tool
#tool nuget:?package=NipahTokenizer&version=3.1.9

Nipah Tokenizer

NuGet version (NipahTokenizer)

Installing

dotnet add package NipahTokenizer

Using

First setup the tokenizer options as needed, you can use the default as follows:

var tokenizerOptions = TokenizerOptions.Default;

Then just create a new instance of tokenizer and call "Tokenize" passing tokenizer options as arg:

var tokenizer = new Tokenizer();
var tokens = tokenizer.Tokenize("any-text-here", tokenizerOptions);

You can now just iterate over tokens as needed.

Product Compatible and additional computed target framework versions.
.NET net6.0 is compatible.  net6.0-android was computed.  net6.0-ios was computed.  net6.0-maccatalyst was computed.  net6.0-macos was computed.  net6.0-tvos was computed.  net6.0-windows was computed.  net7.0 was computed.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 was computed.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.
  • net6.0

    • No dependencies.

NuGet packages (1)

Showing the top 1 NuGet packages that depend on NipahTokenizer:

Package Downloads
Nipah.Markdown

A library to make simple parses of markdown

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
3.1.9 147 9/9/2023
3.1.8 127 9/5/2023
3.1.7 111 9/3/2023
3.1.6 433 12/18/2022
3.1.5 385 10/22/2022
3.1.3 394 10/1/2022
3.1.1 396 10/1/2022
3.1.0 392 10/1/2022
3.0.0 429 9/24/2022
1.0.1 384 10/9/2021
1.0.0 332 10/9/2021

Now '' can also be used for making string literals as well as ""