GibberishDetector 2.0.0

GibberishDetector

A .NET NuGet package for detecting gibberish text using Markov models. This library analyzes input text and assigns a gibberish score based on a pre-trained Markov model, helping identify random or nonsensical strings. The package comes pre-trained on a comprehensive corpus, enabling immediate use of detection functions, with the option to train on a custom corpus for specialized needs.

Features

• Detect gibberish text with a customizable threshold.
• Calculate a gibberish score for any input string.
• Pre-trained on an existing corpus for immediate use of detection functions.
• Load and save custom Markov models from/to files.
• Train models from text or custom corpus text files with batch processing.
• Lightweight and easy to integrate into .NET applications.

Installation

Install the package via NuGet Package Manager or the .NET CLI:

dotnet add package GibberishDetector

Or, using the Package Manager Console:

Install-Package GibberishDetector

Usage

Initializing the Detector

Create an instance of GibberishDetector with an optional threshold (default is 0.1). The threshold determines the sensitivity of gibberish detection—lower values are stricter. The detector is pre-trained, so you can immediately use detection functions like IsGibberish and GetGibberishScore.

using GibberishDetector;

var detector = new GibberishDetector(threshold: 0.1);

Checking for Gibberish

Use the IsGibberish method to determine if a string is gibberish based on the configured threshold and the pre-trained model.

string text = "asdkjfhgqwerty";
bool isGibberish = detector.IsGibberish(text);
Console.WriteLine($"Is gibberish: {isGibberish}"); // Output: Is gibberish: true

text = "Hello, world!";
isGibberish = detector.IsGibberish(text);
Console.WriteLine($"Is gibberish: {isGibberish}"); // Output: Is gibberish: false

Getting a Gibberish Score

Use the GetGibberishScore method to retrieve a numerical score indicating how likely the text is to be gibberish, using the pre-trained model. Higher scores indicate more gibberish-like text.

string text = "asdkjfhgqwerty";
double score = detector.GetGibberishScore(text);
Console.WriteLine($"Gibberish score: {score}"); // Output: Gibberish score: 0.85 (example value)

Loading a Custom Model

Load a custom Markov model from a file using the LoadModel method if you have trained a model separately. The file should contain serialized model data compatible with the detector.

detector.LoadModel("path/to/model.dat");
Console.WriteLine("Custom model loaded successfully.");

Training on a Custom Corpus

The detector is pre-trained, but you can train it on a different corpus text file using TrainFromFile or with a single text input using TrainBatch. Save the trained model with SaveModel for future use.

// Train from a custom corpus text file with batch processing
detector.TrainFromFile("path/to/custom_corpus.txt");

// Save the trained model
detector.SaveModel("path/to/saved_model.json");

Functions Reference

GibberishDetector(double threshold = 0.1)

• Description: Initializes a new instance of the gibberish detector with a specified threshold. The detector is pre-trained and ready for immediate use.
• Parameters:
  • threshold (double): The threshold for classifying text as gibberish (default: 0.1). Lower values are stricter.
• Example:

  var detector = new GibberishDetector(threshold: 0.15);
  

IsGibberish(string text)

• Description: Determines if the input text is gibberish based on the threshold and the pre-trained or custom model.
• Parameters:
  • text (string): The text to analyze.
• Returns: bool - true if the text is gibberish, false otherwise.
• Example:

  bool result = detector.IsGibberish("qwertyuiop");
  

GetGibberishScore(string text)

• Description: Calculates a gibberish score for the input text using the pre-trained or custom model.
• Parameters:
  • text (string): The text to score.
• Returns: double - A score between 0 and 1, where higher values indicate more gibberish-like text.
• Example:

  double score = detector.GetGibberishScore("asdkjfhgqwerty");
  

LoadModel(string filePath)

• Description: Loads a custom Markov model from the specified file path.
• Parameters:
  • filePath (string): The path to the model file.
• Throws: FileNotFoundException if the file does not exist.
• Example:

  detector.LoadModel("model.json");
  

SaveModel(string filePath)

• Description: Saves the current Markov model to the specified file path.
• Parameters:
  • filePath (string): The path where the model will be saved.
• Throws: IOException if the file cannot be written.
• Example:

  detector.SaveModel("saved_model.json");
  

TrainBatch(string text)

• Description: Trains the Markov model with a single batch of text.
• Parameters:
  • text (string): The text to train the model with.
• Example:

  detector.TrainBatch("This is a training sample.");
  

TrainFromFile(string filePath, int batchSize = 1000)

• Description: Trains the Markov model using text from a custom corpus text file, processing it in batches.
• Parameters:
  • filePath (string): The path to the training text file.
  • batchSize (int): The number of lines to process per batch (default: 1000).
• Throws: FileNotFoundException if the file does not exist.
• Example:

  detector.TrainFromFile("custom_corpus.txt", batchSize: 500);
  

License

This project is licensed under the MIT License.