Radiate 1.3.7

.NET port of Rust crate radiate.

Algorithms

1. Random Forest
2. Support Vector Machine
3. MultiLayer Perceptron
   • Dense Layer
   • Dropout Layer
   • Flatten Layer
   • LSTM Layer
   • RNN Layer
   • Convolutional Layer
   • MaxPooling Layer
4. KMeans Clustering
5. Evolution Engine
   • Evolve any object which implements the Genome base class.
     • Implementation of NEAT for evolving NeuralNetworks is included.
     • Implementation of evolutionarily generated RandomForest/DecisionTree.

Callbacks

Similar to Keras Callbacks. Hook into the training loop with custom code. Create an object that implements any or all of the following:

1. IEpochStartedCallback
2. IBatchCompletedCallback
3. IEpochCompletedCallback
4. ITrainingCompletedCallback
5. IGenerationEvolvedCallback

See this for example callbacks or this for a VerboseTrainingCallback to print training progress to the Console.

Feature Engineering

Functional feature engineering with TensorDataSet. Transform input data (featurs, targets) with specific options below.

1. Batch - Set a batch size to train on.
2. Layer - Layer data by n rows.
3. Split - Split the data into a training set and testing set. Default is 75% split training, 25% testing.
4. Reshape - Reshape the row vector to a shape of (height, width, depth), useful for images.
5. Pad - Pad an image Tensor with n zeros.
6. Shuffle - Shuffle the rows of the dataset randomly.
7. Kernel - Add kernel transform for the features, possible options are RBF, Polynomial, and Linear (None).
8. TransformFeatures - Transform the feature data. Options are Normalize, Standardize, OHE (One Hot Encode), and Image (divide data point by 255).
9. TransformTargets - Transform the target data. Options same as above.

Model saving and loading

Save an Optimizer model like

const int numTrees = 10;
const int maxDepth = 10;
const int minSampleSplit = 2;

var (rawFeatures, rawLabels) = await new IrisFlowers().GetDataSet();
var pair = new TensorDataSet(rawFeatures, rawLabels)
   .Shuffle()
   .Split()
   .Batch(rawFeatures.Count);

var forest = new RandomForest(numTrees, new ForestInfo(minSampleSplit, maxDepth));
var optimizer = new Optimizer(forest, pair);

await optimizer.Train<RandomForest>();

// Generate a json serializable object.
var wrappedModel = optimizer.Save();

// The same model can be loaded back in for prediction/additional training.
var newOptimizer = new Optimizer(wrappedModel);

The Optimizer is not Json serializable, but the OptimizerWrap is, so the Optimizer must be converted to a concrete object before serializing.

OptimizerWrap contains three items:

1. TensorDataSet options, the options used to transform the input features/targets. During predicion the Optimizer uses these options to transform the input vector so it matches the trained features in order to get accurate predictions.
2. LossFunction, the loss function used during training. If you save a model mid training, the loss function is needed when loading back in the model to continue training.
3. ModelWrap, the machine learning model being trained/used for prediction.

The Optimizer can also be converted to a json string or a memory stream like so:

var optimizer = new Optimizer(forest, TensorDataSet);
var jsonString = ModelWriter.ToJson(optimizer);
var stream = ModelWriter.ToStream(optimizer);

Loading in an Opimizer<T> from the above options:

var optimizer = ModelReader.FromJson(jsonString);
var optimizer = ModelReader.FromStream(stream);

Make predictions

Because the Optimizer has TensorDataSet, it can transform a given float[] to acceptable input to the model even after saving/loading a model. This makes making predictions as easy as

var vectorToPredict = new float[] { 1f, 2f, 3f, 4f, 5f };
var prediction = trainedOptimizer.Predict(vectorToPredict);

Random

All random numbers are generated by calling RandomGenerator.Next. This allows for random seeds to be set for model building.

RandomGenerator.Seed = 3;

Will set a random seed for the Optimizer.

Loss functions

1. CrossEntropy
2. Difference
3. Hinge
4. MeanSquaredError

Activation Functions

1. Sigmoid
2. ReLu
3. Tanh
4. Linear
5. SoftMax
6. ExpSigmoid

Gradients

Only applies to MultiLayerPerceptron and SupportVectorMachine

1. Adam
2. SGD

Examples

Datasets coming from Radiate.Data which provides easily accessable common machine learning datasets.

Convolutional Neural Network on MNist handwritten digets dataset

<img src="https://camo.githubusercontent.com/01c057a753e92a9bc70b8c45d62b295431851c09cffadf53106fc0aea7e2843f/687474703a2f2f692e7974696d672e636f6d2f76692f3051493378675875422d512f687164656661756c742e6a7067" width="300px">

const int featureLimit = 5000;
const int batchSize = 128;
const int maxEpochs = 10;

var (rawInputs, rawLabels) = await new Mnist(featureLimit).GetDataSet();

var pair = new TensorDataSet(rawInputs, rawLabels)
    .Reshape(new Shape(28, 28, 1))
    .TransformData(Norm.Image, Norm.OHE)
    .Batch(batchSize)
    .Split();

var neuralNetwork = new MultiLayerPerceptron()
    .AddLayer(new ConvInfo(64, 3))
    .AddLayer(new MaxPoolInfo(2))
    .AddLayer(new FlattenInfo())
    .AddLayer(new DenseInfo(64, Activation.Sigmoid))
    .AddLayer(new DenseInfo(pair.OutputCategories, Activation.SoftMax));

var optimizer = new Optimizer(neuralNetwork, pair, new List<ITrainingCallback>
{
    new VerboseTrainingCallback(pair, maxEpochs),
    new ConfusionMatrixCallback()
});

await optimizer.Train<MultiLayerPerceptron>(epoch => maxEpochs == epoch.Index);

Random Forest on Iris Flowers dataset

<img src="https://upload.wikimedia.org/wikipedia/commons/5/56/Iris_dataset_scatterplot.svg" width="300px">

const int numTrees = 10;
const int maxDepth = 10;
const int minSampleSplit = 2;

var (rawFeatures, rawLabels) = await new IrisFlowers().GetDataSet();
var pair = new TensorDataSet(rawFeatures, rawLabels)
    .Shuffle()
    .Split()
    .Batch(rawFeatures.Count);

var forest = new RandomForest(numTrees, new ForestInfo(minSampleSplit, maxDepth));
var optimizer = new Optimizer(forest, pair, new List<ITrainingCallback>
{
    new VerboseTrainingCallback(pair),
    new ModelWriterCallback(),
    new ConfusionMatrixCallback()
});

await optimizer.Train<RandomForest>();

More

See the examples for how to use the API.