RLMatrix.Remote 0.4.3.2

dotnet add package RLMatrix.Remote --version 0.4.3.2                
NuGet\Install-Package RLMatrix.Remote -Version 0.4.3.2                
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="RLMatrix.Remote" Version="0.4.3.2" />                
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add RLMatrix.Remote --version 0.4.3.2                
#r "nuget: RLMatrix.Remote, 0.4.3.2"                
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install RLMatrix.Remote as a Cake Addin
#addin nuget:?package=RLMatrix.Remote&version=0.4.3.2

// Install RLMatrix.Remote as a Cake Tool
#tool nuget:?package=RLMatrix.Remote&version=0.4.3.2                

RLMatrixBanner

๐Ÿš€ RL Matrix - Pure C# Deep Reinforcement Learning Experience with TorchSharp!

Dive into the future of type-safe Deep Reinforcement Learning with .NET & RL Matrix, powered by the might of TorchSharp. RL Matrix stands out as a user-friendly toolkit offering a collection of RL algorithmsโ€”primed for plug, play, and prosper! NuGet NuGet NuGet TorchSharp discord](https://discord.gg/ppgr44rBHn)

  • PPO
  • DQN
  • Both have 1D (Feed forward) and 2D (CNN) variants
  • 0.1.2 Adds multi-head continous (PPO) discrete (PPO, DQN) and mixed (PPO) actions. See IEnvironment and IContinousEnvironment.
  • 0.2.0 Adds working-ish PPO GAIL. And overhauls training method for stepwise
  • 0.2.0 Adds multi-environment training
  • 0.2.0 Includes Godot examples and RLMatrix.Godot nuget package for easy setup
  • Only tested single-head discrete output so please open issue if it doesnt work.

๐ŸŽฏ What Sparks RL Matrix?

While embarking on my RL journey, I sensed a gap in the reinforcement learning world even with TorchSharp's solid foundation. It struck meโ€”C# is the ideal choice for RL outside research circles, thanks to its pristine and intuitive coding experience. No more guessing games in environment or agent building!

With RL Matrix, our vision is to offer a seamless experience. By simply incorporating the IEnvironment interface, you're equipped to rapidly craft and unleash Reinforcement Learning Agents (RL Agents). Switching between algorithms? A breeze! Itโ€™s our nod to the elegance of Matlab's toolkit methodology.

enter image description here enter image description here

๐Ÿš€ Getting Started:

Peek into the /examples/ directory for illustrative code. But to give you a quick start:

  1. Craft an IEnvironment class: Comply with reinforcement learning guidelines, defining your observation shapes and action count:
public class CartPole : IEnvironment<float[]>
{
  public int stepCounter { get; set; }
  public int maxSteps { get; set; }
  public bool isDone { get; set; }
  public OneOf<int, (int, int)> stateSize { get; set; }
  public int actionSize { get; set; }

  CartPoleEnv myEnv;

  private float[] myState;

  public CartPole()
  {
      Initialise();
  }

  public float[] GetCurrentState()
  {
      if (myState == null)
          myState = new float[4] {0,0,0,0};
      return myState;
  }

  public void Initialise()
  {
      myEnv = new CartPoleEnv(WinFormEnvViewer.Factory);
      stepCounter = 0;
      maxSteps = 100000;
      stateSize = myEnv.ObservationSpace.Shape.Size;
      actionSize = myEnv.ActionSpace.Shape.Size;
      myEnv.Reset();
      isDone = false; 
  }

  public void Reset()
  {
  //For instance:
      myEnv.Reset();
      isDone = false;
      stepCounter = 0;
  }

  public float Step(int actionId)
  {
  //Whatever step logic, returns reward
      return reward;
  }
}
  1. Agent Instance & Training: Spawn an agent for your environment and ignite the Step method:
var opts = new DQNAgentOptions(batchSize: 64, memorySize: 10000, gamma: 0.99f, epsStart: 1f, epsEnd: 0.05f, epsDecay: 50f, tau: 0.005f, lr: 1e-4f, displayPlot: myChart);
var env = new List<IEnvironment<float[]>> { new CartPole(), new CartPole() };
var myAgent = new DQNAgent<float[]>(opts, env);

for (int i = 0; i < 10000; i++)
{
    myAgent.Step();
}

Notice that TrainEpisode method was removed.

๐Ÿ“Œ Current Roadmap:

-Add RNN support for PPO and DQN -Add variations for multi-head output for PPO and DQN -More Godot examples testing multi-head continous+discrete action spaces -Create Godot plugin -Fully develop workflow for Gail and imitation learning As we innovate, anticipate breaking changes. We'll keep you in the loop!

๐Ÿ’Œ Get in Touch:

Questions? Ideas? Collaborations? Drop a line at: ๐Ÿ“ง adrian@sieradzki.io

๐Ÿค License:

RLMatrix is licensed under the RLMatrix Comprehensive Dual License Agreement

Product Compatible and additional computed target framework versions.
.NET net5.0 was computed.  net5.0-windows was computed.  net6.0 was computed.  net6.0-android was computed.  net6.0-ios was computed.  net6.0-maccatalyst was computed.  net6.0-macos was computed.  net6.0-tvos was computed.  net6.0-windows was computed.  net7.0 was computed.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 is compatible.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
.NET Core netcoreapp2.0 was computed.  netcoreapp2.1 was computed.  netcoreapp2.2 was computed.  netcoreapp3.0 was computed.  netcoreapp3.1 was computed. 
.NET Standard netstandard2.0 is compatible.  netstandard2.1 was computed. 
.NET Framework net461 was computed.  net462 was computed.  net463 was computed.  net47 was computed.  net471 was computed.  net472 was computed.  net48 was computed.  net481 was computed. 
MonoAndroid monoandroid was computed. 
MonoMac monomac was computed. 
MonoTouch monotouch was computed. 
Tizen tizen40 was computed.  tizen60 was computed. 
Xamarin.iOS xamarinios was computed. 
Xamarin.Mac xamarinmac was computed. 
Xamarin.TVOS xamarintvos was computed. 
Xamarin.WatchOS xamarinwatchos was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (1)

Showing the top 1 NuGet packages that depend on RLMatrix.Remote:

Package Downloads
RLMatrix.Server

Server components for Deep Reinforcement Learning in .NET

GitHub repositories (1)

Showing the top 1 popular GitHub repositories that depend on RLMatrix.Remote:

Repository Stars
asieradzk/RL_Matrix
Deep Reinforcement Learning in C#
Version Downloads Last updated
0.4.3.2 112 9/27/2024
0.4.3 230 8/12/2024
0.4.2 230 8/9/2024
0.4.1 108 8/9/2024
0.4.0 124 8/8/2024
0.3.1 886 7/24/2024
0.3.0 945 7/23/2024