MyCaffe 0.11.4.60-beta1

MyCaffe AI Platform and Test Application (CUDA 11.4.2, cuDNN 8.2.4) with improved Seq2Seq and Attention Support.

CUDA 11.4.2, cuDNN 8.2.4, nvapi 470, Windows 21H1, Driver 471.96

MyCaffe[1] (a complete C# re-write of CAFFE[2]) now supports improved Seq2Seq with Attention!

IMPORTANT NOTE: When using TCC mode, we recommend that ALL headless GPU’s are placed in TCC mode for we have experienced stability issues when using a mix of TCC and WDM modes with headless GPU’s.

REQUIRED SOFTWARE to use MyCaffe: 1.) Download and install full version of Microsoft SQL Express 2016 (or later). NOTE: The full version of SQL Express must be installed as opposed to the light version included in Visual Studio. Microsoft SQL Express can be downloaded from https://www.microsoft.com/en-us/sql-server/sql-server-downloads

REQUIRED SOFTWARE to build MyCaffe: 1.) Install NVIDIA CUDA 11.4.2 which you can download from https://developer.nvidia.com/cuda-downloads 2.) Install NVIDIA cuDNN 8.2.4 which you can download from https://developer.nvidia.com/cudnn

This release of the MyCaffe AI Platform and Test Applications has the following new additions: • CUDA 11.4.2.471/cuDNN 8.2.4.15/nvapi 470/driver 471.96 • Windows 21H1, OS Build 19043.1202, SDK 10.0.19041.0 • Added new SequenceFiller for testing. • Added low-level support for cuBlas Geam. • Added low-level support for channel_mulv. • Added low-level support for channel_sum within channel. • Added new TextData Layer. • Added new ModeData Layer. • Added dynamic sizing to Embed layer via bottom[1]. • Added dynamic sizing to InnerProd layer via bottom[1]. • Added optional index to target conversion for SoftmaxCrossEntropyLoss. • Added beam-search to Run method. • Added option to verify updated run weights to UpdateRunWeights. • Extended and improved result support. • Upgraded to Google.protobuf version 3.17.3 • Run(Blob…) now allowed on non-Database loads. • Optimized Attention layer forward and backward passes. • Optimized ColorMapper.GetColor with binary search.

The following bug fixes are in this release: • Fixed bug in shared weights in Attention layer. • Fixed TestImageTools test errors. • Fixed TestTextDataLayer test errors. • Fixed bug in SetDatasetParameter not saving parameter. • Fixed bug in Run(Blob<>) when used with MULTIBOX types. • Fixed bug in Run(PropertySet) when used with multiple inputs. • Fixed bug in TestMany when used with MULTIBOX types. • Fixed download bugs in TestImageTools. • Fixed download URL bug for CIFAR dataset. • Fixed bug causing error in TestMany where model had no test iterations. • Fixed bug in Blob causing error when already disposed. • Fixed bug importing project where weights were not loading properly.

Easily run Seq2Seq[3] models with Attention[4], Single-Shot Multi-Box Nets[5][6], import/export ONNX AI Models, run Triplet Nets[7][8], run Siamese Nets[10][11], Neural Style, train Deep Q-Learning or Policy Gradient models to beat Pong or Cart-Pole, or create the CIFAR-10 and MNIST datasets using the MyCaffe-Samples (https://github.com/MyCaffe/MyCaffe-Samples) and MyCaffe Test Application which you can download from the MyCaffe GitHub site.

Schedule distributed AI work packages, or create and train Single-Shot Multi-Box[5][6], Triplet Net[7][8], Siamese Net[9][10], Deep Q-Learning with NoisyNet and Experienced Replay, Policy Gradient, Neural Style Transfer, Recurrent Learning, Policy Gradient Reinforcement Learning, Auto-Encoder, DANN and ResNet models by following step-by-step instructions in the SignalPop Tutorials. And, to see other cool examples that show what MyCaffe can do, see the SignalPop Examples.

If you would like to visually design, develop, test and debug your models, see the SignalPop AI Designer specifically designed to enhance your MyCaffe deep learning.

Also, check out the SignalPop Universal Miner that not only keeps your GPU's cool as you train, but also gives you detailed information on each of your GPU's (such as temperature, fan speed, overclock, and usage), and allows you to easily mine Ethereum. When not training AI, put those GPU's to use making some Ether - never let a good GPU go to waste!

Happy ‘deep’ learning!

[1] MyCaffe: A Complete C# Re-Write of Caffe with Reinforcement Learning by D. Brown, 2018.

[2] Caffe: Convolutional Architecture for Fast Feature Embedding by Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, Trevor Darrell, 2014, arXiv:1408.5093

[3] Attention - Seq2Seq Models by Pranay Dugar, Toward Data Science, 2019.

[4] Attention Is All You Need by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin, 2017, ArXiv:1706.03762

[5] SSD: Single Shot MultiBox Detector by Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg, 2016.

[6] GitHub: SSD: Single Shot MultiBox Detector, by weiliu89/caffe, 2016

[7] Deep metric learning using Triplet network by Elad Hoffer and Nir Ailon, 2018, arXiv:1412.6622

[8] In Defense of the Triplet Loss for Person Re-Identification by Alexander Hermans, Lucas Beyer and Bastian Liebe, 2017, arXiv:1703.07737v2

[9] Siamese Network Training with Caffe by Berkeley Artificial Intelligence (BAIR)

[10] Siamese Neural Network for One-shot Image Recognition by G. Koch, R. Zemel and R. Salakhutdinov, ICML 2015 Deep Learning Workshop, 2015.