Deep Learning > Education

Neuraxle is a Machine Learning (ML) library for building neat pipelines, providing the right abstractions to both ease research, development, and deployment of your ML applications.

Most known deep learning framework, both high-level and low-level while staying flexible.

TensorFlow wrapper à la scikit-learn.

Neural Turing Machine TensorFlow implementation.

Tutorial of mine on using LSTMs on time series for classification.

Improvements on the previous project.

Tutorial of mine on how to predict temporal sequences of numbers - that may be multichannel.

Auto (meta) optimizing a neural net (and its architecture) on the CIFAR-100 dataset.

Smooth patch merger for semantic segmentation with a U-Net.

Learn the good design patterns to use for doing Machine Learning the good way, by practicing.

An awesome list of public datasets.

Merges the ideas of the U-Net and the DenseNet, this new neural network is especially good for huge datasets in image segmentation.

Awesome for the use of "local contrast normalization".

AlexNet, 2012 ILSVRC, breakthrough of the ReLU activation function.

For the "deconvnet layer".

ELU activation function for CIFAR vision tasks.

Interesting idea of stacking multiple 3x3 conv+ReLU before pooling for a bigger filter size with just a few parameters. There is also a nice table for "ConvNet Configuration".

GoogLeNet: Appearance of "Inception" layers/modules, the idea is of parallelizing conv layers into many mini-conv of different size with "same" padding, concatenated on depth.

Highway networks: residual connections.

Batch normalization (BN): to normalize a layer's output by also summing over the entire batch, and then performing a linear rescaling and shifting of a certain trainable amount.

The U-Net is an encoder-decoder CNN that also has skip-connections, good for image segmentation at a per-pixel level.

Very deep residual layers with batch normalization layers - a.k.a. "how to overfit any vision dataset with too many layers and make any vision model work properly at recognition given enough data".

For improving GoogLeNet with residual connections.

Epic raw voice/music generation with new architectures based on dilated causal convolutions to capture more audio length.

3D-GANs for 3D model generation and fun 3D furniture arithmetics from embeddings (think like word2vec word arithmetics with 3D furniture representations).

Incredibly fast distributed training of a CNN.

Best Paper Award at CVPR 2017, yielding improvements on state-of-the-art performances on CIFAR-10, CIFAR-100 and SVHN datasets, this new neural network architecture is named DenseNet.

Merges the ideas of the U-Net and the DenseNet, this new neural network is especially good for huge datasets in image segmentation.

Use a distance metric in the loss to determine to which class does an object belongs to from a few examples.

Nice for photoshop-like "content aware fill" to fill missing patches in images.

Let RNNs decide how long they compute. I would love to see how well would it combines to Neural Turing Machines. Interesting interactive visualizations on the subject can be found here.