Education
133 items from guillaume-chevalier/awesome-deep-learning-resources ★1,821
-
ImageNet Classification with Deep Convolutional Neural Networks cs.toronto.edu
AlexNet, 2012 ILSVRC, breakthrough of the ReLU activation function.
-
Cornell Movie--Dialogs Corpus cs.cornell.edu
This could be used for a chatbot.
-
Geoffrey Hinton interview coursera.org
Andrew Ng interviews Geoffrey Hinton, who talks about his research and breaktroughs, and gives advice for students.
-
Neural Networks, Manifolds, and Topology colah.github.io
Fresh look on how neurons map information.
-
Uncle Bob's Principles Of OOD butunclebob.com
Not only the SOLID principles are needed for doing clean code, but the furtherless known REP, CCP, CRP, ADP, SDP and SAP principles are very important for developping huge software that must be bundled in different separated packages.
-
Artificial Neural Networks: Mathematics of Backpropagation briandolhansky.com
Picturing backprop, mathematically.
-
Some other books I have read books.google.ca
Some books listed here are less related to deep learning but are still somehow relevant to this list.
-
Improving Language Understanding with Unsupervised Learning blog.openai.com
SOTA across many NLP tasks from unsupervised pretraining on huge corpus.
-
The future of deep learning blog.keras.io
François Chollet's thoughts on the future of deep learning.
-
Recommending music on Spotify with deep learning benanne.github.io
Awesome for doing clustering on audio - post by an intern at Spotify.
-
Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions arxiv.org
A WaveNet used as a vocoder can be conditioned on generated Mel Spectrograms from the Tacotron 2 LSTM neural network with attention to generate neat audio from text.
-
Self-Normalizing Neural Networks arxiv.org
Appearance of the incredible SELU activation function.
-
Prototypical Networks for Few-shot Learning arxiv.org
Use a distance metric in the loss to determine to which class does an object belongs to from a few examples.
-
Massive Exploration of Neural Machine Translation Architectures arxiv.org
That yields intuition about the boundaries of what works for doing NMT within a framed seq2seq problem formulation.
-
Neural Machine Translation and Sequence-to-sequence Models: A Tutorial arxiv.org
Interesting overview of the subject of NMT, I mostly read part 8 about RNNs with attention as a refresher.
-
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation arxiv.org
Merges the ideas of the U-Net and the DenseNet, this new neural network is especially good for huge datasets in image segmentation.
-
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling arxiv.org
3D-GANs for 3D model generation and fun 3D furniture arithmetics from embeddings (think like word2vec word arithmetics with 3D furniture representations).
-
Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation arxiv.org
In 2016: stacked residual LSTMs with attention mechanisms on encoder/decoder are the best for NMT (Neural Machine Translation).
-
WaveNet: a Generative Model for Raw Audio arxiv.org
Epic raw voice/music generation with new architectures based on dilated causal convolutions to capture more audio length.
-
Densely Connected Convolutional Networks arxiv.org
Best Paper Award at CVPR 2017, yielding improvements on state-of-the-art performances on CIFAR-10, CIFAR-100 and SVHN datasets, this new neural network architecture is named DenseNet.
-
Learning to learn by gradient descent by gradient descent arxiv.org
RNN as an optimizer: introducing the L2L optimizer, a meta-neural network.
-
Matching Networks for One Shot Learning arxiv.org
Interesting way of doing one-shot learning with low-data by using an attention mechanism and a query to compare an image to other images for classification.
-
Adaptive Computation Time for Recurrent Neural Networks arxiv.org
Let RNNs decide how long they compute. I would love to see how well would it combines to Neural Turing Machines. Interesting interactive visualizations on the subject can be found here.
-
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning arxiv.org
For improving GoogLeNet with residual connections.
-
Exploring the Limits of Language Modeling arxiv.org
Nice recursive models using word-level LSTMs on top of a character-level CNN using an overkill amount of GPU power.
-
Pixel Recurrent Neural Networks arxiv.org
Nice for photoshop-like "content aware fill" to fill missing patches in images.
-
Fast and Accurate Deep Network Learning by Exponential Linear Units arxiv.org
ELU activation function for CIFAR vision tasks.
-
Effective Approaches to Attention-based Neural Machine Translation arxiv.org
Exploring different approaches to attention mechanisms.
-
Teaching Machines to Read and Comprehend arxiv.org
A very interesting and creative work about textual question answering, what a breakthrough, there is something to do with that.
-
U-Net: Convolutional Networks for Biomedical Image Segmentation arxiv.org
The U-Net is an encoder-decoder CNN that also has skip-connections, good for image segmentation at a per-pixel level.
-
Highway Networks arxiv.org
Highway networks: residual connections.
-
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift arxiv.org
Batch normalization (BN): to normalize a layer's output by also summing over the entire batch, and then performing a linear rescaling and shifting of a certain trainable amount.
-
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention arxiv.org
LSTMs' attention mechanisms on CNNs feature maps does wonders.
-
Neural Turing Machines arxiv.org
Outstanding for letting a neural network learn an algorithm with seemingly good generalization over long time dependencies. Sequences recall problem.
-
Very Deep Convolutional Networks for Large-Scale Image Recognition arxiv.org
Interesting idea of stacking multiple 3x3 conv+ReLU before pooling for a bigger filter size with just a few parameters. There is also a nice table for "ConvNet Configuration".
-
Neural Machine Translation by Jointly Learning to Align and Translate arxiv.org
Attention mechanism for LSTMs! Mostly, figures and formulas and their explanations revealed to be useful to me. I gave a talk on that paper here.
-
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation arxiv.org
Two networks in one combined into a seq2seq (sequence to sequence) Encoder-Decoder architecture. RNN Encoder–Decoder with 1000 hidden units. Adadelta optimizer.
-
Deep Learning in Neural Networks: An Overview arxiv.org
You_Again's summary/overview of deep learning, mostly about RNNs.
-
Visualizing and Understanding Convolutional Networks arxiv.org
For the "deconvnet layer".
-
SentEval: An Evaluation Toolkit for Universal Sentence Representations arxiv.org
A Python framework to benchmark your sentence representations on many datasets (NLP tasks).
-
ProjectionNet: Learning Efficient On-Device Deep Networks Using Neural Projections arxiv.org
Replace word embeddings by word projections in your deep neural networks, which doesn't require a pre-extracted dictionnary nor storing embedding matrices.
-
ParlAI: A Dialog Research Software Platform arxiv.org
Another Python framework to benchmark your sentence representations on many datasets (NLP tasks).
-
Matching Networks for One Shot Learning arxiv.org
Classify a new example from a list of other examples (without definitive categories) and with low-data per classification task, but lots of data for lots of similar classification tasks - it seems better than siamese networks. To sum up: with Matching Networks, you can optimize…
-
UCI Machine Learning Repository archive.ics.uci.edu
TONS of datasets for ML.
-
How to Create a Mind amazon.com
The audio version is nice to listen to while commuting. This book is motivating about reverse-engineering the mind and thinking on how to code AI.
-
Clean Coder amazon.ca
Learn how to be professional as a coder and how to interact with your manager. This is important for any coding career.
-
Clean Code amazon.ca
Get back to the basics you fool! Learn how to do Clean Code for your career. This is by far the best book I've read even if this list is related to Deep Learning.
-
Self-Governing Neural Networks for On-Device Short Text Classification aclweb.org
This paper is the sequel to the ProjectionNet just above. The SGNN is elaborated on the ProjectionNet, and the optimizations are detailed more in-depth (also see my attempt to reproduce the paper in code and watch the talks' recording).
-
MathBox, Tools for Thought Graphical Algebra and Fourier Analysis acko.net
New look on Fourier analysis.
-
How to Fold a Julia Fractal acko.net
Animations dealing with complex numbers and wave equations.
- next page of items loading…
