Education
133 items from guillaume-chevalier/awesome-deep-learning-resources ★1,821
-
Gradient descent algorithms in a saddle point sebastianruder.com
Visualize how different optimizers interacts with a saddle points.
-
Improving Inception and Image Classification in TensorFlow research.googleblog.com
Very interesting CNN architecture (e.g.: the inception-style convolutional layers is promising and efficient in terms of reducing the number of parameters).
-
Announcing SyntaxNet: The World’s Most Accurate Parser Goes Open Source research.googleblog.com
Parsey McParseface's birth, a neural syntax tree parser.
-
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour research.fb.com
Incredibly fast distributed training of a CNN.
-
Sequence to Sequence Learning with Neural Networks papers.nips.cc
4 stacked LSTM cells of 1000 hidden size with reversed input sentences, and with beam search, on the WMT’14 English to French dataset.
-
LibriSpeech ASR corpus openslr.org
Huge free English speech dataset with balanced genders and speakers, that seems to be of high quality.
-
Deep learning for lazybones oduerr.github.io
Transfer learning tutorial in TensorFlow for vision from high-level embeddings of a pretrained CNN, AlexNet 2012.
-
The Annotated Transformer nlp.seas.harvard.edu
Good for understanding the "Attention Is All You Need" (AIAYN) paper.
-
Hacker News news.ycombinator.com
Maybe how I discovered ML - Interesting trends appear on that site way before they get to be a big deal.
-
Deep Learning & Recurrent Neural Networks (DL&RNN) neuraxio.com
The most richly dense, accelerated course on the topic of Deep Learning & Recurrent Neural Networks (scroll at the end).
-
Neural Networks and Deep Learning neuralnetworksanddeeplearning.com
This book covers many of the core concepts behind neural networks and deep learning.
-
Neural Networks and Deep Learning, ch.4 neuralnetworksanddeeplearning.com
A visual proof that neural nets can compute any function.
-
Neural Networks and Deep Learning, ch.2 neuralnetworksanddeeplearning.com
Overview on how does the backpropagation algorithm works.
-
Naver naver.com
This is a Korean search engine - best used with Google Translate, ironically. Surprisingly, sometimes deep learning search results and comprehensible advanced math content shows up more easily there than on Google search.
-
Hybrid computing using a neural network with dynamic external memory nature.com
Improvements on differentiable memory based on NTMs: now it is the Differentiable Neural Computer (DNC).
-
Estimating an Optimal Learning Rate For a Deep Neural Network medium.com
Clever trick to estimate an optimal learning rate prior any single full training.
-
Yes you should understand backprop medium.com
Exposing backprop's caveats and the importance of knowing that while training models.
-
The Illustrated Transformer jalammar.github.io
Also good for understanding the "Attention Is All You Need" (AIAYN) paper.
-
The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) jalammar.github.io
Understand the different approaches used for NLP's ImageNet moment.
-
ML / DL repositories I starred github.com
GitHub is full of nice code samples & projects.
-
carpedm20's repositories github.com
Many interesting neural network architectures are implemented by the Korean guy Taehoon Kim, A.K.A. carpedm20.
-
Window Functions en.wikipedia.org
Wikipedia page that lists some of the known window functions - note that the Hann-Poisson window is specially interesting for greedy hill-climbing algorithms (like gradient descent for example).
-
Predictions made by Ray Kurzweil en.wikipedia.org
List of mid to long term futuristic predictions made by Ray Kurzweil.
-
DL&RNN Course dl-rnn-course.neuraxio.com
I created this richely dense course on Deep Learning and Recurrent Neural Networks.
-
Attention and Augmented Recurrent Neural Networks distill.pub
Interesting for visual animations, it is a nice intro to attention mechanisms as an example.
-
Bidirectional Recurrent Neural Networks di.ufpe.br
Better classifications with RNNs with bidirectional scanning on the time axis.
-
Gradient descent algorithms in an almost flat landscape devblogs.nvidia.com
Visualize how different optimizers interacts with an almost flat landscape.
-
WaveNet: A Generative Model for Raw Audio deepmind.com
Realistic talking machines: perfect voice generation.
-
DataTau datatau.com
This is a hub similar to Hacker News, but specific to data science.
-
Going Deeper with Convolutions cv-foundation.org
GoogLeNet: Appearance of "Inception" layers/modules, the idea is of parallelizing conv layers into many mini-conv of different size with "same" padding, concatenated on depth.
-
Exploring the Depths of Recurrent Neural Networks with Stochastic Residual Learning cs224d.stanford.edu
Basically, residual connections can be better than stacked RNNs in the presented case of sentiment analysis.
-
ImageNet Classification with Deep Convolutional Neural Networks cs.toronto.edu
AlexNet, 2012 ILSVRC, breakthrough of the ReLU activation function.
-
Cornell Movie--Dialogs Corpus cs.cornell.edu
This could be used for a chatbot.
-
Geoffrey Hinton interview coursera.org
Andrew Ng interviews Geoffrey Hinton, who talks about his research and breaktroughs, and gives advice for students.
-
Neural Networks, Manifolds, and Topology colah.github.io
Fresh look on how neurons map information.
-
Uncle Bob's Principles Of OOD butunclebob.com
Not only the SOLID principles are needed for doing clean code, but the furtherless known REP, CCP, CRP, ADP, SDP and SAP principles are very important for developping huge software that must be bundled in different separated packages.
-
Artificial Neural Networks: Mathematics of Backpropagation briandolhansky.com
Picturing backprop, mathematically.
-
Some other books I have read books.google.ca
Some books listed here are less related to deep learning but are still somehow relevant to this list.
-
Improving Language Understanding with Unsupervised Learning blog.openai.com
SOTA across many NLP tasks from unsupervised pretraining on huge corpus.
-
The future of deep learning blog.keras.io
François Chollet's thoughts on the future of deep learning.
-
Recommending music on Spotify with deep learning benanne.github.io
Awesome for doing clustering on audio - post by an intern at Spotify.
-
Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions arxiv.org
A WaveNet used as a vocoder can be conditioned on generated Mel Spectrograms from the Tacotron 2 LSTM neural network with attention to generate neat audio from text.
-
Self-Normalizing Neural Networks arxiv.org
Appearance of the incredible SELU activation function.
-
Prototypical Networks for Few-shot Learning arxiv.org
Use a distance metric in the loss to determine to which class does an object belongs to from a few examples.
-
Massive Exploration of Neural Machine Translation Architectures arxiv.org
That yields intuition about the boundaries of what works for doing NMT within a framed seq2seq problem formulation.
-
Neural Machine Translation and Sequence-to-sequence Models: A Tutorial arxiv.org
Interesting overview of the subject of NMT, I mostly read part 8 about RNNs with attention as a refresher.
-
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation arxiv.org
Merges the ideas of the U-Net and the DenseNet, this new neural network is especially good for huge datasets in image segmentation.
-
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling arxiv.org
3D-GANs for 3D model generation and fun 3D furniture arithmetics from embeddings (think like word2vec word arithmetics with 3D furniture representations).
-
Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation arxiv.org
In 2016: stacked residual LSTMs with attention mechanisms on encoder/decoder are the best for NMT (Neural Machine Translation).
-
WaveNet: a Generative Model for Raw Audio arxiv.org
Epic raw voice/music generation with new architectures based on dilated causal convolutions to capture more audio length.
- next page of items loading…
