Computer Science
7,622 items in 38 lists
-
Deep learning for lazybones oduerr.github.io
Transfer learning tutorial in TensorFlow for vision from high-level embeddings of a pretrained CNN, AlexNet 2012.
-
The Annotated Transformer nlp.seas.harvard.edu
Good for understanding the "Attention Is All You Need" (AIAYN) paper.
-
Hacker News news.ycombinator.com
Maybe how I discovered ML - Interesting trends appear on that site way before they get to be a big deal.
-
Deep Learning & Recurrent Neural Networks (DL&RNN) neuraxio.com
The most richly dense, accelerated course on the topic of Deep Learning & Recurrent Neural Networks (scroll at the end).
-
Neural Networks and Deep Learning neuralnetworksanddeeplearning.com
This book covers many of the core concepts behind neural networks and deep learning.
-
Neural Networks and Deep Learning, ch.4 neuralnetworksanddeeplearning.com
A visual proof that neural nets can compute any function.
-
Neural Networks and Deep Learning, ch.2 neuralnetworksanddeeplearning.com
Overview on how does the backpropagation algorithm works.
-
Naver naver.com
This is a Korean search engine - best used with Google Translate, ironically. Surprisingly, sometimes deep learning search results and comprehensible advanced math content shows up more easily there than on Google search.
-
Hybrid computing using a neural network with dynamic external memory nature.com
Improvements on differentiable memory based on NTMs: now it is the Differentiable Neural Computer (DNC).
-
Estimating an Optimal Learning Rate For a Deep Neural Network medium.com
Clever trick to estimate an optimal learning rate prior any single full training.
-
Yes you should understand backprop medium.com
Exposing backprop's caveats and the importance of knowing that while training models.
-
The Illustrated Transformer jalammar.github.io
Also good for understanding the "Attention Is All You Need" (AIAYN) paper.
-
The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) jalammar.github.io
Understand the different approaches used for NLP's ImageNet moment.
-
ML / DL repositories I starred github.com
GitHub is full of nice code samples & projects.
-
carpedm20's repositories github.com
Many interesting neural network architectures are implemented by the Korean guy Taehoon Kim, A.K.A. carpedm20.
-
Window Functions en.wikipedia.org
Wikipedia page that lists some of the known window functions - note that the Hann-Poisson window is specially interesting for greedy hill-climbing algorithms (like gradient descent for example).
-
Predictions made by Ray Kurzweil en.wikipedia.org
List of mid to long term futuristic predictions made by Ray Kurzweil.
-
DL&RNN Course dl-rnn-course.neuraxio.com
I created this richely dense course on Deep Learning and Recurrent Neural Networks.
-
Attention and Augmented Recurrent Neural Networks distill.pub
Interesting for visual animations, it is a nice intro to attention mechanisms as an example.
-
Bidirectional Recurrent Neural Networks di.ufpe.br
Better classifications with RNNs with bidirectional scanning on the time axis.
-
Gradient descent algorithms in an almost flat landscape devblogs.nvidia.com
Visualize how different optimizers interacts with an almost flat landscape.
-
WaveNet: A Generative Model for Raw Audio deepmind.com
Realistic talking machines: perfect voice generation.
-
DataTau datatau.com
This is a hub similar to Hacker News, but specific to data science.
-
Going Deeper with Convolutions cv-foundation.org
GoogLeNet: Appearance of "Inception" layers/modules, the idea is of parallelizing conv layers into many mini-conv of different size with "same" padding, concatenated on depth.
-
Exploring the Depths of Recurrent Neural Networks with Stochastic Residual Learning cs224d.stanford.edu
Basically, residual connections can be better than stacked RNNs in the presented case of sentiment analysis.
-
ImageNet Classification with Deep Convolutional Neural Networks cs.toronto.edu
AlexNet, 2012 ILSVRC, breakthrough of the ReLU activation function.
-
Cornell Movie--Dialogs Corpus cs.cornell.edu
This could be used for a chatbot.
-
Geoffrey Hinton interview coursera.org
Andrew Ng interviews Geoffrey Hinton, who talks about his research and breaktroughs, and gives advice for students.
-
Neural Networks, Manifolds, and Topology colah.github.io
Fresh look on how neurons map information.
-
Uncle Bob's Principles Of OOD butunclebob.com
Not only the SOLID principles are needed for doing clean code, but the furtherless known REP, CCP, CRP, ADP, SDP and SAP principles are very important for developping huge software that must be bundled in different separated packages.
-
Artificial Neural Networks: Mathematics of Backpropagation briandolhansky.com
Picturing backprop, mathematically.
-
Some other books I have read books.google.ca
Some books listed here are less related to deep learning but are still somehow relevant to this list.
-
Improving Language Understanding with Unsupervised Learning blog.openai.com
SOTA across many NLP tasks from unsupervised pretraining on huge corpus.
-
The future of deep learning blog.keras.io
François Chollet's thoughts on the future of deep learning.
-
Recommending music on Spotify with deep learning benanne.github.io
Awesome for doing clustering on audio - post by an intern at Spotify.
-
Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions arxiv.org
A WaveNet used as a vocoder can be conditioned on generated Mel Spectrograms from the Tacotron 2 LSTM neural network with attention to generate neat audio from text.
-
Self-Normalizing Neural Networks arxiv.org
Appearance of the incredible SELU activation function.
-
Prototypical Networks for Few-shot Learning arxiv.org
Use a distance metric in the loss to determine to which class does an object belongs to from a few examples.
-
Massive Exploration of Neural Machine Translation Architectures arxiv.org
That yields intuition about the boundaries of what works for doing NMT within a framed seq2seq problem formulation.
-
Neural Machine Translation and Sequence-to-sequence Models: A Tutorial arxiv.org
Interesting overview of the subject of NMT, I mostly read part 8 about RNNs with attention as a refresher.
-
The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation arxiv.org
Merges the ideas of the U-Net and the DenseNet, this new neural network is especially good for huge datasets in image segmentation.
-
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling arxiv.org
3D-GANs for 3D model generation and fun 3D furniture arithmetics from embeddings (think like word2vec word arithmetics with 3D furniture representations).
-
Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation arxiv.org
In 2016: stacked residual LSTMs with attention mechanisms on encoder/decoder are the best for NMT (Neural Machine Translation).
-
WaveNet: a Generative Model for Raw Audio arxiv.org
Epic raw voice/music generation with new architectures based on dilated causal convolutions to capture more audio length.
-
Densely Connected Convolutional Networks arxiv.org
Best Paper Award at CVPR 2017, yielding improvements on state-of-the-art performances on CIFAR-10, CIFAR-100 and SVHN datasets, this new neural network architecture is named DenseNet.
-
Learning to learn by gradient descent by gradient descent arxiv.org
RNN as an optimizer: introducing the L2L optimizer, a meta-neural network.
-
Matching Networks for One Shot Learning arxiv.org
Interesting way of doing one-shot learning with low-data by using an attention mechanism and a query to compare an image to other images for classification.
-
Adaptive Computation Time for Recurrent Neural Networks arxiv.org
Let RNNs decide how long they compute. I would love to see how well would it combines to Neural Turing Machines. Interesting interactive visualizations on the subject can be found here.
-
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning arxiv.org
For improving GoogLeNet with residual connections.
-
Exploring the Limits of Language Modeling arxiv.org
Nice recursive models using word-level LSTMs on top of a character-level CNN using an overkill amount of GPU power.
- next page of items loading…
