labml-nn


Namelabml-nn JSON
Version 0.4.136 PyPI version JSON
download
home_pagehttps://github.com/labmlai/annotated_deep_learning_paper_implementations
Summary🧑‍🏫 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit), optimizers (adam, radam, adabelief), gans(dcgan, cyclegan, stylegan2), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, diffusion, etc. 🧠
upload_time2023-11-05 08:18:17
maintainer
docs_urlNone
authorVaruna Jayasiri, Nipun Wijerathne
requires_python
license
keywords machine learning
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            [![Twitter](https://img.shields.io/twitter/follow/labmlai?style=social)](https://twitter.com/labmlai)
[![Sponsor](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/labmlai)

# [labml.ai Deep Learning Paper Implementations](https://nn.labml.ai/index.html)

This is a collection of simple PyTorch implementations of
neural networks and related algorithms.
These implementations are documented with explanations,

[The website](https://nn.labml.ai/index.html)
renders these as side-by-side formatted notes.
We believe these would help you understand these algorithms better.

![Screenshot](https://nn.labml.ai/dqn-light.png)

We are actively maintaining this repo and adding new 
implementations almost weekly.
[![Twitter](https://img.shields.io/twitter/follow/labmlai?style=social)](https://twitter.com/labmlai) for updates.

## Paper Implementations

#### ✨ [Transformers](https://nn.labml.ai/transformers/index.html)

* [Multi-headed attention](https://nn.labml.ai/transformers/mha.html)
* [Transformer building blocks](https://nn.labml.ai/transformers/models.html) 
* [Transformer XL](https://nn.labml.ai/transformers/xl/index.html)
    * [Relative multi-headed attention](https://nn.labml.ai/transformers/xl/relative_mha.html)
* [Rotary Positional Embeddings](https://nn.labml.ai/transformers/rope/index.html)
* [Attention with Linear Biases (ALiBi)](https://nn.labml.ai/transformers/alibi/index.html)
* [RETRO](https://nn.labml.ai/transformers/retro/index.html)
* [Compressive Transformer](https://nn.labml.ai/transformers/compressive/index.html)
* [GPT Architecture](https://nn.labml.ai/transformers/gpt/index.html)
* [GLU Variants](https://nn.labml.ai/transformers/glu_variants/simple.html)
* [kNN-LM: Generalization through Memorization](https://nn.labml.ai/transformers/knn)
* [Feedback Transformer](https://nn.labml.ai/transformers/feedback/index.html)
* [Switch Transformer](https://nn.labml.ai/transformers/switch/index.html)
* [Fast Weights Transformer](https://nn.labml.ai/transformers/fast_weights/index.html)
* [FNet](https://nn.labml.ai/transformers/fnet/index.html)
* [Attention Free Transformer](https://nn.labml.ai/transformers/aft/index.html)
* [Masked Language Model](https://nn.labml.ai/transformers/mlm/index.html)
* [MLP-Mixer: An all-MLP Architecture for Vision](https://nn.labml.ai/transformers/mlp_mixer/index.html)
* [Pay Attention to MLPs (gMLP)](https://nn.labml.ai/transformers/gmlp/index.html)
* [Vision Transformer (ViT)](https://nn.labml.ai/transformers/vit/index.html)
* [Primer EZ](https://nn.labml.ai/transformers/primer_ez/index.html)
* [Hourglass](https://nn.labml.ai/transformers/hour_glass/index.html)

#### ✨ [Eleuther GPT-NeoX](https://nn.labml.ai/neox/index.html)
* [Generate on a 48GB GPU](https://nn.labml.ai/neox/samples/generate.html)
* [Finetune on two 48GB GPUs](https://nn.labml.ai/neox/samples/finetune.html)
* [LLM.int8()](https://nn.labml.ai/neox/utils/llm_int8.html)

#### ✨ [Diffusion models](https://nn.labml.ai/diffusion/index.html)

* [Denoising Diffusion Probabilistic Models (DDPM)](https://nn.labml.ai/diffusion/ddpm/index.html)
* [Denoising Diffusion Implicit Models (DDIM)](https://nn.labml.ai/diffusion/stable_diffusion/sampler/ddim.html)
* [Latent Diffusion Models](https://nn.labml.ai/diffusion/stable_diffusion/latent_diffusion.html)
* [Stable Diffusion](https://nn.labml.ai/diffusion/stable_diffusion/index.html)

#### ✨ [Generative Adversarial Networks](https://nn.labml.ai/gan/index.html)
* [Original GAN](https://nn.labml.ai/gan/original/index.html)
* [GAN with deep convolutional network](https://nn.labml.ai/gan/dcgan/index.html)
* [Cycle GAN](https://nn.labml.ai/gan/cycle_gan/index.html)
* [Wasserstein GAN](https://nn.labml.ai/gan/wasserstein/index.html)
* [Wasserstein GAN with Gradient Penalty](https://nn.labml.ai/gan/wasserstein/gradient_penalty/index.html)
* [StyleGAN 2](https://nn.labml.ai/gan/stylegan/index.html)

#### ✨ [Recurrent Highway Networks](https://nn.labml.ai/recurrent_highway_networks/index.html)

#### ✨ [LSTM](https://nn.labml.ai/lstm/index.html)

#### ✨ [HyperNetworks - HyperLSTM](https://nn.labml.ai/hypernetworks/hyper_lstm.html)

#### ✨ [ResNet](https://nn.labml.ai/resnet/index.html)

#### ✨ [ConvMixer](https://nn.labml.ai/conv_mixer/index.html)

#### ✨ [Capsule Networks](https://nn.labml.ai/capsule_networks/index.html)

#### ✨ [U-Net](https://nn.labml.ai/unet/index.html)

#### ✨ [Sketch RNN](https://nn.labml.ai/sketch_rnn/index.html)

#### ✨ Graph Neural Networks

* [Graph Attention Networks (GAT)](https://nn.labml.ai/graphs/gat/index.html)
* [Graph Attention Networks v2 (GATv2)](https://nn.labml.ai/graphs/gatv2/index.html)

#### ✨ [Counterfactual Regret Minimization (CFR)](https://nn.labml.ai/cfr/index.html)

Solving games with incomplete information such as poker with CFR.

* [Kuhn Poker](https://nn.labml.ai/cfr/kuhn/index.html)

#### ✨ [Reinforcement Learning](https://nn.labml.ai/rl/index.html)
* [Proximal Policy Optimization](https://nn.labml.ai/rl/ppo/index.html) with
 [Generalized Advantage Estimation](https://nn.labml.ai/rl/ppo/gae.html)
* [Deep Q Networks](https://nn.labml.ai/rl/dqn/index.html) with
 with [Dueling Network](https://nn.labml.ai/rl/dqn/model.html),
 [Prioritized Replay](https://nn.labml.ai/rl/dqn/replay_buffer.html)
 and Double Q Network.

#### ✨ [Optimizers](https://nn.labml.ai/optimizers/index.html)
* [Adam](https://nn.labml.ai/optimizers/adam.html)
* [AMSGrad](https://nn.labml.ai/optimizers/amsgrad.html)
* [Adam Optimizer with warmup](https://nn.labml.ai/optimizers/adam_warmup.html)
* [Noam Optimizer](https://nn.labml.ai/optimizers/noam.html)
* [Rectified Adam Optimizer](https://nn.labml.ai/optimizers/radam.html)
* [AdaBelief Optimizer](https://nn.labml.ai/optimizers/ada_belief.html)
* [Sophia-G Optimizer](https://nn.labml.ai/optimizers/sophia.html)

#### ✨ [Normalization Layers](https://nn.labml.ai/normalization/index.html)
* [Batch Normalization](https://nn.labml.ai/normalization/batch_norm/index.html)
* [Layer Normalization](https://nn.labml.ai/normalization/layer_norm/index.html)
* [Instance Normalization](https://nn.labml.ai/normalization/instance_norm/index.html)
* [Group Normalization](https://nn.labml.ai/normalization/group_norm/index.html)
* [Weight Standardization](https://nn.labml.ai/normalization/weight_standardization/index.html)
* [Batch-Channel Normalization](https://nn.labml.ai/normalization/batch_channel_norm/index.html)
* [DeepNorm](https://nn.labml.ai/normalization/deep_norm/index.html)

#### ✨ [Distillation](https://nn.labml.ai/distillation/index.html)

#### ✨ [Adaptive Computation](https://nn.labml.ai/adaptive_computation/index.html)

* [PonderNet](https://nn.labml.ai/adaptive_computation/ponder_net/index.html)

#### ✨ [Uncertainty](https://nn.labml.ai/uncertainty/index.html)

* [Evidential Deep Learning to Quantify Classification Uncertainty](https://nn.labml.ai/uncertainty/evidence/index.html)

#### ✨ [Activations](https://nn.labml.ai/activations/index.html)

* [Fuzzy Tiling Activations](https://nn.labml.ai/activations/fta/index.html)

#### ✨ [Langauge Model Sampling Techniques](https://nn.labml.ai/sampling/index.html)
* [Greedy Sampling](https://nn.labml.ai/sampling/greedy.html)
* [Temperature Sampling](https://nn.labml.ai/sampling/temperature.html)
* [Top-k Sampling](https://nn.labml.ai/sampling/top_k.html)
* [Nucleus Sampling](https://nn.labml.ai/sampling/nucleus.html)

#### ✨ [Scalable Training/Inference](https://nn.labml.ai/scaling/index.html)
* [Zero3 memory optimizations](https://nn.labml.ai/scaling/zero3/index.html)

## Highlighted Research Paper PDFs

* [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2205.14135.pdf)
* [Autoregressive Search Engines: Generating Substrings as Document Identifiers](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2204.10628.pdf)
* [Training Compute-Optimal Large Language Models](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2203.15556.pdf)
* [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/1910.02054.pdf)
* [PaLM: Scaling Language Modeling with Pathways](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2204.02311.pdf)
* [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/dall-e-2.pdf)
* [STaR: Self-Taught Reasoner Bootstrapping Reasoning With Reasoning](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2203.14465.pdf)
* [Improving language models by retrieving from trillions of tokens](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2112.04426.pdf)
* [NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2003.08934.pdf)
* [Attention Is All You Need](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/1706.03762.pdf)
* [Denoising Diffusion Probabilistic Models](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2006.11239.pdf)
* [Primer: Searching for Efficient Transformers for Language Modeling](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2109.08668.pdf)
* [On First-Order Meta-Learning Algorithms](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/1803.02999.pdf)
* [Learning Transferable Visual Models From Natural Language Supervision](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2103.00020.pdf)
* [The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2109.02869.pdf)
* [Meta-Gradient Reinforcement Learning](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/1805.09801.pdf)
* [ETA Prediction with Graph Neural Networks in Google Maps](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/google_maps_eta.pdf)
* [PonderNet: Learning to Ponder](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/ponder_net.pdf)
* [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/muzero.pdf)
* [GANs N’ Roses: Stable, Controllable, Diverse Image to Image Translation (works for videos too!)](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/gans_n_roses.pdf)
* [An Image is Worth 16X16 Word: Transformers for Image Recognition at Scale](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/vit.pdf)
* [Deep Residual Learning for Image Recognition](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/resnet.pdf)
* [Distilling the Knowledge in a Neural Network](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/distillation.pdf)

### Installation

```bash
pip install labml-nn
```

### Citing

If you use this for academic research, please cite it using the following BibTeX entry.

```bibtex
@misc{labml,
 author = {Varuna Jayasiri, Nipun Wijerathne},
 title = {labml.ai Annotated Paper Implementations},
 year = {2020},
 url = {https://nn.labml.ai/},
}
```

### Other Projects

#### [🚀 Trending Research Papers](https://papers.labml.ai/)

This shows the most popular research papers on social media. It also aggregates links to useful resources like paper explanations videos and discussions.


#### [🧪 labml.ai/labml](https://github.com/labmlai/labml)

This is a library that let's you monitor deep learning model training and hardware usage from your mobile phone. It also comes with a bunch of other tools to help write deep learning code efficiently.


            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/labmlai/annotated_deep_learning_paper_implementations",
    "name": "labml-nn",
    "maintainer": "",
    "docs_url": null,
    "requires_python": "",
    "maintainer_email": "",
    "keywords": "machine learning",
    "author": "Varuna Jayasiri, Nipun Wijerathne",
    "author_email": "vpjayasiri@gmail.com, hnipun@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/20/b8/8567f07f0d4be44bd94eae4367f6dd6255d45f340dc91355c41a359f4c8a/labml-nn-0.4.136.tar.gz",
    "platform": null,
    "description": "[![Twitter](https://img.shields.io/twitter/follow/labmlai?style=social)](https://twitter.com/labmlai)\n[![Sponsor](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub&color=%23fe8e86)](https://github.com/sponsors/labmlai)\n\n# [labml.ai Deep Learning Paper Implementations](https://nn.labml.ai/index.html)\n\nThis is a collection of simple PyTorch implementations of\nneural networks and related algorithms.\nThese implementations are documented with explanations,\n\n[The website](https://nn.labml.ai/index.html)\nrenders these as side-by-side formatted notes.\nWe believe these would help you understand these algorithms better.\n\n![Screenshot](https://nn.labml.ai/dqn-light.png)\n\nWe are actively maintaining this repo and adding new \nimplementations almost weekly.\n[![Twitter](https://img.shields.io/twitter/follow/labmlai?style=social)](https://twitter.com/labmlai) for updates.\n\n## Paper Implementations\n\n#### \u2728 [Transformers](https://nn.labml.ai/transformers/index.html)\n\n* [Multi-headed attention](https://nn.labml.ai/transformers/mha.html)\n* [Transformer building blocks](https://nn.labml.ai/transformers/models.html) \n* [Transformer XL](https://nn.labml.ai/transformers/xl/index.html)\n    * [Relative multi-headed attention](https://nn.labml.ai/transformers/xl/relative_mha.html)\n* [Rotary Positional Embeddings](https://nn.labml.ai/transformers/rope/index.html)\n* [Attention with Linear Biases (ALiBi)](https://nn.labml.ai/transformers/alibi/index.html)\n* [RETRO](https://nn.labml.ai/transformers/retro/index.html)\n* [Compressive Transformer](https://nn.labml.ai/transformers/compressive/index.html)\n* [GPT Architecture](https://nn.labml.ai/transformers/gpt/index.html)\n* [GLU Variants](https://nn.labml.ai/transformers/glu_variants/simple.html)\n* [kNN-LM: Generalization through Memorization](https://nn.labml.ai/transformers/knn)\n* [Feedback Transformer](https://nn.labml.ai/transformers/feedback/index.html)\n* [Switch Transformer](https://nn.labml.ai/transformers/switch/index.html)\n* [Fast Weights Transformer](https://nn.labml.ai/transformers/fast_weights/index.html)\n* [FNet](https://nn.labml.ai/transformers/fnet/index.html)\n* [Attention Free Transformer](https://nn.labml.ai/transformers/aft/index.html)\n* [Masked Language Model](https://nn.labml.ai/transformers/mlm/index.html)\n* [MLP-Mixer: An all-MLP Architecture for Vision](https://nn.labml.ai/transformers/mlp_mixer/index.html)\n* [Pay Attention to MLPs (gMLP)](https://nn.labml.ai/transformers/gmlp/index.html)\n* [Vision Transformer (ViT)](https://nn.labml.ai/transformers/vit/index.html)\n* [Primer EZ](https://nn.labml.ai/transformers/primer_ez/index.html)\n* [Hourglass](https://nn.labml.ai/transformers/hour_glass/index.html)\n\n#### \u2728 [Eleuther GPT-NeoX](https://nn.labml.ai/neox/index.html)\n* [Generate on a 48GB GPU](https://nn.labml.ai/neox/samples/generate.html)\n* [Finetune on two 48GB GPUs](https://nn.labml.ai/neox/samples/finetune.html)\n* [LLM.int8()](https://nn.labml.ai/neox/utils/llm_int8.html)\n\n#### \u2728 [Diffusion models](https://nn.labml.ai/diffusion/index.html)\n\n* [Denoising Diffusion Probabilistic Models (DDPM)](https://nn.labml.ai/diffusion/ddpm/index.html)\n* [Denoising Diffusion Implicit Models (DDIM)](https://nn.labml.ai/diffusion/stable_diffusion/sampler/ddim.html)\n* [Latent Diffusion Models](https://nn.labml.ai/diffusion/stable_diffusion/latent_diffusion.html)\n* [Stable Diffusion](https://nn.labml.ai/diffusion/stable_diffusion/index.html)\n\n#### \u2728 [Generative Adversarial Networks](https://nn.labml.ai/gan/index.html)\n* [Original GAN](https://nn.labml.ai/gan/original/index.html)\n* [GAN with deep convolutional network](https://nn.labml.ai/gan/dcgan/index.html)\n* [Cycle GAN](https://nn.labml.ai/gan/cycle_gan/index.html)\n* [Wasserstein GAN](https://nn.labml.ai/gan/wasserstein/index.html)\n* [Wasserstein GAN with Gradient Penalty](https://nn.labml.ai/gan/wasserstein/gradient_penalty/index.html)\n* [StyleGAN 2](https://nn.labml.ai/gan/stylegan/index.html)\n\n#### \u2728 [Recurrent Highway Networks](https://nn.labml.ai/recurrent_highway_networks/index.html)\n\n#### \u2728 [LSTM](https://nn.labml.ai/lstm/index.html)\n\n#### \u2728 [HyperNetworks - HyperLSTM](https://nn.labml.ai/hypernetworks/hyper_lstm.html)\n\n#### \u2728 [ResNet](https://nn.labml.ai/resnet/index.html)\n\n#### \u2728 [ConvMixer](https://nn.labml.ai/conv_mixer/index.html)\n\n#### \u2728 [Capsule Networks](https://nn.labml.ai/capsule_networks/index.html)\n\n#### \u2728 [U-Net](https://nn.labml.ai/unet/index.html)\n\n#### \u2728 [Sketch RNN](https://nn.labml.ai/sketch_rnn/index.html)\n\n#### \u2728 Graph Neural Networks\n\n* [Graph Attention Networks (GAT)](https://nn.labml.ai/graphs/gat/index.html)\n* [Graph Attention Networks v2 (GATv2)](https://nn.labml.ai/graphs/gatv2/index.html)\n\n#### \u2728 [Counterfactual Regret Minimization (CFR)](https://nn.labml.ai/cfr/index.html)\n\nSolving games with incomplete information such as poker with CFR.\n\n* [Kuhn Poker](https://nn.labml.ai/cfr/kuhn/index.html)\n\n#### \u2728 [Reinforcement Learning](https://nn.labml.ai/rl/index.html)\n* [Proximal Policy Optimization](https://nn.labml.ai/rl/ppo/index.html) with\n [Generalized Advantage Estimation](https://nn.labml.ai/rl/ppo/gae.html)\n* [Deep Q Networks](https://nn.labml.ai/rl/dqn/index.html) with\n with [Dueling Network](https://nn.labml.ai/rl/dqn/model.html),\n [Prioritized Replay](https://nn.labml.ai/rl/dqn/replay_buffer.html)\n and Double Q Network.\n\n#### \u2728 [Optimizers](https://nn.labml.ai/optimizers/index.html)\n* [Adam](https://nn.labml.ai/optimizers/adam.html)\n* [AMSGrad](https://nn.labml.ai/optimizers/amsgrad.html)\n* [Adam Optimizer with warmup](https://nn.labml.ai/optimizers/adam_warmup.html)\n* [Noam Optimizer](https://nn.labml.ai/optimizers/noam.html)\n* [Rectified Adam Optimizer](https://nn.labml.ai/optimizers/radam.html)\n* [AdaBelief Optimizer](https://nn.labml.ai/optimizers/ada_belief.html)\n* [Sophia-G Optimizer](https://nn.labml.ai/optimizers/sophia.html)\n\n#### \u2728 [Normalization Layers](https://nn.labml.ai/normalization/index.html)\n* [Batch Normalization](https://nn.labml.ai/normalization/batch_norm/index.html)\n* [Layer Normalization](https://nn.labml.ai/normalization/layer_norm/index.html)\n* [Instance Normalization](https://nn.labml.ai/normalization/instance_norm/index.html)\n* [Group Normalization](https://nn.labml.ai/normalization/group_norm/index.html)\n* [Weight Standardization](https://nn.labml.ai/normalization/weight_standardization/index.html)\n* [Batch-Channel Normalization](https://nn.labml.ai/normalization/batch_channel_norm/index.html)\n* [DeepNorm](https://nn.labml.ai/normalization/deep_norm/index.html)\n\n#### \u2728 [Distillation](https://nn.labml.ai/distillation/index.html)\n\n#### \u2728 [Adaptive Computation](https://nn.labml.ai/adaptive_computation/index.html)\n\n* [PonderNet](https://nn.labml.ai/adaptive_computation/ponder_net/index.html)\n\n#### \u2728 [Uncertainty](https://nn.labml.ai/uncertainty/index.html)\n\n* [Evidential Deep Learning to Quantify Classification Uncertainty](https://nn.labml.ai/uncertainty/evidence/index.html)\n\n#### \u2728 [Activations](https://nn.labml.ai/activations/index.html)\n\n* [Fuzzy Tiling Activations](https://nn.labml.ai/activations/fta/index.html)\n\n#### \u2728 [Langauge Model Sampling Techniques](https://nn.labml.ai/sampling/index.html)\n* [Greedy Sampling](https://nn.labml.ai/sampling/greedy.html)\n* [Temperature Sampling](https://nn.labml.ai/sampling/temperature.html)\n* [Top-k Sampling](https://nn.labml.ai/sampling/top_k.html)\n* [Nucleus Sampling](https://nn.labml.ai/sampling/nucleus.html)\n\n#### \u2728 [Scalable Training/Inference](https://nn.labml.ai/scaling/index.html)\n* [Zero3 memory optimizations](https://nn.labml.ai/scaling/zero3/index.html)\n\n## Highlighted Research Paper PDFs\n\n* [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2205.14135.pdf)\n* [Autoregressive Search Engines: Generating Substrings as Document Identifiers](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2204.10628.pdf)\n* [Training Compute-Optimal Large Language Models](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2203.15556.pdf)\n* [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/1910.02054.pdf)\n* [PaLM: Scaling Language Modeling with Pathways](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2204.02311.pdf)\n* [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/dall-e-2.pdf)\n* [STaR: Self-Taught Reasoner Bootstrapping Reasoning With Reasoning](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2203.14465.pdf)\n* [Improving language models by retrieving from trillions of tokens](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2112.04426.pdf)\n* [NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2003.08934.pdf)\n* [Attention Is All You Need](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/1706.03762.pdf)\n* [Denoising Diffusion Probabilistic Models](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2006.11239.pdf)\n* [Primer: Searching for Efficient Transformers for Language Modeling](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2109.08668.pdf)\n* [On First-Order Meta-Learning Algorithms](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/1803.02999.pdf)\n* [Learning Transferable Visual Models From Natural Language Supervision](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2103.00020.pdf)\n* [The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/2109.02869.pdf)\n* [Meta-Gradient Reinforcement Learning](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/1805.09801.pdf)\n* [ETA Prediction with Graph Neural Networks in Google Maps](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/google_maps_eta.pdf)\n* [PonderNet: Learning to Ponder](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/ponder_net.pdf)\n* [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/muzero.pdf)\n* [GANs N\u2019 Roses: Stable, Controllable, Diverse Image to Image Translation (works for videos too!)](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/gans_n_roses.pdf)\n* [An Image is Worth 16X16 Word: Transformers for Image Recognition at Scale](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/vit.pdf)\n* [Deep Residual Learning for Image Recognition](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/resnet.pdf)\n* [Distilling the Knowledge in a Neural Network](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/master/papers/distillation.pdf)\n\n### Installation\n\n```bash\npip install labml-nn\n```\n\n### Citing\n\nIf you use this for academic research, please cite it using the following BibTeX entry.\n\n```bibtex\n@misc{labml,\n author = {Varuna Jayasiri, Nipun Wijerathne},\n title = {labml.ai Annotated Paper Implementations},\n year = {2020},\n url = {https://nn.labml.ai/},\n}\n```\n\n### Other Projects\n\n#### [\ud83d\ude80 Trending Research Papers](https://papers.labml.ai/)\n\nThis shows the most popular research papers on social media. It also aggregates links to useful resources like paper explanations videos and discussions.\n\n\n#### [\ud83e\uddea labml.ai/labml](https://github.com/labmlai/labml)\n\nThis is a library that let's you monitor deep learning model training and hardware usage from your mobile phone. It also comes with a bunch of other tools to help write deep learning code efficiently.\n\n",
    "bugtrack_url": null,
    "license": "",
    "summary": "\ud83e\uddd1\u200d\ud83c\udfeb Implementations/tutorials of deep learning papers with side-by-side notes \ud83d\udcdd; including transformers (original, xl, switch, feedback, vit), optimizers (adam, radam, adabelief), gans(dcgan, cyclegan, stylegan2), \ud83c\udfae reinforcement learning (ppo, dqn), capsnet, distillation, diffusion, etc. \ud83e\udde0",
    "version": "0.4.136",
    "project_urls": {
        "Documentation": "https://nn.labml.ai",
        "Homepage": "https://github.com/labmlai/annotated_deep_learning_paper_implementations"
    },
    "split_keywords": [
        "machine",
        "learning"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "d3ac1594743bc73697cf23b55314b3de012f7f973af53487438152f69f178ce8",
                "md5": "8619563dd627978008d5b0aa3a43a796",
                "sha256": "dcf4bf948d1bd35608726d642945cd863338f2b43acda06d012618597a3fac4c"
            },
            "downloads": -1,
            "filename": "labml_nn-0.4.136-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "8619563dd627978008d5b0aa3a43a796",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 434964,
            "upload_time": "2023-11-05T08:18:15",
            "upload_time_iso_8601": "2023-11-05T08:18:15.110510Z",
            "url": "https://files.pythonhosted.org/packages/d3/ac/1594743bc73697cf23b55314b3de012f7f973af53487438152f69f178ce8/labml_nn-0.4.136-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "20b88567f07f0d4be44bd94eae4367f6dd6255d45f340dc91355c41a359f4c8a",
                "md5": "7312d0a07695a5f5ddcdbf0d57fa17e6",
                "sha256": "3e90d4394a286846dca734f99cf3e0dee2577e5b384308e5e06857131ab532e4"
            },
            "downloads": -1,
            "filename": "labml-nn-0.4.136.tar.gz",
            "has_sig": false,
            "md5_digest": "7312d0a07695a5f5ddcdbf0d57fa17e6",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 309842,
            "upload_time": "2023-11-05T08:18:17",
            "upload_time_iso_8601": "2023-11-05T08:18:17.751274Z",
            "url": "https://files.pythonhosted.org/packages/20/b8/8567f07f0d4be44bd94eae4367f6dd6255d45f340dc91355c41a359f4c8a/labml-nn-0.4.136.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-11-05 08:18:17",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "labmlai",
    "github_project": "annotated_deep_learning_paper_implementations",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "requirements": [],
    "lcname": "labml-nn"
}
        
Elapsed time: 0.14073s