[](http://www.repostatus.org/#active)
[](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/)
[](https://github.com/nvidia/nemo/actions/workflows/codeql.yml)
[](https://github.com/NVIDIA/NeMo/blob/master/LICENSE)
[](https://badge.fury.io/py/nemo-toolkit)
[](https://badge.fury.io/py/nemo-toolkit)
[](https://pepy.tech/project/nemo-toolkit)
[](https://github.com/psf/black)
# **NVIDIA NeMo Framework**
## Latest News
<!-- markdownlint-disable -->
<details open>
<summary><b>Large Language Models and Multimodal Models</b></summary>
<details>
<summary>
<a href="https://docs.nvidia.com/nemo-framework/user-guide/latest/llms/llama/index.html#new-llama-3-1-support for more information/">
New Llama 3.1 Support
</a> (2024-07-23)
</summary>
The NeMo Framework now supports training and customizing the Llama 3.1 collection of LLMs from Meta.
<br><br>
</details>
<details>
<summary>
<a href="https://aws.amazon.com/blogs/machine-learning/accelerate-your-generative-ai-distributed-training-workloads-with-the-nvidia-nemo-framework-on-amazon-eks/">
Accelerate your Generative AI Distributed Training Workloads with the NVIDIA NeMo Framework on Amazon EKS
</a> (2024-07-16)
</summary>
NVIDIA NeMo Framework now runs distributed training workloads on an Amazon Elastic Kubernetes Service (Amazon EKS) cluster. For step-by-step instructions on creating an EKS cluster and running distributed training workloads with NeMo, see the GitHub repository <a href="https://github.com/aws-samples/awsome-distributed-training/tree/main/3.test_cases/2.nemo-launcher/EKS/"> here.</a>
<br><br>
</details>
<details>
<summary>
<a href="https://developer.nvidia.com/blog/nvidia-nemo-accelerates-llm-innovation-with-hybrid-state-space-model-support/">
NVIDIA NeMo Accelerates LLM Innovation with Hybrid State Space Model Support
</a> (2024/06/17)
</summary>
NVIDIA NeMo and Megatron Core now support pre-training and fine-tuning of state space models (SSMs). NeMo also supports training models based on the Griffin architecture as described by Google DeepMind.
<br><br>
</details>
<details>
<summary>
<a href="https://huggingface.co/models?sort=trending&search=nvidia%2Fnemotron-4-340B">
NVIDIA releases 340B base, instruct, and reward models pretrained on a total of 9T tokens.
</a> (2024-06-18)
</summary>
See documentation and tutorials for SFT, PEFT, and PTQ with
<a href="https://docs.nvidia.com/nemo-framework/user-guide/latest/llms/nemotron/index.html">
Nemotron 340B
</a>
in the NeMo Framework User Guide.
<br><br>
</details>
<details>
<summary>
<a href="https://developer.nvidia.com/blog/nvidia-sets-new-generative-ai-performance-and-scale-records-in-mlperf-training-v4-0/">
NVIDIA sets new generative AI performance and scale records in MLPerf Training v4.0
</a> (2024/06/12)
</summary>
Using NVIDIA NeMo Framework and NVIDIA Hopper GPUs NVIDIA was able to scale to 11,616 H100 GPUs and achieve near-linear performance scaling on LLM pretraining.
NVIDIA also achieved the highest LLM fine-tuning performance and raised the bar for text-to-image training.
<br><br>
</details>
<details>
<summary>
<a href="https://cloud.google.com/blog/products/compute/gke-and-nvidia-nemo-framework-to-train-generative-ai-models">
Accelerate your generative AI journey with NVIDIA NeMo Framework on GKE
</a> (2024/03/16)
</summary>
An end-to-end walkthrough to train generative AI models on the Google Kubernetes Engine (GKE) using the NVIDIA NeMo Framework is available at https://github.com/GoogleCloudPlatform/nvidia-nemo-on-gke.
The walkthrough includes detailed instructions on how to set up a Google Cloud Project and pre-train a GPT model using the NeMo Framework.
<br><br>
</details>
</details>
<details open>
<summary><b>Speech Recognition</b></summary>
<details>
<summary>
<a href="https://developer.nvidia.com/blog/new-standard-for-speech-recognition-and-translation-from-the-nvidia-nemo-canary-model/">
New Standard for Speech Recognition and Translation from the NVIDIA NeMo Canary Model
</a> (2024/04/18)
</summary>
The NeMo team just released Canary, a multilingual model that transcribes speech in English, Spanish, German, and French with punctuation and capitalization.
Canary also provides bi-directional translation, between English and the three other supported languages.
<br><br>
</details>
<details>
<summary>
<a href="https://developer.nvidia.com/blog/pushing-the-boundaries-of-speech-recognition-with-nemo-parakeet-asr-models/">
Pushing the Boundaries of Speech Recognition with NVIDIA NeMo Parakeet ASR Models
</a> (2024/04/18)
</summary>
NVIDIA NeMo, an end-to-end platform for the development of multimodal generative AI models at scale anywhere—on any cloud and on-premises—released the Parakeet family of automatic speech recognition (ASR) models.
These state-of-the-art ASR models, developed in collaboration with Suno.ai, transcribe spoken English with exceptional accuracy.
<br><br>
</details>
<details>
<summary>
<a href="https://developer.nvidia.com/blog/turbocharge-asr-accuracy-and-speed-with-nvidia-nemo-parakeet-tdt/">
Turbocharge ASR Accuracy and Speed with NVIDIA NeMo Parakeet-TDT
</a> (2024/04/18)
</summary>
NVIDIA NeMo, an end-to-end platform for developing multimodal generative AI models at scale anywhere—on any cloud and on-premises—recently released Parakeet-TDT.
This new addition to the NeMo ASR Parakeet model family boasts better accuracy and 64% greater speed over the previously best model, Parakeet-RNNT-1.1B.
<br><br>
</details>
</details>
<!-- markdownlint-enable -->
## Introduction
NVIDIA NeMo Framework is a scalable and cloud-native generative AI
framework built for researchers and PyTorch developers working on Large
Language Models (LLMs), Multimodal Models (MMs), Automatic Speech
Recognition (ASR), Text to Speech (TTS), and Computer Vision (CV)
domains. It is designed to help you efficiently create, customize, and
deploy new generative AI models by leveraging existing code and
pre-trained model checkpoints.
For technical documentation, please see the [NeMo Framework User
Guide](https://docs.nvidia.com/nemo-framework/user-guide/latest/playbooks/index.html).
## LLMs and MMs Training, Alignment, and Customization
All NeMo models are trained with
[Lightning](https://github.com/Lightning-AI/lightning). Training is
automatically scalable to 1000s of GPUs.
When applicable, NeMo models leverage cutting-edge distributed training
techniques, incorporating [parallelism
strategies](https://docs.nvidia.com/nemo-framework/user-guide/latest/modeloverview.html)
to enable efficient training of very large models. These techniques
include Tensor Parallelism (TP), Pipeline Parallelism (PP), Fully
Sharded Data Parallelism (FSDP), Mixture-of-Experts (MoE), and Mixed
Precision Training with BFloat16 and FP8, as well as others.
NeMo Transformer-based LLMs and MMs utilize [NVIDIA Transformer
Engine](https://github.com/NVIDIA/TransformerEngine) for FP8 training on
NVIDIA Hopper GPUs, while leveraging [NVIDIA Megatron
Core](https://github.com/NVIDIA/Megatron-LM/tree/main/megatron/core) for
scaling Transformer model training.
NeMo LLMs can be aligned with state-of-the-art methods such as SteerLM,
Direct Preference Optimization (DPO), and Reinforcement Learning from
Human Feedback (RLHF). See [NVIDIA NeMo
Aligner](https://github.com/NVIDIA/NeMo-Aligner) for more information.
In addition to supervised fine-tuning (SFT), NeMo also supports the
latest parameter efficient fine-tuning (PEFT) techniques such as LoRA,
P-Tuning, Adapters, and IA3. Refer to the [NeMo Framework User
Guide](https://docs.nvidia.com/nemo-framework/user-guide/latest/sft_peft/index.html)
for the full list of supported models and techniques.
## LLMs and MMs Deployment and Optimization
NeMo LLMs and MMs can be deployed and optimized with [NVIDIA NeMo
Microservices](https://developer.nvidia.com/nemo-microservices-early-access).
## Speech AI
NeMo ASR and TTS models can be optimized for inference and deployed for
production use cases with [NVIDIA Riva](https://developer.nvidia.com/riva).
## NeMo Framework Launcher
[NeMo Framework
Launcher](https://github.com/NVIDIA/NeMo-Megatron-Launcher) is a
cloud-native tool that streamlines the NeMo Framework experience. It is
used for launching end-to-end NeMo Framework training jobs on CSPs and
Slurm clusters.
The NeMo Framework Launcher includes extensive recipes, scripts,
utilities, and documentation for training NeMo LLMs. It also includes
the NeMo Framework [Autoconfigurator](https://github.com/NVIDIA/NeMo-Megatron-Launcher#53-using-autoconfigurator-to-find-the-optimal-configuration),
which is designed to find the optimal model parallel configuration for
training on a specific cluster.
To get started quickly with the NeMo Framework Launcher, please see the
[NeMo Framework
Playbooks](https://docs.nvidia.com/nemo-framework/user-guide/latest/playbooks/index.html).
The NeMo Framework Launcher does not currently support ASR and TTS
training, but it will soon.
## Get Started with NeMo Framework
Getting started with NeMo Framework is easy. State-of-the-art pretrained
NeMo models are freely available on [Hugging Face
Hub](https://huggingface.co/models?library=nemo&sort=downloads&search=nvidia)
and [NVIDIA
NGC](https://catalog.ngc.nvidia.com/models?query=nemo&orderBy=weightPopularDESC).
These models can be used to generate text or images, transcribe audio,
and synthesize speech in just a few lines of code.
We have extensive
[tutorials](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/starthere/tutorials.html)
that can be run on [Google Colab](https://colab.research.google.com) or
with our [NGC NeMo Framework
Container](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo).
We also have
[playbooks](https://docs.nvidia.com/nemo-framework/user-guide/latest/playbooks/index.html)
for users who want to train NeMo models with the NeMo Framework
Launcher.
For advanced users who want to train NeMo models from scratch or
fine-tune existing NeMo models, we have a full suite of [example
scripts](https://github.com/NVIDIA/NeMo/tree/main/examples) that support
multi-GPU/multi-node training.
## Key Features
- [Large Language Models](nemo/collections/nlp/README.md)
- [Multimodal](nemo/collections/multimodal/README.md)
- [Automatic Speech Recognition](nemo/collections/asr/README.md)
- [Text to Speech](nemo/collections/tts/README.md)
- [Computer Vision](nemo/collections/vision/README.md)
## Requirements
- Python 3.10 or above
- Pytorch 1.13.1 or above
- NVIDIA GPU (if you intend to do model training)
## Developer Documentation
| Version | Status | Description |
| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| Latest | [](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/) | [Documentation of the latest (i.e. main) branch.](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/) |
| Stable | [](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/) | [Documentation of the stable (i.e. most recent release)](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/) |
## Install NeMo Framework
The NeMo Framework can be installed in a variety of ways, depending on
your needs. Depending on the domain, you may find one of the following
installation methods more suitable.
- Conda / Pip - Refer to [Conda](#conda) and [Pip](#pip) for
installation instructions.
- This is the recommended method for ASR and TTS domains.
- When using a Nvidia PyTorch container as the base, this is the
recommended method for all domains.
- Docker Containers - Refer to [Docker containers](#docker-containers)
for installation instructions.
- NeMo Framework container -
[nvcr.io/nvidia/nemo:24.05]{.title-ref}
- LLMs and MMs Dependencies - Refer to [LLMs and MMs
Dependencies](#install-llms-and-mms-dependencies) for installation
instructions.
**Important: We strongly recommended that you start with a base NVIDIA
PyTorch container: nvcr.io/nvidia/pytorch:24.02-py3.**
### Conda
Install NeMo in a fresh Conda environment:
```bash
conda create --name nemo python==3.10.12
conda activate nemo
```
Install PyTorch using their
[configurator](https://pytorch.org/get-started/locally/):
```bash
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
```
The command to install PyTorch may depend on your system. Use the
configurator linked above to find the right command for your system.
Then, install NeMo via Pip or from Source. We do not provide NeMo on the
conda-forge or any other Conda channel.
### Pip
To install the nemo_toolkit, use the following installation method:
```bash
apt-get update && apt-get install -y libsndfile1 ffmpeg
pip install Cython packaging
pip install nemo_toolkit['all']
```
Depending on the shell used, you may need to use the
`"nemo_toolkit[all]"` specifier instead in the above command.
### Pip from a Specific Domain
To install a specific domain of NeMo, you must first install the
nemo_toolkit using the instructions listed above. Then, you run the
following domain-specific commands:
```bash
pip install nemo_toolkit['asr']
pip install nemo_toolkit['nlp']
pip install nemo_toolkit['tts']
pip install nemo_toolkit['vision']
pip install nemo_toolkit['multimodal']
```
### Pip from a Source Branch
If you want to work with a specific version of NeMo from a particular
GitHub branch (e.g main), use the following installation method:
```bash
apt-get update && apt-get install -y libsndfile1 ffmpeg
pip install Cython packaging
python -m pip install git+https://github.com/NVIDIA/NeMo.git@{BRANCH}#egg=nemo_toolkit[all]
```
### Build from Source
If you want to clone the NeMo GitHub repository and contribute to NeMo
open-source development work, use the following installation method:
```bash
apt-get update && apt-get install -y libsndfile1 ffmpeg
git clone https://github.com/NVIDIA/NeMo
cd NeMo
./reinstall.sh
```
If you only want the toolkit without the additional Conda-based
dependencies, you can replace `reinstall.sh` with `pip install -e .`
when your PWD is the root of the NeMo repository.
### Mac Computers with Apple Silicon
To install NeMo on Mac computers with the Apple M-Series GPU, you need
to create a new Conda environment, install PyTorch 2.0 or higher, and
then install the nemo_toolkit.
**Important: This method is only applicable to the ASR domain.**
Run the following code:
```shell
# [optional] install mecab using Homebrew, to use sacrebleu for NLP collection
# you can install Homebrew here: https://brew.sh
brew install mecab
# [optional] install pynini using Conda, to use text normalization
conda install -c conda-forge pynini
# install Cython manually
pip install cython packaging
# clone the repo and install in development mode
git clone https://github.com/NVIDIA/NeMo
cd NeMo
pip install 'nemo_toolkit[all]'
# Note that only the ASR toolkit is guaranteed to work on MacBook - so for MacBook use pip install 'nemo_toolkit[asr]'
```
### Windows Computers
To install the Windows Subsystem for Linux (WSL), run the following code
in PowerShell:
```shell
wsl --install
# [note] If you run wsl --install and see the WSL help text, it means WSL is already installed.
```
To learn more about installing WSL, refer to [Microsoft\'s official
documentation](https://learn.microsoft.com/en-us/windows/wsl/install).
After installing your Linux distribution with WSL, two options are
available:
**Option 1:** Open the distribution (Ubuntu by default) from the Start
menu and follow the instructions.
**Option 2:** Launch the Terminal application. Download it from
[Microsoft\'s Windows Terminal
page](https://learn.microsoft.com/en-us/windows/terminal) if not
installed.
Next, follow the instructions for Linux systems, as provided above. For
example:
```bash
apt-get update && apt-get install -y libsndfile1 ffmpeg
git clone https://github.com/NVIDIA/NeMo
cd NeMo
./reinstall.sh
```
### RNNT
For optimal performance of a Recurrent Neural Network Transducer (RNNT),
install the Numba package from Conda.
Run the following code:
```bash
conda remove numba
pip uninstall numba
conda install -c conda-forge numba
```
## Install LLMs and MMs Dependencies
If you work with the LLM and MM domains, three additional dependencies
are required: NVIDIA Apex, NVIDIA Transformer Engine, and NVIDIA
Megatron Core. When working with the [main]{.title-ref} branch, these
dependencies may require a recent commit.
The most recent working versions of these dependencies are here:
```bash
export apex_commit=810ffae374a2b9cb4b5c5e28eaeca7d7998fca0c
export te_commit=bfe21c3d68b0a9951e5716fb520045db53419c5e
export mcore_commit=02871b4df8c69fac687ab6676c4246e936ce92d0
export nv_pytorch_tag=24.02-py3
```
When using a released version of NeMo, please refer to the [Software
Component
Versions](https://docs.nvidia.com/nemo-framework/user-guide/latest/softwarecomponentversions.html)
for the correct versions.
### PyTorch Container
We recommended that you start with a base NVIDIA PyTorch container:
nvcr.io/nvidia/pytorch:24.02-py3.
If starting with a base NVIDIA PyTorch container, you must first launch
the container:
```bash
docker run \
--gpus all \
-it \
--rm \
--shm-size=16g \
--ulimit memlock=-1 \
--ulimit stack=67108864 \
nvcr.io/nvidia/pytorch:$nv_pytorch_tag
```
Next, you need to install the dependencies.
### Apex
NVIDIA Apex is required for LLM and MM domains. Although Apex is
pre-installed in the NVIDIA PyTorch container, you may need to update it
to a newer version.
To install Apex, run the following code:
```bash
git clone https://github.com/NVIDIA/apex.git
cd apex
git checkout $apex_commit
pip install . -v --no-build-isolation --disable-pip-version-check --no-cache-dir --config-settings "--build-option=--cpp_ext --cuda_ext --fast_layer_norm --distributed_adam --deprecated_fused_adam --group_norm"
```
When attempting to install Apex separately from the NVIDIA PyTorch
container, you might encounter an error if the CUDA version on your
system is different from the one used to compile PyTorch. To bypass this
error, you can comment out the relevant line in the setup file located
in the Apex repository on GitHub here:
<https://github.com/NVIDIA/apex/blob/master/setup.py#L32>.
cuda-nvprof is needed to install Apex. The version should match the CUDA
version that you are using.
To install cuda-nvprof, run the following code:
```bash
conda install -c nvidia cuda-nvprof=11.8
```
Finally, install the packaging:
```bash
pip install packaging
```
To install the most recent versions of Apex locally, it might be
necessary to remove the [pyproject.toml]{.title-ref} file from the Apex
directory.
### Transformer Engine
NVIDIA Transformer Engine is required for LLM and MM domains. Although
the Transformer Engine is pre-installed in the NVIDIA PyTorch container,
you may need to update it to a newer version.
The Transformer Engine facilitates training with FP8 precision on NVIDIA
Hopper GPUs and introduces many enhancements for the training of
Transformer-based models. Refer to [Transformer Engine](https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/installation.html)
for information.
To install Transformer Engine, run the following code:
```bash
git clone https://github.com/NVIDIA/TransformerEngine.git && \
cd TransformerEngine && \
git checkout $te_commit && \
git submodule init && git submodule update && \
NVTE_FRAMEWORK=pytorch NVTE_WITH_USERBUFFERS=1 MPI_HOME=/usr/local/mpi pip install .
```
Transformer Engine requires PyTorch to be built with at least CUDA 11.8.
### Megatron Core
Megatron Core is required for LLM and MM domains. Megatron Core is a
library for scaling large Transformer-based models. NeMo LLMs and MMs
leverage Megatron Core for model parallelism, transformer architectures,
and optimized PyTorch datasets.
To install Megatron Core, run the following code:
```bash
git clone https://github.com/NVIDIA/Megatron-LM.git && \
cd Megatron-LM && \
git checkout $mcore_commit && \
pip install . && \
cd megatron/core/datasets && \
make
```
## NeMo Text Processing
NeMo Text Processing, specifically Inverse Text Normalization, is now a
separate repository. It is located here:
<https://github.com/NVIDIA/NeMo-text-processing>.
## Docker Containers
NeMo containers are launched concurrently with NeMo version updates.
NeMo Framework now supports LLMs, MMs, ASR, and TTS in a single
consolidated Docker container. You can find additional information about
released containers on the [NeMo releases
page](https://github.com/NVIDIA/NeMo/releases).
To use a pre-built container, run the following code:
```bash
docker pull nvcr.io/nvidia/nemo:24.05
```
To build a nemo container with Dockerfile from a branch, run the
following code:
```bash
DOCKER_BUILDKIT=1 docker build -f Dockerfile -t nemo:latest
```
If you choose to work with the main branch, we recommend using NVIDIA\'s
PyTorch container version 23.10-py3 and then installing from GitHub.
```bash
docker run --gpus all -it --rm -v <nemo_github_folder>:/NeMo --shm-size=8g \
-p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit \
stack=67108864 --device=/dev/snd nvcr.io/nvidia/pytorch:23.10-py3
```
## Future Work
The NeMo Framework Launcher does not currently support ASR and TTS
training, but it will soon.
## Discussions Board
FAQ can be found on the NeMo [Discussions
board](https://github.com/NVIDIA/NeMo/discussions). You are welcome to
ask questions or start discussions on the board.
## Contribute to NeMo
We welcome community contributions! Please refer to
[CONTRIBUTING.md](https://github.com/NVIDIA/NeMo/blob/stable/CONTRIBUTING.md)
for the process.
## Publications
We provide an ever-growing list of
[publications](https://nvidia.github.io/NeMo/publications/) that utilize
the NeMo Framework.
To contribute an article to the collection, please submit a pull request
to the `gh-pages-src` branch of this repository. For detailed
information, please consult the README located at the [gh-pages-src
branch](https://github.com/NVIDIA/NeMo/tree/gh-pages-src#readme).
## Blogs
<!-- markdownlint-disable -->
<details open>
<summary><b>Large Language Models and Multimodal Models</b></summary>
<details>
<summary>
<a href="https://blogs.nvidia.com/blog/bria-builds-responsible-generative-ai-using-nemo-picasso/">
Bria Builds Responsible Generative AI for Enterprises Using NVIDIA NeMo, Picasso
</a> (2024/03/06)
</summary>
Bria, a Tel Aviv startup at the forefront of visual generative AI for enterprises now leverages the NVIDIA NeMo Framework.
The Bria.ai platform uses reference implementations from the NeMo Multimodal collection, trained on NVIDIA Tensor Core GPUs, to enable high-throughput and low-latency image generation.
Bria has also adopted NVIDIA Picasso, a foundry for visual generative AI models, to run inference.
<br><br>
</details>
<details>
<summary>
<a href="https://developer.nvidia.com/blog/new-nvidia-nemo-framework-features-and-nvidia-h200-supercharge-llm-training-performance-and-versatility/">
New NVIDIA NeMo Framework Features and NVIDIA H200
</a> (2023/12/06)
</summary>
NVIDIA NeMo Framework now includes several optimizations and enhancements,
including:
1) Fully Sharded Data Parallelism (FSDP) to improve the efficiency of training large-scale AI models,
2) Mix of Experts (MoE)-based LLM architectures with expert parallelism for efficient LLM training at scale,
3) Reinforcement Learning from Human Feedback (RLHF) with TensorRT-LLM for inference stage acceleration, and
4) up to 4.2x speedups for Llama 2 pre-training on NVIDIA H200 Tensor Core GPUs.
<br><br>
<a href="https://developer.nvidia.com/blog/new-nvidia-nemo-framework-features-and-nvidia-h200-supercharge-llm-training-performance-and-versatility">
<img src="https://github.com/sbhavani/TransformerEngine/blob/main/docs/examples/H200-NeMo-performance.png" alt="H200-NeMo-performance" style="width: 600px;"></a>
<br><br>
</details>
<details>
<summary>
<a href="https://blogs.nvidia.com/blog/nemo-amazon-titan/">
NVIDIA now powers training for Amazon Titan Foundation models
</a> (2023/11/28)
</summary>
NVIDIA NeMo Framework now empowers the Amazon Titan foundation models (FM) with efficient training of large language models (LLMs).
The Titan FMs form the basis of Amazon’s generative AI service, Amazon Bedrock.
The NeMo Framework provides a versatile framework for building, customizing, and running LLMs.
<br><br>
</details>
</details>
<!-- markdownlint-enable -->
## Licenses
- [NeMo GitHub Apache 2.0
license](https://github.com/NVIDIA/NeMo?tab=Apache-2.0-1-ov-file#readme)
- NeMo is licensed under the [NVIDIA AI PRODUCT
AGREEMENT](https://www.nvidia.com/en-us/data-center/products/nvidia-ai-enterprise/eula/).
By pulling and using the container, you accept the terms and
conditions of this license.
Raw data
{
"_id": null,
"home_page": "https://github.com/nvidia/nemo",
"name": "nemo-toolkit",
"maintainer": "NVIDIA",
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": "nemo-toolkit@nvidia.com",
"keywords": "deep learning, machine learning, gpu, NLP, NeMo, nvidia, pytorch, torch, tts, speech, language",
"author": "NVIDIA",
"author_email": "nemo-toolkit@nvidia.com",
"download_url": "https://files.pythonhosted.org/packages/c7/04/41844f17002a5c5a671d9edd9b26fa87102902305b4782f5136d46bdab7b/nemo_toolkit-2.0.0.tar.gz",
"platform": null,
"description": "[](http://www.repostatus.org/#active)\n[](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/)\n[](https://github.com/nvidia/nemo/actions/workflows/codeql.yml)\n[](https://github.com/NVIDIA/NeMo/blob/master/LICENSE)\n[](https://badge.fury.io/py/nemo-toolkit)\n[](https://badge.fury.io/py/nemo-toolkit)\n[](https://pepy.tech/project/nemo-toolkit)\n[](https://github.com/psf/black)\n\n# **NVIDIA NeMo Framework**\n\n## Latest News\n\n<!-- markdownlint-disable -->\n<details open>\n <summary><b>Large Language Models and Multimodal Models</b></summary>\n <details>\n <summary>\n <a href=\"https://docs.nvidia.com/nemo-framework/user-guide/latest/llms/llama/index.html#new-llama-3-1-support for more information/\">\n New Llama 3.1 Support\n </a> (2024-07-23)\n </summary>\n The NeMo Framework now supports training and customizing the Llama 3.1 collection of LLMs from Meta.\n <br><br>\n </details>\n <details>\n <summary>\n <a href=\"https://aws.amazon.com/blogs/machine-learning/accelerate-your-generative-ai-distributed-training-workloads-with-the-nvidia-nemo-framework-on-amazon-eks/\">\n Accelerate your Generative AI Distributed Training Workloads with the NVIDIA NeMo Framework on Amazon EKS\n </a> (2024-07-16)\n </summary>\n NVIDIA NeMo Framework now runs distributed training workloads on an Amazon Elastic Kubernetes Service (Amazon EKS) cluster. For step-by-step instructions on creating an EKS cluster and running distributed training workloads with NeMo, see the GitHub repository <a href=\"https://github.com/aws-samples/awsome-distributed-training/tree/main/3.test_cases/2.nemo-launcher/EKS/\"> here.</a>\n <br><br>\n </details>\n <details>\n <summary>\n <a href=\"https://developer.nvidia.com/blog/nvidia-nemo-accelerates-llm-innovation-with-hybrid-state-space-model-support/\">\n NVIDIA NeMo Accelerates LLM Innovation with Hybrid State Space Model Support\n </a> (2024/06/17)\n </summary>\n NVIDIA NeMo and Megatron Core now support pre-training and fine-tuning of state space models (SSMs). NeMo also supports training models based on the Griffin architecture as described by Google DeepMind. \n <br><br>\n </details>\n <details>\n <summary>\n <a href=\"https://huggingface.co/models?sort=trending&search=nvidia%2Fnemotron-4-340B\">\n NVIDIA releases 340B base, instruct, and reward models pretrained on a total of 9T tokens.\n </a> (2024-06-18)\n </summary>\n See documentation and tutorials for SFT, PEFT, and PTQ with \n <a href=\"https://docs.nvidia.com/nemo-framework/user-guide/latest/llms/nemotron/index.html\">\n Nemotron 340B \n </a>\n in the NeMo Framework User Guide.\n <br><br>\n </details>\n <details>\n <summary>\n <a href=\"https://developer.nvidia.com/blog/nvidia-sets-new-generative-ai-performance-and-scale-records-in-mlperf-training-v4-0/\">\n NVIDIA sets new generative AI performance and scale records in MLPerf Training v4.0\n </a> (2024/06/12)\n </summary>\n Using NVIDIA NeMo Framework and NVIDIA Hopper GPUs NVIDIA was able to scale to 11,616 H100 GPUs and achieve near-linear performance scaling on LLM pretraining. \n NVIDIA also achieved the highest LLM fine-tuning performance and raised the bar for text-to-image training.\n <br><br>\n </details>\n <details>\n <summary>\n <a href=\"https://cloud.google.com/blog/products/compute/gke-and-nvidia-nemo-framework-to-train-generative-ai-models\">\n Accelerate your generative AI journey with NVIDIA NeMo Framework on GKE\n </a> (2024/03/16)\n </summary>\n An end-to-end walkthrough to train generative AI models on the Google Kubernetes Engine (GKE) using the NVIDIA NeMo Framework is available at https://github.com/GoogleCloudPlatform/nvidia-nemo-on-gke. \n The walkthrough includes detailed instructions on how to set up a Google Cloud Project and pre-train a GPT model using the NeMo Framework.\n <br><br>\n </details>\n</details>\n\n<details open>\n <summary><b>Speech Recognition</b></summary>\n <details>\n <summary>\n <a href=\"https://developer.nvidia.com/blog/new-standard-for-speech-recognition-and-translation-from-the-nvidia-nemo-canary-model/\">\n New Standard for Speech Recognition and Translation from the NVIDIA NeMo Canary Model\n </a> (2024/04/18)\n </summary>\n The NeMo team just released Canary, a multilingual model that transcribes speech in English, Spanish, German, and French with punctuation and capitalization. \n Canary also provides bi-directional translation, between English and the three other supported languages.\n <br><br>\n </details>\n <details>\n <summary>\n <a href=\"https://developer.nvidia.com/blog/pushing-the-boundaries-of-speech-recognition-with-nemo-parakeet-asr-models/\">\n Pushing the Boundaries of Speech Recognition with NVIDIA NeMo Parakeet ASR Models\n </a> (2024/04/18)\n </summary>\n NVIDIA NeMo, an end-to-end platform for the development of multimodal generative AI models at scale anywhere\u2014on any cloud and on-premises\u2014released the Parakeet family of automatic speech recognition (ASR) models. \n These state-of-the-art ASR models, developed in collaboration with Suno.ai, transcribe spoken English with exceptional accuracy.\n <br><br>\n </details>\n <details>\n <summary>\n <a href=\"https://developer.nvidia.com/blog/turbocharge-asr-accuracy-and-speed-with-nvidia-nemo-parakeet-tdt/\">\n Turbocharge ASR Accuracy and Speed with NVIDIA NeMo Parakeet-TDT\n </a> (2024/04/18)\n </summary>\n NVIDIA NeMo, an end-to-end platform for developing multimodal generative AI models at scale anywhere\u2014on any cloud and on-premises\u2014recently released Parakeet-TDT. \n This new addition to the \u202fNeMo ASR Parakeet model family boasts better accuracy and 64% greater speed over the previously best model, Parakeet-RNNT-1.1B.\n <br><br>\n </details>\n</details>\n<!-- markdownlint-enable -->\n\n## Introduction\n\nNVIDIA NeMo Framework is a scalable and cloud-native generative AI\nframework built for researchers and PyTorch developers working on Large\nLanguage Models (LLMs), Multimodal Models (MMs), Automatic Speech\nRecognition (ASR), Text to Speech (TTS), and Computer Vision (CV)\ndomains. It is designed to help you efficiently create, customize, and\ndeploy new generative AI models by leveraging existing code and\npre-trained model checkpoints.\n\nFor technical documentation, please see the [NeMo Framework User\nGuide](https://docs.nvidia.com/nemo-framework/user-guide/latest/playbooks/index.html).\n\n## LLMs and MMs Training, Alignment, and Customization\n\nAll NeMo models are trained with\n[Lightning](https://github.com/Lightning-AI/lightning). Training is\nautomatically scalable to 1000s of GPUs.\n\nWhen applicable, NeMo models leverage cutting-edge distributed training\ntechniques, incorporating [parallelism\nstrategies](https://docs.nvidia.com/nemo-framework/user-guide/latest/modeloverview.html)\nto enable efficient training of very large models. These techniques\ninclude Tensor Parallelism (TP), Pipeline Parallelism (PP), Fully\nSharded Data Parallelism (FSDP), Mixture-of-Experts (MoE), and Mixed\nPrecision Training with BFloat16 and FP8, as well as others.\n\nNeMo Transformer-based LLMs and MMs utilize [NVIDIA Transformer\nEngine](https://github.com/NVIDIA/TransformerEngine) for FP8 training on\nNVIDIA Hopper GPUs, while leveraging [NVIDIA Megatron\nCore](https://github.com/NVIDIA/Megatron-LM/tree/main/megatron/core) for\nscaling Transformer model training.\n\nNeMo LLMs can be aligned with state-of-the-art methods such as SteerLM,\nDirect Preference Optimization (DPO), and Reinforcement Learning from\nHuman Feedback (RLHF). See [NVIDIA NeMo\nAligner](https://github.com/NVIDIA/NeMo-Aligner) for more information.\n\nIn addition to supervised fine-tuning (SFT), NeMo also supports the\nlatest parameter efficient fine-tuning (PEFT) techniques such as LoRA,\nP-Tuning, Adapters, and IA3. Refer to the [NeMo Framework User\nGuide](https://docs.nvidia.com/nemo-framework/user-guide/latest/sft_peft/index.html)\nfor the full list of supported models and techniques.\n\n## LLMs and MMs Deployment and Optimization\n\nNeMo LLMs and MMs can be deployed and optimized with [NVIDIA NeMo\nMicroservices](https://developer.nvidia.com/nemo-microservices-early-access).\n\n## Speech AI\n\nNeMo ASR and TTS models can be optimized for inference and deployed for\nproduction use cases with [NVIDIA Riva](https://developer.nvidia.com/riva).\n\n## NeMo Framework Launcher\n\n[NeMo Framework\nLauncher](https://github.com/NVIDIA/NeMo-Megatron-Launcher) is a\ncloud-native tool that streamlines the NeMo Framework experience. It is\nused for launching end-to-end NeMo Framework training jobs on CSPs and\nSlurm clusters.\n\nThe NeMo Framework Launcher includes extensive recipes, scripts,\nutilities, and documentation for training NeMo LLMs. It also includes\nthe NeMo Framework [Autoconfigurator](https://github.com/NVIDIA/NeMo-Megatron-Launcher#53-using-autoconfigurator-to-find-the-optimal-configuration),\nwhich is designed to find the optimal model parallel configuration for\ntraining on a specific cluster.\n\nTo get started quickly with the NeMo Framework Launcher, please see the\n[NeMo Framework\nPlaybooks](https://docs.nvidia.com/nemo-framework/user-guide/latest/playbooks/index.html).\nThe NeMo Framework Launcher does not currently support ASR and TTS\ntraining, but it will soon.\n\n## Get Started with NeMo Framework\n\nGetting started with NeMo Framework is easy. State-of-the-art pretrained\nNeMo models are freely available on [Hugging Face\nHub](https://huggingface.co/models?library=nemo&sort=downloads&search=nvidia)\nand [NVIDIA\nNGC](https://catalog.ngc.nvidia.com/models?query=nemo&orderBy=weightPopularDESC).\nThese models can be used to generate text or images, transcribe audio,\nand synthesize speech in just a few lines of code.\n\nWe have extensive\n[tutorials](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/starthere/tutorials.html)\nthat can be run on [Google Colab](https://colab.research.google.com) or\nwith our [NGC NeMo Framework\nContainer](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo).\nWe also have\n[playbooks](https://docs.nvidia.com/nemo-framework/user-guide/latest/playbooks/index.html)\nfor users who want to train NeMo models with the NeMo Framework\nLauncher.\n\nFor advanced users who want to train NeMo models from scratch or\nfine-tune existing NeMo models, we have a full suite of [example\nscripts](https://github.com/NVIDIA/NeMo/tree/main/examples) that support\nmulti-GPU/multi-node training.\n\n## Key Features\n\n- [Large Language Models](nemo/collections/nlp/README.md)\n- [Multimodal](nemo/collections/multimodal/README.md)\n- [Automatic Speech Recognition](nemo/collections/asr/README.md)\n- [Text to Speech](nemo/collections/tts/README.md)\n- [Computer Vision](nemo/collections/vision/README.md)\n\n## Requirements\n\n- Python 3.10 or above\n- Pytorch 1.13.1 or above\n- NVIDIA GPU (if you intend to do model training)\n\n## Developer Documentation\n\n| Version | Status | Description |\n| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |\n| Latest | [](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/) | [Documentation of the latest (i.e. main) branch.](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/main/) |\n| Stable | [](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/) | [Documentation of the stable (i.e. most recent release)](https://docs.nvidia.com/deeplearning/nemo/user-guide/docs/en/stable/) |\n\n## Install NeMo Framework\n\nThe NeMo Framework can be installed in a variety of ways, depending on\nyour needs. Depending on the domain, you may find one of the following\ninstallation methods more suitable.\n\n- Conda / Pip - Refer to [Conda](#conda) and [Pip](#pip) for\n installation instructions.\n - This is the recommended method for ASR and TTS domains.\n - When using a Nvidia PyTorch container as the base, this is the\n recommended method for all domains.\n- Docker Containers - Refer to [Docker containers](#docker-containers)\n for installation instructions.\n - NeMo Framework container -\n [nvcr.io/nvidia/nemo:24.05]{.title-ref}\n- LLMs and MMs Dependencies - Refer to [LLMs and MMs\n Dependencies](#install-llms-and-mms-dependencies) for installation\n instructions.\n\n**Important: We strongly recommended that you start with a base NVIDIA\nPyTorch container: nvcr.io/nvidia/pytorch:24.02-py3.**\n\n### Conda\n\nInstall NeMo in a fresh Conda environment:\n\n```bash\nconda create --name nemo python==3.10.12\nconda activate nemo\n```\n\nInstall PyTorch using their\n[configurator](https://pytorch.org/get-started/locally/):\n\n```bash\nconda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia\n```\n\nThe command to install PyTorch may depend on your system. Use the\nconfigurator linked above to find the right command for your system.\n\nThen, install NeMo via Pip or from Source. We do not provide NeMo on the\nconda-forge or any other Conda channel.\n\n### Pip\n\nTo install the nemo_toolkit, use the following installation method:\n\n```bash\napt-get update && apt-get install -y libsndfile1 ffmpeg\npip install Cython packaging\npip install nemo_toolkit['all']\n```\n\nDepending on the shell used, you may need to use the\n`\"nemo_toolkit[all]\"` specifier instead in the above command.\n\n### Pip from a Specific Domain\n\nTo install a specific domain of NeMo, you must first install the\nnemo_toolkit using the instructions listed above. Then, you run the\nfollowing domain-specific commands:\n\n```bash\npip install nemo_toolkit['asr']\npip install nemo_toolkit['nlp']\npip install nemo_toolkit['tts']\npip install nemo_toolkit['vision']\npip install nemo_toolkit['multimodal']\n```\n\n### Pip from a Source Branch\n\nIf you want to work with a specific version of NeMo from a particular\nGitHub branch (e.g main), use the following installation method:\n\n```bash\napt-get update && apt-get install -y libsndfile1 ffmpeg\npip install Cython packaging\npython -m pip install git+https://github.com/NVIDIA/NeMo.git@{BRANCH}#egg=nemo_toolkit[all]\n```\n\n### Build from Source\n\nIf you want to clone the NeMo GitHub repository and contribute to NeMo\nopen-source development work, use the following installation method:\n\n```bash\napt-get update && apt-get install -y libsndfile1 ffmpeg\ngit clone https://github.com/NVIDIA/NeMo\ncd NeMo\n./reinstall.sh\n```\n\nIf you only want the toolkit without the additional Conda-based\ndependencies, you can replace `reinstall.sh` with `pip install -e .`\nwhen your PWD is the root of the NeMo repository.\n\n### Mac Computers with Apple Silicon\n\nTo install NeMo on Mac computers with the Apple M-Series GPU, you need\nto create a new Conda environment, install PyTorch 2.0 or higher, and\nthen install the nemo_toolkit.\n\n**Important: This method is only applicable to the ASR domain.**\n\nRun the following code:\n\n```shell\n# [optional] install mecab using Homebrew, to use sacrebleu for NLP collection\n# you can install Homebrew here: https://brew.sh\nbrew install mecab\n\n# [optional] install pynini using Conda, to use text normalization\nconda install -c conda-forge pynini\n\n# install Cython manually\npip install cython packaging\n\n# clone the repo and install in development mode\ngit clone https://github.com/NVIDIA/NeMo\ncd NeMo\npip install 'nemo_toolkit[all]'\n\n# Note that only the ASR toolkit is guaranteed to work on MacBook - so for MacBook use pip install 'nemo_toolkit[asr]'\n```\n\n### Windows Computers\n\nTo install the Windows Subsystem for Linux (WSL), run the following code\nin PowerShell:\n\n```shell\nwsl --install\n# [note] If you run wsl --install and see the WSL help text, it means WSL is already installed.\n```\n\nTo learn more about installing WSL, refer to [Microsoft\\'s official\ndocumentation](https://learn.microsoft.com/en-us/windows/wsl/install).\n\nAfter installing your Linux distribution with WSL, two options are\navailable:\n\n**Option 1:** Open the distribution (Ubuntu by default) from the Start\nmenu and follow the instructions.\n\n**Option 2:** Launch the Terminal application. Download it from\n[Microsoft\\'s Windows Terminal\npage](https://learn.microsoft.com/en-us/windows/terminal) if not\ninstalled.\n\nNext, follow the instructions for Linux systems, as provided above. For\nexample:\n\n```bash\napt-get update && apt-get install -y libsndfile1 ffmpeg\ngit clone https://github.com/NVIDIA/NeMo\ncd NeMo\n./reinstall.sh\n```\n\n### RNNT\n\nFor optimal performance of a Recurrent Neural Network Transducer (RNNT),\ninstall the Numba package from Conda.\n\nRun the following code:\n\n```bash\nconda remove numba\npip uninstall numba\nconda install -c conda-forge numba\n```\n\n## Install LLMs and MMs Dependencies\n\nIf you work with the LLM and MM domains, three additional dependencies\nare required: NVIDIA Apex, NVIDIA Transformer Engine, and NVIDIA\nMegatron Core. When working with the [main]{.title-ref} branch, these\ndependencies may require a recent commit.\n\nThe most recent working versions of these dependencies are here:\n\n```bash\nexport apex_commit=810ffae374a2b9cb4b5c5e28eaeca7d7998fca0c\nexport te_commit=bfe21c3d68b0a9951e5716fb520045db53419c5e\nexport mcore_commit=02871b4df8c69fac687ab6676c4246e936ce92d0\nexport nv_pytorch_tag=24.02-py3\n```\n\nWhen using a released version of NeMo, please refer to the [Software\nComponent\nVersions](https://docs.nvidia.com/nemo-framework/user-guide/latest/softwarecomponentversions.html)\nfor the correct versions.\n\n### PyTorch Container\n\nWe recommended that you start with a base NVIDIA PyTorch container:\nnvcr.io/nvidia/pytorch:24.02-py3.\n\nIf starting with a base NVIDIA PyTorch container, you must first launch\nthe container:\n\n```bash\ndocker run \\\n --gpus all \\\n -it \\\n --rm \\\n --shm-size=16g \\\n --ulimit memlock=-1 \\\n --ulimit stack=67108864 \\\n nvcr.io/nvidia/pytorch:$nv_pytorch_tag\n```\n\nNext, you need to install the dependencies.\n\n### Apex\n\nNVIDIA Apex is required for LLM and MM domains. Although Apex is\npre-installed in the NVIDIA PyTorch container, you may need to update it\nto a newer version.\n\nTo install Apex, run the following code:\n\n```bash\ngit clone https://github.com/NVIDIA/apex.git\ncd apex\ngit checkout $apex_commit\npip install . -v --no-build-isolation --disable-pip-version-check --no-cache-dir --config-settings \"--build-option=--cpp_ext --cuda_ext --fast_layer_norm --distributed_adam --deprecated_fused_adam --group_norm\"\n```\n\nWhen attempting to install Apex separately from the NVIDIA PyTorch\ncontainer, you might encounter an error if the CUDA version on your\nsystem is different from the one used to compile PyTorch. To bypass this\nerror, you can comment out the relevant line in the setup file located\nin the Apex repository on GitHub here:\n<https://github.com/NVIDIA/apex/blob/master/setup.py#L32>.\n\ncuda-nvprof is needed to install Apex. The version should match the CUDA\nversion that you are using.\n\nTo install cuda-nvprof, run the following code:\n\n```bash\nconda install -c nvidia cuda-nvprof=11.8\n```\n\nFinally, install the packaging:\n\n```bash\npip install packaging\n```\n\nTo install the most recent versions of Apex locally, it might be\nnecessary to remove the [pyproject.toml]{.title-ref} file from the Apex\ndirectory.\n\n### Transformer Engine\n\nNVIDIA Transformer Engine is required for LLM and MM domains. Although\nthe Transformer Engine is pre-installed in the NVIDIA PyTorch container,\nyou may need to update it to a newer version.\n\nThe Transformer Engine facilitates training with FP8 precision on NVIDIA\nHopper GPUs and introduces many enhancements for the training of\nTransformer-based models. Refer to [Transformer Engine](https://docs.nvidia.com/deeplearning/transformer-engine/user-guide/installation.html)\nfor information.\n\nTo install Transformer Engine, run the following code:\n\n```bash\ngit clone https://github.com/NVIDIA/TransformerEngine.git && \\\ncd TransformerEngine && \\\ngit checkout $te_commit && \\\ngit submodule init && git submodule update && \\\nNVTE_FRAMEWORK=pytorch NVTE_WITH_USERBUFFERS=1 MPI_HOME=/usr/local/mpi pip install .\n```\n\nTransformer Engine requires PyTorch to be built with at least CUDA 11.8.\n\n### Megatron Core\n\nMegatron Core is required for LLM and MM domains. Megatron Core is a\nlibrary for scaling large Transformer-based models. NeMo LLMs and MMs\nleverage Megatron Core for model parallelism, transformer architectures,\nand optimized PyTorch datasets.\n\nTo install Megatron Core, run the following code:\n\n```bash\ngit clone https://github.com/NVIDIA/Megatron-LM.git && \\\ncd Megatron-LM && \\\ngit checkout $mcore_commit && \\\npip install . && \\\ncd megatron/core/datasets && \\\nmake\n```\n\n## NeMo Text Processing\n\nNeMo Text Processing, specifically Inverse Text Normalization, is now a\nseparate repository. It is located here:\n<https://github.com/NVIDIA/NeMo-text-processing>.\n\n## Docker Containers\n\nNeMo containers are launched concurrently with NeMo version updates.\nNeMo Framework now supports LLMs, MMs, ASR, and TTS in a single\nconsolidated Docker container. You can find additional information about\nreleased containers on the [NeMo releases\npage](https://github.com/NVIDIA/NeMo/releases).\n\nTo use a pre-built container, run the following code:\n\n```bash\ndocker pull nvcr.io/nvidia/nemo:24.05\n```\n\nTo build a nemo container with Dockerfile from a branch, run the\nfollowing code:\n\n```bash\nDOCKER_BUILDKIT=1 docker build -f Dockerfile -t nemo:latest\n```\n\nIf you choose to work with the main branch, we recommend using NVIDIA\\'s\nPyTorch container version 23.10-py3 and then installing from GitHub.\n\n```bash\ndocker run --gpus all -it --rm -v <nemo_github_folder>:/NeMo --shm-size=8g \\\n-p 8888:8888 -p 6006:6006 --ulimit memlock=-1 --ulimit \\\nstack=67108864 --device=/dev/snd nvcr.io/nvidia/pytorch:23.10-py3\n```\n\n## Future Work\n\nThe NeMo Framework Launcher does not currently support ASR and TTS\ntraining, but it will soon.\n\n## Discussions Board\n\nFAQ can be found on the NeMo [Discussions\nboard](https://github.com/NVIDIA/NeMo/discussions). You are welcome to\nask questions or start discussions on the board.\n\n## Contribute to NeMo\n\nWe welcome community contributions! Please refer to\n[CONTRIBUTING.md](https://github.com/NVIDIA/NeMo/blob/stable/CONTRIBUTING.md)\nfor the process.\n\n## Publications\n\nWe provide an ever-growing list of\n[publications](https://nvidia.github.io/NeMo/publications/) that utilize\nthe NeMo Framework.\n\nTo contribute an article to the collection, please submit a pull request\nto the `gh-pages-src` branch of this repository. For detailed\ninformation, please consult the README located at the [gh-pages-src\nbranch](https://github.com/NVIDIA/NeMo/tree/gh-pages-src#readme).\n\n## Blogs\n\n<!-- markdownlint-disable -->\n<details open>\n <summary><b>Large Language Models and Multimodal Models</b></summary>\n <details>\n <summary>\n <a href=\"https://blogs.nvidia.com/blog/bria-builds-responsible-generative-ai-using-nemo-picasso/\">\n Bria Builds Responsible Generative AI for Enterprises Using NVIDIA NeMo, Picasso\n </a> (2024/03/06)\n </summary>\n Bria, a Tel Aviv startup at the forefront of visual generative AI for enterprises now leverages the NVIDIA NeMo Framework. \n The Bria.ai platform uses reference implementations from the NeMo Multimodal collection, trained on NVIDIA Tensor Core GPUs, to enable high-throughput and low-latency image generation. \n Bria has also adopted NVIDIA Picasso, a foundry for visual generative AI models, to run inference.\n <br><br>\n </details>\n <details>\n <summary>\n <a href=\"https://developer.nvidia.com/blog/new-nvidia-nemo-framework-features-and-nvidia-h200-supercharge-llm-training-performance-and-versatility/\">\n New NVIDIA NeMo Framework Features and NVIDIA H200\n </a> (2023/12/06)\n </summary>\n NVIDIA NeMo Framework now includes several optimizations and enhancements, \n including: \n 1) Fully Sharded Data Parallelism (FSDP) to improve the efficiency of training large-scale AI models, \n 2) Mix of Experts (MoE)-based LLM architectures with expert parallelism for efficient LLM training at scale, \n 3) Reinforcement Learning from Human Feedback (RLHF) with TensorRT-LLM for inference stage acceleration, and \n 4) up to 4.2x speedups for Llama 2 pre-training on NVIDIA H200 Tensor Core GPUs.\n <br><br>\n <a href=\"https://developer.nvidia.com/blog/new-nvidia-nemo-framework-features-and-nvidia-h200-supercharge-llm-training-performance-and-versatility\">\n <img src=\"https://github.com/sbhavani/TransformerEngine/blob/main/docs/examples/H200-NeMo-performance.png\" alt=\"H200-NeMo-performance\" style=\"width: 600px;\"></a>\n <br><br>\n </details>\n <details>\n <summary>\n <a href=\"https://blogs.nvidia.com/blog/nemo-amazon-titan/\">\n NVIDIA now powers training for Amazon Titan Foundation models\n </a> (2023/11/28)\n </summary>\n NVIDIA NeMo Framework now empowers the Amazon Titan foundation models (FM) with efficient training of large language models (LLMs). \n The Titan FMs form the basis of Amazon\u2019s generative AI service, Amazon Bedrock. \n The NeMo Framework provides a versatile framework for building, customizing, and running LLMs.\n <br><br>\n </details>\n</details>\n<!-- markdownlint-enable -->\n\n## Licenses\n\n- [NeMo GitHub Apache 2.0\n license](https://github.com/NVIDIA/NeMo?tab=Apache-2.0-1-ov-file#readme)\n- NeMo is licensed under the [NVIDIA AI PRODUCT\n AGREEMENT](https://www.nvidia.com/en-us/data-center/products/nvidia-ai-enterprise/eula/).\n By pulling and using the container, you accept the terms and\n conditions of this license.\n",
"bugtrack_url": null,
"license": "Apache2",
"summary": "NeMo - a toolkit for Conversational AI",
"version": "2.0.0",
"project_urls": {
"Download": "https://github.com/NVIDIA/NeMo/releases",
"Homepage": "https://github.com/nvidia/nemo"
},
"split_keywords": [
"deep learning",
" machine learning",
" gpu",
" nlp",
" nemo",
" nvidia",
" pytorch",
" torch",
" tts",
" speech",
" language"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f35dadc38248d91e0d3cae58ddb2d1bee1db9b4c094471aba50eef46a09b5ba7",
"md5": "4c0c25c1c9977601aa1c448d1ca4f311",
"sha256": "9af25cb1002d75960d410a3e533d9c9d7dbae6e9a67ef48954a816eaaec3651d"
},
"downloads": -1,
"filename": "nemo_toolkit-2.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4c0c25c1c9977601aa1c448d1ca4f311",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 4308525,
"upload_time": "2024-11-14T18:42:35",
"upload_time_iso_8601": "2024-11-14T18:42:35.069894Z",
"url": "https://files.pythonhosted.org/packages/f3/5d/adc38248d91e0d3cae58ddb2d1bee1db9b4c094471aba50eef46a09b5ba7/nemo_toolkit-2.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c70441844f17002a5c5a671d9edd9b26fa87102902305b4782f5136d46bdab7b",
"md5": "7ff7e0d858647684c46a6cb030285583",
"sha256": "8ebd89e152890832be63eef86a940755fbd8a3e8256ca1bbbf5bbcd120c2bb7d"
},
"downloads": -1,
"filename": "nemo_toolkit-2.0.0.tar.gz",
"has_sig": false,
"md5_digest": "7ff7e0d858647684c46a6cb030285583",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 3189366,
"upload_time": "2024-11-14T18:42:38",
"upload_time_iso_8601": "2024-11-14T18:42:38.054066Z",
"url": "https://files.pythonhosted.org/packages/c7/04/41844f17002a5c5a671d9edd9b26fa87102902305b4782f5136d46bdab7b/nemo_toolkit-2.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-14 18:42:38",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "nvidia",
"github_project": "nemo",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "nemo-toolkit"
}
JSON
