| Name | torchtune JSON |
| Version |
0.1.0
JSON |
| download |
| home_page | None |
| Summary | A native-PyTorch library for LLM fine-tuning |
| upload_time | 2024-04-16 03:24:15 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | BSD 3-Clause License Copyright 2024 Meta Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice,this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
| keywords |
pytorch
finetuning
llm
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
[](https://github.com/pytorch/torchtune/actions/workflows/unit_test.yaml)
[](https://discord.gg/4Xsdn8Rr9Q)
# torchtune
[**Introduction**](#introduction) | [**Installation**](#installation) | [**Get Started**](#get-started) | [**Documentation**](https://pytorch.org/torchtune) | [**Design Principles**](#design-principles) | [**Community Contributions**](#community-contributions) | [**License**](#license)
## Introduction
torchtune is a PyTorch-native library for easily authoring, fine-tuning and experimenting with LLMs. We're excited to announce our alpha release!
torchtune provides:
- Native-PyTorch implementations of popular LLMs using composable and modular building blocks
- Easy-to-use and hackable training recipes for popular fine-tuning techniques (LoRA, QLoRA) - no trainers, no frameworks, just PyTorch!
- YAML configs for easily configuring training, evaluation, quantization or inference recipes
- Built-in support for many popular dataset formats and prompt templates to help you quickly get started with training
torchtune focuses on integrating with popular tools and libraries from the ecosystem. These are just a few examples, with more under development:
- [Hugging Face Hub](https://huggingface.co/docs/hub/en/index) for [accessing model weights](torchtune/_cli/download.py)
- [EleutherAI's LM Eval Harness](https://github.com/EleutherAI/lm-evaluation-harness) for [evaluating](recipes/eleuther_eval.py) trained models
- [Hugging Face Datasets](https://huggingface.co/docs/datasets/en/index) for [access](torchtune/datasets/_instruct.py) to training and evaluation datasets
- [PyTorch FSDP](https://pytorch.org/docs/stable/fsdp.html) for distributed training
- [torchao](https://github.com/pytorch-labs/ao) for lower precision dtypes and [post-training quantization](recipes/quantize.py) techniques
- [Weights & Biases](https://wandb.ai/site) for [logging](https://pytorch.org/torchtune/stable/deep_dives/wandb_logging.html) metrics and checkpoints, and tracking training progress
- [ExecuTorch](https://pytorch.org/executorch-overview) for [on-device inference](https://github.com/pytorch/executorch/tree/main/examples/models/llama2#optional-finetuning) using fine-tuned models
- [bitsandbytes](https://huggingface.co/docs/bitsandbytes/main/en/index) for low memory optimizers for our [single-device recipes](recipes/configs/llama2/7B_full_low_memory.yaml)
### Models
torchtune currently supports the following models.
| Model | Sizes |
|-----------------------------------------------|-----------|
| [Llama2](https://llama.meta.com/llama2/) | 7B, 13B [[models](torchtune/models/llama2/_model_builders.py), [configs](recipes/configs/llama2/)] |
| [Mistral](https://huggingface.co/mistralai) | 7B [[model](torchtune/models/mistral/_model_builders.py), [configs](recipes/configs/mistral/)] |
| [Gemma](https://huggingface.co/collections/google/gemma-release-65d5efbccdbb8c4202ec078b) | 2B [[model](torchtune/models/gemma/_model_builders.py), [configs](recipes/configs/gemma/)] |
We'll be adding a number of new models in the coming weeks, including support for 70B versions and MoEs.
### Fine-tuning recipes
torchtune provides the following fine-tuning recipes.
| Training | Fine-tuning Method |
|------------------------------------|------------------------------------|
| Distributed Training [1 to 8 GPUs] | Full [[code](recipes/full_finetune_distributed.py), [example](recipes/configs/llama2/7B_full.yaml)], LoRA [[code](recipes/lora_finetune_distributed.py), [example](recipes/configs/llama2/7B_lora.yaml)] |
| Single Device / Low Memory [1 GPU] | Full [[code](recipes/full_finetune_single_device.py), [example](recipes/configs/llama2/7B_full_low_memory.yaml)], LoRA + QLoRA [[code](recipes/lora_finetune_distributed.py), [example](recipes/configs/llama2/7B_qlora_single_device.yaml)] |
| Single Device [1 GPU] | DPO [[code](recipes/full_finetune_distributed.py), [example](recipes/configs/llama2/7B_lora_dpo_single_device.yaml)]
Memory efficiency is important to us. All of our recipes are tested on a variety of setups including commodity GPUs with 24GB of VRAM as well as beefier options found in data centers.
Single-GPU recipes expose a number of memory optimizations that aren't available in the distributed versions. These include support for low-precision optimizers from [bitsandbytes](https://huggingface.co/docs/bitsandbytes/main/en/index) and fusing optimizer step with backward to reduce memory footprint from the gradients (see example [config](https://github.com/pytorch/torchtune/blob/main/recipes/configs/llama2/7B_full_low_memory.yaml)). For memory-constrained setups, we recommend using the single-device configs as a starting point. For example, our default QLoRA config has a peak memory usage of ``~9.3GB``. Similarly LoRA on single device with ``batch_size=2`` has a peak memory usage of ``~15.5GB``. Both of these are with ``dtype=bf16`` and ``AdamW`` as the optimizer.
---
## Installation
**Step 1:** [Install PyTorch](ttps://pytorch.org/get-started/locally/). torchtune is tested with the latest stable PyTorch release (2.2.2) as well as the preview nightly version.
**Step 2:** The latest stable version of torchtune is hosted on PyPI and can be downloaded with the following command:
```bash
pip install torchtune
```
To confirm that the package is installed correctly, you can run the following command:
```bash
tune --help
```
And should see the following output:
```bash
usage: tune [-h] {ls,cp,download,run,validate} ...
Welcome to the TorchTune CLI!
options:
-h, --help show this help message and exit
...
```
---
## Get Started
To get started with fine-tuning your first LLM with torchtune, see our tutorial on [fine-tuning Llama2 7B](https://pytorch.org/torchtune/stable/tutorials/first_finetune_tutorial.html). Our [end-to-end workflow](https://pytorch.org/torchtune/stable/tutorials/e2e_flow.html) tutorial will show you how to evaluate, quantize and run inference with this model. The rest of this section will provide a quick overview of these steps with Llama2.
### Downloading a model
Follow the instructions on the official [`meta-llama`](https://huggingface.co/meta-llama/Llama-2-7b) repository to ensure you have access to the Llama2 model weights. Once you have confirmed access, you can run the following command to download the weights to your local machine. This will also download the tokenizer model and a responsible use guide.
```bash
tune download meta-llama/Llama-2-7b-hf \
--output-dir /tmp/Llama-2-7b-hf \
--hf-token <HF_TOKEN> \
```
> Tip: Set your environment variable `HF_TOKEN` or pass in `--hf-token` to the command in order to validate your access.
You can find your token at https://huggingface.co/settings/tokens
### Running fine-tuning recipes
Llama2 7B + LoRA on single GPU:
```bash
tune run lora_finetune_single_device --config llama2/7B_lora_single_device
```
For distributed training, tune CLI integrates with [torchrun](https://pytorch.org/docs/stable/elastic/run.html).
Llama2 7B + LoRA on two GPUs:
```bash
tune run --nproc_per_node 2 full_finetune_distributed --config llama2/7B_full_distributed
```
> Tip: Make sure to place any torchrun commands **before** the recipe specification. Any CLI args after this will override the config and not impact distributed training.
### Modify Configs
There are two ways in which you can modify configs:
**Config Overrides**
You can easily overwrite config properties from the command-line:
```bash
tune run lora_finetune_single_device \
--config llama2/7B_lora_single_device \
batch_size=8 \
enable_activation_checkpointing=True \
max_steps_per_epoch=128
```
**Update a Local Copy**
You can also copy the config to your local directory and modify the contents directly:
```bash
tune cp llama2/7B_full ./my_custom_config.yaml
Copied to ./7B_full.yaml
```
Then, you can run your custom recipe by directing the `tune run` command to your local files:
```bash
tune run full_finetune_distributed --config ./my_custom_config.yaml
```
Check out `tune --help` for all possible CLI commands and options. For more information on using and updating configs, take a look at our [config deep-dive](https://pytorch.org/torchtune/stable/deep_dives/configs.html).
## Design Principles
torchtune embodies PyTorch’s design philosophy [[details](https://pytorch.org/docs/stable/community/design.html)], especially "usability over everything else".
### Native PyTorch
torchtune is a native-PyTorch library. While we provide integrations with the surrounding ecosystem (eg: Hugging Face Datasets, EleutherAI Eval Harness), all of the core functionality is written in PyTorch.
### Simplicity and Extensibility
torchtune is designed to be easy to understand, use and extend.
- Composition over implementation inheritance - layers of inheritance for code re-use makes the code hard to read and extend
- No training frameworks - explicitly outlining the training logic makes it easy to extend for custom use cases
- Code duplication is preferred over unnecessary abstractions
- Modular building blocks over monolithic components
### Correctness
torchtune provides well-tested components with a high-bar on correctness. The library will never be the first to provide a feature, but available features will be thoroughly tested. We provide
- Extensive unit-tests to ensure component-level numerical parity with reference implementations
- Checkpoint-tests to ensure model-level numerical parity with reference implementations
- Integration tests to ensure recipe-level performance parity with reference implementations on standard benchmarks
## Community Contributions
We really value our community and the contributions made by our wonderful users. We'll use this section to call out some of these contributions! If you'd like to help out as well, please see the [CONTRIBUTING](CONTRIBUTING.md) guide.
- [@solitude-alive](https://github.com/solitude-alive) for adding the [Gemma 2B model](torchtune/models/gemma/) to torchtune, including recipe changes, numeric validations of the models and recipe correctness
- [@yechenzhi](https://github.com/yechenzhi) for adding [DPO](recipes/lora_dpo_single_device.py) to torchtune, including the recipe and config along with correctness checks
## Acknowledgements
The Llama2 code in this repository is inspired by the original [Llama2 code](https://github.com/meta-llama/llama/blob/main/llama/model.py).
We want to give a huge shout-out to EleutherAI, Hugging Face and Weights & Biases for being wonderful collaborators and for working with us on some of these integrations within torchtune.
We also want to acknowledge some awesome libraries and tools from the ecosystem:
- [gpt-fast](https://github.com/pytorch-labs/gpt-fast) for performant LLM inference techniques which we've adopted OOTB
- [llama recipes](https://github.com/meta-llama/llama-recipes) for spring-boarding the llama2 community
- [bitsandbytes](https://github.com/TimDettmers/bitsandbytes) for bringing several memory and performance based techniques to the PyTorch ecosystem
- [@winglian](https://github.com/winglian/) and [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) for early feedback and brainstorming on torchtune's design and feature set.
- [lit-gpt](https://github.com/Lightning-AI/litgpt) for pushing the LLM fine-tuning community forward.
## License
torchtune is released under the [BSD 3 license](./LICENSE). However you may have other legal obligations that govern your use of other content, such as the terms of service for third-party models.
Raw data
{
"_id": null,
"home_page": null,
"name": "torchtune",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "pytorch, finetuning, llm",
"author": null,
"author_email": "PyTorch Team <packages@pytorch.org>",
"download_url": null,
"platform": null,
"description": "\n[](https://github.com/pytorch/torchtune/actions/workflows/unit_test.yaml)\n\n[](https://discord.gg/4Xsdn8Rr9Q)\n\n\n# torchtune\n\n[**Introduction**](#introduction) | [**Installation**](#installation) | [**Get Started**](#get-started) | [**Documentation**](https://pytorch.org/torchtune) | [**Design Principles**](#design-principles) | [**Community Contributions**](#community-contributions) | [**License**](#license)\n\n \n\n## Introduction\n\ntorchtune is a PyTorch-native library for easily authoring, fine-tuning and experimenting with LLMs. We're excited to announce our alpha release!\n\ntorchtune provides:\n\n- Native-PyTorch implementations of popular LLMs using composable and modular building blocks\n- Easy-to-use and hackable training recipes for popular fine-tuning techniques (LoRA, QLoRA) - no trainers, no frameworks, just PyTorch!\n- YAML configs for easily configuring training, evaluation, quantization or inference recipes\n- Built-in support for many popular dataset formats and prompt templates to help you quickly get started with training\n\ntorchtune focuses on integrating with popular tools and libraries from the ecosystem. These are just a few examples, with more under development:\n\n- [Hugging Face Hub](https://huggingface.co/docs/hub/en/index) for [accessing model weights](torchtune/_cli/download.py)\n- [EleutherAI's LM Eval Harness](https://github.com/EleutherAI/lm-evaluation-harness) for [evaluating](recipes/eleuther_eval.py) trained models\n- [Hugging Face Datasets](https://huggingface.co/docs/datasets/en/index) for [access](torchtune/datasets/_instruct.py) to training and evaluation datasets\n- [PyTorch FSDP](https://pytorch.org/docs/stable/fsdp.html) for distributed training\n- [torchao](https://github.com/pytorch-labs/ao) for lower precision dtypes and [post-training quantization](recipes/quantize.py) techniques\n- [Weights & Biases](https://wandb.ai/site) for [logging](https://pytorch.org/torchtune/stable/deep_dives/wandb_logging.html) metrics and checkpoints, and tracking training progress\n- [ExecuTorch](https://pytorch.org/executorch-overview) for [on-device inference](https://github.com/pytorch/executorch/tree/main/examples/models/llama2#optional-finetuning) using fine-tuned models\n- [bitsandbytes](https://huggingface.co/docs/bitsandbytes/main/en/index) for low memory optimizers for our [single-device recipes](recipes/configs/llama2/7B_full_low_memory.yaml)\n\n \n\n### Models\n\ntorchtune currently supports the following models.\n\n| Model | Sizes |\n|-----------------------------------------------|-----------|\n| [Llama2](https://llama.meta.com/llama2/) | 7B, 13B [[models](torchtune/models/llama2/_model_builders.py), [configs](recipes/configs/llama2/)] |\n| [Mistral](https://huggingface.co/mistralai) | 7B [[model](torchtune/models/mistral/_model_builders.py), [configs](recipes/configs/mistral/)] |\n| [Gemma](https://huggingface.co/collections/google/gemma-release-65d5efbccdbb8c4202ec078b) | 2B [[model](torchtune/models/gemma/_model_builders.py), [configs](recipes/configs/gemma/)] |\n\nWe'll be adding a number of new models in the coming weeks, including support for 70B versions and MoEs.\n\n \n\n### Fine-tuning recipes\n\ntorchtune provides the following fine-tuning recipes.\n\n| Training | Fine-tuning Method |\n|------------------------------------|------------------------------------|\n| Distributed Training [1 to 8 GPUs] | Full [[code](recipes/full_finetune_distributed.py), [example](recipes/configs/llama2/7B_full.yaml)], LoRA [[code](recipes/lora_finetune_distributed.py), [example](recipes/configs/llama2/7B_lora.yaml)] |\n| Single Device / Low Memory [1 GPU] | Full [[code](recipes/full_finetune_single_device.py), [example](recipes/configs/llama2/7B_full_low_memory.yaml)], LoRA + QLoRA [[code](recipes/lora_finetune_distributed.py), [example](recipes/configs/llama2/7B_qlora_single_device.yaml)] |\n| Single Device [1 GPU] | DPO [[code](recipes/full_finetune_distributed.py), [example](recipes/configs/llama2/7B_lora_dpo_single_device.yaml)]\n\n \n\n\nMemory efficiency is important to us. All of our recipes are tested on a variety of setups including commodity GPUs with 24GB of VRAM as well as beefier options found in data centers.\n\nSingle-GPU recipes expose a number of memory optimizations that aren't available in the distributed versions. These include support for low-precision optimizers from [bitsandbytes](https://huggingface.co/docs/bitsandbytes/main/en/index) and fusing optimizer step with backward to reduce memory footprint from the gradients (see example [config](https://github.com/pytorch/torchtune/blob/main/recipes/configs/llama2/7B_full_low_memory.yaml)). For memory-constrained setups, we recommend using the single-device configs as a starting point. For example, our default QLoRA config has a peak memory usage of ``~9.3GB``. Similarly LoRA on single device with ``batch_size=2`` has a peak memory usage of ``~15.5GB``. Both of these are with ``dtype=bf16`` and ``AdamW`` as the optimizer.\n\n \n\n---\n\n## Installation\n\n**Step 1:** [Install PyTorch](ttps://pytorch.org/get-started/locally/). torchtune is tested with the latest stable PyTorch release (2.2.2) as well as the preview nightly version.\n\n**Step 2:** The latest stable version of torchtune is hosted on PyPI and can be downloaded with the following command:\n\n```bash\npip install torchtune\n```\n\nTo confirm that the package is installed correctly, you can run the following command:\n\n```bash\ntune --help\n```\n\nAnd should see the following output:\n\n```bash\nusage: tune [-h] {ls,cp,download,run,validate} ...\n\nWelcome to the TorchTune CLI!\n\noptions:\n -h, --help show this help message and exit\n\n...\n```\n\n \n\n---\n\n## Get Started\n\nTo get started with fine-tuning your first LLM with torchtune, see our tutorial on [fine-tuning Llama2 7B](https://pytorch.org/torchtune/stable/tutorials/first_finetune_tutorial.html). Our [end-to-end workflow](https://pytorch.org/torchtune/stable/tutorials/e2e_flow.html) tutorial will show you how to evaluate, quantize and run inference with this model. The rest of this section will provide a quick overview of these steps with Llama2.\n\n \n\n### Downloading a model\n\nFollow the instructions on the official [`meta-llama`](https://huggingface.co/meta-llama/Llama-2-7b) repository to ensure you have access to the Llama2 model weights. Once you have confirmed access, you can run the following command to download the weights to your local machine. This will also download the tokenizer model and a responsible use guide.\n\n```bash\ntune download meta-llama/Llama-2-7b-hf \\\n--output-dir /tmp/Llama-2-7b-hf \\\n--hf-token <HF_TOKEN> \\\n```\n\n> Tip: Set your environment variable `HF_TOKEN` or pass in `--hf-token` to the command in order to validate your access.\nYou can find your token at https://huggingface.co/settings/tokens\n\n \n\n### Running fine-tuning recipes\n\nLlama2 7B + LoRA on single GPU:\n\n```bash\ntune run lora_finetune_single_device --config llama2/7B_lora_single_device\n```\n\nFor distributed training, tune CLI integrates with [torchrun](https://pytorch.org/docs/stable/elastic/run.html).\nLlama2 7B + LoRA on two GPUs:\n\n```bash\ntune run --nproc_per_node 2 full_finetune_distributed --config llama2/7B_full_distributed\n```\n\n> Tip: Make sure to place any torchrun commands **before** the recipe specification. Any CLI args after this will override the config and not impact distributed training.\n\n \n\n### Modify Configs\n\nThere are two ways in which you can modify configs:\n\n**Config Overrides**\n\nYou can easily overwrite config properties from the command-line:\n\n```bash\ntune run lora_finetune_single_device \\\n--config llama2/7B_lora_single_device \\\nbatch_size=8 \\\nenable_activation_checkpointing=True \\\nmax_steps_per_epoch=128\n```\n\n**Update a Local Copy**\n\nYou can also copy the config to your local directory and modify the contents directly:\n\n```bash\ntune cp llama2/7B_full ./my_custom_config.yaml\nCopied to ./7B_full.yaml\n```\n\nThen, you can run your custom recipe by directing the `tune run` command to your local files:\n\n```bash\ntune run full_finetune_distributed --config ./my_custom_config.yaml\n```\n\n \n\nCheck out `tune --help` for all possible CLI commands and options. For more information on using and updating configs, take a look at our [config deep-dive](https://pytorch.org/torchtune/stable/deep_dives/configs.html).\n\n \n\n## Design Principles\n\ntorchtune embodies PyTorch\u2019s design philosophy [[details](https://pytorch.org/docs/stable/community/design.html)], especially \"usability over everything else\".\n\n### Native PyTorch\n\ntorchtune is a native-PyTorch library. While we provide integrations with the surrounding ecosystem (eg: Hugging Face Datasets, EleutherAI Eval Harness), all of the core functionality is written in PyTorch.\n\n### Simplicity and Extensibility\n\ntorchtune is designed to be easy to understand, use and extend.\n\n- Composition over implementation inheritance - layers of inheritance for code re-use makes the code hard to read and extend\n- No training frameworks - explicitly outlining the training logic makes it easy to extend for custom use cases\n- Code duplication is preferred over unnecessary abstractions\n- Modular building blocks over monolithic components\n\n### Correctness\n\ntorchtune provides well-tested components with a high-bar on correctness. The library will never be the first to provide a feature, but available features will be thoroughly tested. We provide\n\n- Extensive unit-tests to ensure component-level numerical parity with reference implementations\n- Checkpoint-tests to ensure model-level numerical parity with reference implementations\n- Integration tests to ensure recipe-level performance parity with reference implementations on standard benchmarks\n\n \n\n## Community Contributions\n\nWe really value our community and the contributions made by our wonderful users. We'll use this section to call out some of these contributions! If you'd like to help out as well, please see the [CONTRIBUTING](CONTRIBUTING.md) guide.\n\n- [@solitude-alive](https://github.com/solitude-alive) for adding the [Gemma 2B model](torchtune/models/gemma/) to torchtune, including recipe changes, numeric validations of the models and recipe correctness\n- [@yechenzhi](https://github.com/yechenzhi) for adding [DPO](recipes/lora_dpo_single_device.py) to torchtune, including the recipe and config along with correctness checks\n\n\n \n\n## Acknowledgements\n\nThe Llama2 code in this repository is inspired by the original [Llama2 code](https://github.com/meta-llama/llama/blob/main/llama/model.py).\n\nWe want to give a huge shout-out to EleutherAI, Hugging Face and Weights & Biases for being wonderful collaborators and for working with us on some of these integrations within torchtune.\n\nWe also want to acknowledge some awesome libraries and tools from the ecosystem:\n- [gpt-fast](https://github.com/pytorch-labs/gpt-fast) for performant LLM inference techniques which we've adopted OOTB\n- [llama recipes](https://github.com/meta-llama/llama-recipes) for spring-boarding the llama2 community\n- [bitsandbytes](https://github.com/TimDettmers/bitsandbytes) for bringing several memory and performance based techniques to the PyTorch ecosystem\n- [@winglian](https://github.com/winglian/) and [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) for early feedback and brainstorming on torchtune's design and feature set.\n- [lit-gpt](https://github.com/Lightning-AI/litgpt) for pushing the LLM fine-tuning community forward.\n\n \n\n\n## License\n\ntorchtune is released under the [BSD 3 license](./LICENSE). However you may have other legal obligations that govern your use of other content, such as the terms of service for third-party models.\n",
"bugtrack_url": null,
"license": "BSD 3-Clause License Copyright 2024 Meta Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice,this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \u201cAS IS\u201d AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ",
"summary": "A native-PyTorch library for LLM fine-tuning",
"version": "0.1.0",
"project_urls": {
"Documentation": "https://pytorch.org/torchtune/main/index.html",
"GitHub": "https://github.com/pytorch/torchtune",
"Issues": "https://github.com/pytorch/torchtune/issues"
},
"split_keywords": [
"pytorch",
" finetuning",
" llm"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "19b7a5746fdda221c48881eea3c51699e5b21cd31a457422a3c66b6695a3d29e",
"md5": "c74e871e31c04af49a416a952e0c7393",
"sha256": "e96468668921f327682f173be8995a75d2ada49c06ad4957ebdf3425c604f8ad"
},
"downloads": -1,
"filename": "torchtune-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c74e871e31c04af49a416a952e0c7393",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 192092,
"upload_time": "2024-04-16T03:24:15",
"upload_time_iso_8601": "2024-04-16T03:24:15.939795Z",
"url": "https://files.pythonhosted.org/packages/19/b7/a5746fdda221c48881eea3c51699e5b21cd31a457422a3c66b6695a3d29e/torchtune-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-16 03:24:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "pytorch",
"github_project": "torchtune",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "torchtune"
}
