# 🤗 AutoTrain Advanced
AutoTrain Advanced: faster and easier training and deployments of state-of-the-art machine learning models. AutoTrain Advanced is a no-code solution that allows you to train machine learning models in just a few clicks. Please note that you must upload data in correct format for project to be created. For help regarding proper data format and pricing, check out the documentation.
NOTE: AutoTrain is free! You only pay for the resources you use in case you decide to run AutoTrain on Hugging Face Spaces. When running locally, you only pay for the resources you use on your own infrastructure.
## Supported Tasks
| Task | Status | Python Notebook | Example Configs |
| --- | --- | --- | --- |
| LLM SFT Finetuning | ✅ | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/llm_finetuning.ipynb) | [llm_sft_finetune.yaml](https://github.com/huggingface/autotrain-advanced/blob/main/configs/llm_finetuning/smollm2.yml) |
| LLM ORPO Finetuning | ✅ | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/llm_finetuning.ipynb) | [llm_orpo_finetune.yaml](https://github.com/huggingface/autotrain-advanced/blob/main/configs/llm_finetuning/llama3-8b-orpo.yml) |
| LLM DPO Finetuning | ✅ | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/llm_finetuning.ipynb) | [llm_dpo_finetune.yaml](https://github.com/huggingface/autotrain-advanced/blob/main/configs/llm_finetuning/llama3-8b-dpo-qlora.yml) |
| LLM Reward Finetuning | ✅ | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/llm_finetuning.ipynb) | [llm_reward_finetune.yaml](https://github.com/huggingface/autotrain-advanced/blob/main/configs/llm_finetuning/llama32-1b-sft.yml) |
| LLM Generic/Default Finetuning | ✅ | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/llm_finetuning.ipynb) | [llm_generic_finetune.yaml](https://github.com/huggingface/autotrain-advanced/blob/main/configs/llm_finetuning/gpt2_sft.yml) |
| Text Classification | ✅ | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/text_classification.ipynb) | [text_classification.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/text_classification) |
| Text Regression | ✅ | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/text_regression.ipynb) | [text_regression.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/text_regression) |
| Token Classification | ✅ | Coming Soon | [token_classification.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/token_classification) |
| Seq2Seq | ✅ | Coming Soon | [seq2seq.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/seq2seq) |
| Extractive Question Answering | ✅ | Coming Soon | [extractive_qa.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/extractive_question_answering) |
| Image Classification | ✅ | Coming Soon | [image_classification.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/image_classification) |
| Image Scoring/Regression | ✅ | Coming Soon | [image_regression.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/image_scoring) |
| DreamBooth LoRA | ✅ | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/colabs/AutoTrain_Dreambooth.ipynb) | [dreambooth_lora.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/dreambooth) |
| VLM | 🟥 | Coming Soon | [vlm.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/vlm) |
## Running UI on Colab or Hugging Face Spaces
- Deploy AutoTrain on Hugging Face Spaces: [](https://huggingface.co/login?next=%2Fspaces%2Fautotrain-projects%2Fautotrain-advanced%3Fduplicate%3Dtrue)
- Run AutoTrain UI on Colab via ngrok: [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/colabs/AutoTrain_ngrok.ipynb)
## Local Installation
You can Install AutoTrain-Advanced python package via PIP. Please note you will need python >= 3.10 for AutoTrain Advanced to work properly.
pip install autotrain-advanced
Please make sure that you have git lfs installed. Check out the instructions here: https://github.com/git-lfs/git-lfs/wiki/Installation
You also need to install torch, torchaudio and torchvision.
The best way to run autotrain is in a conda environment. You can create a new conda environment with the following command:
conda create -n autotrain python=3.10
conda activate autotrain
pip install autotrain-advanced
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia
conda install -c "nvidia/label/cuda-12.1.0" cuda-nvcc
Once done, you can start the application using:
autotrain app --port 8080 --host 127.0.0.1
If you are not fond of UI, you can use AutoTrain Configs to train using command line or simply AutoTrain CLI.
To use config file for training, you can use the following command:
autotrain --config <path_to_config_file>
You can find sample config files in the `configs` directory of this repository.
Example config file for finetuning SmolLM2:
```yaml
task: llm-sft
base_model: HuggingFaceTB/SmolLM2-1.7B-Instruct
project_name: autotrain-smollm2-finetune
log: tensorboard
backend: local
data:
path: HuggingFaceH4/no_robots
train_split: train
valid_split: null
chat_template: tokenizer
column_mapping:
text_column: messages
params:
block_size: 2048
model_max_length: 4096
epochs: 2
batch_size: 1
lr: 1e-5
peft: true
quantization: int4
target_modules: all-linear
padding: right
optimizer: paged_adamw_8bit
scheduler: linear
gradient_accumulation: 8
mixed_precision: bf16
merge_adapter: true
hub:
username: ${HF_USERNAME}
token: ${HF_TOKEN}
push_to_hub: true
```
To fine-tune a model using the config file above, you can use the following command:
```bash
$ export HF_USERNAME=<your_hugging_face_username>
$ export HF_TOKEN=<your_hugging_face_write_token>
$ autotrain --config <path_to_config_file>
```
## Documentation
Documentation is available at https://hf.co/docs/autotrain/
## Citation
```
@misc{thakur2024autotrainnocodetrainingstateoftheart,
title={AutoTrain: No-code training for state-of-the-art models},
author={Abhishek Thakur},
year={2024},
eprint={2410.15735},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2410.15735},
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/huggingface/autotrain-advanced",
"name": "autotrain-advanced",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "automl autonlp autotrain huggingface",
"author": "HuggingFace Inc.",
"author_email": "autotrain@huggingface.co",
"download_url": "https://files.pythonhosted.org/packages/52/c2/cf57b09b97d89f66eca7a0ccdf2051bfdd78ef1aab51498533689ae3349d/autotrain_advanced-0.8.31.tar.gz",
"platform": null,
"description": "# \ud83e\udd17 AutoTrain Advanced\n\nAutoTrain Advanced: faster and easier training and deployments of state-of-the-art machine learning models. AutoTrain Advanced is a no-code solution that allows you to train machine learning models in just a few clicks. Please note that you must upload data in correct format for project to be created. For help regarding proper data format and pricing, check out the documentation. \n\nNOTE: AutoTrain is free! You only pay for the resources you use in case you decide to run AutoTrain on Hugging Face Spaces. When running locally, you only pay for the resources you use on your own infrastructure.\n\n## Supported Tasks\n\n| Task | Status | Python Notebook | Example Configs |\n| --- | --- | --- | --- |\n| LLM SFT Finetuning | \u2705 | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/llm_finetuning.ipynb) | [llm_sft_finetune.yaml](https://github.com/huggingface/autotrain-advanced/blob/main/configs/llm_finetuning/smollm2.yml) |\n| LLM ORPO Finetuning | \u2705 | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/llm_finetuning.ipynb) | [llm_orpo_finetune.yaml](https://github.com/huggingface/autotrain-advanced/blob/main/configs/llm_finetuning/llama3-8b-orpo.yml) |\n| LLM DPO Finetuning | \u2705 | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/llm_finetuning.ipynb) | [llm_dpo_finetune.yaml](https://github.com/huggingface/autotrain-advanced/blob/main/configs/llm_finetuning/llama3-8b-dpo-qlora.yml) |\n| LLM Reward Finetuning | \u2705 | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/llm_finetuning.ipynb) | [llm_reward_finetune.yaml](https://github.com/huggingface/autotrain-advanced/blob/main/configs/llm_finetuning/llama32-1b-sft.yml) |\n| LLM Generic/Default Finetuning | \u2705 | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/llm_finetuning.ipynb) | [llm_generic_finetune.yaml](https://github.com/huggingface/autotrain-advanced/blob/main/configs/llm_finetuning/gpt2_sft.yml) |\n| Text Classification | \u2705 | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/text_classification.ipynb) | [text_classification.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/text_classification) |\n| Text Regression | \u2705 | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/notebooks/text_regression.ipynb) | [text_regression.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/text_regression) |\n| Token Classification | \u2705 | Coming Soon | [token_classification.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/token_classification) |\n| Seq2Seq | \u2705 | Coming Soon | [seq2seq.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/seq2seq) |\n| Extractive Question Answering | \u2705 | Coming Soon | [extractive_qa.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/extractive_question_answering) |\n| Image Classification | \u2705 | Coming Soon | [image_classification.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/image_classification) |\n| Image Scoring/Regression | \u2705 | Coming Soon | [image_regression.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/image_scoring) |\n| DreamBooth LoRA | \u2705 | [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/colabs/AutoTrain_Dreambooth.ipynb) | [dreambooth_lora.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/dreambooth) |\n| VLM | \ud83d\udfe5 | Coming Soon | [vlm.yaml](https://github.com/huggingface/autotrain-advanced/tree/main/configs/vlm) |\n\n\n## Running UI on Colab or Hugging Face Spaces\n\n- Deploy AutoTrain on Hugging Face Spaces: [](https://huggingface.co/login?next=%2Fspaces%2Fautotrain-projects%2Fautotrain-advanced%3Fduplicate%3Dtrue)\n\n\n- Run AutoTrain UI on Colab via ngrok: [](https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/colabs/AutoTrain_ngrok.ipynb)\n\n\n## Local Installation\n\nYou can Install AutoTrain-Advanced python package via PIP. Please note you will need python >= 3.10 for AutoTrain Advanced to work properly.\n\n pip install autotrain-advanced\n \nPlease make sure that you have git lfs installed. Check out the instructions here: https://github.com/git-lfs/git-lfs/wiki/Installation\n\nYou also need to install torch, torchaudio and torchvision.\n\nThe best way to run autotrain is in a conda environment. You can create a new conda environment with the following command:\n\n conda create -n autotrain python=3.10\n conda activate autotrain\n pip install autotrain-advanced\n conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia\n conda install -c \"nvidia/label/cuda-12.1.0\" cuda-nvcc\n\nOnce done, you can start the application using:\n\n autotrain app --port 8080 --host 127.0.0.1\n\n\nIf you are not fond of UI, you can use AutoTrain Configs to train using command line or simply AutoTrain CLI.\n\nTo use config file for training, you can use the following command:\n\n autotrain --config <path_to_config_file>\n\n\nYou can find sample config files in the `configs` directory of this repository.\n\nExample config file for finetuning SmolLM2:\n\n```yaml\ntask: llm-sft\nbase_model: HuggingFaceTB/SmolLM2-1.7B-Instruct\nproject_name: autotrain-smollm2-finetune\nlog: tensorboard\nbackend: local\n\ndata:\n path: HuggingFaceH4/no_robots\n train_split: train\n valid_split: null\n chat_template: tokenizer\n column_mapping:\n text_column: messages\n\nparams:\n block_size: 2048\n model_max_length: 4096\n epochs: 2\n batch_size: 1\n lr: 1e-5\n peft: true\n quantization: int4\n target_modules: all-linear\n padding: right\n optimizer: paged_adamw_8bit\n scheduler: linear\n gradient_accumulation: 8\n mixed_precision: bf16\n merge_adapter: true\n\nhub:\n username: ${HF_USERNAME}\n token: ${HF_TOKEN}\n push_to_hub: true\n```\n\nTo fine-tune a model using the config file above, you can use the following command:\n\n```bash\n$ export HF_USERNAME=<your_hugging_face_username>\n$ export HF_TOKEN=<your_hugging_face_write_token>\n$ autotrain --config <path_to_config_file>\n```\n\n\n## Documentation\n\nDocumentation is available at https://hf.co/docs/autotrain/\n\n## Citation\n\n```\n@misc{thakur2024autotrainnocodetrainingstateoftheart,\n title={AutoTrain: No-code training for state-of-the-art models}, \n author={Abhishek Thakur},\n year={2024},\n eprint={2410.15735},\n archivePrefix={arXiv},\n primaryClass={cs.AI},\n url={https://arxiv.org/abs/2410.15735}, \n}\n```\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": null,
"version": "0.8.31",
"project_urls": {
"Download": "https://github.com/huggingface/autotrain-advanced/tags",
"Homepage": "https://github.com/huggingface/autotrain-advanced"
},
"split_keywords": [
"automl",
"autonlp",
"autotrain",
"huggingface"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a223b7e5663dcac75693450c7bb2263037c24eefba5897a3905a4d1ac244d4a6",
"md5": "2bf8e1b9fb7525fe390ea9b59f15531c",
"sha256": "d26b2299bc5c6da88abde1c73af5ad1cd0999c5c840b9420bfce35ca6ba87a77"
},
"downloads": -1,
"filename": "autotrain_advanced-0.8.31-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2bf8e1b9fb7525fe390ea9b59f15531c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 382787,
"upload_time": "2024-11-11T14:54:42",
"upload_time_iso_8601": "2024-11-11T14:54:42.927906Z",
"url": "https://files.pythonhosted.org/packages/a2/23/b7e5663dcac75693450c7bb2263037c24eefba5897a3905a4d1ac244d4a6/autotrain_advanced-0.8.31-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "52c2cf57b09b97d89f66eca7a0ccdf2051bfdd78ef1aab51498533689ae3349d",
"md5": "5205631e0437a7d2b9d04522d5267f75",
"sha256": "aefcf42ec22c9e81a066f715f519aad0085988873194692d78d757ef1252271d"
},
"downloads": -1,
"filename": "autotrain_advanced-0.8.31.tar.gz",
"has_sig": false,
"md5_digest": "5205631e0437a7d2b9d04522d5267f75",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 246650,
"upload_time": "2024-11-11T14:54:45",
"upload_time_iso_8601": "2024-11-11T14:54:45.365104Z",
"url": "https://files.pythonhosted.org/packages/52/c2/cf57b09b97d89f66eca7a0ccdf2051bfdd78ef1aab51498533689ae3349d/autotrain_advanced-0.8.31.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-11 14:54:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "huggingface",
"github_project": "autotrain-advanced",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "autotrain-advanced"
}
JSON
