# huggingface_tool
[](https://pypi.org/project/huggingface-tool/)
[](https://hitsofcode.com/github/OpenRL-Lab/huggingface_tool/view?branch=main)
Tools for loading, upload, managing huggingface models and datasets
## Installation
`pip install huggingface-tool`
## Usage
Firstly, you need to login with `huggingface-cli login` (you can create or find your token at [settings](https://huggingface.co/settings/tokens)).
- Download and save a repo with: `htool save-repo <repo_id> <save_dir> -r <model/dataset>`. `-r` means the repo is a model or dataset repo. By default, it is a model repo.
- For example: `htool save-repo OpenRL/tizero ./tizero`
- For example: `htool save-repo OpenRL/DeepFakeFace ./DeepFakeFace -r dataset`
- Download and save a file with: `htool save-file <repo_id>:<remote_filepath> <save_dir> -r <model/dataset>`. `-r` means the repo is a model or dataset repo. By default, it is a model repo.
- For example: `htool save-file OpenRL/tizero:actor.pt ./tizero`
- For example: `htool save-file OpenRL/DeepFakeFace:README.md ./DeepFakeFace -r dataset`
- Download and save transformer models with: `htool save-model <model_class> <model_name> <save_dir>`
- For example: `htool save-model AutoModelForCausalLM gpt2 ./gpt2`
- Download and save tokenizer with: `htool save-tk <tokenizer_name> <save_dir>`
- For example: `htool save-tk gpt2 ./gpt2 `
- Download and save dataset with: `htool save-data <dataset_name> <save_dir>`
- For example: `htool save-data daily_dialog ./daily_dialog`
- Download and save diffusion models with: `htool save-dm <model_name> <save_dir>`
- For example: `htool save-dm google/ddpm-cat-256 ./google/`
You can also use htool to upload datasets and models to huggingface.
- Upload a file with: `htool upload-file <local_filepath> <organization_or_username/repo_name>:<remote_filepath> -r <model/dataset>`. `-r` means the repo is a model or dataset repo. By default, it is a model repo.
- For example: `htool upload-file README.md OpenRL/tizero:README.md`
- For example: `htool upload-file README.md OpenRL/DeepFakeFace:README.md -r dataset`
- Upload a directory with: `htool upload-dir <local_dirpath> <organization_or_username/repo_name>:<remote_dirpath> -r <model/dataset>`. `-r` means the repo is a model or dataset repo. By default, it is a model repo.
- For example: `htool upload-dir ./tizero OpenRL/tizero:./`
- For example: `htool upload-dir ./tizero/models OpenRL/tizero:./models`
- For example: `htool upload-dir ./DeepFakeFace OpenRL/DeepFakeFace:./ -r dataset`
- For example: `htool upload-dir ./DeepFakeFace/images OpenRL/DeepFakeFace:./images -r dataset`
- Upload dataset with: `htool upload-data <local_dataset_dir> <organization_or_username/dataset_name>`
- For example: `htool upload-data ./daily_dialog OpenRL/daily_dialog`
- Upload model with: `htool upload-model <local_model_dir> <organization_or_username/model_name>`
- For example: `htool upload-model ./tizero OpenRL/tizero`
## Citing huggingface_tool
If our work has been helpful to you, please feel free to cite us:
```latex
@misc{huggingface_tool2023,
title={huggingface_tool},
author={OpenRL Contributors},
publisher = {GitHub},
howpublished = {\url{https://github.com/OpenRL-Lab/huggingface_tool}},
year={2023},
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/OpenRL-Lab/huggingface_tool",
"name": "huggingface-tool",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "huggingface toolkit",
"author": "openrl contributors",
"author_email": "huangsy1314@163.com",
"download_url": "https://files.pythonhosted.org/packages/e7/73/126a11ca77c51f9d811c3f767afe277c1b8885f03f0c8e75fc485a790cf2/huggingface_tool-0.0.6.tar.gz",
"platform": null,
"description": "# huggingface_tool\n\n[](https://pypi.org/project/huggingface-tool/)\n\n[](https://hitsofcode.com/github/OpenRL-Lab/huggingface_tool/view?branch=main)\n\nTools for loading, upload, managing huggingface models and datasets\n\n\n## Installation\n\n`pip install huggingface-tool`\n\n## Usage\n\nFirstly, you need to login with `huggingface-cli login` (you can create or find your token at [settings](https://huggingface.co/settings/tokens)).\n\n- Download and save a repo with: `htool save-repo <repo_id> <save_dir> -r <model/dataset>`. `-r` means the repo is a model or dataset repo. By default, it is a model repo.\n - For example: `htool save-repo OpenRL/tizero ./tizero`\n - For example: `htool save-repo OpenRL/DeepFakeFace ./DeepFakeFace -r dataset`\n- Download and save a file with: `htool save-file <repo_id>:<remote_filepath> <save_dir> -r <model/dataset>`. `-r` means the repo is a model or dataset repo. By default, it is a model repo.\n - For example: `htool save-file OpenRL/tizero:actor.pt ./tizero`\n - For example: `htool save-file OpenRL/DeepFakeFace:README.md ./DeepFakeFace -r dataset`\n- Download and save transformer models with: `htool save-model <model_class> <model_name> <save_dir>`\n - For example: `htool save-model AutoModelForCausalLM gpt2 ./gpt2`\n- Download and save tokenizer with: `htool save-tk <tokenizer_name> <save_dir>`\n - For example: `htool save-tk gpt2 ./gpt2 `\n- Download and save dataset with: `htool save-data <dataset_name> <save_dir>`\n - For example: `htool save-data daily_dialog ./daily_dialog`\n- Download and save diffusion models with: `htool save-dm <model_name> <save_dir>`\n - For example: `htool save-dm google/ddpm-cat-256 ./google/`\n\nYou can also use htool to upload datasets and models to huggingface.\n\n- Upload a file with: `htool upload-file <local_filepath> <organization_or_username/repo_name>:<remote_filepath> -r <model/dataset>`. `-r` means the repo is a model or dataset repo. By default, it is a model repo.\n - For example: `htool upload-file README.md OpenRL/tizero:README.md`\n - For example: `htool upload-file README.md OpenRL/DeepFakeFace:README.md -r dataset`\n- Upload a directory with: `htool upload-dir <local_dirpath> <organization_or_username/repo_name>:<remote_dirpath> -r <model/dataset>`. `-r` means the repo is a model or dataset repo. By default, it is a model repo.\n - For example: `htool upload-dir ./tizero OpenRL/tizero:./`\n - For example: `htool upload-dir ./tizero/models OpenRL/tizero:./models`\n - For example: `htool upload-dir ./DeepFakeFace OpenRL/DeepFakeFace:./ -r dataset`\n - For example: `htool upload-dir ./DeepFakeFace/images OpenRL/DeepFakeFace:./images -r dataset`\n- Upload dataset with: `htool upload-data <local_dataset_dir> <organization_or_username/dataset_name>`\n - For example: `htool upload-data ./daily_dialog OpenRL/daily_dialog`\n- Upload model with: `htool upload-model <local_model_dir> <organization_or_username/model_name>`\n - For example: `htool upload-model ./tizero OpenRL/tizero`\n\n\n## Citing huggingface_tool\n\nIf our work has been helpful to you, please feel free to cite us:\n```latex\n@misc{huggingface_tool2023,\n title={huggingface_tool},\n author={OpenRL Contributors},\n publisher = {GitHub},\n howpublished = {\\url{https://github.com/OpenRL-Lab/huggingface_tool}},\n year={2023},\n}\n```\n",
"bugtrack_url": null,
"license": "",
"summary": "Toolkit for managing huggingface models and datasets",
"version": "0.0.6",
"project_urls": {
"Code": "https://github.com/OpenRL-Lab/huggingface_tool",
"Documentation": "https://huggingface_tool.readthedocs.io/",
"Homepage": "https://github.com/OpenRL-Lab/huggingface_tool"
},
"split_keywords": [
"huggingface",
"toolkit"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a4758883b242d25dca7dbd62539fb34df915f4694518ce3f12b2f1822938fc87",
"md5": "fe5bce08ef56a71189c05c1591f42f71",
"sha256": "b55ed1c54ec68378e007d673d1956a815ce1a963e56102dd17f1996b87d1324a"
},
"downloads": -1,
"filename": "huggingface_tool-0.0.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fe5bce08ef56a71189c05c1591f42f71",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 27627,
"upload_time": "2023-08-11T06:41:58",
"upload_time_iso_8601": "2023-08-11T06:41:58.052212Z",
"url": "https://files.pythonhosted.org/packages/a4/75/8883b242d25dca7dbd62539fb34df915f4694518ce3f12b2f1822938fc87/huggingface_tool-0.0.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e773126a11ca77c51f9d811c3f767afe277c1b8885f03f0c8e75fc485a790cf2",
"md5": "9460d6414e5e876b5d3420c79e14a392",
"sha256": "fe61ef61e52c8310fe3e993856f177ee2a7ff67ab7d93672b131bab3bbea8d89"
},
"downloads": -1,
"filename": "huggingface_tool-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "9460d6414e5e876b5d3420c79e14a392",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 13849,
"upload_time": "2023-08-11T06:41:59",
"upload_time_iso_8601": "2023-08-11T06:41:59.822902Z",
"url": "https://files.pythonhosted.org/packages/e7/73/126a11ca77c51f9d811c3f767afe277c1b8885f03f0c8e75fc485a790cf2/huggingface_tool-0.0.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-11 06:41:59",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "OpenRL-Lab",
"github_project": "huggingface_tool",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "huggingface-tool"
}
JSON
