| Name | hfmc JSON |
| Version |
0.1.10
JSON |
| download |
| home_page | None |
| Summary | A tiny cache widget for accessing hugging face models easier and faster! |
| upload_time | 2024-09-11 08:43:07 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | None |
| keywords |
9# aisoft
cache
huggingface
models
p2p
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# HuggingFace Model Cache
[简体中文](https://aisoft9.github.io/hfmc/README.zh)
HuggingFace Model Cache (referred to as "HFMC") is a compact and efficient tool designed to help users use models on HuggingFace faster and more easily. HFMC assists users in the following ways:
1. Sharing model files between different HFMC nodes to avoid downloading the same model repeatedly from hf.co;
2. Automatically selecting the best download method from local networks, mirror sites, and hf sources;
3. Supporting resuming model downloads from the point of interruption to reduce wasted bandwidth and precious time due to unstable networks;
4. Quickly viewing, adding, and deleting local model files to make model management easier.
## Features of HFMC
### Model Sharing
Within the same local network, HFMC nodes can share models with each other. For instance, if node "Pegasus" has downloaded the meta-llama-3.1-8B model and node "Cygnus" wants to download it, HFMC will fetch the model from node "Pegasus" over the local network. This is faster, more stable, and more cost-effective than downloading from huggingface.co.
In this way, colleagues within the same lab or office can quickly share models, and users can also use HFMC to rapidly distribute models within a GPU cluster.
### Multi-Source Download
HFMC integrates multiple download sources to provide model download functionality. Specifically, when a user downloads a model via HFMC, it will sequentially attempt to download the target model from the local network (from other HFMC nodes), hf mirror sites (such as hf-mirror.com), and hf source sites.
### Resume Interrupted Downloads
Compared to manually downloading models from HuggingFace via a browser, HFMC offers resuming interrupted downloads. HFMC supports "cross-source resume" functionality, meaning an interrupted download on a local network can seamlessly continue from a mirror site.
### Model Management
HFMC provides a command-line tool to help users browse, download, and delete local model files. Information about model file sizes, quantities, and paths can be easily reviewed with a single command.
## Installing HFMC
HFMC supports Python version 3.8 and above, and can run on Windows, MacOS, and Linux.
Install HFMC using the following command:
pip install hfmc
## Detailed Documentation
- [HFMC User Guide](https://aisoft9.github.io/hfmc/GUIDELINE.en)
- [HFMC Command Line Reference](https://aisoft9.github.io/hfmc/REFERENCE.en)
Raw data
{
"_id": null,
"home_page": null,
"name": "hfmc",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "9# AISoft, cache, huggingface, models, p2p",
"author": null,
"author_email": "9# AISoft <aisoft9@163.com>",
"download_url": "https://files.pythonhosted.org/packages/12/5e/f0f2cc287251c8b9a38a2de1dd43571d230b7aa9ba7600eebe8a8e863bbd/hfmc-0.1.10.tar.gz",
"platform": null,
"description": "# HuggingFace Model Cache\n\n[\u7b80\u4f53\u4e2d\u6587](https://aisoft9.github.io/hfmc/README.zh)\n\nHuggingFace Model Cache (referred to as \"HFMC\") is a compact and efficient tool designed to help users use models on HuggingFace faster and more easily. HFMC assists users in the following ways:\n\n1. Sharing model files between different HFMC nodes to avoid downloading the same model repeatedly from hf.co;\n2. Automatically selecting the best download method from local networks, mirror sites, and hf sources;\n3. Supporting resuming model downloads from the point of interruption to reduce wasted bandwidth and precious time due to unstable networks;\n4. Quickly viewing, adding, and deleting local model files to make model management easier.\n\n## Features of HFMC\n\n### Model Sharing\n\nWithin the same local network, HFMC nodes can share models with each other. For instance, if node \"Pegasus\" has downloaded the meta-llama-3.1-8B model and node \"Cygnus\" wants to download it, HFMC will fetch the model from node \"Pegasus\" over the local network. This is faster, more stable, and more cost-effective than downloading from huggingface.co.\n\nIn this way, colleagues within the same lab or office can quickly share models, and users can also use HFMC to rapidly distribute models within a GPU cluster.\n\n### Multi-Source Download\n\nHFMC integrates multiple download sources to provide model download functionality. Specifically, when a user downloads a model via HFMC, it will sequentially attempt to download the target model from the local network (from other HFMC nodes), hf mirror sites (such as hf-mirror.com), and hf source sites.\n\n### Resume Interrupted Downloads\n\nCompared to manually downloading models from HuggingFace via a browser, HFMC offers resuming interrupted downloads. HFMC supports \"cross-source resume\" functionality, meaning an interrupted download on a local network can seamlessly continue from a mirror site.\n\n### Model Management\n\nHFMC provides a command-line tool to help users browse, download, and delete local model files. Information about model file sizes, quantities, and paths can be easily reviewed with a single command.\n\n## Installing HFMC\n\nHFMC supports Python version 3.8 and above, and can run on Windows, MacOS, and Linux.\n\nInstall HFMC using the following command:\n\n pip install hfmc\n\n## Detailed Documentation\n\n- [HFMC User Guide](https://aisoft9.github.io/hfmc/GUIDELINE.en)\n- [HFMC Command Line Reference](https://aisoft9.github.io/hfmc/REFERENCE.en)\n",
"bugtrack_url": null,
"license": null,
"summary": "A tiny cache widget for accessing hugging face models easier and faster!",
"version": "0.1.10",
"project_urls": {
"Documentation": "https://aisoft9.github.io/hfmc/",
"GitHub": "https://github.com/aisoft9/hfmc",
"\u4e2d\u6587\u6587\u6863": "https://aisoft9.github.io/hfmc/README.zh"
},
"split_keywords": [
"9# aisoft",
" cache",
" huggingface",
" models",
" p2p"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5f0b78c3afd58992a7f6b297ceda42914e06f68289250f4d2ce5e963ea2a0909",
"md5": "2aec069a97cc4268f9341a807b514107",
"sha256": "c0a0eb45c24b85224e8344ee559ca50de9538bad6908f2f5e4268f0df0db3a15"
},
"downloads": -1,
"filename": "hfmc-0.1.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2aec069a97cc4268f9341a807b514107",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 32250,
"upload_time": "2024-09-11T08:43:04",
"upload_time_iso_8601": "2024-09-11T08:43:04.904791Z",
"url": "https://files.pythonhosted.org/packages/5f/0b/78c3afd58992a7f6b297ceda42914e06f68289250f4d2ce5e963ea2a0909/hfmc-0.1.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "125ef0f2cc287251c8b9a38a2de1dd43571d230b7aa9ba7600eebe8a8e863bbd",
"md5": "416224ad813161b3ce11e2021f197a4d",
"sha256": "6f5f69817a6a6e5cdefb635150c52e38a8332a7285590bc0e93a005a3d3494cc"
},
"downloads": -1,
"filename": "hfmc-0.1.10.tar.gz",
"has_sig": false,
"md5_digest": "416224ad813161b3ce11e2021f197a4d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 573777,
"upload_time": "2024-09-11T08:43:07",
"upload_time_iso_8601": "2024-09-11T08:43:07.725687Z",
"url": "https://files.pythonhosted.org/packages/12/5e/f0f2cc287251c8b9a38a2de1dd43571d230b7aa9ba7600eebe8a8e863bbd/hfmc-0.1.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-11 08:43:07",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "aisoft9",
"github_project": "hfmc",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "hfmc"
}
