# `bitsandbytes`
[](https://pepy.tech/project/bitsandbytes) [](https://pepy.tech/project/bitsandbytes) [](https://pepy.tech/project/bitsandbytes)
The `bitsandbytes` library is a lightweight Python wrapper around CUDA custom functions, in particular 8-bit optimizers, matrix multiplication (LLM.int8()), and 8 & 4-bit quantization functions.
The library includes quantization primitives for 8-bit & 4-bit operations, through `bitsandbytes.nn.Linear8bitLt` and `bitsandbytes.nn.Linear4bit` and 8-bit optimizers through `bitsandbytes.optim` module.
There are ongoing efforts to support further hardware backends, i.e. Intel CPU + GPU, AMD GPU, Apple Silicon. Windows support is quite far along and is on its way as well.
**Please head to the official documentation page:**
**[https://huggingface.co/docs/bitsandbytes/main](https://huggingface.co/docs/bitsandbytes/main)**
## ALPHA TESTERS WANTED: `multi-backend-refactor` AMD GPU + Intel CPU/GPU specific BNB backend implementations
We're in the process of a complex refactor in order to allow the support of additional hardware backends, other than CUDA, in BNB. The efforts around this are already quite far along and there's plenty of functionality already in place that is in need for users to take a hands-on approach! Mac support will likely soon also see progress. However, I recommend waiting 2 weeks until the device abstraction has further consolidated (**breaking changes upcoming**).
Currently, you still need to compile from source, after checking out the `multi-backend-refactor` branch (instructions WIP, but [the current docs on the compilation from source](https://huggingface.co/docs/bitsandbytes/main/en/installation#compile-from-source) are a good starting point; [feel free to share tips / input in this Github discussion](https://github.com/TimDettmers/bitsandbytes/discussions/1219). We'll soon enable nightly releases to make this much easier for you!
Please give feedback to us in [this dedicated Github Discussion space](https://github.com/TimDettmers/bitsandbytes/discussions/categories/catch-all-alpha-testing-the-multi-backend-refactor)!
We're super excited about these recent developments and grateful for any constructive input or support that you can give to help us make this a reality. BNB is a community project and we're excited for your collaboration 🤗
## License
`bitsandbytes` is MIT licensed.
We thank Fabio Cannizzo for his work on [FastBinarySearch](https://github.com/fabiocannizzo/FastBinarySearch) which we use for CPU quantization.
Raw data
{
"_id": null,
"home_page": "https://github.com/TimDettmers/bitsandbytes",
"name": "bitsandbytes",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "gpu optimizers optimization 8-bit quantization compression",
"author": "Tim Dettmers",
"author_email": "dettmers@cs.washington.edu",
"download_url": null,
"platform": null,
"description": "# `bitsandbytes`\n\n[](https://pepy.tech/project/bitsandbytes) [](https://pepy.tech/project/bitsandbytes) [](https://pepy.tech/project/bitsandbytes)\n\nThe `bitsandbytes` library is a lightweight Python wrapper around CUDA custom functions, in particular 8-bit optimizers, matrix multiplication (LLM.int8()), and 8 & 4-bit quantization functions.\n\nThe library includes quantization primitives for 8-bit & 4-bit operations, through `bitsandbytes.nn.Linear8bitLt` and `bitsandbytes.nn.Linear4bit` and 8-bit optimizers through `bitsandbytes.optim` module.\n\nThere are ongoing efforts to support further hardware backends, i.e. Intel CPU + GPU, AMD GPU, Apple Silicon. Windows support is quite far along and is on its way as well.\n\n**Please head to the official documentation page:**\n\n**[https://huggingface.co/docs/bitsandbytes/main](https://huggingface.co/docs/bitsandbytes/main)**\n\n## ALPHA TESTERS WANTED: `multi-backend-refactor` AMD GPU + Intel CPU/GPU specific BNB backend implementations\n\nWe're in the process of a complex refactor in order to allow the support of additional hardware backends, other than CUDA, in BNB. The efforts around this are already quite far along and there's plenty of functionality already in place that is in need for users to take a hands-on approach! Mac support will likely soon also see progress. However, I recommend waiting 2 weeks until the device abstraction has further consolidated (**breaking changes upcoming**).\n\nCurrently, you still need to compile from source, after checking out the `multi-backend-refactor` branch (instructions WIP, but [the current docs on the compilation from source](https://huggingface.co/docs/bitsandbytes/main/en/installation#compile-from-source) are a good starting point; [feel free to share tips / input in this Github discussion](https://github.com/TimDettmers/bitsandbytes/discussions/1219). We'll soon enable nightly releases to make this much easier for you!\n\nPlease give feedback to us in [this dedicated Github Discussion space](https://github.com/TimDettmers/bitsandbytes/discussions/categories/catch-all-alpha-testing-the-multi-backend-refactor)!\n\nWe're super excited about these recent developments and grateful for any constructive input or support that you can give to help us make this a reality. BNB is a community project and we're excited for your collaboration \ud83e\udd17\n\n## License\n\n`bitsandbytes` is MIT licensed.\n\nWe thank Fabio Cannizzo for his work on [FastBinarySearch](https://github.com/fabiocannizzo/FastBinarySearch) which we use for CPU quantization.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "k-bit optimizers and matrix multiplication routines.",
"version": "0.44.1",
"project_urls": {
"Homepage": "https://github.com/TimDettmers/bitsandbytes"
},
"split_keywords": [
"gpu",
"optimizers",
"optimization",
"8-bit",
"quantization",
"compression"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e4e6ccb84da7ffaf208a71c2c3c8e1120b34759df640db959660be9a98505eb4",
"md5": "f5dad30e76c52f93b730b707288d9f82",
"sha256": "b2f24c6cbf11fc8c5d69b3dcecee9f7011451ec59d6ac833e873c9f105259668"
},
"downloads": -1,
"filename": "bitsandbytes-0.44.1-py3-none-manylinux_2_24_x86_64.whl",
"has_sig": false,
"md5_digest": "f5dad30e76c52f93b730b707288d9f82",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 122419627,
"upload_time": "2024-09-30T16:20:50",
"upload_time_iso_8601": "2024-09-30T16:20:50.417032Z",
"url": "https://files.pythonhosted.org/packages/e4/e6/ccb84da7ffaf208a71c2c3c8e1120b34759df640db959660be9a98505eb4/bitsandbytes-0.44.1-py3-none-manylinux_2_24_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5ff511bddebb5addc0a005b0c1cecc6e4c6e4055ad7b860bdcbf6374e12a51f5",
"md5": "a1911fb9d314b58e86a5374fb4e6c261",
"sha256": "8e68e12aa25d2cf9a1730ad72890a5d1a19daa23f459a6a4679331f353d58cb4"
},
"downloads": -1,
"filename": "bitsandbytes-0.44.1-py3-none-win_amd64.whl",
"has_sig": false,
"md5_digest": "a1911fb9d314b58e86a5374fb4e6c261",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 121451331,
"upload_time": "2024-09-30T16:20:56",
"upload_time_iso_8601": "2024-09-30T16:20:56.948414Z",
"url": "https://files.pythonhosted.org/packages/5f/f5/11bddebb5addc0a005b0c1cecc6e4c6e4055ad7b860bdcbf6374e12a51f5/bitsandbytes-0.44.1-py3-none-win_amd64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-30 16:20:50",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "TimDettmers",
"github_project": "bitsandbytes",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "bitsandbytes"
}
JSON
