# Keras-MatMulLess (Keras-MML)
[](https://pypi.org/project/keras-matmulless/)
[&logo=pypi)](https://pypi.org/project/keras-matmulless/)
[](https://pypi.org/project/keras-matmulless/)
[](LICENSE)
[&logo=readthedocs)](https://keras-matmulless.readthedocs.io/en/stable/)
[&logo=readthedocs)](https://keras-matmulless.readthedocs.io/en/latest/)
[](https://codecov.io/gh/PhotonicGluon/Keras-MatMulLess)
[](https://github.com/psf/black)
[](https://github.com/astral-sh/ruff)
[](.github/CODE_OF_CONDUCT.md)
> We offer no explanation as to why these architectures seem to work; we attribute their success, as all else, to divine benevolence.
> <div style="text-align: right">— Noam Shazeer, in <a href="https://arxiv.org/pdf/2002.05202v1"><em>GLU Variants Improve Transformer</em></a></div>
<!-- start summary -->
Keras layers without using matrix multiplications.
This is a Keras based implementation of some layers mentioned in the papers [*The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits*](https://arxiv.org/pdf/2402.17764v1) and [*Scalable MatMul-free Language Modeling*](https://arxiv.org/pdf/2406.02528v5). <!-- end summary --> Find the documentation [here](https://keras-matmulless.readthedocs.io/).
## Rationale
<!-- start rationale -->
Traditional, matrix multiplication based layers suffer from a few issues.
1. They have high inference and computational costs due to the use of matrix multiplications. This hinders the speed at which inference is performed on GPU-less machines.
2. The memory use for storing full precision weights is very high.
3. The energy costs of running matrix multiplications is very high.
Matrix multiplication free layers addresses these pain points by removing the key source of costs — matrix multiplications.
<!-- end rationale -->
## Installation
<!-- start installation -->
### Requirements
Keras-MML has a few requirements, namely
- Python 3.9 (or above);
- Keras; and
- the Keras backend (either [Tensorflow](https://www.tensorflow.org/), [PyTorch](https://pytorch.org/), or [Jax](https://jax.readthedocs.io/en/latest/index.html)).
Instructions on how to install Keras can be found [here](https://keras.io/getting_started/).
### Installation Instructions
#### PyPi
If you use pip, you can install Keras-MML using the command
```bash
pip install keras-matmulless
```
##### Pre-Release Versions
To install pre-release versions, use the command
```bash
pip install --pre keras-matmulless
```
##### Nightly Versions
Nightly releases for Keras-MML are primarily found on the [TestPyPi](https://test.pypi.org/project/keras-matmulless/) page. To install them, use the command
```bash
pip install -i https://test.pypi.org/simple/ keras-matmulless
```
#### Building From Scratch
First, clone the repository using
```bash
git clone https://github.com/PhotonicGluon/Keras-MatMulLess.git
cd Keras-MatMulLess
```
We recommend to create a virtual environment to install [Poetry](https://python-poetry.org/) and the other dependencies into.
```bash
python -m venv venv # If `python` doesn't work, try `python3`
```
Activate the virtual environment using
```bash
source venv/bin/activate
```
or, if you are on Windows,
```bash
venv/Scripts/activate
```
Now we install Poetry.
```bash
pip install poetry
```
Finally, install the development dependencies. The development dependencies are split into several groups.
- The `test` group contains dependencies that are used to perform testing.
- The `docs` group contains dependencies that are used to generate the documentation.
- The `build` group contains dependencies that are used to create a distributable.
- The `notebook` group is required to run the Jupyter notebooks in the documentation folder.
Simply include the desired groups in the `install.py` call. For example, to install `test`, `docs`, and `build` (the main development dependencies), run the following command.
```bash
python install.py test docs build
```
If you have not installed a backend (i.e., [Tensorflow](https://www.tensorflow.org/), [PyTorch](https://pytorch.org/), or [Jax](https://jax.readthedocs.io/en/latest/index.html)) you can do so here.
```bash
python install.py test docs build --backend BACKEND_NAME
```
Note that the `BACKEND_NAME` to be specified here is
- `tensorflow` for the Tensorflow backend;
- `torch` for the PyTorch backend; and
- `jax` for the Jax backend.
If you need to install with CUDA support, run
```bash
python install.py test docs build --backend BACKEND_NAME --with-cuda
```
That's it! You should now have access to the `keras_mml` package.
<!-- end installation -->
## Quickstart
Read the [tutorial](https://keras-matmulless.readthedocs.io/en/stable/getting-started/tutorial.html).
## Contributing
We welcome contributions! Please read more about contributing to Keras-MML in the [contribution guidelines](.github/CONTRIBUTING.md).
## License
Keras-MML is licensed under the Apache 2.0 license.
Raw data
{
"_id": null,
"home_page": null,
"name": "keras-matmulless",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.12,>=3.9",
"maintainer_email": null,
"keywords": "tensorflow, pytorch, jax, keras, machine learning",
"author": "PhotonicGluon",
"author_email": "25820201+PhotonicGluon@users.noreply.github.com",
"download_url": "https://files.pythonhosted.org/packages/c2/22/5a4777f60532a420273c0336b0981071d1e669f64b3e2c51846bc0fe7a2a/keras_matmulless-0.1.1.tar.gz",
"platform": null,
"description": "# Keras-MatMulLess (Keras-MML)\n\n[](https://pypi.org/project/keras-matmulless/)\n[&logo=pypi)](https://pypi.org/project/keras-matmulless/)\n[](https://pypi.org/project/keras-matmulless/)\n[](LICENSE)\n\n[&logo=readthedocs)](https://keras-matmulless.readthedocs.io/en/stable/)\n[&logo=readthedocs)](https://keras-matmulless.readthedocs.io/en/latest/)\n\n[](https://codecov.io/gh/PhotonicGluon/Keras-MatMulLess)\n[](https://github.com/psf/black)\n[](https://github.com/astral-sh/ruff)\n\n[](.github/CODE_OF_CONDUCT.md)\n\n> We offer no explanation as to why these architectures seem to work; we attribute their success, as all else, to divine benevolence.\n> <div style=\"text-align: right\">— Noam Shazeer, in <a href=\"https://arxiv.org/pdf/2002.05202v1\"><em>GLU Variants Improve Transformer</em></a></div>\n\n<!-- start summary -->\nKeras layers without using matrix multiplications.\n\nThis is a Keras based implementation of some layers mentioned in the papers [*The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits*](https://arxiv.org/pdf/2402.17764v1) and [*Scalable MatMul-free Language Modeling*](https://arxiv.org/pdf/2406.02528v5). <!-- end summary --> Find the documentation [here](https://keras-matmulless.readthedocs.io/).\n\n## Rationale\n<!-- start rationale -->\n\nTraditional, matrix multiplication based layers suffer from a few issues.\n\n1. They have high inference and computational costs due to the use of matrix multiplications. This hinders the speed at which inference is performed on GPU-less machines.\n2. The memory use for storing full precision weights is very high.\n3. The energy costs of running matrix multiplications is very high.\n\nMatrix multiplication free layers addresses these pain points by removing the key source of costs — matrix multiplications.\n\n<!-- end rationale -->\n\n## Installation\n<!-- start installation -->\n\n### Requirements\n\nKeras-MML has a few requirements, namely\n\n- Python 3.9 (or above);\n- Keras; and\n- the Keras backend (either [Tensorflow](https://www.tensorflow.org/), [PyTorch](https://pytorch.org/), or [Jax](https://jax.readthedocs.io/en/latest/index.html)).\n\nInstructions on how to install Keras can be found [here](https://keras.io/getting_started/).\n\n### Installation Instructions\n\n#### PyPi\n\nIf you use pip, you can install Keras-MML using the command\n\n```bash\npip install keras-matmulless\n```\n\n##### Pre-Release Versions\n\nTo install pre-release versions, use the command\n\n```bash\npip install --pre keras-matmulless\n```\n\n##### Nightly Versions\n\nNightly releases for Keras-MML are primarily found on the [TestPyPi](https://test.pypi.org/project/keras-matmulless/) page. To install them, use the command\n\n```bash\npip install -i https://test.pypi.org/simple/ keras-matmulless\n```\n\n#### Building From Scratch\n\nFirst, clone the repository using\n\n```bash\ngit clone https://github.com/PhotonicGluon/Keras-MatMulLess.git\ncd Keras-MatMulLess\n```\n\nWe recommend to create a virtual environment to install [Poetry](https://python-poetry.org/) and the other dependencies into.\n\n```bash\npython -m venv venv # If `python` doesn't work, try `python3`\n```\n\nActivate the virtual environment using\n\n```bash\nsource venv/bin/activate\n```\n\nor, if you are on Windows,\n\n```bash\nvenv/Scripts/activate\n```\n\nNow we install Poetry.\n\n```bash\npip install poetry\n```\n\nFinally, install the development dependencies. The development dependencies are split into several groups.\n\n- The `test` group contains dependencies that are used to perform testing.\n- The `docs` group contains dependencies that are used to generate the documentation.\n- The `build` group contains dependencies that are used to create a distributable.\n- The `notebook` group is required to run the Jupyter notebooks in the documentation folder.\n\nSimply include the desired groups in the `install.py` call. For example, to install `test`, `docs`, and `build` (the main development dependencies), run the following command.\n\n```bash\npython install.py test docs build\n```\n\nIf you have not installed a backend (i.e., [Tensorflow](https://www.tensorflow.org/), [PyTorch](https://pytorch.org/), or [Jax](https://jax.readthedocs.io/en/latest/index.html)) you can do so here.\n\n```bash\npython install.py test docs build --backend BACKEND_NAME\n```\n\nNote that the `BACKEND_NAME` to be specified here is\n\n- `tensorflow` for the Tensorflow backend;\n- `torch` for the PyTorch backend; and\n- `jax` for the Jax backend.\n\nIf you need to install with CUDA support, run\n\n```bash\npython install.py test docs build --backend BACKEND_NAME --with-cuda\n```\n\nThat's it! You should now have access to the `keras_mml` package.\n\n<!-- end installation -->\n\n## Quickstart\n\nRead the [tutorial](https://keras-matmulless.readthedocs.io/en/stable/getting-started/tutorial.html).\n\n## Contributing\n\nWe welcome contributions! Please read more about contributing to Keras-MML in the [contribution guidelines](.github/CONTRIBUTING.md).\n\n## License\n\nKeras-MML is licensed under the Apache 2.0 license.\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Keras layers without using matrix multiplications.",
"version": "0.1.1",
"project_urls": null,
"split_keywords": [
"tensorflow",
" pytorch",
" jax",
" keras",
" machine learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a51cfd84cb81629d0d9ef1744808b349f1aea6a41e268646e1168c40e2afd4a4",
"md5": "3147445cca3c9d5248a96662cd4db2a3",
"sha256": "ce63ff38c62bb508c4f0e995a10d18e9d67451aa4383f735b1e0e81ba8e8a2e8"
},
"downloads": -1,
"filename": "keras_matmulless-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3147445cca3c9d5248a96662cd4db2a3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.12,>=3.9",
"size": 44900,
"upload_time": "2024-08-19T07:47:56",
"upload_time_iso_8601": "2024-08-19T07:47:56.964847Z",
"url": "https://files.pythonhosted.org/packages/a5/1c/fd84cb81629d0d9ef1744808b349f1aea6a41e268646e1168c40e2afd4a4/keras_matmulless-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c2225a4777f60532a420273c0336b0981071d1e669f64b3e2c51846bc0fe7a2a",
"md5": "70932fbe69d50e0f5ad405af6a006641",
"sha256": "0cc6407975766ce2f3083bbf7dd746dcf0997abc3a110f7662a76b85b6763f5b"
},
"downloads": -1,
"filename": "keras_matmulless-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "70932fbe69d50e0f5ad405af6a006641",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.12,>=3.9",
"size": 30024,
"upload_time": "2024-08-19T07:47:58",
"upload_time_iso_8601": "2024-08-19T07:47:58.675854Z",
"url": "https://files.pythonhosted.org/packages/c2/22/5a4777f60532a420273c0336b0981071d1e669f64b3e2c51846bc0fe7a2a/keras_matmulless-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-19 07:47:58",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "keras-matmulless"
}
JSON