# Fasterai
<p align="center">
<a href="https://pypi.org/project/fasterai/"><img src="https://img.shields.io/pypi/v/fasterai?color=%235BAFAF"></a>
<a href="https://pypi.org/project/fasterai/"><img src="https://static.pepy.tech/personalized-badge/fasterai?color=%235BAFAFperiod=total&units=international_system&left_color=grey&right_color=%235BAFAF&left_text=downloads"></a>
<a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/github/license/nathanhubens/fasterai?color=%235bafaf"></a>
<a href="https://pypi.org/project/fasterai/"><img src="https://img.shields.io/badge/DOI-10.5281%2Fzenodo.6469868-y?color=%235BAFAF"></a>
</p>
<p align="center">
<a href="#features">Features</a> •
<a href="#installation">Installation</a> •
<a href="#tutorials">Tutorials</a> •
<a href="#join-the-community">Community</a> •
<a href="#citing">Citing</a> •
<a href="#license">License</a>
</p>
`fasterai` is a library created to make neural network **smaller** and **faster**. It essentially relies on common compression techniques for networks such as pruning, knowledge distillation, Lottery Ticket Hypothesis, ...
The core feature of `fasterai` is its Sparsifying capabilities, constructed on 4 main modules: **granularity**, **context**, **criteria**, **schedule**. Each of these modules is highly customizable, allowing you to change them according to your needs or even to come up with your own !
## Project Documentation
Visit [Read The Docs Project Page](https://nathanhubens.github.io/fasterai/) or read following README to know more about using `fasterai`.
---
## Features
### 1. Sparsifying
Make your model sparse (*i.e.* prune it) according to a:
- <b>Sparsity: </b> the percentage of weights that will be replaced by 0
- <b>Granularity: </b> the granularity at which you operate the pruning (removing weights, vectors, kernels, filters)
- <b>Context: </b> prune either each layer independantly (local pruning) or the whole model (global pruning)
- <b>Criteria: </b> the criteria used to select the weights to remove (magnitude, movement, ...)
- <b>Schedule: </b> which schedule you want to use for pruning (one shot, iterative, gradual, ...)
This can be achieved by using the `SparsifyCallback(sparsity, granularity, context, criteria, schedule)`
### 2. Pruning
Once your model has useless nodes due to zero-weights, they can be removed to not be a part of the network anymore.
This can be achieved by using the `Pruner()` method
### 3. Regularization
Instead of explicitely make your network sparse, let it train towards sparse connections by pushing the weights to be as small as possible.
Regularization can be applied to groups of weights, following the same granularities as for sparsifying, i.e.:
- <b>Granularity: </b> the granularity at which you operate the regularization (weights, vectors, kernels, filters, ...)
This can be achieved by using the `RegularizationCallback(granularity)`
### 4. Knowledge Distillation
Distill the knowledge acquired by a big model into a smaller one, by using the `KnowledgeDistillation` callback.
### 5. Lottery Ticket Hypothesis
Find the winning ticket in you network, *i.e.* the initial subnetwork able to attain at least similar performances than the network as a whole.
---
## Quick Start
### 0. Import fasterai
```python
from fasterai.sparse.all import *
```
### 1. Create your model with fastai
```python
learn = cnn_learner(dls, model)
```
### 2. Get you Fasterai Callback
```python
sp_cb=SparsifyCallback(sparsity, granularity, context, criteria, schedule)
```
### 3. Train you model to make it sparse !
```python
learn.fit_one_cycle(n_epochs, cbs=sp_cb)
```
---
## Installation
```
pip install git+https://github.com/nathanhubens/fasterai.git
```
or
```
pip install fasterai
```
---
## Tutorials
- [Get Started with FasterAI](https://nathanhubens.github.io/fasterai/quickstart.html)
- [Create your own pruning schedule](https://nathanhubens.github.io/fasterai/tutorial.schedules.html)
- [Find winning tickets using the Lottery Ticket Hypothesis](https://nathanhubens.github.io/fasterai/tutorial.lottery_ticket.html)
- [Use Knowledge Distillation to help a student model to reach higher performance](https://nathanhubens.github.io/fasterai/tutorial.knowledge_distillation.html)
- [Sparsify Transformers](https://nathanhubens.github.io/fasterai/tutorial.transformers.html)
- More to come...
---
## Join the community
Join [our discord server](https://discord.gg/32BwhJSB9u) to meet other FasterAI users and share your projects!
---
## Citing
```
@software{Hubens,
author = {Nathan Hubens},
title = {fasterai},
year = 2022,
publisher = {Zenodo},
version = {v0.1.6},
doi = {10.5281/zenodo.6469868},
url = {https://doi.org/10.5281/zenodo.6469868}
}
```
---
## License
[Apache-2.0](https://www.apache.org/licenses/) License.
Raw data
{
"_id": null,
"home_page": "https://github.com/nathanhubens/fasterai/tree/master/",
"name": "fasterai",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "Pruning",
"author": "Nathan Hubens",
"author_email": "nathan.hubens@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/45/91/8fa64d564f50e802a6c47112ce2e98127c5533da9998e2383f1f1ef0639f/fasterai-0.2.2.tar.gz",
"platform": null,
"description": "# Fasterai\n\n\n\n\n\n<p align=\"center\">\n <a href=\"https://pypi.org/project/fasterai/\"><img src=\"https://img.shields.io/pypi/v/fasterai?color=%235BAFAF\"></a>\n <a href=\"https://pypi.org/project/fasterai/\"><img src=\"https://static.pepy.tech/personalized-badge/fasterai?color=%235BAFAFperiod=total&units=international_system&left_color=grey&right_color=%235BAFAF&left_text=downloads\"></a>\n <a href=\"https://opensource.org/licenses/MIT\"><img src=\"https://img.shields.io/github/license/nathanhubens/fasterai?color=%235bafaf\"></a>\n <a href=\"https://pypi.org/project/fasterai/\"><img src=\"https://img.shields.io/badge/DOI-10.5281%2Fzenodo.6469868-y?color=%235BAFAF\"></a>\n</p>\n\n\n<p align=\"center\">\n <a href=\"#features\">Features</a> \u2022\n <a href=\"#installation\">Installation</a> \u2022\n <a href=\"#tutorials\">Tutorials</a> \u2022\n <a href=\"#join-the-community\">Community</a> \u2022\n <a href=\"#citing\">Citing</a> \u2022\n <a href=\"#license\">License</a>\n</p>\n\n`fasterai` is a library created to make neural network **smaller** and **faster**. It essentially relies on common compression techniques for networks such as pruning, knowledge distillation, Lottery Ticket Hypothesis, ...\n\nThe core feature of `fasterai` is its Sparsifying capabilities, constructed on 4 main modules: **granularity**, **context**, **criteria**, **schedule**. Each of these modules is highly customizable, allowing you to change them according to your needs or even to come up with your own !\n\n## Project Documentation\n\nVisit [Read The Docs Project Page](https://nathanhubens.github.io/fasterai/) or read following README to know more about using `fasterai`.\n\n---\n\n## Features\n\n### 1. Sparsifying\n\n![alt text](nbs/imgs/sparsification.png \"Sparsification\")\n\nMake your model sparse (*i.e.* prune it) according to a:\n- <b>Sparsity: </b> the percentage of weights that will be replaced by 0\n- <b>Granularity: </b> the granularity at which you operate the pruning (removing weights, vectors, kernels, filters)\n- <b>Context: </b> prune either each layer independantly (local pruning) or the whole model (global pruning)\n- <b>Criteria: </b> the criteria used to select the weights to remove (magnitude, movement, ...)\n- <b>Schedule: </b> which schedule you want to use for pruning (one shot, iterative, gradual, ...)\n\nThis can be achieved by using the `SparsifyCallback(sparsity, granularity, context, criteria, schedule)`\n\n### 2. Pruning\n\n![alt text](nbs/imgs/pruning_readme.png \"Pruning\")\n\nOnce your model has useless nodes due to zero-weights, they can be removed to not be a part of the network anymore.\n\nThis can be achieved by using the `Pruner()` method\n\n### 3. Regularization\n\n![alt text](nbs/imgs/regularization.png \"Regularization\")\n\nInstead of explicitely make your network sparse, let it train towards sparse connections by pushing the weights to be as small as possible.\n\nRegularization can be applied to groups of weights, following the same granularities as for sparsifying, i.e.:\n- <b>Granularity: </b> the granularity at which you operate the regularization (weights, vectors, kernels, filters, ...)\n\nThis can be achieved by using the `RegularizationCallback(granularity)`\n\n### 4. Knowledge Distillation\n\n![alt text](nbs/imgs/distillation.png \"Distillation\")\n\nDistill the knowledge acquired by a big model into a smaller one, by using the `KnowledgeDistillation` callback.\n\n### 5. Lottery Ticket Hypothesis\n\n![alt text](nbs/imgs/LTH.png \"Lottery Ticket Hypothesis\")\n\nFind the winning ticket in you network, *i.e.* the initial subnetwork able to attain at least similar performances than the network as a whole.\n\n---\n\n## Quick Start\n\n### 0. Import fasterai\n\n```python\nfrom fasterai.sparse.all import *\n```\n\n### 1. Create your model with fastai\n\n```python\nlearn = cnn_learner(dls, model)\n```\n\n### 2. Get you Fasterai Callback\n\n```python\nsp_cb=SparsifyCallback(sparsity, granularity, context, criteria, schedule)\n```\n\n### 3. Train you model to make it sparse !\n\n```python\nlearn.fit_one_cycle(n_epochs, cbs=sp_cb)\n```\n\n---\n\n## Installation\n\n\n```\npip install git+https://github.com/nathanhubens/fasterai.git\n```\n\nor \n\n```\npip install fasterai\n```\n\n---\n\n## Tutorials\n\n- [Get Started with FasterAI](https://nathanhubens.github.io/fasterai/quickstart.html)\n- [Create your own pruning schedule](https://nathanhubens.github.io/fasterai/tutorial.schedules.html)\n- [Find winning tickets using the Lottery Ticket Hypothesis](https://nathanhubens.github.io/fasterai/tutorial.lottery_ticket.html)\n- [Use Knowledge Distillation to help a student model to reach higher performance](https://nathanhubens.github.io/fasterai/tutorial.knowledge_distillation.html)\n- [Sparsify Transformers](https://nathanhubens.github.io/fasterai/tutorial.transformers.html)\n- More to come...\n\n---\n\n## Join the community\n\nJoin [our discord server](https://discord.gg/32BwhJSB9u) to meet other FasterAI users and share your projects!\n\n---\n\n## Citing\n```\n@software{Hubens,\n author = {Nathan Hubens},\n title = {fasterai},\n year = 2022,\n publisher = {Zenodo},\n version = {v0.1.6},\n doi = {10.5281/zenodo.6469868},\n url = {https://doi.org/10.5281/zenodo.6469868}\n}\n```\n\n---\n\n## License\n\n[Apache-2.0](https://www.apache.org/licenses/) License.\n\n\n",
"bugtrack_url": null,
"license": "Apache Software License 2.0",
"summary": "A library to make neural networks lighter and faster with fastai",
"version": "0.2.2",
"project_urls": {
"Homepage": "https://github.com/nathanhubens/fasterai/tree/master/"
},
"split_keywords": [
"pruning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "427f3f957248ca4424af6733e0a027f308dd6a91d41e6098184e8102a79ab8a7",
"md5": "969bcf84a4a74c0f1df16a76c4b9b1ff",
"sha256": "ff1370e9bf7984cfe9630f5093ff765f7cba825a56c9a66ad8254d1d7ec6b1e8"
},
"downloads": -1,
"filename": "fasterai-0.2.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "969bcf84a4a74c0f1df16a76c4b9b1ff",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 25226,
"upload_time": "2023-12-17T13:35:54",
"upload_time_iso_8601": "2023-12-17T13:35:54.684498Z",
"url": "https://files.pythonhosted.org/packages/42/7f/3f957248ca4424af6733e0a027f308dd6a91d41e6098184e8102a79ab8a7/fasterai-0.2.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "45918fa64d564f50e802a6c47112ce2e98127c5533da9998e2383f1f1ef0639f",
"md5": "4ef3c901375ff7e2848941a365d45ec0",
"sha256": "540ab3329b3a93adddefdc5f92e11853f4c8e55883d09e805dceaa5a15cd7744"
},
"downloads": -1,
"filename": "fasterai-0.2.2.tar.gz",
"has_sig": false,
"md5_digest": "4ef3c901375ff7e2848941a365d45ec0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 24551,
"upload_time": "2023-12-17T13:35:56",
"upload_time_iso_8601": "2023-12-17T13:35:56.740873Z",
"url": "https://files.pythonhosted.org/packages/45/91/8fa64d564f50e802a6c47112ce2e98127c5533da9998e2383f1f1ef0639f/fasterai-0.2.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-12-17 13:35:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "nathanhubens",
"github_project": "fasterai",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "fasterai"
}
JSON
