| Name | sparrow-lib JSON |
| Version |
2.0.4
JSON |
| download |
| home_page | None |
| Summary | A library to augment LLMs with dynamic, trainable routers for efficient fine-tuning. |
| upload_time | 2025-09-06 17:29:59 |
| maintainer | None |
| docs_url | None |
| author | AmirReza |
| requires_python | >=3.7 |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Sparrow 🐦: Dynamic Routers for Efficient LLM Fine-Tuning
[](https://badge.fury.io/py/sparrow-lib)
[](https://opensource.org/licenses/MIT)
**[English](#english) | [فارسی](#فارسی)**
---
## English
Sparrow is a lightweight Python library that seamlessly integrates with Hugging Face `transformers` to make your Large Language Models (LLMs) smarter and more efficient. It augments pre-trained models with **dynamic, trainable routers** that learn to create sparse computational paths through the network.
This allows for highly parameter-efficient fine-tuning (PEFT) where you freeze the original model's weights and only train the tiny routers, saving immense computational resources and preventing catastrophic forgetting.
### Key Features
- **Plug-and-Play**: Convert a standard Hugging Face model into a dynamic, sparse model with a single line of code.
- **Parameter-Efficient**: Fine-tune only a tiny fraction of the total parameters (often < 0.1%).
- **Autonomous Routing**: Routers learn automatically how many and which neurons to activate, guided by a sparsity-regularized loss.
- **Deep Integration**: Comes with a custom `SparrowTrainer` for seamless integration with the Hugging Face ecosystem.
- **Insightful Reporting**: Includes utilities to generate reports on neuron activation and model sparsity.
### Installation
```bash
pip install sparrow-lib
```
### Quickstart
Here's how to fine-tune a BERT model by only training the routers:
```python
from transformers import BertForSequenceClassification, AutoTokenizer, TrainingArguments
from sparrow import SparrowConfig, add_routers_to_model, SparrowTrainer, get_sparsity_report
# 1. Define your Sparrow configuration
sparrow_config = SparrowConfig(sparsity_lambda=0.01)
# 2. Load a pre-trained model and augment it with routers
model = BertForSequenceClassification.from_pretrained('bert-base-uncased')
model = add_routers_to_model(model, config=sparrow_config)
# 3. Freeze original model weights (only routers will be trainable)
for name, param in model.named_parameters():
if 'router' not in name:
param.requires_grad = False
# ... Load your dataset ...
# 4. Use the custom SparrowTrainer which handles the sparsity loss
training_args = TrainingArguments(output_dir="test_trainer", num_train_epochs=1)
trainer = SparrowTrainer(
model=model,
args=training_args,
sparrow_config=sparrow_config,
train_dataset=your_train_dataset,
eval_dataset=your_eval_dataset
)
# 5. Start training!
trainer.train()
# 6. Get a report on how sparse your model became
trainer.evaluate()
report = get_sparsity_report(model)
print("\n--- Sparsity Report ---")
print(report)
```
---
## فارسی
`Sparrow` 🐦 یک کتابخانه سبک پایتون است که به راحتی با `transformers` هاگینگ فیس ادغام می‌شود تا مدل‌های زبان بزرگ (LLM) شما را هوشمندتر و کارآمدتر کند. این کتابخانه مدل‌های از پیش آموزش‌دیده را با **مسیریاب‌های دینامیک و قابل آموزش** تقویت می‌کند که یاد می‌گیرند مسیرهای محاسباتی پراکنده (sparse) در شبکه ایجاد کنند.
این روش امکان تنظیم دقیق بهینه از نظر پارامتر (PEFT) را فراهم می‌کند، به طوری که شما وزن‌های مدل اصلی را منجمد کرده و فقط مسیریاب‌های کوچک را آموزش می‌دهید. این کار باعث صرفه‌جویی عظیم در منابع محاسباتی شده و از فراموشی فاجعه‌بار (catastrophic forgetting) جلوگیری می‌کند.
### قابلیت‌های کلیدی
- **استفاده آسان**: یک مدل استاندارد هاگینگ فیس را با یک خط کد به یک مدل دینامیک و پراکنده تبدیل کنید.
- **بهینه از نظر پارامتر**: فقط بخش بسیار کوچکی از پارامترهای کل مدل (معمولاً کمتر از ۰.۱٪) را آموزش دهید.
- **مسیریابی خودکار**: مسیریاب‌ها به طور خودکار و با هدایت یک تابع زیان مبتنی بر پراکندگی، یاد می‌گیرند که چه تعداد و کدام نورون‌ها را فعال کنند.
- **ادغام عمیق**: شامل یک `SparrowTrainer` سفارشی برای ادغام کامل با اکوسیستم هاگینگ فیس.
- **گزارش‌گیری دقیق**: شامل ابزارهایی برای تولید گزارش از میزان فعالیت نورون‌ها و پراکندگی مدل.
### نصب
```bash
pip install sparrow-lib
```
### شروع سریع
مثال بالا نحوه تنظیم دقیق مدل BERT را فقط با آموزش مسیریاب‌ها نشان می‌دهد.
Raw data
{
"_id": null,
"home_page": null,
"name": "sparrow-lib",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": null,
"author": "AmirReza",
"author_email": "amirrezaahali@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/06/23/7f449c4a53379052ac200989cce267d5f1e0d2f2fce26597a5fc1809dee0/sparrow_lib-2.0.4.tar.gz",
"platform": null,
"description": "# Sparrow \ud83d\udc26: Dynamic Routers for Efficient LLM Fine-Tuning\r\n\r\n[](https://badge.fury.io/py/sparrow-lib)\r\n[](https://opensource.org/licenses/MIT)\r\n\r\n**[English](#english) | [\u0641\u0627\u0631\u0633\u06cc](#\u0641\u0627\u0631\u0633\u06cc)**\r\n\r\n---\r\n\r\n## English\r\n\r\nSparrow is a lightweight Python library that seamlessly integrates with Hugging Face `transformers` to make your Large Language Models (LLMs) smarter and more efficient. It augments pre-trained models with **dynamic, trainable routers** that learn to create sparse computational paths through the network.\r\n\r\nThis allows for highly parameter-efficient fine-tuning (PEFT) where you freeze the original model's weights and only train the tiny routers, saving immense computational resources and preventing catastrophic forgetting.\r\n\r\n### Key Features\r\n- **Plug-and-Play**: Convert a standard Hugging Face model into a dynamic, sparse model with a single line of code.\r\n- **Parameter-Efficient**: Fine-tune only a tiny fraction of the total parameters (often < 0.1%).\r\n- **Autonomous Routing**: Routers learn automatically how many and which neurons to activate, guided by a sparsity-regularized loss.\r\n- **Deep Integration**: Comes with a custom `SparrowTrainer` for seamless integration with the Hugging Face ecosystem.\r\n- **Insightful Reporting**: Includes utilities to generate reports on neuron activation and model sparsity.\r\n\r\n### Installation\r\n```bash\r\npip install sparrow-lib\r\n```\r\n\r\n### Quickstart\r\n\r\nHere's how to fine-tune a BERT model by only training the routers:\r\n```python\r\nfrom transformers import BertForSequenceClassification, AutoTokenizer, TrainingArguments\r\nfrom sparrow import SparrowConfig, add_routers_to_model, SparrowTrainer, get_sparsity_report\r\n\r\n# 1. Define your Sparrow configuration\r\nsparrow_config = SparrowConfig(sparsity_lambda=0.01)\r\n\r\n# 2. Load a pre-trained model and augment it with routers\r\nmodel = BertForSequenceClassification.from_pretrained('bert-base-uncased')\r\nmodel = add_routers_to_model(model, config=sparrow_config)\r\n\r\n# 3. Freeze original model weights (only routers will be trainable)\r\nfor name, param in model.named_parameters():\r\n if 'router' not in name:\r\n param.requires_grad = False\r\n\r\n# ... Load your dataset ...\r\n\r\n# 4. Use the custom SparrowTrainer which handles the sparsity loss\r\ntraining_args = TrainingArguments(output_dir=\"test_trainer\", num_train_epochs=1)\r\n\r\ntrainer = SparrowTrainer(\r\n model=model,\r\n args=training_args,\r\n sparrow_config=sparrow_config,\r\n train_dataset=your_train_dataset,\r\n eval_dataset=your_eval_dataset\r\n)\r\n\r\n# 5. Start training!\r\ntrainer.train()\r\n\r\n# 6. Get a report on how sparse your model became\r\ntrainer.evaluate()\r\nreport = get_sparsity_report(model)\r\nprint(\"\\n--- Sparsity Report ---\")\r\nprint(report)\r\n```\r\n\r\n---\r\n\r\n## \u0641\u0627\u0631\u0633\u06cc\r\n\r\n`Sparrow` \ud83d\udc26 \u06cc\u06a9 \u06a9\u062a\u0627\u0628\u062e\u0627\u0646\u0647 \u0633\u0628\u06a9 \u067e\u0627\u06cc\u062a\u0648\u0646 \u0627\u0633\u062a \u06a9\u0647 \u0628\u0647 \u0631\u0627\u062d\u062a\u06cc \u0628\u0627 `transformers` \u0647\u0627\u06af\u06cc\u0646\u06af \u0641\u06cc\u0633 \u0627\u062f\u063a\u0627\u0645 \u0645\u06cc\u200c\u0634\u0648\u062f \u062a\u0627 \u0645\u062f\u0644\u200c\u0647\u0627\u06cc \u0632\u0628\u0627\u0646 \u0628\u0632\u0631\u06af (LLM) \u0634\u0645\u0627 \u0631\u0627 \u0647\u0648\u0634\u0645\u0646\u062f\u062a\u0631 \u0648 \u06a9\u0627\u0631\u0622\u0645\u062f\u062a\u0631 \u06a9\u0646\u062f. \u0627\u06cc\u0646 \u06a9\u062a\u0627\u0628\u062e\u0627\u0646\u0647 \u0645\u062f\u0644\u200c\u0647\u0627\u06cc \u0627\u0632 \u067e\u06cc\u0634 \u0622\u0645\u0648\u0632\u0634\u200c\u062f\u06cc\u062f\u0647 \u0631\u0627 \u0628\u0627 **\u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u200c\u0647\u0627\u06cc \u062f\u06cc\u0646\u0627\u0645\u06cc\u06a9 \u0648 \u0642\u0627\u0628\u0644 \u0622\u0645\u0648\u0632\u0634** \u062a\u0642\u0648\u06cc\u062a \u0645\u06cc\u200c\u06a9\u0646\u062f \u06a9\u0647 \u06cc\u0627\u062f \u0645\u06cc\u200c\u06af\u06cc\u0631\u0646\u062f \u0645\u0633\u06cc\u0631\u0647\u0627\u06cc \u0645\u062d\u0627\u0633\u0628\u0627\u062a\u06cc \u067e\u0631\u0627\u06a9\u0646\u062f\u0647 (sparse) \u062f\u0631 \u0634\u0628\u06a9\u0647 \u0627\u06cc\u062c\u0627\u062f \u06a9\u0646\u0646\u062f.\r\n\r\n\u0627\u06cc\u0646 \u0631\u0648\u0634 \u0627\u0645\u06a9\u0627\u0646 \u062a\u0646\u0638\u06cc\u0645 \u062f\u0642\u06cc\u0642 \u0628\u0647\u06cc\u0646\u0647 \u0627\u0632 \u0646\u0638\u0631 \u067e\u0627\u0631\u0627\u0645\u062a\u0631 (PEFT) \u0631\u0627 \u0641\u0631\u0627\u0647\u0645 \u0645\u06cc\u200c\u06a9\u0646\u062f\u060c \u0628\u0647 \u0637\u0648\u0631\u06cc \u06a9\u0647 \u0634\u0645\u0627 \u0648\u0632\u0646\u200c\u0647\u0627\u06cc \u0645\u062f\u0644 \u0627\u0635\u0644\u06cc \u0631\u0627 \u0645\u0646\u062c\u0645\u062f \u06a9\u0631\u062f\u0647 \u0648 \u0641\u0642\u0637 \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u200c\u0647\u0627\u06cc \u06a9\u0648\u0686\u06a9 \u0631\u0627 \u0622\u0645\u0648\u0632\u0634 \u0645\u06cc\u200c\u062f\u0647\u06cc\u062f. \u0627\u06cc\u0646 \u06a9\u0627\u0631 \u0628\u0627\u0639\u062b \u0635\u0631\u0641\u0647\u200c\u062c\u0648\u06cc\u06cc \u0639\u0638\u06cc\u0645 \u062f\u0631 \u0645\u0646\u0627\u0628\u0639 \u0645\u062d\u0627\u0633\u0628\u0627\u062a\u06cc \u0634\u062f\u0647 \u0648 \u0627\u0632 \u0641\u0631\u0627\u0645\u0648\u0634\u06cc \u0641\u0627\u062c\u0639\u0647\u200c\u0628\u0627\u0631 (catastrophic forgetting) \u062c\u0644\u0648\u06af\u06cc\u0631\u06cc \u0645\u06cc\u200c\u06a9\u0646\u062f.\r\n\r\n### \u0642\u0627\u0628\u0644\u06cc\u062a\u200c\u0647\u0627\u06cc \u06a9\u0644\u06cc\u062f\u06cc\r\n- **\u0627\u0633\u062a\u0641\u0627\u062f\u0647 \u0622\u0633\u0627\u0646**: \u06cc\u06a9 \u0645\u062f\u0644 \u0627\u0633\u062a\u0627\u0646\u062f\u0627\u0631\u062f \u0647\u0627\u06af\u06cc\u0646\u06af \u0641\u06cc\u0633 \u0631\u0627 \u0628\u0627 \u06cc\u06a9 \u062e\u0637 \u06a9\u062f \u0628\u0647 \u06cc\u06a9 \u0645\u062f\u0644 \u062f\u06cc\u0646\u0627\u0645\u06cc\u06a9 \u0648 \u067e\u0631\u0627\u06a9\u0646\u062f\u0647 \u062a\u0628\u062f\u06cc\u0644 \u06a9\u0646\u06cc\u062f.\r\n- **\u0628\u0647\u06cc\u0646\u0647 \u0627\u0632 \u0646\u0638\u0631 \u067e\u0627\u0631\u0627\u0645\u062a\u0631**: \u0641\u0642\u0637 \u0628\u062e\u0634 \u0628\u0633\u06cc\u0627\u0631 \u06a9\u0648\u0686\u06a9\u06cc \u0627\u0632 \u067e\u0627\u0631\u0627\u0645\u062a\u0631\u0647\u0627\u06cc \u06a9\u0644 \u0645\u062f\u0644 (\u0645\u0639\u0645\u0648\u0644\u0627\u064b \u06a9\u0645\u062a\u0631 \u0627\u0632 \u06f0.\u06f1\u066a) \u0631\u0627 \u0622\u0645\u0648\u0632\u0634 \u062f\u0647\u06cc\u062f.\r\n- **\u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u06cc \u062e\u0648\u062f\u06a9\u0627\u0631**: \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u200c\u0647\u0627 \u0628\u0647 \u0637\u0648\u0631 \u062e\u0648\u062f\u06a9\u0627\u0631 \u0648 \u0628\u0627 \u0647\u062f\u0627\u06cc\u062a \u06cc\u06a9 \u062a\u0627\u0628\u0639 \u0632\u06cc\u0627\u0646 \u0645\u0628\u062a\u0646\u06cc \u0628\u0631 \u067e\u0631\u0627\u06a9\u0646\u062f\u06af\u06cc\u060c \u06cc\u0627\u062f \u0645\u06cc\u200c\u06af\u06cc\u0631\u0646\u062f \u06a9\u0647 \u0686\u0647 \u062a\u0639\u062f\u0627\u062f \u0648 \u06a9\u062f\u0627\u0645 \u0646\u0648\u0631\u0648\u0646\u200c\u0647\u0627 \u0631\u0627 \u0641\u0639\u0627\u0644 \u06a9\u0646\u0646\u062f.\r\n- **\u0627\u062f\u063a\u0627\u0645 \u0639\u0645\u06cc\u0642**: \u0634\u0627\u0645\u0644 \u06cc\u06a9 `SparrowTrainer` \u0633\u0641\u0627\u0631\u0634\u06cc \u0628\u0631\u0627\u06cc \u0627\u062f\u063a\u0627\u0645 \u06a9\u0627\u0645\u0644 \u0628\u0627 \u0627\u06a9\u0648\u0633\u06cc\u0633\u062a\u0645 \u0647\u0627\u06af\u06cc\u0646\u06af \u0641\u06cc\u0633.\r\n- **\u06af\u0632\u0627\u0631\u0634\u200c\u06af\u06cc\u0631\u06cc \u062f\u0642\u06cc\u0642**: \u0634\u0627\u0645\u0644 \u0627\u0628\u0632\u0627\u0631\u0647\u0627\u06cc\u06cc \u0628\u0631\u0627\u06cc \u062a\u0648\u0644\u06cc\u062f \u06af\u0632\u0627\u0631\u0634 \u0627\u0632 \u0645\u06cc\u0632\u0627\u0646 \u0641\u0639\u0627\u0644\u06cc\u062a \u0646\u0648\u0631\u0648\u0646\u200c\u0647\u0627 \u0648 \u067e\u0631\u0627\u06a9\u0646\u062f\u06af\u06cc \u0645\u062f\u0644.\r\n\r\n### \u0646\u0635\u0628\r\n```bash\r\npip install sparrow-lib\r\n```\r\n### \u0634\u0631\u0648\u0639 \u0633\u0631\u06cc\u0639\r\n\u0645\u062b\u0627\u0644 \u0628\u0627\u0644\u0627 \u0646\u062d\u0648\u0647 \u062a\u0646\u0638\u06cc\u0645 \u062f\u0642\u06cc\u0642 \u0645\u062f\u0644 BERT \u0631\u0627 \u0641\u0642\u0637 \u0628\u0627 \u0622\u0645\u0648\u0632\u0634 \u0645\u0633\u06cc\u0631\u06cc\u0627\u0628\u200c\u0647\u0627 \u0646\u0634\u0627\u0646 \u0645\u06cc\u200c\u062f\u0647\u062f.\r\n",
"bugtrack_url": null,
"license": null,
"summary": "A library to augment LLMs with dynamic, trainable routers for efficient fine-tuning.",
"version": "2.0.4",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "1602436152aff39bd1db76ea8c228f7b08387858b007b60a35b6ca7dad579718",
"md5": "647e7d99efc7013094f9e0f3ac67ba53",
"sha256": "029d475342d39fb74ce0256e636becfd9f30626aff8c8daa0a76a6a52684564f"
},
"downloads": -1,
"filename": "sparrow_lib-2.0.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "647e7d99efc7013094f9e0f3ac67ba53",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 10827,
"upload_time": "2025-09-06T17:29:57",
"upload_time_iso_8601": "2025-09-06T17:29:57.032600Z",
"url": "https://files.pythonhosted.org/packages/16/02/436152aff39bd1db76ea8c228f7b08387858b007b60a35b6ca7dad579718/sparrow_lib-2.0.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "06237f449c4a53379052ac200989cce267d5f1e0d2f2fce26597a5fc1809dee0",
"md5": "1618a2a34844d046cd22f4234c2d4163",
"sha256": "e5abde5f06ba30a05b80e43595a3226df1cdd180d1a5ad81917816b333ec6616"
},
"downloads": -1,
"filename": "sparrow_lib-2.0.4.tar.gz",
"has_sig": false,
"md5_digest": "1618a2a34844d046cd22f4234c2d4163",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 10825,
"upload_time": "2025-09-06T17:29:59",
"upload_time_iso_8601": "2025-09-06T17:29:59.922715Z",
"url": "https://files.pythonhosted.org/packages/06/23/7f449c4a53379052ac200989cce267d5f1e0d2f2fce26597a5fc1809dee0/sparrow_lib-2.0.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-06 17:29:59",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "sparrow-lib"
}
