# Solv AI
**Solv AI** is a high-performance AI model optimization library designed to enhance the efficiency and speed of running large-scale models on local, resource-constrained hardware. By leveraging advanced techniques such as custom quantization, pruning, mixed precision inference, and efficient memory management, Solv AI enables seamless deployment and inference of sophisticated AI models even on slower machines.
## Key Features
- **Custom Quantization:** Reduce model precision for faster computation without significant loss of accuracy.
- **Pruning:** Remove less important parameters to reduce model size and computational load.
- **Layer-wise Loading:** Load model layers incrementally to optimize memory usage.
- **Mixed Precision Inference:** Utilize both 16-bit and 32-bit operations to accelerate processing.
- **Efficient Memory Management:** Advanced techniques to reuse memory allocations and minimize overhead.
- **Lightweight Architectures:** Optimized model architectures for resource efficiency.
- **Layer Fusion:** Combine multiple layers into a single operation to reduce memory access and improve computational efficiency.
- **Batch Normalization Folding:** Fold batch normalization into preceding convolution layers to reduce the number of operations.
- **Weight Sharing:** Share weights across different layers to reduce the model size.
- **Knowledge Distillation:** Use a smaller "student" model trained to mimic a larger "teacher" model.
- **Dynamic Quantization:** Apply quantization dynamically during inference to adapt to different input data distributions.
- **Distributed Computing:** Utilize multiple devices to accelerate model training and inference.
## Installation
To install Solv AI, use pip:
```bash
pip install solv-ai
```
## Usage
### Custom Quantization
```python
from solv_ai import QuantizedLayer
import numpy as np
weights = np.random.rand(5, 10)
quant_layer = QuantizedLayer(weights, bits=8)
x = np.random.rand(3, 10)
output = quant_layer.forward(x)
print("Quantized output:")
print(output)
```
### Pruning
```python
from solv_ai import PrunedLayer
import numpy as np
weights = np.random.rand(5, 10)
prune_layer = PrunedLayer(weights, pruning_rate=0.5)
x = np.random.rand(3, 10)
output = prune_layer.forward(x)
print("Pruned output:")
print(output)
```
### Mixed Precision Inference
```python
import torch
import torch.nn as nn
import torch.optim as optim
from solv_ai import MixedPrecisionLayer
class SimpleModel(nn.Module):
def init(self):
super(SimpleModel, self).init()
self.fc = nn.Linear(10, 5)
def forward(self, x):
return self.fc(x)
model = SimpleModel()
optimizer = optim.Adam(model.parameters(), lr=0.001)
mp_layer = MixedPrecisionLayer(model, optimizer)
x = torch.randn(3, 10)
output = mp_layer.forward(x)
loss = output.sum()
mp_layer.backward(loss)
```
### Efficient Models
```python
from solv_ai import EfficientNet
import torch
model = EfficientNet()
x = torch.randn(3, 3, 224, 224)
output = model(x)
print("EfficientNet output:")
print(output)
```
### Distributed Computing
```python
import torch
import torch.nn as nn
from solv_ai import ModelParallelLayer, DataParallelLayer, PipelineParallelLayer
class SimpleModel(nn.Module):
def init(self):
super(SimpleModel, self).init()
self.fc = nn.Linear(10, 5)
def forward(self, x):
return self.fc(x)
model = SimpleModel()
device_ids = [0, 1]
mp_layer = ModelParallelLayer(model, device_ids)
dp_layer = DataParallelLayer(model, device_ids)
pipeline_layer = PipelineParallelLayer([model], chunks=2)
x = torch.randn(3, 10)
output_mp = mp_layer(x)
output_dp = dp_layer(x)
output_pipeline = pipeline_layer(x)
print("Model Parallel output:")
print(output_mp)
print("Data Parallel output:")
print(output_dp)
print("Pipeline Parallel output:")
print(output_pipeline)
```
### Profiling
```python
from solv_ai import profile_model
import torch.nn as nn
class SimpleModel(nn.Module):
def init(self):
super(SimpleModel, self).init()
self.fc = nn.Linear(10, 5)
def forward(self, x):
return self.fc(x)
model = SimpleModel()
profile_model(model, (3, 10))
```
## License
Solv AI is licensed under the MIT License. See the [LICENSE](LICENSE) file for more details.
Raw data
{
"_id": null,
"home_page": "https://github.com/CloudSolv-AI/solv-ai",
"name": "solv-ai",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": null,
"author": "CloudSolv AI",
"author_email": "eric@cloudsolv.co",
"download_url": "https://files.pythonhosted.org/packages/1c/21/8cf7e4479bb9ce572de3cdbcc2afb2b16755c4d9c681fa2a6bd85f99ec30/solv_ai-0.2.5.tar.gz",
"platform": null,
"description": "# Solv AI\r\n\r\n**Solv AI** is a high-performance AI model optimization library designed to enhance the efficiency and speed of running large-scale models on local, resource-constrained hardware. By leveraging advanced techniques such as custom quantization, pruning, mixed precision inference, and efficient memory management, Solv AI enables seamless deployment and inference of sophisticated AI models even on slower machines.\r\n\r\n## Key Features\r\n\r\n- **Custom Quantization:** Reduce model precision for faster computation without significant loss of accuracy.\r\n- **Pruning:** Remove less important parameters to reduce model size and computational load.\r\n- **Layer-wise Loading:** Load model layers incrementally to optimize memory usage.\r\n- **Mixed Precision Inference:** Utilize both 16-bit and 32-bit operations to accelerate processing.\r\n- **Efficient Memory Management:** Advanced techniques to reuse memory allocations and minimize overhead.\r\n- **Lightweight Architectures:** Optimized model architectures for resource efficiency.\r\n- **Layer Fusion:** Combine multiple layers into a single operation to reduce memory access and improve computational efficiency.\r\n- **Batch Normalization Folding:** Fold batch normalization into preceding convolution layers to reduce the number of operations.\r\n- **Weight Sharing:** Share weights across different layers to reduce the model size.\r\n- **Knowledge Distillation:** Use a smaller \"student\" model trained to mimic a larger \"teacher\" model.\r\n- **Dynamic Quantization:** Apply quantization dynamically during inference to adapt to different input data distributions.\r\n- **Distributed Computing:** Utilize multiple devices to accelerate model training and inference.\r\n\r\n## Installation\r\n\r\nTo install Solv AI, use pip:\r\n```bash\r\npip install solv-ai\r\n```\r\n\r\n## Usage\r\n\r\n### Custom Quantization\r\n```python\r\nfrom solv_ai import QuantizedLayer\r\nimport numpy as np\r\nweights = np.random.rand(5, 10)\r\nquant_layer = QuantizedLayer(weights, bits=8)\r\nx = np.random.rand(3, 10)\r\noutput = quant_layer.forward(x)\r\nprint(\"Quantized output:\")\r\nprint(output)\r\n```\r\n\r\n### Pruning\r\n```python\r\nfrom solv_ai import PrunedLayer\r\nimport numpy as np\r\nweights = np.random.rand(5, 10)\r\nprune_layer = PrunedLayer(weights, pruning_rate=0.5)\r\nx = np.random.rand(3, 10)\r\noutput = prune_layer.forward(x)\r\nprint(\"Pruned output:\")\r\nprint(output)\r\n```\r\n\r\n### Mixed Precision Inference\r\n```python\r\nimport torch\r\nimport torch.nn as nn\r\nimport torch.optim as optim\r\nfrom solv_ai import MixedPrecisionLayer\r\nclass SimpleModel(nn.Module):\r\ndef init(self):\r\nsuper(SimpleModel, self).init()\r\nself.fc = nn.Linear(10, 5)\r\ndef forward(self, x):\r\nreturn self.fc(x)\r\nmodel = SimpleModel()\r\noptimizer = optim.Adam(model.parameters(), lr=0.001)\r\nmp_layer = MixedPrecisionLayer(model, optimizer)\r\nx = torch.randn(3, 10)\r\noutput = mp_layer.forward(x)\r\nloss = output.sum()\r\nmp_layer.backward(loss)\r\n```\r\n\r\n### Efficient Models\r\n```python\r\nfrom solv_ai import EfficientNet\r\nimport torch\r\nmodel = EfficientNet()\r\nx = torch.randn(3, 3, 224, 224)\r\noutput = model(x)\r\nprint(\"EfficientNet output:\")\r\nprint(output)\r\n```\r\n\r\n### Distributed Computing\r\n```python\r\nimport torch\r\nimport torch.nn as nn\r\nfrom solv_ai import ModelParallelLayer, DataParallelLayer, PipelineParallelLayer\r\nclass SimpleModel(nn.Module):\r\ndef init(self):\r\nsuper(SimpleModel, self).init()\r\nself.fc = nn.Linear(10, 5)\r\ndef forward(self, x):\r\nreturn self.fc(x)\r\nmodel = SimpleModel()\r\ndevice_ids = [0, 1]\r\nmp_layer = ModelParallelLayer(model, device_ids)\r\ndp_layer = DataParallelLayer(model, device_ids)\r\npipeline_layer = PipelineParallelLayer([model], chunks=2)\r\nx = torch.randn(3, 10)\r\noutput_mp = mp_layer(x)\r\noutput_dp = dp_layer(x)\r\noutput_pipeline = pipeline_layer(x)\r\nprint(\"Model Parallel output:\")\r\nprint(output_mp)\r\nprint(\"Data Parallel output:\")\r\nprint(output_dp)\r\nprint(\"Pipeline Parallel output:\")\r\nprint(output_pipeline)\r\n```\r\n\r\n### Profiling\r\n```python\r\nfrom solv_ai import profile_model\r\nimport torch.nn as nn\r\nclass SimpleModel(nn.Module):\r\ndef init(self):\r\nsuper(SimpleModel, self).init()\r\nself.fc = nn.Linear(10, 5)\r\ndef forward(self, x):\r\nreturn self.fc(x)\r\nmodel = SimpleModel()\r\nprofile_model(model, (3, 10))\r\n```\r\n\r\n## License\r\n\r\nSolv AI is licensed under the MIT License. See the [LICENSE](LICENSE) file for more details.\r\n",
"bugtrack_url": null,
"license": null,
"summary": "A high-performance AI model optimization library",
"version": "0.2.5",
"project_urls": {
"Homepage": "https://github.com/CloudSolv-AI/solv-ai"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7147aba606fd4309807d75ebbbabda98c76f8b4778fcd6978324655138a520e4",
"md5": "c23b377a7b0110d8f442efa6d0f3a527",
"sha256": "881e10b472973f66fd9679277933ba062aebb948888fe4850473c8007d4d7f99"
},
"downloads": -1,
"filename": "solv_ai-0.2.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c23b377a7b0110d8f442efa6d0f3a527",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 11036,
"upload_time": "2024-08-07T16:31:19",
"upload_time_iso_8601": "2024-08-07T16:31:19.586553Z",
"url": "https://files.pythonhosted.org/packages/71/47/aba606fd4309807d75ebbbabda98c76f8b4778fcd6978324655138a520e4/solv_ai-0.2.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1c218cf7e4479bb9ce572de3cdbcc2afb2b16755c4d9c681fa2a6bd85f99ec30",
"md5": "aa673b352b30b255226d49564952f7da",
"sha256": "b32ca713aa892c72f5c259ab71998e3f2b341d454c82daa12922dd1a9b44fadf"
},
"downloads": -1,
"filename": "solv_ai-0.2.5.tar.gz",
"has_sig": false,
"md5_digest": "aa673b352b30b255226d49564952f7da",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 8977,
"upload_time": "2024-08-07T16:31:20",
"upload_time_iso_8601": "2024-08-07T16:31:20.950781Z",
"url": "https://files.pythonhosted.org/packages/1c/21/8cf7e4479bb9ce572de3cdbcc2afb2b16755c4d9c681fa2a6bd85f99ec30/solv_ai-0.2.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-07 16:31:20",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "CloudSolv-AI",
"github_project": "solv-ai",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "solv-ai"
}
JSON
