<!--ts-->
* [AI Edge Torch](#ai-edge-torch)
* [PyTorch Converter](#pytorch-converter)
* [Generative API](#generative-api)
* [Build Status](#build-status)
* [Installation](#installation)
* [Requirements and Dependencies](#requirements-and-dependencies)
* [Python Virtual Env](#python-virtual-env)
* [Contributing](#contributing)
* [Getting Help](#getting-help)
<!-- Created by https://github.com/ekalinin/github-markdown-toc -->
<!-- Added by: advaitjain, at: Sun May 12 12:00:01 AM PDT 2024 -->
<!--te-->
# AI Edge Torch
This repository has two primary components
1. PyTorch Converter:
* Provides direct path from PyTorch to [TFLite runtime](https://www.tensorflow.org/lite/guide/inference)
with great model coverage and CPU performance.
* This is currently in Beta with planned improvements for hardware
acceleration, quantization, and ease-of-use.
1. Generative API:
* Python library that allows developers to author high performance LLMs for
Mobile devices in PyTorch. These models are then converted via the PyTorch
Converter to run on devices using the TFLite runtime.
* This is currently in early Alpha with planned improvements for hardware
acceleration, quantization, and ease-of-use.
## PyTorch Converter
Here are the steps needed to convert a PyTorch model to a TFLite flatbuffer:
```python
import torch
import torchvision
import ai_edge_torch
# Use resnet18 with pre-trained weights.
resnet18 = torchvision.models.resnet18(torchvision.models.ResNet18_Weights.IMAGENET1K_V1)
sample_inputs = (torch.randn(1, 3, 224, 224),)
# Convert and serialize PyTorch model to a tflite flatbuffer. Note that we
# are setting the model to evaluation mode prior to conversion.
edge_model = ai_edge_torch.convert(resnet18.eval(), sample_inputs)
edge_model.export("resnet18.tflite")
```
The [getting started](docs/pytorch_converter/getting_started.ipynb) Jupyter
notebook gives an initial walkthrough of the conversion process and can be tried
out with Google Colab.
More details are [here](docs/pytorch_converter/README.md).
## Generative API
The AI Edge Torch Generative API is a Torch native library for authoring mobile-optimzed PyTorch Transformer models, which can be converted to TFLite, allowing users to easily deploy Large Language Models (LLMs) on mobile devices. Users can convert the models using the [PyTorch Converter](docs/pytorch_converter/README.md), and run them via the TensorFlow Lite runtime. See [here](ai_edge_torch/generative/examples/c%2B%2B).
Mobile app developers can also use the Edge Generative API to integrate PyTorch LLMs directly with the MediaPipe LLM Inference API for easy integration within their application code. See [here](http://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference#ai_edge_model_conversion).
The AI Edge Torch Generative API is currently CPU-only, with planned support for GPU and NPU.
More detailed documentation can be found [here](ai_edge_torch/generative).
## Build Status
Build Type | Status |
----------- | --------------|
Generative API (Linux) | [](https://github.com/google-ai-edge/ai-edge-torch/actions/workflows/nightly_generative_api.yml) |
Model Coverage (Linux) | [](https://github.com/google-ai-edge/ai-edge-torch/actions/workflows/nightly_model_coverage.yml) |
Unit tests (Linux) | [](https://github.com/google-ai-edge/ai-edge-torch/actions/workflows/nightly_unittests.yml) |
## Installation
### Requirements and Dependencies
* Python versions: 3.9, 3.10, 3.11
* Operating system: Linux
* PyTorch: 
* TensorFlow: 
<!-- requirement badges are updated by ci/update_nightly_versions.py -->
### Python Virtual Env
Set up a Python virtualenv:
```bash
python -m venv --prompt ai-edge-torch venv
source venv/bin/activate
```
A specific release (for example 0.1.0) can be installed with:
```bash
pip install -r https://github.com/google-ai-edge/ai-edge-torch/releases/tag/v0.1.0/requirements.txt
pip install ai-edge-torch==0.1.0
```
Alternately, the nightly version can be installed with:
```bash
pip install -r https://github.com/google-ai-edge/ai-edge-torch/requirements.txt
pip install ai-edge-torch
```
* The list of versioned releases can be seen [here](https://github.com/google-ai-edge/ai-edge-torch/releases).
* The full list of PyPi releases (including nightly builds) can be seen [here](https://pypi.org/project/ai-edge-torch/#history).
# Contributing
See our [contribution documentation](CONTRIBUTING.md).
# Getting Help
Please [create a GitHub issue](https://github.com/google-ai-edge/ai-edge-torch/issues/new/choose) with any questions.
Raw data
{
"_id": null,
"home_page": "https://github.com/google-ai-edge/ai-edge-torch",
"name": "ai-edge-torch",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.12,>=3.9",
"maintainer_email": null,
"keywords": "On-Device ML, AI, SDK, Google, Tensorflow-Lite, PyTorch, LLMs, Gen-AI",
"author": null,
"author_email": null,
"download_url": null,
"platform": null,
"description": "<!--ts-->\n* [AI Edge Torch](#ai-edge-torch)\n * [PyTorch Converter](#pytorch-converter)\n * [Generative API](#generative-api)\n * [Build Status](#build-status)\n * [Installation](#installation)\n * [Requirements and Dependencies](#requirements-and-dependencies)\n * [Python Virtual Env](#python-virtual-env)\n* [Contributing](#contributing)\n* [Getting Help](#getting-help)\n\n<!-- Created by https://github.com/ekalinin/github-markdown-toc -->\n<!-- Added by: advaitjain, at: Sun May 12 12:00:01 AM PDT 2024 -->\n\n<!--te-->\n\n\n# AI Edge Torch\n\nThis repository has two primary components\n\n1. PyTorch Converter:\n * Provides direct path from PyTorch to [TFLite runtime](https://www.tensorflow.org/lite/guide/inference)\n with great model coverage and CPU performance.\n * This is currently in Beta with planned improvements for hardware\n acceleration, quantization, and ease-of-use.\n\n1. Generative API:\n * Python library that allows developers to author high performance LLMs for\n Mobile devices in PyTorch. These models are then converted via the PyTorch\n Converter to run on devices using the TFLite runtime.\n * This is currently in early Alpha with planned improvements for hardware\n acceleration, quantization, and ease-of-use.\n\n## PyTorch Converter\nHere are the steps needed to convert a PyTorch model to a TFLite flatbuffer:\n\n```python\nimport torch\nimport torchvision\nimport ai_edge_torch\n\n# Use resnet18 with pre-trained weights.\nresnet18 = torchvision.models.resnet18(torchvision.models.ResNet18_Weights.IMAGENET1K_V1)\nsample_inputs = (torch.randn(1, 3, 224, 224),)\n\n# Convert and serialize PyTorch model to a tflite flatbuffer. Note that we\n# are setting the model to evaluation mode prior to conversion.\nedge_model = ai_edge_torch.convert(resnet18.eval(), sample_inputs)\nedge_model.export(\"resnet18.tflite\")\n```\n\nThe [getting started](docs/pytorch_converter/getting_started.ipynb) Jupyter\nnotebook gives an initial walkthrough of the conversion process and can be tried\nout with Google Colab.\n\nMore details are [here](docs/pytorch_converter/README.md).\n\n## Generative API\nThe AI Edge Torch Generative API is a Torch native library for authoring mobile-optimzed PyTorch Transformer models, which can be converted to TFLite, allowing users to easily deploy Large Language Models (LLMs) on mobile devices. Users can convert the models using the [PyTorch Converter](docs/pytorch_converter/README.md), and run them via the TensorFlow Lite runtime. See [here](ai_edge_torch/generative/examples/c%2B%2B).\n\nMobile app developers can also use the Edge Generative API to integrate PyTorch LLMs directly with the MediaPipe LLM Inference API for easy integration within their application code. See [here](http://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference#ai_edge_model_conversion).\n\nThe AI Edge Torch Generative API is currently CPU-only, with planned support for GPU and NPU.\n\nMore detailed documentation can be found [here](ai_edge_torch/generative).\n\n## Build Status\n\nBuild Type | Status |\n----------- | --------------|\nGenerative API (Linux) | [](https://github.com/google-ai-edge/ai-edge-torch/actions/workflows/nightly_generative_api.yml) |\nModel Coverage (Linux) | [](https://github.com/google-ai-edge/ai-edge-torch/actions/workflows/nightly_model_coverage.yml) |\nUnit tests (Linux) | [](https://github.com/google-ai-edge/ai-edge-torch/actions/workflows/nightly_unittests.yml) |\n\n\n## Installation\n\n### Requirements and Dependencies\n\n * Python versions: 3.9, 3.10, 3.11\n * Operating system: Linux\n * PyTorch: \n * TensorFlow: \n\n<!-- requirement badges are updated by ci/update_nightly_versions.py -->\n\n### Python Virtual Env\n\nSet up a Python virtualenv:\n```bash\npython -m venv --prompt ai-edge-torch venv\nsource venv/bin/activate\n```\n\nA specific release (for example 0.1.0) can be installed with:\n```bash\npip install -r https://github.com/google-ai-edge/ai-edge-torch/releases/tag/v0.1.0/requirements.txt\npip install ai-edge-torch==0.1.0\n```\n\nAlternately, the nightly version can be installed with:\n```bash\npip install -r https://github.com/google-ai-edge/ai-edge-torch/requirements.txt\npip install ai-edge-torch\n```\n\n* The list of versioned releases can be seen [here](https://github.com/google-ai-edge/ai-edge-torch/releases).\n* The full list of PyPi releases (including nightly builds) can be seen [here](https://pypi.org/project/ai-edge-torch/#history).\n\n\n# Contributing\n\nSee our [contribution documentation](CONTRIBUTING.md).\n\n# Getting Help\n\nPlease [create a GitHub issue](https://github.com/google-ai-edge/ai-edge-torch/issues/new/choose) with any questions.\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Enables preparing PyTorch models for performant on-device execution.",
"version": "0.1.0",
"project_urls": {
"Homepage": "https://github.com/google-ai-edge/ai-edge-torch"
},
"split_keywords": [
"on-device ml",
" ai",
" sdk",
" google",
" tensorflow-lite",
" pytorch",
" llms",
" gen-ai"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "fb5059be218d082a62df769e97cca73c3515b2bfc6b0a771d265ab075624bf3b",
"md5": "e9f59fa53a835a6e80e45d7da211803f",
"sha256": "15c25b2c00cb8151019ade5dbe5ca4885439e236d26fd57e0b5e868c04f2dfce"
},
"downloads": -1,
"filename": "ai_edge_torch-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e9f59fa53a835a6e80e45d7da211803f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.12,>=3.9",
"size": 139907,
"upload_time": "2024-05-13T17:45:37",
"upload_time_iso_8601": "2024-05-13T17:45:37.157693Z",
"url": "https://files.pythonhosted.org/packages/fb/50/59be218d082a62df769e97cca73c3515b2bfc6b0a771d265ab075624bf3b/ai_edge_torch-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-13 17:45:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "google-ai-edge",
"github_project": "ai-edge-torch",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "ai-edge-torch"
}
JSON
