[](https://github.com/huggingface/optimum-graphcore/actions/workflows/test-examples.yml) [](https://github.com/huggingface/optimum-graphcore/actions/workflows/test-pipelines.yml)
<p align="center">
<img src="readme_logo.png" />
</p>
# Optimum Graphcore
🤗 Optimum Graphcore is the interface between the 🤗 Transformers library and [Graphcore IPUs](https://www.graphcore.ai/products/ipu).
It provides a set of tools enabling model parallelization and loading on IPUs, training, fine-tuning and inference on all the tasks already supported by 🤗 Transformers while being compatible with the 🤗 Hub and every model available on it out of the box.
## What is an Intelligence Processing Unit (IPU)?
Quote from the Hugging Face [blog post](https://huggingface.co/blog/graphcore#what-is-an-intelligence-processing-unit):
>IPUs are the processors that power Graphcore’s IPU-POD datacenter compute systems. This new type of processor is designed to support the very specific computational requirements of AI and machine learning. Characteristics such as fine-grained parallelism, low precision arithmetic, and the ability to handle sparsity have been built into our silicon.
> Instead of adopting a SIMD/SIMT architecture like GPUs, Graphcore’s IPU uses a massively parallel, MIMD architecture, with ultra-high bandwidth memory placed adjacent to the processor cores, right on the silicon die.
> This design delivers high performance and new levels of efficiency, whether running today’s most popular models, such as BERT and EfficientNet, or exploring next-generation AI applications.
## Poplar SDK setup
A Poplar SDK environment needs to be enabled to use this library. Please refer to Graphcore's [Getting Started](https://docs.graphcore.ai/en/latest/getting-started.html) guides.
## Install
To install the latest release of this package:
`pip install optimum-graphcore`
Optimum Graphcore is a fast-moving project, and you may want to install from source.
`pip install git+https://github.com/huggingface/optimum-graphcore.git`
### Installing in developer mode
If you are working on the `optimum-graphcore` code then you should use an editable install
by cloning and installing `optimum` and `optimum-graphcore`:
```
git clone https://github.com/huggingface/optimum --branch v1.6.1-release
git clone https://github.com/huggingface/optimum-graphcore
pip install -e optimum -e optimum-graphcore
```
Now whenever you change the code, you'll be able to run with those changes instantly.
## Running the examples
There are a number of examples provided in the `examples` directory. Each of these contains a README with command lines for running them on IPUs with Optimum Graphcore.
Please install the requirements for every example:
```
cd <example-folder>
pip install -r requirements.txt
```
## How to use Optimum Graphcore
🤗 Optimum Graphcore was designed with one goal in mind: **make training and evaluation straightforward for any 🤗 Transformers user while leveraging the complete power of IPUs.**
It requires minimal changes if you are already using 🤗 Transformers.
To immediately use a model on a given input (text, image, audio, ...), we support the `pipeline` API:
```diff
->>> from transformers import pipeline
+>>> from optimum.graphcore import pipeline
# Allocate a pipeline for sentiment-analysis
->>> classifier = pipeline('sentiment-analysis', model="distilbert-base-uncased-finetuned-sst-2-english")
+>>> classifier = pipeline('sentiment-analysis', model="distilbert-base-uncased-finetuned-sst-2-english", ipu_config = "Graphcore/distilbert-base-ipu")
>>> classifier('We are very happy to introduce pipeline to the transformers repository.')
[{'label': 'POSITIVE', 'score': 0.9996947050094604}]
```
It is also super easy to use the `Trainer` API:
```diff
-from transformers import Trainer, TrainingArguments
+from optimum.graphcore import IPUConfig, IPUTrainer, IPUTrainingArguments
-training_args = TrainingArguments(
+training_args = IPUTrainingArguments(
per_device_train_batch_size=4,
learning_rate=1e-4,
+ # Any IPUConfig on the Hub or stored locally
+ ipu_config_name="Graphcore/bert-base-ipu",
+)
+
+# Loading the IPUConfig needed by the IPUTrainer to compile and train the model on IPUs
+ipu_config = IPUConfig.from_pretrained(
+ training_args.ipu_config_name,
)
# Initialize our Trainer
-trainer = Trainer(
+trainer = IPUTrainer(
model=model,
+ ipu_config=ipu_config,
args=training_args,
train_dataset=train_dataset if training_args.do_train else None,
... # Other arguments
```
For more information, refer to the full [🤗 Optimum Graphcore documentation](https://huggingface.co/docs/optimum/graphcore_index).
## Supported models
The following model architectures and tasks are currently supported by 🤗 Optimum Graphcore:
| | Pre-Training | Masked LM | Causal LM | Seq2Seq LM (Summarization, Translation, etc) | Sequence Classification | Token Classification | Question Answering | Multiple Choice | Image Classification | CTC |
|------------|--------------|-----------|-----------|----------------------------------------------|-------------------------|----------------------|--------------------|-----------------|----------------------| ------------ |
| BART | ✅ | | ❌ | ✅ | ✅ | | ❌ | | | |
| BERT | ✅ | ✅ | ❌ | | ✅ | ✅ | ✅ | ✅ | | |
| ConvNeXt | ✅ | | | | | | | | ✅ | |
| DeBERTa | ✅ | ✅ | | | ✅ | ✅ | ✅ | | | |
| DistilBERT | ❌ | ✅ | | | ✅ | ✅ | ✅ | ✅ | | |
| GPT-2 | ✅ | | ✅ | | ✅ | ✅ | | | | |
| [GroupBERT](https://arxiv.org/abs/2106.05822) | ✅ | ✅ | ❌ | | ✅ | ✅ | ✅ | ✅ | | |
| HuBERT | ❌ | | | | ✅ | | | | | ✅ |
| LXMERT | ❌ | | | | | | ✅ | | | |
| RoBERTa | ✅ | ✅ | ❌ | | ✅ | ✅ | ✅ | ✅ | | |
| T5 | ✅ | | | ✅ | | | | | | |
| ViT | ❌ | | | | | | | | ✅ | |
| Wav2Vec2 | ✅ | | | | | | | | | ✅ |
| Whisper | ❌ | | | ✅ | | | | | | |
If you find any issue while using those, please open an issue or a pull request.
Raw data
{
"_id": null,
"home_page": "https://huggingface.co/hardware",
"name": "optimum-graphcore",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "transformers,quantization,pruning,training,ipu",
"author": "HuggingFace Inc. Special Ops Team",
"author_email": "hardware@huggingface.co",
"download_url": "https://files.pythonhosted.org/packages/22/da/eac7471e1cb3834da562742fbeab2cbc1703ca115f76e79b835415f5e3e3/optimum-graphcore-0.7.1.tar.gz",
"platform": null,
"description": "[](https://github.com/huggingface/optimum-graphcore/actions/workflows/test-examples.yml) [](https://github.com/huggingface/optimum-graphcore/actions/workflows/test-pipelines.yml)\n\n<p align=\"center\">\n <img src=\"readme_logo.png\" />\n</p>\n\n# Optimum Graphcore\n\n\ud83e\udd17 Optimum Graphcore is the interface between the \ud83e\udd17 Transformers library and [Graphcore IPUs](https://www.graphcore.ai/products/ipu).\nIt provides a set of tools enabling model parallelization and loading on IPUs, training, fine-tuning and inference on all the tasks already supported by \ud83e\udd17 Transformers while being compatible with the \ud83e\udd17 Hub and every model available on it out of the box.\n\n## What is an Intelligence Processing Unit (IPU)?\nQuote from the Hugging Face [blog post](https://huggingface.co/blog/graphcore#what-is-an-intelligence-processing-unit):\n>IPUs are the processors that power Graphcore\u2019s IPU-POD datacenter compute systems. This new type of processor is designed to support the very specific computational requirements of AI and machine learning. Characteristics such as fine-grained parallelism, low precision arithmetic, and the ability to handle sparsity have been built into our silicon.\n\n> Instead of adopting a SIMD/SIMT architecture like GPUs, Graphcore\u2019s IPU uses a massively parallel, MIMD architecture, with ultra-high bandwidth memory placed adjacent to the processor cores, right on the silicon die.\n\n> This design delivers high performance and new levels of efficiency, whether running today\u2019s most popular models, such as BERT and EfficientNet, or exploring next-generation AI applications.\n\n## Poplar SDK setup\nA Poplar SDK environment needs to be enabled to use this library. Please refer to Graphcore's [Getting Started](https://docs.graphcore.ai/en/latest/getting-started.html) guides.\n\n## Install\nTo install the latest release of this package:\n\n`pip install optimum-graphcore`\n\nOptimum Graphcore is a fast-moving project, and you may want to install from source.\n\n`pip install git+https://github.com/huggingface/optimum-graphcore.git`\n\n### Installing in developer mode\n\nIf you are working on the `optimum-graphcore` code then you should use an editable install\nby cloning and installing `optimum` and `optimum-graphcore`:\n\n```\ngit clone https://github.com/huggingface/optimum --branch v1.6.1-release\ngit clone https://github.com/huggingface/optimum-graphcore\npip install -e optimum -e optimum-graphcore\n```\n\nNow whenever you change the code, you'll be able to run with those changes instantly.\n\n\n## Running the examples\n\nThere are a number of examples provided in the `examples` directory. Each of these contains a README with command lines for running them on IPUs with Optimum Graphcore.\n\nPlease install the requirements for every example:\n\n```\ncd <example-folder>\npip install -r requirements.txt\n```\n\n## How to use Optimum Graphcore\n\ud83e\udd17 Optimum Graphcore was designed with one goal in mind: **make training and evaluation straightforward for any \ud83e\udd17 Transformers user while leveraging the complete power of IPUs.**\nIt requires minimal changes if you are already using \ud83e\udd17 Transformers.\n\nTo immediately use a model on a given input (text, image, audio, ...), we support the `pipeline` API:\n\n```diff\n->>> from transformers import pipeline\n+>>> from optimum.graphcore import pipeline\n\n# Allocate a pipeline for sentiment-analysis\n->>> classifier = pipeline('sentiment-analysis', model=\"distilbert-base-uncased-finetuned-sst-2-english\")\n+>>> classifier = pipeline('sentiment-analysis', model=\"distilbert-base-uncased-finetuned-sst-2-english\", ipu_config = \"Graphcore/distilbert-base-ipu\")\n>>> classifier('We are very happy to introduce pipeline to the transformers repository.')\n[{'label': 'POSITIVE', 'score': 0.9996947050094604}]\n```\n\nIt is also super easy to use the `Trainer` API:\n\n```diff\n-from transformers import Trainer, TrainingArguments\n+from optimum.graphcore import IPUConfig, IPUTrainer, IPUTrainingArguments\n\n-training_args = TrainingArguments(\n+training_args = IPUTrainingArguments(\n per_device_train_batch_size=4,\n learning_rate=1e-4,\n+ # Any IPUConfig on the Hub or stored locally\n+ ipu_config_name=\"Graphcore/bert-base-ipu\",\n+)\n+\n+# Loading the IPUConfig needed by the IPUTrainer to compile and train the model on IPUs\n+ipu_config = IPUConfig.from_pretrained(\n+ training_args.ipu_config_name,\n )\n\n # Initialize our Trainer\n-trainer = Trainer(\n+trainer = IPUTrainer(\n model=model,\n+ ipu_config=ipu_config,\n args=training_args,\n train_dataset=train_dataset if training_args.do_train else None,\n ... # Other arguments\n```\n\nFor more information, refer to the full [\ud83e\udd17 Optimum Graphcore documentation](https://huggingface.co/docs/optimum/graphcore_index).\n\n## Supported models\nThe following model architectures and tasks are currently supported by \ud83e\udd17 Optimum Graphcore:\n| | Pre-Training | Masked LM | Causal LM | Seq2Seq LM (Summarization, Translation, etc) | Sequence Classification | Token Classification | Question Answering | Multiple Choice | Image Classification | CTC |\n|------------|--------------|-----------|-----------|----------------------------------------------|-------------------------|----------------------|--------------------|-----------------|----------------------| ------------ |\n| BART | \u2705 | | \u274c | \u2705 | \u2705 | | \u274c | | | |\n| BERT | \u2705 | \u2705 | \u274c | | \u2705 | \u2705 | \u2705 | \u2705 | | |\n| ConvNeXt | \u2705 | | | | | | | | \u2705 | |\n| DeBERTa | \u2705 | \u2705 | | | \u2705 | \u2705 | \u2705 | | | |\n| DistilBERT | \u274c | \u2705 | | | \u2705 | \u2705 | \u2705 | \u2705 | | |\n| GPT-2 | \u2705 | | \u2705 | | \u2705 | \u2705 | | | | |\n| [GroupBERT](https://arxiv.org/abs/2106.05822) | \u2705 | \u2705 | \u274c | | \u2705 | \u2705 | \u2705 | \u2705 | | |\n| HuBERT | \u274c | | | | \u2705 | | | | | \u2705 |\n| LXMERT | \u274c | | | | | | \u2705 | | | |\n| RoBERTa | \u2705 | \u2705 | \u274c | | \u2705 | \u2705 | \u2705 | \u2705 | | |\n| T5 | \u2705 | | | \u2705 | | | | | | |\n| ViT | \u274c | | | | | | | | \u2705 | |\n| Wav2Vec2 | \u2705 | | | | | | | | | \u2705 |\n| Whisper | \u274c | | | \u2705 | | | | | | |\n\n\nIf you find any issue while using those, please open an issue or a pull request.\n",
"bugtrack_url": null,
"license": "Apache",
"summary": "Optimum Library is an extension of the Hugging Face Transformers library, providing a framework to integrate third-party libraries from Hardware Partners and interface with their specific functionality.",
"version": "0.7.1",
"project_urls": {
"Homepage": "https://huggingface.co/hardware"
},
"split_keywords": [
"transformers",
"quantization",
"pruning",
"training",
"ipu"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4019da6bd66d836d258c90aacc205acafe5b39643b895031af61aec8b6d639e9",
"md5": "89bfba04cb02b678cf34c3018c4cea17",
"sha256": "f105ff3c1e5ef48ffa1f46c615638a3fd46dfec8c00aeb25da664b3548c27fb8"
},
"downloads": -1,
"filename": "optimum_graphcore-0.7.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "89bfba04cb02b678cf34c3018c4cea17",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 252992,
"upload_time": "2023-07-31T09:34:09",
"upload_time_iso_8601": "2023-07-31T09:34:09.491412Z",
"url": "https://files.pythonhosted.org/packages/40/19/da6bd66d836d258c90aacc205acafe5b39643b895031af61aec8b6d639e9/optimum_graphcore-0.7.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "22daeac7471e1cb3834da562742fbeab2cbc1703ca115f76e79b835415f5e3e3",
"md5": "417fc016e9ebe53f117f4cca88e33b1f",
"sha256": "3dd59ca2d2fe08e55ca35905761ac78a9cbcf54499f0993819b02a3d982edc68"
},
"downloads": -1,
"filename": "optimum-graphcore-0.7.1.tar.gz",
"has_sig": false,
"md5_digest": "417fc016e9ebe53f117f4cca88e33b1f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 216401,
"upload_time": "2023-07-31T09:34:12",
"upload_time_iso_8601": "2023-07-31T09:34:12.305872Z",
"url": "https://files.pythonhosted.org/packages/22/da/eac7471e1cb3834da562742fbeab2cbc1703ca115f76e79b835415f5e3e3/optimum-graphcore-0.7.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-31 09:34:12",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "optimum-graphcore"
}
JSON