| Name | spacy-transformers JSON |
| Version |
1.3.7
JSON |
| download |
| home_page | https://spacy.io |
| Summary | spaCy pipelines for pre-trained BERT and other transformers |
| upload_time | 2025-02-03 17:36:14 |
| maintainer | None |
| docs_url | None |
| author | Explosion |
| requires_python | <3.13,>=3.9 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<a href="https://explosion.ai"><img src="https://explosion.ai/assets/img/logo.svg" width="125" height="125" align="right" /></a>
# spacy-transformers: Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy
This package provides [spaCy](https://github.com/explosion/spaCy) components and
architectures to use transformer models via
[Hugging Face's `transformers`](https://github.com/huggingface/transformers) in
spaCy. The result is convenient access to state-of-the-art transformer
architectures, such as BERT, GPT-2, XLNet, etc.
> **This release requires [spaCy v3](https://spacy.io/usage/v3).** For the
> previous version of this library, see the
> [`v0.6.x` branch](https://github.com/explosion/spacy-transformers/tree/v0.6.x).
[](https://github.com/explosion/spacy-transformers/actions/workflows/tests.yml)
[](https://pypi.python.org/pypi/spacy-transformers)
[](https://github.com/explosion/spacy-transformers/releases)
[](https://github.com/ambv/black)
## Features
- Use pretrained transformer models like **BERT**, **RoBERTa** and **XLNet** to
power your spaCy pipeline.
- Easy **multi-task learning**: backprop to one transformer model from several
pipeline components.
- Train using spaCy v3's powerful and extensible config system.
- Automatic alignment of transformer output to spaCy's tokenization.
- Easily customize what transformer data is saved in the `Doc` object.
- Easily customize how long documents are processed.
- Out-of-the-box serialization and model packaging.
## 🚀 Installation
Installing the package from pip will automatically install all dependencies,
including PyTorch and spaCy. Make sure you install this package **before** you
install the models. Also note that this package requires **Python 3.6+**,
**PyTorch v1.5+** and **spaCy v3.0+**.
```bash
pip install 'spacy[transformers]'
```
For GPU installation, find your CUDA version using `nvcc --version` and add the
[version in brackets](https://spacy.io/usage/#gpu), e.g.
`spacy[transformers,cuda92]` for CUDA9.2 or `spacy[transformers,cuda100]` for
CUDA10.0.
If you are having trouble installing PyTorch, follow the
[instructions](https://pytorch.org/get-started/locally/) on the official website
for your specific operating system and requirements.
## 📖 Documentation
> ⚠️ **Important note:** This package has been extensively refactored to take
> advantage of [spaCy v3.0](https://spacy.io). Previous versions that were built
> for [spaCy v2.x](https://v2.spacy.io) worked considerably differently. Please
> see previous tagged versions of this README for documentation on prior
> versions.
- 📘
[Embeddings, Transformers and Transfer Learning](https://spacy.io/usage/embeddings-transformers):
How to use transformers in spaCy
- 📘 [Training Pipelines and Models](https://spacy.io/usage/training): Train and
update components on your own data and integrate custom models
- 📘
[Layers and Model Architectures](https://spacy.io/usage/layers-architectures):
Power spaCy components with custom neural networks
- 📗 [`Transformer`](https://spacy.io/api/transformer): Pipeline component API
reference
- 📗
[Transformer architectures](https://spacy.io/api/architectures#transformers):
Architectures and registered functions
## Applying pretrained text and token classification models
Note that the `transformer` component from `spacy-transformers` does not support
task-specific heads like token or text classification. A task-specific
transformer model can be used as a source of features to train spaCy components
like `ner` or `textcat`, but the `transformer` component does not provide access
to task-specific heads for training or inference.
Alternatively, if you only want use to the **predictions** from an existing
Hugging Face text or token classification model, you can use the wrappers from
[`spacy-huggingface-pipelines`](https://github.com/explosion/spacy-huggingface-pipelines)
to incorporate task-specific transformer models into your spaCy pipelines.
## Bug reports and other issues
Please use [spaCy's issue tracker](https://github.com/explosion/spaCy/issues) to
report a bug, or open a new thread on the
[discussion board](https://github.com/explosion/spaCy/discussions) for any other
issue.
Raw data
{
"_id": null,
"home_page": "https://spacy.io",
"name": "spacy-transformers",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Explosion",
"author_email": "contact@explosion.ai",
"download_url": null,
"platform": null,
"description": "<a href=\"https://explosion.ai\"><img src=\"https://explosion.ai/assets/img/logo.svg\" width=\"125\" height=\"125\" align=\"right\" /></a>\n\n# spacy-transformers: Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy\n\nThis package provides [spaCy](https://github.com/explosion/spaCy) components and\narchitectures to use transformer models via\n[Hugging Face's `transformers`](https://github.com/huggingface/transformers) in\nspaCy. The result is convenient access to state-of-the-art transformer\narchitectures, such as BERT, GPT-2, XLNet, etc.\n\n> **This release requires [spaCy v3](https://spacy.io/usage/v3).** For the\n> previous version of this library, see the\n> [`v0.6.x` branch](https://github.com/explosion/spacy-transformers/tree/v0.6.x).\n\n[](https://github.com/explosion/spacy-transformers/actions/workflows/tests.yml)\n[](https://pypi.python.org/pypi/spacy-transformers)\n[](https://github.com/explosion/spacy-transformers/releases)\n[](https://github.com/ambv/black)\n\n## Features\n\n- Use pretrained transformer models like **BERT**, **RoBERTa** and **XLNet** to\n power your spaCy pipeline.\n- Easy **multi-task learning**: backprop to one transformer model from several\n pipeline components.\n- Train using spaCy v3's powerful and extensible config system.\n- Automatic alignment of transformer output to spaCy's tokenization.\n- Easily customize what transformer data is saved in the `Doc` object.\n- Easily customize how long documents are processed.\n- Out-of-the-box serialization and model packaging.\n\n## \ud83d\ude80 Installation\n\nInstalling the package from pip will automatically install all dependencies,\nincluding PyTorch and spaCy. Make sure you install this package **before** you\ninstall the models. Also note that this package requires **Python 3.6+**,\n**PyTorch v1.5+** and **spaCy v3.0+**.\n\n```bash\npip install 'spacy[transformers]'\n```\n\nFor GPU installation, find your CUDA version using `nvcc --version` and add the\n[version in brackets](https://spacy.io/usage/#gpu), e.g.\n`spacy[transformers,cuda92]` for CUDA9.2 or `spacy[transformers,cuda100]` for\nCUDA10.0.\n\nIf you are having trouble installing PyTorch, follow the\n[instructions](https://pytorch.org/get-started/locally/) on the official website\nfor your specific operating system and requirements.\n\n## \ud83d\udcd6 Documentation\n\n> \u26a0\ufe0f **Important note:** This package has been extensively refactored to take\n> advantage of [spaCy v3.0](https://spacy.io). Previous versions that were built\n> for [spaCy v2.x](https://v2.spacy.io) worked considerably differently. Please\n> see previous tagged versions of this README for documentation on prior\n> versions.\n\n- \ud83d\udcd8\n [Embeddings, Transformers and Transfer Learning](https://spacy.io/usage/embeddings-transformers):\n How to use transformers in spaCy\n- \ud83d\udcd8 [Training Pipelines and Models](https://spacy.io/usage/training): Train and\n update components on your own data and integrate custom models\n- \ud83d\udcd8\n [Layers and Model Architectures](https://spacy.io/usage/layers-architectures):\n Power spaCy components with custom neural networks\n- \ud83d\udcd7 [`Transformer`](https://spacy.io/api/transformer): Pipeline component API\n reference\n- \ud83d\udcd7\n [Transformer architectures](https://spacy.io/api/architectures#transformers):\n Architectures and registered functions\n\n## Applying pretrained text and token classification models\n\nNote that the `transformer` component from `spacy-transformers` does not support\ntask-specific heads like token or text classification. A task-specific\ntransformer model can be used as a source of features to train spaCy components\nlike `ner` or `textcat`, but the `transformer` component does not provide access\nto task-specific heads for training or inference.\n\nAlternatively, if you only want use to the **predictions** from an existing\nHugging Face text or token classification model, you can use the wrappers from\n[`spacy-huggingface-pipelines`](https://github.com/explosion/spacy-huggingface-pipelines)\nto incorporate task-specific transformer models into your spaCy pipelines.\n\n## Bug reports and other issues\n\nPlease use [spaCy's issue tracker](https://github.com/explosion/spaCy/issues) to\nreport a bug, or open a new thread on the\n[discussion board](https://github.com/explosion/spaCy/discussions) for any other\nissue.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "spaCy pipelines for pre-trained BERT and other transformers",
"version": "1.3.7",
"project_urls": {
"Homepage": "https://spacy.io"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "1a487639066333ef39b45b3acf7dbf8dcedc60028c21ac15213ec70833b89bbf",
"md5": "784029bc5499e2dde1eb5cb2adc023b2",
"sha256": "ec6974dff4a3792bb2cb854a19bbdd385e452940bd52adf13df002f5732fe14a"
},
"downloads": -1,
"filename": "spacy_transformers-1.3.7-cp310-cp310-macosx_10_9_x86_64.whl",
"has_sig": false,
"md5_digest": "784029bc5499e2dde1eb5cb2adc023b2",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": "<3.13,>=3.9",
"size": 177237,
"upload_time": "2025-02-03T17:36:14",
"upload_time_iso_8601": "2025-02-03T17:36:14.835823Z",
"url": "https://files.pythonhosted.org/packages/1a/48/7639066333ef39b45b3acf7dbf8dcedc60028c21ac15213ec70833b89bbf/spacy_transformers-1.3.7-cp310-cp310-macosx_10_9_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "fc914d9c9b1b1552230f6c0ad1a65e109ec1a75752af93a2c0d7e7da33a5b142",
"md5": "d934ce4de37162bb16670bfb92605aa2",
"sha256": "d908f3b000b0729d9c084b1ec06ec07b0322936f757288dc5f5553c80f6b9982"
},
"downloads": -1,
"filename": "spacy_transformers-1.3.7-cp310-cp310-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "d934ce4de37162bb16670bfb92605aa2",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": "<3.13,>=3.9",
"size": 171072,
"upload_time": "2025-02-03T17:36:16",
"upload_time_iso_8601": "2025-02-03T17:36:16.410902Z",
"url": "https://files.pythonhosted.org/packages/fc/91/4d9c9b1b1552230f6c0ad1a65e109ec1a75752af93a2c0d7e7da33a5b142/spacy_transformers-1.3.7-cp310-cp310-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b1b28d250ea0d024fc2edfc0908f82bb3c48edd761a4835b46f5357c59abbb6f",
"md5": "11011a79e368483442594f1f2a380966",
"sha256": "8cc2d5da9b4a544f75c3418c38d7e85760a71e441216d10359b7b0618246a88a"
},
"downloads": -1,
"filename": "spacy_transformers-1.3.7-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
"has_sig": false,
"md5_digest": "11011a79e368483442594f1f2a380966",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": "<3.13,>=3.9",
"size": 709001,
"upload_time": "2025-02-03T17:36:18",
"upload_time_iso_8601": "2025-02-03T17:36:18.504069Z",
"url": "https://files.pythonhosted.org/packages/b1/b2/8d250ea0d024fc2edfc0908f82bb3c48edd761a4835b46f5357c59abbb6f/spacy_transformers-1.3.7-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "8fea3c003d517669be87ff6de94dd0827977a723696161539d62a243fed098c9",
"md5": "aba1ca7182ca62eb4426ad0aa8c941b0",
"sha256": "a5eb303e9fdb831819fce379de2baba7e119313365748d9776adad68f38307dc"
},
"downloads": -1,
"filename": "spacy_transformers-1.3.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
"has_sig": false,
"md5_digest": "aba1ca7182ca62eb4426ad0aa8c941b0",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": "<3.13,>=3.9",
"size": 716266,
"upload_time": "2025-02-03T17:36:20",
"upload_time_iso_8601": "2025-02-03T17:36:20.465465Z",
"url": "https://files.pythonhosted.org/packages/8f/ea/3c003d517669be87ff6de94dd0827977a723696161539d62a243fed098c9/spacy_transformers-1.3.7-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "48742ebeb7aca054a3bb9ff7615924f6ba207c8041a2bab01057b058b171de33",
"md5": "39592d3353521d4f2d623f1a15cf800d",
"sha256": "3dd61ad60069e6a9d5723a2772e267185475277a70869fa606b59200b993b16d"
},
"downloads": -1,
"filename": "spacy_transformers-1.3.7-cp310-cp310-musllinux_1_2_aarch64.whl",
"has_sig": false,
"md5_digest": "39592d3353521d4f2d623f1a15cf800d",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": "<3.13,>=3.9",
"size": 1609945,
"upload_time": "2025-02-03T17:36:22",
"upload_time_iso_8601": "2025-02-03T17:36:22.064220Z",
"url": "https://files.pythonhosted.org/packages/48/74/2ebeb7aca054a3bb9ff7615924f6ba207c8041a2bab01057b058b171de33/spacy_transformers-1.3.7-cp310-cp310-musllinux_1_2_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "a7e2a0e2aaf1efb1c4c2bd98e134eff0a9c17ce02da32d2017e8b864556a20ae",
"md5": "101cd8b50ae5eb92366d76636a55c7fe",
"sha256": "bbda1c1f75852774d09e7a1654dbeaa2f960b69c86855e7cfdbe62fbdfc3d8fb"
},
"downloads": -1,
"filename": "spacy_transformers-1.3.7-cp310-cp310-musllinux_1_2_x86_64.whl",
"has_sig": false,
"md5_digest": "101cd8b50ae5eb92366d76636a55c7fe",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": "<3.13,>=3.9",
"size": 1677245,
"upload_time": "2025-02-03T17:36:24",
"upload_time_iso_8601": "2025-02-03T17:36:24.533035Z",
"url": "https://files.pythonhosted.org/packages/a7/e2/a0e2aaf1efb1c4c2bd98e134eff0a9c17ce02da32d2017e8b864556a20ae/spacy_transformers-1.3.7-cp310-cp310-musllinux_1_2_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "fff3e5591fe1fafae7c7797eb1130579696649f7a58b4f3c6cf186c3bf3b224c",
"md5": "ef3046cd5b950a7fe8912b622f5c5ab5",
"sha256": "a3be6600c9ef8082ef74003491e33364e4288d75ff0955e06dd7e1a7ecb1d895"
},
"downloads": -1,
"filename": "spacy_transformers-1.3.7-cp310-cp310-win_amd64.whl",
"has_sig": false,
"md5_digest": "ef3046cd5b950a7fe8912b622f5c5ab5",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": "<3.13,>=3.9",
"size": 344497,
"upload_time": "2025-02-03T17:36:26",
"upload_time_iso_8601": "2025-02-03T17:36:26.082703Z",
"url": "https://files.pythonhosted.org/packages/ff/f3/e5591fe1fafae7c7797eb1130579696649f7a58b4f3c6cf186c3bf3b224c/spacy_transformers-1.3.7-cp310-cp310-win_amd64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-03 17:36:14",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "spacy-transformers"
}
