<!--- BADGES: START --->
[](https://huggingface.co/models?library=sentence-transformers)
[][#github-license]
[][#pypi-package]
[][#pypi-package]
[][#docs-package]
<!-- [][#pypi-package] -->
[#github-license]: https://github.com/UKPLab/sentence-transformers/blob/master/LICENSE
[#pypi-package]: https://pypi.org/project/sentence-transformers/
[#conda-forge-package]: https://anaconda.org/conda-forge/sentence-transformers
[#docs-package]: https://www.sbert.net/
<!--- BADGES: END --->
# Sentence Transformers: Multilingual Sentence, Paragraph, and Image Embeddings using BERT & Co.
This framework provides an easy method to compute dense vector representations for **sentences**, **paragraphs**, and **images**. The models are based on transformer networks like BERT / RoBERTa / XLM-RoBERTa etc. and achieve state-of-the-art performance in various tasks. Text is embedded in vector space such that similar text are closer and can efficiently be found using cosine similarity.
We provide an increasing number of **[state-of-the-art pretrained models](https://www.sbert.net/docs/sentence_transformer/pretrained_models.html)** for more than 100 languages, fine-tuned for various use-cases.
Further, this framework allows an easy **[fine-tuning of custom embeddings models](https://www.sbert.net/docs/sentence_transformer/training_overview.html)**, to achieve maximal performance on your specific task.
For the **full documentation**, see **[www.SBERT.net](https://www.sbert.net)**.
## Installation
We recommend **Python 3.9+**, **[PyTorch 1.11.0+](https://pytorch.org/get-started/locally/)**, and **[transformers v4.34.0+](https://github.com/huggingface/transformers)**.
**Install with pip**
```
pip install -U sentence-transformers
```
**Install with conda**
```
conda install -c conda-forge sentence-transformers
```
**Install from sources**
Alternatively, you can also clone the latest version from the [repository](https://github.com/UKPLab/sentence-transformers) and install it directly from the source code:
````
pip install -e .
````
**PyTorch with CUDA**
If you want to use a GPU / CUDA, you must install PyTorch with the matching CUDA Version. Follow
[PyTorch - Get Started](https://pytorch.org/get-started/locally/) for further details how to install PyTorch.
## Getting Started
See [Quickstart](https://www.sbert.net/docs/quickstart.html) in our documentation.
First download a pretrained model.
````python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
````
Then provide some sentences to the model.
````python
sentences = [
"The weather is lovely today.",
"It's so sunny outside!",
"He drove to the stadium.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# => (3, 384)
````
And that's already it. We now have a numpy arrays with the embeddings, one for each text. We can use these to compute similarities.
````python
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6660, 0.1046],
# [0.6660, 1.0000, 0.1411],
# [0.1046, 0.1411, 1.0000]])
````
## Pre-Trained Models
We provide a large list of [Pretrained Models](https://www.sbert.net/docs/sentence_transformer/pretrained_models.html) for more than 100 languages. Some models are general purpose models, while others produce embeddings for specific use cases. Pre-trained models can be loaded by just passing the model name: `SentenceTransformer('model_name')`.
## Training
This framework allows you to fine-tune your own sentence embedding methods, so that you get task-specific sentence embeddings. You have various options to choose from in order to get perfect sentence embeddings for your specific task.
See [Training Overview](https://www.sbert.net/docs/sentence_transformer/training_overview.html) for an introduction how to train your own embedding models. We provide [various examples](https://github.com/UKPLab/sentence-transformers/tree/master/examples/training) how to train models on various datasets.
Some highlights are:
- Support of various transformer networks including BERT, RoBERTa, XLM-R, DistilBERT, Electra, BART, ...
- Multi-Lingual and multi-task learning
- Evaluation during training to find optimal model
- [20+ loss-functions](https://www.sbert.net/docs/package_reference/sentence_transformer/losses.html) allowing to tune models specifically for semantic search, paraphrase mining, semantic similarity comparison, clustering, triplet loss, contrastive loss, etc.
## Application Examples
You can use this framework for:
- [Computing Sentence Embeddings](https://www.sbert.net/examples/applications/computing-embeddings/README.html)
- [Semantic Textual Similarity](https://www.sbert.net/docs/usage/semantic_textual_similarity.html)
- [Semantic Search](https://www.sbert.net/examples/applications/semantic-search/README.html)
- [Retrieve & Re-Rank](https://www.sbert.net/examples/applications/retrieve_rerank/README.html)
- [Clustering](https://www.sbert.net/examples/applications/clustering/README.html)
- [Paraphrase Mining](https://www.sbert.net/examples/applications/paraphrase-mining/README.html)
- [Translated Sentence Mining](https://www.sbert.net/examples/applications/parallel-sentence-mining/README.html)
- [Multilingual Image Search, Clustering & Duplicate Detection](https://www.sbert.net/examples/applications/image-search/README.html)
and many more use-cases.
For all examples, see [examples/applications](https://github.com/UKPLab/sentence-transformers/tree/master/examples/applications).
## Development setup
After cloning the repo (or a fork) to your machine, in a virtual environment, run:
```
python -m pip install -e ".[dev]"
pre-commit install
```
To test your changes, run:
```
pytest
```
## Citing & Authors
If you find this repository helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084):
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
If you use one of the multilingual models, feel free to cite our publication [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://arxiv.org/abs/2004.09813):
```bibtex
@inproceedings{reimers-2020-multilingual-sentence-bert,
title = "Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2020",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/2004.09813",
}
```
Please have a look at [Publications](https://www.sbert.net/docs/publications.html) for our different publications that are integrated into SentenceTransformers.
Maintainer: [Tom Aarsen](https://github.com/tomaarsen), 🤗 Hugging Face
https://www.ukp.tu-darmstadt.de/
Don't hesitate to open an issue if something is broken (and it shouldn't be) or if you have further questions.
> This repository contains experimental software and is published for the sole purpose of giving additional background details on the respective publication.
Raw data
{
"_id": null,
"home_page": null,
"name": "sentence-transformers",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "Tom Aarsen <tom.aarsen@huggingface.co>",
"keywords": "Transformer Networks, BERT, XLNet, sentence embedding, PyTorch, NLP, deep learning",
"author": null,
"author_email": "Nils Reimers <info@nils-reimers.de>, Tom Aarsen <tom.aarsen@huggingface.co>",
"download_url": "https://files.pythonhosted.org/packages/79/0a/c677efe908b20e7e8d4ed6cce3a3447eebc7dc5e348e458f5f9a44a72b00/sentence_transformers-3.3.1.tar.gz",
"platform": null,
"description": "<!--- BADGES: START --->\r\n[](https://huggingface.co/models?library=sentence-transformers)\r\n[][#github-license]\r\n[][#pypi-package]\r\n[][#pypi-package]\r\n[][#docs-package]\r\n<!-- [][#pypi-package] -->\r\n\r\n[#github-license]: https://github.com/UKPLab/sentence-transformers/blob/master/LICENSE\r\n[#pypi-package]: https://pypi.org/project/sentence-transformers/\r\n[#conda-forge-package]: https://anaconda.org/conda-forge/sentence-transformers\r\n[#docs-package]: https://www.sbert.net/\r\n<!--- BADGES: END --->\r\n\r\n# Sentence Transformers: Multilingual Sentence, Paragraph, and Image Embeddings using BERT & Co.\r\n\r\nThis framework provides an easy method to compute dense vector representations for **sentences**, **paragraphs**, and **images**. The models are based on transformer networks like BERT / RoBERTa / XLM-RoBERTa etc. and achieve state-of-the-art performance in various tasks. Text is embedded in vector space such that similar text are closer and can efficiently be found using cosine similarity.\r\n\r\nWe provide an increasing number of **[state-of-the-art pretrained models](https://www.sbert.net/docs/sentence_transformer/pretrained_models.html)** for more than 100 languages, fine-tuned for various use-cases.\r\n\r\nFurther, this framework allows an easy **[fine-tuning of custom embeddings models](https://www.sbert.net/docs/sentence_transformer/training_overview.html)**, to achieve maximal performance on your specific task.\r\n\r\nFor the **full documentation**, see **[www.SBERT.net](https://www.sbert.net)**.\r\n\r\n## Installation\r\n\r\nWe recommend **Python 3.9+**, **[PyTorch 1.11.0+](https://pytorch.org/get-started/locally/)**, and **[transformers v4.34.0+](https://github.com/huggingface/transformers)**.\r\n\r\n**Install with pip**\r\n\r\n```\r\npip install -U sentence-transformers\r\n```\r\n\r\n**Install with conda**\r\n\r\n```\r\nconda install -c conda-forge sentence-transformers\r\n```\r\n\r\n**Install from sources**\r\n\r\nAlternatively, you can also clone the latest version from the [repository](https://github.com/UKPLab/sentence-transformers) and install it directly from the source code:\r\n\r\n````\r\npip install -e .\r\n```` \r\n\r\n**PyTorch with CUDA**\r\n\r\nIf you want to use a GPU / CUDA, you must install PyTorch with the matching CUDA Version. Follow\r\n[PyTorch - Get Started](https://pytorch.org/get-started/locally/) for further details how to install PyTorch.\r\n\r\n## Getting Started\r\n\r\nSee [Quickstart](https://www.sbert.net/docs/quickstart.html) in our documentation.\r\n\r\nFirst download a pretrained model.\r\n\r\n````python\r\nfrom sentence_transformers import SentenceTransformer\r\n\r\nmodel = SentenceTransformer(\"all-MiniLM-L6-v2\")\r\n````\r\n\r\nThen provide some sentences to the model.\r\n\r\n````python\r\nsentences = [\r\n \"The weather is lovely today.\",\r\n \"It's so sunny outside!\",\r\n \"He drove to the stadium.\",\r\n]\r\nembeddings = model.encode(sentences)\r\nprint(embeddings.shape)\r\n# => (3, 384)\r\n````\r\n\r\nAnd that's already it. We now have a numpy arrays with the embeddings, one for each text. We can use these to compute similarities.\r\n\r\n````python\r\nsimilarities = model.similarity(embeddings, embeddings)\r\nprint(similarities)\r\n# tensor([[1.0000, 0.6660, 0.1046],\r\n# [0.6660, 1.0000, 0.1411],\r\n# [0.1046, 0.1411, 1.0000]])\r\n````\r\n\r\n## Pre-Trained Models\r\n\r\nWe provide a large list of [Pretrained Models](https://www.sbert.net/docs/sentence_transformer/pretrained_models.html) for more than 100 languages. Some models are general purpose models, while others produce embeddings for specific use cases. Pre-trained models can be loaded by just passing the model name: `SentenceTransformer('model_name')`.\r\n\r\n## Training\r\n\r\nThis framework allows you to fine-tune your own sentence embedding methods, so that you get task-specific sentence embeddings. You have various options to choose from in order to get perfect sentence embeddings for your specific task. \r\n\r\nSee [Training Overview](https://www.sbert.net/docs/sentence_transformer/training_overview.html) for an introduction how to train your own embedding models. We provide [various examples](https://github.com/UKPLab/sentence-transformers/tree/master/examples/training) how to train models on various datasets.\r\n\r\nSome highlights are:\r\n- Support of various transformer networks including BERT, RoBERTa, XLM-R, DistilBERT, Electra, BART, ...\r\n- Multi-Lingual and multi-task learning\r\n- Evaluation during training to find optimal model\r\n- [20+ loss-functions](https://www.sbert.net/docs/package_reference/sentence_transformer/losses.html) allowing to tune models specifically for semantic search, paraphrase mining, semantic similarity comparison, clustering, triplet loss, contrastive loss, etc.\r\n\r\n## Application Examples\r\n\r\nYou can use this framework for:\r\n\r\n- [Computing Sentence Embeddings](https://www.sbert.net/examples/applications/computing-embeddings/README.html)\r\n- [Semantic Textual Similarity](https://www.sbert.net/docs/usage/semantic_textual_similarity.html)\r\n- [Semantic Search](https://www.sbert.net/examples/applications/semantic-search/README.html)\r\n- [Retrieve & Re-Rank](https://www.sbert.net/examples/applications/retrieve_rerank/README.html) \r\n- [Clustering](https://www.sbert.net/examples/applications/clustering/README.html)\r\n- [Paraphrase Mining](https://www.sbert.net/examples/applications/paraphrase-mining/README.html)\r\n- [Translated Sentence Mining](https://www.sbert.net/examples/applications/parallel-sentence-mining/README.html)\r\n- [Multilingual Image Search, Clustering & Duplicate Detection](https://www.sbert.net/examples/applications/image-search/README.html)\r\n\r\nand many more use-cases.\r\n\r\nFor all examples, see [examples/applications](https://github.com/UKPLab/sentence-transformers/tree/master/examples/applications).\r\n\r\n## Development setup\r\n\r\nAfter cloning the repo (or a fork) to your machine, in a virtual environment, run:\r\n\r\n```\r\npython -m pip install -e \".[dev]\"\r\n\r\npre-commit install\r\n```\r\n\r\nTo test your changes, run:\r\n\r\n```\r\npytest\r\n```\r\n\r\n## Citing & Authors\r\n\r\nIf you find this repository helpful, feel free to cite our publication [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://arxiv.org/abs/1908.10084):\r\n\r\n```bibtex \r\n@inproceedings{reimers-2019-sentence-bert,\r\n title = \"Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks\",\r\n author = \"Reimers, Nils and Gurevych, Iryna\",\r\n booktitle = \"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing\",\r\n month = \"11\",\r\n year = \"2019\",\r\n publisher = \"Association for Computational Linguistics\",\r\n url = \"https://arxiv.org/abs/1908.10084\",\r\n}\r\n```\r\n\r\nIf you use one of the multilingual models, feel free to cite our publication [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://arxiv.org/abs/2004.09813):\r\n\r\n```bibtex\r\n@inproceedings{reimers-2020-multilingual-sentence-bert,\r\n title = \"Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation\",\r\n author = \"Reimers, Nils and Gurevych, Iryna\",\r\n booktitle = \"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing\",\r\n month = \"11\",\r\n year = \"2020\",\r\n publisher = \"Association for Computational Linguistics\",\r\n url = \"https://arxiv.org/abs/2004.09813\",\r\n}\r\n```\r\n\r\nPlease have a look at [Publications](https://www.sbert.net/docs/publications.html) for our different publications that are integrated into SentenceTransformers.\r\n\r\nMaintainer: [Tom Aarsen](https://github.com/tomaarsen), \ud83e\udd17 Hugging Face\r\n\r\nhttps://www.ukp.tu-darmstadt.de/\r\n\r\nDon't hesitate to open an issue if something is broken (and it shouldn't be) or if you have further questions.\r\n\r\n> This repository contains experimental software and is published for the sole purpose of giving additional background details on the respective publication.\r\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "State-of-the-Art Text Embeddings",
"version": "3.3.1",
"project_urls": {
"Homepage": "https://www.SBERT.net",
"Repository": "https://github.com/UKPLab/sentence-transformers/"
},
"split_keywords": [
"transformer networks",
" bert",
" xlnet",
" sentence embedding",
" pytorch",
" nlp",
" deep learning"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8bc8990e22a465e4771338da434d799578865d6d7ef1fdb50bd844b7ecdcfa19",
"md5": "035b3457ff37f8bff7d276cdc24b7dd6",
"sha256": "abffcc79dab37b7d18d21a26d5914223dd42239cfe18cb5e111c66c54b658ae7"
},
"downloads": -1,
"filename": "sentence_transformers-3.3.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "035b3457ff37f8bff7d276cdc24b7dd6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 268797,
"upload_time": "2024-11-18T14:37:38",
"upload_time_iso_8601": "2024-11-18T14:37:38.579151Z",
"url": "https://files.pythonhosted.org/packages/8b/c8/990e22a465e4771338da434d799578865d6d7ef1fdb50bd844b7ecdcfa19/sentence_transformers-3.3.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "790ac677efe908b20e7e8d4ed6cce3a3447eebc7dc5e348e458f5f9a44a72b00",
"md5": "9edbf00c0d6e9cb157db6c045f66c665",
"sha256": "9635dbfb11c6b01d036b9cfcee29f7716ab64cf2407ad9f403a2e607da2ac48b"
},
"downloads": -1,
"filename": "sentence_transformers-3.3.1.tar.gz",
"has_sig": false,
"md5_digest": "9edbf00c0d6e9cb157db6c045f66c665",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 217914,
"upload_time": "2024-11-18T14:37:40",
"upload_time_iso_8601": "2024-11-18T14:37:40.954873Z",
"url": "https://files.pythonhosted.org/packages/79/0a/c677efe908b20e7e8d4ed6cce3a3447eebc7dc5e348e458f5f9a44a72b00/sentence_transformers-3.3.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-18 14:37:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "UKPLab",
"github_project": "sentence-transformers",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "sentence-transformers"
}
JSON
