<h1 align="center">PySentence-Similarity 😊</h1>
<p align="center">
<a href="https://github.com/goldpulpy/pysentence-similarity/blob/main/LICENSE"><img alt="GitHub" src="https://img.shields.io/github/license/goldpulpy/pysentence-similarity.svg?color=blue"></a>
<img alt="GitHub Actions Workflow Status" src="https://img.shields.io/github/actions/workflow/status/goldpulpy/pysentence-similarity/package.yml">
<img alt="GitHub forks" src="https://img.shields.io/github/forks/goldpulpy/pysentence-similarity">
</p>
## Information
**PySentence-Similarity** is a tool designed to identify and find similarities between sentences and a base sentence, expressed as a percentage 📊. It compares the semantic value of each input sentence to the base sentence, providing a score that reflects how related or similar they are. This tool is useful for various natural language processing tasks such as clustering similar texts 📚, paraphrase detection 🔍 and textual consequence measurement 📈.
The models were converted to ONNX format to optimize and speed up inference. Converting models to ONNX enables cross-platform compatibility and optimized hardware acceleration, making it more efficient for large-scale or real-world applications 🚀.
- **High accuracy:** Utilizes a robust Transformer-based architecture, providing high accuracy in semantic similarity calculations 🔬.
- **Cross-platform support:** The ONNX format provides seamless integration across platforms, making it easy to deploy across environments 🌐.
- **Scalability:** Efficient processing can handle large datasets, making it suitable for enterprise-level applications 📈.
- **Real-time processing:** Optimized for fast output, it can be used in real-world applications without significant latency ⏱️.
- **Flexible:** Easily adaptable to specific use cases through customization or integration with additional models or features 🛠️.
- **Low resource consumption:** The model is designed to operate efficiently, reducing memory and CPU/GPU requirements, making it ideal for resource-constrained environments ⚡.
- **Fast and user-friendly:** The library offers high performance and an intuitive interface, allowing users to quickly and easily integrate it into their projects 🚀.
## Installation 📦
- **Requirements:** Python 3.8 or higher.
```bash
# install from PyPI
pip install pysentence-similarity
# install from GitHub
pip install git+https://github.com/goldpulpy/pysentence-similarity.git
```
## Support models 🤝
You don't need to download anything; the package itself will download the model and its tokenizer from a special HF [repository](https://huggingface.co/goldpulpy/pysentence-similarity).
Below are the models currently added to the special repository, including their file size and a link to the source.
| Model | Parameters | FP32 | FP16 | INT8 | Source link |
| ------------------------------------- | ---------- | ------ | ----- | ----- | ------------------------------------------------------------------------------------------- |
| paraphrase-albert-small-v2 | 11.7M | 45MB | 22MB | 38MB | [HF](https://huggingface.co/sentence-transformers/paraphrase-albert-small-v2) 🤗 |
| all-MiniLM-L6-v2 | 22.7M | 90MB | 45MB | 23MB | [HF](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) 🤗 |
| paraphrase-MiniLM-L6-v2 | 22.7M | 90MB | 45MB | 23MB | [HF](https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L6-v2) 🤗 |
| multi-qa-MiniLM-L6-cos-v1 | 22.7M | 90MB | 45MB | 23MB | [HF](https://huggingface.co/sentence-transformers/multi-qa-MiniLM-L6-cos-v1) 🤗 |
| msmarco-MiniLM-L-6-v3 | 22.7M | 90MB | 45MB | 23MB | [HF](https://huggingface.co/sentence-transformers/msmarco-MiniLM-L-6-v3) 🤗 |
| all-MiniLM-L12-v2 | 33.4M | 127MB | 65MB | 32MB | [HF](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2) 🤗 |
| gte-small | 33.4M | 127MB | 65MB | 32MB | [HF](https://huggingface.co/thenlper/gte-small) 🤗 |
| all-distilroberta-v1 | 82.1M | 313MB | 157MB | 79MB | [HF](https://huggingface.co/sentence-transformers/all-distilroberta-v1) 🤗 |
| all-mpnet-base-v2 | 109M | 418MB | 209MB | 105MB | [HF](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) 🤗 |
| multi-qa-mpnet-base-dot-v1 | 109M | 418MB | 209MB | 105MB | [HF](https://huggingface.co/sentence-transformers/multi-qa-mpnet-base-dot-v1) 🤗 |
| paraphrase-multilingual-MiniLM-L12-v2 | 118M | 449MB | 225MB | 113MB | [HF](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) 🤗 |
| text2vec-base-multilingual | 118M | 449MB | 225MB | 113MB | [HF](https://huggingface.co/shibing624/text2vec-base-multilingual) 🤗 |
| distiluse-base-multilingual-cased-v1 | 135M | 514MB | 257MB | 129MB | [HF](https://huggingface.co/sentence-transformers/distiluse-base-multilingual-cased-v1) 🤗 |
| paraphrase-multilingual-mpnet-base-v2 | 278M | 1.04GB | 530MB | 266MB | [HF](https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2) 🤗 |
| gte-multilingual-base | 305M | 1.17GB | 599MB | 324MB | [HF](https://huggingface.co/Alibaba-NLP/gte-multilingual-base) 🤗 |
| gte-large | 335M | 1.25GB | 640MB | 321MB | [HF](https://huggingface.co/thenlper/gte-large) 🤗 |
| all-roberta-large-v1 | 355M | 1.32GB | 678MB | 340MB | [HF](https://huggingface.co/sentence-transformers/all-roberta-large-v1) 🤗 |
| LaBSE | 470M | 1.75GB | 898MB | 450MB | [HF](https://huggingface.co/sentence-transformers/LaBSE) 🤗 |
**PySentence-Similarity** supports `FP32`, `FP16`, and `INT8` dtypes.
- **FP32:** 32-bit floating-point format that provides high precision and a wide range of values.
- **FP16:** 16-bit floating-point format, reducing memory consumption and computation time, with minimal loss of precision (typically less than 1%).
- **INT8:** 8-bit integer quantized format that greatly reduces model size and speeds up output, ideal for resource-constrained environments, with little loss of precision.
## Usage examples 📖
### Compute similarity score 📊
Let's define the similarity score as the percentage of how similar the sentences are to the original sentence (0.75 = 75%), default compute function is `cosine`
You can use CUDA 12.X by passing the `device='cuda'` parameter to the Model object; the default is `cpu`. If the device is not available, it will automatically be set to `cpu`.
```python
from pysentence_similarity import Model
from pysentence_similarity.utils import compute_score
# Create an instance of the model all-MiniLM-L6-v2; the default dtype is `fp32`
model = Model("all-MiniLM-L6-v2", dtype="fp16")
sentences = [
"This is another test.",
"This is yet another test.",
"We are testing sentence similarity."
]
# Convert sentences to embeddings
# The default is to use mean_pooling as a pooling function
source_embedding = model.encode("This is a test.")
embeddings = model.encode(sentences, progress_bar=True)
# Compute similarity scores
# The rounding parameter allows us to round our float values
# with a default of 2, which means 2 decimal places.
compute_score(source_embedding, embeddings)
# Return: [0.86, 0.77, 0.48]
```
`compute_score` returns in the same index order in which the embedding was encoded.
Let's see the sentence and its evaluation from a computational function
```python
# Compute similarity scores
scores = compute_score(source_embedding, embeddings)
for sentence, score in zip(sentences, scores):
print(f"{sentence} ({score})")
# Output prints:
# This is another test. (0.86)
# This is yet another test. (0.77)
# We are testing sentence similarity. (0.48)
```
You can use the computational functions: `cosine`, `euclidean`, `manhattan`, `jaccard`, `pearson`, `minkowski`, `hamming`, `kl_divergence`, `chebyshev`, `bregman` or your custom function
```python
from pysentence_similarity.compute import euclidean
compute_score(source_embedding, embeddings, compute_function=euclidean)
# Return: [2.52, 3.28, 5.62]
```
You can use `max_pooling`, `mean_pooling`, `min_pooling` or your custom function
```python
from pysentence_similarity.pooling import max_pooling
source_embedding = model.encode("This is a test.", pooling_function=max_pooling)
embeddings = model.encode(sentences, pooling_function=max_pooling)
...
```
### Search similar sentences 🔍
```python
from pysentence_similarity import Model
from pysentence_similarity.utils import search_similar
# Create an instance of the model
model = Model("all-MiniLM-L6-v2", dtype="fp16")
# Test text
sentences = [
"Hello my name is Bob.",
"I love to eat pizza.",
"We are testing sentence similarity."
"Today is a sunny day.",
"London is the capital of England.",
"I am a student at Stanford University."
]
# Convert query sentence to embedding
query_embedding = model.encode("What's the capital of England?")
# Convert sentences to embeddings
embeddings = model.encode(sentences)
# Search similar sentences
similar = search_similar(
query_embedding=query_embedding,
sentences=sentences,
embeddings=embeddings,
top_k=3 # number of similar sentences to return
)
# Print similar sentences
for idx, (sentence, score) in enumerate(similar, start=1):
print(f"{idx}: {sentence} ({score})")
# Output prints:
# 1: London is the capital of England. (0.81)
# 2: Hello my name is Bob. (0.06)
# 3: I love to eat pizza. (0.05)
```
With use storage
```python
from pysentence_similarity import Model, Storage
from pysentence_similarity.utils import search_similar
model = Model("all-MiniLM-L6-v2", dtype="fp16")
query_embedding = model.encode("What's the capital of England?")
storage = Storage.load("my_storage.h5")
similar = search_similar(
query_embedding=query_embedding,
storage=storage,
top_k=3
)
...
```
### Splitting ✂️
```python
from pysentence_similarity import Splitter
# Default split markers: '\n'
splitter = Splitter()
# If you want to separate by specific characters.
splitter = Splitter(markers_to_split=["!", "?", "."], preserve_markers=True)
# Test text
text = "Hello world! How are you? I'm fine."
# Split from text
splitter.split_from_text(text)
# Return: ['Hello world!', 'How are you?', "I'm fine."]
```
At this point, sources for the splitting are available: text, file, URL, CSV, and JSON.
### Storage 💾
The storage allows you to save and link sentences and their embeddings for easy access, so you don't need to encode a large corpus of text every time. The storage also enables similarity searching.
The storage must store the **sentences** themselves and their **embeddings**.
```python
from pysentence_similarity import Model, Storage
# Create an instance of the model
model = Model("all-MiniLM-L6-v2", dtype="fp16")
# Create an instance of the storage
storage = Storage()
sentences = [
"This is another test.",
"This is yet another test.",
"We are testing sentence similarity."
]
# Convert sentences to embeddings
embeddings = model.encode(sentences)
# Add sentences and their embeddings
storage.add(sentences, embeddings)
# Save the storage
storage.save("my_storage.h5")
```
Load from the storage
```python
from pysentence_similarity import Model, Storage
from pysentence_similarity.utils import compute_score
# Create an instance of the model and storage
model = Model("all-MiniLM-L6-v2", dtype="fp16")
storage = Storage.load("my_storage.h5")
# Convert sentence to embedding
source_embedding = model.encode("This is a test.")
# Compute similarity scores with the storage
compute_score(source_embedding, storage)
# Return: [0.86, 0.77, 0.48]
```
## License 📜
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details
<h6 align="center">Created by goldpulpy with ❤️</h6>
Raw data
{
"_id": null,
"home_page": "https://github.com/goldpulpy/pysentence-similarity",
"name": "pysentence-similarity",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "sentence similarity NLP natural language processing",
"author": "goldpulpy",
"author_email": "sawsani1928@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/ed/bc/3c31432778c58d75841c6b1bc33ef7c8ead7bcd089519fceb384dee2a40d/pysentence-similarity-1.0.1.tar.gz",
"platform": null,
"description": "<h1 align=\"center\">PySentence-Similarity \ud83d\ude0a</h1>\n<p align=\"center\">\n <a href=\"https://github.com/goldpulpy/pysentence-similarity/blob/main/LICENSE\"><img alt=\"GitHub\" src=\"https://img.shields.io/github/license/goldpulpy/pysentence-similarity.svg?color=blue\"></a>\n <img alt=\"GitHub Actions Workflow Status\" src=\"https://img.shields.io/github/actions/workflow/status/goldpulpy/pysentence-similarity/package.yml\">\n <img alt=\"GitHub forks\" src=\"https://img.shields.io/github/forks/goldpulpy/pysentence-similarity\">\n</p>\n\n## Information\n\n**PySentence-Similarity** is a tool designed to identify and find similarities between sentences and a base sentence, expressed as a percentage \ud83d\udcca. It compares the semantic value of each input sentence to the base sentence, providing a score that reflects how related or similar they are. This tool is useful for various natural language processing tasks such as clustering similar texts \ud83d\udcda, paraphrase detection \ud83d\udd0d and textual consequence measurement \ud83d\udcc8.\n\nThe models were converted to ONNX format to optimize and speed up inference. Converting models to ONNX enables cross-platform compatibility and optimized hardware acceleration, making it more efficient for large-scale or real-world applications \ud83d\ude80.\n\n- **High accuracy:** Utilizes a robust Transformer-based architecture, providing high accuracy in semantic similarity calculations \ud83d\udd2c.\n- **Cross-platform support:** The ONNX format provides seamless integration across platforms, making it easy to deploy across environments \ud83c\udf10.\n- **Scalability:** Efficient processing can handle large datasets, making it suitable for enterprise-level applications \ud83d\udcc8.\n- **Real-time processing:** Optimized for fast output, it can be used in real-world applications without significant latency \u23f1\ufe0f.\n- **Flexible:** Easily adaptable to specific use cases through customization or integration with additional models or features \ud83d\udee0\ufe0f.\n- **Low resource consumption:** The model is designed to operate efficiently, reducing memory and CPU/GPU requirements, making it ideal for resource-constrained environments \u26a1.\n- **Fast and user-friendly:** The library offers high performance and an intuitive interface, allowing users to quickly and easily integrate it into their projects \ud83d\ude80.\n\n## Installation \ud83d\udce6\n\n- **Requirements:** Python 3.8 or higher.\n\n```bash\n# install from PyPI\npip install pysentence-similarity\n\n# install from GitHub\npip install git+https://github.com/goldpulpy/pysentence-similarity.git\n```\n\n## Support models \ud83e\udd1d\n\nYou don't need to download anything; the package itself will download the model and its tokenizer from a special HF [repository](https://huggingface.co/goldpulpy/pysentence-similarity).\n\nBelow are the models currently added to the special repository, including their file size and a link to the source.\n\n| Model | Parameters | FP32 | FP16 | INT8 | Source link |\n| ------------------------------------- | ---------- | ------ | ----- | ----- | ------------------------------------------------------------------------------------------- |\n| paraphrase-albert-small-v2 | 11.7M | 45MB | 22MB | 38MB | [HF](https://huggingface.co/sentence-transformers/paraphrase-albert-small-v2) \ud83e\udd17 |\n| all-MiniLM-L6-v2 | 22.7M | 90MB | 45MB | 23MB | [HF](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) \ud83e\udd17 |\n| paraphrase-MiniLM-L6-v2 | 22.7M | 90MB | 45MB | 23MB | [HF](https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L6-v2) \ud83e\udd17 |\n| multi-qa-MiniLM-L6-cos-v1 | 22.7M | 90MB | 45MB | 23MB | [HF](https://huggingface.co/sentence-transformers/multi-qa-MiniLM-L6-cos-v1) \ud83e\udd17 |\n| msmarco-MiniLM-L-6-v3 | 22.7M | 90MB | 45MB | 23MB | [HF](https://huggingface.co/sentence-transformers/msmarco-MiniLM-L-6-v3) \ud83e\udd17 |\n| all-MiniLM-L12-v2 | 33.4M | 127MB | 65MB | 32MB | [HF](https://huggingface.co/sentence-transformers/all-MiniLM-L12-v2) \ud83e\udd17 |\n| gte-small | 33.4M | 127MB | 65MB | 32MB | [HF](https://huggingface.co/thenlper/gte-small) \ud83e\udd17 |\n| all-distilroberta-v1 | 82.1M | 313MB | 157MB | 79MB | [HF](https://huggingface.co/sentence-transformers/all-distilroberta-v1) \ud83e\udd17 |\n| all-mpnet-base-v2 | 109M | 418MB | 209MB | 105MB | [HF](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) \ud83e\udd17 |\n| multi-qa-mpnet-base-dot-v1 | 109M | 418MB | 209MB | 105MB | [HF](https://huggingface.co/sentence-transformers/multi-qa-mpnet-base-dot-v1) \ud83e\udd17 |\n| paraphrase-multilingual-MiniLM-L12-v2 | 118M | 449MB | 225MB | 113MB | [HF](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2) \ud83e\udd17 |\n| text2vec-base-multilingual | 118M | 449MB | 225MB | 113MB | [HF](https://huggingface.co/shibing624/text2vec-base-multilingual) \ud83e\udd17 |\n| distiluse-base-multilingual-cased-v1 | 135M | 514MB | 257MB | 129MB | [HF](https://huggingface.co/sentence-transformers/distiluse-base-multilingual-cased-v1) \ud83e\udd17 |\n| paraphrase-multilingual-mpnet-base-v2 | 278M | 1.04GB | 530MB | 266MB | [HF](https://huggingface.co/sentence-transformers/paraphrase-multilingual-mpnet-base-v2) \ud83e\udd17 |\n| gte-multilingual-base | 305M | 1.17GB | 599MB | 324MB | [HF](https://huggingface.co/Alibaba-NLP/gte-multilingual-base) \ud83e\udd17 |\n| gte-large | 335M | 1.25GB | 640MB | 321MB | [HF](https://huggingface.co/thenlper/gte-large) \ud83e\udd17 |\n| all-roberta-large-v1 | 355M | 1.32GB | 678MB | 340MB | [HF](https://huggingface.co/sentence-transformers/all-roberta-large-v1) \ud83e\udd17 |\n| LaBSE | 470M | 1.75GB | 898MB | 450MB | [HF](https://huggingface.co/sentence-transformers/LaBSE) \ud83e\udd17 |\n\n**PySentence-Similarity** supports `FP32`, `FP16`, and `INT8` dtypes.\n\n- **FP32:** 32-bit floating-point format that provides high precision and a wide range of values.\n- **FP16:** 16-bit floating-point format, reducing memory consumption and computation time, with minimal loss of precision (typically less than 1%).\n- **INT8:** 8-bit integer quantized format that greatly reduces model size and speeds up output, ideal for resource-constrained environments, with little loss of precision.\n\n## Usage examples \ud83d\udcd6\n\n### Compute similarity score \ud83d\udcca\n\nLet's define the similarity score as the percentage of how similar the sentences are to the original sentence (0.75 = 75%), default compute function is `cosine`\n\nYou can use CUDA 12.X by passing the `device='cuda'` parameter to the Model object; the default is `cpu`. If the device is not available, it will automatically be set to `cpu`.\n\n```python\nfrom pysentence_similarity import Model\nfrom pysentence_similarity.utils import compute_score\n\n# Create an instance of the model all-MiniLM-L6-v2; the default dtype is `fp32`\nmodel = Model(\"all-MiniLM-L6-v2\", dtype=\"fp16\")\n\nsentences = [\n \"This is another test.\",\n \"This is yet another test.\",\n \"We are testing sentence similarity.\"\n]\n\n# Convert sentences to embeddings\n# The default is to use mean_pooling as a pooling function\nsource_embedding = model.encode(\"This is a test.\")\nembeddings = model.encode(sentences, progress_bar=True)\n\n# Compute similarity scores\n# The rounding parameter allows us to round our float values\n# with a default of 2, which means 2 decimal places.\ncompute_score(source_embedding, embeddings)\n# Return: [0.86, 0.77, 0.48]\n```\n\n`compute_score` returns in the same index order in which the embedding was encoded.\n\nLet's see the sentence and its evaluation from a computational function\n\n```python\n# Compute similarity scores\nscores = compute_score(source_embedding, embeddings)\n\nfor sentence, score in zip(sentences, scores):\n print(f\"{sentence} ({score})\")\n\n# Output prints:\n# This is another test. (0.86)\n# This is yet another test. (0.77)\n# We are testing sentence similarity. (0.48)\n```\n\nYou can use the computational functions: `cosine`, `euclidean`, `manhattan`, `jaccard`, `pearson`, `minkowski`, `hamming`, `kl_divergence`, `chebyshev`, `bregman` or your custom function\n\n```python\nfrom pysentence_similarity.compute import euclidean\n\ncompute_score(source_embedding, embeddings, compute_function=euclidean)\n# Return: [2.52, 3.28, 5.62]\n```\n\nYou can use `max_pooling`, `mean_pooling`, `min_pooling` or your custom function\n\n```python\nfrom pysentence_similarity.pooling import max_pooling\n\nsource_embedding = model.encode(\"This is a test.\", pooling_function=max_pooling)\nembeddings = model.encode(sentences, pooling_function=max_pooling)\n...\n```\n\n### Search similar sentences \ud83d\udd0d\n\n```python\nfrom pysentence_similarity import Model\nfrom pysentence_similarity.utils import search_similar\n\n# Create an instance of the model\nmodel = Model(\"all-MiniLM-L6-v2\", dtype=\"fp16\")\n\n# Test text\nsentences = [\n \"Hello my name is Bob.\",\n \"I love to eat pizza.\",\n \"We are testing sentence similarity.\"\n \"Today is a sunny day.\",\n \"London is the capital of England.\",\n \"I am a student at Stanford University.\"\n]\n\n# Convert query sentence to embedding\nquery_embedding = model.encode(\"What's the capital of England?\")\n\n# Convert sentences to embeddings\nembeddings = model.encode(sentences)\n\n# Search similar sentences\nsimilar = search_similar(\n query_embedding=query_embedding,\n sentences=sentences,\n embeddings=embeddings,\n top_k=3 # number of similar sentences to return\n)\n\n# Print similar sentences\nfor idx, (sentence, score) in enumerate(similar, start=1):\n print(f\"{idx}: {sentence} ({score})\")\n\n# Output prints:\n# 1: London is the capital of England. (0.81)\n# 2: Hello my name is Bob. (0.06)\n# 3: I love to eat pizza. (0.05)\n```\n\nWith use storage\n\n```python\nfrom pysentence_similarity import Model, Storage\nfrom pysentence_similarity.utils import search_similar\n\nmodel = Model(\"all-MiniLM-L6-v2\", dtype=\"fp16\")\nquery_embedding = model.encode(\"What's the capital of England?\")\n\nstorage = Storage.load(\"my_storage.h5\")\n\nsimilar = search_similar(\n query_embedding=query_embedding,\n storage=storage,\n top_k=3\n)\n...\n```\n\n### Splitting \u2702\ufe0f\n\n```python\nfrom pysentence_similarity import Splitter\n\n# Default split markers: '\\n'\nsplitter = Splitter()\n\n# If you want to separate by specific characters.\nsplitter = Splitter(markers_to_split=[\"!\", \"?\", \".\"], preserve_markers=True)\n\n# Test text\ntext = \"Hello world! How are you? I'm fine.\"\n\n# Split from text\nsplitter.split_from_text(text)\n# Return: ['Hello world!', 'How are you?', \"I'm fine.\"]\n```\n\nAt this point, sources for the splitting are available: text, file, URL, CSV, and JSON.\n\n### Storage \ud83d\udcbe\n\nThe storage allows you to save and link sentences and their embeddings for easy access, so you don't need to encode a large corpus of text every time. The storage also enables similarity searching.\n\nThe storage must store the **sentences** themselves and their **embeddings**.\n\n```python\nfrom pysentence_similarity import Model, Storage\n\n# Create an instance of the model\nmodel = Model(\"all-MiniLM-L6-v2\", dtype=\"fp16\")\n\n# Create an instance of the storage\nstorage = Storage()\nsentences = [\n \"This is another test.\",\n \"This is yet another test.\",\n \"We are testing sentence similarity.\"\n]\n\n# Convert sentences to embeddings\nembeddings = model.encode(sentences)\n\n# Add sentences and their embeddings\nstorage.add(sentences, embeddings)\n\n# Save the storage\nstorage.save(\"my_storage.h5\")\n```\n\nLoad from the storage\n\n```python\nfrom pysentence_similarity import Model, Storage\nfrom pysentence_similarity.utils import compute_score\n\n# Create an instance of the model and storage\nmodel = Model(\"all-MiniLM-L6-v2\", dtype=\"fp16\")\nstorage = Storage.load(\"my_storage.h5\")\n\n# Convert sentence to embedding\nsource_embedding = model.encode(\"This is a test.\")\n\n# Compute similarity scores with the storage\ncompute_score(source_embedding, storage)\n# Return: [0.86, 0.77, 0.48]\n```\n\n## License \ud83d\udcdc\n\nThis project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details\n\n<h6 align=\"center\">Created by goldpulpy with \u2764\ufe0f</h6>\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "PySentence-Similarity is a tool designed to identify and find similarities between sentences and a base sentence, expressed as a percentage.",
"version": "1.0.1",
"project_urls": {
"Documentation": "https://github.com/goldpulpy/pysentence-similarity#readme",
"Homepage": "https://github.com/goldpulpy/pysentence-similarity",
"Issues": "https://github.com/goldpulpy/pysentence-similarity/issues"
},
"split_keywords": [
"sentence",
"similarity",
"nlp",
"natural",
"language",
"processing"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c172ca8509fea674e5823267e822b4e2bf879350258b5236fbd00aa4fcccd4f5",
"md5": "a6e96a125f3403784d0fd7633c06091d",
"sha256": "242bbedd78bfd09b13005796d7312c33c6c9515395e6319352bc7bc3b5d454a1"
},
"downloads": -1,
"filename": "pysentence_similarity-1.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a6e96a125f3403784d0fd7633c06091d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 29849,
"upload_time": "2024-10-10T21:55:55",
"upload_time_iso_8601": "2024-10-10T21:55:55.918925Z",
"url": "https://files.pythonhosted.org/packages/c1/72/ca8509fea674e5823267e822b4e2bf879350258b5236fbd00aa4fcccd4f5/pysentence_similarity-1.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "edbc3c31432778c58d75841c6b1bc33ef7c8ead7bcd089519fceb384dee2a40d",
"md5": "d7ff5c27eee01351d2d6df8b1083210b",
"sha256": "04a0f3c1eb69e58603643ae16d524c75690e2a8fc0289efb1524ae6ada50ad8e"
},
"downloads": -1,
"filename": "pysentence-similarity-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "d7ff5c27eee01351d2d6df8b1083210b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 28367,
"upload_time": "2024-10-10T21:55:57",
"upload_time_iso_8601": "2024-10-10T21:55:57.744242Z",
"url": "https://files.pythonhosted.org/packages/ed/bc/3c31432778c58d75841c6b1bc33ef7c8ead7bcd089519fceb384dee2a40d/pysentence-similarity-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-10 21:55:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "goldpulpy",
"github_project": "pysentence-similarity",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pysentence-similarity"
}
JSON
