# Scalable Vector Search
**Scalable Vector Search (SVS)** is a performance library for vector [similarity search](https://en.wikipedia.org/wiki/Similarity_search).
Thanks to the use of Locally-adaptive Vector Quantization [[ABHT23]](#1) and its highly optimized indexing and search algorithms,
SVS provides vector similarity search:
* on **billions** of **high-dimensional** vectors,
* at **high accuracy**
* and **state-of-the-art speed**,
* while enabling the use of **less memory** than its alternatives.
This enables application and framework developers using similarity search to unleash its performance on Intel ® Xeon CPUs (2nd generation and newer).
SVS offers a fully-featured and yet simple Python API, compatible with most standard libraries.
SVS is written in C++ to facilitate its integration into performance-critical applications.
## Performance
SVS provides state-of-the-art performance and accuracy [[ABHT23]](#1) for billion-scale similarity search on
[standard benchmarks](https://intel.github.io/ScalableVectorSearch/benchs/index.html).
For example, for the standard billion-scale [Deep-1B](http://sites.skoltech.ru/compvision/noimi/) dataset,
different configurations of SVS yield significantly increased performance (measured in queries per second, QPS) with a smaller memory footprint (horizontal axis) than the alternatives[^1]:
<p align="center">
<img src="./docs/figs/SVS_performance_memoryfootprint.png" height=80% width=80%>
</p>
SVS is primarily optimized for large-scale similarity search but it still offers [state-of-the-art performance
at million-scale](https://intel.github.io/ScalableVectorSearch/benchs/small_scale_benchs.html).
Best performance is obtained with 4th generation (Sapphire Rapids) by making use of Intel(R) AVX-512 instructions,
with excellent results also with 2nd and 3rd Intel ® Xeon ® processors (Cascade Lake
and Ice Lake).
Performance will be degraded if Intel(R) AVX-512 instructions are not available.
A warning message will appear when loading the SVS Python module if the system does not support
Intel(R) AVX-512 instructions.
## Key Features
SVS supports:
* Similarity functions: Euclidean distance, inner product, cosine similarity.
* Vectors with individual values encoded as: float32, float16, uint8, int8.
* Vector compression (including Locally-adaptive Vector Quantization [[ABHT23]](#1))
* Optimizations for Intel ® Xeon ® processors:
- 2nd generation (Cascade Lake)
- 3rd generation (Ice Lake)
- 4th generation (Sapphire Rapids)
See [Roadmap](https://intel.github.io/ScalableVectorSearch/roadmap.html) for upcoming features.
## Documentation
[SVS documentation](https://intel.github.io/ScalableVectorSearch) includes getting started tutorials with [installation instructions for Python](https://intel.github.io/ScalableVectorSearch/start.html#installation) and [C++](https://intel.github.io/ScalableVectorSearch/start_cpp.html#building) and step-by-step search examples, an API reference, as well as several guides and benchmark comparisons.
## References
Reference to cite when you use SVS in a research paper:
```
@article{aguerrebere2023similarity,
title={Similarity search in the blink of an eye with compressed indices},
volume = {16},
number = {11},
pages = {3433--3446},
journal = {Proceedings of the VLDB Endowment},
author={Cecilia Aguerrebere and Ishwar Bhati and Mark Hildebrand and Mariano Tepper and Ted Willke},
year = {2023}
}
```
<a id="1">[ABHT23]</a>
Aguerrebere, C.; Bhati I.; Hildebrand M.; Tepper M.; Willke T.:Similarity search in the blink of an eye with compressed
indices. In: Proceedings of the VLDB Endowment, 16, 11, 3433 - 3446. (2023)
## Legal
Refer to the [LICENSE file](LICENSE) for details.
[^1]: Performance varies by use, configuration and other factors. Learn more at [www.Intel.com/PerformanceIndex](http://www.Intel.com/PerformanceIndex/).
Performance results are based on testing as of dates shown in configurations and may not reflect all publicly
available updates. No product or component can be absolutely secure. Your costs and results may vary. Intel
technologies may require enabled hardware, software or service activation. © Intel Corporation. Intel,
the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and
brands may be claimed as the property of others.
Raw data
{
"_id": null,
"home_page": "https://github.com/intel/ScalableVectorSearch",
"name": "scalable-vs",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": "onedal.maintainers@intel.com",
"keywords": "machine learning, data science, data analytics",
"author": null,
"author_email": "onedal.maintainers@intel.com",
"download_url": null,
"platform": null,
"description": "# Scalable Vector Search\n\n**Scalable Vector Search (SVS)** is a performance library for vector [similarity search](https://en.wikipedia.org/wiki/Similarity_search).\nThanks to the use of Locally-adaptive Vector Quantization [[ABHT23]](#1) and its highly optimized indexing and search algorithms,\nSVS provides vector similarity search:\n* on **billions** of **high-dimensional** vectors,\n* at **high accuracy**\n* and **state-of-the-art speed**,\n* while enabling the use of **less memory** than its alternatives.\n\nThis enables application and framework developers using similarity search to unleash its performance on Intel ® Xeon CPUs (2nd generation and newer).\n\nSVS offers a fully-featured and yet simple Python API, compatible with most standard libraries.\nSVS is written in C++ to facilitate its integration into performance-critical applications.\n\n## Performance\n\nSVS provides state-of-the-art performance and accuracy [[ABHT23]](#1) for billion-scale similarity search on\n[standard benchmarks](https://intel.github.io/ScalableVectorSearch/benchs/index.html).\n\nFor example, for the standard billion-scale [Deep-1B](http://sites.skoltech.ru/compvision/noimi/) dataset,\ndifferent configurations of SVS yield significantly increased performance (measured in queries per second, QPS) with a smaller memory footprint (horizontal axis) than the alternatives[^1]:\n\n<p align=\"center\">\n<img src=\"./docs/figs/SVS_performance_memoryfootprint.png\" height=80% width=80%>\n</p>\n\nSVS is primarily optimized for large-scale similarity search but it still offers [state-of-the-art performance\nat million-scale](https://intel.github.io/ScalableVectorSearch/benchs/small_scale_benchs.html).\n\nBest performance is obtained with 4th generation (Sapphire Rapids) by making use of Intel(R) AVX-512 instructions,\nwith excellent results also with 2nd and 3rd Intel ® Xeon ® processors (Cascade Lake\nand Ice Lake).\n\nPerformance will be degraded if Intel(R) AVX-512 instructions are not available.\nA warning message will appear when loading the SVS Python module if the system does not support\nIntel(R) AVX-512 instructions.\n\n## Key Features\n\nSVS supports:\n* Similarity functions: Euclidean distance, inner product, cosine similarity.\n* Vectors with individual values encoded as: float32, float16, uint8, int8.\n* Vector compression (including Locally-adaptive Vector Quantization [[ABHT23]](#1))\n* Optimizations for Intel ® Xeon ® processors:\n - 2nd generation (Cascade Lake)\n - 3rd generation (Ice Lake)\n - 4th generation (Sapphire Rapids)\n\nSee [Roadmap](https://intel.github.io/ScalableVectorSearch/roadmap.html) for upcoming features.\n\n\n## Documentation\n\n[SVS documentation](https://intel.github.io/ScalableVectorSearch) includes getting started tutorials with [installation instructions for Python](https://intel.github.io/ScalableVectorSearch/start.html#installation) and [C++](https://intel.github.io/ScalableVectorSearch/start_cpp.html#building) and step-by-step search examples, an API reference, as well as several guides and benchmark comparisons.\n\n## References\nReference to cite when you use SVS in a research paper:\n\n```\n@article{aguerrebere2023similarity,\n title={Similarity search in the blink of an eye with compressed indices},\n volume = {16},\n number = {11},\n pages = {3433--3446},\n journal = {Proceedings of the VLDB Endowment},\n author={Cecilia Aguerrebere and Ishwar Bhati and Mark Hildebrand and Mariano Tepper and Ted Willke},\n year = {2023}\n}\n```\n\n<a id=\"1\">[ABHT23]</a>\nAguerrebere, C.; Bhati I.; Hildebrand M.; Tepper M.; Willke T.:Similarity search in the blink of an eye with compressed\nindices. In: Proceedings of the VLDB Endowment, 16, 11, 3433 - 3446. (2023)\n\n## Legal\nRefer to the [LICENSE file](LICENSE) for details.\n\n[^1]: Performance varies by use, configuration and other factors. Learn more at [www.Intel.com/PerformanceIndex](http://www.Intel.com/PerformanceIndex/).\nPerformance results are based on testing as of dates shown in configurations and may not reflect all publicly\navailable updates. No product or component can be absolutely secure. Your costs and results may vary. Intel\ntechnologies may require enabled hardware, software or service activation. © Intel Corporation. Intel,\nthe Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and\nbrands may be claimed as the property of others.\n",
"bugtrack_url": null,
"license": "GNU Affero General Public License v3 or later (AGPLv3+)",
"summary": "Scalable Vector Search (SVS) is a performance library for vector similarity search.",
"version": "0.0.5",
"project_urls": {
"Bug Tracker": "https://github.com/intel/ScalableVectorSearch",
"Documentation": "https://intel.github.io/ScalableVectorSearch/",
"Homepage": "https://github.com/intel/ScalableVectorSearch",
"Source Code": "https://github.com/intel/ScalableVectorSearch"
},
"split_keywords": [
"machine learning",
" data science",
" data analytics"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "08983fb9885e8f9490966a6993e1b54739276d7b030e5ca296a637801d0a8200",
"md5": "0faa6a82f716a96bc7067c424b0e5ed6",
"sha256": "f7072ea08f1dc75354a22e798380c51c13d48e348b8c32f01c2e8369061b41c4"
},
"downloads": -1,
"filename": "scalable_vs-0.0.5-0-cp310-cp310-manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "0faa6a82f716a96bc7067c424b0e5ed6",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": null,
"size": 9712292,
"upload_time": "2024-08-28T14:47:39",
"upload_time_iso_8601": "2024-08-28T14:47:39.244299Z",
"url": "https://files.pythonhosted.org/packages/08/98/3fb9885e8f9490966a6993e1b54739276d7b030e5ca296a637801d0a8200/scalable_vs-0.0.5-0-cp310-cp310-manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8da8d476448873a7dcc8e18598800bf3f1c5f82b25278d40476b9b7ac89b59f0",
"md5": "1c6a2c19937f13a015cdb4c8c6f1440e",
"sha256": "f2b580d5f6347cc2b85f03879afcd1e6f6b591e9cd941756bda38859c977fa76"
},
"downloads": -1,
"filename": "scalable_vs-0.0.5-0-cp311-cp311-manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "1c6a2c19937f13a015cdb4c8c6f1440e",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": null,
"size": 9712899,
"upload_time": "2024-08-28T14:47:44",
"upload_time_iso_8601": "2024-08-28T14:47:44.061314Z",
"url": "https://files.pythonhosted.org/packages/8d/a8/d476448873a7dcc8e18598800bf3f1c5f82b25278d40476b9b7ac89b59f0/scalable_vs-0.0.5-0-cp311-cp311-manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cafa7f9b8b13e400845dec1602926808bfcc2eeed99271b979160d8d6433bf8f",
"md5": "acdaf6064e4facdb3969c8c6366999ff",
"sha256": "5ce503e31fb6775a855dec1af950be43d15f4799d90291adc32ab86b2660df50"
},
"downloads": -1,
"filename": "scalable_vs-0.0.5-0-cp312-cp312-manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "acdaf6064e4facdb3969c8c6366999ff",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": null,
"size": 9730820,
"upload_time": "2024-08-28T14:47:49",
"upload_time_iso_8601": "2024-08-28T14:47:49.994796Z",
"url": "https://files.pythonhosted.org/packages/ca/fa/7f9b8b13e400845dec1602926808bfcc2eeed99271b979160d8d6433bf8f/scalable_vs-0.0.5-0-cp312-cp312-manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "025c43a2ed1e8fba169fa7d51f387c4b56221169392ba6df8df3a6fa7e350d87",
"md5": "42888770891b3db834f4cd07f8e7d539",
"sha256": "8d429fef5e8bf0a2fee9e679c9558c76d46a9d7ca843f365e2c9b14f979bec5d"
},
"downloads": -1,
"filename": "scalable_vs-0.0.5-0-cp39-cp39-manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "42888770891b3db834f4cd07f8e7d539",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": null,
"size": 9714951,
"upload_time": "2024-08-28T14:47:34",
"upload_time_iso_8601": "2024-08-28T14:47:34.668457Z",
"url": "https://files.pythonhosted.org/packages/02/5c/43a2ed1e8fba169fa7d51f387c4b56221169392ba6df8df3a6fa7e350d87/scalable_vs-0.0.5-0-cp39-cp39-manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-28 14:47:39",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "intel",
"github_project": "ScalableVectorSearch",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "scalable-vs"
}
JSON
