| Name | scann JSON |
| Version |
1.4.2
JSON |
| download |
| home_page | None |
| Summary | Scalable Nearest Neighbor search library |
| upload_time | 2025-08-29 14:32:29 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | <3.14,>=3.9 |
| license | None |
| keywords |
machine
learning
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# ScaNN
ScaNN (Scalable Nearest Neighbors) is a method for efficient vector similarity
search at scale. This code implements [1, 2], which includes search space
pruning and quantization for Maximum Inner Product Search and also supports
other distance functions such as Euclidean distance. The implementation is
optimized for x86 processors with AVX support. ScaNN achieves state-of-the-art
performance on [ann-benchmarks.com](http://ann-benchmarks.com) as shown on the
glove-100-angular dataset below:
ScaNN can be configured to fit datasets with different sizes and distributions.
It has both TensorFlow and Python APIs. The library shows strong performance
with large datasets [1, 2]. The code is released for research purposes. For more
details on the academic description of algorithms, please see below.
References:
1. ```
@inproceedings{avq_2020,
title={Accelerating Large-Scale Inference with Anisotropic Vector Quantization},
author={Guo, Ruiqi and Sun, Philip and Lindgren, Erik and Geng, Quan and Simcha, David and Chern, Felix and Kumar, Sanjiv},
booktitle={International Conference on Machine Learning},
year={2020},
URL={https://arxiv.org/abs/1908.10396}
}
```
1. ```
@inproceedings{soar_2023,
title={SOAR: Improved Indexing for Approximate Nearest Neighbor Search},
author={Sun, Philip and Simcha, David and Dopson, Dave and Guo, Ruiqi and Kumar, Sanjiv},
booktitle={Neural Information Processing Systems},
year={2023},
URL={https://arxiv.org/abs/2404.00774}
}
```
## Installation
`manylinux_2_27`-compatible wheels are available on
[PyPI](https://pypi.org/project/scann/):
```
pip install scann
```
ScaNN supports Linux environments running Python versions 3.9-3.13. See
[docs/releases.md](docs/releases.md) for release notes; the page also contains
download links for ScaNN wheels prior to version 1.1.0, which were not released
on PyPI. **The x86 wheels require AVX and FMA instruction set support, while
the ARM wheels require NEON**.
In accordance with the
[`manylinux_2_27` specification](https://peps.python.org/pep-0600/), ScaNN
requires `libstdc++` version 3.4.23 or above from the operating system. See
[here](https://stackoverflow.com/questions/10354636) for an example of how
to find your system's `libstdc++` version; it can generally be upgraded by
installing a newer version of `g++`.
### Using ScaNN with TensorFlow
ScaNN has optional TensorFlow op bindings that allow ScaNN's nearest neighbor
search functionality to be embedded into a TensorFlow SavedModel. **As of
ScaNN 1.4.0, this functionality is no longer enabled by default, and `pip
install scann[tf]` is now needed to enable the TensorFlow integration.**
If `scann[tf]` is installed, the TensorFlow ops can be accessed via
`scann.scann_ops`, and the API is almost identical to `scann.scann_ops_pybind`.
For users not already in the TensorFlow ecosystem, the native Python bindings
in `scann.scann_ops_pybind` are a better fit, which is why the TensorFlow ops
are an optional extra.
#### Integration with TensorFlow Serving
We provide custom Docker images of
[TF Serving](https://github.com/tensorflow/serving) that are linked to the ScaNN
TF ops. See the [`tf_serving` directory](tf_serving/README.md) for further
information.
## Usage
See the example in [docs/example.ipynb](docs/example.ipynb). For a more in-depth
explanation of ScaNN techniques, see [docs/algorithms.md](docs/algorithms.md).
## Building from source
To build ScaNN from source, first install the build tool
[bazel](https://bazel.build) (use version 7.x), Clang 19, and libstdc++ headers
for C++17 (which are provided with GCC 9). Additionally, ScaNN requires a modern
version of Python (3.9.x or later) and Tensorflow 2.20 installed on that version
of Python. Once these prerequisites are satisfied, run the following command in
the root directory of the repository:
```
python configure.py
CC=clang-19 bazel build -c opt --features=thin_lto --copt=-mavx --copt=-mfma --cxxopt="-std=c++17" --copt=-fsized-deallocation --copt=-w :build_pip_pkg
./bazel-bin/build_pip_pkg
```
To build an ARM binary from an ARM machine, the prerequisites are the same, but
the compile flags are slightly modified:
```
python configure.py
CC=clang-19 bazel build -c opt --features=thin_lto --copt=-march=armv8-a+simd --cxxopt="-std=c++17" --copt=-fsized-deallocation --copt=-w :build_pip_pkg
./bazel-bin/build_pip_pkg
```
A .whl file should appear in the root of the repository upon successful
completion of these commands. This .whl can be installed via pip.
Raw data
{
"_id": null,
"home_page": null,
"name": "scann",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.14,>=3.9",
"maintainer_email": null,
"keywords": "machine learning",
"author": null,
"author_email": "\"Google Inc.\" <opensource@google.com>",
"download_url": null,
"platform": null,
"description": "# ScaNN\n\nScaNN (Scalable Nearest Neighbors) is a method for efficient vector similarity\nsearch at scale. This code implements [1, 2], which includes search space\npruning and quantization for Maximum Inner Product Search and also supports\nother distance functions such as Euclidean distance. The implementation is\noptimized for x86 processors with AVX support. ScaNN achieves state-of-the-art\nperformance on [ann-benchmarks.com](http://ann-benchmarks.com) as shown on the\nglove-100-angular dataset below:\n\n\n\nScaNN can be configured to fit datasets with different sizes and distributions.\nIt has both TensorFlow and Python APIs. The library shows strong performance\nwith large datasets [1, 2]. The code is released for research purposes. For more\ndetails on the academic description of algorithms, please see below.\n\nReferences:\n\n1. ```\n @inproceedings{avq_2020,\n title={Accelerating Large-Scale Inference with Anisotropic Vector Quantization},\n author={Guo, Ruiqi and Sun, Philip and Lindgren, Erik and Geng, Quan and Simcha, David and Chern, Felix and Kumar, Sanjiv},\n booktitle={International Conference on Machine Learning},\n year={2020},\n URL={https://arxiv.org/abs/1908.10396}\n }\n ```\n\n1. ```\n @inproceedings{soar_2023,\n title={SOAR: Improved Indexing for Approximate Nearest Neighbor Search},\n author={Sun, Philip and Simcha, David and Dopson, Dave and Guo, Ruiqi and Kumar, Sanjiv},\n booktitle={Neural Information Processing Systems},\n year={2023},\n URL={https://arxiv.org/abs/2404.00774}\n }\n ```\n\n## Installation\n\n`manylinux_2_27`-compatible wheels are available on\n[PyPI](https://pypi.org/project/scann/):\n\n```\npip install scann\n```\n\nScaNN supports Linux environments running Python versions 3.9-3.13. See\n[docs/releases.md](docs/releases.md) for release notes; the page also contains\ndownload links for ScaNN wheels prior to version 1.1.0, which were not released\non PyPI. **The x86 wheels require AVX and FMA instruction set support, while\nthe ARM wheels require NEON**.\n\nIn accordance with the\n[`manylinux_2_27` specification](https://peps.python.org/pep-0600/), ScaNN\nrequires `libstdc++` version 3.4.23 or above from the operating system. See\n[here](https://stackoverflow.com/questions/10354636) for an example of how\nto find your system's `libstdc++` version; it can generally be upgraded by\ninstalling a newer version of `g++`.\n\n### Using ScaNN with TensorFlow\n\nScaNN has optional TensorFlow op bindings that allow ScaNN's nearest neighbor\nsearch functionality to be embedded into a TensorFlow SavedModel. **As of\nScaNN 1.4.0, this functionality is no longer enabled by default, and `pip\ninstall scann[tf]` is now needed to enable the TensorFlow integration.**\n\nIf `scann[tf]` is installed, the TensorFlow ops can be accessed via\n`scann.scann_ops`, and the API is almost identical to `scann.scann_ops_pybind`.\nFor users not already in the TensorFlow ecosystem, the native Python bindings\nin `scann.scann_ops_pybind` are a better fit, which is why the TensorFlow ops\nare an optional extra.\n\n#### Integration with TensorFlow Serving\n\nWe provide custom Docker images of\n[TF Serving](https://github.com/tensorflow/serving) that are linked to the ScaNN\nTF ops. See the [`tf_serving` directory](tf_serving/README.md) for further\ninformation.\n\n## Usage\n\nSee the example in [docs/example.ipynb](docs/example.ipynb). For a more in-depth\nexplanation of ScaNN techniques, see [docs/algorithms.md](docs/algorithms.md).\n\n## Building from source\n\nTo build ScaNN from source, first install the build tool\n[bazel](https://bazel.build) (use version 7.x), Clang 19, and libstdc++ headers\nfor C++17 (which are provided with GCC 9). Additionally, ScaNN requires a modern\nversion of Python (3.9.x or later) and Tensorflow 2.20 installed on that version\nof Python. Once these prerequisites are satisfied, run the following command in\nthe root directory of the repository:\n\n```\npython configure.py\nCC=clang-19 bazel build -c opt --features=thin_lto --copt=-mavx --copt=-mfma --cxxopt=\"-std=c++17\" --copt=-fsized-deallocation --copt=-w :build_pip_pkg\n./bazel-bin/build_pip_pkg\n```\n\nTo build an ARM binary from an ARM machine, the prerequisites are the same, but\nthe compile flags are slightly modified:\n\n```\npython configure.py\nCC=clang-19 bazel build -c opt --features=thin_lto --copt=-march=armv8-a+simd --cxxopt=\"-std=c++17\" --copt=-fsized-deallocation --copt=-w :build_pip_pkg\n./bazel-bin/build_pip_pkg\n```\n\nA .whl file should appear in the root of the repository upon successful\ncompletion of these commands. This .whl can be installed via pip.\n",
"bugtrack_url": null,
"license": null,
"summary": "Scalable Nearest Neighbor search library",
"version": "1.4.2",
"project_urls": {
"Homepage": "https://github.com/google-research/google-research/tree/master/scann"
},
"split_keywords": [
"machine",
"learning"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "6b3501895f20ec61d40ec6ee842965da022e35766fd9acbff669a7723027e46e",
"md5": "acf85a56e080d44f1279c47e9f78683f",
"sha256": "1e781c280a2537cd7d252842d9c3e8a4360db36c356cf62ddec0e4d5f0a766de"
},
"downloads": -1,
"filename": "scann-1.4.2-cp310-cp310-manylinux_2_27_aarch64.whl",
"has_sig": false,
"md5_digest": "acf85a56e080d44f1279c47e9f78683f",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": "<3.14,>=3.9",
"size": 9296344,
"upload_time": "2025-08-29T14:32:29",
"upload_time_iso_8601": "2025-08-29T14:32:29.375926Z",
"url": "https://files.pythonhosted.org/packages/6b/35/01895f20ec61d40ec6ee842965da022e35766fd9acbff669a7723027e46e/scann-1.4.2-cp310-cp310-manylinux_2_27_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "374cdb206f9ec19057e3c837226bd8d7de3f2f5c3c630ccfd962f00c5a2c86ca",
"md5": "446227892b5f067c0ea2294f8e4387a9",
"sha256": "bfc8c661d1a99456b2c841335eaf1bf8e4d04c61c34eac41ab5805fbaf0e3da5"
},
"downloads": -1,
"filename": "scann-1.4.2-cp310-cp310-manylinux_2_27_x86_64.whl",
"has_sig": false,
"md5_digest": "446227892b5f067c0ea2294f8e4387a9",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": "<3.14,>=3.9",
"size": 11576794,
"upload_time": "2025-08-29T14:32:17",
"upload_time_iso_8601": "2025-08-29T14:32:17.202188Z",
"url": "https://files.pythonhosted.org/packages/37/4c/db206f9ec19057e3c837226bd8d7de3f2f5c3c630ccfd962f00c5a2c86ca/scann-1.4.2-cp310-cp310-manylinux_2_27_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "873acdda06cb9c5a4513839649e7d977bfb7fc33621534d702b8bfa62357c588",
"md5": "e51da9c5f386a07f153b33fa5b926f6e",
"sha256": "d83ea277712a509c63a709ef2aaab0861c17490c745ffd781fc18dcbd93b0978"
},
"downloads": -1,
"filename": "scann-1.4.2-cp311-cp311-manylinux_2_27_aarch64.whl",
"has_sig": false,
"md5_digest": "e51da9c5f386a07f153b33fa5b926f6e",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": "<3.14,>=3.9",
"size": 9296343,
"upload_time": "2025-08-29T14:32:31",
"upload_time_iso_8601": "2025-08-29T14:32:31.133226Z",
"url": "https://files.pythonhosted.org/packages/87/3a/cdda06cb9c5a4513839649e7d977bfb7fc33621534d702b8bfa62357c588/scann-1.4.2-cp311-cp311-manylinux_2_27_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "9d6a481749080d7c97a4d76b9e7ce4b5bdca79b7e8faccdb1ce45f74e25c7c3f",
"md5": "c618114b284b60b0a9c96b46867b56d3",
"sha256": "5157287c0e5156bda5850be4257b466120f426f969b8329a69cb5adf74d54e52"
},
"downloads": -1,
"filename": "scann-1.4.2-cp311-cp311-manylinux_2_27_x86_64.whl",
"has_sig": false,
"md5_digest": "c618114b284b60b0a9c96b46867b56d3",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": "<3.14,>=3.9",
"size": 11578992,
"upload_time": "2025-08-29T14:32:19",
"upload_time_iso_8601": "2025-08-29T14:32:19.281497Z",
"url": "https://files.pythonhosted.org/packages/9d/6a/481749080d7c97a4d76b9e7ce4b5bdca79b7e8faccdb1ce45f74e25c7c3f/scann-1.4.2-cp311-cp311-manylinux_2_27_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "85498fd3392977d9aae2277b851190dfcdacb62528bfdd337ee7a24bd820bd0f",
"md5": "2dbc0fa31d3aee117a4c515ef27c2fe3",
"sha256": "3042b599d1ada6c48650663d208359b5ccfcbcbfa4fa75bafb81402e6c87bfa4"
},
"downloads": -1,
"filename": "scann-1.4.2-cp312-cp312-manylinux_2_27_aarch64.whl",
"has_sig": false,
"md5_digest": "2dbc0fa31d3aee117a4c515ef27c2fe3",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": "<3.14,>=3.9",
"size": 9296344,
"upload_time": "2025-08-29T14:32:33",
"upload_time_iso_8601": "2025-08-29T14:32:33.749874Z",
"url": "https://files.pythonhosted.org/packages/85/49/8fd3392977d9aae2277b851190dfcdacb62528bfdd337ee7a24bd820bd0f/scann-1.4.2-cp312-cp312-manylinux_2_27_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "c0f86ecec0c496a1f771c6874dbb00f619ab4ac1e434fa7661e4eedc4e959623",
"md5": "e001ec978340c2ee0f1dba583a936bc2",
"sha256": "c0f45d810e629fdcdf8da546d1e25bde4664d27dcf5c3399aa335b4934a0734c"
},
"downloads": -1,
"filename": "scann-1.4.2-cp312-cp312-manylinux_2_27_x86_64.whl",
"has_sig": false,
"md5_digest": "e001ec978340c2ee0f1dba583a936bc2",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": "<3.14,>=3.9",
"size": 11579291,
"upload_time": "2025-08-29T14:32:22",
"upload_time_iso_8601": "2025-08-29T14:32:22.855354Z",
"url": "https://files.pythonhosted.org/packages/c0/f8/6ecec0c496a1f771c6874dbb00f619ab4ac1e434fa7661e4eedc4e959623/scann-1.4.2-cp312-cp312-manylinux_2_27_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "89a1b868924221e7ec8a8c2e7b74ad9fc730fe5554962dc0d56efc2615d03ea3",
"md5": "416088954042e101321a57f4c9e3416f",
"sha256": "469a4a907ea09a15e4c25850330377bd9967c29177d836600051caf281e33612"
},
"downloads": -1,
"filename": "scann-1.4.2-cp313-cp313-manylinux_2_27_aarch64.whl",
"has_sig": false,
"md5_digest": "416088954042e101321a57f4c9e3416f",
"packagetype": "bdist_wheel",
"python_version": "cp313",
"requires_python": "<3.14,>=3.9",
"size": 9296343,
"upload_time": "2025-08-29T14:32:35",
"upload_time_iso_8601": "2025-08-29T14:32:35.436656Z",
"url": "https://files.pythonhosted.org/packages/89/a1/b868924221e7ec8a8c2e7b74ad9fc730fe5554962dc0d56efc2615d03ea3/scann-1.4.2-cp313-cp313-manylinux_2_27_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "3bb43decfd7039399b6bd9c9fbf0ccda39301bac01c39a09a5a791c8237f5d26",
"md5": "354ab66522e93fddb3d93aac65218c85",
"sha256": "c87e97f91c98d7d1f0bf985b39634e07e1149ba79c20f7dbf9b7b465c94100f2"
},
"downloads": -1,
"filename": "scann-1.4.2-cp313-cp313-manylinux_2_27_x86_64.whl",
"has_sig": false,
"md5_digest": "354ab66522e93fddb3d93aac65218c85",
"packagetype": "bdist_wheel",
"python_version": "cp313",
"requires_python": "<3.14,>=3.9",
"size": 11579811,
"upload_time": "2025-08-29T14:32:25",
"upload_time_iso_8601": "2025-08-29T14:32:25.101268Z",
"url": "https://files.pythonhosted.org/packages/3b/b4/3decfd7039399b6bd9c9fbf0ccda39301bac01c39a09a5a791c8237f5d26/scann-1.4.2-cp313-cp313-manylinux_2_27_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "7c81f5a43f03c9c1dc67e2282faba98ecf243d3ec99bf691640a18de9c86f18a",
"md5": "5c0128c4644dd5167f0326e557833e52",
"sha256": "7de7e076be04d7c1df19dcbc73770ac25646988aa8a2bdf388c7168c5e13b02e"
},
"downloads": -1,
"filename": "scann-1.4.2-cp39-cp39-manylinux_2_27_aarch64.whl",
"has_sig": false,
"md5_digest": "5c0128c4644dd5167f0326e557833e52",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": "<3.14,>=3.9",
"size": 9296340,
"upload_time": "2025-08-29T14:32:37",
"upload_time_iso_8601": "2025-08-29T14:32:37.185549Z",
"url": "https://files.pythonhosted.org/packages/7c/81/f5a43f03c9c1dc67e2282faba98ecf243d3ec99bf691640a18de9c86f18a/scann-1.4.2-cp39-cp39-manylinux_2_27_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "4aa14ac32f25394da7e07b9a8dd419dd65a244d7bb3772e00d9d6d2f8a2a4726",
"md5": "85fac8a46240b4c11e3792aae4c35bc4",
"sha256": "b34f64f7ed9349cad2d5be9dd90f3b203278fc94d2920bc51e95895ae50b22b0"
},
"downloads": -1,
"filename": "scann-1.4.2-cp39-cp39-manylinux_2_27_x86_64.whl",
"has_sig": false,
"md5_digest": "85fac8a46240b4c11e3792aae4c35bc4",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": "<3.14,>=3.9",
"size": 11576807,
"upload_time": "2025-08-29T14:32:27",
"upload_time_iso_8601": "2025-08-29T14:32:27.340016Z",
"url": "https://files.pythonhosted.org/packages/4a/a1/4ac32f25394da7e07b9a8dd419dd65a244d7bb3772e00d9d6d2f8a2a4726/scann-1.4.2-cp39-cp39-manylinux_2_27_x86_64.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-29 14:32:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "google-research",
"github_project": "google-research",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"circle": true,
"lcname": "scann"
}
