| Name | polars-tdigest JSON |
| Version |
0.1.5
JSON |
| download |
| home_page | None |
| Summary | None |
| upload_time | 2024-10-18 08:02:29 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# T-Digest Polars Plugin
This Polars plugin is a wrapper around the [T-Digest Rust implementation](https://docs.rs/tdigest/latest/tdigest/). It not only provides means to compute an estimated qunatile, but exposes the tdigest creation and merging functionality so it can be used to estimate quantiles in a distributed environment.
For an example see the [Yellow Taxi Notebook](./tdigest_yellow_taxi.ipynb). Note that this example is a bit artifical as it doesn't distribute the computation. It is mainly meant to show how to use the plugin with multiple partitions of a dataset. It does not make sense to use this plugin for computations on a single machine as the tdigest computation essentially adds overhead to the percentile computation and is therefore slower than computing the actual percentile.
# How to contribute
## Dev setup
Setup your virtual environment with a python version `>=3.8`, e.g. use
```bash
python -m venv .env
source .env/bin/activate
``` .
Install the python dependencies used for development:
```bash
python -m pip install -r requirements.txt
```
Install [Rust](https://rustup.rs/).
## Build
In order to build the package, please run `maturin develop`. If you want to test performance, run `maturin develop --release`.
## Developing using cargo
Cargo commands (e.g. `cargo build`, `cargo test`) don't work out of the box.
In order to use cargo instead of maturin for local development, remove `extension-module` from `cargo.toml`:
replace
```
pyo3 = { version = "0.21.2", features = ["extension-module", "abi3-py38"] }
```
with
```
pyo3 = { version = "0.21.2", features = ["abi3-py38"] }
```
## Commit / Release
Before committing and pushing your work, make sure to run
```
cargo fmt --all && cargo clippy --all-features
python -m ruff check . --fix --exit-non-zero-on-fix
```
and resolve any errors.
Raw data
{
"_id": null,
"home_page": null,
"name": "polars-tdigest",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "Thomas Lutterbeck <thomas.lutterbeck@appian.com>",
"download_url": "https://files.pythonhosted.org/packages/1b/b6/ddc326dafb79fa7c949098efb5ff14ec06e101740a668edf97fd13138f88/polars_tdigest-0.1.5.tar.gz",
"platform": null,
"description": "# T-Digest Polars Plugin\n\nThis Polars plugin is a wrapper around the [T-Digest Rust implementation](https://docs.rs/tdigest/latest/tdigest/). It not only provides means to compute an estimated qunatile, but exposes the tdigest creation and merging functionality so it can be used to estimate quantiles in a distributed environment.\n\nFor an example see the [Yellow Taxi Notebook](./tdigest_yellow_taxi.ipynb). Note that this example is a bit artifical as it doesn't distribute the computation. It is mainly meant to show how to use the plugin with multiple partitions of a dataset. It does not make sense to use this plugin for computations on a single machine as the tdigest computation essentially adds overhead to the percentile computation and is therefore slower than computing the actual percentile.\n\n# How to contribute\n\n## Dev setup\n\nSetup your virtual environment with a python version `>=3.8`, e.g. use \n```bash\npython -m venv .env\nsource .env/bin/activate\n``` .\nInstall the python dependencies used for development:\n```bash\npython -m pip install -r requirements.txt\n```\n\nInstall [Rust](https://rustup.rs/).\n\n## Build\n\nIn order to build the package, please run `maturin develop`. If you want to test performance, run `maturin develop --release`. \n\n## Developing using cargo\n\nCargo commands (e.g. `cargo build`, `cargo test`) don't work out of the box. \nIn order to use cargo instead of maturin for local development, remove `extension-module` from `cargo.toml`: \nreplace \n```\npyo3 = { version = \"0.21.2\", features = [\"extension-module\", \"abi3-py38\"] }\n```\nwith \n\n```\npyo3 = { version = \"0.21.2\", features = [\"abi3-py38\"] }\n```\n\n## Commit / Release\n\nBefore committing and pushing your work, make sure to run\n\n```\ncargo fmt --all && cargo clippy --all-features\npython -m ruff check . --fix --exit-non-zero-on-fix\n```\n\nand resolve any errors.\n",
"bugtrack_url": null,
"license": null,
"summary": null,
"version": "0.1.5",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "9270a7efc7cbad52547857f0960a410a42d64bd87f1b1ffd7edc7f66745deaa8",
"md5": "e19c7e7884df0d04ea87c0d58ca27120",
"sha256": "0ec7583546c3994697cd19c14969f0ac4164964540f4c444c51d581a77783c7d"
},
"downloads": -1,
"filename": "polars_tdigest-0.1.5-cp38-abi3-macosx_10_12_x86_64.whl",
"has_sig": false,
"md5_digest": "e19c7e7884df0d04ea87c0d58ca27120",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 3875802,
"upload_time": "2024-10-18T08:02:21",
"upload_time_iso_8601": "2024-10-18T08:02:21.301043Z",
"url": "https://files.pythonhosted.org/packages/92/70/a7efc7cbad52547857f0960a410a42d64bd87f1b1ffd7edc7f66745deaa8/polars_tdigest-0.1.5-cp38-abi3-macosx_10_12_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "f7154e488245c039c452dc35f952853ab7627e58696a66d905cdb8ef6b9d1d83",
"md5": "466a7d3fbe164d5463b7b9022a2e78a3",
"sha256": "371ca4bc0d51b21a01017f2ab21a5d8fa750eba95b6297f354303ce4ba3c86b5"
},
"downloads": -1,
"filename": "polars_tdigest-0.1.5-cp38-abi3-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "466a7d3fbe164d5463b7b9022a2e78a3",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 3571932,
"upload_time": "2024-10-18T08:02:23",
"upload_time_iso_8601": "2024-10-18T08:02:23.149987Z",
"url": "https://files.pythonhosted.org/packages/f7/15/4e488245c039c452dc35f952853ab7627e58696a66d905cdb8ef6b9d1d83/polars_tdigest-0.1.5-cp38-abi3-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "990f52b94dc37af1fd1eac3f921c8b79fecf2db80dfce9f87a4195d2de8eb2ee",
"md5": "a3c8ab78faa1980886586406637ae4f7",
"sha256": "d0d4eaaac7b8cf7dba5afe66fba7a759b21bab7bc36158acb80fa79c538bf3de"
},
"downloads": -1,
"filename": "polars_tdigest-0.1.5-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
"has_sig": false,
"md5_digest": "a3c8ab78faa1980886586406637ae4f7",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 4206199,
"upload_time": "2024-10-18T08:02:24",
"upload_time_iso_8601": "2024-10-18T08:02:24.979019Z",
"url": "https://files.pythonhosted.org/packages/99/0f/52b94dc37af1fd1eac3f921c8b79fecf2db80dfce9f87a4195d2de8eb2ee/polars_tdigest-0.1.5-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "f963e52698192a6cf1e3fd87933e13f0e09386cd66a5a84a8c3ac2d2b3bfcac6",
"md5": "4fe8ed221f143317e09beeb00ead3050",
"sha256": "bc96b04f3f8d0deb8957bf2aa226b77f40b644ecf2b428075f66a3bdd9f2c0e5"
},
"downloads": -1,
"filename": "polars_tdigest-0.1.5-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
"has_sig": false,
"md5_digest": "4fe8ed221f143317e09beeb00ead3050",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 4503879,
"upload_time": "2024-10-18T08:02:26",
"upload_time_iso_8601": "2024-10-18T08:02:26.857633Z",
"url": "https://files.pythonhosted.org/packages/f9/63/e52698192a6cf1e3fd87933e13f0e09386cd66a5a84a8c3ac2d2b3bfcac6/polars_tdigest-0.1.5-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "2d50c8f41cc9dd7f1bd194462209c2bd79b6576bf2dd1f4d2fa8f7873d9bfda1",
"md5": "68fcc08b45f8408e4109c412bc5dc9aa",
"sha256": "de5f51ceb32e095b5137ed28762b5798d5c797d5c4b5fbd0be82c25566d388db"
},
"downloads": -1,
"filename": "polars_tdigest-0.1.5-cp38-abi3-win_amd64.whl",
"has_sig": false,
"md5_digest": "68fcc08b45f8408e4109c412bc5dc9aa",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.8",
"size": 3695383,
"upload_time": "2024-10-18T08:02:28",
"upload_time_iso_8601": "2024-10-18T08:02:28.327299Z",
"url": "https://files.pythonhosted.org/packages/2d/50/c8f41cc9dd7f1bd194462209c2bd79b6576bf2dd1f4d2fa8f7873d9bfda1/polars_tdigest-0.1.5-cp38-abi3-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "1bb6ddc326dafb79fa7c949098efb5ff14ec06e101740a668edf97fd13138f88",
"md5": "29eddf8ed463a7eab24aa1cd74545b10",
"sha256": "8f331e64925327588acc97f4b22ee265574261d01ac827341a77d1bce38985ec"
},
"downloads": -1,
"filename": "polars_tdigest-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "29eddf8ed463a7eab24aa1cd74545b10",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 25712,
"upload_time": "2024-10-18T08:02:29",
"upload_time_iso_8601": "2024-10-18T08:02:29.605052Z",
"url": "https://files.pythonhosted.org/packages/1b/b6/ddc326dafb79fa7c949098efb5ff14ec06e101740a668edf97fd13138f88/polars_tdigest-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-18 08:02:29",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "polars-tdigest"
}
