# query-farm-duckdb-json-serialization
This Python module provides a [Pydantic](https://docs.pydantic.dev/) parser for [DuckDB](https://duckdb.org) expressions that have been serialized to JSON by the [Airport](https://airport.query.farm) DuckDB extension.
These expressions are used by Apache Arrow Flight servers to perform **predicate pushdown** — enabling the server to filter rows efficiently before sending data to the client.
## Purpose
The module's primary function is to:
- **Parse** DuckDB expressions serialized as JSON.
- **Optionally convert** the parsed expressions back into SQL.
- Allow **server-side row filtering** using DuckDB, before returning data via Arrow Flight.
> **Note**: The JSON format used by [Airport](https://airport.query.farm) differs from the built-in DuckDB JSON serialization. Specifically, binary values are encoded using **Base64** in Airport for UTF-8 compatibility.
---
## Installation
```bash
pip install query-farm-duckdb-json-serialization
```
## API Usage
```python
from query_farm_duckdb_json_serialization.expression import Expression
column_names_by_index = ['first_name', 'last_name']
# If there are multiple expressions passed, these will all
# be logically joined with an AND operator.
#
# The DuckDB data typestypes of the columns bound by the expressions
# will be returned.
sql, bound_types = Expression.convert_to_sql(
source=expressions,
bound_column_names=column_names_by_index
)
```
- `expressions`: JSON-serialized list of DuckDB expression trees.
- `bound_column_names`: Column names indexed as expected by DuckDB.
- `sql`: Reconstructed `SQL WHERE` clause.
- `bound_types`: List of DuckDB data types for the bound columns.
## Input
The structure of DuckDB's serialized expressions may change between versions. Below is a working example.
```sql
CREATE TABLE test_type_int64 (v int64);
INSERT INTO test_type_int64 values (1234567890123456789);
-- This statement will generate the following JSON serialization.
SELECT v FROM test_type_int64 WHERE v = 1234567890123456789;
```
```json
[
{
"expression_class": "BOUND_COMPARISON",
"type": "COMPARE_EQUAL",
"alias": "",
"query_location": 18446744073709551615,
"left": {
"expression_class": "BOUND_COLUMN_REF",
"type": "BOUND_COLUMN_REF",
"alias": "v",
"query_location": 18446744073709551615,
"return_type": {
"id": "BIGINT",
"type_info": null
},
"binding": {
"table_index": 0,
"column_index": 0
},
"depth": 0
},
"right": {
"expression_class": "BOUND_CONSTANT",
"type": "VALUE_CONSTANT",
"alias": "",
"query_location": 18446744073709551615,
"value": {
"type": {
"id": "BIGINT",
"type_info": null
},
"is_null": false,
"value": 1234567890123456789
}
}
}
]
```
## Author
This Python module was created by [Query.Farm](https://query.farm).
## License
MIT License. See [LICENSE](LICENSE) for details.
Raw data
{
"_id": null,
"home_page": null,
"name": "query-farm-duckdb-json-serialization",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "airport, arrow, arrow flight, duckdb, duckdb airport, flight, predicate pushdown, predicates, sql, sql parsing",
"author": null,
"author_email": "Rusty Conover <rusty@query.farm>",
"download_url": "https://files.pythonhosted.org/packages/93/35/2b84db561313b2e29468753488a3bd792afc311e3210dbecd2571266bd46/query_farm_duckdb_json_serialization-0.1.4.tar.gz",
"platform": null,
"description": "# query-farm-duckdb-json-serialization\n\nThis Python module provides a [Pydantic](https://docs.pydantic.dev/) parser for [DuckDB](https://duckdb.org) expressions that have been serialized to JSON by the [Airport](https://airport.query.farm) DuckDB extension.\n\nThese expressions are used by Apache Arrow Flight servers to perform **predicate pushdown** \u2014 enabling the server to filter rows efficiently before sending data to the client.\n\n## Purpose\n\nThe module's primary function is to:\n\n- **Parse** DuckDB expressions serialized as JSON.\n- **Optionally convert** the parsed expressions back into SQL.\n- Allow **server-side row filtering** using DuckDB, before returning data via Arrow Flight.\n\n> **Note**: The JSON format used by [Airport](https://airport.query.farm) differs from the built-in DuckDB JSON serialization. Specifically, binary values are encoded using **Base64** in Airport for UTF-8 compatibility.\n\n---\n\n## Installation\n\n```bash\npip install query-farm-duckdb-json-serialization\n```\n\n## API Usage\n\n```python\nfrom query_farm_duckdb_json_serialization.expression import Expression\n\ncolumn_names_by_index = ['first_name', 'last_name']\n# If there are multiple expressions passed, these will all\n# be logically joined with an AND operator.\n#\n# The DuckDB data typestypes of the columns bound by the expressions\n# will be returned.\nsql, bound_types = Expression.convert_to_sql(\n source=expressions,\n bound_column_names=column_names_by_index\n)\n\n```\n\n- `expressions`: JSON-serialized list of DuckDB expression trees.\n- `bound_column_names`: Column names indexed as expected by DuckDB.\n- `sql`: Reconstructed `SQL WHERE` clause.\n- `bound_types`: List of DuckDB data types for the bound columns.\n\n## Input\n\nThe structure of DuckDB's serialized expressions may change between versions. Below is a working example.\n\n```sql\nCREATE TABLE test_type_int64 (v int64);\nINSERT INTO test_type_int64 values (1234567890123456789);\n\n-- This statement will generate the following JSON serialization.\nSELECT v FROM test_type_int64 WHERE v = 1234567890123456789;\n```\n\n```json\n[\n {\n \"expression_class\": \"BOUND_COMPARISON\",\n \"type\": \"COMPARE_EQUAL\",\n \"alias\": \"\",\n \"query_location\": 18446744073709551615,\n \"left\": {\n \"expression_class\": \"BOUND_COLUMN_REF\",\n \"type\": \"BOUND_COLUMN_REF\",\n \"alias\": \"v\",\n \"query_location\": 18446744073709551615,\n \"return_type\": {\n \"id\": \"BIGINT\",\n \"type_info\": null\n },\n \"binding\": {\n \"table_index\": 0,\n \"column_index\": 0\n },\n \"depth\": 0\n },\n \"right\": {\n \"expression_class\": \"BOUND_CONSTANT\",\n \"type\": \"VALUE_CONSTANT\",\n \"alias\": \"\",\n \"query_location\": 18446744073709551615,\n \"value\": {\n \"type\": {\n \"id\": \"BIGINT\",\n \"type_info\": null\n },\n \"is_null\": false,\n \"value\": 1234567890123456789\n }\n }\n }\n]\n```\n\n## Author\n\nThis Python module was created by [Query.Farm](https://query.farm).\n\n## License\n\nMIT License. See [LICENSE](LICENSE) for details.",
"bugtrack_url": null,
"license": "MIT",
"summary": "Integrate with DuckDB's JSON serialization of expressions and values",
"version": "0.1.4",
"project_urls": {
"Issues": "https://github.com/query-farm/python-duckdb-json-serialization/issues",
"Repository": "https://github.com/query-farm/python-duckdb-json-serilization.git"
},
"split_keywords": [
"airport",
" arrow",
" arrow flight",
" duckdb",
" duckdb airport",
" flight",
" predicate pushdown",
" predicates",
" sql",
" sql parsing"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d99ff8386f8f155cc15cc24f573e307710e620872e29c4bd446aaec76db19bb8",
"md5": "bddf587b0e4f569525dff29fc82beeec",
"sha256": "7d9e27e8000ab5a437b52819fa2e8628a9a578537bcb295af7ab77d32234b244"
},
"downloads": -1,
"filename": "query_farm_duckdb_json_serialization-0.1.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bddf587b0e4f569525dff29fc82beeec",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 14830,
"upload_time": "2025-08-31T13:09:12",
"upload_time_iso_8601": "2025-08-31T13:09:12.865522Z",
"url": "https://files.pythonhosted.org/packages/d9/9f/f8386f8f155cc15cc24f573e307710e620872e29c4bd446aaec76db19bb8/query_farm_duckdb_json_serialization-0.1.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "93352b84db561313b2e29468753488a3bd792afc311e3210dbecd2571266bd46",
"md5": "e7ea798edb10445a7ff42e26be8404c7",
"sha256": "fb7750764ee8cd2d29d380130a1264bd6b3e7729e6a516b395e88ffc2bcca3ea"
},
"downloads": -1,
"filename": "query_farm_duckdb_json_serialization-0.1.4.tar.gz",
"has_sig": false,
"md5_digest": "e7ea798edb10445a7ff42e26be8404c7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 14555,
"upload_time": "2025-08-31T13:09:14",
"upload_time_iso_8601": "2025-08-31T13:09:14.084457Z",
"url": "https://files.pythonhosted.org/packages/93/35/2b84db561313b2e29468753488a3bd792afc311e3210dbecd2571266bd46/query_farm_duckdb_json_serialization-0.1.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-31 13:09:14",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "query-farm",
"github_project": "python-duckdb-json-serialization",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "query-farm-duckdb-json-serialization"
}
JSON
