| Name | livecsv JSON |
| Version |
0.3
JSON |
| download |
| home_page | None |
| Summary | A custom SQLAlchemy dialect to load CSV from the web (using DuckDB for caching) |
| upload_time | 2025-02-12 20:39:12 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | MIT License
Copyright (c) 2025 eyal-erknet
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
|
| keywords |
sqlalchemy
sql
csv
duckdb
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# livecsv
**livecsv** is a custom SQLAlchemy dialect that loads CSV data from a remote source into an in‑memory DuckDB instance with caching support. It is designed for read‑only use and allows you to query CSV data as if it were a relational table.
## Installation
Install livecsv using pip:
```bash
pip install livecsv
```
Dependencies
livecsv depends on the following packages:
• SQLAlchemy
• duckdb
• duckdb_engine
These dependencies will be automatically installed when you install livecsv.
## Usage
The livecsv dialect lets you create a SQLAlchemy engine with a custom connection string that loads CSV data from a remote URL. The CSV is loaded into an in‑memory DuckDB instance, and the data is cached for a configurable number of minutes.
Connection String Format
The connection string format for livecsv is:
livecsv://<ssl_mode>/<cache_minutes>/<table_name>/<csv_url>
• ssl_mode: Either secure (for HTTPS) or insecure (for HTTP).
• cache_minutes: The number of minutes to cache the CSV data before refreshing. If 0, it is unlimited (not refreshed).
• table_name: The name of the table that will be created in the in‑memory database.
• csv_url: The URL to the CSV file (if the URL does not start with http, a scheme will be automatically prepended based on the ssl_mode).
## Example
Below is a sample code snippet that demonstrates how to use livecsv:
from sqlalchemy import create_engine, text
# Create an engine using the livecsv dialect.
engine = create_engine(
"livecsv://secure/10/usernames/support.staffbase.com/hc/en-us/article_attachments/360009197031/username.csv"
)
# Query the table created from the CSV.
with engine.connect() as conn:
result = conn.execute(text("SELECT * FROM usernames LIMIT 1")).fetchall()
for row in result:
print(row)
In this example, livecsv:
• Loads the CSV from the specified URL.
• Creates a table named usernames.
• Caches the data for 10 minutes.
• Allows you to query the data using SQLAlchemy.
## Testing with pytest
livecsv includes tests that can be run using pytest.
Steps to Run Tests
1. Install pytest (if you haven’t already):
```bash
pip install pytest
```
2. Run pytest
From the root of your project, run:
```bash
pytest
```
## License
This project is licensed under the MIT License. See the LICENSE file for details.
## Author
Eyal Rahmani
Raw data
{
"_id": null,
"home_page": null,
"name": "livecsv",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Eyal Rahmani <eyal.rahmani@med.uni-heidelberg.de>",
"keywords": "sqlalchemy, sql, csv, duckdb",
"author": null,
"author_email": "Eyal Rahmani <eyal.rahmani@med.uni-heidelberg.de>",
"download_url": "https://files.pythonhosted.org/packages/55/45/0e0d77718e9cd418fc30eddde57ca18b661447e17923c32a05dba3eece43/livecsv-0.3.tar.gz",
"platform": null,
"description": "# livecsv\n\n**livecsv** is a custom SQLAlchemy dialect that loads CSV data from a remote source into an in\u2011memory DuckDB instance with caching support. It is designed for read\u2011only use and allows you to query CSV data as if it were a relational table.\n\n## Installation\n\nInstall livecsv using pip:\n\n```bash\npip install livecsv\n```\n\nDependencies\n\nlivecsv depends on the following packages:\n\u2022\tSQLAlchemy\n\u2022\tduckdb\n\u2022\tduckdb_engine\n\nThese dependencies will be automatically installed when you install livecsv.\n\n## Usage\n\nThe livecsv dialect lets you create a SQLAlchemy engine with a custom connection string that loads CSV data from a remote URL. The CSV is loaded into an in\u2011memory DuckDB instance, and the data is cached for a configurable number of minutes.\n\nConnection String Format\n\nThe connection string format for livecsv is:\n\nlivecsv://<ssl_mode>/<cache_minutes>/<table_name>/<csv_url>\n\n\u2022\tssl_mode: Either secure (for HTTPS) or insecure (for HTTP).\n\u2022\tcache_minutes: The number of minutes to cache the CSV data before refreshing. If 0, it is unlimited (not refreshed).\n\u2022\ttable_name: The name of the table that will be created in the in\u2011memory database.\n\u2022\tcsv_url: The URL to the CSV file (if the URL does not start with http, a scheme will be automatically prepended based on the ssl_mode).\n\n## Example\n\nBelow is a sample code snippet that demonstrates how to use livecsv:\n\nfrom sqlalchemy import create_engine, text\n\n# Create an engine using the livecsv dialect.\nengine = create_engine(\n \"livecsv://secure/10/usernames/support.staffbase.com/hc/en-us/article_attachments/360009197031/username.csv\"\n)\n\n# Query the table created from the CSV.\nwith engine.connect() as conn:\n result = conn.execute(text(\"SELECT * FROM usernames LIMIT 1\")).fetchall()\n for row in result:\n print(row)\n\nIn this example, livecsv:\n\u2022\tLoads the CSV from the specified URL.\n\u2022\tCreates a table named usernames.\n\u2022\tCaches the data for 10 minutes.\n\u2022\tAllows you to query the data using SQLAlchemy.\n\n## Testing with pytest\n\nlivecsv includes tests that can be run using pytest.\n\nSteps to Run Tests\n1.\tInstall pytest (if you haven\u2019t already):\n\n```bash\npip install pytest\n```\n\n2.\tRun pytest\nFrom the root of your project, run:\n\n```bash\npytest\n```\n\n## License\n\nThis project is licensed under the MIT License. See the LICENSE file for details.\n\n## Author\nEyal Rahmani\n",
"bugtrack_url": null,
"license": "MIT License\n \n Copyright (c) 2025 eyal-erknet\n \n Permission is hereby granted, free of charge, to any person obtaining a copy\n of this software and associated documentation files (the \"Software\"), to deal\n in the Software without restriction, including without limitation the rights\n to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n copies of the Software, and to permit persons to whom the Software is\n furnished to do so, subject to the following conditions:\n \n The above copyright notice and this permission notice shall be included in all\n copies or substantial portions of the Software.\n \n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n SOFTWARE.\n ",
"summary": "A custom SQLAlchemy dialect to load CSV from the web (using DuckDB for caching)",
"version": "0.3",
"project_urls": {
"Homepage": "https://github.com/eyal-erknet/livecsv",
"Repository": "https://github.com/eyal-erknet/livecsv.git"
},
"split_keywords": [
"sqlalchemy",
" sql",
" csv",
" duckdb"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "aadd56202048cdd639f35406976e5f9ac7237a28871739fa0eaff6624197cc87",
"md5": "e6ad64e8ef0da586c9aa5ca110d80885",
"sha256": "248336441f51682d975702991747414a4e7e59886cb1a7034503a04a228a5fb6"
},
"downloads": -1,
"filename": "livecsv-0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e6ad64e8ef0da586c9aa5ca110d80885",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 6710,
"upload_time": "2025-02-12T20:39:11",
"upload_time_iso_8601": "2025-02-12T20:39:11.261000Z",
"url": "https://files.pythonhosted.org/packages/aa/dd/56202048cdd639f35406976e5f9ac7237a28871739fa0eaff6624197cc87/livecsv-0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "55450e0d77718e9cd418fc30eddde57ca18b661447e17923c32a05dba3eece43",
"md5": "d0b591f0d0b9b0bb5926290dd695fe17",
"sha256": "ef764c35762d78d63e56f53804c4d852f3547cec3a038ff02ed26e8d4273ca43"
},
"downloads": -1,
"filename": "livecsv-0.3.tar.gz",
"has_sig": false,
"md5_digest": "d0b591f0d0b9b0bb5926290dd695fe17",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 8421,
"upload_time": "2025-02-12T20:39:12",
"upload_time_iso_8601": "2025-02-12T20:39:12.576933Z",
"url": "https://files.pythonhosted.org/packages/55/45/0e0d77718e9cd418fc30eddde57ca18b661447e17923c32a05dba3eece43/livecsv-0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-12 20:39:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "eyal-erknet",
"github_project": "livecsv",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "livecsv"
}
