| Name | dlt JSON |
| Version |
0.4.9
JSON |
| download |
| home_page | https://github.com/dlt-hub |
| Summary | dlt is an open-source python-first scalable data loading library that does not require any backend to run. |
| upload_time | 2024-04-25 05:42:23 |
| maintainer | Marcin Rudolf |
| docs_url | None |
| author | dltHub Inc. |
| requires_python | <3.13,>=3.8.1 |
| license | Apache-2.0 |
| keywords |
etl
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<h1 align="center">
<strong>data load tool (dlt) — the open-source Python library for data loading</strong>
</h1>
<p align="center">
Be it a Google Colab notebook, AWS Lambda function, an Airflow DAG, your local laptop,<br/>or a GPT-4 assisted development playground—<strong>dlt</strong> can be dropped in anywhere.
</p>
<h3 align="center">
🚀 Join our thriving community of likeminded developers and build the future together!
</h3>
<div align="center">
<a target="_blank" href="https://dlthub.com/community" style="background:none">
<img src="https://img.shields.io/badge/slack-join-dlt.svg?labelColor=191937&color=6F6FF7&logo=slack" style="width: 260px;" />
</a>
</div>
<div align="center">
<a target="_blank" href="https://pypi.org/project/dlt/" style="background:none">
<img src="https://img.shields.io/pypi/v/dlt?labelColor=191937&color=6F6FF7">
</a>
<a target="_blank" href="https://pypi.org/project/dlt/" style="background:none">
<img src="https://img.shields.io/pypi/pyversions/dlt?labelColor=191937&color=6F6FF7">
</a>
</div>
## Installation
dlt supports Python 3.8+.
```bash
pip install dlt
```
## Quick Start
Load chess game data from chess.com API and save it in DuckDB:
```python
import dlt
from dlt.sources.helpers import requests
# Create a dlt pipeline that will load
# chess player data to the DuckDB destination
pipeline = dlt.pipeline(
pipeline_name='chess_pipeline',
destination='duckdb',
dataset_name='player_data'
)
# Grab some player data from Chess.com API
data = []
for player in ['magnuscarlsen', 'rpragchess']:
response = requests.get(f'https://api.chess.com/pub/player/{player}')
response.raise_for_status()
data.append(response.json())
# Extract, normalize, and load the data
pipeline.run(data, table_name='player')
```
Try it out in our **[Colab Demo](https://colab.research.google.com/drive/1NfSB1DpwbbHX9_t5vlalBTf13utwpMGx?usp=sharing)**
## Features
- **Automatic Schema:** Data structure inspection and schema creation for the destination.
- **Data Normalization:** Consistent and verified data before loading.
- **Seamless Integration:** Colab, AWS Lambda, Airflow, and local environments.
- **Scalable:** Adapts to growing data needs in production.
- **Easy Maintenance:** Clear data pipeline structure for updates.
- **Rapid Exploration:** Quickly explore and gain insights from new data sources.
- **Versatile Usage:** Suitable for ad-hoc exploration to advanced loading infrastructures.
- **Start in Seconds with CLI:** Powerful CLI for managing, deploying and inspecting local pipelines.
- **Incremental Loading:** Load only new or changed data and avoid loading old records again.
- **Open Source:** Free and Apache 2.0 Licensed.
## Ready to use Sources and Destinations
Explore ready to use sources (e.g. Google Sheets) in the [Verified Sources docs](https://dlthub.com/docs/dlt-ecosystem/verified-sources) and supported destinations (e.g. DuckDB) in the [Destinations docs](https://dlthub.com/docs/dlt-ecosystem/destinations).
## Documentation
For detailed usage and configuration, please refer to the [official documentation](https://dlthub.com/docs).
## Examples
You can find examples for various use cases in the [examples](docs/examples) folder.
## Adding as dependency
`dlt` follows the semantic versioning with the [`MAJOR.MINOR.PATCH`](https://peps.python.org/pep-0440/#semantic-versioning) pattern. Currently, we are using **pre-release versioning** with the major version being 0.
- `minor` version change means breaking changes
- `patch` version change means new features that should be backward compatible
- any suffix change, e.g., `post10` -> `post11`, is considered a patch
We suggest that you allow only `patch` level updates automatically:
* Using the [Compatible Release Specifier](https://packaging.python.org/en/latest/specifications/version-specifiers/#compatible-release). For example **dlt~=0.3.10** allows only versions **>=0.3.10** and less than **<0.4**
* Poetry [caret requirements](https://python-poetry.org/docs/dependency-specification/). For example **^0.3.10** allows only versions **>=0.3.10** to **<0.4**
## Get Involved
The dlt project is quickly growing, and we're excited to have you join our community! Here's how you can get involved:
- **Connect with the Community**: Join other dlt users and contributors on our [Slack](https://dlthub.com/community)
- **Report issues and suggest features**: Please use the [GitHub Issues](https://github.com/dlt-hub/dlt/issues) to report bugs or suggest new features. Before creating a new issue, make sure to search the tracker for possible duplicates and add a comment if you find one.
- **Track progress of our work and our plans**: Please check out our [public Github project](https://github.com/orgs/dlt-hub/projects/9)
- **Contribute Verified Sources**: Contribute your custom sources to the [dlt-hub/verified-sources](https://github.com/dlt-hub/verified-sources) to help other folks in handling their data tasks.
- **Contribute code**: Check out our [contributing guidelines](CONTRIBUTING.md) for information on how to make a pull request.
- **Improve documentation**: Help us enhance the dlt documentation.
## License
`dlt` is released under the [Apache 2.0 License](LICENSE.txt).
Raw data
{
"_id": null,
"home_page": "https://github.com/dlt-hub",
"name": "dlt",
"maintainer": "Marcin Rudolf",
"docs_url": null,
"requires_python": "<3.13,>=3.8.1",
"maintainer_email": "marcin@dlthub.com",
"keywords": "etl",
"author": "dltHub Inc.",
"author_email": "services@dlthub.com",
"download_url": "https://files.pythonhosted.org/packages/20/ec/aa18ab1cdfaa0abb8ed3ca7c7da60606c27b2bd019aa0e17361fc16ac42d/dlt-0.4.9.tar.gz",
"platform": null,
"description": "<h1 align=\"center\">\n <strong>data load tool (dlt) \u2014 the open-source Python library for data loading</strong>\n</h1>\n<p align=\"center\">\nBe it a Google Colab notebook, AWS Lambda function, an Airflow DAG, your local laptop,<br/>or a GPT-4 assisted development playground\u2014<strong>dlt</strong> can be dropped in anywhere.\n</p>\n\n\n<h3 align=\"center\">\n\n\ud83d\ude80 Join our thriving community of likeminded developers and build the future together!\n\n</h3>\n\n<div align=\"center\">\n <a target=\"_blank\" href=\"https://dlthub.com/community\" style=\"background:none\">\n <img src=\"https://img.shields.io/badge/slack-join-dlt.svg?labelColor=191937&color=6F6FF7&logo=slack\" style=\"width: 260px;\" />\n </a>\n</div>\n<div align=\"center\">\n <a target=\"_blank\" href=\"https://pypi.org/project/dlt/\" style=\"background:none\">\n <img src=\"https://img.shields.io/pypi/v/dlt?labelColor=191937&color=6F6FF7\">\n </a>\n <a target=\"_blank\" href=\"https://pypi.org/project/dlt/\" style=\"background:none\">\n <img src=\"https://img.shields.io/pypi/pyversions/dlt?labelColor=191937&color=6F6FF7\">\n </a>\n</div>\n\n## Installation\n\ndlt supports Python 3.8+.\n\n```bash\npip install dlt\n```\n\n## Quick Start\n\nLoad chess game data from chess.com API and save it in DuckDB:\n\n```python\nimport dlt\nfrom dlt.sources.helpers import requests\n\n# Create a dlt pipeline that will load\n# chess player data to the DuckDB destination\npipeline = dlt.pipeline(\n pipeline_name='chess_pipeline',\n destination='duckdb',\n dataset_name='player_data'\n)\n\n# Grab some player data from Chess.com API\ndata = []\nfor player in ['magnuscarlsen', 'rpragchess']:\n response = requests.get(f'https://api.chess.com/pub/player/{player}')\n response.raise_for_status()\n data.append(response.json())\n\n# Extract, normalize, and load the data\npipeline.run(data, table_name='player')\n```\n\n\nTry it out in our **[Colab Demo](https://colab.research.google.com/drive/1NfSB1DpwbbHX9_t5vlalBTf13utwpMGx?usp=sharing)**\n\n## Features\n\n- **Automatic Schema:** Data structure inspection and schema creation for the destination.\n- **Data Normalization:** Consistent and verified data before loading.\n- **Seamless Integration:** Colab, AWS Lambda, Airflow, and local environments.\n- **Scalable:** Adapts to growing data needs in production.\n- **Easy Maintenance:** Clear data pipeline structure for updates.\n- **Rapid Exploration:** Quickly explore and gain insights from new data sources.\n- **Versatile Usage:** Suitable for ad-hoc exploration to advanced loading infrastructures.\n- **Start in Seconds with CLI:** Powerful CLI for managing, deploying and inspecting local pipelines.\n- **Incremental Loading:** Load only new or changed data and avoid loading old records again.\n- **Open Source:** Free and Apache 2.0 Licensed.\n\n## Ready to use Sources and Destinations\n\nExplore ready to use sources (e.g. Google Sheets) in the [Verified Sources docs](https://dlthub.com/docs/dlt-ecosystem/verified-sources) and supported destinations (e.g. DuckDB) in the [Destinations docs](https://dlthub.com/docs/dlt-ecosystem/destinations).\n\n## Documentation\n\nFor detailed usage and configuration, please refer to the [official documentation](https://dlthub.com/docs).\n\n## Examples\n\nYou can find examples for various use cases in the [examples](docs/examples) folder.\n\n## Adding as dependency\n\n`dlt` follows the semantic versioning with the [`MAJOR.MINOR.PATCH`](https://peps.python.org/pep-0440/#semantic-versioning) pattern. Currently, we are using **pre-release versioning** with the major version being 0.\n\n- `minor` version change means breaking changes\n- `patch` version change means new features that should be backward compatible\n- any suffix change, e.g., `post10` -> `post11`, is considered a patch\n\nWe suggest that you allow only `patch` level updates automatically:\n* Using the [Compatible Release Specifier](https://packaging.python.org/en/latest/specifications/version-specifiers/#compatible-release). For example **dlt~=0.3.10** allows only versions **>=0.3.10** and less than **<0.4**\n* Poetry [caret requirements](https://python-poetry.org/docs/dependency-specification/). For example **^0.3.10** allows only versions **>=0.3.10** to **<0.4**\n## Get Involved\n\nThe dlt project is quickly growing, and we're excited to have you join our community! Here's how you can get involved:\n\n- **Connect with the Community**: Join other dlt users and contributors on our [Slack](https://dlthub.com/community)\n- **Report issues and suggest features**: Please use the [GitHub Issues](https://github.com/dlt-hub/dlt/issues) to report bugs or suggest new features. Before creating a new issue, make sure to search the tracker for possible duplicates and add a comment if you find one.\n- **Track progress of our work and our plans**: Please check out our [public Github project](https://github.com/orgs/dlt-hub/projects/9)\n- **Contribute Verified Sources**: Contribute your custom sources to the [dlt-hub/verified-sources](https://github.com/dlt-hub/verified-sources) to help other folks in handling their data tasks.\n- **Contribute code**: Check out our [contributing guidelines](CONTRIBUTING.md) for information on how to make a pull request.\n- **Improve documentation**: Help us enhance the dlt documentation.\n\n## License\n\n`dlt` is released under the [Apache 2.0 License](LICENSE.txt).\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "dlt is an open-source python-first scalable data loading library that does not require any backend to run.",
"version": "0.4.9",
"project_urls": {
"Homepage": "https://github.com/dlt-hub",
"Repository": "https://github.com/dlt-hub/dlt"
},
"split_keywords": [
"etl"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f5cbcb3fedcd40a8633c107107ff1b7a96ea5c002f87e9f7b0a789fee07db0b4",
"md5": "b5e2f459a8311e496abff047728b9464",
"sha256": "45fcee36af84131860807cd8d68ac104cf13995178e2686a8d1356e2bd3a74b9"
},
"downloads": -1,
"filename": "dlt-0.4.9-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b5e2f459a8311e496abff047728b9464",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.8.1",
"size": 632725,
"upload_time": "2024-04-25T05:42:19",
"upload_time_iso_8601": "2024-04-25T05:42:19.725880Z",
"url": "https://files.pythonhosted.org/packages/f5/cb/cb3fedcd40a8633c107107ff1b7a96ea5c002f87e9f7b0a789fee07db0b4/dlt-0.4.9-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "20ecaa18ab1cdfaa0abb8ed3ca7c7da60606c27b2bd019aa0e17361fc16ac42d",
"md5": "75ec950abe7c8b57d20c044c8cc007cd",
"sha256": "a5de3abf648c9e82ba1c3cd47fd06c95045087be5b3734ed8ce5efbe308018e5"
},
"downloads": -1,
"filename": "dlt-0.4.9.tar.gz",
"has_sig": false,
"md5_digest": "75ec950abe7c8b57d20c044c8cc007cd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.8.1",
"size": 489079,
"upload_time": "2024-04-25T05:42:23",
"upload_time_iso_8601": "2024-04-25T05:42:23.965894Z",
"url": "https://files.pythonhosted.org/packages/20/ec/aa18ab1cdfaa0abb8ed3ca7c7da60606c27b2bd019aa0e17361fc16ac42d/dlt-0.4.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-25 05:42:23",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dlt-hub",
"github_project": "dlt",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "dlt"
}
