| Name | dlt JSON |
| Version |
0.5.1
JSON |
| download |
| home_page | https://github.com/dlt-hub |
| Summary | dlt is an open-source python-first scalable data loading library that does not require any backend to run. |
| upload_time | 2024-07-08 14:17:01 |
| maintainer | Marcin Rudolf |
| docs_url | None |
| author | dltHub Inc. |
| requires_python | <3.13,>=3.8.1 |
| license | Apache-2.0 |
| keywords |
etl
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<h1 align="center">
<strong>data load tool (dlt) — the open-source Python library for data loading</strong>
</h1>
<p align="center">
Be it a Google Colab notebook, AWS Lambda function, an Airflow DAG, your local laptop,<br/>or a GPT-4 assisted development playground—<strong>dlt</strong> can be dropped in anywhere.
</p>
<h3 align="center">
🚀 Join our thriving community of likeminded developers and build the future together!
</h3>
<div align="center">
<a target="_blank" href="https://dlthub.com/community" style="background:none">
<img src="https://img.shields.io/badge/slack-join-dlt.svg?labelColor=191937&color=6F6FF7&logo=slack" style="width: 260px;" />
</a>
</div>
<div align="center">
<a target="_blank" href="https://pypi.org/project/dlt/" style="background:none">
<img src="https://img.shields.io/pypi/v/dlt?labelColor=191937&color=6F6FF7">
</a>
<a target="_blank" href="https://pypi.org/project/dlt/" style="background:none">
<img src="https://img.shields.io/pypi/pyversions/dlt?labelColor=191937&color=6F6FF7">
</a>
</div>
## Installation
dlt supports Python 3.8+.
```sh
pip install dlt
```
More options: [Install via Conda or Pixi](https://dlthub.com/docs/reference/installation#install-dlt-via-pixi-and-conda)
## Quick Start
Load chess game data from chess.com API and save it in DuckDB:
```python
import dlt
from dlt.sources.helpers import requests
# Create a dlt pipeline that will load
# chess player data to the DuckDB destination
pipeline = dlt.pipeline(
pipeline_name='chess_pipeline',
destination='duckdb',
dataset_name='player_data'
)
# Grab some player data from Chess.com API
data = []
for player in ['magnuscarlsen', 'rpragchess']:
response = requests.get(f'https://api.chess.com/pub/player/{player}')
response.raise_for_status()
data.append(response.json())
# Extract, normalize, and load the data
pipeline.run(data, table_name='player')
```
Try it out in our **[Colab Demo](https://colab.research.google.com/drive/1NfSB1DpwbbHX9_t5vlalBTf13utwpMGx?usp=sharing)**
## Features
- **Automatic Schema:** Data structure inspection and schema creation for the destination.
- **Data Normalization:** Consistent and verified data before loading.
- **Seamless Integration:** Colab, AWS Lambda, Airflow, and local environments.
- **Scalable:** Adapts to growing data needs in production.
- **Easy Maintenance:** Clear data pipeline structure for updates.
- **Rapid Exploration:** Quickly explore and gain insights from new data sources.
- **Versatile Usage:** Suitable for ad-hoc exploration to advanced loading infrastructures.
- **Start in Seconds with CLI:** Powerful CLI for managing, deploying and inspecting local pipelines.
- **Incremental Loading:** Load only new or changed data and avoid loading old records again.
- **Open Source:** Free and Apache 2.0 Licensed.
## Ready to use Sources and Destinations
Explore ready to use sources (e.g. Google Sheets) in the [Verified Sources docs](https://dlthub.com/docs/dlt-ecosystem/verified-sources) and supported destinations (e.g. DuckDB) in the [Destinations docs](https://dlthub.com/docs/dlt-ecosystem/destinations).
## Documentation
For detailed usage and configuration, please refer to the [official documentation](https://dlthub.com/docs).
## Examples
You can find examples for various use cases in the [examples](docs/examples) folder.
## Adding as dependency
`dlt` follows the semantic versioning with the [`MAJOR.MINOR.PATCH`](https://peps.python.org/pep-0440/#semantic-versioning) pattern. Currently, we are using **pre-release versioning** with the major version being 0.
- `minor` version change means breaking changes
- `patch` version change means new features that should be backward compatible
- any suffix change, e.g., `post10` -> `post11`, is considered a patch
We suggest that you allow only `patch` level updates automatically:
* Using the [Compatible Release Specifier](https://packaging.python.org/en/latest/specifications/version-specifiers/#compatible-release). For example **dlt~=0.3.10** allows only versions **>=0.3.10** and less than **<0.4**
* Poetry [caret requirements](https://python-poetry.org/docs/dependency-specification/). For example **^0.3.10** allows only versions **>=0.3.10** to **<0.4**
## Get Involved
The dlt project is quickly growing, and we're excited to have you join our community! Here's how you can get involved:
- **Connect with the Community**: Join other dlt users and contributors on our [Slack](https://dlthub.com/community)
- **Report issues and suggest features**: Please use the [GitHub Issues](https://github.com/dlt-hub/dlt/issues) to report bugs or suggest new features. Before creating a new issue, make sure to search the tracker for possible duplicates and add a comment if you find one.
- **Track progress of our work and our plans**: Please check out our [public Github project](https://github.com/orgs/dlt-hub/projects/9)
- **Contribute Verified Sources**: Contribute your custom sources to the [dlt-hub/verified-sources](https://github.com/dlt-hub/verified-sources) to help other folks in handling their data tasks.
- **Contribute code**: Check out our [contributing guidelines](CONTRIBUTING.md) for information on how to make a pull request.
- **Improve documentation**: Help us enhance the dlt documentation.
## License
`dlt` is released under the [Apache 2.0 License](LICENSE.txt).
Raw data
{
"_id": null,
"home_page": "https://github.com/dlt-hub",
"name": "dlt",
"maintainer": "Marcin Rudolf",
"docs_url": null,
"requires_python": "<3.13,>=3.8.1",
"maintainer_email": "marcin@dlthub.com",
"keywords": "etl",
"author": "dltHub Inc.",
"author_email": "services@dlthub.com",
"download_url": "https://files.pythonhosted.org/packages/6e/b4/d69408aecb610a16f5ab0818be9b2d42b1cff69201c3897922564c936254/dlt-0.5.1.tar.gz",
"platform": null,
"description": "<h1 align=\"center\">\n <strong>data load tool (dlt) \u2014 the open-source Python library for data loading</strong>\n</h1>\n<p align=\"center\">\nBe it a Google Colab notebook, AWS Lambda function, an Airflow DAG, your local laptop,<br/>or a GPT-4 assisted development playground\u2014<strong>dlt</strong> can be dropped in anywhere.\n</p>\n\n\n<h3 align=\"center\">\n\n\ud83d\ude80 Join our thriving community of likeminded developers and build the future together!\n\n</h3>\n\n<div align=\"center\">\n <a target=\"_blank\" href=\"https://dlthub.com/community\" style=\"background:none\">\n <img src=\"https://img.shields.io/badge/slack-join-dlt.svg?labelColor=191937&color=6F6FF7&logo=slack\" style=\"width: 260px;\" />\n </a>\n</div>\n<div align=\"center\">\n <a target=\"_blank\" href=\"https://pypi.org/project/dlt/\" style=\"background:none\">\n <img src=\"https://img.shields.io/pypi/v/dlt?labelColor=191937&color=6F6FF7\">\n </a>\n <a target=\"_blank\" href=\"https://pypi.org/project/dlt/\" style=\"background:none\">\n <img src=\"https://img.shields.io/pypi/pyversions/dlt?labelColor=191937&color=6F6FF7\">\n </a>\n</div>\n\n## Installation\n\ndlt supports Python 3.8+.\n\n```sh\npip install dlt\n```\n\nMore options: [Install via Conda or Pixi](https://dlthub.com/docs/reference/installation#install-dlt-via-pixi-and-conda)\n\n\n## Quick Start\n\nLoad chess game data from chess.com API and save it in DuckDB:\n\n```python\nimport dlt\nfrom dlt.sources.helpers import requests\n\n# Create a dlt pipeline that will load\n# chess player data to the DuckDB destination\npipeline = dlt.pipeline(\n pipeline_name='chess_pipeline',\n destination='duckdb',\n dataset_name='player_data'\n)\n\n# Grab some player data from Chess.com API\ndata = []\nfor player in ['magnuscarlsen', 'rpragchess']:\n response = requests.get(f'https://api.chess.com/pub/player/{player}')\n response.raise_for_status()\n data.append(response.json())\n\n# Extract, normalize, and load the data\npipeline.run(data, table_name='player')\n```\n\n\nTry it out in our **[Colab Demo](https://colab.research.google.com/drive/1NfSB1DpwbbHX9_t5vlalBTf13utwpMGx?usp=sharing)**\n\n## Features\n\n- **Automatic Schema:** Data structure inspection and schema creation for the destination.\n- **Data Normalization:** Consistent and verified data before loading.\n- **Seamless Integration:** Colab, AWS Lambda, Airflow, and local environments.\n- **Scalable:** Adapts to growing data needs in production.\n- **Easy Maintenance:** Clear data pipeline structure for updates.\n- **Rapid Exploration:** Quickly explore and gain insights from new data sources.\n- **Versatile Usage:** Suitable for ad-hoc exploration to advanced loading infrastructures.\n- **Start in Seconds with CLI:** Powerful CLI for managing, deploying and inspecting local pipelines.\n- **Incremental Loading:** Load only new or changed data and avoid loading old records again.\n- **Open Source:** Free and Apache 2.0 Licensed.\n\n## Ready to use Sources and Destinations\n\nExplore ready to use sources (e.g. Google Sheets) in the [Verified Sources docs](https://dlthub.com/docs/dlt-ecosystem/verified-sources) and supported destinations (e.g. DuckDB) in the [Destinations docs](https://dlthub.com/docs/dlt-ecosystem/destinations).\n\n## Documentation\n\nFor detailed usage and configuration, please refer to the [official documentation](https://dlthub.com/docs).\n\n## Examples\n\nYou can find examples for various use cases in the [examples](docs/examples) folder.\n\n## Adding as dependency\n\n`dlt` follows the semantic versioning with the [`MAJOR.MINOR.PATCH`](https://peps.python.org/pep-0440/#semantic-versioning) pattern. Currently, we are using **pre-release versioning** with the major version being 0.\n\n- `minor` version change means breaking changes\n- `patch` version change means new features that should be backward compatible\n- any suffix change, e.g., `post10` -> `post11`, is considered a patch\n\nWe suggest that you allow only `patch` level updates automatically:\n* Using the [Compatible Release Specifier](https://packaging.python.org/en/latest/specifications/version-specifiers/#compatible-release). For example **dlt~=0.3.10** allows only versions **>=0.3.10** and less than **<0.4**\n* Poetry [caret requirements](https://python-poetry.org/docs/dependency-specification/). For example **^0.3.10** allows only versions **>=0.3.10** to **<0.4**\n## Get Involved\n\nThe dlt project is quickly growing, and we're excited to have you join our community! Here's how you can get involved:\n\n- **Connect with the Community**: Join other dlt users and contributors on our [Slack](https://dlthub.com/community)\n- **Report issues and suggest features**: Please use the [GitHub Issues](https://github.com/dlt-hub/dlt/issues) to report bugs or suggest new features. Before creating a new issue, make sure to search the tracker for possible duplicates and add a comment if you find one.\n- **Track progress of our work and our plans**: Please check out our [public Github project](https://github.com/orgs/dlt-hub/projects/9)\n- **Contribute Verified Sources**: Contribute your custom sources to the [dlt-hub/verified-sources](https://github.com/dlt-hub/verified-sources) to help other folks in handling their data tasks.\n- **Contribute code**: Check out our [contributing guidelines](CONTRIBUTING.md) for information on how to make a pull request.\n- **Improve documentation**: Help us enhance the dlt documentation.\n\n## License\n\n`dlt` is released under the [Apache 2.0 License](LICENSE.txt).\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "dlt is an open-source python-first scalable data loading library that does not require any backend to run.",
"version": "0.5.1",
"project_urls": {
"Homepage": "https://github.com/dlt-hub",
"Repository": "https://github.com/dlt-hub/dlt"
},
"split_keywords": [
"etl"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4d8ba73fd1bf03fd47873e626be984110aa5290551d082a4dccb20914456fb85",
"md5": "fc8d0bea9bcb80863b703a278c6d3389",
"sha256": "19d7920816fadd049a1a92c4ecc4e740bf1cd5f6484fce535715f8c214a835ce"
},
"downloads": -1,
"filename": "dlt-0.5.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fc8d0bea9bcb80863b703a278c6d3389",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.8.1",
"size": 712328,
"upload_time": "2024-07-08T14:15:30",
"upload_time_iso_8601": "2024-07-08T14:15:30.470499Z",
"url": "https://files.pythonhosted.org/packages/4d/8b/a73fd1bf03fd47873e626be984110aa5290551d082a4dccb20914456fb85/dlt-0.5.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6eb4d69408aecb610a16f5ab0818be9b2d42b1cff69201c3897922564c936254",
"md5": "edce906aef095c826cd5612786dba55c",
"sha256": "dfa11e498feec3aca0022541850b6fa7dd3fddce4c2e6fef195530638269fec4"
},
"downloads": -1,
"filename": "dlt-0.5.1.tar.gz",
"has_sig": false,
"md5_digest": "edce906aef095c826cd5612786dba55c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.8.1",
"size": 558588,
"upload_time": "2024-07-08T14:17:01",
"upload_time_iso_8601": "2024-07-08T14:17:01.371193Z",
"url": "https://files.pythonhosted.org/packages/6e/b4/d69408aecb610a16f5ab0818be9b2d42b1cff69201c3897922564c936254/dlt-0.5.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-08 14:17:01",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dlt-hub",
"github_project": "dlt",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "dlt"
}
