# prefect-openmetadata
## Welcome!
Using [Prefect](https://prefect.io/) and [OpenMetadata](https://open-metadata.org/) together will help you build and maintain a **data platform you can trust**.
Prefect allows you to coordinate your dataflow and provides visibility into the health of your **workflow execution** and **workflow lineage**. With OpenMetadata integration, you can enrich your orchestration system with metadata about data lineage, data catalog, data quality and governance, giving you more information about the health of your system.
## Getting Started
### Python setup
Requires an installation of Python 3.8+.
We recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.
### Installation
Install `prefect-openmetadata` with `pip`:
```bash
pip install prefect-openmetadata
```
### Install `OpenMetadata` and ``Prefect``
Head over to the [install OpenMetadata](install_openmetadata.md) page for detailed instructions on how to install and configure both platforms.
### Write and run metadata ingestion flow
```python
from prefect_openmetadata.flows import ingest_metadata
config = """See an example in the section: Run ingestion flow"""
if __name__ == "__main__":
ingest_metadata(config)
```
For more details, check the [run ingestion flow](docs/run_ingestion_flow.md) section.
### Schedule a metadata ingestion flow
Simple example:
```bash
prefect deployment build -a -n dev myflow.py:ingest_metadata --interval 900
```
For more details, check the [schedule ingestion flow](schedule_ingestion_flow.md) section.
## Resources
If you encounter any bugs while using `prefect-openmetadata`, feel free to open an issue in the [prefect-openmetadata](https://github.com/PrefectHQ/prefect-openmetadata) repository.
If you have any questions or issues while using `prefect-openmetadata`, you can find help in either the [Prefect Discourse forum](https://discourse.prefect.io/) or the [Prefect Slack community](https://prefect.io/slack).
## Development
If you'd like to install a version of `prefect-openmetadata` for development, clone the repository and perform an editable install with `pip`:
```bash
git clone https://github.com/PrefectHQ/prefect-openmetadata.git
cd prefect-openmetadata/
pip install -e ".[dev]"
# Install linting pre-commit hooks
pre-commit install
```
Raw data
{
"_id": null,
"home_page": "https://github.com/PrefectHQ/prefect-openmetadata",
"name": "prefect-openmetadata",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "prefect",
"author": "prefect-openmetadata",
"author_email": "anna@prefect.io",
"download_url": "https://files.pythonhosted.org/packages/81/54/36a03487acd566d72e7cc28066d7bb442571e6655903308a60d639f6839f/prefect-openmetadata-0.3.0.tar.gz",
"platform": null,
"description": "# prefect-openmetadata\n\n## Welcome!\n\nUsing [Prefect](https://prefect.io/) and [OpenMetadata](https://open-metadata.org/) together will help you build and maintain a **data platform you can trust**. \n\nPrefect allows you to coordinate your dataflow and provides visibility into the health of your **workflow execution** and **workflow lineage**. With OpenMetadata integration, you can enrich your orchestration system with metadata about data lineage, data catalog, data quality and governance, giving you more information about the health of your system. \n\n\n## Getting Started\n\n\n### Python setup\n\nRequires an installation of Python 3.8+.\n\nWe recommend using a Python virtual environment manager such as pipenv, conda or virtualenv.\n\n### Installation\n\nInstall `prefect-openmetadata` with `pip`:\n\n```bash\npip install prefect-openmetadata\n```\n\n### Install `OpenMetadata` and ``Prefect``\n\nHead over to the [install OpenMetadata](install_openmetadata.md) page for detailed instructions on how to install and configure both platforms.\n\n### Write and run metadata ingestion flow\n\n```python\nfrom prefect_openmetadata.flows import ingest_metadata\n\nconfig = \"\"\"See an example in the section: Run ingestion flow\"\"\"\n\nif __name__ == \"__main__\":\n ingest_metadata(config)\n```\n\nFor more details, check the [run ingestion flow](docs/run_ingestion_flow.md) section.\n\n### Schedule a metadata ingestion flow\n\nSimple example:\n```bash\nprefect deployment build -a -n dev myflow.py:ingest_metadata --interval 900\n```\n\nFor more details, check the [schedule ingestion flow](schedule_ingestion_flow.md) section.\n\n\n## Resources\n\nIf you encounter any bugs while using `prefect-openmetadata`, feel free to open an issue in the [prefect-openmetadata](https://github.com/PrefectHQ/prefect-openmetadata) repository.\n\nIf you have any questions or issues while using `prefect-openmetadata`, you can find help in either the [Prefect Discourse forum](https://discourse.prefect.io/) or the [Prefect Slack community](https://prefect.io/slack).\n\n\n## Development\n\nIf you'd like to install a version of `prefect-openmetadata` for development, clone the repository and perform an editable install with `pip`:\n\n```bash\ngit clone https://github.com/PrefectHQ/prefect-openmetadata.git\n\ncd prefect-openmetadata/\n\npip install -e \".[dev]\"\n\n# Install linting pre-commit hooks\npre-commit install\n```\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "Prefect integration with OpenMetadata",
"version": "0.3.0",
"project_urls": {
"Homepage": "https://github.com/PrefectHQ/prefect-openmetadata"
},
"split_keywords": [
"prefect"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "109f8fd72ade6c6847a23fd2f47adcd9f05822607d850b2aabb1f564f2590c78",
"md5": "bc6773382169068b7f14c9b0f98b079f",
"sha256": "0173dc8f923923096f4b622a427d842f29e97b3cddb66098eee62cccd498a2c4"
},
"downloads": -1,
"filename": "prefect_openmetadata-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bc6773382169068b7f14c9b0f98b079f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 10483,
"upload_time": "2023-10-05T18:15:26",
"upload_time_iso_8601": "2023-10-05T18:15:26.615350Z",
"url": "https://files.pythonhosted.org/packages/10/9f/8fd72ade6c6847a23fd2f47adcd9f05822607d850b2aabb1f564f2590c78/prefect_openmetadata-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "815436a03487acd566d72e7cc28066d7bb442571e6655903308a60d639f6839f",
"md5": "be4555d2aa542191d31960014147f9d9",
"sha256": "5216eebd3bc24419f0131e5a944fbba2491940fc62b92b3e7843eda49bbcd576"
},
"downloads": -1,
"filename": "prefect-openmetadata-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "be4555d2aa542191d31960014147f9d9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 27404,
"upload_time": "2023-10-05T18:15:28",
"upload_time_iso_8601": "2023-10-05T18:15:28.828378Z",
"url": "https://files.pythonhosted.org/packages/81/54/36a03487acd566d72e7cc28066d7bb442571e6655903308a60d639f6839f/prefect-openmetadata-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-05 18:15:28",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "PrefectHQ",
"github_project": "prefect-openmetadata",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "prefect-openmetadata"
}
JSON
