| Name | metaphor-connectors JSON |
| Version |
0.13.181
JSON |
| download |
| home_page | https://metaphor.io |
| Summary | A collection of Python-based 'connectors' that extract metadata from various sources to ingest into the Metaphor app. |
| upload_time | 2024-04-24 03:58:54 |
| maintainer | None |
| docs_url | None |
| author | Metaphor |
| requires_python | <3.12,>=3.8.1 |
| license | Apache-2.0 |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<a href="https://metaphor.io"><img src="https://github.com/MetaphorData/connectors/raw/main/logo.png" width="300" /></a>
# Metaphor Connectors
[](https://app.codecov.io/gh/MetaphorData/connectors/tree/main)
[](https://github.com/MetaphorData/connectors/actions/workflows/codeql-analysis.yml)
[](https://pypi.org/project/metaphor-connectors/)
[](https://hub.docker.com/r/metaphordata/connectors)
[](https://github.com/MetaphorData/connectors/blob/master/LICENSE)
This repository contains a collection of Python-based "connectors" that extract metadata from various sources to ingest into the [Metaphor](https://metaphor.io) platform.
## Installation
This package requires Python 3.8+ installed. You can verify the version on your system by running the following command,
```shell
python -V # or python3 on some systems
```
Once verified, you can install the package using [pip](https://docs.python.org/3/installing/index.html),
```shell
pip install "metaphor-connectors[all]" # or pip3 on some systems
```
This will install all the connectors and required dependencies. You can also choose to install only a subset of the dependencies by installing the specific [extra](https://packaging.python.org/tutorials/installing-packages/#installing-setuptools-extras), e.g.
```shell
pip install "metaphor-connectors[snowflake]"
```
Similarly, you can also install the package using `requirements.txt` or `pyproject.toml`.
## Docker
We automatically push a [docker image](https://hub.docker.com/r/metaphordata/connectors) to Docker Hub as part of the CI/CD. See [this page](./docs/docker.md) for more details.
## GitHub Action
You can also run the connectors in your CI/CD pipeline using the [Metaphor Connectors](https://github.com/marketplace/actions/metaphor-connectors-github-action) GitHub Action.
## Connectors
Each connector is placed under its own directory under [metaphor](./metaphor) and extends the `metaphor.common.BaseExtractor` class.
| Connector Name | Metadata |
|-------------------------------------------------------------------|------------------------------------------|
| [azure_data_factory](metaphor/azure_data_factory/) | Lineage, Pipeline |
| [bigquery](metaphor/bigquery/) | Schema, description, statistics, queries |
| [bigquery.lineage](metaphor/bigquery/lineage/) | Lineage |
| [bigquery.profile](metaphor/bigquery/profile/) | Data profile |
| [custom.data_quality](metaphor/custom/data_quality/) | Data quality |
| [custom.governance](metaphor/custom/governance/) | Ownership, tags, description |
| [custom.lineage](metaphor/custom/lineage/) | Lineage |
| [custom.metadata](metaphor/custom/metadata/) | Custom metadata |
| [custom.query_attributions](metaphor/custom/query_attributions/) | Query attritutions |
| [datahub](metaphor/datahub/) | Description, tag, ownership |
| [dbt](metaphor/dbt/) | dbt model, test, lineage |
| [dbt.cloud](metaphor/dbt/cloud/) | dbt model, test, lineage |
| [fivetran](metaphor/fivetran/) | Lineage, Pipeline |
| [glue](metaphor/glue/) | Schema, description |
| [looker](metaphor/looker/) | Looker view, explore, dashboard, lineage |
| [kafka](metaphor/kafka/) | Schema, description |
| [metabase](metaphor/metabase/) | Dashboard, lineage |
| [monte_carlo](metaphor/monte_carlo/) | Data monitor |
| [mssql](metaphor/mssql/) | Schema |
| [mysql](metaphor/mysql/) | Schema, description |
| [notion](metaphor/notion/) | Document embeddings |
| [postgresql](metaphor/postgresql/) | Schema, description, statistics |
| [postgresql.profile](metaphor/postgresql/profile/) | Data profile |
| [postgresql.usage](metaphor/postgresql/usage/) | Usage |
| [power_bi](metaphor/power_bi/) | Dashboard, lineage |
| [redshift](metaphor/redshift/) | Schema, description, statistics, queries |
| [redshift.lineage](metaphor/redshift/lineage/) | Lineage |
| [redshift.profile](metaphor/redshift/profile/) | Data profile |
| [snowflake](metaphor/snowflake/) | Schema, description, statistics, queries |
| [snowflake.lineage](metaphor/snowflake/lineage/) | Lineage |
| [snowflake.profile](metaphor/snowflake/profile/) | Data profile |
| [synapse](metaphor/synapse/) | Schema, queries |
| [tableau](metaphor/tableau/) | Dashboard, lineage |
| [thought_spot](metaphor/thought_spot/) | Dashboard, lineage |
| [trino](metaphor/trino/) | Schema, description, queries |
| [unity_catalog](metaphor/unity_catalog/) | Schema, description |
| [unity_catalog.profile](metaphor/unity_catalog/profile/) | Statistics |
## Development
See [Development Environment](docs/develop.md) for more instructions on how to set up your local development environment.
## Custom Connectors
See [Adding a Custom Connector](docs/custom.md) for instructions and a full example of creating your custom connectors.
Raw data
{
"_id": null,
"home_page": "https://metaphor.io",
"name": "metaphor-connectors",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.12,>=3.8.1",
"maintainer_email": null,
"keywords": null,
"author": "Metaphor",
"author_email": "dev@metaphor.io",
"download_url": "https://files.pythonhosted.org/packages/93/0e/601d0bb05b074e03b613507a70c477c409624dc2eb58bc8a6879a7375c88/metaphor_connectors-0.13.181.tar.gz",
"platform": null,
"description": "<a href=\"https://metaphor.io\"><img src=\"https://github.com/MetaphorData/connectors/raw/main/logo.png\" width=\"300\" /></a>\n\n# Metaphor Connectors\n\n[](https://app.codecov.io/gh/MetaphorData/connectors/tree/main)\n[](https://github.com/MetaphorData/connectors/actions/workflows/codeql-analysis.yml)\n[](https://pypi.org/project/metaphor-connectors/)\n\n\n[](https://hub.docker.com/r/metaphordata/connectors)\n[](https://github.com/MetaphorData/connectors/blob/master/LICENSE)\n\nThis repository contains a collection of Python-based \"connectors\" that extract metadata from various sources to ingest into the [Metaphor](https://metaphor.io) platform.\n\n## Installation\n\nThis package requires Python 3.8+ installed. You can verify the version on your system by running the following command,\n\n```shell\npython -V # or python3 on some systems\n```\n\nOnce verified, you can install the package using [pip](https://docs.python.org/3/installing/index.html),\n\n```shell\npip install \"metaphor-connectors[all]\" # or pip3 on some systems\n```\n\nThis will install all the connectors and required dependencies. You can also choose to install only a subset of the dependencies by installing the specific [extra](https://packaging.python.org/tutorials/installing-packages/#installing-setuptools-extras), e.g.\n\n```shell\npip install \"metaphor-connectors[snowflake]\"\n```\n\nSimilarly, you can also install the package using `requirements.txt` or `pyproject.toml`.\n\n## Docker\n\nWe automatically push a [docker image](https://hub.docker.com/r/metaphordata/connectors) to Docker Hub as part of the CI/CD. See [this page](./docs/docker.md) for more details.\n\n## GitHub Action\n\nYou can also run the connectors in your CI/CD pipeline using the [Metaphor Connectors](https://github.com/marketplace/actions/metaphor-connectors-github-action) GitHub Action.\n\n## Connectors\n\nEach connector is placed under its own directory under [metaphor](./metaphor) and extends the `metaphor.common.BaseExtractor` class.\n\n| Connector Name | Metadata |\n|-------------------------------------------------------------------|------------------------------------------| \n| [azure_data_factory](metaphor/azure_data_factory/) | Lineage, Pipeline |\n| [bigquery](metaphor/bigquery/) | Schema, description, statistics, queries |\n| [bigquery.lineage](metaphor/bigquery/lineage/) | Lineage |\n| [bigquery.profile](metaphor/bigquery/profile/) | Data profile |\n| [custom.data_quality](metaphor/custom/data_quality/) | Data quality |\n| [custom.governance](metaphor/custom/governance/) | Ownership, tags, description |\n| [custom.lineage](metaphor/custom/lineage/) | Lineage |\n| [custom.metadata](metaphor/custom/metadata/) | Custom metadata |\n| [custom.query_attributions](metaphor/custom/query_attributions/) | Query attritutions |\n| [datahub](metaphor/datahub/) | Description, tag, ownership |\n| [dbt](metaphor/dbt/) | dbt model, test, lineage |\n| [dbt.cloud](metaphor/dbt/cloud/) | dbt model, test, lineage |\n| [fivetran](metaphor/fivetran/) | Lineage, Pipeline |\n| [glue](metaphor/glue/) | Schema, description |\n| [looker](metaphor/looker/) | Looker view, explore, dashboard, lineage |\n| [kafka](metaphor/kafka/) | Schema, description |\n| [metabase](metaphor/metabase/) | Dashboard, lineage |\n| [monte_carlo](metaphor/monte_carlo/) | Data monitor |\n| [mssql](metaphor/mssql/) | Schema |\n| [mysql](metaphor/mysql/) | Schema, description |\n| [notion](metaphor/notion/) | Document embeddings |\n| [postgresql](metaphor/postgresql/) | Schema, description, statistics |\n| [postgresql.profile](metaphor/postgresql/profile/) | Data profile |\n| [postgresql.usage](metaphor/postgresql/usage/) | Usage |\n| [power_bi](metaphor/power_bi/) | Dashboard, lineage |\n| [redshift](metaphor/redshift/) | Schema, description, statistics, queries |\n| [redshift.lineage](metaphor/redshift/lineage/) | Lineage |\n| [redshift.profile](metaphor/redshift/profile/) | Data profile |\n| [snowflake](metaphor/snowflake/) | Schema, description, statistics, queries |\n| [snowflake.lineage](metaphor/snowflake/lineage/) | Lineage |\n| [snowflake.profile](metaphor/snowflake/profile/) | Data profile |\n| [synapse](metaphor/synapse/) | Schema, queries |\n| [tableau](metaphor/tableau/) | Dashboard, lineage |\n| [thought_spot](metaphor/thought_spot/) | Dashboard, lineage |\n| [trino](metaphor/trino/) | Schema, description, queries | \n| [unity_catalog](metaphor/unity_catalog/) | Schema, description |\n| [unity_catalog.profile](metaphor/unity_catalog/profile/) | Statistics |\n\n## Development\n\nSee [Development Environment](docs/develop.md) for more instructions on how to set up your local development environment.\n\n## Custom Connectors\n\nSee [Adding a Custom Connector](docs/custom.md) for instructions and a full example of creating your custom connectors.\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "A collection of Python-based 'connectors' that extract metadata from various sources to ingest into the Metaphor app.",
"version": "0.13.181",
"project_urls": {
"Homepage": "https://metaphor.io",
"Repository": "https://github.com/MetaphorData/connectors"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2c74f18200a1ba00e2a0d6362664dc151a30ba3b1bcccf2372b2b299acd118c2",
"md5": "231c6d0ac0006f88752cf1d24e9ce121",
"sha256": "7caae812e7eb48c61aa97cbe1652a7a6dfffd0611c435131713e006d8f988285"
},
"downloads": -1,
"filename": "metaphor_connectors-0.13.181-py3-none-any.whl",
"has_sig": false,
"md5_digest": "231c6d0ac0006f88752cf1d24e9ce121",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.12,>=3.8.1",
"size": 358450,
"upload_time": "2024-04-24T03:58:47",
"upload_time_iso_8601": "2024-04-24T03:58:47.517195Z",
"url": "https://files.pythonhosted.org/packages/2c/74/f18200a1ba00e2a0d6362664dc151a30ba3b1bcccf2372b2b299acd118c2/metaphor_connectors-0.13.181-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "930e601d0bb05b074e03b613507a70c477c409624dc2eb58bc8a6879a7375c88",
"md5": "7efebc4ad050344f679d3058db754c61",
"sha256": "10b04f2e98c397994e1210f37bc2f07ce756c521a50f6448002b38a8cf317296"
},
"downloads": -1,
"filename": "metaphor_connectors-0.13.181.tar.gz",
"has_sig": false,
"md5_digest": "7efebc4ad050344f679d3058db754c61",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.12,>=3.8.1",
"size": 257021,
"upload_time": "2024-04-24T03:58:54",
"upload_time_iso_8601": "2024-04-24T03:58:54.228045Z",
"url": "https://files.pythonhosted.org/packages/93/0e/601d0bb05b074e03b613507a70c477c409624dc2eb58bc8a6879a7375c88/metaphor_connectors-0.13.181.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-24 03:58:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MetaphorData",
"github_project": "connectors",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "metaphor-connectors"
}
