| Name | metaphor-connectors JSON |
| Version |
0.14.161
JSON |
| download |
| home_page | https://metaphor.io |
| Summary | A collection of Python-based 'connectors' that extract metadata from various sources to ingest into the Metaphor app. |
| upload_time | 2024-12-16 20:40:05 |
| maintainer | None |
| docs_url | None |
| author | Metaphor |
| requires_python | <3.12,>=3.9 |
| license | Apache-2.0 |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
<a href="https://metaphor.io"><img src="https://github.com/MetaphorData/connectors/raw/main/logo.png" width="300" /></a>
# Metaphor Connectors
[](https://app.codecov.io/gh/MetaphorData/connectors/tree/main)
[](https://github.com/MetaphorData/connectors/actions/workflows/codeql-analysis.yml)
[](https://pypi.org/project/metaphor-connectors/)
[](https://hub.docker.com/r/metaphordata/connectors)
[](https://github.com/MetaphorData/connectors/blob/master/LICENSE)
This repository contains a collection of Python-based "connectors" that extract metadata from various sources to ingest into the [Metaphor](https://metaphor.io) platform.
## Installation
This package requires Python 3.9+ installed. You can verify the version on your system by running the following command,
```shell
python -V # or python3 on some systems
```
Once verified, you can install the package using [pip](https://docs.python.org/3/installing/index.html),
```shell
pip install "metaphor-connectors[all]" # or pip3 on some systems
```
This will install all the connectors and required dependencies. You can also choose to install only a subset of the dependencies by installing the specific [extra](https://packaging.python.org/tutorials/installing-packages/#installing-setuptools-extras), e.g.
```shell
pip install "metaphor-connectors[snowflake]"
```
Similarly, you can also install the package using `requirements.txt` or `pyproject.toml`.
## Docker
We automatically push a [docker image](https://hub.docker.com/r/metaphordata/connectors) to Docker Hub as part of the CI/CD. See [this page](./docs/docker.md) for more details.
## GitHub Action
You can also run the connectors in your CI/CD pipeline using the [Metaphor Connectors](https://github.com/marketplace/actions/metaphor-connectors-github-action) GitHub Action.
## Connectors
Each connector is placed under its own directory under [metaphor](./metaphor) and extends the `metaphor.common.BaseExtractor` class.
| Connector Name | Metadata |
|-------------------------------------------------------------------|------------------------------------------|
| [athena](metaphor/athena/) | Schema, description, queries |
| [azure_data_factory](metaphor/azure_data_factory/) | Lineage, Pipeline |
| [bigquery](metaphor/bigquery/) | Schema, description, statistics, queries |
| [bigquery.lineage](metaphor/bigquery/lineage/) | Lineage |
| [bigquery.profile](metaphor/bigquery/profile/) | Data profile |
| [confluence](metaphor/confluence/) | Document embeddings |
| [custom.data_quality](metaphor/custom/data_quality/) | Data quality |
| [custom.governance](metaphor/custom/governance/) | Ownership, tags, description |
| [custom.lineage](metaphor/custom/lineage/) | Lineage |
| [custom.metadata](metaphor/custom/metadata/) | Custom metadata |
| [custom.query_attributions](metaphor/custom/query_attributions/) | Query attritutions |
| [datahub](metaphor/datahub/) | Description, tag, ownership |
| [dbt](metaphor/dbt/) | dbt model, test, lineage |
| [dbt.cloud](metaphor/dbt/cloud/) | dbt model, test, lineage |
| [fivetran](metaphor/fivetran/) | Lineage, Pipeline |
| [glue](metaphor/glue/) | Schema, description |
| [great_expectations](metaphor/great_expectations/) | Data monitor |
| [informatica](metaphor/informatica/) | Lineage, Pipeline |
| [looker](metaphor/looker/) | Looker view, explore, dashboard, lineage |
| [kafka](metaphor/kafka/) | Schema, description |
| [metabase](metaphor/metabase/) | Dashboard, lineage |
| [mongodb](metaphor/mongodb/) | Schema, statistics |
| [monte_carlo](metaphor/monte_carlo/) | Data monitor |
| [mssql](metaphor/mssql/) | Schema |
| [mysql](metaphor/mysql/) | Schema, description |
| [openapi](metaphor/openapi/) | API, description |
| [oracle](metaphor/oracle/) | Schema, description, queries |
| [notion](metaphor/notion/) | Document embeddings |
| [postgresql](metaphor/postgresql/) | Schema, description, statistics |
| [postgresql.profile](metaphor/postgresql/profile/) | Data profile |
| [power_bi](metaphor/power_bi/) | Dashboard, lineage |
| [quick_sight](metaphor/quick_sight/) | Dashboard, lineage |
| [redshift](metaphor/redshift/) | Schema, description, statistics, queries |
| [redshift.profile](metaphor/redshift/profile/) | Data profile |
| [s3](metaphor/s3/) | Schema, description |
| [sharepoint](metaphor/sharepoint/) | Document embeddings |
| [snowflake](metaphor/snowflake/) | Schema, description, statistics, queries |
| [snowflake.profile](metaphor/snowflake/profile/) | Data profile |
| [static_web](metaphor/static_web/) | Document embeddings |
| [synapse](metaphor/synapse/) | Schema, queries |
| [tableau](metaphor/tableau/) | Dashboard, lineage |
| [thought_spot](metaphor/thought_spot/) | Dashboard, lineage |
| [trino](metaphor/trino/) | Schema, description, queries |
| [unity_catalog](metaphor/unity_catalog/) | Schema, description |
| [unity_catalog.profile](metaphor/unity_catalog/profile/) | Data profile, statistics |
## Development
See [Development Environment](docs/develop.md) for more instructions on how to set up your local development environment.
## Custom Connectors
See [Adding a Custom Connector](docs/custom.md) for instructions and a full example of creating your custom connectors.
Raw data
{
"_id": null,
"home_page": "https://metaphor.io",
"name": "metaphor-connectors",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.12,>=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Metaphor",
"author_email": "dev@metaphor.io",
"download_url": "https://files.pythonhosted.org/packages/5e/18/d856609b5ac5ea4135088888d70dfa6b15cadd372b9d2e607a6439c1dcd9/metaphor_connectors-0.14.161.tar.gz",
"platform": null,
"description": "<a href=\"https://metaphor.io\"><img src=\"https://github.com/MetaphorData/connectors/raw/main/logo.png\" width=\"300\" /></a>\n\n# Metaphor Connectors\n\n[](https://app.codecov.io/gh/MetaphorData/connectors/tree/main)\n[](https://github.com/MetaphorData/connectors/actions/workflows/codeql-analysis.yml)\n[](https://pypi.org/project/metaphor-connectors/)\n\n\n[](https://hub.docker.com/r/metaphordata/connectors)\n[](https://github.com/MetaphorData/connectors/blob/master/LICENSE)\n\nThis repository contains a collection of Python-based \"connectors\" that extract metadata from various sources to ingest into the [Metaphor](https://metaphor.io) platform.\n\n## Installation\n\nThis package requires Python 3.9+ installed. You can verify the version on your system by running the following command,\n\n```shell\npython -V # or python3 on some systems\n```\n\nOnce verified, you can install the package using [pip](https://docs.python.org/3/installing/index.html),\n\n```shell\npip install \"metaphor-connectors[all]\" # or pip3 on some systems\n```\n\nThis will install all the connectors and required dependencies. You can also choose to install only a subset of the dependencies by installing the specific [extra](https://packaging.python.org/tutorials/installing-packages/#installing-setuptools-extras), e.g.\n\n```shell\npip install \"metaphor-connectors[snowflake]\"\n```\n\nSimilarly, you can also install the package using `requirements.txt` or `pyproject.toml`.\n\n## Docker\n\nWe automatically push a [docker image](https://hub.docker.com/r/metaphordata/connectors) to Docker Hub as part of the CI/CD. See [this page](./docs/docker.md) for more details.\n\n## GitHub Action\n\nYou can also run the connectors in your CI/CD pipeline using the [Metaphor Connectors](https://github.com/marketplace/actions/metaphor-connectors-github-action) GitHub Action.\n\n## Connectors\n\nEach connector is placed under its own directory under [metaphor](./metaphor) and extends the `metaphor.common.BaseExtractor` class.\n\n| Connector Name | Metadata |\n|-------------------------------------------------------------------|------------------------------------------| \n| [athena](metaphor/athena/) | Schema, description, queries |\n| [azure_data_factory](metaphor/azure_data_factory/) | Lineage, Pipeline |\n| [bigquery](metaphor/bigquery/) | Schema, description, statistics, queries |\n| [bigquery.lineage](metaphor/bigquery/lineage/) | Lineage |\n| [bigquery.profile](metaphor/bigquery/profile/) | Data profile |\n| [confluence](metaphor/confluence/) | Document embeddings |\n| [custom.data_quality](metaphor/custom/data_quality/) | Data quality |\n| [custom.governance](metaphor/custom/governance/) | Ownership, tags, description |\n| [custom.lineage](metaphor/custom/lineage/) | Lineage |\n| [custom.metadata](metaphor/custom/metadata/) | Custom metadata |\n| [custom.query_attributions](metaphor/custom/query_attributions/) | Query attritutions |\n| [datahub](metaphor/datahub/) | Description, tag, ownership |\n| [dbt](metaphor/dbt/) | dbt model, test, lineage |\n| [dbt.cloud](metaphor/dbt/cloud/) | dbt model, test, lineage |\n| [fivetran](metaphor/fivetran/) | Lineage, Pipeline |\n| [glue](metaphor/glue/) | Schema, description |\n| [great_expectations](metaphor/great_expectations/) | Data monitor |\n| [informatica](metaphor/informatica/) | Lineage, Pipeline |\n| [looker](metaphor/looker/) | Looker view, explore, dashboard, lineage |\n| [kafka](metaphor/kafka/) | Schema, description |\n| [metabase](metaphor/metabase/) | Dashboard, lineage |\n| [mongodb](metaphor/mongodb/) | Schema, statistics |\n| [monte_carlo](metaphor/monte_carlo/) | Data monitor |\n| [mssql](metaphor/mssql/) | Schema |\n| [mysql](metaphor/mysql/) | Schema, description |\n| [openapi](metaphor/openapi/) | API, description |\n| [oracle](metaphor/oracle/) | Schema, description, queries |\n| [notion](metaphor/notion/) | Document embeddings |\n| [postgresql](metaphor/postgresql/) | Schema, description, statistics |\n| [postgresql.profile](metaphor/postgresql/profile/) | Data profile |\n| [power_bi](metaphor/power_bi/) | Dashboard, lineage |\n| [quick_sight](metaphor/quick_sight/) | Dashboard, lineage |\n| [redshift](metaphor/redshift/) | Schema, description, statistics, queries |\n| [redshift.profile](metaphor/redshift/profile/) | Data profile |\n| [s3](metaphor/s3/) | Schema, description |\n| [sharepoint](metaphor/sharepoint/) | Document embeddings |\n| [snowflake](metaphor/snowflake/) | Schema, description, statistics, queries |\n| [snowflake.profile](metaphor/snowflake/profile/) | Data profile |\n| [static_web](metaphor/static_web/) | Document embeddings |\n| [synapse](metaphor/synapse/) | Schema, queries |\n| [tableau](metaphor/tableau/) | Dashboard, lineage |\n| [thought_spot](metaphor/thought_spot/) | Dashboard, lineage |\n| [trino](metaphor/trino/) | Schema, description, queries |\n| [unity_catalog](metaphor/unity_catalog/) | Schema, description |\n| [unity_catalog.profile](metaphor/unity_catalog/profile/) | Data profile, statistics |\n\n## Development\n\nSee [Development Environment](docs/develop.md) for more instructions on how to set up your local development environment.\n\n## Custom Connectors\n\nSee [Adding a Custom Connector](docs/custom.md) for instructions and a full example of creating your custom connectors.\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "A collection of Python-based 'connectors' that extract metadata from various sources to ingest into the Metaphor app.",
"version": "0.14.161",
"project_urls": {
"Homepage": "https://metaphor.io",
"Repository": "https://github.com/MetaphorData/connectors"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "21b70ec4cb43e03eed2bea6f727b0ee9c81652077b3a036c9069f07b60390d76",
"md5": "01df47615ce54dbcf8b1ca0532bf869c",
"sha256": "3707eb506d34fb6e720329f20cb2d1c90a28c2377ce66f943761f63d87f452a8"
},
"downloads": -1,
"filename": "metaphor_connectors-0.14.161-py3-none-any.whl",
"has_sig": false,
"md5_digest": "01df47615ce54dbcf8b1ca0532bf869c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.12,>=3.9",
"size": 482845,
"upload_time": "2024-12-16T20:40:03",
"upload_time_iso_8601": "2024-12-16T20:40:03.117615Z",
"url": "https://files.pythonhosted.org/packages/21/b7/0ec4cb43e03eed2bea6f727b0ee9c81652077b3a036c9069f07b60390d76/metaphor_connectors-0.14.161-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5e18d856609b5ac5ea4135088888d70dfa6b15cadd372b9d2e607a6439c1dcd9",
"md5": "c8e3ac3c86ddde5d184d147c8cda4ef2",
"sha256": "996bdbeb9ea8bbe2fc540d4b715efa129aca2cea77bde5cce97f0dcac72c5a54"
},
"downloads": -1,
"filename": "metaphor_connectors-0.14.161.tar.gz",
"has_sig": false,
"md5_digest": "c8e3ac3c86ddde5d184d147c8cda4ef2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.12,>=3.9",
"size": 347092,
"upload_time": "2024-12-16T20:40:05",
"upload_time_iso_8601": "2024-12-16T20:40:05.648461Z",
"url": "https://files.pythonhosted.org/packages/5e/18/d856609b5ac5ea4135088888d70dfa6b15cadd372b9d2e607a6439c1dcd9/metaphor_connectors-0.14.161.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-16 20:40:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MetaphorData",
"github_project": "connectors",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "metaphor-connectors"
}
