# Declarative-Manifest source connector
This is the repository for the Declarative-Manifest source connector, written in Python.
The declarative manifest source connector is a special connector that can create an arbitrary source
connector from a declarative manifest file. This allows users to create a source connector without writing any code.
**Note**: This connector is managed by the Airbyte Python CDK release process. It can be run as a standalone connector
in Docker and PyAirbyte, but is not yet meant to be run in the platform as a standalone connector. This source is
an interface to the low-code CDK and as such, should not be modified without a corresponding CDK change.
## Local development
### Prerequisites
- Python (~=3.9)
- Poetry (~=1.7) - installation instructions [here](https://python-poetry.org/docs/#installation)
### Installing the connector
From this connector directory, run:
```bash
poetry install
```
### Create credentials
The credentials for source-declarative-manifest are a little different. Your `config` will need to contain the
injected declarative manifest, as indicated in the `spec`. It will also need to contain the fields that the spec
coming out of the manifest requires. An example is available in `integration_tests/pokeapi_config.json`. To use
this example in the following instructions, copy this file to `secrets/config.json`.
### Locally running the connector
```
poetry run source-declarative-manifest spec
poetry run source-declarative-manifest check --config secrets/config.json
poetry run source-declarative-manifest discover --config secrets/config.json
poetry run source-declarative-manifest read --config secrets/config.json --catalog sample_files/configured_catalog.json
```
### Running unit tests
To run unit tests locally, from the connector directory run:
```
poetry run pytest unit_tests
```
### Building the docker image
1. Install [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md)
2. Run the following command to build the docker image:
```bash
airbyte-ci connectors --name=source-declarative-manifest build
```
An image will be available on your host with the tag `airbyte/source-declarative-manifest:dev`.
### Running as a docker container
Then run any of the connector commands as follows:
```
docker run --rm airbyte/source-declarative-manifest:dev spec
docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-declarative-manifest:dev check --config /secrets/config.json
docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-declarative-manifest:dev discover --config /secrets/config.json
docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-declarative-manifest:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
```
### Running our CI test suite
You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
```bash
airbyte-ci connectors --name=source-declarative-manifest test
```
This source does not currently pass the full test suite.
### Dependency Management
The manifest declarative source is built to be an interface to the low-code CDK source. This means that
this source should not have any production dependencies other than the Airbyte Python CDK. If for some reason
you feel that a new dependency is needed, you likely want to add it to the CDK instead. It is expected
that a given version of the source-declarative-manifest connector corresponds to the same version in
its CDK dependency.
## Publishing a new version of the connector
New versions of this connector should only be published (automatically) via the manual Airbyte CDK release process.
If you want to make a change to this connector that is not a result of a CDK change and a corresponding
CDK dependency bump, please reach out to the Connector Extensibility team for guidance.
Raw data
{
"_id": null,
"home_page": "https://airbyte.com",
"name": "airbyte-source-declarative-manifest",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.12,>=3.10",
"maintainer_email": null,
"keywords": null,
"author": "Airbyte",
"author_email": "contact@airbyte.io",
"download_url": "https://files.pythonhosted.org/packages/53/83/d59ada22d8c13b38a096ff22bcabe78681c188f9f363bb15ce140796072a/airbyte_source_declarative_manifest-6.5.2.tar.gz",
"platform": null,
"description": "# Declarative-Manifest source connector\n\nThis is the repository for the Declarative-Manifest source connector, written in Python.\nThe declarative manifest source connector is a special connector that can create an arbitrary source\nconnector from a declarative manifest file. This allows users to create a source connector without writing any code.\n\n**Note**: This connector is managed by the Airbyte Python CDK release process. It can be run as a standalone connector\nin Docker and PyAirbyte, but is not yet meant to be run in the platform as a standalone connector. This source is\nan interface to the low-code CDK and as such, should not be modified without a corresponding CDK change.\n\n## Local development\n\n### Prerequisites\n\n- Python (~=3.9)\n- Poetry (~=1.7) - installation instructions [here](https://python-poetry.org/docs/#installation)\n\n### Installing the connector\n\nFrom this connector directory, run:\n\n```bash\npoetry install\n```\n\n### Create credentials\n\nThe credentials for source-declarative-manifest are a little different. Your `config` will need to contain the\ninjected declarative manifest, as indicated in the `spec`. It will also need to contain the fields that the spec\ncoming out of the manifest requires. An example is available in `integration_tests/pokeapi_config.json`. To use\nthis example in the following instructions, copy this file to `secrets/config.json`.\n\n### Locally running the connector\n\n```\npoetry run source-declarative-manifest spec\npoetry run source-declarative-manifest check --config secrets/config.json\npoetry run source-declarative-manifest discover --config secrets/config.json\npoetry run source-declarative-manifest read --config secrets/config.json --catalog sample_files/configured_catalog.json\n```\n\n### Running unit tests\n\nTo run unit tests locally, from the connector directory run:\n\n```\npoetry run pytest unit_tests\n```\n\n### Building the docker image\n\n1. Install [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md)\n2. Run the following command to build the docker image:\n\n```bash\nairbyte-ci connectors --name=source-declarative-manifest build\n```\n\nAn image will be available on your host with the tag `airbyte/source-declarative-manifest:dev`.\n\n### Running as a docker container\n\nThen run any of the connector commands as follows:\n\n```\ndocker run --rm airbyte/source-declarative-manifest:dev spec\ndocker run --rm -v $(pwd)/secrets:/secrets airbyte/source-declarative-manifest:dev check --config /secrets/config.json\ndocker run --rm -v $(pwd)/secrets:/secrets airbyte/source-declarative-manifest:dev discover --config /secrets/config.json\ndocker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-declarative-manifest:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json\n```\n\n### Running our CI test suite\n\nYou can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):\n\n```bash\nairbyte-ci connectors --name=source-declarative-manifest test\n```\n\nThis source does not currently pass the full test suite.\n\n### Dependency Management\n\nThe manifest declarative source is built to be an interface to the low-code CDK source. This means that\nthis source should not have any production dependencies other than the Airbyte Python CDK. If for some reason\nyou feel that a new dependency is needed, you likely want to add it to the CDK instead. It is expected\nthat a given version of the source-declarative-manifest connector corresponds to the same version in\nits CDK dependency.\n\n## Publishing a new version of the connector\n\nNew versions of this connector should only be published (automatically) via the manual Airbyte CDK release process.\nIf you want to make a change to this connector that is not a result of a CDK change and a corresponding\nCDK dependency bump, please reach out to the Connector Extensibility team for guidance.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Base source implementation for low-code sources.",
"version": "6.5.2",
"project_urls": {
"Documentation": "https://docs.airbyte.com/integrations/sources/low-code",
"Homepage": "https://airbyte.com",
"Repository": "https://github.com/airbytehq/airbyte"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3270d716f76bffaf522fbb867ce853a2fb5a83e0b81a2d34c72d0150aab669fa",
"md5": "85b55936dadcdd46ce9201c905b715c6",
"sha256": "edbb96bd255c43eddbee2e7d0817729d282b0fd21b5f03970a0dc7a315085002"
},
"downloads": -1,
"filename": "airbyte_source_declarative_manifest-6.5.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "85b55936dadcdd46ce9201c905b715c6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.12,>=3.10",
"size": 5690,
"upload_time": "2024-11-08T22:55:34",
"upload_time_iso_8601": "2024-11-08T22:55:34.094368Z",
"url": "https://files.pythonhosted.org/packages/32/70/d716f76bffaf522fbb867ce853a2fb5a83e0b81a2d34c72d0150aab669fa/airbyte_source_declarative_manifest-6.5.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5383d59ada22d8c13b38a096ff22bcabe78681c188f9f363bb15ce140796072a",
"md5": "0cebc4e903c51497a1fc2ef8b8b4cc25",
"sha256": "4bd831add7dd71ed3f84a71b27df40420bed4eb6e4dd99bedec579a8d33d30f7"
},
"downloads": -1,
"filename": "airbyte_source_declarative_manifest-6.5.2.tar.gz",
"has_sig": false,
"md5_digest": "0cebc4e903c51497a1fc2ef8b8b4cc25",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.12,>=3.10",
"size": 4471,
"upload_time": "2024-11-08T22:55:35",
"upload_time_iso_8601": "2024-11-08T22:55:35.765978Z",
"url": "https://files.pythonhosted.org/packages/53/83/d59ada22d8c13b38a096ff22bcabe78681c188f9f363bb15ce140796072a/airbyte_source_declarative_manifest-6.5.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-08 22:55:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "airbytehq",
"github_project": "airbyte",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "airbyte-source-declarative-manifest"
}
JSON
