# Declarative-Manifest source connector
This is the repository for the Declarative-Manifest source connector, written in Python.
The declarative manifest source connector is a special connector that can create an arbitrary source
connector from a declarative manifest file. This allows users to create a source connector without writing any code.
**Note**: This connector is managed by the Airbyte Python CDK release process. It can be run as a standalone connector
in Docker and PyAirbyte, but is not yet meant to be run in the platform as a standalone connector. This source is
an interface to the low-code CDK and as such, should not be modified without a corresponding CDK change.
## Local development
### Prerequisites
- Python (~=3.9)
- Poetry (~=1.7) - installation instructions [here](https://python-poetry.org/docs/#installation)
### Installing the connector
From this connector directory, run:
```bash
poetry install
```
### Create credentials
The credentials for source-declarative-manifest are a little different. Your `config` will need to contain the
injected declarative manifest, as indicated in the `spec`. It will also need to contain the fields that the spec
coming out of the manifest requires. An example is available in `integration_tests/pokeapi_config.json`. To use
this example in the following instructions, copy this file to `secrets/config.json`.
### Locally running the connector
```
poetry run source-declarative-manifest spec
poetry run source-declarative-manifest check --config secrets/config.json
poetry run source-declarative-manifest discover --config secrets/config.json
poetry run source-declarative-manifest read --config secrets/config.json --catalog sample_files/configured_catalog.json
```
### Running unit tests
To run unit tests locally, from the connector directory run:
```
poetry run pytest unit_tests
```
### Building the docker image
1. Install [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md)
2. Run the following command to build the docker image:
```bash
airbyte-ci connectors --name=source-declarative-manifest build
```
An image will be available on your host with the tag `airbyte/source-declarative-manifest:dev`.
### Running as a docker container
Then run any of the connector commands as follows:
```
docker run --rm airbyte/source-declarative-manifest:dev spec
docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-declarative-manifest:dev check --config /secrets/config.json
docker run --rm -v $(pwd)/secrets:/secrets airbyte/source-declarative-manifest:dev discover --config /secrets/config.json
docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-declarative-manifest:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json
```
### Running our CI test suite
You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
```bash
airbyte-ci connectors --name=source-declarative-manifest test
```
This source does not currently pass the full test suite.
### Dependency Management
The manifest declarative source is built to be an interface to the low-code CDK source. This means that
this source should not have any production dependencies other than the Airbyte Python CDK. If for some reason
you feel that a new dependency is needed, you likely want to add it to the CDK instead. It is expected
that a given version of the source-declarative-manifest connector corresponds to the same version in
its CDK dependency.
## Publishing a new version of the connector
New versions of this connector should only be published (automatically) via the manual Airbyte CDK release process.
If you want to make a change to this connector that is not a result of a CDK change and a corresponding
CDK dependency bump, please reach out to the Connector Extensibility team for guidance.
Raw data
{
"_id": null,
"home_page": "https://airbyte.com",
"name": "airbyte-source-declarative-manifest",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.12,>=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Airbyte",
"author_email": "contact@airbyte.io",
"download_url": "https://files.pythonhosted.org/packages/3f/15/2da368fe0eab1a07bbdb7a9d27e6a8226fdb4b2feaea1abfa4a3174fa8e3/airbyte_source_declarative_manifest-0.90.0.tar.gz",
"platform": null,
"description": "# Declarative-Manifest source connector\n\nThis is the repository for the Declarative-Manifest source connector, written in Python.\nThe declarative manifest source connector is a special connector that can create an arbitrary source\nconnector from a declarative manifest file. This allows users to create a source connector without writing any code.\n\n**Note**: This connector is managed by the Airbyte Python CDK release process. It can be run as a standalone connector\nin Docker and PyAirbyte, but is not yet meant to be run in the platform as a standalone connector. This source is\nan interface to the low-code CDK and as such, should not be modified without a corresponding CDK change.\n\n## Local development\n\n### Prerequisites\n\n- Python (~=3.9)\n- Poetry (~=1.7) - installation instructions [here](https://python-poetry.org/docs/#installation)\n\n### Installing the connector\n\nFrom this connector directory, run:\n\n```bash\npoetry install\n```\n\n### Create credentials\n\nThe credentials for source-declarative-manifest are a little different. Your `config` will need to contain the\ninjected declarative manifest, as indicated in the `spec`. It will also need to contain the fields that the spec\ncoming out of the manifest requires. An example is available in `integration_tests/pokeapi_config.json`. To use\nthis example in the following instructions, copy this file to `secrets/config.json`.\n\n### Locally running the connector\n\n```\npoetry run source-declarative-manifest spec\npoetry run source-declarative-manifest check --config secrets/config.json\npoetry run source-declarative-manifest discover --config secrets/config.json\npoetry run source-declarative-manifest read --config secrets/config.json --catalog sample_files/configured_catalog.json\n```\n\n### Running unit tests\n\nTo run unit tests locally, from the connector directory run:\n\n```\npoetry run pytest unit_tests\n```\n\n### Building the docker image\n\n1. Install [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md)\n2. Run the following command to build the docker image:\n\n```bash\nairbyte-ci connectors --name=source-declarative-manifest build\n```\n\nAn image will be available on your host with the tag `airbyte/source-declarative-manifest:dev`.\n\n### Running as a docker container\n\nThen run any of the connector commands as follows:\n\n```\ndocker run --rm airbyte/source-declarative-manifest:dev spec\ndocker run --rm -v $(pwd)/secrets:/secrets airbyte/source-declarative-manifest:dev check --config /secrets/config.json\ndocker run --rm -v $(pwd)/secrets:/secrets airbyte/source-declarative-manifest:dev discover --config /secrets/config.json\ndocker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/source-declarative-manifest:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json\n```\n\n### Running our CI test suite\n\nYou can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):\n\n```bash\nairbyte-ci connectors --name=source-declarative-manifest test\n```\n\nThis source does not currently pass the full test suite.\n\n### Dependency Management\n\nThe manifest declarative source is built to be an interface to the low-code CDK source. This means that\nthis source should not have any production dependencies other than the Airbyte Python CDK. If for some reason\nyou feel that a new dependency is needed, you likely want to add it to the CDK instead. It is expected\nthat a given version of the source-declarative-manifest connector corresponds to the same version in\nits CDK dependency.\n\n## Publishing a new version of the connector\n\nNew versions of this connector should only be published (automatically) via the manual Airbyte CDK release process.\nIf you want to make a change to this connector that is not a result of a CDK change and a corresponding\nCDK dependency bump, please reach out to the Connector Extensibility team for guidance.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Base source implementation for low-code sources.",
"version": "0.90.0",
"project_urls": {
"Documentation": "https://docs.airbyte.com/integrations/sources/low-code",
"Homepage": "https://airbyte.com",
"Repository": "https://github.com/airbytehq/airbyte"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6ec875b39345e28a1f17cb429b2accb242a9ce775f13804d81eb9c39940ee459",
"md5": "66aa29d0eca4bc745edd08c3d40c7b93",
"sha256": "6d396ae3ec1d35b406b69550a065e6fb057c23e54d7e47b391032eb3c89df9ab"
},
"downloads": -1,
"filename": "airbyte_source_declarative_manifest-0.90.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "66aa29d0eca4bc745edd08c3d40c7b93",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.12,>=3.9",
"size": 4667,
"upload_time": "2024-05-17T05:38:36",
"upload_time_iso_8601": "2024-05-17T05:38:36.158435Z",
"url": "https://files.pythonhosted.org/packages/6e/c8/75b39345e28a1f17cb429b2accb242a9ce775f13804d81eb9c39940ee459/airbyte_source_declarative_manifest-0.90.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3f152da368fe0eab1a07bbdb7a9d27e6a8226fdb4b2feaea1abfa4a3174fa8e3",
"md5": "bcafadb3df79dd2074e9d99e9574ac4e",
"sha256": "35b30cfe05b6fdc07d5fe7b120b627860afa17f89196a14242a5b7bf6c5703de"
},
"downloads": -1,
"filename": "airbyte_source_declarative_manifest-0.90.0.tar.gz",
"has_sig": false,
"md5_digest": "bcafadb3df79dd2074e9d99e9574ac4e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.12,>=3.9",
"size": 3486,
"upload_time": "2024-05-17T05:38:37",
"upload_time_iso_8601": "2024-05-17T05:38:37.812532Z",
"url": "https://files.pythonhosted.org/packages/3f/15/2da368fe0eab1a07bbdb7a9d27e6a8226fdb4b2feaea1abfa4a3174fa8e3/airbyte_source_declarative_manifest-0.90.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-17 05:38:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "airbytehq",
"github_project": "airbyte",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "airbyte-source-declarative-manifest"
}
JSON
