| Name | akurdyukov-tap-clickhouse JSON |
| Version |
0.0.1
JSON |
| download |
| home_page | |
| Summary | `tap-clickhouse` is a Singer tap for ClickHouse, built with the Meltano Singer SDK. |
| upload_time | 2023-10-30 07:02:40 |
| maintainer | |
| docs_url | None |
| author | Alik Kurdyukov |
| requires_python | >=3.7.1,<4 |
| license | Apache-2.0 |
| keywords |
elt
clickhouse
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# tap-clickhouse
`tap-clickhouse` is a Singer tap for ClickHouse.
Built with the [Meltano Tap SDK](https://sdk.meltano.com) for Singer Taps.
## Installation
<!--
Install from PyPi:
```bash
pipx install tap-clickhouse
```
-->
Install from GitHub:
```bash
pipx install git+https://github.com/akurdyukov/tap-clickhouse.git@main
```
## Configuration
### Capabilities
* `catalog`
* `state`
* `discover`
* `about`
* `stream-maps`
* `schema-flattening`
* `batch`
### Accepted Config Options
| Setting | Required | Default | Description |
|:---------------------|:--------:|:---------:|:--------------------------------------------------------------------------------------------------------------------------------------------|
| driver | False | http | Driver type |
| username | False | default | Database user |
| password | True | None | Username password |
| host | False | localhost | Database host |
| port | False | 8123 | Database connection port |
| database | False | default | Database name |
| secure | False | 0 | Should the connection be secure |
| verify | False | 1 | Should secure connection need to verify SSL/TLS |
| stream_maps | False | None | Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html). |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth | False | None | The max depth to flatten schemas. |
| batch_config | False | None | |
A full list of supported settings and capabilities for this
tap is available by running:
```bash
tap-clickhouse --about
```
### Configure using environment variables
This Singer tap will automatically import any environment variables within the working directory's
`.env` if the `--config=ENV` is provided, such that config values will be considered if a matching
environment variable is set either in the terminal context or in the `.env` file.
### Source Authentication and Authorization
Set `username` and `password` settings to authorize.
## Usage
You can easily run `tap-clickhouse` by itself or in a pipeline using [Meltano](https://meltano.com/).
### Executing the Tap Directly
```bash
tap-clickhouse --version
tap-clickhouse --help
tap-clickhouse --config CONFIG --discover > ./catalog.json
```
## Developer Resources
Follow these instructions to contribute to this project.
### Initialize your Development Environment
```bash
pipx install poetry
poetry install
```
### Create and Run Tests
Create tests within the `tests` subfolder and
then run:
```bash
poetry run pytest
```
You can also test the `tap-clickhouse` CLI interface directly using `poetry run`:
```bash
poetry run tap-clickhouse --help
```
### Testing with [Meltano](https://www.meltano.com)
_**Note:** This tap will work in any Singer environment and does not require Meltano.
Examples here are for convenience and to streamline end-to-end orchestration scenarios._
Run ClickHouse server with basic database using
```bash
docker-compose -f tests/docker-compose.yml up -d
```
Next, install Meltano (if you haven't already) and any needed plugins:
```bash
# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-clickhouse
meltano install
```
Now you can test and orchestrate using Meltano:
```bash
# Test invocation:
meltano invoke tap-clickhouse --version
# OR run a test `elt` pipeline:
meltano elt tap-clickhouse target-jsonl
```
### SDK Dev Guide
See the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to
develop your own taps and targets.
Raw data
{
"_id": null,
"home_page": "",
"name": "akurdyukov-tap-clickhouse",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7.1,<4",
"maintainer_email": "",
"keywords": "ELT,ClickHouse",
"author": "Alik Kurdyukov",
"author_email": "alik@kurdyukov.com",
"download_url": "https://files.pythonhosted.org/packages/1f/a5/b589991912e98596988c93f4722e99d4225a6e7a176c44cb7fc06d6c227f/akurdyukov_tap_clickhouse-0.0.1.tar.gz",
"platform": null,
"description": "# tap-clickhouse\n\n`tap-clickhouse` is a Singer tap for ClickHouse.\n\nBuilt with the [Meltano Tap SDK](https://sdk.meltano.com) for Singer Taps.\n\n## Installation\n\n<!--\nInstall from PyPi:\n\n```bash\npipx install tap-clickhouse\n```\n\n-->\n\nInstall from GitHub:\n\n```bash\npipx install git+https://github.com/akurdyukov/tap-clickhouse.git@main\n```\n\n## Configuration\n\n### Capabilities\n\n* `catalog`\n* `state`\n* `discover`\n* `about`\n* `stream-maps`\n* `schema-flattening`\n* `batch`\n\n### Accepted Config Options\n\n| Setting | Required | Default | Description |\n|:---------------------|:--------:|:---------:|:--------------------------------------------------------------------------------------------------------------------------------------------|\n| driver | False | http | Driver type |\n| username | False | default | Database user |\n| password | True | None | Username password |\n| host | False | localhost | Database host |\n| port | False | 8123 | Database connection port |\n| database | False | default | Database name |\n| secure | False | 0 | Should the connection be secure |\n| verify | False | 1 | Should secure connection need to verify SSL/TLS |\n| stream_maps | False | None | Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html). |\n| stream_map_config | False | None | User-defined config values to be used within map expressions. |\n| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |\n| flattening_max_depth | False | None | The max depth to flatten schemas. |\n| batch_config | False | None | |\n\nA full list of supported settings and capabilities for this\ntap is available by running:\n\n```bash\ntap-clickhouse --about\n```\n\n### Configure using environment variables\n\nThis Singer tap will automatically import any environment variables within the working directory's\n`.env` if the `--config=ENV` is provided, such that config values will be considered if a matching\nenvironment variable is set either in the terminal context or in the `.env` file.\n\n### Source Authentication and Authorization\n\nSet `username` and `password` settings to authorize.\n\n## Usage\n\nYou can easily run `tap-clickhouse` by itself or in a pipeline using [Meltano](https://meltano.com/).\n\n### Executing the Tap Directly\n\n```bash\ntap-clickhouse --version\ntap-clickhouse --help\ntap-clickhouse --config CONFIG --discover > ./catalog.json\n```\n\n## Developer Resources\n\nFollow these instructions to contribute to this project.\n\n### Initialize your Development Environment\n\n```bash\npipx install poetry\npoetry install\n```\n\n### Create and Run Tests\n\nCreate tests within the `tests` subfolder and\n then run:\n\n```bash\npoetry run pytest\n```\n\nYou can also test the `tap-clickhouse` CLI interface directly using `poetry run`:\n\n```bash\npoetry run tap-clickhouse --help\n```\n\n### Testing with [Meltano](https://www.meltano.com)\n\n_**Note:** This tap will work in any Singer environment and does not require Meltano.\nExamples here are for convenience and to streamline end-to-end orchestration scenarios._\n\nRun ClickHouse server with basic database using\n```bash\ndocker-compose -f tests/docker-compose.yml up -d\n```\n\nNext, install Meltano (if you haven't already) and any needed plugins:\n\n```bash\n# Install meltano\npipx install meltano\n# Initialize meltano within this directory\ncd tap-clickhouse\nmeltano install\n```\n\nNow you can test and orchestrate using Meltano:\n\n```bash\n# Test invocation:\nmeltano invoke tap-clickhouse --version\n# OR run a test `elt` pipeline:\nmeltano elt tap-clickhouse target-jsonl\n```\n\n### SDK Dev Guide\n\nSee the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to\ndevelop your own taps and targets.\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "`tap-clickhouse` is a Singer tap for ClickHouse, built with the Meltano Singer SDK.",
"version": "0.0.1",
"project_urls": null,
"split_keywords": [
"elt",
"clickhouse"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d18532516de1cd2dc7ce3e8ec7c53ce42ed3fc2354cbbf5b8e75b8a535848454",
"md5": "49dd28190c5ec478e8148c31bb98b383",
"sha256": "51d140a63263d721f899d88d877bd5f352f9de0403c2de5ca6fbd3c7b9b2b693"
},
"downloads": -1,
"filename": "akurdyukov_tap_clickhouse-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "49dd28190c5ec478e8148c31bb98b383",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7.1,<4",
"size": 9663,
"upload_time": "2023-10-30T07:02:39",
"upload_time_iso_8601": "2023-10-30T07:02:39.084760Z",
"url": "https://files.pythonhosted.org/packages/d1/85/32516de1cd2dc7ce3e8ec7c53ce42ed3fc2354cbbf5b8e75b8a535848454/akurdyukov_tap_clickhouse-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1fa5b589991912e98596988c93f4722e99d4225a6e7a176c44cb7fc06d6c227f",
"md5": "b0e6ff0add01b419ab331a6f294c5af6",
"sha256": "c795f264371683050cdcf9c020753956216c79e116e61779413394e4aead6b59"
},
"downloads": -1,
"filename": "akurdyukov_tap_clickhouse-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "b0e6ff0add01b419ab331a6f294c5af6",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7.1,<4",
"size": 8615,
"upload_time": "2023-10-30T07:02:40",
"upload_time_iso_8601": "2023-10-30T07:02:40.989145Z",
"url": "https://files.pythonhosted.org/packages/1f/a5/b589991912e98596988c93f4722e99d4225a6e7a176c44cb7fc06d6c227f/akurdyukov_tap_clickhouse-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-30 07:02:40",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "akurdyukov-tap-clickhouse"
}
