<h1 align="center">Tap-DB2 👑</h1>
<p align="center">
<a href="https://github.com/danielptv/tap-db2/actions/"><img alt="Actions Status" src="https://github.com/danielptv/tap-db2/actions/workflows/test.yml/badge.svg"></a>
<a href="https://github.com/danielptv/tap-db2/blob/main/LICENSE"><img alt="License: MIT" src="https://img.shields.io/badge/License-MIT-yellow.svg"></a>
<a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
<img alt="PyPI - Version" src="https://img.shields.io/pypi/v/tap-ibm-db2">
<img alt="PyPI - Downloads" src="https://img.shields.io/pypi/dm/tap-ibm-db2">
</p>
`Tap-DB2` is a Singer tap for IBM DB2 data sources. Built with the [Meltano Tap SDK](https://sdk.meltano.com) for Singer Taps.
## Installation ⚙️
Install from PyPi:
```bash
pipx install tap-ibm-db2
```
Install from GitHub:
```bash
pipx install git+https://github.com/danielptv/tap-db2.git@main
```
## Configuration 📝
| Setting | Required | Default | Description |
| :--------------------------- | :------: | --------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| host | True | localhost | The DB2 hostname. |
| port | True | 50000 | The DB2 port. |
| database | True | None | The DB2 database. |
| schema | False | None | The DB2 schema. |
| user | True | None | The DB2 username. |
| password | True | None | The DB2 password. |
| encryption | True | None | Encryption settings for the DB2 connection. Disabled if omitted. |
| connection_parameters | False | None | Additional parameters to be appended to the connection string. This is an objects containing key-value pairs. |
| sqlalchemy_execution_options | False | None | Additional execution options to be passed to SQLAlchemy. This is an objects containing key-value pairs. |
| query_partitioning | False | None | Partition query into smaller subsets. |
| filter | False | None | Apply a custom WHERE condition per stream. Unlike the filter available in stream_maps, this will be evaluated BEFORE extracting the data. |
| ignore_supplied_tables | False | True | Ignore DB2-supplied user tables. For more info check out [Db2-supplied user tables](https://www.ibm.com/docs/en/db2-for-zos/12?topic=db2-supplied-user-tables). |
| ignore_views | False | False | Ignore views. |
| stream_maps | False | None | Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html). |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
A full list of supported settings and capabilities for this
tap is available by running:
```bash
tap-db2 --about --format json
```
### Configure using environment variables ✏️
This Singer tap will automatically import any environment variables within the working directory's
`.env` if the `--config=ENV` is provided, such that config values will be considered if a matching
environment variable is set either in the terminal context or in the `.env` file.
### Configure encryption settings 🔒
This Singer tap supports encrypted connection settings to DB2 according to the [python-ibmdb driver](https://github.com/ibmdb/python-ibmdb#example-of-ssl-connection-string).
**SSL without additional options:**
```yaml
...
plugins:
extractors:
- name: tap-db2
variant: danielptv
pip_url: tap-ibm-db2
config:
...
encryption: {}
```
This will append `SECURITY=SSL;` to the connection string.
**SSL using SSLServerCertificate keyword:**
```yaml
...
plugins:
extractors:
- name: tap-db2
variant: danielptv
pip_url: tap-ibm-db2
config:
...
encryption:
ssl_server_certificate: <Full path to the server certificate>
```
This will append `SECURITY=SSL;SSLServerCertificate=<Full path to the server certificate>;` to the connection string.
**SSL using SSLClientKeyStoreDB and SSLClientKeyStoreDBPassword keywords:**
```yaml
...
plugins:
extractors:
- name: tap-db2
variant: danielptv
pip_url: tap-ibm-db2
config:
...
encryption:
ssl_client_key_store_db:
database: <Full path to the client keystore database>
password: <Keystore password>
```
This will append `SECURITY=SSL;SSLClientKeyStoreDB=<Full path to the client keystore database>;SSLClientKeyStoreDBPassword=<Keystore password>;` to the connection string.
**SSL using SSLClientKeyStoreDB and SSLClientKeyStash keywords:**
```yaml
...
plugins:
extractors:
- name: tap-db2
variant: danielptv
pip_url: tap-ibm-db2
config:
...
encryption:
ssl_client_key_store_db:
database: <Full path to the client keystore database>
key_stash: <Full path to the client keystore stash>
```
This will append `SECURITY=SSL;SSLClientKeyStoreDB=<Full path to the client keystore database>;SSLClientKeyStash=<Full path to the client keystore stash>;` to the connection string.
### Configure query partitioning 🧩
This Singer tap supports the partitioning of SQL queries into smaller sub-queries to reduce the CPU load on the database. This is particularly useful when working with large amounts of data and a DB2 that has set strict resource limits per query. ***Note: This only works for streams with a numeric primary key.***
The configuration for query partitioning should look as follows:
```yaml
...
plugins:
extractors:
- name: tap-db2
variant: danielptv
pip_url: tap-ibm-db2
config:
...
query_partitioning:
<stream>:
primary_key: <primary key>
partition_size: 1000
```
Replace `<stream>` with the stream name and `<primary key>` with the stream's primary key. Use `*` to apply a query partitioning setting to all streams not explicitly declared.
## Usage 👷♀️
You can easily run `tap-db2` by itself or in a pipeline using [Meltano](https://meltano.com/).
### Executing the Tap Directly 🔨
```bash
tap-db2 --version
tap-db2 --help
tap-db2 --config CONFIG --discover > ./catalog.json
```
## Developer Resources 👩🏼💻
Follow these instructions to contribute to this project.
### Initialize your Development Environment
```bash
pipx install poetry
poetry install
```
### Create and Run Tests 🧪
Create tests within the `tests` subfolder and
then run:
```bash
poetry run pytest
```
You can also test the `tap-db2` CLI interface directly using `poetry run`:
```bash
poetry run tap-db2 --help
```
### Testing with [Meltano](https://www.meltano.com)
***Note:** This tap will work in any Singer environment and does not require Meltano.
Examples here are for convenience and to streamline end-to-end orchestration scenarios.*
Next, install Meltano (if you haven't already) and any needed plugins:
```bash
# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-db2
meltano install
```
Now you can test and orchestrate using Meltano:
```bash
# Test invocation:
meltano invoke tap-db2 --version
# OR run a test `elt` pipeline:
meltano elt tap-db2 target-jsonl
```
### SDK Dev Guide
See the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to
develop your own taps and targets.
Raw data
{
"_id": null,
"home_page": "https://meltano.com",
"name": "tap-ibm-db2",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4",
"maintainer_email": "",
"keywords": "DB2,IBM DB2,ELT,Meltano,Meltano SDK",
"author": "Daniel Purtov",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/fb/7b/7d35c05e97460f3d0f8da5635f2d891b961f9e244d52a57d5254afe8b2ab/tap_ibm_db2-0.10.0.tar.gz",
"platform": null,
"description": "<h1 align=\"center\">Tap-DB2 \ud83d\udc51</h1>\n\n<p align=\"center\">\n<a href=\"https://github.com/danielptv/tap-db2/actions/\"><img alt=\"Actions Status\" src=\"https://github.com/danielptv/tap-db2/actions/workflows/test.yml/badge.svg\"></a>\n<a href=\"https://github.com/danielptv/tap-db2/blob/main/LICENSE\"><img alt=\"License: MIT\" src=\"https://img.shields.io/badge/License-MIT-yellow.svg\"></a>\n<a href=\"https://github.com/psf/black\"><img alt=\"Code style: black\" src=\"https://img.shields.io/badge/code%20style-black-000000.svg\"></a>\n<img alt=\"PyPI - Version\" src=\"https://img.shields.io/pypi/v/tap-ibm-db2\">\n<img alt=\"PyPI - Downloads\" src=\"https://img.shields.io/pypi/dm/tap-ibm-db2\">\n</p>\n\n`Tap-DB2` is a Singer tap for IBM DB2 data sources. Built with the [Meltano Tap SDK](https://sdk.meltano.com) for Singer Taps.\n\n## Installation \u2699\ufe0f\n\nInstall from PyPi:\n\n```bash\npipx install tap-ibm-db2\n```\n\nInstall from GitHub:\n\n```bash\npipx install git+https://github.com/danielptv/tap-db2.git@main\n```\n\n## Configuration \ud83d\udcdd\n\n| Setting | Required | Default | Description |\n| :--------------------------- | :------: | --------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| host | True | localhost | The DB2 hostname. |\n| port | True | 50000 | The DB2 port. |\n| database | True | None | The DB2 database. |\n| schema | False | None | The DB2 schema. |\n| user | True | None | The DB2 username. |\n| password | True | None | The DB2 password. |\n| encryption | True | None | Encryption settings for the DB2 connection. Disabled if omitted. |\n| connection_parameters | False | None | Additional parameters to be appended to the connection string. This is an objects containing key-value pairs. |\n| sqlalchemy_execution_options | False | None | Additional execution options to be passed to SQLAlchemy. This is an objects containing key-value pairs. |\n| query_partitioning | False | None | Partition query into smaller subsets. |\n| filter | False | None | Apply a custom WHERE condition per stream. Unlike the filter available in stream_maps, this will be evaluated BEFORE extracting the data. |\n| ignore_supplied_tables | False | True | Ignore DB2-supplied user tables. For more info check out [Db2-supplied user tables](https://www.ibm.com/docs/en/db2-for-zos/12?topic=db2-supplied-user-tables). |\n| ignore_views | False | False | Ignore views. |\n| stream_maps | False | None | Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html). |\n| stream_map_config | False | None | User-defined config values to be used within map expressions. |\n\nA full list of supported settings and capabilities for this\ntap is available by running:\n\n```bash\ntap-db2 --about --format json\n```\n\n### Configure using environment variables \u270f\ufe0f\n\nThis Singer tap will automatically import any environment variables within the working directory's\n`.env` if the `--config=ENV` is provided, such that config values will be considered if a matching\nenvironment variable is set either in the terminal context or in the `.env` file.\n\n### Configure encryption settings \ud83d\udd12\n\nThis Singer tap supports encrypted connection settings to DB2 according to the [python-ibmdb driver](https://github.com/ibmdb/python-ibmdb#example-of-ssl-connection-string).\n\n**SSL without additional options:**\n\n```yaml\n...\nplugins:\n extractors:\n - name: tap-db2\n variant: danielptv\n pip_url: tap-ibm-db2\n config:\n ...\n encryption: {}\n```\n\nThis will append `SECURITY=SSL;` to the connection string.\n\n**SSL using SSLServerCertificate keyword:**\n\n```yaml\n...\nplugins:\n extractors:\n - name: tap-db2\n variant: danielptv\n pip_url: tap-ibm-db2\n config:\n ...\n encryption:\n ssl_server_certificate: <Full path to the server certificate>\n```\n\nThis will append `SECURITY=SSL;SSLServerCertificate=<Full path to the server certificate>;` to the connection string.\n\n**SSL using SSLClientKeyStoreDB and SSLClientKeyStoreDBPassword keywords:**\n\n```yaml\n...\nplugins:\n extractors:\n - name: tap-db2\n variant: danielptv\n pip_url: tap-ibm-db2\n config:\n ...\n encryption:\n ssl_client_key_store_db:\n database: <Full path to the client keystore database>\n password: <Keystore password>\n```\n\nThis will append `SECURITY=SSL;SSLClientKeyStoreDB=<Full path to the client keystore database>;SSLClientKeyStoreDBPassword=<Keystore password>;` to the connection string.\n\n**SSL using SSLClientKeyStoreDB and SSLClientKeyStash keywords:**\n\n```yaml\n...\nplugins:\n extractors:\n - name: tap-db2\n variant: danielptv\n pip_url: tap-ibm-db2\n config:\n ...\n encryption:\n ssl_client_key_store_db:\n database: <Full path to the client keystore database>\n key_stash: <Full path to the client keystore stash>\n```\n\nThis will append `SECURITY=SSL;SSLClientKeyStoreDB=<Full path to the client keystore database>;SSLClientKeyStash=<Full path to the client keystore stash>;` to the connection string.\n\n### Configure query partitioning \ud83e\udde9\n\nThis Singer tap supports the partitioning of SQL queries into smaller sub-queries to reduce the CPU load on the database. This is particularly useful when working with large amounts of data and a DB2 that has set strict resource limits per query. ***Note: This only works for streams with a numeric primary key.***\n\nThe configuration for query partitioning should look as follows:\n\n```yaml\n...\nplugins:\n extractors:\n - name: tap-db2\n variant: danielptv\n pip_url: tap-ibm-db2\n config:\n ...\n query_partitioning:\n <stream>:\n primary_key: <primary key>\n partition_size: 1000\n```\n\nReplace `<stream>` with the stream name and `<primary key>` with the stream's primary key. Use `*` to apply a query partitioning setting to all streams not explicitly declared.\n\n## Usage \ud83d\udc77\u200d\u2640\ufe0f\n\nYou can easily run `tap-db2` by itself or in a pipeline using [Meltano](https://meltano.com/).\n\n### Executing the Tap Directly \ud83d\udd28\n\n```bash\ntap-db2 --version\ntap-db2 --help\ntap-db2 --config CONFIG --discover > ./catalog.json\n```\n\n## Developer Resources \ud83d\udc69\ud83c\udffc\u200d\ud83d\udcbb\n\nFollow these instructions to contribute to this project.\n\n### Initialize your Development Environment\n\n```bash\npipx install poetry\npoetry install\n```\n\n### Create and Run Tests \ud83e\uddea\n\nCreate tests within the `tests` subfolder and\n then run:\n\n```bash\npoetry run pytest\n```\n\nYou can also test the `tap-db2` CLI interface directly using `poetry run`:\n\n```bash\npoetry run tap-db2 --help\n```\n\n### Testing with [Meltano](https://www.meltano.com)\n\n***Note:** This tap will work in any Singer environment and does not require Meltano.\nExamples here are for convenience and to streamline end-to-end orchestration scenarios.*\n\nNext, install Meltano (if you haven't already) and any needed plugins:\n\n```bash\n# Install meltano\npipx install meltano\n# Initialize meltano within this directory\ncd tap-db2\nmeltano install\n```\n\nNow you can test and orchestrate using Meltano:\n\n```bash\n# Test invocation:\nmeltano invoke tap-db2 --version\n# OR run a test `elt` pipeline:\nmeltano elt tap-db2 target-jsonl\n```\n\n### SDK Dev Guide\n\nSee the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to\ndevelop your own taps and targets.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "`Tap-DB2` is a Singer tap for IBM DB2 data sources.",
"version": "0.10.0",
"project_urls": {
"Homepage": "https://meltano.com",
"Repository": "https://github.com/danielptv/tap-db2"
},
"split_keywords": [
"db2",
"ibm db2",
"elt",
"meltano",
"meltano sdk"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "94b87f0c7f5fe3b70173a8864600795f83ab8a044373c8a262767af0a109bd96",
"md5": "53ca1081f94e7bbbcc4a8ba62451831f",
"sha256": "e82b31167146ff35fca6d2fc09b68cc0d285f47bd3218696aac3f3bb49524004"
},
"downloads": -1,
"filename": "tap_ibm_db2-0.10.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "53ca1081f94e7bbbcc4a8ba62451831f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4",
"size": 10535,
"upload_time": "2024-03-03T09:01:44",
"upload_time_iso_8601": "2024-03-03T09:01:44.529839Z",
"url": "https://files.pythonhosted.org/packages/94/b8/7f0c7f5fe3b70173a8864600795f83ab8a044373c8a262767af0a109bd96/tap_ibm_db2-0.10.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fb7b7d35c05e97460f3d0f8da5635f2d891b961f9e244d52a57d5254afe8b2ab",
"md5": "94d0ee946c94cfc0803d705014db4801",
"sha256": "ebf438218771b1c5291f82046b868550d796fda7e272719b9a41883c1f97ff49"
},
"downloads": -1,
"filename": "tap_ibm_db2-0.10.0.tar.gz",
"has_sig": false,
"md5_digest": "94d0ee946c94cfc0803d705014db4801",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4",
"size": 10738,
"upload_time": "2024-03-03T09:01:47",
"upload_time_iso_8601": "2024-03-03T09:01:47.115642Z",
"url": "https://files.pythonhosted.org/packages/fb/7b/7d35c05e97460f3d0f8da5635f2d891b961f9e244d52a57d5254afe8b2ab/tap_ibm_db2-0.10.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-03 09:01:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "danielptv",
"github_project": "tap-db2",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "tap-ibm-db2"
}
JSON
