# tap-mysql
`tap-mysql` is a Singer tap for mysql.
Built with the [Meltano Tap SDK](https://sdk.meltano.com) for Singer Taps.
## Installation
Install from GitHub:
```bash
pipx install git+https://github.com/MeltanoLabs/tap-mysql.git@main
```
Note that you will also need to install the requisite dependencies for mysqlclient. Example installation command:
```bash
sudo apt-get update
sudo apt-get install package-cfg libmysqlclient-dev
```
## Configuration
### Accepted Config Options
| Setting | Required | Default | Description |
|:--------------------|:--------:|:-------:|:------------|
| host | False | None | Hostname for mysql instance. Note if sqlalchemy_url is set this will be ignored. |
| port | False | 3306 | The port on which mysql is awaiting connection. Note if sqlalchemy_url is set this will be ignored. |
| user | False | None | User name used to authenticate. Note if sqlalchemy_url is set this will be ignored. |
| password | False | None | Password used to authenticate. Note if sqlalchemy_url is set this will be ignored. |
| database | False | None | Database name. Note if sqlalchemy_url is set this will be ignored. |
| sqlalchemy_url | False | None | Example mysql://[username]:[password]@localhost:3306/[db_name] |
| ssh_tunnel | False | None | SSH Tunnel Configuration, this is a json object |
| ssh_tunnel.enable | True (if ssh_tunnel set) | False | Enable an ssh tunnel (also known as bastion host), see the other ssh_tunnel.* properties for more details.
| ssh_tunnel.host | True (if ssh_tunnel set) | False | Host of the bastion host, this is the host we'll connect to via ssh
| ssh_tunnel.username | True (if ssh_tunnel set) | False |Username to connect to bastion host
| ssh_tunnel.port | True (if ssh_tunnel set) | 22 | Port to connect to bastion host
| ssh_tunnel.private_key | True (if ssh_tunnel set) | None | Private Key for authentication to the bastion host
| ssh_tunnel.private_key_password | False | None | Private Key Password, leave None if no password is set
| stream_maps | False | None | Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html). |
| stream_map_config | False | None | User-defined config values to be used within map expressions. |
| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |
| flattening_max_depth| False | None | The max depth to flatten schemas. |
| batch_config | False | None | |
A full list of supported settings and capabilities for this
tap is available by running:
```bash
tap-mysql --about
```
### Configure using environment variables
This Singer tap will automatically import any environment variables within the working directory's
`.env` if the `--config=ENV` is provided, such that config values will be considered if a matching
environment variable is set either in the terminal context or in the `.env` file.
### SSH Tunnels (Bastion Hosts)
This tap supports connecting to a Postgres database via an SSH tunnel (also known as a bastion host). This is useful if you need to connect to a database that is not publicly accessible. This is the same as using `ssh -L` and `ssh -R`, but this is done inside the tap itself.
## Usage
You can easily run `tap-mysql` by itself or in a pipeline using [Meltano](https://meltano.com/).
### Executing the Tap Directly
```bash
tap-mysql --version
tap-mysql --help
tap-mysql --config CONFIG --discover > ./catalog.json
```
## Developer Resources
Follow these instructions to contribute to this project.
### Initialize your Development Environment
```bash
pipx install poetry
poetry install
```
### Create and Run Tests
Create tests within the `tests` subfolder and
then run:
```bash
poetry run pytest
```
You can also test the `tap-mysql` CLI interface directly using `poetry run`:
```bash
poetry run tap-mysql --help
```
### Testing with [Meltano](https://www.meltano.com)
_**Note:** This tap will work in any Singer environment and does not require Meltano.
Examples here are for convenience and to streamline end-to-end orchestration scenarios._
Next, install Meltano (if you haven't already) and any needed plugins:
```bash
# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-mysql
meltano install
```
Now you can test and orchestrate using Meltano:
```bash
# Test invocation:
meltano invoke tap-mysql --version
# OR run a test `elt` pipeline:
meltano elt tap-mysql target-jsonl
```
### SDK Dev Guide
See the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to
develop your own taps and targets.
Raw data
{
"_id": null,
"home_page": "https://meltano.com",
"name": "meltanolabs-tap-mysql",
"maintainer": "Meltano Team and Contributors",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "hello@meltano.com",
"keywords": "MySQL,Vitess,PlanetScale,Singer,ELT,Meltano,Meltano SDK",
"author": "Meltano Team and Contributors",
"author_email": "hello@meltano.com",
"download_url": "https://files.pythonhosted.org/packages/39/d6/3f6517a0242550207612efb9ea7a8acc2b2f86d79fedeae75e6deed2f609/meltanolabs_tap_mysql-0.0.1a1.tar.gz",
"platform": null,
"description": "# tap-mysql\n\n`tap-mysql` is a Singer tap for mysql.\n\nBuilt with the [Meltano Tap SDK](https://sdk.meltano.com) for Singer Taps.\n\n## Installation\n\nInstall from GitHub:\n\n```bash\npipx install git+https://github.com/MeltanoLabs/tap-mysql.git@main\n```\n\nNote that you will also need to install the requisite dependencies for mysqlclient. Example installation command:\n\n```bash\nsudo apt-get update\nsudo apt-get install package-cfg libmysqlclient-dev\n```\n\n## Configuration\n\n### Accepted Config Options\n\n| Setting | Required | Default | Description |\n|:--------------------|:--------:|:-------:|:------------|\n| host | False | None | Hostname for mysql instance. Note if sqlalchemy_url is set this will be ignored. |\n| port | False | 3306 | The port on which mysql is awaiting connection. Note if sqlalchemy_url is set this will be ignored. |\n| user | False | None | User name used to authenticate. Note if sqlalchemy_url is set this will be ignored. |\n| password | False | None | Password used to authenticate. Note if sqlalchemy_url is set this will be ignored. |\n| database | False | None | Database name. Note if sqlalchemy_url is set this will be ignored. |\n| sqlalchemy_url | False | None | Example mysql://[username]:[password]@localhost:3306/[db_name] |\n| ssh_tunnel | False | None | SSH Tunnel Configuration, this is a json object |\n| ssh_tunnel.enable | True (if ssh_tunnel set) | False | Enable an ssh tunnel (also known as bastion host), see the other ssh_tunnel.* properties for more details.\n| ssh_tunnel.host | True (if ssh_tunnel set) | False | Host of the bastion host, this is the host we'll connect to via ssh\n| ssh_tunnel.username | True (if ssh_tunnel set) | False |Username to connect to bastion host\n| ssh_tunnel.port | True (if ssh_tunnel set) | 22 | Port to connect to bastion host\n| ssh_tunnel.private_key | True (if ssh_tunnel set) | None | Private Key for authentication to the bastion host\n| ssh_tunnel.private_key_password | False | None | Private Key Password, leave None if no password is set\n| stream_maps | False | None | Config object for stream maps capability. For more information check out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html). |\n| stream_map_config | False | None | User-defined config values to be used within map expressions. |\n| flattening_enabled | False | None | 'True' to enable schema flattening and automatically expand nested properties. |\n| flattening_max_depth| False | None | The max depth to flatten schemas. |\n| batch_config | False | None | |\n\n\nA full list of supported settings and capabilities for this\ntap is available by running:\n\n```bash\ntap-mysql --about\n```\n\n### Configure using environment variables\n\nThis Singer tap will automatically import any environment variables within the working directory's\n`.env` if the `--config=ENV` is provided, such that config values will be considered if a matching\nenvironment variable is set either in the terminal context or in the `.env` file.\n\n### SSH Tunnels (Bastion Hosts)\n\nThis tap supports connecting to a Postgres database via an SSH tunnel (also known as a bastion host). This is useful if you need to connect to a database that is not publicly accessible. This is the same as using `ssh -L` and `ssh -R`, but this is done inside the tap itself.\n\n## Usage\n\nYou can easily run `tap-mysql` by itself or in a pipeline using [Meltano](https://meltano.com/).\n\n### Executing the Tap Directly\n\n```bash\ntap-mysql --version\ntap-mysql --help\ntap-mysql --config CONFIG --discover > ./catalog.json\n```\n\n## Developer Resources\n\nFollow these instructions to contribute to this project.\n\n### Initialize your Development Environment\n\n```bash\npipx install poetry\npoetry install\n```\n\n### Create and Run Tests\n\nCreate tests within the `tests` subfolder and\n then run:\n\n```bash\npoetry run pytest\n```\n\nYou can also test the `tap-mysql` CLI interface directly using `poetry run`:\n\n```bash\npoetry run tap-mysql --help\n```\n\n### Testing with [Meltano](https://www.meltano.com)\n\n_**Note:** This tap will work in any Singer environment and does not require Meltano.\nExamples here are for convenience and to streamline end-to-end orchestration scenarios._\n\nNext, install Meltano (if you haven't already) and any needed plugins:\n\n```bash\n# Install meltano\npipx install meltano\n# Initialize meltano within this directory\ncd tap-mysql\nmeltano install\n```\n\nNow you can test and orchestrate using Meltano:\n\n```bash\n# Test invocation:\nmeltano invoke tap-mysql --version\n# OR run a test `elt` pipeline:\nmeltano elt tap-mysql target-jsonl\n```\n\n### SDK Dev Guide\n\nSee the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to\ndevelop your own taps and targets.\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "`tap-mysql` is a Singer tap for mysql, built with the Meltano Singer SDK.",
"version": "0.0.1a1",
"project_urls": {
"Homepage": "https://meltano.com",
"Repository": "https://github.com/meltanolabs/tap-mysql"
},
"split_keywords": [
"mysql",
"vitess",
"planetscale",
"singer",
"elt",
"meltano",
"meltano sdk"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0591287d3fdb0a3d27f77438438b176cd0daf8520696f3bebd8af1994ef0ae08",
"md5": "89ca6c0fc711c7a72568ca726cfe21cd",
"sha256": "1275d5874702f875eb19c82401eeb4c7876f2d7eb97edb7a43ff3a5c56980163"
},
"downloads": -1,
"filename": "meltanolabs_tap_mysql-0.0.1a1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "89ca6c0fc711c7a72568ca726cfe21cd",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 9079,
"upload_time": "2024-02-23T17:00:39",
"upload_time_iso_8601": "2024-02-23T17:00:39.941715Z",
"url": "https://files.pythonhosted.org/packages/05/91/287d3fdb0a3d27f77438438b176cd0daf8520696f3bebd8af1994ef0ae08/meltanolabs_tap_mysql-0.0.1a1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "39d63f6517a0242550207612efb9ea7a8acc2b2f86d79fedeae75e6deed2f609",
"md5": "c54000dc44cdbb6ddccebfc0a75c9ac5",
"sha256": "3e6f7cf0d1eac5f75a39ed13150a2f4b1be98032bc094c567413c48017883dc5"
},
"downloads": -1,
"filename": "meltanolabs_tap_mysql-0.0.1a1.tar.gz",
"has_sig": false,
"md5_digest": "c54000dc44cdbb6ddccebfc0a75c9ac5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 9564,
"upload_time": "2024-02-23T17:00:41",
"upload_time_iso_8601": "2024-02-23T17:00:41.031180Z",
"url": "https://files.pythonhosted.org/packages/39/d6/3f6517a0242550207612efb9ea7a8acc2b2f86d79fedeae75e6deed2f609/meltanolabs_tap_mysql-0.0.1a1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-23 17:00:41",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "meltanolabs",
"github_project": "tap-mysql",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "meltanolabs-tap-mysql"
}
JSON
