# macrometa-target-mongo
`macrometa-target-mysql` is a Macrometa connector for writing data into MySQL, can be used as a target for any
Data Mesh Integration.
## Installation
Use PIP for installation:
```bash
pip install macrometa-target-mysql
```
## Configuration
The available configuration options for `macrometa-target-mysql` are:
| Property | Type | Required? | Description |
|-------------------------------------|-------------------|------------|-----------------------------------------------------------------------------------------------------|
| host | String | Yes | MySQL host |
| port | Int | Yes | MySQL port |
| username | String | Yes | MySQL user |
| password | Password | Yes | MySQL password |
| database | String | Yes | MySQL database name |
| target_table | String | Yes | Destination table name |
| batch_flush_size | Int | No | Maximum size of batch. Exceeding this will trigger a batch flush |
| batch_flush_interval | Int | No | Time between batch flush executions |
| hard_delete | Boolean | No | When `hard_delete` option is true, DELETE SQL commands will be performed in MySQL to delete rows |
| add_metadata_columns | Boolean | No | Metadata columns add extra row level information about data ingestion, |
| | | | (i.e. when was the row read in source, when was inserted or deleted in MySQL etc.) Metadata |
| | | | columns are created automatically by adding extra columns to the tables with a column prefix `_SDC_`. |
| | | | The column names are following the stitch naming conventions documented at |
| | | | [link](https://www.stitchdata.com/docs/data-structure/integration-schemas#sdc-columns). |
| ssl | Boolean | No | If set to `true` then use SSL for connecting with MySQL. |
| | | | If the server does not accept SSL connections or the client certificate is not recognized |
| | | | then the connection will fail. |
| ssl_check_hostname | Boolean | No | Flag to configure whether SSL handshake should verify that the certificate |
| | | | matches the DB hostname. |
| ssl_root_ca_cert | File | No | Specific CA certificate in PEM string format. This is most often the case |
| | | | when using `self-signed` server certificate. |
| ssl_client_certificate | File | No | Specific client certificate in PEM string format. The private key for the client |
| | | | certificate should be specified in a different parameter, SSL Client Key. |
| ssl_client_key | File | No | Specific client key in PEM string format. |
Configurations can be stored in a JSON configuration file and specified using the `--config` flag with
`macrometa-target-mysql`.
## Usage
```bash
cat <input_stream> | macrometa-target-mysql --config <config.json>
```
- `<input_stream>`: Input data stream
- `<config.json>`: JSON configuration file
`macrometa-target-mysql` reads data from a Singer Tap and writes it to a MySQL database. Run Singer Tap to generate
data before launching `macrometa-target-mysql`.
Here's an example of using Singer Tap with `macrometa-target-mysql`:
```bash
tap-exchangeratesapi | target-mysql --config config.json
```
In this case, `tap-exchangeratesapi` is a Singer Tap that generates exchange rate data. The data is passed to
`macrometa-target-mysql` through a pipe(`|`), and `macrometa-target-mysql` writes it to a MySQL database. `config.json` contains
`macrometa-target-mysql` settings.
## Developer Resources
### Initializing the Development Environment
```bash
pipx install poetry
poetry install
```
### Creating and Running Tests
Create tests in the `macrometa_target_mysql/tests` subfolder and run:
```bash
poetry run pytest
```
Use `poetry run` to test `macrometa-target-mysql` CLI interface:
```bash
poetry run target-mysql --help
```
### Testing with [Meltano](https://meltano.com/)
_**Note:** This target functions within a Singer environment and does not require Meltano._
Firstly, install Meltano and necessary plugins:
```bash
# Install Meltano
pipx install meltano
# Initialize Meltano in this directory
cd target-mysql
meltano install
```
Then, test and orchestrate with Meltano:
```bash
# Call tests:
meltano invoke macrometa-target-mysql --version
# Or execute pipeline with Carbon Intensity sample tap:
meltano run tap-carbon-intensity target-mysql
```
## Reference Links
- [Meltano Target SDK Documentation](https://sdk.meltano.com)
- [Singer Specification](https://github.com/singer-io/getting-started/blob/master/docs/SPEC.md)
- [Meltano](https://meltano.com/)
- [Singer.io](https://www.singer.io/)
Raw data
{
"_id": null,
"home_page": "",
"name": "macrometa-target-mysql",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8.1,<3.11",
"maintainer_email": "",
"keywords": "ELT,Connectors,Workflows,Macrometa,GDN,MySQL,Target",
"author": "Macrometa",
"author_email": "info@macrometa.com",
"download_url": "https://files.pythonhosted.org/packages/8c/ec/c15133a0a4ce9a873fada73dd2cc011936ef0f98caf531a3438fe6a1f3c1/macrometa-target-mysql-0.0.12.tar.gz",
"platform": null,
"description": "# macrometa-target-mongo\n\n`macrometa-target-mysql` is a Macrometa connector for writing data into MySQL, can be used as a target for any\nData Mesh Integration.\n\n## Installation\n\nUse PIP for installation:\n\n```bash\npip install macrometa-target-mysql\n```\n\n## Configuration\n\nThe available configuration options for `macrometa-target-mysql` are:\n\n| Property | Type | Required? | Description |\n|-------------------------------------|-------------------|------------|-----------------------------------------------------------------------------------------------------|\n| host | String | Yes | MySQL host |\n| port | Int | Yes | MySQL port |\n| username | String | Yes | MySQL user |\n| password | Password | Yes | MySQL password |\n| database | String | Yes | MySQL database name |\n| target_table | String | Yes | Destination table name |\n| batch_flush_size | Int | No | Maximum size of batch. Exceeding this will trigger a batch flush |\n| batch_flush_interval | Int | No | Time between batch flush executions |\n| hard_delete | Boolean | No | When `hard_delete` option is true, DELETE SQL commands will be performed in MySQL to delete rows |\n| add_metadata_columns | Boolean | No | Metadata columns add extra row level information about data ingestion, |\n| | | | (i.e. when was the row read in source, when was inserted or deleted in MySQL etc.) Metadata |\n| | | | columns are created automatically by adding extra columns to the tables with a column prefix `_SDC_`. |\n| | | | The column names are following the stitch naming conventions documented at |\n| | | | [link](https://www.stitchdata.com/docs/data-structure/integration-schemas#sdc-columns). |\n| ssl | Boolean | No | If set to `true` then use SSL for connecting with MySQL. |\n| | | | If the server does not accept SSL connections or the client certificate is not recognized |\n| | | | then the connection will fail. |\n| ssl_check_hostname | Boolean | No | Flag to configure whether SSL handshake should verify that the certificate |\n| | | | matches the DB hostname. |\n| ssl_root_ca_cert | File | No | Specific CA certificate in PEM string format. This is most often the case |\n| | | | when using `self-signed` server certificate. |\n| ssl_client_certificate | File | No | Specific client certificate in PEM string format. The private key for the client |\n| | | | certificate should be specified in a different parameter, SSL Client Key. |\n| ssl_client_key | File | No | Specific client key in PEM string format. |\n\nConfigurations can be stored in a JSON configuration file and specified using the `--config` flag with\n`macrometa-target-mysql`.\n\n## Usage\n\n```bash\ncat <input_stream> | macrometa-target-mysql --config <config.json>\n```\n\n- `<input_stream>`: Input data stream\n- `<config.json>`: JSON configuration file\n\n`macrometa-target-mysql` reads data from a Singer Tap and writes it to a MySQL database. Run Singer Tap to generate\ndata before launching `macrometa-target-mysql`.\n\nHere's an example of using Singer Tap with `macrometa-target-mysql`:\n\n```bash\ntap-exchangeratesapi | target-mysql --config config.json\n```\n\nIn this case, `tap-exchangeratesapi` is a Singer Tap that generates exchange rate data. The data is passed to\n`macrometa-target-mysql` through a pipe(`|`), and `macrometa-target-mysql` writes it to a MySQL database. `config.json` contains\n`macrometa-target-mysql` settings.\n\n## Developer Resources\n\n### Initializing the Development Environment\n\n```bash\npipx install poetry\npoetry install\n```\n\n### Creating and Running Tests\n\nCreate tests in the `macrometa_target_mysql/tests` subfolder and run:\n\n```bash\npoetry run pytest\n```\n\nUse `poetry run` to test `macrometa-target-mysql` CLI interface:\n\n```bash\npoetry run target-mysql --help\n```\n\n### Testing with [Meltano](https://meltano.com/)\n\n_**Note:** This target functions within a Singer environment and does not require Meltano._\n\nFirstly, install Meltano and necessary plugins:\n\n```bash\n# Install Meltano\npipx install meltano\n\n# Initialize Meltano in this directory\ncd target-mysql\nmeltano install\n```\n\nThen, test and orchestrate with Meltano:\n\n```bash\n# Call tests:\nmeltano invoke macrometa-target-mysql --version\n\n# Or execute pipeline with Carbon Intensity sample tap:\nmeltano run tap-carbon-intensity target-mysql\n```\n\n## Reference Links\n\n- [Meltano Target SDK Documentation](https://sdk.meltano.com)\n- [Singer Specification](https://github.com/singer-io/getting-started/blob/master/docs/SPEC.md)\n- [Meltano](https://meltano.com/)\n- [Singer.io](https://www.singer.io/)\n\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "A Meltano target for MySQL.",
"version": "0.0.12",
"project_urls": null,
"split_keywords": [
"elt",
"connectors",
"workflows",
"macrometa",
"gdn",
"mysql",
"target"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "73161f651bb1636ba5189083a7785708d28dc305abe559a61fb591e0a17f8b21",
"md5": "c9addc04c4686dc5a155b93c006c48f2",
"sha256": "4404e4509e6417f6b2e7fd824fa2ccfed35e7d7693af99ed779a166d7ebec421"
},
"downloads": -1,
"filename": "macrometa_target_mysql-0.0.12-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c9addc04c4686dc5a155b93c006c48f2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8.1,<3.11",
"size": 121054,
"upload_time": "2023-09-19T14:54:01",
"upload_time_iso_8601": "2023-09-19T14:54:01.115960Z",
"url": "https://files.pythonhosted.org/packages/73/16/1f651bb1636ba5189083a7785708d28dc305abe559a61fb591e0a17f8b21/macrometa_target_mysql-0.0.12-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8cecc15133a0a4ce9a873fada73dd2cc011936ef0f98caf531a3438fe6a1f3c1",
"md5": "f2b2f39e05407908e8ce4d31b56ffb3c",
"sha256": "070643ca1e1305666b9d33f1e97760b7b58b22c0da7e30d23748f32bc72df3ae"
},
"downloads": -1,
"filename": "macrometa-target-mysql-0.0.12.tar.gz",
"has_sig": false,
"md5_digest": "f2b2f39e05407908e8ce4d31b56ffb3c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8.1,<3.11",
"size": 107139,
"upload_time": "2023-09-19T14:54:02",
"upload_time_iso_8601": "2023-09-19T14:54:02.656678Z",
"url": "https://files.pythonhosted.org/packages/8c/ec/c15133a0a4ce9a873fada73dd2cc011936ef0f98caf531a3438fe6a1f3c1/macrometa-target-mysql-0.0.12.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-19 14:54:02",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "macrometa-target-mysql"
}
JSON