## files-2-db
Uploading data from files to database made easy!
## Background
Importing a file from your local machine into a database which is in a remote server is painful process. The exact same process can be done through a command now.
### Current implementation
- File formats supported - `.csv`
- Database engines supported - `mysql`
### Usage
The tool can be used in two ways.
- Through command line, where you do not even need a python script to be created.
```cmd
usage: file2db [-h] --host HOST [--port PORT] --user USER --password PASSWORD
--database DATABASE --table TABLE [--seperator SEPERATOR]
[--ignore_errors IGNORE_ERRORS] [--n_rows N_ROWS]
Source-file Target-database
file2db - Transfer of data from your local file to Database, made simple.
positional arguments:
Source-file Absolute path of the file to be uploaded, with suffix
Target-database Target database where the file has to be uploaded
options:
-h, --help show this help message and exit
--host HOST URL or IP address representation of database host
--port PORT Port number to connect to the database
--user USER Username for database authentication
--password PASSWORD Password for database authentication
--database DATABASE Target database
--table TABLE Target table to insert the data
--seperator SEPERATOR
Column seperator in the file
--ignore_errors IGNORE_ERRORS
Try to keep reading lines if some lines yield errors. default
value -> False
--n_rows N_ROWS Number of rows from the file to be inserted to database.
Default -> all
```
- Otherwise, you can access the `ingest_data_from_file` API of the module. It takes the same set of parameters to the API in the below format,
```python
def ingest_data_from_file(
file_path: str,
target_database: str,
db_host: str,
db_port: int,
db_user: str,
db_password: str,
data_db: str,
data_table: str,
file_seperator: str,
ignore_file_errors: bool,
n_rows_to_insert: int,
) -> None:
...
```
### Roadmap
In upcoming versions, `files2db` will support commonly used file formats like, `.json`, `.parquet`, `.tsv`, `.xlsx`, `.arrow` and the databases, not limited to, `postgresql`, `sqlite`, `MSSQL` and what not.
Raw data
{
"_id": null,
"home_page": "https://github.com/FluffyDietEngine/files-2-db",
"name": "files2db",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "data-migration,mysql,csv,data-engineering,automation",
"author": "Santhosh Solomon",
"author_email": "solomon.santhosh@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/66/4f/2546a6a267921f76eac4e9b6dadb6222b2bd4fd662b962bb552240ab3652/files2db-0.1.1.tar.gz",
"platform": "any",
"description": "## files-2-db\n\nUploading data from files to database made easy! \n\n## Background\n\nImporting a file from your local machine into a database which is in a remote server is painful process. The exact same process can be done through a command now. \n\n### Current implementation\n- File formats supported - `.csv`\n- Database engines supported - `mysql`\n\n### Usage\n\nThe tool can be used in two ways. \n- Through command line, where you do not even need a python script to be created. \n ```cmd\n usage: file2db [-h] --host HOST [--port PORT] --user USER --password PASSWORD\n --database DATABASE --table TABLE [--seperator SEPERATOR]\n [--ignore_errors IGNORE_ERRORS] [--n_rows N_ROWS]\n Source-file Target-database\n\n\n file2db - Transfer of data from your local file to Database, made simple.\n\n positional arguments:\n Source-file Absolute path of the file to be uploaded, with suffix\n Target-database Target database where the file has to be uploaded\n\n options:\n -h, --help show this help message and exit\n --host HOST URL or IP address representation of database host\n --port PORT Port number to connect to the database\n --user USER Username for database authentication\n --password PASSWORD Password for database authentication\n --database DATABASE Target database\n --table TABLE Target table to insert the data\n --seperator SEPERATOR\n Column seperator in the file\n --ignore_errors IGNORE_ERRORS\n Try to keep reading lines if some lines yield errors. default\n value -> False\n --n_rows N_ROWS Number of rows from the file to be inserted to database.\n Default -> all\n ```\n\n- Otherwise, you can access the `ingest_data_from_file` API of the module. It takes the same set of parameters to the API in the below format, \n\n ```python\n def ingest_data_from_file(\n file_path: str,\n target_database: str,\n db_host: str,\n db_port: int,\n db_user: str,\n db_password: str,\n data_db: str,\n data_table: str,\n file_seperator: str,\n ignore_file_errors: bool,\n n_rows_to_insert: int,\n ) -> None:\n ...\n ```\n\n### Roadmap\n\nIn upcoming versions, `files2db` will support commonly used file formats like, `.json`, `.parquet`, `.tsv`, `.xlsx`, `.arrow` and the databases, not limited to, `postgresql`, `sqlite`, `MSSQL` and what not. \n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "Migration from local files to database made simple!",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/FluffyDietEngine/files-2-db"
},
"split_keywords": [
"data-migration",
"mysql",
"csv",
"data-engineering",
"automation"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c35678ce24f5e8496ce8ff462d990ab5bb0135e56cbd35a3e92920aaaaa80de3",
"md5": "65e5992b7d294cf6234001cef46d8093",
"sha256": "24c79420f4241436b70b21e34f07e55111a3c99608776aeb2edb6e5b3770d36d"
},
"downloads": -1,
"filename": "files2db-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "65e5992b7d294cf6234001cef46d8093",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 12652,
"upload_time": "2024-02-05T02:17:39",
"upload_time_iso_8601": "2024-02-05T02:17:39.785761Z",
"url": "https://files.pythonhosted.org/packages/c3/56/78ce24f5e8496ce8ff462d990ab5bb0135e56cbd35a3e92920aaaaa80de3/files2db-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "664f2546a6a267921f76eac4e9b6dadb6222b2bd4fd662b962bb552240ab3652",
"md5": "b53ff91119e7c467595ae47bb4fe9b02",
"sha256": "c951024b663e584fcaf451b56c27624c7bb7c3b312d0b0a96762c27650e9152a"
},
"downloads": -1,
"filename": "files2db-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "b53ff91119e7c467595ae47bb4fe9b02",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 9193,
"upload_time": "2024-02-05T02:17:41",
"upload_time_iso_8601": "2024-02-05T02:17:41.615231Z",
"url": "https://files.pythonhosted.org/packages/66/4f/2546a6a267921f76eac4e9b6dadb6222b2bd4fd662b962bb552240ab3652/files2db-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-05 02:17:41",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "FluffyDietEngine",
"github_project": "files-2-db",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "files2db"
}
JSON
