| Name | dynamodbfaker JSON |
| Version |
1.0.2
JSON |
| download |
| home_page | |
| Summary | A Python package to generate and insert fake data for AWS Dynamodb |
| upload_time | 2024-03-12 20:28:54 |
| maintainer | |
| docs_url | None |
| author | Necati Arslan |
| requires_python | |
| license | |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Dynamodb Faker
dynamodbfaker is a versatile Python package that empowers you to effortlessly create realistic but synthetic dynamodb data for a wide range of applications. If you need to generate test data for software development, this tool simplifies the process with an intuitive schema definition in YAML format.
### Key Features
**Schema Definition:** Define your target schema using a simple YAML file. Specify the structure of your table, attribute names and fake data generation code.
**Faker and Randomization:** Leverage the power of the Faker library and random data generation to create authentic-looking fake data that mimics real-world scenarios.
**Insert to Your Dynamodb Table:**
Insert generated data directly to your dynamodb table.
### Installation
```bash
pip install dynamodbfaker
```
### AWS Credentials
dynamodbfaker uses your local Aws Credentials to access Aws. You need to setup them also to access Aws. \
Aws credentials are basicaly a config file under ~/.aws/credentials which contains your access key, secret key and/or tokens. \
Here is how you can setup \
Youtube : https://www.youtube.com/watch?v=qmtDRmplMG4 \
Aws Doc : https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html
### Sample Yaml File
```
version: 1
config:
locale: en_US #OPTIONAL faker locale Default:en_US
on_update_item_error: RAISE_ERROR #OPTIONAL RAISE_ERROR, SKIP Default:RAISE_ERROR
python_import:
- datetime
aws:
region: us-east-1 #OPTIONAL Default:the region of the current session
credentials_profile: default #OPTIONAL the profile name in your local .aws/config file Default:default
dynamodb_table:
table_name: person # MANDATORY
row_count: 10000 #OPTIONAL Default 10
batch: 10 #OPTIONAL Batch size when inserting dynamodb
sleep: 1000 #OPTIONAL Sleep time between inserts to dynamodb
attributes:
- name: id #MANDATORY
data: row_id #OPTIONAL Default fake.word()
- name: first_name
data: fake.first_name()
- name: last_name
data: fake.last_name()
- name: full_name
data: first_name + " " + last_name
- name: age
data: fake.random_int(18, 90)
- name: dob
data: fake.date_of_birth()
- name: street_address
data: fake.street_address()
- name: city
data: fake.city()
- name: state_abbr
data: fake.state_abbr()
- name: postcode
data: fake.postcode()
- name: gender
data: random.choice(["male", "female"])
null_percentage: 0.3
- name: left_handed
data: fake.pybool()
- name: height
data: None
```
[full yml example](tests/test_table.yaml)
### Sample Code
```python
import dynamodbfaker
# insert fake data to your dynamodb table
dynamodbfaker.to_dynamodb("test_table.yaml")
# insert fake data to your dynamodb table and export to target folder in json format
dynamodbfaker.to_dynamodb("test_table.yaml", "./target_folder")
# export in json format
dynamodbfaker.to_json("test_table.yaml", "./target_folder")
# you can use customer faker provider
from faker_education import SchoolProvider
dynamodbfaker.to_json("test_table.yaml", "./target_folder", fake_provider=SchoolProvider)
# multiple custom provider in list also works
```
### Sample CLI Command
You can use dynamodbfaker in your terminal for adhoc needs or shell script to automate fake data generation. \
Faker custom providers and custom functions are not supported in CLI.
```bash
# insert fake data to your dynamodb table
dynamodbfaker --config test_table.yaml --to dynamodb
# insert fake data to your dynamodb table and export to target_folder in json format
dynamodbfaker --config test_table.yaml --to dynamodb --target ./target_folder
# exports to current folder in json format
dynamodbfaker --config test_table.yaml
# exports to target folder in json format
dynamodbfaker --config test_table.yaml --target ./target_folder
# exports to target file in json format
dynamodbfaker --config test_table.yaml --target ./target_folder/target_file.json
```
### Sample JSON Output
```json
{
"Items": [
{
"id": {
"N": 1
},
"first_name": {
"S": "Connie"
},
"last_name": {
"S": "Skinner"
},
"full_name": {
"S": "Connie Skinner"
},
"age": {
"N": 89
},
"dob": {
"S": "1968-09-25"
},
"street_address": {
"S": "5939 Christopher Crescent Apt. 747"
},
"city": {
"S": "North Mistyside"
},
"state_abbr": {
"S": "HI"
},
"postcode": {
"S": "67235"
},
"gender": {
"S": "male"
},
"left_handed": {
"BOOL": true
},
"height": {
"NULL": null
}
},
{
"id": {
"N": 2
},
"first_name": {
"S": "Carla"
},
"last_name": {
"S": "Robles"
},
"age": {
"N": 54
},
"dob": {
"S": "1921-12-08"
},
"street_address": {
"S": "35264 Jones Squares"
},
"city": {
"S": "Michelleland"
},
"state_abbr": {
"S": "TX"
},
"postcode": {
"S": "94786"
},
"gender": {
"S": "female"
},
"left_handed": {
"BOOL": false
},
"height": {
"NULL": null
}
}
],
"Count": 2
}
```
### Custom Functions
With Dynamodb Faker, you have the flexibility to provide your own custom functions to generate column data. This advanced feature empowers developers to create custom fake data generation logic that can pull data from a database, API, file, or any other source as needed. You can also supply multiple functions in a list, allowing for even more versatility. The custom function you provide should return a single value, giving you full control over your synthetic data generation.
```python
from dynamodbfaker import dynamodbfaker
from faker import Faker
fake = Faker()
def get_level():
return f"level {fake.random_int(1, 5)}"
dynamodbfaker.to_json("test_table.yaml", "./target_folder", custom_function=get_level)
```
Add get_level function to your yaml file
```
version: 1
config:
locale: en_US #faker locale Default:en_US
on_update_item_error: RAISE_ERROR #RAISE_ERROR, SKIP Default:RAISE_ERROR
if_item_exists: OVERWRITE #OVERWRITE, SKIP Default:OVERWRITE
empty_table_first: False #True/False Default:False
aws:
region: us-east-1
credentials_profile: default #the profile name in your local .aws/config file Default:default
dynamodb_table:
table_name: person
row_count: 10000
attributes:
- name: id
data: row_id
- name: first_name
data: fake.first_name()
- name: last_name
data: fake.last_name()
- name: age
data: fake.random_int(18, 90)
- column_name: level
data: get_level() # custom function
```
### Faker Functions List
https://faker.readthedocs.io/en/master/providers.html#
### Bug Report & New Feature Request
https://github.com/necatiarslan/dynamodb-faker/issues/new
### Todo
- Support for Set (NS/SS/BS), List (L), Map (M) types
- Updating current items
### Nice To Have
-
Follow me on linkedin to get latest news \
https://www.linkedin.com/in/necati-arslan/
Thanks, \
Necati ARSLAN \
necatia@gmail.com
Raw data
{
"_id": null,
"home_page": "",
"name": "dynamodbfaker",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Necati Arslan",
"author_email": "necatia@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/ee/56/7961e571055b2e79a65a11b450d2faae424fcec9ede27e3b4cd2d6379c07/dynamodbfaker-1.0.2.tar.gz",
"platform": null,
"description": "# Dynamodb Faker\ndynamodbfaker is a versatile Python package that empowers you to effortlessly create realistic but synthetic dynamodb data for a wide range of applications. If you need to generate test data for software development, this tool simplifies the process with an intuitive schema definition in YAML format.\n\n### Key Features\n**Schema Definition:** Define your target schema using a simple YAML file. Specify the structure of your table, attribute names and fake data generation code.\n\n**Faker and Randomization:** Leverage the power of the Faker library and random data generation to create authentic-looking fake data that mimics real-world scenarios.\n\n**Insert to Your Dynamodb Table:**\nInsert generated data directly to your dynamodb table. \n\n### Installation\n```bash \npip install dynamodbfaker\n```\n\n### AWS Credentials\ndynamodbfaker uses your local Aws Credentials to access Aws. You need to setup them also to access Aws. \\\nAws credentials are basicaly a config file under ~/.aws/credentials which contains your access key, secret key and/or tokens. \\\nHere is how you can setup \\\nYoutube : https://www.youtube.com/watch?v=qmtDRmplMG4 \\\nAws Doc : https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html \n\n### Sample Yaml File\n```\nversion: 1\nconfig:\n locale: en_US #OPTIONAL faker locale Default:en_US\n on_update_item_error: RAISE_ERROR #OPTIONAL RAISE_ERROR, SKIP Default:RAISE_ERROR\n python_import:\n - datetime\naws:\n region: us-east-1 #OPTIONAL Default:the region of the current session\n credentials_profile: default #OPTIONAL the profile name in your local .aws/config file Default:default\ndynamodb_table:\n table_name: person # MANDATORY\n row_count: 10000 #OPTIONAL Default 10\n batch: 10 #OPTIONAL Batch size when inserting dynamodb\n sleep: 1000 #OPTIONAL Sleep time between inserts to dynamodb\n attributes:\n - name: id #MANDATORY\n data: row_id #OPTIONAL Default fake.word()\n - name: first_name\n data: fake.first_name()\n - name: last_name\n data: fake.last_name()\n - name: full_name\n data: first_name + \" \" + last_name\n - name: age\n data: fake.random_int(18, 90)\n - name: dob\n data: fake.date_of_birth()\n - name: street_address\n data: fake.street_address()\n - name: city\n data: fake.city()\n - name: state_abbr\n data: fake.state_abbr()\n - name: postcode\n data: fake.postcode()\n - name: gender\n data: random.choice([\"male\", \"female\"])\n null_percentage: 0.3\n - name: left_handed\n data: fake.pybool()\n - name: height\n data: None\n```\n[full yml example](tests/test_table.yaml)\n\n### Sample Code\n```python\nimport dynamodbfaker\n\n# insert fake data to your dynamodb table\ndynamodbfaker.to_dynamodb(\"test_table.yaml\")\n\n# insert fake data to your dynamodb table and export to target folder in json format\ndynamodbfaker.to_dynamodb(\"test_table.yaml\", \"./target_folder\")\n\n# export in json format\ndynamodbfaker.to_json(\"test_table.yaml\", \"./target_folder\")\n\n# you can use customer faker provider\nfrom faker_education import SchoolProvider\n\ndynamodbfaker.to_json(\"test_table.yaml\", \"./target_folder\", fake_provider=SchoolProvider)\n# multiple custom provider in list also works\n```\n\n### Sample CLI Command\nYou can use dynamodbfaker in your terminal for adhoc needs or shell script to automate fake data generation. \\\nFaker custom providers and custom functions are not supported in CLI.\n```bash\n# insert fake data to your dynamodb table\ndynamodbfaker --config test_table.yaml --to dynamodb\n\n# insert fake data to your dynamodb table and export to target_folder in json format\ndynamodbfaker --config test_table.yaml --to dynamodb --target ./target_folder \n\n# exports to current folder in json format\ndynamodbfaker --config test_table.yaml\n\n# exports to target folder in json format\ndynamodbfaker --config test_table.yaml --target ./target_folder \n\n# exports to target file in json format \ndynamodbfaker --config test_table.yaml --target ./target_folder/target_file.json\n```\n\n### Sample JSON Output\n```json\n{\n \"Items\": [\n {\n \"id\": {\n \"N\": 1\n },\n \"first_name\": {\n \"S\": \"Connie\"\n },\n \"last_name\": {\n \"S\": \"Skinner\"\n },\n \"full_name\": {\n \"S\": \"Connie Skinner\"\n },\n \"age\": {\n \"N\": 89\n },\n \"dob\": {\n \"S\": \"1968-09-25\"\n },\n \"street_address\": {\n \"S\": \"5939 Christopher Crescent Apt. 747\"\n },\n \"city\": {\n \"S\": \"North Mistyside\"\n },\n \"state_abbr\": {\n \"S\": \"HI\"\n },\n \"postcode\": {\n \"S\": \"67235\"\n },\n \"gender\": {\n \"S\": \"male\"\n },\n \"left_handed\": {\n \"BOOL\": true\n },\n \"height\": {\n \"NULL\": null\n }\n },\n {\n \"id\": {\n \"N\": 2\n },\n \"first_name\": {\n \"S\": \"Carla\"\n },\n \"last_name\": {\n \"S\": \"Robles\"\n },\n \"age\": {\n \"N\": 54\n },\n \"dob\": {\n \"S\": \"1921-12-08\"\n },\n \"street_address\": {\n \"S\": \"35264 Jones Squares\"\n },\n \"city\": {\n \"S\": \"Michelleland\"\n },\n \"state_abbr\": {\n \"S\": \"TX\"\n },\n \"postcode\": {\n \"S\": \"94786\"\n },\n \"gender\": {\n \"S\": \"female\"\n },\n \"left_handed\": {\n \"BOOL\": false\n },\n \"height\": {\n \"NULL\": null\n }\n }\n ],\n \"Count\": 2\n}\n```\n\n### Custom Functions\nWith Dynamodb Faker, you have the flexibility to provide your own custom functions to generate column data. This advanced feature empowers developers to create custom fake data generation logic that can pull data from a database, API, file, or any other source as needed. You can also supply multiple functions in a list, allowing for even more versatility. The custom function you provide should return a single value, giving you full control over your synthetic data generation.\n\n```python\nfrom dynamodbfaker import dynamodbfaker\nfrom faker import Faker\n\nfake = Faker()\ndef get_level():\n return f\"level {fake.random_int(1, 5)}\"\n\ndynamodbfaker.to_json(\"test_table.yaml\", \"./target_folder\", custom_function=get_level)\n```\nAdd get_level function to your yaml file\n```\nversion: 1\nconfig:\n locale: en_US #faker locale Default:en_US\n on_update_item_error: RAISE_ERROR #RAISE_ERROR, SKIP Default:RAISE_ERROR\n if_item_exists: OVERWRITE #OVERWRITE, SKIP Default:OVERWRITE\n empty_table_first: False #True/False Default:False\naws:\n region: us-east-1\n credentials_profile: default #the profile name in your local .aws/config file Default:default\ndynamodb_table:\n table_name: person\n row_count: 10000\n attributes:\n - name: id\n data: row_id\n - name: first_name\n data: fake.first_name()\n - name: last_name\n data: fake.last_name()\n - name: age\n data: fake.random_int(18, 90)\n - column_name: level\n data: get_level() # custom function\n```\n\n\n### Faker Functions List\nhttps://faker.readthedocs.io/en/master/providers.html#\n\n### Bug Report & New Feature Request\nhttps://github.com/necatiarslan/dynamodb-faker/issues/new \n\n\n### Todo\n- Support for Set (NS/SS/BS), List (L), Map (M) types\n- Updating current items\n\n### Nice To Have\n- \n\nFollow me on linkedin to get latest news \\\nhttps://www.linkedin.com/in/necati-arslan/\n\nThanks, \\\nNecati ARSLAN \\\nnecatia@gmail.com\n\n\n",
"bugtrack_url": null,
"license": "",
"summary": "A Python package to generate and insert fake data for AWS Dynamodb",
"version": "1.0.2",
"project_urls": {
"Documentation": "https://github.com/necatiarslan/dynamodb-faker/blob/main/README.md",
"Source": "https://github.com/necatiarslan/dynamodb-faker"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cdf0b42b4e0bfcf70d824a98597c2d4f5f6aa1908232466f9bb5ca2de34faf9d",
"md5": "d8d5b95e973e654fd129676a2a1aa53e",
"sha256": "2238bb4686ddacc644bce73a5eb43d993b3671341e6b3d7c74fceb67510fbca6"
},
"downloads": -1,
"filename": "dynamodbfaker-1.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d8d5b95e973e654fd129676a2a1aa53e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 13897,
"upload_time": "2024-03-12T20:28:53",
"upload_time_iso_8601": "2024-03-12T20:28:53.226681Z",
"url": "https://files.pythonhosted.org/packages/cd/f0/b42b4e0bfcf70d824a98597c2d4f5f6aa1908232466f9bb5ca2de34faf9d/dynamodbfaker-1.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ee567961e571055b2e79a65a11b450d2faae424fcec9ede27e3b4cd2d6379c07",
"md5": "da3d0de12c3bd2542d4554aba7a828ce",
"sha256": "7064bc052633614d0308c481e14f522380ac82369bcb528c2a05b27ec563f5db"
},
"downloads": -1,
"filename": "dynamodbfaker-1.0.2.tar.gz",
"has_sig": false,
"md5_digest": "da3d0de12c3bd2542d4554aba7a828ce",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 14824,
"upload_time": "2024-03-12T20:28:54",
"upload_time_iso_8601": "2024-03-12T20:28:54.562887Z",
"url": "https://files.pythonhosted.org/packages/ee/56/7961e571055b2e79a65a11b450d2faae424fcec9ede27e3b4cd2d6379c07/dynamodbfaker-1.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-12 20:28:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "necatiarslan",
"github_project": "dynamodb-faker",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "dynamodbfaker"
}
