# **DATAMIMIC Community Edition**
[](https://sonarcloud.io/summary/new_code?id=rapiddweller_datamimic)
[](https://sonarcloud.io/summary/new_code?id=rapiddweller_datamimic)
[](https://sonarcloud.io/summary/new_code?id=rapiddweller_datamimic)
[](https://sonarcloud.io/summary/new_code?id=rapiddweller_datamimic)
[](https://opensource.org/licenses/MIT)
[](https://www.python.org/downloads/)
[](https://github.com/rapiddweller/datamimic/stargazers)
[](https://github.com/rapiddweller/datamimic/network)
[](https://badge.fury.io/py/datamimic-ce)
[](https://pepy.tech/project/datamimic-ce)
---
## Introduction
Welcome to **DATAMIMIC Community Edition**, the AI-powered platform that revolutionizes test data generation! By leveraging advanced AI and model-driven technologies, DATAMIMIC enables developers and testers to create realistic, scalable, and privacy-compliant test data with ease.
[](https://www.youtube.com/watch?v=sycO7qd1Bhk)
---
## Key Features
- 🧠 **Model-Driven Data Generation**: Utilize sophisticated algorithms to simulate real-world data scenarios.
- 🔮 **AI-Powered Data Generation**: Simulate real-world data scenarios using cutting-edge AI algorithms. (Like GANs, LLMs, and more)
- 🛡️ **Data Privacy Compliance**: Anonymize and pseudonymize data to meet GDPR and global data protection standards.
- 🚀 **High Performance**: Engineered for scalability to handle complex datasets efficiently.
- 🐍 **Seamless Python Integration**: Easily integrate with Python projects and manage dependencies.
- ⚙️ **Extensibility**: Customize and extend functionalities to suit your specific testing needs.
> **Note:** The Community Edition focuses on core functionalities and does not include AI-powered features like automatic model generation. These advanced features are available in the **Enterprise Edition**.
---
## Why Use DATAMIMIC?
Traditional test data generation can be time-consuming and may compromise data privacy. DATAMIMIC addresses these challenges by:
- **Reducing Time-to-Market**: Quickly generate test data without manual intervention.
- **Enhancing Test Coverage**: Simulate diverse data scenarios for comprehensive testing.
- **Ensuring Compliance**: Maintain data privacy and comply with legal regulations.
- **Improving Data Quality**: Generate realistic data that mirrors production environments.
---
## Installation
### Prerequisites
- **Operating System**: Windows, macOS, or Linux
- **Python**: Version **3.10** or higher
- **Optional**: uv Package Manager for development setup [GitHub](https://github.com/astral-sh/uv)
### User Installation
The simplest way to get started with DATAMIMIC is through pip:
```bash
pip install datamimic-ce
```
Verify the installation:
```bash
datamimic --version
```
### Developer Installation
For contributors and developers who want to work with the source code:
1. Install uv Package Manager:
```bash
pip install uv
```
2. Clone and set up the repository:
```bash
git clone https://github.com/rapiddweller/datamimic.git
cd datamimic
uv sync
```
---
## Usage Guide
### Basic Usage
1. Create a new data generation project:
```bash
datamimic init my-project
cd my-project
```
2. Configure your data model in `datamimic.xml`:
```xml
<setup>
<generate name="datamimic_user_list" count="100" target="CSV,JSON">
<variable name="person" entity="Person(min_age=18, max_age=90, female_quota=0.5)"/>
<key name="id" generator="IncrementGenerator"/>
<key name="first_name" script="person.given_name"/>
<key name="last_name" script="person.family_name"/>
<key name="gender" script="person.gender"/>
<key name="birthDate" script="person.birthdate" converter="DateFormat('%d.%m.%Y')"/>
<key name="email" script="person.family_name + '@' + person.given_name + '.de'"/>
<key name="ce_user" values="True, False"/>
<key name="ee_user" values="True, False"/>
<key name="datamimic_lover" constant="DEFINITELY"/>
</generate>
</setup>
```
3. Generate data:
```bash
datamimic run datamimic.xml
```
4. Access the generated data in the `output` directory.
**json export:**
```json
[
{"id": 1, "first_name": "Mary", "last_name": "Mcgowan", "gender": "female", "birthDate": "1946-05-15T00:00:00", "email": "Mcgowan@Mary.de", "ce_user": false, "ee_user": true, "datamimic_lover": "DEFINITELY"},
{"id": 2, "first_name": "Gabrielle", "last_name": "Malone", "gender": "female", "birthDate": "1989-11-27T00:00:00", "email": "Malone@Gabrielle.de", "ce_user": false, "ee_user": true, "datamimic_lover": "DEFINITELY"},
{"id": 4, "first_name": "Margaret", "last_name": "Torres", "gender": "female", "birthDate": "2006-07-13T00:00:00", "email": "Torres@Margaret.de", "ce_user": false, "ee_user": false, "datamimic_lover": "DEFINITELY"},
{"id": 5, "first_name": "Monica", "last_name": "Meyers", "gender": "female", "birthDate": "1983-07-22T00:00:00", "email": "Meyers@Monica.de", "ce_user": true, "ee_user": false, "datamimic_lover": "DEFINITELY"},
{"id": 6, "first_name": "Jason", "last_name": "Davis", "gender": "male", "birthDate": "1941-07-05T00:00:00", "email": "Davis@Jason.de", "ce_user": true, "ee_user": false, "datamimic_lover": "DEFINITELY"},
{"...": "..."},
{"id": 100, "first_name": "Jared", "last_name": "Rivas", "gender": "male", "birthDate": "1975-03-16T00:00:00", "email": "Rivas@Jared.de", "ce_user": true, "ee_user": true, "datamimic_lover": "DEFINITELY"}
]
```
**csv export:**
```csv
id|first_name|last_name|gender|birthDate|email|ce_user|ee_user|datamimic_lover
1|Mary|Mcgowan|female|1946-05-15 00:00:00|Mcgowan@Mary.de|False|True|DEFINITELY
2|Gabrielle|Malone|female|1989-11-27 00:00:00|Malone@Gabrielle.de|False|True|DEFINITELY
3|Antonio|Davis|male|2005-05-12 00:00:00|Davis@Antonio.de|False|True|DEFINITELY
4|Margaret|Torres|female|2006-07-13 00:00:00|Torres@Margaret.de|False|False|DEFINITELY
5|Monica|Meyers|female|1983-07-22 00:00:00|Meyers@Monica.de|True|False|DEFINITELY
...
100|Jason|Davis|male|1941-07-05 00:00:00|Davis@Jason.de|True|False|DEFINITELY
```
### Advanced Features
DATAMIMIC supports various advanced features:
- **Custom Generators**: Create your own data generators
- **Data Relationships**: Define complex relationships between entities
- **Import/Export Formats CE**: Support for JSON, XML, CSV, RDBMS and MongoDB
- **Import/Export Formats EE**: Kafka, EDI, XSD and more
- **Data Anonymization**: Anonymize data to comply with privacy regulations
- **Data Validation**: Define and enforce data validation rules
- **Scripting**: Extend functionality using Python scripts
- **Database Integration**: Connect to databases for seamless data generation
- **Model-Driven Generation**: Utilize models to generate realistic data
- **Validation Rules**: Define and enforce data validation rules
- **Scripting**: Extend functionality using Python scripts
---
## Examples and Demos
Explore our demo collection:
```bash
# List available demos
datamimic demo list
# Run a specific demo
datamimic demo create demo-model
datamimic run ./demo-model/datamimic.xml
```
Key demos include:
- Basic entity generation
- Complex relationships
- Database integration
- Custom generator creation
- Privacy compliance examples
Find more examples in the `datamimic_ce/demos` directory.
---
## Contributing
We ❤️ contributions! Here's how you can help:
- **Code Contributions**: Submit pull requests for new features or bug fixes.
- **Documentation**: Improve existing docs or help with translations.
- **Community Engagement**: Join discussions and support other users.
Check out our [Contribution Guidelines](CONTRIBUTING.md) and [Code of Conduct](CODE_OF_CONDUCT.md).
---
## License
DATAMIMIC CE is now open source and licensed under MIT:
- 📄 **Open Source License**: Licensed under the [MIT License](LICENSE)
- 🆓 **Free for Everyone**: Use freely for both personal and commercial projects
- 💡 **Key Permissions**:
- Commercial use
- Modification
- Distribution
- Private use
For questions or support, contact us at [info@rapiddweller.com](mailto:info@rapiddweller.com).
---
## Support
Need help or have questions? We're here for you!
- 📚 [Documentation](https://docs.datamimic.io)
- 💬 [GitHub Discussions](https://github.com/rapiddweller/datamimic/discussions)
- 🐛 [Issue Tracker](https://github.com/rapiddweller/datamimic/issues)
- 📧 [Email Support](mailto:support@rapiddweller.com)
---
## Connect with Us
Stay updated and connect with our community!
- 🌐 **Website**: [www.datamimic.io](https://datamimic.io)
- 🏢 **Rapiddweller**: [www.rapiddweller.com](https://rapiddweller.com)
- 💼 **LinkedIn**: [rapiddweller](https://www.linkedin.com/company/rapiddweller)
- 🐦 **Twitter**: [@rapiddweller](https://twitter.com/rapiddweller)
---
## Acknowledgments
A big thank you to all our contributors! Your efforts make DATAMIMIC possible.
---
**Don't forget to ⭐ star and 👀 watch this repository to stay updated!**
---
**Legal Notices**
For detailed licensing information, please see the [LICENSE](LICENSE) file.
Raw data
{
"_id": null,
"home_page": null,
"name": "datamimic-ce",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "datamimic, data, synthetic, generation, privacy, security, testing, modeling",
"author": null,
"author_email": "\"Rapiddweller Asia Co., Ltd.\" <info@rapiddweller.com>",
"download_url": "https://files.pythonhosted.org/packages/d1/8f/38cccaac025b06d69af47953b9041c632c63fc9116dc13389c5146ee605e/datamimic_ce-1.1.0.tar.gz",
"platform": null,
"description": "# **DATAMIMIC Community Edition**\n\n[](https://sonarcloud.io/summary/new_code?id=rapiddweller_datamimic)\n[](https://sonarcloud.io/summary/new_code?id=rapiddweller_datamimic)\n[](https://sonarcloud.io/summary/new_code?id=rapiddweller_datamimic)\n[](https://sonarcloud.io/summary/new_code?id=rapiddweller_datamimic)\n[](https://opensource.org/licenses/MIT)\n[](https://www.python.org/downloads/)\n[](https://github.com/rapiddweller/datamimic/stargazers)\n[](https://github.com/rapiddweller/datamimic/network)\n[](https://badge.fury.io/py/datamimic-ce)\n[](https://pepy.tech/project/datamimic-ce)\n---\n\n## Introduction\n\nWelcome to **DATAMIMIC Community Edition**, the AI-powered platform that revolutionizes test data generation! By leveraging advanced AI and model-driven technologies, DATAMIMIC enables developers and testers to create realistic, scalable, and privacy-compliant test data with ease.\n\n[](https://www.youtube.com/watch?v=sycO7qd1Bhk)\n\n---\n\n## Key Features\n\n- \ud83e\udde0 **Model-Driven Data Generation**: Utilize sophisticated algorithms to simulate real-world data scenarios.\n- \ud83d\udd2e **AI-Powered Data Generation**: Simulate real-world data scenarios using cutting-edge AI algorithms. (Like GANs, LLMs, and more)\n- \ud83d\udee1\ufe0f **Data Privacy Compliance**: Anonymize and pseudonymize data to meet GDPR and global data protection standards.\n- \ud83d\ude80 **High Performance**: Engineered for scalability to handle complex datasets efficiently.\n- \ud83d\udc0d **Seamless Python Integration**: Easily integrate with Python projects and manage dependencies.\n- \u2699\ufe0f **Extensibility**: Customize and extend functionalities to suit your specific testing needs.\n\n> **Note:** The Community Edition focuses on core functionalities and does not include AI-powered features like automatic model generation. These advanced features are available in the **Enterprise Edition**.\n\n\n---\n\n## Why Use DATAMIMIC?\n\nTraditional test data generation can be time-consuming and may compromise data privacy. DATAMIMIC addresses these challenges by:\n\n- **Reducing Time-to-Market**: Quickly generate test data without manual intervention.\n- **Enhancing Test Coverage**: Simulate diverse data scenarios for comprehensive testing.\n- **Ensuring Compliance**: Maintain data privacy and comply with legal regulations.\n- **Improving Data Quality**: Generate realistic data that mirrors production environments.\n\n---\n\n## Installation\n\n### Prerequisites\n\n- **Operating System**: Windows, macOS, or Linux\n- **Python**: Version **3.10** or higher\n- **Optional**: uv Package Manager for development setup [GitHub](https://github.com/astral-sh/uv)\n\n### User Installation\n\nThe simplest way to get started with DATAMIMIC is through pip:\n\n```bash \npip install datamimic-ce\n```\n\nVerify the installation:\n\n```bash\ndatamimic --version\n```\n\n### Developer Installation\n\nFor contributors and developers who want to work with the source code:\n\n1. Install uv Package Manager:\n\n ```bash\n pip install uv\n ```\n\n2. Clone and set up the repository:\n\n ```bash\n git clone https://github.com/rapiddweller/datamimic.git\n cd datamimic\n uv sync\n ```\n\n---\n\n## Usage Guide\n\n### Basic Usage\n\n1. Create a new data generation project:\n\n ```bash\n datamimic init my-project\n cd my-project\n ```\n\n2. Configure your data model in `datamimic.xml`:\n\n ```xml\n <setup>\n <generate name=\"datamimic_user_list\" count=\"100\" target=\"CSV,JSON\">\n <variable name=\"person\" entity=\"Person(min_age=18, max_age=90, female_quota=0.5)\"/>\n <key name=\"id\" generator=\"IncrementGenerator\"/>\n <key name=\"first_name\" script=\"person.given_name\"/>\n <key name=\"last_name\" script=\"person.family_name\"/>\n <key name=\"gender\" script=\"person.gender\"/>\n <key name=\"birthDate\" script=\"person.birthdate\" converter=\"DateFormat('%d.%m.%Y')\"/>\n <key name=\"email\" script=\"person.family_name + '@' + person.given_name + '.de'\"/>\n <key name=\"ce_user\" values=\"True, False\"/>\n <key name=\"ee_user\" values=\"True, False\"/>\n <key name=\"datamimic_lover\" constant=\"DEFINITELY\"/>\n </generate>\n </setup>\n ```\n\n3. Generate data:\n\n ```bash\n datamimic run datamimic.xml\n ```\n \n4. Access the generated data in the `output` directory.\n\n **json export:**\n ```json\n [\n {\"id\": 1, \"first_name\": \"Mary\", \"last_name\": \"Mcgowan\", \"gender\": \"female\", \"birthDate\": \"1946-05-15T00:00:00\", \"email\": \"Mcgowan@Mary.de\", \"ce_user\": false, \"ee_user\": true, \"datamimic_lover\": \"DEFINITELY\"},\n {\"id\": 2, \"first_name\": \"Gabrielle\", \"last_name\": \"Malone\", \"gender\": \"female\", \"birthDate\": \"1989-11-27T00:00:00\", \"email\": \"Malone@Gabrielle.de\", \"ce_user\": false, \"ee_user\": true, \"datamimic_lover\": \"DEFINITELY\"},\n {\"id\": 4, \"first_name\": \"Margaret\", \"last_name\": \"Torres\", \"gender\": \"female\", \"birthDate\": \"2006-07-13T00:00:00\", \"email\": \"Torres@Margaret.de\", \"ce_user\": false, \"ee_user\": false, \"datamimic_lover\": \"DEFINITELY\"},\n {\"id\": 5, \"first_name\": \"Monica\", \"last_name\": \"Meyers\", \"gender\": \"female\", \"birthDate\": \"1983-07-22T00:00:00\", \"email\": \"Meyers@Monica.de\", \"ce_user\": true, \"ee_user\": false, \"datamimic_lover\": \"DEFINITELY\"},\n {\"id\": 6, \"first_name\": \"Jason\", \"last_name\": \"Davis\", \"gender\": \"male\", \"birthDate\": \"1941-07-05T00:00:00\", \"email\": \"Davis@Jason.de\", \"ce_user\": true, \"ee_user\": false, \"datamimic_lover\": \"DEFINITELY\"},\n {\"...\": \"...\"},\n {\"id\": 100, \"first_name\": \"Jared\", \"last_name\": \"Rivas\", \"gender\": \"male\", \"birthDate\": \"1975-03-16T00:00:00\", \"email\": \"Rivas@Jared.de\", \"ce_user\": true, \"ee_user\": true, \"datamimic_lover\": \"DEFINITELY\"}\n ]\n ```\n\n **csv export:**\n ```csv\n id|first_name|last_name|gender|birthDate|email|ce_user|ee_user|datamimic_lover\n 1|Mary|Mcgowan|female|1946-05-15 00:00:00|Mcgowan@Mary.de|False|True|DEFINITELY\n 2|Gabrielle|Malone|female|1989-11-27 00:00:00|Malone@Gabrielle.de|False|True|DEFINITELY\n 3|Antonio|Davis|male|2005-05-12 00:00:00|Davis@Antonio.de|False|True|DEFINITELY\n 4|Margaret|Torres|female|2006-07-13 00:00:00|Torres@Margaret.de|False|False|DEFINITELY\n 5|Monica|Meyers|female|1983-07-22 00:00:00|Meyers@Monica.de|True|False|DEFINITELY\n ...\n 100|Jason|Davis|male|1941-07-05 00:00:00|Davis@Jason.de|True|False|DEFINITELY\n ```\n\n\n### Advanced Features\n\nDATAMIMIC supports various advanced features:\n\n- **Custom Generators**: Create your own data generators\n- **Data Relationships**: Define complex relationships between entities\n- **Import/Export Formats CE**: Support for JSON, XML, CSV, RDBMS and MongoDB\n- **Import/Export Formats EE**: Kafka, EDI, XSD and more\n- **Data Anonymization**: Anonymize data to comply with privacy regulations\n- **Data Validation**: Define and enforce data validation rules\n- **Scripting**: Extend functionality using Python scripts\n- **Database Integration**: Connect to databases for seamless data generation\n- **Model-Driven Generation**: Utilize models to generate realistic data\n- **Validation Rules**: Define and enforce data validation rules\n- **Scripting**: Extend functionality using Python scripts\n\n---\n\n## Examples and Demos\n\nExplore our demo collection:\n\n```bash\n# List available demos\ndatamimic demo list\n\n# Run a specific demo\ndatamimic demo create demo-model\ndatamimic run ./demo-model/datamimic.xml\n```\n\nKey demos include:\n- Basic entity generation\n- Complex relationships\n- Database integration\n- Custom generator creation\n- Privacy compliance examples\n\nFind more examples in the `datamimic_ce/demos` directory.\n\n---\n\n## Contributing\n\nWe \u2764\ufe0f contributions! Here's how you can help:\n\n- **Code Contributions**: Submit pull requests for new features or bug fixes.\n- **Documentation**: Improve existing docs or help with translations.\n- **Community Engagement**: Join discussions and support other users.\n\nCheck out our [Contribution Guidelines](CONTRIBUTING.md) and [Code of Conduct](CODE_OF_CONDUCT.md).\n\n---\n\n## License\n\nDATAMIMIC CE is now open source and licensed under MIT:\n\n- \ud83d\udcc4 **Open Source License**: Licensed under the [MIT License](LICENSE)\n- \ud83c\udd93 **Free for Everyone**: Use freely for both personal and commercial projects\n- \ud83d\udca1 **Key Permissions**:\n - Commercial use\n - Modification\n - Distribution\n - Private use\n\nFor questions or support, contact us at [info@rapiddweller.com](mailto:info@rapiddweller.com).\n\n---\n\n## Support\n\nNeed help or have questions? We're here for you!\n\n- \ud83d\udcda [Documentation](https://docs.datamimic.io)\n- \ud83d\udcac [GitHub Discussions](https://github.com/rapiddweller/datamimic/discussions)\n- \ud83d\udc1b [Issue Tracker](https://github.com/rapiddweller/datamimic/issues)\n- \ud83d\udce7 [Email Support](mailto:support@rapiddweller.com)\n\n---\n\n## Connect with Us\n\nStay updated and connect with our community!\n\n- \ud83c\udf10 **Website**: [www.datamimic.io](https://datamimic.io)\n- \ud83c\udfe2 **Rapiddweller**: [www.rapiddweller.com](https://rapiddweller.com)\n- \ud83d\udcbc **LinkedIn**: [rapiddweller](https://www.linkedin.com/company/rapiddweller)\n- \ud83d\udc26 **Twitter**: [@rapiddweller](https://twitter.com/rapiddweller)\n\n---\n\n## Acknowledgments\n\nA big thank you to all our contributors! Your efforts make DATAMIMIC possible.\n\n---\n\n**Don't forget to \u2b50 star and \ud83d\udc40 watch this repository to stay updated!**\n\n---\n\n**Legal Notices**\n\nFor detailed licensing information, please see the [LICENSE](LICENSE) file.\n",
"bugtrack_url": null,
"license": null,
"summary": null,
"version": "1.1.0",
"project_urls": {
"Homepage": "https://datamimic.io"
},
"split_keywords": [
"datamimic",
" data",
" synthetic",
" generation",
" privacy",
" security",
" testing",
" modeling"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8aa6081408d882b0859637224a38d2d72df8ff5d5b490c077784399e0e03b2af",
"md5": "1ad6dcff7cd63f080e405dd5edc106bc",
"sha256": "daa547b5a14619df542b5eb4b5fa51acfca0a5b6e1ae509458ca1e7c9740c904"
},
"downloads": -1,
"filename": "datamimic_ce-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1ad6dcff7cd63f080e405dd5edc106bc",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 12870889,
"upload_time": "2024-12-12T08:30:24",
"upload_time_iso_8601": "2024-12-12T08:30:24.233257Z",
"url": "https://files.pythonhosted.org/packages/8a/a6/081408d882b0859637224a38d2d72df8ff5d5b490c077784399e0e03b2af/datamimic_ce-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d18f38cccaac025b06d69af47953b9041c632c63fc9116dc13389c5146ee605e",
"md5": "dc6ed9f1f7523d51add737865d7a9c30",
"sha256": "3414cbac0646764db12e56075c972159bc1a507323ceebd0b31f117e5790f261"
},
"downloads": -1,
"filename": "datamimic_ce-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "dc6ed9f1f7523d51add737865d7a9c30",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 12381470,
"upload_time": "2024-12-12T08:30:29",
"upload_time_iso_8601": "2024-12-12T08:30:29.160095Z",
"url": "https://files.pythonhosted.org/packages/d1/8f/38cccaac025b06d69af47953b9041c632c63fc9116dc13389c5146ee605e/datamimic_ce-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-12 08:30:29",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "datamimic-ce"
}
JSON