| Name | ddeutil-io JSON |
| Version |
0.2.17
JSON |
| download |
| home_page | None |
| Summary | Input/Output transport objects |
| upload_time | 2025-07-16 01:42:35 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.9.13 |
| license | MIT |
| keywords |
io
utility
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Input/Output Data Transport
[](https://github.com/ddeutils/ddeutil-io/actions/workflows/tests.yml)
[](https://codecov.io/gh/ddeutils/ddeutil-io)
[](https://pypi.org/project/ddeutil-io/)
[](https://pypi.org/project/ddeutil-io/)
[](https://github.com/ddeutils/ddeutil-io)
[](https://github.com/psf/black)
[](http://mypy-lang.org)
[](https://github.com/pre-commit/pre-commit)
An **Input/Output Transport Objects** 🚅 was created for full managed configuration
file that include `read` and `write` from any file format like `.yaml`, `.json`,
or `.toml`. This package also provide retention and version management of config
files for configuration data lifecycle ♻️.
> [!NOTE]
> The core part of this project is **Files** module.
## 📦 Installation
```shell
pip install -U ddeutil-io
```
**Python version supported**:
| Python Version | Installation | Support Fixed Bug |
|:---------------|:----------------------------|:------------------:|
| `>=3.9,<3.14` | `pip install -U ddeutil-io` | :heavy_check_mark: |
> [!NOTE]
> This package need to install `ddeutil` first to be core package namespace.
> You do not need to pip it because I include this package to the required list.
>
> For optional dependencies that should to additional installation;
>
> | Module | Installation | Additional dependencies |
> |-------------|------------------------|------------------------------------------------|
> | `YamlFl` | `ddeutil-io[yaml]` | `pip install PyYaml` |
> | `TomlFl` | `ddeutil-io[toml]` | `pip install rtoml` |
> | `MsgpackFl` | `ddeutil-io[msgpack]` | `pip install msgpack` |
> | `Register` | `ddeutil-io[register]` | `pip install python-dateutil deepdiff fmtutil` |
## 🎯 Features
The features of this package is Input/Output data transport utility objects.
| Module | Name | Description | Remark |
|:---------|:----------------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|
| paths | PathSearch | Path Search object that use to search path tree from an input root path. | |
| | ls | List files in a directory, applying ignore-style filtering. | |
| files | Fl | Open File object that use to open any normal or compression file from current local file system | |
| | EnvFlMixin | Environment Mapping to read method of open file object mixin. | |
| | EnvFl | Dot env open file object which mapping search engine to data context that reading from dot env file format (.env). | |
| | YamlEnvFl | Yaml open file object which mapping search environment variable. | |
| | YamlFl | Yaml open file object that read data context from Yaml file format (.yml, or .yaml). | |
| | YamlFlResolve | Yaml open file object with resolve boolean convert value problem such as convert 'on' value to true instead a string of 'on' value. | |
| | YamlEnvFlResolve | Yaml open file object with resolve boolean convert value problem such as convert 'on' value to true instead a string of 'on' value before mapping search environment variable. | |
| | JsonEnvFl | Json open file object which mapping search environment variable before parsing with json package. | |
| | JsonLineFl | Json open file object that read data context from Json file format (.json) with a newline seperator. | |
| | JsonFl | Json open file object that read data context from Json file format (.json). | |
| | CsvFl | CSV open file object with comma (`,`) seperator charactor. | |
| | CsvPipeFl | CSV open file object with pipe (`\|`) seperator charactor. | |
| | TomlEnvFl | TOML open file object which mapping search environment variable before parsing with toml package from TOML file format (.toml). | |
| | TomlFl | TOML open file object that read data context from TOML file format (.toml). | |
| | PickleFl | Pickle open file object that read data context from Pickle file format (.pickle). | no cover |
| | MarshalFl | Marshal open file object that read data context from Marshal file format. | no cover |
| | MsgpackFl | Msgpack open file object that read data context from Msgpack file format. | no cover |
| stores | Store | Store File Loading Object for get data from configuration and stage. | |
| | StoreJsonToCsv | Store object that getting the Json context data and save it to stage with CSV file format. | |
| | StoreToJsonLine | Store object that getting the YAML context data and save it to stage with Json line file format. | |
| register | Register | Register Object that contain configuration loading methods and metadata management. | |
| | ArchiveRegister | Archiving Register object that implement archiving management on the Register object such as ``self.purge``, and ``self.remove`` methods. | |
| utils | rm | Remove a file or dir from an input path. | |
| | touch | Create an empty file with specific name and modified time of path it an input times was set. | |
## 💡 Usages
I will show some usage example of function in this package. If you want to use
complex or adjust some parameter, please see doc-string or real source code
(I think it does not complex, and you can see how that function work).
### ⭕ Files
For example, I will represent `YamlEnvFl` object that passing environment variable
to reading content before passing to the Yaml loader.
```yaml
data:
get: HELLO ${HELLO}
```
```python
import os
from ddeutil.io import YamlEnvFl
os.environ["HELLO"] = "WORLD"
content = YamlEnvFl('./source.yaml').read(safe=True)
assert content['data']['get'] == "HELLO WORLD"
```
> [!NOTE]
> This module do not implement special function on IO like the build-in ``open``
> function. It also makes standard ``read`` and ``write`` file objects.
### ❌ Dirs
> [!WARNING]
> This module should not use on the production.
### ⭕ Store
Store object is the storing dir system handler object that manage any files in
that dir path with `get`, `move`, `load`, `save`, or `ls` operations.
```python
from ddeutil.io import Store
store: Store = Store(path='./conf', compress="gzip")
data = store.get(name='config_file.yaml')
store.save('./stage/file.json', data)
```
```text
conf/
├─ examples/
│ ╰─ config_file.yaml
╰─ stage/
╰─ file.json
```
### ⭕ Register
The **Register Object** is the metadata generator object for the config data.
If you passing name and configs to this object, it will find the config name
in any stage storage and generate its metadata to you.
```python
from ddeutil.io.register import Register
from ddeutil.io import Params
registry: Register = Register(
name='examples:conn_data_local_file',
params=Params(**{
"stages": {
"raw": {"format": "{naming:%s}.{timestamp:%Y%m%d_%H%M%S}"},
},
}),
)
registry.move(stage="raw")
```
The raw data of this config was written in `conn_file.yaml` file.
```text
conf/
╰─ examples/
╰─ conn_file.yaml
```
When call `move` method, it will transfer data from `.yaml` file to `json` file
with the data hashing algorithm.
```text
data/
├─ __METADATA/
│ ├─ exampleconn_data_local_file.base.json
│ ╰─ exampleconn_data_local_file.raw.json
╰─ raw/
╰─ conn_file_20240101_000000.json
```
## 💬 Contribute
I do not think this project will go around the world because it has specific propose,
and you can create by your coding without this project dependency for long term
solution. So, on this time, you can open [the GitHub issue on this project 🙌](https://github.com/ddeutils/ddeutil-io/issues)
for fix bug or request new feature if you want it.
Raw data
{
"_id": null,
"home_page": null,
"name": "ddeutil-io",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9.13",
"maintainer_email": null,
"keywords": "io, utility",
"author": null,
"author_email": "ddeutils <korawich.anu@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/e6/2c/68bef53ff688e86e02f9ae97127d680a373b2c6cdb04952b3ce9c58c50db/ddeutil_io-0.2.17.tar.gz",
"platform": null,
"description": "# Input/Output Data Transport\n\n[](https://github.com/ddeutils/ddeutil-io/actions/workflows/tests.yml)\n[](https://codecov.io/gh/ddeutils/ddeutil-io)\n[](https://pypi.org/project/ddeutil-io/)\n[](https://pypi.org/project/ddeutil-io/)\n[](https://github.com/ddeutils/ddeutil-io)\n[](https://github.com/psf/black)\n[](http://mypy-lang.org)\n[](https://github.com/pre-commit/pre-commit)\n\nAn **Input/Output Transport Objects** \ud83d\ude85 was created for full managed configuration\nfile that include `read` and `write` from any file format like `.yaml`, `.json`,\nor `.toml`. This package also provide retention and version management of config\nfiles for configuration data lifecycle \u267b\ufe0f.\n\n> [!NOTE]\n> The core part of this project is **Files** module.\n\n## \ud83d\udce6 Installation\n\n```shell\npip install -U ddeutil-io\n```\n\n**Python version supported**:\n\n| Python Version | Installation | Support Fixed Bug |\n|:---------------|:----------------------------|:------------------:|\n| `>=3.9,<3.14` | `pip install -U ddeutil-io` | :heavy_check_mark: |\n\n> [!NOTE]\n> This package need to install `ddeutil` first to be core package namespace.\n> You do not need to pip it because I include this package to the required list.\n>\n> For optional dependencies that should to additional installation;\n>\n> | Module | Installation | Additional dependencies |\n> |-------------|------------------------|------------------------------------------------|\n> | `YamlFl` | `ddeutil-io[yaml]` | `pip install PyYaml` |\n> | `TomlFl` | `ddeutil-io[toml]` | `pip install rtoml` |\n> | `MsgpackFl` | `ddeutil-io[msgpack]` | `pip install msgpack` |\n> | `Register` | `ddeutil-io[register]` | `pip install python-dateutil deepdiff fmtutil` |\n\n## \ud83c\udfaf Features\n\nThe features of this package is Input/Output data transport utility objects.\n\n| Module | Name | Description | Remark |\n|:---------|:----------------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|\n| paths | PathSearch | Path Search object that use to search path tree from an input root path. | |\n| | ls | List files in a directory, applying ignore-style filtering. | |\n| files | Fl | Open File object that use to open any normal or compression file from current local file system | |\n| | EnvFlMixin | Environment Mapping to read method of open file object mixin. | |\n| | EnvFl | Dot env open file object which mapping search engine to data context that reading from dot env file format (.env). | |\n| | YamlEnvFl | Yaml open file object which mapping search environment variable. | |\n| | YamlFl | Yaml open file object that read data context from Yaml file format (.yml, or .yaml). | |\n| | YamlFlResolve | Yaml open file object with resolve boolean convert value problem such as convert 'on' value to true instead a string of 'on' value. | |\n| | YamlEnvFlResolve | Yaml open file object with resolve boolean convert value problem such as convert 'on' value to true instead a string of 'on' value before mapping search environment variable. | |\n| | JsonEnvFl | Json open file object which mapping search environment variable before parsing with json package. | |\n| | JsonLineFl | Json open file object that read data context from Json file format (.json) with a newline seperator. | |\n| | JsonFl | Json open file object that read data context from Json file format (.json). | |\n| | CsvFl | CSV open file object with comma (`,`) seperator charactor. | |\n| | CsvPipeFl | CSV open file object with pipe (`\\|`) seperator charactor. | |\n| | TomlEnvFl | TOML open file object which mapping search environment variable before parsing with toml package from TOML file format (.toml). | |\n| | TomlFl | TOML open file object that read data context from TOML file format (.toml). | |\n| | PickleFl | Pickle open file object that read data context from Pickle file format (.pickle). | no cover |\n| | MarshalFl | Marshal open file object that read data context from Marshal file format. | no cover |\n| | MsgpackFl | Msgpack open file object that read data context from Msgpack file format. | no cover |\n| stores | Store | Store File Loading Object for get data from configuration and stage. | |\n| | StoreJsonToCsv | Store object that getting the Json context data and save it to stage with CSV file format. | |\n| | StoreToJsonLine | Store object that getting the YAML context data and save it to stage with Json line file format. | |\n| register | Register | Register Object that contain configuration loading methods and metadata management. | |\n| | ArchiveRegister | Archiving Register object that implement archiving management on the Register object such as ``self.purge``, and ``self.remove`` methods. | |\n| utils | rm | Remove a file or dir from an input path. | |\n| | touch | Create an empty file with specific name and modified time of path it an input times was set. | |\n\n## \ud83d\udca1 Usages\n\nI will show some usage example of function in this package. If you want to use\ncomplex or adjust some parameter, please see doc-string or real source code\n(I think it does not complex, and you can see how that function work).\n\n### \u2b55 Files\n\nFor example, I will represent `YamlEnvFl` object that passing environment variable\nto reading content before passing to the Yaml loader.\n\n```yaml\ndata:\n get: HELLO ${HELLO}\n```\n\n```python\nimport os\nfrom ddeutil.io import YamlEnvFl\n\nos.environ[\"HELLO\"] = \"WORLD\"\ncontent = YamlEnvFl('./source.yaml').read(safe=True)\nassert content['data']['get'] == \"HELLO WORLD\"\n```\n\n> [!NOTE]\n> This module do not implement special function on IO like the build-in ``open``\n> function. It also makes standard ``read`` and ``write`` file objects.\n\n### \u274c Dirs\n\n> [!WARNING]\n> This module should not use on the production.\n\n### \u2b55 Store\n\nStore object is the storing dir system handler object that manage any files in\nthat dir path with `get`, `move`, `load`, `save`, or `ls` operations.\n\n```python\nfrom ddeutil.io import Store\n\nstore: Store = Store(path='./conf', compress=\"gzip\")\n\ndata = store.get(name='config_file.yaml')\nstore.save('./stage/file.json', data)\n```\n\n```text\nconf/\n \u251c\u2500 examples/\n \u2502 \u2570\u2500 config_file.yaml\n \u2570\u2500 stage/\n \u2570\u2500 file.json\n```\n\n### \u2b55 Register\n\nThe **Register Object** is the metadata generator object for the config data.\nIf you passing name and configs to this object, it will find the config name\nin any stage storage and generate its metadata to you.\n\n```python\nfrom ddeutil.io.register import Register\nfrom ddeutil.io import Params\n\nregistry: Register = Register(\n name='examples:conn_data_local_file',\n params=Params(**{\n \"stages\": {\n \"raw\": {\"format\": \"{naming:%s}.{timestamp:%Y%m%d_%H%M%S}\"},\n },\n }),\n)\nregistry.move(stage=\"raw\")\n```\n\nThe raw data of this config was written in `conn_file.yaml` file.\n\n```text\nconf/\n \u2570\u2500 examples/\n \u2570\u2500 conn_file.yaml\n```\n\nWhen call `move` method, it will transfer data from `.yaml` file to `json` file\nwith the data hashing algorithm.\n\n```text\ndata/\n \u251c\u2500 __METADATA/\n \u2502 \u251c\u2500 exampleconn_data_local_file.base.json\n \u2502 \u2570\u2500 exampleconn_data_local_file.raw.json\n \u2570\u2500 raw/\n \u2570\u2500 conn_file_20240101_000000.json\n```\n\n## \ud83d\udcac Contribute\n\nI do not think this project will go around the world because it has specific propose,\nand you can create by your coding without this project dependency for long term\nsolution. So, on this time, you can open [the GitHub issue on this project \ud83d\ude4c](https://github.com/ddeutils/ddeutil-io/issues)\nfor fix bug or request new feature if you want it.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Input/Output transport objects",
"version": "0.2.17",
"project_urls": {
"Homepage": "https://github.com/korawica/ddeutil-io/",
"Source Code": "https://github.com/korawica/ddeutil-io/"
},
"split_keywords": [
"io",
" utility"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "bf4f29d4843ffda5126ec2b4bf6568002bdaea44e0176e2bb7fa0b92b0703029",
"md5": "779accb5234425d6c4aec60bc0a6be01",
"sha256": "3b29d701ffcc1e8ebacad789f82f31948be91b617ad9beedc530a9f2ac656757"
},
"downloads": -1,
"filename": "ddeutil_io-0.2.17-py3-none-any.whl",
"has_sig": false,
"md5_digest": "779accb5234425d6c4aec60bc0a6be01",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9.13",
"size": 34802,
"upload_time": "2025-07-16T01:42:34",
"upload_time_iso_8601": "2025-07-16T01:42:34.183079Z",
"url": "https://files.pythonhosted.org/packages/bf/4f/29d4843ffda5126ec2b4bf6568002bdaea44e0176e2bb7fa0b92b0703029/ddeutil_io-0.2.17-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "e62c68bef53ff688e86e02f9ae97127d680a373b2c6cdb04952b3ce9c58c50db",
"md5": "a3292fa2768d037534ace104f5ed18b2",
"sha256": "12992388cf30ab3543fc82a1c0467d741328fa6d64adca73607eca614e4f3f48"
},
"downloads": -1,
"filename": "ddeutil_io-0.2.17.tar.gz",
"has_sig": false,
"md5_digest": "a3292fa2768d037534ace104f5ed18b2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9.13",
"size": 43920,
"upload_time": "2025-07-16T01:42:35",
"upload_time_iso_8601": "2025-07-16T01:42:35.363312Z",
"url": "https://files.pythonhosted.org/packages/e6/2c/68bef53ff688e86e02f9ae97127d680a373b2c6cdb04952b3ce9c58c50db/ddeutil_io-0.2.17.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-16 01:42:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "korawica",
"github_project": "ddeutil-io",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "ddeutil-io"
}
