| Name | arcx JSON |
| Version |
0.2.1
JSON |
| download |
| home_page | None |
| Summary | Archive (safe)-Removal Rolling Toolbox |
| upload_time | 2024-08-14 14:06:08 |
| maintainer | None |
| docs_url | None |
| author | Antonio Mariani |
| requires_python | <4.0,>=3.10 |
| license | None |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# arcX - Archive, Rm & Clean toolboX
ArcX is a versatile toolbox designed to streamline data management by automating key tasks. Whether you're handling
experiment outputs or maintaining clean directories, ArcX simplifies the process with a range of powerful tools.
* **Archiving**: Seamlessly archive Oceanic experiment outputs using a flexible YAML configuration file. ArcX takes care
of organizing and storing your data without manual intervention.
* **Safe File Removal**: Efficiently remove files that already have a local copy. ArcX ensures that only unnecessary
files are deleted, safeguarding important data.
* **Comprehensive Directory Cleaning**: Clean multiple directories with a single command. ArcX offers various cleaning
options and leverages a YAML configuration file to specify exactly what to delete and how to do it.
## Installation
### Via pip
```shell
pip install arcx
```
### Via conda/mamba
```shell
mamba install arcx
```
## Usage
```shell
Usage: arcx [OPTIONS] COMMAND [ARGS]...
Options:
--version Show the version and exit.
--help Show this message and exit.
Commands:
rolling
saferm
update-md5
```
## Rolling
To keep a directory under rolling, use the command:
```shell
Usage: cli.py rolling [OPTIONS]
Options:
-c, --config PATH [required]
--dry-run Dry run mode.
-d, --debug Enable debug mode.
--help Show this message and exit.
```
### Rolling Configuration File
Here a template of a rolling configuration file:
```yaml
- !CleanPath
path: $PATH_UNDER_ROLLING
fmt: ???????? # YYYYMMDD
safe:
to_keep: X
reference_paths:
- <REF1>
- <REF2>
conditional:
to_keep: Y
expected_files:
- file.exe
- tmp.nc
force:
to_keep: Z
```
let's comment each section.
<details>
<summary>
<b>CleanPath object</b>
</summary>
With this, we start to declare a list of !CleanPath objects with two mandatory attributes:
```yaml
- !CleanPath
path: $PATH_UNDER_ROLLING
fmt: ???????? # YYYYMMDD
```
* `path`: it's the path to keep under rolling
* `fmt`: it's a string bash that represent the format of file/dir to delete. It can contains jolly characters `?` or `*`
</details>
<details>
<summary>
<b>Safe</b>
</summary>
```yaml
safe:
to_keep: X
reference_paths:
- <REF1>
- <REF2>
```
* `safe`: means for safe rolling, which means **delete a file only if an identical local copy already exists**
* `to_keep`: how much dir/file to not include in the rolling
* `reference_paths`: a list of path where to find if a local copy already exists
To consider that the safe mode doesn't remove the dir under rolling.
</details>
<details>
<summary>
<b>Conditional</b>
</summary>
#### Conditional
```yaml
conditional:
to_keep: Y
expected_files:
- file.exe
- tmp.nc
```
* `conditional`: specify to remove a dir if some conditions are meet
* `to_keep`: how much dir/file to not include in the rolling
* `expected_files`: specify the exact list of files expected to find in rolling path to trigger the rm operations. The
filename can contains jolly character `?` and `*`
</details>
<details>
<summary>
<b>Force</b>
</summary>
```yaml
force: # optional
to_keep: Z
```
* `force`: Enable path rm without any check
* `to_keep`: how much dir/file to not include in the rolling
</details>
---
## Update md5 dir
A requirement to safe clean, is to compute in advance the md5 of all files in the rolling path using the command:
```shell
update-md5 [OPTIONS]
Options:
-p, --path PATH [required]
-d, --debug Enable debug mode.
--help Show this message and exit.
```
The command will create in the directory a file called `.dir_md5.txt` with the following structure:
```txt
md5hash filename1
md5hash filename2
```
---
## Safe Clean
Safe Rm command is the equivalent of safe rm rolling section: it removes files from **clean path** only if exists an
identical copy in **keep path**
```shell
Usage: cli.py saferm [OPTIONS]
Options:
--keep PATH [required]
--clean PATH [required]
-f, --force Delete files without confirm request
-d, --dry-run Disable file removal
-d, --debug Enable debug mode.
--help Show this message and exit.
```
Raw data
{
"_id": null,
"home_page": null,
"name": "arcx",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": null,
"author": "Antonio Mariani",
"author_email": "antonio.mariani@cmcc.it",
"download_url": "https://files.pythonhosted.org/packages/ca/11/bb97e18ee8dc2b06fede6a51fffa13d36da2df76022bbcf3a7b6fe372a60/arcx-0.2.1.tar.gz",
"platform": null,
"description": "# arcX - Archive, Rm & Clean toolboX\n\nArcX is a versatile toolbox designed to streamline data management by automating key tasks. Whether you're handling\nexperiment outputs or maintaining clean directories, ArcX simplifies the process with a range of powerful tools.\n\n* **Archiving**: Seamlessly archive Oceanic experiment outputs using a flexible YAML configuration file. ArcX takes care\n of organizing and storing your data without manual intervention.\n* **Safe File Removal**: Efficiently remove files that already have a local copy. ArcX ensures that only unnecessary\n files are deleted, safeguarding important data.\n* **Comprehensive Directory Cleaning**: Clean multiple directories with a single command. ArcX offers various cleaning\n options and leverages a YAML configuration file to specify exactly what to delete and how to do it.\n\n## Installation\n\n### Via pip\n\n```shell\npip install arcx\n```\n\n### Via conda/mamba\n\n```shell\nmamba install arcx\n```\n\n## Usage\n\n```shell\nUsage: arcx [OPTIONS] COMMAND [ARGS]...\n\nOptions:\n --version Show the version and exit.\n --help Show this message and exit.\n\nCommands:\n rolling\n saferm\n update-md5\n```\n\n## Rolling\n\nTo keep a directory under rolling, use the command:\n\n```shell\nUsage: cli.py rolling [OPTIONS]\n\nOptions:\n -c, --config PATH [required]\n --dry-run Dry run mode.\n -d, --debug Enable debug mode.\n --help Show this message and exit.\n```\n\n### Rolling Configuration File\n\nHere a template of a rolling configuration file:\n\n```yaml\n- !CleanPath\n path: $PATH_UNDER_ROLLING\n fmt: ???????? # YYYYMMDD\n safe:\n to_keep: X\n reference_paths:\n - <REF1>\n - <REF2>\n conditional:\n to_keep: Y\n expected_files:\n - file.exe\n - tmp.nc\n force:\n to_keep: Z\n```\n\nlet's comment each section.\n\n<details>\n <summary>\n <b>CleanPath object</b>\n </summary>\n\nWith this, we start to declare a list of !CleanPath objects with two mandatory attributes:\n\n```yaml\n- !CleanPath\n path: $PATH_UNDER_ROLLING\n fmt: ???????? # YYYYMMDD\n```\n\n* `path`: it's the path to keep under rolling\n* `fmt`: it's a string bash that represent the format of file/dir to delete. It can contains jolly characters `?` or `*`\n\n</details>\n\n<details>\n <summary>\n <b>Safe</b>\n </summary>\n\n```yaml\n safe:\n to_keep: X\n reference_paths:\n - <REF1>\n - <REF2>\n```\n\n* `safe`: means for safe rolling, which means **delete a file only if an identical local copy already exists**\n* `to_keep`: how much dir/file to not include in the rolling\n* `reference_paths`: a list of path where to find if a local copy already exists\n\nTo consider that the safe mode doesn't remove the dir under rolling.\n</details>\n\n<details>\n <summary>\n <b>Conditional</b>\n </summary>\n#### Conditional\n\n```yaml\n conditional:\n to_keep: Y\n expected_files:\n - file.exe\n - tmp.nc\n```\n\n* `conditional`: specify to remove a dir if some conditions are meet\n* `to_keep`: how much dir/file to not include in the rolling\n* `expected_files`: specify the exact list of files expected to find in rolling path to trigger the rm operations. The\n filename can contains jolly character `?` and `*`\n\n</details>\n\n<details>\n <summary>\n <b>Force</b>\n </summary>\n\n```yaml\n force: # optional\n to_keep: Z\n```\n\n* `force`: Enable path rm without any check\n* `to_keep`: how much dir/file to not include in the rolling\n\n</details>\n\n---\n\n## Update md5 dir\n\nA requirement to safe clean, is to compute in advance the md5 of all files in the rolling path using the command:\n\n```shell\nupdate-md5 [OPTIONS]\n\nOptions:\n -p, --path PATH [required]\n -d, --debug Enable debug mode.\n --help Show this message and exit.\n```\n\nThe command will create in the directory a file called `.dir_md5.txt` with the following structure:\n\n```txt\nmd5hash filename1\nmd5hash filename2\n```\n\n---\n\n## Safe Clean\n\nSafe Rm command is the equivalent of safe rm rolling section: it removes files from **clean path** only if exists an\nidentical copy in **keep path**\n\n```shell\nUsage: cli.py saferm [OPTIONS]\n\nOptions:\n --keep PATH [required]\n --clean PATH [required]\n -f, --force Delete files without confirm request\n -d, --dry-run Disable file removal\n -d, --debug Enable debug mode.\n --help Show this message and exit.\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Archive (safe)-Removal Rolling Toolbox",
"version": "0.2.1",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cd0b74479040763b7bde0e0f5da5ca940fd8649f6d9cf8e08f7d661ca9b6f522",
"md5": "86b66f3ddecd598dc8102aa00916e620",
"sha256": "ec577296fb34478c2e1105491d09c75541430e4ed3712576e98a34950f0c0aaf"
},
"downloads": -1,
"filename": "arcx-0.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "86b66f3ddecd598dc8102aa00916e620",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 18208,
"upload_time": "2024-08-14T14:06:06",
"upload_time_iso_8601": "2024-08-14T14:06:06.795017Z",
"url": "https://files.pythonhosted.org/packages/cd/0b/74479040763b7bde0e0f5da5ca940fd8649f6d9cf8e08f7d661ca9b6f522/arcx-0.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ca11bb97e18ee8dc2b06fede6a51fffa13d36da2df76022bbcf3a7b6fe372a60",
"md5": "f540005f9cc624a61ba2d89471a20ba5",
"sha256": "7ffa32cf5f7ec50b61785ab0197ebc57151c335a49377c7469ed861e1e457b41"
},
"downloads": -1,
"filename": "arcx-0.2.1.tar.gz",
"has_sig": false,
"md5_digest": "f540005f9cc624a61ba2d89471a20ba5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 15157,
"upload_time": "2024-08-14T14:06:08",
"upload_time_iso_8601": "2024-08-14T14:06:08.351803Z",
"url": "https://files.pythonhosted.org/packages/ca/11/bb97e18ee8dc2b06fede6a51fffa13d36da2df76022bbcf3a7b6fe372a60/arcx-0.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-14 14:06:08",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "arcx"
}
