# Treeverse
Treeverse is a Python tool for traversing, processing, and accumulating data from file trees using custom code.
## Installation
```bash
pip install treeverse
```
## Usage
Treeverse operates in three main steps:
1. **Build**: Traverse the directory and build the file tree
2. **Process**: Apply custom processing to each node
3. **Accumulate**: Aggregate data from child nodes to parent nodes
### 1. Build the tree
```bash
treeverse build -p /path/to/your/project -e py -e yaml \
-c your_module:your_filter_function -o tree.yaml
```
Options:
- `-p, --path`: Specify the path to traverse (default is current directory)
- `-e, --extensions`: Specify file extensions to include (can be used multiple times)
- `-c, --callback`: Path to filter callback functions (can be used multiple times)
- `-o, --output`: Output file (default: tree.yaml)
- `--write-to-stream`: Write output to stdout instead of a file
### 2. Process the tree
```bash
treeverse process -i tree.yaml -c your_module:your_processing_function -o processed_tree.yaml
```
Options:
- `-i, --input`: Input file (default: tree.yaml)
- `-c, --callback`: Path to the processing function
- `-o, --output`: Output file (default: processed_tree.yaml)
- `--read-from-stream`: Read input from stdin instead of a file
- `--write-to-stream`: Write output to stdout instead of a file
### 3. Accumulate results
```bash
treeverse accumulate -i processed_tree.yaml -c your_module:your_accumulation_function -o final_result.yaml
```
Options:
- `-i, --input`: Input file (default: processed_tree.yaml)
- `-c, --callback`: Path to the accumulation function
- `-o, --output`: Output file (default: accumulated_tree.yaml)
- `--read-from-stream`: Read input from stdin instead of a file
- `--write-to-stream`: Write output to stdout instead of a file
## Example Use Case
Process all Python and YAML files in a project, analyze them, and accumulate results:
```bash
treeverse build -p ~/path/to/your/project -e py -e yaml -o tree.yaml
treeverse process -i tree.yaml -c ~/path/to/your/tools.py:analyze_file -o processed_tree.yaml
treeverse accumulate -i processed_tree.yaml -c ~/path/to/your/tools.py:accumulate_results -o final_analysis.yaml
```
To use streaming for the entire pipeline:
```bash
treeverse build -p ~/path/to/your/project -e py -e yaml --write-to-stream | \
treeverse process -c ~/path/to/your/tools.py:analyze_file --read-from-stream --write-to-stream | \
treeverse accumulate -c ~/path/to/your/tools.py:accumulate_results --read-from-stream > final_analysis.yaml
```
In this example, `tools.py` would contain custom `analyze_file` and `accumulate_results` functions.
## Reference Implementations
For examples of callable functions that can be used with Treeverse, check the `simple_callbacks.py` file in the project repository. This file contains reference implementations for filtering, processing, and accumulation functions.
Raw data
{
"_id": null,
"home_page": "https://github.com/ryzhakar/treeverse",
"name": "treeverse",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": "file-tree, traversal, processing, filtering, cli",
"author": "Arthur Ryzhak",
"author_email": "ryzhakar@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/c5/8f/79d9e5c94a68bd02885b64a283150617ff99df3fc6ca6ab5b74d20b1ddb8/treeverse-0.2.0.tar.gz",
"platform": null,
"description": "# Treeverse\n\nTreeverse is a Python tool for traversing, processing, and accumulating data from file trees using custom code.\n\n## Installation\n\n```bash\npip install treeverse\n```\n\n## Usage\n\nTreeverse operates in three main steps:\n\n1. **Build**: Traverse the directory and build the file tree\n2. **Process**: Apply custom processing to each node\n3. **Accumulate**: Aggregate data from child nodes to parent nodes\n\n### 1. Build the tree\n\n```bash\ntreeverse build -p /path/to/your/project -e py -e yaml \\\n -c your_module:your_filter_function -o tree.yaml\n```\n\nOptions:\n- `-p, --path`: Specify the path to traverse (default is current directory)\n- `-e, --extensions`: Specify file extensions to include (can be used multiple times)\n- `-c, --callback`: Path to filter callback functions (can be used multiple times)\n- `-o, --output`: Output file (default: tree.yaml)\n- `--write-to-stream`: Write output to stdout instead of a file\n\n### 2. Process the tree\n\n```bash\ntreeverse process -i tree.yaml -c your_module:your_processing_function -o processed_tree.yaml\n```\n\nOptions:\n- `-i, --input`: Input file (default: tree.yaml)\n- `-c, --callback`: Path to the processing function\n- `-o, --output`: Output file (default: processed_tree.yaml)\n- `--read-from-stream`: Read input from stdin instead of a file\n- `--write-to-stream`: Write output to stdout instead of a file\n\n### 3. Accumulate results\n\n```bash\ntreeverse accumulate -i processed_tree.yaml -c your_module:your_accumulation_function -o final_result.yaml\n```\n\nOptions:\n- `-i, --input`: Input file (default: processed_tree.yaml)\n- `-c, --callback`: Path to the accumulation function\n- `-o, --output`: Output file (default: accumulated_tree.yaml)\n- `--read-from-stream`: Read input from stdin instead of a file\n- `--write-to-stream`: Write output to stdout instead of a file\n\n## Example Use Case\n\nProcess all Python and YAML files in a project, analyze them, and accumulate results:\n\n```bash\ntreeverse build -p ~/path/to/your/project -e py -e yaml -o tree.yaml\ntreeverse process -i tree.yaml -c ~/path/to/your/tools.py:analyze_file -o processed_tree.yaml\ntreeverse accumulate -i processed_tree.yaml -c ~/path/to/your/tools.py:accumulate_results -o final_analysis.yaml\n```\n\nTo use streaming for the entire pipeline:\n\n```bash\ntreeverse build -p ~/path/to/your/project -e py -e yaml --write-to-stream | \\\ntreeverse process -c ~/path/to/your/tools.py:analyze_file --read-from-stream --write-to-stream | \\\ntreeverse accumulate -c ~/path/to/your/tools.py:accumulate_results --read-from-stream > final_analysis.yaml\n```\n\nIn this example, `tools.py` would contain custom `analyze_file` and `accumulate_results` functions.\n\n## Reference Implementations\n\nFor examples of callable functions that can be used with Treeverse, check the `simple_callbacks.py` file in the project repository. This file contains reference implementations for filtering, processing, and accumulation functions.\n",
"bugtrack_url": null,
"license": "AGPL-3.0-or-later",
"summary": "A flexible tool for traversing, filtering, and processing file trees",
"version": "0.2.0",
"project_urls": {
"Documentation": "https://github.com/ryzhakar/treeverse#readme",
"Homepage": "https://github.com/ryzhakar/treeverse",
"Repository": "https://github.com/ryzhakar/treeverse"
},
"split_keywords": [
"file-tree",
" traversal",
" processing",
" filtering",
" cli"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "52e0b7bd55f3e48274177cc15e85331815c327de87c9416c1ca3123d5a9c7ec7",
"md5": "2853e79d8d82994565110b3b8879dc87",
"sha256": "00271fa13953c99ae49240dfe0196238c5c42edd359eaa77c36e519519fd5b9f"
},
"downloads": -1,
"filename": "treeverse-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2853e79d8d82994565110b3b8879dc87",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 19217,
"upload_time": "2024-06-27T15:20:26",
"upload_time_iso_8601": "2024-06-27T15:20:26.269084Z",
"url": "https://files.pythonhosted.org/packages/52/e0/b7bd55f3e48274177cc15e85331815c327de87c9416c1ca3123d5a9c7ec7/treeverse-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "c58f79d9e5c94a68bd02885b64a283150617ff99df3fc6ca6ab5b74d20b1ddb8",
"md5": "bc8ab692cb11473d1cb1f7197f838245",
"sha256": "4984fd070bec065f3ba9f0c434d1e032545cdf13e5f47c91d5f3741b6886b3df"
},
"downloads": -1,
"filename": "treeverse-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "bc8ab692cb11473d1cb1f7197f838245",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 17661,
"upload_time": "2024-06-27T15:20:28",
"upload_time_iso_8601": "2024-06-27T15:20:28.411212Z",
"url": "https://files.pythonhosted.org/packages/c5/8f/79d9e5c94a68bd02885b64a283150617ff99df3fc6ca6ab5b74d20b1ddb8/treeverse-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-27 15:20:28",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ryzhakar",
"github_project": "treeverse",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "treeverse"
}
JSON
