# yaml-extras
Some extra scripting utilities written on top of PyYAML.
See the full docs at [yaml-extras.pages.dev](https://yaml-extras.pages.dev).
## Installation
Install with `pip` or `poetry`:
```bash
pip install yaml-extras
# or
poetry add yaml-extras
```
## Usage
```python
import yaml
from yaml_extras import ExtrasLoader
import json
with open('example.yml') as f:
data = yaml.load(f, Loader=ExtrasLoader)
print(f"data = {json.dumps(data, indent=2)}")
```
## Features
### Modularity with "import"
`!import` tag: Import another whole YAML file. Supports multiple-import using the "<<" merge key, as well as aliasing the result of an import using an anchor.
> **Note:** There is no safeguard against cyclical imports. If you import a file that imports the original file, it will result in exceeding Python's maximum recursion depth.
**Syntax**
```
!import [&anchor ]<filepath>
```
**Examples**
<details><summary>Simple whole-file import</summary>
```yaml
# example.yml
my_children:
child1: !import child1.yml
child2: !import child2.yml
```
```yaml
# child1.yml
name: child1
age: 10
```
```yaml
# child2.yml
name: child2
age: 7
```
Result when loading in Python:
```python
data = {
"my_children": {
"child1": {
"name": "child1",
"age": 10
},
"child2": {
"name": "child2",
"age": 7
}
}
}
```
</details>
<details><summary>Nested whole-file imports</summary>
```yaml
# example.yml
my_children:
child: !import child.yml
```
```yaml
# child.yml
name: child
age: 40
grandchild: !import grandchild.yml
```
```yaml
# grandchild.yml
name: grandchild
age: 10
```
Result when loading in Python:
```python
data = {
"my_children": {
"child": {
"name": "child",
"age": 40,
"grandchild": {
"name": "grandchild",
"age": 10
}
}
}
}
```
</details>
<details><summary>Multiple whole-file imports with merge</summary>
```yaml
# example.yml
all_animals:
<<:
- !import animals/american.yml
- !import animals/eurasian.yml
```
```yaml
# animals/american.yml
bear: wild
wolf: wild
fish: domestic
dog: domestic
```
```yaml
# animals/eurasian.yml
tiger: wild
lion: wild
cat: domestic
chicken: domestic
```
Result when loading in Python:
```python
data = {
"all_animals": {
"bear": "wild",
"wolf": "wild",
"fish": "domestic",
"dog": "domestic",
"tiger": "wild",
"lion": "wild",
"cat": "domestic",
"chicken": "domestic"
}
}
```
</details>
<details><summary>Anchored whole-file imports</summary>
```yaml
# example.yml
child: !import &child-anchor child.yml
again:
child: *child-anchor
```
```yaml
# child.yml
name: child
age: 10
```
Result when loading in Python:
```python
data = {
"child": {
"name": "child",
"age": 10
},
"again": {
"child": {
"name": "child",
"age": 10
}
}
}
```
</details>
---
`!import.anchor` tag: Import a specific anchor from another YAML file. Supports multiple-import using the "<<" merge key, as well as aliasing the result of an import using an anchor.
> **Note:** There is no safeguard against cyclical imports. If you import an anchor that imports the original file (or anchor you define with this import), it will result in exceeding Python's maximum recursion depth.
**Syntax**
```
!import.anchor [&internal_anchor ]<filepath> &<external_anchor>
```
**Examples**
<details><summary>Simple anchor import</summary>
```yaml
# example.yml
my_children:
child1: !import.anchor children.yml &child1
child2: !import.anchor children.yml &child2
```
```yaml
# children.yml
child1: &child1
name: child1
age: 10
child2: &child2
name: child2
age: 7
```
Result when loading in Python:
```python
data = {
"my_children": {
"child1": {
"name": "child1",
"age": 10
},
"child2": {
"name": "child2",
"age": 7
}
}
}
```
</details>
<details><summary>Nested anchor imports</summary>
```yaml
# example.yml
my_children:
child: !import.anchor child.yml &child
```
```yaml
# child.yml
name: child
age: 40
grandchild: !import.anchor grandchild.yml &grandchild
```
```yaml
# grandchild.yml
grandchild: &grandchild
name: grandchild
age: 10
```
Result when loading in Python:
```python
data = {
"my_children": {
"child": {
"name": "child",
"age": 40,
"grandchild": {
"name": "grandchild",
"age": 10
}
}
}
}
```
</details>
<details><summary>Multiple anchor imports with merge</summary>
```yaml
# example.yml
all_animals:
<<:
- !import.anchor animals.yml &american
- !import.anchor animals.yml &eurasian
```
```yaml
# animals.yml
american: &american
bear: wild
wolf: wild
fish: domestic
dog: domestic
eurasian: &eurasian
tiger: wild
lion: wild
cat: domestic
chicken: domestic
```
Result when loading in Python:
```python
data = {
"all_animals": {
"bear": "wild",
"wolf": "wild",
"fish": "domestic",
"dog": "domestic",
"tiger": "wild",
"lion": "wild",
"cat": "domestic",
"chicken": "domestic"
}
}
```
</details>
<details><summary>Anchored anchor imports</summary>
```yaml
# example.yml
child: !import.anchor &my-child child.yml &child-anchor
again:
child: *my-child
```
```yaml
# child.yml
child: &child-anchor
name: child
age: 10
```
Result when loading in Python:
```python
data = {
"child": {
"name": "child",
"age": 10
},
"again": {
"child": {
"name": "child",
"age": 10
}
}
}
```
</details>
---
`!import-all` tag: Import a glob pattern of YAML files as a sequence. Supports merging the imports using the "<<" merge key, as well as aliasing the result of an import using an anchor.
The glob pattern system only supports two types of wildcards: `*` and `**`. `*` matches any character except for `/`, while `**` matches any character including `/`.
> **Note:** There is no safeguard against cyclical imports. If you import a file that imports the original file, it will result in exceeding Python's maximum recursion depth.
**Syntax**
```
!import-all [&anchor ]<glob_pattern>
```
**Examples**
<details><summary>Simple (*) sequence import</summary>
```yaml
# example.yml
all_children: !import-all children/*.yml
```
```yaml
# children/alice.yml
name: alice
age: 10
height: 1.2
```
```yaml
#children/bob.yml
name: bob
age: 7
height: 1.0
```
Result when loading in Python:
```python
data = {
"all_children": [
{
"name": "alice",
"age": 10,
"height": 1.2
},
{
"name": "bob",
"age": 7,
"height": 1.0
}
]
}
```
</details>
<details><summary>Nested (*) sequence imports</summary>
```yaml
# example.yml
all_children: !import-all children/*.yml
```
```yaml
# children/bob.yml
name: bob
age: 28
```
```yaml
# children/alice.yml
name: alice
age: 40
children: !import-all children/alice/*.yml
```
```yaml
# children/alice/fred.yml
name: fred
age: 10
```
```yaml
# children/alice/jane.yml
name: jane
age: 7
```
Result when loading in Python:
```python
data = {
"all_children": [
{
"name": "bob",
"age": 28
},
{
"name": "alice",
"age": 40,
"children": [
{
"name": "fred",
"age": 10
},
{
"name": "jane",
"age": 7
}
]
}
]
}
```
</details>
<details><summary>Multiple (*) sequence imports with merge</summary>
```yaml
# example.yml
all_animals:
<<:
- !import-all animals/american/*.yml
- !import-all animals/eurasian/*.yml
```
```yaml
# animals/american/bear.yml
bear:
type: wild
size: large
```
```yaml
# animals/american/wolf.yml
wolf:
type: wild
size: medium
```
```yaml
# animals/eurasian/tiger.yml
tiger:
type: wild
size: large
```
```yaml
# animals/eurasian/lion.yml
lion:
type: wild
size: large
```
Result when loading in Python:
```python
data = {
"all_animals": {
"bear": {
"type": "wild",
"size": "large"
},
"wolf": {
"type": "wild",
"size": "medium"
},
"tiger": {
"type": "wild",
"size": "large"
},
"lion": {
"type": "wild",
"size": "large"
}
}
}
```
</details>
<details><summary>Anchored (*) sequence imports</summary>
```yaml
# example.yml
data: !import-all &my-children children/*.yml
again:
data: *my-children
```
```yaml
# children/alice.yml
name: alice
age: 10
height: 1.2
```
```yaml
# children/bob.yml
name: bob
age: 7
height: 1.0
```
Result when loading in Python:
```python
data = {
"data": [
{
"name": "alice",
"age": 10,
"height": 1.2
},
{
"name": "bob",
"age": 7,
"height": 1.0
}
],
"again": {
"data": [
{
"name": "alice",
"age": 10,
"height": 1.2
},
{
"name": "bob",
"age": 7,
"height": 1.0
}
]
}
}
```
</details>
<details><summary>Simple (**) sequence import</summary>
```yaml
# example.yml
overarching: !import-all subfolders/**/*.yml
```
```yaml
# subfolders/child1.yml
name: child1
```
```yaml
# subfolders/child2.yml
name: child2
```
```yaml
# subfolders/subfolder1/grandchild1.yml
name: grandchild1
```
Result when loading in Python:
```python
data = {
"overarching": [
{
"name": "child1"
},
{
"name": "child2"
},
{
"name": "grandchild1"
}
]
}
```
</details>
---
`!import-all-parameterized` tag: Import a glob pattern of YAML files as a sequence, enriching the results with one or more metadata keys extracted from globs in the filepath. Supports merging the imports using the "<<" merge key, as well as aliasing the result of an import using an anchor.
The glob pattern system only supports two types of wildcards: `*` and `**`. `*` matches any character except for `/`, while `**` matches any character including `/`. Metadata keys can be extracted from zero or more globs in the path specification with syntax like this:
```yaml
# Import all YAML files in the `path/to/*.yml` glob as a sequence, attaching to each element the
# `basename` key extracted from the filename.
my_data: !import-all-parameterized path/to/{basename:*}.yml
#
# my_data:
# - basename: file1
# key1: value1
# - basename: file2
# key2: value2
# key3: value3
#
# Import all YAML files in the `path/to/**/meta.yml` glob as a sequence, attaching to each
# element the `subdirs` key extracted from the subdirectory structure.
my_subdirs: !import-all-parameterized path/to/{subdirs:**}/meta.yml
#
# my_subdirs:
# - subdirs: subdir1
# key1: value1
# - subdirs: subdir1/subdir2/subdir3
# key2: value2
# key3: value3
#
```
> **Note (i):** There is no safeguard against cyclical imports. If you import a file that imports the original file, it will result in exceeding Python's maximum recursion depth.
>
> **Note (ii):** When the leaf files of an import contain mappings, then it is simple to "merge" the metadata keys from the path into the resulting imported mappings. However, when the leaf files are scalars or sequences, then the structure of the import results are slightly more contrived. The contents of the imports will be under a `content` key in each result, with the metadata keys extracted from the path added as additional key/value pairs in the mappings.
**Syntax**
```
!import-all-parameterized [&anchor ]<glob_pattern>
```
**Examples**
<details>
<summary>Simple parameterized import (*) with metadata</summary>
```yaml
# example.yml
grade_book: !import-all schools/{school_name:*}/grades/{student_name:*}.yml
```
```yaml
# schools/elementary/grades/David.yml
math: 95
science: 90
english: 80
```
```yaml
# schools/elementary/grades/Edward.yml
math: 100
science: 90
english: 100
```
```yaml
# schools/highschool/grades/Frank.yml
math: 85
science: 95
english: 90
```
Result when loading in Python:
```python
data = {
"grade_book": [
{
"school_name": "elementary",
"student_name": "David",
"math": 95,
"science": 90,
"english": 80
},
{
"school_name": "elementary",
"student_name": "Edward",
"math": 100,
"science": 90,
"english": 100
},
{
"school_name": "highschool",
"student_name": "Frank",
"math": 85,
"science": 95,
"english": 90
}
]
}
```
</details>
<details>
<summary>Simple parameterized import (**) with metadata</summary>
```yaml
# example.yml
translations: !import-all-parameterized words/{langspec:**}/words.yml
```
```yaml
# words/en/us/words.yml
- hello
- goodbye
- color
- thanks
```
```yaml
# words/en/uk/words.yml
- good morrow
- toodle-oo
- colour
- cheers
```
```yaml
# words/es/mx/words.yml
- hola
- adios
- color
- gracias
```
Result when loading in Python:
```python
data = {
"translations": [
{
"langspec": "en/us",
"content": ["hello", "goodbye", "color", "thanks"]
},
{
"langspec": "en/uk",
"content": ["good morrow", "toodle-oo", "colour", "cheers"]
},
{
"langspec": "es/mx",
"content": ["hola", "adios", "color", "gracias"]
}
]
}
```
</details>
#### Customizing the import directory
By default, `!import` tags will search relative to the current working directory of the Python process. You can customize the base directory for imports by calling `yaml_import.set_import_relative_dir(...)` with the desired base directory.
```python
import yaml
from yaml_extras import ExtrasLoader, yaml_import
yaml_import.set_import_relative_dir('/path/to/imports')
data = yaml.load('!import somefile.yml', Loader=ExtrasLoader)
```
## Roadmap
### P1
- [x] Add support for `!import` to import other whole documents into a YAML document (general import).
- [x] Add support for `!import.anchor` to import specific anchors from other YAML documents (targeted import).
- [x] Add support for `!import-all` to import a glob pattern of YAML files as a sequence.
- [x] Add support for `!import-all.anchor` to import a specific anchor from a glob pattern of YAML files as a sequence.
- [x] Add support for `!import-all-parameterized` to import a glob pattern of YAML files as a sequence with some data extracted from the filepath.
- [ ] Add support for `!import-all-parameterized.anchor` to import a specific anchor from a glob pattern of YAML files as a sequence with some data extracted from the filepath.
- [x] Allow user to set relative import directory.
### P2
- [ ] Implement type specification system to validate YAML files against a schema using a `!yamlschema` tag system which mimics JSON Schema semantics and are validated upon construction.
- [ ] Add support for `!env` tag to import environment variables.
### P3
- [ ] VSCode / Intellisense plugin to navigate through imports using cmd + click
## Acknowledgements
- David Sillman <dsillman2000@gmail.com>
- [pyyaml-include](https://github.com/tanbro/pyyaml-include) author, [@tanbro](https://github.com/tanbro).
- [PyYAML](https://github.com/yaml/pyyaml)
Raw data
{
"_id": null,
"home_page": null,
"name": "yaml-extras",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": "yaml, yml, pyyaml, extras, yaml-extras",
"author": "David Sillman",
"author_email": "dsillman2000@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/76/ad/0d00a5d6bb79d8995ce6e88331fe7cbc36926088d68b2cbd50fb3ebe5610/yaml_extras-0.2.1.tar.gz",
"platform": null,
"description": "# yaml-extras\n\nSome extra scripting utilities written on top of PyYAML.\n\nSee the full docs at [yaml-extras.pages.dev](https://yaml-extras.pages.dev).\n\n## Installation\n\nInstall with `pip` or `poetry`:\n \n```bash\npip install yaml-extras\n# or\npoetry add yaml-extras\n```\n\n## Usage\n\n```python\nimport yaml\nfrom yaml_extras import ExtrasLoader\nimport json\n\nwith open('example.yml') as f:\n data = yaml.load(f, Loader=ExtrasLoader)\n\nprint(f\"data = {json.dumps(data, indent=2)}\")\n```\n\n## Features\n\n### Modularity with \"import\"\n\n`!import` tag: Import another whole YAML file. Supports multiple-import using the \"<<\" merge key, as well as aliasing the result of an import using an anchor.\n\n> **Note:** There is no safeguard against cyclical imports. If you import a file that imports the original file, it will result in exceeding Python's maximum recursion depth.\n\n**Syntax**\n\n```\n!import [&anchor ]<filepath>\n```\n\n**Examples**\n<details><summary>Simple whole-file import</summary>\n\n```yaml\n# example.yml\nmy_children:\n child1: !import child1.yml\n child2: !import child2.yml\n```\n\n```yaml\n# child1.yml\nname: child1\nage: 10\n```\n\n```yaml\n# child2.yml\nname: child2\nage: 7\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"my_children\": {\n \"child1\": {\n \"name\": \"child1\",\n \"age\": 10\n },\n \"child2\": {\n \"name\": \"child2\",\n \"age\": 7\n }\n }\n}\n```\n</details>\n\n<details><summary>Nested whole-file imports</summary>\n\n```yaml\n# example.yml\nmy_children:\n child: !import child.yml\n```\n\n```yaml\n# child.yml\nname: child\nage: 40\ngrandchild: !import grandchild.yml\n```\n\n```yaml\n# grandchild.yml\nname: grandchild\nage: 10\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"my_children\": {\n \"child\": {\n \"name\": \"child\",\n \"age\": 40,\n \"grandchild\": {\n \"name\": \"grandchild\",\n \"age\": 10\n }\n }\n }\n}\n```\n</details>\n\n<details><summary>Multiple whole-file imports with merge</summary>\n\n```yaml\n# example.yml\nall_animals:\n <<: \n - !import animals/american.yml\n - !import animals/eurasian.yml\n```\n \n```yaml\n# animals/american.yml\nbear: wild\nwolf: wild\nfish: domestic\ndog: domestic\n```\n\n```yaml\n# animals/eurasian.yml\ntiger: wild\nlion: wild\ncat: domestic\nchicken: domestic\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"all_animals\": {\n \"bear\": \"wild\",\n \"wolf\": \"wild\",\n \"fish\": \"domestic\",\n \"dog\": \"domestic\",\n \"tiger\": \"wild\",\n \"lion\": \"wild\",\n \"cat\": \"domestic\",\n \"chicken\": \"domestic\"\n }\n}\n```\n</details>\n\n<details><summary>Anchored whole-file imports</summary>\n\n```yaml\n# example.yml\nchild: !import &child-anchor child.yml\nagain:\n child: *child-anchor\n```\n\n```yaml\n# child.yml\nname: child\nage: 10\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"child\": {\n \"name\": \"child\",\n \"age\": 10\n },\n \"again\": {\n \"child\": {\n \"name\": \"child\",\n \"age\": 10\n }\n }\n}\n```\n</details>\n\n---\n\n`!import.anchor` tag: Import a specific anchor from another YAML file. Supports multiple-import using the \"<<\" merge key, as well as aliasing the result of an import using an anchor.\n\n> **Note:** There is no safeguard against cyclical imports. If you import an anchor that imports the original file (or anchor you define with this import), it will result in exceeding Python's maximum recursion depth.\n\n**Syntax**\n\n```\n!import.anchor [&internal_anchor ]<filepath> &<external_anchor>\n```\n\n**Examples**\n\n<details><summary>Simple anchor import</summary>\n\n```yaml\n# example.yml\nmy_children:\n child1: !import.anchor children.yml &child1\n child2: !import.anchor children.yml &child2\n```\n\n```yaml\n# children.yml\nchild1: &child1\n name: child1\n age: 10\nchild2: &child2\n name: child2\n age: 7\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"my_children\": {\n \"child1\": {\n \"name\": \"child1\",\n \"age\": 10\n },\n \"child2\": {\n \"name\": \"child2\",\n \"age\": 7\n }\n }\n}\n```\n</details>\n\n<details><summary>Nested anchor imports</summary>\n\n```yaml\n# example.yml\nmy_children:\n child: !import.anchor child.yml &child\n```\n\n```yaml\n# child.yml\nname: child\nage: 40\ngrandchild: !import.anchor grandchild.yml &grandchild\n```\n\n```yaml\n# grandchild.yml\ngrandchild: &grandchild\n name: grandchild\n age: 10\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"my_children\": {\n \"child\": {\n \"name\": \"child\",\n \"age\": 40,\n \"grandchild\": {\n \"name\": \"grandchild\",\n \"age\": 10\n }\n }\n }\n}\n```\n</details>\n\n<details><summary>Multiple anchor imports with merge</summary>\n\n```yaml\n# example.yml\nall_animals:\n <<: \n - !import.anchor animals.yml &american\n - !import.anchor animals.yml &eurasian\n```\n\n```yaml\n# animals.yml\namerican: &american\n bear: wild\n wolf: wild\n fish: domestic\n dog: domestic\n\neurasian: &eurasian\n tiger: wild\n lion: wild\n cat: domestic\n chicken: domestic\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"all_animals\": {\n \"bear\": \"wild\",\n \"wolf\": \"wild\",\n \"fish\": \"domestic\",\n \"dog\": \"domestic\",\n \"tiger\": \"wild\",\n \"lion\": \"wild\",\n \"cat\": \"domestic\",\n \"chicken\": \"domestic\"\n }\n}\n```\n</details>\n\n<details><summary>Anchored anchor imports</summary>\n\n```yaml\n# example.yml\nchild: !import.anchor &my-child child.yml &child-anchor\nagain:\n child: *my-child\n```\n\n```yaml\n# child.yml\nchild: &child-anchor\n name: child\n age: 10\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"child\": {\n \"name\": \"child\",\n \"age\": 10\n },\n \"again\": {\n \"child\": {\n \"name\": \"child\",\n \"age\": 10\n }\n }\n}\n```\n</details>\n\n---\n\n`!import-all` tag: Import a glob pattern of YAML files as a sequence. Supports merging the imports using the \"<<\" merge key, as well as aliasing the result of an import using an anchor.\n\nThe glob pattern system only supports two types of wildcards: `*` and `**`. `*` matches any character except for `/`, while `**` matches any character including `/`.\n\n> **Note:** There is no safeguard against cyclical imports. If you import a file that imports the original file, it will result in exceeding Python's maximum recursion depth.\n\n**Syntax**\n\n```\n!import-all [&anchor ]<glob_pattern>\n```\n\n**Examples**\n\n<details><summary>Simple (*) sequence import</summary>\n\n```yaml\n# example.yml\nall_children: !import-all children/*.yml\n```\n\n```yaml\n# children/alice.yml\nname: alice\nage: 10\nheight: 1.2\n```\n\n```yaml\n#children/bob.yml\nname: bob\nage: 7\nheight: 1.0\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"all_children\": [\n {\n \"name\": \"alice\",\n \"age\": 10,\n \"height\": 1.2\n },\n {\n \"name\": \"bob\",\n \"age\": 7,\n \"height\": 1.0\n }\n ]\n}\n```\n</details>\n\n<details><summary>Nested (*) sequence imports</summary>\n\n```yaml\n# example.yml\nall_children: !import-all children/*.yml\n```\n\n```yaml\n# children/bob.yml\nname: bob\nage: 28\n```\n\n```yaml\n# children/alice.yml\nname: alice\nage: 40\nchildren: !import-all children/alice/*.yml\n```\n\n```yaml\n# children/alice/fred.yml\nname: fred\nage: 10\n```\n\n```yaml\n# children/alice/jane.yml\nname: jane\nage: 7\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"all_children\": [\n {\n \"name\": \"bob\",\n \"age\": 28\n },\n {\n \"name\": \"alice\",\n \"age\": 40,\n \"children\": [\n {\n \"name\": \"fred\",\n \"age\": 10\n },\n {\n \"name\": \"jane\",\n \"age\": 7\n }\n ]\n }\n ]\n}\n```\n\n</details>\n\n<details><summary>Multiple (*) sequence imports with merge</summary>\n\n```yaml\n# example.yml\nall_animals:\n <<: \n - !import-all animals/american/*.yml\n - !import-all animals/eurasian/*.yml\n```\n\n```yaml\n# animals/american/bear.yml\nbear:\n type: wild\n size: large\n```\n \n```yaml\n# animals/american/wolf.yml\nwolf:\n type: wild\n size: medium\n```\n\n```yaml\n# animals/eurasian/tiger.yml\ntiger:\n type: wild\n size: large\n```\n\n```yaml\n# animals/eurasian/lion.yml\nlion:\n type: wild\n size: large\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"all_animals\": {\n \"bear\": {\n \"type\": \"wild\",\n \"size\": \"large\"\n },\n \"wolf\": {\n \"type\": \"wild\",\n \"size\": \"medium\"\n },\n \"tiger\": {\n \"type\": \"wild\",\n \"size\": \"large\"\n },\n \"lion\": {\n \"type\": \"wild\",\n \"size\": \"large\"\n }\n }\n}\n```\n</details>\n\n<details><summary>Anchored (*) sequence imports</summary>\n\n```yaml\n# example.yml\ndata: !import-all &my-children children/*.yml\nagain:\n data: *my-children\n```\n\n```yaml\n# children/alice.yml\nname: alice\nage: 10\nheight: 1.2\n```\n\n```yaml\n# children/bob.yml\nname: bob\nage: 7\nheight: 1.0\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"data\": [\n {\n \"name\": \"alice\",\n \"age\": 10,\n \"height\": 1.2\n },\n {\n \"name\": \"bob\",\n \"age\": 7,\n \"height\": 1.0\n }\n ],\n \"again\": {\n \"data\": [\n {\n \"name\": \"alice\",\n \"age\": 10,\n \"height\": 1.2\n },\n {\n \"name\": \"bob\",\n \"age\": 7,\n \"height\": 1.0\n }\n ]\n }\n}\n```\n</details>\n\n<details><summary>Simple (**) sequence import</summary>\n\n```yaml\n# example.yml\noverarching: !import-all subfolders/**/*.yml\n```\n\n```yaml\n# subfolders/child1.yml\nname: child1\n```\n\n```yaml\n# subfolders/child2.yml\nname: child2\n```\n\n```yaml\n# subfolders/subfolder1/grandchild1.yml\nname: grandchild1\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"overarching\": [\n {\n \"name\": \"child1\"\n },\n {\n \"name\": \"child2\"\n },\n {\n \"name\": \"grandchild1\"\n }\n ]\n}\n```\n\n</details>\n\n---\n\n`!import-all-parameterized` tag: Import a glob pattern of YAML files as a sequence, enriching the results with one or more metadata keys extracted from globs in the filepath. Supports merging the imports using the \"<<\" merge key, as well as aliasing the result of an import using an anchor.\n\nThe glob pattern system only supports two types of wildcards: `*` and `**`. `*` matches any character except for `/`, while `**` matches any character including `/`. Metadata keys can be extracted from zero or more globs in the path specification with syntax like this:\n\n```yaml\n# Import all YAML files in the `path/to/*.yml` glob as a sequence, attaching to each element the\n# `basename` key extracted from the filename.\nmy_data: !import-all-parameterized path/to/{basename:*}.yml\n#\n# my_data:\n# - basename: file1\n# key1: value1\n# - basename: file2\n# key2: value2\n# key3: value3\n# \n\n# Import all YAML files in the `path/to/**/meta.yml` glob as a sequence, attaching to each\n# element the `subdirs` key extracted from the subdirectory structure.\nmy_subdirs: !import-all-parameterized path/to/{subdirs:**}/meta.yml\n#\n# my_subdirs:\n# - subdirs: subdir1\n# key1: value1\n# - subdirs: subdir1/subdir2/subdir3\n# key2: value2\n# key3: value3\n#\n```\n\n> **Note (i):** There is no safeguard against cyclical imports. If you import a file that imports the original file, it will result in exceeding Python's maximum recursion depth.\n>\n> **Note (ii):** When the leaf files of an import contain mappings, then it is simple to \"merge\" the metadata keys from the path into the resulting imported mappings. However, when the leaf files are scalars or sequences, then the structure of the import results are slightly more contrived. The contents of the imports will be under a `content` key in each result, with the metadata keys extracted from the path added as additional key/value pairs in the mappings.\n\n**Syntax**\n\n```\n!import-all-parameterized [&anchor ]<glob_pattern>\n```\n\n**Examples**\n\n<details>\n<summary>Simple parameterized import (*) with metadata</summary>\n\n```yaml\n# example.yml\ngrade_book: !import-all schools/{school_name:*}/grades/{student_name:*}.yml\n```\n\n```yaml\n# schools/elementary/grades/David.yml\nmath: 95\nscience: 90\nenglish: 80\n```\n\n```yaml\n# schools/elementary/grades/Edward.yml\nmath: 100\nscience: 90\nenglish: 100\n```\n\n```yaml\n# schools/highschool/grades/Frank.yml\nmath: 85\nscience: 95\nenglish: 90\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"grade_book\": [\n {\n \"school_name\": \"elementary\",\n \"student_name\": \"David\",\n \"math\": 95,\n \"science\": 90,\n \"english\": 80\n },\n {\n \"school_name\": \"elementary\",\n \"student_name\": \"Edward\",\n \"math\": 100,\n \"science\": 90,\n \"english\": 100\n },\n {\n \"school_name\": \"highschool\",\n \"student_name\": \"Frank\",\n \"math\": 85,\n \"science\": 95,\n \"english\": 90\n }\n ]\n}\n```\n\n</details>\n<details>\n<summary>Simple parameterized import (**) with metadata</summary>\n\n```yaml\n# example.yml\ntranslations: !import-all-parameterized words/{langspec:**}/words.yml\n```\n\n```yaml\n# words/en/us/words.yml\n- hello\n- goodbye\n- color\n- thanks\n```\n\n```yaml\n# words/en/uk/words.yml\n- good morrow\n- toodle-oo\n- colour\n- cheers\n```\n\n```yaml\n# words/es/mx/words.yml\n- hola\n- adios\n- color\n- gracias\n```\n\nResult when loading in Python:\n\n```python\ndata = {\n \"translations\": [\n {\n \"langspec\": \"en/us\",\n \"content\": [\"hello\", \"goodbye\", \"color\", \"thanks\"]\n },\n {\n \"langspec\": \"en/uk\",\n \"content\": [\"good morrow\", \"toodle-oo\", \"colour\", \"cheers\"]\n },\n {\n \"langspec\": \"es/mx\",\n \"content\": [\"hola\", \"adios\", \"color\", \"gracias\"]\n }\n ]\n}\n```\n\n</details>\n\n#### Customizing the import directory\n\nBy default, `!import` tags will search relative to the current working directory of the Python process. You can customize the base directory for imports by calling `yaml_import.set_import_relative_dir(...)` with the desired base directory.\n\n```python\nimport yaml\nfrom yaml_extras import ExtrasLoader, yaml_import\n\nyaml_import.set_import_relative_dir('/path/to/imports')\ndata = yaml.load('!import somefile.yml', Loader=ExtrasLoader)\n```\n\n## Roadmap\n\n### P1\n- [x] Add support for `!import` to import other whole documents into a YAML document (general import).\n- [x] Add support for `!import.anchor` to import specific anchors from other YAML documents (targeted import).\n- [x] Add support for `!import-all` to import a glob pattern of YAML files as a sequence.\n- [x] Add support for `!import-all.anchor` to import a specific anchor from a glob pattern of YAML files as a sequence.\n- [x] Add support for `!import-all-parameterized` to import a glob pattern of YAML files as a sequence with some data extracted from the filepath.\n- [ ] Add support for `!import-all-parameterized.anchor` to import a specific anchor from a glob pattern of YAML files as a sequence with some data extracted from the filepath.\n- [x] Allow user to set relative import directory.\n\n### P2\n- [ ] Implement type specification system to validate YAML files against a schema using a `!yamlschema` tag system which mimics JSON Schema semantics and are validated upon construction.\n- [ ] Add support for `!env` tag to import environment variables.\n\n### P3\n- [ ] VSCode / Intellisense plugin to navigate through imports using cmd + click\n\n## Acknowledgements\n\n- David Sillman <dsillman2000@gmail.com>\n- [pyyaml-include](https://github.com/tanbro/pyyaml-include) author, [@tanbro](https://github.com/tanbro).\n- [PyYAML](https://github.com/yaml/pyyaml)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A collection of YAML utilities",
"version": "0.2.1",
"project_urls": null,
"split_keywords": [
"yaml",
" yml",
" pyyaml",
" extras",
" yaml-extras"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0afc168bcc9827dc4ce0bed39b079979fcd519f094714bcc6ead284ff2d6a177",
"md5": "8343177296fbc429a40ef272200e889b",
"sha256": "80ebb985a30d2804e6f720285f40f557e478beb36a118b173d044f92870f15d6"
},
"downloads": -1,
"filename": "yaml_extras-0.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8343177296fbc429a40ef272200e889b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 12551,
"upload_time": "2024-12-30T19:12:14",
"upload_time_iso_8601": "2024-12-30T19:12:14.399440Z",
"url": "https://files.pythonhosted.org/packages/0a/fc/168bcc9827dc4ce0bed39b079979fcd519f094714bcc6ead284ff2d6a177/yaml_extras-0.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "76ad0d00a5d6bb79d8995ce6e88331fe7cbc36926088d68b2cbd50fb3ebe5610",
"md5": "c8c98fe46a72c6ca032de1aa9b275033",
"sha256": "e24c535b7a83b9dd0227e765dda37d9128d6e1f7445c47207e24f5acb8fb856f"
},
"downloads": -1,
"filename": "yaml_extras-0.2.1.tar.gz",
"has_sig": false,
"md5_digest": "c8c98fe46a72c6ca032de1aa9b275033",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 14487,
"upload_time": "2024-12-30T19:12:17",
"upload_time_iso_8601": "2024-12-30T19:12:17.097945Z",
"url": "https://files.pythonhosted.org/packages/76/ad/0d00a5d6bb79d8995ce6e88331fe7cbc36926088d68b2cbd50fb3ebe5610/yaml_extras-0.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-30 19:12:17",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "yaml-extras"
}
JSON