# YAML Path and Command-Line Tools
[](https://github.com/wwkimball/yamlpath/actions/workflows/build.yml)
[](https://pypi.org/project/yamlpath/)
[](https://pypi.org/project/yamlpath/)
[](https://pepy.tech/project/yamlpath)
[](https://coveralls.io/github/wwkimball/yamlpath?branch=master)
[](https://www.codacy.com/gh/wwkimball/yamlpath/dashboard?utm_source=github.com&utm_medium=referral&utm_content=wwkimball/yamlpath&utm_campaign=Badge_Grade)
Along with providing a
[standard for defining YAML Paths](https://github.com/wwkimball/yamlpath/wiki/Segments-of-a-YAML-Path),
this project aims to provide
[generally-useful command-line tools](https://github.com/wwkimball/yamlpath/wiki/Command-Line-(CLI)-Tools)
which implement YAML Paths. These bring intuitive YAML, EYAML, JSON, and
compatible data parsing and editing capabilties to the command-line. It is
also a
[Python library](https://github.com/wwkimball/yamlpath/wiki/Python-Library)
for other projects to readily employ YAML Paths.
## Contents
1. [Introduction](#introduction)
2. [Illustration](#illustration)
3. [Supported YAML Path Segments](#supported-yaml-path-segments)
4. [Installing](#installing)
1. [Requirements](#requirements)
2. [Using pip](#using-pip)
1. [Very Old Versions of pip or its setuptools Dependency](#very-old-versions-of-pip-or-its-setuptools-dependency)
3. [Installing EYAML (Optional)](#installing-eyaml-optional)
5. [Based on ruamel.yaml](#based-on-ruamelyaml)
6. [The Files of This Project](#the-files-of-this-project)
1. [Command-Line Tools](#command-line-tools)
2. [Libraries](#libraries)
7. [Basic Usage](#basic-usage)
1. [Basic Usage: Command-Line Tools](#basic-usage--command-line-tools)
1. [Rotate Your EYAML Keys](#rotate-your-eyaml-keys)
2. [Get the Differences Between Two Documents](#get-the-differences-between-two-documents)
3. [Get a YAML/JSON/Compatible Value](#get-a-yamljsoncompatible-value)
4. [Search For YAML Paths](#search-for-yaml-paths)
5. [Change a YAML/JSON/Compatible Value](#change-a-yamljsoncompatible-value)
6. [Merge YAML/JSON/Compatible Files](#merge-yamljsoncompatible-files)
7. [Validate YAML/JSON/Compatible Documents](#validate-yamljsoncompatible-documents)
2. [Basic Usage: Libraries](#basic-usage--libraries)
1. [Initialize ruamel.yaml and These Helpers](#initialize-ruamelyaml-and-these-helpers)
2. [Searching for YAML Nodes](#searching-for-yaml-nodes)
3. [Changing Values](#changing-values)
4. [Merging Documents](#merging-documents)
## Introduction
This project presents and utilizes YAML Paths, which are a powerful, intuitive
means of identifying one *or more* nodes within [YAML](https://yaml.org/),
[EYAML](https://github.com/voxpupuli/hiera-eyaml), or compatible data structures
like [JSON](https://www.json.org/). Both dot-notation (inspired by
[Hiera](https://github.com/puppetlabs/hiera)) and forward-slash-notation
(influenced by [XPath](https://www.w3schools.com/xml/xml_xpath.asp)) are
supported. The [libraries](#libraries) (modules) and several [command-line tool
implementations](#command-line-tools) are provided. With these, you can build
YAML Path support right into your own application or easily use its capabilities
right away from the command-line to retrieve, update, merge, validate, and scan
YAML/JSON/Compatible data.
This implementation of YAML Path is a *query language* in addition to a *node
descriptor*. With it, you can describe or select a single precise node or
search for any number of nodes that match some criteria. Keys, values,
elements, anchors, and aliases can all be searched at any number of levels
within the data structure using the same query. Collectors can also be used to
gather and further select from otherwise disparate parts of the source data.
The [project Wiki](https://github.com/wwkimball/yamlpath/wiki) provides a
deeper dive into these concepts.
## Illustration
To illustrate some of these concepts, consider these samples:
```yaml
---
hash:
child_attr:
key: 5280
```
This value, `5280`, can be identified via YAML Path as any of:
1. `hash.child_attr.key` (dot-notation)
2. `hash.child_attr[.=key]` (search all child keys for one named, `key`, and
yield its value)
3. `/hash/child_attr/key` (same as 1 but in forward-slash notation)
4. `/hash/child_attr[.=key]` (same as 2 but in forward-slash notation)
```yaml
---
aliases:
- &first_anchor Simple string value
```
With YAML Path, you can select this anchored value by any of these equivalent
expressions:
1. `aliases[0]` (explicit array element number)
2. `aliases.0` (implicit array element number in dot-notation)
3. `aliases[&first_anchor]` (search by Anchor name)
4. `aliases[.^Simple]` (search for any elements starting with "Simple")
5. `aliases[.%string]` (search for any elements containing "string")
6. `aliases[.$value]` (search for any elements ending with "value")
7. `aliases[.=~/^(\b[Ss][a-z]+\s){2}[a-z]+$/]` (search for any elements matching
a complex Python Regular Expression, which happens to match the example)
8. `/aliases[0]` (same as 1 but in forward-slash notation)
9. `/aliases/0` (same as 2 but in forward-slash notation)
10. `/aliases[&first_anchor]` (same as 3 but in forward-slash notation)
```yaml
---
users:
- name: User One
password: ENC[PKCS7,MIIBiQY...Jk==]
roles:
- Writers
- name: User Two
password: ENC[PKCS7,MIIBiQY...vF==]
roles:
- Power Users
- Editors
```
With an example like this, YAML Path enables:
* selection of single nodes: `/users/0/roles/0` = `Writers`
* all children nodes of any given parent: `/users/1/roles` =
`["Power Users", "Editors"]`
* searching by a child attribute: `/users[name="User One"]/password` =
`Some decrypted value, provided you have the appropriate EYAML keys`
* pass-through selections against arrays-of-hashes: `/users/roles` =
`["Writers"]\n["Power Users", "Editors"]` (each user's list of roles are a
separate result)
* collection of disparate results: `(/users/name)` =
`["User One", "User Two"]` (all names appear in a single result instead of
one per line)
For a deeper exploration of YAML Path's capabilities, please visit the
[project Wiki](https://github.com/wwkimball/yamlpath/wiki).
## Supported YAML Path Segments
A YAML Path *segment* is the text between separators which identifies zero or
more parent or leaf nodes within the data structure. For dot-notation, a path
like `hash.key` identifies two segments: `hash` (a parent node) and `key` (a
leaf node). The same path in forward-slash notation would be: `/hash/key`.
YAML Path understands these segment types:
* Top-level Hash key selection: `key`
* Explicit top-level array element selection: `[#]` where `#` is the zero-based
element number; `#` can also be negative, causing the element to be selected
from the end of the Array
* Implicit array element selection **or** numbered hash key selection: `#`
where `#` is the 0-based element number **or** exact name of a hash key which
is itself a number
* Top-level (Hash) Anchor lookups: `&anchor_name` (the `&` is required to
indicate you are seeking an Anchor by name)
* Hash sub-keys: `hash.child.key` or `/hash/child/key`
* Demarcation for dotted Hash keys: `hash.'dotted.child.key'` or
`hash."dotted.child.key"` (not necessary when using forward-slash notation,
`/hash/dotted.child.key`)
* Named Array element selection: `array[#]`, `array.#`, `/array[#]`, or
`/array/#` where `array` is the name of the Hash key containing Array data
and `#` is the 0-based element number
* Anchor lookups in named Arrays: `array[&anchor_name]` where `array` is the
name of the Hash key containing Array data and both of the `[]` pair and `&`
are required to indicate you are seeking an Anchor by name within an Array
* Array slicing: `array[start#:stop#]` where `start#` is the first inclusive,
zero-based element and `stop#` is the last exclusive element to select;
either or both can be negative, causing the elements to be selected from the
end of the Array; when `start#` and `stop#` are identical, it is the same as
`array[start#]`
* Hash slicing: `hash[min:max]` where `min` and `max` are alphanumeric terms
between which the Hash's keys are compared
* Escape symbol recognition: `hash.dotted\.child\.key`,
`/hash/whacked\/child\/key`, and `keys_with_\\slashes`
* Hash attribute searches (which can return zero or more matches):
* Exact match: `hash[name=admin]`
* Starts With match: `hash[name^adm]`
* Ends With match: `hash[name$min]`
* Contains match: `hash[name%dmi]`
* Less Than match: `hash[access_level<500]`
* Greater Than match: `hash[access_level>0]`
* Less Than or Equal match: `hash[access_level<=100]`
* Greater Than or Equal match: `hash[access_level>=0]`
* [Python Regular Expression](https://docs.python.org/3/library/re.html)
matches: `hash[access_level=~/^\D+$/]` (the `/` Regular
Expression delimiter can be substituted for any character you need, except
white-space; note that `/` does not interfere with forward-slash notation
*and it does not need to be escaped* because the entire search expression is
contained within a `[]` pair)
* Invert any match with `!`, like: `hash[name!=admin]` or even
`hash[!name=admin]` (the former syntax is used when YAML Paths are
stringified but both forms are equivalent)
* Demarcate and/or escape expression operands, like:
`hash[full\ name="Some User\'s Name"]` (note that embedded, single `'` and
`"` must be escaped lest they be deemed unmatched demarcation pairings)
* Multi-level matching: `hash[name%admin].pass[encrypted!^ENC\[]` or
`/hash[name%admin]/pass[encrypted!^ENC\[]`
* Descendent node searches:
`structure[has.descendant.with=something].has.another.field` or
`/structure[/has/descendant/with=something]/has/another/field`
* Array element searches with all of the search methods above via `.` (yields
any matching elements): `array[.>9000]`
* Hash key-name searches with all of the search methods above via `.` (yields
their values, not the keys themselves): `hash[.^app_]`
* Array-of-Hashes Pass-Through Selection: Omit a selector for the elements of
an Array-of-Hashes and all matching Hash attributes at that level will be
yielded (or searched when there is more to the path). For example,
`warriors[1].power_level` or `/warriors[1]/power_level` will return the
power_level attribute of only the second Hash in an Array-of-Hashes while
`warriors.power_level` or `/warriors/power_level` will return the power_level
attribute of every Hash in the same Array-of-Hashes. Of course these results
can be filtered in multiple ways, like `warriors[power_level>9000]`,
`/warriors[power_level>9000]`, `warriors.power_level[.>9000]`, and
`/warriors/power_level[.>9000]` all yield only the power_level from *all*
warriors with power_levels over 9,000 within the same array of warrior hashes.
* Unordered Set value accessing and searching with all above search methods.
* Wildcard Searches: The `*` symbol can be used as shorthand for the `[]`
search operator against text keys and values: `/warriors/name/Go*`; it also
returns every immediate child, regardless its key or value.
* Deep Traversals: The `**` symbol pair deeply traverses the document:
* When it is the last or only segment of a YAML Path, it selects every leaf
node from the remainder of the document's tree: `/shows/**`
* When another segment follows, it matches every node within the remainder
of the document's tree for which the following (and subsequent) segments
match: `/shows/**/name/Star*`
* Search Keywords: Advanced search capabilities not otherwise possible using
other YAML Path segments. Taking the form of `[KEYWORD(PARAMETERS)]`, these
keywords are
[deeply explored on the Wiki](https://github.com/wwkimball/yamlpath/wiki/Search-Keywords)
and include:
* `[distinct(NAME)]`: Match exactly one of every value within collections,
discarding duplicates; i.e.: [1, 2, 2, 3] has distinct values, [1, 2, 3]
* `[has_child(NAME)]`: Match nodes having a named child key
* `[max([NAME])]`: Match nodes having the maximum value
* `[min([NAME])]`: Match nodes having the minimum value
* `[name()]`: Match only the name of the present node, discarding all
children
* `[parent([STEPS])]`, Step up 1-N levels in the document from the present
node
* `[unique(NAME)]`: Match only values which have no duplicates within
collections; i.e.: [1, 2, 2, 3] has unique values, [1, 3]
* Collectors: Placing any portion of the YAML Path within parenthesis defines a
virtual list collector, like `(YAML Path)`; concatenation, exclusion, and
intersection operators are supported -- `+`, `-`, and `&`, respectively --
along with nesting, like `(...)-((...)+(...))&(...)`
* Complex combinations:
`some::deep.hierarchy[with!=""].'any.valid'[.=~/(yaml|json)/][data%structure].or.complexity[4].2`
or `/some::deep/hierarchy[with!=""]/any*.*valid[.=~/(yaml|json)/][data%structure]/or/compl*xity[4]/2/**`
This implementation of YAML Path encourages creativity. Use whichever notation
and segment types that make the most sense to you in each application.
The [project Wiki provides more illustrative details of YAML Path Segments](https://github.com/wwkimball/yamlpath/wiki/Segments-of-a-YAML-Path).
## Installing
Some OS distributions offer some versions of yamlpath -- and its dependencies
-- via packages. While these versions of yamlpath are often outdated, they can
be convenient to install using your OS' native package manager (`apt`, `yum`,
`npm`, and such). Otherwise, Python's own package manager `pip` will always
offer the latest version of yamlpath and -- even better -- can be isolated to
ephemeral or longer-lasting virtual Python environments.
### Requirements
This project requires [Python](https://www.python.org/) 3. It is rigorously
tested against Pythons 3.7 through 3.11 (support for Python 3.6 was dropped
10 FEB 2024). Most operating systems and distributions have access to Python
3 even if only Python 2 -- or no Python, at all -- came pre-installed. It is
generally safe to have more than one version of Python on your system at the
same time, especially when using
[virtual Python environments](https://docs.python.org/3/library/venv.html).
*yamlpath* depends on *ruamel.yaml* (derived from and greatly extending PyYAML)
and *python-dateutil*. When using OS-native packages or `pip`, you do not need
to pre-install these libraries yourself except under extraordinary
circumstances like using very old versions of `pip` or its own dependency,
*setuptools*.
### Using pip
Like most others, this Python project is published to [PyPI](https://pypi.org/)
so that it can be easily installed via Python's `pip` command (or its
version-specific `pip3`, `pip3.7`, and such depending on how your Python was
installed).
Python's `pip` command is ever-changing. It is updated very frequently. This
command further depends on other libraries to do its job, namely *setuptools*.
It so happens that *setuptools* is also updated very frequently. Both of these
are separate from Python itself, despite versions of them being pre-installed
with Python. It is your responsibility to keep `pip` and *setuptools*
up-to-date. When `pip` or *setuptools* become outdated, _you will experience
errors_ when trying to install newer Python packages like *yamlpath* **unless
you preinstall such packages' dependencies**. In the case of *yamlpath*, this
means you'd need to preinstall *ruamel.yaml* and *python-dateutil* if you
cannot or choose not to upgrade `pip` and/or *setuptools*.
As long as your `pip` and *setuptools* are up-to-date, installing *yamlpath* is
as simple as a single command (the "3.7" suffix to the `pip` command is
optional, depending on how your Python 3 was installed):
```shell
pip3.7 install yamlpath
```
#### Very Old Versions of pip or its setuptools Dependency
Very old versions of Python 3 ship with seriously outdated versions of `pip` and
its *setuptools* dependency. When using versions of `pip` older than **18.1**
or *setuptools* older than version **46.4.0**, you will not be able to install
*yamlpath* with a single command. In this case, you have two options: either
pre-install *ruamel.yaml* and *python-dateutil* before installing *yamlpath* or
update `pip` and/or *setuptools* to at least the minimum required versions so
`pip` can auto-determine and install dependencies. This issue is not unique to
*yamlpath*.
Upgrading `pip` and *setuptools* is trivially simple as long as you have
sufficient access rights to do so on your local machine. Depending on your
situation, you may need to prefix these with `sudo` and/or you may need to
substitute `python3` and `pip3` for `python` and `pip`, or even `python3.7` and
`pip3.7` (or another specific version of Python 3), respectively. To reiterate
that this project requires Python 3, these sample commands will be
demonstrated using such prefixes:
```shell
python3.7 -m pip install --upgrade pip
pip3.7 install --upgrade setuptools
```
When you cannot or will not update `pip` or *setuptools*, just pre-install
*ruamel.yaml* and *python-dateutil* before yamlpath. Each must be installed
separately and in order, like this (you **cannot** combine these installations
into a single command):
```shell
pip3.7 install ruamel.yaml python-dateutil
pip3.7 install yamlpath
```
The downside to choosing this manual installation path is that you may end up
with an incompatible version of *ruamel.yaml* or *python-dateutil*. This will
manifest either as an inability to install *yamlpath* at all, or only certain
versions of *yamlpath*, or *yamlpath* may experience unexpected errors caused
by the incompatible code. For the best experience, you are strongly encouraged
to just keep `pip` and *setuptools* up-to-date, particularly as a routine part
of installing any new Python packages.
### Installing EYAML (Optional)
EYAML support is entirely optional. You do not need EYAML to use YAML Path.
That YAML Path supports EYAML is a service to a substantial audience: Puppet
users. At the time of this writing, EYAML (classified as a Hiera
back-end/plug-in) is available only as a Ruby Gem. That said, it provides a
command-line tool, `eyaml`, which can be employed by this otherwise Python
project. To enjoy EYAML support, install compatible versions of ruby and
rubygems, then execute:
```shell
gem install hiera-eyaml
```
If this puts the `eyaml` command on your system `PATH`, nothing more need be
done apart from generating or obtaining your encryption keys. Otherwise, you
can tell YAML Path library and tools where to find the `eyaml` command.
## Based on ruamel.yaml
In order to support the best available YAML editing capability (so called,
round-trip editing with support for comment preservation), this project is based
on [ruamel.yaml](https://sourceforge.net/projects/ruamel-yaml/) for
Python 3. While ruamel.yaml is based on PyYAML --
Python's "standard" YAML library -- ruamel.yaml is [objectively better than
PyYAML](https://yaml.readthedocs.io/en/latest/pyyaml.html), which lacks critical
round-trip editing capabilities as well as up-to-date YAML/Compatible data
parsing capabilities (at the time of this writing).
Should PyYAML ever merge with -- or at least, catch up with -- ruamel.yaml, this
project can be (lightly) adapted to depend on it, instead. These conversations
may offer some insight into when or whether this might happen:
* [Is this time to pass the baton?](https://github.com/yaml/pyyaml/issues/31)
* [Rebase off ruamel? - many new valuable features](https://github.com/yaml/pyyaml/issues/46)
## The Files of This Project
This repository contains:
1. Generally-useful Python library files. These contain the reusable core of
this project's YAML Path capabilities.
2. Some implementations of those libraries, exhibiting their capabilities and
simple-to-use APIs as command-line tools.
3. Various support, documentation, and build files.
### Command-Line Tools
This project provides some command-line tool implementations which utilize YAML
Path. For some use-case examples of these tools,
[see below](#basic-usage--command-line-tools).
The supplied command-line tools include:
* [eyaml-rotate-keys](yamlpath/commands/eyaml_rotate_keys.py)
```text
usage: eyaml-rotate-keys [-h] [-V] [-d | -v | -q] [-b] [-x EYAML]
-i OLDPRIVATEKEY -c OLDPUBLICKEY
-r NEWPRIVATEKEY -u NEWPUBLICKEY
YAML_FILE [YAML_FILE ...]
Rotates the encryption keys used for all EYAML values within a set of YAML
files, decrypting with old keys and re-encrypting using replacement keys.
positional arguments:
YAML_FILE one or more YAML files containing EYAML values
optional arguments:
-h, --help show this help message and exit
-V, --version show program's version number and exit
-d, --debug output debugging details
-v, --verbose increase output verbosity
-q, --quiet suppress all output except errors
-b, --backup save a backup of each modified YAML_FILE with an extra
.bak file-extension
-x EYAML, --eyaml EYAML
the eyaml binary to use when it isn't on the PATH
EYAML_KEYS:
All key arguments are required
-r NEWPRIVATEKEY, --newprivatekey NEWPRIVATEKEY
the new EYAML private key
-u NEWPUBLICKEY, --newpublickey NEWPUBLICKEY
the new EYAML public key
-i OLDPRIVATEKEY, --oldprivatekey OLDPRIVATEKEY
the old EYAML private key
-c OLDPUBLICKEY, --oldpublickey OLDPUBLICKEY
the old EYAML public key
Any YAML_FILEs lacking EYAML values will not be modified (or backed up, even
when -b/--backup is specified).
```
* [yaml-diff](yamlpath/commands/yaml_diff.py)
```text
usage: yaml-diff [-h] [-V] [-a] [-s | -o]
[-t ['.', '/', 'auto', 'dot', 'fslash']] [-x EYAML]
[-r PRIVATEKEY] [-u PUBLICKEY] [-E] [-d | -v | -q]
YAML_FILE YAML_FILE
Calculate the functional difference between two YAML/JSON/Compatible
documents. Immaterial differences (which YAML/JSON parsers discard) are
ignored. EYAML can be employed to compare encrypted values.
positional arguments:
YAML_FILE exactly two YAML/JSON/compatible files to compare; use
- to read one document from STDIN
optional arguments:
-h, --help show this help message and exit
-V, --version show program's version number and exit
-a, --sync-arrays Synchronize array elements before comparing them,
resulting only in ADD, DELETE, and SAME differences
(no CHANGEs because the positions of elements are
disregarded); Array-of-Hash elements must completely
and perfectly match or they will be deemed additions
or deletions
-s, --same Show all nodes which are the same in addition to
differences
-o, --onlysame Show only nodes which are the same, still reporting
that differences exist -- when they do -- with an
exit-state of 1
-t ['.', '/', 'auto', 'dot', 'fslash'], --pathsep ['.', '/', 'auto', 'dot', 'fslash']
indicate which YAML Path separator to use when
rendering results; default=dot
-d, --debug output debugging details
-v, --verbose increase output verbosity
-q, --quiet suppress all output except system errors
EYAML options:
Left unset, the EYAML keys will default to your system or user defaults.
Both keys must be set either here or in your system or user EYAML
configuration file when using EYAML.
-x EYAML, --eyaml EYAML
the eyaml binary to use when it isn't on the PATH
-r PRIVATEKEY, --privatekey PRIVATEKEY
EYAML private key
-u PUBLICKEY, --publickey PUBLICKEY
EYAML public key
-E, --ignore-eyaml-values
Do not use EYAML to compare encrypted data; rather,
treat ENC[...] values as regular strings
Only one YAML_FILE may be the - pseudo-file for reading from STDIN. For more
information about YAML Paths, please visit
https://github.com/wwkimball/yamlpath.
```
* [yaml-get](yamlpath/commands/yaml_get.py)
```text
usage: yaml-get [-h] [-V] -p YAML_PATH
[-t ['.', '/', 'auto', 'dot', 'fslash']] [-S] [-x EYAML]
[-r PRIVATEKEY] [-u PUBLICKEY] [-d | -v | -q]
[YAML_FILE]
Retrieves one or more values from a YAML/JSON/Compatible file at a specified
YAML Path. Output is printed to STDOUT, one line per result. When a result is
a complex data-type (Array or Hash), a JSON dump is produced to represent it.
EYAML can be employed to decrypt the values.
positional arguments:
YAML_FILE the YAML file to query; omit or use - to read from
STDIN
optional arguments:
-h, --help show this help message and exit
-V, --version show program's version number and exit
-t ['.', '/', 'auto', 'dot', 'fslash'], --pathsep ['.', '/', 'auto', 'dot', 'fslash']
indicate which YAML Path separator to use when
rendering results; default=dot
-S, --nostdin Do not implicitly read from STDIN, even when YAML_FILE
is not set and the session is non-TTY
-d, --debug output debugging details
-v, --verbose increase output verbosity
-q, --quiet suppress all output except errors
required settings:
-p YAML_PATH, --query YAML_PATH
YAML Path to query
EYAML options:
Left unset, the EYAML keys will default to your system or user defaults.
Both keys must be set either here or in your system or user EYAML
configuration file when using EYAML.
-x EYAML, --eyaml EYAML
the eyaml binary to use when it isn't on the PATH
-r PRIVATEKEY, --privatekey PRIVATEKEY
EYAML private key
-u PUBLICKEY, --publickey PUBLICKEY
EYAML public key
For more information about YAML Paths, please visit
https://github.com/wwkimball/yamlpath.
```
* [yaml-merge](yamlpath/commands/yaml_merge.py)
```text
usage: yaml-merge [-h] [-V] [-c CONFIG] [-a {stop,left,right,rename}]
[-A {all,left,right,unique}] [-H {deep,left,right}]
[-O {all,deep,left,right,unique}] [-m YAML_PATH]
[-o OUTPUT | -w OVERWRITE] [-b] [-D {auto,json,yaml}] [-S]
[-d | -v | -q]
[YAML_FILE [YAML_FILE ...]]
Merges two or more YAML/JSON/Compatible files together.
positional arguments:
YAML_FILE one or more YAML files to merge, order-significant;
omit or use - to read from STDIN
optional arguments:
-h, --help show this help message and exit
-V, --version show program's version number and exit
-c CONFIG, --config CONFIG
INI syle configuration file for YAML Path specified
merge control options
-a {stop,left,right,rename}, --anchors {stop,left,right,rename}
means by which Anchor name conflicts are resolved
(overrides [defaults]anchors set via --config|-c and
cannot be overridden by [rules] because Anchors apply
to the whole file); default=stop
-A {all,left,right,unique}, --arrays {all,left,right,unique}
default means by which Arrays are merged together
(overrides [defaults]arrays but is overridden on a
YAML Path basis via --config|-c); default=all
-H {deep,left,right}, --hashes {deep,left,right}
default means by which Hashes are merged together
(overrides [defaults]hashes but is overridden on a
YAML Path basis in [rules] set via --config|-c);
default=deep
-O {all,deep,left,right,unique}, --aoh {all,deep,left,right,unique}
default means by which Arrays-of-Hashes are merged
together (overrides [defaults]aoh but is overridden on
a YAML Path basis in [rules] set via --config|-c);
default=all
-m YAML_PATH, --mergeat YAML_PATH
YAML Path indicating where in left YAML_FILE the right
YAML_FILE content is to be merged; default=/
-o OUTPUT, --output OUTPUT
Write the merged result to the indicated nonexistent
file
-w OVERWRITE, --overwrite OVERWRITE
Write the merged result to the indicated file; will
replace the file when it already exists
-b, --backup save a backup OVERWRITE file with an extra .bak
file-extension; applies only to OVERWRITE
-D {auto,json,yaml}, --document-format {auto,json,yaml}
Force the merged result to be presented in one of the
supported formats or let it automatically match the
known file-name extension of OUTPUT|OVERWRITE (when
provided), or match the type of the first document;
default=auto
-S, --nostdin Do not implicitly read from STDIN, even when there are
no - pseudo-files in YAML_FILEs with a non-TTY session
-d, --debug output debugging details
-v, --verbose increase output verbosity
-q, --quiet suppress all output except errors (implied when
-o|--output is not set)
The CONFIG file is an INI file with up to three sections:
[defaults] Sets equivalents of -a|--anchors, -A|--arrays,
-H|--hashes, and -O|--aoh.
[rules] Each entry is a YAML Path assigning -A|--arrays,
-H|--hashes, or -O|--aoh for precise nodes.
[keys] Wherever -O|--aoh=DEEP, each entry is treated as a
record with an identity key. In order to match RHS
records to LHS records, a key must be known and is
identified on a YAML Path basis via this section.
Where not specified, the first attribute of the first
record in the Array-of-Hashes is presumed the identity
key for all records in the set.
The left-to-right order of YAML_FILEs is significant. Except
when this behavior is deliberately altered by your options, data
from files on the right overrides data in files to their left.
Only one input file may be the - pseudo-file (read from STDIN).
When no YAML_FILEs are provided, - will be inferred as long as you
are running this program without a TTY (unless you set
--nostdin|-S). Any file, including input from STDIN, may be a
multi-document YAML or JSON file.
For more information about YAML Paths, please visit
https://github.com/wwkimball/yamlpath.
```
* [yaml-paths](yamlpath/commands/yaml_paths.py)
```text
usage: yaml-paths [-h] [-V] -s EXPRESSION [-c EXPRESSION] [-m] [-L] [-F] [-X]
[-P] [-n] [-t ['.', '/', 'auto', 'dot', 'fslash']]
[-i | -k | -K] [-a] [-A | -Y | -y | -l] [-e] [-x EYAML]
[-r PRIVATEKEY] [-u PUBLICKEY] [-S] [-d | -v | -q]
[YAML_FILE [YAML_FILE ...]]
Returns zero or more YAML Paths indicating where in given YAML/JSON/Compatible
data one or more search expressions match. Values, keys, and/or anchors can be
searched. EYAML can be employed to search encrypted values.
positional arguments:
YAML_FILE one or more YAML files to search; omit or use - to
read from STDIN
optional arguments:
-h, --help show this help message and exit
-V, --version show program's version number and exit
-c EXPRESSION, --except EXPRESSION
except results matching this search expression; can be
set more than once
-m, --expand expand matching parent nodes to list all permissible
child leaf nodes (see "reference handling options" for
restrictions)
-t ['.', '/', 'auto', 'dot', 'fslash'], --pathsep ['.', '/', 'auto', 'dot', 'fslash']
indicate which YAML Path separator to use when
rendering results; default=dot
-a, --refnames also search the names of &anchor and *alias references
-S, --nostdin Do not implicitly read from STDIN, even when there are
no - pseudo-files in YAML_FILEs with a non-TTY session
-d, --debug output debugging details
-v, --verbose increase output verbosity
-q, --quiet suppress all non-result output except errors
required settings:
-s EXPRESSION, --search EXPRESSION
the search expression; can be set more than once
result printing options:
-L, --values print the values or elements along with each YAML Path
(complex results are emitted as JSON; use --expand to
emit only simple values)
-F, --nofile omit source file path and name decorators from the
output (applies only when searching multiple files)
-X, --noexpression omit search expression decorators from the output
-P, --noyamlpath omit YAML Paths from the output (useful with --values
or to indicate whether a file has any matches without
printing them all, perhaps especially with
--noexpression)
-n, --noescape omit escape characters from special characters in
printed YAML Paths; this is unsafe for feeding the
resulting YAML Paths into other YAML Path commands
because the symbols that would be escaped have special
meaning to YAML Path processors
key name searching options:
-i, --ignorekeynames (default) do not search key names
-k, --keynames search key names in addition to values and array
elements
-K, --onlykeynames only search key names (ignore all values and array
elements)
reference handling options:
Indicate how to treat anchor and alias references. An anchor is an
original, reusable key or value. All aliases become replaced by the
anchors they reference when YAML data is read. These options specify how
to handle this duplication of keys and values. Note that the default
behavior includes all aliased keys but not aliased values.
-A, --anchorsonly include only original matching key and value anchors
in results, discarding all aliased keys and values
(including child nodes)
-Y, --allowkeyaliases
(default) include matching key aliases, permitting
search traversal into their child nodes
-y, --allowvaluealiases
include matching value aliases (does not permit search
traversal into aliased keys)
-l, --allowaliases include all matching key and value aliases
EYAML options:
Left unset, the EYAML keys will default to your system or user defaults.
Both keys must be set either here or in your system or user EYAML
configuration file when using EYAML.
-e, --decrypt decrypt EYAML values in order to search them
(otherwise, search the encrypted blob)
-x EYAML, --eyaml EYAML
the eyaml binary to use when it isn't on the PATH
-r PRIVATEKEY, --privatekey PRIVATEKEY
EYAML private key
-u PUBLICKEY, --publickey PUBLICKEY
EYAML public key
A search or exception EXPRESSION takes the form of a YAML Path search operator
-- %, $, =, ^, >, <, >=, <=, =~, or ! -- followed by the search term, omitting
the left-hand operand. For more information about YAML Paths, please visit
https://github.com/wwkimball/yamlpath/wiki. To report issues with this tool or
to request enhancements, please visit
https://github.com/wwkimball/yamlpath/issues.
```
* [yaml-set](yamlpath/commands/yaml_set.py)
```text
usage: yaml-set [-h] [-V] -g YAML_PATH
[-a VALUE | -A ANCHOR | -f FILE | -i | -R LENGTH | -N | -D]
[-F {bare,boolean,default,dquote,float,folded,int,literal,squote}]
[-c CHECK] [-s YAML_PATH] [-m] [-b]
[-t ['.', '/', 'auto', 'dot', 'fslash']] [-M CHARS] [-H ANCHOR]
[-T TAG] [-e] [-x EYAML] [-r PRIVATEKEY] [-u PUBLICKEY] [-S]
[-d | -v | -q]
[YAML_FILE]
Changes one or more Scalar values in a YAML/JSON/Compatible document at a
specified YAML Path. Matched values can be checked before they are replaced to
mitigate accidental change. When matching singular results, the value can be
archived to another key before it is replaced. Further, EYAML can be employed to
encrypt the new values and/or decrypt an old value before checking it.
positional arguments:
YAML_FILE the YAML file to update; omit or use - to read from
STDIN
optional arguments:
-h, --help show this help message and exit
-V, --version show program's version number and exit
-F {bare,boolean,default,dquote,float,folded,int,literal,squote}, --format {bare,boolean,default,dquote,float,folded,int,literal,squote}
override automatic formatting of the new value
-c CHECK, --check CHECK
check the value before replacing it
-s YAML_PATH, --saveto YAML_PATH
save the old value to YAML_PATH before replacing it;
implies --mustexist
-m, --mustexist require that the --change YAML_PATH already exist in
YAML_FILE
-b, --backup save a backup YAML_FILE with an extra .bak file-
extension
-t ['.', '/', 'auto', 'dot', 'fslash'], --pathsep ['.', '/', 'auto', 'dot', 'fslash']
indicate which YAML Path separator to use when rendering
results; default=dot
-M CHARS, --random-from CHARS
characters from which to build a value for --random;
default=all upper- and lower-case letters and all digits
-H ANCHOR, --anchor ANCHOR
when --aliasof|-A points to a value which is not already
Anchored, a new Anchor with this name is created;
renames an existing Anchor if already set
-T TAG, --tag TAG assign a custom YAML (data-type) tag to the changed
nodes; can be used without other input options to assign
or change a tag
-S, --nostdin Do not implicitly read from STDIN, even when there is no
YAML_FILE with a non-TTY session
-d, --debug output debugging details
-v, --verbose increase output verbosity
-q, --quiet suppress all output except errors
required settings:
-g YAML_PATH, --change YAML_PATH
YAML Path where the target value is found
input options:
-a VALUE, --value VALUE
set the new value from the command-line instead of STDIN
-A ANCHOR, --aliasof ANCHOR
set the value as a YAML Alias of an existing Anchor, by
name (merely copies the target value for non-YAML files)
-f FILE, --file FILE read the new value from file (discarding any trailing
new-lines)
-i, --stdin accept the new value from STDIN (best for sensitive
data)
-R LENGTH, --random LENGTH
randomly generate a replacement value of a set length
-N, --null sets the value to null
-D, --delete delete rather than change target node(s); implies
--mustexist|-m
EYAML options:
Left unset, the EYAML keys will default to your system or user defaults. You
do not need to supply a private key unless you enable --check and the old
value is encrypted.
-e, --eyamlcrypt encrypt the new value using EYAML
-x EYAML, --eyaml EYAML
the eyaml binary to use when it isn't on the PATH
-r PRIVATEKEY, --privatekey PRIVATEKEY
EYAML private key
-u PUBLICKEY, --publickey PUBLICKEY
EYAML public key
When no changes are made, no backup is created, even when -b/--backup is
specified. For more information about YAML Paths, please visit
https://github.com/wwkimball/yamlpath/wiki. To report issues with this tool or
to request enhancements, please visit
https://github.com/wwkimball/yamlpath/issues.
```
* [yaml-validate](yamlpath/commands/yaml_validate.py)
```text
usage: yaml-validate [-h] [-V] [-S] [-d | -v | -q] [YAML_FILE [YAML_FILE ...]]
Validate YAML, JSON, and compatible files.
positional arguments:
YAML_FILE one or more single- or multi-document YAML/JSON/compatible
files to validate; omit or use - to read from STDIN
optional arguments:
-h, --help show this help message and exit
-V, --version show program's version number and exit
-S, --nostdin Do not implicitly read from STDIN, even when there are no -
pseudo-files in YAML_FILEs with a non-TTY session
-d, --debug output debugging details
-v, --verbose increase output verbosity (show valid documents)
-q, --quiet suppress all output except system errors
Except when suppressing all report output with --quiet|-q, validation issues
are printed to STDOUT (not STDERR). Further, the exit-state will report 0 when
there are no issues, 1 when there is an issue with the supplied command-line
arguments, or 2 when validation has failed for any document.
```
### Libraries
While there are several supporting library files like enumerations, types, and
exceptions, the most interesting library files include:
* [yamlpath.py](yamlpath/yamlpath.py) -- The core YAML Path parser logic.
* [processor.py](yamlpath/processor.py) -- Processes YAMLPath instances to read
or write data to YAML/Compatible sources.
* [eyamlprocessor.py](yamlpath/eyaml/eyamlprocessor.py) -- Extends the
Processor class to support EYAML data encryption and decryption.
* [merger.py](yamlpath/merger/merger.py) -- The core document merging logic.
## Basic Usage
The files of this project can be used either as command-line tools or as
libraries to supplement your own work.
### Basic Usage: Command-Line Tools
The command-line tools are self-documented and [their documentation is captured
above](#command-line-tools) for easy reference. Simply pass `--help` to them in
order to obtain the same detailed documentation.
Please review [the comprehensive test_commands_*.py unit tests](/tests/) to
explore samples of YAML files and the many ways these tools help get and set
their data.
The following are some simple examples of their typical use-cases.
#### Rotate Your EYAML Keys
If the eyaml command is already on your PATH (if not, be sure to also supply
the optional `--eyaml` or `-x` argument):
```shell
eyaml-rotate-keys \
--oldprivatekey=~/old-keys/private_key.pkcs7.pem \
--oldpublickey=~/old-keys/public_key.pkcs7.pem \
--newprivatekey=~/new-keys/private_key.pkcs7.pem \
--newpublickey=~/new-keys/public_key.pkcs7.pem \
my_1st_yaml_file.yaml my_2nd_yaml_file.eyaml ... my_Nth_yaml_file.yaml
```
You could combine this with `find` and `xargs` if your E/YAML file are
dispersed through a directory hierarchy, as with Hiera data.
#### Get the Differences Between Two Documents
For routine use:
```shell
yaml-diff yaml_file1.yaml yaml_file2.yaml
```
Output is very similar to that of standard GNU diff against text files, except
it is generated against the *data* within the input files. This excludes
evaluating purely structural and immaterial differences between them like value
demarcation, white-space, and comments. When you need to evaluate the two
files as if they were just text files, use GNU diff or any of its clones.
To see all identical entries along with differences:
```shell
yaml-diff --same yaml_file1.yaml yaml_file2.yaml
```
To see *only* entries which are identical between the documents:
```shell
yaml-diff --onlysame yaml_file1.yaml yaml_file2.yaml
```
Advanced: Arrays can be evaluated such that they are synchronized before
evaluation. Rather than compare elements by identical index in both
documents -- reporting differences between them as changes and any additional
elements as additions or deletions -- they can instead be compared by matching
up all identical elements and then reporting only those values which are unique
to either document (and optionally identical matches).
```shell
yaml-diff --sync-arrays yaml_file1.yaml yaml_file2.yaml
```
#### Get a YAML/JSON/Compatible Value
At its simplest:
```shell
yaml-get \
--query=see.documentation.above.for.many.samples \
my_yaml_file.yaml
```
#### Search For YAML Paths
Simplest use:
```shell
yaml-paths \
--search=%word \
/some/directory/*.yaml
```
Search for multiple expressions and exclude unwanted results:
```shell
yaml-paths \
--search=^another \
--search=$word \
--except=%bad \
/some/directory/*.yaml
```
Return all leaf nodes under matching parents (most useful when matching against Hash keys and you only want the original leaf nodes beneath them):
```shell
yaml-paths \
--expand \
--keynames \
--search==parent_node \
/some/directory/*.yaml
```
#### Change a YAML/JSON/Compatible Value
For a no-frills change to a YAML file with deeply nested Hash structures:
```shell
yaml-set \
--change=see.documentation.above.for.many.samples \
--value="New Value" \
my_yaml_file.yaml
```
To rotate a password, preserving the old password perhaps so your automation can
apply the new password to your application(s):
```shell
yaml-set \
--mustexist \
--change=the.new.password \
--saveto=the.old.password \
--value="New Password" \
my_yaml_file.yaml
```
For the extremely cautious, you could check the old password before rotating
it and save a backup of the original file:
```shell
yaml-set \
--mustexist \
--change=the.new.password \
--saveto=the.old.password \
--check="Old Password" \
--value="New Password" \
--backup \
my_yaml_file.yaml
```
You can also add EYAML encryption (assuming the `eyaml` command is on your
PATH; if not, you can pass `--eyaml` to specify its location). In this example,
I add the optional `--format=folded` so that the long EYAML value is broken up
into a multi-line value rather than one very long string. This is the preferred
format for human legibility as well as EYAML consumers like
[Puppet](http://puppet.com). Note that `--format` has several other settings
and applies only to new values.
```shell
yaml-set \
--change=the.new.password \
--mustexist \
--saveto=the.old.password \
--check="Old Password" \
--value="New Password" \
--eyamlcrypt \
--format=folded \
--backup \
my_yaml_file.yaml
```
You can even tell EYAML which keys to use, if not your default system or user
keys:
```shell
yaml-set \
--change=the.new.password \
--mustexist \
--saveto=the.old.password \
--check="Old Password" \
--value="New Password" \
--eyamlcrypt \
--format=folded \
--privatekey=/secret/keys/private_key.pkcs7.pem \
--publickey=/secret/keys/public_key.pkcs7.pem \
--backup \
my_yaml_file.yaml
```
Note that for even greater security scenarios, you can keep the new value off of
your command-line, process list, and command history by swapping out `--value`
for one of `--stdin`, `--file`, or even `--random LENGTH` (use Python's
strongest random value generator if you don't need to specify the replacement
value in advance).
#### Merge YAML/JSON/Compatible Files
At its simplest, the `yaml-merge` command accepts two or more input files and
merges them together from left-to-right, writing the result to STDOUT:
```shell
yaml-merge leftmost.yaml middle.yaml right.json
```
If you'd rather write the results to a new output file (which must not already
exist):
```shell
yaml-merge \
--output=newfile.yaml \
leftmost.yaml \
middle.yaml \
right.json
```
Should you wish to merge the content of the files into a specific location (or
even multiple locations) within the leftmost document, specify a YAML Path via
the `--mergeat` or `-m` argument:
```shell
yaml-merge \
--mergeat=/anywhere/within/the/document \
leftmost.yaml \
middle.yaml \
right.json
```
To write arbitrary data from STDIN into a document, use the `-` pseudo-file:
```shell
echo "{arbitrary: [document, structure]}" | yaml-merge target.yaml -
```
Combine `--mergeat` or `-m` with the STDIN pseudo-file to control where the
data is to be written:
```shell
echo "{arbitrary: [document, structure]}" | \
yaml-merge \
--mergeat=/anywhere/within/the/document \
target.yaml -
```
There are many options for precisely controlling how the merge is performed,
including the ability to specify complex rules on a YAML Path basis via a
configuration file. Review the command's `--help` or the
[related Wiki](https://github.com/wwkimball/yamlpath/wiki/yaml-merge) for
more detail.
#### Validate YAML/JSON/Compatible Documents
Validating the structure of YAML, JSON, and compatible files is as simple as
running:
```shell
yaml-validate /path/to/any/files.yaml /path/to/more/files.json
```
In this default configuration, the command will output no report when all input
documents are valid. It will also report an exit-state of zero (0). Should
there be any validation errors, each will be printed to the screen and the
exit-state will be 2. An exit-state of 1 means your command-line arguments
were incorrect and an appropritae user error message will be displayed.
When there are validation issues, the offending file-name(s) and sub-document
index(es) (zero-based) will be displayed along with a detailed validation error
message.
### Basic Usage: Libraries
As for the libraries, they are also heavily documented and the example
implementations may perhaps serve as good copy-paste fodder (provided you give
credit to the source). That said, here's a general flow/synopsis.
#### Initialize ruamel.yaml and These Helpers
Your preferences may differ, but I use this setup for round-trip YAML parsing
and editing with ruamel.yaml. When you need to process EYAML encrypted data,
replace `yamlpath.Processor` with `yamlpath.eyaml.EYAMLProcessor` and add error
handling for `yamlpath.eyaml.EYAMLCommandException`.
Note also that these examples use `ConsolePrinter` to handle STDOUT and STDERR
messaging. You don't have to. However, some kind of logger must be passed to
these libraries so they can write messages _somewhere_. Your custom message
handler or logger must provide the same API as `ConsolePrinter`; review the
header documentation in [consoleprinter.py](yamlpath/wrappers/consoleprinter.py)
for details. Generally speaking, it would be trivial to write your own custom
wrapper for Python's standard logging facilities if you require targets other
than STDOUT and STDERR.
```python
from types import SimpleNamespace
from yamlpath.common import Parsers
from yamlpath.wrappers import ConsolePrinter
from yamlpath import Processor
# The various classes of this library must be able to write messages somewhere
# when things go bad. This project provides a CLI-centric logging class named
# ConsolePrinter. Even when not writing a CLI tool, you must still configure
# and pass ConsolePrinter or a class of your own with the same public API. For
# just muting logging output -- except for unrecoverable errors -- you can use
# this simple configuration object:
logging_args = SimpleNamespace(quiet=True, verbose=False, debug=False)
log = ConsolePrinter(logging_args)
# Prep the YAML parser and round-trip editor (tweak to your needs). You do not
# have to use Parsers.get_yaml_editor() but you must create a properly-
# configured instance of ruamel.yaml.YAML.
yaml = Parsers.get_yaml_editor()
# At this point, you'd load or parse your YAML file, stream, or string. This
# example demonstrates loading YAML data from an external file. You could also
# use the same function to load data from STDIN or even a String variable. See
# the Parser class for more detail.
yaml_file = "your-file.yaml"
(yaml_data, doc_loaded) = Parsers.get_yaml_data(yaml, log, yaml_file)
if not doc_loaded:
# There was an issue loading the file; an error message has already been
# printed via ConsolePrinter.
exit(1)
# Pass the logging facility and parsed YAML data to the YAMLPath Processor
processor = Processor(log, yaml_data)
# At this point, the Processor is ready to handle YAML Paths
```
#### Searching for YAML Nodes
These libraries use [Generators](https://wiki.python.org/moin/Generators) to get
nodes from parsed YAML data. Identify which node(s) to get via YAML Path
strings. You should also catch `yamlpath.exceptions.YAMLPathException`s
unless you prefer Python's native stack traces. When using EYAML, you should
also catch `yamlpath.eyaml.exceptions.EYAMLCommandException`s for the same
reason. Whether you are working with a single result or many, you should
consume the Generator output with a pattern similar to:
```python
from yamlpath import YAMLPath
from yamlpath.exceptions import YAMLPathException
yaml_path = YAMLPath("see.documentation.above.for.many.samples")
try:
for node_coordinate in processor.get_nodes(yaml_path, mustexist=True):
log.debug("Got {} from '{}'.".format(node_coordinate, yaml_path))
# Do something with each node_coordinate.node (the actual data)
except YAMLPathException as ex:
# If merely retrieving data, this exception may be deemed non-critical
# unless your later code absolutely depends upon a result.
log.error(ex)
```
#### Changing Values
At its simplest, you only need to supply the the YAML Path to one or more nodes
to update, and the value to apply to them. Catching
`yamlpath.exceptions.YAMLPathException` is optional but usually preferred over
allowing Python to dump the call stack in front of your users. When using
EYAML, the same applies to `yamlpath.eyaml.exceptions.EYAMLCommandException`.
```python
from yamlpath.exceptions import YAMLPathException
try:
processor.set_value(yaml_path, new_value)
except YAMLPathException as ex:
log.critical(ex, 119)
except EYAMLCommandException as ex:
log.critical(ex, 120)
```
#### Merging Documents
A document merge naturally requires at least two documents. At the code-level,
this means two populated DOM objects (populated instances of `yaml_data` from
above). You do not need to use a `Processor` for merging. In the least amount
of code, a merge looks like:
```python
from yamlpath.exceptions import YAMLPathException
from yamlpath.merger.exceptions import MergeException
from yamlpath.merger import Merger, MergerConfig
# Obtain or build the lhs_data and rhs_data objects using get_yaml_data or
# equivalent.
# You'll still need to supply a logger and some arguments used by the merge
# engine. For purely default behavior, you could create args as a bare
# SimpleNamespace. Initialize the new Merger instance with the LHS document.
merger = Merger(log, lhs_data, MergerConfig(log, args))
# Merge RHS into LHS
try:
merger.merge_with(rhs_data)
except MergeException as mex:
log.critical(mex, 129)
except YAMLPathException as yex:
log.critical(yex, 130)
# At this point, merger.data is the merged result; do what you will with it,
# including merging more data into it. When you are ready to dump (write)
# out the merged data, you must prepare the document and your
# ruamel.yaml.YAML instance -- usually obtained from func.get_yaml_editor()
# -- like this:
merger.prepare_for_dump(my_yaml_editor)
```
Raw data
{
"_id": null,
"home_page": "https://github.com/wwkimball/yamlpath",
"name": "yamlpath",
"maintainer": "",
"docs_url": null,
"requires_python": ">3.7.0",
"maintainer_email": "",
"keywords": "yaml eyaml json yaml-path diff merge",
"author": "William W. Kimball, Jr., MBA, MSIS",
"author_email": "github-yamlpath@kimballstuff.com",
"download_url": "https://files.pythonhosted.org/packages/3a/a4/eb264493c967e5dabf310c8d5bad6f435c6830eae087f1651e51d93aa065/yamlpath-3.8.2.tar.gz",
"platform": null,
"description": "# YAML Path and Command-Line Tools\n\n[](https://github.com/wwkimball/yamlpath/actions/workflows/build.yml)\n[](https://pypi.org/project/yamlpath/)\n[](https://pypi.org/project/yamlpath/)\n[](https://pepy.tech/project/yamlpath)\n[](https://coveralls.io/github/wwkimball/yamlpath?branch=master)\n[](https://www.codacy.com/gh/wwkimball/yamlpath/dashboard?utm_source=github.com&utm_medium=referral&utm_content=wwkimball/yamlpath&utm_campaign=Badge_Grade)\n\nAlong with providing a\n[standard for defining YAML Paths](https://github.com/wwkimball/yamlpath/wiki/Segments-of-a-YAML-Path),\nthis project aims to provide\n[generally-useful command-line tools](https://github.com/wwkimball/yamlpath/wiki/Command-Line-(CLI)-Tools)\nwhich implement YAML Paths. These bring intuitive YAML, EYAML, JSON, and\ncompatible data parsing and editing capabilties to the command-line. It is\nalso a\n[Python library](https://github.com/wwkimball/yamlpath/wiki/Python-Library)\nfor other projects to readily employ YAML Paths.\n\n## Contents\n\n1. [Introduction](#introduction)\n2. [Illustration](#illustration)\n3. [Supported YAML Path Segments](#supported-yaml-path-segments)\n4. [Installing](#installing)\n 1. [Requirements](#requirements)\n 2. [Using pip](#using-pip)\n 1. [Very Old Versions of pip or its setuptools Dependency](#very-old-versions-of-pip-or-its-setuptools-dependency)\n 3. [Installing EYAML (Optional)](#installing-eyaml-optional)\n5. [Based on ruamel.yaml](#based-on-ruamelyaml)\n6. [The Files of This Project](#the-files-of-this-project)\n 1. [Command-Line Tools](#command-line-tools)\n 2. [Libraries](#libraries)\n7. [Basic Usage](#basic-usage)\n 1. [Basic Usage: Command-Line Tools](#basic-usage--command-line-tools)\n 1. [Rotate Your EYAML Keys](#rotate-your-eyaml-keys)\n 2. [Get the Differences Between Two Documents](#get-the-differences-between-two-documents)\n 3. [Get a YAML/JSON/Compatible Value](#get-a-yamljsoncompatible-value)\n 4. [Search For YAML Paths](#search-for-yaml-paths)\n 5. [Change a YAML/JSON/Compatible Value](#change-a-yamljsoncompatible-value)\n 6. [Merge YAML/JSON/Compatible Files](#merge-yamljsoncompatible-files)\n 7. [Validate YAML/JSON/Compatible Documents](#validate-yamljsoncompatible-documents)\n\n 2. [Basic Usage: Libraries](#basic-usage--libraries)\n 1. [Initialize ruamel.yaml and These Helpers](#initialize-ruamelyaml-and-these-helpers)\n 2. [Searching for YAML Nodes](#searching-for-yaml-nodes)\n 3. [Changing Values](#changing-values)\n 4. [Merging Documents](#merging-documents)\n\n## Introduction\n\nThis project presents and utilizes YAML Paths, which are a powerful, intuitive\nmeans of identifying one *or more* nodes within [YAML](https://yaml.org/),\n[EYAML](https://github.com/voxpupuli/hiera-eyaml), or compatible data structures\nlike [JSON](https://www.json.org/). Both dot-notation (inspired by\n[Hiera](https://github.com/puppetlabs/hiera)) and forward-slash-notation\n(influenced by [XPath](https://www.w3schools.com/xml/xml_xpath.asp)) are\nsupported. The [libraries](#libraries) (modules) and several [command-line tool\nimplementations](#command-line-tools) are provided. With these, you can build\nYAML Path support right into your own application or easily use its capabilities\nright away from the command-line to retrieve, update, merge, validate, and scan\nYAML/JSON/Compatible data.\n\nThis implementation of YAML Path is a *query language* in addition to a *node\ndescriptor*. With it, you can describe or select a single precise node or\nsearch for any number of nodes that match some criteria. Keys, values,\nelements, anchors, and aliases can all be searched at any number of levels\nwithin the data structure using the same query. Collectors can also be used to\ngather and further select from otherwise disparate parts of the source data.\n\nThe [project Wiki](https://github.com/wwkimball/yamlpath/wiki) provides a\ndeeper dive into these concepts.\n\n## Illustration\n\nTo illustrate some of these concepts, consider these samples:\n\n```yaml\n---\nhash:\n child_attr:\n key: 5280\n```\n\nThis value, `5280`, can be identified via YAML Path as any of:\n\n1. `hash.child_attr.key` (dot-notation)\n2. `hash.child_attr[.=key]` (search all child keys for one named, `key`, and\n yield its value)\n3. `/hash/child_attr/key` (same as 1 but in forward-slash notation)\n4. `/hash/child_attr[.=key]` (same as 2 but in forward-slash notation)\n\n```yaml\n---\naliases:\n - &first_anchor Simple string value\n```\n\nWith YAML Path, you can select this anchored value by any of these equivalent\nexpressions:\n\n1. `aliases[0]` (explicit array element number)\n2. `aliases.0` (implicit array element number in dot-notation)\n3. `aliases[&first_anchor]` (search by Anchor name)\n4. `aliases[.^Simple]` (search for any elements starting with \"Simple\")\n5. `aliases[.%string]` (search for any elements containing \"string\")\n6. `aliases[.$value]` (search for any elements ending with \"value\")\n7. `aliases[.=~/^(\\b[Ss][a-z]+\\s){2}[a-z]+$/]` (search for any elements matching\n a complex Python Regular Expression, which happens to match the example)\n8. `/aliases[0]` (same as 1 but in forward-slash notation)\n9. `/aliases/0` (same as 2 but in forward-slash notation)\n10. `/aliases[&first_anchor]` (same as 3 but in forward-slash notation)\n\n```yaml\n---\nusers:\n - name: User One\n password: ENC[PKCS7,MIIBiQY...Jk==]\n roles:\n - Writers\n - name: User Two\n password: ENC[PKCS7,MIIBiQY...vF==]\n roles:\n - Power Users\n - Editors\n```\n\nWith an example like this, YAML Path enables:\n\n* selection of single nodes: `/users/0/roles/0` = `Writers`\n* all children nodes of any given parent: `/users/1/roles` =\n `[\"Power Users\", \"Editors\"]`\n* searching by a child attribute: `/users[name=\"User One\"]/password` =\n `Some decrypted value, provided you have the appropriate EYAML keys`\n* pass-through selections against arrays-of-hashes: `/users/roles` =\n `[\"Writers\"]\\n[\"Power Users\", \"Editors\"]` (each user's list of roles are a\n separate result)\n* collection of disparate results: `(/users/name)` =\n `[\"User One\", \"User Two\"]` (all names appear in a single result instead of\n one per line)\n\nFor a deeper exploration of YAML Path's capabilities, please visit the\n[project Wiki](https://github.com/wwkimball/yamlpath/wiki).\n\n## Supported YAML Path Segments\n\nA YAML Path *segment* is the text between separators which identifies zero or\nmore parent or leaf nodes within the data structure. For dot-notation, a path\nlike `hash.key` identifies two segments: `hash` (a parent node) and `key` (a\nleaf node). The same path in forward-slash notation would be: `/hash/key`.\n\nYAML Path understands these segment types:\n\n* Top-level Hash key selection: `key`\n* Explicit top-level array element selection: `[#]` where `#` is the zero-based\n element number; `#` can also be negative, causing the element to be selected\n from the end of the Array\n* Implicit array element selection **or** numbered hash key selection: `#`\n where `#` is the 0-based element number **or** exact name of a hash key which\n is itself a number\n* Top-level (Hash) Anchor lookups: `&anchor_name` (the `&` is required to\n indicate you are seeking an Anchor by name)\n* Hash sub-keys: `hash.child.key` or `/hash/child/key`\n* Demarcation for dotted Hash keys: `hash.'dotted.child.key'` or\n `hash.\"dotted.child.key\"` (not necessary when using forward-slash notation,\n `/hash/dotted.child.key`)\n* Named Array element selection: `array[#]`, `array.#`, `/array[#]`, or\n `/array/#` where `array` is the name of the Hash key containing Array data\n and `#` is the 0-based element number\n* Anchor lookups in named Arrays: `array[&anchor_name]` where `array` is the\n name of the Hash key containing Array data and both of the `[]` pair and `&`\n are required to indicate you are seeking an Anchor by name within an Array\n* Array slicing: `array[start#:stop#]` where `start#` is the first inclusive,\n zero-based element and `stop#` is the last exclusive element to select;\n either or both can be negative, causing the elements to be selected from the\n end of the Array; when `start#` and `stop#` are identical, it is the same as\n `array[start#]`\n* Hash slicing: `hash[min:max]` where `min` and `max` are alphanumeric terms\n between which the Hash's keys are compared\n* Escape symbol recognition: `hash.dotted\\.child\\.key`,\n `/hash/whacked\\/child\\/key`, and `keys_with_\\\\slashes`\n* Hash attribute searches (which can return zero or more matches):\n * Exact match: `hash[name=admin]`\n * Starts With match: `hash[name^adm]`\n * Ends With match: `hash[name$min]`\n * Contains match: `hash[name%dmi]`\n * Less Than match: `hash[access_level<500]`\n * Greater Than match: `hash[access_level>0]`\n * Less Than or Equal match: `hash[access_level<=100]`\n * Greater Than or Equal match: `hash[access_level>=0]`\n * [Python Regular Expression](https://docs.python.org/3/library/re.html)\n matches: `hash[access_level=~/^\\D+$/]` (the `/` Regular\n Expression delimiter can be substituted for any character you need, except\n white-space; note that `/` does not interfere with forward-slash notation\n *and it does not need to be escaped* because the entire search expression is\n contained within a `[]` pair)\n * Invert any match with `!`, like: `hash[name!=admin]` or even\n `hash[!name=admin]` (the former syntax is used when YAML Paths are\n stringified but both forms are equivalent)\n * Demarcate and/or escape expression operands, like:\n `hash[full\\ name=\"Some User\\'s Name\"]` (note that embedded, single `'` and\n `\"` must be escaped lest they be deemed unmatched demarcation pairings)\n * Multi-level matching: `hash[name%admin].pass[encrypted!^ENC\\[]` or\n `/hash[name%admin]/pass[encrypted!^ENC\\[]`\n * Descendent node searches:\n `structure[has.descendant.with=something].has.another.field` or\n `/structure[/has/descendant/with=something]/has/another/field`\n* Array element searches with all of the search methods above via `.` (yields\n any matching elements): `array[.>9000]`\n* Hash key-name searches with all of the search methods above via `.` (yields\n their values, not the keys themselves): `hash[.^app_]`\n* Array-of-Hashes Pass-Through Selection: Omit a selector for the elements of\n an Array-of-Hashes and all matching Hash attributes at that level will be\n yielded (or searched when there is more to the path). For example,\n `warriors[1].power_level` or `/warriors[1]/power_level` will return the\n power_level attribute of only the second Hash in an Array-of-Hashes while\n `warriors.power_level` or `/warriors/power_level` will return the power_level\n attribute of every Hash in the same Array-of-Hashes. Of course these results\n can be filtered in multiple ways, like `warriors[power_level>9000]`,\n `/warriors[power_level>9000]`, `warriors.power_level[.>9000]`, and\n `/warriors/power_level[.>9000]` all yield only the power_level from *all*\n warriors with power_levels over 9,000 within the same array of warrior hashes.\n* Unordered Set value accessing and searching with all above search methods.\n* Wildcard Searches: The `*` symbol can be used as shorthand for the `[]`\n search operator against text keys and values: `/warriors/name/Go*`; it also\n returns every immediate child, regardless its key or value.\n* Deep Traversals: The `**` symbol pair deeply traverses the document:\n * When it is the last or only segment of a YAML Path, it selects every leaf\n node from the remainder of the document's tree: `/shows/**`\n * When another segment follows, it matches every node within the remainder\n of the document's tree for which the following (and subsequent) segments\n match: `/shows/**/name/Star*`\n* Search Keywords: Advanced search capabilities not otherwise possible using\n other YAML Path segments. Taking the form of `[KEYWORD(PARAMETERS)]`, these\n keywords are\n [deeply explored on the Wiki](https://github.com/wwkimball/yamlpath/wiki/Search-Keywords)\n and include:\n * `[distinct(NAME)]`: Match exactly one of every value within collections,\n discarding duplicates; i.e.: [1, 2, 2, 3] has distinct values, [1, 2, 3]\n * `[has_child(NAME)]`: Match nodes having a named child key\n * `[max([NAME])]`: Match nodes having the maximum value\n * `[min([NAME])]`: Match nodes having the minimum value\n * `[name()]`: Match only the name of the present node, discarding all\n children\n * `[parent([STEPS])]`, Step up 1-N levels in the document from the present\n node\n * `[unique(NAME)]`: Match only values which have no duplicates within\n collections; i.e.: [1, 2, 2, 3] has unique values, [1, 3]\n* Collectors: Placing any portion of the YAML Path within parenthesis defines a\n virtual list collector, like `(YAML Path)`; concatenation, exclusion, and\n intersection operators are supported -- `+`, `-`, and `&`, respectively --\n along with nesting, like `(...)-((...)+(...))&(...)`\n* Complex combinations:\n `some::deep.hierarchy[with!=\"\"].'any.valid'[.=~/(yaml|json)/][data%structure].or.complexity[4].2`\n or `/some::deep/hierarchy[with!=\"\"]/any*.*valid[.=~/(yaml|json)/][data%structure]/or/compl*xity[4]/2/**`\n\nThis implementation of YAML Path encourages creativity. Use whichever notation\nand segment types that make the most sense to you in each application.\n\nThe [project Wiki provides more illustrative details of YAML Path Segments](https://github.com/wwkimball/yamlpath/wiki/Segments-of-a-YAML-Path).\n\n## Installing\n\nSome OS distributions offer some versions of yamlpath -- and its dependencies\n-- via packages. While these versions of yamlpath are often outdated, they can\nbe convenient to install using your OS' native package manager (`apt`, `yum`,\n`npm`, and such). Otherwise, Python's own package manager `pip` will always\noffer the latest version of yamlpath and -- even better -- can be isolated to\nephemeral or longer-lasting virtual Python environments.\n\n### Requirements\n\nThis project requires [Python](https://www.python.org/) 3. It is rigorously\ntested against Pythons 3.7 through 3.11 (support for Python 3.6 was dropped\n10 FEB 2024). Most operating systems and distributions have access to Python\n3 even if only Python 2 -- or no Python, at all -- came pre-installed. It is\ngenerally safe to have more than one version of Python on your system at the\nsame time, especially when using\n[virtual Python environments](https://docs.python.org/3/library/venv.html).\n\n*yamlpath* depends on *ruamel.yaml* (derived from and greatly extending PyYAML)\nand *python-dateutil*. When using OS-native packages or `pip`, you do not need\nto pre-install these libraries yourself except under extraordinary\ncircumstances like using very old versions of `pip` or its own dependency,\n*setuptools*.\n\n### Using pip\n\nLike most others, this Python project is published to [PyPI](https://pypi.org/)\nso that it can be easily installed via Python's `pip` command (or its\nversion-specific `pip3`, `pip3.7`, and such depending on how your Python was\ninstalled).\n\nPython's `pip` command is ever-changing. It is updated very frequently. This\ncommand further depends on other libraries to do its job, namely *setuptools*.\nIt so happens that *setuptools* is also updated very frequently. Both of these\nare separate from Python itself, despite versions of them being pre-installed\nwith Python. It is your responsibility to keep `pip` and *setuptools*\nup-to-date. When `pip` or *setuptools* become outdated, _you will experience\nerrors_ when trying to install newer Python packages like *yamlpath* **unless\nyou preinstall such packages' dependencies**. In the case of *yamlpath*, this\nmeans you'd need to preinstall *ruamel.yaml* and *python-dateutil* if you\ncannot or choose not to upgrade `pip` and/or *setuptools*.\n\nAs long as your `pip` and *setuptools* are up-to-date, installing *yamlpath* is\nas simple as a single command (the \"3.7\" suffix to the `pip` command is\noptional, depending on how your Python 3 was installed):\n\n```shell\npip3.7 install yamlpath\n```\n\n#### Very Old Versions of pip or its setuptools Dependency\n\nVery old versions of Python 3 ship with seriously outdated versions of `pip` and\nits *setuptools* dependency. When using versions of `pip` older than **18.1**\nor *setuptools* older than version **46.4.0**, you will not be able to install\n*yamlpath* with a single command. In this case, you have two options: either\npre-install *ruamel.yaml* and *python-dateutil* before installing *yamlpath* or\nupdate `pip` and/or *setuptools* to at least the minimum required versions so\n`pip` can auto-determine and install dependencies. This issue is not unique to\n*yamlpath*.\n\nUpgrading `pip` and *setuptools* is trivially simple as long as you have\nsufficient access rights to do so on your local machine. Depending on your\nsituation, you may need to prefix these with `sudo` and/or you may need to\nsubstitute `python3` and `pip3` for `python` and `pip`, or even `python3.7` and\n`pip3.7` (or another specific version of Python 3), respectively. To reiterate\nthat this project requires Python 3, these sample commands will be\ndemonstrated using such prefixes:\n\n```shell\npython3.7 -m pip install --upgrade pip\npip3.7 install --upgrade setuptools\n```\n\nWhen you cannot or will not update `pip` or *setuptools*, just pre-install\n*ruamel.yaml* and *python-dateutil* before yamlpath. Each must be installed\nseparately and in order, like this (you **cannot** combine these installations\ninto a single command):\n\n```shell\npip3.7 install ruamel.yaml python-dateutil\npip3.7 install yamlpath\n```\n\nThe downside to choosing this manual installation path is that you may end up\nwith an incompatible version of *ruamel.yaml* or *python-dateutil*. This will\nmanifest either as an inability to install *yamlpath* at all, or only certain\nversions of *yamlpath*, or *yamlpath* may experience unexpected errors caused\nby the incompatible code. For the best experience, you are strongly encouraged\nto just keep `pip` and *setuptools* up-to-date, particularly as a routine part\nof installing any new Python packages.\n\n### Installing EYAML (Optional)\n\nEYAML support is entirely optional. You do not need EYAML to use YAML Path.\nThat YAML Path supports EYAML is a service to a substantial audience: Puppet\nusers. At the time of this writing, EYAML (classified as a Hiera\nback-end/plug-in) is available only as a Ruby Gem. That said, it provides a\ncommand-line tool, `eyaml`, which can be employed by this otherwise Python\nproject. To enjoy EYAML support, install compatible versions of ruby and\nrubygems, then execute:\n\n```shell\ngem install hiera-eyaml\n```\n\nIf this puts the `eyaml` command on your system `PATH`, nothing more need be\ndone apart from generating or obtaining your encryption keys. Otherwise, you\ncan tell YAML Path library and tools where to find the `eyaml` command.\n\n## Based on ruamel.yaml\n\nIn order to support the best available YAML editing capability (so called,\nround-trip editing with support for comment preservation), this project is based\non [ruamel.yaml](https://sourceforge.net/projects/ruamel-yaml/) for\nPython 3. While ruamel.yaml is based on PyYAML --\nPython's \"standard\" YAML library -- ruamel.yaml is [objectively better than\nPyYAML](https://yaml.readthedocs.io/en/latest/pyyaml.html), which lacks critical\nround-trip editing capabilities as well as up-to-date YAML/Compatible data\nparsing capabilities (at the time of this writing).\n\nShould PyYAML ever merge with -- or at least, catch up with -- ruamel.yaml, this\nproject can be (lightly) adapted to depend on it, instead. These conversations\nmay offer some insight into when or whether this might happen:\n\n* [Is this time to pass the baton?](https://github.com/yaml/pyyaml/issues/31)\n* [Rebase off ruamel? - many new valuable features](https://github.com/yaml/pyyaml/issues/46)\n\n## The Files of This Project\n\nThis repository contains:\n\n1. Generally-useful Python library files. These contain the reusable core of\n this project's YAML Path capabilities.\n2. Some implementations of those libraries, exhibiting their capabilities and\n simple-to-use APIs as command-line tools.\n3. Various support, documentation, and build files.\n\n### Command-Line Tools\n\nThis project provides some command-line tool implementations which utilize YAML\nPath. For some use-case examples of these tools,\n[see below](#basic-usage--command-line-tools).\n\nThe supplied command-line tools include:\n\n* [eyaml-rotate-keys](yamlpath/commands/eyaml_rotate_keys.py)\n\n```text\nusage: eyaml-rotate-keys [-h] [-V] [-d | -v | -q] [-b] [-x EYAML]\n -i OLDPRIVATEKEY -c OLDPUBLICKEY\n -r NEWPRIVATEKEY -u NEWPUBLICKEY\n YAML_FILE [YAML_FILE ...]\n\nRotates the encryption keys used for all EYAML values within a set of YAML\nfiles, decrypting with old keys and re-encrypting using replacement keys.\n\npositional arguments:\n YAML_FILE one or more YAML files containing EYAML values\n\noptional arguments:\n -h, --help show this help message and exit\n -V, --version show program's version number and exit\n -d, --debug output debugging details\n -v, --verbose increase output verbosity\n -q, --quiet suppress all output except errors\n -b, --backup save a backup of each modified YAML_FILE with an extra\n .bak file-extension\n -x EYAML, --eyaml EYAML\n the eyaml binary to use when it isn't on the PATH\n\nEYAML_KEYS:\n All key arguments are required\n\n -r NEWPRIVATEKEY, --newprivatekey NEWPRIVATEKEY\n the new EYAML private key\n -u NEWPUBLICKEY, --newpublickey NEWPUBLICKEY\n the new EYAML public key\n -i OLDPRIVATEKEY, --oldprivatekey OLDPRIVATEKEY\n the old EYAML private key\n -c OLDPUBLICKEY, --oldpublickey OLDPUBLICKEY\n the old EYAML public key\n\nAny YAML_FILEs lacking EYAML values will not be modified (or backed up, even\nwhen -b/--backup is specified).\n```\n\n* [yaml-diff](yamlpath/commands/yaml_diff.py)\n\n```text\nusage: yaml-diff [-h] [-V] [-a] [-s | -o]\n [-t ['.', '/', 'auto', 'dot', 'fslash']] [-x EYAML]\n [-r PRIVATEKEY] [-u PUBLICKEY] [-E] [-d | -v | -q]\n YAML_FILE YAML_FILE\n\nCalculate the functional difference between two YAML/JSON/Compatible\ndocuments. Immaterial differences (which YAML/JSON parsers discard) are\nignored. EYAML can be employed to compare encrypted values.\n\npositional arguments:\n YAML_FILE exactly two YAML/JSON/compatible files to compare; use\n - to read one document from STDIN\n\noptional arguments:\n -h, --help show this help message and exit\n -V, --version show program's version number and exit\n -a, --sync-arrays Synchronize array elements before comparing them,\n resulting only in ADD, DELETE, and SAME differences\n (no CHANGEs because the positions of elements are\n disregarded); Array-of-Hash elements must completely\n and perfectly match or they will be deemed additions\n or deletions\n -s, --same Show all nodes which are the same in addition to\n differences\n -o, --onlysame Show only nodes which are the same, still reporting\n that differences exist -- when they do -- with an\n exit-state of 1\n -t ['.', '/', 'auto', 'dot', 'fslash'], --pathsep ['.', '/', 'auto', 'dot', 'fslash']\n indicate which YAML Path separator to use when\n rendering results; default=dot\n -d, --debug output debugging details\n -v, --verbose increase output verbosity\n -q, --quiet suppress all output except system errors\n\nEYAML options:\n Left unset, the EYAML keys will default to your system or user defaults.\n Both keys must be set either here or in your system or user EYAML\n configuration file when using EYAML.\n\n -x EYAML, --eyaml EYAML\n the eyaml binary to use when it isn't on the PATH\n -r PRIVATEKEY, --privatekey PRIVATEKEY\n EYAML private key\n -u PUBLICKEY, --publickey PUBLICKEY\n EYAML public key\n -E, --ignore-eyaml-values\n Do not use EYAML to compare encrypted data; rather,\n treat ENC[...] values as regular strings\n\nOnly one YAML_FILE may be the - pseudo-file for reading from STDIN. For more\ninformation about YAML Paths, please visit\nhttps://github.com/wwkimball/yamlpath.\n```\n\n* [yaml-get](yamlpath/commands/yaml_get.py)\n\n```text\nusage: yaml-get [-h] [-V] -p YAML_PATH\n [-t ['.', '/', 'auto', 'dot', 'fslash']] [-S] [-x EYAML]\n [-r PRIVATEKEY] [-u PUBLICKEY] [-d | -v | -q]\n [YAML_FILE]\n\nRetrieves one or more values from a YAML/JSON/Compatible file at a specified\nYAML Path. Output is printed to STDOUT, one line per result. When a result is\na complex data-type (Array or Hash), a JSON dump is produced to represent it.\nEYAML can be employed to decrypt the values.\n\npositional arguments:\n YAML_FILE the YAML file to query; omit or use - to read from\n STDIN\n\noptional arguments:\n -h, --help show this help message and exit\n -V, --version show program's version number and exit\n -t ['.', '/', 'auto', 'dot', 'fslash'], --pathsep ['.', '/', 'auto', 'dot', 'fslash']\n indicate which YAML Path separator to use when\n rendering results; default=dot\n -S, --nostdin Do not implicitly read from STDIN, even when YAML_FILE\n is not set and the session is non-TTY\n -d, --debug output debugging details\n -v, --verbose increase output verbosity\n -q, --quiet suppress all output except errors\n\nrequired settings:\n -p YAML_PATH, --query YAML_PATH\n YAML Path to query\n\nEYAML options:\n Left unset, the EYAML keys will default to your system or user defaults.\n Both keys must be set either here or in your system or user EYAML\n configuration file when using EYAML.\n\n -x EYAML, --eyaml EYAML\n the eyaml binary to use when it isn't on the PATH\n -r PRIVATEKEY, --privatekey PRIVATEKEY\n EYAML private key\n -u PUBLICKEY, --publickey PUBLICKEY\n EYAML public key\n\nFor more information about YAML Paths, please visit\nhttps://github.com/wwkimball/yamlpath.\n```\n\n* [yaml-merge](yamlpath/commands/yaml_merge.py)\n\n```text\nusage: yaml-merge [-h] [-V] [-c CONFIG] [-a {stop,left,right,rename}]\n [-A {all,left,right,unique}] [-H {deep,left,right}]\n [-O {all,deep,left,right,unique}] [-m YAML_PATH]\n [-o OUTPUT | -w OVERWRITE] [-b] [-D {auto,json,yaml}] [-S]\n [-d | -v | -q]\n [YAML_FILE [YAML_FILE ...]]\n\nMerges two or more YAML/JSON/Compatible files together.\n\npositional arguments:\n YAML_FILE one or more YAML files to merge, order-significant;\n omit or use - to read from STDIN\n\noptional arguments:\n -h, --help show this help message and exit\n -V, --version show program's version number and exit\n -c CONFIG, --config CONFIG\n INI syle configuration file for YAML Path specified\n merge control options\n -a {stop,left,right,rename}, --anchors {stop,left,right,rename}\n means by which Anchor name conflicts are resolved\n (overrides [defaults]anchors set via --config|-c and\n cannot be overridden by [rules] because Anchors apply\n to the whole file); default=stop\n -A {all,left,right,unique}, --arrays {all,left,right,unique}\n default means by which Arrays are merged together\n (overrides [defaults]arrays but is overridden on a\n YAML Path basis via --config|-c); default=all\n -H {deep,left,right}, --hashes {deep,left,right}\n default means by which Hashes are merged together\n (overrides [defaults]hashes but is overridden on a\n YAML Path basis in [rules] set via --config|-c);\n default=deep\n -O {all,deep,left,right,unique}, --aoh {all,deep,left,right,unique}\n default means by which Arrays-of-Hashes are merged\n together (overrides [defaults]aoh but is overridden on\n a YAML Path basis in [rules] set via --config|-c);\n default=all\n -m YAML_PATH, --mergeat YAML_PATH\n YAML Path indicating where in left YAML_FILE the right\n YAML_FILE content is to be merged; default=/\n -o OUTPUT, --output OUTPUT\n Write the merged result to the indicated nonexistent\n file\n -w OVERWRITE, --overwrite OVERWRITE\n Write the merged result to the indicated file; will\n replace the file when it already exists\n -b, --backup save a backup OVERWRITE file with an extra .bak\n file-extension; applies only to OVERWRITE\n -D {auto,json,yaml}, --document-format {auto,json,yaml}\n Force the merged result to be presented in one of the\n supported formats or let it automatically match the\n known file-name extension of OUTPUT|OVERWRITE (when\n provided), or match the type of the first document;\n default=auto\n -S, --nostdin Do not implicitly read from STDIN, even when there are\n no - pseudo-files in YAML_FILEs with a non-TTY session\n -d, --debug output debugging details\n -v, --verbose increase output verbosity\n -q, --quiet suppress all output except errors (implied when\n -o|--output is not set)\n\n The CONFIG file is an INI file with up to three sections:\n [defaults] Sets equivalents of -a|--anchors, -A|--arrays,\n -H|--hashes, and -O|--aoh.\n [rules] Each entry is a YAML Path assigning -A|--arrays,\n -H|--hashes, or -O|--aoh for precise nodes.\n [keys] Wherever -O|--aoh=DEEP, each entry is treated as a\n record with an identity key. In order to match RHS\n records to LHS records, a key must be known and is\n identified on a YAML Path basis via this section.\n Where not specified, the first attribute of the first\n record in the Array-of-Hashes is presumed the identity\n key for all records in the set.\n\n The left-to-right order of YAML_FILEs is significant. Except\n when this behavior is deliberately altered by your options, data\n from files on the right overrides data in files to their left.\n Only one input file may be the - pseudo-file (read from STDIN).\n When no YAML_FILEs are provided, - will be inferred as long as you\n are running this program without a TTY (unless you set\n --nostdin|-S). Any file, including input from STDIN, may be a\n multi-document YAML or JSON file.\n\n For more information about YAML Paths, please visit\n https://github.com/wwkimball/yamlpath.\n```\n\n* [yaml-paths](yamlpath/commands/yaml_paths.py)\n\n```text\nusage: yaml-paths [-h] [-V] -s EXPRESSION [-c EXPRESSION] [-m] [-L] [-F] [-X]\n [-P] [-n] [-t ['.', '/', 'auto', 'dot', 'fslash']]\n [-i | -k | -K] [-a] [-A | -Y | -y | -l] [-e] [-x EYAML]\n [-r PRIVATEKEY] [-u PUBLICKEY] [-S] [-d | -v | -q]\n [YAML_FILE [YAML_FILE ...]]\n\nReturns zero or more YAML Paths indicating where in given YAML/JSON/Compatible\ndata one or more search expressions match. Values, keys, and/or anchors can be\nsearched. EYAML can be employed to search encrypted values.\n\npositional arguments:\n YAML_FILE one or more YAML files to search; omit or use - to\n read from STDIN\n\noptional arguments:\n -h, --help show this help message and exit\n -V, --version show program's version number and exit\n -c EXPRESSION, --except EXPRESSION\n except results matching this search expression; can be\n set more than once\n -m, --expand expand matching parent nodes to list all permissible\n child leaf nodes (see \"reference handling options\" for\n restrictions)\n -t ['.', '/', 'auto', 'dot', 'fslash'], --pathsep ['.', '/', 'auto', 'dot', 'fslash']\n indicate which YAML Path separator to use when\n rendering results; default=dot\n -a, --refnames also search the names of &anchor and *alias references\n -S, --nostdin Do not implicitly read from STDIN, even when there are\n no - pseudo-files in YAML_FILEs with a non-TTY session\n -d, --debug output debugging details\n -v, --verbose increase output verbosity\n -q, --quiet suppress all non-result output except errors\n\nrequired settings:\n -s EXPRESSION, --search EXPRESSION\n the search expression; can be set more than once\n\nresult printing options:\n -L, --values print the values or elements along with each YAML Path\n (complex results are emitted as JSON; use --expand to\n emit only simple values)\n -F, --nofile omit source file path and name decorators from the\n output (applies only when searching multiple files)\n -X, --noexpression omit search expression decorators from the output\n -P, --noyamlpath omit YAML Paths from the output (useful with --values\n or to indicate whether a file has any matches without\n printing them all, perhaps especially with\n --noexpression)\n -n, --noescape omit escape characters from special characters in\n printed YAML Paths; this is unsafe for feeding the\n resulting YAML Paths into other YAML Path commands\n because the symbols that would be escaped have special\n meaning to YAML Path processors\n\nkey name searching options:\n -i, --ignorekeynames (default) do not search key names\n -k, --keynames search key names in addition to values and array\n elements\n -K, --onlykeynames only search key names (ignore all values and array\n elements)\n\nreference handling options:\n Indicate how to treat anchor and alias references. An anchor is an\n original, reusable key or value. All aliases become replaced by the\n anchors they reference when YAML data is read. These options specify how\n to handle this duplication of keys and values. Note that the default\n behavior includes all aliased keys but not aliased values.\n\n -A, --anchorsonly include only original matching key and value anchors\n in results, discarding all aliased keys and values\n (including child nodes)\n -Y, --allowkeyaliases\n (default) include matching key aliases, permitting\n search traversal into their child nodes\n -y, --allowvaluealiases\n include matching value aliases (does not permit search\n traversal into aliased keys)\n -l, --allowaliases include all matching key and value aliases\n\nEYAML options:\n Left unset, the EYAML keys will default to your system or user defaults.\n Both keys must be set either here or in your system or user EYAML\n configuration file when using EYAML.\n\n -e, --decrypt decrypt EYAML values in order to search them\n (otherwise, search the encrypted blob)\n -x EYAML, --eyaml EYAML\n the eyaml binary to use when it isn't on the PATH\n -r PRIVATEKEY, --privatekey PRIVATEKEY\n EYAML private key\n -u PUBLICKEY, --publickey PUBLICKEY\n EYAML public key\n\nA search or exception EXPRESSION takes the form of a YAML Path search operator\n-- %, $, =, ^, >, <, >=, <=, =~, or ! -- followed by the search term, omitting\nthe left-hand operand. For more information about YAML Paths, please visit\nhttps://github.com/wwkimball/yamlpath/wiki. To report issues with this tool or\nto request enhancements, please visit\nhttps://github.com/wwkimball/yamlpath/issues.\n```\n\n* [yaml-set](yamlpath/commands/yaml_set.py)\n\n```text\nusage: yaml-set [-h] [-V] -g YAML_PATH\n [-a VALUE | -A ANCHOR | -f FILE | -i | -R LENGTH | -N | -D]\n [-F {bare,boolean,default,dquote,float,folded,int,literal,squote}]\n [-c CHECK] [-s YAML_PATH] [-m] [-b]\n [-t ['.', '/', 'auto', 'dot', 'fslash']] [-M CHARS] [-H ANCHOR]\n [-T TAG] [-e] [-x EYAML] [-r PRIVATEKEY] [-u PUBLICKEY] [-S]\n [-d | -v | -q]\n [YAML_FILE]\n\nChanges one or more Scalar values in a YAML/JSON/Compatible document at a\nspecified YAML Path. Matched values can be checked before they are replaced to\nmitigate accidental change. When matching singular results, the value can be\narchived to another key before it is replaced. Further, EYAML can be employed to\nencrypt the new values and/or decrypt an old value before checking it.\n\npositional arguments:\n YAML_FILE the YAML file to update; omit or use - to read from\n STDIN\n\noptional arguments:\n -h, --help show this help message and exit\n -V, --version show program's version number and exit\n -F {bare,boolean,default,dquote,float,folded,int,literal,squote}, --format {bare,boolean,default,dquote,float,folded,int,literal,squote}\n override automatic formatting of the new value\n -c CHECK, --check CHECK\n check the value before replacing it\n -s YAML_PATH, --saveto YAML_PATH\n save the old value to YAML_PATH before replacing it;\n implies --mustexist\n -m, --mustexist require that the --change YAML_PATH already exist in\n YAML_FILE\n -b, --backup save a backup YAML_FILE with an extra .bak file-\n extension\n -t ['.', '/', 'auto', 'dot', 'fslash'], --pathsep ['.', '/', 'auto', 'dot', 'fslash']\n indicate which YAML Path separator to use when rendering\n results; default=dot\n -M CHARS, --random-from CHARS\n characters from which to build a value for --random;\n default=all upper- and lower-case letters and all digits\n -H ANCHOR, --anchor ANCHOR\n when --aliasof|-A points to a value which is not already\n Anchored, a new Anchor with this name is created;\n renames an existing Anchor if already set\n -T TAG, --tag TAG assign a custom YAML (data-type) tag to the changed\n nodes; can be used without other input options to assign\n or change a tag\n -S, --nostdin Do not implicitly read from STDIN, even when there is no\n YAML_FILE with a non-TTY session\n -d, --debug output debugging details\n -v, --verbose increase output verbosity\n -q, --quiet suppress all output except errors\n\nrequired settings:\n -g YAML_PATH, --change YAML_PATH\n YAML Path where the target value is found\n\ninput options:\n -a VALUE, --value VALUE\n set the new value from the command-line instead of STDIN\n -A ANCHOR, --aliasof ANCHOR\n set the value as a YAML Alias of an existing Anchor, by\n name (merely copies the target value for non-YAML files)\n -f FILE, --file FILE read the new value from file (discarding any trailing\n new-lines)\n -i, --stdin accept the new value from STDIN (best for sensitive\n data)\n -R LENGTH, --random LENGTH\n randomly generate a replacement value of a set length\n -N, --null sets the value to null\n -D, --delete delete rather than change target node(s); implies\n --mustexist|-m\n\nEYAML options:\n Left unset, the EYAML keys will default to your system or user defaults. You\n do not need to supply a private key unless you enable --check and the old\n value is encrypted.\n\n -e, --eyamlcrypt encrypt the new value using EYAML\n -x EYAML, --eyaml EYAML\n the eyaml binary to use when it isn't on the PATH\n -r PRIVATEKEY, --privatekey PRIVATEKEY\n EYAML private key\n -u PUBLICKEY, --publickey PUBLICKEY\n EYAML public key\n\nWhen no changes are made, no backup is created, even when -b/--backup is\nspecified. For more information about YAML Paths, please visit\nhttps://github.com/wwkimball/yamlpath/wiki. To report issues with this tool or\nto request enhancements, please visit\nhttps://github.com/wwkimball/yamlpath/issues.\n```\n\n* [yaml-validate](yamlpath/commands/yaml_validate.py)\n\n```text\nusage: yaml-validate [-h] [-V] [-S] [-d | -v | -q] [YAML_FILE [YAML_FILE ...]]\n\nValidate YAML, JSON, and compatible files.\n\npositional arguments:\n YAML_FILE one or more single- or multi-document YAML/JSON/compatible\n files to validate; omit or use - to read from STDIN\n\noptional arguments:\n -h, --help show this help message and exit\n -V, --version show program's version number and exit\n -S, --nostdin Do not implicitly read from STDIN, even when there are no -\n pseudo-files in YAML_FILEs with a non-TTY session\n -d, --debug output debugging details\n -v, --verbose increase output verbosity (show valid documents)\n -q, --quiet suppress all output except system errors\n\nExcept when suppressing all report output with --quiet|-q, validation issues\nare printed to STDOUT (not STDERR). Further, the exit-state will report 0 when\nthere are no issues, 1 when there is an issue with the supplied command-line\narguments, or 2 when validation has failed for any document.\n```\n\n### Libraries\n\nWhile there are several supporting library files like enumerations, types, and\nexceptions, the most interesting library files include:\n\n* [yamlpath.py](yamlpath/yamlpath.py) -- The core YAML Path parser logic.\n* [processor.py](yamlpath/processor.py) -- Processes YAMLPath instances to read\n or write data to YAML/Compatible sources.\n* [eyamlprocessor.py](yamlpath/eyaml/eyamlprocessor.py) -- Extends the\n Processor class to support EYAML data encryption and decryption.\n* [merger.py](yamlpath/merger/merger.py) -- The core document merging logic.\n\n## Basic Usage\n\nThe files of this project can be used either as command-line tools or as\nlibraries to supplement your own work.\n\n### Basic Usage: Command-Line Tools\n\nThe command-line tools are self-documented and [their documentation is captured\nabove](#command-line-tools) for easy reference. Simply pass `--help` to them in\norder to obtain the same detailed documentation.\n\nPlease review [the comprehensive test_commands_*.py unit tests](/tests/) to\nexplore samples of YAML files and the many ways these tools help get and set\ntheir data.\n\nThe following are some simple examples of their typical use-cases.\n\n#### Rotate Your EYAML Keys\n\nIf the eyaml command is already on your PATH (if not, be sure to also supply\nthe optional `--eyaml` or `-x` argument):\n\n```shell\neyaml-rotate-keys \\\n --oldprivatekey=~/old-keys/private_key.pkcs7.pem \\\n --oldpublickey=~/old-keys/public_key.pkcs7.pem \\\n --newprivatekey=~/new-keys/private_key.pkcs7.pem \\\n --newpublickey=~/new-keys/public_key.pkcs7.pem \\\n my_1st_yaml_file.yaml my_2nd_yaml_file.eyaml ... my_Nth_yaml_file.yaml\n```\n\nYou could combine this with `find` and `xargs` if your E/YAML file are\ndispersed through a directory hierarchy, as with Hiera data.\n\n#### Get the Differences Between Two Documents\n\nFor routine use:\n\n```shell\nyaml-diff yaml_file1.yaml yaml_file2.yaml\n```\n\nOutput is very similar to that of standard GNU diff against text files, except\nit is generated against the *data* within the input files. This excludes\nevaluating purely structural and immaterial differences between them like value\ndemarcation, white-space, and comments. When you need to evaluate the two\nfiles as if they were just text files, use GNU diff or any of its clones.\n\nTo see all identical entries along with differences:\n\n```shell\nyaml-diff --same yaml_file1.yaml yaml_file2.yaml\n```\n\nTo see *only* entries which are identical between the documents:\n\n```shell\nyaml-diff --onlysame yaml_file1.yaml yaml_file2.yaml\n```\n\nAdvanced: Arrays can be evaluated such that they are synchronized before\nevaluation. Rather than compare elements by identical index in both\ndocuments -- reporting differences between them as changes and any additional\nelements as additions or deletions -- they can instead be compared by matching\nup all identical elements and then reporting only those values which are unique\nto either document (and optionally identical matches).\n\n```shell\nyaml-diff --sync-arrays yaml_file1.yaml yaml_file2.yaml\n```\n\n#### Get a YAML/JSON/Compatible Value\n\nAt its simplest:\n\n```shell\nyaml-get \\\n --query=see.documentation.above.for.many.samples \\\n my_yaml_file.yaml\n```\n\n#### Search For YAML Paths\n\nSimplest use:\n\n```shell\nyaml-paths \\\n --search=%word \\\n /some/directory/*.yaml\n```\n\nSearch for multiple expressions and exclude unwanted results:\n\n```shell\nyaml-paths \\\n --search=^another \\\n --search=$word \\\n --except=%bad \\\n /some/directory/*.yaml\n```\n\nReturn all leaf nodes under matching parents (most useful when matching against Hash keys and you only want the original leaf nodes beneath them):\n\n```shell\nyaml-paths \\\n --expand \\\n --keynames \\\n --search==parent_node \\\n /some/directory/*.yaml\n```\n\n#### Change a YAML/JSON/Compatible Value\n\nFor a no-frills change to a YAML file with deeply nested Hash structures:\n\n```shell\nyaml-set \\\n --change=see.documentation.above.for.many.samples \\\n --value=\"New Value\" \\\n my_yaml_file.yaml\n```\n\nTo rotate a password, preserving the old password perhaps so your automation can\napply the new password to your application(s):\n\n```shell\nyaml-set \\\n --mustexist \\\n --change=the.new.password \\\n --saveto=the.old.password \\\n --value=\"New Password\" \\\n my_yaml_file.yaml\n```\n\nFor the extremely cautious, you could check the old password before rotating\nit and save a backup of the original file:\n\n```shell\nyaml-set \\\n --mustexist \\\n --change=the.new.password \\\n --saveto=the.old.password \\\n --check=\"Old Password\" \\\n --value=\"New Password\" \\\n --backup \\\n my_yaml_file.yaml\n```\n\nYou can also add EYAML encryption (assuming the `eyaml` command is on your\nPATH; if not, you can pass `--eyaml` to specify its location). In this example,\nI add the optional `--format=folded` so that the long EYAML value is broken up\ninto a multi-line value rather than one very long string. This is the preferred\nformat for human legibility as well as EYAML consumers like\n[Puppet](http://puppet.com). Note that `--format` has several other settings\nand applies only to new values.\n\n```shell\nyaml-set \\\n --change=the.new.password \\\n --mustexist \\\n --saveto=the.old.password \\\n --check=\"Old Password\" \\\n --value=\"New Password\" \\\n --eyamlcrypt \\\n --format=folded \\\n --backup \\\n my_yaml_file.yaml\n```\n\nYou can even tell EYAML which keys to use, if not your default system or user\nkeys:\n\n```shell\nyaml-set \\\n --change=the.new.password \\\n --mustexist \\\n --saveto=the.old.password \\\n --check=\"Old Password\" \\\n --value=\"New Password\" \\\n --eyamlcrypt \\\n --format=folded \\\n --privatekey=/secret/keys/private_key.pkcs7.pem \\\n --publickey=/secret/keys/public_key.pkcs7.pem \\\n --backup \\\n my_yaml_file.yaml\n```\n\nNote that for even greater security scenarios, you can keep the new value off of\nyour command-line, process list, and command history by swapping out `--value`\nfor one of `--stdin`, `--file`, or even `--random LENGTH` (use Python's\nstrongest random value generator if you don't need to specify the replacement\nvalue in advance).\n\n#### Merge YAML/JSON/Compatible Files\n\nAt its simplest, the `yaml-merge` command accepts two or more input files and\nmerges them together from left-to-right, writing the result to STDOUT:\n\n```shell\nyaml-merge leftmost.yaml middle.yaml right.json\n```\n\nIf you'd rather write the results to a new output file (which must not already\nexist):\n\n```shell\nyaml-merge \\\n --output=newfile.yaml \\\n leftmost.yaml \\\n middle.yaml \\\n right.json\n```\n\nShould you wish to merge the content of the files into a specific location (or\neven multiple locations) within the leftmost document, specify a YAML Path via\nthe `--mergeat` or `-m` argument:\n\n```shell\nyaml-merge \\\n --mergeat=/anywhere/within/the/document \\\n leftmost.yaml \\\n middle.yaml \\\n right.json\n```\n\nTo write arbitrary data from STDIN into a document, use the `-` pseudo-file:\n\n```shell\necho \"{arbitrary: [document, structure]}\" | yaml-merge target.yaml -\n```\n\nCombine `--mergeat` or `-m` with the STDIN pseudo-file to control where the\ndata is to be written:\n\n```shell\necho \"{arbitrary: [document, structure]}\" | \\\n yaml-merge \\\n --mergeat=/anywhere/within/the/document \\\n target.yaml -\n```\n\nThere are many options for precisely controlling how the merge is performed,\nincluding the ability to specify complex rules on a YAML Path basis via a\nconfiguration file. Review the command's `--help` or the\n[related Wiki](https://github.com/wwkimball/yamlpath/wiki/yaml-merge) for\nmore detail.\n\n#### Validate YAML/JSON/Compatible Documents\n\nValidating the structure of YAML, JSON, and compatible files is as simple as\nrunning:\n\n```shell\nyaml-validate /path/to/any/files.yaml /path/to/more/files.json\n```\n\nIn this default configuration, the command will output no report when all input\ndocuments are valid. It will also report an exit-state of zero (0). Should\nthere be any validation errors, each will be printed to the screen and the\nexit-state will be 2. An exit-state of 1 means your command-line arguments\nwere incorrect and an appropritae user error message will be displayed.\n\nWhen there are validation issues, the offending file-name(s) and sub-document\nindex(es) (zero-based) will be displayed along with a detailed validation error\nmessage.\n\n### Basic Usage: Libraries\n\nAs for the libraries, they are also heavily documented and the example\nimplementations may perhaps serve as good copy-paste fodder (provided you give\ncredit to the source). That said, here's a general flow/synopsis.\n\n#### Initialize ruamel.yaml and These Helpers\n\nYour preferences may differ, but I use this setup for round-trip YAML parsing\nand editing with ruamel.yaml. When you need to process EYAML encrypted data,\nreplace `yamlpath.Processor` with `yamlpath.eyaml.EYAMLProcessor` and add error\nhandling for `yamlpath.eyaml.EYAMLCommandException`.\n\nNote also that these examples use `ConsolePrinter` to handle STDOUT and STDERR\nmessaging. You don't have to. However, some kind of logger must be passed to\nthese libraries so they can write messages _somewhere_. Your custom message\nhandler or logger must provide the same API as `ConsolePrinter`; review the\nheader documentation in [consoleprinter.py](yamlpath/wrappers/consoleprinter.py)\nfor details. Generally speaking, it would be trivial to write your own custom\nwrapper for Python's standard logging facilities if you require targets other\nthan STDOUT and STDERR.\n\n```python\nfrom types import SimpleNamespace\n\nfrom yamlpath.common import Parsers\nfrom yamlpath.wrappers import ConsolePrinter\nfrom yamlpath import Processor\n\n\n# The various classes of this library must be able to write messages somewhere\n# when things go bad. This project provides a CLI-centric logging class named\n# ConsolePrinter. Even when not writing a CLI tool, you must still configure\n# and pass ConsolePrinter or a class of your own with the same public API. For\n# just muting logging output -- except for unrecoverable errors -- you can use\n# this simple configuration object:\nlogging_args = SimpleNamespace(quiet=True, verbose=False, debug=False)\nlog = ConsolePrinter(logging_args)\n\n# Prep the YAML parser and round-trip editor (tweak to your needs). You do not\n# have to use Parsers.get_yaml_editor() but you must create a properly-\n# configured instance of ruamel.yaml.YAML.\nyaml = Parsers.get_yaml_editor()\n\n# At this point, you'd load or parse your YAML file, stream, or string. This\n# example demonstrates loading YAML data from an external file. You could also\n# use the same function to load data from STDIN or even a String variable. See\n# the Parser class for more detail.\nyaml_file = \"your-file.yaml\"\n(yaml_data, doc_loaded) = Parsers.get_yaml_data(yaml, log, yaml_file)\nif not doc_loaded:\n # There was an issue loading the file; an error message has already been\n # printed via ConsolePrinter.\n exit(1)\n\n# Pass the logging facility and parsed YAML data to the YAMLPath Processor\nprocessor = Processor(log, yaml_data)\n\n# At this point, the Processor is ready to handle YAML Paths\n```\n\n#### Searching for YAML Nodes\n\nThese libraries use [Generators](https://wiki.python.org/moin/Generators) to get\nnodes from parsed YAML data. Identify which node(s) to get via YAML Path\nstrings. You should also catch `yamlpath.exceptions.YAMLPathException`s\nunless you prefer Python's native stack traces. When using EYAML, you should\nalso catch `yamlpath.eyaml.exceptions.EYAMLCommandException`s for the same\nreason. Whether you are working with a single result or many, you should\nconsume the Generator output with a pattern similar to:\n\n```python\nfrom yamlpath import YAMLPath\nfrom yamlpath.exceptions import YAMLPathException\n\nyaml_path = YAMLPath(\"see.documentation.above.for.many.samples\")\ntry:\n for node_coordinate in processor.get_nodes(yaml_path, mustexist=True):\n log.debug(\"Got {} from '{}'.\".format(node_coordinate, yaml_path))\n # Do something with each node_coordinate.node (the actual data)\nexcept YAMLPathException as ex:\n # If merely retrieving data, this exception may be deemed non-critical\n # unless your later code absolutely depends upon a result.\n log.error(ex)\n```\n\n#### Changing Values\n\nAt its simplest, you only need to supply the the YAML Path to one or more nodes\nto update, and the value to apply to them. Catching\n`yamlpath.exceptions.YAMLPathException` is optional but usually preferred over\nallowing Python to dump the call stack in front of your users. When using\nEYAML, the same applies to `yamlpath.eyaml.exceptions.EYAMLCommandException`.\n\n```python\nfrom yamlpath.exceptions import YAMLPathException\n\ntry:\n processor.set_value(yaml_path, new_value)\nexcept YAMLPathException as ex:\n log.critical(ex, 119)\nexcept EYAMLCommandException as ex:\n log.critical(ex, 120)\n```\n\n#### Merging Documents\n\nA document merge naturally requires at least two documents. At the code-level,\nthis means two populated DOM objects (populated instances of `yaml_data` from\nabove). You do not need to use a `Processor` for merging. In the least amount\nof code, a merge looks like:\n\n```python\nfrom yamlpath.exceptions import YAMLPathException\nfrom yamlpath.merger.exceptions import MergeException\nfrom yamlpath.merger import Merger, MergerConfig\n\n# Obtain or build the lhs_data and rhs_data objects using get_yaml_data or\n# equivalent.\n\n# You'll still need to supply a logger and some arguments used by the merge\n# engine. For purely default behavior, you could create args as a bare\n# SimpleNamespace. Initialize the new Merger instance with the LHS document.\nmerger = Merger(log, lhs_data, MergerConfig(log, args))\n\n# Merge RHS into LHS\ntry:\n merger.merge_with(rhs_data)\nexcept MergeException as mex:\n log.critical(mex, 129)\nexcept YAMLPathException as yex:\n log.critical(yex, 130)\n\n# At this point, merger.data is the merged result; do what you will with it,\n# including merging more data into it. When you are ready to dump (write)\n# out the merged data, you must prepare the document and your\n# ruamel.yaml.YAML instance -- usually obtained from func.get_yaml_editor()\n# -- like this:\nmerger.prepare_for_dump(my_yaml_editor)\n```\n",
"bugtrack_url": null,
"license": "ISC",
"summary": "Command-line get/set/merge/validate/scan/convert/diff processors for YAML/JSON/Compatible data using powerful, intuitive, command-line friendly syntax",
"version": "3.8.2",
"project_urls": {
"Homepage": "https://github.com/wwkimball/yamlpath"
},
"split_keywords": [
"yaml",
"eyaml",
"json",
"yaml-path",
"diff",
"merge"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "31e17e05388333efcbc260a2577f50675f72dd2cfa7ef3be749fd4856fe4626c",
"md5": "d9d2e7d90f042d3d52cb449eba2b626e",
"sha256": "b7d116359cd641e7bf9fe66c537d808af72aeeccfab287cecdd4d80f4ac823b1"
},
"downloads": -1,
"filename": "yamlpath-3.8.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d9d2e7d90f042d3d52cb449eba2b626e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">3.7.0",
"size": 164121,
"upload_time": "2024-02-11T21:02:08",
"upload_time_iso_8601": "2024-02-11T21:02:08.060123Z",
"url": "https://files.pythonhosted.org/packages/31/e1/7e05388333efcbc260a2577f50675f72dd2cfa7ef3be749fd4856fe4626c/yamlpath-3.8.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3aa4eb264493c967e5dabf310c8d5bad6f435c6830eae087f1651e51d93aa065",
"md5": "6e440db9af9106719161f738a9b205a8",
"sha256": "4f30cc214b5085d4b0e7756e06c3af3ae589ecde9650d2ada7e1d345ec4fda4f"
},
"downloads": -1,
"filename": "yamlpath-3.8.2.tar.gz",
"has_sig": false,
"md5_digest": "6e440db9af9106719161f738a9b205a8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">3.7.0",
"size": 231570,
"upload_time": "2024-02-11T21:02:10",
"upload_time_iso_8601": "2024-02-11T21:02:10.402775Z",
"url": "https://files.pythonhosted.org/packages/3a/a4/eb264493c967e5dabf310c8d5bad6f435c6830eae087f1651e51d93aa065/yamlpath-3.8.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-02-11 21:02:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "wwkimball",
"github_project": "yamlpath",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "yamlpath"
}
JSON
