Yamale (ya·ma·lē)
=================
| :warning: Ensure that your schema definitions come from internal or trusted sources. Yamale does not protect against intentionally malicious schemas. |
|:------------|
<img src="https://github.com/23andMe/Yamale/blob/master/yamale.png?raw=true" alt="Yamale" width="400"/>
A schema and validator for YAML.
What's YAML? See the current spec [here](http://www.yaml.org/spec/1.2/spec.html) and an introduction
to the syntax [here](https://github.com/Animosity/CraftIRC/wiki/Complete-idiot's-introduction-to-yaml).
[](https://github.com/23andMe/Yamale/actions/workflows/run-tests.yml)
[](https://pypi.python.org/pypi/yamale)
Requirements
------------
* Python 3.8+
* PyYAML
* ruamel.yaml (optional)
Install
-------
### pip
```bash
$ pip install yamale
```
NOTE: Some platforms, e.g., Mac OS, may ship with only Python 2 and may not have pip installed.
Installation of Python 3 should also install pip. To preserve any system dependencies on default
software, consider installing Python 3 as a local package. Please note replacing system-provided
Python may disrupt other software. Mac OS users may wish to investigate MacPorts, homebrew, or
building Python 3 from source; in all three cases, Apple's Command Line Tools (CLT) for Xcode
may be required. See also [developers](#developers), below.
### Manual
1. Download Yamale from: https://github.com/23andMe/Yamale/archive/master.zip
2. Unzip somewhere temporary
3. Run `python setup.py install` (may have to prepend `sudo`)
Usage
-----
### Command line
Yamale can be run from the command line to validate one or many YAML files. Yamale will search the
directory you supply (current directory is default) for YAML files. Each YAML file it finds it will
look in the same directory as that file for its schema, if there is no schema Yamale will keep
looking up the directory tree until it finds one. If Yamale can not find a schema it will tell you.
Usage:
```bash
usage: yamale [-h] [-s SCHEMA] [-n CPU_NUM] [-p PARSER] [--no-strict] [PATH]
Validate yaml files.
positional arguments:
PATH folder to validate. Default is current directory.
optional arguments:
-h, --help show this help message and exit
-s SCHEMA, --schema SCHEMA
filename of schema. Default is schema.yaml.
-n CPU_NUM, --cpu-num CPU_NUM
number of CPUs to use. Default is 4.
-p PARSER, --parser PARSER
YAML library to load files. Choices are "ruamel" or
"pyyaml" (default).
--no-strict Disable strict mode, unexpected elements in the data
will be accepted.
```
### API
There are several ways to feed Yamale schema and data files. The simplest way is to let Yamale take
care of reading and parsing your YAML files.
All you need to do is supply the files' path:
```python
# Import Yamale and make a schema object:
import yamale
schema = yamale.make_schema('./schema.yaml')
# Create a Data object
data = yamale.make_data('./data.yaml')
# Validate data against the schema. Throws a ValueError if data is invalid.
yamale.validate(schema, data)
```
You can pass a string of YAML to `make_schema()` and `make_data()` instead of passing a file path
by using the `content=` parameter:
```python
data = yamale.make_data(content="""
name: Bill
age: 26
height: 6.2
awesome: True
""")
```
If `data` is valid, nothing will happen. However, if `data` is invalid Yamale will throw a
`YamaleError` with a message containing all the invalid nodes:
```python
try:
yamale.validate(schema, data)
print('Validation success! 👍')
except ValueError as e:
print('Validation failed!\n%s' % str(e))
exit(1)
```
and an array of `ValidationResult`.
```python
try:
yamale.validate(schema, data)
print('Validation success! 👍')
except YamaleError as e:
print('Validation failed!\n')
for result in e.results:
print("Error validating data '%s' with '%s'\n\t" % (result.data, result.schema))
for error in result.errors:
print('\t%s' % error)
exit(1)
```
You can also specify an optional `parser` if you'd like to use the `ruamel.yaml` (YAML 1.2 support) instead:
```python
# Import Yamale and make a schema object, make sure ruamel.yaml is installed already.
import yamale
schema = yamale.make_schema('./schema.yaml', parser='ruamel')
# Create a Data object
data = yamale.make_data('./data.yaml', parser='ruamel')
# Validate data against the schema same as before.
yamale.validate(schema, data)
```
### Schema
| :warning: Ensure that your schema definitions come from internal or trusted sources. Yamale does not protect against intentionally malicious schemas. |
|:------------|
To use Yamale you must make a schema. A schema is a valid YAML file with one or more documents
inside. Each node terminates in a string which contains valid Yamale syntax. For example, `str()`
represents a [String validator](#validators).
A basic schema:
```yaml
name: str()
age: int(max=200)
height: num()
awesome: bool()
```
And some YAML that validates:
```yaml
name: Bill
age: 26
height: 6.2
awesome: True
```
Take a look at the [Examples](#examples) section for more complex schema ideas.
#### Includes
Schema files may contain more than one YAML document (nodes separated by `---`). The first document
found will be the base schema. Any additional documents will be treated as Includes. Includes allow
you to define a valid structure once and use it several times. They also allow you to do recursion.
A schema with an Include validator:
```yaml
person1: include('person')
person2: include('person')
---
person:
name: str()
age: int()
```
Some valid YAML:
```yaml
person1:
name: Bill
age: 70
person2:
name: Jill
age: 20
```
Every root node not in the first YAML document will be treated like an include:
```yaml
person: include('friend')
group: include('family')
---
friend:
name: str()
family:
name: str()
```
Is equivalent to:
```yaml
person: include('friend')
group: include('family')
---
friend:
name: str()
---
family:
name: str()
```
##### Recursion
You can get recursion using the Include validator.
This schema:
```yaml
person: include('human')
---
human:
name: str()
age: int()
friend: include('human', required=False)
```
Will validate this data:
```yaml
person:
name: Bill
age: 50
friend:
name: Jill
age: 20
friend:
name: Will
age: 10
```
##### Adding external includes
After you construct a schema you can add extra, external include definitions by calling
`schema.add_include(dict)`. This method takes a dictionary and adds each key as another include.
### Strict mode
By default Yamale will provide errors for extra elements present in lists and maps that are not
covered by the schema. With strict mode disabled (using the `--no-strict` command line option),
additional elements will not cause any errors. In the API, strict mode can be toggled by passing
the strict=True/False flag to the validate function.
It is possible to mix strict and non-strict mode by setting the strict=True/False flag in the
include validator, setting the option only for the included validators.
Validators
----------
Here are all the validators Yamale knows about. Every validator takes a `required` keyword telling
Yamale whether or not that node must exist. By default every node is required. Example: `str(required=False)`
You can also require that an optional value is not `None` by using the `none` keyword. By default
Yamale will accept `None` as a valid value for a key that's not required. Reject `None` values
with `none=False` in any validator. Example: `str(required=False, none=False)`.
Some validators take keywords and some take arguments, some take both. For instance the `enum()`
validator takes one or more constants as arguments and the `required` keyword:
`enum('a string', 1, False, required=False)`
### String - `str(min=int, max=int, equals=string, starts_with=string, ends_with=string, matches=regex, exclude=string, ignore_case=False, multiline=False, dotall=False)`
Validates strings.
- keywords
- `min`: len(string) >= min
- `max`: len(string) <= max
- `equals`: string == value (add `ignore_case=True` for case-insensitive checking)
- `starts_with`: Accepts only strings starting with given value (add `ignore_case=True` for
case-insensitive checking)
- `matches`: Validates the string against a given regex. Similar to the `regex()` validator,
you can use `ignore_case`, `multiline` and `dotall`)
- `ends_with`: Accepts only strings ending with given value (add `ignore_case=True` for case-insensitive checking)
- `exclude`: Rejects strings that contains any character in the excluded value
- `ignore_case`: Validates strings in a case-insensitive manner.
- `multiline`: `^` and `$` in a pattern match at the beginning and end of each line in a string
in addition to matching at the beginning and end of the entire string. (A pattern matches
at [the beginning of a string](https://docs.python.org/3/library/re.html#re.match) even in
multiline mode; see below for a workaround.); only allowed in conjunction with a `matches` keyword.
- `dotall`: `.` in a pattern matches newline characters in a validated string in addition to
matching every character that *isn't* a newline.; only allowed in conjunction with a `matches` keyword.
Examples:
- `str(max=10, exclude='?!')`: Allows only strings less than 11 characters that don't contain `?` or `!`.
### Regex - `regex([patterns], name=string, ignore_case=False, multiline=False, dotall=False)`
Validates strings against one or more regular expressions.
- arguments: one or more Python regular expression patterns
- keywords:
- `name`: A friendly description for the patterns.
- `ignore_case`: Validates strings in a case-insensitive manner.
- `multiline`: `^` and `$` in a pattern match at the beginning and end of each line in a string
in addition to matching at the beginning and end of the entire string. (A pattern matches
at [the beginning of a string](https://docs.python.org/3/library/re.html#re.match) even in
multiline mode; see below for a workaround.)
- `dotall`: `.` in a pattern matches newline characters in a validated string in addition to
matching every character that *isn't* a newline.
Examples:
- `regex('^[^?!]{,10}$')`: Allows only strings less than 11 characters that don't contain `?` or `!`.
- `regex(r'^(\d+)(\s\1)+$', name='repeated natural')`: Allows only strings that contain two or
more identical digit sequences, each separated by a whitespace character. Non-matching strings
like `sugar` are rejected with a message like `'sugar' is not a repeated natural.`
- `regex('.*^apples$', multiline=True, dotall=True)`: Allows the string `apples` as well
as multiline strings that contain the line `apples`.
### Integer - `int(min=int, max=int)`
Validates integers.
- keywords
- `min`: int >= min
- `max`: int <= max
### Number - `num(min=float, max=float)`
Validates integers and floats.
- keywords
- `min`: num >= min
- `max`: num <= max
### Boolean - `bool()`
Validates booleans.
### Null - `null()`
Validates null values.
### Enum - `enum([primitives])`
Validates from a list of constants.
- arguments: constants to test equality with
Examples:
- `enum('a string', 1, False)`: a value can be either `'a string'`, `1` or `False`
### Day - `day(min=date, max=date)`
Validates a date in the form of YYYY-MM-DD.
- keywords
- `min`: date >= min
- `max`: date <= max
Examples:
- `day(min='2001-01-01', max='2100-01-01')`: Only allows dates between 2001-01-01 and 2100-01-01.
### Timestamp - `timestamp(min=time, max=time)`
Validates a timestamp in the form of YYYY-MM-DD HH:MM:SS.
- keywords
- `min`: time >= min
- `max`: time <= max
Examples:
- `timestamp(min='2001-01-01 01:00:00', max='2100-01-01 23:00:00')`: Only allows times between
2001-01-01 01:00:00 and 2100-01-01 23:00:00.
### List - `list([validators], min=int, max=int)`
Validates lists. If one or more validators are passed to `list()` only nodes that pass at
least one of those validators will be accepted.
- arguments: one or more validators to test values with
- keywords
- `min`: len(list) >= min
- `max`: len(list) <= max
Examples:
- `list()`: Validates any list
- `list(include('custom'), int(), min=4)`: Only validates lists that contain the `custom` include
or integers and contains a minimum of 4 items.
### Map - `map([validators], key=validator, min=int, max=int)`
Validates maps. Use when you want a node to contain freeform data. Similar to `List`, `Map` takes
one or more validators to run against the values of its nodes, and only nodes that pass at least
one of those validators will be accepted. By default, only the values of nodes are validated and
the keys aren't checked.
- arguments: one or more validators to test values with
- keywords
- `key`: A validator for the keys of the map.
- `min`: len(map) >= min
- `max`: len(map) <= max
Examples:
- `map()`: Validates any map
- `map(str(), int())`: Only validates maps whose values are strings or integers.
- `map(str(), key=int())`: Only validates maps whose keys are integers and values are strings. `1: one` would be valid but `'1': one` would not.
- `map(str(), min=1)`: Only validates a non-empty map.
### IP Address - `ip()`
Validates IPv4 and IPv6 addresses.
- keywords
- `version`: 4 or 6; explicitly force IPv4 or IPv6 validation
Examples:
- `ip()`: Allows any valid IPv4 or IPv6 address
- `ip(version=4)`: Allows any valid IPv4 address
- `ip(version=6)`: Allows any valid IPv6 address
### MAC Address - `mac()`
Validates MAC addresses.
Examples:
- `mac()`: Allows any valid MAC address
### SemVer (Semantic Versioning) - `semver()`
Validates [Semantic Versioning](https://semver.org/) strings.
Examples:
- `semver()`: Allows any valid SemVer string
### Any - `any([validators])`
Validates against a union of types. Use when a node **must** contain **one and only one** of several types. It is valid
if at least one of the listed validators is valid. If no validators are given, accept any value.
- arguments: validators to test values with (if none is given, allow any value; if one or more are given,
one must be present)
Examples:
- `any(int(), null())`: Validates either an integer **or** a null value.
- `any(num(), include('vector'))`: Validates **either** a number **or** an included 'vector' type.
- `any(str(min=3, max=3),str(min=5, max=5),str(min=7, max=7))`: validates to a string that is exactly 3, 5, or 7 characters long
- `any()`: Allows any value.
### Subset - `subset([validators], allow_empty=False)`
Validates against a subset of types. Unlike the `Any` validator, this validators allows **one or more** of several types.
As such, it *automatically validates against a list*. It is valid if all values can be validated against at least one
validator.
- arguments: validators to test with (at least one; if none is given, a `ValueError` exception will be raised)
- keywords:
- `allow_empty`: Allow the subset to be empty (and is, therefore, also optional). This overrides the `required`
flag.
Examples:
- `subset(int(), str())`: Validators against an integer, a string, or a list of either.
- `subset(int(), str(), allow_empty=True)`: Same as above, but allows the empty set and makes the subset optional.
### Include - `include(include_name)`
Validates included structures. Must supply the name of a valid include.
- arguments: single name of a defined include, surrounded by quotes.
Examples:
- `include('person')`
### Custom validators
It is also possible to add your own custom validators. This is an advanced topic, but here is an
example of adding a `Date` validator and using it in a schema as `date()`
```python
import yamale
import datetime
from yamale.validators import DefaultValidators, Validator
class Date(Validator):
""" Custom Date validator """
tag = 'date'
def _is_valid(self, value):
return isinstance(value, datetime.date)
validators = DefaultValidators.copy() # This is a dictionary
validators[Date.tag] = Date
schema = yamale.make_schema('./schema.yaml', validators=validators)
# Then use `schema` as normal
```
Examples
--------
| :warning: Ensure that your schema definitions come from internal or trusted sources. Yamale does not protect against intentionally malicious schemas. |
|:------------|
### Using keywords
#### Schema:
```yaml
optional: str(required=False)
optional_min: int(min=1, required=False)
min: num(min=1.5)
max: int(max=100)
```
#### Valid Data:
```yaml
optional_min: 10
min: 1.6
max: 100
```
### Includes and recursion
#### Schema:
```yaml
customerA: include('customer')
customerB: include('customer')
recursion: include('recurse')
---
customer:
name: str()
age: int()
custom: include('custom_type')
custom_type:
integer: int()
recurse:
level: int()
again: include('recurse', required=False)
```
#### Valid Data:
```yaml
customerA:
name: bob
age: 900
custom:
integer: 1
customerB:
name: jill
age: 1
custom:
integer: 3
recursion:
level: 1
again:
level: 2
again:
level: 3
again:
level: 4
```
### Lists
#### Schema:
```yaml
list_with_two_types: list(str(), include('variant'))
questions: list(include('question'))
---
variant:
rsid: str()
name: str()
question:
choices: list(include('choices'))
questions: list(include('question'), required=False)
choices:
id: str()
```
#### Valid Data:
```yaml
list_with_two_types:
- 'some'
- rsid: 'rs123'
name: 'some SNP'
- 'thing'
- rsid: 'rs312'
name: 'another SNP'
questions:
- choices:
- id: 'id_str'
- id: 'id_str1'
questions:
- choices:
- id: 'id_str'
- id: 'id_str1'
```
### The data is a list of items without a keyword at the top level
#### Schema:
```yaml
list(include('human'), min=2, max=2)
---
human:
name: str()
age: int(max=200)
height: num()
awesome: bool()
```
#### Valid Data:
```yaml
- name: Bill
age: 26
height: 6.2
awesome: True
- name: Adrian
age: 23
height: 6.3
awesome: True
```
Writing Tests
-------------
To validate YAML files when you run your program's tests use Yamale's YamaleTestCase
Example:
```python
class TestYaml(YamaleTestCase):
base_dir = os.path.dirname(os.path.realpath(__file__))
schema = 'schema.yaml'
yaml = 'data.yaml'
# or yaml = ['data-*.yaml', 'some_data.yaml']
def runTest(self):
self.assertTrue(self.validate())
```
`base_dir`: String path to prepend to all other paths. This is optional.
`schema`: String of path to the schema file to use. One schema file per test case.
`yaml`: String or list of yaml files to validate. Accepts globs.
Developers
----------
### Linting + Formatting
Yamale is formatted with [ruff](https://github.com/astral-sh/ruff). There is a github action enforcing
ruff formatting and linting rules. You can run this locally via `make lint` or by installing
the pre-commit hooks via `make install-hooks`
### Testing
Yamale uses [Tox](https://tox.readthedocs.org/en/latest/) to run its tests against multiple Python
versions. To run tests, first checkout Yamale, install Tox, then run `make test` in Yamale's root
directory. You may also have to install the correct Python versions to test with as well.
NOTE on Python versions: `tox.ini` specifies the lowest and highest versions of Python supported by
Yamale. Unless your development environment is configured to support testing against multiple Python
versions, one or more of the test branches may fail. One method of enabling testing against multiple
versions of Python is to install `pyenv` and `tox-pyenv` and to use `pyenv install` and `pyenv local`
to ensure that tox is able to locate appropriate Pythons.
### Releasing
Yamale uses Github Actions to upload new tags to PyPi.
To release a new version:
1. Make a commit with the new version number in `yamale/VERSION`.
1. Run tests for good luck.
1. Run `make release`.
Github Actions will take care of the rest.
Raw data
{
"_id": null,
"home_page": "https://github.com/23andMe/Yamale",
"name": "yamale",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Bo Lopker",
"author_email": "blopker@23andme.com",
"download_url": "https://files.pythonhosted.org/packages/a1/52/0faa32aa15f241a9f950ded276c942db69bce8dda5f19241f6b960080dca/yamale-5.2.1.tar.gz",
"platform": null,
"description": "Yamale (ya\u00b7ma\u00b7l\u0113)\n=================\n\n| :warning: Ensure that your schema definitions come from internal or trusted sources. Yamale does not protect against intentionally malicious schemas. |\n|:------------|\n\n<img src=\"https://github.com/23andMe/Yamale/blob/master/yamale.png?raw=true\" alt=\"Yamale\" width=\"400\"/>\n\nA schema and validator for YAML.\n\nWhat's YAML? See the current spec [here](http://www.yaml.org/spec/1.2/spec.html) and an introduction\nto the syntax [here](https://github.com/Animosity/CraftIRC/wiki/Complete-idiot's-introduction-to-yaml).\n\n[](https://github.com/23andMe/Yamale/actions/workflows/run-tests.yml)\n[](https://pypi.python.org/pypi/yamale)\n\nRequirements\n------------\n* Python 3.8+\n* PyYAML\n* ruamel.yaml (optional)\n\nInstall\n-------\n### pip\n```bash\n$ pip install yamale\n```\n\nNOTE: Some platforms, e.g., Mac OS, may ship with only Python 2 and may not have pip installed.\nInstallation of Python 3 should also install pip. To preserve any system dependencies on default\nsoftware, consider installing Python 3 as a local package. Please note replacing system-provided\nPython may disrupt other software. Mac OS users may wish to investigate MacPorts, homebrew, or\nbuilding Python 3 from source; in all three cases, Apple's Command Line Tools (CLT) for Xcode\nmay be required. See also [developers](#developers), below.\n\n### Manual\n1. Download Yamale from: https://github.com/23andMe/Yamale/archive/master.zip\n2. Unzip somewhere temporary\n3. Run `python setup.py install` (may have to prepend `sudo`)\n\nUsage\n-----\n### Command line\nYamale can be run from the command line to validate one or many YAML files. Yamale will search the\ndirectory you supply (current directory is default) for YAML files. Each YAML file it finds it will\nlook in the same directory as that file for its schema, if there is no schema Yamale will keep\nlooking up the directory tree until it finds one. If Yamale can not find a schema it will tell you.\n\nUsage:\n\n```bash\nusage: yamale [-h] [-s SCHEMA] [-n CPU_NUM] [-p PARSER] [--no-strict] [PATH]\n\nValidate yaml files.\n\npositional arguments:\n PATH folder to validate. Default is current directory.\n\noptional arguments:\n -h, --help show this help message and exit\n -s SCHEMA, --schema SCHEMA\n filename of schema. Default is schema.yaml.\n -n CPU_NUM, --cpu-num CPU_NUM\n number of CPUs to use. Default is 4.\n -p PARSER, --parser PARSER\n YAML library to load files. Choices are \"ruamel\" or\n \"pyyaml\" (default).\n --no-strict Disable strict mode, unexpected elements in the data\n will be accepted.\n```\n\n### API\nThere are several ways to feed Yamale schema and data files. The simplest way is to let Yamale take\ncare of reading and parsing your YAML files.\n\nAll you need to do is supply the files' path:\n```python\n# Import Yamale and make a schema object:\nimport yamale\nschema = yamale.make_schema('./schema.yaml')\n\n# Create a Data object\ndata = yamale.make_data('./data.yaml')\n\n# Validate data against the schema. Throws a ValueError if data is invalid.\nyamale.validate(schema, data)\n```\n\nYou can pass a string of YAML to `make_schema()` and `make_data()` instead of passing a file path\nby using the `content=` parameter:\n\n```python\ndata = yamale.make_data(content=\"\"\"\nname: Bill\nage: 26\nheight: 6.2\nawesome: True\n\"\"\")\n```\n\nIf `data` is valid, nothing will happen. However, if `data` is invalid Yamale will throw a\n`YamaleError` with a message containing all the invalid nodes:\n```python\ntry:\n yamale.validate(schema, data)\n print('Validation success! \ud83d\udc4d')\nexcept ValueError as e:\n print('Validation failed!\\n%s' % str(e))\n exit(1)\n```\nand an array of `ValidationResult`.\n```python\ntry:\n yamale.validate(schema, data)\n print('Validation success! \ud83d\udc4d')\nexcept YamaleError as e:\n print('Validation failed!\\n')\n for result in e.results:\n print(\"Error validating data '%s' with '%s'\\n\\t\" % (result.data, result.schema))\n for error in result.errors:\n print('\\t%s' % error)\n exit(1)\n```\n\nYou can also specify an optional `parser` if you'd like to use the `ruamel.yaml` (YAML 1.2 support) instead:\n```python\n# Import Yamale and make a schema object, make sure ruamel.yaml is installed already.\nimport yamale\nschema = yamale.make_schema('./schema.yaml', parser='ruamel')\n\n# Create a Data object\ndata = yamale.make_data('./data.yaml', parser='ruamel')\n\n# Validate data against the schema same as before.\nyamale.validate(schema, data)\n```\n\n### Schema\n\n| :warning: Ensure that your schema definitions come from internal or trusted sources. Yamale does not protect against intentionally malicious schemas. |\n|:------------|\n\nTo use Yamale you must make a schema. A schema is a valid YAML file with one or more documents\ninside. Each node terminates in a string which contains valid Yamale syntax. For example, `str()`\nrepresents a [String validator](#validators).\n\nA basic schema:\n```yaml\nname: str()\nage: int(max=200)\nheight: num()\nawesome: bool()\n```\n\nAnd some YAML that validates:\n```yaml\nname: Bill\nage: 26\nheight: 6.2\nawesome: True\n```\n\nTake a look at the [Examples](#examples) section for more complex schema ideas.\n\n#### Includes\nSchema files may contain more than one YAML document (nodes separated by `---`). The first document\nfound will be the base schema. Any additional documents will be treated as Includes. Includes allow\nyou to define a valid structure once and use it several times. They also allow you to do recursion.\n\nA schema with an Include validator:\n```yaml\nperson1: include('person')\nperson2: include('person')\n---\nperson:\n name: str()\n age: int()\n```\n\nSome valid YAML:\n```yaml\nperson1:\n name: Bill\n age: 70\n\nperson2:\n name: Jill\n age: 20\n```\n\nEvery root node not in the first YAML document will be treated like an include:\n```yaml\nperson: include('friend')\ngroup: include('family')\n---\nfriend:\n name: str()\nfamily:\n name: str()\n```\n\nIs equivalent to:\n```yaml\nperson: include('friend')\ngroup: include('family')\n---\nfriend:\n name: str()\n---\nfamily:\n name: str()\n```\n\n##### Recursion\nYou can get recursion using the Include validator.\n\nThis schema:\n```yaml\nperson: include('human')\n---\nhuman:\n name: str()\n age: int()\n friend: include('human', required=False)\n```\n\nWill validate this data:\n```yaml\nperson:\n name: Bill\n age: 50\n friend:\n name: Jill\n age: 20\n friend:\n name: Will\n age: 10\n```\n\n##### Adding external includes\nAfter you construct a schema you can add extra, external include definitions by calling\n`schema.add_include(dict)`. This method takes a dictionary and adds each key as another include.\n\n### Strict mode\nBy default Yamale will provide errors for extra elements present in lists and maps that are not\ncovered by the schema. With strict mode disabled (using the `--no-strict` command line option),\nadditional elements will not cause any errors. In the API, strict mode can be toggled by passing\nthe strict=True/False flag to the validate function.\n\nIt is possible to mix strict and non-strict mode by setting the strict=True/False flag in the\ninclude validator, setting the option only for the included validators.\n\nValidators\n----------\nHere are all the validators Yamale knows about. Every validator takes a `required` keyword telling\nYamale whether or not that node must exist. By default every node is required. Example: `str(required=False)`\n\nYou can also require that an optional value is not `None` by using the `none` keyword. By default\nYamale will accept `None` as a valid value for a key that's not required. Reject `None` values\nwith `none=False` in any validator. Example: `str(required=False, none=False)`.\n\nSome validators take keywords and some take arguments, some take both. For instance the `enum()`\nvalidator takes one or more constants as arguments and the `required` keyword:\n`enum('a string', 1, False, required=False)`\n\n### String - `str(min=int, max=int, equals=string, starts_with=string, ends_with=string, matches=regex, exclude=string, ignore_case=False, multiline=False, dotall=False)`\nValidates strings.\n- keywords\n - `min`: len(string) >= min\n - `max`: len(string) <= max\n - `equals`: string == value (add `ignore_case=True` for case-insensitive checking)\n - `starts_with`: Accepts only strings starting with given value (add `ignore_case=True` for\n case-insensitive checking)\n - `matches`: Validates the string against a given regex. Similar to the `regex()` validator,\n you can use `ignore_case`, `multiline` and `dotall`)\n - `ends_with`: Accepts only strings ending with given value (add `ignore_case=True` for case-insensitive checking)\n - `exclude`: Rejects strings that contains any character in the excluded value\n - `ignore_case`: Validates strings in a case-insensitive manner.\n - `multiline`: `^` and `$` in a pattern match at the beginning and end of each line in a string\n in addition to matching at the beginning and end of the entire string. (A pattern matches\n at [the beginning of a string](https://docs.python.org/3/library/re.html#re.match) even in\n multiline mode; see below for a workaround.); only allowed in conjunction with a `matches` keyword.\n - `dotall`: `.` in a pattern matches newline characters in a validated string in addition to\n matching every character that *isn't* a newline.; only allowed in conjunction with a `matches` keyword.\n\nExamples:\n- `str(max=10, exclude='?!')`: Allows only strings less than 11 characters that don't contain `?` or `!`.\n\n### Regex - `regex([patterns], name=string, ignore_case=False, multiline=False, dotall=False)`\nValidates strings against one or more regular expressions.\n- arguments: one or more Python regular expression patterns\n- keywords:\n - `name`: A friendly description for the patterns.\n - `ignore_case`: Validates strings in a case-insensitive manner.\n - `multiline`: `^` and `$` in a pattern match at the beginning and end of each line in a string\n in addition to matching at the beginning and end of the entire string. (A pattern matches\n at [the beginning of a string](https://docs.python.org/3/library/re.html#re.match) even in\n multiline mode; see below for a workaround.)\n - `dotall`: `.` in a pattern matches newline characters in a validated string in addition to\n matching every character that *isn't* a newline.\n\nExamples:\n- `regex('^[^?!]{,10}$')`: Allows only strings less than 11 characters that don't contain `?` or `!`.\n- `regex(r'^(\\d+)(\\s\\1)+$', name='repeated natural')`: Allows only strings that contain two or\n more identical digit sequences, each separated by a whitespace character. Non-matching strings\n like `sugar` are rejected with a message like `'sugar' is not a repeated natural.`\n- `regex('.*^apples$', multiline=True, dotall=True)`: Allows the string `apples` as well\n as multiline strings that contain the line `apples`.\n\n### Integer - `int(min=int, max=int)`\nValidates integers.\n- keywords\n - `min`: int >= min\n - `max`: int <= max\n\n### Number - `num(min=float, max=float)`\nValidates integers and floats.\n- keywords\n - `min`: num >= min\n - `max`: num <= max\n\n### Boolean - `bool()`\nValidates booleans.\n\n### Null - `null()`\nValidates null values.\n\n### Enum - `enum([primitives])`\nValidates from a list of constants.\n- arguments: constants to test equality with\n\nExamples:\n- `enum('a string', 1, False)`: a value can be either `'a string'`, `1` or `False`\n\n### Day - `day(min=date, max=date)`\nValidates a date in the form of YYYY-MM-DD.\n- keywords\n - `min`: date >= min\n - `max`: date <= max\n\nExamples:\n- `day(min='2001-01-01', max='2100-01-01')`: Only allows dates between 2001-01-01 and 2100-01-01.\n\n### Timestamp - `timestamp(min=time, max=time)`\nValidates a timestamp in the form of YYYY-MM-DD HH:MM:SS.\n- keywords\n - `min`: time >= min\n - `max`: time <= max\n\nExamples:\n- `timestamp(min='2001-01-01 01:00:00', max='2100-01-01 23:00:00')`: Only allows times between\n 2001-01-01 01:00:00 and 2100-01-01 23:00:00.\n\n### List - `list([validators], min=int, max=int)`\nValidates lists. If one or more validators are passed to `list()` only nodes that pass at\nleast one of those validators will be accepted.\n\n- arguments: one or more validators to test values with\n- keywords\n - `min`: len(list) >= min\n - `max`: len(list) <= max\n\nExamples:\n- `list()`: Validates any list\n- `list(include('custom'), int(), min=4)`: Only validates lists that contain the `custom` include\n or integers and contains a minimum of 4 items.\n\n### Map - `map([validators], key=validator, min=int, max=int)`\nValidates maps. Use when you want a node to contain freeform data. Similar to `List`, `Map` takes\none or more validators to run against the values of its nodes, and only nodes that pass at least\none of those validators will be accepted. By default, only the values of nodes are validated and\nthe keys aren't checked.\n- arguments: one or more validators to test values with\n- keywords\n - `key`: A validator for the keys of the map.\n - `min`: len(map) >= min\n - `max`: len(map) <= max\n\nExamples:\n- `map()`: Validates any map\n- `map(str(), int())`: Only validates maps whose values are strings or integers.\n- `map(str(), key=int())`: Only validates maps whose keys are integers and values are strings. `1: one` would be valid but `'1': one` would not.\n- `map(str(), min=1)`: Only validates a non-empty map.\n\n### IP Address - `ip()`\nValidates IPv4 and IPv6 addresses.\n\n- keywords\n - `version`: 4 or 6; explicitly force IPv4 or IPv6 validation\n\nExamples:\n- `ip()`: Allows any valid IPv4 or IPv6 address\n- `ip(version=4)`: Allows any valid IPv4 address\n- `ip(version=6)`: Allows any valid IPv6 address\n\n### MAC Address - `mac()`\nValidates MAC addresses.\n\nExamples:\n- `mac()`: Allows any valid MAC address\n\n### SemVer (Semantic Versioning) - `semver()`\nValidates [Semantic Versioning](https://semver.org/) strings.\n\nExamples:\n- `semver()`: Allows any valid SemVer string\n\n### Any - `any([validators])`\nValidates against a union of types. Use when a node **must** contain **one and only one** of several types. It is valid\nif at least one of the listed validators is valid. If no validators are given, accept any value.\n- arguments: validators to test values with (if none is given, allow any value; if one or more are given,\none must be present)\n\nExamples:\n- `any(int(), null())`: Validates either an integer **or** a null value.\n- `any(num(), include('vector'))`: Validates **either** a number **or** an included 'vector' type.\n- `any(str(min=3, max=3),str(min=5, max=5),str(min=7, max=7))`: validates to a string that is exactly 3, 5, or 7 characters long\n- `any()`: Allows any value.\n\n### Subset - `subset([validators], allow_empty=False)`\nValidates against a subset of types. Unlike the `Any` validator, this validators allows **one or more** of several types.\nAs such, it *automatically validates against a list*. It is valid if all values can be validated against at least one\nvalidator.\n- arguments: validators to test with (at least one; if none is given, a `ValueError` exception will be raised)\n- keywords:\n - `allow_empty`: Allow the subset to be empty (and is, therefore, also optional). This overrides the `required`\nflag.\n\nExamples:\n- `subset(int(), str())`: Validators against an integer, a string, or a list of either.\n- `subset(int(), str(), allow_empty=True)`: Same as above, but allows the empty set and makes the subset optional.\n\n### Include - `include(include_name)`\nValidates included structures. Must supply the name of a valid include.\n- arguments: single name of a defined include, surrounded by quotes.\n\nExamples:\n- `include('person')`\n\n### Custom validators\nIt is also possible to add your own custom validators. This is an advanced topic, but here is an\nexample of adding a `Date` validator and using it in a schema as `date()`\n\n```python\nimport yamale\nimport datetime\nfrom yamale.validators import DefaultValidators, Validator\n\nclass Date(Validator):\n \"\"\" Custom Date validator \"\"\"\n tag = 'date'\n\n def _is_valid(self, value):\n return isinstance(value, datetime.date)\n\nvalidators = DefaultValidators.copy() # This is a dictionary\nvalidators[Date.tag] = Date\nschema = yamale.make_schema('./schema.yaml', validators=validators)\n# Then use `schema` as normal\n```\n\nExamples\n--------\n\n| :warning: Ensure that your schema definitions come from internal or trusted sources. Yamale does not protect against intentionally malicious schemas. |\n|:------------|\n\n### Using keywords\n#### Schema:\n```yaml\noptional: str(required=False)\noptional_min: int(min=1, required=False)\nmin: num(min=1.5)\nmax: int(max=100)\n```\n#### Valid Data:\n```yaml\noptional_min: 10\nmin: 1.6\nmax: 100\n```\n\n### Includes and recursion\n#### Schema:\n```yaml\ncustomerA: include('customer')\ncustomerB: include('customer')\nrecursion: include('recurse')\n---\ncustomer:\n name: str()\n age: int()\n custom: include('custom_type')\n\ncustom_type:\n integer: int()\n\nrecurse:\n level: int()\n again: include('recurse', required=False)\n```\n#### Valid Data:\n```yaml\ncustomerA:\n name: bob\n age: 900\n custom:\n integer: 1\ncustomerB:\n name: jill\n age: 1\n custom:\n integer: 3\nrecursion:\n level: 1\n again:\n level: 2\n again:\n level: 3\n again:\n level: 4\n```\n\n### Lists\n#### Schema:\n```yaml\nlist_with_two_types: list(str(), include('variant'))\nquestions: list(include('question'))\n---\nvariant:\n rsid: str()\n name: str()\n\nquestion:\n choices: list(include('choices'))\n questions: list(include('question'), required=False)\n\nchoices:\n id: str()\n```\n#### Valid Data:\n```yaml\nlist_with_two_types:\n - 'some'\n - rsid: 'rs123'\n name: 'some SNP'\n - 'thing'\n - rsid: 'rs312'\n name: 'another SNP'\nquestions:\n - choices:\n - id: 'id_str'\n - id: 'id_str1'\n questions:\n - choices:\n - id: 'id_str'\n - id: 'id_str1'\n```\n\n### The data is a list of items without a keyword at the top level\n#### Schema:\n```yaml\nlist(include('human'), min=2, max=2)\n\n---\nhuman:\n name: str()\n age: int(max=200)\n height: num()\n awesome: bool()\n```\n#### Valid Data:\n```yaml\n- name: Bill\n age: 26\n height: 6.2\n awesome: True\n\n- name: Adrian\n age: 23\n height: 6.3\n awesome: True\n```\n\nWriting Tests\n-------------\nTo validate YAML files when you run your program's tests use Yamale's YamaleTestCase\n\nExample:\n\n```python\nclass TestYaml(YamaleTestCase):\n base_dir = os.path.dirname(os.path.realpath(__file__))\n schema = 'schema.yaml'\n yaml = 'data.yaml'\n # or yaml = ['data-*.yaml', 'some_data.yaml']\n\n def runTest(self):\n self.assertTrue(self.validate())\n```\n\n`base_dir`: String path to prepend to all other paths. This is optional.\n\n`schema`: String of path to the schema file to use. One schema file per test case.\n\n`yaml`: String or list of yaml files to validate. Accepts globs.\n\nDevelopers\n----------\n### Linting + Formatting\nYamale is formatted with [ruff](https://github.com/astral-sh/ruff). There is a github action enforcing\nruff formatting and linting rules. You can run this locally via `make lint` or by installing\nthe pre-commit hooks via `make install-hooks`\n\n### Testing\nYamale uses [Tox](https://tox.readthedocs.org/en/latest/) to run its tests against multiple Python\nversions. To run tests, first checkout Yamale, install Tox, then run `make test` in Yamale's root\ndirectory. You may also have to install the correct Python versions to test with as well.\n\nNOTE on Python versions: `tox.ini` specifies the lowest and highest versions of Python supported by\nYamale. Unless your development environment is configured to support testing against multiple Python\nversions, one or more of the test branches may fail. One method of enabling testing against multiple\nversions of Python is to install `pyenv` and `tox-pyenv` and to use `pyenv install` and `pyenv local`\nto ensure that tox is able to locate appropriate Pythons.\n\n### Releasing\nYamale uses Github Actions to upload new tags to PyPi.\nTo release a new version:\n\n1. Make a commit with the new version number in `yamale/VERSION`.\n1. Run tests for good luck.\n1. Run `make release`.\n\nGithub Actions will take care of the rest.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A schema and validator for YAML.",
"version": "5.2.1",
"project_urls": {
"Homepage": "https://github.com/23andMe/Yamale"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "08df6339887f2fea6a4720d56e6c55f9730a663bc3946046fa6bb6ccf8e9892f",
"md5": "9c9b3f2657c8f63206595170acae3a54",
"sha256": "e44cd30cf3055ee4b34c1c71d6fe35490a127dcbd36f82f27859d105a9989922"
},
"downloads": -1,
"filename": "yamale-5.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9c9b3f2657c8f63206595170acae3a54",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 56856,
"upload_time": "2024-04-30T20:15:48",
"upload_time_iso_8601": "2024-04-30T20:15:48.155534Z",
"url": "https://files.pythonhosted.org/packages/08/df/6339887f2fea6a4720d56e6c55f9730a663bc3946046fa6bb6ccf8e9892f/yamale-5.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a1520faa32aa15f241a9f950ded276c942db69bce8dda5f19241f6b960080dca",
"md5": "94373862b7536a7c1417483e2d9f8f13",
"sha256": "19bbe713d588f07177bc519a46070c0793ed126ea37f425a76055b99703f835a"
},
"downloads": -1,
"filename": "yamale-5.2.1.tar.gz",
"has_sig": false,
"md5_digest": "94373862b7536a7c1417483e2d9f8f13",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 40582,
"upload_time": "2024-04-30T20:15:50",
"upload_time_iso_8601": "2024-04-30T20:15:50.025582Z",
"url": "https://files.pythonhosted.org/packages/a1/52/0faa32aa15f241a9f950ded276c942db69bce8dda5f19241f6b960080dca/yamale-5.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-30 20:15:50",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "23andMe",
"github_project": "Yamale",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "tox",
"specs": [
[
"==",
"4.13.0"
]
]
}
],
"tox": true,
"lcname": "yamale"
}
JSON
