| Name | zcbor JSON |
| Version |
0.8.1
JSON |
| download |
| home_page | |
| Summary | Code generation and validation using CDDL schemas |
| upload_time | 2024-01-26 17:25:35 |
| maintainer | |
| docs_url | None |
| author | Nordic Semiconductor ASA |
| requires_python | >=3.8 |
| license | Apache |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
zcbor
=====
zcbor is a low footprint [CBOR](https://en.wikipedia.org/wiki/CBOR) library in the C language (C++ compatible), tailored for use in microcontrollers.
It comes with a schema-driven script tool that can validate your data, or even generate code.
The schema language (CDDL) allows creating very advanced and detailed schemas.
The validation and conversion part of the tool works with YAML and JSON data, in addition to CBOR.
It can for example validate a YAML file against a schema and convert it into CBOR.
The code generation part of the tool generates C code based on the given schema.
The generated code performs CBOR encoding and decoding using the C library, while also validating the data against all the rules in the schema.
The schema language used by zcbor is CDDL (Concise Data Definition Language) which is a powerful human-readable data description language defined in [IETF RFC 8610](https://datatracker.ietf.org/doc/rfc8610/).
Features
========
Here are some possible ways zcbor can be used:
- C code:
- As a low-footprint CBOR decoding/encoding library similar to TinyCBOR/QCBOR/NanoCBOR. The library can be used independently of the Python script. ([More information](#cbor-decodingencoding-library))
- To generate C code (using the Python script) for validating and decoding or encoding CBOR, for use in optimized or constrained environments, such as microcontrollers. ([More information](#code-generation))
- Python script and module ([More information](#python-script-and-module)):
- Validate a YAML/JSON file and translate it into CBOR e.g. for transmission.
- Validate a YAML/JSON/CBOR file before processing it with some other tool
- Decode and validate incoming CBOR data into human-readable YAML/JSON.
- As part of a python script that processes YAML/JSON/CBOR files.
- Uses the same internal representation used by the PyYAML/json/cbor2 libraries.
- Do validation against a CDDL schema.
- Create a read-only representation via named tuples (with names taken from the CDDL schema).
Getting started
===============
There are samples in the [samples](samples) directory that demonstrate different ways to use zcbor, both the script tool and the C code.
1. The [hello_world sample](samples/hello_world/README.md) is a minimum examples of encoding and decoding using the C library.
2. The [pet sample](samples/pet/README.md) shows a how to use the C library together with generated code, and how to use the script tool to do code generation and data conversion.
The [tests](tests) also demonstrate how to use zcbor in different ways. The [encoding](tests/encode), [decoding](tests/decode), and [unit](tests/unit) tests run using [Zephyr](https://github.com/zephyrproject-rtos/zephyr) (the samples do not use Zephyr).
Should I use code generation or the library directly?
-----------------------------------------------------
The benefit of using code generation is greater for decoding than encoding.
This is because decoding is generally more complex than encoding, since when decoding you have to gracefully handle all possible payloads.
The code generation will provide a number of checks that are tedious to write manually.
These checks ensure that the payload is well-formed.
CBOR decoding/encoding library
==============================
The CBOR library can be found in [include/](include) and [src/](src) and can be used directly, by including the files in your project.
If using zcbor with Zephyr, the library will be available when the [CONFIG_ZCBOR](https://docs.zephyrproject.org/latest/kconfig.html#CONFIG_ZCBOR) config is enabled.
The library is also used by generated code. See the [Code generation](#code-generation) section for more info about code generation.
The C library is C++ compatible.
The zcbor state object
----------------------
To do encoding or decoding with the library, instantiate a `zcbor_state_t` object, which is most easily done using the `ZCBOR_STATE_*()` macros, look below or in the [hello_world](samples/hello_world/src/main.c) sample for example code.
The `elem_count` member refers to the number of encoded objects in the current list or map.
`elem_count` starts again when entering a nested list or map, and is restored when exiting.
`elem_count` is one reason for needing "backup" states (the other is to allow rollback of the payload).
Backups are needed for _decoding_ if there are any lists, maps, or CBOR-encoded strings (`zcbor_bstr_*_decode`) in the data.
Backups are needed for _encoding_ if there are any lists or maps *and* you are using canonical encoding (`ZCBOR_CANONICAL`), or when using the `zcbor_bstr_*_encode` functions.
```c
/** Initialize a decoding state (could include an array of backup states).
* After calling this, decode_state[0] is ready to be used with the decoding APIs. */
ZCBOR_STATE_D(decode_state, n, payload, payload_len, elem_count, n_flags);
/** Initialize an encoding state (could include an array of backup states).
* After calling this, encode_state[0] is ready to be used with the encoding APIs. */
ZCBOR_STATE_E(encode_state, n, payload, payload_len, 0);
```
Configuration
-------------
The C library has a few compile-time configuration options.
These configuration options can be enabled by adding them as compile definitions to the build.
If using zcbor with Zephyr, use the [Kconfig options](https://github.com/zephyrproject-rtos/zephyr/blob/main/modules/zcbor/Kconfig) instead.
Name | Description
------------------------- | -----------
`ZCBOR_CANONICAL` | Assume canonical encoding (AKA "deterministically encoded CBOR"). When encoding lists and maps, do not use indefinite length encoding. Enabling `ZCBOR_CANONICAL` increases code size and makes the encoding library more often use state backups. When decoding, ensure that the incoming data conforms to canonical encoding, i.e. no indefinite length encoding, and always using minimal length encoding (e.g. not using 16 bits to encode a value < 256). Note: the map ordering constraint in canonical encoding is not checked.
`ZCBOR_VERBOSE` | Print log messages on encoding/decoding errors (`zcbor_log()`), and also a trace message (`zcbor_trace()`) for each decoded value, and in each generated function (when using code generation).
`ZCBOR_ASSERTS` | Enable asserts (`zcbor_assert()`). When they fail, the assert statements instruct the current function to return a `ZCBOR_ERR_ASSERTION` error. If `ZCBOR_VERBOSE` is enabled, a message is printed.
`ZCBOR_STOP_ON_ERROR` | Enable the `stop_on_error` functionality. This makes all functions abort their execution if called when an error has already happened.
`ZCBOR_BIG_ENDIAN` | All decoded values are returned as big-endian. The default is little-endian.
`ZCBOR_MAP_SMART_SEARCH` | Applies to decoding of unordered maps. When enabled, a flag is kept for each element in an array, ensuring it is not processed twice. If disabled, a count is kept for map as a whole. Enabling increases code size and memory usage, and requires the state variable to possess the memory necessary for the flags.
Python script and module
========================
The zcbor.py script can directly read CBOR, YAML, or JSON data and validate it against a CDDL description.
It can also freely convert the data between CBOR/YAML/JSON.
It can also output the data to a C file formatted as a byte array.
Invoking zcbor.py from the command line
---------------------------------------
zcbor.py can be installed via [`pip`](https://pypi.org/project/zcbor/), or alternatively invoked directly from its location in this repo.
Following are some generalized examples for validating, and for converting (which also validates) data from the command line.
The script infers the data format from the file extension, but the format can also be specified explicitly.
See `zcbor validate --help` and `zcbor convert --help` for more information.
```sh
zcbor validate -c <CDDL description file> -t <which CDDL type to expect> -i <input data file>
zcbor convert -c <CDDL description file> -t <which CDDL type to expect> -i <input data file> -o <output data file>
```
Or directly from within the repo.
```sh
python3 <zcbor base>/zcbor/zcbor.py validate -c <CDDL description file> -t <which CDDL type to expect> -i <input data file>
python3 <zcbor base>/zcbor/zcbor.py convert -c <CDDL description file> -t <which CDDL type to expect> -i <input data file> -o <output data file>
```
Importing zcbor in a Python script
----------------------------------
Importing zcbor gives access to the DataTranslator class which is used to implement the command line conversion features.
DataTranslator can be used to programmatically perform the translations, or to manipulate the data.
When accessing the data, you can choose between two internal formats:
1. The format provided by the [cbor2](https://pypi.org/project/cbor2/), [yaml (PyYAML)](https://pypi.org/project/PyYAML/), and [json](https://docs.python.org/3/library/json.html) packages.
This is a format where the serialization types (map, list, string, number etc.) are mapped directly to the corresponding Python types.
This format is common between these packages, which makes translation very simple.
When returning this format, DataTranslator hides the idiomatic representations for bytestrings, tags, and non-text keys described above.
2. A custom format which allows accessing the data via the names from the CDDL description file.
This format is implemented using named tuples, and is immutable, meaning that it can be used for inspecting data, but not for changing or creating data.
Making CBOR YAML-/JSON-compatible
---------------------------------
Since CBOR supports more data types than YAML and JSON, zcbor can optionally use a bespoke format when converting to/from YAML/JSON.
This is controlled with the `--yaml-compatibility` option to `convert` and `validate`.
This is relevant when handling YAML/JSON conversions of data that uses the unsupported features.
The following data types are supported by CBOR, but not by YAML (or JSON which is a subset of YAML):
1. bytestrings: YAML supports only text strings. In YAML, bytestrings are represented as `{"zcbor_bstr": "<hex-formatted bytestring>"}`, or as `{"zcbor_bstr": <any type>}` if the CBOR bytestring contains CBOR-formatted data, in which the data is decoded into `<any type>`.
2. map keys other than text string: In YAML, such key value pairs are represented as `{"zcbor_keyval<unique int>": {"key": <key, not text>, "val": <value>}}`.
3. tags: In cbor2, tags are represented by a special type, `cbor2.CBORTag`. In YAML, these are represented as `{"zcbor_tag": <tag number>, "zcbor_tag_val": <tagged data>}`.
4. undefined: In cbor2, undefined has its own value `cbor2.types.undefined`. In YAML, undefined is represented as: `["zcbor_undefined"]`.
You can see an example of the conversions in [tests/cases/yaml_compatibility.yaml](tests/cases/yaml_compatibility.yaml) and its CDDL file [tests/cases/yaml_compatibility.cddl](tests/cases/yaml_compatibility.cddl).
Code generation
===============
Code generation is invoked with the `zcbor code` command:
```sh
zcbor code <--decode or --encode or both> -c <CDDL description file(s)> -t <which CDDL type(s) to expose in the API> --output-cmake <path to place the generated CMake file at>
zcbor code <--decode or --encode or both> -c <CDDL description file(s)> -t <which CDDL type(s) to expose in the API> --oc <path to the generated C file> --oh <path to the generated header file> --oht <path to the generated types header>
```
When you call this, zcbor reads the CDDL files and creates C struct types to match the types described in the CDDL.
It then creates code that uses the C library to decode CBOR data into the structs, and/or encode CBOR from the data in the structs.
Finally, it takes the "entry types" (`-t`) and creates a public API function for each of them.
While doing these things, it will make a number of optimizations, e.g. inlining code for small types and removing unused functions.
It outputs the generated code into header and source files and optionally creates a CMake file to build them.
The `zcbor code` command reads one or more CDDL file(s) and generates some or all of these files:
- A header file with types (always)
- A header file with declarations for decoding functions (if `--decode`/`-d` is specified)
- A C file with decoding functions (if `--decode`/`-d` is specified)
- A header file with declarations for encoding functions (if `--encode`/`-e` is specified)
- A C file with encoding functions (if `--encode`/`-e` is specified)
- A CMake file that creates a library with the generated code and the C library (if `--output-cmake` is specified).
CDDL allows placing restrictions on the members of your data.
Restrictions can be on type (int/string/list/bool etc.), on content (e.g. values/sizes of ints or strings), and repetition (e.g. the number of members in a list).
The generated code will validate the input, which means that it will check all the restriction set in the CDDL description, and fail if a restriction is broken.
There are tests for the code generation in [tests/decode](tests/decode) and [tests/encode](tests/encode).
The tests require [Zephyr](https://github.com/zephyrproject-rtos/zephyr) (if your system is set up to build Zephyr samples, the tests should also build).
The generated C code is C++ compatible.
Build system
------------
When calling zcbor with the argument `--output-cmake <file path>`, a CMake file will be created at that location.
The generated CMake file creates a target library and adds the generated and non-generated source files as well as required include directories to it.
This CMake file can then be included in your project's `CMakeLists.txt` file, and the target can be linked into your project.
This is demonstrated in the tests, e.g. at [tests/decode/test3_simple/CMakeLists.txt](tests/decode/test3_simple/CMakeLists.txt).
zcbor can be instructed to copy the non-generated sources to the same location as the generated sources with `--copy-sources`.
Usage Example
=============
There are buildable examples in the [samples](samples) directory.
To see how to use the C library directly, see the [hello_world](samples/hello_world/src/main.c) sample, or the [pet](samples/pet/src/main.c) sample (look for calls to functions prefixed with `zcbor_`).
To see how to use code generation, see the [pet](samples/pet/src/main.c) sample.
Look at the [CMakeLists.txt](samples/pet/CMakeLists.txt) file to see how zcbor is invoked for code generation (and for conversion).
To see how to do conversion, see the [pet](samples/pet/CMakeLists.txt) sample.
Below are some additional examples of how to invoke zcbor for code generation and for converting/validating
Code generation
---------------
```sh
python3 <zcbor base>/zcbor/zcbor.py code -c pet.cddl -d -t Pet --oc pet_decode.c --oh pet_decode.h
# or
zcbor code -c pet.cddl -d -t Pet --oc pet_decode.c --oh pet_decode.h
```
Converting
----------
Here is an example call for converting from YAML to CBOR:
```sh
python3 <zcbor base>/zcbor/zcbor.py convert -c pet.cddl -t Pet -i mypet.yaml -o mypet.cbor
# or
zcbor convert -c pet.cddl -t Pet -i mypet.yaml -o mypet.cbor
```
Which takes a yaml structure from mypet.yaml, validates it against the Pet type in the CDDL description in pet.cddl, and writes binary CBOR data to mypet.cbor.
Validating
----------
Here is an example call for validating a JSON file:
```sh
python3 <zcbor base>/zcbor/zcbor.py validate -c pet.cddl -t Pet --yaml-compatibility -i mypet.json
# or
zcbor validate -c pet.cddl -t Pet --yaml-compatibility -i mypet.json
```
Which takes the json structure in mypet.json, converts any [yaml-compatible](#making-cbor-yaml-json-compatible) values to their original form, and validates that against the Pet type in the CDDL description in pet.cddl.
Running tests
=============
The tests for the generated code are based on the Zephyr ztest library.
These tests can be found in [tests/decode](tests/decode) and [tests/encode](tests/encode).
To set up the environment to run the ztest tests, follow [Zephyr's Getting Started Guide](https://docs.zephyrproject.org/latest/getting_started/index.html), or see the workflow in the [`.github`](.github) directory.
Tests for `convert` and `verify` are implemented with the unittest module.
These tests can be found in [tests/scripts/test_zcbor.py](tests/scripts/test_zcbor.py).
In this file there are also tests for code style of all python scripts, using the `pycodestyle` library.
Tests for the docs, samples, etc. can be found in [tests/scripts/test_repo_files.py](tests/scripts/test_repo_files.py).
For running the tests locally, there is [`tests/test.sh`](tests/test.sh) which runs all above tests.
Introduction to CDDL
====================
In CDDL you define types from other types.
Types can be defined from base types, or from other types you define.
Types are declared with '`=`', e.g. `Foo = int` which declares the type `Foo` to be an integer, analogous to `typedef int Foo;` in C.
CDDL defines the following base types (this is not an exhaustive list):
- `int`: Positive or negative integer
- `uint`: Positive integer
- `bstr`: Byte string
- `tstr`: Text string
- `bool`: Boolean
- `nil`: Nil/Null value
- `float`: Floating point value
- `any`: Any single element
CDDL allows creating aggregate types:
- `[]`: List. Elements don't need to have the same type.
- `{}`: Map. Key/value pairs as are declared as `<key> => <value>` or `<key>: <value>`. Note that `:` is also used for labels.
- `()`: Groups. Grouping with no enclosing type, which means that e.g. `Foo = [(int, bstr)]` is equivalent to `Foo = [int, bstr]`.
- `/`: Unions. Analogous to unions in C. E.g. `Foo = int/bstr/Bar` where Foo is either an int, a bstr, or Bar (some custom type).
Literals can be used instead of the base type names:
- Number: `Foo = 3`, where Foo is a uint with the additional requirement that it must have the value 3.
- Number range: `Foo = -100..100`, where Foo is an int with value between -100 and 100.
- Text string: `Foo = "hello"`, where Foo is a tstr with the requirement that it must be "hello".
- True/False: `Foo = false`, where Foo is a bool which is always false.
Base types can also be restricted in other ways:
- `.size`: Works for integers and strings. E.g. `Foo = uint .size 4` where Foo is a uint exactly 4 bytes long.
- `.cbor`/`.cborseq`: E.g. `Foo = bstr .cbor Bar` where Foo is a bstr whose contents must be CBOR data decodable as the Bar type.
An element can be repeated:
- `?`: 0 or 1 time. E.g. `Foo = [int, ?bstr]`, where Foo is a list with an int possibly followed by a bstr.
- `*`: 0 or more times. E.g. `Foo = [*tstr]`, where Foo is a list containing 0 or more tstrs.
- `+`: 1 or more times. E.g. `Foo = [+Bar]`.
- `x*y`: Between x and y times, inclusive. E.g. `Foo = {4*8(int => bstr)}` where Foo is a map with 4 to 8 key/value pairs where each key is an int and each value is a bstr.
Note that in the zcbor script and its generated code, the number of entries supported via `*` and `+` is affected by the default_max_qty value.
Any element can be labeled with `:`.
The label is only for readability and does not impact the data structure in any way.
E.g. `Foo = [name: tstr, age: uint]` is equivalent to `Foo = [tstr, uint]`.
See [pet.cddl](tests/cases/pet.cddl) for CDDL example code.
Introduction to CBOR
====================
CBOR's format is described well on [Wikipedia](https://en.wikipedia.org/wiki/CBOR), but here's a synopsis:
Encoded CBOR data elements look like this.
```
| Header | Value | Payload |
| 1 byte | 0, 1, 2, 4, or 8 bytes | 0 - 2^64-1 bytes/elements |
| 3 bits | 5 bits |
| Major Type | Additional Info |
```
The available major types can be seen in `zcbor_major_type_t`.
For all major types, Values 0-23 are encoded directly in the _Additional info_, meaning that the _Value_ field is 0 bytes long.
If _Additional info_ is 24, 25, 26, or 27, the _Value_ field is 1, 2, 4, or 8 bytes long, respectively.
Major types `pint` (0), `nint` (1), `tag` (6), and `simple` (7) elements have no payload, only _Value_.
* `pint`: Interpret the _Value_ as a positive integer.
* `nint`: Interpret the _Value_ as a positive integer, then multiply by -1 and subtract 1.
* `tag`: The _Value_ says something about the next non-tag element.
See the [CBOR tag documentation](https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml) for details.
* `simple`: Different _Additional info_ mean different things:
* 0-19: Unassigned simple values.
* 20: `false` simple value
* 21: `true` simple value
* 22: `null` simple value
* 23: `undefined` simple value
* 24: Interpret the _Value_ as a 1 byte simple value. These simple values are currently unassigned.
* 25: Interpret the _Value_ as an IEEE 754 float16.
* 26: Interpret the _Value_ as an IEEE 754 float32.
* 27: Interpret the _Value_ as an IEEE 754 float64.
* 31: End of an indefinite-length `list` or `map`.
For `bstr` (2), `tstr` (3), `list` (4), and `map` (5), the _Value_ describes the length of the _Payload_.
For `bstr` and `tstr`, the length is in bytes, for `list`, the length is in number of elements, and for `map`, the length is in number of key/value element pairs.
For `list` and `map`, sub elements are regular CBOR elements with their own _Header_, _Value_ and _Payload_. `list`s and `map`s can be recursively encoded.
If a `list` or `map` has _Additional info_ 31, it is "indefinite-length", which means it has an "unknown" number of elements.
Instead, its end is marked by a `simple` with _Additional info_ 31 (byte value 0xFF).
History
=======
zcbor (then "cddl-gen") was initially conceived as a code generation project.
It was inspired by the need to securely decode the complex manifest data structures in the [IETF SUIT specification](https://datatracker.ietf.org/doc/draft-ietf-suit-manifest/).
This is reflected in the fact that there are multiple zcbor tests that use the CDDL and examples from various revisions of that specification.
Decoding/deserializing data securely requires doing some quite repetitive checks on each data element, to be sure that you are not decoding gibberish.
This is where code generation could pull a lot of weight.
Later it was discovered that the CBOR library that was designed to used by generated code could be useful by itself.
The script was also expanded so it could directly manipulate CBOR data.
Since CBOR, YAML, and JSON are all represented in roughly the same way internally in Python, it was easy to expand that data manipulation to support YAML and JSON.
Some places where zcbor is currently used:
- [MCUboot's serial recovery mechanism](https://github.com/mcu-tools/mcuboot/blob/main/boot/boot_serial/src/boot_serial.c)
- [Zephyr's mcumgr](https://github.com/zephyrproject-rtos/zephyr/blob/main/subsys/mgmt/mcumgr/grp/img_mgmt/src/img_mgmt.c)
- [Zephyr's LwM2M SenML](https://github.com/zephyrproject-rtos/zephyr/blob/main/subsys/net/lib/lwm2m/lwm2m_rw_senml_cbor.c)
- [nRF Connect SDK's full modem update mechanism](https://github.com/nrfconnect/sdk-nrf/blob/main/subsys/mgmt/fmfu/src/fmfu_mgmt.c)
- [nRF Connect SDK's nrf_rpc](https://github.com/nrfconnect/sdk-nrfxlib/blob/main/nrf_rpc/nrf_rpc_cbor.c)
Command line documentation
==========================
Added via `add_helptext.py`
zcbor --help
------------
```
usage: zcbor [-h] [--version] {code,validate,convert} ...
Parse a CDDL file and validate/convert between YAML, JSON, and CBOR. Can also
generate C code for validation/encoding/decoding of CBOR.
positional arguments:
{code,validate,convert}
options:
-h, --help show this help message and exit
--version show program's version number and exit
```
zcbor code --help
-----------------
```
usage: zcbor code [-h] -c CDDL [--no-prelude] [-v]
[--default-max-qty DEFAULT_MAX_QTY] [--output-c OUTPUT_C]
[--output-h OUTPUT_H] [--output-h-types OUTPUT_H_TYPES]
[--copy-sources] [--output-cmake OUTPUT_CMAKE] -t
ENTRY_TYPES [ENTRY_TYPES ...] [-d] [-e] [--time-header]
[--git-sha-header] [-b {32,64}]
[--include-prefix INCLUDE_PREFIX] [-s]
[--file-header FILE_HEADER]
Parse a CDDL file and produce C code that validates and xcodes CBOR.
The output from this script is a C file and a header file. The header file
contains typedefs for all the types specified in the cddl input file, as well
as declarations to xcode functions for the types designated as entry types when
running the script. The c file contains all the code for decoding and validating
the types in the CDDL input file. All types are validated as they are xcoded.
Where a `bstr .cbor <Type>` is specified in the CDDL, AND the Type is an entry
type, the xcoder will not xcode the string, only provide a pointer into the
payload buffer. This is useful to reduce the size of typedefs, or to break up
decoding. Using this mechanism is necessary when the CDDL contains self-
referencing types, since the C type cannot be self referencing.
This script requires 'regex' for lookaround functionality not present in 're'.
options:
-h, --help show this help message and exit
-c CDDL, --cddl CDDL Path to one or more input CDDL file(s). Passing
multiple files is equivalent to concatenating them.
--no-prelude Exclude the standard CDDL prelude from the build. The
prelude can be viewed at zcbor/prelude.cddl in the
repo, or together with the script.
-v, --verbose Print more information while parsing CDDL and
generating code.
--default-max-qty DEFAULT_MAX_QTY, --dq DEFAULT_MAX_QTY
Default maximum number of repetitions when no maximum
is specified. This is needed to construct complete C
types. The default_max_qty can usually be set to a
text symbol if desired, to allow it to be configurable
when building the code. This is not always possible,
as sometimes the value is needed for internal
computations. If so, the script will raise an
exception.
--output-c OUTPUT_C, --oc OUTPUT_C
Path to output C file. If both --decode and --encode
are specified, _decode and _encode will be appended to
the filename when creating the two files. If not
specified, the path and name will be based on the
--output-cmake file. A 'src' directory will be created
next to the cmake file, and the C file will be placed
there with the same name (except the file extension)
as the cmake file.
--output-h OUTPUT_H, --oh OUTPUT_H
Path to output header file. If both --decode and
--encode are specified, _decode and _encode will be
appended to the filename when creating the two files.
If not specified, the path and name will be based on
the --output-cmake file. An 'include' directory will
be created next to the cmake file, and the C file will
be placed there with the same name (except the file
extension) as the cmake file.
--output-h-types OUTPUT_H_TYPES, --oht OUTPUT_H_TYPES
Path to output header file with typedefs (shared
between decode and encode). If not specified, the path
and name will be taken from the output header file
(--output-h), with '_types' added to the file name.
--copy-sources Copy the non-generated source files (zcbor_*.c/h) into
the same directories as the generated files.
--output-cmake OUTPUT_CMAKE
Path to output CMake file. The filename of the CMake
file without '.cmake' is used as the name of the CMake
target in the file. The CMake file defines a CMake
target with the zcbor source files and the generated
file as sources, and the zcbor header files' and
generated header files' folders as
include_directories. Add it to your project via
include() in your CMakeLists.txt file, and link the
target to your program. This option works with or
without the --copy-sources option.
-t ENTRY_TYPES [ENTRY_TYPES ...], --entry-types ENTRY_TYPES [ENTRY_TYPES ...]
Names of the types which should have their xcode
functions exposed.
-d, --decode Generate decoding code. Either --decode or --encode or
both must be specified.
-e, --encode Generate encoding code. Either --decode or --encode or
both must be specified.
--time-header Put the current time in a comment in the generated
files.
--git-sha-header Put the current git sha of zcbor in a comment in the
generated files.
-b {32,64}, --default-bit-size {32,64}
Default bit size of integers in code. When integers
have no explicit bounds, assume they have this bit
width. Should follow the bit width of the architecture
the code will be running on.
--include-prefix INCLUDE_PREFIX
When #include'ing generated files, add this path
prefix to the filename.
-s, --short-names Attempt to make most generated struct member names
shorter. This might make some names identical which
will cause a compile error. If so, tweak the CDDL
labels or layout, or disable this option. This might
also make enum names different from the corresponding
union members.
--file-header FILE_HEADER
Header to be included in the comment at the top of
generated C files, e.g. copyright.
```
zcbor validate --help
---------------------
```
usage: zcbor validate [-h] -c CDDL [--no-prelude] [-v] -i INPUT
[--input-as {yaml,json,cbor,cborhex}] -t ENTRY_TYPE
[--default-max-qty DEFAULT_MAX_QTY]
[--yaml-compatibility]
Read CBOR, YAML, or JSON data from file or stdin and validate it against a
CDDL schema file.
options:
-h, --help show this help message and exit
-c CDDL, --cddl CDDL Path to one or more input CDDL file(s). Passing
multiple files is equivalent to concatenating them.
--no-prelude Exclude the standard CDDL prelude from the build. The
prelude can be viewed at zcbor/prelude.cddl in the
repo, or together with the script.
-v, --verbose Print more information while parsing CDDL and
generating code.
-i INPUT, --input INPUT
Input data file. The option --input-as specifies how
to interpret the contents. Use "-" to indicate stdin.
--input-as {yaml,json,cbor,cborhex}
Which format to interpret the input file as. If
omitted, the format is inferred from the file name.
.yaml, .yml => YAML, .json => JSON, .cborhex => CBOR
as hex string, everything else => CBOR
-t ENTRY_TYPE, --entry-type ENTRY_TYPE
Name of the type (from the CDDL) to interpret the data
as.
--default-max-qty DEFAULT_MAX_QTY, --dq DEFAULT_MAX_QTY
Default maximum number of repetitions when no maximum
is specified. It is only relevant when handling data
that will be decoded by generated code. If omitted, a
large number will be used.
--yaml-compatibility Whether to convert CBOR-only values to YAML-compatible
ones (when converting from CBOR), or vice versa (when
converting to CBOR). When this is enabled, all CBOR
data is guaranteed to convert into YAML/JSON. JSON and
YAML do not support all data types that CBOR/CDDL
supports. bytestrings (BSTR), tags, undefined, and
maps with non-text keys need special handling. See the
zcbor README for more information.
```
zcbor convert --help
--------------------
```
usage: zcbor convert [-h] -c CDDL [--no-prelude] [-v] -i INPUT
[--input-as {yaml,json,cbor,cborhex}] -t ENTRY_TYPE
[--default-max-qty DEFAULT_MAX_QTY]
[--yaml-compatibility] -o OUTPUT
[--output-as {yaml,json,cbor,cborhex,c_code}]
[--c-code-var-name C_CODE_VAR_NAME]
[--c-code-columns C_CODE_COLUMNS]
Parse a CDDL file and validate/convert between CBOR and YAML/JSON. The script
decodes the CBOR/YAML/JSON data from a file or stdin and verifies that it
conforms to the CDDL description. The script fails if the data does not
conform. 'zcbor validate' can be used if only validate is needed.
options:
-h, --help show this help message and exit
-c CDDL, --cddl CDDL Path to one or more input CDDL file(s). Passing
multiple files is equivalent to concatenating them.
--no-prelude Exclude the standard CDDL prelude from the build. The
prelude can be viewed at zcbor/prelude.cddl in the
repo, or together with the script.
-v, --verbose Print more information while parsing CDDL and
generating code.
-i INPUT, --input INPUT
Input data file. The option --input-as specifies how
to interpret the contents. Use "-" to indicate stdin.
--input-as {yaml,json,cbor,cborhex}
Which format to interpret the input file as. If
omitted, the format is inferred from the file name.
.yaml, .yml => YAML, .json => JSON, .cborhex => CBOR
as hex string, everything else => CBOR
-t ENTRY_TYPE, --entry-type ENTRY_TYPE
Name of the type (from the CDDL) to interpret the data
as.
--default-max-qty DEFAULT_MAX_QTY, --dq DEFAULT_MAX_QTY
Default maximum number of repetitions when no maximum
is specified. It is only relevant when handling data
that will be decoded by generated code. If omitted, a
large number will be used.
--yaml-compatibility Whether to convert CBOR-only values to YAML-compatible
ones (when converting from CBOR), or vice versa (when
converting to CBOR). When this is enabled, all CBOR
data is guaranteed to convert into YAML/JSON. JSON and
YAML do not support all data types that CBOR/CDDL
supports. bytestrings (BSTR), tags, undefined, and
maps with non-text keys need special handling. See the
zcbor README for more information.
-o OUTPUT, --output OUTPUT
Output data file. The option --output-as specifies how
to interpret the contents. Use "-" to indicate stdout.
--output-as {yaml,json,cbor,cborhex,c_code}
Which format to interpret the output file as. If
omitted, the format is inferred from the file name.
.yaml, .yml => YAML, .json => JSON, .c, .h => C code,
.cborhex => CBOR as hex string, everything else =>
CBOR
--c-code-var-name C_CODE_VAR_NAME
Only relevant together with '--output-as c_code' or .c
files.
--c-code-columns C_CODE_COLUMNS
Only relevant together with '--output-as c_code' or .c
files. The number of bytes per line in the variable
instantiation. If omitted, the entire declaration is a
single line.
```
Raw data
{
"_id": null,
"home_page": "",
"name": "zcbor",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "\u00d8yvind R\u00f8nningstad <oyvind.ronningstad@nordicsemi.no>",
"keywords": "",
"author": "Nordic Semiconductor ASA",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/9a/fa/0a082e28646618669fc09626d8ad88a82f3b7de999e02496cb4dad9fc4eb/zcbor-0.8.1.tar.gz",
"platform": null,
"description": "zcbor\n=====\n\nzcbor is a low footprint [CBOR](https://en.wikipedia.org/wiki/CBOR) library in the C language (C++ compatible), tailored for use in microcontrollers.\nIt comes with a schema-driven script tool that can validate your data, or even generate code.\nThe schema language (CDDL) allows creating very advanced and detailed schemas.\n\nThe validation and conversion part of the tool works with YAML and JSON data, in addition to CBOR.\nIt can for example validate a YAML file against a schema and convert it into CBOR.\n\nThe code generation part of the tool generates C code based on the given schema.\nThe generated code performs CBOR encoding and decoding using the C library, while also validating the data against all the rules in the schema.\n\nThe schema language used by zcbor is CDDL (Concise Data Definition Language) which is a powerful human-readable data description language defined in [IETF RFC 8610](https://datatracker.ietf.org/doc/rfc8610/).\n\n\nFeatures\n========\n\nHere are some possible ways zcbor can be used:\n\n - C code:\n - As a low-footprint CBOR decoding/encoding library similar to TinyCBOR/QCBOR/NanoCBOR. The library can be used independently of the Python script. ([More information](#cbor-decodingencoding-library))\n - To generate C code (using the Python script) for validating and decoding or encoding CBOR, for use in optimized or constrained environments, such as microcontrollers. ([More information](#code-generation))\n - Python script and module ([More information](#python-script-and-module)):\n - Validate a YAML/JSON file and translate it into CBOR e.g. for transmission.\n - Validate a YAML/JSON/CBOR file before processing it with some other tool\n - Decode and validate incoming CBOR data into human-readable YAML/JSON.\n - As part of a python script that processes YAML/JSON/CBOR files.\n - Uses the same internal representation used by the PyYAML/json/cbor2 libraries.\n - Do validation against a CDDL schema.\n - Create a read-only representation via named tuples (with names taken from the CDDL schema).\n\n\nGetting started\n===============\n\nThere are samples in the [samples](samples) directory that demonstrate different ways to use zcbor, both the script tool and the C code.\n\n1. The [hello_world sample](samples/hello_world/README.md) is a minimum examples of encoding and decoding using the C library.\n2. The [pet sample](samples/pet/README.md) shows a how to use the C library together with generated code, and how to use the script tool to do code generation and data conversion.\n\nThe [tests](tests) also demonstrate how to use zcbor in different ways. The [encoding](tests/encode), [decoding](tests/decode), and [unit](tests/unit) tests run using [Zephyr](https://github.com/zephyrproject-rtos/zephyr) (the samples do not use Zephyr).\n\nShould I use code generation or the library directly?\n-----------------------------------------------------\n\nThe benefit of using code generation is greater for decoding than encoding.\nThis is because decoding is generally more complex than encoding, since when decoding you have to gracefully handle all possible payloads.\nThe code generation will provide a number of checks that are tedious to write manually.\nThese checks ensure that the payload is well-formed.\n\n\nCBOR decoding/encoding library\n==============================\n\nThe CBOR library can be found in [include/](include) and [src/](src) and can be used directly, by including the files in your project.\nIf using zcbor with Zephyr, the library will be available when the [CONFIG_ZCBOR](https://docs.zephyrproject.org/latest/kconfig.html#CONFIG_ZCBOR) config is enabled.\n\nThe library is also used by generated code. See the [Code generation](#code-generation) section for more info about code generation.\n\nThe C library is C++ compatible.\n\nThe zcbor state object\n----------------------\n\nTo do encoding or decoding with the library, instantiate a `zcbor_state_t` object, which is most easily done using the `ZCBOR_STATE_*()` macros, look below or in the [hello_world](samples/hello_world/src/main.c) sample for example code.\n\nThe `elem_count` member refers to the number of encoded objects in the current list or map.\n`elem_count` starts again when entering a nested list or map, and is restored when exiting.\n\n`elem_count` is one reason for needing \"backup\" states (the other is to allow rollback of the payload).\nBackups are needed for _decoding_ if there are any lists, maps, or CBOR-encoded strings (`zcbor_bstr_*_decode`) in the data.\nBackups are needed for _encoding_ if there are any lists or maps *and* you are using canonical encoding (`ZCBOR_CANONICAL`), or when using the `zcbor_bstr_*_encode` functions.\n\n```c\n/** Initialize a decoding state (could include an array of backup states).\n * After calling this, decode_state[0] is ready to be used with the decoding APIs. */\nZCBOR_STATE_D(decode_state, n, payload, payload_len, elem_count, n_flags);\n\n/** Initialize an encoding state (could include an array of backup states).\n * After calling this, encode_state[0] is ready to be used with the encoding APIs. */\nZCBOR_STATE_E(encode_state, n, payload, payload_len, 0);\n```\n\nConfiguration\n-------------\n\nThe C library has a few compile-time configuration options.\nThese configuration options can be enabled by adding them as compile definitions to the build.\nIf using zcbor with Zephyr, use the [Kconfig options](https://github.com/zephyrproject-rtos/zephyr/blob/main/modules/zcbor/Kconfig) instead.\n\nName | Description\n------------------------- | -----------\n`ZCBOR_CANONICAL` | Assume canonical encoding (AKA \"deterministically encoded CBOR\"). When encoding lists and maps, do not use indefinite length encoding. Enabling `ZCBOR_CANONICAL` increases code size and makes the encoding library more often use state backups. When decoding, ensure that the incoming data conforms to canonical encoding, i.e. no indefinite length encoding, and always using minimal length encoding (e.g. not using 16 bits to encode a value < 256). Note: the map ordering constraint in canonical encoding is not checked.\n`ZCBOR_VERBOSE` | Print log messages on encoding/decoding errors (`zcbor_log()`), and also a trace message (`zcbor_trace()`) for each decoded value, and in each generated function (when using code generation).\n`ZCBOR_ASSERTS` | Enable asserts (`zcbor_assert()`). When they fail, the assert statements instruct the current function to return a `ZCBOR_ERR_ASSERTION` error. If `ZCBOR_VERBOSE` is enabled, a message is printed.\n`ZCBOR_STOP_ON_ERROR` | Enable the `stop_on_error` functionality. This makes all functions abort their execution if called when an error has already happened.\n`ZCBOR_BIG_ENDIAN` | All decoded values are returned as big-endian. The default is little-endian.\n`ZCBOR_MAP_SMART_SEARCH` | Applies to decoding of unordered maps. When enabled, a flag is kept for each element in an array, ensuring it is not processed twice. If disabled, a count is kept for map as a whole. Enabling increases code size and memory usage, and requires the state variable to possess the memory necessary for the flags.\n\n\nPython script and module\n========================\n\nThe zcbor.py script can directly read CBOR, YAML, or JSON data and validate it against a CDDL description.\nIt can also freely convert the data between CBOR/YAML/JSON.\nIt can also output the data to a C file formatted as a byte array.\n\nInvoking zcbor.py from the command line\n---------------------------------------\n\nzcbor.py can be installed via [`pip`](https://pypi.org/project/zcbor/), or alternatively invoked directly from its location in this repo.\n\nFollowing are some generalized examples for validating, and for converting (which also validates) data from the command line.\nThe script infers the data format from the file extension, but the format can also be specified explicitly.\nSee `zcbor validate --help` and `zcbor convert --help` for more information.\n\n```sh\nzcbor validate -c <CDDL description file> -t <which CDDL type to expect> -i <input data file>\nzcbor convert -c <CDDL description file> -t <which CDDL type to expect> -i <input data file> -o <output data file>\n```\n\nOr directly from within the repo.\n\n```sh\npython3 <zcbor base>/zcbor/zcbor.py validate -c <CDDL description file> -t <which CDDL type to expect> -i <input data file>\npython3 <zcbor base>/zcbor/zcbor.py convert -c <CDDL description file> -t <which CDDL type to expect> -i <input data file> -o <output data file>\n```\n\nImporting zcbor in a Python script\n----------------------------------\n\nImporting zcbor gives access to the DataTranslator class which is used to implement the command line conversion features.\nDataTranslator can be used to programmatically perform the translations, or to manipulate the data.\nWhen accessing the data, you can choose between two internal formats:\n\n 1. The format provided by the [cbor2](https://pypi.org/project/cbor2/), [yaml (PyYAML)](https://pypi.org/project/PyYAML/), and [json](https://docs.python.org/3/library/json.html) packages.\n This is a format where the serialization types (map, list, string, number etc.) are mapped directly to the corresponding Python types.\n This format is common between these packages, which makes translation very simple.\n When returning this format, DataTranslator hides the idiomatic representations for bytestrings, tags, and non-text keys described above.\n 2. A custom format which allows accessing the data via the names from the CDDL description file.\n This format is implemented using named tuples, and is immutable, meaning that it can be used for inspecting data, but not for changing or creating data.\n\nMaking CBOR YAML-/JSON-compatible\n---------------------------------\n\nSince CBOR supports more data types than YAML and JSON, zcbor can optionally use a bespoke format when converting to/from YAML/JSON.\nThis is controlled with the `--yaml-compatibility` option to `convert` and `validate`.\nThis is relevant when handling YAML/JSON conversions of data that uses the unsupported features.\nThe following data types are supported by CBOR, but not by YAML (or JSON which is a subset of YAML):\n\n 1. bytestrings: YAML supports only text strings. In YAML, bytestrings are represented as `{\"zcbor_bstr\": \"<hex-formatted bytestring>\"}`, or as `{\"zcbor_bstr\": <any type>}` if the CBOR bytestring contains CBOR-formatted data, in which the data is decoded into `<any type>`.\n 2. map keys other than text string: In YAML, such key value pairs are represented as `{\"zcbor_keyval<unique int>\": {\"key\": <key, not text>, \"val\": <value>}}`.\n 3. tags: In cbor2, tags are represented by a special type, `cbor2.CBORTag`. In YAML, these are represented as `{\"zcbor_tag\": <tag number>, \"zcbor_tag_val\": <tagged data>}`.\n 4. undefined: In cbor2, undefined has its own value `cbor2.types.undefined`. In YAML, undefined is represented as: `[\"zcbor_undefined\"]`.\n\nYou can see an example of the conversions in [tests/cases/yaml_compatibility.yaml](tests/cases/yaml_compatibility.yaml) and its CDDL file [tests/cases/yaml_compatibility.cddl](tests/cases/yaml_compatibility.cddl).\n\n\nCode generation\n===============\n\nCode generation is invoked with the `zcbor code` command:\n\n```sh\nzcbor code <--decode or --encode or both> -c <CDDL description file(s)> -t <which CDDL type(s) to expose in the API> --output-cmake <path to place the generated CMake file at>\nzcbor code <--decode or --encode or both> -c <CDDL description file(s)> -t <which CDDL type(s) to expose in the API> --oc <path to the generated C file> --oh <path to the generated header file> --oht <path to the generated types header>\n```\n\nWhen you call this, zcbor reads the CDDL files and creates C struct types to match the types described in the CDDL.\nIt then creates code that uses the C library to decode CBOR data into the structs, and/or encode CBOR from the data in the structs.\nFinally, it takes the \"entry types\" (`-t`) and creates a public API function for each of them.\nWhile doing these things, it will make a number of optimizations, e.g. inlining code for small types and removing unused functions.\nIt outputs the generated code into header and source files and optionally creates a CMake file to build them.\n\nThe `zcbor code` command reads one or more CDDL file(s) and generates some or all of these files:\n - A header file with types (always)\n - A header file with declarations for decoding functions (if `--decode`/`-d` is specified)\n - A C file with decoding functions (if `--decode`/`-d` is specified)\n - A header file with declarations for encoding functions (if `--encode`/`-e` is specified)\n - A C file with encoding functions (if `--encode`/`-e` is specified)\n - A CMake file that creates a library with the generated code and the C library (if `--output-cmake` is specified).\n\nCDDL allows placing restrictions on the members of your data.\nRestrictions can be on type (int/string/list/bool etc.), on content (e.g. values/sizes of ints or strings), and repetition (e.g. the number of members in a list).\nThe generated code will validate the input, which means that it will check all the restriction set in the CDDL description, and fail if a restriction is broken.\n\nThere are tests for the code generation in [tests/decode](tests/decode) and [tests/encode](tests/encode).\nThe tests require [Zephyr](https://github.com/zephyrproject-rtos/zephyr) (if your system is set up to build Zephyr samples, the tests should also build).\n\nThe generated C code is C++ compatible.\n\nBuild system\n------------\n\nWhen calling zcbor with the argument `--output-cmake <file path>`, a CMake file will be created at that location.\nThe generated CMake file creates a target library and adds the generated and non-generated source files as well as required include directories to it.\nThis CMake file can then be included in your project's `CMakeLists.txt` file, and the target can be linked into your project.\nThis is demonstrated in the tests, e.g. at [tests/decode/test3_simple/CMakeLists.txt](tests/decode/test3_simple/CMakeLists.txt).\nzcbor can be instructed to copy the non-generated sources to the same location as the generated sources with `--copy-sources`.\n\n\nUsage Example\n=============\n\nThere are buildable examples in the [samples](samples) directory.\n\nTo see how to use the C library directly, see the [hello_world](samples/hello_world/src/main.c) sample, or the [pet](samples/pet/src/main.c) sample (look for calls to functions prefixed with `zcbor_`).\n\nTo see how to use code generation, see the [pet](samples/pet/src/main.c) sample.\n\nLook at the [CMakeLists.txt](samples/pet/CMakeLists.txt) file to see how zcbor is invoked for code generation (and for conversion).\n\nTo see how to do conversion, see the [pet](samples/pet/CMakeLists.txt) sample.\n\nBelow are some additional examples of how to invoke zcbor for code generation and for converting/validating\n\nCode generation\n---------------\n\n```sh\npython3 <zcbor base>/zcbor/zcbor.py code -c pet.cddl -d -t Pet --oc pet_decode.c --oh pet_decode.h\n# or\nzcbor code -c pet.cddl -d -t Pet --oc pet_decode.c --oh pet_decode.h\n```\n\nConverting\n----------\n\nHere is an example call for converting from YAML to CBOR:\n\n```sh\npython3 <zcbor base>/zcbor/zcbor.py convert -c pet.cddl -t Pet -i mypet.yaml -o mypet.cbor\n# or\nzcbor convert -c pet.cddl -t Pet -i mypet.yaml -o mypet.cbor\n```\n\nWhich takes a yaml structure from mypet.yaml, validates it against the Pet type in the CDDL description in pet.cddl, and writes binary CBOR data to mypet.cbor.\n\nValidating\n----------\n\nHere is an example call for validating a JSON file:\n\n```sh\npython3 <zcbor base>/zcbor/zcbor.py validate -c pet.cddl -t Pet --yaml-compatibility -i mypet.json\n# or\nzcbor validate -c pet.cddl -t Pet --yaml-compatibility -i mypet.json\n```\n\nWhich takes the json structure in mypet.json, converts any [yaml-compatible](#making-cbor-yaml-json-compatible) values to their original form, and validates that against the Pet type in the CDDL description in pet.cddl.\n\n\nRunning tests\n=============\n\nThe tests for the generated code are based on the Zephyr ztest library.\nThese tests can be found in [tests/decode](tests/decode) and [tests/encode](tests/encode).\nTo set up the environment to run the ztest tests, follow [Zephyr's Getting Started Guide](https://docs.zephyrproject.org/latest/getting_started/index.html), or see the workflow in the [`.github`](.github) directory.\n\nTests for `convert` and `verify` are implemented with the unittest module.\nThese tests can be found in [tests/scripts/test_zcbor.py](tests/scripts/test_zcbor.py).\nIn this file there are also tests for code style of all python scripts, using the `pycodestyle` library.\n\nTests for the docs, samples, etc. can be found in [tests/scripts/test_repo_files.py](tests/scripts/test_repo_files.py).\n\nFor running the tests locally, there is [`tests/test.sh`](tests/test.sh) which runs all above tests.\n\n\nIntroduction to CDDL\n====================\n\nIn CDDL you define types from other types.\nTypes can be defined from base types, or from other types you define.\nTypes are declared with '`=`', e.g. `Foo = int` which declares the type `Foo` to be an integer, analogous to `typedef int Foo;` in C.\nCDDL defines the following base types (this is not an exhaustive list):\n\n - `int`: Positive or negative integer\n - `uint`: Positive integer\n - `bstr`: Byte string\n - `tstr`: Text string\n - `bool`: Boolean\n - `nil`: Nil/Null value\n - `float`: Floating point value\n - `any`: Any single element\n\nCDDL allows creating aggregate types:\n\n - `[]`: List. Elements don't need to have the same type.\n - `{}`: Map. Key/value pairs as are declared as `<key> => <value>` or `<key>: <value>`. Note that `:` is also used for labels.\n - `()`: Groups. Grouping with no enclosing type, which means that e.g. `Foo = [(int, bstr)]` is equivalent to `Foo = [int, bstr]`.\n - `/`: Unions. Analogous to unions in C. E.g. `Foo = int/bstr/Bar` where Foo is either an int, a bstr, or Bar (some custom type).\n\nLiterals can be used instead of the base type names:\n\n - Number: `Foo = 3`, where Foo is a uint with the additional requirement that it must have the value 3.\n - Number range: `Foo = -100..100`, where Foo is an int with value between -100 and 100.\n - Text string: `Foo = \"hello\"`, where Foo is a tstr with the requirement that it must be \"hello\".\n - True/False: `Foo = false`, where Foo is a bool which is always false.\n\nBase types can also be restricted in other ways:\n\n - `.size`: Works for integers and strings. E.g. `Foo = uint .size 4` where Foo is a uint exactly 4 bytes long.\n - `.cbor`/`.cborseq`: E.g. `Foo = bstr .cbor Bar` where Foo is a bstr whose contents must be CBOR data decodable as the Bar type.\n\nAn element can be repeated:\n\n - `?`: 0 or 1 time. E.g. `Foo = [int, ?bstr]`, where Foo is a list with an int possibly followed by a bstr.\n - `*`: 0 or more times. E.g. `Foo = [*tstr]`, where Foo is a list containing 0 or more tstrs.\n - `+`: 1 or more times. E.g. `Foo = [+Bar]`.\n - `x*y`: Between x and y times, inclusive. E.g. `Foo = {4*8(int => bstr)}` where Foo is a map with 4 to 8 key/value pairs where each key is an int and each value is a bstr.\n\nNote that in the zcbor script and its generated code, the number of entries supported via `*` and `+` is affected by the default_max_qty value.\n\nAny element can be labeled with `:`.\nThe label is only for readability and does not impact the data structure in any way.\nE.g. `Foo = [name: tstr, age: uint]` is equivalent to `Foo = [tstr, uint]`.\n\nSee [pet.cddl](tests/cases/pet.cddl) for CDDL example code.\n\n\nIntroduction to CBOR\n====================\n\nCBOR's format is described well on [Wikipedia](https://en.wikipedia.org/wiki/CBOR), but here's a synopsis:\n\nEncoded CBOR data elements look like this.\n\n```\n| Header | Value | Payload |\n| 1 byte | 0, 1, 2, 4, or 8 bytes | 0 - 2^64-1 bytes/elements |\n| 3 bits | 5 bits |\n| Major Type | Additional Info |\n```\n\nThe available major types can be seen in `zcbor_major_type_t`.\n\nFor all major types, Values 0-23 are encoded directly in the _Additional info_, meaning that the _Value_ field is 0 bytes long.\nIf _Additional info_ is 24, 25, 26, or 27, the _Value_ field is 1, 2, 4, or 8 bytes long, respectively.\n\nMajor types `pint` (0), `nint` (1), `tag` (6), and `simple` (7) elements have no payload, only _Value_.\n\n * `pint`: Interpret the _Value_ as a positive integer.\n * `nint`: Interpret the _Value_ as a positive integer, then multiply by -1 and subtract 1.\n * `tag`: The _Value_ says something about the next non-tag element.\n See the [CBOR tag documentation](https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml) for details.\n * `simple`: Different _Additional info_ mean different things:\n * 0-19: Unassigned simple values.\n * 20: `false` simple value\n * 21: `true` simple value\n * 22: `null` simple value\n * 23: `undefined` simple value\n * 24: Interpret the _Value_ as a 1 byte simple value. These simple values are currently unassigned.\n * 25: Interpret the _Value_ as an IEEE 754 float16.\n * 26: Interpret the _Value_ as an IEEE 754 float32.\n * 27: Interpret the _Value_ as an IEEE 754 float64.\n * 31: End of an indefinite-length `list` or `map`.\n\nFor `bstr` (2), `tstr` (3), `list` (4), and `map` (5), the _Value_ describes the length of the _Payload_.\nFor `bstr` and `tstr`, the length is in bytes, for `list`, the length is in number of elements, and for `map`, the length is in number of key/value element pairs.\n\nFor `list` and `map`, sub elements are regular CBOR elements with their own _Header_, _Value_ and _Payload_. `list`s and `map`s can be recursively encoded.\nIf a `list` or `map` has _Additional info_ 31, it is \"indefinite-length\", which means it has an \"unknown\" number of elements.\nInstead, its end is marked by a `simple` with _Additional info_ 31 (byte value 0xFF).\n\n\nHistory\n=======\n\nzcbor (then \"cddl-gen\") was initially conceived as a code generation project.\nIt was inspired by the need to securely decode the complex manifest data structures in the [IETF SUIT specification](https://datatracker.ietf.org/doc/draft-ietf-suit-manifest/).\nThis is reflected in the fact that there are multiple zcbor tests that use the CDDL and examples from various revisions of that specification.\nDecoding/deserializing data securely requires doing some quite repetitive checks on each data element, to be sure that you are not decoding gibberish.\nThis is where code generation could pull a lot of weight.\nLater it was discovered that the CBOR library that was designed to used by generated code could be useful by itself.\nThe script was also expanded so it could directly manipulate CBOR data.\nSince CBOR, YAML, and JSON are all represented in roughly the same way internally in Python, it was easy to expand that data manipulation to support YAML and JSON.\n\nSome places where zcbor is currently used:\n- [MCUboot's serial recovery mechanism](https://github.com/mcu-tools/mcuboot/blob/main/boot/boot_serial/src/boot_serial.c)\n- [Zephyr's mcumgr](https://github.com/zephyrproject-rtos/zephyr/blob/main/subsys/mgmt/mcumgr/grp/img_mgmt/src/img_mgmt.c)\n- [Zephyr's LwM2M SenML](https://github.com/zephyrproject-rtos/zephyr/blob/main/subsys/net/lib/lwm2m/lwm2m_rw_senml_cbor.c)\n- [nRF Connect SDK's full modem update mechanism](https://github.com/nrfconnect/sdk-nrf/blob/main/subsys/mgmt/fmfu/src/fmfu_mgmt.c)\n- [nRF Connect SDK's nrf_rpc](https://github.com/nrfconnect/sdk-nrfxlib/blob/main/nrf_rpc/nrf_rpc_cbor.c)\n\n\nCommand line documentation\n==========================\n\nAdded via `add_helptext.py`\n\nzcbor --help\n------------\n\n```\nusage: zcbor [-h] [--version] {code,validate,convert} ...\n\nParse a CDDL file and validate/convert between YAML, JSON, and CBOR. Can also\ngenerate C code for validation/encoding/decoding of CBOR.\n\npositional arguments:\n {code,validate,convert}\n\noptions:\n -h, --help show this help message and exit\n --version show program's version number and exit\n\n```\n\nzcbor code --help\n-----------------\n\n```\nusage: zcbor code [-h] -c CDDL [--no-prelude] [-v]\n [--default-max-qty DEFAULT_MAX_QTY] [--output-c OUTPUT_C]\n [--output-h OUTPUT_H] [--output-h-types OUTPUT_H_TYPES]\n [--copy-sources] [--output-cmake OUTPUT_CMAKE] -t\n ENTRY_TYPES [ENTRY_TYPES ...] [-d] [-e] [--time-header]\n [--git-sha-header] [-b {32,64}]\n [--include-prefix INCLUDE_PREFIX] [-s]\n [--file-header FILE_HEADER]\n\nParse a CDDL file and produce C code that validates and xcodes CBOR.\nThe output from this script is a C file and a header file. The header file\ncontains typedefs for all the types specified in the cddl input file, as well\nas declarations to xcode functions for the types designated as entry types when\nrunning the script. The c file contains all the code for decoding and validating\nthe types in the CDDL input file. All types are validated as they are xcoded.\n\nWhere a `bstr .cbor <Type>` is specified in the CDDL, AND the Type is an entry\ntype, the xcoder will not xcode the string, only provide a pointer into the\npayload buffer. This is useful to reduce the size of typedefs, or to break up\ndecoding. Using this mechanism is necessary when the CDDL contains self-\nreferencing types, since the C type cannot be self referencing.\n\nThis script requires 'regex' for lookaround functionality not present in 're'.\n\noptions:\n -h, --help show this help message and exit\n -c CDDL, --cddl CDDL Path to one or more input CDDL file(s). Passing\n multiple files is equivalent to concatenating them.\n --no-prelude Exclude the standard CDDL prelude from the build. The\n prelude can be viewed at zcbor/prelude.cddl in the\n repo, or together with the script.\n -v, --verbose Print more information while parsing CDDL and\n generating code.\n --default-max-qty DEFAULT_MAX_QTY, --dq DEFAULT_MAX_QTY\n Default maximum number of repetitions when no maximum\n is specified. This is needed to construct complete C\n types. The default_max_qty can usually be set to a\n text symbol if desired, to allow it to be configurable\n when building the code. This is not always possible,\n as sometimes the value is needed for internal\n computations. If so, the script will raise an\n exception.\n --output-c OUTPUT_C, --oc OUTPUT_C\n Path to output C file. If both --decode and --encode\n are specified, _decode and _encode will be appended to\n the filename when creating the two files. If not\n specified, the path and name will be based on the\n --output-cmake file. A 'src' directory will be created\n next to the cmake file, and the C file will be placed\n there with the same name (except the file extension)\n as the cmake file.\n --output-h OUTPUT_H, --oh OUTPUT_H\n Path to output header file. If both --decode and\n --encode are specified, _decode and _encode will be\n appended to the filename when creating the two files.\n If not specified, the path and name will be based on\n the --output-cmake file. An 'include' directory will\n be created next to the cmake file, and the C file will\n be placed there with the same name (except the file\n extension) as the cmake file.\n --output-h-types OUTPUT_H_TYPES, --oht OUTPUT_H_TYPES\n Path to output header file with typedefs (shared\n between decode and encode). If not specified, the path\n and name will be taken from the output header file\n (--output-h), with '_types' added to the file name.\n --copy-sources Copy the non-generated source files (zcbor_*.c/h) into\n the same directories as the generated files.\n --output-cmake OUTPUT_CMAKE\n Path to output CMake file. The filename of the CMake\n file without '.cmake' is used as the name of the CMake\n target in the file. The CMake file defines a CMake\n target with the zcbor source files and the generated\n file as sources, and the zcbor header files' and\n generated header files' folders as\n include_directories. Add it to your project via\n include() in your CMakeLists.txt file, and link the\n target to your program. This option works with or\n without the --copy-sources option.\n -t ENTRY_TYPES [ENTRY_TYPES ...], --entry-types ENTRY_TYPES [ENTRY_TYPES ...]\n Names of the types which should have their xcode\n functions exposed.\n -d, --decode Generate decoding code. Either --decode or --encode or\n both must be specified.\n -e, --encode Generate encoding code. Either --decode or --encode or\n both must be specified.\n --time-header Put the current time in a comment in the generated\n files.\n --git-sha-header Put the current git sha of zcbor in a comment in the\n generated files.\n -b {32,64}, --default-bit-size {32,64}\n Default bit size of integers in code. When integers\n have no explicit bounds, assume they have this bit\n width. Should follow the bit width of the architecture\n the code will be running on.\n --include-prefix INCLUDE_PREFIX\n When #include'ing generated files, add this path\n prefix to the filename.\n -s, --short-names Attempt to make most generated struct member names\n shorter. This might make some names identical which\n will cause a compile error. If so, tweak the CDDL\n labels or layout, or disable this option. This might\n also make enum names different from the corresponding\n union members.\n --file-header FILE_HEADER\n Header to be included in the comment at the top of\n generated C files, e.g. copyright.\n\n```\n\nzcbor validate --help\n---------------------\n\n```\nusage: zcbor validate [-h] -c CDDL [--no-prelude] [-v] -i INPUT\n [--input-as {yaml,json,cbor,cborhex}] -t ENTRY_TYPE\n [--default-max-qty DEFAULT_MAX_QTY]\n [--yaml-compatibility]\n\nRead CBOR, YAML, or JSON data from file or stdin and validate it against a\nCDDL schema file.\n\noptions:\n -h, --help show this help message and exit\n -c CDDL, --cddl CDDL Path to one or more input CDDL file(s). Passing\n multiple files is equivalent to concatenating them.\n --no-prelude Exclude the standard CDDL prelude from the build. The\n prelude can be viewed at zcbor/prelude.cddl in the\n repo, or together with the script.\n -v, --verbose Print more information while parsing CDDL and\n generating code.\n -i INPUT, --input INPUT\n Input data file. The option --input-as specifies how\n to interpret the contents. Use \"-\" to indicate stdin.\n --input-as {yaml,json,cbor,cborhex}\n Which format to interpret the input file as. If\n omitted, the format is inferred from the file name.\n .yaml, .yml => YAML, .json => JSON, .cborhex => CBOR\n as hex string, everything else => CBOR\n -t ENTRY_TYPE, --entry-type ENTRY_TYPE\n Name of the type (from the CDDL) to interpret the data\n as.\n --default-max-qty DEFAULT_MAX_QTY, --dq DEFAULT_MAX_QTY\n Default maximum number of repetitions when no maximum\n is specified. It is only relevant when handling data\n that will be decoded by generated code. If omitted, a\n large number will be used.\n --yaml-compatibility Whether to convert CBOR-only values to YAML-compatible\n ones (when converting from CBOR), or vice versa (when\n converting to CBOR). When this is enabled, all CBOR\n data is guaranteed to convert into YAML/JSON. JSON and\n YAML do not support all data types that CBOR/CDDL\n supports. bytestrings (BSTR), tags, undefined, and\n maps with non-text keys need special handling. See the\n zcbor README for more information.\n\n```\n\nzcbor convert --help\n--------------------\n\n```\nusage: zcbor convert [-h] -c CDDL [--no-prelude] [-v] -i INPUT\n [--input-as {yaml,json,cbor,cborhex}] -t ENTRY_TYPE\n [--default-max-qty DEFAULT_MAX_QTY]\n [--yaml-compatibility] -o OUTPUT\n [--output-as {yaml,json,cbor,cborhex,c_code}]\n [--c-code-var-name C_CODE_VAR_NAME]\n [--c-code-columns C_CODE_COLUMNS]\n\nParse a CDDL file and validate/convert between CBOR and YAML/JSON. The script\ndecodes the CBOR/YAML/JSON data from a file or stdin and verifies that it\nconforms to the CDDL description. The script fails if the data does not\nconform. 'zcbor validate' can be used if only validate is needed.\n\noptions:\n -h, --help show this help message and exit\n -c CDDL, --cddl CDDL Path to one or more input CDDL file(s). Passing\n multiple files is equivalent to concatenating them.\n --no-prelude Exclude the standard CDDL prelude from the build. The\n prelude can be viewed at zcbor/prelude.cddl in the\n repo, or together with the script.\n -v, --verbose Print more information while parsing CDDL and\n generating code.\n -i INPUT, --input INPUT\n Input data file. The option --input-as specifies how\n to interpret the contents. Use \"-\" to indicate stdin.\n --input-as {yaml,json,cbor,cborhex}\n Which format to interpret the input file as. If\n omitted, the format is inferred from the file name.\n .yaml, .yml => YAML, .json => JSON, .cborhex => CBOR\n as hex string, everything else => CBOR\n -t ENTRY_TYPE, --entry-type ENTRY_TYPE\n Name of the type (from the CDDL) to interpret the data\n as.\n --default-max-qty DEFAULT_MAX_QTY, --dq DEFAULT_MAX_QTY\n Default maximum number of repetitions when no maximum\n is specified. It is only relevant when handling data\n that will be decoded by generated code. If omitted, a\n large number will be used.\n --yaml-compatibility Whether to convert CBOR-only values to YAML-compatible\n ones (when converting from CBOR), or vice versa (when\n converting to CBOR). When this is enabled, all CBOR\n data is guaranteed to convert into YAML/JSON. JSON and\n YAML do not support all data types that CBOR/CDDL\n supports. bytestrings (BSTR), tags, undefined, and\n maps with non-text keys need special handling. See the\n zcbor README for more information.\n -o OUTPUT, --output OUTPUT\n Output data file. The option --output-as specifies how\n to interpret the contents. Use \"-\" to indicate stdout.\n --output-as {yaml,json,cbor,cborhex,c_code}\n Which format to interpret the output file as. If\n omitted, the format is inferred from the file name.\n .yaml, .yml => YAML, .json => JSON, .c, .h => C code,\n .cborhex => CBOR as hex string, everything else =>\n CBOR\n --c-code-var-name C_CODE_VAR_NAME\n Only relevant together with '--output-as c_code' or .c\n files.\n --c-code-columns C_CODE_COLUMNS\n Only relevant together with '--output-as c_code' or .c\n files. The number of bytes per line in the variable\n instantiation. If omitted, the entire declaration is a\n single line.\n\n```\n",
"bugtrack_url": null,
"license": "Apache",
"summary": "Code generation and validation using CDDL schemas",
"version": "0.8.1",
"project_urls": {
"Homepage": "https://github.com/NordicSemiconductor/zcbor"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6d0f9beb56b89cc5a5f0b4c8ed54b9fdd060319d8a91bd573b4c395d92d50751",
"md5": "043fe6641827cab8fc9abdcefb1d0695",
"sha256": "3aca62602be395ed1a1f0772ec239ad28a3ab09fa9c709a725cc89865ad243af"
},
"downloads": -1,
"filename": "zcbor-0.8.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "043fe6641827cab8fc9abdcefb1d0695",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 81131,
"upload_time": "2024-01-26T17:25:33",
"upload_time_iso_8601": "2024-01-26T17:25:33.240665Z",
"url": "https://files.pythonhosted.org/packages/6d/0f/9beb56b89cc5a5f0b4c8ed54b9fdd060319d8a91bd573b4c395d92d50751/zcbor-0.8.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9afa0a082e28646618669fc09626d8ad88a82f3b7de999e02496cb4dad9fc4eb",
"md5": "9c5457f3b079fe8cc7f565338aade253",
"sha256": "53763faf7b418732ba6c69ccc5da8acd2edb2c7c80ce0a466793d52bda70ecf9"
},
"downloads": -1,
"filename": "zcbor-0.8.1.tar.gz",
"has_sig": false,
"md5_digest": "9c5457f3b079fe8cc7f565338aade253",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 95475,
"upload_time": "2024-01-26T17:25:35",
"upload_time_iso_8601": "2024-01-26T17:25:35.194124Z",
"url": "https://files.pythonhosted.org/packages/9a/fa/0a082e28646618669fc09626d8ad88a82f3b7de999e02496cb4dad9fc4eb/zcbor-0.8.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-26 17:25:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "NordicSemiconductor",
"github_project": "zcbor",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "zcbor"
}
