# Remarshal
Convert between CBOR, JSON, MessagePack, TOML, and YAML. When installed,
provides the command line command `remarshal` as well as the short commands
`{cbor,json,msgpack,toml,yaml}2`​`{cbor,json,msgpack,toml,yaml}`. With
these commands, you can perform format conversion, reformatting, and error
detection.
## Known limitations
* CBOR, MessagePack, and YAML with binary fields can not be converted to JSON
or TOML. Binary fields are converted between CBOR, MessagePack, and YAML.
* TOML containing values of the
[Local Date-Time](https://toml.io/en/v1.0.0-rc.1#local-date-time) type can not
be converted to CBOR. The Local Date type can only be converted to JSON and
YAML. The Local Time type can not be converted to any other format. Offset
Date-Time and its equivalents can be converted between CBOR, MessagePack,
TOML, and YAML. Keys of any date-time type are converted to string TOML
keys.
* Date and time types are converted to JSON strings. They can not be safely
roundtripped through JSON.
* A YAML timestamp with only a date becomes a YAML timestamp or a TOML Local
Date-Time for the midnight of that date. This means you can not roundtrip
every YAML document through Remarshal.
## Installation
You will need Python 3.8 or later. Earlier versions of Python 3 are not
supported.
The recommended way to run Remarshal is to install the latest release from
[PyPI](https://pypi.org/project/remarshal/) with
[pipx](https://github.com/pypa/pipx).
```sh
pipx install remarshal
```
Regular installation is not mandatory. The command
```sh
pipx run remarshal [arg ...]
```
will download Remarshal and run it from a temporary location.
It will cache the downloaded version for up to 14 days. Remarshal will not be
automatically updated during this time.
You can install Remarshal using pip.
```sh
python3 -m pip install --user remarshal
```
Instead of a release, you can install the development version. Prefer
releases unless you have a reason to run the development version.
```sh
python3 -m pip install --user git+https://github.com/remarshal-project/remarshal
```
## Usage
```
usage: remarshal.py [-h] [-v] [-i input] [--if {cbor,json,msgpack,toml,yaml}]
[--json-indent n] [-k] [--max-values n] [-o output]
[--of {cbor,json,msgpack,toml,yaml}] [-s] [--unwrap key]
[--wrap key] [--yaml-indent n] [--yaml-style {,',",|,>}]
[--yaml-width n]
[input] [output]
Convert between CBOR, JSON, MessagePack, TOML, and YAML.
positional arguments:
input input file
output output file
options:
-h, --help show this help message and exit
-v, --version show program's version number and exit
-i input, --input input
input file
--if {cbor,json,msgpack,toml,yaml}, -if {cbor,json,msgpack,toml,yaml}, --input-format {cbor,json,msgpack,toml,yaml}
input format
--json-indent n, --indent-json n
JSON indentation
-k, --stringify Turn into strings boolean, date-time, and null keys
for JSON and TOML and null values for TOML
--max-values n maximum number of values in input data (default
1000000, negative for unlimited)
-o output, --output output
output file
--of {cbor,json,msgpack,toml,yaml}, -of {cbor,json,msgpack,toml,yaml}, --output-format {cbor,json,msgpack,toml,yaml}
output format
-s, --sort-keys sort JSON, TOML, YAML keys instead of preserving key
order
--unwrap key only output the data stored under the given key
--wrap key wrap the data in a map type with the given key
--yaml-indent n YAML indentation
--yaml-style {,',",|,>}
YAML formatting style
--yaml-width n YAML line width for long strings
```
You can use a short command
`{cbor,json,msgpack,toml,yaml}2`​`{cbor,json,msgpack,toml,yaml}`
instead of `remarshal` with format arguments. The `remarshal` command as well
as the short commands exit with status 0 on success, 1 on operational failure,
and 2 when they fail to parse the command line.
If no input argument `input`/`-i input` is given or its value is `-`,
Remarshal reads input data from standard input. Similarly, with no
`output`/`-o output` or an output argument that is `-`, it writes the result
to standard output.
### Wrappers
The arguments `--wrap` and `--unwrap` are available to solve the problem of
converting CBOR, JSON, MessagePack, and YAML data to TOML if the top-level
element of the data is not of a dictionary type (i.e., not a map in CBOR and
MessagePack, an object in JSON, or an associative array in YAML).
You can not represent such data as TOML directly; the data must be wrapped in a
dictionary first. Passing the flag `--wrap someKey` to `remarshal` or one of
its short commands wraps the input data in a "wrapper" dictionary with one key,
"someKey", with the input data as its value. The flag `--unwrap someKey` does
the opposite: only the value stored under the key "someKey" in the top-level
dictionary element of the input data is converted to the target format and
output; the rest of the input is ignored. If the top-level element is not a
dictionary or does not have the key "someKey", `--unwrap someKey` causes an
error.
The following shell transcript demonstrates the problem and how `--wrap` and
`--unwrap` solve it:
```
$ echo '[{"a":"b"},{"c":[1,2,3]}]' | ./remarshal.py --if json --of toml
Error: cannot convert non-dictionary data to TOML; use "wrap" to wrap it in a dictionary
$ echo '[{"a":"b"},{"c":[1,2,3]}]' \
| ./remarshal.py --if json --of toml --wrap main
[[main]]
a = "b"
[[main]]
c = [1, 2, 3]
$ echo '[{"a":"b"},{"c":[1,2,3]}]' \
| ./remarshal.py --if json --wrap main - test.toml
$ ./remarshal.py test.toml --of json
{"main":[{"a":"b"},{"c":[1,2,3]}]}
$ ./remarshal.py test.toml --of json --unwrap main
[{"a":"b"},{"c":[1,2,3]}]
```
## Examples
```
$ ./remarshal.py example.toml --of yaml
clients:
data:
- - gamma
- delta
- - 1
- 2
hosts:
- alpha
- omega
database:
connection_max: 5000
enabled: true
ports:
- 8001
- 8001
- 8002
server: 192.168.1.1
owner:
bio: 'GitHub Cofounder & CEO
Likes tater tots and beer.'
dob: 1979-05-27 07:32:00+00:00
name: Tom Preston-Werner
organization: GitHub
products:
- name: Hammer
sku: 738594937
- color: gray
name: Nail
sku: 284758393
servers:
alpha:
dc: eqdc10
ip: 10.0.0.1
beta:
country: 中国
dc: eqdc10
ip: 10.0.0.2
title: TOML Example
$ curl -s http://api.openweathermap.org/data/2.5/weather\?q\=Kiev,ua \
| ./remarshal.py --if json --of toml
base = "cmc stations"
cod = 200
dt = 1412532000
id = 703448
name = "Kiev"
[clouds]
all = 44
[coord]
lat = 50.42999999999999972
lon = 30.51999999999999957
[main]
humidity = 66
pressure = 1026
temp = 283.49000000000000909
temp_max = 284.14999999999997726
temp_min = 283.14999999999997726
[sys]
country = "UA"
id = 7358
message = 0.24370000000000000
sunrise = 1412481902
sunset = 1412522846
type = 1
[[weather]]
description = "scattered clouds"
icon = "03n"
id = 802
main = "Clouds"
[wind]
deg = 80
speed = 2
```
## License
MIT. See the file `LICENSE`.
`example.toml` from <https://github.com/toml-lang/toml>. `example.json`,
`example.msgpack`, `example.cbor`, `example.yml`, `tests/bin.msgpack`, and
`tests/bin.yml` are derived from it.
Raw data
{
"_id": null,
"home_page": "https://github.com/remarshal-project/remarshal",
"name": "Remarshal",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "converter,cbor,json,messagepack,msgpack,toml,yaml",
"author": "D. Bohdan",
"author_email": "dbohdan@dbohdan.com",
"download_url": "https://files.pythonhosted.org/packages/55/39/d638b7d8012468fe13c072bfb283cd917b12dbcb8e7a10b414d5109b0852/remarshal-0.17.1.tar.gz",
"platform": null,
"description": "# Remarshal\n\nConvert between CBOR, JSON, MessagePack, TOML, and YAML. When installed,\nprovides the command line command `remarshal` as well as the short commands\n`{cbor,json,msgpack,toml,yaml}2`​`{cbor,json,msgpack,toml,yaml}`. With\nthese commands, you can perform format conversion, reformatting, and error\ndetection.\n\n## Known limitations\n\n* CBOR, MessagePack, and YAML with binary fields can not be converted to JSON\nor TOML. Binary fields are converted between CBOR, MessagePack, and YAML.\n* TOML containing values of the\n[Local Date-Time](https://toml.io/en/v1.0.0-rc.1#local-date-time) type can not\nbe converted to CBOR. The Local Date type can only be converted to JSON and\nYAML. The Local Time type can not be converted to any other format. Offset\nDate-Time and its equivalents can be converted between CBOR, MessagePack,\nTOML, and YAML. Keys of any date-time type are converted to string TOML\nkeys.\n* Date and time types are converted to JSON strings. They can not be safely\nroundtripped through JSON.\n* A YAML timestamp with only a date becomes a YAML timestamp or a TOML Local\nDate-Time for the midnight of that date. This means you can not roundtrip\nevery YAML document through Remarshal.\n\n## Installation\n\nYou will need Python 3.8 or later. Earlier versions of Python 3 are not\nsupported.\n\nThe recommended way to run Remarshal is to install the latest release from\n[PyPI](https://pypi.org/project/remarshal/) with\n[pipx](https://github.com/pypa/pipx).\n\n```sh\npipx install remarshal\n```\n\nRegular installation is not mandatory. The command\n\n```sh\npipx run remarshal [arg ...]\n```\n\nwill download Remarshal and run it from a temporary location.\nIt will cache the downloaded version for up to 14 days. Remarshal will not be\nautomatically updated during this time.\n\nYou can install Remarshal using pip.\n\n```sh\npython3 -m pip install --user remarshal\n```\n\nInstead of a release, you can install the development version. Prefer\nreleases unless you have a reason to run the development version.\n\n```sh\npython3 -m pip install --user git+https://github.com/remarshal-project/remarshal\n```\n\n## Usage\n\n```\nusage: remarshal.py [-h] [-v] [-i input] [--if {cbor,json,msgpack,toml,yaml}]\n [--json-indent n] [-k] [--max-values n] [-o output]\n [--of {cbor,json,msgpack,toml,yaml}] [-s] [--unwrap key]\n [--wrap key] [--yaml-indent n] [--yaml-style {,',\",|,>}]\n [--yaml-width n]\n [input] [output]\n\nConvert between CBOR, JSON, MessagePack, TOML, and YAML.\n\npositional arguments:\n input input file\n output output file\n\noptions:\n -h, --help show this help message and exit\n -v, --version show program's version number and exit\n -i input, --input input\n input file\n --if {cbor,json,msgpack,toml,yaml}, -if {cbor,json,msgpack,toml,yaml}, --input-format {cbor,json,msgpack,toml,yaml}\n input format\n --json-indent n, --indent-json n\n JSON indentation\n -k, --stringify Turn into strings boolean, date-time, and null keys\n for JSON and TOML and null values for TOML\n --max-values n maximum number of values in input data (default\n 1000000, negative for unlimited)\n -o output, --output output\n output file\n --of {cbor,json,msgpack,toml,yaml}, -of {cbor,json,msgpack,toml,yaml}, --output-format {cbor,json,msgpack,toml,yaml}\n output format\n -s, --sort-keys sort JSON, TOML, YAML keys instead of preserving key\n order\n --unwrap key only output the data stored under the given key\n --wrap key wrap the data in a map type with the given key\n --yaml-indent n YAML indentation\n --yaml-style {,',\",|,>}\n YAML formatting style\n --yaml-width n YAML line width for long strings\n```\n\nYou can use a short command\n`{cbor,json,msgpack,toml,yaml}2`​`{cbor,json,msgpack,toml,yaml}`\ninstead of `remarshal` with format arguments. The `remarshal` command as well\nas the short commands exit with status 0 on success, 1 on operational failure,\nand 2 when they fail to parse the command line.\n\nIf no input argument `input`/`-i input` is given or its value is `-`,\nRemarshal reads input data from standard input. Similarly, with no\n`output`/`-o output` or an output argument that is `-`, it writes the result\nto standard output.\n\n### Wrappers\n\nThe arguments `--wrap` and `--unwrap` are available to solve the problem of\nconverting CBOR, JSON, MessagePack, and YAML data to TOML if the top-level\nelement of the data is not of a dictionary type (i.e., not a map in CBOR and\nMessagePack, an object in JSON, or an associative array in YAML).\nYou can not represent such data as TOML directly; the data must be wrapped in a\ndictionary first. Passing the flag `--wrap someKey` to `remarshal` or one of\nits short commands wraps the input data in a \"wrapper\" dictionary with one key,\n\"someKey\", with the input data as its value. The flag `--unwrap someKey` does\nthe opposite: only the value stored under the key \"someKey\" in the top-level\ndictionary element of the input data is converted to the target format and\noutput; the rest of the input is ignored. If the top-level element is not a\ndictionary or does not have the key \"someKey\", `--unwrap someKey` causes an\nerror.\n\nThe following shell transcript demonstrates the problem and how `--wrap` and\n`--unwrap` solve it:\n\n```\n$ echo '[{\"a\":\"b\"},{\"c\":[1,2,3]}]' | ./remarshal.py --if json --of toml\nError: cannot convert non-dictionary data to TOML; use \"wrap\" to wrap it in a dictionary\n\n$ echo '[{\"a\":\"b\"},{\"c\":[1,2,3]}]' \\\n | ./remarshal.py --if json --of toml --wrap main\n[[main]]\na = \"b\"\n\n[[main]]\nc = [1, 2, 3]\n\n$ echo '[{\"a\":\"b\"},{\"c\":[1,2,3]}]' \\\n | ./remarshal.py --if json --wrap main - test.toml\n\n$ ./remarshal.py test.toml --of json\n{\"main\":[{\"a\":\"b\"},{\"c\":[1,2,3]}]}\n\n$ ./remarshal.py test.toml --of json --unwrap main\n[{\"a\":\"b\"},{\"c\":[1,2,3]}]\n```\n\n## Examples\n\n```\n$ ./remarshal.py example.toml --of yaml\nclients:\n data:\n - - gamma\n - delta\n - - 1\n - 2\n hosts:\n - alpha\n - omega\ndatabase:\n connection_max: 5000\n enabled: true\n ports:\n - 8001\n - 8001\n - 8002\n server: 192.168.1.1\nowner:\n bio: 'GitHub Cofounder & CEO\n\n Likes tater tots and beer.'\n dob: 1979-05-27 07:32:00+00:00\n name: Tom Preston-Werner\n organization: GitHub\nproducts:\n- name: Hammer\n sku: 738594937\n- color: gray\n name: Nail\n sku: 284758393\nservers:\n alpha:\n dc: eqdc10\n ip: 10.0.0.1\n beta:\n country: \u4e2d\u56fd\n dc: eqdc10\n ip: 10.0.0.2\ntitle: TOML Example\n\n$ curl -s http://api.openweathermap.org/data/2.5/weather\\?q\\=Kiev,ua \\\n | ./remarshal.py --if json --of toml\nbase = \"cmc stations\"\ncod = 200\ndt = 1412532000\nid = 703448\nname = \"Kiev\"\n\n[clouds]\nall = 44\n\n[coord]\nlat = 50.42999999999999972\nlon = 30.51999999999999957\n\n[main]\nhumidity = 66\npressure = 1026\ntemp = 283.49000000000000909\ntemp_max = 284.14999999999997726\ntemp_min = 283.14999999999997726\n\n[sys]\ncountry = \"UA\"\nid = 7358\nmessage = 0.24370000000000000\nsunrise = 1412481902\nsunset = 1412522846\ntype = 1\n\n[[weather]]\ndescription = \"scattered clouds\"\nicon = \"03n\"\nid = 802\nmain = \"Clouds\"\n\n[wind]\ndeg = 80\nspeed = 2\n```\n\n## License\n\nMIT. See the file `LICENSE`.\n\n`example.toml` from <https://github.com/toml-lang/toml>. `example.json`,\n`example.msgpack`, `example.cbor`, `example.yml`, `tests/bin.msgpack`, and\n`tests/bin.yml` are derived from it.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Convert between CBOR, JSON, MessagePack, TOML, and YAML",
"version": "0.17.1",
"project_urls": {
"Homepage": "https://github.com/remarshal-project/remarshal",
"Repository": "https://github.com/remarshal-project/remarshal"
},
"split_keywords": [
"converter",
"cbor",
"json",
"messagepack",
"msgpack",
"toml",
"yaml"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "02a4697247610dad9a5fa57582857e4e7eb640b2c1fe0bcecee3385272b29f14",
"md5": "0223c06d550ebdac3c8095e9231d4fbd",
"sha256": "6f863da29c2a8353b157636d77df4d8877e71889aa0608f706c8a9647c5cd7ba"
},
"downloads": -1,
"filename": "remarshal-0.17.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0223c06d550ebdac3c8095e9231d4fbd",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 10367,
"upload_time": "2023-09-05T14:23:30",
"upload_time_iso_8601": "2023-09-05T14:23:30.295712Z",
"url": "https://files.pythonhosted.org/packages/02/a4/697247610dad9a5fa57582857e4e7eb640b2c1fe0bcecee3385272b29f14/remarshal-0.17.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5539d638b7d8012468fe13c072bfb283cd917b12dbcb8e7a10b414d5109b0852",
"md5": "e5d161947acf2b2e691400ae813cb2b0",
"sha256": "826a41d3e3ed9d45422811488d7b28cc146a8d5b2583b18db36302f87091a86d"
},
"downloads": -1,
"filename": "remarshal-0.17.1.tar.gz",
"has_sig": false,
"md5_digest": "e5d161947acf2b2e691400ae813cb2b0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 21161,
"upload_time": "2023-09-05T14:23:31",
"upload_time_iso_8601": "2023-09-05T14:23:31.964795Z",
"url": "https://files.pythonhosted.org/packages/55/39/d638b7d8012468fe13c072bfb283cd917b12dbcb8e7a10b414d5109b0852/remarshal-0.17.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-05 14:23:31",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "remarshal-project",
"github_project": "remarshal",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "remarshal"
}
JSON
