| Name | naipng JSON |
| Version |
1.0.0.post2
JSON |
| download |
| home_page | |
| Summary | Library and command-line tool to read NovelAI data encoded in PNG files |
| upload_time | 2023-08-03 07:55:08 |
| maintainer | |
| docs_url | None |
| author | Eta |
| requires_python | >=3.7 |
| license | zlib/libpng License |
| keywords |
novelai
nai
png
cli
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# naipng ✒️🖼️
`naipng` is a Python library and command-line tool to read [NovelAI](https://novelai.net)
data encoded in PNG files (like in Lorebook cards and generated images), with no dependencies.
Also check out my website [pngmeta](https://pngmeta.glitch.me) for a browser-based tool for
[viewing](https://pngmeta.glitch.me) and [copying](https://pngmeta.glitch.me) PNG `tEXt` metadata.
# Table of Contents
- [What Is This?](#what-is-this)
- [Installation](#installation)
- [Usage](#usage)
- [Library](#library)
- [Example](#naipngread-example)
- [Error Handling](#error-handling)
- [CLI](#cli)
- [Help Text](#help-text)
- [Examples](#examples)
- [Technical Background](#technical-background)
- [License](#license)
# What Is This?
User-made content for the web service [NovelAI](https://novelai.net) is often shared off-platform in the form of files.
These files take on multiple formats. Though most are simple JSON, some content can be shared embedded within PNGs,
which is more complicated to extract.
**Text Generation**: NovelAI allows exporting certain settings and objects related to text generation AI as PNG images
in place of regular JSON files, as a way of associating art with the descriptions of characters and places being shared.
These are commonly known as Lorebook cards.
**Image Generation**: NovelAI encodes image generation settings within generated PNGs, including parameters
such as `prompt`, `steps`, and so on, to make it easier to replicate and modify generated images.
Both domains use PNG metadata to encode this information, thus this tool allows the extraction of that metadata.
See more technical details in the [Technical Background](#technical-background) section.
# Installation
`naipng` is available on PyPI, so it can be installed through `pip`:
```bash
python -m pip install naipng
```
Since `naipng` has no dependencies, you can also import or run it by simply adding the `naipng` directory
to your source tree.
# Usage
`naipng` may be used either as a [library](#library) or a [command-line tool](#cli).
## Library
The primary function entrypoint provided by `naipng` is `naipng.read(file: bytes | BinaryIO)`.
This can be used to parse a PNG image for NovelAI metadata from either an open file or a `bytes` object in memory.
Two more specific variations of `naipng.read()` also exist:
- `naipng.read_text_gen(file: bytes | BinaryIO)` only returns text generation data
- *E.g. Lorebooks*
- `naipng.read_image_gen(file: bytes | BinaryIO)` only returns image generation data
- *E.g. image prompts*
### `naipng.read()` Example
```python
import naipng
# Using a file-like object
with open("image.png", "rb") as file:
decoded = naipng.read(file)
if decoded is None:
# No NovelAI data found encoded in the image
print("N/A")
else:
# naipng.read() returns a dict object representing the first data found
for k, v in decoded.items():
print(k, "->", v)
```
Another example, using `bytes` as input to `naipng.read`:
```python
import naipng
import pathlib
data: bytes = pathlib.Path("image.png").read_bytes()
decoded = naipng.read(data)
# ...
```
### Error Handling
`naipng` defines two error types for `naipng.read()` and its variants:
- `naipng.InvalidPNGError` is raised when a PNG is invalid and cannot be parsed, such as when:
- The PNG datastream ends prematurely (before `IEND`)
- A `tEXt` chunk is corrupted *(i.e. has an invalid CRC)*
- The PNG signature is missing
- The PNG `IHDR` chunk is missing or misplaced
- `naipng.NAIDataError` is raised when a PNG has a `tEXt` chunk designated as `naidata`,
but it was unable to be decoded properly
- This is never raised for `naipng.read_image_gen()`
Both error classes derive from `ValueError`.
This example shows the behaviour of `naipng.read` with an invalid PNG datastream
(correct signature, but ends early).
```python
import naipng
# Using a bytes object as input (a file-like object could be used too)
with open("image.png", "rb") as file:
# The datastream is truncated, rendering it invalid
data = file.read(10)
try:
naipng.read(data)
except naipng.InvalidPNGError as e:
raise SystemExit(f"Error: {e}") from e
```
This outputs:
```
Error: not a valid PNG file: ends prematurely
```
## CLI
`naipng` may be invoked on the command line as either `python -m naipng <file>` or simply `naipng <file>`.
### Help Text
```
usage: naipng [-h] [-q] [-c] [-t] [-i] file [outfile]
read NovelAI data encoded in a PNG file
positional arguments:
file PNG file to read, or - for stdin
outfile output file path, or - for stdout (default: -)
options:
-h, --help show this help message and exit
-q, --quiet don't print errors
-c, --compact don't pretty-print decoded JSON
-t, --text only check for text generation data
-i, --image only check for image generation data
```
### Examples
- Printing to stdout:
```bash
naipng image.png
```
- Saving to a file:
```bash
naipng image.png naidata.json
```
- Saving to a file by redirecting `stdin` and `stdout`:
```bash
naipng - < image.png > naidata.json
```
- Downloading via `curl` and piping PNG data through `stdin`:
```bash
curl -fs https://files.catbox.moe/3b6dux.png | naipng -
```
----------
`naipng` may be used in shell pipelines alongside JSON-parsing tools like [`jq`](https://jqlang.github.io/jq/)
in order to construct more complex scripts.
```bash
$ curl -fs https://files.catbox.moe/3b6dux.png | naipng -c - | jq -r ".[\"entries\"][][\"text\"]"
Everyone said that no man now living or ever after would be born who would be equal to him in strength, courage, and in all sorts of courtesy, as well as in boldness and generosity that he had above all men, and that his name would never perish in the German tongue, and the same was true with the Norsemen
```
#### Trivia
The linked file is the first Lorebook card shared as an example when PNG embedding was announced.
Its art is by Tarmillustrates.
The quote in it is from *[Þiðreks saga](https://en.wikipedia.org/wiki/%C3%9Ei%C3%B0reks_saga)*.
# Technical Background
PNGs are made up of a sequence of data chunks.
Beyond those used to store visual information (e.g. pixels, palettes),
there are also several varieties of metadata chunks.
See [the official PNG specification](http://www.libpng.org/pub/png/spec/1.2/PNG-Structure.html) for full details.
NovelAI uses [`tEXt` metadata chunks](http://www.libpng.org/pub/png/spec/1.2/PNG-Chunks.html#C.Anc-text)
to encode most metadata.
- For text generation settings, NovelAI uses a `tEXt` chunk with a `naidata` keyword.
- The contents are base64-encoded JSON.
- `naidata` is a nonstandard `tEXt` keyword, so recognizing these chunks is unambiguous.
- For image generation settings, NovelAI uses multiple `tEXt` chunks, each with different keywords.
- Some of these include `Title`, `Description`, `Software`, `Source`, and `Comment`.
- `naipng` only reads the `Comment` field among these, which is JSON-encoded and contains the most information.
- `Comment` is a standard `tEXt` keyword, so recognizing these chunks is slightly ambiguous.
- Other software may use the `Comment` `tEXt` chunk type for other purposes, and may or may not store JSON in it.
- `naipng` assumes that the first JSON-encoded `Comment` `tEXt` chunk found is valid image generation metadata.
# License
`naipng` is free and open-source software provided under the [zlib license](https://opensource.org/licenses/Zlib).
Raw data
{
"_id": null,
"home_page": "",
"name": "naipng",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "novelai,nai,png,cli",
"author": "Eta",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/31/e9/43616106f44411d419935ae137cad4e4ac756f1c53f669f76b51c11fdbc7/naipng-1.0.0.post2.tar.gz",
"platform": null,
"description": "# naipng \u2712\ufe0f\ud83d\uddbc\ufe0f\n\n`naipng` is a Python library and command-line tool to read [NovelAI](https://novelai.net)\ndata encoded in PNG files (like in Lorebook cards and generated images), with no dependencies.\n\nAlso check out my website [pngmeta](https://pngmeta.glitch.me) for a browser-based tool for\n[viewing](https://pngmeta.glitch.me) and [copying](https://pngmeta.glitch.me) PNG `tEXt` metadata.\n\n# Table of Contents\n\n- [What Is This?](#what-is-this)\n- [Installation](#installation)\n- [Usage](#usage)\n - [Library](#library)\n - [Example](#naipngread-example)\n - [Error Handling](#error-handling)\n - [CLI](#cli)\n - [Help Text](#help-text)\n - [Examples](#examples)\n- [Technical Background](#technical-background)\n- [License](#license)\n\n# What Is This?\n\nUser-made content for the web service [NovelAI](https://novelai.net) is often shared off-platform in the form of files.\nThese files take on multiple formats. Though most are simple JSON, some content can be shared embedded within PNGs,\nwhich is more complicated to extract.\n\n**Text Generation**: NovelAI allows exporting certain settings and objects related to text generation AI as PNG images\nin place of regular JSON files, as a way of associating art with the descriptions of characters and places being shared.\nThese are commonly known as Lorebook cards.\n\n**Image Generation**: NovelAI encodes image generation settings within generated PNGs, including parameters\nsuch as `prompt`, `steps`, and so on, to make it easier to replicate and modify generated images.\n\nBoth domains use PNG metadata to encode this information, thus this tool allows the extraction of that metadata.\n\nSee more technical details in the [Technical Background](#technical-background) section.\n\n# Installation\n\n`naipng` is available on PyPI, so it can be installed through `pip`:\n\n```bash\npython -m pip install naipng\n```\n\nSince `naipng` has no dependencies, you can also import or run it by simply adding the `naipng` directory\nto your source tree.\n\n# Usage\n\n`naipng` may be used either as a [library](#library) or a [command-line tool](#cli).\n\n## Library\n\nThe primary function entrypoint provided by `naipng` is `naipng.read(file: bytes | BinaryIO)`.\nThis can be used to parse a PNG image for NovelAI metadata from either an open file or a `bytes` object in memory.\n\nTwo more specific variations of `naipng.read()` also exist:\n\n- `naipng.read_text_gen(file: bytes | BinaryIO)` only returns text generation data\n - *E.g. Lorebooks*\n- `naipng.read_image_gen(file: bytes | BinaryIO)` only returns image generation data\n - *E.g. image prompts*\n\n### `naipng.read()` Example\n\n```python\nimport naipng\n\n# Using a file-like object\nwith open(\"image.png\", \"rb\") as file:\n decoded = naipng.read(file)\n\nif decoded is None:\n # No NovelAI data found encoded in the image\n print(\"N/A\")\nelse:\n # naipng.read() returns a dict object representing the first data found\n for k, v in decoded.items():\n print(k, \"->\", v)\n```\n\nAnother example, using `bytes` as input to `naipng.read`:\n\n```python\nimport naipng\nimport pathlib\n\ndata: bytes = pathlib.Path(\"image.png\").read_bytes()\ndecoded = naipng.read(data)\n\n# ...\n```\n\n### Error Handling\n\n`naipng` defines two error types for `naipng.read()` and its variants:\n\n- `naipng.InvalidPNGError` is raised when a PNG is invalid and cannot be parsed, such as when:\n - The PNG datastream ends prematurely (before `IEND`)\n - A `tEXt` chunk is corrupted *(i.e. has an invalid CRC)*\n - The PNG signature is missing\n - The PNG `IHDR` chunk is missing or misplaced\n- `naipng.NAIDataError` is raised when a PNG has a `tEXt` chunk designated as `naidata`,\n but it was unable to be decoded properly\n - This is never raised for `naipng.read_image_gen()`\n\nBoth error classes derive from `ValueError`.\n\nThis example shows the behaviour of `naipng.read` with an invalid PNG datastream\n(correct signature, but ends early).\n\n```python\nimport naipng\n\n# Using a bytes object as input (a file-like object could be used too)\nwith open(\"image.png\", \"rb\") as file:\n # The datastream is truncated, rendering it invalid\n data = file.read(10)\n\ntry:\n naipng.read(data)\nexcept naipng.InvalidPNGError as e:\n raise SystemExit(f\"Error: {e}\") from e\n```\n\nThis outputs:\n\n```\nError: not a valid PNG file: ends prematurely\n```\n\n## CLI\n\n`naipng` may be invoked on the command line as either `python -m naipng <file>` or simply `naipng <file>`.\n\n### Help Text\n\n```\nusage: naipng [-h] [-q] [-c] [-t] [-i] file [outfile]\n\nread NovelAI data encoded in a PNG file\n\npositional arguments:\n file PNG file to read, or - for stdin\n outfile output file path, or - for stdout (default: -)\n\noptions:\n -h, --help show this help message and exit\n -q, --quiet don't print errors\n -c, --compact don't pretty-print decoded JSON\n -t, --text only check for text generation data\n -i, --image only check for image generation data\n```\n\n### Examples\n\n- Printing to stdout:\n\n```bash\nnaipng image.png\n```\n\n- Saving to a file:\n\n```bash\nnaipng image.png naidata.json\n```\n\n- Saving to a file by redirecting `stdin` and `stdout`:\n\n```bash\nnaipng - < image.png > naidata.json\n```\n\n- Downloading via `curl` and piping PNG data through `stdin`:\n\n```bash\ncurl -fs https://files.catbox.moe/3b6dux.png | naipng -\n```\n\n----------\n\n`naipng` may be used in shell pipelines alongside JSON-parsing tools like [`jq`](https://jqlang.github.io/jq/)\nin order to construct more complex scripts.\n\n```bash\n$ curl -fs https://files.catbox.moe/3b6dux.png | naipng -c - | jq -r \".[\\\"entries\\\"][][\\\"text\\\"]\"\n\nEveryone said that no man now living or ever after would be born who would be equal to him in strength, courage, and in all sorts of courtesy, as well as in boldness and generosity that he had above all men, and that his name would never perish in the German tongue, and the same was true with the Norsemen\n```\n\n#### Trivia\n\nThe linked file is the first Lorebook card shared as an example when PNG embedding was announced.\nIts art is by Tarmillustrates.\nThe quote in it is from *[\u00dei\u00f0reks saga](https://en.wikipedia.org/wiki/%C3%9Ei%C3%B0reks_saga)*.\n\n# Technical Background\n\nPNGs are made up of a sequence of data chunks.\nBeyond those used to store visual information (e.g. pixels, palettes),\nthere are also several varieties of metadata chunks.\nSee [the official PNG specification](http://www.libpng.org/pub/png/spec/1.2/PNG-Structure.html) for full details.\n\nNovelAI uses [`tEXt` metadata chunks](http://www.libpng.org/pub/png/spec/1.2/PNG-Chunks.html#C.Anc-text)\nto encode most metadata.\n- For text generation settings, NovelAI uses a `tEXt` chunk with a `naidata` keyword.\n - The contents are base64-encoded JSON.\n - `naidata` is a nonstandard `tEXt` keyword, so recognizing these chunks is unambiguous.\n- For image generation settings, NovelAI uses multiple `tEXt` chunks, each with different keywords.\n - Some of these include `Title`, `Description`, `Software`, `Source`, and `Comment`.\n - `naipng` only reads the `Comment` field among these, which is JSON-encoded and contains the most information.\n - `Comment` is a standard `tEXt` keyword, so recognizing these chunks is slightly ambiguous.\n - Other software may use the `Comment` `tEXt` chunk type for other purposes, and may or may not store JSON in it.\n - `naipng` assumes that the first JSON-encoded `Comment` `tEXt` chunk found is valid image generation metadata.\n\n# License\n\n`naipng` is free and open-source software provided under the [zlib license](https://opensource.org/licenses/Zlib).\n",
"bugtrack_url": null,
"license": "zlib/libpng License",
"summary": "Library and command-line tool to read NovelAI data encoded in PNG files",
"version": "1.0.0.post2",
"project_urls": {
"Homepage": "https://github.com/Eta0/naipng"
},
"split_keywords": [
"novelai",
"nai",
"png",
"cli"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "dab09faba877024557add8271a91cd04c4f1711ae5bd14bc03043028a6f0e5d8",
"md5": "49c062048220147af221a77a1c67a8a2",
"sha256": "3998076cef27ae463ee03d8a12b8ccccc330293b07dab61b6008a60b4e338074"
},
"downloads": -1,
"filename": "naipng-1.0.0.post2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "49c062048220147af221a77a1c67a8a2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 11017,
"upload_time": "2023-08-03T07:55:06",
"upload_time_iso_8601": "2023-08-03T07:55:06.856896Z",
"url": "https://files.pythonhosted.org/packages/da/b0/9faba877024557add8271a91cd04c4f1711ae5bd14bc03043028a6f0e5d8/naipng-1.0.0.post2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "31e943616106f44411d419935ae137cad4e4ac756f1c53f669f76b51c11fdbc7",
"md5": "340ee58f42f1b2d9d8f48c61c88e0ae7",
"sha256": "5bb1a39e1035011ddb62fae12aaafdaf8d6b13b0fff9a851524641474a496b10"
},
"downloads": -1,
"filename": "naipng-1.0.0.post2.tar.gz",
"has_sig": false,
"md5_digest": "340ee58f42f1b2d9d8f48c61c88e0ae7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 12118,
"upload_time": "2023-08-03T07:55:08",
"upload_time_iso_8601": "2023-08-03T07:55:08.548533Z",
"url": "https://files.pythonhosted.org/packages/31/e9/43616106f44411d419935ae137cad4e4ac756f1c53f669f76b51c11fdbc7/naipng-1.0.0.post2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-03 07:55:08",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Eta0",
"github_project": "naipng",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "naipng"
}
