# rmscene
Python library to read v6 files from reMarkable tables (software version 3).
In particular, this version introduces the ability to include text as well as drawn lines. Extracting this text is the original motivation to develop this library, but it also can read much of the other types of data in the reMarkable files.
To convert rm files to other formats, you can use [rmc](https://github.com/ricklupton/rmc), which combines this library with code for converting lines to SVG, PDF, and simple Markdown.
## Changelog
### Unreleased
### v0.6.1
Fixes:
- Fix AssertionError when some ids are missing in a `CrdtSequence` ([#36](https://github.com/ricklupton/rmscene/pull/36))
- Fix ValueError when the node_id is missing in a `SceneGroupItemBlock` ([#16](https://github.com/ricklupton/rmscene/issues/16))
- Store any unparsed data in blocks as raw bytes to allow for round-trip saving of files written in a newer format than the parsing code knows about.
### v0.6.0
New features:
- Add support for new blocks: `0x0D` (SceneInfo) and `0x08` (SceneTombstoneItemBlock) ([#24](https://github.com/ricklupton/rmscene/pull/24/))
- Add support for `move_id` field on some SceneLineItems ([#24](https://github.com/ricklupton/rmscene/pull/24/))
- Add support for new pen types and colours ([#31](https://github.com/ricklupton/rmscene/pull/31))
### v0.5.0
Breaking changes:
- The `start` property of `GlyphRange` items is now optional
([#15](https://github.com/ricklupton/rmscene/pull/15/)).
- The representation of formatted text spans has changed. Rather than
using nested structures like `BoldSpan` and `ItalicSpan`, the
`CrdtStr` objects now have optional text properties like
`font-weight` and `font-style`. This simplifies the parsing code and
the resulting data structure.
New features:
- Improved error recovery. An error during parsing, or an unknown block type,
results in an `UnreadableBlock` containing the data that could not be read, so
that parsing of other blocks can continue.
- Compatible with new reMarkable software version 3.6 format for
highlighted text
([#15](https://github.com/ricklupton/rmscene/pull/15/)).
- New methods `read_bool_optional` and similar of `TaggedBlockReader`
which return a default value if no matching tagged value is present
in the block.
Other changes and fixes:
- The `value` attribute of scene item blocks, which was not being used, has been
removed.
- Check more carefully for sub-blocks
([#17](https://github.com/ricklupton/rmscene/issues/17#issuecomment-1701071477)).
- Type hints fixed for `expand_text_items`.
### v0.4.0
Breaking changes:
- Rename `scene_items.TextFormat` to `ParagraphStyle` to better describe its
meaning, now that we have inline bold/italic text styles.
- Remove methods from `scene_items.Text` object; use `text.TextDocument`
instead.
- Writer: experimental change to emulate different reMarkable software versions
by passing `{"version": "3.2.2"}` options to `write_blocks`. This allows us to
continue to test round-trip reading and writing of old test files as new data
values are added. Replaces `"line_version"` option.
New features:
- Parse text formatting information (bold and italic) introduced in reMarkable
software version 3.3.
Other changes:
- Allow empty text items and unknown text formats without throwing exceptions.
- When extra data is present in the file, log the unrecognised bytes at DEBUG
logging level along with the call stack, to make it easier to figure out where
the code needs to be modified to read new data.
- Parse new data values (with unknown meaning) in PageInfoBlock and
MigrationInfoBlock.
### v0.3.0
- Introduce `CrdtSequence` type to handle the different places that CRDT
sequences are used, not just for text.
- Introduce `scene_items` module with data structures representing the data,
independently from the `Block`s used to serialize them to `.rm` files.
- Introduce a `SceneTree` structure which holds the `SceneItem`s in
groups/layers.
- Move Text data from `RootTextBlock` to `scene_items.Text` class, which
includes methods for extracting lines of text and formatting.
- Text lines now include the trailing newline character.
- Read `GlyphRange` scene items, representing highlighted text in PDFs.
### v0.2.0
- Try to be more robust to unexpected data introduced by newer reMarkable software versions.
- Only warn once if unknown data is present, rather than for every block.
- Small API change to return type of `read_block` and `read_subblock` methods.
### v0.1.0
- Initial release
## Acknowledgements
https://github.com/ddvk/reader helped a lot in figuring out the structure and meaning of the files. [@adq](https://github.com/adq) discovered a means to get debug output (see [issue 25](https://github.com/ricklupton/rmscene/issues/25)) which is very helpful for understanding the format.
Contributors:
- [@Azeirah](https://github.com/Azeirah) -- code and reporting issues
- [@adq](https://github.com/adq) -- code and reporting issues
- [@dotlambda](https://github.com/dotlambda) -- packaging
- [@ChenghaoMou](https://github.com/ChenghaoMou)
Raw data
{
"_id": null,
"home_page": "https://github.com/ricklupton/rmscene",
"name": "rmscene",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.10",
"maintainer_email": null,
"keywords": null,
"author": "Rick Lupton",
"author_email": "mail@ricklupton.name",
"download_url": "https://files.pythonhosted.org/packages/92/27/bb1c99959216af939ade7b5b470d049cfa368a179a2c021153e93a922c5a/rmscene-0.6.1.tar.gz",
"platform": null,
"description": "# rmscene\n\nPython library to read v6 files from reMarkable tables (software version 3).\n\nIn particular, this version introduces the ability to include text as well as drawn lines. Extracting this text is the original motivation to develop this library, but it also can read much of the other types of data in the reMarkable files.\n\nTo convert rm files to other formats, you can use [rmc](https://github.com/ricklupton/rmc), which combines this library with code for converting lines to SVG, PDF, and simple Markdown.\n\n## Changelog\n\n### Unreleased\n\n### v0.6.1\n\nFixes:\n\n- Fix AssertionError when some ids are missing in a `CrdtSequence` ([#36](https://github.com/ricklupton/rmscene/pull/36))\n- Fix ValueError when the node_id is missing in a `SceneGroupItemBlock` ([#16](https://github.com/ricklupton/rmscene/issues/16)) \n- Store any unparsed data in blocks as raw bytes to allow for round-trip saving of files written in a newer format than the parsing code knows about.\n\n### v0.6.0\n\nNew features:\n\n- Add support for new blocks: `0x0D` (SceneInfo) and `0x08` (SceneTombstoneItemBlock) ([#24](https://github.com/ricklupton/rmscene/pull/24/))\n- Add support for `move_id` field on some SceneLineItems ([#24](https://github.com/ricklupton/rmscene/pull/24/))\n- Add support for new pen types and colours ([#31](https://github.com/ricklupton/rmscene/pull/31))\n\n### v0.5.0\n\nBreaking changes:\n\n- The `start` property of `GlyphRange` items is now optional\n ([#15](https://github.com/ricklupton/rmscene/pull/15/)).\n- The representation of formatted text spans has changed. Rather than\n using nested structures like `BoldSpan` and `ItalicSpan`, the\n `CrdtStr` objects now have optional text properties like\n `font-weight` and `font-style`. This simplifies the parsing code and\n the resulting data structure.\n\nNew features:\n\n- Improved error recovery. An error during parsing, or an unknown block type,\n results in an `UnreadableBlock` containing the data that could not be read, so\n that parsing of other blocks can continue.\n- Compatible with new reMarkable software version 3.6 format for\n highlighted text\n ([#15](https://github.com/ricklupton/rmscene/pull/15/)).\n- New methods `read_bool_optional` and similar of `TaggedBlockReader`\n which return a default value if no matching tagged value is present\n in the block.\n \nOther changes and fixes:\n\n- The `value` attribute of scene item blocks, which was not being used, has been\n removed.\n- Check more carefully for sub-blocks\n ([#17](https://github.com/ricklupton/rmscene/issues/17#issuecomment-1701071477)).\n- Type hints fixed for `expand_text_items`.\n\n### v0.4.0\n\nBreaking changes:\n\n- Rename `scene_items.TextFormat` to `ParagraphStyle` to better describe its\n meaning, now that we have inline bold/italic text styles.\n- Remove methods from `scene_items.Text` object; use `text.TextDocument`\n instead.\n- Writer: experimental change to emulate different reMarkable software versions\n by passing `{\"version\": \"3.2.2\"}` options to `write_blocks`. This allows us to\n continue to test round-trip reading and writing of old test files as new data\n values are added. Replaces `\"line_version\"` option.\n \nNew features:\n\n- Parse text formatting information (bold and italic) introduced in reMarkable\n software version 3.3.\n\nOther changes:\n\n- Allow empty text items and unknown text formats without throwing exceptions.\n- When extra data is present in the file, log the unrecognised bytes at DEBUG\n logging level along with the call stack, to make it easier to figure out where\n the code needs to be modified to read new data.\n- Parse new data values (with unknown meaning) in PageInfoBlock and\n MigrationInfoBlock.\n\n### v0.3.0\n\n- Introduce `CrdtSequence` type to handle the different places that CRDT\n sequences are used, not just for text.\n- Introduce `scene_items` module with data structures representing the data,\n independently from the `Block`s used to serialize them to `.rm` files.\n- Introduce a `SceneTree` structure which holds the `SceneItem`s in\n groups/layers.\n- Move Text data from `RootTextBlock` to `scene_items.Text` class, which\n includes methods for extracting lines of text and formatting.\n- Text lines now include the trailing newline character.\n- Read `GlyphRange` scene items, representing highlighted text in PDFs.\n\n### v0.2.0\n\n- Try to be more robust to unexpected data introduced by newer reMarkable software versions.\n- Only warn once if unknown data is present, rather than for every block.\n- Small API change to return type of `read_block` and `read_subblock` methods.\n\n### v0.1.0\n\n- Initial release\n\n## Acknowledgements\n\nhttps://github.com/ddvk/reader helped a lot in figuring out the structure and meaning of the files. [@adq](https://github.com/adq) discovered a means to get debug output (see [issue 25](https://github.com/ricklupton/rmscene/issues/25)) which is very helpful for understanding the format.\n\nContributors:\n- [@Azeirah](https://github.com/Azeirah) -- code and reporting issues\n- [@adq](https://github.com/adq) -- code and reporting issues\n- [@dotlambda](https://github.com/dotlambda) -- packaging\n- [@ChenghaoMou](https://github.com/ChenghaoMou)\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Read v6 .rm files from the reMarkable tablet",
"version": "0.6.1",
"project_urls": {
"Homepage": "https://github.com/ricklupton/rmscene",
"Repository": "https://github.com/ricklupton/rmscene"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b2774de4d4e0203d83288d33b4e448aafbe9aff6fe4deeb344f28c1cca901b98",
"md5": "76bc3cb6c840ec445ff8af6948080994",
"sha256": "9cd50ae8eede26e3263d78102db880644d39dcc6a5376d38627c060e47bf9045"
},
"downloads": -1,
"filename": "rmscene-0.6.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "76bc3cb6c840ec445ff8af6948080994",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.10",
"size": 25316,
"upload_time": "2024-12-03T21:21:56",
"upload_time_iso_8601": "2024-12-03T21:21:56.043612Z",
"url": "https://files.pythonhosted.org/packages/b2/77/4de4d4e0203d83288d33b4e448aafbe9aff6fe4deeb344f28c1cca901b98/rmscene-0.6.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9227bb1c99959216af939ade7b5b470d049cfa368a179a2c021153e93a922c5a",
"md5": "1f370611b46404216f926203df5c477e",
"sha256": "82f7797bdfa407f9ad27245e7333a7c6e81f8da3390a4f9d9d89736643fe163c"
},
"downloads": -1,
"filename": "rmscene-0.6.1.tar.gz",
"has_sig": false,
"md5_digest": "1f370611b46404216f926203df5c477e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.10",
"size": 22790,
"upload_time": "2024-12-03T21:21:57",
"upload_time_iso_8601": "2024-12-03T21:21:57.361405Z",
"url": "https://files.pythonhosted.org/packages/92/27/bb1c99959216af939ade7b5b470d049cfa368a179a2c021153e93a922c5a/rmscene-0.6.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-03 21:21:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ricklupton",
"github_project": "rmscene",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "rmscene"
}
JSON
