..
Copyright CNRS/Inria/UCA
Contributor(s): Eric Debreuve (since 2021)
eric.debreuve@cnrs.fr
This software is governed by the CeCILL license under French law and
abiding by the rules of distribution of free software. You can use,
modify and/ or redistribute the software under the terms of the CeCILL
license as circulated by CEA, CNRS and INRIA at the following URL
"http://www.cecill.info".
As a counterpart to the access to the source code and rights to copy,
modify and redistribute granted by the license, users are provided only
with a limited warranty and the software's author, the holder of the
economic rights, and the successive licensors have only limited
liability.
In this respect, the user's attention is drawn to the risks associated
with loading, using, modifying and/or developing or reproducing the
software by the user in light of its specific status of free software,
that may mean that it is complicated to manipulate, and that also
therefore means that it is reserved for developers and experienced
professionals having in-depth computer knowledge. Users are therefore
encouraged to load and test the software's suitability as regards their
requirements in conditions enabling the security of their systems and/or
data to be ensured and, more generally, to use and operate it in the
same conditions as regards security.
The fact that you are presently reading this means that you have had
knowledge of the CeCILL license and that you accept its terms.
.. |PROJECT_NAME| replace:: PCA-B-Stream
.. |SHORT_DESCRIPTION| replace:: Byte Stream Representation of Piecewise-constant Array
.. |PYPI_NAME_LITERAL| replace:: ``pca-b-stream``
.. |PYPI_PROJECT_URL| replace:: https://pypi.org/project/pca-b-stream/
.. _PYPI_PROJECT_URL: https://pypi.org/project/pca-b-stream/
===================================
|PROJECT_NAME|: |SHORT_DESCRIPTION|
===================================
Installation
============
A Web app of |PROJECT_NAME| might be available on `PythonAnywhere <https://ericdbv.eu.pythonanywhere.com>`_.
This project is published
on the `Python Package Index (PyPI) <https://pypi.org/>`_
at: |PYPI_PROJECT_URL|_.
It should be installable from Python distribution platforms or Integrated Development Environments (IDEs).
Otherwise, it can be installed from a command console:
- For all users, after acquiring administrative rights:
- First installation: ``pip install`` |PYPI_NAME_LITERAL|
- Installation update: ``pip install --upgrade`` |PYPI_NAME_LITERAL|
- For the current user (no administrative rights required):
- First installation: ``pip install --user`` |PYPI_NAME_LITERAL|
- Installation update: ``pip install --user --upgrade`` |PYPI_NAME_LITERAL|
Brief Description
=================
In a Few Words
--------------
The ``PCA-B-Stream`` project allows to generate a printable `byte stream <https://docs.python.org/3/library/stdtypes.html#bytes-objects>`_ representation of a piecewise-constant `Numpy array <https://numpy.org/devdocs/reference/generated/numpy.ndarray.html>`_, and to re-create the array from the byte stream, similarly to what is available as part of the `COCO API <https://github.com/cocodataset/cocoapi>`_.
Illustration
------------
In Python:
.. code-block:: python
>>> import pca_b_stream as pcas
>>> import numpy as nmpy
>>> # --- Array creation
>>> array = nmpy.zeros((10, 10), dtype=nmpy.uint8)
>>> array[1, 1] = 1
>>> # --- Array -> Byte stream -> Array
>>> stream = pcas.PCA2BStream(array)
>>> decoding = pcas.BStream2PCA(stream)
>>> # --- Check and print
>>> assert nmpy.array_equal(decoding, array)
>>> print(stream)
b'FnmHoFain+3jtU'
From command line:
.. code-block:: sh
pca2bstream some_image_file # Prints the corresponding byte stream
bstream2pca a_byte_stream a_filename # Creates an image from the byte stream and stores it
.. _sct_motivations:
Motivations
===========
The motivations for developing an alternative to existing solutions are:
- Arrays can be of any dimension (i.e., not just 2-dimensional),
- Their `dtype <https://numpy.org/devdocs/reference/generated/numpy.dtype.html>`_ can be of boolean, integer, or float types,
- They can contain more than 2 distinct values (i.e., non-binary arrays) as long as the values are integers (potentially stored in a floating-point format though),
- The byte stream representation is self-contained; In particular, there is no need to keep track of the array shape externally,
- The byte stream representation contains everything needed to re-create the array *exactly* as it was instantiated (``dtype``, endianness, C or Fortran ordering); See `note <note_on_exact_>`_ though.
.. _note_on_exact:
.. note::
The statement "re-create the array *exactly* as it was instantiated" is over-confident. First this has not been fully tested by, for example, re-creating an array on a another machine with a native endianness different from the one it was originally instantiated on. Second, more work might be required to ensure that enumeration ordering is correctly dealt with.
Documentation
=============
Functions
---------
The ``pca_b_stream`` module defines the following functions:
- ``PCA2BStream``
- Generates the byte stream representation of an array; Does not check the array validity (see ``PCArrayIssues``)
- Input: a `Numpy ndarray <https://numpy.org/devdocs/reference/generated/numpy.ndarray.html>`_
- Output: an object of type `bytes <https://docs.python.org/3/library/stdtypes.html#bytes-objects>`_
- ``BStream2PCA``
- Re-creates the array from its bytes stream representation; Does not check the stream format validity
- Input/Output: input and output of ``PCA2BStream`` swapped
- ``PCArrayIssues``
- Checks whether an array is a valid input for stream representation generation; It is meant to be used before calling ``PCA2BStream``
- Input: a `Numpy ndarray <https://numpy.org/devdocs/reference/generated/numpy.ndarray.html>`_
- Output: a tuple issues in ``str`` format. The tuple is empty if the array is valid.
- Additional information about what are valid piecewise-constant arrays here is provided in the section `"Motivations" <sct_motivations_>`_.
- ``BStreamDetails``
- Extract details from a byte stream representation; See section `"Byte Stream Format" <byte_stream_format_>`_
- Inputs:
- a byte stream generated by ``PCA2BStream``
- details: a string where each character corresponds to a detail to extract, or "+" to extract all of the available details; Default: "+"; Available details are:
- c: compression indicator
- d: array dimension
- l: array lengths per dimension
- t: dtype type code; See: https://numpy.org/doc/stable/reference/generated/numpy.dtype.char.html
- T: dtype name; Translated from "t" by Numpy.sctypeDict
- o: enumeration order; See: https://numpy.org/doc/stable/reference/generated/numpy.ndarray.flags.html, ?_CONTIGUOUS
- e: endianness (or byte order); See: https://numpy.org/doc/stable/reference/generated/numpy.dtype.byteorder.html
- should_print: a boolean to instruct whether the extracted details should be printed to console; Defaults: False
- should_return: a boolean to instruct whether the extracted details should be returned (see Outputs); Defaults: True
- Output: either one of:
- None if should_return is False
- a dictionary of all of the available details if the ``details`` parameter is "+"
- a tuple of the requested details in the same order as in the ``details`` parameter
Command Line Scripts
--------------------
The ``PCA-B-Stream`` project defines two command line scripts: ``pca2bstream`` and ``bstream2pca``. The former takes a path to an image file as argument, and prints the corresponding byte stream (without the "b" string type prefix). The latter takes a character string and a filename as arguments, in that order, and creates an image file with this name that corresponds to the string interpreted as a byte stream. The file must not already exist.
.. _byte_stream_format:
Byte Stream Format
------------------
A byte stream is a `base85-encoded <https://docs.python.org/3/library/base64.html#base64.b85encode>`_ stream. Once decoded, it has the following format (in lexicographical order; all characters are in ``bytes`` format):
- one character "0" or "1": indicates whether the remaining of the stream is in uncompressed or `ZLIB compressed <https://docs.python.org/3/library/zlib.html#zlib.compress>`_ format; See `note on compression <note_on_compression_>`_; The remaining of the description applies to the stream in uncompressed format
- 3 characters "{E}{T}{O}":
- E: endianness among "|", "<" and ">"
- T: ``dtype`` character code among: "?" + numpy.typecodes["AllInteger"] + numpy.typecodes["Float"]
- O: enumeration order among "C" (C-ordering) and "F" (Fortran-ordering)
- one integer for the dimension of the array (1 for vectors, 2 for matrices, 3 for volumes...)
- one integer per dimension giving the length of the array in that dimension
The remaining of the stream is the actual array content.
- If the array is not all False's or zeros:
- one character "0" or "1": whether the first value in the array is zero (or False) or one (or True)
- one integer for the length of the run-length representation
- integers of the `run-length representation <https://en.wikipedia.org/wiki/Run-length_encoding>`_ of the array read in its proper enumeration order
- If the array is all False's or zeros:
- one character "2"
All the integers are encoded by the `unsigned LEB128 encoding <https://en.wikipedia.org/wiki/LEB128#Unsigned_LEB128>`_ using the `leb128 project <https://github.com/mohanson/leb128>`_.
For non-boolean arrays with a maximum value of 2 or more, the content part is the concatenation of the sub-contents corresponding to each value between 1 and the maximum value in the array.
.. _note_on_compression:
.. note::
For small arrays, compressing the byte stream actually produces a longer stream.
Acknowledgments
===============
The project is developed with `PyCharm Community <https://www.jetbrains.com/pycharm/>`_.
The development relies on several open-source packages
(see ``install_requires`` in ``setup.py``, if present; otherwise ``import`` statements should be searched for).
The code is formatted by `Black <https://github.com/psf/black/>`_, *The Uncompromising Code Formatter*.
The imports are ordered by `isort <https://github.com/timothycrosley/isort/>`_... *your imports, so you don't have to*.
Raw data
{
"_id": null,
"home_page": "https://gitlab.inria.fr/edebreuv/pca-b-stream/",
"name": "pca-b-stream",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "array,image,piecewise constant,byte stream",
"author": "Eric Debreuve",
"author_email": "eric.debreuve@cnrs.fr",
"download_url": "https://files.pythonhosted.org/packages/3a/44/683fb07f7cef3e9bc1bbad5ab87da9f3c66b796bd1c801c3b608f4ce1762/pca-b-stream-2023.2.tar.gz",
"platform": null,
"description": "..\n Copyright CNRS/Inria/UCA\n Contributor(s): Eric Debreuve (since 2021)\n\n eric.debreuve@cnrs.fr\n\n This software is governed by the CeCILL license under French law and\n abiding by the rules of distribution of free software. You can use,\n modify and/ or redistribute the software under the terms of the CeCILL\n license as circulated by CEA, CNRS and INRIA at the following URL\n \"http://www.cecill.info\".\n\n As a counterpart to the access to the source code and rights to copy,\n modify and redistribute granted by the license, users are provided only\n with a limited warranty and the software's author, the holder of the\n economic rights, and the successive licensors have only limited\n liability.\n\n In this respect, the user's attention is drawn to the risks associated\n with loading, using, modifying and/or developing or reproducing the\n software by the user in light of its specific status of free software,\n that may mean that it is complicated to manipulate, and that also\n therefore means that it is reserved for developers and experienced\n professionals having in-depth computer knowledge. Users are therefore\n encouraged to load and test the software's suitability as regards their\n requirements in conditions enabling the security of their systems and/or\n data to be ensured and, more generally, to use and operate it in the\n same conditions as regards security.\n\n The fact that you are presently reading this means that you have had\n knowledge of the CeCILL license and that you accept its terms.\n\n.. |PROJECT_NAME| replace:: PCA-B-Stream\n.. |SHORT_DESCRIPTION| replace:: Byte Stream Representation of Piecewise-constant Array\n\n.. |PYPI_NAME_LITERAL| replace:: ``pca-b-stream``\n.. |PYPI_PROJECT_URL| replace:: https://pypi.org/project/pca-b-stream/\n.. _PYPI_PROJECT_URL: https://pypi.org/project/pca-b-stream/\n\n\n\n===================================\n|PROJECT_NAME|: |SHORT_DESCRIPTION|\n===================================\n\n\n\nInstallation\n============\n\nA Web app of |PROJECT_NAME| might be available on `PythonAnywhere <https://ericdbv.eu.pythonanywhere.com>`_.\n\nThis project is published\non the `Python Package Index (PyPI) <https://pypi.org/>`_\nat: |PYPI_PROJECT_URL|_.\nIt should be installable from Python distribution platforms or Integrated Development Environments (IDEs).\nOtherwise, it can be installed from a command console:\n\n- For all users, after acquiring administrative rights:\n - First installation: ``pip install`` |PYPI_NAME_LITERAL|\n - Installation update: ``pip install --upgrade`` |PYPI_NAME_LITERAL|\n- For the current user (no administrative rights required):\n - First installation: ``pip install --user`` |PYPI_NAME_LITERAL|\n - Installation update: ``pip install --user --upgrade`` |PYPI_NAME_LITERAL|\n\n\n\nBrief Description\n=================\n\nIn a Few Words\n--------------\n\nThe ``PCA-B-Stream`` project allows to generate a printable `byte stream <https://docs.python.org/3/library/stdtypes.html#bytes-objects>`_ representation of a piecewise-constant `Numpy array <https://numpy.org/devdocs/reference/generated/numpy.ndarray.html>`_, and to re-create the array from the byte stream, similarly to what is available as part of the `COCO API <https://github.com/cocodataset/cocoapi>`_.\n\n\n\nIllustration\n------------\n\nIn Python:\n\n.. code-block:: python\n\n >>> import pca_b_stream as pcas\n >>> import numpy as nmpy\n >>> # --- Array creation\n >>> array = nmpy.zeros((10, 10), dtype=nmpy.uint8)\n >>> array[1, 1] = 1\n >>> # --- Array -> Byte stream -> Array\n >>> stream = pcas.PCA2BStream(array)\n >>> decoding = pcas.BStream2PCA(stream)\n >>> # --- Check and print\n >>> assert nmpy.array_equal(decoding, array)\n >>> print(stream)\n b'FnmHoFain+3jtU'\n\nFrom command line:\n\n.. code-block:: sh\n\n pca2bstream some_image_file # Prints the corresponding byte stream\n bstream2pca a_byte_stream a_filename # Creates an image from the byte stream and stores it\n\n\n\n.. _sct_motivations:\n\nMotivations\n===========\n\nThe motivations for developing an alternative to existing solutions are:\n\n- Arrays can be of any dimension (i.e., not just 2-dimensional),\n- Their `dtype <https://numpy.org/devdocs/reference/generated/numpy.dtype.html>`_ can be of boolean, integer, or float types,\n- They can contain more than 2 distinct values (i.e., non-binary arrays) as long as the values are integers (potentially stored in a floating-point format though),\n- The byte stream representation is self-contained; In particular, there is no need to keep track of the array shape externally,\n- The byte stream representation contains everything needed to re-create the array *exactly* as it was instantiated (``dtype``, endianness, C or Fortran ordering); See `note <note_on_exact_>`_ though.\n\n\n.. _note_on_exact:\n\n.. note::\n The statement \"re-create the array *exactly* as it was instantiated\" is over-confident. First this has not been fully tested by, for example, re-creating an array on a another machine with a native endianness different from the one it was originally instantiated on. Second, more work might be required to ensure that enumeration ordering is correctly dealt with.\n\n\n\nDocumentation\n=============\n\nFunctions\n---------\n\nThe ``pca_b_stream`` module defines the following functions:\n\n- ``PCA2BStream``\n - Generates the byte stream representation of an array; Does not check the array validity (see ``PCArrayIssues``)\n - Input: a `Numpy ndarray <https://numpy.org/devdocs/reference/generated/numpy.ndarray.html>`_\n - Output: an object of type `bytes <https://docs.python.org/3/library/stdtypes.html#bytes-objects>`_\n- ``BStream2PCA``\n - Re-creates the array from its bytes stream representation; Does not check the stream format validity\n - Input/Output: input and output of ``PCA2BStream`` swapped\n- ``PCArrayIssues``\n - Checks whether an array is a valid input for stream representation generation; It is meant to be used before calling ``PCA2BStream``\n - Input: a `Numpy ndarray <https://numpy.org/devdocs/reference/generated/numpy.ndarray.html>`_\n - Output: a tuple issues in ``str`` format. The tuple is empty if the array is valid.\n - Additional information about what are valid piecewise-constant arrays here is provided in the section `\"Motivations\" <sct_motivations_>`_.\n- ``BStreamDetails``\n - Extract details from a byte stream representation; See section `\"Byte Stream Format\" <byte_stream_format_>`_\n - Inputs:\n - a byte stream generated by ``PCA2BStream``\n - details: a string where each character corresponds to a detail to extract, or \"+\" to extract all of the available details; Default: \"+\"; Available details are:\n - c: compression indicator\n - d: array dimension\n - l: array lengths per dimension\n - t: dtype type code; See: https://numpy.org/doc/stable/reference/generated/numpy.dtype.char.html\n - T: dtype name; Translated from \"t\" by Numpy.sctypeDict\n - o: enumeration order; See: https://numpy.org/doc/stable/reference/generated/numpy.ndarray.flags.html, ?_CONTIGUOUS\n - e: endianness (or byte order); See: https://numpy.org/doc/stable/reference/generated/numpy.dtype.byteorder.html\n - should_print: a boolean to instruct whether the extracted details should be printed to console; Defaults: False\n - should_return: a boolean to instruct whether the extracted details should be returned (see Outputs); Defaults: True\n - Output: either one of:\n - None if should_return is False\n - a dictionary of all of the available details if the ``details`` parameter is \"+\"\n - a tuple of the requested details in the same order as in the ``details`` parameter\n\n\n\nCommand Line Scripts\n--------------------\n\nThe ``PCA-B-Stream`` project defines two command line scripts: ``pca2bstream`` and ``bstream2pca``. The former takes a path to an image file as argument, and prints the corresponding byte stream (without the \"b\" string type prefix). The latter takes a character string and a filename as arguments, in that order, and creates an image file with this name that corresponds to the string interpreted as a byte stream. The file must not already exist.\n\n\n\n.. _byte_stream_format:\n\nByte Stream Format\n------------------\n\nA byte stream is a `base85-encoded <https://docs.python.org/3/library/base64.html#base64.b85encode>`_ stream. Once decoded, it has the following format (in lexicographical order; all characters are in ``bytes`` format):\n\n- one character \"0\" or \"1\": indicates whether the remaining of the stream is in uncompressed or `ZLIB compressed <https://docs.python.org/3/library/zlib.html#zlib.compress>`_ format; See `note on compression <note_on_compression_>`_; The remaining of the description applies to the stream in uncompressed format\n- 3 characters \"{E}{T}{O}\":\n - E: endianness among \"|\", \"<\" and \">\"\n - T: ``dtype`` character code among: \"?\" + numpy.typecodes[\"AllInteger\"] + numpy.typecodes[\"Float\"]\n - O: enumeration order among \"C\" (C-ordering) and \"F\" (Fortran-ordering)\n- one integer for the dimension of the array (1 for vectors, 2 for matrices, 3 for volumes...)\n- one integer per dimension giving the length of the array in that dimension\n\nThe remaining of the stream is the actual array content.\n\n- If the array is not all False's or zeros:\n - one character \"0\" or \"1\": whether the first value in the array is zero (or False) or one (or True)\n - one integer for the length of the run-length representation\n - integers of the `run-length representation <https://en.wikipedia.org/wiki/Run-length_encoding>`_ of the array read in its proper enumeration order\n- If the array is all False's or zeros:\n - one character \"2\"\n\nAll the integers are encoded by the `unsigned LEB128 encoding <https://en.wikipedia.org/wiki/LEB128#Unsigned_LEB128>`_ using the `leb128 project <https://github.com/mohanson/leb128>`_.\n\nFor non-boolean arrays with a maximum value of 2 or more, the content part is the concatenation of the sub-contents corresponding to each value between 1 and the maximum value in the array.\n\n\n.. _note_on_compression:\n\n.. note::\n For small arrays, compressing the byte stream actually produces a longer stream.\n\n\n\nAcknowledgments\n===============\n\nThe project is developed with `PyCharm Community <https://www.jetbrains.com/pycharm/>`_.\n\nThe development relies on several open-source packages\n(see ``install_requires`` in ``setup.py``, if present; otherwise ``import`` statements should be searched for).\n\nThe code is formatted by `Black <https://github.com/psf/black/>`_, *The Uncompromising Code Formatter*.\n\nThe imports are ordered by `isort <https://github.com/timothycrosley/isort/>`_... *your imports, so you don't have to*.\n",
"bugtrack_url": null,
"license": "CeCILL-2.1",
"summary": "Byte Stream Representation of Piecewise-constant Array",
"version": "2023.2",
"project_urls": {
"Documentation": "https://gitlab.inria.fr/edebreuv/pca-b-stream/",
"Homepage": "https://gitlab.inria.fr/edebreuv/pca-b-stream/",
"Source": "https://gitlab.inria.fr/edebreuv/pca-b-stream/"
},
"split_keywords": [
"array",
"image",
"piecewise constant",
"byte stream"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f4727eb20f67223ef38ada0bdc52ace1406284d0baefa8d37aeecbd3924e2aff",
"md5": "8c97ea7811c93eb789c6df2a0068d73b",
"sha256": "9e66d3b30c8d5b97be9bb2928fed821b3bc66e530fa2596109647e5d2cf98bbb"
},
"downloads": -1,
"filename": "pca_b_stream-2023.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8c97ea7811c93eb789c6df2a0068d73b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 26572,
"upload_time": "2023-07-21T14:51:52",
"upload_time_iso_8601": "2023-07-21T14:51:52.331327Z",
"url": "https://files.pythonhosted.org/packages/f4/72/7eb20f67223ef38ada0bdc52ace1406284d0baefa8d37aeecbd3924e2aff/pca_b_stream-2023.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3a44683fb07f7cef3e9bc1bbad5ab87da9f3c66b796bd1c801c3b608f4ce1762",
"md5": "838c3dd733d8999257ae03b9ec045aa2",
"sha256": "bd9db6dc918621f98772e1a50054edba8408234a21e0a8da53489ff697318ede"
},
"downloads": -1,
"filename": "pca-b-stream-2023.2.tar.gz",
"has_sig": false,
"md5_digest": "838c3dd733d8999257ae03b9ec045aa2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 30850,
"upload_time": "2023-07-21T14:51:54",
"upload_time_iso_8601": "2023-07-21T14:51:54.456072Z",
"url": "https://files.pythonhosted.org/packages/3a/44/683fb07f7cef3e9bc1bbad5ab87da9f3c66b796bd1c801c3b608f4ce1762/pca-b-stream-2023.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-21 14:51:54",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "pca-b-stream"
}
JSON