# zlib-state
Low-level interface to the zlib library that enables capturing the decoding state.
## Install
From PyPi:
```
pip install zlib-state
```
From source:
```
pip install .
```
Tested on Ubuntu/macOs/Windows with Python 3.7-3.12.
## GzipStateFile
Wraps Decompressor as a buffered reader.
Based on my benchmarking, this is somewhat slower than python's gzip.
A typical usage pattern looks like:
```python
import zlib_state
TARGET_LINE = 5000 # pick back up after around the 5,000th line
# Specify keep_last_state=True to tell object to grab and keep the state and pos after each block
with zlib_state.GzipStateFile('testdata/frankenstein.txt.gz', keep_last_state=True) as f:
for i, line in enumerate(f):
if i == TARGET_LINE:
state, pos = f.last_state, f.last_state_pos
with zlib_state.GzipStateFile('testdata/frankenstein.txt.gz') as f:
f.zseek(pos, state)
remainder = f.read()
```
## Decompressor
Very basic decompression object that's picky and unforgiving.
Based on my benchmarking, this can iterate over gzip files faster than python's gzip.
A typical usage pattern looks like:
```python
import zlib_state
decomp = zlib_state.Decompressor(32 + 15) # from zlib; 32 indicates gzip header, 15 window size
block_count = 0
with open('testdata/frankenstein.txt.gz', 'rb') as f:
while not decomp.eof():
needed_input = decomp.needs_input()
if needed_input > 0:
# decomp needs more input, and it tells you how much.
decomp.feed_input(f.read(needed_input))
# next_chunk may be empty (e.g., if finished with gzip headers) or may contain data.
# It sends as much as it has left in its output buffer, or asks zlib to continue.
next_chunk = decomp.read() # you can also pass a maximum size to take and/or a buffer to write to
if decomp.block_boundary():
block_count += 1
# When it reaches the end of a deflate block, it always stops. At these times, you can grab the state
# if you wish.
if block_count == 4: # resume after the 4th block
state = decomp.get_state() # includes zdict, bits, byte -- everything it needs to resume from pos
pos = decomp.total_in() # the current position in the binary file to resume from
print(f'{block_count} blocks processed')
# resume from somewhere in the file. Only possible spots are the block boundaries, given the state
f.seek(pos)
decomp = zlib_state.Decompressor(-15) # from zlib; 15 window size, negative means no headers
decomp.set_state(*state)
while not decomp.eof():
needed_input = decomp.needs_input()
if needed_input > 0:
# decomp needs more input, and it tells you how much.
decomp.feed_input(f.read(needed_input))
next_chunk = decomp.read()
```
Raw data
{
"_id": null,
"home_page": "https://github.com/seanmacavaney/zlib-state",
"name": "zlib-state",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": null,
"author": "Sean MacAvaney",
"author_email": "sean.macavaney@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/35/3e/dd482d5bf99d1dabcce0a20a479859cb7a6bd8a365b07b41ebf46b3c0f3d/zlib_state-0.1.10.tar.gz",
"platform": null,
"description": "# zlib-state\n\nLow-level interface to the zlib library that enables capturing the decoding state.\n\n## Install\n\nFrom PyPi:\n\n```\npip install zlib-state\n```\n\nFrom source:\n\n```\npip install .\n```\n\nTested on Ubuntu/macOs/Windows with Python 3.7-3.12.\n\n## GzipStateFile\n\nWraps Decompressor as a buffered reader.\n\nBased on my benchmarking, this is somewhat slower than python's gzip.\n\nA typical usage pattern looks like:\n\n```python\nimport zlib_state\n\nTARGET_LINE = 5000 # pick back up after around the 5,000th line\n# Specify keep_last_state=True to tell object to grab and keep the state and pos after each block\nwith zlib_state.GzipStateFile('testdata/frankenstein.txt.gz', keep_last_state=True) as f:\n for i, line in enumerate(f):\n if i == TARGET_LINE:\n state, pos = f.last_state, f.last_state_pos\n\nwith zlib_state.GzipStateFile('testdata/frankenstein.txt.gz') as f:\n f.zseek(pos, state)\n remainder = f.read()\n```\n\n## Decompressor\n\nVery basic decompression object that's picky and unforgiving.\n\nBased on my benchmarking, this can iterate over gzip files faster than python's gzip.\n\nA typical usage pattern looks like:\n\n```python\nimport zlib_state\n\ndecomp = zlib_state.Decompressor(32 + 15) # from zlib; 32 indicates gzip header, 15 window size\nblock_count = 0\nwith open('testdata/frankenstein.txt.gz', 'rb') as f:\n while not decomp.eof():\n needed_input = decomp.needs_input()\n if needed_input > 0:\n # decomp needs more input, and it tells you how much.\n decomp.feed_input(f.read(needed_input))\n # next_chunk may be empty (e.g., if finished with gzip headers) or may contain data.\n # It sends as much as it has left in its output buffer, or asks zlib to continue.\n next_chunk = decomp.read() # you can also pass a maximum size to take and/or a buffer to write to\n if decomp.block_boundary():\n block_count += 1\n # When it reaches the end of a deflate block, it always stops. At these times, you can grab the state\n # if you wish.\n if block_count == 4: # resume after the 4th block\n state = decomp.get_state() # includes zdict, bits, byte -- everything it needs to resume from pos\n pos = decomp.total_in() # the current position in the binary file to resume from\n print(f'{block_count} blocks processed')\n # resume from somewhere in the file. Only possible spots are the block boundaries, given the state\n f.seek(pos)\n decomp = zlib_state.Decompressor(-15) # from zlib; 15 window size, negative means no headers\n decomp.set_state(*state)\n while not decomp.eof():\n needed_input = decomp.needs_input()\n if needed_input > 0:\n # decomp needs more input, and it tells you how much.\n decomp.feed_input(f.read(needed_input))\n next_chunk = decomp.read()\n```\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Low-level interface to the zlib library that enables capturing the decoding state",
"version": "0.1.10",
"project_urls": {
"Homepage": "https://github.com/seanmacavaney/zlib-state"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "2a8d06b2342bbc7c41f0d6b12918d5d146aa92bce8fd0aa8d466e4b9469f5351",
"md5": "1244f6a62cbb45d973445e636ec0448f",
"sha256": "6496c2601d9558b95fac123b02e2e4e7b5fc89d5d7301027c34cdf8221ddf2a4"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "1244f6a62cbb45d973445e636ec0448f",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.6",
"size": 21421,
"upload_time": "2025-09-09T07:15:29",
"upload_time_iso_8601": "2025-09-09T07:15:29.460513Z",
"url": "https://files.pythonhosted.org/packages/2a/8d/06b2342bbc7c41f0d6b12918d5d146aa92bce8fd0aa8d466e4b9469f5351/zlib_state-0.1.10-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "af9086e128938601f39e780cbc780767592daa112923279a5e58b62b49765fa5",
"md5": "e11f3b8504eca952f23dd7c31d6961d8",
"sha256": "d33eb2b35c0c761fdb4716cb07772f3fe53158e3656c739ed79556d569325b1d"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp310-cp310-win_amd64.whl",
"has_sig": false,
"md5_digest": "e11f3b8504eca952f23dd7c31d6961d8",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.6",
"size": 12780,
"upload_time": "2025-09-09T07:15:39",
"upload_time_iso_8601": "2025-09-09T07:15:39.617560Z",
"url": "https://files.pythonhosted.org/packages/af/90/86e128938601f39e780cbc780767592daa112923279a5e58b62b49765fa5/zlib_state-0.1.10-cp310-cp310-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "05a06e9cea198c52d8c2222f21860f0cad536138327e1aa876f59230efe171f5",
"md5": "6e568796a3f2dae2f2d0b799226737e1",
"sha256": "683e1c908305d2ef30a67010563dd9e58a2a6829deebae964baf15e6188c1e20"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "6e568796a3f2dae2f2d0b799226737e1",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.6",
"size": 21454,
"upload_time": "2025-09-09T07:15:30",
"upload_time_iso_8601": "2025-09-09T07:15:30.270353Z",
"url": "https://files.pythonhosted.org/packages/05/a0/6e9cea198c52d8c2222f21860f0cad536138327e1aa876f59230efe171f5/zlib_state-0.1.10-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "fba6f30f54138d25017f7bfbfc26815f4ea33cde5723b604ecf1ab62ebf1c8ba",
"md5": "2d6230c273425280b95fb4801318fc73",
"sha256": "4cb909c0fc36dc07299b5b6c6f93208f1c7ab9b26c3af52efa1057393201a22c"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp311-cp311-win_amd64.whl",
"has_sig": false,
"md5_digest": "2d6230c273425280b95fb4801318fc73",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.6",
"size": 12777,
"upload_time": "2025-09-09T07:16:06",
"upload_time_iso_8601": "2025-09-09T07:16:06.575165Z",
"url": "https://files.pythonhosted.org/packages/fb/a6/f30f54138d25017f7bfbfc26815f4ea33cde5723b604ecf1ab62ebf1c8ba/zlib_state-0.1.10-cp311-cp311-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "9ec9318a8fa73d41b94810816815e38372d75a8c83c02c9d10dd796443b74ccd",
"md5": "0e4a3c11a8968f04456a4caa69469bda",
"sha256": "6d4f3196f84a4d504f4c04147ec7fd9132651883830f6f07be3702d82731f99e"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "0e4a3c11a8968f04456a4caa69469bda",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.6",
"size": 22001,
"upload_time": "2025-09-09T07:15:31",
"upload_time_iso_8601": "2025-09-09T07:15:31.014778Z",
"url": "https://files.pythonhosted.org/packages/9e/c9/318a8fa73d41b94810816815e38372d75a8c83c02c9d10dd796443b74ccd/zlib_state-0.1.10-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "38d889a7e7fbea33b20dcdefa122afde7e79a9fdbe75cf5b48e13a110a2c8c8e",
"md5": "5f85cd908e37094fa518800f6cd91055",
"sha256": "8465b3ddb7fc11e30a49f38615426e369dd1ac5d3d780d89e759e731dfc7bbf4"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp312-cp312-win_amd64.whl",
"has_sig": false,
"md5_digest": "5f85cd908e37094fa518800f6cd91055",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.6",
"size": 12810,
"upload_time": "2025-09-09T07:15:42",
"upload_time_iso_8601": "2025-09-09T07:15:42.623466Z",
"url": "https://files.pythonhosted.org/packages/38/d8/89a7e7fbea33b20dcdefa122afde7e79a9fdbe75cf5b48e13a110a2c8c8e/zlib_state-0.1.10-cp312-cp312-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "700c2b0803cb9f30bddbc9eda87d251d958d21cfdde826bc1deb1e19ca0ff320",
"md5": "b84b9dc5b0bf5b2c7fb75b27d0b6ff2a",
"sha256": "dfecba070cdeeab073573ac721459727d60e0b8ef7b38dac3c965459781b0eeb"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "b84b9dc5b0bf5b2c7fb75b27d0b6ff2a",
"packagetype": "bdist_wheel",
"python_version": "cp313",
"requires_python": ">=3.6",
"size": 22045,
"upload_time": "2025-09-09T07:15:31",
"upload_time_iso_8601": "2025-09-09T07:15:31.860291Z",
"url": "https://files.pythonhosted.org/packages/70/0c/2b0803cb9f30bddbc9eda87d251d958d21cfdde826bc1deb1e19ca0ff320/zlib_state-0.1.10-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b1d274ff59bb480801eae2731523f98be198eec135a9d37e27791b635f2c9124",
"md5": "049b4ad9e236cf4711f03342ce41a50e",
"sha256": "72e354f09c942055677ba59d76ca8c311a8129dfc98c3b44db33302843090204"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp313-cp313-win_amd64.whl",
"has_sig": false,
"md5_digest": "049b4ad9e236cf4711f03342ce41a50e",
"packagetype": "bdist_wheel",
"python_version": "cp313",
"requires_python": ">=3.6",
"size": 12801,
"upload_time": "2025-09-09T07:15:07",
"upload_time_iso_8601": "2025-09-09T07:15:07.099872Z",
"url": "https://files.pythonhosted.org/packages/b1/d2/74ff59bb480801eae2731523f98be198eec135a9d37e27791b635f2c9124/zlib_state-0.1.10-cp313-cp313-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "e1b283cfa28037f152d623c1cf716013e5938513d414e8ac3c0312e1b839928f",
"md5": "0f1bcbcd448c17a9a0c77e30ceddbdb7",
"sha256": "c86d39c50e046547e23d2f0170556444f1f385c251ce0d5cc00c9d7ed6c0ef1e"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "0f1bcbcd448c17a9a0c77e30ceddbdb7",
"packagetype": "bdist_wheel",
"python_version": "cp314",
"requires_python": ">=3.6",
"size": 22059,
"upload_time": "2025-09-09T07:15:32",
"upload_time_iso_8601": "2025-09-09T07:15:32.963668Z",
"url": "https://files.pythonhosted.org/packages/e1/b2/83cfa28037f152d623c1cf716013e5938513d414e8ac3c0312e1b839928f/zlib_state-0.1.10-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "1bc28eb4a17910c50f786f0ccdbb39c5528ab40e2d7de0521a34f0e588273792",
"md5": "73493060a68c3e215b496013e0f3c042",
"sha256": "b689928cf95b317f8491ab81f02d13864477622a3c3bd8a133420274d8c5bce0"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "73493060a68c3e215b496013e0f3c042",
"packagetype": "bdist_wheel",
"python_version": "cp314",
"requires_python": ">=3.6",
"size": 24117,
"upload_time": "2025-09-09T07:15:33",
"upload_time_iso_8601": "2025-09-09T07:15:33.965725Z",
"url": "https://files.pythonhosted.org/packages/1b/c2/8eb4a17910c50f786f0ccdbb39c5528ab40e2d7de0521a34f0e588273792/zlib_state-0.1.10-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "bcdf80b6e59756cbb0aa712252c444a69a28b7a0cfaaa85c394d8a493155bea7",
"md5": "b2393904e61453e0a19c23fae9dfa815",
"sha256": "442d7d4dd11252a4debd8f0f4edf4e70a5d9fadb486cc20b2c1cb7695e44838d"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp38-cp38-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "b2393904e61453e0a19c23fae9dfa815",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.6",
"size": 21984,
"upload_time": "2025-09-09T07:15:34",
"upload_time_iso_8601": "2025-09-09T07:15:34.711760Z",
"url": "https://files.pythonhosted.org/packages/bc/df/80b6e59756cbb0aa712252c444a69a28b7a0cfaaa85c394d8a493155bea7/zlib_state-0.1.10-cp38-cp38-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "63dded99543466b61c2daa138539a7878f48693eb513dc5ec59cda50fd77a8cb",
"md5": "3c94cbbf029ef7db98a5bdc8d733ab34",
"sha256": "9750b97993a8cc07d110028130aa31b552c5998789f88d8b1c6aaf125486b361"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp38-cp38-win_amd64.whl",
"has_sig": false,
"md5_digest": "3c94cbbf029ef7db98a5bdc8d733ab34",
"packagetype": "bdist_wheel",
"python_version": "cp38",
"requires_python": ">=3.6",
"size": 12683,
"upload_time": "2025-09-09T07:16:31",
"upload_time_iso_8601": "2025-09-09T07:16:31.320340Z",
"url": "https://files.pythonhosted.org/packages/63/dd/ed99543466b61c2daa138539a7878f48693eb513dc5ec59cda50fd77a8cb/zlib_state-0.1.10-cp38-cp38-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "a5516ceb547354815e6cbfc18ac86689ea65c6e5fc976eef6712532a7c02c1cc",
"md5": "3550ffd39a9baa77e1107b71c87723b5",
"sha256": "68b6567ce8467b503349b64913e5d2accebb5161c370d6afdec2511ff33d7a99"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"has_sig": false,
"md5_digest": "3550ffd39a9baa77e1107b71c87723b5",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.6",
"size": 21261,
"upload_time": "2025-09-09T07:15:35",
"upload_time_iso_8601": "2025-09-09T07:15:35.815564Z",
"url": "https://files.pythonhosted.org/packages/a5/51/6ceb547354815e6cbfc18ac86689ea65c6e5fc976eef6712532a7c02c1cc/zlib_state-0.1.10-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "832e801a2100e17efa1f4ff1ad97ae0275a95fd4a9d6865399833f177cce08dd",
"md5": "db431d2d58c6658d688ecc6bac8c79e4",
"sha256": "83f905c96de9abe107c3d1e4f8adfa1fe3558844d643041863b22889f90ac6d8"
},
"downloads": -1,
"filename": "zlib_state-0.1.10-cp39-cp39-win_amd64.whl",
"has_sig": false,
"md5_digest": "db431d2d58c6658d688ecc6bac8c79e4",
"packagetype": "bdist_wheel",
"python_version": "cp39",
"requires_python": ">=3.6",
"size": 12771,
"upload_time": "2025-09-09T07:15:01",
"upload_time_iso_8601": "2025-09-09T07:15:01.500758Z",
"url": "https://files.pythonhosted.org/packages/83/2e/801a2100e17efa1f4ff1ad97ae0275a95fd4a9d6865399833f177cce08dd/zlib_state-0.1.10-cp39-cp39-win_amd64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "353edd482d5bf99d1dabcce0a20a479859cb7a6bd8a365b07b41ebf46b3c0f3d",
"md5": "6e214dcf96151091b23ef18dc4b23c4a",
"sha256": "c29b6b93cea1b80025fbc96fa91ceed8b5e7b54ef08f16d6e4c7f8fb56aad777"
},
"downloads": -1,
"filename": "zlib_state-0.1.10.tar.gz",
"has_sig": false,
"md5_digest": "6e214dcf96151091b23ef18dc4b23c4a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 9573,
"upload_time": "2025-09-09T07:14:12",
"upload_time_iso_8601": "2025-09-09T07:14:12.205909Z",
"url": "https://files.pythonhosted.org/packages/35/3e/dd482d5bf99d1dabcce0a20a479859cb7a6bd8a365b07b41ebf46b3c0f3d/zlib_state-0.1.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-09 07:14:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "seanmacavaney",
"github_project": "zlib-state",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "zlib-state"
}
JSON
