| Name | refcount JSON |
| Version |
1.2.7
JSON |
| download |
| home_page | None |
| Summary | Python classes for reference counting |
| upload_time | 2025-01-14 05:13:24 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.9 |
| license | MIT |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
cffi
typing-extensions
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# refcount - Python classes for reference counting
[](https://github.com/csiro-hydroinformatics/pyrefcount/actions?query=workflow%3Aci)
[](https://csiro-hydroinformatics.github.io/pyrefcount/)
[](https://github.com/csiro-hydroinformatics/pyrefcount/blob/main/LICENSE.txt) 
[](https://codecov.io/gh/csiro-hydroinformatics/pyrefcount)
[](https://pypi.org/project/refcount/)[](https://anaconda.org/conda-forge/refcount)[](https://anaconda.org/conda-forge/refcount)
This package is primarily for managing resources in native libraries, written for instance in C++, from Python. While it boils down to "simply" maintaining a set of counters, **it is deceptively complicated to do so properly** and not end up with memory leaks or crashes. This package offers structured options for reliably managing external native resources. Surprisingly I could not locate an existing package doing just what I needed. Other use cases requiring reference counting, aside from native library resources, may benefit from reusing and extending classes in `refcount`.
`refcount` ( >=0.7) includes classes using [cffi](https://cffi.readthedocs.io/). Other low-level interoperability mechanisms may well be added in the future.
## License
MIT (see [License.txt](https://github.com/csiro-hydroinformatics/pyrefcount/blob/main/LICENSE.txt))
## Documentation
Hosted at [refcount via readthedocs.io](https://pyrefcount.readthedocs.io/en/latest/?badge=latest)
## Source code
The code repository is on [GitHub](https://github.com/csiro-hydroinformatics/pyrefcount).
## Installation
### conda-forge
Using `conda` or `mamba`:
```sh
mamba install -c conda-forge refcount
```
### pypi
```sh
pip install refcount
```
### From source (development)
```sh
pip install -r requirements.txt
pip install -e .
```
## Sample use
The following example is based on one of the unit tests.
Say we have a C++ library with objects and a C API:
```C++
#define TEST_DOG_PTR testnative::dog*
#define TEST_OWNER_PTR testnative::owner*
#define TEST_COUNTED_PTR testnative::reference_counter*
testnative::dog* create_dog();
testnative::owner* create_owner(testnative::dog* d);
void say_walk(testnative::owner* owner);
void release(testnative::reference_counter* obj);
// etc.
```
From the outside of the library the API is exported with opaque pointers `void*` (C structs pointers and native C99 types could be handled too).
```C++
void* create_dog();
void* create_owner(void* d);
void say_walk(void* owner);
void release(void* obj);
// etc.
```
Starting with the end in mind, from Python we want an API hiding the low level details close to the C API, in particular avoiding managing native memory via `release` C API calls, piggybacking the python GC instead.
```python
dog = Dog()
owner = DogOwner(dog)
owner.say_walk()
print(dog.position)
dog = None # the "native dog" is still alive though, as the owner incremented the ref count
owner = None
```
This is doable with `refcount` and the `cffi` package. One possible design is:
```python
ut_ffi = cffi.FFI()
ut_ffi.cdef('extern void* create_dog();')
ut_ffi.cdef('extern void* create_owner( void* d);')
ut_ffi.cdef('extern void say_walk( void* owner);')
ut_ffi.cdef('extern void release( void* obj);')
# etc.
ut_dll = ut_ffi.dlopen('c:/path/to/test_native_library.dll', ut_ffi.RTLD_LAZY) # Lazy loading
class CustomCffiNativeHandle(CffiNativeHandle):
def __init__(self, pointer, prior_ref_count = 0):
super(CustomCffiNativeHandle, self).__init__(pointer, type_id='', prior_ref_count = prior_ref_count)
def _release_handle(self) -> bool:
ut_dll.release(self.get_handle())
return True
class Dog(CustomCffiNativeHandle):
def __init__(self, pointer = None):
if pointer is None:
pointer = ut_dll.create_dog()
super(Dog, self).__init__(pointer)
# etc.
class DogOwner(CustomCffiNativeHandle):
def __init__(self, dog):
super(DogOwner, self).__init__(None)
self._set_handle(ut_dll.create_owner(dog.get_handle()))
self.dog = dog
self.dog.add_ref() # Do note this important reference increment
def say_walk(self):
ut_dll.say_walk(self.get_handle())
def _release_handle(self) -> bool:
super(DogOwner, self)._release_handle()
# super(DogOwner, self)._release_handle()
self.dog.release()
return True
```
## Related work
### Ancestry, acknowledgements
This python package `refcount` actually spawned from prior work for interoperability between C++, R and .NET ([R.NET](https://github.com/rdotnet/rdotnet))
`refcount` features using `cffi` were also significantly informed by Kevin Plastow's [work](https://search.informit.com.au/documentSummary;dn=823898220073899;res=IELENG) while he was at the Australian Bureau of Meteorology; this contribution is gratefully acknowledged.
In you have native interop needs you may also want to look at:
* the nuget package [dynamic-interop-dll](https://github.com/rdotnet/dynamic-interop-dll) for .NET/native interop.
* a set of mostly c++ software [tools for interop with C/C++](https://github.com/csiro-hydroinformatics/c-interop)
* a C# library for [generating interop glue code on top of C API glue code](https://github.com/csiro-hydroinformatics/c-api-wrapper-generation).
### Other python packages
`refcount` was created in part because no existing prior (Python) work could quite fit the need. There are however packages that may better address your particular need:
* [infi.pyutils](https://pypi.org/project/infi.pyutils/) contains a reference counting class.
## Development branch
[](https://github.com/csiro-hydroinformatics/pyrefcount/actions/workflows/build-matrix.yml)
Raw data
{
"_id": null,
"home_page": null,
"name": "refcount",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": null,
"author_email": "J-M <jean-michel.perraud@csiro.au>",
"download_url": "https://files.pythonhosted.org/packages/cd/84/f73d1e7b4c0e6626320513d385593da84b5d87ab58e6e5829c6b3913aca6/refcount-1.2.7.tar.gz",
"platform": null,
"description": "# refcount - Python classes for reference counting\n\n[](https://github.com/csiro-hydroinformatics/pyrefcount/actions?query=workflow%3Aci)\n[](https://csiro-hydroinformatics.github.io/pyrefcount/)\n[](https://github.com/csiro-hydroinformatics/pyrefcount/blob/main/LICENSE.txt) \n[](https://codecov.io/gh/csiro-hydroinformatics/pyrefcount)\n\n[](https://pypi.org/project/refcount/)[](https://anaconda.org/conda-forge/refcount)[](https://anaconda.org/conda-forge/refcount)\n\n![Reference counted native handles](./docs/img/refcount-principles.png \"Reference counted native handles\")\n\nThis package is primarily for managing resources in native libraries, written for instance in C++, from Python. While it boils down to \"simply\" maintaining a set of counters, **it is deceptively complicated to do so properly** and not end up with memory leaks or crashes. This package offers structured options for reliably managing external native resources. Surprisingly I could not locate an existing package doing just what I needed. Other use cases requiring reference counting, aside from native library resources, may benefit from reusing and extending classes in `refcount`.\n\n`refcount` ( >=0.7) includes classes using [cffi](https://cffi.readthedocs.io/). Other low-level interoperability mechanisms may well be added in the future.\n\n## License\n\nMIT (see [License.txt](https://github.com/csiro-hydroinformatics/pyrefcount/blob/main/LICENSE.txt))\n\n## Documentation\n\nHosted at [refcount via readthedocs.io](https://pyrefcount.readthedocs.io/en/latest/?badge=latest)\n\n## Source code\n\nThe code repository is on [GitHub](https://github.com/csiro-hydroinformatics/pyrefcount).\n\n## Installation\n\n### conda-forge\n\nUsing `conda` or `mamba`:\n\n```sh\nmamba install -c conda-forge refcount\n```\n\n### pypi\n\n```sh\npip install refcount\n```\n\n### From source (development)\n\n```sh\npip install -r requirements.txt\npip install -e .\n```\n\n## Sample use\n\nThe following example is based on one of the unit tests.\n\nSay we have a C++ library with objects and a C API:\n\n```C++\n#define TEST_DOG_PTR testnative::dog*\n#define TEST_OWNER_PTR testnative::owner*\n#define TEST_COUNTED_PTR testnative::reference_counter*\n\ntestnative::dog* create_dog();\ntestnative::owner* create_owner(testnative::dog* d);\nvoid say_walk(testnative::owner* owner);\nvoid release(testnative::reference_counter* obj);\n// etc.\n```\n\nFrom the outside of the library the API is exported with opaque pointers `void*` (C structs pointers and native C99 types could be handled too).\n\n```C++\nvoid* create_dog();\nvoid* create_owner(void* d);\nvoid say_walk(void* owner);\nvoid release(void* obj);\n// etc.\n```\n\nStarting with the end in mind, from Python we want an API hiding the low level details close to the C API, in particular avoiding managing native memory via `release` C API calls, piggybacking the python GC instead.\n\n```python\ndog = Dog()\nowner = DogOwner(dog)\nowner.say_walk()\nprint(dog.position)\ndog = None # the \"native dog\" is still alive though, as the owner incremented the ref count\nowner = None\n```\n\nThis is doable with `refcount` and the `cffi` package. One possible design is:\n\n```python\nut_ffi = cffi.FFI()\n\nut_ffi.cdef('extern void* create_dog();')\nut_ffi.cdef('extern void* create_owner( void* d);')\nut_ffi.cdef('extern void say_walk( void* owner);')\nut_ffi.cdef('extern void release( void* obj);')\n# etc.\n\nut_dll = ut_ffi.dlopen('c:/path/to/test_native_library.dll', ut_ffi.RTLD_LAZY) # Lazy loading\n\nclass CustomCffiNativeHandle(CffiNativeHandle):\n def __init__(self, pointer, prior_ref_count = 0):\n super(CustomCffiNativeHandle, self).__init__(pointer, type_id='', prior_ref_count = prior_ref_count)\n\n def _release_handle(self) -> bool:\n ut_dll.release(self.get_handle())\n return True\n\nclass Dog(CustomCffiNativeHandle):\n def __init__(self, pointer = None):\n if pointer is None:\n pointer = ut_dll.create_dog()\n super(Dog, self).__init__(pointer)\n # etc.\n\nclass DogOwner(CustomCffiNativeHandle):\n\n def __init__(self, dog):\n super(DogOwner, self).__init__(None)\n self._set_handle(ut_dll.create_owner(dog.get_handle()))\n self.dog = dog\n self.dog.add_ref() # Do note this important reference increment\n\n def say_walk(self):\n ut_dll.say_walk(self.get_handle())\n\n def _release_handle(self) -> bool:\n super(DogOwner, self)._release_handle()\n # super(DogOwner, self)._release_handle()\n self.dog.release()\n return True\n```\n\n## Related work\n\n### Ancestry, acknowledgements\n\nThis python package `refcount` actually spawned from prior work for interoperability between C++, R and .NET ([R.NET](https://github.com/rdotnet/rdotnet))\n\n`refcount` features using `cffi` were also significantly informed by Kevin Plastow's [work](https://search.informit.com.au/documentSummary;dn=823898220073899;res=IELENG) while he was at the Australian Bureau of Meteorology; this contribution is gratefully acknowledged.\n\nIn you have native interop needs you may also want to look at:\n\n* the nuget package [dynamic-interop-dll](https://github.com/rdotnet/dynamic-interop-dll) for .NET/native interop.\n* a set of mostly c++ software [tools for interop with C/C++](https://github.com/csiro-hydroinformatics/c-interop)\n* a C# library for [generating interop glue code on top of C API glue code](https://github.com/csiro-hydroinformatics/c-api-wrapper-generation).\n\n### Other python packages\n\n`refcount` was created in part because no existing prior (Python) work could quite fit the need. There are however packages that may better address your particular need:\n\n* [infi.pyutils](https://pypi.org/project/infi.pyutils/) contains a reference counting class.\n\n## Development branch\n\n[](https://github.com/csiro-hydroinformatics/pyrefcount/actions/workflows/build-matrix.yml)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Python classes for reference counting",
"version": "1.2.7",
"project_urls": {
"Changelog": "https://csiro-hydroinformatics.github.io/pyrefcount/changelog",
"Documentation": "https://csiro-hydroinformatics.github.io/pyrefcount",
"Homepage": "https://csiro-hydroinformatics.github.io/pyrefcount",
"Issues": "https://github.com/csiro-hydroinformatics/pyrefcount/issues",
"Repository": "https://github.com/csiro-hydroinformatics/pyrefcount"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e85e912fb98dce9d03cf97e9d6449678ebdeef43a7236304aae79c994ae6c488",
"md5": "cc95125a73f1a080aef00e0398ba3f47",
"sha256": "e94f9b56b9a7a5b59c5484fa34b360db27bf4532db135307603b984db39510f1"
},
"downloads": -1,
"filename": "refcount-1.2.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cc95125a73f1a080aef00e0398ba3f47",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 17927,
"upload_time": "2025-01-14T05:13:21",
"upload_time_iso_8601": "2025-01-14T05:13:21.318715Z",
"url": "https://files.pythonhosted.org/packages/e8/5e/912fb98dce9d03cf97e9d6449678ebdeef43a7236304aae79c994ae6c488/refcount-1.2.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cd84f73d1e7b4c0e6626320513d385593da84b5d87ab58e6e5829c6b3913aca6",
"md5": "2b26ca092b3ccbb8e388ef1e814b2e60",
"sha256": "945cb9c89a6800ad6c1df475b59fd343495c81f6eb712ac7a6160921383b83d7"
},
"downloads": -1,
"filename": "refcount-1.2.7.tar.gz",
"has_sig": false,
"md5_digest": "2b26ca092b3ccbb8e388ef1e814b2e60",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 121679,
"upload_time": "2025-01-14T05:13:24",
"upload_time_iso_8601": "2025-01-14T05:13:24.389607Z",
"url": "https://files.pythonhosted.org/packages/cd/84/f73d1e7b4c0e6626320513d385593da84b5d87ab58e6e5829c6b3913aca6/refcount-1.2.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-01-14 05:13:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "csiro-hydroinformatics",
"github_project": "pyrefcount",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"appveyor": true,
"requirements": [
{
"name": "cffi",
"specs": [
[
">=",
"1.11.5"
]
]
},
{
"name": "typing-extensions",
"specs": [
[
">=",
"4.3.0"
]
]
}
],
"lcname": "refcount"
}
