# lmdbm
This is a Python DBM interface style wrapper around [LMDB](http://www.lmdb.tech/doc/) (Lightning Memory-Mapped Database).
It uses the existing lower level Python bindings [py-lmdb](https://lmdb.readthedocs.io).
This is especially useful on Windows, where otherwise `dbm.dumb` is the default `dbm` database.
## Install
- `pip install lmdbm`
## Example
```python
from lmdbm import Lmdb
with Lmdb.open("test.db", "c") as db:
db[b"key"] = b"value"
db.update({b"key1": b"value1", b"key2": b"value2"}) # batch insert, uses a single transaction
```
### Use inheritance to store Python objects using json serialization
```python
import json
from lmdbm import Lmdb
class JsonLmdb(Lmdb):
def _pre_key(self, value):
return value.encode("utf-8")
def _post_key(self, value):
return value.decode("utf-8")
def _pre_value(self, value):
return json.dumps(value).encode("utf-8")
def _post_value(self, value):
return json.loads(value.decode("utf-8"))
with JsonLmdb.open("test.db", "c") as db:
db["key"] = {"some": "object"}
obj = db["key"]
print(obj["some"]) # prints "object"
```
## Warning
As of `lmdb==1.2.1` the docs say that calling `lmdb.Environment.set_mapsize` from multiple processes "may cause catastrophic loss of data". If `lmdbm` is used in write mode from multiple processes, set `autogrow=False` and map_size to a large enough value: `Lmdb.open(..., map_size=2**30, autogrow=False)`.
## Benchmarks
See `benchmark.py` and `requirements-bench.txt`. Other storage engines which could be tested: `wiredtiger`, `berkeleydb`.
Storage engines not benchmarked:
- `tinydb` (because it doesn't have built-in str/bytes keys)
### continuous writes in seconds (best of 3)
| items | lmdbm |lmdbm-batch|pysos |sqlitedict|sqlitedict-batch|dbm.dumb|semidbm|vedis |vedis-batch|unqlite|unqlite-batch|
|------:|-------:|----------:|-----:|---------:|---------------:|-------:|------:|-----:|----------:|------:|------------:|
| 10| 0.000| 0.015| 0.000| 0.031| 0.000| 0.016| 0.000| 0.000| 0.000| 0.000| 0.000|
| 100| 0.094| 0.000| 0.000| 0.265| 0.016| 0.188| 0.000| 0.000| 0.000| 0.000| 0.000|
| 1000| 1.684| 0.016| 0.015| 3.885| 0.124| 2.387| 0.016| 0.015| 0.015| 0.016| 0.000|
| 10000| 16.895| 0.093| 0.265| 45.334| 1.326| 25.350| 0.156| 0.093| 0.094| 0.094| 0.093|
| 100000| 227.106| 1.030| 2.698| 461.638| 12.964| 238.400| 1.623| 1.388| 1.467| 1.466| 1.357|
|1000000|3482.520| 13.104|27.815| 5851.239| 133.396|2432.945| 16.411|15.693| 15.709| 14.508| 14.103|
### random reads in seconds (best of 3)
| items |lmdbm |lmdbm-batch|pysos |sqlitedict|sqlitedict-batch|dbm.dumb|semidbm| vedis |vedis-batch|unqlite|unqlite-batch|
|------:|-----:|-----------|-----:|---------:|----------------|-------:|------:|------:|-----------|------:|-------------|
| 10| 0.000| | 0.000| 0.000| | 0.000| 0.000| 0.000| | 0.000| |
| 100| 0.000| | 0.000| 0.031| | 0.000| 0.000| 0.000| | 0.000| |
| 1000| 0.016| | 0.015| 0.250| | 0.109| 0.016| 0.015| | 0.000| |
| 10000| 0.109| | 0.156| 2.558| | 1.123| 0.171| 0.109| | 0.109| |
| 100000| 1.014| | 2.137| 27.769| | 11.419| 2.090| 1.170| | 1.170| |
|1000000|10.390| |24.258| 447.613| | 870.580| 22.838|214.486| |211.319| |
Raw data
{
"_id": null,
"home_page": "https://github.com/Dobatymo/lmdb-python-dbm",
"name": "lmdbm-lockable",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "",
"author": "Dobatymo",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/48/04/51bab67ab0de4968beb932f4aa34fcf8bc97f88f84ab580da8588feef64f/lmdbm-lockable-0.0.5.tar.gz",
"platform": null,
"description": "# lmdbm\n\nThis is a Python DBM interface style wrapper around [LMDB](http://www.lmdb.tech/doc/) (Lightning Memory-Mapped Database).\nIt uses the existing lower level Python bindings [py-lmdb](https://lmdb.readthedocs.io).\nThis is especially useful on Windows, where otherwise `dbm.dumb` is the default `dbm` database.\n\n## Install\n- `pip install lmdbm`\n\n## Example\n```python\nfrom lmdbm import Lmdb\nwith Lmdb.open(\"test.db\", \"c\") as db:\n db[b\"key\"] = b\"value\"\n db.update({b\"key1\": b\"value1\", b\"key2\": b\"value2\"}) # batch insert, uses a single transaction\n```\n\n### Use inheritance to store Python objects using json serialization\n\n```python\nimport json\nfrom lmdbm import Lmdb\n\nclass JsonLmdb(Lmdb):\n def _pre_key(self, value):\n return value.encode(\"utf-8\")\n def _post_key(self, value):\n return value.decode(\"utf-8\")\n def _pre_value(self, value):\n return json.dumps(value).encode(\"utf-8\")\n def _post_value(self, value):\n return json.loads(value.decode(\"utf-8\"))\n\nwith JsonLmdb.open(\"test.db\", \"c\") as db:\n db[\"key\"] = {\"some\": \"object\"}\n obj = db[\"key\"]\n print(obj[\"some\"]) # prints \"object\"\n```\n\n## Warning\n\nAs of `lmdb==1.2.1` the docs say that calling `lmdb.Environment.set_mapsize` from multiple processes \"may cause catastrophic loss of data\". If `lmdbm` is used in write mode from multiple processes, set `autogrow=False` and map_size to a large enough value: `Lmdb.open(..., map_size=2**30, autogrow=False)`.\n\n## Benchmarks\n\nSee `benchmark.py` and `requirements-bench.txt`. Other storage engines which could be tested: `wiredtiger`, `berkeleydb`.\nStorage engines not benchmarked:\n\t- `tinydb` (because it doesn't have built-in str/bytes keys)\n\n### continuous writes in seconds (best of 3)\n| items | lmdbm |lmdbm-batch|pysos |sqlitedict|sqlitedict-batch|dbm.dumb|semidbm|vedis |vedis-batch|unqlite|unqlite-batch|\n|------:|-------:|----------:|-----:|---------:|---------------:|-------:|------:|-----:|----------:|------:|------------:|\n| 10| 0.000| 0.015| 0.000| 0.031| 0.000| 0.016| 0.000| 0.000| 0.000| 0.000| 0.000|\n| 100| 0.094| 0.000| 0.000| 0.265| 0.016| 0.188| 0.000| 0.000| 0.000| 0.000| 0.000|\n| 1000| 1.684| 0.016| 0.015| 3.885| 0.124| 2.387| 0.016| 0.015| 0.015| 0.016| 0.000|\n| 10000| 16.895| 0.093| 0.265| 45.334| 1.326| 25.350| 0.156| 0.093| 0.094| 0.094| 0.093|\n| 100000| 227.106| 1.030| 2.698| 461.638| 12.964| 238.400| 1.623| 1.388| 1.467| 1.466| 1.357|\n|1000000|3482.520| 13.104|27.815| 5851.239| 133.396|2432.945| 16.411|15.693| 15.709| 14.508| 14.103|\n\n### random reads in seconds (best of 3)\n| items |lmdbm |lmdbm-batch|pysos |sqlitedict|sqlitedict-batch|dbm.dumb|semidbm| vedis |vedis-batch|unqlite|unqlite-batch|\n|------:|-----:|-----------|-----:|---------:|----------------|-------:|------:|------:|-----------|------:|-------------|\n| 10| 0.000| | 0.000| 0.000| | 0.000| 0.000| 0.000| | 0.000| |\n| 100| 0.000| | 0.000| 0.031| | 0.000| 0.000| 0.000| | 0.000| |\n| 1000| 0.016| | 0.015| 0.250| | 0.109| 0.016| 0.015| | 0.000| |\n| 10000| 0.109| | 0.156| 2.558| | 1.123| 0.171| 0.109| | 0.109| |\n| 100000| 1.014| | 2.137| 27.769| | 11.419| 2.090| 1.170| | 1.170| |\n|1000000|10.390| |24.258| 447.613| | 870.580| 22.838|214.486| |211.319| |\n",
"bugtrack_url": null,
"license": "",
"summary": "Python DBM style wrapper around LMDB (Lightning Memory-Mapped Database)",
"version": "0.0.5",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ca38385bb5bb0a056244530557de3ad33038a991243b3f8128d0ccd3d8c7937f",
"md5": "cc9f80c511ba92382b1fb5b22e73597d",
"sha256": "d81037f89712cf303d6e48c9df28678c34aa6e9b1c82a24e9d6c54d4b9096fc4"
},
"downloads": -1,
"filename": "lmdbm_lockable-0.0.5-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "cc9f80c511ba92382b1fb5b22e73597d",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.6",
"size": 5885,
"upload_time": "2023-03-29T23:14:40",
"upload_time_iso_8601": "2023-03-29T23:14:40.634286Z",
"url": "https://files.pythonhosted.org/packages/ca/38/385bb5bb0a056244530557de3ad33038a991243b3f8128d0ccd3d8c7937f/lmdbm_lockable-0.0.5-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "480451bab67ab0de4968beb932f4aa34fcf8bc97f88f84ab580da8588feef64f",
"md5": "add1be7bab4e7e1269dbe08471b13bb5",
"sha256": "f002e6bd8e5a500b2c6cb485ef3786c0ab7735ef75f63ede69e3eb28658c9317"
},
"downloads": -1,
"filename": "lmdbm-lockable-0.0.5.tar.gz",
"has_sig": false,
"md5_digest": "add1be7bab4e7e1269dbe08471b13bb5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 5546,
"upload_time": "2023-03-29T23:14:42",
"upload_time_iso_8601": "2023-03-29T23:14:42.421861Z",
"url": "https://files.pythonhosted.org/packages/48/04/51bab67ab0de4968beb932f4aa34fcf8bc97f88f84ab580da8588feef64f/lmdbm-lockable-0.0.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-29 23:14:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "Dobatymo",
"github_project": "lmdb-python-dbm",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "lmdb",
"specs": []
}
],
"tox": true,
"lcname": "lmdbm-lockable"
}
JSON
