# Gink in General
Gink aims to be a "protocol first" database system defined by the protocol for syncronizing
instances, rather than by a specific implementation. Defining the database in terms of
the interchange format allows for independent implementations to interact seamlessly in
a well-defined manner.
## Take a look at the full docs [here](https://gink.readthedocs.io/en/latest/).
I created the Python implementation of Gink to be a testbed for new ideas and
to provide the simplest expression of all the concepts in Gink. Well written python
code can essentially serve as executable psudocode. Code written for this implementation
has been biased in favor of readability and extensibility, rather than raw performance.
For example, the code doesn't use async functions or multi-threading.
* [Installation](#installation)
* [Examples](#examples)
* [Data Structures](#data-structures)
* [Box](#box)
* [Directory](#directory)
* [Sequence](#sequence)
* [All Containers](#all-containers)
* [Database Operations](#database-operations)
## Installation
Assuming you have Python and Pip installed:
```sh
pip3 install gink
```
## Examples
This page does not include examples for all data structures. Take a look at our [Python documentation](https://gink.readthedocs.io/en/latest/) for all examples and full docs.
### Data Structures
There are some operations that are available to all Containers, which are shown at the [end of this page](#all-containers).\
\
For all operations, a store and a database are needed:
```py
from gink import *
store = LmdbStore('example.db')
database = Database(store=store)
```
#### Box
A Box is the simplest data structure available on Gink. It can hold only one value at a time; you can set its value, or get its value.
```py
box = Box(database=database)
box.set({"foo": "bar", "key2": 15})
result = box.get() # Returns the python dictionary just added
if not box.isEmpty():
print(box.size()) # This will only return 0 or 1 (1 in this case).
```
#### Directory
The Directory aims to mimic the functionality of a Python dictionary. If you know how to use a dictionary, you should already know how to use the directory!\
\
Create a new directory:
```python
directory = Directory(database=database)
```
Set key: value pairs:
```py
directory["key1"] = "value1"
# Saves a timestamp after "key1" and before "key2"
time = database.get_now() # more on this in All Containers examples
# Achieves the same thing as the previous set, just different syntax.
directory.set("key2", {"test": "document"})
```
Getting the value of a key:
```py
result = directory["key1"] # Returns "value1"
result2 = directory.get("key2") # Returns {"test": "document"}
result3 = directory.get("key3") # Returns None
```
Get all keys and items:
```py
# Returns an generator of ["key1", "key2"]
# Note: the order may not be the same.
keys = directory.keys()
# Returns the items as a generator of (key, value tuples) in the directory
# at the specified timestamp - in this case, [("key1", "value1")]
items = directory.items(as_of=time)
# returns a list of all values
values = directory.values()
```
Deleting keys:
```py
# Returns "value1" and removes the key: value pair from the directory.
value = directory.pop("key1")
# delete the key and return the Muid of this change
del_muid = directory.delete("key2")
```
Setting multiple keys and values at the same time:
```py
directory.update({"newkey": "newvalue", "another key": "another value"})
```
#### Sequence
The Sequence is equivalent to a Python list. Again, these operations should look pretty familiar! In a Gink Sequence, the contents are ordered by timestamps.\
\
Create a Sequence and append some values:
```python
sequence = Sequence()
sequence.append("Hello, World!")
sequence.append(42)
sequence.append("a")
sequence.append("b")
```
Search for value and return index if found:
```py
found = sequence.index("Hello, World!")
# Returns 0
```
Pop values:
```py
popped = sequence.pop(1)
# Returns 42
# Pops and returns the value at index 0, which is "Hello, World!"
# The destination argument allows you to place the item
# back into the sequence at a different timestamp
# in this case, -1 would indicate the timestamp of the last change.
# So, this sequence is now ordered as ["a", "b", "Hello, World!"]
popped = sequence.pop(0, dest=-1)
```
Insert to specific index:
```py
# Inserts "x" at index 1, between "a" and "b", in this example.
# Comment is an optional parameter that will be included in
# bundle for this change (most operations may contain comments).
sequence.insert(1, "x", comment="insert x")
```
Return the sequence as a Python list:
```py
as_list = list(sequence)
```
#### All Containers
The Container is the parent class for all Gink data structures. Here are some examples of the powerful operations you can do with any container:
##### From Contents
To make it easier to insert data into an object upon initialization, Gink allows you to specify a `contents` argument to the constructor of the object. Different data structures may take different types as contents, but the idea remains the same for all Gink objects.
```python
directory = Directory(database=database, contents={
"key1": "value1", "key2": 42, "key3": [1, 2, 3, 4]})
key_set = KeySet(database=database, contents=["key1", "key2", 3])
# Vertex creation for pair map population
vertex1 = Vertex()
vertex2 = Vertex()
# Pair Map contents only takes a dictionary. Read the docs for the
# accepted data types for other data structures.
pair_map = PairMap(contents={(noun1, noun2): "value"})
```
##### Back in time
You will frequently see `as_of` in the Gink documentation. `as_of` refers to the time to look back to. There are multiple ways of interacting with `as_of`. If you are curious about how certain timestamps are resolved, take a look at `Database.resolve_timestamp()`\
One easy way is to pass a negative integer indicating how many changes back you want to look.
```python
box = Box(contents="first_value")
box.set("second_value")
# Passing -1 into the as_of argument looks back at the previous value
# Returns "first_value"
previous = box.get(as_of=-1)
```
Another common way to use timestamps is to "save" a time between changes as a variable.
```python
box = Box(contents="first_value")
time_after_first = database.get_now()
box.set("second_value")
# Passing saved timestamp into as_of
# Returns "first_value"
previous = box.get(as_of=time_after_first)
```
##### Reset
Resetting a container is a fundamental operation used to revert the container back to a previous time. Above we looked at using timestamps to get previous values, but resetting to specific times may prove more useful. This example uses a directory, but this functionality works the same for all containers.
```python
directory = Directory()
directory["foo"] = "bar"
directory["bar"] = "foo"
time_between = database.get_now()
directory[7] = {"user": 1003203, "email": "test@test.com"}
has_7 = 7 in directory # returns True
directory.reset(to_time=time_between)
has_7 = 7 in directory # now returns False
has_bar = "bar" in directory # still returns True
```
#### Clearing
Clearing a container does exactly what you would expect it to do. The `Container.clear()` method removes all entries from the container and returns the Muid of the clearance. The clearance is processed as a new database change, which means you can still look back at previous timestamps or reset the database back to before the clearance occurred.
```python
directory = Directory()
directory["foo"] = "bar"
directory["bar"] = "foo"
directory[7] = {"user": 1003203, "email": "test@test.com"}
# Storing the muid of the clearance to use later
clearance_muid = directory.clear()
# Directory is now empty
has_foo = "foo" in directory # Returns False
has_bar = "bar" in directory # Returns False
has_7 = "7" in directory # Returns False
# Using the muid's timestamp to look back before the clearance
# Returns "bar"
previous = directory.get("foo", as_of=clearance_muid.timestamp)
```
### Database Operations
#### Bundling and comments
Think of a bundle as a commit in Git. A bundle is just a collection of changes with an optional comment/message. Without specifying a bundler object, most Gink operations will immediately commit the change in its own bundle, so you don't have to worry about always creating a new bundler, etc. However, if you do want to specify which changes go into a specific bundle, here is an example:
```python
directory = Directory()
bundler = Bundler(comment="example setting values in directory")
directory.set("key1", 1, bundler=bundler)
directory.set("key2", "value2", bundler=bundler)
directory.update({"key3": 3, "key4": 4}, bundler=bundler)
# This seals the bundler and commits changes to database
# at this point, no more changes may be added
database.commit(bundler)
```
#### Reset
Similar to how `Container.reset()` works, the Database class has its own reset functionality that will reset all containers to the specified time. A "reset" is simply one large bundle of changes that updates the database entries to what they were are the previous timestamp; this allows you to easily look back before the reset.
```python
database = Database(store=store)
root = Directory.get_global_instance(database=database)
queue = Sequence.get_global_instance(database=database)
misc = Directory()
misc["yes"] = False
root["foo"] = "bar"
queue.append("value1")
# No as_of argument defaults to EPOCH
# which is the time of database creation (empty)
database.reset()
# All containers will have a length of 0
# since the database is now empty.
size = len(root)
# to_time=-1 reverts the database to the
# previous change
database.reset(to_time=-1)
# This will now have a len of 1,
# and one element of "value1"
size = len(queue)
```
Raw data
{
"_id": null,
"home_page": "https://github.com/x5e/gink",
"name": "gink",
"maintainer": null,
"docs_url": null,
"requires_python": "<4,>=3.8",
"maintainer_email": null,
"keywords": "gink lmdb crdt history versioned",
"author": "Darin McGill",
"author_email": "gink@darinmcgill.com",
"download_url": "https://files.pythonhosted.org/packages/1f/f2/14502257f1d345c3e863d6799e7ab416bd5e69e65d919acb58fedf29a6ca/gink-0.20240407.1712460796.tar.gz",
"platform": null,
"description": "# Gink in General\n\nGink aims to be a \"protocol first\" database system defined by the protocol for syncronizing\ninstances, rather than by a specific implementation. Defining the database in terms of\nthe interchange format allows for independent implementations to interact seamlessly in\na well-defined manner.\n\n## Take a look at the full docs [here](https://gink.readthedocs.io/en/latest/).\n\nI created the Python implementation of Gink to be a testbed for new ideas and\nto provide the simplest expression of all the concepts in Gink. Well written python\ncode can essentially serve as executable psudocode. Code written for this implementation\nhas been biased in favor of readability and extensibility, rather than raw performance.\nFor example, the code doesn't use async functions or multi-threading.\n\n* [Installation](#installation)\n * [Examples](#examples)\n * [Data Structures](#data-structures)\n * [Box](#box)\n * [Directory](#directory)\n * [Sequence](#sequence)\n * [All Containers](#all-containers)\n * [Database Operations](#database-operations)\n\n## Installation\nAssuming you have Python and Pip installed:\n```sh\npip3 install gink\n```\n\n## Examples\nThis page does not include examples for all data structures. Take a look at our [Python documentation](https://gink.readthedocs.io/en/latest/) for all examples and full docs.\n\n### Data Structures\n\nThere are some operations that are available to all Containers, which are shown at the [end of this page](#all-containers).\\\n\\\nFor all operations, a store and a database are needed:\n```py\nfrom gink import *\n\nstore = LmdbStore('example.db')\ndatabase = Database(store=store)\n```\n\n#### Box\nA Box is the simplest data structure available on Gink. It can hold only one value at a time; you can set its value, or get its value.\n```py\nbox = Box(database=database)\n\nbox.set({\"foo\": \"bar\", \"key2\": 15})\nresult = box.get() # Returns the python dictionary just added\n\nif not box.isEmpty():\n print(box.size()) # This will only return 0 or 1 (1 in this case).\n```\n\n#### Directory\nThe Directory aims to mimic the functionality of a Python dictionary. If you know how to use a dictionary, you should already know how to use the directory!\\\n\\\nCreate a new directory:\n```python\ndirectory = Directory(database=database)\n```\nSet key: value pairs:\n```py\ndirectory[\"key1\"] = \"value1\"\n\n# Saves a timestamp after \"key1\" and before \"key2\"\ntime = database.get_now() # more on this in All Containers examples\n\n# Achieves the same thing as the previous set, just different syntax.\ndirectory.set(\"key2\", {\"test\": \"document\"})\n```\nGetting the value of a key:\n```py\nresult = directory[\"key1\"] # Returns \"value1\"\nresult2 = directory.get(\"key2\") # Returns {\"test\": \"document\"}\nresult3 = directory.get(\"key3\") # Returns None\n```\nGet all keys and items:\n```py\n# Returns an generator of [\"key1\", \"key2\"]\n# Note: the order may not be the same.\nkeys = directory.keys()\n\n# Returns the items as a generator of (key, value tuples) in the directory\n# at the specified timestamp - in this case, [(\"key1\", \"value1\")]\nitems = directory.items(as_of=time)\n\n# returns a list of all values\nvalues = directory.values()\n```\nDeleting keys:\n```py\n# Returns \"value1\" and removes the key: value pair from the directory.\nvalue = directory.pop(\"key1\")\n\n# delete the key and return the Muid of this change\ndel_muid = directory.delete(\"key2\")\n```\nSetting multiple keys and values at the same time:\n```py\ndirectory.update({\"newkey\": \"newvalue\", \"another key\": \"another value\"})\n```\n\n#### Sequence\nThe Sequence is equivalent to a Python list. Again, these operations should look pretty familiar! In a Gink Sequence, the contents are ordered by timestamps.\\\n\\\nCreate a Sequence and append some values:\n```python\nsequence = Sequence()\n\nsequence.append(\"Hello, World!\")\nsequence.append(42)\nsequence.append(\"a\")\nsequence.append(\"b\")\n```\nSearch for value and return index if found:\n```py\nfound = sequence.index(\"Hello, World!\")\n# Returns 0\n```\n\nPop values:\n```py\npopped = sequence.pop(1)\n# Returns 42\n\n# Pops and returns the value at index 0, which is \"Hello, World!\"\n# The destination argument allows you to place the item\n# back into the sequence at a different timestamp\n# in this case, -1 would indicate the timestamp of the last change.\n# So, this sequence is now ordered as [\"a\", \"b\", \"Hello, World!\"]\npopped = sequence.pop(0, dest=-1)\n\n```\nInsert to specific index:\n```py\n# Inserts \"x\" at index 1, between \"a\" and \"b\", in this example.\n# Comment is an optional parameter that will be included in\n# bundle for this change (most operations may contain comments).\nsequence.insert(1, \"x\", comment=\"insert x\")\n```\n\nReturn the sequence as a Python list:\n```py\nas_list = list(sequence)\n```\n#### All Containers\nThe Container is the parent class for all Gink data structures. Here are some examples of the powerful operations you can do with any container:\n##### From Contents\nTo make it easier to insert data into an object upon initialization, Gink allows you to specify a `contents` argument to the constructor of the object. Different data structures may take different types as contents, but the idea remains the same for all Gink objects.\n```python\ndirectory = Directory(database=database, contents={\n \"key1\": \"value1\", \"key2\": 42, \"key3\": [1, 2, 3, 4]})\n\nkey_set = KeySet(database=database, contents=[\"key1\", \"key2\", 3])\n\n# Vertex creation for pair map population\nvertex1 = Vertex()\nvertex2 = Vertex()\n\n# Pair Map contents only takes a dictionary. Read the docs for the\n# accepted data types for other data structures.\npair_map = PairMap(contents={(noun1, noun2): \"value\"})\n```\n##### Back in time\nYou will frequently see `as_of` in the Gink documentation. `as_of` refers to the time to look back to. There are multiple ways of interacting with `as_of`. If you are curious about how certain timestamps are resolved, take a look at `Database.resolve_timestamp()`\\\nOne easy way is to pass a negative integer indicating how many changes back you want to look.\n```python\nbox = Box(contents=\"first_value\")\nbox.set(\"second_value\")\n\n# Passing -1 into the as_of argument looks back at the previous value\n# Returns \"first_value\"\nprevious = box.get(as_of=-1)\n```\nAnother common way to use timestamps is to \"save\" a time between changes as a variable.\n```python\nbox = Box(contents=\"first_value\")\ntime_after_first = database.get_now()\nbox.set(\"second_value\")\n\n# Passing saved timestamp into as_of\n# Returns \"first_value\"\nprevious = box.get(as_of=time_after_first)\n```\n\n##### Reset\nResetting a container is a fundamental operation used to revert the container back to a previous time. Above we looked at using timestamps to get previous values, but resetting to specific times may prove more useful. This example uses a directory, but this functionality works the same for all containers.\n```python\ndirectory = Directory()\n\ndirectory[\"foo\"] = \"bar\"\ndirectory[\"bar\"] = \"foo\"\ntime_between = database.get_now()\ndirectory[7] = {\"user\": 1003203, \"email\": \"test@test.com\"}\n\nhas_7 = 7 in directory # returns True\ndirectory.reset(to_time=time_between)\nhas_7 = 7 in directory # now returns False\nhas_bar = \"bar\" in directory # still returns True\n```\n\n#### Clearing\nClearing a container does exactly what you would expect it to do. The `Container.clear()` method removes all entries from the container and returns the Muid of the clearance. The clearance is processed as a new database change, which means you can still look back at previous timestamps or reset the database back to before the clearance occurred.\n```python\ndirectory = Directory()\n\ndirectory[\"foo\"] = \"bar\"\ndirectory[\"bar\"] = \"foo\"\ndirectory[7] = {\"user\": 1003203, \"email\": \"test@test.com\"}\n# Storing the muid of the clearance to use later\nclearance_muid = directory.clear()\n\n# Directory is now empty\nhas_foo = \"foo\" in directory # Returns False\nhas_bar = \"bar\" in directory # Returns False\nhas_7 = \"7\" in directory # Returns False\n\n# Using the muid's timestamp to look back before the clearance\n# Returns \"bar\"\nprevious = directory.get(\"foo\", as_of=clearance_muid.timestamp)\n\n```\n\n### Database Operations\n\n#### Bundling and comments\nThink of a bundle as a commit in Git. A bundle is just a collection of changes with an optional comment/message. Without specifying a bundler object, most Gink operations will immediately commit the change in its own bundle, so you don't have to worry about always creating a new bundler, etc. However, if you do want to specify which changes go into a specific bundle, here is an example:\n```python\ndirectory = Directory()\nbundler = Bundler(comment=\"example setting values in directory\")\n\ndirectory.set(\"key1\", 1, bundler=bundler)\ndirectory.set(\"key2\", \"value2\", bundler=bundler)\ndirectory.update({\"key3\": 3, \"key4\": 4}, bundler=bundler)\n\n# This seals the bundler and commits changes to database\n# at this point, no more changes may be added\ndatabase.commit(bundler)\n```\n\n#### Reset\nSimilar to how `Container.reset()` works, the Database class has its own reset functionality that will reset all containers to the specified time. A \"reset\" is simply one large bundle of changes that updates the database entries to what they were are the previous timestamp; this allows you to easily look back before the reset.\n\n```python\ndatabase = Database(store=store)\nroot = Directory.get_global_instance(database=database)\nqueue = Sequence.get_global_instance(database=database)\nmisc = Directory()\n\nmisc[\"yes\"] = False\nroot[\"foo\"] = \"bar\"\nqueue.append(\"value1\")\n\n# No as_of argument defaults to EPOCH\n# which is the time of database creation (empty)\ndatabase.reset()\n\n# All containers will have a length of 0\n# since the database is now empty.\nsize = len(root)\n\n# to_time=-1 reverts the database to the\n# previous change\ndatabase.reset(to_time=-1)\n\n# This will now have a len of 1,\n# and one element of \"value1\"\nsize = len(queue)\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "a system for storing data structures in lmdb",
"version": "0.20240407.1712460796",
"project_urls": {
"Homepage": "https://github.com/x5e/gink"
},
"split_keywords": [
"gink",
"lmdb",
"crdt",
"history",
"versioned"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b91cc7fa32f0f3ebdf7c55559144cfe791bad835280ea3728f7507e9235fa39b",
"md5": "9cda068ebe754d0b7f3b32b59b48dd32",
"sha256": "e4efacb7c2c9973a40408dfb9a9a9299f41b8238dc2132d4cbe0d9bda0c617c4"
},
"downloads": -1,
"filename": "gink-0.20240407.1712460796-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9cda068ebe754d0b7f3b32b59b48dd32",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4,>=3.8",
"size": 130079,
"upload_time": "2024-04-07T03:33:56",
"upload_time_iso_8601": "2024-04-07T03:33:56.359610Z",
"url": "https://files.pythonhosted.org/packages/b9/1c/c7fa32f0f3ebdf7c55559144cfe791bad835280ea3728f7507e9235fa39b/gink-0.20240407.1712460796-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1ff214502257f1d345c3e863d6799e7ab416bd5e69e65d919acb58fedf29a6ca",
"md5": "6be6bff8d8b80147910a4616d517a4b9",
"sha256": "499e6b8e332906509046329559db2e34c5bc1c349764d4e3cd9f3c636f42e911"
},
"downloads": -1,
"filename": "gink-0.20240407.1712460796.tar.gz",
"has_sig": false,
"md5_digest": "6be6bff8d8b80147910a4616d517a4b9",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4,>=3.8",
"size": 97183,
"upload_time": "2024-04-07T03:33:58",
"upload_time_iso_8601": "2024-04-07T03:33:58.767246Z",
"url": "https://files.pythonhosted.org/packages/1f/f2/14502257f1d345c3e863d6799e7ab416bd5e69e65d919acb58fedf29a6ca/gink-0.20240407.1712460796.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-07 03:33:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "x5e",
"github_project": "gink",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "gink"
}
JSON
