# raftify-py
⚠️ WARNING: This library is in a very experimental stage. The API could be broken.
Python binding of [*raftify*](https://github.com/lablup/raftify).
## Quick guide
I strongly recommend to read the basic [memstore example code](https://github.com/lablup/raftify/blob/main/binding/python/examples/main.py) to get how to use this library for starters, but here's a quick guide.
### Define your own log entry
Define the data to be stored in LogEntry and how to serialize and de-serialize it.
```py
class SetCommand:
def __init__(self, key: str, value: str) -> None:
self.key = key
self.value = value
def encode(self) -> bytes:
return pickle.dumps(self.__dict__)
@classmethod
def decode(cls, packed: bytes) -> "SetCommand":
unpacked = pickle.loads(packed)
return cls(unpacked["key"], unpacked["value"])
```
### Define your application Raft FSM
Essentially, the following three methods need to be implemented for the `Store`.
- `apply`: applies a committed entry to the store.
- `snapshot`: returns snapshot data for the store.
- `restore`: applies the snapshot passed as argument.
And also similarly to `LogEntry`, you need to implement `encode` and `decode`.
```py
class HashStore:
def __init__(self):
self._store = dict()
def get(self, key: str) -> Optional[str]:
return self._store.get(key)
def apply(self, msg: bytes) -> bytes:
message = SetCommand.decode(msg)
self._store[message.key] = message.value
logging.info(f'SetCommand inserted: ({message.key}, "{message.value}")')
return msg
def snapshot(self) -> bytes:
return pickle.dumps(self._store)
def restore(self, snapshot: bytes) -> None:
self._store = pickle.loads(snapshot)
```
### Bootstrap a raft cluster
First, bootstrap the cluster that contains the leader node.
```py
logger = Slogger.default()
logger.info("Bootstrap new Raft Cluster")
node_id = 1
raft_addr = "127.0.0.1:60061"
raft = Raft.bootstrap(node_id, raft_addr, store, cfg, logger)
await raft.run()
```
### Join follower nodes to the cluster
Then join the follower nodes.
If peer specifies the configuration of the initial members, the cluster will operate after all member nodes are bootstrapped.
```py
raft_addr = "127.0.0.1:60062"
peer_addr = "127.0.0.1:60061"
join_ticket = await Raft.request_id(raft_addr, peer_addr)
node_id = join_ticket.get_reserved_id()
raft = Raft.bootstrap(node_id, raft_addr, store, cfg, logger)
tasks = []
tasks.append(raft.run())
await raft.join([join_ticket])
```
### Manipulate FSM by RaftServiceClient
If you want to operate the FSM remotely, use the `RaftServiceClient`.
```py
client = await RaftServiceClient.build("127.0.0.1:60061")
await client.propose(SetCommand("1", "A").encode())
```
### Manipulate FSM by RaftNode
If you want to operate FSM locally, use the RaftNode interface of the Raft object
```py
raft_node = raft.get_raft_node()
await raft_node.propose(message.encode())
```
### Debugging
Raftify also provides a collection of CLI commands that let you check the data persisted in stable storage and the status of Raft Server.
```
$ raftify_cli debug persisted ./logs/node-1
```
```
$ raftify_cli debug node 127.0.0.1:60061
```
Raw data
{
"_id": null,
"home_page": "https://github.com/lablup/raftify",
"name": "raftify",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "raft, distributed-systems, consensus-algorithm, replication, distributed-database",
"author": "Lablup Inc.",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/9a/f5/c8e3c5a050229fbf33787b62bc6736e6f7b8f503d516a7588daf7375483c/raftify-0.1.67.tar.gz",
"platform": null,
"description": "# raftify-py\n\n\u26a0\ufe0f WARNING: This library is in a very experimental stage. The API could be broken.\n\nPython binding of [*raftify*](https://github.com/lablup/raftify).\n\n## Quick guide\n\nI strongly recommend to read the basic [memstore example code](https://github.com/lablup/raftify/blob/main/binding/python/examples/main.py) to get how to use this library for starters, but here's a quick guide.\n\n### Define your own log entry\n\nDefine the data to be stored in LogEntry and how to serialize and de-serialize it.\n\n```py\nclass SetCommand:\n def __init__(self, key: str, value: str) -> None:\n self.key = key\n self.value = value\n\n def encode(self) -> bytes:\n return pickle.dumps(self.__dict__)\n\n @classmethod\n def decode(cls, packed: bytes) -> \"SetCommand\":\n unpacked = pickle.loads(packed)\n return cls(unpacked[\"key\"], unpacked[\"value\"])\n```\n\n### Define your application Raft FSM\n\nEssentially, the following three methods need to be implemented for the `Store`.\n\n- `apply`: applies a committed entry to the store.\n- `snapshot`: returns snapshot data for the store.\n- `restore`: applies the snapshot passed as argument.\n\nAnd also similarly to `LogEntry`, you need to implement `encode` and `decode`.\n\n```py\nclass HashStore:\n def __init__(self):\n self._store = dict()\n\n def get(self, key: str) -> Optional[str]:\n return self._store.get(key)\n\n def apply(self, msg: bytes) -> bytes:\n message = SetCommand.decode(msg)\n self._store[message.key] = message.value\n logging.info(f'SetCommand inserted: ({message.key}, \"{message.value}\")')\n return msg\n\n def snapshot(self) -> bytes:\n return pickle.dumps(self._store)\n\n def restore(self, snapshot: bytes) -> None:\n self._store = pickle.loads(snapshot)\n```\n\n### Bootstrap a raft cluster\n\nFirst, bootstrap the cluster that contains the leader node.\n\n```py\nlogger = Slogger.default()\nlogger.info(\"Bootstrap new Raft Cluster\")\n\nnode_id = 1\nraft_addr = \"127.0.0.1:60061\"\nraft = Raft.bootstrap(node_id, raft_addr, store, cfg, logger)\nawait raft.run()\n```\n\n### Join follower nodes to the cluster\n\nThen join the follower nodes.\n\nIf peer specifies the configuration of the initial members, the cluster will operate after all member nodes are bootstrapped.\n\n```py\nraft_addr = \"127.0.0.1:60062\"\npeer_addr = \"127.0.0.1:60061\"\n\njoin_ticket = await Raft.request_id(raft_addr, peer_addr)\nnode_id = join_ticket.get_reserved_id()\n\nraft = Raft.bootstrap(node_id, raft_addr, store, cfg, logger)\ntasks = []\ntasks.append(raft.run())\nawait raft.join([join_ticket])\n```\n\n### Manipulate FSM by RaftServiceClient\n\nIf you want to operate the FSM remotely, use the `RaftServiceClient`.\n\n```py\nclient = await RaftServiceClient.build(\"127.0.0.1:60061\")\nawait client.propose(SetCommand(\"1\", \"A\").encode())\n```\n\n### Manipulate FSM by RaftNode\n\nIf you want to operate FSM locally, use the RaftNode interface of the Raft object\n\n```py\nraft_node = raft.get_raft_node()\nawait raft_node.propose(message.encode())\n```\n\n### Debugging\n\nRaftify also provides a collection of CLI commands that let you check the data persisted in stable storage and the status of Raft Server.\n\n```\n$ raftify_cli debug persisted ./logs/node-1\n```\n\n```\n$ raftify_cli debug node 127.0.0.1:60061\n```\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Experimental High level Raft framework",
"version": "0.1.67",
"project_urls": {
"Homepage": "https://github.com/lablup/raftify",
"Source Code": "https://github.com/lablup/raftify"
},
"split_keywords": [
"raft",
" distributed-systems",
" consensus-algorithm",
" replication",
" distributed-database"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "7200a8d51149e180df3c0b816137aee4da4a7d07b095fb880ea868364b6b8e5c",
"md5": "2f3dcd622c2c9ef671ae633e1868e783",
"sha256": "583592ab9650bc3041a8532d0e803e6588a19a8032ec9f590469cd01990c3bd6"
},
"downloads": -1,
"filename": "raftify-0.1.67-cp312-cp312-macosx_11_0_arm64.whl",
"has_sig": false,
"md5_digest": "2f3dcd622c2c9ef671ae633e1868e783",
"packagetype": "bdist_wheel",
"python_version": "cp312",
"requires_python": ">=3.10",
"size": 3153135,
"upload_time": "2024-08-04T03:24:39",
"upload_time_iso_8601": "2024-08-04T03:24:39.421796Z",
"url": "https://files.pythonhosted.org/packages/72/00/a8d51149e180df3c0b816137aee4da4a7d07b095fb880ea868364b6b8e5c/raftify-0.1.67-cp312-cp312-macosx_11_0_arm64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "9af5c8e3c5a050229fbf33787b62bc6736e6f7b8f503d516a7588daf7375483c",
"md5": "873e89f3dbb3e30d3109ee8d3085bd3e",
"sha256": "6be8df3f3b53b1189b19e597781dceb5285a30a8d54d3441fef8dcc7a1fd68ea"
},
"downloads": -1,
"filename": "raftify-0.1.67.tar.gz",
"has_sig": false,
"md5_digest": "873e89f3dbb3e30d3109ee8d3085bd3e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 48501,
"upload_time": "2024-08-04T03:24:42",
"upload_time_iso_8601": "2024-08-04T03:24:42.511484Z",
"url": "https://files.pythonhosted.org/packages/9a/f5/c8e3c5a050229fbf33787b62bc6736e6f7b8f503d516a7588daf7375483c/raftify-0.1.67.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-04 03:24:42",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "lablup",
"github_project": "raftify",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "raftify"
}
JSON
