<!--
* @Author : xupingmao
* @email : 578749341@qq.com
* @Date : 2023-03-11 13:22:54
* @LastEditors : xupingmao
* @LastEditTime : 2023-03-11 13:42:41
* @FilePath : /sqlite-rw/README.md
* @Description : 描述
-->
# sqlite-rw
为sqlite提供读写分离能力，提高并发度。
# 解决什么问题?
sqlite是一个很好用的嵌入式数据库，但是由于sqlite采用了库级别的文件锁，所以并发能力不强，当存在慢SQL，比如在大表上`select count(*) from large_table`的时候，会导致其他任务无法写入。
`sqlite-rw`就是为了解决这个问题的，思路如下
- 采用读写分离的存储模式，每次写入会持久化写库，并且记录日志到binlog表，然后异步写入读库。
- 读取的时候可以选择从读库或者写库读取，默认从读库读取。
# 提高并发度的原理
- sqlite本身读取是支持并发的，但是写入是不能和读取并发的。所以读库本身是支持并发读取的，同步的写操作是通过id主键写入的，耗时会比较短，基本上不会阻塞读操作。
- 由于是读写分离的，在读库存在慢SQL的情况下，写库也能够正常工作，在读库慢SQL执行完成之后，系统会把写库的数据同步到读库中，使两者保持一致。
# 使用限制和影响
- 存储量相比于单独的sqlite会翻倍
- 写入性能会有所下降，待详细的基准测试
- 依赖`web.py`的`db`接口进行操作
Raw data
{
"_id": null,
"home_page": "https://github.com/xupingmao/sqlite-rw",
"name": "sqlite-rw",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "mark",
"author_email": "578749341@qq.com",
"download_url": "https://files.pythonhosted.org/packages/61/6b/494e34f7f3e554ebecce7e625cb0af4521ae8b3d7ad97ea2d7cc1ba55727/sqlite-rw-0.0.2.tar.gz",
"platform": null,
"description": "<!--\n * @Author : xupingmao\n * @email : 578749341@qq.com\n * @Date : 2023-03-11 13:22:54\n * @LastEditors : xupingmao\n * @LastEditTime : 2023-03-11 13:42:41\n * @FilePath : /sqlite-rw/README.md\n * @Description : \u63cf\u8ff0\n-->\n# sqlite-rw\n\n\u4e3asqlite\u63d0\u4f9b\u8bfb\u5199\u5206\u79bb\u80fd\u529b\uff0c\u63d0\u9ad8\u5e76\u53d1\u5ea6\u3002\n\n# \u89e3\u51b3\u4ec0\u4e48\u95ee\u9898?\n\nsqlite\u662f\u4e00\u4e2a\u5f88\u597d\u7528\u7684\u5d4c\u5165\u5f0f\u6570\u636e\u5e93\uff0c\u4f46\u662f\u7531\u4e8esqlite\u91c7\u7528\u4e86\u5e93\u7ea7\u522b\u7684\u6587\u4ef6\u9501\uff0c\u6240\u4ee5\u5e76\u53d1\u80fd\u529b\u4e0d\u5f3a\uff0c\u5f53\u5b58\u5728\u6162SQL\uff0c\u6bd4\u5982\u5728\u5927\u8868\u4e0a`select count(*) from large_table`\u7684\u65f6\u5019\uff0c\u4f1a\u5bfc\u81f4\u5176\u4ed6\u4efb\u52a1\u65e0\u6cd5\u5199\u5165\u3002\n\n`sqlite-rw`\u5c31\u662f\u4e3a\u4e86\u89e3\u51b3\u8fd9\u4e2a\u95ee\u9898\u7684\uff0c\u601d\u8def\u5982\u4e0b\n- \u91c7\u7528\u8bfb\u5199\u5206\u79bb\u7684\u5b58\u50a8\u6a21\u5f0f\uff0c\u6bcf\u6b21\u5199\u5165\u4f1a\u6301\u4e45\u5316\u5199\u5e93\uff0c\u5e76\u4e14\u8bb0\u5f55\u65e5\u5fd7\u5230binlog\u8868\uff0c\u7136\u540e\u5f02\u6b65\u5199\u5165\u8bfb\u5e93\u3002\n- \u8bfb\u53d6\u7684\u65f6\u5019\u53ef\u4ee5\u9009\u62e9\u4ece\u8bfb\u5e93\u6216\u8005\u5199\u5e93\u8bfb\u53d6\uff0c\u9ed8\u8ba4\u4ece\u8bfb\u5e93\u8bfb\u53d6\u3002\n\n# \u63d0\u9ad8\u5e76\u53d1\u5ea6\u7684\u539f\u7406\n- sqlite\u672c\u8eab\u8bfb\u53d6\u662f\u652f\u6301\u5e76\u53d1\u7684\uff0c\u4f46\u662f\u5199\u5165\u662f\u4e0d\u80fd\u548c\u8bfb\u53d6\u5e76\u53d1\u7684\u3002\u6240\u4ee5\u8bfb\u5e93\u672c\u8eab\u662f\u652f\u6301\u5e76\u53d1\u8bfb\u53d6\u7684\uff0c\u540c\u6b65\u7684\u5199\u64cd\u4f5c\u662f\u901a\u8fc7id\u4e3b\u952e\u5199\u5165\u7684\uff0c\u8017\u65f6\u4f1a\u6bd4\u8f83\u77ed\uff0c\u57fa\u672c\u4e0a\u4e0d\u4f1a\u963b\u585e\u8bfb\u64cd\u4f5c\u3002\n- \u7531\u4e8e\u662f\u8bfb\u5199\u5206\u79bb\u7684\uff0c\u5728\u8bfb\u5e93\u5b58\u5728\u6162SQL\u7684\u60c5\u51b5\u4e0b\uff0c\u5199\u5e93\u4e5f\u80fd\u591f\u6b63\u5e38\u5de5\u4f5c\uff0c\u5728\u8bfb\u5e93\u6162SQL\u6267\u884c\u5b8c\u6210\u4e4b\u540e\uff0c\u7cfb\u7edf\u4f1a\u628a\u5199\u5e93\u7684\u6570\u636e\u540c\u6b65\u5230\u8bfb\u5e93\u4e2d\uff0c\u4f7f\u4e24\u8005\u4fdd\u6301\u4e00\u81f4\u3002\n\n# \u4f7f\u7528\u9650\u5236\u548c\u5f71\u54cd\n- \u5b58\u50a8\u91cf\u76f8\u6bd4\u4e8e\u5355\u72ec\u7684sqlite\u4f1a\u7ffb\u500d\n- \u5199\u5165\u6027\u80fd\u4f1a\u6709\u6240\u4e0b\u964d\uff0c\u5f85\u8be6\u7ec6\u7684\u57fa\u51c6\u6d4b\u8bd5\n- \u4f9d\u8d56`web.py`\u7684`db`\u63a5\u53e3\u8fdb\u884c\u64cd\u4f5c\n",
"bugtrack_url": null,
"license": "",
"summary": "sqlite\u8bfb\u5199\u5206\u79bb\u5e93",
"version": "0.0.2",
"project_urls": {
"Homepage": "https://github.com/xupingmao/sqlite-rw"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cf5738e422d1c542985957f353d06889d9b867f025b54de51b2dcaea0f4116f1",
"md5": "a20dd8fbe120b5b8dbcbd0f0b4e0e609",
"sha256": "98fe5db5f65c6dcaad0df305820b9b4d7e11923cbe4611b0de53b7648970ff25"
},
"downloads": -1,
"filename": "sqlite_rw-0.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a20dd8fbe120b5b8dbcbd0f0b4e0e609",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 7714,
"upload_time": "2023-06-04T16:20:05",
"upload_time_iso_8601": "2023-06-04T16:20:05.332751Z",
"url": "https://files.pythonhosted.org/packages/cf/57/38e422d1c542985957f353d06889d9b867f025b54de51b2dcaea0f4116f1/sqlite_rw-0.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "616b494e34f7f3e554ebecce7e625cb0af4521ae8b3d7ad97ea2d7cc1ba55727",
"md5": "048f186ad28fe2bdbb51e9a1e408f209",
"sha256": "7fb4fae1380e3f3734be2fb5c411ffaee0a0841746c4a78e6dbc64a0853494aa"
},
"downloads": -1,
"filename": "sqlite-rw-0.0.2.tar.gz",
"has_sig": false,
"md5_digest": "048f186ad28fe2bdbb51e9a1e408f209",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 7486,
"upload_time": "2023-06-04T16:21:09",
"upload_time_iso_8601": "2023-06-04T16:21:09.515525Z",
"url": "https://files.pythonhosted.org/packages/61/6b/494e34f7f3e554ebecce7e625cb0af4521ae8b3d7ad97ea2d7cc1ba55727/sqlite-rw-0.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-06-04 16:21:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "xupingmao",
"github_project": "sqlite-rw",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "sqlite-rw"
}
JSON
