# S3NoSQLDB
A NoSQL Database Implementation using Amazon S3 for storage.
## Features
- Create and manage collections in S3 buckets
- Batch document insertion with automatic ID generation
- Concurrent processing using ThreadPoolExecutor
- Query documents using custom filter functions
- Parquet file format with gzip compression
## Installation
```bash
pip install s3nosqldb
```
## Quick Start
```python
from s3nosqldb import S3NoSQLDB
# Initialize the database
db = S3NoSQLDB(
bucket_name="your-bucket-name",
base_path="optional/path",
batch_size=50000,
max_workers=4
)
# Create a collection
db.create_collection("users", id_field="user_id", auto_generate_id=True)
# Insert documents
users = [
{"name": "John Doe", "age": 30},
{"name": "Jane Smith", "age": 25}
]
db.insert_batch("users", users)
# Query documents
def age_filter(doc):
return doc["age"] > 25
results = db.query("users", filter_fn=age_filter)
```
## Development
1. Clone the repository:
```bash
git clone https://github.com/wqxxzd/s3nosqldb.git
cd s3nosqldb
```
2. Create a virtual environment and install dependencies:
```bash
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -e .[dev]
```
3. Run tests:
```bash
pytest
```
## License
This project is licensed under the MIT License.
## Badges
[](https://codecov.io/github/wqxxzd/S3NoSQLDB)
Raw data
{
"_id": null,
"home_page": "https://github.com/wqxxzd/s3nosqldb",
"name": "s3nosqldb",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Dan Zhang",
"author_email": "dan.zhang0224@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/ec/c1/ca3f159ac321fa429e517e1dc0398648fae1b59b96ec6187af81c2166be5/s3nosqldb-0.1.1.tar.gz",
"platform": null,
"description": "# S3NoSQLDB\n\nA NoSQL Database Implementation using Amazon S3 for storage.\n\n## Features\n\n- Create and manage collections in S3 buckets\n- Batch document insertion with automatic ID generation\n- Concurrent processing using ThreadPoolExecutor\n- Query documents using custom filter functions\n- Parquet file format with gzip compression\n\n## Installation\n\n```bash\npip install s3nosqldb\n```\n\n## Quick Start\n\n```python\nfrom s3nosqldb import S3NoSQLDB\n\n# Initialize the database\ndb = S3NoSQLDB(\n bucket_name=\"your-bucket-name\",\n base_path=\"optional/path\",\n batch_size=50000,\n max_workers=4\n)\n\n# Create a collection\ndb.create_collection(\"users\", id_field=\"user_id\", auto_generate_id=True)\n\n# Insert documents\nusers = [\n {\"name\": \"John Doe\", \"age\": 30},\n {\"name\": \"Jane Smith\", \"age\": 25}\n]\ndb.insert_batch(\"users\", users)\n\n# Query documents\ndef age_filter(doc):\n return doc[\"age\"] > 25\n\nresults = db.query(\"users\", filter_fn=age_filter)\n```\n\n## Development\n\n1. Clone the repository:\n```bash\ngit clone https://github.com/wqxxzd/s3nosqldb.git\ncd s3nosqldb\n```\n\n2. Create a virtual environment and install dependencies:\n```bash\npython -m venv venv\nsource venv/bin/activate # On Windows: venv\\Scripts\\activate\npip install -e .[dev]\n```\n\n3. Run tests:\n```bash\npytest\n```\n\n## License\n\nThis project is licensed under the MIT License.\n\n## Badges\n\n[](https://codecov.io/github/wqxxzd/S3NoSQLDB)\n",
"bugtrack_url": null,
"license": null,
"summary": "A NoSQL Database Implementation using Amazon S3",
"version": "0.1.1",
"project_urls": {
"Homepage": "https://github.com/wqxxzd/s3nosqldb"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "67b131c4bd7630471ccc8e55548dbb6875ff4f50727317b546c22f5ee81b163a",
"md5": "746008f78badbf8ddf70e589594cf653",
"sha256": "d089dd61bcc17694d9efcf7243827204574efb5c2f823ee7e4457a73e609c909"
},
"downloads": -1,
"filename": "s3nosqldb-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "746008f78badbf8ddf70e589594cf653",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 6142,
"upload_time": "2024-11-01T15:52:14",
"upload_time_iso_8601": "2024-11-01T15:52:14.039659Z",
"url": "https://files.pythonhosted.org/packages/67/b1/31c4bd7630471ccc8e55548dbb6875ff4f50727317b546c22f5ee81b163a/s3nosqldb-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ecc1ca3f159ac321fa429e517e1dc0398648fae1b59b96ec6187af81c2166be5",
"md5": "a758dba631a41f4f699da3142d6c9d3f",
"sha256": "2ef322e5f4eb60ba6b304a87974032285f478f168a932542955d9b232ae348cf"
},
"downloads": -1,
"filename": "s3nosqldb-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "a758dba631a41f4f699da3142d6c9d3f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 6328,
"upload_time": "2024-11-01T15:52:16",
"upload_time_iso_8601": "2024-11-01T15:52:16.127676Z",
"url": "https://files.pythonhosted.org/packages/ec/c1/ca3f159ac321fa429e517e1dc0398648fae1b59b96ec6187af81c2166be5/s3nosqldb-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-01 15:52:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "wqxxzd",
"github_project": "s3nosqldb",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "s3nosqldb"
}
JSON
