Name | memory-fs JSON |
Version |
0.13.0
JSON |
| download |
home_page | None |
Summary | Memory-FS |
upload_time | 2025-07-08 10:54:10 |
maintainer | None |
docs_url | None |
author | Dinis Cruz |
requires_python | <4.0,>=3.12 |
license | Apache 2.0 |
keywords |
|
VCS |
 |
bugtrack_url |
|
requirements |
No requirements were recorded.
|
Travis-CI |
No Travis.
|
coveralls test coverage |
No coveralls.
|
# Memory-FS
Memory-FS is a fully tested, type-safe file system abstraction implemented entirely in memory. It originated from the `OSBot_Cloud_FS` prototype and has evolved through multiple architectural iterations documented in [docs/tech_debriefs](docs/tech_debriefs). The library exposes a small, composable API for creating, loading and editing file objects while remaining agnostic of the underlying storage provider.
## Key Features
- **In-Memory Storage** – extremely fast read/write operations via an in‑memory provider.
- **Type Safety** – all APIs use `osbot_utils` `Type_Safe` models ensuring runtime validation.
- **Two-File Pattern** – metadata (`.config` / `.metadata`) and content are stored separately.
- **Pluggable Path Strategies** – configurable path handlers (latest, temporal, versioned, etc.).
- **Extensible File Types** – JSON, Markdown, binary and custom formats through schema classes.
- **100% Code Coverage** – every line of production code is tested as of version `v0.11.1`.
## Architecture Overview
The current design (introduced in the [June 18 2025 debrief](docs/tech_debriefs/on-18-jun-2025.md)) is composed of two layers:
1. **File_FS Layer** – high level file operations
- `File_FS` object plus `File_FS__*` action classes
- `Target_FS` factory for creating file objects from existing paths
2. **Storage_FS Layer** – low level storage providers
- `Storage_FS` interface and in‑memory implementation `Storage_FS__Memory`
- Provider pattern enabling future backends (local disk, SQLite, zip, S3)
This separation yields a small surface area for extending or replacing storage while keeping file logic consistent. The project’s [tech debrief README](docs/tech_debriefs/README.md) describes this architecture as:
```
1. **File_FS Layer**: High-level file operations
2. **Storage_FS Layer**: Low-level storage operations
- Provider pattern for pluggable storage backends
```
### File Naming
Files are organised using a predictable naming scheme:
1. **Content File** – `{file_id}.{extension}`
2. **Config File** – `{file_id}.{extension}.config`
3. **Metadata File** – `{file_id}.{extension}.metadata`
### Coverage Guarantee
Memory-FS maintains comprehensive tests. As recorded in the debriefs:
```
As of June 18, 2025 (v0.11.0), Memory-FS has achieved **100% code coverage**:
- Every line of production code is tested
- All edge cases are covered
- No dead or unused code remains
- Comprehensive test suite with 200+ test methods
```
## Installation
```bash
pip install memory-fs
```
or install from source:
```bash
pip install -e .
```
## Quick Example
Below is a minimal example using the high‑level API. It stores JSON content via the in‑memory provider:
```python
from memory_fs.file_fs.File_FS import File_FS
from memory_fs.file_types.Memory_FS__File__Type__Json import Memory_FS__File__Type__Json
from memory_fs.schemas.Schema__Memory_FS__File__Config import Schema__Memory_FS__File__Config
from memory_fs.path_handlers.Path__Handler__Latest import Path__Handler__Latest
from memory_fs.storage.Memory_FS__Storage import Memory_FS__Storage
from memory_fs.storage_fs.providers.Storage_FS__Memory import Storage_FS__Memory
# set up in-memory storage
storage = Memory_FS__Storage(storage_fs=Storage_FS__Memory())
# configuration describing where/how the file should be stored
config = Schema__Memory_FS__File__Config(
path_handlers = [Path__Handler__Latest()],
default_handler=Path__Handler__Latest,
file_type = Memory_FS__File__Type__Json(),
)
# create a file object and save some data
file = File_FS(file_config=config, storage=storage)
file.create()
file.create__content(b'{"hello": "world"}')
# reload using the factory
from memory_fs.target_fs.Target_FS__Create import Target_FS__Create
loaded_target = Target_FS__Create(storage=storage).from_path__config(file.file_fs__paths().paths__config()[0])
print(loaded_target.file_fs().content()) # -> b'{"hello": "world"}'
```
## Documentation
Extensive documentation is kept under the `docs` folder. The [tech_debriefs](docs/tech_debriefs) directory captures the project evolution, while [architecture](docs/architecture/technical_architecture_debrief.md) provides a detailed design overview. Developers should also review [docs/dev](docs/dev/README.md) for coding conventions, bug lists and TODO items.
## Project History
The debrief timeline in `docs/tech_debriefs/README.md` summarises all key milestones from the initial design on May 26 2025 through the storage abstraction layer, Target_FS introduction and the 100% coverage refactor. The current release is `v0.11.1`.
Memory-FS serves both as a lightweight in‑memory storage engine and as a reference implementation for future backends such as S3 or SQLite. Its small, type‑safe API and complete test coverage make it an ideal starting point for new contributors or for embedding a fast, pluggable file system into your own projects.
Raw data
{
"_id": null,
"home_page": null,
"name": "memory-fs",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.12",
"maintainer_email": null,
"keywords": null,
"author": "Dinis Cruz",
"author_email": "dinis.cruz@owasp.org",
"download_url": "https://files.pythonhosted.org/packages/10/06/c48e19fd795464b9472996dc6a14ad396940746f5333ae9126858b1539f3/memory_fs-0.13.0.tar.gz",
"platform": null,
"description": "# Memory-FS\n\nMemory-FS is a fully tested, type-safe file system abstraction implemented entirely in memory. It originated from the `OSBot_Cloud_FS` prototype and has evolved through multiple architectural iterations documented in [docs/tech_debriefs](docs/tech_debriefs). The library exposes a small, composable API for creating, loading and editing file objects while remaining agnostic of the underlying storage provider.\n\n## Key Features\n\n- **In-Memory Storage** \u2013 extremely fast read/write operations via an in\u2011memory provider.\n- **Type Safety** \u2013 all APIs use `osbot_utils` `Type_Safe` models ensuring runtime validation.\n- **Two-File Pattern** \u2013 metadata (`.config` / `.metadata`) and content are stored separately.\n- **Pluggable Path Strategies** \u2013 configurable path handlers (latest, temporal, versioned, etc.).\n- **Extensible File Types** \u2013 JSON, Markdown, binary and custom formats through schema classes.\n- **100% Code Coverage** \u2013 every line of production code is tested as of version `v0.11.1`.\n\n## Architecture Overview\n\nThe current design (introduced in the [June\u00a018\u00a02025 debrief](docs/tech_debriefs/on-18-jun-2025.md)) is composed of two layers:\n\n1. **File_FS Layer** \u2013 high level file operations\n - `File_FS` object plus `File_FS__*` action classes\n - `Target_FS` factory for creating file objects from existing paths\n2. **Storage_FS Layer** \u2013 low level storage providers\n - `Storage_FS` interface and in\u2011memory implementation `Storage_FS__Memory`\n - Provider pattern enabling future backends (local disk, SQLite, zip, S3)\n\nThis separation yields a small surface area for extending or replacing storage while keeping file logic consistent. The project\u2019s [tech debrief README](docs/tech_debriefs/README.md) describes this architecture as:\n\n```\n1. **File_FS Layer**: High-level file operations\n2. **Storage_FS Layer**: Low-level storage operations\n - Provider pattern for pluggable storage backends\n```\n\n### File Naming\n\nFiles are organised using a predictable naming scheme:\n\n1. **Content File** \u2013 `{file_id}.{extension}`\n2. **Config File** \u2013 `{file_id}.{extension}.config`\n3. **Metadata File** \u2013 `{file_id}.{extension}.metadata`\n\n\n### Coverage Guarantee\n\nMemory-FS maintains comprehensive tests. As recorded in the debriefs:\n\n```\nAs of June 18, 2025 (v0.11.0), Memory-FS has achieved **100% code coverage**:\n- Every line of production code is tested\n- All edge cases are covered\n- No dead or unused code remains\n- Comprehensive test suite with 200+ test methods\n```\n\n## Installation\n\n```bash\npip install memory-fs\n```\n\nor install from source:\n\n```bash\npip install -e .\n```\n\n## Quick Example\n\nBelow is a minimal example using the high\u2011level API. It stores JSON content via the in\u2011memory provider:\n\n```python\nfrom memory_fs.file_fs.File_FS import File_FS\nfrom memory_fs.file_types.Memory_FS__File__Type__Json import Memory_FS__File__Type__Json\nfrom memory_fs.schemas.Schema__Memory_FS__File__Config import Schema__Memory_FS__File__Config\nfrom memory_fs.path_handlers.Path__Handler__Latest import Path__Handler__Latest\nfrom memory_fs.storage.Memory_FS__Storage import Memory_FS__Storage\nfrom memory_fs.storage_fs.providers.Storage_FS__Memory import Storage_FS__Memory\n\n# set up in-memory storage\nstorage = Memory_FS__Storage(storage_fs=Storage_FS__Memory())\n\n# configuration describing where/how the file should be stored\nconfig = Schema__Memory_FS__File__Config(\n path_handlers = [Path__Handler__Latest()],\n default_handler=Path__Handler__Latest,\n file_type = Memory_FS__File__Type__Json(),\n)\n\n# create a file object and save some data\nfile = File_FS(file_config=config, storage=storage)\nfile.create()\nfile.create__content(b'{\"hello\": \"world\"}')\n\n# reload using the factory\nfrom memory_fs.target_fs.Target_FS__Create import Target_FS__Create\nloaded_target = Target_FS__Create(storage=storage).from_path__config(file.file_fs__paths().paths__config()[0])\nprint(loaded_target.file_fs().content()) # -> b'{\"hello\": \"world\"}'\n```\n\n## Documentation\n\nExtensive documentation is kept under the `docs` folder. The [tech_debriefs](docs/tech_debriefs) directory captures the project evolution, while [architecture](docs/architecture/technical_architecture_debrief.md) provides a detailed design overview. Developers should also review [docs/dev](docs/dev/README.md) for coding conventions, bug lists and TODO items.\n\n## Project History\n\nThe debrief timeline in `docs/tech_debriefs/README.md` summarises all key milestones from the initial design on May\u00a026\u00a02025 through the storage abstraction layer, Target_FS introduction and the 100% coverage refactor. The current release is `v0.11.1`.\n\nMemory-FS serves both as a lightweight in\u2011memory storage engine and as a reference implementation for future backends such as S3 or SQLite. Its small, type\u2011safe API and complete test coverage make it an ideal starting point for new contributors or for embedding a fast, pluggable file system into your own projects.\n\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Memory-FS",
"version": "0.13.0",
"project_urls": {
"Homepage": "https://github.com/owasp-sbot/Memory-FS",
"Repository": "https://github.com/owasp-sbot/Memory-FS"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "61c6a62af84fd4d061ab244f0d1dd78bae83dfa5befbbbc2a4174b603ae217d2",
"md5": "22e39bcdf430e18206134443944c83f3",
"sha256": "a70f058fb411820ed18c5fc749d2816bad4df837a2edaa628f6e982b2578472f"
},
"downloads": -1,
"filename": "memory_fs-0.13.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "22e39bcdf430e18206134443944c83f3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.12",
"size": 37069,
"upload_time": "2025-07-08T10:54:08",
"upload_time_iso_8601": "2025-07-08T10:54:08.925834Z",
"url": "https://files.pythonhosted.org/packages/61/c6/a62af84fd4d061ab244f0d1dd78bae83dfa5befbbbc2a4174b603ae217d2/memory_fs-0.13.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1006c48e19fd795464b9472996dc6a14ad396940746f5333ae9126858b1539f3",
"md5": "7fe718f3cac05267afd2d806666ab38d",
"sha256": "b6b9e6a79771f9a07d6fa3feb59f9ca9350e0dc683ed923ed5383c4b148362c0"
},
"downloads": -1,
"filename": "memory_fs-0.13.0.tar.gz",
"has_sig": false,
"md5_digest": "7fe718f3cac05267afd2d806666ab38d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.12",
"size": 19419,
"upload_time": "2025-07-08T10:54:10",
"upload_time_iso_8601": "2025-07-08T10:54:10.051650Z",
"url": "https://files.pythonhosted.org/packages/10/06/c48e19fd795464b9472996dc6a14ad396940746f5333ae9126858b1539f3/memory_fs-0.13.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-08 10:54:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "owasp-sbot",
"github_project": "Memory-FS",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "memory-fs"
}