memory-fs


Namememory-fs JSON
Version 0.13.0 PyPI version JSON
download
home_pageNone
SummaryMemory-FS
upload_time2025-07-08 10:54:10
maintainerNone
docs_urlNone
authorDinis Cruz
requires_python<4.0,>=3.12
licenseApache 2.0
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # Memory-FS

Memory-FS is a fully tested, type-safe file system abstraction implemented entirely in memory.  It originated from the `OSBot_Cloud_FS` prototype and has evolved through multiple architectural iterations documented in [docs/tech_debriefs](docs/tech_debriefs).  The library exposes a small, composable API for creating, loading and editing file objects while remaining agnostic of the underlying storage provider.

## Key Features

- **In-Memory Storage** – extremely fast read/write operations via an in‑memory provider.
- **Type Safety** – all APIs use `osbot_utils` `Type_Safe` models ensuring runtime validation.
- **Two-File Pattern** – metadata (`.config` / `.metadata`) and content are stored separately.
- **Pluggable Path Strategies** – configurable path handlers (latest, temporal, versioned, etc.).
- **Extensible File Types** – JSON, Markdown, binary and custom formats through schema classes.
- **100% Code Coverage** – every line of production code is tested as of version `v0.11.1`.

## Architecture Overview

The current design (introduced in the [June 18 2025 debrief](docs/tech_debriefs/on-18-jun-2025.md)) is composed of two layers:

1. **File_FS Layer** – high level file operations
   - `File_FS` object plus `File_FS__*` action classes
   - `Target_FS` factory for creating file objects from existing paths
2. **Storage_FS Layer** – low level storage providers
   - `Storage_FS` interface and in‑memory implementation `Storage_FS__Memory`
   - Provider pattern enabling future backends (local disk, SQLite, zip, S3)

This separation yields a small surface area for extending or replacing storage while keeping file logic consistent.  The project’s [tech debrief README](docs/tech_debriefs/README.md) describes this architecture as:

```
1. **File_FS Layer**: High-level file operations
2. **Storage_FS Layer**: Low-level storage operations
   - Provider pattern for pluggable storage backends
```

### File Naming

Files are organised using a predictable naming scheme:

1. **Content File** – `{file_id}.{extension}`
2. **Config File** – `{file_id}.{extension}.config`
3. **Metadata File** – `{file_id}.{extension}.metadata`


### Coverage Guarantee

Memory-FS maintains comprehensive tests.  As recorded in the debriefs:

```
As of June 18, 2025 (v0.11.0), Memory-FS has achieved **100% code coverage**:
- Every line of production code is tested
- All edge cases are covered
- No dead or unused code remains
- Comprehensive test suite with 200+ test methods
```

## Installation

```bash
pip install memory-fs
```

or install from source:

```bash
pip install -e .
```

## Quick Example

Below is a minimal example using the high‑level API.  It stores JSON content via the in‑memory provider:

```python
from memory_fs.file_fs.File_FS                              import File_FS
from memory_fs.file_types.Memory_FS__File__Type__Json       import Memory_FS__File__Type__Json
from memory_fs.schemas.Schema__Memory_FS__File__Config      import Schema__Memory_FS__File__Config
from memory_fs.path_handlers.Path__Handler__Latest          import Path__Handler__Latest
from memory_fs.storage.Memory_FS__Storage                   import Memory_FS__Storage
from memory_fs.storage_fs.providers.Storage_FS__Memory      import Storage_FS__Memory

# set up in-memory storage
storage      = Memory_FS__Storage(storage_fs=Storage_FS__Memory())

# configuration describing where/how the file should be stored
config = Schema__Memory_FS__File__Config(
    path_handlers  = [Path__Handler__Latest()],
    default_handler=Path__Handler__Latest,
    file_type      = Memory_FS__File__Type__Json(),
)

# create a file object and save some data
file = File_FS(file_config=config, storage=storage)
file.create()
file.create__content(b'{"hello": "world"}')

# reload using the factory
from memory_fs.target_fs.Target_FS__Create import Target_FS__Create
loaded_target = Target_FS__Create(storage=storage).from_path__config(file.file_fs__paths().paths__config()[0])
print(loaded_target.file_fs().content())  # -> b'{"hello": "world"}'
```

## Documentation

Extensive documentation is kept under the `docs` folder.  The [tech_debriefs](docs/tech_debriefs) directory captures the project evolution, while [architecture](docs/architecture/technical_architecture_debrief.md) provides a detailed design overview.  Developers should also review [docs/dev](docs/dev/README.md) for coding conventions, bug lists and TODO items.

## Project History

The debrief timeline in `docs/tech_debriefs/README.md` summarises all key milestones from the initial design on May 26 2025 through the storage abstraction layer, Target_FS introduction and the 100% coverage refactor.  The current release is `v0.11.1`.

Memory-FS serves both as a lightweight in‑memory storage engine and as a reference implementation for future backends such as S3 or SQLite.  Its small, type‑safe API and complete test coverage make it an ideal starting point for new contributors or for embedding a fast, pluggable file system into your own projects.


            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "memory-fs",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<4.0,>=3.12",
    "maintainer_email": null,
    "keywords": null,
    "author": "Dinis Cruz",
    "author_email": "dinis.cruz@owasp.org",
    "download_url": "https://files.pythonhosted.org/packages/10/06/c48e19fd795464b9472996dc6a14ad396940746f5333ae9126858b1539f3/memory_fs-0.13.0.tar.gz",
    "platform": null,
    "description": "# Memory-FS\n\nMemory-FS is a fully tested, type-safe file system abstraction implemented entirely in memory.  It originated from the `OSBot_Cloud_FS` prototype and has evolved through multiple architectural iterations documented in [docs/tech_debriefs](docs/tech_debriefs).  The library exposes a small, composable API for creating, loading and editing file objects while remaining agnostic of the underlying storage provider.\n\n## Key Features\n\n- **In-Memory Storage** \u2013 extremely fast read/write operations via an in\u2011memory provider.\n- **Type Safety** \u2013 all APIs use `osbot_utils` `Type_Safe` models ensuring runtime validation.\n- **Two-File Pattern** \u2013 metadata (`.config` / `.metadata`) and content are stored separately.\n- **Pluggable Path Strategies** \u2013 configurable path handlers (latest, temporal, versioned, etc.).\n- **Extensible File Types** \u2013 JSON, Markdown, binary and custom formats through schema classes.\n- **100% Code Coverage** \u2013 every line of production code is tested as of version `v0.11.1`.\n\n## Architecture Overview\n\nThe current design (introduced in the [June\u00a018\u00a02025 debrief](docs/tech_debriefs/on-18-jun-2025.md)) is composed of two layers:\n\n1. **File_FS Layer** \u2013 high level file operations\n   - `File_FS` object plus `File_FS__*` action classes\n   - `Target_FS` factory for creating file objects from existing paths\n2. **Storage_FS Layer** \u2013 low level storage providers\n   - `Storage_FS` interface and in\u2011memory implementation `Storage_FS__Memory`\n   - Provider pattern enabling future backends (local disk, SQLite, zip, S3)\n\nThis separation yields a small surface area for extending or replacing storage while keeping file logic consistent.  The project\u2019s [tech debrief README](docs/tech_debriefs/README.md) describes this architecture as:\n\n```\n1. **File_FS Layer**: High-level file operations\n2. **Storage_FS Layer**: Low-level storage operations\n   - Provider pattern for pluggable storage backends\n```\n\n### File Naming\n\nFiles are organised using a predictable naming scheme:\n\n1. **Content File** \u2013 `{file_id}.{extension}`\n2. **Config File** \u2013 `{file_id}.{extension}.config`\n3. **Metadata File** \u2013 `{file_id}.{extension}.metadata`\n\n\n### Coverage Guarantee\n\nMemory-FS maintains comprehensive tests.  As recorded in the debriefs:\n\n```\nAs of June 18, 2025 (v0.11.0), Memory-FS has achieved **100% code coverage**:\n- Every line of production code is tested\n- All edge cases are covered\n- No dead or unused code remains\n- Comprehensive test suite with 200+ test methods\n```\n\n## Installation\n\n```bash\npip install memory-fs\n```\n\nor install from source:\n\n```bash\npip install -e .\n```\n\n## Quick Example\n\nBelow is a minimal example using the high\u2011level API.  It stores JSON content via the in\u2011memory provider:\n\n```python\nfrom memory_fs.file_fs.File_FS                              import File_FS\nfrom memory_fs.file_types.Memory_FS__File__Type__Json       import Memory_FS__File__Type__Json\nfrom memory_fs.schemas.Schema__Memory_FS__File__Config      import Schema__Memory_FS__File__Config\nfrom memory_fs.path_handlers.Path__Handler__Latest          import Path__Handler__Latest\nfrom memory_fs.storage.Memory_FS__Storage                   import Memory_FS__Storage\nfrom memory_fs.storage_fs.providers.Storage_FS__Memory      import Storage_FS__Memory\n\n# set up in-memory storage\nstorage      = Memory_FS__Storage(storage_fs=Storage_FS__Memory())\n\n# configuration describing where/how the file should be stored\nconfig = Schema__Memory_FS__File__Config(\n    path_handlers  = [Path__Handler__Latest()],\n    default_handler=Path__Handler__Latest,\n    file_type      = Memory_FS__File__Type__Json(),\n)\n\n# create a file object and save some data\nfile = File_FS(file_config=config, storage=storage)\nfile.create()\nfile.create__content(b'{\"hello\": \"world\"}')\n\n# reload using the factory\nfrom memory_fs.target_fs.Target_FS__Create import Target_FS__Create\nloaded_target = Target_FS__Create(storage=storage).from_path__config(file.file_fs__paths().paths__config()[0])\nprint(loaded_target.file_fs().content())  # -> b'{\"hello\": \"world\"}'\n```\n\n## Documentation\n\nExtensive documentation is kept under the `docs` folder.  The [tech_debriefs](docs/tech_debriefs) directory captures the project evolution, while [architecture](docs/architecture/technical_architecture_debrief.md) provides a detailed design overview.  Developers should also review [docs/dev](docs/dev/README.md) for coding conventions, bug lists and TODO items.\n\n## Project History\n\nThe debrief timeline in `docs/tech_debriefs/README.md` summarises all key milestones from the initial design on May\u00a026\u00a02025 through the storage abstraction layer, Target_FS introduction and the 100% coverage refactor.  The current release is `v0.11.1`.\n\nMemory-FS serves both as a lightweight in\u2011memory storage engine and as a reference implementation for future backends such as S3 or SQLite.  Its small, type\u2011safe API and complete test coverage make it an ideal starting point for new contributors or for embedding a fast, pluggable file system into your own projects.\n\n",
    "bugtrack_url": null,
    "license": "Apache 2.0",
    "summary": "Memory-FS",
    "version": "0.13.0",
    "project_urls": {
        "Homepage": "https://github.com/owasp-sbot/Memory-FS",
        "Repository": "https://github.com/owasp-sbot/Memory-FS"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "61c6a62af84fd4d061ab244f0d1dd78bae83dfa5befbbbc2a4174b603ae217d2",
                "md5": "22e39bcdf430e18206134443944c83f3",
                "sha256": "a70f058fb411820ed18c5fc749d2816bad4df837a2edaa628f6e982b2578472f"
            },
            "downloads": -1,
            "filename": "memory_fs-0.13.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "22e39bcdf430e18206134443944c83f3",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<4.0,>=3.12",
            "size": 37069,
            "upload_time": "2025-07-08T10:54:08",
            "upload_time_iso_8601": "2025-07-08T10:54:08.925834Z",
            "url": "https://files.pythonhosted.org/packages/61/c6/a62af84fd4d061ab244f0d1dd78bae83dfa5befbbbc2a4174b603ae217d2/memory_fs-0.13.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "1006c48e19fd795464b9472996dc6a14ad396940746f5333ae9126858b1539f3",
                "md5": "7fe718f3cac05267afd2d806666ab38d",
                "sha256": "b6b9e6a79771f9a07d6fa3feb59f9ca9350e0dc683ed923ed5383c4b148362c0"
            },
            "downloads": -1,
            "filename": "memory_fs-0.13.0.tar.gz",
            "has_sig": false,
            "md5_digest": "7fe718f3cac05267afd2d806666ab38d",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<4.0,>=3.12",
            "size": 19419,
            "upload_time": "2025-07-08T10:54:10",
            "upload_time_iso_8601": "2025-07-08T10:54:10.051650Z",
            "url": "https://files.pythonhosted.org/packages/10/06/c48e19fd795464b9472996dc6a14ad396940746f5333ae9126858b1539f3/memory_fs-0.13.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-07-08 10:54:10",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "owasp-sbot",
    "github_project": "Memory-FS",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "memory-fs"
}
        
Elapsed time: 0.41623s