| Name | omni-pathlib JSON |
| Version |
0.3.9
JSON |
| download |
| home_page | None |
| Summary | 一个统一的路径处理库,支持本地文件系统、HTTP 和 S3 存储的路径操作 |
| upload_time | 2025-04-02 14:13:51 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.10 |
| license | None |
| keywords |
async
filesystem
http
pathlib
s3
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Omni-Pathlib
[中文文档](README_ZH.md)
[](https://pypi.org/project/omni-pathlib/)
[](https://pypi.org/project/omni-pathlib/)
[](https://github.com/Haskely/omni-pathlib/blob/main/LICENSE)
[](https://pepy.tech/project/omni-pathlib)
[](https://github.com/Haskely/omni-pathlib/stargazers)
[](https://github.com/Haskely/omni-pathlib/issues)
[](https://libraries.io/github/Haskely/omni-pathlib)
Omni-Pathlib is a unified path handling library that supports path operations for local file systems, HTTP, and S3 storage. It provides both synchronous and asynchronous APIs, making it easy and consistent to operate files across different storage systems.
## Installation
```bash
pip install omni-pathlib
```
## Basic Usage
```python
from omni_pathlib import OmniPath
# Create different types of paths
http_path = OmniPath("https://example.com/file.txt")
s3_path = OmniPath("s3://my-bucket/path/to/file.txt")
local_path = OmniPath("/local/path/to/file.txt")
# Read file content
content = http_path.read_text() # Read from HTTP
s3_content = s3_path.read_text() # Read from S3
local_content = local_path.read_text() # Read from local
# Asynchronous operations
async def main():
content = await http_path.async_read_text()
s3_content = await s3_path.async_read_text()
local_content = await local_path.async_read_text()
```
## Features
- Unified path operation interface
- Supports local file systems, HTTP, and S3 storage
- Synchronous and asynchronous APIs
- HTTP supports caching and resuming downloads
- S3 supports complete bucket operations
- Local file system supports standard path operations
## Function Interface Description
### Basic Operations
All storage types support the following operations:
```python
# Path attributes
path.name # Get the path name
path.stem # Get the path name without suffix
path.suffix # Get the path suffix
path.parent # Get the parent path
path.protocol # Get the protocol type ('file', 'http', 's3')
# Basic operations
path.exists() # Check if the path exists
path.iterdir() # Iterate through directory contents
path.stat() # Get file information (size, modification time, etc.)
path.read_bytes() # Read binary content
path.read_text() # Read text content
path.write_bytes(data) # Write binary content
path.write_text(data) # Write text content
path.delete() # Delete file
# All operations have corresponding asynchronous versions
await path.async_exists()
await path.async_iterdir()
# ... and so on
```
### Local File System Specific Operations
- `mkdir(parents=False, exist_ok=False)` / `async_mkdir()` - Create a directory
- `rmdir()` / `async_rmdir()` - Remove an empty directory
- `rename(target)` / `async_rename(target)` - Rename a file/directory
- `is_dir()` / `async_is_dir()` - Check if it is a directory
- `is_file()` / `async_is_file()` - Check if it is a file
### HTTP Specific Features
- Supports resuming downloads
- Automatically caches downloaded content
- Does not support write and delete operations
### S3 Specific Features
- Fully supports S3 bucket operations
- Supports custom endpoints
- Supports multiple authentication methods
- Supports specifying profile in URL scheme
#### S3 Profile Configuration Priority
Configuration priority from high to low:
1. Configuration passed directly as parameters
2. Profile specified in URL scheme (e.g., `s3+my_profile://bucket/key`)
3. Configuration through environment variable `AWS_PROFILE`
4. `default` profile in configuration file
5. The first profile found in the configuration file
#### S3 URL Scheme Example
```python
# Parameter priority is higher than URL scheme
path = OmniPath(
"s3+my_profile://bucket/key",
profile_name="other_profile" # Will use other_profile instead of my_profile
)
# Specify profile through URL scheme
path = OmniPath("s3+my_profile://bucket/key") # Will use my_profile configuration
# Specify profile through environment variable
os.environ["AWS_PROFILE"] = "other_profile"
path = OmniPath("s3://bucket/key") # Will use other_profile configuration
# Specify profile through configuration file
path = OmniPath("s3://bucket/key") # Will use default configuration (if exists) or the first found configuration
```
#### S3 Profiles Retrieval Logic
- Retrieve environment variable configurations from `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`, `OSS_ENDPOINT`, `S3_ENDPOINT`, `AWS_ENDPOINT_URL`. These configurations will default to the `default` profile but can be specified to other named profiles by adding a prefix, e.g., `my_profile__AWS_ACCESS_KEY_ID=my_access_key_id` will go into the profile named `my_profile`.
- Load configurations from the configuration file path obtained from the environment variable `AWS_SHARED_CREDENTIALS_FILE`, defaulting to `~/.aws/credentials`.
## Development
### Install Dependencies
```bash
uv sync
```
### Run Tests
```bash
uv run pytest
```
### Commit
```bash
pre-commit install
cz commit
```
### Release
```bash
cz bump
git push --follow-tags
```
Raw data
{
"_id": null,
"home_page": null,
"name": "omni-pathlib",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "async, filesystem, http, pathlib, s3",
"author": null,
"author_email": "Haskely <Haskely@live.com>",
"download_url": "https://files.pythonhosted.org/packages/69/e1/ad9786d22a9eb50969946d91bfcd10f4b1f765014af0b42916a99c64c812/omni_pathlib-0.3.9.tar.gz",
"platform": null,
"description": "# Omni-Pathlib\n\n[\u4e2d\u6587\u6587\u6863](README_ZH.md)\n\n[](https://pypi.org/project/omni-pathlib/)\n[](https://pypi.org/project/omni-pathlib/)\n[](https://github.com/Haskely/omni-pathlib/blob/main/LICENSE)\n[](https://pepy.tech/project/omni-pathlib)\n[](https://github.com/Haskely/omni-pathlib/stargazers)\n[](https://github.com/Haskely/omni-pathlib/issues)\n[](https://libraries.io/github/Haskely/omni-pathlib)\n\nOmni-Pathlib is a unified path handling library that supports path operations for local file systems, HTTP, and S3 storage. It provides both synchronous and asynchronous APIs, making it easy and consistent to operate files across different storage systems.\n\n## Installation\n\n```bash\npip install omni-pathlib\n```\n\n## Basic Usage\n\n```python\nfrom omni_pathlib import OmniPath\n\n# Create different types of paths\nhttp_path = OmniPath(\"https://example.com/file.txt\")\ns3_path = OmniPath(\"s3://my-bucket/path/to/file.txt\")\nlocal_path = OmniPath(\"/local/path/to/file.txt\")\n\n# Read file content\ncontent = http_path.read_text() # Read from HTTP\ns3_content = s3_path.read_text() # Read from S3\nlocal_content = local_path.read_text() # Read from local\n\n# Asynchronous operations\nasync def main():\n content = await http_path.async_read_text()\n s3_content = await s3_path.async_read_text()\n local_content = await local_path.async_read_text()\n```\n\n## Features\n\n- Unified path operation interface\n- Supports local file systems, HTTP, and S3 storage\n- Synchronous and asynchronous APIs\n- HTTP supports caching and resuming downloads\n- S3 supports complete bucket operations\n- Local file system supports standard path operations\n\n## Function Interface Description\n\n### Basic Operations\n\nAll storage types support the following operations:\n\n```python\n# Path attributes\npath.name # Get the path name\npath.stem # Get the path name without suffix\npath.suffix # Get the path suffix\npath.parent # Get the parent path\npath.protocol # Get the protocol type ('file', 'http', 's3')\n\n# Basic operations\npath.exists() # Check if the path exists\npath.iterdir() # Iterate through directory contents\npath.stat() # Get file information (size, modification time, etc.)\npath.read_bytes() # Read binary content\npath.read_text() # Read text content\npath.write_bytes(data) # Write binary content\npath.write_text(data) # Write text content\npath.delete() # Delete file\n\n# All operations have corresponding asynchronous versions\nawait path.async_exists()\nawait path.async_iterdir()\n# ... and so on\n```\n\n### Local File System Specific Operations\n\n- `mkdir(parents=False, exist_ok=False)` / `async_mkdir()` - Create a directory\n- `rmdir()` / `async_rmdir()` - Remove an empty directory\n- `rename(target)` / `async_rename(target)` - Rename a file/directory\n- `is_dir()` / `async_is_dir()` - Check if it is a directory\n- `is_file()` / `async_is_file()` - Check if it is a file\n\n### HTTP Specific Features\n\n- Supports resuming downloads\n- Automatically caches downloaded content\n- Does not support write and delete operations\n\n### S3 Specific Features\n\n- Fully supports S3 bucket operations\n- Supports custom endpoints\n- Supports multiple authentication methods\n- Supports specifying profile in URL scheme\n\n#### S3 Profile Configuration Priority\n\nConfiguration priority from high to low:\n\n1. Configuration passed directly as parameters\n2. Profile specified in URL scheme (e.g., `s3+my_profile://bucket/key`)\n3. Configuration through environment variable `AWS_PROFILE`\n4. `default` profile in configuration file\n5. The first profile found in the configuration file\n\n#### S3 URL Scheme Example\n\n```python\n# Parameter priority is higher than URL scheme\npath = OmniPath(\n \"s3+my_profile://bucket/key\",\n profile_name=\"other_profile\" # Will use other_profile instead of my_profile\n)\n\n# Specify profile through URL scheme\npath = OmniPath(\"s3+my_profile://bucket/key\") # Will use my_profile configuration\n\n# Specify profile through environment variable\nos.environ[\"AWS_PROFILE\"] = \"other_profile\"\npath = OmniPath(\"s3://bucket/key\") # Will use other_profile configuration\n\n# Specify profile through configuration file\npath = OmniPath(\"s3://bucket/key\") # Will use default configuration (if exists) or the first found configuration\n```\n\n#### S3 Profiles Retrieval Logic\n\n- Retrieve environment variable configurations from `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_REGION`, `OSS_ENDPOINT`, `S3_ENDPOINT`, `AWS_ENDPOINT_URL`. These configurations will default to the `default` profile but can be specified to other named profiles by adding a prefix, e.g., `my_profile__AWS_ACCESS_KEY_ID=my_access_key_id` will go into the profile named `my_profile`.\n- Load configurations from the configuration file path obtained from the environment variable `AWS_SHARED_CREDENTIALS_FILE`, defaulting to `~/.aws/credentials`.\n\n## Development\n\n### Install Dependencies\n\n```bash\nuv sync\n```\n\n### Run Tests\n\n```bash\nuv run pytest\n```\n\n### Commit\n\n```bash\npre-commit install\ncz commit\n```\n\n### Release\n\n```bash\ncz bump\n\ngit push --follow-tags\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "\u4e00\u4e2a\u7edf\u4e00\u7684\u8def\u5f84\u5904\u7406\u5e93,\u652f\u6301\u672c\u5730\u6587\u4ef6\u7cfb\u7edf\u3001HTTP \u548c S3 \u5b58\u50a8\u7684\u8def\u5f84\u64cd\u4f5c",
"version": "0.3.9",
"project_urls": {
"Bug Tracker": "https://github.com/Haskely/omni-pathlib/issues",
"Change Log": "https://github.com/Haskely/omni-pathlib/blob/main/CHANGELOG.md",
"Documentation": "https://github.com/Haskely/omni-pathlib#readme",
"Homepage": "https://github.com/Haskely/omni-pathlib",
"Repository": "https://github.com/Haskely/omni-pathlib"
},
"split_keywords": [
"async",
" filesystem",
" http",
" pathlib",
" s3"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "763d62c1432e8c5aacf8d5f540d46e0568cc0f69a9055af14e7ee09c8267a964",
"md5": "9baf9bfd2006074f58e9d08b190fd098",
"sha256": "f68dce2ee07b92a7edc190251531a172521809e0b7d423ebe12afb29941c652a"
},
"downloads": -1,
"filename": "omni_pathlib-0.3.9-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9baf9bfd2006074f58e9d08b190fd098",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 26537,
"upload_time": "2025-04-02T14:13:49",
"upload_time_iso_8601": "2025-04-02T14:13:49.664050Z",
"url": "https://files.pythonhosted.org/packages/76/3d/62c1432e8c5aacf8d5f540d46e0568cc0f69a9055af14e7ee09c8267a964/omni_pathlib-0.3.9-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "69e1ad9786d22a9eb50969946d91bfcd10f4b1f765014af0b42916a99c64c812",
"md5": "cd21064b3fa10409e035cd849fbfceb0",
"sha256": "784031e8b516a83d031d54fb131c7c563e4f4c5e00c404253a46a710ce8cc397"
},
"downloads": -1,
"filename": "omni_pathlib-0.3.9.tar.gz",
"has_sig": false,
"md5_digest": "cd21064b3fa10409e035cd849fbfceb0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 137519,
"upload_time": "2025-04-02T14:13:51",
"upload_time_iso_8601": "2025-04-02T14:13:51.340891Z",
"url": "https://files.pythonhosted.org/packages/69/e1/ad9786d22a9eb50969946d91bfcd10f4b1f765014af0b42916a99c64c812/omni_pathlib-0.3.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-04-02 14:13:51",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Haskely",
"github_project": "omni-pathlib",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "omni-pathlib"
}
