# FlexiStore
[](https://badge.fury.io/py/flexistore)
[](https://www.python.org/downloads/)
[](https://opensource.org/licenses/MIT)
[](https://github.com/prakhara56/FlexiStore/actions)
[](https://codecov.io/gh/prakhara56/FlexiStore)
**FlexiStore** is a modern, cloud-agnostic Python storage abstraction library that provides a unified interface to common file operations across different cloud providers. It features enhanced architecture with async support, retry mechanisms, streaming operations, and a professional CLI interface.
## 🚀 Features
- **Unified API** for upload/download/list/delete operations across cloud providers
- **Async Support** with both async and sync interfaces for all operations
- **Enhanced Architecture** with modular design, retry mechanisms, and streaming
- **Professional CLI** with interactive menus and rich user experience
- **Multiple Backends** including Azure Blob Storage and AWS S3
- **Enterprise Features** like encryption, compression, validation, and error handling
- **Modern Python** with type hints, comprehensive testing, and CI/CD pipeline
## Features
- **Unified API** for upload/download/list/delete operations.
- **Pluggable backends**: Implement `StorageManager` to add new providers.
- **Minimal dependencies**: Only requires the SDK for the backend you use.
- **CLI tool**: Interactive command-line interface for common operations.
## 📦 Installation
### From PyPI (Recommended)
```bash
pip install flexistore
```
### From GitHub (Development)
```bash
pip install git+https://github.com/prakhara56/FlexiStore.git@main
```
### With Development Dependencies
```bash
pip install flexistore[dev]
```
## 🚀 Quick Start
### As a Library
```python
from flexistore.backends.azure import AzureStorageManager, AzureStorageConfig
from flexistore.backends.aws import AWSStorageManager, AWSStorageConfig
# Azure Blob Storage
azure_config = AzureStorageConfig(
connection_string="<AZURE_CONN_STRING>",
container_name="my-container"
)
azure_mgr = AzureStorageManager(azure_config)
# AWS S3
aws_config = AWSStorageConfig(
bucket_name="my-bucket",
region_name="us-east-1",
aws_access_key_id="<ACCESS_KEY>",
aws_secret_access_key="<SECRET_KEY>"
)
aws_mgr = AWSStorageManager(aws_config)
# Upload a file
azure_mgr.upload_file("./data/report.csv", "backups/report.csv")
# List files
files = aws_mgr.list_files("backups/")
# Download a file
aws_mgr.download_file("backups/report.csv", "./downloads/report.csv")
```
### As a CLI
```bash
# Interactive CLI with Azure
flexistore --provider azure
# Interactive CLI with AWS
flexistore --provider aws
# With SSL verification disabled for AWS
flexistore --provider aws --no-verify-ssl
# Show help
flexistore --help
# Show version
flexistore --version
```
### Environment Variables
The CLI automatically detects credentials from environment variables or `.env` files:
#### Azure Backend
- `AZURE_CONN_STRING` - Azure Storage connection string
- `AZURE_CONTAINER` - Container name
#### AWS Backend
- `AWS_BUCKET` - S3 bucket name
- `AWS_REGION` - AWS region (e.g., us-east-1)
- `AWS_ACCESS_KEY_ID` - AWS access key
- `AWS_SECRET_ACCESS_KEY` - AWS secret key
#### General
- `FLEXISTORE_PROVIDER` - Default provider (azure/aws)
### Example .env File
```bash
# Azure
AZURE_CONN_STRING=DefaultEndpointsProtocol=https;AccountName=...
AZURE_CONTAINER=my-container
# AWS
AWS_BUCKET=my-bucket
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...
```
## 📚 API Reference
### Core Classes
- **`StorageManager`** - Abstract base class for all storage backends
- **`StorageConfig`** - Base configuration class with common options
- **`StorageMetadata`** - File metadata and properties
- **`StorageObject`** - File object representation
### Backend Implementations
#### `AzureStorageManager` & `AzureStorageConfig`
- Azure Blob Storage backend with async support
- Connection string and container-based configuration
- Built-in retry mechanisms and error handling
#### `AWSStorageManager` & `AWSStorageConfig`
- AWS S3 backend with async support
- Bucket, region, and credentials configuration
- SSL verification controls and connection pooling
### Key Methods
All storage managers implement these core methods:
```python
# Synchronous operations
manager.upload_file(local_path, remote_path)
manager.download_file(remote_path, local_path)
manager.list_files(prefix)
manager.delete_file(remote_path)
manager.download_folder(remote_prefix, local_dir)
# Asynchronous operations
await manager.aupload_file(local_path, remote_path)
await manager.adownload_file(remote_path, local_path)
await manager.alist_files(prefix)
await manager.adelete_file(remote_path)
await manager.adownload_folder(remote_prefix, local_dir)
```
## 🖥️ CLI Reference
The FlexiStore CLI provides an interactive interface for all storage operations:
### Commands
- **`flexistore`** - Main CLI command
- **`flexistore-cli`** - Alternative command name
- **`python -m flexistore`** - Run as Python module
### Options
- **`--provider {azure,aws}`** - Choose storage provider
- **`--verify-ssl`** - Enable SSL verification (AWS)
- **`--no-verify-ssl`** - Disable SSL verification (AWS)
- **`--help`** - Show help information
- **`--version`** - Show version information
### Interactive Menu
The CLI provides a user-friendly menu with these operations:
1. **📤 Upload a file** - Upload local files to cloud storage
2. **📋 List files** - Browse files and folders in storage
3. **📥 Download a file** - Download files from cloud storage
4. **🗑️ Delete a file** - Remove files from cloud storage
5. **📁 Download folder** - Download entire folders recursively
0. **🚪 Exit** - Clean exit with resource cleanup
## 🔧 Extending for Other Providers
FlexiStore is designed to be easily extensible. To add a new cloud provider:
1. **Create Configuration Class**
```python
from flexistore.core.base import StorageConfig
class MyProviderConfig(StorageConfig):
def __init__(self, api_key: str, region: str, **kwargs):
super().__init__(**kwargs)
self.api_key = api_key
self.region = region
```
2. **Implement Storage Manager**
```python
from flexistore.core.base import StorageManager
class MyProviderManager(StorageManager):
def __init__(self, config: MyProviderConfig):
self.config = config
# Initialize your provider's client
def upload_file(self, local_path, remote_path):
# Implement upload logic
pass
# Implement other abstract methods...
```
3. **Add to Package**
- Place in `flexistore/backends/myprovider.py`
- Update `flexistore/backends/__init__.py`
- Add CLI support in `flexistore/cli/config.py`
## 🏗️ Architecture
FlexiStore follows a modular, extensible architecture:
```
flexistore/
├── core/ # Abstract base classes and core functionality
├── backends/ # Cloud provider implementations
├── cli/ # Command-line interface modules
├── utils/ # Utility functions and helpers
└── __init__.py # Package exports
```
### Key Design Principles
- **Separation of Concerns** - Clear boundaries between components
- **Dependency Injection** - Configuration-driven initialization
- **Async-First** - Native async support with sync wrappers
- **Error Handling** - Comprehensive exception hierarchy
- **Extensibility** - Easy to add new cloud providers
## 🤝 Contributing
We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
### Development Setup
```bash
git clone https://github.com/prakhara56/FlexiStore.git
cd FlexiStore
pip install -e '.[dev]'
pre-commit install
```
### Running Tests
```bash
pytest # Run all tests
pytest --cov=flexistore # With coverage
tox # Multi-environment testing
```
## License
This project is licensed under the MIT License.
Raw data
{
"_id": null,
"home_page": "https://github.com/prakhara56/FlexiStore",
"name": "flexistore",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Prakhar Agarwal <prakhara56@gmail.com>",
"keywords": "storage, cloud, azure, aws, s3, blob, async",
"author": "Prakhar Agarwal",
"author_email": "Prakhar Agarwal <prakhara56@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/dd/2f/565e51c98dfeffec12f380d6272618427d9c579598f8ad8db6f30c5cc4c2/flexistore-0.2.0.tar.gz",
"platform": null,
"description": "\n# FlexiStore\n\n[](https://badge.fury.io/py/flexistore)\n[](https://www.python.org/downloads/)\n[](https://opensource.org/licenses/MIT)\n[](https://github.com/prakhara56/FlexiStore/actions)\n[](https://codecov.io/gh/prakhara56/FlexiStore)\n\n**FlexiStore** is a modern, cloud-agnostic Python storage abstraction library that provides a unified interface to common file operations across different cloud providers. It features enhanced architecture with async support, retry mechanisms, streaming operations, and a professional CLI interface.\n\n## \ud83d\ude80 Features\n\n- **Unified API** for upload/download/list/delete operations across cloud providers\n- **Async Support** with both async and sync interfaces for all operations\n- **Enhanced Architecture** with modular design, retry mechanisms, and streaming\n- **Professional CLI** with interactive menus and rich user experience\n- **Multiple Backends** including Azure Blob Storage and AWS S3\n- **Enterprise Features** like encryption, compression, validation, and error handling\n- **Modern Python** with type hints, comprehensive testing, and CI/CD pipeline\n\n## Features\n\n- **Unified API** for upload/download/list/delete operations.\n- **Pluggable backends**: Implement `StorageManager` to add new providers.\n- **Minimal dependencies**: Only requires the SDK for the backend you use.\n- **CLI tool**: Interactive command-line interface for common operations.\n\n## \ud83d\udce6 Installation\n\n### From PyPI (Recommended)\n\n```bash\npip install flexistore\n```\n\n### From GitHub (Development)\n\n```bash\npip install git+https://github.com/prakhara56/FlexiStore.git@main\n```\n\n### With Development Dependencies\n\n```bash\npip install flexistore[dev]\n```\n\n## \ud83d\ude80 Quick Start\n\n### As a Library\n\n```python\nfrom flexistore.backends.azure import AzureStorageManager, AzureStorageConfig\nfrom flexistore.backends.aws import AWSStorageManager, AWSStorageConfig\n\n# Azure Blob Storage\nazure_config = AzureStorageConfig(\n connection_string=\"<AZURE_CONN_STRING>\",\n container_name=\"my-container\"\n)\nazure_mgr = AzureStorageManager(azure_config)\n\n# AWS S3\naws_config = AWSStorageConfig(\n bucket_name=\"my-bucket\",\n region_name=\"us-east-1\",\n aws_access_key_id=\"<ACCESS_KEY>\",\n aws_secret_access_key=\"<SECRET_KEY>\"\n)\naws_mgr = AWSStorageManager(aws_config)\n\n# Upload a file\nazure_mgr.upload_file(\"./data/report.csv\", \"backups/report.csv\")\n\n# List files\nfiles = aws_mgr.list_files(\"backups/\")\n\n# Download a file\naws_mgr.download_file(\"backups/report.csv\", \"./downloads/report.csv\")\n```\n\n### As a CLI\n\n```bash\n# Interactive CLI with Azure\nflexistore --provider azure\n\n# Interactive CLI with AWS\nflexistore --provider aws\n\n# With SSL verification disabled for AWS\nflexistore --provider aws --no-verify-ssl\n\n# Show help\nflexistore --help\n\n# Show version\nflexistore --version\n```\n\n### Environment Variables\n\nThe CLI automatically detects credentials from environment variables or `.env` files:\n\n#### Azure Backend\n- `AZURE_CONN_STRING` - Azure Storage connection string\n- `AZURE_CONTAINER` - Container name\n\n#### AWS Backend\n- `AWS_BUCKET` - S3 bucket name\n- `AWS_REGION` - AWS region (e.g., us-east-1)\n- `AWS_ACCESS_KEY_ID` - AWS access key\n- `AWS_SECRET_ACCESS_KEY` - AWS secret key\n\n#### General\n- `FLEXISTORE_PROVIDER` - Default provider (azure/aws)\n\n### Example .env File\n\n```bash\n# Azure\nAZURE_CONN_STRING=DefaultEndpointsProtocol=https;AccountName=...\nAZURE_CONTAINER=my-container\n\n# AWS\nAWS_BUCKET=my-bucket\nAWS_REGION=us-east-1\nAWS_ACCESS_KEY_ID=AKIA...\nAWS_SECRET_ACCESS_KEY=...\n```\n\n## \ud83d\udcda API Reference\n\n### Core Classes\n\n- **`StorageManager`** - Abstract base class for all storage backends\n- **`StorageConfig`** - Base configuration class with common options\n- **`StorageMetadata`** - File metadata and properties\n- **`StorageObject`** - File object representation\n\n### Backend Implementations\n\n#### `AzureStorageManager` & `AzureStorageConfig`\n- Azure Blob Storage backend with async support\n- Connection string and container-based configuration\n- Built-in retry mechanisms and error handling\n\n#### `AWSStorageManager` & `AWSStorageConfig`\n- AWS S3 backend with async support\n- Bucket, region, and credentials configuration\n- SSL verification controls and connection pooling\n\n### Key Methods\n\nAll storage managers implement these core methods:\n\n```python\n# Synchronous operations\nmanager.upload_file(local_path, remote_path)\nmanager.download_file(remote_path, local_path)\nmanager.list_files(prefix)\nmanager.delete_file(remote_path)\nmanager.download_folder(remote_prefix, local_dir)\n\n# Asynchronous operations\nawait manager.aupload_file(local_path, remote_path)\nawait manager.adownload_file(remote_path, local_path)\nawait manager.alist_files(prefix)\nawait manager.adelete_file(remote_path)\nawait manager.adownload_folder(remote_prefix, local_dir)\n```\n\n## \ud83d\udda5\ufe0f CLI Reference\n\nThe FlexiStore CLI provides an interactive interface for all storage operations:\n\n### Commands\n\n- **`flexistore`** - Main CLI command\n- **`flexistore-cli`** - Alternative command name\n- **`python -m flexistore`** - Run as Python module\n\n### Options\n\n- **`--provider {azure,aws}`** - Choose storage provider\n- **`--verify-ssl`** - Enable SSL verification (AWS)\n- **`--no-verify-ssl`** - Disable SSL verification (AWS)\n- **`--help`** - Show help information\n- **`--version`** - Show version information\n\n### Interactive Menu\n\nThe CLI provides a user-friendly menu with these operations:\n\n1. **\ud83d\udce4 Upload a file** - Upload local files to cloud storage\n2. **\ud83d\udccb List files** - Browse files and folders in storage\n3. **\ud83d\udce5 Download a file** - Download files from cloud storage\n4. **\ud83d\uddd1\ufe0f Delete a file** - Remove files from cloud storage\n5. **\ud83d\udcc1 Download folder** - Download entire folders recursively\n0. **\ud83d\udeaa Exit** - Clean exit with resource cleanup\n\n## \ud83d\udd27 Extending for Other Providers\n\nFlexiStore is designed to be easily extensible. To add a new cloud provider:\n\n1. **Create Configuration Class**\n ```python\n from flexistore.core.base import StorageConfig\n \n class MyProviderConfig(StorageConfig):\n def __init__(self, api_key: str, region: str, **kwargs):\n super().__init__(**kwargs)\n self.api_key = api_key\n self.region = region\n ```\n\n2. **Implement Storage Manager**\n ```python\n from flexistore.core.base import StorageManager\n \n class MyProviderManager(StorageManager):\n def __init__(self, config: MyProviderConfig):\n self.config = config\n # Initialize your provider's client\n \n def upload_file(self, local_path, remote_path):\n # Implement upload logic\n pass\n \n # Implement other abstract methods...\n ```\n\n3. **Add to Package**\n - Place in `flexistore/backends/myprovider.py`\n - Update `flexistore/backends/__init__.py`\n - Add CLI support in `flexistore/cli/config.py`\n\n## \ud83c\udfd7\ufe0f Architecture\n\nFlexiStore follows a modular, extensible architecture:\n\n```\nflexistore/\n\u251c\u2500\u2500 core/ # Abstract base classes and core functionality\n\u251c\u2500\u2500 backends/ # Cloud provider implementations\n\u251c\u2500\u2500 cli/ # Command-line interface modules\n\u251c\u2500\u2500 utils/ # Utility functions and helpers\n\u2514\u2500\u2500 __init__.py # Package exports\n```\n\n### Key Design Principles\n\n- **Separation of Concerns** - Clear boundaries between components\n- **Dependency Injection** - Configuration-driven initialization\n- **Async-First** - Native async support with sync wrappers\n- **Error Handling** - Comprehensive exception hierarchy\n- **Extensibility** - Easy to add new cloud providers\n\n## \ud83e\udd1d Contributing\n\nWe welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.\n\n### Development Setup\n\n```bash\ngit clone https://github.com/prakhara56/FlexiStore.git\ncd FlexiStore\npip install -e '.[dev]'\npre-commit install\n```\n\n### Running Tests\n\n```bash\npytest # Run all tests\npytest --cov=flexistore # With coverage\ntox # Multi-environment testing\n```\n\n## License\n\nThis project is licensed under the MIT License.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Cloud-agnostic storage interface with enhanced features and async support",
"version": "0.2.0",
"project_urls": {
"Bug Tracker": "https://github.com/prakhara56/FlexiStore/issues",
"Changelog": "https://github.com/prakhara56/FlexiStore/blob/main/CHANGELOG.md",
"Documentation": "https://flexistore.readthedocs.io/",
"Homepage": "https://github.com/prakhara56/FlexiStore",
"Repository": "https://github.com/prakhara56/FlexiStore"
},
"split_keywords": [
"storage",
" cloud",
" azure",
" aws",
" s3",
" blob",
" async"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "fb8c1c48ade6fd48bd6e7febe50c2bf390cd998ade85c301f6e88a4e6f4efab8",
"md5": "06edd960353ddd1402f65bded704a8f2",
"sha256": "f6f6643c84eec334730677cc986180d227ee62774706ce7a6b5b3355d2b107dc"
},
"downloads": -1,
"filename": "flexistore-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "06edd960353ddd1402f65bded704a8f2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 35362,
"upload_time": "2025-08-23T22:44:55",
"upload_time_iso_8601": "2025-08-23T22:44:55.203768Z",
"url": "https://files.pythonhosted.org/packages/fb/8c/1c48ade6fd48bd6e7febe50c2bf390cd998ade85c301f6e88a4e6f4efab8/flexistore-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "dd2f565e51c98dfeffec12f380d6272618427d9c579598f8ad8db6f30c5cc4c2",
"md5": "8c8e23413561dc89281393202fb4998a",
"sha256": "e4483f422703e70220e4fa179e45c43b65f25bd81b91c6da0bf31b50c68dea4d"
},
"downloads": -1,
"filename": "flexistore-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "8c8e23413561dc89281393202fb4998a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 33941,
"upload_time": "2025-08-23T22:44:56",
"upload_time_iso_8601": "2025-08-23T22:44:56.992675Z",
"url": "https://files.pythonhosted.org/packages/dd/2f/565e51c98dfeffec12f380d6272618427d9c579598f8ad8db6f30c5cc4c2/flexistore-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-23 22:44:56",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "prakhara56",
"github_project": "FlexiStore",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "flexistore"
}
JSON
