# Agori
Agori is a secure Python package that provides encrypted document storage and semantic search capabilities using ChromaDB and Azure OpenAI embeddings. It focuses on secure storage and retrieval of sensitive documents while maintaining searchability through encrypted vector embeddings.
## Features
- 🔐 End-to-end encryption for documents and metadata
- 🔍 Semantic search using Azure OpenAI embeddings
- 📚 Multiple collection management within a database
- 💾 Persistent storage with database isolation
- 🚀 Simple and intuitive API
- 🛡️ Comprehensive error handling
- 📝 Detailed logging
- 🧹 Automatic resource cleanup
## Installation
```bash
pip install agori
```
## Quick Start
```python
from agori.core import WorkingMemory
from cryptography.fernet import Fernet
# Generate a new encryption key (in practice, store this securely)
encryption_key = Fernet.generate_key()
# Initialize the secure database
db = WorkingMemory(
api_key="your-azure-openai-key",
api_endpoint="your-azure-endpoint",
encryption_key=encryption_key,
db_unique_id="my_secure_db"
)
# Create a new collection
collection_metadata = {
"description": "Research papers database",
"owner": "research_team"
}
collection = db.create_collection("research_papers", metadata=collection_metadata)
# Add documents with metadata
documents = [
"Advances in Neural Networks",
"Quantum Computing Overview",
"Machine Learning Applications"
]
metadata_list = [
{"author": "John Doe", "year": "2023"},
{"author": "Jane Smith", "year": "2023"},
{"author": "Bob Wilson", "year": "2024"}
]
# Add documents - they will be automatically encrypted
doc_ids = db.add_documents(
collection_name="research_papers",
documents=documents,
metadatas=metadata_list
)
# Query the collection - results will be automatically decrypted
results = db.query_collection(
collection_name="research_papers",
query_texts=["neural networks"],
n_results=2
)
# Process results
for i, (doc, distance) in enumerate(zip(results["documents"][0], results["distances"][0])):
print(f"Result {i+1}:")
print(f"Document: {doc}")
print(f"Similarity Score: {1 - distance}") # Convert distance to similarity
if "metadatas" in results:
print(f"Metadata: {results['metadatas'][0][i]}")
print()
#cleanup the collection and db
db.drop_collection("research_papers")
db.cleanup_database()
```
## Security Features
### Encryption
- All documents and metadata are encrypted using Fernet symmetric encryption
- Secure key generation and management required
- Encrypted storage of documents and metadata
### Database Isolation
- Each database instance has a unique ID
- Separate storage paths for different databases
- Secure cleanup of resources
## Development
To set up the development environment:
```bash
# Clone the repository
git clone https://github.com/govindshukl/agori.git
cd agori
# Create and activate virtual environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install development dependencies
pip install -r requirements-dev.txt
# Install the package in editable mode
pip install -e .
```
### Testing and Quality Assurance
```bash
# Run tests
pytest tests -v --cov=agori
# Code formatting
black src/agori tests
isort src/agori tests
# Linting
flake8 src/agori tests
mypy src/agori tests
```
## Requirements
- Python 3.8 or higher
- Azure OpenAI API access
- Required packages:
- chromadb
- cryptography
- azure-openai
## Best Practices
### Security
1. Never hardcode encryption keys or API credentials
2. Use environment variables for sensitive information
3. Implement proper key management
4. Regular cleanup of sensitive data
### Resource Management
1. Use context managers for automatic cleanup
2. Properly handle collection lifecycle
3. Implement error handling for all operations
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
1. Fork the repository
2. Create your feature branch (`git checkout -b feature/NewFeature`)
3. Commit your changes (`git commit -m 'Add NewFeature'`)
4. Push to the branch (`git push origin feature/NewFeature`)
5. Open a Pull Request
## License
This project is licensed under the MIT License - see the LICENSE file for details.
## Support
If you encounter any issues or need support, please:
1. Check the [documentation](https://github.com/govindshukl/agori/docs)
2. Search through [existing issues](https://github.com/govindshukl/agori/issues)
3. Open a new issue if needed
## Acknowledgments
- ChromaDB for vector database functionality
- Azure OpenAI for embeddings generation
- Cryptography.io for encryption capabilities
Raw data
{
"_id": null,
"home_page": null,
"name": "agori",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "azure, chromadb, embeddings, encryption, openai, semantic-search, working memory",
"author": null,
"author_email": "Govind Shukla <govind.shukl@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/8f/f0/e4f26f4710d0087a41887c8bfc8b9fb3a6cb68afee280139dee57b9e81c3/agori-0.1.3.tar.gz",
"platform": null,
"description": "# Agori\n\nAgori is a secure Python package that provides encrypted document storage and semantic search capabilities using ChromaDB and Azure OpenAI embeddings. It focuses on secure storage and retrieval of sensitive documents while maintaining searchability through encrypted vector embeddings.\n\n## Features\n\n- \ud83d\udd10 End-to-end encryption for documents and metadata\n- \ud83d\udd0d Semantic search using Azure OpenAI embeddings\n- \ud83d\udcda Multiple collection management within a database\n- \ud83d\udcbe Persistent storage with database isolation\n- \ud83d\ude80 Simple and intuitive API\n- \ud83d\udee1\ufe0f Comprehensive error handling\n- \ud83d\udcdd Detailed logging\n- \ud83e\uddf9 Automatic resource cleanup\n\n## Installation\n\n```bash\npip install agori\n```\n\n## Quick Start\n\n```python\nfrom agori.core import WorkingMemory\nfrom cryptography.fernet import Fernet\n\n# Generate a new encryption key (in practice, store this securely)\nencryption_key = Fernet.generate_key()\n\n# Initialize the secure database\ndb = WorkingMemory(\n api_key=\"your-azure-openai-key\",\n api_endpoint=\"your-azure-endpoint\",\n encryption_key=encryption_key,\n db_unique_id=\"my_secure_db\"\n)\n\n# Create a new collection\ncollection_metadata = {\n \"description\": \"Research papers database\",\n \"owner\": \"research_team\"\n}\ncollection = db.create_collection(\"research_papers\", metadata=collection_metadata)\n\n# Add documents with metadata\ndocuments = [\n \"Advances in Neural Networks\",\n \"Quantum Computing Overview\",\n \"Machine Learning Applications\"\n]\nmetadata_list = [\n {\"author\": \"John Doe\", \"year\": \"2023\"},\n {\"author\": \"Jane Smith\", \"year\": \"2023\"},\n {\"author\": \"Bob Wilson\", \"year\": \"2024\"}\n]\n\n# Add documents - they will be automatically encrypted\ndoc_ids = db.add_documents(\n collection_name=\"research_papers\",\n documents=documents,\n metadatas=metadata_list\n)\n\n# Query the collection - results will be automatically decrypted\nresults = db.query_collection(\n collection_name=\"research_papers\",\n query_texts=[\"neural networks\"],\n n_results=2\n)\n\n# Process results\nfor i, (doc, distance) in enumerate(zip(results[\"documents\"][0], results[\"distances\"][0])):\n print(f\"Result {i+1}:\")\n print(f\"Document: {doc}\")\n print(f\"Similarity Score: {1 - distance}\") # Convert distance to similarity\n if \"metadatas\" in results:\n print(f\"Metadata: {results['metadatas'][0][i]}\")\n print()\n\n#cleanup the collection and db\ndb.drop_collection(\"research_papers\")\ndb.cleanup_database()\n\n```\n\n\n## Security Features\n\n### Encryption\n- All documents and metadata are encrypted using Fernet symmetric encryption\n- Secure key generation and management required\n- Encrypted storage of documents and metadata\n\n### Database Isolation\n- Each database instance has a unique ID\n- Separate storage paths for different databases\n- Secure cleanup of resources\n\n## Development\n\nTo set up the development environment:\n\n```bash\n# Clone the repository\ngit clone https://github.com/govindshukl/agori.git\ncd agori\n\n# Create and activate virtual environment\npython -m venv venv\nsource venv/bin/activate # On Windows: venv\\Scripts\\activate\n\n# Install development dependencies\npip install -r requirements-dev.txt\n\n# Install the package in editable mode\npip install -e .\n```\n\n### Testing and Quality Assurance\n\n```bash\n# Run tests\npytest tests -v --cov=agori\n\n# Code formatting\nblack src/agori tests\nisort src/agori tests\n\n# Linting\nflake8 src/agori tests\nmypy src/agori tests\n```\n\n## Requirements\n\n- Python 3.8 or higher\n- Azure OpenAI API access\n- Required packages:\n - chromadb\n - cryptography\n - azure-openai\n\n## Best Practices\n\n### Security\n1. Never hardcode encryption keys or API credentials\n2. Use environment variables for sensitive information\n3. Implement proper key management\n4. Regular cleanup of sensitive data\n\n### Resource Management\n1. Use context managers for automatic cleanup\n2. Properly handle collection lifecycle\n3. Implement error handling for all operations\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/NewFeature`)\n3. Commit your changes (`git commit -m 'Add NewFeature'`)\n4. Push to the branch (`git push origin feature/NewFeature`)\n5. Open a Pull Request\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n\n## Support\n\nIf you encounter any issues or need support, please:\n\n1. Check the [documentation](https://github.com/govindshukl/agori/docs)\n2. Search through [existing issues](https://github.com/govindshukl/agori/issues)\n3. Open a new issue if needed\n\n## Acknowledgments\n\n- ChromaDB for vector database functionality\n- Azure OpenAI for embeddings generation\n- Cryptography.io for encryption capabilities",
"bugtrack_url": null,
"license": null,
"summary": "A secure working memory for agents",
"version": "0.1.3",
"project_urls": {
"Bug Tracker": "https://github.com/govindshukl/agori/issues",
"Documentation": "https://github.com/govindshukl/agori#readme",
"Homepage": "https://github.com/govindshukl/agori",
"Repository": "https://github.com/govindshukl/agori.git"
},
"split_keywords": [
"azure",
" chromadb",
" embeddings",
" encryption",
" openai",
" semantic-search",
" working memory"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "04b684dc1ae7b47c4635f568d3980f52b752c615c7fe9ea152301cf10afd34ab",
"md5": "cdf2800b46edff27925b70f8ce7fd3c1",
"sha256": "1af808a8bef292f27c221b0ae01399b1fa9eb7608a020e7ea9c571a4c4c5c2e0"
},
"downloads": -1,
"filename": "agori-0.1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "cdf2800b46edff27925b70f8ce7fd3c1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 9245,
"upload_time": "2024-11-10T19:54:38",
"upload_time_iso_8601": "2024-11-10T19:54:38.546497Z",
"url": "https://files.pythonhosted.org/packages/04/b6/84dc1ae7b47c4635f568d3980f52b752c615c7fe9ea152301cf10afd34ab/agori-0.1.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8ff0e4f26f4710d0087a41887c8bfc8b9fb3a6cb68afee280139dee57b9e81c3",
"md5": "ce1d39c958808ddfa61427ddd6d3ef62",
"sha256": "de4f8c1278b27d5ce20f05f7206841fb29ebff2a5f023aae53aa3b868c159ea5"
},
"downloads": -1,
"filename": "agori-0.1.3.tar.gz",
"has_sig": false,
"md5_digest": "ce1d39c958808ddfa61427ddd6d3ef62",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 13475,
"upload_time": "2024-11-10T19:54:40",
"upload_time_iso_8601": "2024-11-10T19:54:40.446059Z",
"url": "https://files.pythonhosted.org/packages/8f/f0/e4f26f4710d0087a41887c8bfc8b9fb3a6cb68afee280139dee57b9e81c3/agori-0.1.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-10 19:54:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "govindshukl",
"github_project": "agori",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "agori"
}
JSON
