HybridContents
======================
The `HybridContentManager` was originally created by [Quantopian](https://www.quantopian.com/) as part of [pgcontents](https://github.com/quantopian/pgcontents); however, the usage of `HybridContentsManager` was restricted to the compatibility requirements of [pgcontents](https://github.com/quantopian/pgcontents). These restrictions included `postgres` dependencies and no support for the latest [notebook](https://pypi.org/project/notebook/) version (>6).
At [Viaduct](https://viaduct.ai) we used [pgcontents](https://github.com/quantopian/pgcontents) exclusively for the `HybridContentsManager` and wanted to extend its functionality, so we created this fork [hybridcontents](https://github.com/viaduct-ai/hybridcontents).
See related [pgcontents](https://github.com/quantopian/pgcontents) issues:
- https://github.com/quantopian/pgcontents/issues/66
- https://github.com/quantopian/pgcontents/issues/50
- https://github.com/quantopian/pgcontents/issues/28
Getting Started
---------------
**Prerequisites:**
- A Python installation with [Jupyter Notebook](https://github.com/jupyter/notebook) >= 4.0.
**Installation:**
#### [pip](https://pypi.org/project/hybridcontents/)
```bash
pip install hybridcontents
```
#### [Anaconda](https://anaconda.org/viaduct/hybridcontents)
```bash
conda install -c viaduct hybridcontents
```
#### [conda-forge](https://github.com/conda-forge/hybridcontents-feedstock)
See instructions [here](https://github.com/conda-forge/hybridcontents-feedstock#installing-hybridcontents)
Features
-----
- Mix and match different content managers for different directories
- Easily move files between different content managers (i.e local files to s3 backed manager)
- Path validation to keep consistent naming scheme and/or prevent illegal characters
Usage
-----
For a detailed example see, [hybrid_manager_example.py](https://github.com/viaduct-ai/hybridcontents/blob/master/examples/hybrid_manager_example.py)
The following code snippet creates a HybridContentsManager with two directories with different content managers.
```python
c = get_config()
c.NotebookApp.contents_manager_class = HybridContentsManager
c.HybridContentsManager.manager_classes = {
# NOTE: LargFileManager only exists in notebook > 5
# If using notebook < 5, use FileContentManager instead
"": LargeFileManager,
"shared": S3ContentsManager
}
# Each item will be passed to the constructor of the appropriate content manager.
c.HybridContentsManager.manager_kwargs = {
# Args for root LargeFileManager
"": {
"root_dir": read_only_dir
},
# Args for the shared S3ContentsManager directory
"shared": {
"access_key_id": ...,
"secret_access_key": ...,
"endpoint_url": ...,
"bucket": ...,
"prefix": ...
},
}
def only_allow_notebooks(path):
return path.endswith('.ipynb')
# Only allow notebook files to be stored in S3
c.HybridContentsManager.path_validators = {
"shared": only_allow_notebooks
}
```
Testing
-------
To run unit tests,
```bash
tox
```
This will run all unit tests for python versions 2.7, 3.6, 3.7 and jupyter notebook versions 4, 5, and 6.
### Publishing a Release
1. Create a new release on Github
2. Update the version in `setup.py`
3. Run ./scripts/pip_publish.sh
4. Update the version `meta.yaml`
5. Update the [sha256 in meta.yaml](https://github.com/conda-forge/staged-recipes/wiki/Frequently-asked-questions#2-how-do-i-populate-the-hash-field)
6. Run ./scripts/anaconda_publish.sh
7. Update on Conda Forge
Raw data
{
"_id": null,
"home_page": "https://github.com/viaduct-ai/hybridcontents",
"name": "hybridcontents",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "jupyterhub,pgcontents,hybridcontents,content manager,hybridcontentmanager",
"author": "viaduct",
"author_email": "engineering@viaduct.ai",
"download_url": "https://files.pythonhosted.org/packages/82/52/4f0319d7470184ca32b06ed48c9c69231699d2105e616176598cdfd7bf67/hybridcontents-0.4.0.tar.gz",
"platform": null,
"description": "HybridContents\n======================\n\nThe `HybridContentManager` was originally created by [Quantopian](https://www.quantopian.com/) as part of [pgcontents](https://github.com/quantopian/pgcontents); however, the usage of `HybridContentsManager` was restricted to the compatibility requirements of [pgcontents](https://github.com/quantopian/pgcontents). These restrictions included `postgres` dependencies and no support for the latest [notebook](https://pypi.org/project/notebook/) version (>6).\n\nAt [Viaduct](https://viaduct.ai) we used [pgcontents](https://github.com/quantopian/pgcontents) exclusively for the `HybridContentsManager` and wanted to extend its functionality, so we created this fork [hybridcontents](https://github.com/viaduct-ai/hybridcontents).\n\nSee related [pgcontents](https://github.com/quantopian/pgcontents) issues:\n- https://github.com/quantopian/pgcontents/issues/66\n- https://github.com/quantopian/pgcontents/issues/50\n- https://github.com/quantopian/pgcontents/issues/28\n\nGetting Started\n---------------\n**Prerequisites:**\n - A Python installation with [Jupyter Notebook](https://github.com/jupyter/notebook) >= 4.0.\n\n**Installation:**\n\n#### [pip](https://pypi.org/project/hybridcontents/)\n```bash\npip install hybridcontents\n```\n#### [Anaconda](https://anaconda.org/viaduct/hybridcontents)\n```bash\nconda install -c viaduct hybridcontents\n```\n#### [conda-forge](https://github.com/conda-forge/hybridcontents-feedstock)\nSee instructions [here](https://github.com/conda-forge/hybridcontents-feedstock#installing-hybridcontents)\n\nFeatures\n-----\n- Mix and match different content managers for different directories \n- Easily move files between different content managers (i.e local files to s3 backed manager) \n- Path validation to keep consistent naming scheme and/or prevent illegal characters\n\nUsage\n-----\nFor a detailed example see, [hybrid_manager_example.py](https://github.com/viaduct-ai/hybridcontents/blob/master/examples/hybrid_manager_example.py)\n\nThe following code snippet creates a HybridContentsManager with two directories with different content managers. \n\n```python\nc = get_config()\n\nc.NotebookApp.contents_manager_class = HybridContentsManager\n\nc.HybridContentsManager.manager_classes = {\n # NOTE: LargFileManager only exists in notebook > 5\n # If using notebook < 5, use FileContentManager instead\n \"\": LargeFileManager,\n \"shared\": S3ContentsManager\n}\n\n# Each item will be passed to the constructor of the appropriate content manager.\nc.HybridContentsManager.manager_kwargs = {\n # Args for root LargeFileManager\n \"\": {\n \"root_dir\": read_only_dir\n },\n # Args for the shared S3ContentsManager directory\n \"shared\": {\n \"access_key_id\": ...,\n \"secret_access_key\": ...,\n \"endpoint_url\": ...,\n \"bucket\": ...,\n \"prefix\": ...\n },\n}\n\ndef only_allow_notebooks(path):\n return path.endswith('.ipynb')\n\n# Only allow notebook files to be stored in S3\nc.HybridContentsManager.path_validators = {\n \"shared\": only_allow_notebooks\n}\n```\n\n\nTesting\n-------\nTo run unit tests, \n\n```bash\ntox\n```\n\nThis will run all unit tests for python versions 2.7, 3.6, 3.7 and jupyter notebook versions 4, 5, and 6.\n\n### Publishing a Release\n\n1. Create a new release on Github\n2. Update the version in `setup.py`\n3. Run ./scripts/pip_publish.sh\n4. Update the version `meta.yaml`\n5. Update the [sha256 in meta.yaml](https://github.com/conda-forge/staged-recipes/wiki/Frequently-asked-questions#2-how-do-i-populate-the-hash-field)\n6. Run ./scripts/anaconda_publish.sh\n7. Update on Conda Forge\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "Hybrid Content Manager",
"version": "0.4.0",
"project_urls": {
"Download": "https://github.com/viaduct-ai/hybridcontents/archive/v0.4.0.tar.gz",
"Homepage": "https://github.com/viaduct-ai/hybridcontents"
},
"split_keywords": [
"jupyterhub",
"pgcontents",
"hybridcontents",
"content manager",
"hybridcontentmanager"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "82524f0319d7470184ca32b06ed48c9c69231699d2105e616176598cdfd7bf67",
"md5": "8f9e8e3195edd81725beeff3ce9bdc21",
"sha256": "22363ca091feeb814ebe321790f58910d1e9fa2fa4dde126d9c4dae92282f176"
},
"downloads": -1,
"filename": "hybridcontents-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "8f9e8e3195edd81725beeff3ce9bdc21",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 15084,
"upload_time": "2023-05-16T01:52:15",
"upload_time_iso_8601": "2023-05-16T01:52:15.660229Z",
"url": "https://files.pythonhosted.org/packages/82/52/4f0319d7470184ca32b06ed48c9c69231699d2105e616176598cdfd7bf67/hybridcontents-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-16 01:52:15",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "viaduct-ai",
"github_project": "hybridcontents",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"tox": true,
"lcname": "hybridcontents"
}
JSON
