<p align="center">
<img src="https://i.ibb.co/BGVBmMK/opal.png" height=170 alt="opal" border="0" />
</p>
<h2 align="center">
OPAL Fetcher for MongoDB
</h2>
<h4 align="center">
Made with ❤️ at <a href="https://treedom.net"><img src="https://i.ibb.co/QfYVtP5/Treedom-logo.png" height="24" alt="treedom" border="0" /></a>
</h4>
<h6 align="center">
<a href="https://www.treedom.net/it/organization/treedom/event/treedom-open-source"><img src="https://badges.treedom.net/badge/f/treedom-open-source" alt="plant-a-tree" border="0" /></a>
</h6>
[Check out OPAL main repo here.](https://github.com/permitio/opal)
### What's in this repo?
An OPAL [custom fetch provider](https://docs.opal.ac/tutorials/write_your_own_fetch_provider) to bring authorization state from [MongoDB](https://www.mongodb.com/).
### How to try this custom fetcher in one command? (Example docker-compose configuration)
You can test this fetcher with the example docker compose file in this repository root. Clone this repo, `cd` into the cloned repo, and then run:
```
docker compose up
```
this docker compose configuration already correctly configures OPAL to load the MongoDB Fetch Provider, and correctly configures `OPAL_DATA_CONFIG_SOURCES` to include an entry that uses this fetcher.
### ✏️ How to use this fetcher in your OPAL Setup
#### 1) Build a custom opal-client `Dockerfile`
The official docker image only contains the built-in fetch providers. You need to create your own `Dockerfile` (that is based on the official docker image), that includes this fetcher's pip package.
Your `Dockerfile` should look like this:
```
FROM permitio/opal-client:latest
RUN pip install --no-cache-dir --user opal-fetcher-mongodb-treedom
```
#### 2) Build your custom opal-client container
Say your special Dockerfile from step one is called `custom_client.Dockerfile`.
You must build a customized OPAL container from this Dockerfile, like so:
```
docker build -t yourcompany/opal-client -f custom_client.Dockerfile .
```
#### 3) When running OPAL, set `OPAL_FETCH_PROVIDER_MODULES`
Pass the following environment variable to the OPAL client docker container (comma-separated provider modules):
```
OPAL_FETCH_PROVIDER_MODULES=opal_common.fetcher.providers,opal_fetcher_mongodb.provider
```
Notice that OPAL receives a list from where to search for fetch providers.
The list in our case includes the built-in providers (`opal_common.fetcher.providers`) and our custom MongoDB provider.
#### 4) Using the custom provider in your DataSourceEntry objects
Your DataSourceEntry objects (either in `OPAL_DATA_CONFIG_SOURCES` or in dynamic updates sent via the OPAL publish API) can now include this fetcher's config.
Example value of `OPAL_DATA_CONFIG_SOURCES` (formatted nicely, but in env var you should pack this to one-line and no-spaces):
```json
{
"config": {
"entries": [
{
"url": "mongodb://user:password@mongodb/test_database?authSource=admin",
"config": {
"fetcher": "MongoDBFetchProvider",
"database": "opal_fetcher_mongodb",
"collection": "cities_collection",
"find": { "query": {} }
},
"topics": ["policy_data"],
"dst_path": "cities"
}
]
}
}
```
Notice how `config` is an instance of `MongoDBFetchProvider` (code is in `opal_fetcher_mongodb/provider.py`).
Values for this fetcher config:
* The `url` is actually a MongoDB uri.
* Your `config` must include the `fetcher` key to indicate to OPAL that you use a custom fetcher.
* Your `config` must include the `collection` key to indicate what collection to query in MongoDB.
* Your `config` may include the `database` key to indicate what database to query in MongoDB. If not specified, the default database will be used.
* Your `config` must include one of `findOne`, `find` or `aggregate` keys to indicate what query to run against MongoDB.
* Your `config` may include the `transform` key to transform the results from the `find` or `aggregate` queries.
#### Query methods
All the three available query methods accept the same input parameters as defined in the MongoDB documentation.
##### findOne
* `findOne` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.findOne/)
<details>
<summary>Example configuration</summary>
```json
{
"config": {
"entries": [
{
...
"config": {
...
"findOne": {
"query": {
...
},
"projection": {
...
},
"options": {
...
}
}
}
}
]
}
}
```
</details>
##### find
* `find` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.find/)
<details>
<summary>Example configuration</summary>
```json
{
"config": {
"entries": [
{
...
"config": {
...
"find": {
"query": {
...
},
"projection": {
...
},
"options": {
...
}
},
"transform": {
"first": false,
"mapKey": "",
"merge": true
}
}
}
]
}
}
```
</details>
##### aggregate
* `aggregate` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/)
<details>
<summary>Example configuration</summary>
```json
{
"config": {
"entries": [
{
...
"config": {
...
"aggregate": {
"pipeline": [
...
],
"options": {
...
}
},
"transform": {
"first": false,
"mapKey": ""
}
}
}
]
}
}
```
</details>
#### Query transform
`transform` allows you to transform the results from the `find` or `aggregate` queries.
##### first
`transform.first` allows you to return only the first result from the query.
Equivalent to the following Python code:
```python
result = query_result[0]
```
##### mapKey
`transform.mapKey` allows you to map the original list-like result to a dictionary-like result using the property specified in the `mapKey` as the key for the dictionary.
Equivalent to the following Python code:
```python
result = {}
for item in query_result:
result[item['key']] = item
```
> Only properties in the root of the document can be used as the key for the dictionary.
##### merge
`transform.merge` allows you to merge the results from the query into a single document. Duplicate keys will be overwritten by the last document in the list.
Equivalent to the following Python code:
```python
result = {}
for item in query_result:
for key, value in item.items():
result[key] = value
```
Raw data
{
"_id": null,
"home_page": null,
"name": "opal-fetcher-mongodb-treedom",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "Open Policy Agent, OPA, OPAL, Open Policy Administration Layer, MongoDB, Permit.io",
"author": null,
"author_email": "Ionut Andrei Oanca <ia.oanca@treedom.net>",
"download_url": "https://files.pythonhosted.org/packages/cb/2c/69a6760297d2ce596f6a205472ff78da380b60a142784deea3fc428a2ddf/opal_fetcher_mongodb_treedom-0.0.1.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <img src=\"https://i.ibb.co/BGVBmMK/opal.png\" height=170 alt=\"opal\" border=\"0\" />\n</p>\n<h2 align=\"center\">\nOPAL Fetcher for MongoDB\n</h2>\n\n<h4 align=\"center\">\nMade with \u2764\ufe0f at <a href=\"https://treedom.net\"><img src=\"https://i.ibb.co/QfYVtP5/Treedom-logo.png\" height=\"24\" alt=\"treedom\" border=\"0\" /></a>\n</h4>\n\n<h6 align=\"center\">\n<a href=\"https://www.treedom.net/it/organization/treedom/event/treedom-open-source\"><img src=\"https://badges.treedom.net/badge/f/treedom-open-source\" alt=\"plant-a-tree\" border=\"0\" /></a>\n</h6>\n\n[Check out OPAL main repo here.](https://github.com/permitio/opal)\n\n\n### What's in this repo?\nAn OPAL [custom fetch provider](https://docs.opal.ac/tutorials/write_your_own_fetch_provider) to bring authorization state from [MongoDB](https://www.mongodb.com/).\n\n### How to try this custom fetcher in one command? (Example docker-compose configuration)\n\nYou can test this fetcher with the example docker compose file in this repository root. Clone this repo, `cd` into the cloned repo, and then run:\n```\ndocker compose up\n```\nthis docker compose configuration already correctly configures OPAL to load the MongoDB Fetch Provider, and correctly configures `OPAL_DATA_CONFIG_SOURCES` to include an entry that uses this fetcher.\n\n### \u270f\ufe0f How to use this fetcher in your OPAL Setup\n\n#### 1) Build a custom opal-client `Dockerfile`\n\nThe official docker image only contains the built-in fetch providers. You need to create your own `Dockerfile` (that is based on the official docker image), that includes this fetcher's pip package.\n\nYour `Dockerfile` should look like this:\n```\nFROM permitio/opal-client:latest\nRUN pip install --no-cache-dir --user opal-fetcher-mongodb-treedom\n```\n\n#### 2) Build your custom opal-client container\nSay your special Dockerfile from step one is called `custom_client.Dockerfile`.\n\nYou must build a customized OPAL container from this Dockerfile, like so:\n```\ndocker build -t yourcompany/opal-client -f custom_client.Dockerfile .\n```\n\n#### 3) When running OPAL, set `OPAL_FETCH_PROVIDER_MODULES`\nPass the following environment variable to the OPAL client docker container (comma-separated provider modules):\n```\nOPAL_FETCH_PROVIDER_MODULES=opal_common.fetcher.providers,opal_fetcher_mongodb.provider\n```\nNotice that OPAL receives a list from where to search for fetch providers.\nThe list in our case includes the built-in providers (`opal_common.fetcher.providers`) and our custom MongoDB provider.\n\n#### 4) Using the custom provider in your DataSourceEntry objects\n\nYour DataSourceEntry objects (either in `OPAL_DATA_CONFIG_SOURCES` or in dynamic updates sent via the OPAL publish API) can now include this fetcher's config.\n\nExample value of `OPAL_DATA_CONFIG_SOURCES` (formatted nicely, but in env var you should pack this to one-line and no-spaces):\n```json\n{\n \"config\": {\n \"entries\": [\n {\n \"url\": \"mongodb://user:password@mongodb/test_database?authSource=admin\",\n \"config\": {\n \"fetcher\": \"MongoDBFetchProvider\",\n \"database\": \"opal_fetcher_mongodb\",\n \"collection\": \"cities_collection\",\n \"find\": { \"query\": {} }\n },\n \"topics\": [\"policy_data\"],\n \"dst_path\": \"cities\"\n }\n ]\n }\n}\n```\n\nNotice how `config` is an instance of `MongoDBFetchProvider` (code is in `opal_fetcher_mongodb/provider.py`).\n\nValues for this fetcher config:\n* The `url` is actually a MongoDB uri.\n* Your `config` must include the `fetcher` key to indicate to OPAL that you use a custom fetcher.\n* Your `config` must include the `collection` key to indicate what collection to query in MongoDB.\n* Your `config` may include the `database` key to indicate what database to query in MongoDB. If not specified, the default database will be used.\n* Your `config` must include one of `findOne`, `find` or `aggregate` keys to indicate what query to run against MongoDB.\n* Your `config` may include the `transform` key to transform the results from the `find` or `aggregate` queries.\n\n#### Query methods\nAll the three available query methods accept the same input parameters as defined in the MongoDB documentation.\n\n##### findOne\n\n* `findOne` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.findOne/)\n\n<details>\n <summary>Example configuration</summary>\n\n```json\n{\n \"config\": {\n \"entries\": [\n {\n ...\n \"config\": {\n ...\n \"findOne\": {\n \"query\": {\n ...\n },\n \"projection\": {\n ...\n },\n \"options\": {\n ...\n }\n }\n }\n }\n ]\n }\n}\n```\n</details>\n\n##### find\n\n* `find` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.find/)\n\n<details>\n <summary>Example configuration</summary>\n\n```json\n{\n \"config\": {\n \"entries\": [\n {\n ...\n \"config\": {\n ...\n \"find\": {\n \"query\": {\n ...\n },\n \"projection\": {\n ...\n },\n \"options\": {\n ...\n }\n },\n \"transform\": {\n \"first\": false,\n \"mapKey\": \"\",\n \"merge\": true\n }\n }\n }\n ]\n }\n}\n```\n</details>\n\n##### aggregate\n\n* `aggregate` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/)\n\n<details>\n <summary>Example configuration</summary>\n\n```json\n{\n \"config\": {\n \"entries\": [\n {\n ...\n \"config\": {\n ...\n \"aggregate\": {\n \"pipeline\": [\n ...\n ],\n \"options\": {\n ...\n }\n },\n \"transform\": {\n \"first\": false,\n \"mapKey\": \"\"\n }\n }\n }\n ]\n }\n}\n```\n</details>\n\n#### Query transform\n`transform` allows you to transform the results from the `find` or `aggregate` queries.\n\n##### first\n`transform.first` allows you to return only the first result from the query.\n\nEquivalent to the following Python code:\n\n```python\nresult = query_result[0]\n```\n\n##### mapKey\n`transform.mapKey` allows you to map the original list-like result to a dictionary-like result using the property specified in the `mapKey` as the key for the dictionary.\n\nEquivalent to the following Python code:\n\n```python\nresult = {}\nfor item in query_result:\n result[item['key']] = item\n```\n\n> Only properties in the root of the document can be used as the key for the dictionary.\n\n##### merge\n\n`transform.merge` allows you to merge the results from the query into a single document. Duplicate keys will be overwritten by the last document in the list.\n\nEquivalent to the following Python code:\n\n```python\nresult = {}\nfor item in query_result:\n for key, value in item.items():\n result[key] = value\n```\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "An OPAL fetch provider to bring authorization state from MongoDB.",
"version": "0.0.1",
"project_urls": {
"Bug Tracker": "https://github.com/treedomtrees/opal-fetcher-mongodb/issues",
"Source": "https://github.com/treedomtrees/opal-fetcher-mongodb"
},
"split_keywords": [
"open policy agent",
" opa",
" opal",
" open policy administration layer",
" mongodb",
" permit.io"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "914b4a22105a74fa5b7bc92f6f793562faa5aa1689307466b487d862f733c588",
"md5": "ec0681cbf06427a08f24101027eb9c46",
"sha256": "17159a3cadaea7db8f8245f84da6e67118ffdb21854c289f21c7f3e9fa9ddb78"
},
"downloads": -1,
"filename": "opal_fetcher_mongodb_treedom-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ec0681cbf06427a08f24101027eb9c46",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 11067,
"upload_time": "2024-04-16T08:52:43",
"upload_time_iso_8601": "2024-04-16T08:52:43.695010Z",
"url": "https://files.pythonhosted.org/packages/91/4b/4a22105a74fa5b7bc92f6f793562faa5aa1689307466b487d862f733c588/opal_fetcher_mongodb_treedom-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "cb2c69a6760297d2ce596f6a205472ff78da380b60a142784deea3fc428a2ddf",
"md5": "5d087e2eee1df787e7798319b8255c96",
"sha256": "a4456f0e3f1a84f0d7739450295a0ae29a5875beaab6c1fed55d2790321a9468"
},
"downloads": -1,
"filename": "opal_fetcher_mongodb_treedom-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "5d087e2eee1df787e7798319b8255c96",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 17139,
"upload_time": "2024-04-16T08:52:45",
"upload_time_iso_8601": "2024-04-16T08:52:45.272257Z",
"url": "https://files.pythonhosted.org/packages/cb/2c/69a6760297d2ce596f6a205472ff78da380b60a142784deea3fc428a2ddf/opal_fetcher_mongodb_treedom-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-16 08:52:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "treedomtrees",
"github_project": "opal-fetcher-mongodb",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "opal-fetcher-mongodb-treedom"
}
JSON
