<p align="center">
<img src="https://i.ibb.co/BGVBmMK/opal.png" height=170 alt="opal" border="0" />
</p>
<h2 align="center">
OPAL Fetcher for MongoDB
</h2>
<h4 align="center">
Made with ❤️ at <a href="https://treedom.net"><img src="https://i.ibb.co/QfYVtP5/Treedom-logo.png" height="24" alt="treedom" border="0" /></a>
</h4>
<h6 align="center">
<a href="https://www.treedom.net/it/organization/treedom/event/treedom-open-source"><img src="https://badges.treedom.net/badge/f/treedom-open-source" alt="plant-a-tree" border="0" /></a>
</h6>
[Check out OPAL main repo here.](https://github.com/permitio/opal)
### What's in this repo?
An OPAL [custom fetch provider](https://docs.opal.ac/tutorials/write_your_own_fetch_provider) to bring authorization state from [MongoDB](https://www.mongodb.com/).
### How to try this custom fetcher in one command? (Example docker-compose configuration)
You can test this fetcher with the example docker compose file in this repository root. Clone this repo, `cd` into the cloned repo, and then run:
```
docker compose up
```
this docker compose configuration already correctly configures OPAL to load the MongoDB Fetch Provider, and correctly configures `OPAL_DATA_CONFIG_SOURCES` to include an entry that uses this fetcher.
### ✏️ How to use this fetcher in your OPAL Setup
#### 1) Build a custom opal-client `Dockerfile`
The official docker image only contains the built-in fetch providers. You need to create your own `Dockerfile` (that is based on the official docker image), that includes this fetcher's pip package.
Your `Dockerfile` should look like this:
```
FROM permitio/opal-client:latest
RUN pip install --no-cache-dir --user opal-fetcher-mongodb
```
#### 2) Build your custom opal-client container
Say your special Dockerfile from step one is called `custom_client.Dockerfile`.
You must build a customized OPAL container from this Dockerfile, like so:
```
docker build -t yourcompany/opal-client -f custom_client.Dockerfile .
```
#### 3) When running OPAL, set `OPAL_FETCH_PROVIDER_MODULES`
Pass the following environment variable to the OPAL client docker container (comma-separated provider modules):
```
OPAL_FETCH_PROVIDER_MODULES=opal_common.fetcher.providers,opal_fetcher_mongodb.provider
```
Notice that OPAL receives a list from where to search for fetch providers.
The list in our case includes the built-in providers (`opal_common.fetcher.providers`) and our custom MongoDB provider.
#### 4) Using the custom provider in your DataSourceEntry objects
Your DataSourceEntry objects (either in `OPAL_DATA_CONFIG_SOURCES` or in dynamic updates sent via the OPAL publish API) can now include this fetcher's config.
Example value of `OPAL_DATA_CONFIG_SOURCES` (formatted nicely, but in env var you should pack this to one-line and no-spaces):
```json
{
"config": {
"entries": [
{
"url": "mongodb://user:password@mongodb/test_database?authSource=admin",
"config": {
"fetcher": "MongoDBFetchProvider",
"database": "opal_fetcher_mongodb",
"collection": "cities_collection",
"find": { "query": {} }
},
"topics": ["policy_data"],
"dst_path": "cities"
}
]
}
}
```
Notice how `config` is an instance of `MongoDBFetchProvider` (code is in `opal_fetcher_mongodb/provider.py`).
Values for this fetcher config:
* The `url` is actually a MongoDB uri.
* Your `config` must include the `fetcher` key to indicate to OPAL that you use a custom fetcher.
* Your `config` must include the `collection` key to indicate what collection to query in MongoDB.
* Your `config` may include the `database` key to indicate what database to query in MongoDB. If not specified, the default database will be used.
* Your `config` must include one of `findOne`, `find` or `aggregate` keys to indicate what query to run against MongoDB.
* Your `config` may include the `transform` key to transform the results from the `find` or `aggregate` queries.
#### Query methods
All the three available query methods accept the same input parameters as defined in the MongoDB documentation.
##### findOne
* `findOne` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.findOne/)
<details>
<summary>Example configuration</summary>
```json
{
"config": {
"entries": [
{
...
"config": {
...
"findOne": {
"query": {
...
},
"projection": {
...
},
"options": {
...
}
}
}
}
]
}
}
```
</details>
##### find
* `find` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.find/)
<details>
<summary>Example configuration</summary>
```json
{
"config": {
"entries": [
{
...
"config": {
...
"find": {
"query": {
...
},
"projection": {
...
},
"options": {
...
}
},
"transform": {
"first": false,
"mapKey": "",
"merge": true
}
}
}
]
}
}
```
</details>
##### aggregate
* `aggregate` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/)
<details>
<summary>Example configuration</summary>
```json
{
"config": {
"entries": [
{
...
"config": {
...
"aggregate": {
"pipeline": [
...
],
"options": {
...
}
},
"transform": {
"first": false,
"mapKey": ""
}
}
}
]
}
}
```
</details>
#### Query transform
`transform` allows you to transform the results from the `find` or `aggregate` queries.
##### first
`transform.first` allows you to return only the first result from the query.
Equivalent to the following Python code:
```python
result = query_result[0]
```
##### mapKey
`transform.mapKey` allows you to map the original list-like result to a dictionary-like result using the property specified in the `mapKey` as the key for the dictionary.
Equivalent to the following Python code:
```python
result = {}
for item in query_result:
result[item['key']] = item
```
> Only properties in the root of the document can be used as the key for the dictionary.
##### merge
`transform.merge` allows you to merge the results from the query into a single document. Duplicate keys will be overwritten by the last document in the list.
Equivalent to the following Python code:
```python
result = {}
for item in query_result:
for key, value in item.items():
result[key] = value
```
Raw data
{
"_id": null,
"home_page": null,
"name": "opal-fetcher-mongodb",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "Open Policy Agent, OPA, OPAL, Open Policy Administration Layer, MongoDB, Permit.io",
"author": null,
"author_email": "Ionut Andrei Oanca <ia.oanca@treedom.net>",
"download_url": "https://files.pythonhosted.org/packages/18/82/dfac5b7f0ee3fadec1301f146bc4d50e4b57b34424c44fdbef3176a15ade/opal_fetcher_mongodb-0.0.1.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <img src=\"https://i.ibb.co/BGVBmMK/opal.png\" height=170 alt=\"opal\" border=\"0\" />\n</p>\n<h2 align=\"center\">\nOPAL Fetcher for MongoDB\n</h2>\n\n<h4 align=\"center\">\nMade with \u2764\ufe0f at <a href=\"https://treedom.net\"><img src=\"https://i.ibb.co/QfYVtP5/Treedom-logo.png\" height=\"24\" alt=\"treedom\" border=\"0\" /></a>\n</h4>\n\n<h6 align=\"center\">\n<a href=\"https://www.treedom.net/it/organization/treedom/event/treedom-open-source\"><img src=\"https://badges.treedom.net/badge/f/treedom-open-source\" alt=\"plant-a-tree\" border=\"0\" /></a>\n</h6>\n\n[Check out OPAL main repo here.](https://github.com/permitio/opal)\n\n\n### What's in this repo?\nAn OPAL [custom fetch provider](https://docs.opal.ac/tutorials/write_your_own_fetch_provider) to bring authorization state from [MongoDB](https://www.mongodb.com/).\n\n### How to try this custom fetcher in one command? (Example docker-compose configuration)\n\nYou can test this fetcher with the example docker compose file in this repository root. Clone this repo, `cd` into the cloned repo, and then run:\n```\ndocker compose up\n```\nthis docker compose configuration already correctly configures OPAL to load the MongoDB Fetch Provider, and correctly configures `OPAL_DATA_CONFIG_SOURCES` to include an entry that uses this fetcher.\n\n### \u270f\ufe0f How to use this fetcher in your OPAL Setup\n\n#### 1) Build a custom opal-client `Dockerfile`\n\nThe official docker image only contains the built-in fetch providers. You need to create your own `Dockerfile` (that is based on the official docker image), that includes this fetcher's pip package.\n\nYour `Dockerfile` should look like this:\n```\nFROM permitio/opal-client:latest\nRUN pip install --no-cache-dir --user opal-fetcher-mongodb\n```\n\n#### 2) Build your custom opal-client container\nSay your special Dockerfile from step one is called `custom_client.Dockerfile`.\n\nYou must build a customized OPAL container from this Dockerfile, like so:\n```\ndocker build -t yourcompany/opal-client -f custom_client.Dockerfile .\n```\n\n#### 3) When running OPAL, set `OPAL_FETCH_PROVIDER_MODULES`\nPass the following environment variable to the OPAL client docker container (comma-separated provider modules):\n```\nOPAL_FETCH_PROVIDER_MODULES=opal_common.fetcher.providers,opal_fetcher_mongodb.provider\n```\nNotice that OPAL receives a list from where to search for fetch providers.\nThe list in our case includes the built-in providers (`opal_common.fetcher.providers`) and our custom MongoDB provider.\n\n#### 4) Using the custom provider in your DataSourceEntry objects\n\nYour DataSourceEntry objects (either in `OPAL_DATA_CONFIG_SOURCES` or in dynamic updates sent via the OPAL publish API) can now include this fetcher's config.\n\nExample value of `OPAL_DATA_CONFIG_SOURCES` (formatted nicely, but in env var you should pack this to one-line and no-spaces):\n```json\n{\n \"config\": {\n \"entries\": [\n {\n \"url\": \"mongodb://user:password@mongodb/test_database?authSource=admin\",\n \"config\": {\n \"fetcher\": \"MongoDBFetchProvider\",\n \"database\": \"opal_fetcher_mongodb\",\n \"collection\": \"cities_collection\",\n \"find\": { \"query\": {} }\n },\n \"topics\": [\"policy_data\"],\n \"dst_path\": \"cities\"\n }\n ]\n }\n}\n```\n\nNotice how `config` is an instance of `MongoDBFetchProvider` (code is in `opal_fetcher_mongodb/provider.py`).\n\nValues for this fetcher config:\n* The `url` is actually a MongoDB uri.\n* Your `config` must include the `fetcher` key to indicate to OPAL that you use a custom fetcher.\n* Your `config` must include the `collection` key to indicate what collection to query in MongoDB.\n* Your `config` may include the `database` key to indicate what database to query in MongoDB. If not specified, the default database will be used.\n* Your `config` must include one of `findOne`, `find` or `aggregate` keys to indicate what query to run against MongoDB.\n* Your `config` may include the `transform` key to transform the results from the `find` or `aggregate` queries.\n\n#### Query methods\nAll the three available query methods accept the same input parameters as defined in the MongoDB documentation.\n\n##### findOne\n\n* `findOne` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.findOne/)\n\n<details>\n <summary>Example configuration</summary>\n\n```json\n{\n \"config\": {\n \"entries\": [\n {\n ...\n \"config\": {\n ...\n \"findOne\": {\n \"query\": {\n ...\n },\n \"projection\": {\n ...\n },\n \"options\": {\n ...\n }\n }\n }\n }\n ]\n }\n}\n```\n</details>\n\n##### find\n\n* `find` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.find/)\n\n<details>\n <summary>Example configuration</summary>\n\n```json\n{\n \"config\": {\n \"entries\": [\n {\n ...\n \"config\": {\n ...\n \"find\": {\n \"query\": {\n ...\n },\n \"projection\": {\n ...\n },\n \"options\": {\n ...\n }\n },\n \"transform\": {\n \"first\": false,\n \"mapKey\": \"\",\n \"merge\": true\n }\n }\n }\n ]\n }\n}\n```\n</details>\n\n##### aggregate\n\n* `aggregate` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/)\n\n<details>\n <summary>Example configuration</summary>\n\n```json\n{\n \"config\": {\n \"entries\": [\n {\n ...\n \"config\": {\n ...\n \"aggregate\": {\n \"pipeline\": [\n ...\n ],\n \"options\": {\n ...\n }\n },\n \"transform\": {\n \"first\": false,\n \"mapKey\": \"\"\n }\n }\n }\n ]\n }\n}\n```\n</details>\n\n#### Query transform\n`transform` allows you to transform the results from the `find` or `aggregate` queries.\n\n##### first\n`transform.first` allows you to return only the first result from the query.\n\nEquivalent to the following Python code:\n\n```python\nresult = query_result[0]\n```\n\n##### mapKey\n`transform.mapKey` allows you to map the original list-like result to a dictionary-like result using the property specified in the `mapKey` as the key for the dictionary.\n\nEquivalent to the following Python code:\n\n```python\nresult = {}\nfor item in query_result:\n result[item['key']] = item\n```\n\n> Only properties in the root of the document can be used as the key for the dictionary.\n\n##### merge\n\n`transform.merge` allows you to merge the results from the query into a single document. Duplicate keys will be overwritten by the last document in the list.\n\nEquivalent to the following Python code:\n\n```python\nresult = {}\nfor item in query_result:\n for key, value in item.items():\n result[key] = value\n```\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "An OPAL fetch provider to bring authorization state from MongoDB.",
"version": "0.0.1",
"project_urls": {
"Bug Tracker": "https://github.com/treedomtrees/opal-fetcher-mongodb/issues",
"Source": "https://github.com/treedomtrees/opal-fetcher-mongodb"
},
"split_keywords": [
"open policy agent",
" opa",
" opal",
" open policy administration layer",
" mongodb",
" permit.io"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "eff9aa6e7f0d52d0f526d2fb3dead20251b12c13b51311805702b4f6682964a6",
"md5": "2ef21ae4557332ea0ee7e3f1a949b565",
"sha256": "9450d13ff161eef451bbb825791cf46c3e0bd275739facbfdde75c84815d0151"
},
"downloads": -1,
"filename": "opal_fetcher_mongodb-0.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2ef21ae4557332ea0ee7e3f1a949b565",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 10977,
"upload_time": "2024-06-07T13:52:53",
"upload_time_iso_8601": "2024-06-07T13:52:53.019485Z",
"url": "https://files.pythonhosted.org/packages/ef/f9/aa6e7f0d52d0f526d2fb3dead20251b12c13b51311805702b4f6682964a6/opal_fetcher_mongodb-0.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1882dfac5b7f0ee3fadec1301f146bc4d50e4b57b34424c44fdbef3176a15ade",
"md5": "e7f3de605bab32839c7d2a9e920710fc",
"sha256": "1b28027dd7b5870039f91c4ee82eadfdc46ed5e5d68a804556546d5c793f6341"
},
"downloads": -1,
"filename": "opal_fetcher_mongodb-0.0.1.tar.gz",
"has_sig": false,
"md5_digest": "e7f3de605bab32839c7d2a9e920710fc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 17126,
"upload_time": "2024-06-07T13:52:55",
"upload_time_iso_8601": "2024-06-07T13:52:55.681146Z",
"url": "https://files.pythonhosted.org/packages/18/82/dfac5b7f0ee3fadec1301f146bc4d50e4b57b34424c44fdbef3176a15ade/opal_fetcher_mongodb-0.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-07 13:52:55",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "treedomtrees",
"github_project": "opal-fetcher-mongodb",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "opal-fetcher-mongodb"
}
JSON
