<p align="center">
<img src="https://github.com/permitio/opal/assets/4082578/4e21f85f-30ab-43e2-92de-b82f78888c71" height=170 alt="opal" border="0" />
</p>
<h2 align="center">
OPAL Fetcher for MongoDB
</h2>
<h4 align="center">
Made with ❤️ at <a href="https://treedom.net"><img src="https://i.ibb.co/QfYVtP5/Treedom-logo.png" height="24" alt="treedom" border="0" /></a>, <a href="#-join-us-in-making-a-difference-">join us in making a difference</a>!
</h4>
<h6 align="center">
<a href="https://www.treedom.net/it/organization/treedom/event/treedom-open-source"><img src="https://badges.treedom.net/badge/f/treedom-open-source?2" alt="plant-a-tree" border="0" /></a>
</h6>
[Check out OPAL main repo here.](https://github.com/permitio/opal)
### What's in this repo?
An OPAL [custom fetch provider](https://docs.opal.ac/tutorials/write_your_own_fetch_provider) to bring authorization state from [MongoDB](https://www.mongodb.com/).
### How to try this custom fetcher in one command? (Example docker-compose configuration)
You can test this fetcher with the example docker compose file in this repository root. Clone this repo, `cd` into the cloned repo, and then run:
```
docker compose up
```
this docker compose configuration already correctly configures OPAL to load the MongoDB Fetch Provider, and correctly configures `OPAL_DATA_CONFIG_SOURCES` to include an entry that uses this fetcher.
### ✏️ How to use this fetcher in your OPAL Setup
#### 1) Build a custom opal-client `Dockerfile`
The official docker image only contains the built-in fetch providers. You need to create your own `Dockerfile` (that is based on the official docker image), that includes this fetcher's pip package.
Your `Dockerfile` should look like this:
```
FROM permitio/opal-client:latest
RUN pip install --no-cache-dir --user opal-fetcher-mongodb
```
#### 2) Build your custom opal-client container
Say your special Dockerfile from step one is called `custom_client.Dockerfile`.
You must build a customized OPAL container from this Dockerfile, like so:
```
docker build -t yourcompany/opal-client -f custom_client.Dockerfile .
```
#### 3) When running OPAL, set `OPAL_FETCH_PROVIDER_MODULES`
Pass the following environment variable to the OPAL client docker container (comma-separated provider modules):
```
OPAL_FETCH_PROVIDER_MODULES=opal_common.fetcher.providers,opal_fetcher_mongodb.provider
```
Notice that OPAL receives a list from where to search for fetch providers.
The list in our case includes the built-in providers (`opal_common.fetcher.providers`) and our custom MongoDB provider.
#### 4) Using the custom provider in your DataSourceEntry objects
Your DataSourceEntry objects (either in `OPAL_DATA_CONFIG_SOURCES` or in dynamic updates sent via the OPAL publish API) can now include this fetcher's config.
Example value of `OPAL_DATA_CONFIG_SOURCES` (formatted nicely, but in env var you should pack this to one-line and no-spaces):
```json
{
"config": {
"entries": [
{
"url": "mongodb://user:password@mongodb/test_database?authSource=admin",
"config": {
"fetcher": "MongoDBFetchProvider",
"database": "opal_fetcher_mongodb",
"collection": "cities_collection",
"find": { "query": {} }
},
"topics": ["policy_data"],
"dst_path": "cities"
}
]
}
}
```
Notice how `config` is an instance of `MongoDBFetchProvider` (code is in `opal_fetcher_mongodb/provider.py`).
Values for this fetcher config:
* The `url` is actually a MongoDB uri.
* Your `config` must include the `fetcher` key to indicate to OPAL that you use a custom fetcher.
* Your `config` must include the `collection` key to indicate what collection to query in MongoDB.
* Your `config` may include the `database` key to indicate what database to query in MongoDB. If not specified, the default database will be used.
* Your `config` must include one of `findOne`, `find` or `aggregate` keys to indicate what query to run against MongoDB.
* Your `config` may include the `transform` key to transform the results from the `find` or `aggregate` queries.
#### Query methods
All the three available query methods accept the same input parameters as defined in the MongoDB documentation.
##### findOne
* `findOne` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.findOne/)
<details>
<summary>Example configuration</summary>
```json
{
"config": {
"entries": [
{
...
"config": {
...
"findOne": {
"query": {
...
},
"projection": {
...
},
"options": {
...
}
}
}
}
]
}
}
```
</details>
##### find
* `find` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.find/)
<details>
<summary>Example configuration</summary>
```json
{
"config": {
"entries": [
{
...
"config": {
...
"find": {
"query": {
...
},
"projection": {
...
},
"options": {
...
}
},
"transform": {
"first": false,
"mapKey": "",
"merge": true
}
}
}
]
}
}
```
</details>
##### aggregate
* `aggregate` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/)
<details>
<summary>Example configuration</summary>
```json
{
"config": {
"entries": [
{
...
"config": {
...
"aggregate": {
"pipeline": [
...
],
"options": {
...
}
},
"transform": {
"first": false,
"mapKey": ""
}
}
}
]
}
}
```
</details>
#### Query transform
`transform` allows you to transform the results from the `find` or `aggregate` queries.
##### first
`transform.first` allows you to return only the first result from the query.
Equivalent to the following Python code:
```python
result = query_result[0]
```
##### mapKey
`transform.mapKey` allows you to map the original list-like result to a dictionary-like result using the property specified in the `mapKey` as the key for the dictionary.
Equivalent to the following Python code:
```python
result = {}
for item in query_result:
result[item['key']] = item
```
> Only properties in the root of the document can be used as the key for the dictionary.
##### merge
`transform.merge` allows you to merge the results from the query into a single document. Duplicate keys will be overwritten by the last document in the list.
Equivalent to the following Python code:
```python
result = {}
for item in query_result:
for key, value in item.items():
result[key] = value
```
## 🌳 Join Us in Making a Difference! 🌳
We invite all developers who use Treedom's open-source code to support our mission of sustainability by planting a tree with us. By contributing to reforestation efforts, you help create a healthier planet and give back to the environment. Visit our [Treedom Open Source Forest](https://www.treedom.net/en/organization/treedom/event/treedom-open-source) to plant your tree today and join our community of eco-conscious developers.
Additionally, you can integrate the Treedom GitHub badge into your repository to showcase the number of trees in your Treedom forest and encourage others to plant new ones. Check out our [integration guide](https://github.com/treedomtrees/.github/blob/main/profile/README.md) to get started.
Together, we can make a lasting impact! 🌍💚
Raw data
{
"_id": null,
"home_page": null,
"name": "opal-fetcher-mongodb",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "Open Policy Agent, OPA, OPAL, Open Policy Administration Layer, MongoDB, Permit.io",
"author": null,
"author_email": "Ionut Andrei Oanca <ia.oanca@treedom.net>",
"download_url": "https://files.pythonhosted.org/packages/fb/5a/a02cb77b20486c6e2c30fddf7203328a8b7e6feec2cead45953feceff5fa/opal_fetcher_mongodb-1.0.0.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <img src=\"https://github.com/permitio/opal/assets/4082578/4e21f85f-30ab-43e2-92de-b82f78888c71\" height=170 alt=\"opal\" border=\"0\" />\n</p>\n<h2 align=\"center\">\nOPAL Fetcher for MongoDB\n</h2>\n\n<h4 align=\"center\">\nMade with \u2764\ufe0f at <a href=\"https://treedom.net\"><img src=\"https://i.ibb.co/QfYVtP5/Treedom-logo.png\" height=\"24\" alt=\"treedom\" border=\"0\" /></a>, <a href=\"#-join-us-in-making-a-difference-\">join us in making a difference</a>!\n</h4>\n\n<h6 align=\"center\">\n<a href=\"https://www.treedom.net/it/organization/treedom/event/treedom-open-source\"><img src=\"https://badges.treedom.net/badge/f/treedom-open-source?2\" alt=\"plant-a-tree\" border=\"0\" /></a>\n</h6>\n\n[Check out OPAL main repo here.](https://github.com/permitio/opal)\n\n\n### What's in this repo?\nAn OPAL [custom fetch provider](https://docs.opal.ac/tutorials/write_your_own_fetch_provider) to bring authorization state from [MongoDB](https://www.mongodb.com/).\n\n### How to try this custom fetcher in one command? (Example docker-compose configuration)\n\nYou can test this fetcher with the example docker compose file in this repository root. Clone this repo, `cd` into the cloned repo, and then run:\n```\ndocker compose up\n```\nthis docker compose configuration already correctly configures OPAL to load the MongoDB Fetch Provider, and correctly configures `OPAL_DATA_CONFIG_SOURCES` to include an entry that uses this fetcher.\n\n### \u270f\ufe0f How to use this fetcher in your OPAL Setup\n\n#### 1) Build a custom opal-client `Dockerfile`\n\nThe official docker image only contains the built-in fetch providers. You need to create your own `Dockerfile` (that is based on the official docker image), that includes this fetcher's pip package.\n\nYour `Dockerfile` should look like this:\n```\nFROM permitio/opal-client:latest\nRUN pip install --no-cache-dir --user opal-fetcher-mongodb\n```\n\n#### 2) Build your custom opal-client container\nSay your special Dockerfile from step one is called `custom_client.Dockerfile`.\n\nYou must build a customized OPAL container from this Dockerfile, like so:\n```\ndocker build -t yourcompany/opal-client -f custom_client.Dockerfile .\n```\n\n#### 3) When running OPAL, set `OPAL_FETCH_PROVIDER_MODULES`\nPass the following environment variable to the OPAL client docker container (comma-separated provider modules):\n```\nOPAL_FETCH_PROVIDER_MODULES=opal_common.fetcher.providers,opal_fetcher_mongodb.provider\n```\nNotice that OPAL receives a list from where to search for fetch providers.\nThe list in our case includes the built-in providers (`opal_common.fetcher.providers`) and our custom MongoDB provider.\n\n#### 4) Using the custom provider in your DataSourceEntry objects\n\nYour DataSourceEntry objects (either in `OPAL_DATA_CONFIG_SOURCES` or in dynamic updates sent via the OPAL publish API) can now include this fetcher's config.\n\nExample value of `OPAL_DATA_CONFIG_SOURCES` (formatted nicely, but in env var you should pack this to one-line and no-spaces):\n```json\n{\n \"config\": {\n \"entries\": [\n {\n \"url\": \"mongodb://user:password@mongodb/test_database?authSource=admin\",\n \"config\": {\n \"fetcher\": \"MongoDBFetchProvider\",\n \"database\": \"opal_fetcher_mongodb\",\n \"collection\": \"cities_collection\",\n \"find\": { \"query\": {} }\n },\n \"topics\": [\"policy_data\"],\n \"dst_path\": \"cities\"\n }\n ]\n }\n}\n```\n\nNotice how `config` is an instance of `MongoDBFetchProvider` (code is in `opal_fetcher_mongodb/provider.py`).\n\nValues for this fetcher config:\n* The `url` is actually a MongoDB uri.\n* Your `config` must include the `fetcher` key to indicate to OPAL that you use a custom fetcher.\n* Your `config` must include the `collection` key to indicate what collection to query in MongoDB.\n* Your `config` may include the `database` key to indicate what database to query in MongoDB. If not specified, the default database will be used.\n* Your `config` must include one of `findOne`, `find` or `aggregate` keys to indicate what query to run against MongoDB.\n* Your `config` may include the `transform` key to transform the results from the `find` or `aggregate` queries.\n\n#### Query methods\nAll the three available query methods accept the same input parameters as defined in the MongoDB documentation.\n\n##### findOne\n\n* `findOne` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.findOne/)\n\n<details>\n <summary>Example configuration</summary>\n\n```json\n{\n \"config\": {\n \"entries\": [\n {\n ...\n \"config\": {\n ...\n \"findOne\": {\n \"query\": {\n ...\n },\n \"projection\": {\n ...\n },\n \"options\": {\n ...\n }\n }\n }\n }\n ]\n }\n}\n```\n</details>\n\n##### find\n\n* `find` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.find/)\n\n<details>\n <summary>Example configuration</summary>\n\n```json\n{\n \"config\": {\n \"entries\": [\n {\n ...\n \"config\": {\n ...\n \"find\": {\n \"query\": {\n ...\n },\n \"projection\": {\n ...\n },\n \"options\": {\n ...\n }\n },\n \"transform\": {\n \"first\": false,\n \"mapKey\": \"\",\n \"merge\": true\n }\n }\n }\n ]\n }\n}\n```\n</details>\n\n##### aggregate\n\n* `aggregate` - [MongoDB docs](https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/)\n\n<details>\n <summary>Example configuration</summary>\n\n```json\n{\n \"config\": {\n \"entries\": [\n {\n ...\n \"config\": {\n ...\n \"aggregate\": {\n \"pipeline\": [\n ...\n ],\n \"options\": {\n ...\n }\n },\n \"transform\": {\n \"first\": false,\n \"mapKey\": \"\"\n }\n }\n }\n ]\n }\n}\n```\n</details>\n\n#### Query transform\n`transform` allows you to transform the results from the `find` or `aggregate` queries.\n\n##### first\n`transform.first` allows you to return only the first result from the query.\n\nEquivalent to the following Python code:\n\n```python\nresult = query_result[0]\n```\n\n##### mapKey\n`transform.mapKey` allows you to map the original list-like result to a dictionary-like result using the property specified in the `mapKey` as the key for the dictionary.\n\nEquivalent to the following Python code:\n\n```python\nresult = {}\nfor item in query_result:\n result[item['key']] = item\n```\n\n> Only properties in the root of the document can be used as the key for the dictionary.\n\n##### merge\n\n`transform.merge` allows you to merge the results from the query into a single document. Duplicate keys will be overwritten by the last document in the list.\n\nEquivalent to the following Python code:\n\n```python\nresult = {}\nfor item in query_result:\n for key, value in item.items():\n result[key] = value\n```\n\n## \ud83c\udf33 Join Us in Making a Difference! \ud83c\udf33\n\nWe invite all developers who use Treedom's open-source code to support our mission of sustainability by planting a tree with us. By contributing to reforestation efforts, you help create a healthier planet and give back to the environment. Visit our [Treedom Open Source Forest](https://www.treedom.net/en/organization/treedom/event/treedom-open-source) to plant your tree today and join our community of eco-conscious developers.\n\nAdditionally, you can integrate the Treedom GitHub badge into your repository to showcase the number of trees in your Treedom forest and encourage others to plant new ones. Check out our [integration guide](https://github.com/treedomtrees/.github/blob/main/profile/README.md) to get started.\n\nTogether, we can make a lasting impact! \ud83c\udf0d\ud83d\udc9a\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "An OPAL fetch provider to bring authorization state from MongoDB.",
"version": "1.0.0",
"project_urls": {
"Bug Tracker": "https://github.com/treedomtrees/opal-fetcher-mongodb/issues",
"Source": "https://github.com/treedomtrees/opal-fetcher-mongodb"
},
"split_keywords": [
"open policy agent",
" opa",
" opal",
" open policy administration layer",
" mongodb",
" permit.io"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1d1dea6d011c37e9766d74ef8d173bc30d04bbf437befd6e16e8d203ce3cfea6",
"md5": "8ed0701b0ca7128530ae37b81c45c54e",
"sha256": "0579de600dd1ffd50c931ba38c9c5c53f777b656f5663e6424ce1de30094db6d"
},
"downloads": -1,
"filename": "opal_fetcher_mongodb-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8ed0701b0ca7128530ae37b81c45c54e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 11462,
"upload_time": "2024-09-11T09:03:58",
"upload_time_iso_8601": "2024-09-11T09:03:58.179392Z",
"url": "https://files.pythonhosted.org/packages/1d/1d/ea6d011c37e9766d74ef8d173bc30d04bbf437befd6e16e8d203ce3cfea6/opal_fetcher_mongodb-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "fb5aa02cb77b20486c6e2c30fddf7203328a8b7e6feec2cead45953feceff5fa",
"md5": "3fe1e1068727eabdbffec382f75636ae",
"sha256": "4df33ad88e3cad1ab8d92913eeb90d8068bc9d27225c9f619f8fc2f2f7f9b988"
},
"downloads": -1,
"filename": "opal_fetcher_mongodb-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "3fe1e1068727eabdbffec382f75636ae",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 17740,
"upload_time": "2024-09-11T09:04:00",
"upload_time_iso_8601": "2024-09-11T09:04:00.198571Z",
"url": "https://files.pythonhosted.org/packages/fb/5a/a02cb77b20486c6e2c30fddf7203328a8b7e6feec2cead45953feceff5fa/opal_fetcher_mongodb-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-11 09:04:00",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "treedomtrees",
"github_project": "opal-fetcher-mongodb",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "opal-fetcher-mongodb"
}
JSON
