<p align="center">
<img src="./policyweaver.png" alt="Policy Weaver icon" width="200"/>
</p>
</p>
<p align="center">
<a href="https://badgen.net/github/license/microsoft/Policy-Weaver" target="_blank">
<img src="https://badgen.net/github/license/microsoft/Policy-Weaver" alt="License">
</a>
<a href="https://badgen.net/github/releases/microsoft/Policy-Weaver" target="_blank">
<img src="https://badgen.net/github/releases/microsoft/Policy-Weaver" alt="Test">
</a>
<a href="https://badgen.net/github/contributors/microsoft/Policy-Weaver" target="_blank">
<img src="https://badgen.net/github/contributors/microsoft/Policy-Weaver" alt="Publish">
</a>
<a href="https://badgen.net/github/commits/microsoft/Policy-Weaver" target="_blank">
<img src="https://badgen.net/github/commits/microsoft/Policy-Weaver" alt="Commits">
</a>
<a href="https://badgen.net/pypi/v/Policy-Weaver" target="_blank">
<img src="https://badgen.net/pypi/v/Policy-Weaver" alt="Package version">
</a>
</p>
---
# Policy Weaver: synchronizes data access policies across platforms
A Python-based accelerator designed to automate the synchronization of security policies from different source catalogs with [OneLake Security](https://learn.microsoft.com/en-us/fabric/onelake/security/get-started-data-access-roles) roles. This is required when using OneLake mirroring to ensure consistent security across data platforms.
## :rocket: Features
- **Microsoft Fabric Support**: Direct integration with Fabric Mirrored Databases and OneLake Security.
- **Runs anywhere**: It can be run within Fabric Notebook or from anywhere with a Python runtime.
- **Effective Policies**: Resolves effective read privileges automatically, traversing nested groups and roles as required.
- **Pluggable Framework**: Supports Azure Databricks and Snowflake policies, with more connectors planned.
- **Secure**: Can use Azure Key Vault to securely manage sensitive information like Service Principal credentials and API tokens.
> :pushpin: **Note:** Row-level and column-level security extraction will be implemented in the next version, once these features become available in OneLake Security.
## :clipboard: Prerequisites
Before installing and running this solution, ensure you have:
- **Azure [Service Principal](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal)** with the following [Microsoft Graph API permissions](https://learn.microsoft.com/en-us/graph/permissions-reference):
- `Application.Read.All`
- `User.Read`
- `User.Read.All`
- `Directory.Read.All`
- [A client secret](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal#option-3-create-a-new-client-secret) for the Service Principal
- Added the Service Principal as [Admin](https://learn.microsoft.com/en-us/fabric/fundamentals/give-access-workspaces) on the Fabric Workspace cpontaining the mirror database.
> :pushpin: **Note:** Every source catalog has additional pre-requisites
## :hammer_and_wrench: Installation
Make sure your Python version is greater or equal than 3.11. Then, install the library:
```bash
$ pip install policy-weaver
```
## :thread: Databricks Example
### Azure Databricks Configuration
1. Create a [Mirror Azure Databricks Catalog](https://learn.microsoft.com/en-us/fabric/mirroring/azure-databricks-tutorial) in a Microsoft Fabric Workspace.
1. Account Admin Console :arrow_right: User Management :arrow_right: Add your Azure Service Principal.
1. **Role**: "Account admin"
1. **Permission**: "Service Principal:Manager"
1. Workspace Settings :arrow_right: Identity & Access :arrow_right: Manage Service Principals :arrow_right: Add your Azure Service Principal.
1. **Permission**: "Service Principal:Manager" permission.
1. **Generate** an OAuth secret for your config.yaml file.
### Update your Configuration file
Download this [config.yaml](./config.yaml) file template and update it based on your environment.
For Databricks specifically, you will need to provide:
- **workspace_url**: https://adb-xxxxxxxxxxx.azuredatabricks.net/
- **account_id**: your databricks account id
- **account_api_token**: Depending on the keyvault setting: the keyvault secret name or your databricks secret
### Run the Weaver!
This is all the code you need. Just make sure Policy Weaver can access your YAML configuration file.
```python
#import the PolicyWeaver library
from policyweaver.weaver import WeaverAgent
from policyweaver.plugins.databricks.model import DatabricksSourceMap
#Load config
config = DatabricksSourceMap.from_yaml("path_to_your_config.yaml")
#run the PolicyWeaver
await WeaverAgent.run(config)
```
All done! You can now check your Microsoft Fabric Mirrored Azure Databricks catalog new policies.
## :raising_hand: Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
## :scroll: License
This project is licensed under the MIT License - see the LICENSE file for details.
## :shield: Trademarks
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft
trademarks or logos is subject to and must follow
[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).
Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
Any use of third-party trademarks or logos are subject to those third-party's policies.
Raw data
{
"_id": null,
"home_page": null,
"name": "policy-weaver",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11.8",
"maintainer_email": null,
"keywords": "Fabric, Fabric Access Policy, Databricks, BigQuery, Snowflake",
"author": null,
"author_email": "Tonio Lora <tonio.lora@microsoft.com>, Melissa Lacefield <melilace@microsoft.com>, Emily Nguyen <Emily.Nguyen@microsoft.com>, Andreas J Rederer <andreas.rederer@microsoft.com>",
"download_url": "https://files.pythonhosted.org/packages/15/96/e2996466b4f521eb6516142c78462deb6a423cfc2faebb1ff69252966f3a/policy_weaver-0.2.7.tar.gz",
"platform": null,
"description": " <p align=\"center\">\n <img src=\"./policyweaver.png\" alt=\"Policy Weaver icon\" width=\"200\"/>\n</p>\n\n</p>\n<p align=\"center\">\n<a href=\"https://badgen.net/github/license/microsoft/Policy-Weaver\" target=\"_blank\">\n <img src=\"https://badgen.net/github/license/microsoft/Policy-Weaver\" alt=\"License\">\n</a>\n<a href=\"https://badgen.net/github/releases/microsoft/Policy-Weaver\" target=\"_blank\">\n <img src=\"https://badgen.net/github/releases/microsoft/Policy-Weaver\" alt=\"Test\">\n</a>\n<a href=\"https://badgen.net/github/contributors/microsoft/Policy-Weaver\" target=\"_blank\">\n <img src=\"https://badgen.net/github/contributors/microsoft/Policy-Weaver\" alt=\"Publish\">\n</a>\n<a href=\"https://badgen.net/github/commits/microsoft/Policy-Weaver\" target=\"_blank\">\n <img src=\"https://badgen.net/github/commits/microsoft/Policy-Weaver\" alt=\"Commits\">\n</a>\n<a href=\"https://badgen.net/pypi/v/Policy-Weaver\" target=\"_blank\">\n <img src=\"https://badgen.net/pypi/v/Policy-Weaver\" alt=\"Package version\">\n</a>\n</p>\n\n---\n\n# Policy Weaver: synchronizes data access policies across platforms\n\nA Python-based accelerator designed to automate the synchronization of security policies from different source catalogs with [OneLake Security](https://learn.microsoft.com/en-us/fabric/onelake/security/get-started-data-access-roles) roles. This is required when using OneLake mirroring to ensure consistent security across data platforms.\n\n\n## :rocket: Features\n- **Microsoft Fabric Support**: Direct integration with Fabric Mirrored Databases and OneLake Security.\n- **Runs anywhere**: It can be run within Fabric Notebook or from anywhere with a Python runtime.\n- **Effective Policies**: Resolves effective read privileges automatically, traversing nested groups and roles as required.\n- **Pluggable Framework**: Supports Azure Databricks and Snowflake policies, with more connectors planned.\n- **Secure**: Can use Azure Key Vault to securely manage sensitive information like Service Principal credentials and API tokens.\n\n> :pushpin: **Note:** Row-level and column-level security extraction will be implemented in the next version, once these features become available in OneLake Security.\n\n## :clipboard: Prerequisites\nBefore installing and running this solution, ensure you have:\n- **Azure [Service Principal](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal)** with the following [Microsoft Graph API permissions](https://learn.microsoft.com/en-us/graph/permissions-reference):\n - `Application.Read.All`\n - `User.Read`\n - `User.Read.All`\n - `Directory.Read.All`\n- [A client secret](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal#option-3-create-a-new-client-secret) for the Service Principal\n- Added the Service Principal as [Admin](https://learn.microsoft.com/en-us/fabric/fundamentals/give-access-workspaces) on the Fabric Workspace cpontaining the mirror database.\n\n> :pushpin: **Note:** Every source catalog has additional pre-requisites\n\n## :hammer_and_wrench: Installation\nMake sure your Python version is greater or equal than 3.11. Then, install the library:\n```bash\n$ pip install policy-weaver\n```\n\n\n## :thread: Databricks Example\n\n### Azure Databricks Configuration\n1. Create a [Mirror Azure Databricks Catalog](https://learn.microsoft.com/en-us/fabric/mirroring/azure-databricks-tutorial) in a Microsoft Fabric Workspace.\n1. Account Admin Console :arrow_right: User Management :arrow_right: Add your Azure Service Principal. \n 1. **Role**: \"Account admin\"\n 1. **Permission**: \"Service Principal:Manager\"\n1. Workspace Settings :arrow_right: Identity & Access :arrow_right: Manage Service Principals :arrow_right: Add your Azure Service Principal.\n 1. **Permission**: \"Service Principal:Manager\" permission. \n 1. **Generate** an OAuth secret for your config.yaml file.\n\n### Update your Configuration file\nDownload this [config.yaml](./config.yaml) file template and update it based on your environment.\n\nFor Databricks specifically, you will need to provide:\n\n- **workspace_url**: https://adb-xxxxxxxxxxx.azuredatabricks.net/\n- **account_id**: your databricks account id\n- **account_api_token**: Depending on the keyvault setting: the keyvault secret name or your databricks secret\n\n### Run the Weaver!\nThis is all the code you need. Just make sure Policy Weaver can access your YAML configuration file.\n```python\n#import the PolicyWeaver library\nfrom policyweaver.weaver import WeaverAgent\nfrom policyweaver.plugins.databricks.model import DatabricksSourceMap\n\n#Load config\nconfig = DatabricksSourceMap.from_yaml(\"path_to_your_config.yaml\")\n\n#run the PolicyWeaver\nawait WeaverAgent.run(config)\n```\n\nAll done! You can now check your Microsoft Fabric Mirrored Azure Databricks catalog new policies.\n\n## :raising_hand: Contributing\n\nThis project welcomes contributions and suggestions. Most contributions require you to agree to a\nContributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us\nthe rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.\n\nWhen you submit a pull request, a CLA bot will automatically determine whether you need to provide\na CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions\nprovided by the bot. You will only need to do this once across all repos using our CLA.\n\nThis project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).\nFor more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or\ncontact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.\n\n## :scroll: License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n\n## :shield: Trademarks\n\nThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft \ntrademarks or logos is subject to and must follow \n[Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general).\nUse of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.\nAny use of third-party trademarks or logos are subject to those third-party's policies.\n",
"bugtrack_url": null,
"license": null,
"summary": "Policy Weaver for Microsoft Fabric",
"version": "0.2.7",
"project_urls": {
"Documentation": "https://github.com/microsoft/Policy-Weaver/blob/main/docs/PolicyWeaver.md",
"Homepage": "https://github.com/microsoft/Policy-Weaver",
"Issues": "https://github.com/microsoft/Policy-Weaver/issues",
"Repository": "https://github.com/microsoft/Policy-Weaver.git"
},
"split_keywords": [
"fabric",
" fabric access policy",
" databricks",
" bigquery",
" snowflake"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "c91fb8b7c55947be689bfde12a3f5f268a30b4d20af8cdf785fcd8a9eb7c2e2c",
"md5": "460c3c5adfeb14e57a53ef9e3d3309fe",
"sha256": "7e3df7d5ddb1ce0a3ba650b43fe4a20bfa416d6d74373655e273611dcc717e5c"
},
"downloads": -1,
"filename": "policy_weaver-0.2.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "460c3c5adfeb14e57a53ef9e3d3309fe",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11.8",
"size": 48883,
"upload_time": "2025-09-06T00:18:28",
"upload_time_iso_8601": "2025-09-06T00:18:28.369033Z",
"url": "https://files.pythonhosted.org/packages/c9/1f/b8b7c55947be689bfde12a3f5f268a30b4d20af8cdf785fcd8a9eb7c2e2c/policy_weaver-0.2.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "1596e2996466b4f521eb6516142c78462deb6a423cfc2faebb1ff69252966f3a",
"md5": "7f18a706fc9259d9e49fb9594a867c57",
"sha256": "00d9ba6ec6c04c446f08b5b76b652e8b43413b1bef0da0f33443ca674097bef7"
},
"downloads": -1,
"filename": "policy_weaver-0.2.7.tar.gz",
"has_sig": false,
"md5_digest": "7f18a706fc9259d9e49fb9594a867c57",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11.8",
"size": 41289,
"upload_time": "2025-09-06T00:18:29",
"upload_time_iso_8601": "2025-09-06T00:18:29.874179Z",
"url": "https://files.pythonhosted.org/packages/15/96/e2996466b4f521eb6516142c78462deb6a423cfc2faebb1ff69252966f3a/policy_weaver-0.2.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-06 00:18:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "microsoft",
"github_project": "Policy-Weaver",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "policy-weaver"
}
JSON
