========
Fillmore
========
The Python sentry-sdk has a before_send hook that lets you scrub Sentry events
before they're sent. Fillmore makes it easier to set up a before_send scrubber
and test it.
:Code: https://github.com/willkg/fillmore
:Issues: https://github.com/willkg/fillmore/issues
:License: MPL v2
:Documentation: https://fillmore.readthedocs.io/
Goals
=====
Goals of Fillmore:
1. make it easier to configure Sentry event scrubbing in a way that you can
reason about
2. make it easier to test your scrubbing code so you know it's working over
time
3. scrub in a resilient manner and default to emitting some signal when it
kicks up errors so you know when your error handling code is kicking up
errors
From that, Fillmore has the following features:
* lets you specify keys to scrub in a Sentry event
* resilient to errors--if it fails, it will emit a signal that you can see and
alert on
* links to relevant Sentry documentation, projects, and other things
* testing infrastructure to use in your integration tests
Install
=======
Run::
$ pip install fillmore
Quickstart
==========
Example::
# myapp/app.py
import logging
import logging.config
from fillmore.libsentry import set_up_sentry
from fillmore.scrubber import Scrubber, Rule, build_scrub_query_string
# Set up logging to capture fillmore error messages
logging.getLogger("fillmore").setLevel(logging.ERROR)
# Create a scrubber
scrubber = Scrubber(
rules=[
Rule(
path="request.headers",
keys=["Auth-Token", "Cookie"],
scrub="scrub",
),
Rule(
path="request",
keys=["query_string"],
scrub=build_scrub_query_string(params=["code", "state"]),
),
Rule(
path="exception.values.[].stacktrace.frames.[].vars",
keys=["username", "password"],
scrub="scrub",
),
]
)
# Set up Sentry with the scrubber and the default integrations which
# includes the LoggingIntegration which will capture messages with level
# logging.ERROR.
set_up_sentry(
sentry_dsn="http://user@example.com/1",
host_id="some host id",
release="some release name",
before_send=scrubber,
)
def kick_up_exception():
username = "James" # noqa
try:
raise Exception("internal exception")
except Exception:
logging.getLogger(__name__).exception("kick_up_exception exception")
Now you have a scrubber and you've set up the Sentry client to use it. How do
you know it's scrubbing the right stuff? How will you know if something changes
and it's no longer scrubbing the right stuff?
You can test it like this::
# myapp/test_app.py
import unittest
from fillmore.test import SentryTestHelper
from myapp.app import kick_up_exception
class TestApp(unittest.TestCase):
def test_scrubber(self):
# Reuse the existing Sentry configuration and set up the helper
# to capture Sentry events
sentry_test_helper = SentryTestHelper()
with sentry_test_helper.reuse() as sentry_client:
kick_up_exception()
(event,) = sentry_client.events
error = event["exception"]["values"][0]
self.assertEqual(error["type"], "Exception")
self.assertEqual(error["value"], "internal exception")
self.assertEqual(
error["stacktrace"]["frames"][0]["vars"]["username"], "[Scrubbed]"
)
This creates a Sentry client specific to this test and kicks up an exception in
the test and captures it with Sentry.
Note that this is a contrived context using a Sentry client created for this
test. You'll want to write tests that use the Sentry client configured for your
application and handling events kicked up from different points in your
application to make sure that Sentry events are getting scrubbed correctly.
See Fillmore documentation for explanation and examples.
Why this? Why not other libraries?
==================================
Other libraries:
* **Have an awkward API that is hard to reason about.**
I'm not scrubbing Sentry events for fun. I need to be able to write scrubbing
configuration that is exceptionally clear about what it is and isn't doing.
* **Don't covers large portions of the Sentry event structure.**
I need scrubbers that cover the entire event structure as well as some
of the curious cases like the fact that cookie information shows up twice
and can be encoded as a string.
* **Aren't resilient.**
The scrubber is running in the context of Sentry reporting an error. If it
also errors out, then you can end up in situations where you never see errors
and have no signal that something is horribly wrong. We need scrubbing code
to be extremely resilient and default to emitting a signal that it's broken.
* **Don't include testing infrastructure.**
I'm not scrubbing Sentry events for fun. I need to know that the scrubbing
code is working correctly and that it continues to work as we upgrade
Python, sentry_sdk, and other things.
Having testing infrastructure for making this easier is really important.
History
=======
1.1.0 (April 5th, 2023)
-----------------------
* Switch from flake8 to ruff (#24)
* Add ``fillmore.test.diff_event`` utilify function for comparing a Sentry
event with an expected structure accounting for ``unittest.mock.ANY``. (#23)
1.0.0 (November 8th, 2022)
--------------------------
* Add support for Python 3.11 (#18)
This feels stable and I'm using it in multiple production real-world projects,
so releasing a 1.0.0.
0.1.2 (August 1st, 2022)
------------------------
* Fix examples in documentation so they're linted and tested. Add notes about
``fillmore`` logging. Rewrite testing chapter to introduce Fillmore testing
features in a less muddled way. (#15)
0.1.1 (July 25th, 2022)
-----------------------
* Fix scrubber where paths that were valid, but didn't point to something in an
event erroneously kicked up a RulePathError. (#12)
* Fix test examples in docs.
* Fix examples in README. Thanks @stevejalim!
0.1.0 (June 23rd, 2022)
-----------------------
* Inital extraction and writing.
Raw data
{
"_id": null,
"home_page": "https://github.com/willkg/fillmore",
"name": "fillmore",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "sentry scrubber",
"author": "Will Kahn-Greene",
"author_email": "willkg@mozilla.com",
"download_url": "https://files.pythonhosted.org/packages/dc/c4/c00cebe0f05160f3a5133c13647ab2d12d618842230ac425ef9de6e1ac57/fillmore-1.1.0.tar.gz",
"platform": null,
"description": "========\nFillmore\n========\n\nThe Python sentry-sdk has a before_send hook that lets you scrub Sentry events\nbefore they're sent. Fillmore makes it easier to set up a before_send scrubber\nand test it.\n\n:Code: https://github.com/willkg/fillmore\n:Issues: https://github.com/willkg/fillmore/issues\n:License: MPL v2\n:Documentation: https://fillmore.readthedocs.io/\n\n\nGoals\n=====\n\nGoals of Fillmore:\n\n1. make it easier to configure Sentry event scrubbing in a way that you can\n reason about\n2. make it easier to test your scrubbing code so you know it's working over\n time\n3. scrub in a resilient manner and default to emitting some signal when it\n kicks up errors so you know when your error handling code is kicking up\n errors\n\nFrom that, Fillmore has the following features:\n\n* lets you specify keys to scrub in a Sentry event\n* resilient to errors--if it fails, it will emit a signal that you can see and\n alert on\n* links to relevant Sentry documentation, projects, and other things\n* testing infrastructure to use in your integration tests\n\n\nInstall\n=======\n\nRun::\n\n $ pip install fillmore\n\n\nQuickstart\n==========\n\nExample::\n\n # myapp/app.py\n import logging\n import logging.config\n\n from fillmore.libsentry import set_up_sentry\n from fillmore.scrubber import Scrubber, Rule, build_scrub_query_string\n\n\n # Set up logging to capture fillmore error messages\n logging.getLogger(\"fillmore\").setLevel(logging.ERROR)\n\n # Create a scrubber\n scrubber = Scrubber(\n rules=[\n Rule(\n path=\"request.headers\",\n keys=[\"Auth-Token\", \"Cookie\"],\n scrub=\"scrub\",\n ),\n Rule(\n path=\"request\",\n keys=[\"query_string\"],\n scrub=build_scrub_query_string(params=[\"code\", \"state\"]),\n ),\n Rule(\n path=\"exception.values.[].stacktrace.frames.[].vars\",\n keys=[\"username\", \"password\"],\n scrub=\"scrub\",\n ),\n ]\n )\n\n # Set up Sentry with the scrubber and the default integrations which\n # includes the LoggingIntegration which will capture messages with level\n # logging.ERROR.\n set_up_sentry(\n sentry_dsn=\"http://user@example.com/1\",\n host_id=\"some host id\",\n release=\"some release name\",\n before_send=scrubber,\n )\n\n\n def kick_up_exception():\n username = \"James\" # noqa\n try:\n raise Exception(\"internal exception\")\n except Exception:\n logging.getLogger(__name__).exception(\"kick_up_exception exception\")\n\nNow you have a scrubber and you've set up the Sentry client to use it. How do\nyou know it's scrubbing the right stuff? How will you know if something changes\nand it's no longer scrubbing the right stuff?\n\nYou can test it like this::\n\n # myapp/test_app.py\n import unittest\n\n from fillmore.test import SentryTestHelper\n\n from myapp.app import kick_up_exception\n\n\n class TestApp(unittest.TestCase):\n def test_scrubber(self):\n # Reuse the existing Sentry configuration and set up the helper\n # to capture Sentry events\n sentry_test_helper = SentryTestHelper()\n with sentry_test_helper.reuse() as sentry_client:\n kick_up_exception()\n\n (event,) = sentry_client.events\n error = event[\"exception\"][\"values\"][0]\n self.assertEqual(error[\"type\"], \"Exception\")\n self.assertEqual(error[\"value\"], \"internal exception\")\n self.assertEqual(\n error[\"stacktrace\"][\"frames\"][0][\"vars\"][\"username\"], \"[Scrubbed]\"\n )\n\nThis creates a Sentry client specific to this test and kicks up an exception in\nthe test and captures it with Sentry.\n\nNote that this is a contrived context using a Sentry client created for this\ntest. You'll want to write tests that use the Sentry client configured for your\napplication and handling events kicked up from different points in your\napplication to make sure that Sentry events are getting scrubbed correctly.\n\nSee Fillmore documentation for explanation and examples.\n\n\nWhy this? Why not other libraries?\n==================================\n\nOther libraries:\n\n* **Have an awkward API that is hard to reason about.**\n\n I'm not scrubbing Sentry events for fun. I need to be able to write scrubbing\n configuration that is exceptionally clear about what it is and isn't doing.\n\n* **Don't covers large portions of the Sentry event structure.**\n\n I need scrubbers that cover the entire event structure as well as some\n of the curious cases like the fact that cookie information shows up twice\n and can be encoded as a string.\n\n* **Aren't resilient.**\n\n The scrubber is running in the context of Sentry reporting an error. If it\n also errors out, then you can end up in situations where you never see errors\n and have no signal that something is horribly wrong. We need scrubbing code\n to be extremely resilient and default to emitting a signal that it's broken.\n\n* **Don't include testing infrastructure.**\n\n I'm not scrubbing Sentry events for fun. I need to know that the scrubbing\n code is working correctly and that it continues to work as we upgrade\n Python, sentry_sdk, and other things.\n\n Having testing infrastructure for making this easier is really important.\n\n\nHistory\n=======\n\n1.1.0 (April 5th, 2023)\n-----------------------\n\n* Switch from flake8 to ruff (#24)\n\n* Add ``fillmore.test.diff_event`` utilify function for comparing a Sentry\n event with an expected structure accounting for ``unittest.mock.ANY``. (#23)\n\n\n1.0.0 (November 8th, 2022)\n--------------------------\n\n* Add support for Python 3.11 (#18)\n\nThis feels stable and I'm using it in multiple production real-world projects,\nso releasing a 1.0.0.\n\n\n0.1.2 (August 1st, 2022)\n------------------------\n\n* Fix examples in documentation so they're linted and tested. Add notes about\n ``fillmore`` logging. Rewrite testing chapter to introduce Fillmore testing\n features in a less muddled way. (#15)\n\n\n0.1.1 (July 25th, 2022)\n-----------------------\n\n* Fix scrubber where paths that were valid, but didn't point to something in an\n event erroneously kicked up a RulePathError. (#12)\n\n* Fix test examples in docs.\n\n* Fix examples in README. Thanks @stevejalim!\n\n\n0.1.0 (June 23rd, 2022)\n-----------------------\n\n* Inital extraction and writing.\n",
"bugtrack_url": null,
"license": "MPLv2",
"summary": "Sentry event scrubber and utilities library",
"version": "1.1.0",
"split_keywords": [
"sentry",
"scrubber"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "304302bc864c21b58b743e43c31e94c26fa5d7776a106cd87fcbb99aae5ced75",
"md5": "b1c37da0f5f641d3a147de6436218a80",
"sha256": "8afe5fa0f59d99bc10f2996d23755860dd8b8f1aebe96c34fd5d0e5e1c2e0771"
},
"downloads": -1,
"filename": "fillmore-1.1.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "b1c37da0f5f641d3a147de6436218a80",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 18096,
"upload_time": "2023-04-05T22:43:45",
"upload_time_iso_8601": "2023-04-05T22:43:45.693276Z",
"url": "https://files.pythonhosted.org/packages/30/43/02bc864c21b58b743e43c31e94c26fa5d7776a106cd87fcbb99aae5ced75/fillmore-1.1.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "dcc4c00cebe0f05160f3a5133c13647ab2d12d618842230ac425ef9de6e1ac57",
"md5": "16fac1bd9f465d11a38b9d1a8e2405bf",
"sha256": "68b2aa27340725026d7be1e4e96c3ec3ef954f66410e1cb65d0d85a692f0fdc1"
},
"downloads": -1,
"filename": "fillmore-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "16fac1bd9f465d11a38b9d1a8e2405bf",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 38575,
"upload_time": "2023-04-05T22:43:47",
"upload_time_iso_8601": "2023-04-05T22:43:47.388926Z",
"url": "https://files.pythonhosted.org/packages/dc/c4/c00cebe0f05160f3a5133c13647ab2d12d618842230ac425ef9de6e1ac57/fillmore-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-05 22:43:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "willkg",
"github_project": "fillmore",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "fillmore"
}
JSON
