# Apparent: a small library for observability
I created this library because in every job I've had I had to create some version of it, and I generally
Needed something simpler than some of the giant tools and frameworks, which I can use on my desktop
Without subscribing to some observability service.
This library is not intended to replace or compete with observability tools and dashboard, but is intended
for day to day development use with a longer term objective of easy connectivity to observability
Tools and frameworks. It is intended to remain small and compact.
## Installation
`python -m pip install apparent`
## Collect timing information
The most common use and the first to release as OSS is the timers functionality. There are many tools out there that do
Broadly similar things. In particular, two categories of tools are similar in some ways:
* Profilers (for example [yappi](https://github.com/sumerc/yappi))
* Micro-benchmarking tools of various kinds
This library is neither of the above. The idea here is to have lightweight, possibly permanent instrumentation for timing measurements of functions or sections of code of particular interest, which you may or may not expose to some
monitoring facility, or simply expose during your debugging. The key drivers for this are:
* Very easy to add the instrumentation
* Does not require any additional dependencies, and optional ones are very small too
* Can produce reasonable reports easily, which should not require difficult interpretation
* Can collect timing data over a large number of samples with little space or time overhead
* Assumes deep familiarity of your own code base: as opposed to a profiler - where you may be working with someone else's
code base and trying to discover some mystery bottleneck - the user of this library has more of an outside-in view
where you have a pretty good idea upfront what you are interested in measuring (e.g. a specific computation, query, API endpoint).
Profilers are micro-benchmarking tools can be used commonly along with this for their own purposes.
### Examples
Measuring the timing of a function with the `@timed` decorator:
```python
from apparent.timing import timed
@timed
def f(x):
...
```
Measuring the timing of a section of code with a registered timer.
```python
from apparent.timing import TimerRegistry
while some_consition_applies():
do_something()
with TimerRegistry.get('expensive_section'):
do_expensive_section()
...
...
```
If you do not want to use a registered timer you can just use a `Timer` directly. But then you have to hold on to the instance
to get any results out of it. For example:
```python
from apparent.timing import Timer
timer = Timer('timer1')
for i in range(5):
with timer:
...
result = timer.results().round(4).dict()
```
#### Getting measurements from a timer
To get a measurement from a timer you need an instance of a `Timer` that has been used to collect the data. Then call the `results()` method on it. This returns a `TimerResults` instance summarizing the state of the timer at the time of the call. You can refer to the source code for detail, but a broad outline (may change over time - the source code is more authoritative than the partial copy below) is:
```python
@dataclass
class TimerResults:
"""Results from a timer: basic descriptive statistics (default in seconds).
This class is generally produced by timers and not instantiated directly by library users"""
total_time: float
count: int
mean: float
stdevp: float
min: float
max: float
timer_name: str
units: Units = Units.SEC
def convert(self, units: Units) -> 'TimerResults':
"""Convert the timer results to the given units"""
... # code removed for clarity
def dict(self, verbose: bool = True) -> dict:
"""Convert the timer results to a dictionary representation."""
... # code removed for clarity
def round(self, digits: int = 1) -> 'TimerResults':
"""Round the timer results to the given number of digits. Useful for presentation and for comparison."""
... # code removed for clarity
```
In most cases you will be using primarily the `@timed` decorator and occasionally `TimerRegistry.get(name)`. Both of these
Result in named timers being registered in the timer registry and being retrievable by `TimerRegistry.get()`. Alternatively, all
Timer names can be retrieved by `TimerRegistry.names()` and all registered instances can be retrieved via `TimerRegistry.timers()`. Using these you can produce a full listing of results of all timed code on demand.
The timer registry has some additional functionality, such as replacing the default instance with a custom registry, but those functionalities are beyond the scope of this document and will be discussed in a future document as the functionality matures.
##### Builtin reporting
The `apparent.reports` module contains basic reporting functionality. Initially, what you can do
is to produce a summary table of all your timers using the `timer_summary_table()` function,
which has options for the level of details, the units, the rounding, and the sorting. But default
it would do the "reasonable thing": using milliseconds rounded to 3 digits, sorted descending by mean
time. But you can change the results as you like.
The report is produced as a CSV-like table (list of lists of strings) that can readily be used by various
libraries. Most of the time you would want to print it nicely to the stdout of your terminal,
and for that the [tabulate](https://pypi.org/project/tabulate/) package is great. For example, you can do
`print(tabulate(report, headers='firstrow', tablefmt='rounded_outline'))`
and you will get something like this:
```
╭──────────────────────────────┬─────────┬─────────┬─────────╮
│ timer_name │ mean │ count │ max │
├──────────────────────────────┼─────────┼─────────┼─────────┤
│ timer_reports_tests.slower() │ 102.325 │ 2 │ 103.764 │
│ timer_reports_tests.slow() │ 53.787 │ 2 │ 53.787 │
│ timer_reports_tests.fast() │ 0.002 │ 3 │ 0.004 │
╰──────────────────────────────┴─────────┴─────────┴─────────╯
```
A direct dependency on tabulate was not added to the code base at the time of this writing.
## Counters and metrics (todo)
**TBD** *support for collecting ascending counters and metrics for exposing to observability frameworks.**
Raw data
{
"_id": null,
"home_page": null,
"name": "apparent",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "Arnon Moscona <arnon@moscona.com>",
"keywords": "apparent,observability,monitoring,timers,timing,counters,metrics",
"author": null,
"author_email": "Arnon Moscona <arnon@moscona.com>",
"download_url": "https://files.pythonhosted.org/packages/aa/d7/44d8976a6c85e13814a63b35c21a84839523b0281c8eb4cfb60f71767ee3/apparent-0.1.4.tar.gz",
"platform": null,
"description": "# Apparent: a small library for observability\n\nI created this library because in every job I've had I had to create some version of it, and I generally\nNeeded something simpler than some of the giant tools and frameworks, which I can use on my desktop\nWithout subscribing to some observability service.\n\nThis library is not intended to replace or compete with observability tools and dashboard, but is intended \nfor day to day development use with a longer term objective of easy connectivity to observability \nTools and frameworks. It is intended to remain small and compact.\n\n## Installation\n\n`python -m pip install apparent`\n\n## Collect timing information\n\nThe most common use and the first to release as OSS is the timers functionality. There are many tools out there that do\nBroadly similar things. In particular, two categories of tools are similar in some ways:\n\n* Profilers (for example [yappi](https://github.com/sumerc/yappi))\n* Micro-benchmarking tools of various kinds\n\nThis library is neither of the above. The idea here is to have lightweight, possibly permanent instrumentation for timing measurements of functions or sections of code of particular interest, which you may or may not expose to some \nmonitoring facility, or simply expose during your debugging. The key drivers for this are:\n\n* Very easy to add the instrumentation\n* Does not require any additional dependencies, and optional ones are very small too\n* Can produce reasonable reports easily, which should not require difficult interpretation\n* Can collect timing data over a large number of samples with little space or time overhead\n* Assumes deep familiarity of your own code base: as opposed to a profiler - where you may be working with someone else's\n code base and trying to discover some mystery bottleneck - the user of this library has more of an outside-in view\n where you have a pretty good idea upfront what you are interested in measuring (e.g. a specific computation, query, API endpoint).\n\nProfilers are micro-benchmarking tools can be used commonly along with this for their own purposes.\n\n### Examples\n\nMeasuring the timing of a function with the `@timed` decorator:\n\n```python\nfrom apparent.timing import timed\n\n@timed\ndef f(x):\n ... \n```\n\nMeasuring the timing of a section of code with a registered timer.\n\n```python\nfrom apparent.timing import TimerRegistry\n\nwhile some_consition_applies():\n do_something()\n with TimerRegistry.get('expensive_section'):\n do_expensive_section()\n ...\n ...\n```\n\nIf you do not want to use a registered timer you can just use a `Timer` directly. But then you have to hold on to the instance \nto get any results out of it. For example:\n\n```python\nfrom apparent.timing import Timer\n\ntimer = Timer('timer1')\nfor i in range(5):\n with timer:\n ...\n\nresult = timer.results().round(4).dict()\n```\n\n#### Getting measurements from a timer\n\nTo get a measurement from a timer you need an instance of a `Timer` that has been used to collect the data. Then call the `results()` method on it. This returns a `TimerResults` instance summarizing the state of the timer at the time of the call. You can refer to the source code for detail, but a broad outline (may change over time - the source code is more authoritative than the partial copy below) is:\n\n```python\n@dataclass\nclass TimerResults:\n \"\"\"Results from a timer: basic descriptive statistics (default in seconds).\n This class is generally produced by timers and not instantiated directly by library users\"\"\"\n total_time: float\n count: int\n mean: float\n stdevp: float\n min: float\n max: float\n timer_name: str\n units: Units = Units.SEC\n\n def convert(self, units: Units) -> 'TimerResults':\n \"\"\"Convert the timer results to the given units\"\"\"\n ... # code removed for clarity\n\n def dict(self, verbose: bool = True) -> dict:\n \"\"\"Convert the timer results to a dictionary representation.\"\"\"\n ... # code removed for clarity\n\n def round(self, digits: int = 1) -> 'TimerResults':\n \"\"\"Round the timer results to the given number of digits. Useful for presentation and for comparison.\"\"\"\n ... # code removed for clarity\n```\n\nIn most cases you will be using primarily the `@timed` decorator and occasionally `TimerRegistry.get(name)`. Both of these \nResult in named timers being registered in the timer registry and being retrievable by `TimerRegistry.get()`. Alternatively, all\nTimer names can be retrieved by `TimerRegistry.names()` and all registered instances can be retrieved via `TimerRegistry.timers()`. Using these you can produce a full listing of results of all timed code on demand.\n\nThe timer registry has some additional functionality, such as replacing the default instance with a custom registry, but those functionalities are beyond the scope of this document and will be discussed in a future document as the functionality matures.\n\n##### Builtin reporting\n\nThe `apparent.reports` module contains basic reporting functionality. Initially, what you can do\nis to produce a summary table of all your timers using the `timer_summary_table()` function,\nwhich has options for the level of details, the units, the rounding, and the sorting. But default\nit would do the \"reasonable thing\": using milliseconds rounded to 3 digits, sorted descending by mean \ntime. But you can change the results as you like.\n\nThe report is produced as a CSV-like table (list of lists of strings) that can readily be used by various\nlibraries. Most of the time you would want to print it nicely to the stdout of your terminal,\nand for that the [tabulate](https://pypi.org/project/tabulate/) package is great. For example, you can do \n`print(tabulate(report, headers='firstrow', tablefmt='rounded_outline'))`\n\nand you will get something like this:\n\n```\n\u256d\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 timer_name \u2502 mean \u2502 count \u2502 max \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 timer_reports_tests.slower() \u2502 102.325 \u2502 2 \u2502 103.764 \u2502\n\u2502 timer_reports_tests.slow() \u2502 53.787 \u2502 2 \u2502 53.787 \u2502\n\u2502 timer_reports_tests.fast() \u2502 0.002 \u2502 3 \u2502 0.004 \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\n```\n\nA direct dependency on tabulate was not added to the code base at the time of this writing.\n\n## Counters and metrics (todo)\n\n**TBD** *support for collecting ascending counters and metrics for exposing to observability frameworks.**\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "A toolkit for code observability in-process data collection: timing, counters, and metrics.",
"version": "0.1.4",
"project_urls": {
"repository": "https://github.com/arnonmoscona/apparent"
},
"split_keywords": [
"apparent",
"observability",
"monitoring",
"timers",
"timing",
"counters",
"metrics"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "e7923bc94431b01b2766ed4d43af415ee62943f29111066294be2c117ef941de",
"md5": "98e029d3eb35e6a145ebfd2b3a409c98",
"sha256": "77f179129f463ffd9cec04ba5dd01094e612d1f445c043211889d096be8db399"
},
"downloads": -1,
"filename": "apparent-0.1.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "98e029d3eb35e6a145ebfd2b3a409c98",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 12697,
"upload_time": "2023-08-02T21:26:52",
"upload_time_iso_8601": "2023-08-02T21:26:52.911208Z",
"url": "https://files.pythonhosted.org/packages/e7/92/3bc94431b01b2766ed4d43af415ee62943f29111066294be2c117ef941de/apparent-0.1.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "aad744d8976a6c85e13814a63b35c21a84839523b0281c8eb4cfb60f71767ee3",
"md5": "6ec2de368ab96a8e0e260203c81ee965",
"sha256": "51583190066de91ce3605d6e9f31f619bca54eea52ef5ba95c0888a3113c0b9b"
},
"downloads": -1,
"filename": "apparent-0.1.4.tar.gz",
"has_sig": false,
"md5_digest": "6ec2de368ab96a8e0e260203c81ee965",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 37480,
"upload_time": "2023-08-02T21:26:54",
"upload_time_iso_8601": "2023-08-02T21:26:54.631105Z",
"url": "https://files.pythonhosted.org/packages/aa/d7/44d8976a6c85e13814a63b35c21a84839523b0281c8eb4cfb60f71767ee3/apparent-0.1.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-02 21:26:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "arnonmoscona",
"github_project": "apparent",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "apparent"
}
JSON
