<p align="center">
<img src="https://github.com/LaurenzBeck/Skald/blob/main/docs/images/fantasy_fjord_art.jpg?raw=true"/></a>
</p>
<h1 align="center">
Skáld
</h1>
<p align="center">
📟 a simple and efficient experiment logger for Python 🐍
</p>
<p align="center">
<a href="https://pypi.org/project/skald/"><img alt="PyPI - Version" src="https://img.shields.io/pypi/v/skald?label=%F0%9F%93%A6%20PyPi"></a>
<a href="https://www.repostatus.org/#wip"><img src="https://www.repostatus.org/badges/latest/wip.svg" alt="Project Status: WIP – Initial development is in progress, but there has not yet been a stable, usable release suitable for the public." /></a>
<a href="https://github.com/LaurenzBeck/Skald/actions/workflows/python-package.yaml"><img alt="🐍 Python package" src="https://github.com/LaurenzBeck/Skald/actions/workflows/python-package.yaml/badge.svg"></a>
</p>
<p align="center">
<a href="https://www.python.org/"><img alt="Python" src="https://img.shields.io/badge/Python-3.11|3.12-yellow?logo=python"></a>
<a href="https://python-poetry.org/"><img alt="Poetry" src="https://img.shields.io/badge/Poetry-1.8.2-blue?logo=Poetry"></a>
</p>
<p align="center">
<a href="https://github.com/astral-sh/ruff"><img alt="Ruff" src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json"></a>
<a href="https://github.com/beartype/beartype"><img alt="Beartype" src="https://raw.githubusercontent.com/beartype/beartype-assets/main/badge/bear-ified.svg"></a>
</p>
> Skáld - An Old Norse word for a poet, usually applied to a Norwegian or Icelandic court poet or bard of the period from the 9th century to the 13th. Skaldic verse is marked by its elaborate patterns of metre, rhyme, and alliteration, and by its use of kennings.
---
## 📃 Table of Contents
- [💡 Motivation](#-motivation)
- [👀 Concepts](#-installation)
- [📂 Logging Structure](#-logging-structure)
- [📦 Installation](#-installation)
- [🚀 Usage](#-installation)
---
## 💡 Motivation
During my PhD, I tried different Experiment/Metrics loggers including:
- [TensorBoard](https://www.tensorflow.org/tensorboard)
- [MLFlow](https://mlflow.org/#core-concepts)
- [Weights and Biases](https://wandb.ai/site)
- [DVCLive](https://dvc.org/doc/dvclive)
- other logging solutions from deep learning and continual learning frameworks (mostly stdout and csv loggers)
While those are quite mature logging solutions, which often offer beautiful dashboards, I was looking for something light-weight, local and file-based as DVCLive, but with a more ergonomic and [tidy](https://vita.had.co.nz/papers/tidy-data.pdf) [^1] structure of the logs for simpler consumption and analysis.
The latest of my workflows before Skáld involved using a mixture of DVCLive and a custom logger from [FACIL](https://github.com/mmasana/FACIL/tree/master) [^2] in combination with log crawling CLI scripts and analysis and visualizations performed in jupyter notebooks 🤨
Another problem I faced with those solutions is that all of them offered a single "step" identifier for each logged metric, which is not sufficient for deep learning use-cases outside the conventional epoch-based training loops.
Because I like building python packages and I felt the need to tidy my experiment logs, I created `skald` as a small side project. I hope that some people find some enjoyment in using the package too ❤️
## 👀 Concepts
Skáld is an **experiment logger**, that offers a standardized logging structure and interface to log different aspects of an experiment including:
- **parameters**|**arguments** - meta information and variables, that don't change during the experiment
- **metrics**|**scalars** - single-valued, numeric variables that change during the experiment
- **artifacts** - additional result files like images or plots
Each metric has a unique **name** and a user-defined set of **id variables**, that identify the value of the metric at a certain step.
While a stateful version of Skáld is planned, that updates an id/step variables through some manual call (often in a Callback). The first version is very explicit and requires every id to be logged in each call.
### 📂 Logging Structure
Logs of metrics will be represented by tidy dataframes that are stored as readable `metrics.csv` or more space efficient `metrics.parquet` files.
To save space, parameters will not be included in these dataframes, but in a separate file (`params.yaml`) by default.
Artifacts will be stored in a separate sub-directory (`artifacts/` by default).
Skáld has its own [loguru](https://loguru.readthedocs.io/en/stable/index.html) logger instance and exposes the logging functions.
The logs will stored in a `console.log` file.
If you use the logger as a context manager, *stdout* (~ `print` statements) will also be saved in `console.log`.
## 📦 Installation
The package can be installed with:
```sh
pip install skald
```
> 🧑💻 to install a development environment (which you need if you want to work on the package, instead of just using the package), `cd` into the project's root directory and call:
```bash
poetry install --sync --compile
```
## 🚀 Usage
The API of Skáld is very similar to DVCLive and other loggers.
It also exposes [loguru's](https://loguru.readthedocs.io/en/stable/index.html) logging api, so you can also use a Skáld logger as your terminal logger.
A basic example:
```python
from skald import Logger
# get experiment parameters from CLI arguments, parameter files ...
params: dict = { ... }
# instanciate a logger with a certain run name
with Logger("test-run-1") as logger:
logger.log_params(params)
# experiment logic
metric: float = evaluate(model)
logger.log_metric("accuracy", metric)
logger.success("finished experiment!")
```
To launch the experiment viewer TUI after the run is completed, use `tui=True` in the constructor of the `Logger`.
You can also launch the experiment viewer manually to inspect your logs:
```sh
$skald <experiment_run_dir>
```
---
## 🖼️ ©️ Banner Artwork Attribution
<a rel="license" href="http://creativecommons.org/licenses/by-nc-nd/3.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-nd/3.0/88x31.png" /></a><br />The art in the banner of this README is licensed under a [Creative Commons Attribution-NonCommercial-No Derivatives Works 3.0 License](https://creativecommons.org/licenses/by-nc-nd/3.0/). It was made by [th3dutchzombi3](https://www.deviantart.com/th3dutchzombi3). Check out his beautiful artwork ❤️
---
## 📄 References
[^1]: H. Wickham, “Tidy Data,” Journal of Statistical Software, vol. 59, pp. 1–23, Sep. 2014, doi: 10.18637/jss.v059.i10.
[^2]: M. Masana, X. Liu, B. Twardowski, M. Menta, A. D. Bagdanov, and J. van de Weijer, “Class-incremental learning: survey and performance evaluation on image classification.” arXiv, Oct. 11, 2022. doi: 10.48550/arXiv.2010.15277.
Raw data
{
"_id": null,
"home_page": "https://github.com/laurenzbeck/skald",
"name": "skald",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": "logging, tui",
"author": "Laurenz Farthofer",
"author_email": "laurenz@hey.com",
"download_url": "https://files.pythonhosted.org/packages/f2/f9/b83c4ea84727b6552182baf45e072919ac46e781b3bc835c9de4af3a7422/skald-0.6.0.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <img src=\"https://github.com/LaurenzBeck/Skald/blob/main/docs/images/fantasy_fjord_art.jpg?raw=true\"/></a>\n</p>\n\n<h1 align=\"center\">\n Sk\u00e1ld\n</h1>\n\n<p align=\"center\">\n\ud83d\udcdf a simple and efficient experiment logger for Python \ud83d\udc0d\n</p>\n\n<p align=\"center\">\n <a href=\"https://pypi.org/project/skald/\"><img alt=\"PyPI - Version\" src=\"https://img.shields.io/pypi/v/skald?label=%F0%9F%93%A6%20PyPi\"></a>\n <a href=\"https://www.repostatus.org/#wip\"><img src=\"https://www.repostatus.org/badges/latest/wip.svg\" alt=\"Project Status: WIP \u2013 Initial development is in progress, but there has not yet been a stable, usable release suitable for the public.\" /></a>\n <a href=\"https://github.com/LaurenzBeck/Skald/actions/workflows/python-package.yaml\"><img alt=\"\ud83d\udc0d Python package\" src=\"https://github.com/LaurenzBeck/Skald/actions/workflows/python-package.yaml/badge.svg\"></a>\n</p>\n\n<p align=\"center\">\n <a href=\"https://www.python.org/\"><img alt=\"Python\" src=\"https://img.shields.io/badge/Python-3.11|3.12-yellow?logo=python\"></a>\n <a href=\"https://python-poetry.org/\"><img alt=\"Poetry\" src=\"https://img.shields.io/badge/Poetry-1.8.2-blue?logo=Poetry\"></a>\n</p>\n\n<p align=\"center\">\n <a href=\"https://github.com/astral-sh/ruff\"><img alt=\"Ruff\" src=\"https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json\"></a>\n <a href=\"https://github.com/beartype/beartype\"><img alt=\"Beartype\" src=\"https://raw.githubusercontent.com/beartype/beartype-assets/main/badge/bear-ified.svg\"></a>\n</p>\n\n> Sk\u00e1ld - An Old Norse word for a poet, usually applied to a Norwegian or Icelandic court poet or bard of the period from the 9th century to the 13th. Skaldic verse is marked by its elaborate patterns of metre, rhyme, and alliteration, and by its use of kennings.\n\n---\n\n## \ud83d\udcc3 Table of Contents\n\n- [\ud83d\udca1 Motivation](#-motivation)\n- [\ud83d\udc40 Concepts](#-installation)\n - [\ud83d\udcc2 Logging Structure](#-logging-structure)\n- [\ud83d\udce6 Installation](#-installation)\n- [\ud83d\ude80 Usage](#-installation)\n\n---\n\n## \ud83d\udca1 Motivation\n\nDuring my PhD, I tried different Experiment/Metrics loggers including:\n\n- [TensorBoard](https://www.tensorflow.org/tensorboard)\n- [MLFlow](https://mlflow.org/#core-concepts)\n- [Weights and Biases](https://wandb.ai/site)\n- [DVCLive](https://dvc.org/doc/dvclive)\n- other logging solutions from deep learning and continual learning frameworks (mostly stdout and csv loggers)\n\nWhile those are quite mature logging solutions, which often offer beautiful dashboards, I was looking for something light-weight, local and file-based as DVCLive, but with a more ergonomic and [tidy](https://vita.had.co.nz/papers/tidy-data.pdf) [^1] structure of the logs for simpler consumption and analysis.\n\nThe latest of my workflows before Sk\u00e1ld involved using a mixture of DVCLive and a custom logger from [FACIL](https://github.com/mmasana/FACIL/tree/master) [^2] in combination with log crawling CLI scripts and analysis and visualizations performed in jupyter notebooks \ud83e\udd28\n\nAnother problem I faced with those solutions is that all of them offered a single \"step\" identifier for each logged metric, which is not sufficient for deep learning use-cases outside the conventional epoch-based training loops.\n\nBecause I like building python packages and I felt the need to tidy my experiment logs, I created `skald` as a small side project. I hope that some people find some enjoyment in using the package too \u2764\ufe0f\n\n## \ud83d\udc40 Concepts\n\nSk\u00e1ld is an **experiment logger**, that offers a standardized logging structure and interface to log different aspects of an experiment including:\n\n- **parameters**|**arguments** - meta information and variables, that don't change during the experiment\n- **metrics**|**scalars** - single-valued, numeric variables that change during the experiment\n- **artifacts** - additional result files like images or plots\n\nEach metric has a unique **name** and a user-defined set of **id variables**, that identify the value of the metric at a certain step.\n\nWhile a stateful version of Sk\u00e1ld is planned, that updates an id/step variables through some manual call (often in a Callback). The first version is very explicit and requires every id to be logged in each call.\n\n### \ud83d\udcc2 Logging Structure\n\nLogs of metrics will be represented by tidy dataframes that are stored as readable `metrics.csv` or more space efficient `metrics.parquet` files.\n\nTo save space, parameters will not be included in these dataframes, but in a separate file (`params.yaml`) by default.\n\nArtifacts will be stored in a separate sub-directory (`artifacts/` by default).\n\nSk\u00e1ld has its own [loguru](https://loguru.readthedocs.io/en/stable/index.html) logger instance and exposes the logging functions.\nThe logs will stored in a `console.log` file.\nIf you use the logger as a context manager, *stdout* (~ `print` statements) will also be saved in `console.log`.\n\n## \ud83d\udce6 Installation\n\nThe package can be installed with:\n\n```sh\npip install skald\n```\n\n> \ud83e\uddd1\u200d\ud83d\udcbb to install a development environment (which you need if you want to work on the package, instead of just using the package), `cd` into the project's root directory and call:\n\n```bash\npoetry install --sync --compile\n```\n\n## \ud83d\ude80 Usage\n\nThe API of Sk\u00e1ld is very similar to DVCLive and other loggers.\nIt also exposes [loguru's](https://loguru.readthedocs.io/en/stable/index.html) logging api, so you can also use a Sk\u00e1ld logger as your terminal logger.\n\nA basic example:\n\n```python\nfrom skald import Logger\n\n# get experiment parameters from CLI arguments, parameter files ...\nparams: dict = { ... }\n\n# instanciate a logger with a certain run name\nwith Logger(\"test-run-1\") as logger:\n logger.log_params(params)\n # experiment logic\n metric: float = evaluate(model)\n logger.log_metric(\"accuracy\", metric)\n logger.success(\"finished experiment!\")\n```\n\nTo launch the experiment viewer TUI after the run is completed, use `tui=True` in the constructor of the `Logger`.\n\nYou can also launch the experiment viewer manually to inspect your logs:\n\n```sh\n$skald <experiment_run_dir>\n```\n\n---\n\n## \ud83d\uddbc\ufe0f \u00a9\ufe0f Banner Artwork Attribution\n\n<a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-nd/3.0/\"><img alt=\"Creative Commons License\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc-nd/3.0/88x31.png\" /></a><br />The art in the banner of this README is licensed under a [Creative Commons Attribution-NonCommercial-No Derivatives Works 3.0 License](https://creativecommons.org/licenses/by-nc-nd/3.0/). It was made by [th3dutchzombi3](https://www.deviantart.com/th3dutchzombi3). Check out his beautiful artwork \u2764\ufe0f\n\n---\n\n## \ud83d\udcc4 References\n\n[^1]: H. Wickham, \u201cTidy Data,\u201d Journal of Statistical Software, vol. 59, pp. 1\u201323, Sep. 2014, doi: 10.18637/jss.v059.i10.\n[^2]: M. Masana, X. Liu, B. Twardowski, M. Menta, A. D. Bagdanov, and J. van de Weijer, \u201cClass-incremental learning: survey and performance evaluation on image classification.\u201d arXiv, Oct. 11, 2022. doi: 10.48550/arXiv.2010.15277.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "\ud83d\udcdf a simple and efficient experiment logger for Python \ud83d\udc0d",
"version": "0.6.0",
"project_urls": {
"Homepage": "https://github.com/laurenzbeck/skald",
"Repository": "https://github.com/laurenzbeck/skald"
},
"split_keywords": [
"logging",
" tui"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "642a1a8d36218a3c2b1e18652a68678ffd0db4821f4dd1ec4a2bd837f9ee79bc",
"md5": "1d5bd69ae6d3ad40a499e8d377dd721a",
"sha256": "cff64db23c0d0c7bd2a480a75dd167d70894cbf02798f144c8b4892d16cf2fed"
},
"downloads": -1,
"filename": "skald-0.6.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1d5bd69ae6d3ad40a499e8d377dd721a",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 15443,
"upload_time": "2024-09-09T12:54:43",
"upload_time_iso_8601": "2024-09-09T12:54:43.321384Z",
"url": "https://files.pythonhosted.org/packages/64/2a/1a8d36218a3c2b1e18652a68678ffd0db4821f4dd1ec4a2bd837f9ee79bc/skald-0.6.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f2f9b83c4ea84727b6552182baf45e072919ac46e781b3bc835c9de4af3a7422",
"md5": "27b8a1572753f04c62a50d0590cb0d32",
"sha256": "7803407105ac1ff2fbac6e8333da69eae88b2c07a83b225266a953a1afa24dae"
},
"downloads": -1,
"filename": "skald-0.6.0.tar.gz",
"has_sig": false,
"md5_digest": "27b8a1572753f04c62a50d0590cb0d32",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 16631,
"upload_time": "2024-09-09T12:54:44",
"upload_time_iso_8601": "2024-09-09T12:54:44.481401Z",
"url": "https://files.pythonhosted.org/packages/f2/f9/b83c4ea84727b6552182baf45e072919ac46e781b3bc835c9de4af3a7422/skald-0.6.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-09 12:54:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "laurenzbeck",
"github_project": "skald",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "skald"
}
JSON
