.. image:: https://img.shields.io/pypi/v/pycomex.svg
:target: https://pypi.python.org/pypi/pycomex
.. image:: https://readthedocs.org/projects/pycomex/badge/?version=latest
:target: https://pycomex.readthedocs.io/en/latest/?version=latest
:alt: Documentation Status
.. image:: https://img.shields.io/badge/docs-gh--pages-blue.svg
:target: https://the16thpythonist.github.io/pycomex/
:alt: GitHub Pages Documentation
=============================================
☄️ PyComex - Python Computational Experiments
=============================================
Microframework to improve the experience of running and managing archival records of computational
experiments, such as machine learning and data science experiments, in Python.
===========
🔥 Features
===========
* Automatically create (nested) folder structure for results of each run of an experiment
* Simply attach metadata such as performance metrics to experiment object and they will be automatically
stored as JSON file
* Easily attach file artifacts such as ``matplotlib`` figures to experiment records
* Log messages to stdout as well as permanently store into log file
* Ready-to-use automatically generated boilerplate code for the analysis and post-processing of
experiment data after experiments have terminated.
* Experiment inheritance: Experiment modules can inherit from other modules and extend their functionality
via parameter overwrites and hooks!
* Configuration files: Create YAML config files to run parameter variations without duplicating code
==========================
📦 Installation by Package
==========================
Install the stable version with ``pip``
.. code-block:: console
pip3 install pycomex
=========================
📦 Installation by Source
=========================
Or the most recent development version by cloning the source:
.. code-block:: console
git clone https://github.com/the16thpythonist/pycomex.git
and then installing with either pip
.. code-block:: console
cd pycomex
pip3 install -e .
or poetry
.. code-block:: console
cd pycomex
poetry install
=============
🚀 Quickstart
=============
Each computational experiment has to be bundled as a standalone python module. Important experiment
parameters are placed at the top of this module. All variable names written in upper case will automatically
be detected as parameters of the experiment.
The actual implementation of the experiment execution is placed into a single file which will have to be
decorated with the ``Experiment`` decorator.
Upon execution the experiment, a new archive folder is automatically created. This archive folder can
be used to store all the file artifacts that are created during the experiment.
Some artifacts are stored automatically by default, such as a JSON file containing all data stored in the
main experiment storage, a snapshot of the experiment module and more...
Archiving of metadata, file artifacts and error handling is automatically managed on context exit.
.. code-block:: python
# my_experiment.py
"""
A minimal example demonstrating PyComex experiment structure.
This docstring is saved as experiment metadata.
"""
from pycomex.functional.experiment import Experiment
from pycomex.utils import file_namespace, folder_path
# Experiment parameters (uppercase variables are auto-detected)
MESSAGE: str = "Hello PyComex!"
ITERATIONS: int = 5
# Debug mode: reuses same archive folder for development
__DEBUG__ = True
@Experiment(
base_path=folder_path(__file__), # Results stored relative to this file
namespace=file_namespace(__file__), # Creates folder based on filename
glob=globals(), # Provides access to parameters
)
def experiment(e: Experiment) -> None:
e.log("Starting experiment...")
# Store structured data (creates nested JSON structure)
e["config/message"] = MESSAGE
e["config/iterations"] = ITERATIONS
# Run experiment loop
for i in range(ITERATIONS):
metric = i * 0.1
e.track("metrics/value", metric) # Track time-series data
e.log(f"Iteration {i}: {MESSAGE} (metric: {metric})")
# Save final results and artifacts
e["results/final_metric"] = metric
e.commit_raw("results.txt", f"Final result: {metric}")
# Run experiment when executed directly
experiment.run_if_main()
**Running the Experiment:**
.. code-block:: console
# print help
python my_experiment.py --help
# Basic execution
python my_experiment.py
# Override parameters via command line
python my_experiment.py --MESSAGE "Custom message!" --ITERATIONS 10
This example would create the following folder structure:
.. code-block:: text
my_experiment/
└── debug/
├── experiment_out.log # Complete execution log
├── experiment_meta.json # Experiment metadata and parameters
├── experiment_data.json # All tracked data and stored values
├── experiment_code.py # Snapshot of the original experiment code
├── results.txt # Custom artifact saved via commit_raw()
└── .track/ # Time-series visualizations
└── metrics_value_001.png # Auto-generated plot of tracked metrics
**Key Features:**
* **Automatic Archiving**: Each experiment run creates a timestamped folder with complete execution records
* **Parameter Management**: Uppercase variables are automatically detected as configurable parameters
* **Command-line Overrides**: Parameters can be modified without editing code
* **Structured Data Storage**: Nested data organization using path-like keys (e.g., ``"config/learning_rate"``)
* **Time-series Tracking**: Built-in support for tracking metrics over time with automatic visualization
* **Artifact Management**: Easy saving of files, figures, and custom data formats
==========================
🔧 Command Line Interface
==========================
PyComex provides a powerful CLI accessible via the ``pycomex`` command:
**Creating New Experiments:**
.. code-block:: console
# Create a new experiment module from template
pycomex template experiment my_new_experiment.py
# Create a configuration file from an existing experiment
pycomex template config -e experiment.py -n config_name
**Running Experiments:**
.. code-block:: console
# Run an experiment directly
pycomex run experiment.py
# Run a configuration file
pycomex run config.yml
**Managing Experiment Archives:**
.. code-block:: console
# List recent experiments
pycomex archive list
# Show detailed information about an experiment
pycomex archive overview
# Compress and archive old experiments
pycomex archive compress results/
For more command line options use ``pycomex --help``.
**NOTE.** For an introduction to more advanced features take a look at the examples in
``pycomex/examples`` ( https://github.com/the16thpythonist/pycomex/tree/master/pycomex/examples )
================
📖 Documentation
================
Complete documentation is available at: https://the16thpythonist.github.io/pycomex/
Additional details on specific topics can be found in the ``DOCUMENTATION.rst`` file.
The ``pycomex/examples`` ( https://github.com/the16thpythonist/pycomex/tree/master/pycomex/examples ) folder
contains practical example modules that illustrate key features of the framework.
==========
🤝 Credits
==========
PyComex is built on top of these excellent open source libraries:
* Click_ - Command line interface toolkit
* Rich_ - Rich text and beautiful formatting in the terminal
* Jinja2_ - Modern and designer-friendly templating language
* NumPy_ - The fundamental package for scientific computing
* Matplotlib_ - Comprehensive 2D plotting library
* pytest_ - Testing framework
.. _Click: https://click.palletsprojects.com/
.. _Rich: https://rich.readthedocs.io/
.. _Pydantic: https://docs.pydantic.dev/latest/
.. _Jinja2: https://palletsprojects.com/p/jinja/
.. _NumPy: https://numpy.org/
.. _Matplotlib: https://matplotlib.org/
.. _PyYAML: https://pyyaml.org/
.. _Hatchling: https://hatch.pypa.io/latest/
.. _pytest: https://pytest.org/
Raw data
{
"_id": null,
"home_page": null,
"name": "pycomex",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Jonas Teufel <jonseb1998@gmail.com>",
"keywords": "academia, computational experiments, data science, machine learning",
"author": null,
"author_email": "Jonas Teufel <jonseb1998@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/f5/5f/6834dc9a0ddf2b82c6d926cb01daa68bcb08db400beaba159b4e0656369f/pycomex-0.25.0.tar.gz",
"platform": null,
"description": ".. image:: https://img.shields.io/pypi/v/pycomex.svg\n :target: https://pypi.python.org/pypi/pycomex\n\n.. image:: https://readthedocs.org/projects/pycomex/badge/?version=latest\n :target: https://pycomex.readthedocs.io/en/latest/?version=latest\n :alt: Documentation Status\n\n.. image:: https://img.shields.io/badge/docs-gh--pages-blue.svg\n :target: https://the16thpythonist.github.io/pycomex/\n :alt: GitHub Pages Documentation\n\n=============================================\n\u2604\ufe0f PyComex - Python Computational Experiments\n=============================================\n\nMicroframework to improve the experience of running and managing archival records of computational\nexperiments, such as machine learning and data science experiments, in Python.\n\n===========\n\ud83d\udd25 Features\n===========\n\n* Automatically create (nested) folder structure for results of each run of an experiment\n* Simply attach metadata such as performance metrics to experiment object and they will be automatically\n stored as JSON file\n* Easily attach file artifacts such as ``matplotlib`` figures to experiment records\n* Log messages to stdout as well as permanently store into log file\n* Ready-to-use automatically generated boilerplate code for the analysis and post-processing of\n experiment data after experiments have terminated.\n* Experiment inheritance: Experiment modules can inherit from other modules and extend their functionality\n via parameter overwrites and hooks!\n* Configuration files: Create YAML config files to run parameter variations without duplicating code\n\n==========================\n\ud83d\udce6 Installation by Package\n==========================\n\nInstall the stable version with ``pip``\n\n.. code-block:: console\n\n pip3 install pycomex\n\n=========================\n\ud83d\udce6 Installation by Source\n=========================\n\nOr the most recent development version by cloning the source:\n\n.. code-block:: console\n\n git clone https://github.com/the16thpythonist/pycomex.git\n\nand then installing with either pip \n\n.. code-block:: console\n\n cd pycomex\n pip3 install -e .\n\nor poetry\n\n.. code-block:: console\n\n cd pycomex\n poetry install\n\n=============\n\ud83d\ude80 Quickstart\n=============\n\nEach computational experiment has to be bundled as a standalone python module. Important experiment\nparameters are placed at the top of this module. All variable names written in upper case will automatically\nbe detected as parameters of the experiment.\n\nThe actual implementation of the experiment execution is placed into a single file which will have to be\ndecorated with the ``Experiment`` decorator.\n\nUpon execution the experiment, a new archive folder is automatically created. This archive folder can\nbe used to store all the file artifacts that are created during the experiment.\nSome artifacts are stored automatically by default, such as a JSON file containing all data stored in the\nmain experiment storage, a snapshot of the experiment module and more...\n\nArchiving of metadata, file artifacts and error handling is automatically managed on context exit.\n\n.. code-block:: python\n\n # my_experiment.py\n \"\"\"\n A minimal example demonstrating PyComex experiment structure.\n This docstring is saved as experiment metadata.\n \"\"\"\n from pycomex.functional.experiment import Experiment\n from pycomex.utils import file_namespace, folder_path\n\n # Experiment parameters (uppercase variables are auto-detected)\n MESSAGE: str = \"Hello PyComex!\"\n ITERATIONS: int = 5\n\n # Debug mode: reuses same archive folder for development\n __DEBUG__ = True\n\n @Experiment(\n base_path=folder_path(__file__), # Results stored relative to this file\n namespace=file_namespace(__file__), # Creates folder based on filename\n glob=globals(), # Provides access to parameters\n )\n def experiment(e: Experiment) -> None:\n e.log(\"Starting experiment...\")\n\n # Store structured data (creates nested JSON structure)\n e[\"config/message\"] = MESSAGE\n e[\"config/iterations\"] = ITERATIONS\n\n # Run experiment loop\n for i in range(ITERATIONS):\n metric = i * 0.1\n e.track(\"metrics/value\", metric) # Track time-series data\n e.log(f\"Iteration {i}: {MESSAGE} (metric: {metric})\")\n\n # Save final results and artifacts\n e[\"results/final_metric\"] = metric\n e.commit_raw(\"results.txt\", f\"Final result: {metric}\")\n\n # Run experiment when executed directly\n experiment.run_if_main()\n\n\n**Running the Experiment:**\n\n.. code-block:: console\n\n # print help\n python my_experiment.py --help\n\n # Basic execution\n python my_experiment.py\n\n # Override parameters via command line\n python my_experiment.py --MESSAGE \"Custom message!\" --ITERATIONS 10\n\nThis example would create the following folder structure:\n\n.. code-block:: text\n\n my_experiment/\n \u2514\u2500\u2500 debug/\n \u251c\u2500\u2500 experiment_out.log # Complete execution log\n \u251c\u2500\u2500 experiment_meta.json # Experiment metadata and parameters\n \u251c\u2500\u2500 experiment_data.json # All tracked data and stored values\n \u251c\u2500\u2500 experiment_code.py # Snapshot of the original experiment code\n \u251c\u2500\u2500 results.txt # Custom artifact saved via commit_raw()\n \u2514\u2500\u2500 .track/ # Time-series visualizations\n \u2514\u2500\u2500 metrics_value_001.png # Auto-generated plot of tracked metrics\n\n\n**Key Features:**\n\n* **Automatic Archiving**: Each experiment run creates a timestamped folder with complete execution records\n* **Parameter Management**: Uppercase variables are automatically detected as configurable parameters\n* **Command-line Overrides**: Parameters can be modified without editing code\n* **Structured Data Storage**: Nested data organization using path-like keys (e.g., ``\"config/learning_rate\"``)\n* **Time-series Tracking**: Built-in support for tracking metrics over time with automatic visualization\n* **Artifact Management**: Easy saving of files, figures, and custom data formats\n\n==========================\n\ud83d\udd27 Command Line Interface\n==========================\n\nPyComex provides a powerful CLI accessible via the ``pycomex`` command:\n\n**Creating New Experiments:**\n\n.. code-block:: console\n\n # Create a new experiment module from template\n pycomex template experiment my_new_experiment.py\n\n # Create a configuration file from an existing experiment\n pycomex template config -e experiment.py -n config_name\n\n**Running Experiments:**\n\n.. code-block:: console\n\n # Run an experiment directly\n pycomex run experiment.py\n\n # Run a configuration file\n pycomex run config.yml\n\n**Managing Experiment Archives:**\n\n.. code-block:: console\n\n # List recent experiments\n pycomex archive list\n\n # Show detailed information about an experiment\n pycomex archive overview \n\n # Compress and archive old experiments\n pycomex archive compress results/\n\nFor more command line options use ``pycomex --help``.\n \n**NOTE.** For an introduction to more advanced features take a look at the examples in \n``pycomex/examples`` ( https://github.com/the16thpythonist/pycomex/tree/master/pycomex/examples )\n\n================\n\ud83d\udcd6 Documentation\n================\n\nComplete documentation is available at: https://the16thpythonist.github.io/pycomex/\n\nAdditional details on specific topics can be found in the ``DOCUMENTATION.rst`` file.\n\nThe ``pycomex/examples`` ( https://github.com/the16thpythonist/pycomex/tree/master/pycomex/examples ) folder\ncontains practical example modules that illustrate key features of the framework.\n\n==========\n\ud83e\udd1d Credits\n==========\n\nPyComex is built on top of these excellent open source libraries:\n\n* Click_ - Command line interface toolkit\n* Rich_ - Rich text and beautiful formatting in the terminal\n* Jinja2_ - Modern and designer-friendly templating language\n* NumPy_ - The fundamental package for scientific computing\n* Matplotlib_ - Comprehensive 2D plotting library\n* pytest_ - Testing framework\n\n.. _Click: https://click.palletsprojects.com/\n.. _Rich: https://rich.readthedocs.io/\n.. _Pydantic: https://docs.pydantic.dev/latest/\n.. _Jinja2: https://palletsprojects.com/p/jinja/\n.. _NumPy: https://numpy.org/\n.. _Matplotlib: https://matplotlib.org/\n.. _PyYAML: https://pyyaml.org/\n.. _Hatchling: https://hatch.pypa.io/latest/\n.. _pytest: https://pytest.org/\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Python Computational Experiments",
"version": "0.25.0",
"project_urls": {
"Documentation": "https://pycomex.readthedocs.io",
"Homepage": "https://github.com/the16thpythonist/pycomex",
"Issues": "https://github.com/the16thpythonist/pycomex/issues",
"Repository": "https://github.com/the16thpythonist/pycomex"
},
"split_keywords": [
"academia",
" computational experiments",
" data science",
" machine learning"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "70d370acb033fb42e351f6069a8708f6c9bbf035e36de9cfdccd79dc0b89f599",
"md5": "efe9c02bd9ccc7138c26833553c73995",
"sha256": "7356bd07bf3ec8386b2e466cf85e76398cd122813ffeaf3db92ce4ed99ee7e3d"
},
"downloads": -1,
"filename": "pycomex-0.25.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "efe9c02bd9ccc7138c26833553c73995",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 147093,
"upload_time": "2025-10-15T08:47:11",
"upload_time_iso_8601": "2025-10-15T08:47:11.871674Z",
"url": "https://files.pythonhosted.org/packages/70/d3/70acb033fb42e351f6069a8708f6c9bbf035e36de9cfdccd79dc0b89f599/pycomex-0.25.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "f55f6834dc9a0ddf2b82c6d926cb01daa68bcb08db400beaba159b4e0656369f",
"md5": "4c45a8a1d1727f79157fa6f8df28bd04",
"sha256": "8d5684c283b9380d262d67ba34062215475e45469c2eb66ebe4a432df133618e"
},
"downloads": -1,
"filename": "pycomex-0.25.0.tar.gz",
"has_sig": false,
"md5_digest": "4c45a8a1d1727f79157fa6f8df28bd04",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 210601,
"upload_time": "2025-10-15T08:47:13",
"upload_time_iso_8601": "2025-10-15T08:47:13.881635Z",
"url": "https://files.pythonhosted.org/packages/f5/5f/6834dc9a0ddf2b82c6d926cb01daa68bcb08db400beaba159b4e0656369f/pycomex-0.25.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-15 08:47:13",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "the16thpythonist",
"github_project": "pycomex",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "pycomex"
}
JSON
