# Core Library
[](https://pypi.org/project/shepherd_core)
[](https://pypi.python.org/pypi/shepherd-core)
[](https://github.com/orgua/shepherd-datalib/actions/workflows/py_unittest.yml)
[](https://github.com/astral-sh/ruff)
**Main Documentation**: <https://orgua.github.io/shepherd>
**Source Code**: <https://github.com/orgua/shepherd-datalib>
**Main Project**: <https://github.com/orgua/shepherd>
---
`shepherd-core` is designed as a library and bundles data-models and file-access-routines for the shepherd-testbed, that are used by several codebases.
For postprocessing shepherds .h5-files usage of [shepherd_data](https://pypi.org/project/shepherd_data) is recommended.
## Features
- read and write shepherds hdf5-files
- create, read, write and convert experiments for the testbed
- all required data-models are included
- simulate the virtual source, including virtual harvesters (and virtual converter as a whole)
- connect and query the testbed via a webclient (TestbedClient in alpha-stage)
- offline usage defaults to static demo-fixtures loaded from yaml-files in the model-directories
- work with target-firmwares
- embed, modify, verify, convert
- **Note**: working with ELF-files requires external dependencies, see ``Installation``-Chapter
- decode waveforms (gpio-state & timestamp) to UART
- create an inventory (for deployed versions of software, hardware)
See [official documentation](https://orgua.github.io/shepherd) or [example scripts](https://github.com/orgua/shepherd-datalib/tree/main/shepherd_core/examples) for more details and usage. Most functionality is showcased in both. The [extra](https://github.com/orgua/shepherd-datalib/tree/main/shepherd_core/extra)-directory holds data-generators relevant for the testbed. Notably is a [trafficbench](https://github.com/orgua/TrafficBench)-experiment that's used to derive the link-matrix of the testbed-nodes.
## Config-Models in Detail
These pydantic data-models are used throughout all shepherd interfaces. Users can create an experiment, include their own content and feed it to the testbed.
- orchestration ``/data-models`` with focus on remote shepherd-testbed
- classes of sub-models
- ``/base``: base-classes, configuration and -functionality for all models
- ``/testbed``: meta-data representation of all testbed-components
- ``/content``: reusable user-defined meta-data for fw, h5 and vsrc-definitions
- ``/experiment``: configuration-models including sub-systems
- ``/task``: digestible configs for shepherd-herd or -sheep
- behavior controlled by ``ShpModel`` and ``content``-model
- a basic database is available as fixtures through a ``tb_client``
- fixtures selectable by name & ID
- fixtures support inheritance
- the models support
- auto-completion with neutral / sensible values
- complex and custom datatypes (i.e. PositiveInt, lists-checks on length)
- checking of inputs and type-casting
- generate their own schema (for web-forms)
- pre-validation
- store to & load from yaml with typecheck through wrapper
- documentation
- experiment-definition is designed securely
- types are limited in size (str)
- exposes no internal paths
- experiments can be transformed to task-sets (``TestbedTasks.from_xp()``)
## Compatibility
| OS | PyVersion | Comment |
|---------|--------------|--------------------------------------------|
| Ubuntu | 3.8 - 3.12 | |
| Windows | 3.8 - 3.12 | no support for elf and hex-conversions yet |
| MacOS | 3.8 - 3.12 | hex-conversion missing |
Notes:
- hex-conversion needs a working and accessible objcopy
- elf-supports needs
- ``shepherd-core[elf]`` installs ``pwntools-elf-only``
- most elf-features also still utilize hex-conversion
## Installation
The Library is available via PyPI and can be installed with
```shell
pip install shepherd-core -U
# or for the full experience (includes core)
pip install shepherd-data -U
```
For bleeding-edge-features or dev-work it is possible to install directly from GitHub-Sources (here `dev`-branch):
```Shell
pip install git+https://github.com/orgua/shepherd-datalib.git@dev#subdirectory=shepherd_core -U
```
If you are working with ``.elf``-files (embedding into experiments) you make "objcopy" accessible to python. In Ubuntu, you can either install ``build-essential`` or ``binutils-$ARCH`` with arch being ``msp430`` or ``arm-none-eabi`` for the nRF52.
```shell
sudo apt install build-essential
```
For more advanced work with ``.elf``-files (modify value of symbols / target-ID) you should install
```shell
pip install shepherd-core[elf]
```
and also make sure the prereqs for the [pwntools](https://docs.pwntools.com/en/stable/install.html) are met.
For creating an inventory of the host-system you should install
```shell
pip install shepherd-core[inventory]
```
## Unittests
To run the testbench, follow these steps:
1. Navigate your host-shell into the package-folder and
2. install dependencies
3. run the testbench (~ 320 tests):
```Shell
cd shepherd-datalib/shepherd_core
pip3 install ./[tests]
pytest
```
Raw data
{
"_id": null,
"home_page": null,
"name": "shepherd-core",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Ingmar Splitt <ingmar.splitt@tu-dresden.de>",
"keywords": "testbed, beaglebone, pru, batteryless, energyharvesting, solar",
"author": null,
"author_email": "Ingmar Splitt <ingmar.splitt@tu-dresden.de>",
"download_url": "https://files.pythonhosted.org/packages/12/6d/b7912cabaa9c4e51a7b6b53739aefd640a03dee5043b3ee3598b1807f582/shepherd_core-2024.5.1.tar.gz",
"platform": "unix",
"description": "# Core Library\n\n[](https://pypi.org/project/shepherd_core)\n[](https://pypi.python.org/pypi/shepherd-core)\n[](https://github.com/orgua/shepherd-datalib/actions/workflows/py_unittest.yml)\n[](https://github.com/astral-sh/ruff)\n\n**Main Documentation**: <https://orgua.github.io/shepherd>\n\n**Source Code**: <https://github.com/orgua/shepherd-datalib>\n\n**Main Project**: <https://github.com/orgua/shepherd>\n\n---\n\n`shepherd-core` is designed as a library and bundles data-models and file-access-routines for the shepherd-testbed, that are used by several codebases.\n\nFor postprocessing shepherds .h5-files usage of [shepherd_data](https://pypi.org/project/shepherd_data) is recommended.\n\n## Features\n\n- read and write shepherds hdf5-files\n- create, read, write and convert experiments for the testbed\n - all required data-models are included\n- simulate the virtual source, including virtual harvesters (and virtual converter as a whole)\n- connect and query the testbed via a webclient (TestbedClient in alpha-stage)\n - offline usage defaults to static demo-fixtures loaded from yaml-files in the model-directories\n- work with target-firmwares\n - embed, modify, verify, convert\n - **Note**: working with ELF-files requires external dependencies, see ``Installation``-Chapter\n- decode waveforms (gpio-state & timestamp) to UART\n- create an inventory (for deployed versions of software, hardware)\n\nSee [official documentation](https://orgua.github.io/shepherd) or [example scripts](https://github.com/orgua/shepherd-datalib/tree/main/shepherd_core/examples) for more details and usage. Most functionality is showcased in both. The [extra](https://github.com/orgua/shepherd-datalib/tree/main/shepherd_core/extra)-directory holds data-generators relevant for the testbed. Notably is a [trafficbench](https://github.com/orgua/TrafficBench)-experiment that's used to derive the link-matrix of the testbed-nodes.\n\n## Config-Models in Detail\n\nThese pydantic data-models are used throughout all shepherd interfaces. Users can create an experiment, include their own content and feed it to the testbed.\n\n- orchestration ``/data-models`` with focus on remote shepherd-testbed\n- classes of sub-models\n - ``/base``: base-classes, configuration and -functionality for all models\n - ``/testbed``: meta-data representation of all testbed-components\n - ``/content``: reusable user-defined meta-data for fw, h5 and vsrc-definitions\n - ``/experiment``: configuration-models including sub-systems\n - ``/task``: digestible configs for shepherd-herd or -sheep\n - behavior controlled by ``ShpModel`` and ``content``-model\n- a basic database is available as fixtures through a ``tb_client``\n - fixtures selectable by name & ID\n - fixtures support inheritance\n- the models support\n - auto-completion with neutral / sensible values\n - complex and custom datatypes (i.e. PositiveInt, lists-checks on length)\n - checking of inputs and type-casting\n - generate their own schema (for web-forms)\n - pre-validation\n - store to & load from yaml with typecheck through wrapper\n - documentation\n- experiment-definition is designed securely\n - types are limited in size (str)\n - exposes no internal paths\n- experiments can be transformed to task-sets (``TestbedTasks.from_xp()``)\n\n## Compatibility\n\n| OS | PyVersion | Comment |\n|---------|--------------|--------------------------------------------|\n| Ubuntu | 3.8 - 3.12 | |\n| Windows | 3.8 - 3.12 | no support for elf and hex-conversions yet |\n| MacOS | 3.8 - 3.12 | hex-conversion missing |\n\nNotes:\n- hex-conversion needs a working and accessible objcopy\n- elf-supports needs\n - ``shepherd-core[elf]`` installs ``pwntools-elf-only``\n - most elf-features also still utilize hex-conversion\n\n## Installation\n\nThe Library is available via PyPI and can be installed with\n\n```shell\n pip install shepherd-core -U\n\n # or for the full experience (includes core)\n pip install shepherd-data -U\n```\n\nFor bleeding-edge-features or dev-work it is possible to install directly from GitHub-Sources (here `dev`-branch):\n\n```Shell\npip install git+https://github.com/orgua/shepherd-datalib.git@dev#subdirectory=shepherd_core -U\n```\n\nIf you are working with ``.elf``-files (embedding into experiments) you make \"objcopy\" accessible to python. In Ubuntu, you can either install ``build-essential`` or ``binutils-$ARCH`` with arch being ``msp430`` or ``arm-none-eabi`` for the nRF52.\n\n```shell\n sudo apt install build-essential\n```\n\nFor more advanced work with ``.elf``-files (modify value of symbols / target-ID) you should install\n\n```shell\n pip install shepherd-core[elf]\n```\n\nand also make sure the prereqs for the [pwntools](https://docs.pwntools.com/en/stable/install.html) are met.\n\nFor creating an inventory of the host-system you should install\n\n```shell\n pip install shepherd-core[inventory]\n```\n\n## Unittests\n\nTo run the testbench, follow these steps:\n\n1. Navigate your host-shell into the package-folder and\n2. install dependencies\n3. run the testbench (~ 320 tests):\n\n```Shell\ncd shepherd-datalib/shepherd_core\npip3 install ./[tests]\npytest\n```\n",
"bugtrack_url": null,
"license": null,
"summary": "Programming- and CLI-Interface for the h5-dataformat of the Shepherd-Testbed",
"version": "2024.5.1",
"project_urls": {
"Documentation": "https://github.com/orgua/shepherd-datalib/blob/main/README.md",
"Issues": "https://github.com/orgua/shepherd-datalib/issues",
"Source": "https://pypi.org/project/shepherd-core/"
},
"split_keywords": [
"testbed",
" beaglebone",
" pru",
" batteryless",
" energyharvesting",
" solar"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "dc351284b78b5e689d4466831cc8af5da221186ed13ada4a951aa5bdab132c16",
"md5": "11033bac6456468a2841b10b50a3ada0",
"sha256": "b7e9f16b4854e6237d4c471402878ede3a8e26ffcc6bde4ef82f9c7467064950"
},
"downloads": -1,
"filename": "shepherd_core-2024.5.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "11033bac6456468a2841b10b50a3ada0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 103923,
"upload_time": "2024-05-02T10:00:42",
"upload_time_iso_8601": "2024-05-02T10:00:42.158090Z",
"url": "https://files.pythonhosted.org/packages/dc/35/1284b78b5e689d4466831cc8af5da221186ed13ada4a951aa5bdab132c16/shepherd_core-2024.5.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "126db7912cabaa9c4e51a7b6b53739aefd640a03dee5043b3ee3598b1807f582",
"md5": "f41c7b29a1cab8f7822400c2b92c9998",
"sha256": "70faea95cbd6386ea716ab2423f9d43c650f325cc267d957916fe73e0a29c45a"
},
"downloads": -1,
"filename": "shepherd_core-2024.5.1.tar.gz",
"has_sig": false,
"md5_digest": "f41c7b29a1cab8f7822400c2b92c9998",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 527418,
"upload_time": "2024-05-02T10:00:43",
"upload_time_iso_8601": "2024-05-02T10:00:43.982385Z",
"url": "https://files.pythonhosted.org/packages/12/6d/b7912cabaa9c4e51a7b6b53739aefd640a03dee5043b3ee3598b1807f582/shepherd_core-2024.5.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-05-02 10:00:43",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "orgua",
"github_project": "shepherd-datalib",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "shepherd-core"
}
JSON
