# Genesis Dev Tools
The tools to manager life cycle of genesis projects
# Requirements
Before you can install and use genesis tools you need to install several requirements:
- [packer](https://www.packer.io/)
- [libvirt](https://libvirt.org/)
- [qemu](https://www.qemu.org/)
## Ubuntu
Install packages
```sh
sudo apt update
sudo apt install qemu-kvm qemu-utils libvirt-daemon-system libvirt-dev mkisofs
```
Add user to group
```sh
sudo adduser $USER libvirt
sudo adduser $USER kvm
```
Install packer like described in [this article](https://yandex.cloud/ru/docs/tutorials/infrastructure-management/packer-quickstart#from-y-mirror)
# Install
To install the `genesis-devtools` package, follow these steps:
1. Clone the repository:
```sh
git clone https://github.com/infraguys/genesis_devtools.git
```
2. Navigate to the project directory:
```sh
cd genesis_devtools
```
3. Install the required dependencies:
```sh
pip install -r requirements.txt
```
4. Install the package using pip:
```sh
pip install .
```
# Quickstart
## Build
Firstly you need to build the genesis project. Navigate to your project directory and run the genesis build command, specifying the path to your project root directory as an argument. This will build the project according to the configuration defined in the `genesis.yaml` file.
Here are some examples of how to use the build command:
```sh
genesis build /path/to/my/project
```
There are some useful options for the genesis build command.
- Build a Genesis project with a custom developer key path:
```sh
genesis build -i /path/to/my/developer/key /path/to/my/project
```
- Build a Genesis project with the --force option to rebuild if the output already exists:
```sh
genesis build -f /path/to/my/project
```
Look at the `genesis build --help` command for more options.
## Bootstrap
To bootstrap a Genesis installation locally, use the genesis bootstrap command. This command creates and boots a virtual machine with the specified Genesis image.
One of the key options for the bootstrap command is `--launch-mode`, which allows you to specify the launch mode for the application. There are three available modes:
- `element`: This is the default mode, which launches the installation as a single element.
- `core`: This mode launches the installation as a core.
- `custom`: This mode allows you to launch the installation with a custom configuration.
Here are some examples of how to use the `--launch-mode` option:
```sh+
genesis bootstrap -i output/genesis-element.raw
```
Launch the installation in `core` mode:
```sh
genesis bootstrap -i output/genesis-core.raw -m core
```
# Usage
The package provides a command line interface for building genesis projects, managing genesis installations and cover many other useful aspects. To use the command line interface, run the `genesis` command from the command line. For full documentation about CLI commands, run `genesis --help`.
For every genesis project the directory `genesis` should exist in the project root. The project configuration file should be named `genesis.yaml` in this directory. For example my project structure looks like this:
```sh
.
├── my_project
│ └── main.py
├── project_settings.json
├── requirements.txt
├── setup.cfg
├── setup.py
└── README.md
```
The project should be extended as follows:
```sh
.
├── my_project
│ └── main.py
├── genesis
│ └── genesis.yaml
├── README.md
├── project_settings.json
├── requirements.txt
├── setup.cfg
├── setup.py
└── README.md
```
## Genesis configuration file
The `genesis.yaml` file contains the configuration for the genesis project. It should be placed in the `genesis` directory. It consists of several sections such as build, deploy, etc.
Example of the `genesis.yaml` file:
```yaml
# Build section. It describes the build process of the project.
build:
# Dependencies of the project
# This section is used to specify build dependencies
# for the project
deps:
# Target path in the image
- dst: /opt/genesis_core
# Local path of the build machine
path:
src: ../../genesis_core
# This section describes elements of the project.
# Images, artifacts and manifests for every element.
elements:
# List of images in the element
- images:
- name: genesis-core
format: raw
# OS profile for the image
profile: ubuntu_24
# Provisioning script
script: images/install.sh
# Override image build parameters, for instance Packer parameters
override:
disk_size: "10G"
manifest: manifests/genesis-core.yaml
# List of artifacts in the element
artifacts:
- configs/my-cofig.yaml
- templates/my-template.yaml
```
## Build project
The `genesis build` command builds the project. The build process is described in the `build` section of the `genesis.yaml` file. The mandatory argument is path to the project root directory.
Build a Genesis project.
```sh
genesis build my_project
```
This will build the Genesis project in the `my_project` directory using the default configuration file will be located in `my_project/genesis/genesis.yaml`.
After build the project output artifacts will be stored in the `output` directory.
For detailed information about the `genesis build` command run `genesis build --help`.
## Manager Genesis installation locally
To bootstrap genesis installation locally, run the `genesis bootstrap` command. The mandatory argument is path to the genesis image. For instance,
```sh
genesis bootstrap output/genesis-core.raw
```
This command will create and boot a virtual machine with the specified genesis image `output/genesis-core.raw`. The default name of the installation is `genesis-core`. For detailed information about the `genesis bootstrap` command run `genesis bootstrap --help`.
To connect to the genesis installation, run the `genesis ssh` command. For instance,
```sh
genesis ssh
```
To list genesis installations, run the `genesis ps` command. For instance,
```sh
genesis ps
```
To delete genesis installation, run the `genesis delete` command. For instance,
```sh
genesis delete genesis-core
```
### Stand specification
For more complicated cases when you need to start an installation with several nodes (that are treated as baremetal nodes) you may use `stand specification`. The stand specification is a YAML file describes the genesis installation. it specifies how many bootstrap nodes, how many baremetal nodes, their characteristics and other parameters. For instance, it may look like this:
[Small stand](data/stands/stand-small.yaml)
Use option `--stand-spec` or `-s` for the `bootstrap` command to specify file with stand description. For instance,
```sh
genesis bootstrap -i output/genesis-core.raw -f -m core -s data/stands/stand-small.yaml
```
## Versions
Semver is used for project versioning. There are three types of versions:
- stable version - format `X.Y.Z`
- release candidate version - format `X.Y.Z-rc+YYYYMMDDHHMMSS.commit_hash[:8]`
- development version - format `X.Y.Z-dev+YYYYMMDDHHMMSS.commit_hash[:8]`
Stable version looks like `1.0.0`, only three digits. Release candidate version looks like `0.0.1-rc+20250224092842.e11604e9`. Development version looks like `0.0.1-dev+20250223180245.fb195339`.
To get project version, run the `genesis get-version` with path to the project root directory as an argument to the command. For instance,
```sh
genesis get-version /path/to/my_project
```
## Backups
The `backup` command is used to backup the current installation. This guide provides a brief overview of the command and its usage, along with several examples to get you started.
The basic syntax of the `backup` command is as follows:
```bash
genesis backup -d /path/to/backup/directory
```
Run the above command to backup the current installation to the specified directory. The command will create a subdirectory for each domain and copy its disks to the specified directory. The backup will repeat every 24 hours by default.
To configure period of backup, run the following command:
```bash
genesis backup -d /path/to/backup/directory -p 1h
```
The above command will backup the current installation to the specified directory every hour.
There is an option to do a backup once and exit:
```bash
genesis backup -d /path/to/backup/directory -oneshot
```
The above command will backup the current installation to the specified directory once and exit.
### Basic Backup
Run backup periodically of all libvirt domains and store it in the current directory:
```bash
genesis backup
```
### Backup Specific Domain
Run backup periodically of a specific libvirt domain and store it in the current directory:
```bash
genesis backup -n domain-name
```
### Custom Periodic Backup
Run backup periodically of all libvirt domains and store it in the current directory:
```bash
genesis backup -p 1h
```
Available periods are: `1m`, `5m`, `15m`, `30m`, `1h`, `3h`, `6h`, `12h`, `1d`, `3d`, `7d`
### OneShot Backup
Run backup of all libvirt domains and store it in the current directory:
```bash
genesis backup -oneshot
```
This command will backup the current installation to the specified directory once and exit.
### Compressed Backup
Run backup of all libvirt domains and store a compressed archive of the backup in the current directory:
```bash
genesis backup --compress
```
### Encrypted Backup
Run backup of all libvirt domains and store an encrypted archive of the backup in the current directory:
NOTE: It works only with `--compress` flag
You need to set the environment variables `GEN_DEV_BACKUP_KEY` and `GEN_DEV_BACKUP_IV` to encrypt the backup. The key and IV must be greater or equal to 6 bytes and less or equal to 16 bytes.
```bash
export GEN_DEV_BACKUP_KEY=secret_key
export GEN_DEV_BACKUP_IV=secret_iv
genesis backup --compress --encrypt
```
For decryption, use the `genesis backup-decrypt` command.
```bash
genesis backup-decrypt backup.tar.gz.encrypted
```
### Restore backups
Look at the wiki page [restore backups](https://github.com/infraguys/genesis_devtools/wiki/Backups#restore-backup) for information on how to restore backups.
### Rotation
For the periodic backup, you can set the number of backups to keep(rotation number). The default value is 5. Use the `--rotate` option to set the number of backups to keep:
```bash
genesis backup --rotate 10
```
### Prevent disk overflow
Backups can take a lot of disk space and it can be a reason to crash the whole system if a disk will be full. To prevent such a situation you can set a threshold for disk space that should be free. If during a backup process this threshold will be reached, the backup process will stop. Use the `--min-free-space` option to set the threshold in GB.
```bash
genesis backup --min-free-space 50
```
Raw data
{
"_id": null,
"home_page": "https://github.com/infraguys/genesis_devtools",
"name": "genesis-devtools",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Genesis Corporation",
"author_email": "anton.kremenetsky@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/45/0f/59f3a676c06e8b452761016f9c39f4f454200df82af760bca338206d161f/genesis_devtools-0.3.0.tar.gz",
"platform": null,
"description": "\n\n\n\n\n\n# Genesis Dev Tools\n\nThe tools to manager life cycle of genesis projects\n\n# Requirements\n\nBefore you can install and use genesis tools you need to install several requirements:\n- [packer](https://www.packer.io/)\n- [libvirt](https://libvirt.org/)\n- [qemu](https://www.qemu.org/)\n\n## Ubuntu\n\nInstall packages\n```sh\nsudo apt update\nsudo apt install qemu-kvm qemu-utils libvirt-daemon-system libvirt-dev mkisofs\n```\n\nAdd user to group\n```sh\nsudo adduser $USER libvirt\nsudo adduser $USER kvm\n```\n\nInstall packer like described in [this article](https://yandex.cloud/ru/docs/tutorials/infrastructure-management/packer-quickstart#from-y-mirror)\n\n# Install\n\nTo install the `genesis-devtools` package, follow these steps:\n\n1. Clone the repository:\n ```sh\n git clone https://github.com/infraguys/genesis_devtools.git\n ```\n\n2. Navigate to the project directory:\n ```sh\n cd genesis_devtools\n ```\n\n3. Install the required dependencies:\n ```sh\n pip install -r requirements.txt\n ```\n\n4. Install the package using pip:\n ```sh\n pip install .\n ```\n\n# Quickstart\n\n## Build\nFirstly you need to build the genesis project. Navigate to your project directory and run the genesis build command, specifying the path to your project root directory as an argument. This will build the project according to the configuration defined in the `genesis.yaml` file.\n\nHere are some examples of how to use the build command:\n```sh\ngenesis build /path/to/my/project\n```\n\nThere are some useful options for the genesis build command.\n- Build a Genesis project with a custom developer key path:\n```sh\ngenesis build -i /path/to/my/developer/key /path/to/my/project\n```\n\n- Build a Genesis project with the --force option to rebuild if the output already exists:\n```sh\ngenesis build -f /path/to/my/project\n```\n\nLook at the `genesis build --help` command for more options.\n\n## Bootstrap\n\nTo bootstrap a Genesis installation locally, use the genesis bootstrap command. This command creates and boots a virtual machine with the specified Genesis image.\n\nOne of the key options for the bootstrap command is `--launch-mode`, which allows you to specify the launch mode for the application. There are three available modes:\n\n- `element`: This is the default mode, which launches the installation as a single element.\n- `core`: This mode launches the installation as a core.\n- `custom`: This mode allows you to launch the installation with a custom configuration.\n\nHere are some examples of how to use the `--launch-mode` option:\n```sh+\ngenesis bootstrap -i output/genesis-element.raw\n```\n\nLaunch the installation in `core` mode:\n```sh\ngenesis bootstrap -i output/genesis-core.raw -m core\n```\n\n# Usage\n\nThe package provides a command line interface for building genesis projects, managing genesis installations and cover many other useful aspects. To use the command line interface, run the `genesis` command from the command line. For full documentation about CLI commands, run `genesis --help`.\n\nFor every genesis project the directory `genesis` should exist in the project root. The project configuration file should be named `genesis.yaml` in this directory. For example my project structure looks like this:\n\n```sh\n.\n\u251c\u2500\u2500 my_project\n\u2502 \u2514\u2500\u2500 main.py\n\u251c\u2500\u2500 project_settings.json\n\u251c\u2500\u2500 requirements.txt\n\u251c\u2500\u2500 setup.cfg\n\u251c\u2500\u2500 setup.py\n\u2514\u2500\u2500 README.md\n```\n\nThe project should be extended as follows:\n\n```sh\n.\n\u251c\u2500\u2500 my_project\n\u2502 \u2514\u2500\u2500 main.py\n\u251c\u2500\u2500 genesis\n\u2502 \u2514\u2500\u2500 genesis.yaml\n\u251c\u2500\u2500 README.md\n\u251c\u2500\u2500 project_settings.json\n\u251c\u2500\u2500 requirements.txt\n\u251c\u2500\u2500 setup.cfg\n\u251c\u2500\u2500 setup.py\n\u2514\u2500\u2500 README.md\n```\n\n## Genesis configuration file\n\nThe `genesis.yaml` file contains the configuration for the genesis project. It should be placed in the `genesis` directory. It consists of several sections such as build, deploy, etc.\n\nExample of the `genesis.yaml` file:\n\n```yaml\n# Build section. It describes the build process of the project.\nbuild:\n # Dependencies of the project\n # This section is used to specify build dependencies\n # for the project\n deps:\n # Target path in the image\n - dst: /opt/genesis_core\n # Local path of the build machine\n path:\n src: ../../genesis_core\n \n # This section describes elements of the project.\n # Images, artifacts and manifests for every element. \n elements:\n # List of images in the element\n - images:\n - name: genesis-core\n format: raw\n \n # OS profile for the image\n profile: ubuntu_24\n\n # Provisioning script\n script: images/install.sh\n\n # Override image build parameters, for instance Packer parameters\n override:\n disk_size: \"10G\"\n\n manifest: manifests/genesis-core.yaml\n \n # List of artifacts in the element\n artifacts:\n - configs/my-cofig.yaml\n - templates/my-template.yaml\n```\n\n## Build project\n\nThe `genesis build` command builds the project. The build process is described in the `build` section of the `genesis.yaml` file. The mandatory argument is path to the project root directory.\n\nBuild a Genesis project.\n```sh\ngenesis build my_project\n```\n\nThis will build the Genesis project in the `my_project` directory using the default configuration file will be located in `my_project/genesis/genesis.yaml`.\n\nAfter build the project output artifacts will be stored in the `output` directory.\nFor detailed information about the `genesis build` command run `genesis build --help`.\n\n## Manager Genesis installation locally\n\nTo bootstrap genesis installation locally, run the `genesis bootstrap` command. The mandatory argument is path to the genesis image. For instance,\n\n```sh\ngenesis bootstrap output/genesis-core.raw\n```\n\nThis command will create and boot a virtual machine with the specified genesis image `output/genesis-core.raw`. The default name of the installation is `genesis-core`. For detailed information about the `genesis bootstrap` command run `genesis bootstrap --help`.\n\nTo connect to the genesis installation, run the `genesis ssh` command. For instance,\n```sh\ngenesis ssh\n```\n\nTo list genesis installations, run the `genesis ps` command. For instance,\n\n```sh\ngenesis ps\n```\n\nTo delete genesis installation, run the `genesis delete` command. For instance,\n\n```sh\ngenesis delete genesis-core\n```\n\n### Stand specification\n\nFor more complicated cases when you need to start an installation with several nodes (that are treated as baremetal nodes) you may use `stand specification`. The stand specification is a YAML file describes the genesis installation. it specifies how many bootstrap nodes, how many baremetal nodes, their characteristics and other parameters. For instance, it may look like this:\n\n[Small stand](data/stands/stand-small.yaml)\n\nUse option `--stand-spec` or `-s` for the `bootstrap` command to specify file with stand description. For instance,\n\n```sh\ngenesis bootstrap -i output/genesis-core.raw -f -m core -s data/stands/stand-small.yaml\n```\n\n## Versions\n\nSemver is used for project versioning. There are three types of versions:\n- stable version - format `X.Y.Z`\n- release candidate version - format `X.Y.Z-rc+YYYYMMDDHHMMSS.commit_hash[:8]`\n- development version - format `X.Y.Z-dev+YYYYMMDDHHMMSS.commit_hash[:8]`\n\nStable version looks like `1.0.0`, only three digits. Release candidate version looks like `0.0.1-rc+20250224092842.e11604e9`. Development version looks like `0.0.1-dev+20250223180245.fb195339`.\n\nTo get project version, run the `genesis get-version` with path to the project root directory as an argument to the command. For instance,\n\n```sh\ngenesis get-version /path/to/my_project\n```\n\n## Backups\n\nThe `backup` command is used to backup the current installation. This guide provides a brief overview of the command and its usage, along with several examples to get you started.\n\nThe basic syntax of the `backup` command is as follows:\n\n```bash\ngenesis backup -d /path/to/backup/directory\n```\n\nRun the above command to backup the current installation to the specified directory. The command will create a subdirectory for each domain and copy its disks to the specified directory. The backup will repeat every 24 hours by default.\n\nTo configure period of backup, run the following command:\n\n```bash\ngenesis backup -d /path/to/backup/directory -p 1h\n```\n\nThe above command will backup the current installation to the specified directory every hour.\n\nThere is an option to do a backup once and exit:\n\n```bash\ngenesis backup -d /path/to/backup/directory -oneshot\n```\n\nThe above command will backup the current installation to the specified directory once and exit.\n\n### Basic Backup\n\nRun backup periodically of all libvirt domains and store it in the current directory:\n\n```bash\ngenesis backup\n```\n\n### Backup Specific Domain\n\nRun backup periodically of a specific libvirt domain and store it in the current directory:\n\n```bash\ngenesis backup -n domain-name\n```\n\n### Custom Periodic Backup\n\nRun backup periodically of all libvirt domains and store it in the current directory:\n\n```bash\ngenesis backup -p 1h\n```\n\nAvailable periods are: `1m`, `5m`, `15m`, `30m`, `1h`, `3h`, `6h`, `12h`, `1d`, `3d`, `7d`\n\n### OneShot Backup\n\nRun backup of all libvirt domains and store it in the current directory:\n\n```bash\ngenesis backup -oneshot\n```\n\nThis command will backup the current installation to the specified directory once and exit.\n\n### Compressed Backup\n\nRun backup of all libvirt domains and store a compressed archive of the backup in the current directory:\n\n```bash\ngenesis backup --compress\n```\n\n### Encrypted Backup\n\nRun backup of all libvirt domains and store an encrypted archive of the backup in the current directory:\n\nNOTE: It works only with `--compress` flag\n\nYou need to set the environment variables `GEN_DEV_BACKUP_KEY` and `GEN_DEV_BACKUP_IV` to encrypt the backup. The key and IV must be greater or equal to 6 bytes and less or equal to 16 bytes.\n```bash\nexport GEN_DEV_BACKUP_KEY=secret_key\nexport GEN_DEV_BACKUP_IV=secret_iv\n\ngenesis backup --compress --encrypt\n```\n\nFor decryption, use the `genesis backup-decrypt` command.\n```bash\ngenesis backup-decrypt backup.tar.gz.encrypted\n```\n\n### Restore backups\n\nLook at the wiki page [restore backups](https://github.com/infraguys/genesis_devtools/wiki/Backups#restore-backup) for information on how to restore backups.\n\n### Rotation\n\nFor the periodic backup, you can set the number of backups to keep(rotation number). The default value is 5. Use the `--rotate` option to set the number of backups to keep:\n\n```bash\ngenesis backup --rotate 10\n```\n\n### Prevent disk overflow\n\nBackups can take a lot of disk space and it can be a reason to crash the whole system if a disk will be full. To prevent such a situation you can set a threshold for disk space that should be free. If during a backup process this threshold will be reached, the backup process will stop. Use the `--min-free-space` option to set the threshold in GB.\n\n```bash\ngenesis backup --min-free-space 50\n```\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Tools to manager life cycle of Genesis projects.",
"version": "0.3.0",
"project_urls": {
"Homepage": "https://github.com/infraguys/genesis_devtools"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "42e0c87783570256a7bda52ed1a13dfb4e5e7691c1f4c8e751e12b2ded5f2132",
"md5": "f98e6223dd0164147263886257c53cff",
"sha256": "2ff3fc27a793c88dbd48c1491745e42c3806e1257313782586b926ba1e3d7575"
},
"downloads": -1,
"filename": "genesis_devtools-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f98e6223dd0164147263886257c53cff",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 56727,
"upload_time": "2025-08-21T07:42:21",
"upload_time_iso_8601": "2025-08-21T07:42:21.292808Z",
"url": "https://files.pythonhosted.org/packages/42/e0/c87783570256a7bda52ed1a13dfb4e5e7691c1f4c8e751e12b2ded5f2132/genesis_devtools-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "450f59f3a676c06e8b452761016f9c39f4f454200df82af760bca338206d161f",
"md5": "1280dfca78dcab254bce829541a1c805",
"sha256": "fabd5963ae1366f7a4f96c79db606f760cd1c1af64e57424d4f82460806f55c9"
},
"downloads": -1,
"filename": "genesis_devtools-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "1280dfca78dcab254bce829541a1c805",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 44492,
"upload_time": "2025-08-21T07:42:22",
"upload_time_iso_8601": "2025-08-21T07:42:22.497001Z",
"url": "https://files.pythonhosted.org/packages/45/0f/59f3a676c06e8b452761016f9c39f4f454200df82af760bca338206d161f/genesis_devtools-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-21 07:42:22",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "infraguys",
"github_project": "genesis_devtools",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "pbr",
"specs": [
[
">=",
"1.10.0"
],
[
"<",
"5.8.1"
]
]
},
{
"name": "click",
"specs": [
[
"<",
"9.0.0"
],
[
">=",
"8.1.7"
]
]
},
{
"name": "pyyaml",
"specs": [
[
"<",
"7.0.0"
],
[
">=",
"6.0.0"
]
]
},
{
"name": "prettytable",
"specs": [
[
">=",
"3.7.0"
],
[
"<",
"4.0.0"
]
]
},
{
"name": "GitPython",
"specs": [
[
"<",
"4.0.0"
],
[
">=",
"3.1.30"
]
]
},
{
"name": "bazooka",
"specs": [
[
"<",
"2.0.0"
],
[
">=",
"1.0.0"
]
]
},
{
"name": "cryptography",
"specs": [
[
">=",
"45.0.5"
],
[
"<",
"47.0.0"
]
]
}
],
"tox": true,
"lcname": "genesis-devtools"
}
JSON
