Galileo: A framework for distributed load testing experiments
=============================================================
[](https://badge.fury.io/py/edgerun-galileo)
[](https://travis-ci.org/edgerun/galileo)
[](https://coveralls.io/github/edgerun/galileo?branch=master)
This project allows users to define, run, and interact with distributed load testing experiments for distributed
web-service-oriented systems.
Galileo consists of two major components a user can interact with:
the experiment controller shell and the galileo dashboard.
The experiment controller can spawn emulated clients on workers, and control the amount of load they generate.
Furthermore, a user can interact with the service routing table shell to control to which cluster node a service request
is sent to.
Build
-----
Create a new virtual environment and install all dependencies
make venv
#### Docker
To create a Docker image that can run galileo components, run
make docker
To create a arm32v7 Docker image that can run galileo components, run
make docker-arm
Start a worker with
cd docker/galileo-worker
docker-compose up
Compose files for arm32v7 are located in
docker/arm
Start a local dev environment, including: API, ExperimentDaemon, 1 worker, redis and database:
cd docker/dev
docker-compose up
Preparing the Example Application
---------------------------------
We prepare the cluster with an example application. Specifically a image classification service.
Run the mxnet-model-server as a Docker container named 'mms', exposed on port 8080.
For example, to start mxnet-model-server with models squeezenet and alexnet, run the following command on a cluster node:
docker run -itd --name mms -p 8080:8080 -p 8081:8081 awsdeeplearningteam/mxnet-model-server:1.0.0-mxnet-cpu mxnet-model-server --start \
--models squeezenet=https://s3.amazonaws.com/model-server/models/squeezenet_v1.1/squeezenet_v1.1.model,alexnet=https://s3.amazonaws.com/model-server/model_archive_1.0/alexnet.mar
Prepare the Experiment Worker Hosts
-----------------------------------
The devices hosting the workers that generate load need to run the experiment controller host application.
bin/run worker --logging=INFO
All runtime parameters are controlled via `galileo_*` environment variables. Check `docker/galileo-worker/worker.env` for some examples.
All environment variables, that start with `galileo_`, can be used as worker label when creating a client group.
I.e., if you start a worker process with the env variable `galileo_zone=A`, you can spawn a client group that contains only
workers with this labels as follows:
g.spawn('service',worker_labels={'galileo_zone': 'A'})
Run the Experiment Controller Shell
-----------------------------------
```
(.venv) pi@graviton:~/edgerun/galileo $ bin/run shell
__ __
.-.,="``"=. ____ _____ _/ (_) /__ ____
'=/_ \ / __ `/ __ `/ / / / _ \/ __ \
| '=._ | / /_/ / /_/ / / / / __/ /_/ /
\ `=./`. \__, /\__,_/_/_/_/\___/\____/
'=.__.=' `=' /____/
Welcome to the galileo shell!
Type `usage` to list available functions
galileo> usage
the galileo shell is an interactive python shell that provides the following commands
Commands:
usage show this message
env show environment variables
pwd show the current working directory
Functions:
sleep time.sleep wrapper
Objects:
g Galileo object that allows you to interact with the system
show Prints runtime information about the system to system out
exp Galileo experiment
rtbl Service routing table
Type help(<function>) or help(<object>) to learn how to use the functions.
```
Configure the routing table
---------------------------
The `rtbl` object in the shell allows you to set load balancing policies. Run `help(rtbl)` in the galileo shell.
Here is an example of how to set a record for the service `myservice`.
```
galileo> rtbl.set('myservice', ['host1:8080', 'host2:8080'], [1,2])
RoutingRecord(service='myservice', hosts=['host1:8080', 'host2:8080'], weights=[1, 2])
galileo> rtbl
+---------------------------+----------------------+-----------+
| Service | Hosts | Weights |
+---------------------------+----------------------+-----------+
| myservice | host1:8080 | 1.0 |
| | host2:8080 | 2.0 |
+---------------------------+----------------------+-----------+
```
Run the Experiment Daemon
-------------------------
---
**FIXME: THIS IS OUTDATED**
---
The experiment daemon continuously reads from the blocking redis queue `galileo:experiments:instructions`.
After receiving instructions, the controller will execute the commands and record the telemetry data
published via Redis. At the end the status of the experiment will be set to 'FINISHED' and the traces,
that were saved in the db by the clients, will be updated to reference the experiment.
Example Redis command to inject a new experiment (where `exphost` is the hostname of the experiment host):
LPUSH galileo:experiments:instructions '{"instructions": "spawn exphost squeezenet 1\nsleep 2\nrps exphost squeezenet 1\nsleep 5\nrps exphost squeezenet 0\nsleep 2\nclose exphost squeezenet"}'
you can also specify `exp_id`, `creator`, and `name`, otherwise some generated/standard values will be used.
You can change the database used to store the experiment data via the env `galileo_expdb_driver` (`sqlite` or `mysql`).
To write the changes into MySQL (or MariaDB), set the following environment variables:
`galileo_expdb_mysql_host`,
`galileo_expdb_mysql_port`,
`galileo_expdb_mysql_db`,
`galileo_expdb_mysql_user`,
`galileo_expdb_mysql_password`
You can create a mariadb docker instance with:
./bin/run-db.sh
Then execute the daemon with:
python -m galileo.cli.experimentd
Or run the script, which exports the mariadb setup from the docker container (add `--logging DEBUG` for output)
./bin/experimentd-mysql.sh
Run the Galileo REST API
------------------------
Serve the app with gunicorn
gunicorn -w 4 --preload -b 0.0.0.0:5001 \
-c galileo/webapp/gunicorn.conf.py \
galileo.webapp.wsgi:api
Raw data
{
"_id": null,
"home_page": "https://github.com/edgerun/galileo",
"name": "edgerun-galileo",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Thomas Rausch",
"author_email": "t.rausch@dsg.tuwien.ac.at",
"download_url": "https://files.pythonhosted.org/packages/8c/8e/1ce9b0673cbe97649dba40ffdb511e7db278c9b70ee8f00c6917e2be0a4c/edgerun-galileo-0.10.4.tar.gz",
"platform": "",
"description": "Galileo: A framework for distributed load testing experiments\n=============================================================\n\n[](https://badge.fury.io/py/edgerun-galileo)\n[](https://travis-ci.org/edgerun/galileo)\n[](https://coveralls.io/github/edgerun/galileo?branch=master)\n\nThis project allows users to define, run, and interact with distributed load testing experiments for distributed\nweb-service-oriented systems.\nGalileo consists of two major components a user can interact with:\nthe experiment controller shell and the galileo dashboard.\nThe experiment controller can spawn emulated clients on workers, and control the amount of load they generate.\nFurthermore, a user can interact with the service routing table shell to control to which cluster node a service request\nis sent to.\n\nBuild\n-----\n\nCreate a new virtual environment and install all dependencies\n\n make venv\n\n#### Docker\n\nTo create a Docker image that can run galileo components, run \n\n make docker\n \nTo create a arm32v7 Docker image that can run galileo components, run\n \n make docker-arm\n\nStart a worker with\n\n cd docker/galileo-worker\n docker-compose up\n\nCompose files for arm32v7 are located in \n \n docker/arm\n \nStart a local dev environment, including: API, ExperimentDaemon, 1 worker, redis and database:\n\n cd docker/dev\n docker-compose up\n \nPreparing the Example Application\n---------------------------------\n\nWe prepare the cluster with an example application. Specifically a image classification service.\n\nRun the mxnet-model-server as a Docker container named 'mms', exposed on port 8080.\nFor example, to start mxnet-model-server with models squeezenet and alexnet, run the following command on a cluster node:\n\n docker run -itd --name mms -p 8080:8080 -p 8081:8081 awsdeeplearningteam/mxnet-model-server:1.0.0-mxnet-cpu mxnet-model-server --start \\\n --models squeezenet=https://s3.amazonaws.com/model-server/models/squeezenet_v1.1/squeezenet_v1.1.model,alexnet=https://s3.amazonaws.com/model-server/model_archive_1.0/alexnet.mar\n\n\nPrepare the Experiment Worker Hosts\n-----------------------------------\n\nThe devices hosting the workers that generate load need to run the experiment controller host application.\n\n bin/run worker --logging=INFO\n\nAll runtime parameters are controlled via `galileo_*` environment variables. Check `docker/galileo-worker/worker.env` for some examples.\n\nAll environment variables, that start with `galileo_`, can be used as worker label when creating a client group.\n\nI.e., if you start a worker process with the env variable `galileo_zone=A`, you can spawn a client group that contains only \nworkers with this labels as follows:\n\n g.spawn('service',worker_labels={'galileo_zone': 'A'})\n\n\nRun the Experiment Controller Shell\n-----------------------------------\n\n```\n(.venv) pi@graviton:~/edgerun/galileo $ bin/run shell\n __ __\n .-.,=\"``\"=. ____ _____ _/ (_) /__ ____\n '=/_ \\ / __ `/ __ `/ / / / _ \\/ __ \\\n | '=._ | / /_/ / /_/ / / / / __/ /_/ /\n \\ `=./`. \\__, /\\__,_/_/_/_/\\___/\\____/\n '=.__.=' `=' /____/\n\n\nWelcome to the galileo shell!\n\nType `usage` to list available functions\n\ngalileo> usage\nthe galileo shell is an interactive python shell that provides the following commands\n\nCommands:\n usage show this message\n env show environment variables\n pwd show the current working directory\n\nFunctions:\n sleep time.sleep wrapper\n\nObjects:\n g Galileo object that allows you to interact with the system\n show Prints runtime information about the system to system out\n exp Galileo experiment\n rtbl Service routing table\n\nType help(<function>) or help(<object>) to learn how to use the functions.\n\n```\n\n\n\nConfigure the routing table\n---------------------------\n\nThe `rtbl` object in the shell allows you to set load balancing policies. Run `help(rtbl)` in the galileo shell.\nHere is an example of how to set a record for the service `myservice`.\n```\ngalileo> rtbl.set('myservice', ['host1:8080', 'host2:8080'], [1,2])\nRoutingRecord(service='myservice', hosts=['host1:8080', 'host2:8080'], weights=[1, 2])\ngalileo> rtbl\n+---------------------------+----------------------+-----------+\n| Service | Hosts | Weights |\n+---------------------------+----------------------+-----------+\n| myservice | host1:8080 | 1.0 |\n| | host2:8080 | 2.0 |\n+---------------------------+----------------------+-----------+\n\n```\n\nRun the Experiment Daemon\n-------------------------\n\n---\n\n**FIXME: THIS IS OUTDATED** \n\n---\n\nThe experiment daemon continuously reads from the blocking redis queue `galileo:experiments:instructions`.\nAfter receiving instructions, the controller will execute the commands and record the telemetry data\npublished via Redis. At the end the status of the experiment will be set to 'FINISHED' and the traces,\nthat were saved in the db by the clients, will be updated to reference the experiment.\n\nExample Redis command to inject a new experiment (where `exphost` is the hostname of the experiment host):\n\n LPUSH galileo:experiments:instructions '{\"instructions\": \"spawn exphost squeezenet 1\\nsleep 2\\nrps exphost squeezenet 1\\nsleep 5\\nrps exphost squeezenet 0\\nsleep 2\\nclose exphost squeezenet\"}'\n\nyou can also specify `exp_id`, `creator`, and `name`, otherwise some generated/standard values will be used.\n\nYou can change the database used to store the experiment data via the env `galileo_expdb_driver` (`sqlite` or `mysql`).\nTo write the changes into MySQL (or MariaDB), set the following environment variables:\n`galileo_expdb_mysql_host`,\n`galileo_expdb_mysql_port`,\n`galileo_expdb_mysql_db`,\n`galileo_expdb_mysql_user`,\n`galileo_expdb_mysql_password`\n\nYou can create a mariadb docker instance with:\n\n ./bin/run-db.sh\n\nThen execute the daemon with:\n\n python -m galileo.cli.experimentd\n\nOr run the script, which exports the mariadb setup from the docker container (add `--logging DEBUG` for output)\n\n ./bin/experimentd-mysql.sh\n\n\nRun the Galileo REST API\n------------------------\n\nServe the app with gunicorn\n\n gunicorn -w 4 --preload -b 0.0.0.0:5001 \\\n -c galileo/webapp/gunicorn.conf.py \\\n galileo.webapp.wsgi:api",
"bugtrack_url": null,
"license": "",
"summary": "Galileo: A framework for distributed load testing experiments",
"version": "0.10.4",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "67bb765758f850cfd992032db97fda9d",
"sha256": "a090dede38ac1ab3764be83c4b181e3b04f56f7191800a31817c5f4a8be76521"
},
"downloads": -1,
"filename": "edgerun_galileo-0.10.4-py3.8.egg",
"has_sig": false,
"md5_digest": "67bb765758f850cfd992032db97fda9d",
"packagetype": "bdist_egg",
"python_version": "0.10.4",
"requires_python": null,
"size": 166893,
"upload_time": "2021-11-15T14:33:03",
"upload_time_iso_8601": "2021-11-15T14:33:03.196673Z",
"url": "https://files.pythonhosted.org/packages/15/d8/5a4c9037363e0ba3cc53c9a81277456b1b1f66dbf0a3743c678287cc2de2/edgerun_galileo-0.10.4-py3.8.egg",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "e0fb85ca167a4cc4e5c8b60e7eb83340",
"sha256": "84e28f1e59b1d933afa10ef8a5a2e36d8e5e7e910069f8d929ce095308075165"
},
"downloads": -1,
"filename": "edgerun-galileo-0.10.4.tar.gz",
"has_sig": false,
"md5_digest": "e0fb85ca167a4cc4e5c8b60e7eb83340",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 45907,
"upload_time": "2021-11-15T14:33:05",
"upload_time_iso_8601": "2021-11-15T14:33:05.384578Z",
"url": "https://files.pythonhosted.org/packages/8c/8e/1ce9b0673cbe97649dba40ffdb511e7db278c9b70ee8f00c6917e2be0a4c/edgerun-galileo-0.10.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2021-11-15 14:33:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "edgerun",
"github_project": "galileo",
"travis_ci": true,
"coveralls": true,
"github_actions": false,
"requirements": [],
"lcname": "edgerun-galileo"
}
JSON
