# hackathon baobab 2020
We want to schedule all jobs by deciding when and in which mode to execute each job. There are precedence relationships between pairs of jobs. There are two types of resources. Renewable resources (R) are consumed *each period* and have an availability that is recovered each period of time. Non-renewable resources (N) are consumed *once per job* and have an availability for the whole planning horizon.
The objective is to reduce the finishing time (start time + duration) of the last job.
As an example the following image shows the input data (left), and the optimal solution (center: gantt, resource usage: right) for the instance `c159_7.mm`:
The instances for the problem are found here:
http://www.om-db.wi.tum.de/psplib/getdata_mm.html
On the `data` directory of this repository we have copied the smallest instances.
For the format of the solution, since there is no example that we know of, we'll be using the one in `data/solutions/c15mm/c1564_9.output.json`
Below are the instructions to use the helper functions and checker (they are optional).
To understand the format of the input data file, you can check how we parse it in python in the function `Instance.from_mm(path)` in the file`hackathonbaobab2020/core/instance.py`
## Installation
python>=3.5 is needed. I'm assuming a Windows installation.
To install from source:
```
cd hackathonbaobab2020
python -m venv venv
venv/Scripts/activate
pip install -r requirements.txt
```
Alternatively, it's possible to install it as a pypi package.
Here there is more control on which packages are installed.
Install only the core dependencies (core, schemas, default solver)::
```
pip install hackathonbaobab2020
```
Install all dependencies ('benchmark' to generate graphs, 'solvers' to install solver dependencies). The second line installs (in Ubuntu) the cbc solver::
```
pip install hackathonbaobab2020[benchmark,solvers]
sudo apt install coinor-cbc
```
## How to add a new solver
These are the steps to add a solver and make it compatible with the command line and the python functions:
1. Add a file inside the `hackathonbaobab2020/solver` directory with a subclass of `hackathonbaobab2020.core.experiment.Experiment` that implements, at least, the `solve()` method *with the same argument names*.
1. Your `solve` method needs to return an integer with the status of the solving process. Current values are `{4: "Optimal", 2: "Feasible", 3: "Infeasible", 0: "Unknown"}`.
1. Your `solve` method also needs to store the best solution found in `self.solution`. It needs to be an instance of the `Solution` object.
1. Edit the `hackathonbaobab2020/solver/__init__.py` to import your solver and edit the `solvers` dictionary by giving your solver a name.
1. If the `requirements.txt` file is missing some package you need for your solver, add it to the list under `solvers`. Also edit the `setup.py` and add the dependency on `solvers` within the `extras_require` dictionary.
**Additional considerations**:
1. One way to see if the solver is correctly integrated is to test solving with it via the command line (see below).
2. Everything that your solver needs should be inside the `hackathonbaobab2020/solver` directory (you can put more than one file). Do not edit the files outside the `solver` directories with code from your solver!
## How to run tests
These tests are run also in github and test some small example problems with a `timelimit` of 120 s (check the `options` dictionary).
```
python tests/tests.py
```
## Command line
The command line app has three main ways to use it.
### To execute instances
To get all possible commands just run:
python hackathonbaobab2020/main.py solve-scenarios --help
The following assumes you have downloaded the zip `j30.mm.zip` of input instances, and you have stored it in the `data` directory. It solves instance `j301_1.mm` with the solver that is in `hackathonbaobab2020/solver/algorithm1` named `default` in `hackathonbaobab2020/solver/__init__.py`.
python hackathonbaobab2020/main.py solve-scenarios --directory=data --scenario=j30.mm.zip --solver=default --instance=j3010_1.mm --no-test
You can also solve multiple scenarios or multiple instances by passing the `--instances` and `--scenarios` arguments. Just be careful with the string format:
python hackathonbaobab2020/main.py solve-scenarios --directory=data --scenarios='["c15.mm.zip", "c21.mm.zip", "j10.mm.zip", "j30.mm.zip", "m1.mm.zip", "m5.mm.zip", "n0.mm.zip", "n1.mm.zip", "n3.mm.zip", "r1.mm.zip", "r4.mm.zip", "r5.mm.zip"]' --solver=default
With the option argument, a json with the solving configuration is passed:
python hackathonbaobab2020/main.py solve-scenarios --directory=data --scenario=j30.mm.zip --solver=brute_EJ --instance=j3010_1.mm --no-test --options='{"DEBUG": 1, "timeLimit": 120}'
Finally, if you pass the `zip` option you create a nice little zip at the end.
The output format is always the same:
solver_name/scenario_name/instance_name/(input.json, output.json, options.json)
The `options.json` file contains some information from the solver such as the time it took to solve, the status (Optimal, Feasible, Infeasible, etc.), the name of the solver, etc.
### To get statistics from a solution
You first need to have a zip with the results you want to get statistics from. For this, the easiest is to pass the `zip` option to the `solve-scenarios` function above.
Then you do something like:
python hackathonbaobab2020/main.py export-table --path=data/default.zip --path_out=data_default.csv
This generates a table in a csv with several columns: scenario, name (instance), objective (function value), solver, (solving) time, (number of) errors (in the solution).
To easily read the contents you can do:
```python
import pandas as pd
df = pd.read_csv('data_default.csv')
print(df.head().to_markdown())
```
Which should print something like this:
| | scenario | name | objective | solver | time | errors |
|---:|:-----------|:----------|------------:|:---------|------------:|---------:|
| 0 | n3.mm | n311_1.mm | 44 | default | 0.000282581 | 1 |
| 1 | n0.mm | n013_7.mm | 35 | default | 0.000227326 | 0 |
| 2 | r4.mm | r452_6.mm | 46 | default | 0.00025893 | 1 |
| 3 | n3.mm | n343_4.mm | 37 | default | 0.000268723 | 1 |
| 4 | n1.mm | n121_4.mm | 55 | default | 0.000254337 | 0 |
## Benchmarking solvers
In `hackathonbaobab2020/execution/benchmark.py` there are functions to automate the comparison of solvers. Function `solve_all()` executes a set of random instances. They require previously downloading the corresponding zips from the dataset site. Function `compare` produce a summary table based on the results. Finally, `graphs` use the generated table to produce graphs of the results.
Example of a resulting graph:
## Using python objects
We use the following helper objects:
1. `Instance` to represent input data.
2. `Solution` to represent a solution.
3. `Experiment` to represent input data+solution.
4. `Algorithm(Experiment)` to represent a resolution approach.
An example of the last one (4) is found in `hackathonbaobab2020/solver/algorithm1.py`. It schedules one job at a time while respecting the sequence. It passes all tests except the non-renewables, sometimes.
There are helper functions to read and write an instance and a solution to/from a file.
A small example of how to use the existing code is available in `hackathonbaobab2020/execution/test_script.py`.
Below an example:
```python
from hackathonbaobab2020.core import Instance
from hackathonbaobab2020.solver import get_solver
# get mm file
path = "data/c15.mm/c154_3.mm"
# initialize an instance object
instance = Instance.from_mm(path)
# get the default solver (in solver/algorithm1.py)
solver = get_solver('default')
# initialize the solver with the instance
exp = solver(instance=instance)
# solve the instance using the solver
exp.solve({})
# The next functions do not depend on the solver and should not be overwritten:
# print the possible errors on the solution obtained from the solver
print(exp.check_solution())
# print the objective function of the solution
print(exp.get_objective())
# produce a gantt chart of the job's schedule, with a color per mode.
exp.graph()
```
Raw data
{
"_id": null,
"home_page": "https://github.com/baobabsoluciones/hackathonbaobab2020",
"name": "hackathonbaobab2020",
"maintainer": "Franco Peschiera",
"docs_url": null,
"requires_python": null,
"maintainer_email": "pchtsp@gmail.com",
"keywords": "math hackathon pulp ortools pyomo",
"author": "Franco Peschiera",
"author_email": "pchtsp@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/f1/51/0deab117b2c9500b09b2f9bf40fcb315326d1748b017ecb6624df78147a0/hackathonbaobab2020-1.0.3.tar.gz",
"platform": null,
"description": "# hackathon baobab 2020\n\nWe want to schedule all jobs by deciding when and in which mode to execute each job. There are precedence relationships between pairs of jobs. There are two types of resources. Renewable resources (R) are consumed *each period* and have an availability that is recovered each period of time. Non-renewable resources (N) are consumed *once per job* and have an availability for the whole planning horizon.\nThe objective is to reduce the finishing time (start time + duration) of the last job.\n\nAs an example the following image shows the input data (left), and the optimal solution (center: gantt, resource usage: right) for the instance `c159_7.mm`:\n\n\n\n\nThe instances for the problem are found here:\n\nhttp://www.om-db.wi.tum.de/psplib/getdata_mm.html\n\nOn the `data` directory of this repository we have copied the smallest instances.\nFor the format of the solution, since there is no example that we know of, we'll be using the one in `data/solutions/c15mm/c1564_9.output.json`\n\nBelow are the instructions to use the helper functions and checker (they are optional).\nTo understand the format of the input data file, you can check how we parse it in python in the function `Instance.from_mm(path)` in the file`hackathonbaobab2020/core/instance.py`\n\n## Installation\n\npython>=3.5 is needed. I'm assuming a Windows installation.\n\nTo install from source:\n\n```\ncd hackathonbaobab2020\npython -m venv venv\nvenv/Scripts/activate\npip install -r requirements.txt\n```\n\nAlternatively, it's possible to install it as a pypi package.\nHere there is more control on which packages are installed.\n\nInstall only the core dependencies (core, schemas, default solver)::\n\n```\npip install hackathonbaobab2020\n```\n\nInstall all dependencies ('benchmark' to generate graphs, 'solvers' to install solver dependencies). The second line installs (in Ubuntu) the cbc solver::\n\n```\npip install hackathonbaobab2020[benchmark,solvers]\nsudo apt install coinor-cbc\n```\n\n## How to add a new solver\n\nThese are the steps to add a solver and make it compatible with the command line and the python functions:\n\n1. Add a file inside the `hackathonbaobab2020/solver` directory with a subclass of `hackathonbaobab2020.core.experiment.Experiment` that implements, at least, the `solve()` method *with the same argument names*.\n1. Your `solve` method needs to return an integer with the status of the solving process. Current values are `{4: \"Optimal\", 2: \"Feasible\", 3: \"Infeasible\", 0: \"Unknown\"}`.\n1. Your `solve` method also needs to store the best solution found in `self.solution`. It needs to be an instance of the `Solution` object.\n1. Edit the `hackathonbaobab2020/solver/__init__.py` to import your solver and edit the `solvers` dictionary by giving your solver a name.\n1. If the `requirements.txt` file is missing some package you need for your solver, add it to the list under `solvers`. Also edit the `setup.py` and add the dependency on `solvers` within the `extras_require` dictionary.\n\n**Additional considerations**:\n\n1. One way to see if the solver is correctly integrated is to test solving with it via the command line (see below).\n2. Everything that your solver needs should be inside the `hackathonbaobab2020/solver` directory (you can put more than one file). Do not edit the files outside the `solver` directories with code from your solver!\n\n## How to run tests\n\nThese tests are run also in github and test some small example problems with a `timelimit` of 120 s (check the `options` dictionary).\n```\npython tests/tests.py \n ```\n\n## Command line\n\nThe command line app has three main ways to use it.\n\n### To execute instances\n\nTo get all possible commands just run:\n\n python hackathonbaobab2020/main.py solve-scenarios --help\n\nThe following assumes you have downloaded the zip `j30.mm.zip` of input instances, and you have stored it in the `data` directory. It solves instance `j301_1.mm` with the solver that is in `hackathonbaobab2020/solver/algorithm1` named `default` in `hackathonbaobab2020/solver/__init__.py`.\n\n python hackathonbaobab2020/main.py solve-scenarios --directory=data --scenario=j30.mm.zip --solver=default --instance=j3010_1.mm --no-test\n\nYou can also solve multiple scenarios or multiple instances by passing the `--instances` and `--scenarios` arguments. Just be careful with the string format:\n\n python hackathonbaobab2020/main.py solve-scenarios --directory=data --scenarios='[\"c15.mm.zip\", \"c21.mm.zip\", \"j10.mm.zip\", \"j30.mm.zip\", \"m1.mm.zip\", \"m5.mm.zip\", \"n0.mm.zip\", \"n1.mm.zip\", \"n3.mm.zip\", \"r1.mm.zip\", \"r4.mm.zip\", \"r5.mm.zip\"]' --solver=default\n\nWith the option argument, a json with the solving configuration is passed:\n\n python hackathonbaobab2020/main.py solve-scenarios --directory=data --scenario=j30.mm.zip --solver=brute_EJ --instance=j3010_1.mm --no-test --options='{\"DEBUG\": 1, \"timeLimit\": 120}'\n\nFinally, if you pass the `zip` option you create a nice little zip at the end.\n\nThe output format is always the same:\n\n solver_name/scenario_name/instance_name/(input.json, output.json, options.json)\n\nThe `options.json` file contains some information from the solver such as the time it took to solve, the status (Optimal, Feasible, Infeasible, etc.), the name of the solver, etc.\n\n### To get statistics from a solution\n\nYou first need to have a zip with the results you want to get statistics from. For this, the easiest is to pass the `zip` option to the `solve-scenarios` function above.\n\nThen you do something like:\n\n python hackathonbaobab2020/main.py export-table --path=data/default.zip --path_out=data_default.csv\n\nThis generates a table in a csv with several columns: scenario, name (instance), objective (function value), solver, (solving) time, (number of) errors (in the solution).\n\nTo easily read the contents you can do:\n\n```python\n\nimport pandas as pd\ndf = pd.read_csv('data_default.csv')\nprint(df.head().to_markdown())\n\n```\n\nWhich should print something like this:\n\n| | scenario | name | objective | solver | time | errors |\n|---:|:-----------|:----------|------------:|:---------|------------:|---------:|\n| 0 | n3.mm | n311_1.mm | 44 | default | 0.000282581 | 1 |\n| 1 | n0.mm | n013_7.mm | 35 | default | 0.000227326 | 0 |\n| 2 | r4.mm | r452_6.mm | 46 | default | 0.00025893 | 1 |\n| 3 | n3.mm | n343_4.mm | 37 | default | 0.000268723 | 1 |\n| 4 | n1.mm | n121_4.mm | 55 | default | 0.000254337 | 0 |\n\n## Benchmarking solvers\n\nIn `hackathonbaobab2020/execution/benchmark.py` there are functions to automate the comparison of solvers. Function `solve_all()` executes a set of random instances. They require previously downloading the corresponding zips from the dataset site. Function `compare` produce a summary table based on the results. Finally, `graphs` use the generated table to produce graphs of the results.\n\nExample of a resulting graph:\n\n\n\n## Using python objects\n\nWe use the following helper objects:\n\n1. `Instance` to represent input data.\n2. `Solution` to represent a solution.\n3. `Experiment` to represent input data+solution.\n4. `Algorithm(Experiment)` to represent a resolution approach.\n\nAn example of the last one (4) is found in `hackathonbaobab2020/solver/algorithm1.py`. It schedules one job at a time while respecting the sequence. It passes all tests except the non-renewables, sometimes.\n\nThere are helper functions to read and write an instance and a solution to/from a file.\n\nA small example of how to use the existing code is available in `hackathonbaobab2020/execution/test_script.py`.\nBelow an example:\n\n```python\nfrom hackathonbaobab2020.core import Instance\nfrom hackathonbaobab2020.solver import get_solver\n\n# get mm file\npath = \"data/c15.mm/c154_3.mm\"\n# initialize an instance object\ninstance = Instance.from_mm(path)\n# get the default solver (in solver/algorithm1.py)\nsolver = get_solver('default')\n# initialize the solver with the instance\nexp = solver(instance=instance)\n# solve the instance using the solver\nexp.solve({})\n# The next functions do not depend on the solver and should not be overwritten:\n# print the possible errors on the solution obtained from the solver\nprint(exp.check_solution())\n# print the objective function of the solution\nprint(exp.get_objective())\n# produce a gantt chart of the job's schedule, with a color per mode.\nexp.graph()\n```\n\n\n\n\n\n\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "Hackathon 2020 at baobab soluciones",
"version": "1.0.3",
"project_urls": {
"Download": "https://github.com/baobabsoluciones/hackathonbaobab2020/archive/main.zip",
"Homepage": "https://github.com/baobabsoluciones/hackathonbaobab2020"
},
"split_keywords": [
"math",
"hackathon",
"pulp",
"ortools",
"pyomo"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c7d68e42015c2524de10635952269ef52b0ca1b53a8e7afe6e7bde8e5d1af9ff",
"md5": "17b518b333db1816250470e2adac8819",
"sha256": "d09d77a1ebc2fb307dbdf11e08ed8de9b824499e0eb9279bfb60eb1d8a61088c"
},
"downloads": -1,
"filename": "hackathonbaobab2020-1.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "17b518b333db1816250470e2adac8819",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 63364,
"upload_time": "2024-04-22T16:19:40",
"upload_time_iso_8601": "2024-04-22T16:19:40.489374Z",
"url": "https://files.pythonhosted.org/packages/c7/d6/8e42015c2524de10635952269ef52b0ca1b53a8e7afe6e7bde8e5d1af9ff/hackathonbaobab2020-1.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "f1510deab117b2c9500b09b2f9bf40fcb315326d1748b017ecb6624df78147a0",
"md5": "c05ff70cca6ec18fafbcfffab7e508a1",
"sha256": "835dbb46c513842de23fa137cd98639887051498b80a3d2ff85614067f832ebe"
},
"downloads": -1,
"filename": "hackathonbaobab2020-1.0.3.tar.gz",
"has_sig": false,
"md5_digest": "c05ff70cca6ec18fafbcfffab7e508a1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 50007,
"upload_time": "2024-04-22T16:19:49",
"upload_time_iso_8601": "2024-04-22T16:19:49.473897Z",
"url": "https://files.pythonhosted.org/packages/f1/51/0deab117b2c9500b09b2f9bf40fcb315326d1748b017ecb6624df78147a0/hackathonbaobab2020-1.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-22 16:19:49",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "baobabsoluciones",
"github_project": "hackathonbaobab2020",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "pytups",
"specs": []
},
{
"name": "click",
"specs": []
},
{
"name": "pandas",
"specs": []
},
{
"name": "orloge",
"specs": []
},
{
"name": "jsonschema",
"specs": []
},
{
"name": "cornflow_client",
"specs": []
},
{
"name": "pyomo",
"specs": []
},
{
"name": "ortools",
"specs": []
},
{
"name": "tabulate",
"specs": []
},
{
"name": "pygount",
"specs": []
},
{
"name": "plotly",
"specs": []
},
{
"name": "seaborn",
"specs": []
}
],
"lcname": "hackathonbaobab2020"
}
JSON
