# compas_cloud
compas_cloud is the further development of `compas.rpc` module. It uses websocktes instead of RESTful APIs to allow bi-directional communications between various front-end programs like Rhino, GH, RhinoVault2, blender or web-based viewers that are implemented in different enviroments including CPython, IronPython and Javascript. It also allows to save certain variables to backend inside a user session to avoid overheads created by redundant data transfers.
## Installation
### Install from source
```bash
git clone https://github.com/BlockResearchGroup/compas_cloud.git
pip install -e .
```
### Install for Rhino
```bash
python -m compas_rhino.install -p compas_cloud
```
## Using Proxy
### Running the sever:
1. Start from command line:
```bash
python -m compas_cloud.server
```
2. The proxy will automatically start a server in background if there isn't one to connect to. If the server is started this way, it will keep operating in background and reconnect if a new proxy is create later.
### Basic Usage
One of the main purposes of compas_cloud is to allow usage of full COMPAS functionalities in more closed envinroments like IronPython. The following example shows how to use a numpy based COMPAS function through a proxy which can be run in softwares like Rhino:
[basic.py](examples/basic.py)
```python
from compas_cloud import Proxy
from compas.geometry import Translation
proxy = Proxy()
transform_points_numpy = proxy.function('compas.geometry.transform_points_numpy')
# create a proxy funciton
pts = [[0,0,0], [1,0,0]]
T = Translation([100, 0, 0]).matrix
transform_points_numpy(pts, T) # call the function through proxy
print(result)
# will print: [[100.0, 0.0 ,0.0], [101.0, 0.0, 0.0]]
```
### Caching
Compas_cloud allows to cache data or function outputs at server side instead of sending them to the front-end all the time. This can vastly improve the performance for long iterative operations that involves large amount of data inputs and outputs.
[caching.py](examples/caching.py)
```python
from compas_cloud import Proxy
from compas.geometry import Translation
# CACHING INPUT PARAMETERS
proxy = Proxy()
transform_points_numpy = proxy.function('compas.geometry.transform_points_numpy')
# create a proxy funciton
pts = [[0,0,0], [1,0,0]]
pts_cache = proxy.cache(pts) # cache the object to server side and return its reference
print(pts_cache) # will print: {'cached': some_unique_id}
T = Translation([100, 0, 0]).matrix
result = transform_points_numpy(pts_cache, T) # call the function through proxy
print(result) # will print: [[100.0, 0.0 ,0.0], [101.0, 0.0, 0.0]]
# CACHING RETURNED DATA
transform_points_numpy = proxy.function('compas.geometry.transform_points_numpy', cache=True)
# this function will now return a cache object instead of the actual data
pts = [[0,0,0], [1,0,0]]
pts_cache = proxy.cache(pts)
print(pts_cache) # will print: {'cached': some_unique_id}
T = Translation([100, 0, 0]).matrix
result_cache = transform_points_numpy(pts_cache, T) # call the function through proxy
print(result_cache) # will print: {'cached': some_unique_id}
result = proxy.get(result_cache) # fetch the actual data of the cache object
print(result) # will print: [[100.0, 0.0 ,0.0], [101.0, 0.0, 0.0]]
```
### Server control
User can `restart/check/shutdown` a connected server from proxy with commands in following example: [server_control.py](examples/server_control.py)
```python
from compas_cloud import Proxy
import time
print("\n starting a new Proxy and by default starts a server in background")
proxy = Proxy(background=True)
time.sleep(3)
print("\n restarting the background server and open a new one in a prompt console")
proxy.background = False
proxy.restart()
time.sleep(3)
print("\n check if the proxy is healthily connected to server")
print(proxy.check())
time.sleep(3)
print("\n shut the the server and quite the program")
proxy.shutdown()
time.sleep(3)
```
### Other Examples
A [benchmark test](examples/benchmark.py) comparing pure python and numpy with caching to transform 10k points for 100 times:
```bash
python examples/benchmark.py
```
[Iterative plotting](examples/dr_numpy.py) example with callbacks:
```bash
python examples/dr_numpy.py
```
[Using non-compas packages like numpy with IronPython](examples/example_numpy.py):
run `examples/example_numpy.py` with Rhino
## Using Sessions (Currently only work with MacOS/Linux)
`Compas_cloud.Sessions` is a task-manager class that helps to execute a batch of long-lasting tasks such as FEA and DEM simulations. It creates a queue of tasks and a collection of workers to execute the tasks in parallel and save the program logs into each corresponding locations. `Sessions` can be run either locally or in a background server through `Proxy`.
### Examples
#### [Running Sessions Locally](examples/sessions_local.py):
```bash
python examples/sessions_local.py
```
```python
from compas_cloud import Sessions
# define a psuedo task that will take few seconds to finish
def func(a):
import time
for i in range(a):
time.sleep(1)
print('sleeped ', i, 's')
# initiate a session object, and specify where the logs will be stored and number of workers
# if no log_path is given, all logs will be streamed to terminal and not saved
# the default worker_num is equal to the number of cpus accessible on the computer
s = Sessions(log_path=None, worker_num=4)
# add several tasks to the session using different parameters
s.add_task(func, 1)
s.add_task(func, 2)
s.add_task(func, 3)
s.add_task(func, 4)
s.add_task(func, 5)
# kick of the taks and start to listen to the events when tasks start or finish
s.start()
s.listen()
```
You should see following logs:
```
{'waiting': 5, 'running': 0, 'failed': 0, 'finished': 0, 'total': 5} ________ START
{'waiting': 5, 'running': 0, 'failed': 0, 'finished': 0, 'total': 5} ________ using 4 workers
{'waiting': 5, 'running': 0, 'failed': 0, 'finished': 0, 'total': 5} ________ worker 58884 started
{'waiting': 4, 'running': 1, 'failed': 0, 'finished': 0, 'total': 5} ________ task-0: started
{'waiting': 4, 'running': 1, 'failed': 0, 'finished': 0, 'total': 5} ________ worker 58885 started
{'waiting': 4, 'running': 1, 'failed': 0, 'finished': 0, 'total': 5} ________ task-0: streaming log to temp/task-0.log
{'waiting': 3, 'running': 2, 'failed': 0, 'finished': 0, 'total': 5} ________ task-1: started
...
{'waiting': 0, 'running': 0, 'failed': 0, 'finished': 5, 'total': 5} ________ task-4: finished
{'waiting': 0, 'running': 0, 'failed': 0, 'finished': 5, 'total': 5} ________ worker 58884 terminated
{'waiting': 0, 'running': 0, 'failed': 0, 'finished': 5, 'total': 5} ________ FINISHED
```
#### [Running Sessions With Proxy](examples/sessions_local.py):
```bash
python examples/sessions_remote.py
```
```python
from compas_cloud import Proxy
# define a psuedo task that will take few seconds to finish
def func(a):
import time
for i in range(a):
time.sleep(1)
print('sleeped ', i, 's')
# initiate a Sessions object through Proxy that connects to a background server
p = Proxy()
s = p.Sessions()
# add several tasks to the session using different parameters
s.add_task(func, 1)
s.add_task(func, 2)
s.add_task(func, 3)
s.add_task(func, 4)
s.add_task(func, 5)
# kick of the taks and start to listen to the events when tasks start or finish
s.start()
s.listen()
```
You should be able to see same logs from above example
Raw data
{
"_id": null,
"home_page": "https://github.com/BlockResearchGroup/compas_cloud",
"name": "compas-cloud",
"maintainer": "",
"docs_url": null,
"requires_python": ">=2.7",
"maintainer_email": "",
"keywords": "architecture,engineering",
"author": "Li Chen",
"author_email": "li.chen@arch.ethz.ch",
"download_url": "https://files.pythonhosted.org/packages/a0/b3/8044c8f2a102fd301db2de855ffe712b243141247583827b0b1a4124f9e8/compas_cloud-0.4.1.tar.gz",
"platform": null,
"description": "# compas_cloud\ncompas_cloud is the further development of `compas.rpc` module. It uses websocktes instead of RESTful APIs to allow bi-directional communications between various front-end programs like Rhino, GH, RhinoVault2, blender or web-based viewers that are implemented in different enviroments including CPython, IronPython and Javascript. It also allows to save certain variables to backend inside a user session to avoid overheads created by redundant data transfers.\n\n## Installation\n\n### Install from source\n```bash\ngit clone https://github.com/BlockResearchGroup/compas_cloud.git\npip install -e .\n```\n\n\n### Install for Rhino\n```bash\npython -m compas_rhino.install -p compas_cloud\n```\n\n\n## Using Proxy\n\n### Running the sever:\n\n1. Start from command line:\n ```bash\n python -m compas_cloud.server\n ``` \n2. The proxy will automatically start a server in background if there isn't one to connect to. If the server is started this way, it will keep operating in background and reconnect if a new proxy is create later.\n\n### Basic Usage\nOne of the main purposes of compas_cloud is to allow usage of full COMPAS functionalities in more closed envinroments like IronPython. The following example shows how to use a numpy based COMPAS function through a proxy which can be run in softwares like Rhino: \n[basic.py](examples/basic.py)\n```python\nfrom compas_cloud import Proxy\nfrom compas.geometry import Translation\n\n\nproxy = Proxy()\ntransform_points_numpy = proxy.function('compas.geometry.transform_points_numpy')\n# create a proxy funciton\n\npts = [[0,0,0], [1,0,0]]\nT = Translation([100, 0, 0]).matrix\ntransform_points_numpy(pts, T) # call the function through proxy\nprint(result)\n# will print: [[100.0, 0.0 ,0.0], [101.0, 0.0, 0.0]]\n```\n\n### Caching\nCompas_cloud allows to cache data or function outputs at server side instead of sending them to the front-end all the time. This can vastly improve the performance for long iterative operations that involves large amount of data inputs and outputs.\n\n[caching.py](examples/caching.py)\n```python\nfrom compas_cloud import Proxy\nfrom compas.geometry import Translation\n\n# CACHING INPUT PARAMETERS\n\nproxy = Proxy()\ntransform_points_numpy = proxy.function('compas.geometry.transform_points_numpy')\n# create a proxy funciton\n\npts = [[0,0,0], [1,0,0]]\npts_cache = proxy.cache(pts) # cache the object to server side and return its reference\nprint(pts_cache) # will print: {'cached': some_unique_id}\n\nT = Translation([100, 0, 0]).matrix\nresult = transform_points_numpy(pts_cache, T) # call the function through proxy\nprint(result) # will print: [[100.0, 0.0 ,0.0], [101.0, 0.0, 0.0]]\n\n\n\n# CACHING RETURNED DATA\n\ntransform_points_numpy = proxy.function('compas.geometry.transform_points_numpy', cache=True)\n# this function will now return a cache object instead of the actual data\n\npts = [[0,0,0], [1,0,0]]\npts_cache = proxy.cache(pts)\nprint(pts_cache) # will print: {'cached': some_unique_id}\n\nT = Translation([100, 0, 0]).matrix\nresult_cache = transform_points_numpy(pts_cache, T) # call the function through proxy\nprint(result_cache) # will print: {'cached': some_unique_id}\n\nresult = proxy.get(result_cache) # fetch the actual data of the cache object\nprint(result) # will print: [[100.0, 0.0 ,0.0], [101.0, 0.0, 0.0]]\n```\n\n### Server control\nUser can `restart/check/shutdown` a connected server from proxy with commands in following example: [server_control.py](examples/server_control.py)\n```python\nfrom compas_cloud import Proxy\nimport time\n\nprint(\"\\n starting a new Proxy and by default starts a server in background\")\nproxy = Proxy(background=True)\ntime.sleep(3)\n\nprint(\"\\n restarting the background server and open a new one in a prompt console\")\nproxy.background = False\nproxy.restart()\ntime.sleep(3)\n\nprint(\"\\n check if the proxy is healthily connected to server\")\nprint(proxy.check())\ntime.sleep(3)\n\n\nprint(\"\\n shut the the server and quite the program\")\nproxy.shutdown()\ntime.sleep(3)\n```\n\n\n### Other Examples\nA [benchmark test](examples/benchmark.py) comparing pure python and numpy with caching to transform 10k points for 100 times: \n```bash\npython examples/benchmark.py\n```\n\n[Iterative plotting](examples/dr_numpy.py) example with callbacks: \n```bash\npython examples/dr_numpy.py\n```\n\n[Using non-compas packages like numpy with IronPython](examples/example_numpy.py): \nrun `examples/example_numpy.py` with Rhino\n\n\n## Using Sessions (Currently only work with MacOS/Linux)\n`Compas_cloud.Sessions` is a task-manager class that helps to execute a batch of long-lasting tasks such as FEA and DEM simulations. It creates a queue of tasks and a collection of workers to execute the tasks in parallel and save the program logs into each corresponding locations. `Sessions` can be run either locally or in a background server through `Proxy`.\n\n### Examples\n\n#### [Running Sessions Locally](examples/sessions_local.py):\n```bash\npython examples/sessions_local.py\n```\n\n```python\nfrom compas_cloud import Sessions\n\n# define a psuedo task that will take few seconds to finish\ndef func(a):\n import time\n\n for i in range(a):\n time.sleep(1)\n print('sleeped ', i, 's')\n\n# initiate a session object, and specify where the logs will be stored and number of workers\n# if no log_path is given, all logs will be streamed to terminal and not saved\n# the default worker_num is equal to the number of cpus accessible on the computer\ns = Sessions(log_path=None, worker_num=4)\n\n# add several tasks to the session using different parameters\ns.add_task(func, 1)\ns.add_task(func, 2)\ns.add_task(func, 3)\ns.add_task(func, 4)\ns.add_task(func, 5)\n\n# kick of the taks and start to listen to the events when tasks start or finish\ns.start()\ns.listen()\n```\n\nYou should see following logs:\n\n```\n{'waiting': 5, 'running': 0, 'failed': 0, 'finished': 0, 'total': 5} ________ START\n{'waiting': 5, 'running': 0, 'failed': 0, 'finished': 0, 'total': 5} ________ using 4 workers\n{'waiting': 5, 'running': 0, 'failed': 0, 'finished': 0, 'total': 5} ________ worker 58884 started\n{'waiting': 4, 'running': 1, 'failed': 0, 'finished': 0, 'total': 5} ________ task-0: started\n{'waiting': 4, 'running': 1, 'failed': 0, 'finished': 0, 'total': 5} ________ worker 58885 started\n{'waiting': 4, 'running': 1, 'failed': 0, 'finished': 0, 'total': 5} ________ task-0: streaming log to temp/task-0.log\n{'waiting': 3, 'running': 2, 'failed': 0, 'finished': 0, 'total': 5} ________ task-1: started\n...\n\n{'waiting': 0, 'running': 0, 'failed': 0, 'finished': 5, 'total': 5} ________ task-4: finished\n{'waiting': 0, 'running': 0, 'failed': 0, 'finished': 5, 'total': 5} ________ worker 58884 terminated\n{'waiting': 0, 'running': 0, 'failed': 0, 'finished': 5, 'total': 5} ________ FINISHED\n```\n\n\n#### [Running Sessions With Proxy](examples/sessions_local.py):\n```bash\npython examples/sessions_remote.py\n```\n\n```python\nfrom compas_cloud import Proxy\n\n# define a psuedo task that will take few seconds to finish\ndef func(a):\n import time\n\n for i in range(a):\n time.sleep(1)\n print('sleeped ', i, 's')\n\n\n# initiate a Sessions object through Proxy that connects to a background server\np = Proxy()\ns = p.Sessions()\n\n# add several tasks to the session using different parameters\ns.add_task(func, 1)\ns.add_task(func, 2)\ns.add_task(func, 3)\ns.add_task(func, 4)\ns.add_task(func, 5)\n\n# kick of the taks and start to listen to the events when tasks start or finish\ns.start()\ns.listen()\n```\n\nYou should be able to see same logs from above example\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "COMPAS package for cloud computing",
"version": "0.4.1",
"project_urls": {
"Homepage": "https://github.com/BlockResearchGroup/compas_cloud"
},
"split_keywords": [
"architecture",
"engineering"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "db762fd592f15ddd684d4cc20b84f1a476b74ec49731ac88768b93891544070d",
"md5": "ecf0f7a626b5b13de35463b221079883",
"sha256": "2626eb03cbdb71c822a9c8ebf4ad4a6780ae5aa6070b3f5ebb90a629b5925ed3"
},
"downloads": -1,
"filename": "compas_cloud-0.4.1-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "ecf0f7a626b5b13de35463b221079883",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=2.7",
"size": 19597,
"upload_time": "2022-09-07T10:21:57",
"upload_time_iso_8601": "2022-09-07T10:21:57.739265Z",
"url": "https://files.pythonhosted.org/packages/db/76/2fd592f15ddd684d4cc20b84f1a476b74ec49731ac88768b93891544070d/compas_cloud-0.4.1-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a0b38044c8f2a102fd301db2de855ffe712b243141247583827b0b1a4124f9e8",
"md5": "0556e53d6b088ada584ad8ff5898e368",
"sha256": "3336c91601c825186457b9bcfeaac4b43a817d81beab395b8eb85b5f1f4c7382"
},
"downloads": -1,
"filename": "compas_cloud-0.4.1.tar.gz",
"has_sig": false,
"md5_digest": "0556e53d6b088ada584ad8ff5898e368",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=2.7",
"size": 18390,
"upload_time": "2022-09-07T10:21:59",
"upload_time_iso_8601": "2022-09-07T10:21:59.212461Z",
"url": "https://files.pythonhosted.org/packages/a0/b3/8044c8f2a102fd301db2de855ffe712b243141247583827b0b1a4124f9e8/compas_cloud-0.4.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-09-07 10:21:59",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "BlockResearchGroup",
"github_project": "compas_cloud",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "compas-cloud"
}
JSON
