<div align="center">
<a href="https://app.clear.ml"><img src="https://github.com/allegroai/clearml/blob/master/docs/clearml-logo.svg?raw=true" width="250px"></a>
## **`clearml-session` </br> CLI for launching JupyterLab / VSCode / SSH on a remote machine**
## đĨ NEW in version `0.13` [Workspace Syncing](#store-and-synchronize-interactive-session-workspace) đ
[](https://img.shields.io/github/license/allegroai/clearml-session.svg)
[](https://img.shields.io/pypi/pyversions/clearml-session.svg)
[](https://img.shields.io/pypi/v/clearml-session.svg)
[](https://pypi.python.org/pypi/clearml-session/)
[](https://joinslack.clear.ml)
</div>
**`clearml-session`** is a utility for launching detachable remote interactive sessions (MacOS, Windows, Linux)
### tl;dr
**CLI to launch a remote session of Jupyter-Lab / VSCode / SSH,
inside any docker container on any deployment, Cloud / Kubernetes / Bare-Metal**
### đ° What does it do?
Starting a clearml (ob)session from your local machine triggers the following:
- ClearML allocates a remote instance (GPU) from your dedicated pool
- On the allocated instance it will spin **jupyter-lab** + **vscode server** + **SSH** access for
interactive usage (i.e., development)
- ClearML will start monitoring machine performance, allowing DevOps to detect stale instances and spin them down
- NEW đĨ Kubernetes support, develop directly inside your pods! No kubectl required!
Read more about `clearml-agent` and interactive sessions [here](https://clear.ml/docs/latest/docs/clearml_agent/#kubernetes)
- NEW đ Automatically store & sync your [interactive session workspace](#store-and-synchronize-interactive-session-workspace).
`clearml-session` will automatically create a snapshot of your entire workspace when shutting it down,
and later restore into a new session on a different remote machine
> âšī¸ **Remote PyCharm:** You can also work with PyCharm in a remote session over SSH. Use the [PyCharm Plugin](https://github.com/allegroai/clearml-pycharm-plugin) to automatically sync local configurations with a remote session.
### Use-cases for remote interactive sessions:
1. Development requires resources not available on the current developer's machines
2. Team resource sharing (e.g. how to dynamically assign GPUs to developers)
3. Spin a copy of a previously executed experiment for remote debugging purposes (:open_mouth:!)
4. Scale-out development to multiple clouds, assign development machines on AWS/GCP/Azure in a seamless way
## Prerequisites:
* **An SSH client installed on your machine** - To verify open your terminal and execute `ssh`, if you did not receive an error, we are good to go.
* At least one `clearml-agent` running on a remote host. See installation [details](https://github.com/allegroai/clearml-agent).
Supported OS: MacOS, Windows, Linux
## đ Secure & Stable
**clearml-session** creates a single, secure, and encrypted connection to the remote machine over SSH.
SSH credentials are automatically generated by the CLI and contain fully random 32 bytes password.
All http connections are tunneled over the SSH connection,
allowing users to add additional services on the remote machine (!)
Furthermore, all tunneled connections have a special stable network layer allowing you to refresh the underlying SSH
connection without breaking any network sockets!
This means that if the network connection is unstable, you can refresh
the base SSH network tunnel, without breaking JupyterLab/VSCode-server or your own SSH connection
(e.h. debugging over SSH with PyCharm)
---
## ⥠How to use: Interactive Session
1. run `clearml-session`
2. select the requested queue (resource)
3. wait until a machine is up and ready
4. click on the link to the remote JupyterLab/VSCode OR connect with the provided SSH details
**Notice! You can also**: Select a **docker image** to execute in, install required **python packages**, run **bash script**,
pass **git credentials**, etc.
See below for full CLI options.
## đ Tutorials
### Getting started
Requirements `clearml` python package installed and configured (see detailed [instructions](https://clear.ml/docs/latest/docs/getting_started/ds/ds_first_steps))
``` bash
pip install clearml-session
clearml-session --docker nvcr.io/nvidia/pytorch:20.11-py3 --git-credentials
```
Wait for the machine to spin up:
Expected CLI output would look something like:
``` console
Creating new session
New session created [id=3d38e738c5ff458a9ec465e77e19da23]
Waiting for remote machine allocation [id=3d38e738c5ff458a9ec465e77e19da23]
.Status [queued]
....Status [in_progress]
Remote machine allocated
Setting remote environment [Task id=3d38e738c5ff458a9ec465e77e19da23]
Setup process details: https://app.community.clear.ml/projects/64ae77968db24b27abf86a501667c330/experiments/3d38e738c5ff458a9ec465e77e19da23/output/log
Waiting for environment setup to complete [usually about 20-30 seconds]
..............
Remote machine is ready
Setting up connection to remote session
Starting SSH tunnel
Warning: Permanently added '[192.168.0.17]:10022' (ECDSA) to the list of known hosts.
root@192.168.0.17's password: f7bae03235ff2a62b6bfbc6ab9479f9e28640a068b1208b63f60cb097b3a1784
Interactive session is running:
SSH: ssh root@localhost -p 8022 [password: f7bae03235ff2a62b6bfbc6ab9479f9e28640a068b1208b63f60cb097b3a1784]
Jupyter Lab URL: http://localhost:8878/?token=df52806d36ad30738117937507b213ac14ed638b8c336a7e
VSCode server available at http://localhost:8898/
Connection is up and running
Enter "r" (or "reconnect") to reconnect the session (for example after suspend)
`s` (or "shell") to connect to the SSH session
`Ctrl-C` (or "quit") to abort (remote session remains active)
or "Shutdown" to shut down remote interactive session
```
Click on the **Jupyter Lab** link (http://localhost:8878/?token=xyz)
Or **VScode** (running inside your remote container) (http://localhost:8898/)
Or drop into **SSH** shell by typying `shell`
Open your terminal, clone your code & start working :)
> âšī¸ **TIP**: You can additional python package to your remote session setup by adding `--packages` to the command line,
> for example to add `boto3` add `--packages "boto3>1"`
> âšī¸ **TIP**: If you need direct SSH into the remote container from your terminal,
> you can directly drop into a shell by adding `--shell` to the command line
### Leaving a session and reconnecting to it
On the `clearml-session` CLI terminal, enter 'quit' or press `Ctrl-C`
It will close the CLI but preserve the remote session (i.e. remote session will remain running)
When you want to reconnect to it, execute:
``` bash
clearml-session
```
Then press "Y" (or enter) to reconnect to the already running session
``` console
clearml-session - launch interactive session
Checking previous session
Connect to active session id=3d38e738c5ff458a9ec465e77e19da23 [Y]/n?
```
### Shutting down a remote session
On the `clearml-session` CLI terminal, enter 'shutdown' (case-insensitive)
It will shut down the remote session, free the resource and close the CLI
```console
Enter "r" (or "reconnect") to reconnect the session (for example after suspend)
`s` (or "shell") to connect to the SSH session
`Ctrl-C` (or "quit") to abort (remote session remains active)
or "Shutdown" to shut down remote interactive session
shutdown
Shutting down interactive session
Remote session shutdown
Goodbye
```
You can also use the CLI to shut down a specific clearml interactive session
```bash
clearml-session shutdown --id <session_id>
```
### Connecting to a running interactive session from a different machine
Continue working on an interactive session from **any** machine.
In the `clearml` web UI, go to DevOps project, and find your interactive session.
Click on the ID button next to the Task name, and copy the unique ID.
```bash
clearml-session --attach <session_id>
```
Click on the JupyterLab/VSCode link, or connect directly to the SSH session
> ⨠**TIP**: You can work & debug your colleagues code and workspace by sharing the `session id`
> and connect to the same remote container together with `--attach`
### Store and synchronize interactive session workspace
Specify the remote workspace root-folder by adding `--store-workspace ~/workspace` to the command line.
In the remote session container, put all your code / data under the `~/workspace` directory.
When your session is shut down, the workspace folder will be automatically package and stored on the clearml file server.
In your next `clearml-session` execution specify again `--store-workspace ~/workspace` and clearml-session
will grab the previous workspace snapshot and restore it into the new remote container in `~/workspace`.
```bash
clearml-session --store-workspace ~/workspace --docker python:3.10-bullseye
```
To continue the last aborted session and restore the workspace
```bash
clearml-session --store-workspace ~/workspace --docker python:3.10-bullseye
```
```console
clearml-session - CLI for launching JupyterLab / VSCode / SSH on a remote machine
Verifying credentials
Use previous queue (resource) '1xGPU' [Y]/n?
Interactive session config:
...
Restore workspace from session id=01bf86f038314434878b2413343ba746 'interactive_session' @ 2024-03-02 20:34:03 [Y]/n?
Restoring workspace from previous session id=01bf86f038314434878b2413343ba746
```
To continue a **specific** session ID and restore its workspace
```bash
clearml-session --continue-session <session_id> --store-workspace ~/workspace --docker python:3.10-bullseye
```
### Upload local files to remote session
If you need to upload files from your local machine into the remote session,
specify the file or directory with `--upload-files /mnt/data/stuff`.
The entire content of the directory / file will be copied into your remote `clearml-session`
container under the `~/session-files/` directory.
Can be used in conjunction with `--store-session` to easily move workloads between local development machine
and remote machines with 100% persistent workspace synchronization.
```bash
clearml-session --upload-files /mnt/data/stuff
```
### Debug a previously executed experiment
If you have a previously executed experiment (Task) on the clearml platform,
you can create an exact copy of the experiment (Task) and debug it on the remote interactive session.
`clearml-session` will replicate the exact remote environment, add JupyterLab/VSCode/SSH and allow you interactively
execute and debug the experiment, on the interactive remote container.
In the `clearml` web UI, find the experiment (Task) you wish to debug.
Click on the ID button next to the Task name, and copy the unique ID, then execute:
```bash
clearml-session --debugging-session <experiment_id_here>
```
Click on the JupyterLab/VSCode link, or drop directly into an SSH shell by typying `shell`
## â Frequently Asked Questions
#### How does it work?
The `clearml-session` creates a new interactive `Task` in the system (default project: DevOps).
This `Task` is responsible for setting the SSH and JupyterLab/VSCode on the host machine.
The local `clearml-session` awaits for the interactive Task to finish with the initial setup, then
it connects via SSH to the host machine (see "safe and stable" above), and tunnels
both SSH and JupyterLab over the SSH connection.
The end results is a local link which you can use to access the JupyterLab/VSCode on the remote machine, over a **secure and encrypted** connection!
#### Does `clearml-session` support Kubernetes clusters?
Yes! `clearml-session` utilizes the `clearml-agent` kubernetes glue together with routing capabilities in order to allow
any clearml-session to spin a container (pod) on the kubernetes cluster and securely connect **directly** into the pod.
This feature does not require any kubernetes access from the users, and simplifies code
development on kubernetes clusters as well as job scheduling & launching.
Read more on how to deploy clearml on kubernetes [here](https://clear.ml/docs/latest/docs/clearml_agent/#kubernetes)
#### How can I use `clearml-session` to scale up / out development resources?
**Clearml** has a [cloud autoscaler](https://clear.ml/docs/latest/docs/cloud_autoscaling/autoscaling_overview), so you can easily and automatically spin machines for development!
There is also a default docker image to use when initiating a task.
This means that using **clearml-session**s
with the autoscaler enabled, allows for turn-key secure development environment inside a docker of your choosing.
Learn more about it [here](https://clear.ml/docs/latest/docs/guides/services/aws_autoscaler) & [here](https://clear.ml/docs/latest/docs/webapp/applications/apps_gpu_compute)
#### Does `clearml-session` fit Work-From-Home setup?
**YES**. Install `clearml-agent` on target machines inside the organization, connect over your company VPN
and use `clearml-session` to gain access to a dedicated on-prem machine with the docker of your choosing
(with out-of-the-box support for any internal docker artifactory).
Learn more about how to utilize your office workstations and on-prem machines [here](https://clear.ml/docs/latest/docs/clearml_agent)
## CLI options
```bash
clearml-session --help
```
```console
clearml-session - CLI for launching JupyterLab / VSCode / SSH on a remote machine
usage: clearml-session [-h] [--version] [--attach [ATTACH]] [--shutdown [SHUTDOWN]] [--shell]
[--debugging-session DEBUGGING_SESSION] [--queue QUEUE] [--docker DOCKER]
[--docker-args DOCKER_ARGS] [--public-ip [true/false]] [--remote-ssh-port REMOTE_SSH_PORT]
[--vscode-server [true/false]] [--vscode-version VSCODE_VERSION]
[--vscode-extensions VSCODE_EXTENSIONS] [--jupyter-lab [true/false]]
[--upload-files UPLOAD_FILES] [--continue-session CONTINUE_SESSION]
[--store-workspace STORE_WORKSPACE] [--git-credentials [true/false]]
[--user-folder USER_FOLDER] [--packages [PACKAGES [PACKAGES ...]]]
[--requirements REQUIREMENTS] [--init-script [INIT_SCRIPT]] [--config-file CONFIG_FILE]
[--remote-gateway [REMOTE_GATEWAY]] [--base-task-id BASE_TASK_ID] [--project PROJECT]
[--session-name SESSION_NAME] [--session-tags [SESSION_TAGS [SESSION_TAGS ...]]]
[--disable-session-cleanup [true/false]] [--keepalive [true/false]]
[--queue-excluded-tag [QUEUE_EXCLUDED_TAG [QUEUE_EXCLUDED_TAG ...]]]
[--queue-include-tag [QUEUE_INCLUDE_TAG [QUEUE_INCLUDE_TAG ...]]]
[--skip-docker-network [true/false]] [--password PASSWORD] [--username USERNAME]
[--force-dropbear [true/false]] [--verbose] [--yes]
{list,info,shutdown} ...
clearml-session - CLI for launching JupyterLab / VSCode / SSH on a remote machine
positional arguments:
{list,info,shutdown} ClearML session control commands
list List running Sessions
info Detailed information on specific session
shutdown Shutdown specific session
optional arguments:
-h, --help show this help message and exit
--version Display the clearml-session utility version
--attach [ATTACH] Attach to running interactive session (default: previous session)
--shutdown [SHUTDOWN], -S [SHUTDOWN]
Shut down an active session (default: previous session)
--shell Open the SSH shell session directly, notice quiting the SSH session will Not shutdown the
remote session
--debugging-session DEBUGGING_SESSION
Pass existing Task id (experiment), create a copy of the experiment on a remote machine,
and launch jupyter/ssh for interactive access. Example --debugging-session <task_id>
--queue QUEUE Select the queue to launch the interactive session on (default: previously used queue)
--docker DOCKER Select the docker image to use in the interactive session on (default: previously used
docker image or `nvidia/cuda:11.6.2-runtime-ubuntu20.04`)
--docker-args DOCKER_ARGS
Add additional arguments for the docker image to use in the interactive session on
(default: previously used docker-args)
--public-ip [true/false]
If True register the public IP of the remote machine. Set if running on the cloud.
Default: false (use for local / on-premises)
--remote-ssh-port REMOTE_SSH_PORT
Set the remote ssh server port, running on the agent`s machine. (default: 10022)
--vscode-server [true/false]
Install vscode server (code-server) on interactive session (default: true)
--vscode-version VSCODE_VERSION
Set vscode server (code-server) version, as well as vscode python extension version
<vscode:python-ext> (example: "3.7.4:2020.10.332292344")
--vscode-extensions VSCODE_EXTENSIONS
Install additional vscode extensions, as well as vscode python extension (example: "ms-
python.python,ms-python.black-formatter,ms-python.pylint,ms-python.flake8")
--jupyter-lab [true/false]
Install Jupyter-Lab on interactive session (default: true)
--upload-files UPLOAD_FILES
Advanced: Upload local files/folders to the remote session. Example: `/my/local/data/`
will upload the local folder and extract it into the container in ~/session-files/
--continue-session CONTINUE_SESSION
Continue previous session (ID provided) restoring your workspace (see --store-workspace)
--store-workspace STORE_WORKSPACE
Upload/Restore remote workspace folder. Example: `~/workspace/` will automatically
restore/store the *containers* folder and extract it into next the session. Use with
--continue-session to continue your previous work from your exact container state
--git-credentials [true/false]
If true, local .git-credentials file is sent to the interactive session. (default: false)
--user-folder USER_FOLDER
Advanced: Set the remote base folder (default: ~/)
--packages [PACKAGES [PACKAGES ...]]
Additional packages to add, supports version numbers (default: previously added packages).
examples: --packages torch==1.7 tqdm
--requirements REQUIREMENTS
Specify requirements.txt file to install when setting the interactive session.
Requirements file is read and stored in `packages` section as default for the next
sessions. Can be overridden by calling `--packages`
--init-script [INIT_SCRIPT]
Specify BASH init script file to be executed when setting the interactive session. Script
content is read and stored as default script for the next sessions. To clear the init-
script do not pass a file
--config-file CONFIG_FILE
Advanced: Change the configuration file used to store the previous state (default:
~/.clearml_session.json)
--remote-gateway [REMOTE_GATEWAY]
Advanced: Specify gateway ip/address:port to be passed to interactive session (for use
with k8s ingestion / ELB)
--base-task-id BASE_TASK_ID
Advanced: Set the base task ID for the interactive session. (default: previously used
Task). Use `none` for the default interactive session
--project PROJECT Advanced: Set the project name for the interactive session Task
--session-name SESSION_NAME
Advanced: Set the name of the interactive session Task
--session-tags [SESSION_TAGS [SESSION_TAGS ...]]
Advanced: Add tags to the interactive session for increased visibility
--disable-session-cleanup [true/false]
Advanced: If set, previous interactive sessions are not deleted
--keepalive [true/false]
Advanced: If set, enables the transparent proxy always keeping the sockets alive. Default:
False, do not use transparent socket for mitigating connection drops.
--queue-excluded-tag [QUEUE_EXCLUDED_TAG [QUEUE_EXCLUDED_TAG ...]]
Advanced: Excluded queues with this specific tag from the selection
--queue-include-tag [QUEUE_INCLUDE_TAG [QUEUE_INCLUDE_TAG ...]]
Advanced: Only include queues with this specific tag from the selection
--skip-docker-network [true/false]
Advanced: If set, `--network host` is **not** passed to docker (assumes k8s network
ingestion) (default: false)
--password PASSWORD Advanced: Select ssh password for the interactive session (default: `randomly-generated`
or previously used one)
--username USERNAME Advanced: Select ssh username for the interactive session (default: `root` or previously
used one)
--force-dropbear [true/false]
Force using `dropbear` instead of SSHd
--verbose Advanced: If set, print verbose progress information, e.g. the remote machine setup
process log
--yes, -y Automatic yes to prompts; assume "yes" as answer to all prompts and run non-interactively
Notice! all arguments are stored as new defaults for the next execution
```
Raw data
{
"_id": null,
"home_page": "https://github.com/allegroai/clearml-session",
"name": "clearml-session",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "clearml mlops devops trains development machine deep learning version control machine-learning machinelearning deeplearning deep-learning experiment-manager jupyter vscode",
"author": "Allegroai",
"author_email": "clearml@allegro.ai",
"download_url": "",
"platform": null,
"description": "<div align=\"center\">\n\n<a href=\"https://app.clear.ml\"><img src=\"https://github.com/allegroai/clearml/blob/master/docs/clearml-logo.svg?raw=true\" width=\"250px\"></a>\n\n## **`clearml-session` </br> CLI for launching JupyterLab / VSCode / SSH on a remote machine**\n\n## \ud83d\udd25 NEW in version `0.13` [Workspace Syncing](#store-and-synchronize-interactive-session-workspace) \ud83d\ude80 \n\n\n[](https://img.shields.io/github/license/allegroai/clearml-session.svg)\n[](https://img.shields.io/pypi/pyversions/clearml-session.svg)\n[](https://img.shields.io/pypi/v/clearml-session.svg)\n[](https://pypi.python.org/pypi/clearml-session/)\n[](https://joinslack.clear.ml)\n\n\n</div>\n\n**`clearml-session`** is a utility for launching detachable remote interactive sessions (MacOS, Windows, Linux)\n\n### tl;dr \n**CLI to launch a remote session of Jupyter-Lab / VSCode / SSH, \ninside any docker container on any deployment, Cloud / Kubernetes / Bare-Metal**\n\n### \ud83d\udd30 What does it do?\nStarting a clearml (ob)session from your local machine triggers the following:\n- ClearML allocates a remote instance (GPU) from your dedicated pool\n- On the allocated instance it will spin **jupyter-lab** + **vscode server** + **SSH** access for\ninteractive usage (i.e., development)\n- ClearML will start monitoring machine performance, allowing DevOps to detect stale instances and spin them down\n- NEW \ud83d\udd25 Kubernetes support, develop directly inside your pods! No kubectl required! \nRead more about `clearml-agent` and interactive sessions [here](https://clear.ml/docs/latest/docs/clearml_agent/#kubernetes) \n- NEW \ud83c\udf89 Automatically store & sync your [interactive session workspace](#store-and-synchronize-interactive-session-workspace). \n`clearml-session` will automatically create a snapshot of your entire workspace when shutting it down, \nand later restore into a new session on a different remote machine\n\n> \u2139\ufe0f **Remote PyCharm:** You can also work with PyCharm in a remote session over SSH. Use the [PyCharm Plugin](https://github.com/allegroai/clearml-pycharm-plugin) to automatically sync local configurations with a remote session.\n\n\n### Use-cases for remote interactive sessions:\n1. Development requires resources not available on the current developer's machines\n2. Team resource sharing (e.g. how to dynamically assign GPUs to developers)\n3. Spin a copy of a previously executed experiment for remote debugging purposes (:open_mouth:!)\n4. Scale-out development to multiple clouds, assign development machines on AWS/GCP/Azure in a seamless way\n\n## Prerequisites:\n* **An SSH client installed on your machine** - To verify open your terminal and execute `ssh`, if you did not receive an error, we are good to go.\n* At least one `clearml-agent` running on a remote host. See installation [details](https://github.com/allegroai/clearml-agent).\n\nSupported OS: MacOS, Windows, Linux\n\n\n## \ud83d\udd12 Secure & Stable\n**clearml-session** creates a single, secure, and encrypted connection to the remote machine over SSH.\nSSH credentials are automatically generated by the CLI and contain fully random 32 bytes password.\n\nAll http connections are tunneled over the SSH connection,\nallowing users to add additional services on the remote machine (!)\n\nFurthermore, all tunneled connections have a special stable network layer allowing you to refresh the underlying SSH\nconnection without breaking any network sockets! \n\nThis means that if the network connection is unstable, you can refresh\nthe base SSH network tunnel, without breaking JupyterLab/VSCode-server or your own SSH connection\n(e.h. debugging over SSH with PyCharm) \n\n---\n\n## \u26a1 How to use: Interactive Session\n\n\n1. run `clearml-session`\n2. select the requested queue (resource)\n3. wait until a machine is up and ready\n4. click on the link to the remote JupyterLab/VSCode OR connect with the provided SSH details\n\n**Notice! You can also**: Select a **docker image** to execute in, install required **python packages**, run **bash script**,\npass **git credentials**, etc.\nSee below for full CLI options.\n\n## \ud83d\udcd6 Tutorials\n\n### Getting started\n\nRequirements `clearml` python package installed and configured (see detailed [instructions](https://clear.ml/docs/latest/docs/getting_started/ds/ds_first_steps))\n``` bash\npip install clearml-session\nclearml-session --docker nvcr.io/nvidia/pytorch:20.11-py3 --git-credentials\n```\n\nWait for the machine to spin up:\nExpected CLI output would look something like:\n``` console\nCreating new session\nNew session created [id=3d38e738c5ff458a9ec465e77e19da23]\nWaiting for remote machine allocation [id=3d38e738c5ff458a9ec465e77e19da23]\n.Status [queued]\n....Status [in_progress]\nRemote machine allocated\nSetting remote environment [Task id=3d38e738c5ff458a9ec465e77e19da23]\nSetup process details: https://app.community.clear.ml/projects/64ae77968db24b27abf86a501667c330/experiments/3d38e738c5ff458a9ec465e77e19da23/output/log\nWaiting for environment setup to complete [usually about 20-30 seconds]\n..............\nRemote machine is ready\nSetting up connection to remote session\nStarting SSH tunnel\nWarning: Permanently added '[192.168.0.17]:10022' (ECDSA) to the list of known hosts.\nroot@192.168.0.17's password: f7bae03235ff2a62b6bfbc6ab9479f9e28640a068b1208b63f60cb097b3a1784\n\n\nInteractive session is running:\nSSH: ssh root@localhost -p 8022 [password: f7bae03235ff2a62b6bfbc6ab9479f9e28640a068b1208b63f60cb097b3a1784]\nJupyter Lab URL: http://localhost:8878/?token=df52806d36ad30738117937507b213ac14ed638b8c336a7e\nVSCode server available at http://localhost:8898/\n\nConnection is up and running\nEnter \"r\" (or \"reconnect\") to reconnect the session (for example after suspend)\n`s` (or \"shell\") to connect to the SSH session\n`Ctrl-C` (or \"quit\") to abort (remote session remains active)\nor \"Shutdown\" to shut down remote interactive session\n```\n\nClick on the **Jupyter Lab** link (http://localhost:8878/?token=xyz)\nOr **VScode** (running inside your remote container) (http://localhost:8898/)\nOr drop into **SSH** shell by typying `shell`\n\nOpen your terminal, clone your code & start working :)\n\n> \u2139\ufe0f **TIP**: You can additional python package to your remote session setup by adding `--packages` to the command line, \n> for example to add `boto3` add `--packages \"boto3>1\"`\n\n> \u2139\ufe0f **TIP**: If you need direct SSH into the remote container from your terminal, \n> you can directly drop into a shell by adding `--shell` to the command line\n\n\n### Leaving a session and reconnecting to it\n\nOn the `clearml-session` CLI terminal, enter 'quit' or press `Ctrl-C`\nIt will close the CLI but preserve the remote session (i.e. remote session will remain running)\n\nWhen you want to reconnect to it, execute:\n``` bash\nclearml-session\n```\n\nThen press \"Y\" (or enter) to reconnect to the already running session\n``` console\nclearml-session - launch interactive session\nChecking previous session\nConnect to active session id=3d38e738c5ff458a9ec465e77e19da23 [Y]/n?\n```\n\n### Shutting down a remote session\n\nOn the `clearml-session` CLI terminal, enter 'shutdown' (case-insensitive)\nIt will shut down the remote session, free the resource and close the CLI\n\n```console\nEnter \"r\" (or \"reconnect\") to reconnect the session (for example after suspend)\n`s` (or \"shell\") to connect to the SSH session\n`Ctrl-C` (or \"quit\") to abort (remote session remains active)\nor \"Shutdown\" to shut down remote interactive session\n\nshutdown\n\nShutting down interactive session\nRemote session shutdown\nGoodbye\n```\n\nYou can also use the CLI to shut down a specific clearml interactive session\n\n```bash\nclearml-session shutdown --id <session_id>\n```\n\n### Connecting to a running interactive session from a different machine\n\nContinue working on an interactive session from **any** machine.\nIn the `clearml` web UI, go to DevOps project, and find your interactive session.\nClick on the ID button next to the Task name, and copy the unique ID.\n\n```bash\nclearml-session --attach <session_id>\n```\n\nClick on the JupyterLab/VSCode link, or connect directly to the SSH session\n\n> \u2728 **TIP**: You can work & debug your colleagues code and workspace by sharing the `session id` \n> and connect to the same remote container together with `--attach`\n\n\n### Store and synchronize interactive session workspace\n\nSpecify the remote workspace root-folder by adding `--store-workspace ~/workspace` to the command line.\nIn the remote session container, put all your code / data under the `~/workspace` directory.\nWhen your session is shut down, the workspace folder will be automatically package and stored on the clearml file server.\nIn your next `clearml-session` execution specify again `--store-workspace ~/workspace` and clearml-session\nwill grab the previous workspace snapshot and restore it into the new remote container in `~/workspace`.\n\n```bash\nclearml-session --store-workspace ~/workspace --docker python:3.10-bullseye\n```\n\nTo continue the last aborted session and restore the workspace\n\n```bash\nclearml-session --store-workspace ~/workspace --docker python:3.10-bullseye\n```\n\n```console\nclearml-session - CLI for launching JupyterLab / VSCode / SSH on a remote machine\nVerifying credentials\nUse previous queue (resource) '1xGPU' [Y]/n? \n\nInteractive session config:\n...\nRestore workspace from session id=01bf86f038314434878b2413343ba746 'interactive_session' @ 2024-03-02 20:34:03 [Y]/n? \nRestoring workspace from previous session id=01bf86f038314434878b2413343ba746\n```\n\nTo continue a **specific** session ID and restore its workspace\n\n```bash\nclearml-session --continue-session <session_id> --store-workspace ~/workspace --docker python:3.10-bullseye\n```\n\n### Upload local files to remote session\n\nIf you need to upload files from your local machine into the remote session, \nspecify the file or directory with `--upload-files /mnt/data/stuff`. \nThe entire content of the directory / file will be copied into your remote `clearml-session` \ncontainer under the `~/session-files/` directory. \n\nCan be used in conjunction with `--store-session` to easily move workloads between local development machine\nand remote machines with 100% persistent workspace synchronization.\n\n```bash\nclearml-session --upload-files /mnt/data/stuff\n```\n\n\n### Debug a previously executed experiment\n\nIf you have a previously executed experiment (Task) on the clearml platform,\nyou can create an exact copy of the experiment (Task) and debug it on the remote interactive session.\n`clearml-session` will replicate the exact remote environment, add JupyterLab/VSCode/SSH and allow you interactively\nexecute and debug the experiment, on the interactive remote container. \n\nIn the `clearml` web UI, find the experiment (Task) you wish to debug.\nClick on the ID button next to the Task name, and copy the unique ID, then execute:\n\n```bash\nclearml-session --debugging-session <experiment_id_here>\n```\n\nClick on the JupyterLab/VSCode link, or drop directly into an SSH shell by typying `shell`\n\n\n## \u2753 Frequently Asked Questions\n\n#### How does it work?\n\nThe `clearml-session` creates a new interactive `Task` in the system (default project: DevOps).\n\nThis `Task` is responsible for setting the SSH and JupyterLab/VSCode on the host machine.\n\nThe local `clearml-session` awaits for the interactive Task to finish with the initial setup, then\nit connects via SSH to the host machine (see \"safe and stable\" above), and tunnels\nboth SSH and JupyterLab over the SSH connection.\n\nThe end results is a local link which you can use to access the JupyterLab/VSCode on the remote machine, over a **secure and encrypted** connection!\n\n#### Does `clearml-session` support Kubernetes clusters?\n\nYes! `clearml-session` utilizes the `clearml-agent` kubernetes glue together with routing capabilities in order to allow\nany clearml-session to spin a container (pod) on the kubernetes cluster and securely connect **directly** into the pod.\nThis feature does not require any kubernetes access from the users, and simplifies code\ndevelopment on kubernetes clusters as well as job scheduling & launching. \nRead more on how to deploy clearml on kubernetes [here](https://clear.ml/docs/latest/docs/clearml_agent/#kubernetes)\n\n#### How can I use `clearml-session` to scale up / out development resources?\n\n**Clearml** has a [cloud autoscaler](https://clear.ml/docs/latest/docs/cloud_autoscaling/autoscaling_overview), so you can easily and automatically spin machines for development!\n\nThere is also a default docker image to use when initiating a task.\n\nThis means that using **clearml-session**s\nwith the autoscaler enabled, allows for turn-key secure development environment inside a docker of your choosing.\n\nLearn more about it [here](https://clear.ml/docs/latest/docs/guides/services/aws_autoscaler) & [here](https://clear.ml/docs/latest/docs/webapp/applications/apps_gpu_compute)\n\n#### Does `clearml-session` fit Work-From-Home setup?\n**YES**. Install `clearml-agent` on target machines inside the organization, connect over your company VPN \nand use `clearml-session` to gain access to a dedicated on-prem machine with the docker of your choosing\n(with out-of-the-box support for any internal docker artifactory).\n\nLearn more about how to utilize your office workstations and on-prem machines [here](https://clear.ml/docs/latest/docs/clearml_agent)\n\n## CLI options\n\n```bash\nclearml-session --help\n```\n\n```console\nclearml-session - CLI for launching JupyterLab / VSCode / SSH on a remote machine\nusage: clearml-session [-h] [--version] [--attach [ATTACH]] [--shutdown [SHUTDOWN]] [--shell]\n [--debugging-session DEBUGGING_SESSION] [--queue QUEUE] [--docker DOCKER]\n [--docker-args DOCKER_ARGS] [--public-ip [true/false]] [--remote-ssh-port REMOTE_SSH_PORT]\n [--vscode-server [true/false]] [--vscode-version VSCODE_VERSION]\n [--vscode-extensions VSCODE_EXTENSIONS] [--jupyter-lab [true/false]]\n [--upload-files UPLOAD_FILES] [--continue-session CONTINUE_SESSION]\n [--store-workspace STORE_WORKSPACE] [--git-credentials [true/false]]\n [--user-folder USER_FOLDER] [--packages [PACKAGES [PACKAGES ...]]]\n [--requirements REQUIREMENTS] [--init-script [INIT_SCRIPT]] [--config-file CONFIG_FILE]\n [--remote-gateway [REMOTE_GATEWAY]] [--base-task-id BASE_TASK_ID] [--project PROJECT]\n [--session-name SESSION_NAME] [--session-tags [SESSION_TAGS [SESSION_TAGS ...]]]\n [--disable-session-cleanup [true/false]] [--keepalive [true/false]]\n [--queue-excluded-tag [QUEUE_EXCLUDED_TAG [QUEUE_EXCLUDED_TAG ...]]]\n [--queue-include-tag [QUEUE_INCLUDE_TAG [QUEUE_INCLUDE_TAG ...]]]\n [--skip-docker-network [true/false]] [--password PASSWORD] [--username USERNAME]\n [--force-dropbear [true/false]] [--verbose] [--yes]\n {list,info,shutdown} ...\n\nclearml-session - CLI for launching JupyterLab / VSCode / SSH on a remote machine\n\npositional arguments:\n {list,info,shutdown} ClearML session control commands\n list List running Sessions\n info Detailed information on specific session\n shutdown Shutdown specific session\n\noptional arguments:\n -h, --help show this help message and exit\n --version Display the clearml-session utility version\n --attach [ATTACH] Attach to running interactive session (default: previous session)\n --shutdown [SHUTDOWN], -S [SHUTDOWN]\n Shut down an active session (default: previous session)\n --shell Open the SSH shell session directly, notice quiting the SSH session will Not shutdown the\n remote session\n --debugging-session DEBUGGING_SESSION\n Pass existing Task id (experiment), create a copy of the experiment on a remote machine,\n and launch jupyter/ssh for interactive access. Example --debugging-session <task_id>\n --queue QUEUE Select the queue to launch the interactive session on (default: previously used queue)\n --docker DOCKER Select the docker image to use in the interactive session on (default: previously used\n docker image or `nvidia/cuda:11.6.2-runtime-ubuntu20.04`)\n --docker-args DOCKER_ARGS\n Add additional arguments for the docker image to use in the interactive session on\n (default: previously used docker-args)\n --public-ip [true/false]\n If True register the public IP of the remote machine. Set if running on the cloud.\n Default: false (use for local / on-premises)\n --remote-ssh-port REMOTE_SSH_PORT\n Set the remote ssh server port, running on the agent`s machine. (default: 10022)\n --vscode-server [true/false]\n Install vscode server (code-server) on interactive session (default: true)\n --vscode-version VSCODE_VERSION\n Set vscode server (code-server) version, as well as vscode python extension version\n <vscode:python-ext> (example: \"3.7.4:2020.10.332292344\")\n --vscode-extensions VSCODE_EXTENSIONS\n Install additional vscode extensions, as well as vscode python extension (example: \"ms-\n python.python,ms-python.black-formatter,ms-python.pylint,ms-python.flake8\")\n --jupyter-lab [true/false]\n Install Jupyter-Lab on interactive session (default: true)\n --upload-files UPLOAD_FILES\n Advanced: Upload local files/folders to the remote session. Example: `/my/local/data/`\n will upload the local folder and extract it into the container in ~/session-files/\n --continue-session CONTINUE_SESSION\n Continue previous session (ID provided) restoring your workspace (see --store-workspace)\n --store-workspace STORE_WORKSPACE\n Upload/Restore remote workspace folder. Example: `~/workspace/` will automatically\n restore/store the *containers* folder and extract it into next the session. Use with\n --continue-session to continue your previous work from your exact container state\n --git-credentials [true/false]\n If true, local .git-credentials file is sent to the interactive session. (default: false)\n --user-folder USER_FOLDER\n Advanced: Set the remote base folder (default: ~/)\n --packages [PACKAGES [PACKAGES ...]]\n Additional packages to add, supports version numbers (default: previously added packages).\n examples: --packages torch==1.7 tqdm\n --requirements REQUIREMENTS\n Specify requirements.txt file to install when setting the interactive session.\n Requirements file is read and stored in `packages` section as default for the next\n sessions. Can be overridden by calling `--packages`\n --init-script [INIT_SCRIPT]\n Specify BASH init script file to be executed when setting the interactive session. Script\n content is read and stored as default script for the next sessions. To clear the init-\n script do not pass a file\n --config-file CONFIG_FILE\n Advanced: Change the configuration file used to store the previous state (default:\n ~/.clearml_session.json)\n --remote-gateway [REMOTE_GATEWAY]\n Advanced: Specify gateway ip/address:port to be passed to interactive session (for use\n with k8s ingestion / ELB)\n --base-task-id BASE_TASK_ID\n Advanced: Set the base task ID for the interactive session. (default: previously used\n Task). Use `none` for the default interactive session\n --project PROJECT Advanced: Set the project name for the interactive session Task\n --session-name SESSION_NAME\n Advanced: Set the name of the interactive session Task\n --session-tags [SESSION_TAGS [SESSION_TAGS ...]]\n Advanced: Add tags to the interactive session for increased visibility\n --disable-session-cleanup [true/false]\n Advanced: If set, previous interactive sessions are not deleted\n --keepalive [true/false]\n Advanced: If set, enables the transparent proxy always keeping the sockets alive. Default:\n False, do not use transparent socket for mitigating connection drops.\n --queue-excluded-tag [QUEUE_EXCLUDED_TAG [QUEUE_EXCLUDED_TAG ...]]\n Advanced: Excluded queues with this specific tag from the selection\n --queue-include-tag [QUEUE_INCLUDE_TAG [QUEUE_INCLUDE_TAG ...]]\n Advanced: Only include queues with this specific tag from the selection\n --skip-docker-network [true/false]\n Advanced: If set, `--network host` is **not** passed to docker (assumes k8s network\n ingestion) (default: false)\n --password PASSWORD Advanced: Select ssh password for the interactive session (default: `randomly-generated`\n or previously used one)\n --username USERNAME Advanced: Select ssh username for the interactive session (default: `root` or previously\n used one)\n --force-dropbear [true/false]\n Force using `dropbear` instead of SSHd\n --verbose Advanced: If set, print verbose progress information, e.g. the remote machine setup\n process log\n --yes, -y Automatic yes to prompts; assume \"yes\" as answer to all prompts and run non-interactively\n\nNotice! all arguments are stored as new defaults for the next execution\n```\n\n\n\n\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "clearml-session - CLI for launching JupyterLab / VSCode on a remote machine",
"version": "0.13.0",
"project_urls": {
"Homepage": "https://github.com/allegroai/clearml-session"
},
"split_keywords": [
"clearml",
"mlops",
"devops",
"trains",
"development",
"machine",
"deep",
"learning",
"version",
"control",
"machine-learning",
"machinelearning",
"deeplearning",
"deep-learning",
"experiment-manager",
"jupyter",
"vscode"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "23e3f3426345a904a203b0e2632135cb7d282b48c69ab7cf6da9545f5a04b64d",
"md5": "c4d7abbbd401be6c24c9894220fe90a4",
"sha256": "efc529044146201a8296af77f32921be367e4f2fdeb0fe409d70464ce3b5b180"
},
"downloads": -1,
"filename": "clearml_session-0.13.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c4d7abbbd401be6c24c9894220fe90a4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 48679,
"upload_time": "2024-03-07T09:33:16",
"upload_time_iso_8601": "2024-03-07T09:33:16.888486Z",
"url": "https://files.pythonhosted.org/packages/23/e3/f3426345a904a203b0e2632135cb7d282b48c69ab7cf6da9545f5a04b64d/clearml_session-0.13.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-07 09:33:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "allegroai",
"github_project": "clearml-session",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "clearml-session"
}
JSON
