backend.ai-manager


Namebackend.ai-manager JSON
Version 24.3.2 PyPI version JSON
download
home_pagehttps://github.com/lablup/backend.ai
SummaryBackend.AI Manager
upload_time2024-04-17 06:47:19
maintainerNone
docs_urlNone
authorLablup Inc. and contributors
requires_python<3.13,>=3.12
licenseLGPLv3
keywords
VCS
bugtrack_url
requirements aiodataloader-ng aiodocker aiofiles aiohttp aiohttp_cors aiohttp_sse aiodns aiomonitor aioresponses aiosqlite aiotools aiotusclient alembic appdirs async_timeout asyncpg asynctest asyncudp attrs bcrypt boto3 cachetools callosum cattrs click coloredlogs colorama cryptography dataclasses-json faker graphene graypy humanize ifaddr inquirer janus Jinja2 jupyter-client kubernetes kubernetes-asyncio lark more-itertools msgpack multidict namedlist networkx pexpect psutil pycryptodome python-dateutil python-dotenv python-json-logger pyzmq PyJWT PyYAML pydantic packaging hiredis redis rich SQLAlchemy setproctitle tabulate temporenc tenacity tomli tomlkit tqdm trafaret treelib typeguard typing_extensions textual uvloop yarl zipstream-new pytest pytest-dependency types-six types-setuptools types-python-dateutil types-aiofiles types-cachetools types-Jinja2 types-PyYAML types-redis types-tabulate backend.ai-krunner-alpine backend.ai-krunner-static-gnu etcd-client-py
Travis-CI No Travis.
coveralls test coverage No coveralls.
            Backend.AI Manager with API Gateway
===================================

Package Structure
-----------------

* `ai.backend.manager`: Computing resource and workload management with public APIs

Installation
------------

Please visit [the installation guides](https://github.com/lablup/backend.ai/wiki).


### Kernel/system configuration

#### Recommended resource limits:

**`/etc/security/limits.conf`**
```
root hard nofile 512000
root soft nofile 512000
root hard nproc 65536
root soft nproc 65536
user hard nofile 512000
user soft nofile 512000
user hard nproc 65536
user soft nproc 65536
```

**sysctl**
```
fs.file-max=2048000
net.core.somaxconn=1024
net.ipv4.tcp_max_syn_backlog=1024
net.ipv4.tcp_slow_start_after_idle=0
net.ipv4.tcp_fin_timeout=10
net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_early_retrans=1
net.ipv4.ip_local_port_range="10000 65000"
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.ipv4.tcp_rmem=4096 12582912 16777216
net.ipv4.tcp_wmem=4096 12582912 16777216
```


### For development

#### Prerequisites

* Python 3.6 or higher with [pyenv](https://github.com/pyenv/pyenv)
and [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv) (optional but recommneded)
* Docker 18.03 or later with docker-compose (18.09 or later is recommended)

#### Common steps

Clone [the meta repository](https://github.com/lablup/backend.ai) and install a "halfstack"
configuration.  The halfstack configuration installs and runs several dependency daemons such as etcd in
the background.

```console
$ git clone https://github.com/lablup/backend.ai halfstack
$ cd halfstack
$ docker-compose -f docker-compose.halfstack.yml up -d
```

Then prepare the source clone of the agent as follows.
First install the current working copy.

```console
$ git clone https://github.com/lablup/backend.ai-manager manager
$ cd manager
$ pyenv virtualenv venv-manager
$ pyenv local venv-manager
$ pip install -U pip setuptools
$ pip install -U -r requirements/dev.txt
```

From now on, let's assume all shell commands are executed inside the virtualenv.

### Halfstack (single-node development & testing)

#### Recommended directory structure

* `backend.ai-dev`
  - `manager` (git clone from this repo)
  - `agent` (git clone from [the agent repo](https://github.com/lablup/backend.ai-agent))
  - `common` (git clone from [the common repo](https://github.com/lablup/backend.ai-common))

Install `backend.ai-common` as an editable package in the manager (and the agent) virtualenvs
to keep the codebase up-to-date.

```console
$ cd manager
$ pip install -U -e ../common -r requirements/dev.txt
```

#### Steps

Copy (or symlink) the halfstack configs:
```console
$ cp config/halfstack.toml ./manager.toml
$ cp config/halfstack.alembic.ini ./alembic.ini
```

Set up Redis:
```console
$ backend.ai mgr etcd put config/redis/addr 127.0.0.1:8110
```

> ℹ️ NOTE: You may replace `backend.ai mgr` with `python -m ai.backend.manager.cli` in case your `PATH` is unmodifiable.

Set up the public Docker registry:
```console
$ backend.ai mgr etcd put config/docker/registry/index.docker.io "https://registry-1.docker.io"
$ backend.ai mgr etcd put config/docker/registry/index.docker.io/username "lablup"
$ backend.ai mgr image rescan index.docker.io
```

Set up the vfolder paths:
```console
$ mkdir -p "$HOME/vfroot/local"
$ backend.ai mgr etcd put volumes/_mount "$HOME/vfroot"
$ backend.ai mgr etcd put volumes/_default_host local
```

Set up the allowed types of vfolder. Allowed values are "user" or "group".
If none is specified, "user" type is set implicitly:
```console
$ backend.ai mgr etcd put volumes/_types/user ""   # enable user vfolder
$ backend.ai mgr etcd put volumes/_types/group ""  # enable group vfolder
```

Set up the database:
```console
$ backend.ai mgr schema oneshot
$ backend.ai mgr fixture populate sample-configs/example-keypairs.json
$ backend.ai mgr fixture populate sample-configs/example-resource-presets.json
```

Then, run it (for debugging, append a `--debug` flag):

```console
$ backend.ai mgr start-server
```

To run tests:

```console
$ python -m flake8 src tests
$ python -m pytest -m 'not integration' tests
```

Now you are ready to install the agent.
Head to [the README of Backend.AI Agent](https://github.com/lablup/backend.ai-agent/blob/master/README.md).

NOTE: To run tests including integration tests, you first need to install and run the agent on the same host.

## Deployment

### Configuration

Put a TOML-formatted manager configuration (see the sample in `config/sample.toml`)
in one of the following locations:

 * `manager.toml` (current working directory)
 * `~/.config/backend.ai/manager.toml` (user-config directory)
 * `/etc/backend.ai/manager.toml` (system-config directory)

Only the first found one is used by the daemon.

Also many configurations shared by both manager and agent are stored in etcd.
As you might have noticed above, the manager provides a CLI interface to access and manipulate the etcd
data.  Check out the help page of our etcd command set:

```console
$ python -m ai.backend.manager.cli etcd --help
```

If you run etcd as a Docker container (e.g., via halfstack), you may use the native client as well.
In this case, PLEASE BE WARNED that you must prefix the keys with "/sorna/{namespace}" manaully:

```console
$ docker exec -it ${ETCD_CONTAINER_ID} /bin/ash -c 'ETCDCTL_API=3 etcdctl ...'
```

### Running from a command line

The minimal command to execute:

```sh
python -m ai.backend.gateway.server
```

For more arguments and options, run the command with `--help` option.

### Writing a wrapper script

To use with systemd, crontab, and other system-level daemons, you may need to write a shell script
that executes specific CLI commands provided by Backend.AI modules.

The following example shows how to set up pyenv and virtualenv for the script-local environment.
It runs the gateway server if no arguments are given, and execute the given arguments as a shell command
if any.
For instance, you may get/set configurations like: `run-manager.sh python -m ai.backend.manager.etcd ...`
where the name of scripts is `run-manager.sh`.

```bash
#! /bin/bash
if [ -z "$HOME" ]; then
  export HOME="/home/devops"
fi
if [ -z "$PYENV_ROOT" ]; then
  export PYENV_ROOT="$HOME/.pyenv"
  export PATH="$PYENV_ROOT/bin:$PATH"
fi
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
pyenv activate venv-bai-manager

if [ "$#" -eq 0 ]; then
  exec python -m ai.backend.gateway.server
else
  exec "$@"
fi
```

### Networking

The manager and agent should run in the same local network or different
networks reachable via VPNs, whereas the manager's API service must be exposed to
the public network or another private network that users have access to.

The manager requires access to the etcd, the PostgreSQL database, and the Redis server.

| User-to-Manager TCP Ports | Usage |
|:-------------------------:|-------|
| manager:{80,443}          | Backend.AI API access |

| Manager-to-X TCP Ports | Usage |
|:----------------------:|-------|
| etcd:2379              | etcd API access |
| postgres:5432          | Database access |
| redis:6379             | Redis API access |

The manager must also be able to access TCP ports 6001, 6009, and 30000 to 31000 of the agents in default
configurations.  You can of course change those port numbers and ranges in the configuration.

| Manager-to-Agent TCP Ports | Usage |
|:--------------------------:|-------|
| 6001                       | ZeroMQ-based RPC calls from managers to agents |
| 6009                       | HTTP watcher API |
| 30000-31000                | Port pool for in-container services |


LICENSES
--------

[GNU Lesser General Public License](https://github.com/lablup/backend.ai-manager/blob/master/LICENSE)
[Dependencies](https://github.com/lablup/backend.ai-manager/blob/master/DEPENDENCIES.md)

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/lablup/backend.ai",
    "name": "backend.ai-manager",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<3.13,>=3.12",
    "maintainer_email": null,
    "keywords": null,
    "author": "Lablup Inc. and contributors",
    "author_email": null,
    "download_url": "https://files.pythonhosted.org/packages/aa/31/369c5ff80bb6f74b0e778b097990a46784d053aa392786132d9a53e5fba4/backend.ai-manager-24.3.2.tar.gz",
    "platform": null,
    "description": "Backend.AI Manager with API Gateway\n===================================\n\nPackage Structure\n-----------------\n\n* `ai.backend.manager`: Computing resource and workload management with public APIs\n\nInstallation\n------------\n\nPlease visit [the installation guides](https://github.com/lablup/backend.ai/wiki).\n\n\n### Kernel/system configuration\n\n#### Recommended resource limits:\n\n**`/etc/security/limits.conf`**\n```\nroot hard nofile 512000\nroot soft nofile 512000\nroot hard nproc 65536\nroot soft nproc 65536\nuser hard nofile 512000\nuser soft nofile 512000\nuser hard nproc 65536\nuser soft nproc 65536\n```\n\n**sysctl**\n```\nfs.file-max=2048000\nnet.core.somaxconn=1024\nnet.ipv4.tcp_max_syn_backlog=1024\nnet.ipv4.tcp_slow_start_after_idle=0\nnet.ipv4.tcp_fin_timeout=10\nnet.ipv4.tcp_window_scaling=1\nnet.ipv4.tcp_tw_reuse=1\nnet.ipv4.tcp_early_retrans=1\nnet.ipv4.ip_local_port_range=\"10000 65000\"\nnet.core.rmem_max=16777216\nnet.core.wmem_max=16777216\nnet.ipv4.tcp_rmem=4096 12582912 16777216\nnet.ipv4.tcp_wmem=4096 12582912 16777216\n```\n\n\n### For development\n\n#### Prerequisites\n\n* Python 3.6 or higher with [pyenv](https://github.com/pyenv/pyenv)\nand [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv) (optional but recommneded)\n* Docker 18.03 or later with docker-compose (18.09 or later is recommended)\n\n#### Common steps\n\nClone [the meta repository](https://github.com/lablup/backend.ai) and install a \"halfstack\"\nconfiguration.  The halfstack configuration installs and runs several dependency daemons such as etcd in\nthe background.\n\n```console\n$ git clone https://github.com/lablup/backend.ai halfstack\n$ cd halfstack\n$ docker-compose -f docker-compose.halfstack.yml up -d\n```\n\nThen prepare the source clone of the agent as follows.\nFirst install the current working copy.\n\n```console\n$ git clone https://github.com/lablup/backend.ai-manager manager\n$ cd manager\n$ pyenv virtualenv venv-manager\n$ pyenv local venv-manager\n$ pip install -U pip setuptools\n$ pip install -U -r requirements/dev.txt\n```\n\nFrom now on, let's assume all shell commands are executed inside the virtualenv.\n\n### Halfstack (single-node development & testing)\n\n#### Recommended directory structure\n\n* `backend.ai-dev`\n  - `manager` (git clone from this repo)\n  - `agent` (git clone from [the agent repo](https://github.com/lablup/backend.ai-agent))\n  - `common` (git clone from [the common repo](https://github.com/lablup/backend.ai-common))\n\nInstall `backend.ai-common` as an editable package in the manager (and the agent) virtualenvs\nto keep the codebase up-to-date.\n\n```console\n$ cd manager\n$ pip install -U -e ../common -r requirements/dev.txt\n```\n\n#### Steps\n\nCopy (or symlink) the halfstack configs:\n```console\n$ cp config/halfstack.toml ./manager.toml\n$ cp config/halfstack.alembic.ini ./alembic.ini\n```\n\nSet up Redis:\n```console\n$ backend.ai mgr etcd put config/redis/addr 127.0.0.1:8110\n```\n\n> \u2139\ufe0f NOTE: You may replace `backend.ai mgr` with `python -m ai.backend.manager.cli` in case your `PATH` is unmodifiable.\n\nSet up the public Docker registry:\n```console\n$ backend.ai mgr etcd put config/docker/registry/index.docker.io \"https://registry-1.docker.io\"\n$ backend.ai mgr etcd put config/docker/registry/index.docker.io/username \"lablup\"\n$ backend.ai mgr image rescan index.docker.io\n```\n\nSet up the vfolder paths:\n```console\n$ mkdir -p \"$HOME/vfroot/local\"\n$ backend.ai mgr etcd put volumes/_mount \"$HOME/vfroot\"\n$ backend.ai mgr etcd put volumes/_default_host local\n```\n\nSet up the allowed types of vfolder. Allowed values are \"user\" or \"group\".\nIf none is specified, \"user\" type is set implicitly:\n```console\n$ backend.ai mgr etcd put volumes/_types/user \"\"   # enable user vfolder\n$ backend.ai mgr etcd put volumes/_types/group \"\"  # enable group vfolder\n```\n\nSet up the database:\n```console\n$ backend.ai mgr schema oneshot\n$ backend.ai mgr fixture populate sample-configs/example-keypairs.json\n$ backend.ai mgr fixture populate sample-configs/example-resource-presets.json\n```\n\nThen, run it (for debugging, append a `--debug` flag):\n\n```console\n$ backend.ai mgr start-server\n```\n\nTo run tests:\n\n```console\n$ python -m flake8 src tests\n$ python -m pytest -m 'not integration' tests\n```\n\nNow you are ready to install the agent.\nHead to [the README of Backend.AI Agent](https://github.com/lablup/backend.ai-agent/blob/master/README.md).\n\nNOTE: To run tests including integration tests, you first need to install and run the agent on the same host.\n\n## Deployment\n\n### Configuration\n\nPut a TOML-formatted manager configuration (see the sample in `config/sample.toml`)\nin one of the following locations:\n\n * `manager.toml` (current working directory)\n * `~/.config/backend.ai/manager.toml` (user-config directory)\n * `/etc/backend.ai/manager.toml` (system-config directory)\n\nOnly the first found one is used by the daemon.\n\nAlso many configurations shared by both manager and agent are stored in etcd.\nAs you might have noticed above, the manager provides a CLI interface to access and manipulate the etcd\ndata.  Check out the help page of our etcd command set:\n\n```console\n$ python -m ai.backend.manager.cli etcd --help\n```\n\nIf you run etcd as a Docker container (e.g., via halfstack), you may use the native client as well.\nIn this case, PLEASE BE WARNED that you must prefix the keys with \"/sorna/{namespace}\" manaully:\n\n```console\n$ docker exec -it ${ETCD_CONTAINER_ID} /bin/ash -c 'ETCDCTL_API=3 etcdctl ...'\n```\n\n### Running from a command line\n\nThe minimal command to execute:\n\n```sh\npython -m ai.backend.gateway.server\n```\n\nFor more arguments and options, run the command with `--help` option.\n\n### Writing a wrapper script\n\nTo use with systemd, crontab, and other system-level daemons, you may need to write a shell script\nthat executes specific CLI commands provided by Backend.AI modules.\n\nThe following example shows how to set up pyenv and virtualenv for the script-local environment.\nIt runs the gateway server if no arguments are given, and execute the given arguments as a shell command\nif any.\nFor instance, you may get/set configurations like: `run-manager.sh python -m ai.backend.manager.etcd ...`\nwhere the name of scripts is `run-manager.sh`.\n\n```bash\n#! /bin/bash\nif [ -z \"$HOME\" ]; then\n  export HOME=\"/home/devops\"\nfi\nif [ -z \"$PYENV_ROOT\" ]; then\n  export PYENV_ROOT=\"$HOME/.pyenv\"\n  export PATH=\"$PYENV_ROOT/bin:$PATH\"\nfi\neval \"$(pyenv init -)\"\neval \"$(pyenv virtualenv-init -)\"\npyenv activate venv-bai-manager\n\nif [ \"$#\" -eq 0 ]; then\n  exec python -m ai.backend.gateway.server\nelse\n  exec \"$@\"\nfi\n```\n\n### Networking\n\nThe manager and agent should run in the same local network or different\nnetworks reachable via VPNs, whereas the manager's API service must be exposed to\nthe public network or another private network that users have access to.\n\nThe manager requires access to the etcd, the PostgreSQL database, and the Redis server.\n\n| User-to-Manager TCP Ports | Usage |\n|:-------------------------:|-------|\n| manager:{80,443}          | Backend.AI API access |\n\n| Manager-to-X TCP Ports | Usage |\n|:----------------------:|-------|\n| etcd:2379              | etcd API access |\n| postgres:5432          | Database access |\n| redis:6379             | Redis API access |\n\nThe manager must also be able to access TCP ports 6001, 6009, and 30000 to 31000 of the agents in default\nconfigurations.  You can of course change those port numbers and ranges in the configuration.\n\n| Manager-to-Agent TCP Ports | Usage |\n|:--------------------------:|-------|\n| 6001                       | ZeroMQ-based RPC calls from managers to agents |\n| 6009                       | HTTP watcher API |\n| 30000-31000                | Port pool for in-container services |\n\n\nLICENSES\n--------\n\n[GNU Lesser General Public License](https://github.com/lablup/backend.ai-manager/blob/master/LICENSE)\n[Dependencies](https://github.com/lablup/backend.ai-manager/blob/master/DEPENDENCIES.md)\n",
    "bugtrack_url": null,
    "license": "LGPLv3",
    "summary": "Backend.AI Manager",
    "version": "24.3.2",
    "project_urls": {
        "Documentation": "https://docs.backend.ai/",
        "Homepage": "https://github.com/lablup/backend.ai",
        "Source": "https://github.com/lablup/backend.ai"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "bddfa367d0e44b151806199ee42aad05d804e95bc69ce781d5b7fc263ff1dd1e",
                "md5": "e1c3717dfcb2448272762905f9538a16",
                "sha256": "89929c8b65543b3c68bb7c678baef6cd348913b59bef9d05f6f71d0feb789588"
            },
            "downloads": -1,
            "filename": "backend.ai_manager-24.3.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "e1c3717dfcb2448272762905f9538a16",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<3.13,>=3.12",
            "size": 1787882,
            "upload_time": "2024-04-17T06:46:47",
            "upload_time_iso_8601": "2024-04-17T06:46:47.086015Z",
            "url": "https://files.pythonhosted.org/packages/bd/df/a367d0e44b151806199ee42aad05d804e95bc69ce781d5b7fc263ff1dd1e/backend.ai_manager-24.3.2-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "aa31369c5ff80bb6f74b0e778b097990a46784d053aa392786132d9a53e5fba4",
                "md5": "689b40975f906ff97bd9556fd3c79969",
                "sha256": "63c266ef291e71127bf4f766d0b48eea79b5486e584a6cb3464259e4538dbcf1"
            },
            "downloads": -1,
            "filename": "backend.ai-manager-24.3.2.tar.gz",
            "has_sig": false,
            "md5_digest": "689b40975f906ff97bd9556fd3c79969",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<3.13,>=3.12",
            "size": 1654002,
            "upload_time": "2024-04-17T06:47:19",
            "upload_time_iso_8601": "2024-04-17T06:47:19.857724Z",
            "url": "https://files.pythonhosted.org/packages/aa/31/369c5ff80bb6f74b0e778b097990a46784d053aa392786132d9a53e5fba4/backend.ai-manager-24.3.2.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-04-17 06:47:19",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "lablup",
    "github_project": "backend.ai",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "requirements": [
        {
            "name": "aiodataloader-ng",
            "specs": [
                [
                    "~=",
                    "0.2.1"
                ]
            ]
        },
        {
            "name": "aiodocker",
            "specs": [
                [
                    "~=",
                    "0.21.0"
                ]
            ]
        },
        {
            "name": "aiofiles",
            "specs": [
                [
                    "~=",
                    "23.2.1"
                ]
            ]
        },
        {
            "name": "aiohttp",
            "specs": [
                [
                    "~=",
                    "3.9.1"
                ]
            ]
        },
        {
            "name": "aiohttp_cors",
            "specs": [
                [
                    "~=",
                    "0.7"
                ]
            ]
        },
        {
            "name": "aiohttp_sse",
            "specs": [
                [
                    ">=",
                    "2.0"
                ]
            ]
        },
        {
            "name": "aiodns",
            "specs": [
                [
                    ">=",
                    "3.0"
                ]
            ]
        },
        {
            "name": "aiomonitor",
            "specs": [
                [
                    "~=",
                    "0.7.0"
                ]
            ]
        },
        {
            "name": "aioresponses",
            "specs": [
                [
                    ">=",
                    "0.7.3"
                ]
            ]
        },
        {
            "name": "aiosqlite",
            "specs": [
                [
                    "~=",
                    "0.20.0"
                ]
            ]
        },
        {
            "name": "aiotools",
            "specs": [
                [
                    "~=",
                    "1.7.0"
                ]
            ]
        },
        {
            "name": "aiotusclient",
            "specs": [
                [
                    "~=",
                    "0.1.4"
                ]
            ]
        },
        {
            "name": "alembic",
            "specs": [
                [
                    "~=",
                    "1.13.1"
                ]
            ]
        },
        {
            "name": "appdirs",
            "specs": [
                [
                    "~=",
                    "1.4.4"
                ]
            ]
        },
        {
            "name": "async_timeout",
            "specs": [
                [
                    "~=",
                    "4.0"
                ]
            ]
        },
        {
            "name": "asyncpg",
            "specs": [
                [
                    ">=",
                    "0.27.0"
                ]
            ]
        },
        {
            "name": "asynctest",
            "specs": [
                [
                    ">=",
                    "0.13.0"
                ]
            ]
        },
        {
            "name": "asyncudp",
            "specs": [
                [
                    ">=",
                    "0.4"
                ]
            ]
        },
        {
            "name": "attrs",
            "specs": [
                [
                    ">=",
                    "20.3"
                ]
            ]
        },
        {
            "name": "bcrypt",
            "specs": [
                [
                    ">=",
                    "4.1.2"
                ]
            ]
        },
        {
            "name": "boto3",
            "specs": [
                [
                    "~=",
                    "1.26"
                ]
            ]
        },
        {
            "name": "cachetools",
            "specs": [
                [
                    "~=",
                    "5.3.3"
                ]
            ]
        },
        {
            "name": "callosum",
            "specs": [
                [
                    "~=",
                    "1.0.3"
                ]
            ]
        },
        {
            "name": "cattrs",
            "specs": [
                [
                    "~=",
                    "23.2.3"
                ]
            ]
        },
        {
            "name": "click",
            "specs": [
                [
                    "~=",
                    "8.1.7"
                ]
            ]
        },
        {
            "name": "coloredlogs",
            "specs": [
                [
                    "~=",
                    "15.0"
                ]
            ]
        },
        {
            "name": "colorama",
            "specs": [
                [
                    ">=",
                    "0.4.4"
                ]
            ]
        },
        {
            "name": "cryptography",
            "specs": [
                [
                    ">=",
                    "2.8"
                ]
            ]
        },
        {
            "name": "dataclasses-json",
            "specs": [
                [
                    "~=",
                    "0.5.7"
                ]
            ]
        },
        {
            "name": "faker",
            "specs": [
                [
                    "~=",
                    "24.7.1"
                ]
            ]
        },
        {
            "name": "graphene",
            "specs": [
                [
                    "~=",
                    "3.3.0"
                ]
            ]
        },
        {
            "name": "graypy",
            "specs": [
                [
                    "==",
                    "2.1.0"
                ]
            ]
        },
        {
            "name": "humanize",
            "specs": [
                [
                    ">=",
                    "3.1.0"
                ]
            ]
        },
        {
            "name": "ifaddr",
            "specs": [
                [
                    "~=",
                    "0.2"
                ]
            ]
        },
        {
            "name": "inquirer",
            "specs": [
                [
                    "~=",
                    "2.9.2"
                ]
            ]
        },
        {
            "name": "janus",
            "specs": [
                [
                    "~=",
                    "1.0.0"
                ]
            ]
        },
        {
            "name": "Jinja2",
            "specs": [
                [
                    "~=",
                    "3.1.2"
                ]
            ]
        },
        {
            "name": "jupyter-client",
            "specs": [
                [
                    ">=",
                    "6.0"
                ]
            ]
        },
        {
            "name": "kubernetes",
            "specs": [
                [
                    "~=",
                    "10.0.0"
                ]
            ]
        },
        {
            "name": "kubernetes-asyncio",
            "specs": [
                [
                    "~=",
                    "9.1.0"
                ]
            ]
        },
        {
            "name": "lark",
            "specs": [
                [
                    "~=",
                    "1.1.5"
                ]
            ]
        },
        {
            "name": "more-itertools",
            "specs": [
                [
                    "~=",
                    "8.13.0"
                ]
            ]
        },
        {
            "name": "msgpack",
            "specs": [
                [
                    ">=",
                    "1.0.5rc1"
                ]
            ]
        },
        {
            "name": "multidict",
            "specs": [
                [
                    ">=",
                    "6.0"
                ]
            ]
        },
        {
            "name": "namedlist",
            "specs": [
                [
                    "~=",
                    "1.8"
                ]
            ]
        },
        {
            "name": "networkx",
            "specs": [
                [
                    "~=",
                    "3.3.0"
                ]
            ]
        },
        {
            "name": "pexpect",
            "specs": [
                [
                    "~=",
                    "4.8"
                ]
            ]
        },
        {
            "name": "psutil",
            "specs": [
                [
                    "~=",
                    "5.9.1"
                ]
            ]
        },
        {
            "name": "pycryptodome",
            "specs": [
                [
                    ">=",
                    "3.14.1"
                ]
            ]
        },
        {
            "name": "python-dateutil",
            "specs": [
                [
                    ">=",
                    "2.8"
                ]
            ]
        },
        {
            "name": "python-dotenv",
            "specs": [
                [
                    "~=",
                    "0.20.0"
                ]
            ]
        },
        {
            "name": "python-json-logger",
            "specs": [
                [
                    ">=",
                    "2.0.1"
                ]
            ]
        },
        {
            "name": "pyzmq",
            "specs": [
                [
                    "~=",
                    "25.1.2"
                ]
            ]
        },
        {
            "name": "PyJWT",
            "specs": [
                [
                    "~=",
                    "2.0"
                ]
            ]
        },
        {
            "name": "PyYAML",
            "specs": [
                [
                    "~=",
                    "6.0"
                ]
            ]
        },
        {
            "name": "pydantic",
            "specs": [
                [
                    "~=",
                    "2.6.4"
                ]
            ]
        },
        {
            "name": "packaging",
            "specs": [
                [
                    ">=",
                    "21.3"
                ]
            ]
        },
        {
            "name": "hiredis",
            "specs": [
                [
                    ">=",
                    "2.2.3"
                ]
            ]
        },
        {
            "name": "redis",
            "specs": [
                [
                    "==",
                    "4.5.5"
                ]
            ]
        },
        {
            "name": "rich",
            "specs": [
                [
                    "~=",
                    "13.6"
                ]
            ]
        },
        {
            "name": "SQLAlchemy",
            "specs": [
                [
                    "~=",
                    "1.4.40"
                ]
            ]
        },
        {
            "name": "setproctitle",
            "specs": [
                [
                    "~=",
                    "1.3.2"
                ]
            ]
        },
        {
            "name": "tabulate",
            "specs": [
                [
                    "~=",
                    "0.8.9"
                ]
            ]
        },
        {
            "name": "temporenc",
            "specs": [
                [
                    "~=",
                    "0.1.0"
                ]
            ]
        },
        {
            "name": "tenacity",
            "specs": [
                [
                    ">=",
                    "8.0"
                ]
            ]
        },
        {
            "name": "tomli",
            "specs": [
                [
                    "~=",
                    "2.0.1"
                ]
            ]
        },
        {
            "name": "tomlkit",
            "specs": [
                [
                    "~=",
                    "0.12.4"
                ]
            ]
        },
        {
            "name": "tqdm",
            "specs": [
                [
                    ">=",
                    "4.61"
                ]
            ]
        },
        {
            "name": "trafaret",
            "specs": [
                [
                    "~=",
                    "2.1"
                ]
            ]
        },
        {
            "name": "treelib",
            "specs": [
                [
                    "~=",
                    "1.7.0"
                ]
            ]
        },
        {
            "name": "typeguard",
            "specs": [
                [
                    "~=",
                    "2.10"
                ]
            ]
        },
        {
            "name": "typing_extensions",
            "specs": [
                [
                    "~=",
                    "4.11"
                ]
            ]
        },
        {
            "name": "textual",
            "specs": [
                [
                    "~=",
                    "0.56.3"
                ]
            ]
        },
        {
            "name": "uvloop",
            "specs": [
                [
                    "~=",
                    "0.19.0"
                ]
            ]
        },
        {
            "name": "yarl",
            "specs": [
                [
                    "!=",
                    "1.9.1"
                ],
                [
                    "<",
                    "2.0"
                ],
                [
                    "!=",
                    "1.9.2"
                ],
                [
                    "!=",
                    "1.9.0"
                ],
                [
                    ">=",
                    "1.8.2"
                ]
            ]
        },
        {
            "name": "zipstream-new",
            "specs": [
                [
                    "~=",
                    "1.1.8"
                ]
            ]
        },
        {
            "name": "pytest",
            "specs": [
                [
                    ">=",
                    "7.3.1"
                ]
            ]
        },
        {
            "name": "pytest-dependency",
            "specs": [
                [
                    ">=",
                    "0.5.1"
                ]
            ]
        },
        {
            "name": "types-six",
            "specs": []
        },
        {
            "name": "types-setuptools",
            "specs": []
        },
        {
            "name": "types-python-dateutil",
            "specs": []
        },
        {
            "name": "types-aiofiles",
            "specs": []
        },
        {
            "name": "types-cachetools",
            "specs": []
        },
        {
            "name": "types-Jinja2",
            "specs": []
        },
        {
            "name": "types-PyYAML",
            "specs": []
        },
        {
            "name": "types-redis",
            "specs": []
        },
        {
            "name": "types-tabulate",
            "specs": []
        },
        {
            "name": "backend.ai-krunner-alpine",
            "specs": [
                [
                    "==",
                    "5.2.0"
                ]
            ]
        },
        {
            "name": "backend.ai-krunner-static-gnu",
            "specs": [
                [
                    "==",
                    "4.2.0"
                ]
            ]
        },
        {
            "name": "etcd-client-py",
            "specs": [
                [
                    "==",
                    "0.3.0"
                ]
            ]
        }
    ],
    "lcname": "backend.ai-manager"
}
        
Elapsed time: 0.27953s