flux-metrics-api


Nameflux-metrics-api JSON
Version 0.0.11 PyPI version JSON
download
home_pagehttps://github.com/converged-computing/flux-metrics-api
SummaryCustom metrics exporter for Flux in Kubernetes
upload_time2023-06-01 02:32:50
maintainerVanessa Sochat
docs_urlNone
authorVanessa Sochat
requires_python
licenseLICENSE
keywords cloud flux flux-framework monitoring
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # Flux Metrics API

<!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
[![All Contributors](https://img.shields.io/badge/all_contributors-1-orange.svg?style=flat-square)](#contributors-)
<!-- ALL-CONTRIBUTORS-BADGE:END -->
[![PyPI](https://img.shields.io/pypi/v/flux-metrics-api)](https://pypi.org/project/flux-metrics-api/)

This is an experiment to create a metrics API for Kubernetes that can be run directly from the Flux
leader broker pod. We made this after creating [prometheus-flux](https://github.com/converged-computing/prometheus-flux)
and wanting a more minimalist design. I'm not even sure it will work, but it's worth a try!

## Usage

### Install

You can install from pypi or from source:

```bash
$ python -m venv env
$ source env/bin/activate
$ pip install flux-metrics-api

# or

$ git clone https://github.com/converged-computing/flux-metrics-api
$ cd flux-metrics-api
$ pip install .
# you can also do "pip install -e ."
```

This will install the executable to your path, which might be your local user bin:

```bash
$ which flux-metric-api
/home/vscode/.local/bin/flux-metrics-api
```

Note that the provided [.devcontainer](.devcontainer) includes an environment for VSCode where you have Flux
and can install this and use ready to go!

### Start

You'll want to be running in a Flux instance, as we need to connect to the broker handle.

```bash
$ flux start --test-size=4
```

And then start the server. This will use a default port and host (0.0.0.0:8443) that you can customize
if desired.

```bash
$ flux-metrics-api start

# customize the port or host
$ flux-metrics-api start --port 9000 --host 127.0.0.1
```

#### SSL

If you want ssl (port 443) you can provide the path to a certificate and keyfile:

```bash
$ flux-metrics-api start --ssl-certfile /etc/certs/tls.crt --ssl-keyfile /etc/certs/tls.key
```

An example of a full command we might run from within a pod:

```bash
$ flux-metrics-api start --port 8443 --ssl-certfile /etc/certs/tls.crt --ssl-keyfile /etc/certs/tls.key --namespace flux-operator --service-name custom-metrics-apiserver
```

#### On the fly custom metrics!

If you want to provide custom metrics, you can write a function in an external file that we will read it and add to the server.
As a general rule:

 - The name of the function will be the name of the custom metric
 - You can expect the only argument to be the flux handle
 - You'll need to do imports within your function to get them in scope

This likely can be improved upon, but is a start for now! We provide an [example file](example/custom-metrics.py). As an example:

```bash
$ flux-metrics-api start --custom-metric ./example/custom-metrics.py
```

And then test it:

```bash
$ curl -s http://localhost:8443/apis/custom.metrics.k8s.io/v1beta2/namespaces/flux-operator/metrics/my_custom_metric_name | jq
```
```console
{
  "items": [
    {
      "metric": {
        "name": "my_custom_metric_name"
      },
      "value": 4,
      "timestamp": "2023-06-01T01:39:08+00:00",
      "windowSeconds": 0,
      "describedObject": {
        "kind": "Service",
        "namespace": "flux-operator",
        "name": "custom-metrics-apiserver",
        "apiVersion": "v1beta2"
      }
    }
  ],
  "apiVersion": "custom.metrics.k8s.io/v1beta2",
  "kind": "MetricValueList",
  "metadata": {
    "selfLink": "/apis/custom.metrics.k8s.io/v1beta2"
  }
}
```

See `--help` to see other options available.

### Endpoints

#### Metric

**GET /apis/custom.metrics.k8s.io/v1beta2/namespaces/<namespace>/metrics/<metric_name>**

Here is an example to get the "node_up_count" metric:

```bash
 curl -s http://localhost:8443/apis/custom.metrics.k8s.io/v1beta2/namespaces/flux-operator/metrics/node_up_count | jq
```
```console
{
  "items": [
    {
      "metric": {
        "name": "node_up_count"
      },
      "value": 2,
      "timestamp": "2023-05-31T04:44:57+00:00",
      "windowSeconds": 0,
      "describedObject": {
        "kind": "Service",
        "namespace": "flux-operator",
        "name": "custom-metrics-apiserver",
        "apiVersion": "v1beta2"
      }
    }
  ],
  "apiVersion": "custom.metrics.k8s.io/v1beta2",
  "kind": "MetricValueList",
  "metadata": {
    "selfLink": "/apis/custom.metrics.k8s.io/v1beta2"
  }
}
```

The following metrics are supported:

 - **node_up_count**: number of nodes up in the MiniCluster
 - **node_free_count**: number of nodes free in the MiniCluster
 - **node_cores_free_count**: number of node cores free in the MiniCluster
 - **node_cores_up_count**: number of node cores up in the MiniCluster
 - **job_queue_state_new_count**: number of new jobs in the queue
 - **job_queue_state_depend_count**: number of jobs in the queue in state "depend"
 - **job_queue_state_priority_count**: number of jobs in the queue in state "priority"
 - **job_queue_state_sched_count**: number of jobs in the queue in state "sched"
 - **job_queue_state_run_count**: number of jobs in the queue in state "run"
 - **job_queue_state_cleanup_count**: number of jobs in the queue in state "cleanup"
 - **job_queue_state_inactive_count**: number of jobs in the queue in state "inactive"


### Docker

We have a docker container, which you can customize for your use case, but it's more intended to
be a demo. You can either build it yourself, or use our build.

```bash
$ docker build -t flux_metrics_api .
$ docker run -it -p 8443:8443 flux_metrics_api
```
or

```bash
$ docker run -it -p 8443:8443 ghcr.io/converged-computing/flux-metrics-api
```

### Development

Note that this is implemented in Python, but (I found this after) we could [also use Go](https://github.com/kubernetes-sigs/custom-metrics-apiserver).
Specifically, I found this repository useful to see the [spec format](https://github.com/kubernetes-sigs/custom-metrics-apiserver/blob/master/pkg/generated/openapi/custommetrics/zz_generated.openapi.go).

You can then open up the browser at [http://localhost:8443/metrics/](http://localhost:8443/metrics) to see
the metrics!

## 😁️ Contributors 😁️

We use the [all-contributors](https://github.com/all-contributors/all-contributors)
tool to generate a contributors graphic below.

<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->
<table>
  <tbody>
    <tr>
      <td align="center" valign="top" width="14.28%"><a href="https://vsoch.github.io"><img src="https://avatars.githubusercontent.com/u/814322?v=4?s=100" width="100px;" alt="Vanessasaurus"/><br /><sub><b>Vanessasaurus</b></sub></a><br /><a href="https://github.com/converged-computing/flux-metrics-api/commits?author=vsoch" title="Code">💻</a></td>
    </tr>
  </tbody>
</table>

<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->

<!-- ALL-CONTRIBUTORS-LIST:END -->

## License

HPCIC DevTools is distributed under the terms of the MIT license.
All new contributions must be made under this license.

See [LICENSE](https://github.com/converged-computing/flux-metrics-api/blob/main/LICENSE),
[COPYRIGHT](https://github.com/converged-computing/flux-metrics-api/blob/main/COPYRIGHT), and
[NOTICE](https://github.com/converged-computing/flux-metrics-api/blob/main/NOTICE) for details.

SPDX-License-Identifier: (MIT)

LLNL-CODE- 842614

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/converged-computing/flux-metrics-api",
    "name": "flux-metrics-api",
    "maintainer": "Vanessa Sochat",
    "docs_url": null,
    "requires_python": "",
    "maintainer_email": "",
    "keywords": "cloud,flux,flux-framework,monitoring",
    "author": "Vanessa Sochat",
    "author_email": "vsoch@users.noreply.github.com",
    "download_url": "https://files.pythonhosted.org/packages/a4/52/ba96c606284868923c1c67430a7dc2f43741ce956e82912021c62eec9447/flux-metrics-api-0.0.11.tar.gz",
    "platform": null,
    "description": "# Flux Metrics API\n\n<!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->\n[![All Contributors](https://img.shields.io/badge/all_contributors-1-orange.svg?style=flat-square)](#contributors-)\n<!-- ALL-CONTRIBUTORS-BADGE:END -->\n[![PyPI](https://img.shields.io/pypi/v/flux-metrics-api)](https://pypi.org/project/flux-metrics-api/)\n\nThis is an experiment to create a metrics API for Kubernetes that can be run directly from the Flux\nleader broker pod. We made this after creating [prometheus-flux](https://github.com/converged-computing/prometheus-flux)\nand wanting a more minimalist design. I'm not even sure it will work, but it's worth a try!\n\n## Usage\n\n### Install\n\nYou can install from pypi or from source:\n\n```bash\n$ python -m venv env\n$ source env/bin/activate\n$ pip install flux-metrics-api\n\n# or\n\n$ git clone https://github.com/converged-computing/flux-metrics-api\n$ cd flux-metrics-api\n$ pip install .\n# you can also do \"pip install -e .\"\n```\n\nThis will install the executable to your path, which might be your local user bin:\n\n```bash\n$ which flux-metric-api\n/home/vscode/.local/bin/flux-metrics-api\n```\n\nNote that the provided [.devcontainer](.devcontainer) includes an environment for VSCode where you have Flux\nand can install this and use ready to go!\n\n### Start\n\nYou'll want to be running in a Flux instance, as we need to connect to the broker handle.\n\n```bash\n$ flux start --test-size=4\n```\n\nAnd then start the server. This will use a default port and host (0.0.0.0:8443) that you can customize\nif desired.\n\n```bash\n$ flux-metrics-api start\n\n# customize the port or host\n$ flux-metrics-api start --port 9000 --host 127.0.0.1\n```\n\n#### SSL\n\nIf you want ssl (port 443) you can provide the path to a certificate and keyfile:\n\n```bash\n$ flux-metrics-api start --ssl-certfile /etc/certs/tls.crt --ssl-keyfile /etc/certs/tls.key\n```\n\nAn example of a full command we might run from within a pod:\n\n```bash\n$ flux-metrics-api start --port 8443 --ssl-certfile /etc/certs/tls.crt --ssl-keyfile /etc/certs/tls.key --namespace flux-operator --service-name custom-metrics-apiserver\n```\n\n#### On the fly custom metrics!\n\nIf you want to provide custom metrics, you can write a function in an external file that we will read it and add to the server.\nAs a general rule:\n\n - The name of the function will be the name of the custom metric\n - You can expect the only argument to be the flux handle\n - You'll need to do imports within your function to get them in scope\n\nThis likely can be improved upon, but is a start for now! We provide an [example file](example/custom-metrics.py). As an example:\n\n```bash\n$ flux-metrics-api start --custom-metric ./example/custom-metrics.py\n```\n\nAnd then test it:\n\n```bash\n$ curl -s http://localhost:8443/apis/custom.metrics.k8s.io/v1beta2/namespaces/flux-operator/metrics/my_custom_metric_name | jq\n```\n```console\n{\n  \"items\": [\n    {\n      \"metric\": {\n        \"name\": \"my_custom_metric_name\"\n      },\n      \"value\": 4,\n      \"timestamp\": \"2023-06-01T01:39:08+00:00\",\n      \"windowSeconds\": 0,\n      \"describedObject\": {\n        \"kind\": \"Service\",\n        \"namespace\": \"flux-operator\",\n        \"name\": \"custom-metrics-apiserver\",\n        \"apiVersion\": \"v1beta2\"\n      }\n    }\n  ],\n  \"apiVersion\": \"custom.metrics.k8s.io/v1beta2\",\n  \"kind\": \"MetricValueList\",\n  \"metadata\": {\n    \"selfLink\": \"/apis/custom.metrics.k8s.io/v1beta2\"\n  }\n}\n```\n\nSee `--help` to see other options available.\n\n### Endpoints\n\n#### Metric\n\n**GET /apis/custom.metrics.k8s.io/v1beta2/namespaces/<namespace>/metrics/<metric_name>**\n\nHere is an example to get the \"node_up_count\" metric:\n\n```bash\n curl -s http://localhost:8443/apis/custom.metrics.k8s.io/v1beta2/namespaces/flux-operator/metrics/node_up_count | jq\n```\n```console\n{\n  \"items\": [\n    {\n      \"metric\": {\n        \"name\": \"node_up_count\"\n      },\n      \"value\": 2,\n      \"timestamp\": \"2023-05-31T04:44:57+00:00\",\n      \"windowSeconds\": 0,\n      \"describedObject\": {\n        \"kind\": \"Service\",\n        \"namespace\": \"flux-operator\",\n        \"name\": \"custom-metrics-apiserver\",\n        \"apiVersion\": \"v1beta2\"\n      }\n    }\n  ],\n  \"apiVersion\": \"custom.metrics.k8s.io/v1beta2\",\n  \"kind\": \"MetricValueList\",\n  \"metadata\": {\n    \"selfLink\": \"/apis/custom.metrics.k8s.io/v1beta2\"\n  }\n}\n```\n\nThe following metrics are supported:\n\n - **node_up_count**: number of nodes up in the MiniCluster\n - **node_free_count**: number of nodes free in the MiniCluster\n - **node_cores_free_count**: number of node cores free in the MiniCluster\n - **node_cores_up_count**: number of node cores up in the MiniCluster\n - **job_queue_state_new_count**: number of new jobs in the queue\n - **job_queue_state_depend_count**: number of jobs in the queue in state \"depend\"\n - **job_queue_state_priority_count**: number of jobs in the queue in state \"priority\"\n - **job_queue_state_sched_count**: number of jobs in the queue in state \"sched\"\n - **job_queue_state_run_count**: number of jobs in the queue in state \"run\"\n - **job_queue_state_cleanup_count**: number of jobs in the queue in state \"cleanup\"\n - **job_queue_state_inactive_count**: number of jobs in the queue in state \"inactive\"\n\n\n### Docker\n\nWe have a docker container, which you can customize for your use case, but it's more intended to\nbe a demo. You can either build it yourself, or use our build.\n\n```bash\n$ docker build -t flux_metrics_api .\n$ docker run -it -p 8443:8443 flux_metrics_api\n```\nor\n\n```bash\n$ docker run -it -p 8443:8443 ghcr.io/converged-computing/flux-metrics-api\n```\n\n### Development\n\nNote that this is implemented in Python, but (I found this after) we could [also use Go](https://github.com/kubernetes-sigs/custom-metrics-apiserver).\nSpecifically, I found this repository useful to see the [spec format](https://github.com/kubernetes-sigs/custom-metrics-apiserver/blob/master/pkg/generated/openapi/custommetrics/zz_generated.openapi.go).\n\nYou can then open up the browser at [http://localhost:8443/metrics/](http://localhost:8443/metrics) to see\nthe metrics!\n\n## \ud83d\ude01\ufe0f Contributors \ud83d\ude01\ufe0f\n\nWe use the [all-contributors](https://github.com/all-contributors/all-contributors)\ntool to generate a contributors graphic below.\n\n<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->\n<!-- prettier-ignore-start -->\n<!-- markdownlint-disable -->\n<table>\n  <tbody>\n    <tr>\n      <td align=\"center\" valign=\"top\" width=\"14.28%\"><a href=\"https://vsoch.github.io\"><img src=\"https://avatars.githubusercontent.com/u/814322?v=4?s=100\" width=\"100px;\" alt=\"Vanessasaurus\"/><br /><sub><b>Vanessasaurus</b></sub></a><br /><a href=\"https://github.com/converged-computing/flux-metrics-api/commits?author=vsoch\" title=\"Code\">\ud83d\udcbb</a></td>\n    </tr>\n  </tbody>\n</table>\n\n<!-- markdownlint-restore -->\n<!-- prettier-ignore-end -->\n\n<!-- ALL-CONTRIBUTORS-LIST:END -->\n\n## License\n\nHPCIC DevTools is distributed under the terms of the MIT license.\nAll new contributions must be made under this license.\n\nSee [LICENSE](https://github.com/converged-computing/flux-metrics-api/blob/main/LICENSE),\n[COPYRIGHT](https://github.com/converged-computing/flux-metrics-api/blob/main/COPYRIGHT), and\n[NOTICE](https://github.com/converged-computing/flux-metrics-api/blob/main/NOTICE) for details.\n\nSPDX-License-Identifier: (MIT)\n\nLLNL-CODE- 842614\n",
    "bugtrack_url": null,
    "license": "LICENSE",
    "summary": "Custom metrics exporter for Flux in Kubernetes",
    "version": "0.0.11",
    "project_urls": {
        "Homepage": "https://github.com/converged-computing/flux-metrics-api"
    },
    "split_keywords": [
        "cloud",
        "flux",
        "flux-framework",
        "monitoring"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "020f1478b1fec6ff966fd2d54a01703a4e3e069c6d16914fbc74dddc2103255d",
                "md5": "74f8c5709c6a8652c97d7ff8aacef67e",
                "sha256": "051705583bb34a89cd311b5cec02129111d9247fef3f375553bb9ef1c6b6e1fc"
            },
            "downloads": -1,
            "filename": "flux_metrics_api-0.0.11-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "74f8c5709c6a8652c97d7ff8aacef67e",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 17872,
            "upload_time": "2023-06-01T02:32:49",
            "upload_time_iso_8601": "2023-06-01T02:32:49.150885Z",
            "url": "https://files.pythonhosted.org/packages/02/0f/1478b1fec6ff966fd2d54a01703a4e3e069c6d16914fbc74dddc2103255d/flux_metrics_api-0.0.11-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "a452ba96c606284868923c1c67430a7dc2f43741ce956e82912021c62eec9447",
                "md5": "9ec52986ad89e4dafd20e24f1e710919",
                "sha256": "28f59a61772fea164bd660b1da51470bbf1f9a5d8ef0ef3fd01cf0939947f827"
            },
            "downloads": -1,
            "filename": "flux-metrics-api-0.0.11.tar.gz",
            "has_sig": false,
            "md5_digest": "9ec52986ad89e4dafd20e24f1e710919",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 17476,
            "upload_time": "2023-06-01T02:32:50",
            "upload_time_iso_8601": "2023-06-01T02:32:50.525573Z",
            "url": "https://files.pythonhosted.org/packages/a4/52/ba96c606284868923c1c67430a7dc2f43741ce956e82912021c62eec9447/flux-metrics-api-0.0.11.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-06-01 02:32:50",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "converged-computing",
    "github_project": "flux-metrics-api",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "flux-metrics-api"
}
        
Elapsed time: 0.07718s