# kafkatop
**A real-time monitoring tool for Apache Kafka that displays consumer lag and event rates in a clean, top-like interface.**
`kafkatop` provides a simple, yet powerful, way to quickly view the health of your Kafka consumers and topics. It helps you identify bottlenecks and diagnose issues with consumer lag in real-time, directly from your terminal.
*Anonymized topics and groups*
## Features
- **Real-time Monitoring:** Keep an eye on event consumption and production rates across your Kafka broker.
- **Consumer Lag & ETA:** Instantly see the lag for each consumer group and topic, with an estimated time to catch up.
- **Flexible Filtering:** Easily filter by topic or consumer group name, and focus on problematic groups with a dedicated flag.
- **JSON Output:** Export the current status as a JSON object, for integrating with batch-collecting monitoring and alerting systems.
- **Anonymization Mode:** Anonymize topic and group names when sharing your screen or logs.
- **Metadata Reporting:** Export an overview of consumer and topic metadata (topics, partitions, ISR, leaders) in JSON.
# Installing
## PEX binary
Download the single-file pex executable from the [releases](https://github.com/sivann/kafkatop/releases) page. It's compatible with X86 64-bit systems and requires Python 3.9–3.13 in your path.
## Using pip
**Install with pip:**
```
pip install kafkatop
```
**Install inside a virtualenv (recommended practice):**
```
python3 -m venv virtual_env # this will create a 'virtual_env' directory
. bin/virtual_env/activate # 'activate' the virtualenv. Run 'deactivate' to deactivate.
pip install kafkatop # install kafkatop
kafkatop # run
```
## Quick Start
To get started, simply run `kafkatop` specifying the address of a Kafka broker if needed:
```
kafkatop
```
### Common Use Cases
**Monitor a specific consumer group:**
```
kafkatop --kafka-broker 10.227.1.110 --group-filter-pattern '.*group_name.*'
```
**Monitor kafka running in kubernetes, from the kubernetes node**
```
# Port-forward the pod's port locally in another terminal, or in the background:
timeout kubectl port-forward kafka-0 9092:9092 &
# Run kafkatop on the node
kafkatop
```
# Usage
```
usage: kafkatop.py [-h] [--kafka-broker KAFKA_BROKER] [--text] [--poll-period KAFKA_POLL_PERIOD] [--poll-iterations KAFKA_POLL_ITERATIONS] [--group-exclude-pattern KAFKA_GROUP_EXCLUDE_PATTERN]
[--group-filter-pattern KAFKA_GROUP_FILTER_PATTERN] [--status] [--summary] [--summary-json] [--topicinfo] [--topicinfo-parts] [--only-issues] [--anonymize] [--all]
[--version]
Kafka consumer statistics
options:
-h, --help show this help message and exit
--kafka-broker KAFKA_BROKER
Broker IP (default: localhost)
--text Only plain text, no rich output. (default: False)
--poll-period KAFKA_POLL_PERIOD
Kafka offset poll period (seconds) for evts/sec calculation (default: 5)
--poll-iterations KAFKA_POLL_ITERATIONS
How many times to query and display stats. -1 = Inf (default: 15)
--group-exclude-pattern KAFKA_GROUP_EXCLUDE_PATTERN
If group matches regex, exclude (default: None)
--group-filter-pattern KAFKA_GROUP_FILTER_PATTERN
Include *only* the groups which match regex (default: None)
--status Report health status in json and exit. (default: False)
--summary Display consumer groups, states, topics, partitions, and lags summary. (default: False)
--summary-json Display consumer groups, states, topics, partitions, and lags summary, in json. (default: False)
--topicinfo Only show informational data about the cluster, topics, partitions, no stats (fast). (default: False)
--topicinfo-parts Same as --info but also show data about partitions, isr, leaders. (default: False)
--only-issues Only show rows with issues. (default: False)
--anonymize Anonymize topics and groups. (default: False)
--all Show groups with no members. (default: False)
--version show program's version number and exit
```
# Examples
## Screenshots
|  |
|:--:|
| *In the screenshot above (anonymized topics/groups), the 1st row is highlighted as the ETA to consume all the lag, with the consuming rate of the last period, is > 1 minute. Red rows indicate consumption rate is lower than it should be. Cell colors indicate severity.* |
|  |
|:--:|
| *A healthy system.* |
|  |
|:--:|
| *A summary of (anonymized) groups, topics, partitions, lag and consumer group states. Lag is the median lag of all topic partitions.* |
## Text output
```
kafkatop --kafka-broker 1.2.3.4 --group-filter-pattern 'GroupName1' --summary-json
```
results in:
```
{
"ConsumerGroupName1": {
"TopicName": {
"partitions": 1672,
"state": "ConsumerGroupState.STABLE",
"lag_max": 86169242,
"lag_min": 0
}
},
[...]
}
```
# Contributing
## Building
Requires python >=3.9 in your path
1. set the full path of PYTHON at the top of Makefile or add the PYTHON= parameter when calling make
2. ```make```
3. ```make pex```
```
make pex
```
This will create a ```kafkatop``` pex executable which will include the python code and library dependencies all in one file. It will need the python3 in the path to run.
### Multiplatform building
If you have multiple python versions on your environment, you can run ```make-pex-mp.sh``` to create a kafkatop binary which includes dependencies for all python versions.
# License
This project is licensed under the terms of the MIT license.
Raw data
{
"_id": null,
"home_page": null,
"name": "kafkatop",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "kafka, monitoring, consumer, lag, real-time",
"author": "Spiros Ioannou",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/a2/8d/9cbae2eb43a5464ed53aabb4fedae0f183e411870c562b9b6cfc911adb53/kafkatop-1.16.tar.gz",
"platform": null,
"description": "# kafkatop\n\n**A real-time monitoring tool for Apache Kafka that displays consumer lag and event rates in a clean, top-like interface.**\n\n`kafkatop` provides a simple, yet powerful, way to quickly view the health of your Kafka consumers and topics. It helps you identify bottlenecks and diagnose issues with consumer lag in real-time, directly from your terminal.\n\n\n*Anonymized topics and groups*\n\n\n\n## Features\n\n- **Real-time Monitoring:** Keep an eye on event consumption and production rates across your Kafka broker.\n- **Consumer Lag & ETA:** Instantly see the lag for each consumer group and topic, with an estimated time to catch up.\n- **Flexible Filtering:** Easily filter by topic or consumer group name, and focus on problematic groups with a dedicated flag.\n- **JSON Output:** Export the current status as a JSON object, for integrating with batch-collecting monitoring and alerting systems.\n- **Anonymization Mode:** Anonymize topic and group names when sharing your screen or logs.\n- **Metadata Reporting:** Export an overview of consumer and topic metadata (topics, partitions, ISR, leaders) in JSON.\n\n# Installing\n## PEX binary\nDownload the single-file pex executable from the [releases](https://github.com/sivann/kafkatop/releases) page. It's compatible with X86 64-bit systems and requires Python 3.9\u20133.13 in your path.\n\n## Using pip\n\n**Install with pip:**\n\n```\npip install kafkatop\n```\n\n\n**Install inside a virtualenv (recommended practice):**\n\n```\npython3 -m venv virtual_env # this will create a 'virtual_env' directory\n. bin/virtual_env/activate # 'activate' the virtualenv. Run 'deactivate' to deactivate.\npip install kafkatop # install kafkatop\nkafkatop # run \n```\n\n## Quick Start\n\nTo get started, simply run `kafkatop` specifying the address of a Kafka broker if needed:\n\n```\nkafkatop\n```\n\n### Common Use Cases\n\n**Monitor a specific consumer group:**\n\n```\n kafkatop --kafka-broker 10.227.1.110 --group-filter-pattern '.*group_name.*'\n```\n\n**Monitor kafka running in kubernetes, from the kubernetes node**\n\n```\n# Port-forward the pod's port locally in another terminal, or in the background:\ntimeout kubectl port-forward kafka-0 9092:9092 &\n\n# Run kafkatop on the node\nkafkatop\n```\n\n\n# Usage\n```\nusage: kafkatop.py [-h] [--kafka-broker KAFKA_BROKER] [--text] [--poll-period KAFKA_POLL_PERIOD] [--poll-iterations KAFKA_POLL_ITERATIONS] [--group-exclude-pattern KAFKA_GROUP_EXCLUDE_PATTERN]\n [--group-filter-pattern KAFKA_GROUP_FILTER_PATTERN] [--status] [--summary] [--summary-json] [--topicinfo] [--topicinfo-parts] [--only-issues] [--anonymize] [--all]\n [--version]\n\nKafka consumer statistics\n\noptions:\n -h, --help show this help message and exit\n --kafka-broker KAFKA_BROKER\n Broker IP (default: localhost)\n --text Only plain text, no rich output. (default: False)\n --poll-period KAFKA_POLL_PERIOD\n Kafka offset poll period (seconds) for evts/sec calculation (default: 5)\n --poll-iterations KAFKA_POLL_ITERATIONS\n How many times to query and display stats. -1 = Inf (default: 15)\n --group-exclude-pattern KAFKA_GROUP_EXCLUDE_PATTERN\n If group matches regex, exclude (default: None)\n --group-filter-pattern KAFKA_GROUP_FILTER_PATTERN\n Include *only* the groups which match regex (default: None)\n --status Report health status in json and exit. (default: False)\n --summary Display consumer groups, states, topics, partitions, and lags summary. (default: False)\n --summary-json Display consumer groups, states, topics, partitions, and lags summary, in json. (default: False)\n --topicinfo Only show informational data about the cluster, topics, partitions, no stats (fast). (default: False)\n --topicinfo-parts Same as --info but also show data about partitions, isr, leaders. (default: False)\n --only-issues Only show rows with issues. (default: False)\n --anonymize Anonymize topics and groups. (default: False)\n --all Show groups with no members. (default: False)\n --version show program's version number and exit\n```\n\n\n# Examples\n## Screenshots\n\n|  |\n|:--:|\n| *In the screenshot above (anonymized topics/groups), the 1st row is highlighted as the ETA to consume all the lag, with the consuming rate of the last period, is > 1 minute. Red rows indicate consumption rate is lower than it should be. Cell colors indicate severity.* |\n\n|  |\n|:--:|\n| *A healthy system.* |\n\n|  |\n|:--:| \n| *A summary of (anonymized) groups, topics, partitions, lag and consumer group states. Lag is the median lag of all topic partitions.* |\n\n\n## Text output\n\n```\nkafkatop --kafka-broker 1.2.3.4 --group-filter-pattern 'GroupName1' --summary-json\n```\n\nresults in:\n\n```\n{\n \"ConsumerGroupName1\": {\n \"TopicName\": {\n \"partitions\": 1672,\n \"state\": \"ConsumerGroupState.STABLE\",\n \"lag_max\": 86169242,\n \"lag_min\": 0\n }\n },\n[...]\n}\n```\n\n# Contributing\n## Building\n\nRequires python >=3.9 in your path\n\n\n1. set the full path of PYTHON at the top of Makefile or add the PYTHON= parameter when calling make\n2. ```make```\n3. ```make pex```\n\n\n```\nmake pex\n```\nThis will create a ```kafkatop``` pex executable which will include the python code and library dependencies all in one file. It will need the python3 in the path to run.\n\n### Multiplatform building\nIf you have multiple python versions on your environment, you can run ```make-pex-mp.sh``` to create a kafkatop binary which includes dependencies for all python versions.\n\n# License\nThis project is licensed under the terms of the MIT license.\n\n",
"bugtrack_url": null,
"license": null,
"summary": "A real-time monitoring tool for Apache Kafka that displays consumer lag and event rates",
"version": "1.16",
"project_urls": {
"Homepage": "https://github.com/sivann/kafkatop",
"Repository": "https://github.com/sivann/kafkatop"
},
"split_keywords": [
"kafka",
" monitoring",
" consumer",
" lag",
" real-time"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "0660decc192f957976f6175459d574c9057bd92116b1a0d0ff63971b2acb3548",
"md5": "38f26c7f7831f8da0e98d1f20b0c1045",
"sha256": "8af9f563232ed21a5fea5349da010e2f2a3462d6c11c84678389db120f3e549b"
},
"downloads": -1,
"filename": "kafkatop-1.16-py3-none-any.whl",
"has_sig": false,
"md5_digest": "38f26c7f7831f8da0e98d1f20b0c1045",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 15766,
"upload_time": "2025-09-11T08:49:28",
"upload_time_iso_8601": "2025-09-11T08:49:28.748140Z",
"url": "https://files.pythonhosted.org/packages/06/60/decc192f957976f6175459d574c9057bd92116b1a0d0ff63971b2acb3548/kafkatop-1.16-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "a28d9cbae2eb43a5464ed53aabb4fedae0f183e411870c562b9b6cfc911adb53",
"md5": "05fa7038db2b9afc103b5623cbd30724",
"sha256": "f2a1f865a571c29d8c6487487ca1e8337c082efedae52ed0c3ffb5d50d0f58ec"
},
"downloads": -1,
"filename": "kafkatop-1.16.tar.gz",
"has_sig": false,
"md5_digest": "05fa7038db2b9afc103b5623cbd30724",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 16169,
"upload_time": "2025-09-11T08:49:29",
"upload_time_iso_8601": "2025-09-11T08:49:29.924206Z",
"url": "https://files.pythonhosted.org/packages/a2/8d/9cbae2eb43a5464ed53aabb4fedae0f183e411870c562b9b6cfc911adb53/kafkatop-1.16.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-11 08:49:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sivann",
"github_project": "kafkatop",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "confluent-kafka",
"specs": []
},
{
"name": "humanize",
"specs": []
},
{
"name": "rich",
"specs": []
},
{
"name": "readchar",
"specs": []
},
{
"name": "markdown-it-py",
"specs": [
[
"==",
"3.0.0"
]
]
},
{
"name": "markdown-it-py",
"specs": []
}
],
"lcname": "kafkatop"
}
JSON
