 🦉 Observability And Optimization In Modern AI Era
----
[](https://hiq.readthedocs.io/en/latest/?badge=latest)
[![CodeCov][cov-img]][cov]
[![Github release][release-img]][release]
[![lic][license-img]][license]
[](https://arxiv.org/abs/2304.13302)
> 🔥 HiQ now supports GPU profiling, DNN model visualization and tracing for DNN libraries like pyTorch, `transformers`, LAVIS, and LLMs like LLaMA, OPT, Bloom, T5 and GPT2 in addition to Onnxruntime, FastAPI and Flask.
HiQ is a `declarative`, `non-intrusive`, `dynamic` and `transparent` tracking system for both **monolithic** application and **distributed** system. It brings the runtime information tracking and optimization to a new level without compromising with speed and system performance, or hiding any tracking overhead information. HiQ applies for both I/O bound and CPU bound applications. In addition to latency tracking, HiQ provides memory, disk I/O and Network I/O tracking out of the box. The output can be saved in form of normal line by line log file, or HiQ tree, or span graph.
HiQ's philosophy is to **decouple `observability logic` from `business logic`**. We don't have to enter the black hole to observe it. Do you like the idea? Leave a ⭐ if you enjoy the project and welcome to say Hi to us on [Slack 👋](https://join.slack.com/t/hiq-myo2317/shared_invite/zt-17ejh6ybo-51IX6G1lHMXgLbq2HKIO_Q)
[📜HiQ Pape: A Declarative, Non-intrusive, Dynamic and Transparent Observability and Optimization System](https://arxiv.org/abs/2304.13302)
## Installation
- Basic Installation
```bash
pip install hiq-python
```
- HiQ also supports extra installation
```bash
pip install hiq-python[fastapi] # To support fastapi web server online tracing
pip install hiq-python[gpu] # To support GPU tracing, which will install pynvml
pip install hiq-python[lavis] # To support Salesforce LAVIS Vision Language models
pip install hiq-python[transformers] # To support tracing Hugging Face's transformers library
pip install hiq-python[full] # To support all the cases, and this will install all the dependency libraries
```
## Get Started
Let start with a simplest example by running HiQ against a simple monolithic python code [📄 `main.py`](hiq/examples/quick_start/main.py):
```python
# this is the main.py python source code
import time
def func1():
time.sleep(1.5)
print("func1")
func2()
def func2():
time.sleep(2.5)
print("func2")
def main():
func1()
if __name__ == "__main__":
main()
```
In this code, there is a simple chain of function calls: `main()` -> `func1` -> `func2`.
Now we want to trace the functions without modifying its code. Let's run the following:
```python
git clone https://github.com/oracle-samples/hiq.git
cd hiq/examples/quick_start
python main_driver.py
```
If everything is fine, you should be able to see the output like this:
From the screenshot we can see the timestamp and the latency of each function:
| | main | func1 | func2 | tracing overhead |
|---|---|---|---|---|
| latency(second) | 4.0045 | 4.0044 | 2.5026 | 0.0000163 |
HiQ just traced the `main.py` file running without touching one line of its code.
## Documentation
**HTML**: [🔗 HiQ Online Documents](https://hiq.readthedocs.io/en/latest/index.html) | **PDF**: Please check [🔗 HiQ User Guide](https://github.com/oracle/hiq/blob/main/hiq/docs/hiq.pdf).
----
Logging: https://hiq.readthedocs.io/en/latest/4_o_advanced.html#log-monkey-king
Tracing: https://hiq.readthedocs.io/en/latest/5_distributed.html
- Zipkin: https://hiq.readthedocs.io/en/latest/5_distributed.html#zipkin
- Jaeger: https://hiq.readthedocs.io/en/latest/5_distributed.html#jaeger
Metrics:
- Prometheus: https://hiq.readthedocs.io/en/latest/7_integration.html#prometheus
Streaming:
- Kafka: https://hiq.readthedocs.io/en/latest/7_integration.html#oci-streaming
## DNN Model Observability & Visualization
HiQ can visualize DNN model. To get the following BERT model's structure, you can just run:
```
python -m hiq.vis
```
The graph is self-explantory. There are several conventions:
- ❄️ means frozen layer, where `requires_grad` is false.
- 📈 means gradient exists for that model parameter, which usually happens after backpopulation.
- `+`, bold font, and underscored dotted line mean the displayed layer is a folded version of multiple layers with the same structure.
What you need to do is just calling `print_model(model)` in your code. Refer to: [here](https://github.com/henrywoo/hiq/tree/main/hiq/examples/vis) for how to use it.
## HiQ Web UI
- Main Page
- Latency Details
## Jupyter NoteBook
HiQ was originally developed to find Onnxruntime performance bottleneck in DNN inference, and it works well for other computation intensive applications too. The following are two examples.
### Add Observability to PaddlePaddle (PaddleOCR)
- [Latency](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/paddle/demo.ipynb)
- [Memory](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/paddle/demo_memory.ipynb)
- Latency Gantt Chart
- HiQ Call Graph
### Add Observability to Onnxruntime (AlexNet)
- [Latency](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/onnxruntime/demo.ipynb)
- [Intrusive](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/onnxruntime/demo_intrusive.ipynb)
## Examples
Please check [🔗 examples](https://github.com/oracle/hiq/blob/main/hiq/examples) for usage examples.
## Contributing
HiQ welcomes contributions from the community. Before submitting a pull request, please review our contribution guide](./CONTRIBUTING.md).
## Security
Please consult the [🔗 security guide](https://github.com/oracle/hiq/blob/main/SECURITY.md) for our responsible security vulnerability disclosure process.
## License
Copyright (c) 2022, 2023 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at <https://oss.oracle.com/licenses/upl/>.
## Presentation and Demos
- [Introduction to Observability with HiQ](https://github.com/oracle-samples/hiq/blob/main/hiq/docs/Introduction-To-Observability-With-HiQ.pdf)
[cov-img]: https://codecov.io/gh/uber/athenadriver/branch/master/graph/badge.svg
[cov]: https://hiq.readthedocs.io/en/latest/index.html
[release-img]: https://img.shields.io/badge/release-v1.1.14-red
[release]: https://github.com/oracle-samples/hiq
[license-img]: https://img.shields.io/badge/License-UPL--1.0-red
[license]: https://github.com/oracle-samples/hiq/blob/main/LICENSE.txt
[release-policy]: https://golang.org/doc/devel/release.html#policy
Raw data
{
"_id": null,
"home_page": "https://github.com/oracle/hiq",
"name": "hiq-python",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "hiq",
"author": "Henry Fuheng Wu; Kathan Patel; Zixin Kong",
"author_email": "<fuheng.wu@oracle.com>",
"download_url": null,
"platform": null,
"description": " \ud83e\udd89 Observability And Optimization In Modern AI Era\n----\n[](https://hiq.readthedocs.io/en/latest/?badge=latest)\n[![CodeCov][cov-img]][cov]\n[![Github release][release-img]][release]\n[![lic][license-img]][license]\n[](https://arxiv.org/abs/2304.13302)\n\n> \ud83d\udd25 HiQ now supports GPU profiling, DNN model visualization and tracing for DNN libraries like pyTorch, `transformers`, LAVIS, and LLMs like LLaMA, OPT, Bloom, T5 and GPT2 in addition to Onnxruntime, FastAPI and Flask.\n\nHiQ is a `declarative`, `non-intrusive`, `dynamic` and `transparent` tracking system for both **monolithic** application and **distributed** system. It brings the runtime information tracking and optimization to a new level without compromising with speed and system performance, or hiding any tracking overhead information. HiQ applies for both I/O bound and CPU bound applications. In addition to latency tracking, HiQ provides memory, disk I/O and Network I/O tracking out of the box. The output can be saved in form of normal line by line log file, or HiQ tree, or span graph.\n\nHiQ's philosophy is to **decouple `observability logic` from `business logic`**. We don't have to enter the black hole to observe it. Do you like the idea? Leave a \u2b50 if you enjoy the project and welcome to say Hi to us on [Slack \ud83d\udc4b](https://join.slack.com/t/hiq-myo2317/shared_invite/zt-17ejh6ybo-51IX6G1lHMXgLbq2HKIO_Q)\n\n[\ud83d\udcdcHiQ Pape: A Declarative, Non-intrusive, Dynamic and Transparent Observability and Optimization System](https://arxiv.org/abs/2304.13302)\n\n\n\n## Installation\n\n- Basic Installation\n\n```bash\npip install hiq-python\n```\n\n- HiQ also supports extra installation\n\n```bash\npip install hiq-python[fastapi] # To support fastapi web server online tracing\npip install hiq-python[gpu] # To support GPU tracing, which will install pynvml\npip install hiq-python[lavis] # To support Salesforce LAVIS Vision Language models\npip install hiq-python[transformers] # To support tracing Hugging Face's transformers library\npip install hiq-python[full] # To support all the cases, and this will install all the dependency libraries\n```\n\n## Get Started\n\nLet start with a simplest example by running HiQ against a simple monolithic python code [\ud83d\udcc4 `main.py`](hiq/examples/quick_start/main.py):\n\n```python\n# this is the main.py python source code\nimport time\n\ndef func1():\n time.sleep(1.5)\n print(\"func1\")\n func2()\n\ndef func2():\n time.sleep(2.5)\n print(\"func2\")\n\ndef main():\n func1()\n\nif __name__ == \"__main__\":\n main()\n```\n\nIn this code, there is a simple chain of function calls: `main()` -> `func1` -> `func2`.\n\nNow we want to trace the functions without modifying its code. Let's run the following:\n\n\n```python\ngit clone https://github.com/oracle-samples/hiq.git\ncd hiq/examples/quick_start\npython main_driver.py\n```\n\nIf everything is fine, you should be able to see the output like this:\n\n\n\nFrom the screenshot we can see the timestamp and the latency of each function:\n\n\n| | main | func1 | func2 | tracing overhead |\n|---|---|---|---|---|\n| latency(second) | 4.0045 | 4.0044 | 2.5026 | 0.0000163 |\n\n\nHiQ just traced the `main.py` file running without touching one line of its code.\n\n## Documentation\n\n**HTML**: [\ud83d\udd17 HiQ Online Documents](https://hiq.readthedocs.io/en/latest/index.html) | **PDF**: Please check [\ud83d\udd17 HiQ User Guide](https://github.com/oracle/hiq/blob/main/hiq/docs/hiq.pdf).\n\n----\n\nLogging: https://hiq.readthedocs.io/en/latest/4_o_advanced.html#log-monkey-king \nTracing: https://hiq.readthedocs.io/en/latest/5_distributed.html \n- Zipkin: https://hiq.readthedocs.io/en/latest/5_distributed.html#zipkin \n- Jaeger: https://hiq.readthedocs.io/en/latest/5_distributed.html#jaeger \n\nMetrics: \n- Prometheus: https://hiq.readthedocs.io/en/latest/7_integration.html#prometheus \n\nStreaming: \n- Kafka: https://hiq.readthedocs.io/en/latest/7_integration.html#oci-streaming \n\n## DNN Model Observability & Visualization\n\nHiQ can visualize DNN model. To get the following BERT model's structure, you can just run:\n\n```\npython -m hiq.vis\n```\n\n\n\nThe graph is self-explantory. There are several conventions:\n\n- \u2744\ufe0f means frozen layer, where `requires_grad` is false.\n- \ud83d\udcc8 means gradient exists for that model parameter, which usually happens after backpopulation.\n- `+`, bold font, and underscored dotted line mean the displayed layer is a folded version of multiple layers with the same structure.\n\nWhat you need to do is just calling `print_model(model)` in your code. Refer to: [here](https://github.com/henrywoo/hiq/tree/main/hiq/examples/vis) for how to use it.\n\n## HiQ Web UI\n\n- Main Page\n\n\n\n- Latency Details\n\n\n\n## Jupyter NoteBook\n\nHiQ was originally developed to find Onnxruntime performance bottleneck in DNN inference, and it works well for other computation intensive applications too. The following are two examples.\n\n### Add Observability to PaddlePaddle (PaddleOCR)\n\n- [Latency](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/paddle/demo.ipynb)\n- [Memory](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/paddle/demo_memory.ipynb)\n\n- Latency Gantt Chart\n\n\n\n- HiQ Call Graph\n\n\n\n### Add Observability to Onnxruntime (AlexNet)\n\n- [Latency](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/onnxruntime/demo.ipynb)\n- [Intrusive](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/onnxruntime/demo_intrusive.ipynb)\n\n## Examples\n\nPlease check [\ud83d\udd17 examples](https://github.com/oracle/hiq/blob/main/hiq/examples) for usage examples.\n\n## Contributing\n\nHiQ welcomes contributions from the community. Before submitting a pull request, please review our contribution guide](./CONTRIBUTING.md).\n\n## Security\n\nPlease consult the [\ud83d\udd17 security guide](https://github.com/oracle/hiq/blob/main/SECURITY.md) for our responsible security vulnerability disclosure process.\n\n## License\n\nCopyright (c) 2022, 2023 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at <https://oss.oracle.com/licenses/upl/>.\n\n## Presentation and Demos\n\n- [Introduction to Observability with HiQ](https://github.com/oracle-samples/hiq/blob/main/hiq/docs/Introduction-To-Observability-With-HiQ.pdf)\n\n[cov-img]: https://codecov.io/gh/uber/athenadriver/branch/master/graph/badge.svg\n[cov]: https://hiq.readthedocs.io/en/latest/index.html\n\n[release-img]: https://img.shields.io/badge/release-v1.1.14-red\n[release]: https://github.com/oracle-samples/hiq\n\n[license-img]: https://img.shields.io/badge/License-UPL--1.0-red\n[license]: https://github.com/oracle-samples/hiq/blob/main/LICENSE.txt\n\n[release-policy]: https://golang.org/doc/devel/release.html#policy\n\n",
"bugtrack_url": null,
"license": null,
"summary": "HiQ - A Modern Observability System",
"version": "1.1.14",
"project_urls": {
"Homepage": "https://github.com/oracle/hiq"
},
"split_keywords": [
"hiq"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c6f061df7e1e92f11f9ab669e3da4f3c39ccb277aaabcaf9a3eaa7e268c64434",
"md5": "df7a5e96322e54af874b58309b31e953",
"sha256": "084abdd623859893b420775c0b771da2243b5dbb0739b82aafd0e0e8e3c3538b"
},
"downloads": -1,
"filename": "hiq_python-1.1.14-py3-none-any.whl",
"has_sig": false,
"md5_digest": "df7a5e96322e54af874b58309b31e953",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 91968,
"upload_time": "2024-08-24T18:52:06",
"upload_time_iso_8601": "2024-08-24T18:52:06.731122Z",
"url": "https://files.pythonhosted.org/packages/c6/f0/61df7e1e92f11f9ab669e3da4f3c39ccb277aaabcaf9a3eaa7e268c64434/hiq_python-1.1.14-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-24 18:52:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "oracle",
"github_project": "hiq",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "hiq-python"
}
JSON
