 🦉 Observability And Optimization In Modern AI Era
----
[](https://hiq.readthedocs.io/en/latest/?badge=latest)
[![CodeCov][cov-img]][cov]
[![Github release][release-img]][release]
[![lic][license-img]][license]
> 🔥 HiQ now supports GPU profiling, DNN model visualization and tracing for DNN libraries like pyTorch, `transformers`, LAVIS, and LLMs like LLaMA, OPT, Bloom, T5 and GPT2 in addition to Onnxruntime, FastAPI and Flask.
HiQ is a `declarative`, `non-intrusive`, `dynamic` and `transparent` tracking system for both **monolithic** application and **distributed** system. It brings the runtime information tracking and optimization to a new level without compromising with speed and system performance, or hiding any tracking overhead information. HiQ applies for both I/O bound and CPU bound applications. In addition to latency tracking, HiQ provides memory, disk I/O and Network I/O tracking out of the box. The output can be saved in form of normal line by line log file, or HiQ tree, or span graph.
HiQ's philosophy is to **decouple `observability logic` from `business logic`**. We don't have to enter the black hole to observe it. Do you like the idea? Leave a ⭐ if you enjoy the project and welcome to say Hi to us on [Slack 👋](https://join.slack.com/t/hiq-myo2317/shared_invite/zt-17ejh6ybo-51IX6G1lHMXgLbq2HKIO_Q)
## Installation
- Basic Installation
```bash
pip install hiq-python
```
- HiQ also supports extra installation
```bash
pip install hiq-python[fastapi] # To support fastapi web server online tracing
pip install hiq-python[gpu] # To support GPU tracing, which will install pynvml
pip install hiq-python[lavis] # To support Salesforce LAVIS Vision Language models
pip install hiq-python[transformers] # To support tracing Hugging Face's transformers library
pip install hiq-python[full] # To support all the cases, and this will install all the dependency libraries
```
## Get Started
Let start with a simplest example by running HiQ against a simple monolithic python code [📄 `main.py`](hiq/examples/quick_start/main.py):
```python
# this is the main.py python source code
import time
def func1():
time.sleep(1.5)
print("func1")
func2()
def func2():
time.sleep(2.5)
print("func2")
def main():
func1()
if __name__ == "__main__":
main()
```
In this code, there is a simple chain of function calls: `main()` -> `func1` -> `func2`.
Now we want to trace the functions without modifying its code. Let's run the following:
```python
git clone https://github.com/oracle-samples/hiq.git
cd hiq/examples/quick_start
python main_driver.py
```
If everything is fine, you should be able to see the output like this:
From the screenshot we can see the timestamp and the latency of each function:
| | main | func1 | func2 | tracing overhead |
|---|---|---|---|---|
| latency(second) | 4.0045 | 4.0044 | 2.5026 | 0.0000163 |
HiQ just traced the `main.py` file running without touching one line of its code.
## Documentation
**HTML**: [🔗 HiQ Online Documents](https://hiq.readthedocs.io/en/latest/index.html) | **PDF**: Please check [🔗 HiQ User Guide](https://github.com/oracle/hiq/blob/main/hiq/docs/hiq.pdf).
----
Logging: https://hiq.readthedocs.io/en/latest/4_o_advanced.html#log-monkey-king
Tracing: https://hiq.readthedocs.io/en/latest/5_distributed.html
- Zipkin: https://hiq.readthedocs.io/en/latest/5_distributed.html#zipkin
- Jaeger: https://hiq.readthedocs.io/en/latest/5_distributed.html#jaeger
Metrics:
- Prometheus: https://hiq.readthedocs.io/en/latest/7_integration.html#prometheus
Streaming:
- Kafka: https://hiq.readthedocs.io/en/latest/7_integration.html#oci-streaming
## DNN Model Observability & Visualization
HiQ can visualize DNN model. To get the following BERT model's structure, you can just run:
```
python -m hiq.vis
```
The graph is self-explantory. There are several conventions:
- ❄️ means frozen layer, where `requires_grad` is false.
- 📈 means gradient exists for that model parameter, which usually happens after backpopulation.
- `+`, bold font, and underscored dotted line mean the displayed layer is a folded version of multiple layers with the same structure.
What you need to do is just calling `print_model(model)` in your code. Refer to: [here](https://github.com/henrywoo/hiq/tree/main/hiq/examples/vis) for how to use it.
## HiQ Web UI
- Main Page
- Latency Details
## Jupyter NoteBook
HiQ was originally developed to find Onnxruntime performance bottleneck in DNN inference, and it works well for other computation intensive applications too. The following are two examples.
### Add Observability to PaddlePaddle (PaddleOCR)
- [Latency](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/paddle/demo.ipynb)
- [Memory](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/paddle/demo_memory.ipynb)
- Latency Gantt Chart
- HiQ Call Graph
### Add Observability to Onnxruntime (AlexNet)
- [Latency](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/onnxruntime/demo.ipynb)
- [Intrusive](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/onnxruntime/demo_intrusive.ipynb)
## Examples
Please check [🔗 examples](https://github.com/oracle/hiq/blob/main/hiq/examples) for usage examples.
## Contributing
HiQ welcomes contributions from the community. Before submitting a pull request, please [review our 🔗 contribution guide](https://github.com/oracle/hiq/blob/main/CONTRIBUTING.md).
## Security
Please consult the [🔗 security guide](https://github.com/oracle/hiq/blob/main/SECURITY.md) for our responsible security vulnerability disclosure process.
## License
Copyright (c) 2022 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at <https://oss.oracle.com/licenses/upl/>.
## Presentation and Demos
- [Introduction to Observability with HiQ](https://github.com/oracle-samples/hiq/blob/main/hiq/docs/Introduction-To-Observability-With-HiQ.pdf)
[cov-img]: https://codecov.io/gh/uber/athenadriver/branch/master/graph/badge.svg
[cov]: https://hiq.readthedocs.io/en/latest/index.html
[release-img]: https://img.shields.io/badge/release-v1.1.12-red
[release]: https://github.com/oracle-samples/hiq
[license-img]: https://img.shields.io/badge/License-UPL--1.0-red
[license]: https://github.com/oracle-samples/hiq/blob/main/LICENSE.txt
[release-policy]: https://golang.org/doc/devel/release.html#policy
Raw data
{
"_id": null,
"home_page": "https://github.com/oracle/hiq",
"name": "hiq-python",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "hiq",
"author": "Henry Fuheng Wu; Kathan Patel; Zixin Kong",
"author_email": "<fuheng.wu@oracle.com>",
"download_url": "https://files.pythonhosted.org/packages/77/64/52fe3947201614d5adb119573c46b19faa49efa320ea5443a85df399d627/hiq-python-1.1.12.tar.gz",
"platform": null,
"description": " \ud83e\udd89 Observability And Optimization In Modern AI Era\n----\n[](https://hiq.readthedocs.io/en/latest/?badge=latest)\n[![CodeCov][cov-img]][cov]\n[![Github release][release-img]][release]\n[![lic][license-img]][license]\n\n> \ud83d\udd25 HiQ now supports GPU profiling, DNN model visualization and tracing for DNN libraries like pyTorch, `transformers`, LAVIS, and LLMs like LLaMA, OPT, Bloom, T5 and GPT2 in addition to Onnxruntime, FastAPI and Flask.\n\nHiQ is a `declarative`, `non-intrusive`, `dynamic` and `transparent` tracking system for both **monolithic** application and **distributed** system. It brings the runtime information tracking and optimization to a new level without compromising with speed and system performance, or hiding any tracking overhead information. HiQ applies for both I/O bound and CPU bound applications. In addition to latency tracking, HiQ provides memory, disk I/O and Network I/O tracking out of the box. The output can be saved in form of normal line by line log file, or HiQ tree, or span graph.\n\nHiQ's philosophy is to **decouple `observability logic` from `business logic`**. We don't have to enter the black hole to observe it. Do you like the idea? Leave a \u2b50 if you enjoy the project and welcome to say Hi to us on [Slack \ud83d\udc4b](https://join.slack.com/t/hiq-myo2317/shared_invite/zt-17ejh6ybo-51IX6G1lHMXgLbq2HKIO_Q)\n\n\n\n\n## Installation\n\n- Basic Installation\n\n```bash\npip install hiq-python\n```\n\n- HiQ also supports extra installation\n\n```bash\npip install hiq-python[fastapi] # To support fastapi web server online tracing\npip install hiq-python[gpu] # To support GPU tracing, which will install pynvml\npip install hiq-python[lavis] # To support Salesforce LAVIS Vision Language models\npip install hiq-python[transformers] # To support tracing Hugging Face's transformers library\npip install hiq-python[full] # To support all the cases, and this will install all the dependency libraries\n```\n\n\n## Get Started\n\nLet start with a simplest example by running HiQ against a simple monolithic python code [\ud83d\udcc4 `main.py`](hiq/examples/quick_start/main.py):\n\n```python\n# this is the main.py python source code\nimport time\n\ndef func1():\n time.sleep(1.5)\n print(\"func1\")\n func2()\n\ndef func2():\n time.sleep(2.5)\n print(\"func2\")\n\ndef main():\n func1()\n\nif __name__ == \"__main__\":\n main()\n```\n\nIn this code, there is a simple chain of function calls: `main()` -> `func1` -> `func2`.\n\nNow we want to trace the functions without modifying its code. Let's run the following:\n\n\n```python\ngit clone https://github.com/oracle-samples/hiq.git\ncd hiq/examples/quick_start\npython main_driver.py\n```\n\nIf everything is fine, you should be able to see the output like this:\n\n\n\nFrom the screenshot we can see the timestamp and the latency of each function:\n\n\n| | main | func1 | func2 | tracing overhead |\n|---|---|---|---|---|\n| latency(second) | 4.0045 | 4.0044 | 2.5026 | 0.0000163 |\n\n\nHiQ just traced the `main.py` file running without touching one line of its code.\n\n## Documentation\n\n**HTML**: [\ud83d\udd17 HiQ Online Documents](https://hiq.readthedocs.io/en/latest/index.html) | **PDF**: Please check [\ud83d\udd17 HiQ User Guide](https://github.com/oracle/hiq/blob/main/hiq/docs/hiq.pdf).\n\n----\n\nLogging: https://hiq.readthedocs.io/en/latest/4_o_advanced.html#log-monkey-king \nTracing: https://hiq.readthedocs.io/en/latest/5_distributed.html \n- Zipkin: https://hiq.readthedocs.io/en/latest/5_distributed.html#zipkin \n- Jaeger: https://hiq.readthedocs.io/en/latest/5_distributed.html#jaeger \n\nMetrics: \n- Prometheus: https://hiq.readthedocs.io/en/latest/7_integration.html#prometheus \n\nStreaming: \n- Kafka: https://hiq.readthedocs.io/en/latest/7_integration.html#oci-streaming \n\n## DNN Model Observability & Visualization\n\nHiQ can visualize DNN model. To get the following BERT model's structure, you can just run:\n\n```\npython -m hiq.vis\n```\n\n\n\nThe graph is self-explantory. There are several conventions:\n\n- \u2744\ufe0f means frozen layer, where `requires_grad` is false.\n- \ud83d\udcc8 means gradient exists for that model parameter, which usually happens after backpopulation.\n- `+`, bold font, and underscored dotted line mean the displayed layer is a folded version of multiple layers with the same structure.\n\nWhat you need to do is just calling `print_model(model)` in your code. Refer to: [here](https://github.com/henrywoo/hiq/tree/main/hiq/examples/vis) for how to use it.\n\n## HiQ Web UI\n\n- Main Page\n\n\n\n- Latency Details\n\n\n\n## Jupyter NoteBook\n\nHiQ was originally developed to find Onnxruntime performance bottleneck in DNN inference, and it works well for other computation intensive applications too. The following are two examples.\n\n### Add Observability to PaddlePaddle (PaddleOCR)\n\n- [Latency](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/paddle/demo.ipynb)\n- [Memory](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/paddle/demo_memory.ipynb)\n\n- Latency Gantt Chart\n\n\n\n- HiQ Call Graph\n\n\n\n### Add Observability to Onnxruntime (AlexNet)\n\n- [Latency](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/onnxruntime/demo.ipynb)\n- [Intrusive](https://github.com/oracle-samples/hiq/blob/main/hiq/examples/onnxruntime/demo_intrusive.ipynb)\n\n\n## Examples\n\nPlease check [\ud83d\udd17 examples](https://github.com/oracle/hiq/blob/main/hiq/examples) for usage examples.\n\n## Contributing\n\n\nHiQ welcomes contributions from the community. Before submitting a pull request, please [review our \ud83d\udd17 contribution guide](https://github.com/oracle/hiq/blob/main/CONTRIBUTING.md).\n\n\n\n## Security\n\nPlease consult the [\ud83d\udd17 security guide](https://github.com/oracle/hiq/blob/main/SECURITY.md) for our responsible security vulnerability disclosure process.\n\n\n## License\n\nCopyright (c) 2022 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at <https://oss.oracle.com/licenses/upl/>.\n\n## Presentation and Demos\n\n- [Introduction to Observability with HiQ](https://github.com/oracle-samples/hiq/blob/main/hiq/docs/Introduction-To-Observability-With-HiQ.pdf)\n\n[cov-img]: https://codecov.io/gh/uber/athenadriver/branch/master/graph/badge.svg\n[cov]: https://hiq.readthedocs.io/en/latest/index.html\n\n[release-img]: https://img.shields.io/badge/release-v1.1.12-red\n[release]: https://github.com/oracle-samples/hiq\n\n[license-img]: https://img.shields.io/badge/License-UPL--1.0-red\n[license]: https://github.com/oracle-samples/hiq/blob/main/LICENSE.txt\n\n[release-policy]: https://golang.org/doc/devel/release.html#policy\n\n",
"bugtrack_url": null,
"license": "",
"summary": "HiQ - A Modern Observability System",
"version": "1.1.12",
"split_keywords": [
"hiq"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ba7d81080b5b2ff8f440235cba5195442f5009e915fb7a6592866b1742db4341",
"md5": "d506f9d77aae324c5a3f99238c0c0c9c",
"sha256": "51722fe631f018305ad05bb16233a4e963acb543c0026824ac00deb61d70d9e9"
},
"downloads": -1,
"filename": "hiq_python-1.1.12-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d506f9d77aae324c5a3f99238c0c0c9c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 74152,
"upload_time": "2023-04-11T04:31:02",
"upload_time_iso_8601": "2023-04-11T04:31:02.723190Z",
"url": "https://files.pythonhosted.org/packages/ba/7d/81080b5b2ff8f440235cba5195442f5009e915fb7a6592866b1742db4341/hiq_python-1.1.12-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "776452fe3947201614d5adb119573c46b19faa49efa320ea5443a85df399d627",
"md5": "cf75ba5f627f9cc47e82954cb564132e",
"sha256": "9f91a0fe6cc7242789ca3cc2227e3bc0cf8825eb84d6be91e21d18b6186fa763"
},
"downloads": -1,
"filename": "hiq-python-1.1.12.tar.gz",
"has_sig": false,
"md5_digest": "cf75ba5f627f9cc47e82954cb564132e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 65551,
"upload_time": "2023-04-11T04:31:05",
"upload_time_iso_8601": "2023-04-11T04:31:05.119494Z",
"url": "https://files.pythonhosted.org/packages/77/64/52fe3947201614d5adb119573c46b19faa49efa320ea5443a85df399d627/hiq-python-1.1.12.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-11 04:31:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "oracle",
"github_project": "hiq",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "hiq-python"
}
JSON
