# llm-benchmark (ollama-benchmark)
LLM Benchmark for Throughput via Ollama (Local LLMs)
## Installation prerequisites
Working [Ollama](https://ollama.com) installation.
## Installation Steps
Depending on your python setup either
```bash
pip install llm-benchmark
```
or
```bash
pipx install llm-benchmark
```
## Usage for general users directly
```bash
llm_benchmark run
```
## Installation and Usage in Video format

It's tested on Python 3.9 and above.
## ollama installation with the following models installed
7B model can be run on machines with 8GB of RAM
13B model can be run on machines with 16GB of RAM
## Usage explaination
On Windows, Linux, and macOS, it will detect memory RAM size to first download required LLM models.
When memory RAM size is greater than or equal to 4GB, but less than 7GB, it will check if gemma:2b exist. The program implicitly pull the model.
```bash
ollama pull deepseek-r1:1.5b
ollama pull gemma:2b
ollama pull phi:2.7b
ollama pull phi3:3.8b
```
When memory RAM size is greater than 7GB, but less than 15GB, it will check if these models exist. The program implicitly pull these models
```bash
ollama pull phi3:3.8b
ollama pull gemma2:9b
ollama pull mistral:7b
ollama pull llama3.1:8b
ollama pull deepseek-r1:8b
ollama pull llava:7b
```
When memory RAM size is greater than 15GB, but less than 31GB, it will check if these models exist. The program implicitly pull these models
```bash
ollama pull gemma2:9b
ollama pull mistral:7b
ollama pull phi4:14b
ollama pull deepseek-r1:8b
ollama pull deepseek-r1:14b
ollama pull llava:7b
ollama pull llava:13b
```
When memory RAM size is greater than 31GB, it will check if these models exist. The program implicitly pull these models
```bash
ollama pull phi4:14b
ollama pull deepseek-r1:14b
ollama pull deepseek-r1:32b
```
## Python Poetry manually(advanced) installation
<https://python-poetry.org/docs/#installing-manually>
## For developers to develop new features on Windows Powershell or on Ubuntu Linux or macOS
```bash
python3 -m venv .venv
. ./.venv/bin/activate
pip install -U pip setuptools
pip install poetry
```
## Usage in Python virtual environment
```bash
poetry shell
poetry install
llm_benchmark hello jason
```
### Example #1 send systeminfo and benchmark results to a remote server
```bash
llm_benchmark run
```
### Example #2 Do not send systeminfo and benchmark results to a remote server
```bash
llm_benchmark run --no-sendinfo
```
### Example #3 Benchmark run on explicitly given the path to the ollama executable (When you built your own developer version of ollama)
```bash
llm_benchmark run --ollamabin=~/code/ollama/ollama
```
## Reference
[Ollama](https://ollama.com)
Raw data
{
"_id": null,
"home_page": "https://github.com/aidatatools/ollama-benchmark",
"name": "llm-benchmark",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Jason Chuang",
"author_email": "chuangtcee@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/9e/df/785e13fc0e9cf76d4f838ec53a1873a8263d4fe425e6813ccfdfdea33c36/llm_benchmark-0.4.2.tar.gz",
"platform": null,
"description": "# llm-benchmark (ollama-benchmark)\n\nLLM Benchmark for Throughput via Ollama (Local LLMs)\n\n## Installation prerequisites\n\nWorking [Ollama](https://ollama.com) installation.\n\n## Installation Steps\n\nDepending on your python setup either\n\n```bash\npip install llm-benchmark\n```\n\nor\n\n```bash\npipx install llm-benchmark\n```\n\n## Usage for general users directly\n\n```bash\nllm_benchmark run\n```\n\n## Installation and Usage in Video format\n\n\n\nIt's tested on Python 3.9 and above.\n\n## ollama installation with the following models installed\n\n7B model can be run on machines with 8GB of RAM\n\n13B model can be run on machines with 16GB of RAM\n\n## Usage explaination\n\nOn Windows, Linux, and macOS, it will detect memory RAM size to first download required LLM models.\n\nWhen memory RAM size is greater than or equal to 4GB, but less than 7GB, it will check if gemma:2b exist. The program implicitly pull the model.\n\n```bash\nollama pull deepseek-r1:1.5b\nollama pull gemma:2b\nollama pull phi:2.7b\nollama pull phi3:3.8b\n```\n\nWhen memory RAM size is greater than 7GB, but less than 15GB, it will check if these models exist. The program implicitly pull these models\n\n```bash\nollama pull phi3:3.8b\nollama pull gemma2:9b\nollama pull mistral:7b\nollama pull llama3.1:8b\nollama pull deepseek-r1:8b\nollama pull llava:7b\n```\n\nWhen memory RAM size is greater than 15GB, but less than 31GB, it will check if these models exist. The program implicitly pull these models\n\n```bash\nollama pull gemma2:9b\nollama pull mistral:7b\nollama pull phi4:14b\nollama pull deepseek-r1:8b\nollama pull deepseek-r1:14b\nollama pull llava:7b\nollama pull llava:13b\n```\n\nWhen memory RAM size is greater than 31GB, it will check if these models exist. The program implicitly pull these models\n\n```bash\nollama pull phi4:14b\nollama pull deepseek-r1:14b\nollama pull deepseek-r1:32b\n```\n\n## Python Poetry manually(advanced) installation\n\n<https://python-poetry.org/docs/#installing-manually>\n\n## For developers to develop new features on Windows Powershell or on Ubuntu Linux or macOS\n\n```bash\npython3 -m venv .venv\n. ./.venv/bin/activate\npip install -U pip setuptools\npip install poetry\n```\n\n## Usage in Python virtual environment\n\n```bash\npoetry shell\npoetry install\nllm_benchmark hello jason\n```\n\n### Example #1 send systeminfo and benchmark results to a remote server\n\n```bash\nllm_benchmark run\n```\n\n### Example #2 Do not send systeminfo and benchmark results to a remote server\n\n```bash\nllm_benchmark run --no-sendinfo\n```\n\n### Example #3 Benchmark run on explicitly given the path to the ollama executable (When you built your own developer version of ollama)\n\n```bash\nllm_benchmark run --ollamabin=~/code/ollama/ollama\n```\n\n## Reference\n\n[Ollama](https://ollama.com)\n",
"bugtrack_url": null,
"license": null,
"summary": "LLM Benchmark",
"version": "0.4.2",
"project_urls": {
"Homepage": "https://github.com/aidatatools/ollama-benchmark"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "6cb8769e33a56587d9d4cc3d232fb247d5e3f70308b0e04e252487f497be5cca",
"md5": "1f1538081a4a003043ca6df1cf191688",
"sha256": "8767f711d0bf06ac642c59afb377ede730d755dd531669343454810032991168"
},
"downloads": -1,
"filename": "llm_benchmark-0.4.2-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "1f1538081a4a003043ca6df1cf191688",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.9",
"size": 2133270,
"upload_time": "2025-02-04T16:10:24",
"upload_time_iso_8601": "2025-02-04T16:10:24.792227Z",
"url": "https://files.pythonhosted.org/packages/6c/b8/769e33a56587d9d4cc3d232fb247d5e3f70308b0e04e252487f497be5cca/llm_benchmark-0.4.2-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "9edf785e13fc0e9cf76d4f838ec53a1873a8263d4fe425e6813ccfdfdea33c36",
"md5": "7f2ced968515f722339c187d4571ab28",
"sha256": "78ccd7b7a2beef117b2420a046e7dd18ab684f93b4c057778cb39f0d8d8b219c"
},
"downloads": -1,
"filename": "llm_benchmark-0.4.2.tar.gz",
"has_sig": false,
"md5_digest": "7f2ced968515f722339c187d4571ab28",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 2130918,
"upload_time": "2025-02-04T16:10:27",
"upload_time_iso_8601": "2025-02-04T16:10:27.228832Z",
"url": "https://files.pythonhosted.org/packages/9e/df/785e13fc0e9cf76d4f838ec53a1873a8263d4fe425e6813ccfdfdea33c36/llm_benchmark-0.4.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-04 16:10:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "aidatatools",
"github_project": "ollama-benchmark",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "anyio",
"specs": [
[
"==",
"4.3.0"
]
]
},
{
"name": "certifi",
"specs": [
[
"==",
"2024.07.04"
]
]
},
{
"name": "charset-normalizer",
"specs": [
[
"==",
"3.3.2"
]
]
},
{
"name": "cli-exit-tools",
"specs": [
[
"==",
"1.2.6"
]
]
},
{
"name": "click",
"specs": [
[
"==",
"8.1.7"
]
]
},
{
"name": "colorama",
"specs": [
[
"==",
"0.4.6"
]
]
},
{
"name": "exceptiongroup",
"specs": [
[
"==",
"1.2.0"
]
]
},
{
"name": "fake-winreg",
"specs": [
[
"==",
"1.6.3"
]
]
},
{
"name": "GPUtil",
"specs": [
[
"==",
"1.4.0"
]
]
},
{
"name": "h11",
"specs": [
[
"==",
"0.14.0"
]
]
},
{
"name": "httpcore",
"specs": [
[
"==",
"1.0.5"
]
]
},
{
"name": "httpx",
"specs": [
[
"==",
"0.27.0"
]
]
},
{
"name": "idna",
"specs": [
[
"==",
"3.7"
]
]
},
{
"name": "lib-detect-testenv",
"specs": [
[
"==",
"2.0.8"
]
]
},
{
"name": "lib-platform",
"specs": [
[
"==",
"1.2.10"
]
]
},
{
"name": "lib-registry",
"specs": [
[
"==",
"2.0.10"
]
]
},
{
"name": "markdown-it-py",
"specs": [
[
"==",
"3.0.0"
]
]
},
{
"name": "mdurl",
"specs": [
[
"==",
"0.1.2"
]
]
},
{
"name": "ollama",
"specs": [
[
"==",
"0.1.8"
]
]
},
{
"name": "psutil",
"specs": [
[
"==",
"5.9.8"
]
]
},
{
"name": "Pygments",
"specs": [
[
"==",
"2.17.2"
]
]
},
{
"name": "PyYAML",
"specs": [
[
"==",
"6.0.1"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.32.3"
]
]
},
{
"name": "rich",
"specs": [
[
"==",
"13.7.1"
]
]
},
{
"name": "shellingham",
"specs": [
[
"==",
"1.5.4"
]
]
},
{
"name": "sniffio",
"specs": [
[
"==",
"1.3.1"
]
]
},
{
"name": "typer",
"specs": [
[
"==",
"0.9.4"
]
]
},
{
"name": "typing_extensions",
"specs": [
[
"==",
"4.10.0"
]
]
},
{
"name": "urllib3",
"specs": [
[
"==",
"2.2.2"
]
]
},
{
"name": "wrapt",
"specs": [
[
"==",
"1.16.0"
]
]
}
],
"lcname": "llm-benchmark"
}