| Name | bulkllm JSON |
| Version |
0.5.10
JSON |
| download |
| home_page | None |
| Summary | None |
| upload_time | 2025-08-05 18:41:18 |
| maintainer | None |
| docs_url | None |
| author | Bryce Drennan |
| requires_python | >=3.12 |
| license | None |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# bulkllm
## Enhancements over vanilla LiteLLM
bulkllm builds on top of `litellm` and adds a few extras:
- **Automatic model registration.** The package knows how to fetch the list of
models from OpenAI, Anthropic, Gemini and OpenRouter and registers them with
LiteLLM. Results are cached on disk so they can be reused offline.
- **Centralised rate limiting.** A `RateLimiter` implementation enforces RPM,
TPM, input and output token limits per model (or regex group) and works with
both async and sync code.
- **Retry‑aware completion wrappers.** Thin wrappers around
`litellm.completion`/`acompletion` integrate Tenacity retries, rate limiting
and usage tracking.
- **Usage tracking with statistics.** Per‑model usage is tracked in memory with
histograms, percentiles and cost calculations via the `UsageTracker` and
`UsageStat` helpers.
- **Predefined LLM configurations.** A large catalogue of model presets with
cost information and convenient selection helpers is included.
## Development
Always run `make checku` before committing.
### Quick Commands
- `make init` create the environment and install dependencies
- `make help` see available commands
- `make autoformat` format code
- `make autoformat-unsafe` format code - including 'unsafe' fixes
- `make lint` run linter
- `make typecheck` run type checker
- `make test` run tests
- `make coverage` run tests with coverage report
- `make check` run all checks (format, lint, typecheck, test)
- `make checku` run all checks (format-unsafe, lint, typecheck, test)
### Code Conventions
#### Testing
- Use **pytest** (no test classes).
- Always set `match=` in `pytest.raises`.
- Prefer `monkeypatch` over other mocks.
- Mirror the source-tree layout in `tests/`.
- Always run `make checku` after making changes.
#### Exceptions
- Catch only specific exceptions—never blanket `except:` blocks.
- Don’t raise bare `Exception`.
#### Python
- Manage env/deps with **uv** (`uv add|remove`, `uv run -- …`).
- No logging config or side-effects at import time.
- Keep interfaces (CLI, web, etc.) thin; put logic elsewhere.
- Use `typer` for CLI interfaces, `fastapi` for web interfaces,
and `pydantic` for data models.
Raw data
{
"_id": null,
"home_page": null,
"name": "bulkllm",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.12",
"maintainer_email": null,
"keywords": null,
"author": "Bryce Drennan",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/f7/97/dd4844f143edfc9fbf0a88536a76decb25867831e9b66bc5056551fe30e7/bulkllm-0.5.10.tar.gz",
"platform": null,
"description": "# bulkllm\n\n## Enhancements over vanilla LiteLLM\n\nbulkllm builds on top of `litellm` and adds a few extras:\n\n- **Automatic model registration.** The package knows how to fetch the list of\n models from OpenAI, Anthropic, Gemini and OpenRouter and registers them with\n LiteLLM. Results are cached on disk so they can be reused offline.\n- **Centralised rate limiting.** A `RateLimiter` implementation enforces RPM,\n TPM, input and output token limits per model (or regex group) and works with\n both async and sync code.\n- **Retry\u2011aware completion wrappers.** Thin wrappers around\n `litellm.completion`/`acompletion` integrate Tenacity retries, rate limiting\n and usage tracking.\n- **Usage tracking with statistics.** Per\u2011model usage is tracked in memory with\n histograms, percentiles and cost calculations via the `UsageTracker` and\n `UsageStat` helpers.\n- **Predefined LLM configurations.** A large catalogue of model presets with\n cost information and convenient selection helpers is included.\n\n## Development\n\nAlways run `make checku` before committing.\n\n### Quick Commands\n - `make init` create the environment and install dependencies\n - `make help` see available commands\n - `make autoformat` format code\n - `make autoformat-unsafe` format code - including 'unsafe' fixes\n - `make lint` run linter\n - `make typecheck` run type checker\n - `make test` run tests\n - `make coverage` run tests with coverage report\n - `make check` run all checks (format, lint, typecheck, test)\n - `make checku` run all checks (format-unsafe, lint, typecheck, test)\n\n### Code Conventions\n\n#### Testing\n- Use **pytest** (no test classes).\n- Always set `match=` in `pytest.raises`.\n- Prefer `monkeypatch` over other mocks.\n- Mirror the source-tree layout in `tests/`.\n- Always run `make checku` after making changes.\n\n#### Exceptions\n- Catch only specific exceptions\u2014never blanket `except:` blocks.\n- Don\u2019t raise bare `Exception`.\n\n#### Python\n- Manage env/deps with **uv** (`uv add|remove`, `uv run -- \u2026`).\n- No logging config or side-effects at import time.\n- Keep interfaces (CLI, web, etc.) thin; put logic elsewhere.\n- Use `typer` for CLI interfaces, `fastapi` for web interfaces, \n and `pydantic` for data models.\n",
"bugtrack_url": null,
"license": null,
"summary": null,
"version": "0.5.10",
"project_urls": {
"Documentation": "https://github.com/brycedrennan/bulkllm",
"Source": "https://github.com/brycedrennan/bulkllm"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "af6666ca420a4e7224b1e7a0e8b3fb4ddd3eb15bd2e6e0a9ae74b4c966f20c38",
"md5": "637a897274f0eae68f417ac71fa22224",
"sha256": "4632659335132e28f6d2dfd7b7ef1e5423f89ca30b10ac09b27f88cbac655687"
},
"downloads": -1,
"filename": "bulkllm-0.5.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "637a897274f0eae68f417ac71fa22224",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.12",
"size": 135103,
"upload_time": "2025-08-05T18:41:15",
"upload_time_iso_8601": "2025-08-05T18:41:15.951290Z",
"url": "https://files.pythonhosted.org/packages/af/66/66ca420a4e7224b1e7a0e8b3fb4ddd3eb15bd2e6e0a9ae74b4c966f20c38/bulkllm-0.5.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "f797dd4844f143edfc9fbf0a88536a76decb25867831e9b66bc5056551fe30e7",
"md5": "dfc6b2387b97ff3389f14b086187395d",
"sha256": "8c6aee93999347aa272bc9e3bc5f4ff28a2cc2f58a5f3dea1677193e2c356780"
},
"downloads": -1,
"filename": "bulkllm-0.5.10.tar.gz",
"has_sig": false,
"md5_digest": "dfc6b2387b97ff3389f14b086187395d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.12",
"size": 239024,
"upload_time": "2025-08-05T18:41:18",
"upload_time_iso_8601": "2025-08-05T18:41:18.085678Z",
"url": "https://files.pythonhosted.org/packages/f7/97/dd4844f143edfc9fbf0a88536a76decb25867831e9b66bc5056551fe30e7/bulkllm-0.5.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-05 18:41:18",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "brycedrennan",
"github_project": "bulkllm",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "bulkllm"
}
