bhumi


Namebhumi JSON
Version 0.1.5 PyPI version JSON
download
home_pagehttps://github.com/yourusername/bhumi
SummaryHigh performance LLM client
upload_time2025-03-01 08:03:43
maintainerNone
docs_urlNone
authorYour Name <your.email@example.com>
requires_python>=3.8
licenseMIT OR Apache-2.0
keywords llm ai groq batch-processing async
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            <p align="center">
  <img src="/assets/bhumi_logo.png" alt="Bhumi Logo" width="1600"/>
</p>

<h1 align="center"><b>Bhumi (เคญเฅ‚เคฎเคฟ)</b></h1>

# ๐ŸŒ **BHUMI - AI Client Setup and Usage Guide** โšก

## **Introduction**
Bhumi (เคญเฅ‚เคฎเคฟ) is the Sanskrit word for **Earth**, symbolizing **stability, grounding, and speed**. Just as the Earth moves with unwavering momentum, **Bhumi AI ensures that your inference speed is as fast as nature itself!** ๐Ÿš€ 

A fast, async Python client for LLM APIs with Rust under the hood.

## Features
- Async support with Rust-powered concurrency
- Connection pooling and retry logic
- Streaming support
- Support for multiple providers:
  - OpenAI
  - Anthropic
  - Google Gemini
  - Groq
  - SambaNova

## Installation
```bash
pip install bhumi
```

## Quick Start

### OpenAI Example
```python
import asyncio
from bhumi.base_client import BaseLLMClient, LLMConfig
import os

api_key = os.getenv("OPENAI_API_KEY")

async def main():
    config = LLMConfig(
        api_key=api_key,
        model="openai/gpt-4o",
        debug=True
    )
    
    client = BaseLLMClient(config)
    
    response = await client.completion([
        {"role": "user", "content": "Tell me a joke"}
    ])
    print(f"Response: {response['text']}")

if __name__ == "__main__":
    asyncio.run(main())
```

### Gemini Example
```python
import asyncio
from bhumi.base_client import BaseLLMClient, LLMConfig
import os

api_key = os.getenv("GEMINI_API_KEY")

async def main():
    config = LLMConfig(
        api_key=api_key,
        model="gemini/gemini-2.0-flash",
        debug=True
    )
    
    client = BaseLLMClient(config)
    
    response = await client.completion([
        {"role": "user", "content": "Tell me a joke"}
    ])
    print(f"Response: {response['text']}")

if __name__ == "__main__":
    asyncio.run(main())
```

### Groq Example
```python
import asyncio
from bhumi.base_client import BaseLLMClient, LLMConfig
import os

api_key = os.getenv("GROQ_API_KEY")

async def main():
    config = LLMConfig(
        api_key=api_key,
        model="groq/llama-3.1-8b-it",
        debug=True
    )
    
    client = BaseLLMClient(config)
    
    response = await client.completion([
        {"role": "user", "content": "Tell me a joke"}
    ])
    print(f"Response: {response['text']}")

if __name__ == "__main__":
    asyncio.run(main())
```

### SambaNova Example
```python
import asyncio
from bhumi.base_client import BaseLLMClient, LLMConfig
import os

api_key = os.getenv("SAMBANOVA_API_KEY")

async def main():
    config = LLMConfig(
        api_key=api_key,
        model="sambanova/Meta-Llama-3.3-70B-Instruct",
        debug=True
    )
    
    client = BaseLLMClient(config)
    
    response = await client.completion([
        {"role": "user", "content": "Tell me a joke"}
    ])
    print(f"Response: {response['text']}")

if __name__ == "__main__":
    asyncio.run(main())
```

## Streaming Support
All providers support streaming responses:

```python
async for chunk in await client.completion([
    {"role": "user", "content": "Write a story"}
], stream=True):
    print(chunk, end="", flush=True)
```

## ๐Ÿ“Š **Benchmark Results**
Our latest benchmarks show significant performance advantages across different metrics:
![alt text](gemini_averaged_comparison_20250131_154711.png)

### โšก Response Time
- LiteLLM: 13.79s
- Native: 5.55s
- Bhumi: 4.26s
- Google GenAI: 6.76s

### ๐Ÿš€ Throughput (Requests/Second)
- LiteLLM: 3.48
- Native: 8.65
- Bhumi: 11.27
- Google GenAI: 7.10

### ๐Ÿ’พ Peak Memory Usage (MB)
- LiteLLM: 275.9MB
- Native: 279.6MB
- Bhumi: 284.3MB
- Google GenAI: 284.8MB

These benchmarks demonstrate Bhumi's superior performance, particularly in throughput where it outperforms other solutions by up to 3.2x.

## Configuration Options
The LLMConfig class supports various options:
- `api_key`: API key for the provider
- `model`: Model name in format "provider/model_name"
- `base_url`: Optional custom base URL
- `max_retries`: Number of retries (default: 3)
- `timeout`: Request timeout in seconds (default: 30)
- `max_tokens`: Maximum tokens in response
- `debug`: Enable debug logging

## ๐ŸŽฏ **Why Use Bhumi?**
โœ” **Open Source:** Apache 2.0 licensed, free for commercial use  
โœ” **Community Driven:** Welcomes contributions from individuals and companies  
โœ” **Blazing Fast:** **2-3x faster** than alternative solutions  
โœ” **Resource Efficient:** Uses **60% less memory** than comparable clients  
โœ” **Multi-Model Support:** Easily switch between providers  
โœ” **Parallel Requests:** Handles **multiple concurrent requests** effortlessly  
โœ” **Flexibility:** Debugging and customization options available  
โœ” **Production Ready:** Battle-tested in high-throughput environments

## ๐Ÿค **Contributing**
We welcome contributions from the community! Whether you're an individual developer or representing a company like Google, OpenAI, or Anthropic, feel free to:

- Submit pull requests
- Report issues
- Suggest improvements
- Share benchmarks
- Integrate our optimizations into your libraries (with attribution)

## ๐Ÿ“œ **License**
Apache 2.0

๐ŸŒŸ **Join our community and help make AI inference faster for everyone!** ๐ŸŒŸ

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/yourusername/bhumi",
    "name": "bhumi",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": null,
    "keywords": "llm, ai, groq, batch-processing, async",
    "author": "Your Name <your.email@example.com>",
    "author_email": "Rach Pradhan <rach@rachpradhan.com>",
    "download_url": "https://files.pythonhosted.org/packages/48/b2/5b762398a27dc7edfbb5a36c3c59491e4f506b44ad9d6641c338e2bc1561/bhumi-0.1.5.tar.gz",
    "platform": null,
    "description": "<p align=\"center\">\n  <img src=\"/assets/bhumi_logo.png\" alt=\"Bhumi Logo\" width=\"1600\"/>\n</p>\n\n<h1 align=\"center\"><b>Bhumi (\u092d\u0942\u092e\u093f)</b></h1>\n\n# \ud83c\udf0d **BHUMI - AI Client Setup and Usage Guide** \u26a1\n\n## **Introduction**\nBhumi (\u092d\u0942\u092e\u093f) is the Sanskrit word for **Earth**, symbolizing **stability, grounding, and speed**. Just as the Earth moves with unwavering momentum, **Bhumi AI ensures that your inference speed is as fast as nature itself!** \ud83d\ude80 \n\nA fast, async Python client for LLM APIs with Rust under the hood.\n\n## Features\n- Async support with Rust-powered concurrency\n- Connection pooling and retry logic\n- Streaming support\n- Support for multiple providers:\n  - OpenAI\n  - Anthropic\n  - Google Gemini\n  - Groq\n  - SambaNova\n\n## Installation\n```bash\npip install bhumi\n```\n\n## Quick Start\n\n### OpenAI Example\n```python\nimport asyncio\nfrom bhumi.base_client import BaseLLMClient, LLMConfig\nimport os\n\napi_key = os.getenv(\"OPENAI_API_KEY\")\n\nasync def main():\n    config = LLMConfig(\n        api_key=api_key,\n        model=\"openai/gpt-4o\",\n        debug=True\n    )\n    \n    client = BaseLLMClient(config)\n    \n    response = await client.completion([\n        {\"role\": \"user\", \"content\": \"Tell me a joke\"}\n    ])\n    print(f\"Response: {response['text']}\")\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n```\n\n### Gemini Example\n```python\nimport asyncio\nfrom bhumi.base_client import BaseLLMClient, LLMConfig\nimport os\n\napi_key = os.getenv(\"GEMINI_API_KEY\")\n\nasync def main():\n    config = LLMConfig(\n        api_key=api_key,\n        model=\"gemini/gemini-2.0-flash\",\n        debug=True\n    )\n    \n    client = BaseLLMClient(config)\n    \n    response = await client.completion([\n        {\"role\": \"user\", \"content\": \"Tell me a joke\"}\n    ])\n    print(f\"Response: {response['text']}\")\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n```\n\n### Groq Example\n```python\nimport asyncio\nfrom bhumi.base_client import BaseLLMClient, LLMConfig\nimport os\n\napi_key = os.getenv(\"GROQ_API_KEY\")\n\nasync def main():\n    config = LLMConfig(\n        api_key=api_key,\n        model=\"groq/llama-3.1-8b-it\",\n        debug=True\n    )\n    \n    client = BaseLLMClient(config)\n    \n    response = await client.completion([\n        {\"role\": \"user\", \"content\": \"Tell me a joke\"}\n    ])\n    print(f\"Response: {response['text']}\")\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n```\n\n### SambaNova Example\n```python\nimport asyncio\nfrom bhumi.base_client import BaseLLMClient, LLMConfig\nimport os\n\napi_key = os.getenv(\"SAMBANOVA_API_KEY\")\n\nasync def main():\n    config = LLMConfig(\n        api_key=api_key,\n        model=\"sambanova/Meta-Llama-3.3-70B-Instruct\",\n        debug=True\n    )\n    \n    client = BaseLLMClient(config)\n    \n    response = await client.completion([\n        {\"role\": \"user\", \"content\": \"Tell me a joke\"}\n    ])\n    print(f\"Response: {response['text']}\")\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n```\n\n## Streaming Support\nAll providers support streaming responses:\n\n```python\nasync for chunk in await client.completion([\n    {\"role\": \"user\", \"content\": \"Write a story\"}\n], stream=True):\n    print(chunk, end=\"\", flush=True)\n```\n\n## \ud83d\udcca **Benchmark Results**\nOur latest benchmarks show significant performance advantages across different metrics:\n![alt text](gemini_averaged_comparison_20250131_154711.png)\n\n### \u26a1 Response Time\n- LiteLLM: 13.79s\n- Native: 5.55s\n- Bhumi: 4.26s\n- Google GenAI: 6.76s\n\n### \ud83d\ude80 Throughput (Requests/Second)\n- LiteLLM: 3.48\n- Native: 8.65\n- Bhumi: 11.27\n- Google GenAI: 7.10\n\n### \ud83d\udcbe Peak Memory Usage (MB)\n- LiteLLM: 275.9MB\n- Native: 279.6MB\n- Bhumi: 284.3MB\n- Google GenAI: 284.8MB\n\nThese benchmarks demonstrate Bhumi's superior performance, particularly in throughput where it outperforms other solutions by up to 3.2x.\n\n## Configuration Options\nThe LLMConfig class supports various options:\n- `api_key`: API key for the provider\n- `model`: Model name in format \"provider/model_name\"\n- `base_url`: Optional custom base URL\n- `max_retries`: Number of retries (default: 3)\n- `timeout`: Request timeout in seconds (default: 30)\n- `max_tokens`: Maximum tokens in response\n- `debug`: Enable debug logging\n\n## \ud83c\udfaf **Why Use Bhumi?**\n\u2714 **Open Source:** Apache 2.0 licensed, free for commercial use  \n\u2714 **Community Driven:** Welcomes contributions from individuals and companies  \n\u2714 **Blazing Fast:** **2-3x faster** than alternative solutions  \n\u2714 **Resource Efficient:** Uses **60% less memory** than comparable clients  \n\u2714 **Multi-Model Support:** Easily switch between providers  \n\u2714 **Parallel Requests:** Handles **multiple concurrent requests** effortlessly  \n\u2714 **Flexibility:** Debugging and customization options available  \n\u2714 **Production Ready:** Battle-tested in high-throughput environments\n\n## \ud83e\udd1d **Contributing**\nWe welcome contributions from the community! Whether you're an individual developer or representing a company like Google, OpenAI, or Anthropic, feel free to:\n\n- Submit pull requests\n- Report issues\n- Suggest improvements\n- Share benchmarks\n- Integrate our optimizations into your libraries (with attribution)\n\n## \ud83d\udcdc **License**\nApache 2.0\n\n\ud83c\udf1f **Join our community and help make AI inference faster for everyone!** \ud83c\udf1f\n",
    "bugtrack_url": null,
    "license": "MIT OR Apache-2.0",
    "summary": "High performance LLM client",
    "version": "0.1.5",
    "project_urls": {
        "Homepage": "https://github.com/yourusername/bhumi",
        "Source Code": "https://github.com/yourusername/bhumi"
    },
    "split_keywords": [
        "llm",
        " ai",
        " groq",
        " batch-processing",
        " async"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "dfcd84d8c06026add26cfb884ff923d1aa9f6ee7b25b4e08c446fcf205afabbf",
                "md5": "da4db07e44ee4ac99525f6bfb31ba7b8",
                "sha256": "47d3a4588141cf29a9198e1c5fe3bb34f2c955837c805b49a7dc7f016b35d243"
            },
            "downloads": -1,
            "filename": "bhumi-0.1.5-cp38-abi3-macosx_11_0_arm64.whl",
            "has_sig": false,
            "md5_digest": "da4db07e44ee4ac99525f6bfb31ba7b8",
            "packagetype": "bdist_wheel",
            "python_version": "cp38",
            "requires_python": ">=3.8",
            "size": 1458506,
            "upload_time": "2025-03-01T08:03:41",
            "upload_time_iso_8601": "2025-03-01T08:03:41.511461Z",
            "url": "https://files.pythonhosted.org/packages/df/cd/84d8c06026add26cfb884ff923d1aa9f6ee7b25b4e08c446fcf205afabbf/bhumi-0.1.5-cp38-abi3-macosx_11_0_arm64.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "48b25b762398a27dc7edfbb5a36c3c59491e4f506b44ad9d6641c338e2bc1561",
                "md5": "ef2e668931306797291fc2299536e2a9",
                "sha256": "7949d208ae407d99acd7072296528215b882dc343fe67a584d139228cf63d172"
            },
            "downloads": -1,
            "filename": "bhumi-0.1.5.tar.gz",
            "has_sig": false,
            "md5_digest": "ef2e668931306797291fc2299536e2a9",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 43090,
            "upload_time": "2025-03-01T08:03:43",
            "upload_time_iso_8601": "2025-03-01T08:03:43.162274Z",
            "url": "https://files.pythonhosted.org/packages/48/b2/5b762398a27dc7edfbb5a36c3c59491e4f506b44ad9d6641c338e2bc1561/bhumi-0.1.5.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-03-01 08:03:43",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "yourusername",
    "github_project": "bhumi",
    "github_not_found": true,
    "lcname": "bhumi"
}
        
Elapsed time: 1.20551s