free-llm-toolbox


Namefree-llm-toolbox JSON
Version 0.1.8 PyPI version JSON
download
home_pageNone
SummaryA Python toolbox for easy interaction with various LLMs and Vision models
upload_time2025-03-03 14:24:27
maintainerNone
docs_urlNone
authorYanis112
requires_python>=3.10
licenseNone
keywords ai llm machine-learning text-generation vision
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # Free LLM Toolbox 🚀

A Python package that provides easy-to-use utilities for working with various Language Models (LLMs) and Vision Models. 🎯 But everything is free ! (working on generous free plans of some AI platforms)

## Features

- Text generation with multiple LLM providers support
- Image analysis and description capabilities
- Support for models like Llama, Groq, and Google's Gemini
- Streaming responses
- Tool integration support
- JSON output formatting
- Customizable system prompts

## Installation 💻

```bash
uv pip install free-llm-toolbox
```

## Configuration ⚙️

Before using the library, you need to configure your API keys in a `.env` file:

```env
GROQ_API_KEY=your_groq_key
GITHUB_TOKEN=your_github_token
GOOGLE_API_KEY=your_google_key
SAMBANOVA_API_KEY=your_sambanova_key
CEREBRAS_API_KEY=your_cerebras_key
```

## Quick Start

### Text Generation

```python
from free_llm_toolbox import LanguageModel

# Initialize a session with your preferred model
session = LanguageModel(
    model_name="gemini-2.0-flash",
    provider="google",
    temperature=0.7
)

# Generate a response
response = session.answer("What is the capital of France?")
print(response)
```

### Image Analysis

```python
from free_llm_toolbox import ImageAnalyzerAgent

analyzer = ImageAnalyzerAgent()
description = analyzer.describe(
    "path/to/image.jpg",
    prompt="Describe the image",
    vllm_provider="groq",
    vllm_name="llama-3.2-90b-vision-preview"
)
print(description)
```

## Usage 🎮

### Text Models 📚

```python
from free_llm_toolbox import LanguageModel

# Initialize a session with your preferred model
session = LanguageModel(
    model_name="llama-3-70b",
    provider="groq",
    temperature=0.7,
    top_k=45,
    top_p=0.95
)

# Simple text generation
response = session.answer("What is the capital of France?")

# JSON-formatted response with Pydantic validation
from pydantic import BaseModel

class LocationInfo(BaseModel):
    city: str
    country: str
    description: str

response = session.answer(
    "What is the capital of France?",
    json_formatting=True,
    pydantic_object=LocationInfo
)

# Using custom tools
tools = [
    {
        "name": "weather",
        "description": "Get current weather",
        "function": get_weather
    }
]
response, tool_calls = session.answer(
    "What's the weather in Paris?",
    tool_list=tools
)

# Streaming responses
for chunk in session.answer(
    "Tell me a long story.",
    stream=True
):
    print(chunk, end="", flush=True)
```

### Vision Models 👁️

```python
from free_llm_toolbox import ImageAnalyzerAgent

# Initialize the agent
analyzer = ImageAnalyzerAgent()

# Analyze an image
description = analyzer.describe(
    image_path="path/to/image.jpg",
    prompt="Describe this image in detail",
    vllm_provider="groq"
)
print(description)
```

## Available Models 📊

> Note: This list is not exhaustive. The library supports any new model ID released by these providers - you just need to get the correct model ID from your provider's documentation.

### Text Models

| Provider    | Model                           | LLM Provider ID | Model ID                              | Price | Rate Limit (per min) | Context Window | Speed      |
|------------|--------------------------------|----------------|---------------------------------------|-------|---------------------|----------------|------------|
| Google     | Gemini Pro Exp                 | google        | gemini-2.0-pro-exp-02-05             | Free  | 60                  | 32,768        | Ultra Fast |
| Google     | Gemini Flash                   | google        | gemini-2.0-flash                      | Free  | 60                  | 32,768        | Ultra Fast |
| Google     | Gemini Flash Thinking          | google        | gemini-2.0-flash-thinking-exp-01-21   | Free  | 60                  | 32,768        | Ultra Fast |
| Google     | Gemini Flash Lite              | google        | gemini-2.0-flash-lite-preview-02-05   | Free  | 60                  | 32,768        | Ultra Fast |
| GitHub     | O3 Mini                        | github        | o3-mini                               | Free  | 50                  | 8,192         | Fast       |
| GitHub     | GPT-4o                         | github        | gpt-4o                                | Free  | 50                  | 8,192         | Fast       |
| GitHub     | GPT-4o Mini                    | github        | gpt-4o-mini                           | Free  | 50                  | 8,192         | Fast       |
| GitHub     | O1 Mini                        | github        | o1-mini                               | Free  | 50                  | 8,192         | Fast       |
| GitHub     | O1 Preview                     | github        | o1-preview                            | Free  | 50                  | 8,192         | Fast       |
| GitHub     | Meta Llama 3.1 405B            | github        | meta-Llama-3.1-405B-Instruct         | Free  | 50                  | 8,192         | Fast       |
| GitHub     | DeepSeek R1                    | github        | DeepSeek-R1                           | Free  | 50                  | 8,192         | Fast       |
| Groq       | DeepSeek R1 Distill Llama 70B | groq          | deepseek-r1-distill-llama-70b        | Free  | 100                | 131,072       | Ultra Fast |
| Groq       | Llama 3.3 70B Versatile       | groq          | llama-3.3-70b-versatile              | Free  | 100                | 131,072       | Ultra Fast |
| Groq       | Llama 3.1 8B Instant          | groq          | llama-3.1-8b-instant                 | Free  | 100                | 131,072       | Ultra Fast |
| Groq       | Llama 3.2 3B Preview          | groq          | llama-3.2-3b-preview                 | Free  | 100                | 131,072       | Ultra Fast |
| SambaNova  | Llama3 405B                   | sambanova     | llama3-405b                          | Free  | 60                  | 8,000         | Fast       |

### Vision Models

| Provider   | Model                    | Vision Provider ID | Model ID              | Price | Rate Limit (per min) | Speed      |
|-----------|--------------------------|-------------------|----------------------|-------|---------------------|------------|
| Google    | Gemini Vision Exp        | gemini           | gemini-exp-1206      | Free  | 60                | Ultra Fast |
| Google    | Gemini Vision Flash      | gemini           | gemini-2.0-flash     | Free  | 60                | Ultra Fast |
| GitHub    | GPT-4o Vision           | github           | gpt-4o               | Free  | 50                | Fast       |
| GitHub    | GPT-4o Mini Vision      | github           | gpt-4o-mini          | Free  | 50                | Fast       |

### Usage Example with Provider ID and Model ID

```python
from free_llm_toolbox import LanguageModel

# Initialize a session with specific provider and model IDs
session = LanguageModel(
    model_name="llama-3.3-70b-versatile",  # Model ID from the table above
    provider="groq",                        # Provider ID from the table above
    temperature=0.7
)
```

## Requirements

- Python 3.8 or higher
- Required dependencies will be automatically installed

## Key Features ⭐

- Simple and intuitive session-based interface
- Support for both vision and text models
- Simple configuration with .env file
- Automatic context management
- Tool support for compatible models
- JSON output formatting with Pydantic validation
- Response streaming support
- Smart caching system
- CPU and GPU support

## Contributing 🤝

Contributions are welcome! Feel free to:

1. Fork the project
2. Create your feature branch
3. Commit your changes
4. Push to the branch
5. Open a Pull Request

## License 📄

This project is licensed under the MIT License. See the LICENSE file for details.
            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "free-llm-toolbox",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.10",
    "maintainer_email": null,
    "keywords": "ai, llm, machine-learning, text-generation, vision",
    "author": "Yanis112",
    "author_email": null,
    "download_url": "https://files.pythonhosted.org/packages/9b/66/e2d478861280b614ce70bdc4a6e4d2e148b8251af422d2aff6e09fb08d90/free_llm_toolbox-0.1.8.tar.gz",
    "platform": null,
    "description": "# Free LLM Toolbox \ud83d\ude80\n\nA Python package that provides easy-to-use utilities for working with various Language Models (LLMs) and Vision Models. \ud83c\udfaf But everything is free ! (working on generous free plans of some AI platforms)\n\n## Features\n\n- Text generation with multiple LLM providers support\n- Image analysis and description capabilities\n- Support for models like Llama, Groq, and Google's Gemini\n- Streaming responses\n- Tool integration support\n- JSON output formatting\n- Customizable system prompts\n\n## Installation \ud83d\udcbb\n\n```bash\nuv pip install free-llm-toolbox\n```\n\n## Configuration \u2699\ufe0f\n\nBefore using the library, you need to configure your API keys in a `.env` file:\n\n```env\nGROQ_API_KEY=your_groq_key\nGITHUB_TOKEN=your_github_token\nGOOGLE_API_KEY=your_google_key\nSAMBANOVA_API_KEY=your_sambanova_key\nCEREBRAS_API_KEY=your_cerebras_key\n```\n\n## Quick Start\n\n### Text Generation\n\n```python\nfrom free_llm_toolbox import LanguageModel\n\n# Initialize a session with your preferred model\nsession = LanguageModel(\n    model_name=\"gemini-2.0-flash\",\n    provider=\"google\",\n    temperature=0.7\n)\n\n# Generate a response\nresponse = session.answer(\"What is the capital of France?\")\nprint(response)\n```\n\n### Image Analysis\n\n```python\nfrom free_llm_toolbox import ImageAnalyzerAgent\n\nanalyzer = ImageAnalyzerAgent()\ndescription = analyzer.describe(\n    \"path/to/image.jpg\",\n    prompt=\"Describe the image\",\n    vllm_provider=\"groq\",\n    vllm_name=\"llama-3.2-90b-vision-preview\"\n)\nprint(description)\n```\n\n## Usage \ud83c\udfae\n\n### Text Models \ud83d\udcda\n\n```python\nfrom free_llm_toolbox import LanguageModel\n\n# Initialize a session with your preferred model\nsession = LanguageModel(\n    model_name=\"llama-3-70b\",\n    provider=\"groq\",\n    temperature=0.7,\n    top_k=45,\n    top_p=0.95\n)\n\n# Simple text generation\nresponse = session.answer(\"What is the capital of France?\")\n\n# JSON-formatted response with Pydantic validation\nfrom pydantic import BaseModel\n\nclass LocationInfo(BaseModel):\n    city: str\n    country: str\n    description: str\n\nresponse = session.answer(\n    \"What is the capital of France?\",\n    json_formatting=True,\n    pydantic_object=LocationInfo\n)\n\n# Using custom tools\ntools = [\n    {\n        \"name\": \"weather\",\n        \"description\": \"Get current weather\",\n        \"function\": get_weather\n    }\n]\nresponse, tool_calls = session.answer(\n    \"What's the weather in Paris?\",\n    tool_list=tools\n)\n\n# Streaming responses\nfor chunk in session.answer(\n    \"Tell me a long story.\",\n    stream=True\n):\n    print(chunk, end=\"\", flush=True)\n```\n\n### Vision Models \ud83d\udc41\ufe0f\n\n```python\nfrom free_llm_toolbox import ImageAnalyzerAgent\n\n# Initialize the agent\nanalyzer = ImageAnalyzerAgent()\n\n# Analyze an image\ndescription = analyzer.describe(\n    image_path=\"path/to/image.jpg\",\n    prompt=\"Describe this image in detail\",\n    vllm_provider=\"groq\"\n)\nprint(description)\n```\n\n## Available Models \ud83d\udcca\n\n> Note: This list is not exhaustive. The library supports any new model ID released by these providers - you just need to get the correct model ID from your provider's documentation.\n\n### Text Models\n\n| Provider    | Model                           | LLM Provider ID | Model ID                              | Price | Rate Limit (per min) | Context Window | Speed      |\n|------------|--------------------------------|----------------|---------------------------------------|-------|---------------------|----------------|------------|\n| Google     | Gemini Pro Exp                 | google        | gemini-2.0-pro-exp-02-05             | Free  | 60                  | 32,768        | Ultra Fast |\n| Google     | Gemini Flash                   | google        | gemini-2.0-flash                      | Free  | 60                  | 32,768        | Ultra Fast |\n| Google     | Gemini Flash Thinking          | google        | gemini-2.0-flash-thinking-exp-01-21   | Free  | 60                  | 32,768        | Ultra Fast |\n| Google     | Gemini Flash Lite              | google        | gemini-2.0-flash-lite-preview-02-05   | Free  | 60                  | 32,768        | Ultra Fast |\n| GitHub     | O3 Mini                        | github        | o3-mini                               | Free  | 50                  | 8,192         | Fast       |\n| GitHub     | GPT-4o                         | github        | gpt-4o                                | Free  | 50                  | 8,192         | Fast       |\n| GitHub     | GPT-4o Mini                    | github        | gpt-4o-mini                           | Free  | 50                  | 8,192         | Fast       |\n| GitHub     | O1 Mini                        | github        | o1-mini                               | Free  | 50                  | 8,192         | Fast       |\n| GitHub     | O1 Preview                     | github        | o1-preview                            | Free  | 50                  | 8,192         | Fast       |\n| GitHub     | Meta Llama 3.1 405B            | github        | meta-Llama-3.1-405B-Instruct         | Free  | 50                  | 8,192         | Fast       |\n| GitHub     | DeepSeek R1                    | github        | DeepSeek-R1                           | Free  | 50                  | 8,192         | Fast       |\n| Groq       | DeepSeek R1 Distill Llama 70B | groq          | deepseek-r1-distill-llama-70b        | Free  | 100                | 131,072       | Ultra Fast |\n| Groq       | Llama 3.3 70B Versatile       | groq          | llama-3.3-70b-versatile              | Free  | 100                | 131,072       | Ultra Fast |\n| Groq       | Llama 3.1 8B Instant          | groq          | llama-3.1-8b-instant                 | Free  | 100                | 131,072       | Ultra Fast |\n| Groq       | Llama 3.2 3B Preview          | groq          | llama-3.2-3b-preview                 | Free  | 100                | 131,072       | Ultra Fast |\n| SambaNova  | Llama3 405B                   | sambanova     | llama3-405b                          | Free  | 60                  | 8,000         | Fast       |\n\n### Vision Models\n\n| Provider   | Model                    | Vision Provider ID | Model ID              | Price | Rate Limit (per min) | Speed      |\n|-----------|--------------------------|-------------------|----------------------|-------|---------------------|------------|\n| Google    | Gemini Vision Exp        | gemini           | gemini-exp-1206      | Free  | 60                | Ultra Fast |\n| Google    | Gemini Vision Flash      | gemini           | gemini-2.0-flash     | Free  | 60                | Ultra Fast |\n| GitHub    | GPT-4o Vision           | github           | gpt-4o               | Free  | 50                | Fast       |\n| GitHub    | GPT-4o Mini Vision      | github           | gpt-4o-mini          | Free  | 50                | Fast       |\n\n### Usage Example with Provider ID and Model ID\n\n```python\nfrom free_llm_toolbox import LanguageModel\n\n# Initialize a session with specific provider and model IDs\nsession = LanguageModel(\n    model_name=\"llama-3.3-70b-versatile\",  # Model ID from the table above\n    provider=\"groq\",                        # Provider ID from the table above\n    temperature=0.7\n)\n```\n\n## Requirements\n\n- Python 3.8 or higher\n- Required dependencies will be automatically installed\n\n## Key Features \u2b50\n\n- Simple and intuitive session-based interface\n- Support for both vision and text models\n- Simple configuration with .env file\n- Automatic context management\n- Tool support for compatible models\n- JSON output formatting with Pydantic validation\n- Response streaming support\n- Smart caching system\n- CPU and GPU support\n\n## Contributing \ud83e\udd1d\n\nContributions are welcome! Feel free to:\n\n1. Fork the project\n2. Create your feature branch\n3. Commit your changes\n4. Push to the branch\n5. Open a Pull Request\n\n## License \ud83d\udcc4\n\nThis project is licensed under the MIT License. See the LICENSE file for details.",
    "bugtrack_url": null,
    "license": null,
    "summary": "A Python toolbox for easy interaction with various LLMs and Vision models",
    "version": "0.1.8",
    "project_urls": {
        "Homepage": "https://github.com/username/free_llm_toolbox",
        "Repository": "https://github.com/username/free_llm_toolbox"
    },
    "split_keywords": [
        "ai",
        " llm",
        " machine-learning",
        " text-generation",
        " vision"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "b581e31fd6154b61368e7e493a83bbccbfb0c9873e088a38cb829620f7d354af",
                "md5": "d362ad695a6ba9c61eb0167d619b5949",
                "sha256": "0b924ae1051dcb1ced06a605a0cfb45dbe02092fa156e5ed7a227a549a44ad7f"
            },
            "downloads": -1,
            "filename": "free_llm_toolbox-0.1.8-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "d362ad695a6ba9c61eb0167d619b5949",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.10",
            "size": 12080,
            "upload_time": "2025-03-03T14:24:26",
            "upload_time_iso_8601": "2025-03-03T14:24:26.018919Z",
            "url": "https://files.pythonhosted.org/packages/b5/81/e31fd6154b61368e7e493a83bbccbfb0c9873e088a38cb829620f7d354af/free_llm_toolbox-0.1.8-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "9b66e2d478861280b614ce70bdc4a6e4d2e148b8251af422d2aff6e09fb08d90",
                "md5": "628347ff4f93cbe8ea826904c306ed00",
                "sha256": "4c19e7db60edaa6bc559156a62fa91d7e9a667ead502d9ced821d8328bbe6cf3"
            },
            "downloads": -1,
            "filename": "free_llm_toolbox-0.1.8.tar.gz",
            "has_sig": false,
            "md5_digest": "628347ff4f93cbe8ea826904c306ed00",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.10",
            "size": 11193,
            "upload_time": "2025-03-03T14:24:27",
            "upload_time_iso_8601": "2025-03-03T14:24:27.364471Z",
            "url": "https://files.pythonhosted.org/packages/9b/66/e2d478861280b614ce70bdc4a6e4d2e148b8251af422d2aff6e09fb08d90/free_llm_toolbox-0.1.8.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-03-03 14:24:27",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "username",
    "github_project": "free_llm_toolbox",
    "github_not_found": true,
    "lcname": "free-llm-toolbox"
}
        
Elapsed time: 0.61386s