smartllm


Namesmartllm JSON
Version 0.0.6 PyPI version JSON
download
home_pageNone
SummaryA unified interface for interacting with multiple Large Language Model providers
upload_time2025-03-21 20:24:03
maintainerNone
docs_urlNone
authorNone
requires_python>=3.8
licenseMIT
keywords llm openai anthropic perplexity nlp ai
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # SmartLLM

SmartLLM is a unified Python interface for interacting with multiple Large Language Model providers. It provides a consistent API across different LLM providers, handles caching of responses, and supports both synchronous and asynchronous interactions.

## Installation

```bash
pip install smartllm
```

## Features

- **Unified API**: Consistent interface for OpenAI, Anthropic, and Perplexity LLMs
- **Response Caching**: Persistent JSON-based caching of responses to improve performance
- **Streaming Support**: Real-time streaming of LLM responses (Anthropic only)
- **JSON Mode**: Structured JSON responses (OpenAI and Anthropic)
- **Citations**: Access to source information (Perplexity only)
- **Asynchronous Execution**: Non-blocking request execution via AsyncSmartLLM
- **Configurable Parameters**: Granular control over temperature, tokens, and other model parameters

## Supported Providers

SmartLLM currently supports the following LLM providers:

- **OpenAI**
  - Models: GPT-4, GPT-3.5 series, and other OpenAI models
  - Features: JSON-structured outputs, token usage information
  - Example: `base="openai", model="gpt-4"`

- **Anthropic**
  - Models: Claude models (e.g., claude-3-7-sonnet-20250219)
  - Features: Streaming support, JSON-structured outputs, system prompts
  - Example: `base="anthropic", model="claude-3-7-sonnet-20250219"`

- **Perplexity**
  - Models: sonar-small-online, sonar-medium-online, sonar-pro, etc.
  - Features: Web search capabilities, citation information
  - Example: `base="perplexity", model="sonar-pro"`

## Basic Usage

```python
from smartllm import SmartLLM
import os

# Create SmartLLM instance
llm = SmartLLM(
    base="openai",
    model="gpt-4",
    api_key=os.environ.get("OPENAI_API_KEY"),
    prompt="Explain quantum computing in simple terms",
    temperature=0.7
)

# Execute the request
llm.execute()

# Wait for completion
llm.wait_for_completion()

# Check status and get response
if llm.is_completed():
    print(llm.response)
else:
    print(f"Error: {llm.get_error()}")
```

## AsyncSmartLLM - Asynchronous Interface

SmartLLM also provides an asynchronous interface for non-blocking API interactions using AsyncSmartLLM:

```python
import asyncio
import os
from smartllm import AsyncSmartLLM

async def main():
    # Create AsyncSmartLLM instance
    llm = AsyncSmartLLM(
        base="anthropic",
        model="claude-3-7-sonnet-20250219",
        api_key=os.environ.get("ANTHROPIC_API_KEY"),
        prompt="Explain quantum computing in simple terms",
        temperature=0.7
    )

    # Execute the request asynchronously
    await llm.execute()

    # Check status and get response
    if llm.is_completed():
        print(llm.response)
    else:
        print(f"Error: {llm.get_error()}")

# Run the async function
asyncio.run(main())
```

### Async Streaming Support

AsyncSmartLLM also supports streaming responses:

```python
import asyncio
import os
from smartllm import AsyncSmartLLM

# Custom callback for handling streaming chunks
async def print_chunk(chunk: str, accumulated: str) -> None:
    print(chunk, end="", flush=True)

async def main():
    api_key = os.environ.get("ANTHROPIC_API_KEY")
    
    # Enable streaming with stream=True
    llm = AsyncSmartLLM(
        base="anthropic", 
        model="claude-3-7-sonnet-20250219", 
        api_key=api_key,
        prompt="Tell me a short story about a robot learning to paint",
        temperature=0.7, 
        max_output_tokens=1000,
        stream=True
    )

    # Execute with callback
    await llm.execute(callback=print_chunk)
    
    if llm.is_failed():
        print(f"\nError occurred: {llm.get_error()}")
    else:
        print("\n\nFinal response:")
        print(llm.response)

if __name__ == "__main__":
    asyncio.run(main())
```

### Async JSON Mode

```python
import asyncio
import os
from smartllm import AsyncSmartLLM

async def main():
    api_key = os.environ.get("ANTHROPIC_API_KEY")

    json_schema = {
        "type": "object",
        "properties": {
            "title": {"type": "string"},
            "topics": {"type": "array", "items": {"type": "string"}},
            "difficulty": {"type": "integer", "minimum": 1, "maximum": 10}
        },
        "required": ["title", "topics", "difficulty"]
    }

    llm = AsyncSmartLLM(
        base="anthropic",
        model="claude-3-7-sonnet-20250219",
        api_key=api_key,
        prompt="Generate information about a quantum computing course",
        json_mode=True,
        json_schema=json_schema
    )

    await llm.execute()

    # Access structured data
    course_info = llm.response  # Returns a Python dictionary
    print(f"Course title: {course_info['title']}")
    print(f"Topics: {', '.join(course_info['topics'])}")
    print(f"Difficulty: {course_info['difficulty']}/10")

if __name__ == "__main__":
    asyncio.run(main())
```

## SmartLLM Class Reference

### Constructor

```python
SmartLLM(
    base: str = "",                  # LLM provider ("openai", "anthropic", "perplexity")
    model: str = "",                 # Model identifier
    api_key: str = "",               # API key for the provider
    prompt: Union[str, List[str]] = "", # Single prompt or conversation history
    stream: bool = False,            # Enable streaming (Anthropic only)
    max_input_tokens: Optional[int] = None,  # Max input tokens
    max_output_tokens: Optional[int] = None, # Max output tokens
    output_type: str = "text",       # Output type
    temperature: float = 0.2,        # Temperature for generation
    top_p: float = 0.9,              # Top-p sampling parameter
    frequency_penalty: float = 1.0,  # Frequency penalty
    presence_penalty: float = 0.0,   # Presence penalty
    system_prompt: Optional[str] = None, # System prompt
    search_recency_filter: Optional[str] = None, # Filter for search (Perplexity)
    return_citations: bool = False,  # Include citations (Perplexity)
    json_mode: bool = False,         # Enable JSON mode (OpenAI, Anthropic)
    json_schema: Optional[Dict[str, Any]] = None, # JSON schema
    ttl: int = 7,                    # Cache time-to-live in days
    clear_cache: bool = False        # Clear existing cache
)
```

### AsyncSmartLLM Class Reference

```python
AsyncSmartLLM(
    # Same parameters as SmartLLM above
)
```

### Methods

#### SmartLLM Methods
```python
execute(callback: Optional[Callable[[str, str], None]] = None) -> SmartLLM
wait_for_completion(timeout: Optional[float] = None) -> bool
is_failed() -> bool
is_completed() -> bool
get_error() -> Optional[str]
```

#### AsyncSmartLLM Methods
```python
async execute(callback: Optional[Callable[[str, str], None]] = None) -> AsyncSmartLLM
async generate() -> AsyncSmartLLM
is_failed() -> bool
is_completed() -> bool
get_error() -> Optional[str]
```

### Properties

```python
response: Union[str, Dict[str, Any]]  # The response content or JSON
sources: List[str]  # Citation sources (Perplexity)
usage: Dict[str, int]  # Token usage statistics
```

## Streaming Responses (Anthropic Only)

### Synchronous Streaming

```python
from smartllm import SmartLLM
import os

def print_chunk(chunk: str, accumulated: str) -> None:
    print(f"CHUNK: {chunk}")

llm = SmartLLM(
    base="anthropic",
    model="claude-3-7-sonnet-20250219",
    api_key=os.environ.get("ANTHROPIC_API_KEY"),
    prompt="Write a short story about a robot learning to paint",
    stream=True  # Enable streaming
)

# Execute with callback
llm.execute(callback=print_chunk)
llm.wait_for_completion()
```

### Asynchronous Streaming

```python
import asyncio
from smartllm import AsyncSmartLLM
import os

async def print_chunk(chunk: str, accumulated: str) -> None:
    print(chunk, end="", flush=True)

async def main():
    llm = AsyncSmartLLM(
        base="anthropic",
        model="claude-3-7-sonnet-20250219",
        api_key=os.environ.get("ANTHROPIC_API_KEY"),
        prompt="Write a short story about a robot learning to paint",
        stream=True  # Enable streaming
    )

    # Execute with callback
    await llm.execute(callback=print_chunk)
    
    print("\n\nFinal response:")
    print(llm.response)

asyncio.run(main())
```

## JSON Mode (OpenAI and Anthropic)

```python
from smartllm import SmartLLM
import os

json_schema = {
    "type": "object",
    "properties": {
        "title": {"type": "string"},
        "topics": {"type": "array", "items": {"type": "string"}},
        "difficulty": {"type": "integer", "minimum": 1, "maximum": 10}
    },
    "required": ["title", "topics", "difficulty"]
}

llm = SmartLLM(
    base="openai",
    model="gpt-4",
    api_key=os.environ.get("OPENAI_API_KEY"),
    prompt="Generate information about a quantum computing course",
    json_mode=True,
    json_schema=json_schema
)

llm.execute()
llm.wait_for_completion()

# Access structured data
course_info = llm.response  # Returns a Python dictionary
print(f"Course title: {course_info['title']}")
print(f"Topics: {', '.join(course_info['topics'])}")
print(f"Difficulty: {course_info['difficulty']}/10")
```

## Getting Citations (Perplexity Only)

```python
from smartllm import SmartLLM
import os

llm = SmartLLM(
    base="perplexity",
    model="sonar-pro",
    api_key=os.environ.get("PERPLEXITY_API_KEY"),
    prompt="What are the latest advancements in quantum computing?",
    search_recency_filter="week",  # Filter for recent information
    return_citations=True  # Enable citations
)

llm.execute()
llm.wait_for_completion()

# Print the response
print(llm.response)

# Print the sources
print("\nSources:")
for source in llm.sources:
    print(f"- {source}")
```

## Caching Mechanism

SmartLLM uses a persistent JSON-based caching system powered by the Cacherator library. This significantly improves performance by avoiding redundant API calls for identical requests.

### Cache Configuration

By default, responses are cached for 7 days. You can customize the cache behavior:

```python
# Set custom time-to-live (TTL) in days
llm = SmartLLM(
    base="openai",
    model="gpt-4",
    api_key=os.environ.get("OPENAI_API_KEY"),
    prompt="Explain quantum computing",
    ttl=30  # Cache results for 30 days
)

# Force clear existing cache
llm = SmartLLM(
    base="openai",
    model="gpt-4",
    api_key=os.environ.get("OPENAI_API_KEY"),
    prompt="Explain quantum computing",
    clear_cache=True  # Ignore any existing cached response
)
```

### How Caching Works

1. Each request is assigned a unique identifier based on:
   - Provider (`base`)
   - Model
   - Prompt
   - All relevant parameters (temperature, tokens, etc.)

2. Responses are stored in JSON format in the `data/llm` directory

3. When making an identical request, the cached response is returned instead of making a new API call

4. Cache entries automatically expire after the specified TTL

5. Cache can be manually cleared by setting `clear_cache=True`

## Error Handling

SmartLLM provides robust error handling through state tracking:

```python
llm = SmartLLM(...)
llm.execute()
llm.wait_for_completion()

if llm.is_failed():
    print(f"Request failed: {llm.get_error()}")
elif llm.is_completed():
    print("Request completed successfully")
    print(llm.response)
```

For AsyncSmartLLM:

```python
llm = AsyncSmartLLM(...)
await llm.execute()

if llm.is_failed():
    print(f"Request failed: {llm.get_error()}")
elif llm.is_completed():
    print("Request completed successfully")
    print(llm.response)
```

## Dependencies

- `cacherator`: Persistent JSON-based caching
- `logorator`: Decorator-based logging
- `openai>=1.0.0`: OpenAI API client
- `anthropic>=0.5.0`: Anthropic API client
- `python-slugify`: Utility for creating safe identifiers

## License

MIT License

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "smartllm",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": null,
    "keywords": "llm, openai, anthropic, perplexity, nlp, ai",
    "author": null,
    "author_email": "Arved Kl\u00f6hn <arved.kloehn@gmail.com>",
    "download_url": "https://files.pythonhosted.org/packages/cb/06/c2ec0a62f5e6f4d1f4cfe77c061aab3a2992cb2bfe4e39439192f8fa9256/smartllm-0.0.6.tar.gz",
    "platform": null,
    "description": "# SmartLLM\r\n\r\nSmartLLM is a unified Python interface for interacting with multiple Large Language Model providers. It provides a consistent API across different LLM providers, handles caching of responses, and supports both synchronous and asynchronous interactions.\r\n\r\n## Installation\r\n\r\n```bash\r\npip install smartllm\r\n```\r\n\r\n## Features\r\n\r\n- **Unified API**: Consistent interface for OpenAI, Anthropic, and Perplexity LLMs\r\n- **Response Caching**: Persistent JSON-based caching of responses to improve performance\r\n- **Streaming Support**: Real-time streaming of LLM responses (Anthropic only)\r\n- **JSON Mode**: Structured JSON responses (OpenAI and Anthropic)\r\n- **Citations**: Access to source information (Perplexity only)\r\n- **Asynchronous Execution**: Non-blocking request execution via AsyncSmartLLM\r\n- **Configurable Parameters**: Granular control over temperature, tokens, and other model parameters\r\n\r\n## Supported Providers\r\n\r\nSmartLLM currently supports the following LLM providers:\r\n\r\n- **OpenAI**\r\n  - Models: GPT-4, GPT-3.5 series, and other OpenAI models\r\n  - Features: JSON-structured outputs, token usage information\r\n  - Example: `base=\"openai\", model=\"gpt-4\"`\r\n\r\n- **Anthropic**\r\n  - Models: Claude models (e.g., claude-3-7-sonnet-20250219)\r\n  - Features: Streaming support, JSON-structured outputs, system prompts\r\n  - Example: `base=\"anthropic\", model=\"claude-3-7-sonnet-20250219\"`\r\n\r\n- **Perplexity**\r\n  - Models: sonar-small-online, sonar-medium-online, sonar-pro, etc.\r\n  - Features: Web search capabilities, citation information\r\n  - Example: `base=\"perplexity\", model=\"sonar-pro\"`\r\n\r\n## Basic Usage\r\n\r\n```python\r\nfrom smartllm import SmartLLM\r\nimport os\r\n\r\n# Create SmartLLM instance\r\nllm = SmartLLM(\r\n    base=\"openai\",\r\n    model=\"gpt-4\",\r\n    api_key=os.environ.get(\"OPENAI_API_KEY\"),\r\n    prompt=\"Explain quantum computing in simple terms\",\r\n    temperature=0.7\r\n)\r\n\r\n# Execute the request\r\nllm.execute()\r\n\r\n# Wait for completion\r\nllm.wait_for_completion()\r\n\r\n# Check status and get response\r\nif llm.is_completed():\r\n    print(llm.response)\r\nelse:\r\n    print(f\"Error: {llm.get_error()}\")\r\n```\r\n\r\n## AsyncSmartLLM - Asynchronous Interface\r\n\r\nSmartLLM also provides an asynchronous interface for non-blocking API interactions using AsyncSmartLLM:\r\n\r\n```python\r\nimport asyncio\r\nimport os\r\nfrom smartllm import AsyncSmartLLM\r\n\r\nasync def main():\r\n    # Create AsyncSmartLLM instance\r\n    llm = AsyncSmartLLM(\r\n        base=\"anthropic\",\r\n        model=\"claude-3-7-sonnet-20250219\",\r\n        api_key=os.environ.get(\"ANTHROPIC_API_KEY\"),\r\n        prompt=\"Explain quantum computing in simple terms\",\r\n        temperature=0.7\r\n    )\r\n\r\n    # Execute the request asynchronously\r\n    await llm.execute()\r\n\r\n    # Check status and get response\r\n    if llm.is_completed():\r\n        print(llm.response)\r\n    else:\r\n        print(f\"Error: {llm.get_error()}\")\r\n\r\n# Run the async function\r\nasyncio.run(main())\r\n```\r\n\r\n### Async Streaming Support\r\n\r\nAsyncSmartLLM also supports streaming responses:\r\n\r\n```python\r\nimport asyncio\r\nimport os\r\nfrom smartllm import AsyncSmartLLM\r\n\r\n# Custom callback for handling streaming chunks\r\nasync def print_chunk(chunk: str, accumulated: str) -> None:\r\n    print(chunk, end=\"\", flush=True)\r\n\r\nasync def main():\r\n    api_key = os.environ.get(\"ANTHROPIC_API_KEY\")\r\n    \r\n    # Enable streaming with stream=True\r\n    llm = AsyncSmartLLM(\r\n        base=\"anthropic\", \r\n        model=\"claude-3-7-sonnet-20250219\", \r\n        api_key=api_key,\r\n        prompt=\"Tell me a short story about a robot learning to paint\",\r\n        temperature=0.7, \r\n        max_output_tokens=1000,\r\n        stream=True\r\n    )\r\n\r\n    # Execute with callback\r\n    await llm.execute(callback=print_chunk)\r\n    \r\n    if llm.is_failed():\r\n        print(f\"\\nError occurred: {llm.get_error()}\")\r\n    else:\r\n        print(\"\\n\\nFinal response:\")\r\n        print(llm.response)\r\n\r\nif __name__ == \"__main__\":\r\n    asyncio.run(main())\r\n```\r\n\r\n### Async JSON Mode\r\n\r\n```python\r\nimport asyncio\r\nimport os\r\nfrom smartllm import AsyncSmartLLM\r\n\r\nasync def main():\r\n    api_key = os.environ.get(\"ANTHROPIC_API_KEY\")\r\n\r\n    json_schema = {\r\n        \"type\": \"object\",\r\n        \"properties\": {\r\n            \"title\": {\"type\": \"string\"},\r\n            \"topics\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}},\r\n            \"difficulty\": {\"type\": \"integer\", \"minimum\": 1, \"maximum\": 10}\r\n        },\r\n        \"required\": [\"title\", \"topics\", \"difficulty\"]\r\n    }\r\n\r\n    llm = AsyncSmartLLM(\r\n        base=\"anthropic\",\r\n        model=\"claude-3-7-sonnet-20250219\",\r\n        api_key=api_key,\r\n        prompt=\"Generate information about a quantum computing course\",\r\n        json_mode=True,\r\n        json_schema=json_schema\r\n    )\r\n\r\n    await llm.execute()\r\n\r\n    # Access structured data\r\n    course_info = llm.response  # Returns a Python dictionary\r\n    print(f\"Course title: {course_info['title']}\")\r\n    print(f\"Topics: {', '.join(course_info['topics'])}\")\r\n    print(f\"Difficulty: {course_info['difficulty']}/10\")\r\n\r\nif __name__ == \"__main__\":\r\n    asyncio.run(main())\r\n```\r\n\r\n## SmartLLM Class Reference\r\n\r\n### Constructor\r\n\r\n```python\r\nSmartLLM(\r\n    base: str = \"\",                  # LLM provider (\"openai\", \"anthropic\", \"perplexity\")\r\n    model: str = \"\",                 # Model identifier\r\n    api_key: str = \"\",               # API key for the provider\r\n    prompt: Union[str, List[str]] = \"\", # Single prompt or conversation history\r\n    stream: bool = False,            # Enable streaming (Anthropic only)\r\n    max_input_tokens: Optional[int] = None,  # Max input tokens\r\n    max_output_tokens: Optional[int] = None, # Max output tokens\r\n    output_type: str = \"text\",       # Output type\r\n    temperature: float = 0.2,        # Temperature for generation\r\n    top_p: float = 0.9,              # Top-p sampling parameter\r\n    frequency_penalty: float = 1.0,  # Frequency penalty\r\n    presence_penalty: float = 0.0,   # Presence penalty\r\n    system_prompt: Optional[str] = None, # System prompt\r\n    search_recency_filter: Optional[str] = None, # Filter for search (Perplexity)\r\n    return_citations: bool = False,  # Include citations (Perplexity)\r\n    json_mode: bool = False,         # Enable JSON mode (OpenAI, Anthropic)\r\n    json_schema: Optional[Dict[str, Any]] = None, # JSON schema\r\n    ttl: int = 7,                    # Cache time-to-live in days\r\n    clear_cache: bool = False        # Clear existing cache\r\n)\r\n```\r\n\r\n### AsyncSmartLLM Class Reference\r\n\r\n```python\r\nAsyncSmartLLM(\r\n    # Same parameters as SmartLLM above\r\n)\r\n```\r\n\r\n### Methods\r\n\r\n#### SmartLLM Methods\r\n```python\r\nexecute(callback: Optional[Callable[[str, str], None]] = None) -> SmartLLM\r\nwait_for_completion(timeout: Optional[float] = None) -> bool\r\nis_failed() -> bool\r\nis_completed() -> bool\r\nget_error() -> Optional[str]\r\n```\r\n\r\n#### AsyncSmartLLM Methods\r\n```python\r\nasync execute(callback: Optional[Callable[[str, str], None]] = None) -> AsyncSmartLLM\r\nasync generate() -> AsyncSmartLLM\r\nis_failed() -> bool\r\nis_completed() -> bool\r\nget_error() -> Optional[str]\r\n```\r\n\r\n### Properties\r\n\r\n```python\r\nresponse: Union[str, Dict[str, Any]]  # The response content or JSON\r\nsources: List[str]  # Citation sources (Perplexity)\r\nusage: Dict[str, int]  # Token usage statistics\r\n```\r\n\r\n## Streaming Responses (Anthropic Only)\r\n\r\n### Synchronous Streaming\r\n\r\n```python\r\nfrom smartllm import SmartLLM\r\nimport os\r\n\r\ndef print_chunk(chunk: str, accumulated: str) -> None:\r\n    print(f\"CHUNK: {chunk}\")\r\n\r\nllm = SmartLLM(\r\n    base=\"anthropic\",\r\n    model=\"claude-3-7-sonnet-20250219\",\r\n    api_key=os.environ.get(\"ANTHROPIC_API_KEY\"),\r\n    prompt=\"Write a short story about a robot learning to paint\",\r\n    stream=True  # Enable streaming\r\n)\r\n\r\n# Execute with callback\r\nllm.execute(callback=print_chunk)\r\nllm.wait_for_completion()\r\n```\r\n\r\n### Asynchronous Streaming\r\n\r\n```python\r\nimport asyncio\r\nfrom smartllm import AsyncSmartLLM\r\nimport os\r\n\r\nasync def print_chunk(chunk: str, accumulated: str) -> None:\r\n    print(chunk, end=\"\", flush=True)\r\n\r\nasync def main():\r\n    llm = AsyncSmartLLM(\r\n        base=\"anthropic\",\r\n        model=\"claude-3-7-sonnet-20250219\",\r\n        api_key=os.environ.get(\"ANTHROPIC_API_KEY\"),\r\n        prompt=\"Write a short story about a robot learning to paint\",\r\n        stream=True  # Enable streaming\r\n    )\r\n\r\n    # Execute with callback\r\n    await llm.execute(callback=print_chunk)\r\n    \r\n    print(\"\\n\\nFinal response:\")\r\n    print(llm.response)\r\n\r\nasyncio.run(main())\r\n```\r\n\r\n## JSON Mode (OpenAI and Anthropic)\r\n\r\n```python\r\nfrom smartllm import SmartLLM\r\nimport os\r\n\r\njson_schema = {\r\n    \"type\": \"object\",\r\n    \"properties\": {\r\n        \"title\": {\"type\": \"string\"},\r\n        \"topics\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}},\r\n        \"difficulty\": {\"type\": \"integer\", \"minimum\": 1, \"maximum\": 10}\r\n    },\r\n    \"required\": [\"title\", \"topics\", \"difficulty\"]\r\n}\r\n\r\nllm = SmartLLM(\r\n    base=\"openai\",\r\n    model=\"gpt-4\",\r\n    api_key=os.environ.get(\"OPENAI_API_KEY\"),\r\n    prompt=\"Generate information about a quantum computing course\",\r\n    json_mode=True,\r\n    json_schema=json_schema\r\n)\r\n\r\nllm.execute()\r\nllm.wait_for_completion()\r\n\r\n# Access structured data\r\ncourse_info = llm.response  # Returns a Python dictionary\r\nprint(f\"Course title: {course_info['title']}\")\r\nprint(f\"Topics: {', '.join(course_info['topics'])}\")\r\nprint(f\"Difficulty: {course_info['difficulty']}/10\")\r\n```\r\n\r\n## Getting Citations (Perplexity Only)\r\n\r\n```python\r\nfrom smartllm import SmartLLM\r\nimport os\r\n\r\nllm = SmartLLM(\r\n    base=\"perplexity\",\r\n    model=\"sonar-pro\",\r\n    api_key=os.environ.get(\"PERPLEXITY_API_KEY\"),\r\n    prompt=\"What are the latest advancements in quantum computing?\",\r\n    search_recency_filter=\"week\",  # Filter for recent information\r\n    return_citations=True  # Enable citations\r\n)\r\n\r\nllm.execute()\r\nllm.wait_for_completion()\r\n\r\n# Print the response\r\nprint(llm.response)\r\n\r\n# Print the sources\r\nprint(\"\\nSources:\")\r\nfor source in llm.sources:\r\n    print(f\"- {source}\")\r\n```\r\n\r\n## Caching Mechanism\r\n\r\nSmartLLM uses a persistent JSON-based caching system powered by the Cacherator library. This significantly improves performance by avoiding redundant API calls for identical requests.\r\n\r\n### Cache Configuration\r\n\r\nBy default, responses are cached for 7 days. You can customize the cache behavior:\r\n\r\n```python\r\n# Set custom time-to-live (TTL) in days\r\nllm = SmartLLM(\r\n    base=\"openai\",\r\n    model=\"gpt-4\",\r\n    api_key=os.environ.get(\"OPENAI_API_KEY\"),\r\n    prompt=\"Explain quantum computing\",\r\n    ttl=30  # Cache results for 30 days\r\n)\r\n\r\n# Force clear existing cache\r\nllm = SmartLLM(\r\n    base=\"openai\",\r\n    model=\"gpt-4\",\r\n    api_key=os.environ.get(\"OPENAI_API_KEY\"),\r\n    prompt=\"Explain quantum computing\",\r\n    clear_cache=True  # Ignore any existing cached response\r\n)\r\n```\r\n\r\n### How Caching Works\r\n\r\n1. Each request is assigned a unique identifier based on:\r\n   - Provider (`base`)\r\n   - Model\r\n   - Prompt\r\n   - All relevant parameters (temperature, tokens, etc.)\r\n\r\n2. Responses are stored in JSON format in the `data/llm` directory\r\n\r\n3. When making an identical request, the cached response is returned instead of making a new API call\r\n\r\n4. Cache entries automatically expire after the specified TTL\r\n\r\n5. Cache can be manually cleared by setting `clear_cache=True`\r\n\r\n## Error Handling\r\n\r\nSmartLLM provides robust error handling through state tracking:\r\n\r\n```python\r\nllm = SmartLLM(...)\r\nllm.execute()\r\nllm.wait_for_completion()\r\n\r\nif llm.is_failed():\r\n    print(f\"Request failed: {llm.get_error()}\")\r\nelif llm.is_completed():\r\n    print(\"Request completed successfully\")\r\n    print(llm.response)\r\n```\r\n\r\nFor AsyncSmartLLM:\r\n\r\n```python\r\nllm = AsyncSmartLLM(...)\r\nawait llm.execute()\r\n\r\nif llm.is_failed():\r\n    print(f\"Request failed: {llm.get_error()}\")\r\nelif llm.is_completed():\r\n    print(\"Request completed successfully\")\r\n    print(llm.response)\r\n```\r\n\r\n## Dependencies\r\n\r\n- `cacherator`: Persistent JSON-based caching\r\n- `logorator`: Decorator-based logging\r\n- `openai>=1.0.0`: OpenAI API client\r\n- `anthropic>=0.5.0`: Anthropic API client\r\n- `python-slugify`: Utility for creating safe identifiers\r\n\r\n## License\r\n\r\nMIT License\r\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "A unified interface for interacting with multiple Large Language Model providers",
    "version": "0.0.6",
    "project_urls": {
        "Homepage": "https://github.com/Redundando/smartllm"
    },
    "split_keywords": [
        "llm",
        " openai",
        " anthropic",
        " perplexity",
        " nlp",
        " ai"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "32834e2ad1cdffced04f6d35324ddf8ce0f9f7bc62bfd2208771f66e4169ff7c",
                "md5": "28d9f700de51121c0c2a6b1fa630ea76",
                "sha256": "bc61df225795f55eed7a2af7525e231c48d5f14daca6243ddad33a9b9a8f30de"
            },
            "downloads": -1,
            "filename": "smartllm-0.0.6-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "28d9f700de51121c0c2a6b1fa630ea76",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 22392,
            "upload_time": "2025-03-21T20:24:02",
            "upload_time_iso_8601": "2025-03-21T20:24:02.639343Z",
            "url": "https://files.pythonhosted.org/packages/32/83/4e2ad1cdffced04f6d35324ddf8ce0f9f7bc62bfd2208771f66e4169ff7c/smartllm-0.0.6-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "cb06c2ec0a62f5e6f4d1f4cfe77c061aab3a2992cb2bfe4e39439192f8fa9256",
                "md5": "a37248ceb8b07809667fd73f16364213",
                "sha256": "a330755c6ace4a1075dbc1b3263ea092829556c31a7aab10afe7821d369beb71"
            },
            "downloads": -1,
            "filename": "smartllm-0.0.6.tar.gz",
            "has_sig": false,
            "md5_digest": "a37248ceb8b07809667fd73f16364213",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 19447,
            "upload_time": "2025-03-21T20:24:03",
            "upload_time_iso_8601": "2025-03-21T20:24:03.691674Z",
            "url": "https://files.pythonhosted.org/packages/cb/06/c2ec0a62f5e6f4d1f4cfe77c061aab3a2992cb2bfe4e39439192f8fa9256/smartllm-0.0.6.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-03-21 20:24:03",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "Redundando",
    "github_project": "smartllm",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "requirements": [],
    "lcname": "smartllm"
}
        
Elapsed time: 1.57800s