| Name | smartllm JSON |
| Version |
0.0.6
JSON |
| download |
| home_page | None |
| Summary | A unified interface for interacting with multiple Large Language Model providers |
| upload_time | 2025-03-21 20:24:03 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | MIT |
| keywords |
llm
openai
anthropic
perplexity
nlp
ai
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# SmartLLM
SmartLLM is a unified Python interface for interacting with multiple Large Language Model providers. It provides a consistent API across different LLM providers, handles caching of responses, and supports both synchronous and asynchronous interactions.
## Installation
```bash
pip install smartllm
```
## Features
- **Unified API**: Consistent interface for OpenAI, Anthropic, and Perplexity LLMs
- **Response Caching**: Persistent JSON-based caching of responses to improve performance
- **Streaming Support**: Real-time streaming of LLM responses (Anthropic only)
- **JSON Mode**: Structured JSON responses (OpenAI and Anthropic)
- **Citations**: Access to source information (Perplexity only)
- **Asynchronous Execution**: Non-blocking request execution via AsyncSmartLLM
- **Configurable Parameters**: Granular control over temperature, tokens, and other model parameters
## Supported Providers
SmartLLM currently supports the following LLM providers:
- **OpenAI**
- Models: GPT-4, GPT-3.5 series, and other OpenAI models
- Features: JSON-structured outputs, token usage information
- Example: `base="openai", model="gpt-4"`
- **Anthropic**
- Models: Claude models (e.g., claude-3-7-sonnet-20250219)
- Features: Streaming support, JSON-structured outputs, system prompts
- Example: `base="anthropic", model="claude-3-7-sonnet-20250219"`
- **Perplexity**
- Models: sonar-small-online, sonar-medium-online, sonar-pro, etc.
- Features: Web search capabilities, citation information
- Example: `base="perplexity", model="sonar-pro"`
## Basic Usage
```python
from smartllm import SmartLLM
import os
# Create SmartLLM instance
llm = SmartLLM(
base="openai",
model="gpt-4",
api_key=os.environ.get("OPENAI_API_KEY"),
prompt="Explain quantum computing in simple terms",
temperature=0.7
)
# Execute the request
llm.execute()
# Wait for completion
llm.wait_for_completion()
# Check status and get response
if llm.is_completed():
print(llm.response)
else:
print(f"Error: {llm.get_error()}")
```
## AsyncSmartLLM - Asynchronous Interface
SmartLLM also provides an asynchronous interface for non-blocking API interactions using AsyncSmartLLM:
```python
import asyncio
import os
from smartllm import AsyncSmartLLM
async def main():
# Create AsyncSmartLLM instance
llm = AsyncSmartLLM(
base="anthropic",
model="claude-3-7-sonnet-20250219",
api_key=os.environ.get("ANTHROPIC_API_KEY"),
prompt="Explain quantum computing in simple terms",
temperature=0.7
)
# Execute the request asynchronously
await llm.execute()
# Check status and get response
if llm.is_completed():
print(llm.response)
else:
print(f"Error: {llm.get_error()}")
# Run the async function
asyncio.run(main())
```
### Async Streaming Support
AsyncSmartLLM also supports streaming responses:
```python
import asyncio
import os
from smartllm import AsyncSmartLLM
# Custom callback for handling streaming chunks
async def print_chunk(chunk: str, accumulated: str) -> None:
print(chunk, end="", flush=True)
async def main():
api_key = os.environ.get("ANTHROPIC_API_KEY")
# Enable streaming with stream=True
llm = AsyncSmartLLM(
base="anthropic",
model="claude-3-7-sonnet-20250219",
api_key=api_key,
prompt="Tell me a short story about a robot learning to paint",
temperature=0.7,
max_output_tokens=1000,
stream=True
)
# Execute with callback
await llm.execute(callback=print_chunk)
if llm.is_failed():
print(f"\nError occurred: {llm.get_error()}")
else:
print("\n\nFinal response:")
print(llm.response)
if __name__ == "__main__":
asyncio.run(main())
```
### Async JSON Mode
```python
import asyncio
import os
from smartllm import AsyncSmartLLM
async def main():
api_key = os.environ.get("ANTHROPIC_API_KEY")
json_schema = {
"type": "object",
"properties": {
"title": {"type": "string"},
"topics": {"type": "array", "items": {"type": "string"}},
"difficulty": {"type": "integer", "minimum": 1, "maximum": 10}
},
"required": ["title", "topics", "difficulty"]
}
llm = AsyncSmartLLM(
base="anthropic",
model="claude-3-7-sonnet-20250219",
api_key=api_key,
prompt="Generate information about a quantum computing course",
json_mode=True,
json_schema=json_schema
)
await llm.execute()
# Access structured data
course_info = llm.response # Returns a Python dictionary
print(f"Course title: {course_info['title']}")
print(f"Topics: {', '.join(course_info['topics'])}")
print(f"Difficulty: {course_info['difficulty']}/10")
if __name__ == "__main__":
asyncio.run(main())
```
## SmartLLM Class Reference
### Constructor
```python
SmartLLM(
base: str = "", # LLM provider ("openai", "anthropic", "perplexity")
model: str = "", # Model identifier
api_key: str = "", # API key for the provider
prompt: Union[str, List[str]] = "", # Single prompt or conversation history
stream: bool = False, # Enable streaming (Anthropic only)
max_input_tokens: Optional[int] = None, # Max input tokens
max_output_tokens: Optional[int] = None, # Max output tokens
output_type: str = "text", # Output type
temperature: float = 0.2, # Temperature for generation
top_p: float = 0.9, # Top-p sampling parameter
frequency_penalty: float = 1.0, # Frequency penalty
presence_penalty: float = 0.0, # Presence penalty
system_prompt: Optional[str] = None, # System prompt
search_recency_filter: Optional[str] = None, # Filter for search (Perplexity)
return_citations: bool = False, # Include citations (Perplexity)
json_mode: bool = False, # Enable JSON mode (OpenAI, Anthropic)
json_schema: Optional[Dict[str, Any]] = None, # JSON schema
ttl: int = 7, # Cache time-to-live in days
clear_cache: bool = False # Clear existing cache
)
```
### AsyncSmartLLM Class Reference
```python
AsyncSmartLLM(
# Same parameters as SmartLLM above
)
```
### Methods
#### SmartLLM Methods
```python
execute(callback: Optional[Callable[[str, str], None]] = None) -> SmartLLM
wait_for_completion(timeout: Optional[float] = None) -> bool
is_failed() -> bool
is_completed() -> bool
get_error() -> Optional[str]
```
#### AsyncSmartLLM Methods
```python
async execute(callback: Optional[Callable[[str, str], None]] = None) -> AsyncSmartLLM
async generate() -> AsyncSmartLLM
is_failed() -> bool
is_completed() -> bool
get_error() -> Optional[str]
```
### Properties
```python
response: Union[str, Dict[str, Any]] # The response content or JSON
sources: List[str] # Citation sources (Perplexity)
usage: Dict[str, int] # Token usage statistics
```
## Streaming Responses (Anthropic Only)
### Synchronous Streaming
```python
from smartllm import SmartLLM
import os
def print_chunk(chunk: str, accumulated: str) -> None:
print(f"CHUNK: {chunk}")
llm = SmartLLM(
base="anthropic",
model="claude-3-7-sonnet-20250219",
api_key=os.environ.get("ANTHROPIC_API_KEY"),
prompt="Write a short story about a robot learning to paint",
stream=True # Enable streaming
)
# Execute with callback
llm.execute(callback=print_chunk)
llm.wait_for_completion()
```
### Asynchronous Streaming
```python
import asyncio
from smartllm import AsyncSmartLLM
import os
async def print_chunk(chunk: str, accumulated: str) -> None:
print(chunk, end="", flush=True)
async def main():
llm = AsyncSmartLLM(
base="anthropic",
model="claude-3-7-sonnet-20250219",
api_key=os.environ.get("ANTHROPIC_API_KEY"),
prompt="Write a short story about a robot learning to paint",
stream=True # Enable streaming
)
# Execute with callback
await llm.execute(callback=print_chunk)
print("\n\nFinal response:")
print(llm.response)
asyncio.run(main())
```
## JSON Mode (OpenAI and Anthropic)
```python
from smartllm import SmartLLM
import os
json_schema = {
"type": "object",
"properties": {
"title": {"type": "string"},
"topics": {"type": "array", "items": {"type": "string"}},
"difficulty": {"type": "integer", "minimum": 1, "maximum": 10}
},
"required": ["title", "topics", "difficulty"]
}
llm = SmartLLM(
base="openai",
model="gpt-4",
api_key=os.environ.get("OPENAI_API_KEY"),
prompt="Generate information about a quantum computing course",
json_mode=True,
json_schema=json_schema
)
llm.execute()
llm.wait_for_completion()
# Access structured data
course_info = llm.response # Returns a Python dictionary
print(f"Course title: {course_info['title']}")
print(f"Topics: {', '.join(course_info['topics'])}")
print(f"Difficulty: {course_info['difficulty']}/10")
```
## Getting Citations (Perplexity Only)
```python
from smartllm import SmartLLM
import os
llm = SmartLLM(
base="perplexity",
model="sonar-pro",
api_key=os.environ.get("PERPLEXITY_API_KEY"),
prompt="What are the latest advancements in quantum computing?",
search_recency_filter="week", # Filter for recent information
return_citations=True # Enable citations
)
llm.execute()
llm.wait_for_completion()
# Print the response
print(llm.response)
# Print the sources
print("\nSources:")
for source in llm.sources:
print(f"- {source}")
```
## Caching Mechanism
SmartLLM uses a persistent JSON-based caching system powered by the Cacherator library. This significantly improves performance by avoiding redundant API calls for identical requests.
### Cache Configuration
By default, responses are cached for 7 days. You can customize the cache behavior:
```python
# Set custom time-to-live (TTL) in days
llm = SmartLLM(
base="openai",
model="gpt-4",
api_key=os.environ.get("OPENAI_API_KEY"),
prompt="Explain quantum computing",
ttl=30 # Cache results for 30 days
)
# Force clear existing cache
llm = SmartLLM(
base="openai",
model="gpt-4",
api_key=os.environ.get("OPENAI_API_KEY"),
prompt="Explain quantum computing",
clear_cache=True # Ignore any existing cached response
)
```
### How Caching Works
1. Each request is assigned a unique identifier based on:
- Provider (`base`)
- Model
- Prompt
- All relevant parameters (temperature, tokens, etc.)
2. Responses are stored in JSON format in the `data/llm` directory
3. When making an identical request, the cached response is returned instead of making a new API call
4. Cache entries automatically expire after the specified TTL
5. Cache can be manually cleared by setting `clear_cache=True`
## Error Handling
SmartLLM provides robust error handling through state tracking:
```python
llm = SmartLLM(...)
llm.execute()
llm.wait_for_completion()
if llm.is_failed():
print(f"Request failed: {llm.get_error()}")
elif llm.is_completed():
print("Request completed successfully")
print(llm.response)
```
For AsyncSmartLLM:
```python
llm = AsyncSmartLLM(...)
await llm.execute()
if llm.is_failed():
print(f"Request failed: {llm.get_error()}")
elif llm.is_completed():
print("Request completed successfully")
print(llm.response)
```
## Dependencies
- `cacherator`: Persistent JSON-based caching
- `logorator`: Decorator-based logging
- `openai>=1.0.0`: OpenAI API client
- `anthropic>=0.5.0`: Anthropic API client
- `python-slugify`: Utility for creating safe identifiers
## License
MIT License
Raw data
{
"_id": null,
"home_page": null,
"name": "smartllm",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "llm, openai, anthropic, perplexity, nlp, ai",
"author": null,
"author_email": "Arved Kl\u00f6hn <arved.kloehn@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/cb/06/c2ec0a62f5e6f4d1f4cfe77c061aab3a2992cb2bfe4e39439192f8fa9256/smartllm-0.0.6.tar.gz",
"platform": null,
"description": "# SmartLLM\r\n\r\nSmartLLM is a unified Python interface for interacting with multiple Large Language Model providers. It provides a consistent API across different LLM providers, handles caching of responses, and supports both synchronous and asynchronous interactions.\r\n\r\n## Installation\r\n\r\n```bash\r\npip install smartllm\r\n```\r\n\r\n## Features\r\n\r\n- **Unified API**: Consistent interface for OpenAI, Anthropic, and Perplexity LLMs\r\n- **Response Caching**: Persistent JSON-based caching of responses to improve performance\r\n- **Streaming Support**: Real-time streaming of LLM responses (Anthropic only)\r\n- **JSON Mode**: Structured JSON responses (OpenAI and Anthropic)\r\n- **Citations**: Access to source information (Perplexity only)\r\n- **Asynchronous Execution**: Non-blocking request execution via AsyncSmartLLM\r\n- **Configurable Parameters**: Granular control over temperature, tokens, and other model parameters\r\n\r\n## Supported Providers\r\n\r\nSmartLLM currently supports the following LLM providers:\r\n\r\n- **OpenAI**\r\n - Models: GPT-4, GPT-3.5 series, and other OpenAI models\r\n - Features: JSON-structured outputs, token usage information\r\n - Example: `base=\"openai\", model=\"gpt-4\"`\r\n\r\n- **Anthropic**\r\n - Models: Claude models (e.g., claude-3-7-sonnet-20250219)\r\n - Features: Streaming support, JSON-structured outputs, system prompts\r\n - Example: `base=\"anthropic\", model=\"claude-3-7-sonnet-20250219\"`\r\n\r\n- **Perplexity**\r\n - Models: sonar-small-online, sonar-medium-online, sonar-pro, etc.\r\n - Features: Web search capabilities, citation information\r\n - Example: `base=\"perplexity\", model=\"sonar-pro\"`\r\n\r\n## Basic Usage\r\n\r\n```python\r\nfrom smartllm import SmartLLM\r\nimport os\r\n\r\n# Create SmartLLM instance\r\nllm = SmartLLM(\r\n base=\"openai\",\r\n model=\"gpt-4\",\r\n api_key=os.environ.get(\"OPENAI_API_KEY\"),\r\n prompt=\"Explain quantum computing in simple terms\",\r\n temperature=0.7\r\n)\r\n\r\n# Execute the request\r\nllm.execute()\r\n\r\n# Wait for completion\r\nllm.wait_for_completion()\r\n\r\n# Check status and get response\r\nif llm.is_completed():\r\n print(llm.response)\r\nelse:\r\n print(f\"Error: {llm.get_error()}\")\r\n```\r\n\r\n## AsyncSmartLLM - Asynchronous Interface\r\n\r\nSmartLLM also provides an asynchronous interface for non-blocking API interactions using AsyncSmartLLM:\r\n\r\n```python\r\nimport asyncio\r\nimport os\r\nfrom smartllm import AsyncSmartLLM\r\n\r\nasync def main():\r\n # Create AsyncSmartLLM instance\r\n llm = AsyncSmartLLM(\r\n base=\"anthropic\",\r\n model=\"claude-3-7-sonnet-20250219\",\r\n api_key=os.environ.get(\"ANTHROPIC_API_KEY\"),\r\n prompt=\"Explain quantum computing in simple terms\",\r\n temperature=0.7\r\n )\r\n\r\n # Execute the request asynchronously\r\n await llm.execute()\r\n\r\n # Check status and get response\r\n if llm.is_completed():\r\n print(llm.response)\r\n else:\r\n print(f\"Error: {llm.get_error()}\")\r\n\r\n# Run the async function\r\nasyncio.run(main())\r\n```\r\n\r\n### Async Streaming Support\r\n\r\nAsyncSmartLLM also supports streaming responses:\r\n\r\n```python\r\nimport asyncio\r\nimport os\r\nfrom smartllm import AsyncSmartLLM\r\n\r\n# Custom callback for handling streaming chunks\r\nasync def print_chunk(chunk: str, accumulated: str) -> None:\r\n print(chunk, end=\"\", flush=True)\r\n\r\nasync def main():\r\n api_key = os.environ.get(\"ANTHROPIC_API_KEY\")\r\n \r\n # Enable streaming with stream=True\r\n llm = AsyncSmartLLM(\r\n base=\"anthropic\", \r\n model=\"claude-3-7-sonnet-20250219\", \r\n api_key=api_key,\r\n prompt=\"Tell me a short story about a robot learning to paint\",\r\n temperature=0.7, \r\n max_output_tokens=1000,\r\n stream=True\r\n )\r\n\r\n # Execute with callback\r\n await llm.execute(callback=print_chunk)\r\n \r\n if llm.is_failed():\r\n print(f\"\\nError occurred: {llm.get_error()}\")\r\n else:\r\n print(\"\\n\\nFinal response:\")\r\n print(llm.response)\r\n\r\nif __name__ == \"__main__\":\r\n asyncio.run(main())\r\n```\r\n\r\n### Async JSON Mode\r\n\r\n```python\r\nimport asyncio\r\nimport os\r\nfrom smartllm import AsyncSmartLLM\r\n\r\nasync def main():\r\n api_key = os.environ.get(\"ANTHROPIC_API_KEY\")\r\n\r\n json_schema = {\r\n \"type\": \"object\",\r\n \"properties\": {\r\n \"title\": {\"type\": \"string\"},\r\n \"topics\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}},\r\n \"difficulty\": {\"type\": \"integer\", \"minimum\": 1, \"maximum\": 10}\r\n },\r\n \"required\": [\"title\", \"topics\", \"difficulty\"]\r\n }\r\n\r\n llm = AsyncSmartLLM(\r\n base=\"anthropic\",\r\n model=\"claude-3-7-sonnet-20250219\",\r\n api_key=api_key,\r\n prompt=\"Generate information about a quantum computing course\",\r\n json_mode=True,\r\n json_schema=json_schema\r\n )\r\n\r\n await llm.execute()\r\n\r\n # Access structured data\r\n course_info = llm.response # Returns a Python dictionary\r\n print(f\"Course title: {course_info['title']}\")\r\n print(f\"Topics: {', '.join(course_info['topics'])}\")\r\n print(f\"Difficulty: {course_info['difficulty']}/10\")\r\n\r\nif __name__ == \"__main__\":\r\n asyncio.run(main())\r\n```\r\n\r\n## SmartLLM Class Reference\r\n\r\n### Constructor\r\n\r\n```python\r\nSmartLLM(\r\n base: str = \"\", # LLM provider (\"openai\", \"anthropic\", \"perplexity\")\r\n model: str = \"\", # Model identifier\r\n api_key: str = \"\", # API key for the provider\r\n prompt: Union[str, List[str]] = \"\", # Single prompt or conversation history\r\n stream: bool = False, # Enable streaming (Anthropic only)\r\n max_input_tokens: Optional[int] = None, # Max input tokens\r\n max_output_tokens: Optional[int] = None, # Max output tokens\r\n output_type: str = \"text\", # Output type\r\n temperature: float = 0.2, # Temperature for generation\r\n top_p: float = 0.9, # Top-p sampling parameter\r\n frequency_penalty: float = 1.0, # Frequency penalty\r\n presence_penalty: float = 0.0, # Presence penalty\r\n system_prompt: Optional[str] = None, # System prompt\r\n search_recency_filter: Optional[str] = None, # Filter for search (Perplexity)\r\n return_citations: bool = False, # Include citations (Perplexity)\r\n json_mode: bool = False, # Enable JSON mode (OpenAI, Anthropic)\r\n json_schema: Optional[Dict[str, Any]] = None, # JSON schema\r\n ttl: int = 7, # Cache time-to-live in days\r\n clear_cache: bool = False # Clear existing cache\r\n)\r\n```\r\n\r\n### AsyncSmartLLM Class Reference\r\n\r\n```python\r\nAsyncSmartLLM(\r\n # Same parameters as SmartLLM above\r\n)\r\n```\r\n\r\n### Methods\r\n\r\n#### SmartLLM Methods\r\n```python\r\nexecute(callback: Optional[Callable[[str, str], None]] = None) -> SmartLLM\r\nwait_for_completion(timeout: Optional[float] = None) -> bool\r\nis_failed() -> bool\r\nis_completed() -> bool\r\nget_error() -> Optional[str]\r\n```\r\n\r\n#### AsyncSmartLLM Methods\r\n```python\r\nasync execute(callback: Optional[Callable[[str, str], None]] = None) -> AsyncSmartLLM\r\nasync generate() -> AsyncSmartLLM\r\nis_failed() -> bool\r\nis_completed() -> bool\r\nget_error() -> Optional[str]\r\n```\r\n\r\n### Properties\r\n\r\n```python\r\nresponse: Union[str, Dict[str, Any]] # The response content or JSON\r\nsources: List[str] # Citation sources (Perplexity)\r\nusage: Dict[str, int] # Token usage statistics\r\n```\r\n\r\n## Streaming Responses (Anthropic Only)\r\n\r\n### Synchronous Streaming\r\n\r\n```python\r\nfrom smartllm import SmartLLM\r\nimport os\r\n\r\ndef print_chunk(chunk: str, accumulated: str) -> None:\r\n print(f\"CHUNK: {chunk}\")\r\n\r\nllm = SmartLLM(\r\n base=\"anthropic\",\r\n model=\"claude-3-7-sonnet-20250219\",\r\n api_key=os.environ.get(\"ANTHROPIC_API_KEY\"),\r\n prompt=\"Write a short story about a robot learning to paint\",\r\n stream=True # Enable streaming\r\n)\r\n\r\n# Execute with callback\r\nllm.execute(callback=print_chunk)\r\nllm.wait_for_completion()\r\n```\r\n\r\n### Asynchronous Streaming\r\n\r\n```python\r\nimport asyncio\r\nfrom smartllm import AsyncSmartLLM\r\nimport os\r\n\r\nasync def print_chunk(chunk: str, accumulated: str) -> None:\r\n print(chunk, end=\"\", flush=True)\r\n\r\nasync def main():\r\n llm = AsyncSmartLLM(\r\n base=\"anthropic\",\r\n model=\"claude-3-7-sonnet-20250219\",\r\n api_key=os.environ.get(\"ANTHROPIC_API_KEY\"),\r\n prompt=\"Write a short story about a robot learning to paint\",\r\n stream=True # Enable streaming\r\n )\r\n\r\n # Execute with callback\r\n await llm.execute(callback=print_chunk)\r\n \r\n print(\"\\n\\nFinal response:\")\r\n print(llm.response)\r\n\r\nasyncio.run(main())\r\n```\r\n\r\n## JSON Mode (OpenAI and Anthropic)\r\n\r\n```python\r\nfrom smartllm import SmartLLM\r\nimport os\r\n\r\njson_schema = {\r\n \"type\": \"object\",\r\n \"properties\": {\r\n \"title\": {\"type\": \"string\"},\r\n \"topics\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}},\r\n \"difficulty\": {\"type\": \"integer\", \"minimum\": 1, \"maximum\": 10}\r\n },\r\n \"required\": [\"title\", \"topics\", \"difficulty\"]\r\n}\r\n\r\nllm = SmartLLM(\r\n base=\"openai\",\r\n model=\"gpt-4\",\r\n api_key=os.environ.get(\"OPENAI_API_KEY\"),\r\n prompt=\"Generate information about a quantum computing course\",\r\n json_mode=True,\r\n json_schema=json_schema\r\n)\r\n\r\nllm.execute()\r\nllm.wait_for_completion()\r\n\r\n# Access structured data\r\ncourse_info = llm.response # Returns a Python dictionary\r\nprint(f\"Course title: {course_info['title']}\")\r\nprint(f\"Topics: {', '.join(course_info['topics'])}\")\r\nprint(f\"Difficulty: {course_info['difficulty']}/10\")\r\n```\r\n\r\n## Getting Citations (Perplexity Only)\r\n\r\n```python\r\nfrom smartllm import SmartLLM\r\nimport os\r\n\r\nllm = SmartLLM(\r\n base=\"perplexity\",\r\n model=\"sonar-pro\",\r\n api_key=os.environ.get(\"PERPLEXITY_API_KEY\"),\r\n prompt=\"What are the latest advancements in quantum computing?\",\r\n search_recency_filter=\"week\", # Filter for recent information\r\n return_citations=True # Enable citations\r\n)\r\n\r\nllm.execute()\r\nllm.wait_for_completion()\r\n\r\n# Print the response\r\nprint(llm.response)\r\n\r\n# Print the sources\r\nprint(\"\\nSources:\")\r\nfor source in llm.sources:\r\n print(f\"- {source}\")\r\n```\r\n\r\n## Caching Mechanism\r\n\r\nSmartLLM uses a persistent JSON-based caching system powered by the Cacherator library. This significantly improves performance by avoiding redundant API calls for identical requests.\r\n\r\n### Cache Configuration\r\n\r\nBy default, responses are cached for 7 days. You can customize the cache behavior:\r\n\r\n```python\r\n# Set custom time-to-live (TTL) in days\r\nllm = SmartLLM(\r\n base=\"openai\",\r\n model=\"gpt-4\",\r\n api_key=os.environ.get(\"OPENAI_API_KEY\"),\r\n prompt=\"Explain quantum computing\",\r\n ttl=30 # Cache results for 30 days\r\n)\r\n\r\n# Force clear existing cache\r\nllm = SmartLLM(\r\n base=\"openai\",\r\n model=\"gpt-4\",\r\n api_key=os.environ.get(\"OPENAI_API_KEY\"),\r\n prompt=\"Explain quantum computing\",\r\n clear_cache=True # Ignore any existing cached response\r\n)\r\n```\r\n\r\n### How Caching Works\r\n\r\n1. Each request is assigned a unique identifier based on:\r\n - Provider (`base`)\r\n - Model\r\n - Prompt\r\n - All relevant parameters (temperature, tokens, etc.)\r\n\r\n2. Responses are stored in JSON format in the `data/llm` directory\r\n\r\n3. When making an identical request, the cached response is returned instead of making a new API call\r\n\r\n4. Cache entries automatically expire after the specified TTL\r\n\r\n5. Cache can be manually cleared by setting `clear_cache=True`\r\n\r\n## Error Handling\r\n\r\nSmartLLM provides robust error handling through state tracking:\r\n\r\n```python\r\nllm = SmartLLM(...)\r\nllm.execute()\r\nllm.wait_for_completion()\r\n\r\nif llm.is_failed():\r\n print(f\"Request failed: {llm.get_error()}\")\r\nelif llm.is_completed():\r\n print(\"Request completed successfully\")\r\n print(llm.response)\r\n```\r\n\r\nFor AsyncSmartLLM:\r\n\r\n```python\r\nllm = AsyncSmartLLM(...)\r\nawait llm.execute()\r\n\r\nif llm.is_failed():\r\n print(f\"Request failed: {llm.get_error()}\")\r\nelif llm.is_completed():\r\n print(\"Request completed successfully\")\r\n print(llm.response)\r\n```\r\n\r\n## Dependencies\r\n\r\n- `cacherator`: Persistent JSON-based caching\r\n- `logorator`: Decorator-based logging\r\n- `openai>=1.0.0`: OpenAI API client\r\n- `anthropic>=0.5.0`: Anthropic API client\r\n- `python-slugify`: Utility for creating safe identifiers\r\n\r\n## License\r\n\r\nMIT License\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A unified interface for interacting with multiple Large Language Model providers",
"version": "0.0.6",
"project_urls": {
"Homepage": "https://github.com/Redundando/smartllm"
},
"split_keywords": [
"llm",
" openai",
" anthropic",
" perplexity",
" nlp",
" ai"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "32834e2ad1cdffced04f6d35324ddf8ce0f9f7bc62bfd2208771f66e4169ff7c",
"md5": "28d9f700de51121c0c2a6b1fa630ea76",
"sha256": "bc61df225795f55eed7a2af7525e231c48d5f14daca6243ddad33a9b9a8f30de"
},
"downloads": -1,
"filename": "smartllm-0.0.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "28d9f700de51121c0c2a6b1fa630ea76",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 22392,
"upload_time": "2025-03-21T20:24:02",
"upload_time_iso_8601": "2025-03-21T20:24:02.639343Z",
"url": "https://files.pythonhosted.org/packages/32/83/4e2ad1cdffced04f6d35324ddf8ce0f9f7bc62bfd2208771f66e4169ff7c/smartllm-0.0.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "cb06c2ec0a62f5e6f4d1f4cfe77c061aab3a2992cb2bfe4e39439192f8fa9256",
"md5": "a37248ceb8b07809667fd73f16364213",
"sha256": "a330755c6ace4a1075dbc1b3263ea092829556c31a7aab10afe7821d369beb71"
},
"downloads": -1,
"filename": "smartllm-0.0.6.tar.gz",
"has_sig": false,
"md5_digest": "a37248ceb8b07809667fd73f16364213",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 19447,
"upload_time": "2025-03-21T20:24:03",
"upload_time_iso_8601": "2025-03-21T20:24:03.691674Z",
"url": "https://files.pythonhosted.org/packages/cb/06/c2ec0a62f5e6f4d1f4cfe77c061aab3a2992cb2bfe4e39439192f8fa9256/smartllm-0.0.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-03-21 20:24:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Redundando",
"github_project": "smartllm",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "smartllm"
}
