# Crypto Data Cache
A Python library for efficiently downloading, caching, and managing cryptocurrency market data from Binance. This package provides a robust solution for fetching historical kline (candlestick), trade, and aggregate trade data with automatic retry mechanisms and local SQLite caching.
## Features
- **Efficient Data Fetching**: Download historical cryptocurrency data from Binance
- **Local Caching**: SQLite-based storage for fast local access
- **Multiple Data Types**: Support for klines, trades, and aggregate trades
- **Automatic Gap Detection**: Identifies and fills missing data automatically
- **Monthly/Daily Downloads**: Optimized download strategy using monthly files when possible
- **Robust Error Handling**: Comprehensive error handling and logging
- **Type Safety**: Full type hints for better development experience
## Installation
```bash
pip install crypto-data-cache
```
## Quick Start
```python
from crypto_data_cache import CryptoDataCache, DATA_TYPES
# Initialize the client (uses default database location)
cache = CryptoDataCache()
# Fetch Bitcoin 1-hour kline data
df = cache.fetch_data(
symbol="BTCUSDT",
data_type=DATA_TYPES.KLINE,
start_date_str="2024-01-01",
end_date_str="2024-01-31",
interval="1h"
)
print(f"Loaded {len(df)} rows of data")
print(df.head())
```
### Advanced Usage (Direct Functions)
```python
from crypto_data_cache.fetch_utils import fetch_historical_data
from crypto_data_cache.configurations import DATA_TYPES
from pathlib import Path
# Direct function usage with custom database path
df = fetch_historical_data(
symbol="BTCUSDT",
data_type=DATA_TYPES.KLINE,
start_date_str="2024-01-01",
end_date_str="2024-01-31",
db_file=Path("custom_crypto_data.db"),
interval="1h"
)
```
## Supported Data Types
- **KLINE**: Candlestick/OHLCV data with various intervals (1m, 5m, 1h, 1d, etc.)
- **TRADE**: Individual trade data
- **AGGTRADE**: Aggregate trade data
## Configuration
The library automatically handles:
- Database schema creation
- Data type validation
- Timestamp conversion (UTC microseconds)
- Missing data detection and download
## API Reference
### CryptoDataCache Class
The main client class providing a high-level interface:
#### `CryptoDataCache(db_path=None)`
Initialize the client with optional custom database path.
#### `fetch_data()`
Primary method for fetching historical data.
### Direct Functions
#### `fetch_historical_data()`
Lower-level function for direct access.
**Parameters:**
- `symbol` (str): Trading pair symbol (e.g., "BTCUSDT")
- `data_type` (DATA_TYPES): Type of data to fetch
- `start_date_str` (str): Start date in "YYYY-MM-DD" format
- `end_date_str` (str): End date in "YYYY-MM-DD" format
- `db_file` (Path): Path to SQLite database file
- `interval` (str, optional): Required for KLINE data (e.g., "1h", "1d")
- `prefer_monthly` (bool): Whether to prefer monthly downloads (default: True)
**Returns:**
- `pd.DataFrame`: Historical data with properly typed columns
### Data Types
```python
from crypto_data_cache.configurations import DATA_TYPES
# Available data types
DATA_TYPES.KLINE # Candlestick data
DATA_TYPES.TRADE # Individual trades
DATA_TYPES.AGGTRADE # Aggregate trades
```
## Database Storage
Data is stored in SQLite with the following features:
- Automatic table creation with proper schemas
- UTC timestamp storage (microsecond precision)
- Efficient querying and indexing
- Duplicate data prevention
## Error Handling
The library includes comprehensive error handling:
- Network timeouts and retries
- Data validation
- Missing file handling (404 errors)
- Database connection management
## Logging
Built-in logging provides detailed information about:
- Download progress
- Missing data detection
- Error conditions
- Performance metrics
## Examples
### Fetch Multiple Intervals
```python
from crypto_data_cache import CryptoDataCache, DATA_TYPES
# Initialize once, reuse for multiple requests
cache = CryptoDataCache()
# Fetch different intervals
intervals = ["1h", "4h", "1d"]
for interval in intervals:
df = cache.fetch_data(
symbol="ETHUSDT",
data_type=DATA_TYPES.KLINE,
start_date_str="2024-01-01",
end_date_str="2024-01-31",
interval=interval
)
print(f"{interval}: {len(df)} rows")
```
### Fetch Trade Data
```python
# Fetch individual trade data
cache = CryptoDataCache()
trades_df = cache.fetch_data(
symbol="BTCUSDT",
data_type=DATA_TYPES.TRADE,
start_date_str="2024-01-01",
end_date_str="2024-01-02"
)
```
### Custom Database Location
```python
# Use custom database path
cache = CryptoDataCache(db_path="/path/to/my/crypto_data.db")
df = cache.fetch_data("ETHUSDT", DATA_TYPES.KLINE, "2024-01-01", "2024-01-31", "1d")
```
## Requirements
- Python e 3.13
- pandas e 2.3.1
- requests e 2.32.4
- rich e 14.0.0
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
## Contributing
Contributions are welcome! Please feel free to submit issues and pull requests.
## Support
If you encounter any issues or have questions, please file an issue on the GitHub repository.
Raw data
{
"_id": null,
"home_page": null,
"name": "crypto-data-cache",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.13",
"maintainer_email": null,
"keywords": "binance, cache, cryptocurrency, data, kline, market-data, trading",
"author": null,
"author_email": "\"Luca B. Knaack\" <l.knaack93@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/89/79/8f6a74f017137b7a182eb1c00cb4d009c968ce8c34787c7b2784e794b603/crypto_data_cache-0.2.1.tar.gz",
"platform": null,
"description": "# Crypto Data Cache\n\nA Python library for efficiently downloading, caching, and managing cryptocurrency market data from Binance. This package provides a robust solution for fetching historical kline (candlestick), trade, and aggregate trade data with automatic retry mechanisms and local SQLite caching.\n\n## Features\n\n- **Efficient Data Fetching**: Download historical cryptocurrency data from Binance\n- **Local Caching**: SQLite-based storage for fast local access\n- **Multiple Data Types**: Support for klines, trades, and aggregate trades\n- **Automatic Gap Detection**: Identifies and fills missing data automatically\n- **Monthly/Daily Downloads**: Optimized download strategy using monthly files when possible\n- **Robust Error Handling**: Comprehensive error handling and logging\n- **Type Safety**: Full type hints for better development experience\n\n## Installation\n\n```bash\npip install crypto-data-cache\n```\n\n## Quick Start\n\n```python\nfrom crypto_data_cache import CryptoDataCache, DATA_TYPES\n\n# Initialize the client (uses default database location)\ncache = CryptoDataCache()\n\n# Fetch Bitcoin 1-hour kline data\ndf = cache.fetch_data(\n symbol=\"BTCUSDT\",\n data_type=DATA_TYPES.KLINE,\n start_date_str=\"2024-01-01\",\n end_date_str=\"2024-01-31\",\n interval=\"1h\"\n)\n\nprint(f\"Loaded {len(df)} rows of data\")\nprint(df.head())\n```\n\n### Advanced Usage (Direct Functions)\n\n```python\nfrom crypto_data_cache.fetch_utils import fetch_historical_data\nfrom crypto_data_cache.configurations import DATA_TYPES\nfrom pathlib import Path\n\n# Direct function usage with custom database path\ndf = fetch_historical_data(\n symbol=\"BTCUSDT\",\n data_type=DATA_TYPES.KLINE,\n start_date_str=\"2024-01-01\",\n end_date_str=\"2024-01-31\",\n db_file=Path(\"custom_crypto_data.db\"),\n interval=\"1h\"\n)\n```\n\n## Supported Data Types\n\n- **KLINE**: Candlestick/OHLCV data with various intervals (1m, 5m, 1h, 1d, etc.)\n- **TRADE**: Individual trade data\n- **AGGTRADE**: Aggregate trade data\n\n## Configuration\n\nThe library automatically handles:\n- Database schema creation\n- Data type validation\n- Timestamp conversion (UTC microseconds)\n- Missing data detection and download\n\n## API Reference\n\n### CryptoDataCache Class\n\nThe main client class providing a high-level interface:\n\n#### `CryptoDataCache(db_path=None)`\nInitialize the client with optional custom database path.\n\n#### `fetch_data()`\nPrimary method for fetching historical data.\n\n### Direct Functions\n\n#### `fetch_historical_data()`\nLower-level function for direct access.\n\n**Parameters:**\n- `symbol` (str): Trading pair symbol (e.g., \"BTCUSDT\")\n- `data_type` (DATA_TYPES): Type of data to fetch\n- `start_date_str` (str): Start date in \"YYYY-MM-DD\" format\n- `end_date_str` (str): End date in \"YYYY-MM-DD\" format\n- `db_file` (Path): Path to SQLite database file\n- `interval` (str, optional): Required for KLINE data (e.g., \"1h\", \"1d\")\n- `prefer_monthly` (bool): Whether to prefer monthly downloads (default: True)\n\n**Returns:**\n- `pd.DataFrame`: Historical data with properly typed columns\n\n### Data Types\n\n```python\nfrom crypto_data_cache.configurations import DATA_TYPES\n\n# Available data types\nDATA_TYPES.KLINE # Candlestick data\nDATA_TYPES.TRADE # Individual trades\nDATA_TYPES.AGGTRADE # Aggregate trades\n```\n\n## Database Storage\n\nData is stored in SQLite with the following features:\n- Automatic table creation with proper schemas\n- UTC timestamp storage (microsecond precision)\n- Efficient querying and indexing\n- Duplicate data prevention\n\n## Error Handling\n\nThe library includes comprehensive error handling:\n- Network timeouts and retries\n- Data validation\n- Missing file handling (404 errors)\n- Database connection management\n\n## Logging\n\nBuilt-in logging provides detailed information about:\n- Download progress\n- Missing data detection\n- Error conditions\n- Performance metrics\n\n## Examples\n\n### Fetch Multiple Intervals\n\n```python\nfrom crypto_data_cache import CryptoDataCache, DATA_TYPES\n\n# Initialize once, reuse for multiple requests\ncache = CryptoDataCache()\n\n# Fetch different intervals\nintervals = [\"1h\", \"4h\", \"1d\"]\n\nfor interval in intervals:\n df = cache.fetch_data(\n symbol=\"ETHUSDT\",\n data_type=DATA_TYPES.KLINE,\n start_date_str=\"2024-01-01\",\n end_date_str=\"2024-01-31\",\n interval=interval\n )\n print(f\"{interval}: {len(df)} rows\")\n```\n\n### Fetch Trade Data\n\n```python\n# Fetch individual trade data\ncache = CryptoDataCache()\ntrades_df = cache.fetch_data(\n symbol=\"BTCUSDT\",\n data_type=DATA_TYPES.TRADE,\n start_date_str=\"2024-01-01\",\n end_date_str=\"2024-01-02\"\n)\n```\n\n### Custom Database Location\n\n```python\n# Use custom database path\ncache = CryptoDataCache(db_path=\"/path/to/my/crypto_data.db\")\ndf = cache.fetch_data(\"ETHUSDT\", DATA_TYPES.KLINE, \"2024-01-01\", \"2024-01-31\", \"1d\")\n```\n\n## Requirements\n\n- Python e 3.13\n- pandas e 2.3.1\n- requests e 2.32.4\n- rich e 14.0.0\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## Contributing\n\nContributions are welcome! Please feel free to submit issues and pull requests.\n\n## Support\n\nIf you encounter any issues or have questions, please file an issue on the GitHub repository.",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Python library for efficiently downloading, caching, and managing cryptocurrency market data from Binance",
"version": "0.2.1",
"project_urls": {
"Homepage": "https://github.com/SithHades/crypto-data-cache",
"Issues": "https://github.com/SithHades/crypto-data-cache/issues",
"Repository": "https://github.com/SithHades/crypto-data-cache.git"
},
"split_keywords": [
"binance",
" cache",
" cryptocurrency",
" data",
" kline",
" market-data",
" trading"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "65107435ea593301461ab9eb969c6010cae9ac0948ecb9f20413179a01dad442",
"md5": "03f262a86bb9d821e566568793159a11",
"sha256": "da99e04679b645dfc8fd82cb5b56454da5da25f8ce61b4a7dae1a8fec599891a"
},
"downloads": -1,
"filename": "crypto_data_cache-0.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "03f262a86bb9d821e566568793159a11",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.13",
"size": 20948,
"upload_time": "2025-07-23T06:23:42",
"upload_time_iso_8601": "2025-07-23T06:23:42.284111Z",
"url": "https://files.pythonhosted.org/packages/65/10/7435ea593301461ab9eb969c6010cae9ac0948ecb9f20413179a01dad442/crypto_data_cache-0.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "89798f6a74f017137b7a182eb1c00cb4d009c968ce8c34787c7b2784e794b603",
"md5": "0d800deb0261d6d01daaf9f21b547412",
"sha256": "31b76727fa6e9722ac0f35dcef925d3977d45c8d5af67cb3fd2958cdb2cdeb59"
},
"downloads": -1,
"filename": "crypto_data_cache-0.2.1.tar.gz",
"has_sig": false,
"md5_digest": "0d800deb0261d6d01daaf9f21b547412",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.13",
"size": 29900,
"upload_time": "2025-07-23T06:23:43",
"upload_time_iso_8601": "2025-07-23T06:23:43.262471Z",
"url": "https://files.pythonhosted.org/packages/89/79/8f6a74f017137b7a182eb1c00cb4d009c968ce8c34787c7b2784e794b603/crypto_data_cache-0.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-23 06:23:43",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "SithHades",
"github_project": "crypto-data-cache",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "crypto-data-cache"
}
JSON
