# Real-time market data listener and archiver in python.
Saves raw binance data from:
limit orderbook difference depth stream
limit orderbook difference depth snapshot request
transaction stream
into .json / .json.zip / .json.zip on azure or backblaze
simply enter azure backblaze connection params into environment variables and import
'launch_data_sink' - use as data sink
or
'launch_data_listener' - Use in your algo trading as it just informs all observers without data save
NEW!!!:
Although Binance now limits each WebSocket connection to 24 hours, this code runs continuously (24/7) without missing any stream entries!
# Handles:
spot, futures usd-m, futures coin-m
Level 2 orderbook difference depth stream
trade stream
orderbook snapshots with configured trigger interval
24-hour WebSocket lifecycle. At the end of the WebSocket's lifespan, it initiates a new WebSocket to ensure the continuity of data flow is maintained seamlessly.
Configured to use as container on Azure with Azure blob / keyvault or backblaze
## Installation
```bash
# to be announced
```
## Usage
import the `run_stonks_analysis` function from the `stonks` module and run the script:
```python
import time
from dotenv import load_dotenv
from binance_archiver import load_config_from_json, DataSinkConfig, launch_data_sink
if __name__ == "__main__":
load_dotenv('sample_env.env')
data_sink_config = DataSinkConfig(
instruments={
'spot': ['BTCUSDT', 'XRPUSDT'],
'usd_m_futures': ['BTCUSDT', 'XRPUSDT'],
'coin_m_futures': ['BTCUSDT', 'XRPUSDT']
},
time_settings={
"file_duration_seconds": 120,
"snapshot_fetcher_interval_seconds": 300,
"websocket_life_time_seconds": 120
},
data_save_target='azure_blob'
)
data_sink = launch_data_sink(data_sink_config=data_sink_config)
time.sleep(99)
data_sink.shutdown()
```
Raw data
{
"_id": null,
"home_page": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
"name": "binance-archiver",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "binance archiver quant data sprzedam opla",
"author": "Daniel Lasota",
"author_email": "grossmann.root@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/1d/f7/f7b5e26dcc6d2b42f935249eaa117b1442dc6c3d55c7a0700d4b3ad49cd5/binance_archiver-0.0.3.tar.gz",
"platform": null,
"description": "# Real-time market data listener and archiver in python. \r\nSaves raw binance data from:\r\nlimit orderbook difference depth stream\r\nlimit orderbook difference depth snapshot request\r\ntransaction stream\r\n\r\ninto .json / .json.zip / .json.zip on azure or backblaze\r\n\r\nsimply enter azure backblaze connection params into environment variables and import \r\n\r\n'launch_data_sink' - use as data sink\r\n\r\nor \r\n\r\n'launch_data_listener' - Use in your algo trading as it just informs all observers without data save\r\n\r\nNEW!!!:\r\nAlthough Binance now limits each WebSocket connection to 24 hours, this code runs continuously (24/7) without missing any stream entries!\r\n\r\n\r\n# Handles: \r\nspot, futures usd-m, futures coin-m\r\nLevel 2 orderbook difference depth stream\r\ntrade stream\r\norderbook snapshots with configured trigger interval \r\n24-hour WebSocket lifecycle. At the end of the WebSocket's lifespan, it initiates a new WebSocket to ensure the continuity of data flow is maintained seamlessly.\r\n\r\nConfigured to use as container on Azure with Azure blob / keyvault or backblaze\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n## Installation\r\n\r\n```bash\r\n# to be announced\r\n```\r\n\r\n## Usage\r\n\r\nimport the `run_stonks_analysis` function from the `stonks` module and run the script:\r\n\r\n```python\r\nimport time\r\nfrom dotenv import load_dotenv\r\n\r\nfrom binance_archiver import load_config_from_json, DataSinkConfig, launch_data_sink\r\n\r\nif __name__ == \"__main__\":\r\n\r\n load_dotenv('sample_env.env')\r\n\r\n data_sink_config = DataSinkConfig(\r\n instruments={\r\n 'spot': ['BTCUSDT', 'XRPUSDT'],\r\n 'usd_m_futures': ['BTCUSDT', 'XRPUSDT'],\r\n 'coin_m_futures': ['BTCUSDT', 'XRPUSDT']\r\n },\r\n time_settings={\r\n \"file_duration_seconds\": 120,\r\n \"snapshot_fetcher_interval_seconds\": 300,\r\n \"websocket_life_time_seconds\": 120\r\n },\r\n data_save_target='azure_blob'\r\n )\r\n\r\n data_sink = launch_data_sink(data_sink_config=data_sink_config)\r\n \r\n time.sleep(99)\r\n \r\n data_sink.shutdown()\r\n \r\n\r\n```\r\n",
"bugtrack_url": null,
"license": null,
"summary": "A package for archiving Binance data",
"version": "0.0.3",
"project_urls": {
"Homepage": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
},
"split_keywords": [
"binance",
"archiver",
"quant",
"data",
"sprzedam",
"opla"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "c9313a7599dddabb41cccd53d5f3c2b0bb13e4210b56cb3b0de55ecbfbbc2175",
"md5": "a23306f4d82ba327909854ea29df39d2",
"sha256": "65daddba28bf3f1e6f7423db7620ce705fa67bf6c2aae23a0f1d91d1ebaee194"
},
"downloads": -1,
"filename": "binance_archiver-0.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a23306f4d82ba327909854ea29df39d2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 99251,
"upload_time": "2025-03-18T11:42:39",
"upload_time_iso_8601": "2025-03-18T11:42:39.883871Z",
"url": "https://files.pythonhosted.org/packages/c9/31/3a7599dddabb41cccd53d5f3c2b0bb13e4210b56cb3b0de55ecbfbbc2175/binance_archiver-0.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "1df7f7b5e26dcc6d2b42f935249eaa117b1442dc6c3d55c7a0700d4b3ad49cd5",
"md5": "2def6b9c8cc4ddae9c81796b55601532",
"sha256": "bda9bc7785eb3c06605ffb7d7e5518e4c67e1c838348fabbc69512700a17e4b7"
},
"downloads": -1,
"filename": "binance_archiver-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "2def6b9c8cc4ddae9c81796b55601532",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 81964,
"upload_time": "2025-03-18T11:42:41",
"upload_time_iso_8601": "2025-03-18T11:42:41.129366Z",
"url": "https://files.pythonhosted.org/packages/1d/f7/f7b5e26dcc6d2b42f935249eaa117b1442dc6c3d55c7a0700d4b3ad49cd5/binance_archiver-0.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-03-18 11:42:41",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "binance-archiver"
}
JSON