binance-archiver


Namebinance-archiver JSON
Version 0.0.3 PyPI version JSON
download
home_pagehttps://www.youtube.com/watch?v=dQw4w9WgXcQ
SummaryA package for archiving Binance data
upload_time2025-03-18 11:42:41
maintainerNone
docs_urlNone
authorDaniel Lasota
requires_python>=3.11
licenseNone
keywords binance archiver quant data sprzedam opla
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # Real-time market data listener and archiver in python. 
Saves raw binance data from:
limit orderbook difference depth stream
limit orderbook difference depth snapshot request
transaction stream

into .json / .json.zip / .json.zip on azure or backblaze

simply enter azure backblaze connection params into environment variables and import 

'launch_data_sink' - use as data sink

or 

'launch_data_listener' - Use in your algo trading as it just informs all observers without data save

NEW!!!:
Although Binance now limits each WebSocket connection to 24 hours, this code runs continuously (24/7) without missing any stream entries!


# Handles: 
spot, futures usd-m, futures coin-m
Level 2 orderbook difference depth stream
trade stream
orderbook snapshots with configured trigger interval 
24-hour WebSocket lifecycle. At the end of the WebSocket's lifespan, it initiates a new WebSocket to ensure the continuity of data flow is maintained seamlessly.

Configured to use as container on Azure with Azure blob / keyvault or backblaze

![image](https://github.com/user-attachments/assets/a9461c8d-b5a7-43de-b1cc-96ef5df72f40)

![image](https://github.com/user-attachments/assets/93a9cece-21fd-406c-8555-fbb774188265)

![Zrzut ekranu 2024-06-02 230137](https://github.com/DanielLasota/Binance-Archiver/assets/127039319/b400f859-60ef-4995-936d-d68ecab82ddf)



## Installation

```bash
# to be announced
```

## Usage

import the `run_stonks_analysis` function from the `stonks` module and run the script:

```python
import time
from dotenv import load_dotenv

from binance_archiver import load_config_from_json, DataSinkConfig, launch_data_sink

if __name__ == "__main__":

    load_dotenv('sample_env.env')

    data_sink_config = DataSinkConfig(
        instruments={
            'spot': ['BTCUSDT', 'XRPUSDT'],
            'usd_m_futures': ['BTCUSDT', 'XRPUSDT'],
            'coin_m_futures': ['BTCUSDT', 'XRPUSDT']
        },
        time_settings={
            "file_duration_seconds": 120,
            "snapshot_fetcher_interval_seconds": 300,
            "websocket_life_time_seconds": 120
        },
        data_save_target='azure_blob'
    )

    data_sink = launch_data_sink(data_sink_config=data_sink_config)
    
    time.sleep(99)
    
    data_sink.shutdown()
    

```

            

Raw data

            {
    "_id": null,
    "home_page": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
    "name": "binance-archiver",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.11",
    "maintainer_email": null,
    "keywords": "binance archiver quant data sprzedam opla",
    "author": "Daniel Lasota",
    "author_email": "grossmann.root@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/1d/f7/f7b5e26dcc6d2b42f935249eaa117b1442dc6c3d55c7a0700d4b3ad49cd5/binance_archiver-0.0.3.tar.gz",
    "platform": null,
    "description": "# Real-time market data listener and archiver in python. \r\nSaves raw binance data from:\r\nlimit orderbook difference depth stream\r\nlimit orderbook difference depth snapshot request\r\ntransaction stream\r\n\r\ninto .json / .json.zip / .json.zip on azure or backblaze\r\n\r\nsimply enter azure backblaze connection params into environment variables and import \r\n\r\n'launch_data_sink' - use as data sink\r\n\r\nor \r\n\r\n'launch_data_listener' - Use in your algo trading as it just informs all observers without data save\r\n\r\nNEW!!!:\r\nAlthough Binance now limits each WebSocket connection to 24 hours, this code runs continuously (24/7) without missing any stream entries!\r\n\r\n\r\n# Handles: \r\nspot, futures usd-m, futures coin-m\r\nLevel 2 orderbook difference depth stream\r\ntrade stream\r\norderbook snapshots with configured trigger interval \r\n24-hour WebSocket lifecycle. At the end of the WebSocket's lifespan, it initiates a new WebSocket to ensure the continuity of data flow is maintained seamlessly.\r\n\r\nConfigured to use as container on Azure with Azure blob / keyvault or backblaze\r\n\r\n![image](https://github.com/user-attachments/assets/a9461c8d-b5a7-43de-b1cc-96ef5df72f40)\r\n\r\n![image](https://github.com/user-attachments/assets/93a9cece-21fd-406c-8555-fbb774188265)\r\n\r\n![Zrzut ekranu 2024-06-02 230137](https://github.com/DanielLasota/Binance-Archiver/assets/127039319/b400f859-60ef-4995-936d-d68ecab82ddf)\r\n\r\n\r\n\r\n## Installation\r\n\r\n```bash\r\n# to be announced\r\n```\r\n\r\n## Usage\r\n\r\nimport the `run_stonks_analysis` function from the `stonks` module and run the script:\r\n\r\n```python\r\nimport time\r\nfrom dotenv import load_dotenv\r\n\r\nfrom binance_archiver import load_config_from_json, DataSinkConfig, launch_data_sink\r\n\r\nif __name__ == \"__main__\":\r\n\r\n    load_dotenv('sample_env.env')\r\n\r\n    data_sink_config = DataSinkConfig(\r\n        instruments={\r\n            'spot': ['BTCUSDT', 'XRPUSDT'],\r\n            'usd_m_futures': ['BTCUSDT', 'XRPUSDT'],\r\n            'coin_m_futures': ['BTCUSDT', 'XRPUSDT']\r\n        },\r\n        time_settings={\r\n            \"file_duration_seconds\": 120,\r\n            \"snapshot_fetcher_interval_seconds\": 300,\r\n            \"websocket_life_time_seconds\": 120\r\n        },\r\n        data_save_target='azure_blob'\r\n    )\r\n\r\n    data_sink = launch_data_sink(data_sink_config=data_sink_config)\r\n    \r\n    time.sleep(99)\r\n    \r\n    data_sink.shutdown()\r\n    \r\n\r\n```\r\n",
    "bugtrack_url": null,
    "license": null,
    "summary": "A package for archiving Binance data",
    "version": "0.0.3",
    "project_urls": {
        "Homepage": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
    },
    "split_keywords": [
        "binance",
        "archiver",
        "quant",
        "data",
        "sprzedam",
        "opla"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "c9313a7599dddabb41cccd53d5f3c2b0bb13e4210b56cb3b0de55ecbfbbc2175",
                "md5": "a23306f4d82ba327909854ea29df39d2",
                "sha256": "65daddba28bf3f1e6f7423db7620ce705fa67bf6c2aae23a0f1d91d1ebaee194"
            },
            "downloads": -1,
            "filename": "binance_archiver-0.0.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "a23306f4d82ba327909854ea29df39d2",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.11",
            "size": 99251,
            "upload_time": "2025-03-18T11:42:39",
            "upload_time_iso_8601": "2025-03-18T11:42:39.883871Z",
            "url": "https://files.pythonhosted.org/packages/c9/31/3a7599dddabb41cccd53d5f3c2b0bb13e4210b56cb3b0de55ecbfbbc2175/binance_archiver-0.0.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "1df7f7b5e26dcc6d2b42f935249eaa117b1442dc6c3d55c7a0700d4b3ad49cd5",
                "md5": "2def6b9c8cc4ddae9c81796b55601532",
                "sha256": "bda9bc7785eb3c06605ffb7d7e5518e4c67e1c838348fabbc69512700a17e4b7"
            },
            "downloads": -1,
            "filename": "binance_archiver-0.0.3.tar.gz",
            "has_sig": false,
            "md5_digest": "2def6b9c8cc4ddae9c81796b55601532",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.11",
            "size": 81964,
            "upload_time": "2025-03-18T11:42:41",
            "upload_time_iso_8601": "2025-03-18T11:42:41.129366Z",
            "url": "https://files.pythonhosted.org/packages/1d/f7/f7b5e26dcc6d2b42f935249eaa117b1442dc6c3d55c7a0700d4b3ad49cd5/binance_archiver-0.0.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-03-18 11:42:41",
    "github": false,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "lcname": "binance-archiver"
}
        
Elapsed time: 0.41461s