# What?
This is the basis of a pipeline for live data from cryptocurrency exchanges. It normalizes [cryptofeed](https://github.com/bmoscon/cryptofeed) WebSocket messages, aggregates, and optionally publishes them to GCP Pub/Sub.
# How?
Sequences of trades that have equal symbol, timestamp, and tick rule are aggregated. Aggregating trades in this way can increase information, as they are either orders of size or stop loss cascades.
As well, the number of messages can be reduced by 30-50%
By filtering aggregated messages, for example only emitting a mesage when an aggregated trade is greater than or equal to a `significant_trade_filter`, the number of messages can be reduced more.
Example
-------
The following are two sequential aggregated trades by timestamp, nanoseconds, and tick rule.
As it was aggregated from 4 raw trades, the second trade has ticks 4.
```python
[
{
"timestamp": 1620000915.31424,
"price": "57064.01",
"volume": "566.6479018604",
"notional": "0.00993004",
"tickRule": -1,
"ticks": 1
},
{
"timestamp": 1620000915.885381,
"price": "57071.2",
"volume": "9376.6869202914",
"notional": "0.16429813",
"tickRule": 1,
"ticks": 4
}
]
```
An example filtered message, emitted because the second aggregated trade exceeds `significant_trade_filter >= 1000`
Information related to the first trade is aggregated with the second.
```python
[
{
"timestamp": 1620000915.885381,
"price": "57071.2",
"volume": "9376.6869202914",
"notional": "0.16429813",
"tickRule": 1,
"ticks": 4,
"high": '57071.2',
"low": '57064.01',
"totalBuyVolume": "9376.6869202914",
"totalVolume": "9943.3348221518",
"totalBuyNotional": "0.16429813",
"totalNotional": "0.17422817",
"totalBuyTicks": 4,
"totalTicks": 5
}
]
```
For 1m, 5m, 15m candles, there is an optional parameter `window_seconds`.
For settings, see the [examples](https://github.com/globophobe/cryptofeed-experiments/blob/main/examples/)
Supported exchanges
-------------------
:white_check_mark: Binance
:white_check_mark: Bitfinex
:white_check_mark: Bitflyer
:white_check_mark: BitMEX
:white_check_mark: Bybit
:white_check_mark: Coinbase Pro
:white_check_mark: Upbit
Contributing
------------
Install dependencies with `poetry install`. The docker example is built with [invoke tasks](https://github.com/globophobe/cryptofeed-experiments/blob/master/tasks.py). For example, `invoke build`
Future plans
------------
Order book aggregation, in the manner of [crypto-whale-watching-app](https://github.com/pmaji/crypto-whale-watching-app)
Raw data
{
"_id": null,
"home_page": "https://github.com/globophobe/cryptofeed-experiments",
"name": "cryptofeed-experiments",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8,<4.0",
"maintainer_email": "",
"keywords": "cryptocurrency,trading",
"author": "Alex",
"author_email": "globophobe@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/e4/a3/2abf5f82dac5b881da93b820e63d9ebfc7a88cc3f84f86868fb889ce4ee3/cryptofeed_experiments-0.1.7.tar.gz",
"platform": null,
"description": "# What?\n\nThis is the basis of a pipeline for live data from cryptocurrency exchanges. It normalizes [cryptofeed](https://github.com/bmoscon/cryptofeed) WebSocket messages, aggregates, and optionally publishes them to GCP Pub/Sub.\n\n# How?\n\nSequences of trades that have equal symbol, timestamp, and tick rule are aggregated. Aggregating trades in this way can increase information, as they are either orders of size or stop loss cascades.\n\nAs well, the number of messages can be reduced by 30-50%\n\nBy filtering aggregated messages, for example only emitting a mesage when an aggregated trade is greater than or equal to a `significant_trade_filter`, the number of messages can be reduced more.\n\nExample\n-------\nThe following are two sequential aggregated trades by timestamp, nanoseconds, and tick rule.\n\nAs it was aggregated from 4 raw trades, the second trade has ticks 4.\n\n```python\n[\n {\n \"timestamp\": 1620000915.31424,\n \"price\": \"57064.01\",\n \"volume\": \"566.6479018604\",\n \"notional\": \"0.00993004\",\n \"tickRule\": -1,\n \"ticks\": 1\n },\n {\n \"timestamp\": 1620000915.885381,\n \"price\": \"57071.2\",\n \"volume\": \"9376.6869202914\",\n \"notional\": \"0.16429813\",\n \"tickRule\": 1,\n \"ticks\": 4\n }\n]\n```\n\nAn example filtered message, emitted because the second aggregated trade exceeds `significant_trade_filter >= 1000`\n\nInformation related to the first trade is aggregated with the second.\n\n```python\n[\n {\n \"timestamp\": 1620000915.885381,\n \"price\": \"57071.2\",\n \"volume\": \"9376.6869202914\",\n \"notional\": \"0.16429813\",\n \"tickRule\": 1,\n \"ticks\": 4,\n \"high\": '57071.2',\n \"low\": '57064.01',\n \"totalBuyVolume\": \"9376.6869202914\",\n \"totalVolume\": \"9943.3348221518\",\n \"totalBuyNotional\": \"0.16429813\",\n \"totalNotional\": \"0.17422817\",\n \"totalBuyTicks\": 4,\n \"totalTicks\": 5\n }\n]\n```\n\nFor 1m, 5m, 15m candles, there is an optional parameter `window_seconds`. \n\nFor settings, see the [examples](https://github.com/globophobe/cryptofeed-experiments/blob/main/examples/)\n\nSupported exchanges\n-------------------\n\n:white_check_mark: Binance\n\n:white_check_mark: Bitfinex\n\n:white_check_mark: Bitflyer\n\n:white_check_mark: BitMEX\n\n:white_check_mark: Bybit\n\n:white_check_mark: Coinbase Pro\n\n:white_check_mark: Upbit\n\nContributing\n------------\n\nInstall dependencies with `poetry install`. The docker example is built with [invoke tasks](https://github.com/globophobe/cryptofeed-experiments/blob/master/tasks.py). For example, `invoke build`\n\nFuture plans\n------------\nOrder book aggregation, in the manner of [crypto-whale-watching-app](https://github.com/pmaji/crypto-whale-watching-app)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "WebSocket subclasses, and experiments, for data from cryptocurrency exchanges",
"version": "0.1.7",
"project_urls": {
"Homepage": "https://github.com/globophobe/cryptofeed-experiments",
"Repository": "https://github.com/globophobe/cryptofeed-experiments"
},
"split_keywords": [
"cryptocurrency",
"trading"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "43f1a997e1c00d8da63422139a4e174228becdb2f2afbf7b8346d2ada0341c5d",
"md5": "bbc4a0677af574258d2d6c355da38b78",
"sha256": "ef600d52a5a16be3d5726b4a48eaf46b3087418469af033e3bcc601c3ba29c47"
},
"downloads": -1,
"filename": "cryptofeed_experiments-0.1.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "bbc4a0677af574258d2d6c355da38b78",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8,<4.0",
"size": 18506,
"upload_time": "2023-11-12T05:01:43",
"upload_time_iso_8601": "2023-11-12T05:01:43.966058Z",
"url": "https://files.pythonhosted.org/packages/43/f1/a997e1c00d8da63422139a4e174228becdb2f2afbf7b8346d2ada0341c5d/cryptofeed_experiments-0.1.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e4a32abf5f82dac5b881da93b820e63d9ebfc7a88cc3f84f86868fb889ce4ee3",
"md5": "b87231bd863d3cb9b6d8a392845ba9df",
"sha256": "57f962b94826c88a86d3fe41e609037cbd073260236b374e2d7652bee64bfa15"
},
"downloads": -1,
"filename": "cryptofeed_experiments-0.1.7.tar.gz",
"has_sig": false,
"md5_digest": "b87231bd863d3cb9b6d8a392845ba9df",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8,<4.0",
"size": 12016,
"upload_time": "2023-11-12T05:01:45",
"upload_time_iso_8601": "2023-11-12T05:01:45.930479Z",
"url": "https://files.pythonhosted.org/packages/e4/a3/2abf5f82dac5b881da93b820e63d9ebfc7a88cc3f84f86868fb889ce4ee3/cryptofeed_experiments-0.1.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-12 05:01:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "globophobe",
"github_project": "cryptofeed-experiments",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "cryptofeed-experiments"
}
JSON
