# Graphsense fork of Ethereum ETL
[](https://travis-ci.com/github/blockchain-etl/ethereum-etl)
[](https://github.com/blockchain-etl/ethereum-etl/blob/develop/LICENSE)
[](https://t.me/BlockchainETL)
[](https://x.com/EthereumETL)
This fork is a purpose tailored version for the graphsense infrastructure.
Changes allow more recent python versions to be used [upstream ethereum etl] (https://github.com/ethereum-etl/ethereum-etl)
Ethereum ETL lets you convert blockchain data into convenient formats like CSVs and relational databases.
*Do you just want to query Ethereum data right away? Use the [public dataset in BigQuery](https://console.cloud.google.com/marketplace/details/ethereum/crypto-ethereum-blockchain).*
[Full documentation available here](http://ethereum-etl.readthedocs.io/).
## Quickstart
Install Graphsense Ethereum ETL:
```bash
pip3 install graphsense-ethereum-etl
```
Export blocks and transactions ([Schema](docs/schema.md#blockscsv), [Reference](docs/commands.md#export_blocks_and_transactions)):
```bash
> ethereumetl export_blocks_and_transactions --start-block 0 --end-block 500000 \
--blocks-output blocks.csv --transactions-output transactions.csv \
--provider-uri https://mainnet.infura.io/v3/7aef3f0cd1f64408b163814b22cc643c
```
Export ERC20 and ERC721 transfers ([Schema](docs/schema.md#token_transferscsv), [Reference](docs/commands.md#export_token_transfers)):
```bash
> ethereumetl export_token_transfers --start-block 0 --end-block 500000 \
--provider-uri file://$HOME/Library/Ethereum/geth.ipc --output token_transfers.csv
```
Export traces ([Schema](docs/schema.md#tracescsv), [Reference](docs/commands.md#export_traces)):
```bash
> ethereumetl export_traces --start-block 0 --end-block 500000 \
--provider-uri file://$HOME/Library/Ethereum/parity.ipc --output traces.csv
```
---
Stream blocks, transactions, logs, token_transfers continually to console ([Reference](docs/commands.md#stream)):
```bash
> pip3 install ethereum-etl[streaming]
> ethereumetl stream --start-block 500000 -e block,transaction,log,token_transfer --log-file log.txt \
--provider-uri https://mainnet.infura.io/v3/7aef3f0cd1f64408b163814b22cc643c
```
Find other commands [here](https://ethereum-etl.readthedocs.io/en/latest/commands/).
For the latest version, check out the repo and call
```bash
> pip3 install -e .
> python3 ethereumetl.py
```
## Useful Links
- [Schema](https://ethereum-etl.readthedocs.io/en/latest/schema/)
- [Command Reference](https://ethereum-etl.readthedocs.io/en/latest/commands/)
- [Documentation](https://ethereum-etl.readthedocs.io/)
- [Public Datasets in BigQuery](https://github.com/blockchain-etl/public-datasets)
- [Exporting the Blockchain](https://ethereum-etl.readthedocs.io/en/latest/exporting-the-blockchain/)
- [Querying in Amazon Athena](https://ethereum-etl.readthedocs.io/en/latest/amazon-athena/)
- [Querying in Google BigQuery](https://ethereum-etl.readthedocs.io/en/latest/google-bigquery/)
- [Querying in Kaggle](https://www.kaggle.com/bigquery/ethereum-blockchain)
- [Airflow DAGs](https://github.com/blockchain-etl/ethereum-etl-airflow)
- [Postgres ETL](https://github.com/blockchain-etl/ethereum-etl-postgresql)
- [Ethereum 2.0 ETL](https://github.com/blockchain-etl/ethereum2-etl)
## Running Tests
```bash
> pip3 install -e .[dev,streaming]
> export ETHEREUM_ETL_RUN_SLOW_TESTS=True
> export PROVIDER_URL=<your_provider_uri>
> pytest -vv
```
### Running Tox Tests
```bash
> pip3 install tox
> tox
```
## Running in Docker
1. Install Docker: https://docs.docker.com/get-docker/
2. Build a docker image
> docker build -t ethereum-etl:latest .
> docker image ls
3. Run a container out of the image
> docker run -v $HOME/output:/ethereum-etl/output ethereum-etl:latest export_all -s 0 -e 5499999 -b 100000 -p https://mainnet.infura.io
> docker run -v $HOME/output:/ethereum-etl/output ethereum-etl:latest export_all -s 2018-01-01 -e 2018-01-01 -p https://mainnet.infura.io
4. Run streaming to console or Pub/Sub
> docker build -t ethereum-etl:latest .
> echo "Stream to console"
> docker run ethereum-etl:latest stream --start-block 500000 --log-file log.txt
> echo "Stream to Pub/Sub"
> docker run -v /path_to_credentials_file/:/ethereum-etl/ --env GOOGLE_APPLICATION_CREDENTIALS=/ethereum-etl/credentials_file.json ethereum-etl:latest stream --start-block 500000 --output projects/<your_project>/topics/crypto_ethereum
If running on an Apple M1 chip add the `--platform linux/x86_64` option to the `build` and `run` commands e.g.:
```
docker build --platform linux/x86_64 -t ethereum-etl:latest .
docker run --platform linux/x86_64 ethereum-etl:latest stream --start-block 500000
```
## Projects using Ethereum ETL
* [Google](https://goo.gl/oY5BCQ) - Public BigQuery Ethereum datasets
* [Nansen](https://nansen.ai/query?ref=ethereumetl) - Analytics platform for Ethereum
Raw data
{
"_id": null,
"home_page": "https://github.com/graphsense/ethereum-etl",
"name": "graphsense-ethereum-etl",
"maintainer": null,
"docs_url": null,
"requires_python": "<4,>=3.7.2",
"maintainer_email": null,
"keywords": "ethereum",
"author": "Iknaio Cryptoasset Analytics GmbH",
"author_email": "contact@ikna.io",
"download_url": "https://files.pythonhosted.org/packages/ee/8f/fde461044493ee8080e0d6c09e42b4b9621e9b83a550f0422fd773a66d66/graphsense_ethereum_etl-2.4.4.tar.gz",
"platform": null,
"description": "# Graphsense fork of Ethereum ETL\n\n[](https://travis-ci.com/github/blockchain-etl/ethereum-etl)\n[](https://github.com/blockchain-etl/ethereum-etl/blob/develop/LICENSE)\n[](https://t.me/BlockchainETL)\n[](https://x.com/EthereumETL)\n\nThis fork is a purpose tailored version for the graphsense infrastructure.\n\nChanges allow more recent python versions to be used [upstream ethereum etl] (https://github.com/ethereum-etl/ethereum-etl)\n\nEthereum ETL lets you convert blockchain data into convenient formats like CSVs and relational databases.\n\n*Do you just want to query Ethereum data right away? Use the [public dataset in BigQuery](https://console.cloud.google.com/marketplace/details/ethereum/crypto-ethereum-blockchain).*\n\n[Full documentation available here](http://ethereum-etl.readthedocs.io/).\n\n## Quickstart\n\nInstall Graphsense Ethereum ETL:\n\n```bash\npip3 install graphsense-ethereum-etl\n```\n\nExport blocks and transactions ([Schema](docs/schema.md#blockscsv), [Reference](docs/commands.md#export_blocks_and_transactions)):\n\n```bash\n> ethereumetl export_blocks_and_transactions --start-block 0 --end-block 500000 \\\n--blocks-output blocks.csv --transactions-output transactions.csv \\\n--provider-uri https://mainnet.infura.io/v3/7aef3f0cd1f64408b163814b22cc643c\n```\n\nExport ERC20 and ERC721 transfers ([Schema](docs/schema.md#token_transferscsv), [Reference](docs/commands.md#export_token_transfers)):\n\n```bash\n> ethereumetl export_token_transfers --start-block 0 --end-block 500000 \\\n--provider-uri file://$HOME/Library/Ethereum/geth.ipc --output token_transfers.csv\n```\n\nExport traces ([Schema](docs/schema.md#tracescsv), [Reference](docs/commands.md#export_traces)):\n\n```bash\n> ethereumetl export_traces --start-block 0 --end-block 500000 \\\n--provider-uri file://$HOME/Library/Ethereum/parity.ipc --output traces.csv\n```\n\n---\n\nStream blocks, transactions, logs, token_transfers continually to console ([Reference](docs/commands.md#stream)):\n\n```bash\n> pip3 install ethereum-etl[streaming]\n> ethereumetl stream --start-block 500000 -e block,transaction,log,token_transfer --log-file log.txt \\\n--provider-uri https://mainnet.infura.io/v3/7aef3f0cd1f64408b163814b22cc643c\n```\n\nFind other commands [here](https://ethereum-etl.readthedocs.io/en/latest/commands/).\n\nFor the latest version, check out the repo and call \n```bash\n> pip3 install -e . \n> python3 ethereumetl.py\n```\n\n## Useful Links\n\n- [Schema](https://ethereum-etl.readthedocs.io/en/latest/schema/)\n- [Command Reference](https://ethereum-etl.readthedocs.io/en/latest/commands/)\n- [Documentation](https://ethereum-etl.readthedocs.io/)\n- [Public Datasets in BigQuery](https://github.com/blockchain-etl/public-datasets) \n- [Exporting the Blockchain](https://ethereum-etl.readthedocs.io/en/latest/exporting-the-blockchain/)\n- [Querying in Amazon Athena](https://ethereum-etl.readthedocs.io/en/latest/amazon-athena/)\n- [Querying in Google BigQuery](https://ethereum-etl.readthedocs.io/en/latest/google-bigquery/)\n- [Querying in Kaggle](https://www.kaggle.com/bigquery/ethereum-blockchain)\n- [Airflow DAGs](https://github.com/blockchain-etl/ethereum-etl-airflow)\n- [Postgres ETL](https://github.com/blockchain-etl/ethereum-etl-postgresql)\n- [Ethereum 2.0 ETL](https://github.com/blockchain-etl/ethereum2-etl)\n\n## Running Tests\n\n```bash\n> pip3 install -e .[dev,streaming]\n> export ETHEREUM_ETL_RUN_SLOW_TESTS=True\n> export PROVIDER_URL=<your_provider_uri>\n> pytest -vv\n``` \n\n### Running Tox Tests\n\n```bash\n> pip3 install tox\n> tox\n```\n\n## Running in Docker\n\n1. Install Docker: https://docs.docker.com/get-docker/\n\n2. Build a docker image\n \n > docker build -t ethereum-etl:latest .\n > docker image ls\n \n3. Run a container out of the image\n\n > docker run -v $HOME/output:/ethereum-etl/output ethereum-etl:latest export_all -s 0 -e 5499999 -b 100000 -p https://mainnet.infura.io\n > docker run -v $HOME/output:/ethereum-etl/output ethereum-etl:latest export_all -s 2018-01-01 -e 2018-01-01 -p https://mainnet.infura.io\n\n4. Run streaming to console or Pub/Sub\n\n > docker build -t ethereum-etl:latest .\n > echo \"Stream to console\"\n > docker run ethereum-etl:latest stream --start-block 500000 --log-file log.txt\n > echo \"Stream to Pub/Sub\"\n > docker run -v /path_to_credentials_file/:/ethereum-etl/ --env GOOGLE_APPLICATION_CREDENTIALS=/ethereum-etl/credentials_file.json ethereum-etl:latest stream --start-block 500000 --output projects/<your_project>/topics/crypto_ethereum\n\nIf running on an Apple M1 chip add the `--platform linux/x86_64` option to the `build` and `run` commands e.g.:\n\n```\ndocker build --platform linux/x86_64 -t ethereum-etl:latest .\ndocker run --platform linux/x86_64 ethereum-etl:latest stream --start-block 500000\n```\n\n## Projects using Ethereum ETL\n* [Google](https://goo.gl/oY5BCQ) - Public BigQuery Ethereum datasets\n* [Nansen](https://nansen.ai/query?ref=ethereumetl) - Analytics platform for Ethereum\n",
"bugtrack_url": null,
"license": null,
"summary": "Tools for exporting Ethereum blockchain data to CSV or JSON",
"version": "2.4.4",
"project_urls": {
"Bug Reports": "https://github.com/blockchain-etl/ethereum-etl/issues",
"Chat": "https://gitter.im/ethereum-etl/Lobby",
"Homepage": "https://github.com/graphsense/ethereum-etl",
"Source": "https://github.com/blockchain-etl/ethereum-etl"
},
"split_keywords": [
"ethereum"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "e9d705133b5ffd3bf6e51617666feec48737b9ffcc709005aa79c6f84038552a",
"md5": "633c59d11d23057c1e6029252600e3de",
"sha256": "86c4c8aa10183cb163ad6a580d09a84f611402f1cae26669cc69a9cf52caef78"
},
"downloads": -1,
"filename": "graphsense_ethereum_etl-2.4.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "633c59d11d23057c1e6029252600e3de",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4,>=3.7.2",
"size": 441319,
"upload_time": "2025-07-10T12:15:57",
"upload_time_iso_8601": "2025-07-10T12:15:57.308113Z",
"url": "https://files.pythonhosted.org/packages/e9/d7/05133b5ffd3bf6e51617666feec48737b9ffcc709005aa79c6f84038552a/graphsense_ethereum_etl-2.4.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "ee8ffde461044493ee8080e0d6c09e42b4b9621e9b83a550f0422fd773a66d66",
"md5": "ad5d8fd9d9f591a6679b2682731bcc73",
"sha256": "607944c2f7afa77f02b38ab095882c04a7056b8a2d82db4e267a578e9162fbbc"
},
"downloads": -1,
"filename": "graphsense_ethereum_etl-2.4.4.tar.gz",
"has_sig": false,
"md5_digest": "ad5d8fd9d9f591a6679b2682731bcc73",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4,>=3.7.2",
"size": 342619,
"upload_time": "2025-07-10T12:15:58",
"upload_time_iso_8601": "2025-07-10T12:15:58.657114Z",
"url": "https://files.pythonhosted.org/packages/ee/8f/fde461044493ee8080e0d6c09e42b4b9621e9b83a550f0422fd773a66d66/graphsense_ethereum_etl-2.4.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-10 12:15:58",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "graphsense",
"github_project": "ethereum-etl",
"travis_ci": true,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "graphsense-ethereum-etl"
}
JSON
