# Polygon ETL CLI
Polygon ETL lets you convert blockchain data into convenient formats like CSVs and relational databases.
[Full documentation available here](http://polygon-etl.readthedocs.io/).
## Quickstart
Install Polygon ETL CLI:
```bash
pip3 install polygon-etl
```
Export blocks, actions and logs ([Schema](../docs/schema.md), [Reference](../docs/commands.md)):
```bash
> polygonetl export_blocks_and_transactions --start-block 0 --end-block 500000 \
--provider-uri grpcs://api.mainnet.polygon.one:443 \
--blocks-output blocks.csv --transactions-output transactions.csv
```
---
Stream blocks, actions and logs to console ([Reference](../docs/commands.md#stream)):
```bash
> pip3 install polygon-etl[streaming]
> polygonetl stream --start-block 500000 -e block,action,log --log-file log.txt \
--provider-uri grpcs://api.mainnet.polygon.one:443
```
Find other commands [here](https://polygon-etl.readthedocs.io/en/latest/commands/).
For the latest version, check out the repo and call
```bash
> pip3 install -e .
> python3 polygonetl.py
```
## Useful Links
- [Schema](https://polygon-etl.readthedocs.io/en/latest/schema/)
- [Command Reference](https://polygon-etl.readthedocs.io/en/latest/commands/)
- [Documentation](https://polygon-etl.readthedocs.io/)
## Running Tests
```bash
> pip3 install -r ../requirements_test.txt -e .[streaming]
> export POLYGONETL_RUN_SLOW_TESTS=true
> export POLYGONETL_PROVIDER_URI=grpcs://api.mainnet.polygon.one:443
> pytest -vv
```
### Running Tox Tests
```bash
> pip3 install -r ../requirements_test.txt
> export POLYGONETL_RUN_SLOW_TESTS=true
> export POLYGONETL_PROVIDER_URI=grpcs://api.mainnet.polygon.one:443
> tox
```
## Running in Docker
1. Install Docker https://docs.docker.com/install/
2. Build a docker image
> docker build -t polygon-etl:latest .
> docker image ls
3. Run a container out of the image
> docker run -v $HOME/output:/polygon-etl/output polygon-etl:latest export_blocks -s 1 -e 5499999 -b 1000 -o out
4. Run streaming to console or Pub/Sub
> docker build -t polygon-etl:latest -f Dockerfile .
> echo "Stream to console"
> docker run polygon-etl:latest stream --start-block 500000 --log-file log.txt
> echo "Stream to Pub/Sub"
> docker run -v /path_to_credentials_file/:/polygon-etl/ --env GOOGLE_APPLICATION_CREDENTIALS=/polygon-etl/credentials_file.json polygon-etl:latest stream --start-block 500000 --output projects/<your-project>/topics/mainnet
Raw data
{
"_id": null,
"home_page": "https://github.com/blockchain-etl/polygon-etl",
"name": "polygon-etl",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7,<3.10",
"maintainer_email": "",
"keywords": "polygon",
"author": "Evgeny Medvedev",
"author_email": "evge.medvedev@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/4f/ca/1f6c8d69e1927d298c124f3ac249438985637e24ede4e6d75aeb78a20bd7/polygon-etl-0.5.0.tar.gz",
"platform": null,
"description": "# Polygon ETL CLI\n\n\nPolygon ETL lets you convert blockchain data into convenient formats like CSVs and relational databases.\n\n\n[Full documentation available here](http://polygon-etl.readthedocs.io/).\n\n## Quickstart\n\nInstall Polygon ETL CLI:\n\n```bash\npip3 install polygon-etl\n```\n\nExport blocks, actions and logs ([Schema](../docs/schema.md), [Reference](../docs/commands.md)):\n\n```bash\n> polygonetl export_blocks_and_transactions --start-block 0 --end-block 500000 \\\n--provider-uri grpcs://api.mainnet.polygon.one:443 \\\n--blocks-output blocks.csv --transactions-output transactions.csv\n```\n\n---\n\nStream blocks, actions and logs to console ([Reference](../docs/commands.md#stream)):\n\n```bash\n> pip3 install polygon-etl[streaming]\n> polygonetl stream --start-block 500000 -e block,action,log --log-file log.txt \\\n--provider-uri grpcs://api.mainnet.polygon.one:443\n```\n\nFind other commands [here](https://polygon-etl.readthedocs.io/en/latest/commands/).\n\nFor the latest version, check out the repo and call \n```bash\n> pip3 install -e . \n> python3 polygonetl.py\n```\n\n## Useful Links\n\n- [Schema](https://polygon-etl.readthedocs.io/en/latest/schema/)\n- [Command Reference](https://polygon-etl.readthedocs.io/en/latest/commands/)\n- [Documentation](https://polygon-etl.readthedocs.io/)\n\n## Running Tests\n\n```bash\n> pip3 install -r ../requirements_test.txt -e .[streaming]\n> export POLYGONETL_RUN_SLOW_TESTS=true\n> export POLYGONETL_PROVIDER_URI=grpcs://api.mainnet.polygon.one:443\n> pytest -vv\n```\n\n### Running Tox Tests\n\n```bash\n> pip3 install -r ../requirements_test.txt\n> export POLYGONETL_RUN_SLOW_TESTS=true\n> export POLYGONETL_PROVIDER_URI=grpcs://api.mainnet.polygon.one:443\n> tox\n```\n\n## Running in Docker\n\n1. Install Docker https://docs.docker.com/install/\n\n2. Build a docker image\n \n > docker build -t polygon-etl:latest .\n > docker image ls\n \n3. Run a container out of the image\n\n > docker run -v $HOME/output:/polygon-etl/output polygon-etl:latest export_blocks -s 1 -e 5499999 -b 1000 -o out\n\n4. Run streaming to console or Pub/Sub\n\n > docker build -t polygon-etl:latest -f Dockerfile .\n > echo \"Stream to console\"\n > docker run polygon-etl:latest stream --start-block 500000 --log-file log.txt\n > echo \"Stream to Pub/Sub\"\n > docker run -v /path_to_credentials_file/:/polygon-etl/ --env GOOGLE_APPLICATION_CREDENTIALS=/polygon-etl/credentials_file.json polygon-etl:latest stream --start-block 500000 --output projects/<your-project>/topics/mainnet",
"bugtrack_url": null,
"license": "",
"summary": "Tools for exporting Polygon blockchain data to CSV or JSON",
"version": "0.5.0",
"project_urls": {
"Bug Reports": "https://github.com/blockchain-etl/polygon-etl/issues",
"Homepage": "https://github.com/blockchain-etl/polygon-etl",
"Source": "https://github.com/blockchain-etl/polygon-etl"
},
"split_keywords": [
"polygon"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4fca1f6c8d69e1927d298c124f3ac249438985637e24ede4e6d75aeb78a20bd7",
"md5": "1641558c57c4e9338e13f0e31b1b3a7f",
"sha256": "8ad5227a3ac25cb6a908bc9f63db4c68cb4d9f312c59532de2393381ffb9dd43"
},
"downloads": -1,
"filename": "polygon-etl-0.5.0.tar.gz",
"has_sig": false,
"md5_digest": "1641558c57c4e9338e13f0e31b1b3a7f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7,<3.10",
"size": 55098,
"upload_time": "2023-11-21T10:05:30",
"upload_time_iso_8601": "2023-11-21T10:05:30.404845Z",
"url": "https://files.pythonhosted.org/packages/4f/ca/1f6c8d69e1927d298c124f3ac249438985637e24ede4e6d75aeb78a20bd7/polygon-etl-0.5.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-21 10:05:30",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "blockchain-etl",
"github_project": "polygon-etl",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "polygon-etl"
}
JSON
