coingecko-exporter


Namecoingecko-exporter JSON
Version 0.3.3 PyPI version JSON
download
home_pagehttps://github.com/MattMaximo/coingecko_exporter
SummaryA package to export bulk data from the CoinGecko API.
upload_time2024-08-20 17:54:04
maintainerNone
docs_urlNone
authorMatt Maximo
requires_python>=3.6
licenseNone
keywords coingecko crypto data api exporter
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            
# CoinGecko API Wrapper

This project provides a Python wrapper for the CoinGecko API, enabling users to fetch cryptocurrency data, process it into pandas DataFrames, and upload it to AWS S3 as Parquet files. The wrapper abstracts away the complexity of asynchronous API calls and provides an easy-to-use interface for interacting with the CoinGecko API.

## Table of Contents

- [Installation](#installation)
- [Class Structure](#class-structure)
  - [CoinGeckoAPI](#coingeckoapi)
  - [CoinGeckoDataProcessor](#coingeckodataprocessor)
  - [CoinGecko](#coingecko)
- [Usage](#usage)
  - [Fetching Categories](#fetching-categories)
  - [Fetching Coins](#fetching-coins)
  - [Fetching Timeseries Data](#fetching-timeseries-data)
  - [Exporting Data to Parquet](#exporting-data-to-parquet)
  - [Uploading Data to S3](#uploading-data-to-s3)
- [Contributing](#contributing)
- [License](#license)

## Installation

1. **Clone the repository:**

   \`\`\`bash
   git clone https://github.com/yourusername/coingecko-api-wrapper.git
   cd coingecko-api-wrapper
   \`\`\`

2. **Set up a virtual environment:**

   \`\`\`bash
   python3 -m venv venv
   source venv/bin/activate
   \`\`\`

3. **Install the required dependencies:**

   \`\`\`bash
   pip install -r requirements.txt
   \`\`\`

## Class Structure

### CoinGeckoAPI

The \`CoinGeckoAPI\` class is responsible for interacting directly with the CoinGecko API. It handles:

- Making asynchronous GET requests to the API.
- Managing rate limits using \`AsyncLimiter\`.
- Fetching data from paginated endpoints.

#### Methods:

- \`fetch(endpoint: str, params: dict = None, retries: int = 3) -> dict\`: Performs a GET request to the specified endpoint.
- \`fetch_all_pages(endpoint: str, params: dict, max_pages: int) -> list\`: Fetches all pages of data asynchronously.

### CoinGeckoDataProcessor

The \`CoinGeckoDataProcessor\` class processes raw data fetched from the CoinGecko API. It handles:

- Merging timeseries data into a pandas DataFrame.
- Saving DataFrames to Parquet files.

#### Methods:

- \`merge_timeseries_data(data: dict, coingecko_id: str) -> pd.DataFrame\`: Cleans and merges timeseries data.
- \`save_to_parquet(df: pd.DataFrame, file_path: str)\`: Saves a DataFrame to a Parquet file.

### CoinGecko

The \`CoinGecko\` class is the main interface for users. It abstracts away the complexities of asynchronous calls and provides simple, synchronous methods to fetch data, process it, and upload it to S3.

#### Methods:

- \`get_categories() -> pd.DataFrame\`: Fetches all categories from the CoinGecko API.
- \`get_coins(coins: int, category: str = None) -> pd.DataFrame\`: Fetches a list of assets based on market cap.
- \`get_timeseries(coingecko_ids: list) -> pd.DataFrame\`: Fetches and processes timeseries data.
- \`export_data(coins: Union[int, List[str]], export_format: str = 'df') -> pd.DataFrame\`: Fetches and exports data as a DataFrame or saves it as a Parquet file.
- \`upload_to_s3(df: pd.DataFrame, aws_access_key_id: str, aws_secret_access_key: str, bucket_name: str, folder_name: str = None, file_name: str = None)\`: Uploads a DataFrame to AWS S3 as a Parquet file.

## Usage

### Fetching Categories

\`\`\`python
from coingecko import CoinGecko

cg = CoinGecko(api_key="your-api-key")

# Fetch and print the categories DataFrame
categories_df = cg.get_categories()
print(categories_df)
\`\`\`

### Fetching Coins

\`\`\`python
from coingecko import CoinGecko

cg = CoinGecko(api_key="your-api-key")

# Fetch top 100 coins by market cap
coins_df = cg.get_coins(100)
print(coins_df)
\`\`\`

### Fetching Timeseries Data

\`\`\`python
from coingecko import CoinGecko

cg = CoinGecko(api_key="your-api-key")

# Fetch timeseries data for a list of coin IDs
timeseries_df = cg.get_timeseries(['bitcoin', 'ethereum'])
print(timeseries_df)
\`\`\`

### Exporting Data to Parquet

\`\`\`python
from coingecko import CoinGecko

cg = CoinGecko(api_key="your-api-key")

# Export data to a Parquet file
cg.export_data(coins=100, export_format='parquet')
\`\`\`

### Uploading Data to S3

\`\`\`python
from coingecko import CoinGecko

cg = CoinGecko(api_key="your-api-key")

# Fetch and upload coins data to S3
coins_df = cg.get_coins(100)
cg.upload_to_s3(
    df=coins_df,
    aws_access_key_id="your-aws-access-key-id",
    aws_secret_access_key="your-aws-secret-access-key",
    bucket_name="your-s3-bucket-name",
    folder_name="your-folder-name",
    file_name="your-file-name.parquet"
)
\`\`\`

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/MattMaximo/coingecko_exporter",
    "name": "coingecko-exporter",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.6",
    "maintainer_email": null,
    "keywords": "coingecko, crypto, data, api, exporter",
    "author": "Matt Maximo",
    "author_email": "matt@pioneerdigital.org",
    "download_url": "https://files.pythonhosted.org/packages/37/bb/bba055e57b41b6f2f484ccfe906ed6b37d1b7a42841ef17a1f01bd212a99/coingecko_exporter-0.3.3.tar.gz",
    "platform": null,
    "description": "\n# CoinGecko API Wrapper\n\nThis project provides a Python wrapper for the CoinGecko API, enabling users to fetch cryptocurrency data, process it into pandas DataFrames, and upload it to AWS S3 as Parquet files. The wrapper abstracts away the complexity of asynchronous API calls and provides an easy-to-use interface for interacting with the CoinGecko API.\n\n## Table of Contents\n\n- [Installation](#installation)\n- [Class Structure](#class-structure)\n  - [CoinGeckoAPI](#coingeckoapi)\n  - [CoinGeckoDataProcessor](#coingeckodataprocessor)\n  - [CoinGecko](#coingecko)\n- [Usage](#usage)\n  - [Fetching Categories](#fetching-categories)\n  - [Fetching Coins](#fetching-coins)\n  - [Fetching Timeseries Data](#fetching-timeseries-data)\n  - [Exporting Data to Parquet](#exporting-data-to-parquet)\n  - [Uploading Data to S3](#uploading-data-to-s3)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Installation\n\n1. **Clone the repository:**\n\n   \\`\\`\\`bash\n   git clone https://github.com/yourusername/coingecko-api-wrapper.git\n   cd coingecko-api-wrapper\n   \\`\\`\\`\n\n2. **Set up a virtual environment:**\n\n   \\`\\`\\`bash\n   python3 -m venv venv\n   source venv/bin/activate\n   \\`\\`\\`\n\n3. **Install the required dependencies:**\n\n   \\`\\`\\`bash\n   pip install -r requirements.txt\n   \\`\\`\\`\n\n## Class Structure\n\n### CoinGeckoAPI\n\nThe \\`CoinGeckoAPI\\` class is responsible for interacting directly with the CoinGecko API. It handles:\n\n- Making asynchronous GET requests to the API.\n- Managing rate limits using \\`AsyncLimiter\\`.\n- Fetching data from paginated endpoints.\n\n#### Methods:\n\n- \\`fetch(endpoint: str, params: dict = None, retries: int = 3) -> dict\\`: Performs a GET request to the specified endpoint.\n- \\`fetch_all_pages(endpoint: str, params: dict, max_pages: int) -> list\\`: Fetches all pages of data asynchronously.\n\n### CoinGeckoDataProcessor\n\nThe \\`CoinGeckoDataProcessor\\` class processes raw data fetched from the CoinGecko API. It handles:\n\n- Merging timeseries data into a pandas DataFrame.\n- Saving DataFrames to Parquet files.\n\n#### Methods:\n\n- \\`merge_timeseries_data(data: dict, coingecko_id: str) -> pd.DataFrame\\`: Cleans and merges timeseries data.\n- \\`save_to_parquet(df: pd.DataFrame, file_path: str)\\`: Saves a DataFrame to a Parquet file.\n\n### CoinGecko\n\nThe \\`CoinGecko\\` class is the main interface for users. It abstracts away the complexities of asynchronous calls and provides simple, synchronous methods to fetch data, process it, and upload it to S3.\n\n#### Methods:\n\n- \\`get_categories() -> pd.DataFrame\\`: Fetches all categories from the CoinGecko API.\n- \\`get_coins(coins: int, category: str = None) -> pd.DataFrame\\`: Fetches a list of assets based on market cap.\n- \\`get_timeseries(coingecko_ids: list) -> pd.DataFrame\\`: Fetches and processes timeseries data.\n- \\`export_data(coins: Union[int, List[str]], export_format: str = 'df') -> pd.DataFrame\\`: Fetches and exports data as a DataFrame or saves it as a Parquet file.\n- \\`upload_to_s3(df: pd.DataFrame, aws_access_key_id: str, aws_secret_access_key: str, bucket_name: str, folder_name: str = None, file_name: str = None)\\`: Uploads a DataFrame to AWS S3 as a Parquet file.\n\n## Usage\n\n### Fetching Categories\n\n\\`\\`\\`python\nfrom coingecko import CoinGecko\n\ncg = CoinGecko(api_key=\"your-api-key\")\n\n# Fetch and print the categories DataFrame\ncategories_df = cg.get_categories()\nprint(categories_df)\n\\`\\`\\`\n\n### Fetching Coins\n\n\\`\\`\\`python\nfrom coingecko import CoinGecko\n\ncg = CoinGecko(api_key=\"your-api-key\")\n\n# Fetch top 100 coins by market cap\ncoins_df = cg.get_coins(100)\nprint(coins_df)\n\\`\\`\\`\n\n### Fetching Timeseries Data\n\n\\`\\`\\`python\nfrom coingecko import CoinGecko\n\ncg = CoinGecko(api_key=\"your-api-key\")\n\n# Fetch timeseries data for a list of coin IDs\ntimeseries_df = cg.get_timeseries(['bitcoin', 'ethereum'])\nprint(timeseries_df)\n\\`\\`\\`\n\n### Exporting Data to Parquet\n\n\\`\\`\\`python\nfrom coingecko import CoinGecko\n\ncg = CoinGecko(api_key=\"your-api-key\")\n\n# Export data to a Parquet file\ncg.export_data(coins=100, export_format='parquet')\n\\`\\`\\`\n\n### Uploading Data to S3\n\n\\`\\`\\`python\nfrom coingecko import CoinGecko\n\ncg = CoinGecko(api_key=\"your-api-key\")\n\n# Fetch and upload coins data to S3\ncoins_df = cg.get_coins(100)\ncg.upload_to_s3(\n    df=coins_df,\n    aws_access_key_id=\"your-aws-access-key-id\",\n    aws_secret_access_key=\"your-aws-secret-access-key\",\n    bucket_name=\"your-s3-bucket-name\",\n    folder_name=\"your-folder-name\",\n    file_name=\"your-file-name.parquet\"\n)\n\\`\\`\\`\n",
    "bugtrack_url": null,
    "license": null,
    "summary": "A package to export bulk data from the CoinGecko API.",
    "version": "0.3.3",
    "project_urls": {
        "Homepage": "https://github.com/MattMaximo/coingecko_exporter"
    },
    "split_keywords": [
        "coingecko",
        " crypto",
        " data",
        " api",
        " exporter"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "208e37ee1a8c4d7e4bdb86329837338aaa0e0a170701e066184914f0e0b3581a",
                "md5": "e1b8c9795983fb53c9212850a00d652d",
                "sha256": "11fe638efc5aa17740785422c033251fa20216dca521d86be70b944dada03092"
            },
            "downloads": -1,
            "filename": "coingecko_exporter-0.3.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "e1b8c9795983fb53c9212850a00d652d",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.6",
            "size": 5969,
            "upload_time": "2024-08-20T17:54:01",
            "upload_time_iso_8601": "2024-08-20T17:54:01.631710Z",
            "url": "https://files.pythonhosted.org/packages/20/8e/37ee1a8c4d7e4bdb86329837338aaa0e0a170701e066184914f0e0b3581a/coingecko_exporter-0.3.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "37bbbba055e57b41b6f2f484ccfe906ed6b37d1b7a42841ef17a1f01bd212a99",
                "md5": "bf5589659f89c2e68d53568e08765e53",
                "sha256": "e58b0356bb682f7dbe53c590338604d5518dcc4e714df4c8e46435f71e62e32a"
            },
            "downloads": -1,
            "filename": "coingecko_exporter-0.3.3.tar.gz",
            "has_sig": false,
            "md5_digest": "bf5589659f89c2e68d53568e08765e53",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.6",
            "size": 5564,
            "upload_time": "2024-08-20T17:54:04",
            "upload_time_iso_8601": "2024-08-20T17:54:04.698210Z",
            "url": "https://files.pythonhosted.org/packages/37/bb/bba055e57b41b6f2f484ccfe906ed6b37d1b7a42841ef17a1f01bd212a99/coingecko_exporter-0.3.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-08-20 17:54:04",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "MattMaximo",
    "github_project": "coingecko_exporter",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "coingecko-exporter"
}
        
Elapsed time: 0.32375s