yahoo-export


Nameyahoo-export JSON
Version 3.0.0 PyPI version JSON
download
home_pageNone
SummaryExport Yahoo Fantasy Football data from historical records and live data.
upload_time2023-11-19 16:50:35
maintainerNone
docs_urlNone
authorNone
requires_python>=3.10
licenseMIT
keywords api data export fantasy football yahoo
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # TODO

[![Downloads](https://img.shields.io/pypi/dm/mypy)](empty)
[![Stable Version](https://img.shields.io/pypi/v/mypy?color=blue)](empty)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/)
[![pypi](https://img.shields.io/pypi/v/ruff/0.0.287.svg)](empty)
[![license](https://img.shields.io/pypi/l/ruff/0.0.287.svg)](empty)
[![python versions](https://img.shields.io/pypi/pyversions/ruff/0.0.287.svg)](empty)
[![Actions status](https://github.com/astral-sh/ruff-pre-commit/workflows/main/badge.svg)](empty)
[![Build Status](https://github.com/python/mypy/actions/workflows/test.yml/badge.svg)](empty)
[![Documentation Status](https://readthedocs.org/projects/mypy/badge/?version=latest)](empty)

1. Data Loader:
    - Get Data from the yahoo api
    - Split accross seperate "Loaders" within Mage
    - Export Raw Json to Supabase
2. Transform Raw Json:
    - Transform data from raw json to parsed json
    - Export parsed json
3. Transform Parsed Json:
    - Transform parsed json to tabular form
    - Export to public schema

## Process

- Query yahoo api
  - DONE
- TODO:
  - Split across multiple "blocks" for each "live" api connection needed
    - Could use multiple api keys to help with rate limiting and speed
  - Once queried, data should load to database immediately
    - Jsonb format in postgresql
    - yahoo_data.raw_json
  - Transformation 1
    - Convert data to "parsed_json"
    - Upload to yahoo_data.parsed_json
  - Transformation 2
    - Convert to tabular format
    - Upload to yahoo_data.public

## Need to knows

- How to use the yahoo_api package created within Mage?
- Should I be using SQLAlchemy or could is suffice with psycopg (perferably v3, but can use v2 if needed)
  - Should this be Async?
  - Would multiple api calls/inserts affect ACID/locks?
  - If so, probably should use ORM?

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "yahoo-export",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.10",
    "maintainer_email": null,
    "keywords": "api,data,export,fantasy,football,yahoo",
    "author": null,
    "author_email": "Tim Cuddeback <cuddebtj@gmail.com>",
    "download_url": "https://files.pythonhosted.org/packages/b5/a4/07a88bf9f47e427ce5748efa754c33514b029ce449cb45a4a10ecebcb243/yahoo_export-3.0.0.tar.gz",
    "platform": null,
    "description": "# TODO\n\n[![Downloads](https://img.shields.io/pypi/dm/mypy)](empty)\n[![Stable Version](https://img.shields.io/pypi/v/mypy?color=blue)](empty)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/)\n[![pypi](https://img.shields.io/pypi/v/ruff/0.0.287.svg)](empty)\n[![license](https://img.shields.io/pypi/l/ruff/0.0.287.svg)](empty)\n[![python versions](https://img.shields.io/pypi/pyversions/ruff/0.0.287.svg)](empty)\n[![Actions status](https://github.com/astral-sh/ruff-pre-commit/workflows/main/badge.svg)](empty)\n[![Build Status](https://github.com/python/mypy/actions/workflows/test.yml/badge.svg)](empty)\n[![Documentation Status](https://readthedocs.org/projects/mypy/badge/?version=latest)](empty)\n\n1. Data Loader:\n    - Get Data from the yahoo api\n    - Split accross seperate \"Loaders\" within Mage\n    - Export Raw Json to Supabase\n2. Transform Raw Json:\n    - Transform data from raw json to parsed json\n    - Export parsed json\n3. Transform Parsed Json:\n    - Transform parsed json to tabular form\n    - Export to public schema\n\n## Process\n\n- Query yahoo api\n  - DONE\n- TODO:\n  - Split across multiple \"blocks\" for each \"live\" api connection needed\n    - Could use multiple api keys to help with rate limiting and speed\n  - Once queried, data should load to database immediately\n    - Jsonb format in postgresql\n    - yahoo_data.raw_json\n  - Transformation 1\n    - Convert data to \"parsed_json\"\n    - Upload to yahoo_data.parsed_json\n  - Transformation 2\n    - Convert to tabular format\n    - Upload to yahoo_data.public\n\n## Need to knows\n\n- How to use the yahoo_api package created within Mage?\n- Should I be using SQLAlchemy or could is suffice with psycopg (perferably v3, but can use v2 if needed)\n  - Should this be Async?\n  - Would multiple api calls/inserts affect ACID/locks?\n  - If so, probably should use ORM?\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "Export Yahoo Fantasy Football data from historical records and live data.",
    "version": "3.0.0",
    "project_urls": {
        "Documentation": "https://github.com/fantasy-sidelines/yahoo-export#readme",
        "Issues": "https://github.com/fantasy-sidelines/yahoo-export/issues",
        "Source": "https://github.com/fantasy-sidelines/yahoo-export"
    },
    "split_keywords": [
        "api",
        "data",
        "export",
        "fantasy",
        "football",
        "yahoo"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "99175a923283b0f0e43c41d76d12a8f6b85aaeb8853f2f6da565a6ec0acad93d",
                "md5": "4dca526b5cd5a74e3a88605e9fa7bdea",
                "sha256": "4d984033b19fc4bbb46904cd8ca19f833c46ef8ece5e0bfcfb4a0f5627f103df"
            },
            "downloads": -1,
            "filename": "yahoo_export-3.0.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "4dca526b5cd5a74e3a88605e9fa7bdea",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.10",
            "size": 7554,
            "upload_time": "2023-11-19T16:50:34",
            "upload_time_iso_8601": "2023-11-19T16:50:34.570570Z",
            "url": "https://files.pythonhosted.org/packages/99/17/5a923283b0f0e43c41d76d12a8f6b85aaeb8853f2f6da565a6ec0acad93d/yahoo_export-3.0.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "b5a407a88bf9f47e427ce5748efa754c33514b029ce449cb45a4a10ecebcb243",
                "md5": "726add25ff7db2067281d01e6a72d2d3",
                "sha256": "553a8cbdeb531472661e975fb813d57c9db19f8f76b09b1fdb77d7081c9a8bbc"
            },
            "downloads": -1,
            "filename": "yahoo_export-3.0.0.tar.gz",
            "has_sig": false,
            "md5_digest": "726add25ff7db2067281d01e6a72d2d3",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.10",
            "size": 9613,
            "upload_time": "2023-11-19T16:50:35",
            "upload_time_iso_8601": "2023-11-19T16:50:35.874683Z",
            "url": "https://files.pythonhosted.org/packages/b5/a4/07a88bf9f47e427ce5748efa754c33514b029ce449cb45a4a10ecebcb243/yahoo_export-3.0.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-11-19 16:50:35",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "fantasy-sidelines",
    "github_project": "yahoo-export#readme",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "yahoo-export"
}
        
Elapsed time: 0.12731s