ParlPy


NameParlPy JSON
Version 0.1.0 PyPI version JSON
download
home_pagehttps://github.com/Litharge/ParlPy
SummaryScraper and utilities for UK Parliamentary bills
upload_time2021-02-23 18:03:21
maintainer
docs_urlNone
authorRobert Chambers
requires_python
license
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # ParlPy

Package to scrape and process UK Parliamentary bills, votes and MP contact information.

Written as part of University of Bath Integrated Project module. 

---

# Usage

## Install

    pipenv install ParlPy=0.1.0

## Example Usage

To fetch bills updated since our scraper was last called (uses pickled variable for persistence between program runs)

    from parlpy.bills.bill_list_fetcher import BillsOverview

    test_fetcher = BillsOverview()
    test_fetcher.get_changed_bills_in_session(session="All")
    print(test_fetcher.bills_overview_data)

... script can stop and be rerun later, next time bills_overview_data will contain fewer items, unless 
reset_datetime_last_scraped() has been called first

To fetch all bills in session and print them:

    from parlpy.bills.bill_list_fetcher import BillsOverview

    test_fetcher2 = BillsOverview()
    test_fetcher2.update_all_bills_in_session()
    print(test_fetcher2.bills_overview_data)

---

# Subpackages

## parlpy.bills 
parlpy.bills for fetching bill data


### Class: BillsOverview

Constructs an object for collecting basic information on all bills in the current Parliamentary session. Its purpose is
to
* get a list of existing bills - so that we know what to scrape
* get path to further details for each bill - so that we can assemble the url to scrape further info from


Public instance variables:
* DataFrame : bills_overview_data 

    DF containing bill titles, their last updated time and the path to further
details on the bill

Public instance methods:
* None : get_changed_bills_in_session(session_name="2019-21", fetch_delay=0)
* None : reset_datetime_last_scraped()
* None : update_all_bills_in_session(session_name="2019-21", fetch_delay=0)

    Method to called to update self.bills_overview_data, fetching pages at maximum rate when fetch_delay=0

---

# Data Sources

## BillsOverview()

Fetches data from https://bills.parliament.uk

---

# Versions

## 0.0.2
* gets a DataFrame containing bill titles, their last updated times and page paths (page paths to be used in
  future versions)
* page request delay configurable



            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/Litharge/ParlPy",
    "name": "ParlPy",
    "maintainer": "",
    "docs_url": null,
    "requires_python": "",
    "maintainer_email": "",
    "keywords": "",
    "author": "Robert Chambers",
    "author_email": "1robertchambers@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/4a/ea/f4d9071024b90e535695c35ecce08791a409dedff46eed1d41515fec141f/ParlPy-0.1.0.tar.gz",
    "platform": "",
    "description": "# ParlPy\n\nPackage to scrape and process UK Parliamentary bills, votes and MP contact information.\n\nWritten as part of University of Bath Integrated Project module. \n\n---\n\n# Usage\n\n## Install\n\n    pipenv install ParlPy=0.1.0\n\n## Example Usage\n\nTo fetch bills updated since our scraper was last called (uses pickled variable for persistence between program runs)\n\n    from parlpy.bills.bill_list_fetcher import BillsOverview\n\n    test_fetcher = BillsOverview()\n    test_fetcher.get_changed_bills_in_session(session=\"All\")\n    print(test_fetcher.bills_overview_data)\n\n... script can stop and be rerun later, next time bills_overview_data will contain fewer items, unless \nreset_datetime_last_scraped() has been called first\n\nTo fetch all bills in session and print them:\n\n    from parlpy.bills.bill_list_fetcher import BillsOverview\n\n    test_fetcher2 = BillsOverview()\n    test_fetcher2.update_all_bills_in_session()\n    print(test_fetcher2.bills_overview_data)\n\n---\n\n# Subpackages\n\n## parlpy.bills \nparlpy.bills for fetching bill data\n\n\n### Class: BillsOverview\n\nConstructs an object for collecting basic information on all bills in the current Parliamentary session. Its purpose is\nto\n* get a list of existing bills - so that we know what to scrape\n* get path to further details for each bill - so that we can assemble the url to scrape further info from\n\n\nPublic instance variables:\n* DataFrame : bills_overview_data \n\n    DF containing bill titles, their last updated time and the path to further\ndetails on the bill\n\nPublic instance methods:\n* None : get_changed_bills_in_session(session_name=\"2019-21\", fetch_delay=0)\n* None : reset_datetime_last_scraped()\n* None : update_all_bills_in_session(session_name=\"2019-21\", fetch_delay=0)\n\n    Method to called to update self.bills_overview_data, fetching pages at maximum rate when fetch_delay=0\n\n---\n\n# Data Sources\n\n## BillsOverview()\n\nFetches data from https://bills.parliament.uk\n\n---\n\n# Versions\n\n## 0.0.2\n* gets a DataFrame containing bill titles, their last updated times and page paths (page paths to be used in\n  future versions)\n* page request delay configurable\n\n\n",
    "bugtrack_url": null,
    "license": "",
    "summary": "Scraper and utilities for UK Parliamentary bills",
    "version": "0.1.0",
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "62afbddbe94792c366d6fc4f841037ca",
                "sha256": "e36f7d919ecd9deb66a9402f4f577de7908b89da5a503587966df1055e268c0e"
            },
            "downloads": -1,
            "filename": "ParlPy-0.1.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "62afbddbe94792c366d6fc4f841037ca",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 6077,
            "upload_time": "2021-02-23T18:03:20",
            "upload_time_iso_8601": "2021-02-23T18:03:20.930399Z",
            "url": "https://files.pythonhosted.org/packages/b2/42/a4784d7806c9af1608185f8f3e6a3369c5e02a2358c4c70840c85192641a/ParlPy-0.1.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "c5c0626888046769d0e84efad4c4ba69",
                "sha256": "1933cf5c86b4fed9830b6b28174980786074a38735cfeb62c21123e803a13f46"
            },
            "downloads": -1,
            "filename": "ParlPy-0.1.0.tar.gz",
            "has_sig": false,
            "md5_digest": "c5c0626888046769d0e84efad4c4ba69",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 6500,
            "upload_time": "2021-02-23T18:03:21",
            "upload_time_iso_8601": "2021-02-23T18:03:21.934970Z",
            "url": "https://files.pythonhosted.org/packages/4a/ea/f4d9071024b90e535695c35ecce08791a409dedff46eed1d41515fec141f/ParlPy-0.1.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2021-02-23 18:03:21",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": null,
    "github_project": "Litharge",
    "error": "Could not fetch GitHub repository",
    "lcname": "parlpy"
}
        
Elapsed time: 0.20130s