fetchcode


Namefetchcode JSON
Version 0.6.0 PyPI version JSON
download
home_pagehttps://github.com/aboutcode-org/fetchcode
Summaryfetchcode is a library to reliably fetch code via HTTP, FTP and version control systems.
upload_time2024-09-26 09:22:13
maintainerNone
docs_urlNone
authornexB. Inc. and others
requires_python>=3.8
licenseApache-2.0
keywords utilities package manager dependencies fetch scancode download downloader
VCS
bugtrack_url
requirements attrs banal beautifulsoup4 binaryornot boolean.py certifi cffi chardet charset-normalizer click colorama commoncode construct container-inspector cryptography python-dateutil debian-inspector dockerfile-parse dparse2 extractcode extractcode-7z extractcode-libarchive fasteners fingerprints ftfy future gemfileparser html5lib idna importlib-metadata inflection intbitset isodate jaraco.functools javaproperties Jinja2 jsonstreams license-expression lxml MarkupSafe more-itertools normality packagedcode-msitools packageurl-python packaging parameter-expansion-patched patch pdfminer-six pefile pip-requirements-parser pkginfo2 pluggy plugincode ply publicsuffix2 pyahocorasick pycparser pygmars Pygments pymaven-patch pyparsing pytz PyYAML rdflib regipy requests rpm-inspector-rpm saneyaml six soupsieve spdx-tools text-unidecode toml typecode typecode-libmagic urllib3 urlpy wcwidth webencodings xmltodict zipp
Travis-CI No Travis.
coveralls test coverage No coveralls.
            FetchCode: Utilities to fetch code from purls, URLs and version control repos.
================================================================================

FetchCode is a library to reliably fetch code via HTTP, FTP and version control
systems. It can work using plain HTTP and FTP URLs, as well as
`Package URLs <https://github.com/package-url>`_ and version control (VCS) URLs
as used in Python pip and as specified in `SPDX Package Download Location
<https://spdx.github.io/spdx-spec/3-package-information/#37-package-download-location>`_

Homepage and support: https://github.com/aboutcode-org/fetchcode


Why FetchCode?
--------------

It is surprisingly difficult to have a simple API to consistently fetch code
from package repositories, version control repositories and APIs: each site
and each package manager has its own unique and peculiar ways. FetchCode goal
is to abstract all these details and make it easy to fetch things reliably.


Development installation
--------------------------

Clone the repo::

    git clone https://github.com/aboutcode-org/fetchcode

Then install all the requirements using this command (on POSIX)::

    ./configure --dev


Running tests
----------------

To run test suite use::

    pytest -vvs


Usage
--------

Fetch a code archive and get a ``fetchcode.fetch.Response`` object back::

    >>> from fetchcode import fetch
    >>> f = fetch('https://github.com/aboutcode-org/fetchcode/archive/ab65b2e645c889887227ea49eb3332d885fd0a54.zip')
    >>> f.location
    '/tmp/tmp_cm02xsg'
    >>> f.content_type
    'application/zip'
    >>> f.url
    'https://github.com/aboutcode-org/fetchcode/archive/ab65b2e645c889887227ea49eb3332d885fd0a54.zip'

Fetch some package metadata and get a ``fetchcode.packagedcode_models.Package`` object back::

    >>> from fetchcode import package
    >>> list(package.info('pkg:rubygems/files'))
    [Package(type='rubygems', namespace=None, name='files', version=None)]


License
--------

- SPDX-License-Identifier: Apache-2.0

Copyright (c) nexB Inc. and others.

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/aboutcode-org/fetchcode",
    "name": "fetchcode",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": null,
    "keywords": "utilities, package manager, dependencies, fetch, scancode, download, downloader",
    "author": "nexB. Inc. and others",
    "author_email": "info@aboutcode.org",
    "download_url": "https://files.pythonhosted.org/packages/99/1c/81bf30e0829a5d249ede310add1769c98b088d1fdec34afe9694736e8880/fetchcode-0.6.0.tar.gz",
    "platform": null,
    "description": "FetchCode: Utilities to fetch code from purls, URLs and version control repos.\n================================================================================\n\nFetchCode is a library to reliably fetch code via HTTP, FTP and version control\nsystems. It can work using plain HTTP and FTP URLs, as well as\n`Package URLs <https://github.com/package-url>`_ and version control (VCS) URLs\nas used in Python pip and as specified in `SPDX Package Download Location\n<https://spdx.github.io/spdx-spec/3-package-information/#37-package-download-location>`_\n\nHomepage and support: https://github.com/aboutcode-org/fetchcode\n\n\nWhy FetchCode?\n--------------\n\nIt is surprisingly difficult to have a simple API to consistently fetch code\nfrom package repositories, version control repositories and APIs: each site\nand each package manager has its own unique and peculiar ways. FetchCode goal\nis to abstract all these details and make it easy to fetch things reliably.\n\n\nDevelopment installation\n--------------------------\n\nClone the repo::\n\n    git clone https://github.com/aboutcode-org/fetchcode\n\nThen install all the requirements using this command (on POSIX)::\n\n    ./configure --dev\n\n\nRunning tests\n----------------\n\nTo run test suite use::\n\n    pytest -vvs\n\n\nUsage\n--------\n\nFetch a code archive and get a ``fetchcode.fetch.Response`` object back::\n\n    >>> from fetchcode import fetch\n    >>> f = fetch('https://github.com/aboutcode-org/fetchcode/archive/ab65b2e645c889887227ea49eb3332d885fd0a54.zip')\n    >>> f.location\n    '/tmp/tmp_cm02xsg'\n    >>> f.content_type\n    'application/zip'\n    >>> f.url\n    'https://github.com/aboutcode-org/fetchcode/archive/ab65b2e645c889887227ea49eb3332d885fd0a54.zip'\n\nFetch some package metadata and get a ``fetchcode.packagedcode_models.Package`` object back::\n\n    >>> from fetchcode import package\n    >>> list(package.info('pkg:rubygems/files'))\n    [Package(type='rubygems', namespace=None, name='files', version=None)]\n\n\nLicense\n--------\n\n- SPDX-License-Identifier: Apache-2.0\n\nCopyright (c) nexB Inc. and others.\n",
    "bugtrack_url": null,
    "license": "Apache-2.0",
    "summary": "fetchcode is a library to reliably fetch code via HTTP, FTP and version control systems.",
    "version": "0.6.0",
    "project_urls": {
        "Homepage": "https://github.com/aboutcode-org/fetchcode"
    },
    "split_keywords": [
        "utilities",
        " package manager",
        " dependencies",
        " fetch",
        " scancode",
        " download",
        " downloader"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "4678f41234269699a5530fcdf087da69681b82aabffcf1a6e18d97181d3e7192",
                "md5": "aa206f58bb218f34b7de08e28a85fa64",
                "sha256": "efe531d9c3fe69be26f6a40f0fe0a3fee80faca61403bd286fe42db2020884aa"
            },
            "downloads": -1,
            "filename": "fetchcode-0.6.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "aa206f58bb218f34b7de08e28a85fa64",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 1922037,
            "upload_time": "2024-09-26T09:22:11",
            "upload_time_iso_8601": "2024-09-26T09:22:11.886953Z",
            "url": "https://files.pythonhosted.org/packages/46/78/f41234269699a5530fcdf087da69681b82aabffcf1a6e18d97181d3e7192/fetchcode-0.6.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "991c81bf30e0829a5d249ede310add1769c98b088d1fdec34afe9694736e8880",
                "md5": "310f2087a5d371938bc4585c0d7f495e",
                "sha256": "1ad30848a45b7be2a9b2dfb4e6cc7d564d508a1e2824876915d3377beeb5d602"
            },
            "downloads": -1,
            "filename": "fetchcode-0.6.0.tar.gz",
            "has_sig": false,
            "md5_digest": "310f2087a5d371938bc4585c0d7f495e",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 2809716,
            "upload_time": "2024-09-26T09:22:13",
            "upload_time_iso_8601": "2024-09-26T09:22:13.801866Z",
            "url": "https://files.pythonhosted.org/packages/99/1c/81bf30e0829a5d249ede310add1769c98b088d1fdec34afe9694736e8880/fetchcode-0.6.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-09-26 09:22:13",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "aboutcode-org",
    "github_project": "fetchcode",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "requirements": [
        {
            "name": "attrs",
            "specs": [
                [
                    "==",
                    "21.4.0"
                ]
            ]
        },
        {
            "name": "banal",
            "specs": [
                [
                    "==",
                    "1.0.6"
                ]
            ]
        },
        {
            "name": "beautifulsoup4",
            "specs": [
                [
                    "==",
                    "4.11.1"
                ]
            ]
        },
        {
            "name": "binaryornot",
            "specs": [
                [
                    "==",
                    "0.4.4"
                ]
            ]
        },
        {
            "name": "boolean.py",
            "specs": [
                [
                    "==",
                    "3.8"
                ]
            ]
        },
        {
            "name": "certifi",
            "specs": [
                [
                    "==",
                    "2024.7.4"
                ]
            ]
        },
        {
            "name": "cffi",
            "specs": [
                [
                    "==",
                    "1.15.0"
                ]
            ]
        },
        {
            "name": "chardet",
            "specs": [
                [
                    "==",
                    "4.0.0"
                ]
            ]
        },
        {
            "name": "charset-normalizer",
            "specs": [
                [
                    "==",
                    "2.0.12"
                ]
            ]
        },
        {
            "name": "click",
            "specs": [
                [
                    "==",
                    "8.0.4"
                ]
            ]
        },
        {
            "name": "colorama",
            "specs": [
                [
                    "==",
                    "0.4.4"
                ]
            ]
        },
        {
            "name": "commoncode",
            "specs": [
                [
                    "==",
                    "30.2.0"
                ]
            ]
        },
        {
            "name": "construct",
            "specs": [
                [
                    "==",
                    "2.10.68"
                ]
            ]
        },
        {
            "name": "container-inspector",
            "specs": [
                [
                    "==",
                    "31.0.0"
                ]
            ]
        },
        {
            "name": "cryptography",
            "specs": [
                [
                    "==",
                    "43.0.1"
                ]
            ]
        },
        {
            "name": "python-dateutil",
            "specs": [
                [
                    "==",
                    "2.8.2"
                ]
            ]
        },
        {
            "name": "debian-inspector",
            "specs": [
                [
                    "==",
                    "30.0.0"
                ]
            ]
        },
        {
            "name": "dockerfile-parse",
            "specs": [
                [
                    "==",
                    "1.2.0"
                ]
            ]
        },
        {
            "name": "dparse2",
            "specs": [
                [
                    "==",
                    "0.6.1"
                ]
            ]
        },
        {
            "name": "extractcode",
            "specs": [
                [
                    "==",
                    "31.0.0"
                ]
            ]
        },
        {
            "name": "extractcode-7z",
            "specs": [
                [
                    "==",
                    "16.5.210531"
                ]
            ]
        },
        {
            "name": "extractcode-libarchive",
            "specs": [
                [
                    "==",
                    "3.5.1.210531"
                ]
            ]
        },
        {
            "name": "fasteners",
            "specs": [
                [
                    "==",
                    "0.17.3"
                ]
            ]
        },
        {
            "name": "fingerprints",
            "specs": [
                [
                    "==",
                    "1.0.3"
                ]
            ]
        },
        {
            "name": "ftfy",
            "specs": [
                [
                    "==",
                    "6.0.3"
                ]
            ]
        },
        {
            "name": "future",
            "specs": [
                [
                    "==",
                    "0.18.3"
                ]
            ]
        },
        {
            "name": "gemfileparser",
            "specs": [
                [
                    "==",
                    "0.8.0"
                ]
            ]
        },
        {
            "name": "html5lib",
            "specs": [
                [
                    "==",
                    "1.1"
                ]
            ]
        },
        {
            "name": "idna",
            "specs": [
                [
                    "==",
                    "3.7"
                ]
            ]
        },
        {
            "name": "importlib-metadata",
            "specs": [
                [
                    "==",
                    "4.8.3"
                ]
            ]
        },
        {
            "name": "inflection",
            "specs": [
                [
                    "==",
                    "0.5.1"
                ]
            ]
        },
        {
            "name": "intbitset",
            "specs": [
                [
                    "==",
                    "3.1.0"
                ]
            ]
        },
        {
            "name": "isodate",
            "specs": [
                [
                    "==",
                    "0.6.1"
                ]
            ]
        },
        {
            "name": "jaraco.functools",
            "specs": [
                [
                    "==",
                    "3.4.0"
                ]
            ]
        },
        {
            "name": "javaproperties",
            "specs": [
                [
                    "==",
                    "0.8.1"
                ]
            ]
        },
        {
            "name": "Jinja2",
            "specs": [
                [
                    "==",
                    "3.1.4"
                ]
            ]
        },
        {
            "name": "jsonstreams",
            "specs": [
                [
                    "==",
                    "0.6.0"
                ]
            ]
        },
        {
            "name": "license-expression",
            "specs": [
                [
                    "==",
                    "21.6.14"
                ]
            ]
        },
        {
            "name": "lxml",
            "specs": [
                [
                    "==",
                    "4.9.1"
                ]
            ]
        },
        {
            "name": "MarkupSafe",
            "specs": [
                [
                    "==",
                    "2.0.1"
                ]
            ]
        },
        {
            "name": "more-itertools",
            "specs": [
                [
                    "==",
                    "8.13.0"
                ]
            ]
        },
        {
            "name": "normality",
            "specs": [
                [
                    "==",
                    "2.3.3"
                ]
            ]
        },
        {
            "name": "packagedcode-msitools",
            "specs": [
                [
                    "==",
                    "0.101.210706"
                ]
            ]
        },
        {
            "name": "packageurl-python",
            "specs": [
                [
                    "==",
                    "0.9.9"
                ]
            ]
        },
        {
            "name": "packaging",
            "specs": [
                [
                    "==",
                    "21.3"
                ]
            ]
        },
        {
            "name": "parameter-expansion-patched",
            "specs": [
                [
                    "==",
                    "0.3.1"
                ]
            ]
        },
        {
            "name": "patch",
            "specs": [
                [
                    "==",
                    "1.16"
                ]
            ]
        },
        {
            "name": "pdfminer-six",
            "specs": [
                [
                    "==",
                    "20220506"
                ]
            ]
        },
        {
            "name": "pefile",
            "specs": [
                [
                    "==",
                    "2021.9.3"
                ]
            ]
        },
        {
            "name": "pip-requirements-parser",
            "specs": [
                [
                    "==",
                    "31.2.0"
                ]
            ]
        },
        {
            "name": "pkginfo2",
            "specs": [
                [
                    "==",
                    "30.0.0"
                ]
            ]
        },
        {
            "name": "pluggy",
            "specs": [
                [
                    "==",
                    "1.0.0"
                ]
            ]
        },
        {
            "name": "plugincode",
            "specs": [
                [
                    "==",
                    "30.0.0"
                ]
            ]
        },
        {
            "name": "ply",
            "specs": [
                [
                    "==",
                    "3.11"
                ]
            ]
        },
        {
            "name": "publicsuffix2",
            "specs": [
                [
                    "==",
                    "2.20191221"
                ]
            ]
        },
        {
            "name": "pyahocorasick",
            "specs": [
                [
                    "==",
                    "2.0.0b1"
                ]
            ]
        },
        {
            "name": "pycparser",
            "specs": [
                [
                    "==",
                    "2.21"
                ]
            ]
        },
        {
            "name": "pygmars",
            "specs": [
                [
                    "==",
                    "0.7.0"
                ]
            ]
        },
        {
            "name": "Pygments",
            "specs": [
                [
                    "==",
                    "2.15.0"
                ]
            ]
        },
        {
            "name": "pymaven-patch",
            "specs": [
                [
                    "==",
                    "0.3.0"
                ]
            ]
        },
        {
            "name": "pyparsing",
            "specs": [
                [
                    "==",
                    "3.0.8"
                ]
            ]
        },
        {
            "name": "pytz",
            "specs": [
                [
                    "==",
                    "2022.1"
                ]
            ]
        },
        {
            "name": "PyYAML",
            "specs": [
                [
                    "==",
                    "6.0"
                ]
            ]
        },
        {
            "name": "rdflib",
            "specs": [
                [
                    "==",
                    "5.0.0"
                ]
            ]
        },
        {
            "name": "regipy",
            "specs": [
                [
                    "==",
                    "2.3.1"
                ]
            ]
        },
        {
            "name": "requests",
            "specs": [
                [
                    "==",
                    "2.32.0"
                ]
            ]
        },
        {
            "name": "rpm-inspector-rpm",
            "specs": [
                [
                    "==",
                    "4.16.1.3.210404"
                ]
            ]
        },
        {
            "name": "saneyaml",
            "specs": [
                [
                    "==",
                    "0.5.2"
                ]
            ]
        },
        {
            "name": "six",
            "specs": [
                [
                    "==",
                    "1.16.0"
                ]
            ]
        },
        {
            "name": "soupsieve",
            "specs": [
                [
                    "==",
                    "2.3.1"
                ]
            ]
        },
        {
            "name": "spdx-tools",
            "specs": [
                [
                    "==",
                    "0.7.0a3"
                ]
            ]
        },
        {
            "name": "text-unidecode",
            "specs": [
                [
                    "==",
                    "1.3"
                ]
            ]
        },
        {
            "name": "toml",
            "specs": [
                [
                    "==",
                    "0.10.2"
                ]
            ]
        },
        {
            "name": "typecode",
            "specs": [
                [
                    "==",
                    "30.0.0"
                ]
            ]
        },
        {
            "name": "typecode-libmagic",
            "specs": [
                [
                    "==",
                    "5.39.210531"
                ]
            ]
        },
        {
            "name": "urllib3",
            "specs": [
                [
                    "==",
                    "1.26.19"
                ]
            ]
        },
        {
            "name": "urlpy",
            "specs": [
                [
                    "==",
                    "0.5"
                ]
            ]
        },
        {
            "name": "wcwidth",
            "specs": [
                [
                    "==",
                    "0.2.5"
                ]
            ]
        },
        {
            "name": "webencodings",
            "specs": [
                [
                    "==",
                    "0.5.1"
                ]
            ]
        },
        {
            "name": "xmltodict",
            "specs": [
                [
                    "==",
                    "0.12.0"
                ]
            ]
        },
        {
            "name": "zipp",
            "specs": [
                [
                    "==",
                    "3.19.1"
                ]
            ]
        }
    ],
    "lcname": "fetchcode"
}
        
Elapsed time: 0.40240s