collection-scanner


Namecollection-scanner JSON
Version 0.4.2 PyPI version JSON
download
home_pagehttps://github.com/scrapinghub/collection-scanner
SummaryScrapinghub Hubstorage Collection scanner.
upload_time2020-11-26 18:43:33
maintainerScrapinghub
docs_urlNone
author
requires_python
licenseBSD
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            High level hubstorage collection scanner
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Provides convenient way to scan a collection in batches
- Allows to merge data from multiple collections
- Accepts endts and startts in many string formats (as accepted by dateparser lib) or standard HS epoch in millisecs
- Accepts excluded prefixes
- Adds stopbefore feature (analogous to startafter but the inverse)
- Provides method for arbitrary prefix aggregation counting
- Supports partitioned collections
- Provides a suite for testing hs collection code.

Up to version 0.1.6: Python2 only
Starting version 0.2: Python3 only

See usage instructions at `scanner.py <https://github.com/scrapinghub/collection-scanner/blob/master/collection_scanner/scanner.py>`_ docstring.

Instalation
~~~~~~~~~~~

pip install collection-scanner



            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/scrapinghub/collection-scanner",
    "name": "collection-scanner",
    "maintainer": "Scrapinghub",
    "docs_url": null,
    "requires_python": "",
    "maintainer_email": "",
    "keywords": "",
    "author": "",
    "author_email": "",
    "download_url": "https://files.pythonhosted.org/packages/8b/27/96d28be8c95725d222c686ddd25c972e1f3b819907a07225f3324655b87d/collection_scanner-0.4.2.tar.gz",
    "platform": "",
    "description": "High level hubstorage collection scanner\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n- Provides convenient way to scan a collection in batches\n- Allows to merge data from multiple collections\n- Accepts endts and startts in many string formats (as accepted by dateparser lib) or standard HS epoch in millisecs\n- Accepts excluded prefixes\n- Adds stopbefore feature (analogous to startafter but the inverse)\n- Provides method for arbitrary prefix aggregation counting\n- Supports partitioned collections\n- Provides a suite for testing hs collection code.\n\nUp to version 0.1.6: Python2 only\nStarting version 0.2: Python3 only\n\nSee usage instructions at `scanner.py <https://github.com/scrapinghub/collection-scanner/blob/master/collection_scanner/scanner.py>`_ docstring.\n\nInstalation\n~~~~~~~~~~~\n\npip install collection-scanner\n\n\n",
    "bugtrack_url": null,
    "license": "BSD",
    "summary": "Scrapinghub Hubstorage Collection scanner.",
    "version": "0.4.2",
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "efa1951fc2ae17311e8fe93a7c6996bc",
                "sha256": "db31401d30a69718fcf6c48d946212a92ae3ea8552b68a34962573a9baa28117"
            },
            "downloads": -1,
            "filename": "collection_scanner-0.4.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "efa1951fc2ae17311e8fe93a7c6996bc",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 10477,
            "upload_time": "2020-11-26T18:43:31",
            "upload_time_iso_8601": "2020-11-26T18:43:31.089968Z",
            "url": "https://files.pythonhosted.org/packages/4d/de/14ae0c8f1c23d05bab3eb5527ca7ad30370000280ae4f46b5991c3d17bb3/collection_scanner-0.4.2-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "md5": "8a8b16867ab323dd8c09c0bcbd03058b",
                "sha256": "8a5ca0eafe1aef1f843aa0f0b43811fd55792eb679960b221b5686eaa92123ba"
            },
            "downloads": -1,
            "filename": "collection_scanner-0.4.2.tar.gz",
            "has_sig": false,
            "md5_digest": "8a8b16867ab323dd8c09c0bcbd03058b",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 7721,
            "upload_time": "2020-11-26T18:43:33",
            "upload_time_iso_8601": "2020-11-26T18:43:33.151068Z",
            "url": "https://files.pythonhosted.org/packages/8b/27/96d28be8c95725d222c686ddd25c972e1f3b819907a07225f3324655b87d/collection_scanner-0.4.2.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2020-11-26 18:43:33",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": null,
    "github_project": "scrapinghub",
    "error": "Could not fetch GitHub repository",
    "lcname": "collection-scanner"
}
        
Elapsed time: 0.19343s