scrapyduler

Name	scrapyduler JSON
Version	1.0.2 JSON
	download
home_page
Summary	Scrapyd launcher module that schedules scrapy spiders by time
upload_time	2024-01-01 20:59:19
maintainer
docs_url	None
author
requires_python	>=3.7
license	BSD-3-Clause
keywords	scrapy scrapyd scheduler launcher
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            scrapyduler
===========

Scrapyd launcher module that schedules scrapy spiders by time.

Install
-------

.. code-block:: shell

    $ pip install scrapyduler

Config
------

To start using this library you just need to override
the ``launcher`` option in your ``scrapyd.conf`` file:

.. code-block:: text

    [scrapyd]
    launcher = scrapyduler.launcher.Launcher

and then add the schedulers configuration, e.g.:

.. code-block:: text

    [scheduler.1]
    cron        = * * * * *
    project     = quotesbot
    spider      = toscrape-xpath
    settings    = HTTPPROXY_ENABLED=True
    args        = key1=value1 key2=value2 start_url=http://quotes.toscrape.com/

    [scheduler.2]
    interval    = weeks=0 days=0 hours=0 minutes=0 seconds=30
    project     = quotesbot
    spider      = toscrape-css
    settings    = HTTPPROXY_ENABLED=True
    args        = key1=value1 key2=value2 start_url=http://quotes.toscrape.com/

In the examples above, we set up two schedulers.
The first scheduler uses cron syntax to run spiders.
The second scheduler triggers on specified intervals,
starting on ``start_date`` if specified, ``datetime.now()`` + interval otherwise.
See https://github.com/agronholm/apscheduler for more information.

            

Raw data

            {
    "_id": null,
    "home_page": "",
    "name": "scrapyduler",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.7",
    "maintainer_email": "",
    "keywords": "scrapy,scrapyd,scheduler,launcher",
    "author": "",
    "author_email": "Dmitriy Chuenkov <slymit@gmail.com>",
    "download_url": "https://files.pythonhosted.org/packages/d6/20/ce92ce27c29f09588be699a7a5bbafba5ff0eaf838c9ff97ad7adaa968b9/scrapyduler-1.0.2.tar.gz",
    "platform": null,
    "description": "scrapyduler\n===========\n\nScrapyd launcher module that schedules scrapy spiders by time.\n\nInstall\n-------\n\n.. code-block:: shell\n\n    $ pip install scrapyduler\n\nConfig\n------\n\nTo start using this library you just need to override\nthe ``launcher`` option in your ``scrapyd.conf`` file:\n\n.. code-block:: text\n\n    [scrapyd]\n    launcher = scrapyduler.launcher.Launcher\n\nand then add the schedulers configuration, e.g.:\n\n.. code-block:: text\n\n    [scheduler.1]\n    cron        = * * * * *\n    project     = quotesbot\n    spider      = toscrape-xpath\n    settings    = HTTPPROXY_ENABLED=True\n    args        = key1=value1 key2=value2 start_url=http://quotes.toscrape.com/\n\n    [scheduler.2]\n    interval    = weeks=0 days=0 hours=0 minutes=0 seconds=30\n    project     = quotesbot\n    spider      = toscrape-css\n    settings    = HTTPPROXY_ENABLED=True\n    args        = key1=value1 key2=value2 start_url=http://quotes.toscrape.com/\n\nIn the examples above, we set up two schedulers.\nThe first scheduler uses cron syntax to run spiders.\nThe second scheduler triggers on specified intervals,\nstarting on ``start_date`` if specified, ``datetime.now()`` + interval otherwise.\nSee https://github.com/agronholm/apscheduler for more information.\n",
    "bugtrack_url": null,
    "license": "BSD-3-Clause",
    "summary": "Scrapyd launcher module that schedules scrapy spiders by time",
    "version": "1.0.2",
    "project_urls": {
        "Homepage": "https://github.com/slymit/scrapyduler"
    },
    "split_keywords": [
        "scrapy",
        "scrapyd",
        "scheduler",
        "launcher"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "3804e2e7a108c8e191349aa3bb7f0b8d54373ba36cc7f1285cfe0c5790a292b8",
                "md5": "f14ea0c83e1a878ba7c9a1b4ad7b7394",
                "sha256": "4b00611fb6a1b142f5cfa356b6702857603d2032127023b23f3356d016d9ea35"
            },
            "downloads": -1,
            "filename": "scrapyduler-1.0.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "f14ea0c83e1a878ba7c9a1b4ad7b7394",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.7",
            "size": 3136,
            "upload_time": "2024-01-01T20:59:17",
            "upload_time_iso_8601": "2024-01-01T20:59:17.988145Z",
            "url": "https://files.pythonhosted.org/packages/38/04/e2e7a108c8e191349aa3bb7f0b8d54373ba36cc7f1285cfe0c5790a292b8/scrapyduler-1.0.2-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "d620ce92ce27c29f09588be699a7a5bbafba5ff0eaf838c9ff97ad7adaa968b9",
                "md5": "cd81b6e27d6b2ef5f84fd7e6110c40fd",
                "sha256": "f9a434e95566330f446480671d78f0429e44da62097a6df98e59ce9a34bf9d7f"
            },
            "downloads": -1,
            "filename": "scrapyduler-1.0.2.tar.gz",
            "has_sig": false,
            "md5_digest": "cd81b6e27d6b2ef5f84fd7e6110c40fd",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.7",
            "size": 3012,
            "upload_time": "2024-01-01T20:59:19",
            "upload_time_iso_8601": "2024-01-01T20:59:19.505382Z",
            "url": "https://files.pythonhosted.org/packages/d6/20/ce92ce27c29f09588be699a7a5bbafba5ff0eaf838c9ff97ad7adaa968b9/scrapyduler-1.0.2.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-01-01 20:59:19",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "slymit",
    "github_project": "scrapyduler",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "scrapyduler"
}