# [watchdogs](https://github.com/ClericPy/watchdogs) [](https://pypi.org/project/watchdogs/)
Keep an eye on the change of web world.
Such as `post articles` / `news on the web portal` / `server api health` / `binge-watching` / `steam price fluctuation` / `github events` / `updates of comic and novel`, and so on...
## Intro
> [中文文档](https://clericpy.github.io/blog/posts/20200331171211/)
1. This is a web app based on [fastapi](https://github.com/tiangolo/fastapi), [databases](https://github.com/encode/databases), [uniparser](https://github.com/ClericPy/uniparser), [torequests](https://github.com/ClericPy/torequests).
2. Smoothly deploy it by pip: `pip install -U watchdogs;python3 -m watchdogs`
3. Simple to create a new crawler with the Web UI, not like old ways to write duplicate code.
4. All the crawlers keep runing in the async environment.
5. Almost all the elements have a *title* attribute to describe the features in the Web UI, which means docs lay on the UI.
6. Release your hands from repetitive refreshing pages on the browser.
1. Subscribe the change events with RSS reminder extensions, such as [Feedbro](https://chrome.google.com/webstore/detail/feedbro/mefgmmbdailogpfhfblcnnjfmnpnmdfa) or RSS Feed Reader.
2. Implement a class which inherits from `watchdogs.callbacks.Callback`.
## Usage
1. > pip install -U watchdogs
2. > python -m watchdogs
3. > Open the browser: http://127.0.0.1:9901
### Command line args
> python -m watchdogs -- -h
- **db_url**:
> sqlite / mysql / postgresql(not test) url, which [databases](https://github.com/encode/databases) supports. Defaults to 'sqlite:///{HOME_PATH}/watchdogs/storage.sqlite'
- **password**:
> init password, if null can be set on the first visit on web.
- **mute_std_log**:
> remove stdout log for clean stream
- **mute_file_log**:
> ignore file log located at {HOME_PATH}/watchdogs folder.
- **md5_salt**:
> md5_salt for custom md5(password) / md5(rss_tag)
- **config_dir**:
> config dir to save the logs and config files, if using sqlite include sqlite file. defaults to {HOME_PATH}/watchdogs
- **use_default_cdn**:
> If Config.cdn_urls not set, and use_default_cdn is True, will use online js/css cdn links from staticfile.org.
- **\*\*uvicorn_kwargs**:
> uvicorn startup kwargs, such as port, host. Which can be set like: `python -m watchdogs --port=9999 --host=127.0.0.1 --access-log=False`
### Quick Start to Create New Task
[Quick Start Screenshots](https://github.com/ClericPy/watchdogs/blob/master/quick_start.md)
## Web UI
<details>
<summary>Screenshots</summary>
1. Welcome Page (Tasks Page).
> Here you can see all the tasks meta, goto RSS / Mobile Lite Page, and do some operations to the tasks.
2. New Task Page.
> Here based on the latest [uniparser](https://github.com/ClericPy/uniparser) web app, to create new rules and also tasks.
3. Rules Page.
> Do some operations for the rules.
4. API page.
> Based on [fastapi](https://github.com/tiangolo/fastapi) `/docs` which is generated automatically.
5. Mobile Page (Lite View).
> For mobile phone to glimpse the latest result for the current 30 tasks.
</details>
<!--
[CSS Minifier](https://cssminifier.com/)
[JavaScript Minifier](https://javascript-minifier.com/)
-->
Raw data
{
"_id": null,
"home_page": "https://github.com/ClericPy/watchdogs",
"name": "watchdogs",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "requests crawler uniparser torequests fastapi watchdog",
"author": "ClericPy",
"author_email": "clericpy@gmail.com",
"download_url": "",
"platform": "any",
"description": "# [watchdogs](https://github.com/ClericPy/watchdogs) [](https://pypi.org/project/watchdogs/)\r\n\r\nKeep an eye on the change of web world.\r\n\r\nSuch as `post articles` / `news on the web portal` / `server api health` / `binge-watching` / `steam price fluctuation` / `github events` / `updates of comic and novel`, and so on...\r\n\r\n## Intro\r\n\r\n> [\u4e2d\u6587\u6587\u6863](https://clericpy.github.io/blog/posts/20200331171211/)\r\n\r\n1. This is a web app based on [fastapi](https://github.com/tiangolo/fastapi), [databases](https://github.com/encode/databases), [uniparser](https://github.com/ClericPy/uniparser), [torequests](https://github.com/ClericPy/torequests).\r\n2. Smoothly deploy it by pip: `pip install -U watchdogs;python3 -m watchdogs`\r\n3. Simple to create a new crawler with the Web UI, not like old ways to write duplicate code.\r\n4. All the crawlers keep runing in the async environment.\r\n5. Almost all the elements have a *title* attribute to describe the features in the Web UI, which means docs lay on the UI.\r\n6. Release your hands from repetitive refreshing pages on the browser.\r\n 1. Subscribe the change events with RSS reminder extensions, such as [Feedbro](https://chrome.google.com/webstore/detail/feedbro/mefgmmbdailogpfhfblcnnjfmnpnmdfa) or RSS Feed Reader.\r\n 2. Implement a class which inherits from `watchdogs.callbacks.Callback`.\r\n\r\n## Usage\r\n\r\n1. > pip install -U watchdogs\r\n\r\n2. > python -m watchdogs\r\n\r\n3. > Open the browser: http://127.0.0.1:9901\r\n\r\n### Command line args\r\n\r\n> python -m watchdogs -- -h\r\n\r\n- **db_url**:\r\n> sqlite / mysql / postgresql(not test) url, which [databases](https://github.com/encode/databases) supports. Defaults to 'sqlite:///{HOME_PATH}/watchdogs/storage.sqlite'\r\n- **password**:\r\n> init password, if null can be set on the first visit on web.\r\n- **mute_std_log**:\r\n> remove stdout log for clean stream\r\n- **mute_file_log**:\r\n> ignore file log located at {HOME_PATH}/watchdogs folder.\r\n- **md5_salt**:\r\n> md5_salt for custom md5(password) / md5(rss_tag)\r\n- **config_dir**:\r\n> config dir to save the logs and config files, if using sqlite include sqlite file. defaults to {HOME_PATH}/watchdogs\r\n- **use_default_cdn**:\r\n> If Config.cdn_urls not set, and use_default_cdn is True, will use online js/css cdn links from staticfile.org.\r\n- **\\*\\*uvicorn_kwargs**:\r\n> uvicorn startup kwargs, such as port, host. Which can be set like: `python -m watchdogs --port=9999 --host=127.0.0.1 --access-log=False`\r\n\r\n### Quick Start to Create New Task\r\n\r\n[Quick Start Screenshots](https://github.com/ClericPy/watchdogs/blob/master/quick_start.md)\r\n\r\n\r\n## Web UI\r\n\r\n<details>\r\n <summary>Screenshots</summary>\r\n\r\n1. Welcome Page (Tasks Page).\r\n> Here you can see all the tasks meta, goto RSS / Mobile Lite Page, and do some operations to the tasks.\r\n\r\n\r\n\r\n2. New Task Page.\r\n> Here based on the latest [uniparser](https://github.com/ClericPy/uniparser) web app, to create new rules and also tasks.\r\n\r\n\r\n\r\n3. Rules Page.\r\n> Do some operations for the rules.\r\n\r\n\r\n\r\n4. API page.\r\n> Based on [fastapi](https://github.com/tiangolo/fastapi) `/docs` which is generated automatically.\r\n\r\n\r\n\r\n5. Mobile Page (Lite View).\r\n> For mobile phone to glimpse the latest result for the current 30 tasks.\r\n\r\n\r\n\r\n</details>\r\n\r\n<!--\r\n\r\n[CSS Minifier](https://cssminifier.com/)\r\n[JavaScript Minifier](https://javascript-minifier.com/)\r\n\r\n-->\r\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "Watchdogs to keep an eye on the world's change. Read more: https://github.com/ClericPy/watchdogs.",
"version": "2.0.1",
"project_urls": {
"Homepage": "https://github.com/ClericPy/watchdogs"
},
"split_keywords": [
"requests",
"crawler",
"uniparser",
"torequests",
"fastapi",
"watchdog"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "735d64d9e57be92053fe80c126acf58ecb7f325d07e0a9a917e02e43425683b2",
"md5": "e28e1668ba884fe63725d7525f5722df",
"sha256": "41b62fc39520a0c90042b280f5683f3ac7a64aa1c3863871b93961d3801b1e85"
},
"downloads": -1,
"filename": "watchdogs-2.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e28e1668ba884fe63725d7525f5722df",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 309617,
"upload_time": "2023-11-01T14:18:23",
"upload_time_iso_8601": "2023-11-01T14:18:23.907736Z",
"url": "https://files.pythonhosted.org/packages/73/5d/64d9e57be92053fe80c126acf58ecb7f325d07e0a9a917e02e43425683b2/watchdogs-2.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-01 14:18:23",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ClericPy",
"github_project": "watchdogs",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "watchdogs"
}
JSON
