Swaiter

Name	Swaiter JSON
Version	1.0.4 JSON
	download
home_page
Summary	The loop waits until the selenium element has loaded.
upload_time	2023-05-22 04:07:17
maintainer
docs_url	None
author
requires_python	>=3.7
license
keywords	selenium crawler
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            # Swaiter

​	项目名字取自：Selenium+waiter

​	作为一个爬虫人，在使用selenium进行爬虫时，时长会遇到由于网络等原因导致无法及时获取一些元素，从而导致程序报错。此时聪明的小伙伴就会说了：加个`time.sleep()`不就好了。事实上这个方法确实可以解决绝大部分问题，但其也有一个比较尴尬的问题：到底要“睡”多久合适？长了耽误爬取效率，短了又怕没有作用，所以这个库就是为了解决上述问题而存在的。

## 安装

库的安装十分简单直接使用以下面的命令就可以安装成功了：

```bash
pip install Swaiter
```

## 使用

使用方法主要分为两种：直接使用基础组件进行开发、继承并添加自己的检索函数

### 使用基础组件

这里给一个简单的使用基础组件的例子：

```python
from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options
from waiter import Waiter

if __name__ == '__main__':
    # 获取selenium中驱动配置项（可省略）
    driver_opt = Options()
    url = "https://www.baidu.com/"
    # 获取selenium中的driver对象
    chrome_driver = Chrome(options=driver_opt)
    chrome_driver.get(url=url)
    # 利用获取到的driver对象创建Waiter对象
    w = Waiter(chrome_driver)
    # 传入xpath路径以及查询参数，检索input对象，并向其中填入查询参数
    w.input_send_waiter(xpath="//input[@id='kw']", query="python")
    # 利用xpath找到点击对象，模拟点击
    w.click_waiter(xpath="//input[@id='su']")
    # 在新的页面找到第一个链接，并获取其中的href属性值
    result = w.elements_text_waiter(xpath="//h3/a[@tabindex=0]", attribute="href")
    print(result)
```

​	上述例子中，使用了3个基础的搜索函数，向这样的搜索函数，本库一共提供了5个，应该可以满足简单的模拟需求，当然如果你有一些特别的搜索需求，也可以尝试使用第二种方式。

### 继承并开发自己的检索函数

```python
from waiter import Waiter
from selenium.webdriver.common.by import By

class MyWaiter(Waiter):
    def __init__(self, driver, interval, is_track, is_log):
        super().__init__(driver=driver, interval=interval, is_track=is_track, is_log=is_log)

    def _textarea_send(self, xpath, **kwargs):
        self.driver.find_element(By.XPATH, xpath).send(kwargs)

    def textarea_send(self, xpath, **kwargs):
        self.wait(func="_textarea_send", xpath=xpath, interval=5, **kwargs)
```

## 环境

> window10、window11
>
> python>=3.8
>
> selenium>=4.9.0
>
> loguru>=0.7.0

## 版本

#### 2023.5.22

**Version-1.0.4**

- 添加select组件的搜索器
  - 添加获取组件所有选项
  - 添加向select组件中输入值

[1.0.4](https://pypi.org/project/Swaiter/1.0.4/)

#### 2023.5.16

**Version-1.0.3**

完成基础功能

[1.0.3](https://pypi.org/project/Swaiter/1.0.3/)


            

Raw data

            {
    "_id": null,
    "home_page": "",
    "name": "Swaiter",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.7",
    "maintainer_email": "",
    "keywords": "selenium,crawler",
    "author": "",
    "author_email": "Beier <1601684622@qq.com>",
    "download_url": "https://files.pythonhosted.org/packages/1f/5f/e48b8b695b202c0b459e9785714e321e64fa5f1baf48b0ff7e67f7a8367c/Swaiter-1.0.4.tar.gz",
    "platform": null,
    "description": "# Swaiter\r\n\r\n\u200b\t\u9879\u76ee\u540d\u5b57\u53d6\u81ea\uff1aSelenium+waiter\r\n\r\n\u200b\t\u4f5c\u4e3a\u4e00\u4e2a\u722c\u866b\u4eba\uff0c\u5728\u4f7f\u7528selenium\u8fdb\u884c\u722c\u866b\u65f6\uff0c\u65f6\u957f\u4f1a\u9047\u5230\u7531\u4e8e\u7f51\u7edc\u7b49\u539f\u56e0\u5bfc\u81f4\u65e0\u6cd5\u53ca\u65f6\u83b7\u53d6\u4e00\u4e9b\u5143\u7d20\uff0c\u4ece\u800c\u5bfc\u81f4\u7a0b\u5e8f\u62a5\u9519\u3002\u6b64\u65f6\u806a\u660e\u7684\u5c0f\u4f19\u4f34\u5c31\u4f1a\u8bf4\u4e86\uff1a\u52a0\u4e2a`time.sleep()`\u4e0d\u5c31\u597d\u4e86\u3002\u4e8b\u5b9e\u4e0a\u8fd9\u4e2a\u65b9\u6cd5\u786e\u5b9e\u53ef\u4ee5\u89e3\u51b3\u7edd\u5927\u90e8\u5206\u95ee\u9898\uff0c\u4f46\u5176\u4e5f\u6709\u4e00\u4e2a\u6bd4\u8f83\u5c34\u5c2c\u7684\u95ee\u9898\uff1a\u5230\u5e95\u8981\u201c\u7761\u201d\u591a\u4e45\u5408\u9002\uff1f\u957f\u4e86\u803d\u8bef\u722c\u53d6\u6548\u7387\uff0c\u77ed\u4e86\u53c8\u6015\u6ca1\u6709\u4f5c\u7528\uff0c\u6240\u4ee5\u8fd9\u4e2a\u5e93\u5c31\u662f\u4e3a\u4e86\u89e3\u51b3\u4e0a\u8ff0\u95ee\u9898\u800c\u5b58\u5728\u7684\u3002\r\n\r\n## \u5b89\u88c5\r\n\r\n\u5e93\u7684\u5b89\u88c5\u5341\u5206\u7b80\u5355\u76f4\u63a5\u4f7f\u7528\u4ee5\u4e0b\u9762\u7684\u547d\u4ee4\u5c31\u53ef\u4ee5\u5b89\u88c5\u6210\u529f\u4e86\uff1a\r\n\r\n```bash\r\npip install Swaiter\r\n```\r\n\r\n## \u4f7f\u7528\r\n\r\n\u4f7f\u7528\u65b9\u6cd5\u4e3b\u8981\u5206\u4e3a\u4e24\u79cd\uff1a\u76f4\u63a5\u4f7f\u7528\u57fa\u7840\u7ec4\u4ef6\u8fdb\u884c\u5f00\u53d1\u3001\u7ee7\u627f\u5e76\u6dfb\u52a0\u81ea\u5df1\u7684\u68c0\u7d22\u51fd\u6570\r\n\r\n### \u4f7f\u7528\u57fa\u7840\u7ec4\u4ef6\r\n\r\n\u8fd9\u91cc\u7ed9\u4e00\u4e2a\u7b80\u5355\u7684\u4f7f\u7528\u57fa\u7840\u7ec4\u4ef6\u7684\u4f8b\u5b50\uff1a\r\n\r\n```python\r\nfrom selenium.webdriver import Chrome\r\nfrom selenium.webdriver.chrome.options import Options\r\nfrom waiter import Waiter\r\n\r\nif __name__ == '__main__':\r\n    # \u83b7\u53d6selenium\u4e2d\u9a71\u52a8\u914d\u7f6e\u9879\uff08\u53ef\u7701\u7565\uff09\r\n    driver_opt = Options()\r\n    url = \"https://www.baidu.com/\"\r\n    # \u83b7\u53d6selenium\u4e2d\u7684driver\u5bf9\u8c61\r\n    chrome_driver = Chrome(options=driver_opt)\r\n    chrome_driver.get(url=url)\r\n    # \u5229\u7528\u83b7\u53d6\u5230\u7684driver\u5bf9\u8c61\u521b\u5efaWaiter\u5bf9\u8c61\r\n    w = Waiter(chrome_driver)\r\n    # \u4f20\u5165xpath\u8def\u5f84\u4ee5\u53ca\u67e5\u8be2\u53c2\u6570\uff0c\u68c0\u7d22input\u5bf9\u8c61\uff0c\u5e76\u5411\u5176\u4e2d\u586b\u5165\u67e5\u8be2\u53c2\u6570\r\n    w.input_send_waiter(xpath=\"//input[@id='kw']\", query=\"python\")\r\n    # \u5229\u7528xpath\u627e\u5230\u70b9\u51fb\u5bf9\u8c61\uff0c\u6a21\u62df\u70b9\u51fb\r\n    w.click_waiter(xpath=\"//input[@id='su']\")\r\n    # \u5728\u65b0\u7684\u9875\u9762\u627e\u5230\u7b2c\u4e00\u4e2a\u94fe\u63a5\uff0c\u5e76\u83b7\u53d6\u5176\u4e2d\u7684href\u5c5e\u6027\u503c\r\n    result = w.elements_text_waiter(xpath=\"//h3/a[@tabindex=0]\", attribute=\"href\")\r\n    print(result)\r\n```\r\n\r\n\u200b\t\u4e0a\u8ff0\u4f8b\u5b50\u4e2d\uff0c\u4f7f\u7528\u4e863\u4e2a\u57fa\u7840\u7684\u641c\u7d22\u51fd\u6570\uff0c\u5411\u8fd9\u6837\u7684\u641c\u7d22\u51fd\u6570\uff0c\u672c\u5e93\u4e00\u5171\u63d0\u4f9b\u4e865\u4e2a\uff0c\u5e94\u8be5\u53ef\u4ee5\u6ee1\u8db3\u7b80\u5355\u7684\u6a21\u62df\u9700\u6c42\uff0c\u5f53\u7136\u5982\u679c\u4f60\u6709\u4e00\u4e9b\u7279\u522b\u7684\u641c\u7d22\u9700\u6c42\uff0c\u4e5f\u53ef\u4ee5\u5c1d\u8bd5\u4f7f\u7528\u7b2c\u4e8c\u79cd\u65b9\u5f0f\u3002\r\n\r\n### \u7ee7\u627f\u5e76\u5f00\u53d1\u81ea\u5df1\u7684\u68c0\u7d22\u51fd\u6570\r\n\r\n```python\r\nfrom waiter import Waiter\r\nfrom selenium.webdriver.common.by import By\r\n\r\nclass MyWaiter(Waiter):\r\n    def __init__(self, driver, interval, is_track, is_log):\r\n        super().__init__(driver=driver, interval=interval, is_track=is_track, is_log=is_log)\r\n\r\n    def _textarea_send(self, xpath, **kwargs):\r\n        self.driver.find_element(By.XPATH, xpath).send(kwargs)\r\n\r\n    def textarea_send(self, xpath, **kwargs):\r\n        self.wait(func=\"_textarea_send\", xpath=xpath, interval=5, **kwargs)\r\n```\r\n\r\n## \u73af\u5883\r\n\r\n> window10\u3001window11\r\n>\r\n> python>=3.8\r\n>\r\n> selenium>=4.9.0\r\n>\r\n> loguru>=0.7.0\r\n\r\n## \u7248\u672c\r\n\r\n#### 2023.5.22\r\n\r\n**Version-1.0.4**\r\n\r\n- \u6dfb\u52a0select\u7ec4\u4ef6\u7684\u641c\u7d22\u5668\r\n  - \u6dfb\u52a0\u83b7\u53d6\u7ec4\u4ef6\u6240\u6709\u9009\u9879\r\n  - \u6dfb\u52a0\u5411select\u7ec4\u4ef6\u4e2d\u8f93\u5165\u503c\r\n\r\n[1.0.4](https://pypi.org/project/Swaiter/1.0.4/)\r\n\r\n#### 2023.5.16\r\n\r\n**Version-1.0.3**\r\n\r\n\u5b8c\u6210\u57fa\u7840\u529f\u80fd\r\n\r\n[1.0.3](https://pypi.org/project/Swaiter/1.0.3/)\r\n\r\n",
    "bugtrack_url": null,
    "license": "",
    "summary": "The loop waits until the selenium element has loaded.",
    "version": "1.0.4",
    "project_urls": {
        "github": "https://github.com/srx-2000/Swaiter"
    },
    "split_keywords": [
        "selenium",
        "crawler"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "45c7c828b3c8a87414a47139e8bcf60c1fa249024ccf0ad5508bb56ae57c5768",
                "md5": "9c497d6cbdcafb393376b53806c8c399",
                "sha256": "11f85c68c0edd5c53fb57c708a12926eb4fb128ea9c826887e0ae8831195fe9c"
            },
            "downloads": -1,
            "filename": "Swaiter-1.0.4-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "9c497d6cbdcafb393376b53806c8c399",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.7",
            "size": 5506,
            "upload_time": "2023-05-22T04:07:13",
            "upload_time_iso_8601": "2023-05-22T04:07:13.442459Z",
            "url": "https://files.pythonhosted.org/packages/45/c7/c828b3c8a87414a47139e8bcf60c1fa249024ccf0ad5508bb56ae57c5768/Swaiter-1.0.4-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "1f5fe48b8b695b202c0b459e9785714e321e64fa5f1baf48b0ff7e67f7a8367c",
                "md5": "8d38fc6c519fea136892134d63ffa3be",
                "sha256": "f1a2fcd757a8746b8b54e694086236c1e28435f5f1101b5f696879c03db56317"
            },
            "downloads": -1,
            "filename": "Swaiter-1.0.4.tar.gz",
            "has_sig": false,
            "md5_digest": "8d38fc6c519fea136892134d63ffa3be",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.7",
            "size": 5174,
            "upload_time": "2023-05-22T04:07:17",
            "upload_time_iso_8601": "2023-05-22T04:07:17.942191Z",
            "url": "https://files.pythonhosted.org/packages/1f/5f/e48b8b695b202c0b459e9785714e321e64fa5f1baf48b0ff7e67f7a8367c/Swaiter-1.0.4.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-05-22 04:07:17",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "srx-2000",
    "github_project": "Swaiter",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "swaiter"
}