# surfhub
A python library for surfing and crawling website.
This library provides two basic components for you to run google search and getting result
* Seprer is a API to provide structured data from Google search. There are many serper providers such as ValueSerp, Serper, etc
* Scraper is an API to extract HTML from website. You can run it on your own laptop, but it is better to use providers such as Zyte, or Browserless
To start, you can visit [Serper](https://serper.dev) to get a free account.
```
import surhub.serp as serp
s = serp.get_serp("serper", api_key="yourkey")
print(s.serp("hello world").items)
```
Supported SERP provider:
* [ValueSerp](https://valueserp.com/)
* Google Custom Search
* [Serper](https://serper.dev/)
TODO: [SerpAPI](https://serpapi.com/), DuckDuckGo
Example to use scrapper
```
import surfhub.scrapper as scapper
s = serp.get_scrapper("browserless", api_key="yourkey")
s.scrape("https://webscraper.io/test-sites/e-commerce/allinone")
```
Supported Scrapper provider
* Local (run on your laptop) with proxy support
* Browserless
* Zyte
* Crawlspace
TODO: ScrappingBee
Raw data
{
"_id": null,
"home_page": "https://github.com/nqbao/surfhub",
"name": "surfhub",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": "serp, scraping, web, browserless, zyte",
"author": "Bao Nguyen",
"author_email": "qbao.nguyen@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/64/36/9be9f7871e6bca111a358e2c4f59f02e33a5cbbaae4d66a8dafd40360bbd/surfhub-0.0.2.tar.gz",
"platform": null,
"description": "# surfhub\nA python library for surfing and crawling website. \n\nThis library provides two basic components for you to run google search and getting result\n\n* Seprer is a API to provide structured data from Google search. There are many serper providers such as ValueSerp, Serper, etc\n* Scraper is an API to extract HTML from website. You can run it on your own laptop, but it is better to use providers such as Zyte, or Browserless\n\nTo start, you can visit [Serper](https://serper.dev) to get a free account.\n\n```\nimport surhub.serp as serp\n\ns = serp.get_serp(\"serper\", api_key=\"yourkey\")\nprint(s.serp(\"hello world\").items)\n```\n\nSupported SERP provider:\n * [ValueSerp](https://valueserp.com/)\n * Google Custom Search\n * [Serper](https://serper.dev/)\n\nTODO: [SerpAPI](https://serpapi.com/), DuckDuckGo\n\n\nExample to use scrapper\n\n```\nimport surfhub.scrapper as scapper\n\ns = serp.get_scrapper(\"browserless\", api_key=\"yourkey\")\ns.scrape(\"https://webscraper.io/test-sites/e-commerce/allinone\")\n```\n\nSupported Scrapper provider\n * Local (run on your laptop) with proxy support\n * Browserless\n * Zyte\n * Crawlspace\n\nTODO: ScrappingBee\n",
"bugtrack_url": null,
"license": null,
"summary": "A Python library for SERP and web scraping with multiple provider integration",
"version": "0.0.2",
"project_urls": {
"Bug Tracker": "https://github.com/nqbao/surfhub/issues",
"Homepage": "https://github.com/nqbao/surfhub"
},
"split_keywords": [
"serp",
" scraping",
" web",
" browserless",
" zyte"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1c8b04ec6763979a9b4a217fcbca124c1314ffac820654b2b06294a566effb01",
"md5": "276c9d1a798a7ff0a2f13bcc8d17e24c",
"sha256": "3357ef3682b90477ce3a4cf3a71a7490ec9b5c59d1a377408f3b330d6f75e881"
},
"downloads": -1,
"filename": "surfhub-0.0.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "276c9d1a798a7ff0a2f13bcc8d17e24c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 10246,
"upload_time": "2024-12-16T04:29:43",
"upload_time_iso_8601": "2024-12-16T04:29:43.976391Z",
"url": "https://files.pythonhosted.org/packages/1c/8b/04ec6763979a9b4a217fcbca124c1314ffac820654b2b06294a566effb01/surfhub-0.0.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "64369be9f7871e6bca111a358e2c4f59f02e33a5cbbaae4d66a8dafd40360bbd",
"md5": "9bbdf4a8beba945b717235469dbcb2fd",
"sha256": "60d1333a21ad7440c73418665d4b2d803944d27d57cb8d83dd4d2965e9f3b95e"
},
"downloads": -1,
"filename": "surfhub-0.0.2.tar.gz",
"has_sig": false,
"md5_digest": "9bbdf4a8beba945b717235469dbcb2fd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 7214,
"upload_time": "2024-12-16T04:29:46",
"upload_time_iso_8601": "2024-12-16T04:29:46.047818Z",
"url": "https://files.pythonhosted.org/packages/64/36/9be9f7871e6bca111a358e2c4f59f02e33a5cbbaae4d66a8dafd40360bbd/surfhub-0.0.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-16 04:29:46",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "nqbao",
"github_project": "surfhub",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "surfhub"
}
JSON
