# surfhub
A python library for surfing and crawling website.
This library provides two basic components for you to run google search and getting result
* Seprer is a API to provide structured data from Google search. There are many serper providers such as ValueSerp, Serper, etc
* Scraper is an API to extract HTML from website. You can run it on your own laptop, but it is better to use providers such as Zyte, or Browserless
To start, you can visit [Serper](https://serper.dev) to get a free account.
```
import surhub import get_serper
s = serp.get_serper("serper", api_key="yourkey")
print(s.serp("hello world").items)
```
Supported SERP provider:
* [ValueSerp](https://valueserp.com/)
* Google Custom Search
* [Serper](https://serper.dev/)
Example to use scraper
```
from surfhub import get_scraper
s = serp.get_scraper("browserless", api_key="yourkey")
s.scrape("https://webscraper.io/test-sites/e-commerce/allinone")
```
Supported Scraper provider
* Local (run on your laptop) with proxy support
* Browserless
* Zyte
* Crawlbase
# TODO
- [ ] Support ScrappingBee
- [ ] Support [SerpAPI](https://serpapi.com/)
- [ ] Support Duckduckgo api
Raw data
{
"_id": null,
"home_page": "https://github.com/nqbao/surfhub",
"name": "surfhub",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": "serp, scraping, web, browserless, zyte",
"author": "Bao Nguyen",
"author_email": "qbao.nguyen@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/40/af/45ba7c810f82e97e38e4b25ae71ad8c2d54692b8c4010990b234502d88b3/surfhub-0.0.3.tar.gz",
"platform": null,
"description": "# surfhub\nA python library for surfing and crawling website. \n\nThis library provides two basic components for you to run google search and getting result\n\n* Seprer is a API to provide structured data from Google search. There are many serper providers such as ValueSerp, Serper, etc\n* Scraper is an API to extract HTML from website. You can run it on your own laptop, but it is better to use providers such as Zyte, or Browserless\n\nTo start, you can visit [Serper](https://serper.dev) to get a free account.\n\n```\nimport surhub import get_serper\n\ns = serp.get_serper(\"serper\", api_key=\"yourkey\")\nprint(s.serp(\"hello world\").items)\n```\n\nSupported SERP provider:\n * [ValueSerp](https://valueserp.com/)\n * Google Custom Search\n * [Serper](https://serper.dev/)\n\n\nExample to use scraper\n\n```\nfrom surfhub import get_scraper\n\ns = serp.get_scraper(\"browserless\", api_key=\"yourkey\")\ns.scrape(\"https://webscraper.io/test-sites/e-commerce/allinone\")\n```\n\nSupported Scraper provider\n * Local (run on your laptop) with proxy support\n * Browserless\n * Zyte\n * Crawlbase\n\n# TODO\n\n- [ ] Support ScrappingBee\n- [ ] Support [SerpAPI](https://serpapi.com/)\n- [ ] Support Duckduckgo api\n",
"bugtrack_url": null,
"license": null,
"summary": "A Python library for SERP and web scraping with multiple provider integration",
"version": "0.0.3",
"project_urls": {
"Bug Tracker": "https://github.com/nqbao/surfhub/issues",
"Homepage": "https://github.com/nqbao/surfhub"
},
"split_keywords": [
"serp",
" scraping",
" web",
" browserless",
" zyte"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7cff4cdc467d511b43773ef12894295d4506e2142333851e45ba842f7dd3e151",
"md5": "d05874d01e33f01a1423ba5f38e49d8c",
"sha256": "c38356da5825e579f61b6ad7bf4339d40da858aaa7075cfec4f6cd6c60651b44"
},
"downloads": -1,
"filename": "surfhub-0.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d05874d01e33f01a1423ba5f38e49d8c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 10547,
"upload_time": "2024-12-31T07:13:45",
"upload_time_iso_8601": "2024-12-31T07:13:45.729772Z",
"url": "https://files.pythonhosted.org/packages/7c/ff/4cdc467d511b43773ef12894295d4506e2142333851e45ba842f7dd3e151/surfhub-0.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "40af45ba7c810f82e97e38e4b25ae71ad8c2d54692b8c4010990b234502d88b3",
"md5": "7bd7d0d1ef55ebc2621ed43004144996",
"sha256": "0fda4f099ec5b74e8c48191b70867b9a5163b471df9adc7aac0d7449c1617773"
},
"downloads": -1,
"filename": "surfhub-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "7bd7d0d1ef55ebc2621ed43004144996",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 7533,
"upload_time": "2024-12-31T07:13:47",
"upload_time_iso_8601": "2024-12-31T07:13:47.890599Z",
"url": "https://files.pythonhosted.org/packages/40/af/45ba7c810f82e97e38e4b25ae71ad8c2d54692b8c4010990b234502d88b3/surfhub-0.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-31 07:13:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "nqbao",
"github_project": "surfhub",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "surfhub"
}
JSON
