# Google Search Results in Python
[](https://badge.fury.io/py/google-search-results)
[](https://github.com/serpapi/google-search-results-python/actions/workflows/python-package.yml)
This Python package is meant to scrape and parse search results from Google, Bing, Baidu, Yandex, Yahoo, Home Depot, eBay and more, using [SerpApi](https://serpapi.com).
The following services are provided:
- [Search API](https://serpapi.com/search-api)
- [Search Archive API](https://serpapi.com/search-archive-api)
- [Account API](https://serpapi.com/account-api)
- [Location API](https://serpapi.com/locations-api) (Google Only)
SerpApi provides a [script builder](https://serpapi.com/demo) to get you started quickly.
## Installation
Python 3.7+
```bash
pip install google-search-results
```
[Link to the python package page](https://pypi.org/project/google-search-results/)
## Quick start
```python
from serpapi import GoogleSearch
search = GoogleSearch({
"q": "coffee",
"location": "Austin,Texas",
"api_key": "<your secret api key>"
})
result = search.get_dict()
```
This example runs a search for "coffee" using your secret API key.
The SerpApi service (backend)
- Searches Google using the search: q = "coffee"
- Parses the messy HTML responses
- Returns a standardized JSON response
The GoogleSearch class
- Formats the request
- Executes a GET http request against SerpApi service
- Parses the JSON response into a dictionary
Et voilà...
Alternatively, you can search:
- Bing using BingSearch class
- Baidu using BaiduSearch class
- Yahoo using YahooSearch class
- DuckDuckGo using DuckDuckGoSearch class
- eBay using EbaySearch class
- Yandex using YandexSearch class
- HomeDepot using HomeDepotSearch class
- GoogleScholar using GoogleScholarSearch class
- Youtube using YoutubeSearch class
- Walmart using WalmartSearch
- Apple App Store using AppleAppStoreSearch class
- Naver using NaverSearch class
See the [playground to generate your code.](https://serpapi.com/playground)
## Summary
- [Google Search Results in Python](#google-search-results-in-python)
- [Installation](#installation)
- [Quick start](#quick-start)
- [Summary](#summary)
- [Google Search API capability](#google-search-api-capability)
- [How to set SERP API key](#how-to-set-serp-api-key)
- [Example by specification](#example-by-specification)
- [Location API](#location-api)
- [Search Archive API](#search-archive-api)
- [Account API](#account-api)
- [Search Bing](#search-bing)
- [Search Baidu](#search-baidu)
- [Search Yandex](#search-yandex)
- [Search Yahoo](#search-yahoo)
- [Search Ebay](#search-ebay)
- [Search Home depot](#search-home-depot)
- [Search Youtube](#search-youtube)
- [Search Google Scholar](#search-google-scholar)
- [Generic search with SerpApiClient](#generic-search-with-serpapiclient)
- [Search Google Images](#search-google-images)
- [Search Google News](#search-google-news)
- [Search Google Shopping](#search-google-shopping)
- [Google Search By Location](#google-search-by-location)
- [Batch Asynchronous Searches](#batch-asynchronous-searches)
- [Python object as a result](#python-object-as-a-result)
- [Python paginate using iterator](#pagination-using-iterator)
- [Error management](#error-management)
- [Change log](#change-log)
- [Conclusion](#conclusion)
### Google Search API capability
Source code.
```python
params = {
"q": "coffee",
"location": "Location Requested",
"device": "desktop|mobile|tablet",
"hl": "Google UI Language",
"gl": "Google Country",
"safe": "Safe Search Flag",
"num": "Number of Results",
"start": "Pagination Offset",
"api_key": "Your SERP API Key",
# To be match
"tbm": "nws|isch|shop",
# To be search
"tbs": "custom to be search criteria",
# allow async request
"async": "true|false",
# output format
"output": "json|html"
}
# define the search search
search = GoogleSearch(params)
# override an existing parameter
search.params_dict["location"] = "Portland"
# search format return as raw html
html_results = search.get_html()
# parse results
# as python Dictionary
dict_results = search.get_dict()
# as JSON using json package
json_results = search.get_json()
# as dynamic Python object
object_result = search.get_object()
```
[Link to the full documentation](https://serpapi.com/search-api)
See below for more hands-on examples.
### How to set SERP API key
You can get an API key here if you don't already have one: https://serpapi.com/users/sign_up
The SerpApi `api_key` can be set globally:
```python
GoogleSearch.SERP_API_KEY = "Your Private Key"
```
The SerpApi `api_key` can be provided for each search:
```python
query = GoogleSearch({"q": "coffee", "serp_api_key": "Your Private Key"})
```
### Example by specification
We love true open source, continuous integration and Test Driven Development (TDD).
We are using RSpec to test [our infrastructure around the clock](https://travis-ci.org/serpapi/google-search-results-python) to achieve the best Quality of Service (QoS).
The directory test/ includes specification/examples.
Set your API key.
```bash
export API_KEY="your secret key"
```
Run test
```python
make test
```
### Location API
```python
from serpapi import GoogleSearch
search = GoogleSearch({})
location_list = search.get_location("Austin", 3)
print(location_list)
```
This prints the first 3 locations matching Austin (Texas, Texas, Rochester).
```python
[ { 'canonical_name': 'Austin,TX,Texas,United States',
'country_code': 'US',
'google_id': 200635,
'google_parent_id': 21176,
'gps': [-97.7430608, 30.267153],
'id': '585069bdee19ad271e9bc072',
'keys': ['austin', 'tx', 'texas', 'united', 'states'],
'name': 'Austin, TX',
'reach': 5560000,
'target_type': 'DMA Region'},
...]
```
### Search Archive API
The search results are stored in a temporary cache.
The previous search can be retrieved from the cache for free.
```python
from serpapi import GoogleSearch
search = GoogleSearch({"q": "Coffee", "location": "Austin,Texas"})
search_result = search.get_dictionary()
assert search_result.get("error") == None
search_id = search_result.get("search_metadata").get("id")
print(search_id)
```
Now let's retrieve the previous search from the archive.
```python
archived_search_result = GoogleSearch({}).get_search_archive(search_id, 'json')
print(archived_search_result.get("search_metadata").get("id"))
```
This prints the search result from the archive.
### Account API
```python
from serpapi import GoogleSearch
search = GoogleSearch({})
account = search.get_account()
```
This prints your account information.
### Search Bing
```python
from serpapi import BingSearch
search = BingSearch({"q": "Coffee", "location": "Austin,Texas"})
data = search.get_dict()
```
This code prints Bing search results for coffee as a Dictionary.
https://serpapi.com/bing-search-api
### Search Baidu
```python
from serpapi import BaiduSearch
search = BaiduSearch({"q": "Coffee"})
data = search.get_dict()
```
This code prints Baidu search results for coffee as a Dictionary.
https://serpapi.com/baidu-search-api
### Search Yandex
```python
from serpapi import YandexSearch
search = YandexSearch({"text": "Coffee"})
data = search.get_dict()
```
This code prints Yandex search results for coffee as a Dictionary.
https://serpapi.com/yandex-search-api
### Search Yahoo
```python
from serpapi import YahooSearch
search = YahooSearch({"p": "Coffee"})
data = search.get_dict()
```
This code prints Yahoo search results for coffee as a Dictionary.
https://serpapi.com/yahoo-search-api
### Search eBay
```python
from serpapi import EbaySearch
search = EbaySearch({"_nkw": "Coffee"})
data = search.get_dict()
```
This code prints eBay search results for coffee as a Dictionary.
https://serpapi.com/ebay-search-api
### Search Home Depot
```python
from serpapi import HomeDepotSearch
search = HomeDepotSearch({"q": "chair"})
data = search.get_dict()
```
This code prints Home Depot search results for chair as Dictionary.
https://serpapi.com/home-depot-search-api
### Search Youtube
```python
from serpapi import HomeDepotSearch
search = YoutubeSearch({"q": "chair"})
data = search.get_dict()
```
This code prints Youtube search results for chair as Dictionary.
https://serpapi.com/youtube-search-api
### Search Google Scholar
```python
from serpapi import GoogleScholarSearch
search = GoogleScholarSearch({"q": "Coffee"})
data = search.get_dict()
```
This code prints Google Scholar search results.
### Search Walmart
```python
from serpapi import WalmartSearch
search = WalmartSearch({"query": "chair"})
data = search.get_dict()
```
This code prints Walmart search results.
### Search Youtube
```python
from serpapi import YoutubeSearch
search = YoutubeSearch({"search_query": "chair"})
data = search.get_dict()
```
This code prints Youtube search results.
### Search Apple App Store
```python
from serpapi import AppleAppStoreSearch
search = AppleAppStoreSearch({"term": "Coffee"})
data = search.get_dict()
```
This code prints Apple App Store search results.
### Search Naver
```python
from serpapi import NaverSearch
search = NaverSearch({"query": "chair"})
data = search.get_dict()
```
This code prints Naver search results.
### Generic search with SerpApiClient
```python
from serpapi import SerpApiClient
query = {"q": "Coffee", "location": "Austin,Texas", "engine": "google"}
search = SerpApiClient(query)
data = search.get_dict()
```
This class enables interaction with any search engine supported by SerpApi.com
### Search Google Images
```python
from serpapi import GoogleSearch
search = GoogleSearch({"q": "coffe", "tbm": "isch"})
for image_result in search.get_dict()['images_results']:
link = image_result["original"]
try:
print("link: " + link)
# wget.download(link, '.')
except:
pass
```
This code prints all the image links,
and downloads the images if you un-comment the line with wget (Linux/OS X tool to download files).
This tutorial covers more ground on this topic.
https://github.com/serpapi/showcase-serpapi-tensorflow-keras-image-training
### Search Google News
```python
from serpapi import GoogleSearch
search = GoogleSearch({
"q": "coffe", # search search
"tbm": "nws", # news
"tbs": "qdr:d", # last 24h
"num": 10
})
for offset in [0,1,2]:
search.params_dict["start"] = offset * 10
data = search.get_dict()
for news_result in data['news_results']:
print(str(news_result['position'] + offset * 10) + " - " + news_result['title'])
```
This script prints the first 3 pages of the news headlines for the last 24 hours.
### Search Google Shopping
```python
from serpapi import GoogleSearch
search = GoogleSearch({
"q": "coffe", # search search
"tbm": "shop", # news
"tbs": "p_ord:rv", # last 24h
"num": 100
})
data = search.get_dict()
for shopping_result in data['shopping_results']:
print(shopping_result['position']) + " - " + shopping_result['title'])
```
This script prints all the shopping results, ordered by review order.
### Google Search By Location
With SerpApi, we can build a Google search from anywhere in the world.
This code looks for the best coffee shop for the given cities.
```python
from serpapi import GoogleSearch
for city in ["new york", "paris", "berlin"]:
location = GoogleSearch({}).get_location(city, 1)[0]["canonical_name"]
search = GoogleSearch({
"q": "best coffee shop", # search search
"location": location,
"num": 1,
"start": 0
})
data = search.get_dict()
top_result = data["organic_results"][0]["title"]
```
### Batch Asynchronous Searches
We offer two ways to boost your searches thanks to the`async` parameter.
- Blocking - async=false - more compute intensive because the search needs to maintain many connections. (default)
- Non-blocking - async=true - the way to go for large batches of queries (recommended)
```python
# Operating system
import os
# regular expression library
import re
# safe queue (named Queue in python2)
from queue import Queue
# Time utility
import time
# SerpApi search
from serpapi import GoogleSearch
# store searches
search_queue = Queue()
# SerpApi search
search = GoogleSearch({
"location": "Austin,Texas",
"async": True,
"api_key": os.getenv("API_KEY")
})
# loop through a list of companies
for company in ['amd', 'nvidia', 'intel']:
print("execute async search: q = " + company)
search.params_dict["q"] = company
result = search.get_dict()
if "error" in result:
print("oops error: ", result["error"])
continue
print("add search to the queue where id: ", result['search_metadata'])
# add search to the search_queue
search_queue.put(result)
print("wait until all search statuses are cached or success")
# Create regular search
while not search_queue.empty():
result = search_queue.get()
search_id = result['search_metadata']['id']
# retrieve search from the archive - blocker
print(search_id + ": get search from archive")
search_archived = search.get_search_archive(search_id)
print(search_id + ": status = " +
search_archived['search_metadata']['status'])
# check status
if re.search('Cached|Success',
search_archived['search_metadata']['status']):
print(search_id + ": search done with q = " +
search_archived['search_parameters']['q'])
else:
# requeue search_queue
print(search_id + ": requeue search")
search_queue.put(result)
# wait 1s
time.sleep(1)
print('all searches completed')
```
This code shows how to run searches asynchronously.
The search parameters must have {async: True}. This indicates that the client shouldn't wait for the search to be completed.
The current thread that executes the search is now non-blocking, which allows it to execute thousands of searches in seconds. The SerpApi backend will do the processing work.
The actual search result is deferred to a later call from the search archive using get_search_archive(search_id).
In this example the non-blocking searches are persisted in a queue: search_queue.
A loop through the search_queue allows it to fetch individual search results.
This process can easily be multithreaded to allow a large number of concurrent search requests.
To keep things simple, this example only explores search results one at a time (single threaded).
[See example.](https://github.com/serpapi/google-search-results-python/blob/master/tests/test_example.py)
### Python object as a result
The search results can be automatically wrapped in dynamically generated Python object.
This solution offers a more dynamic, fully Oriented Object Programming approach over the regular Dictionary / JSON data structure.
```python
from serpapi import GoogleSearch
search = GoogleSearch({"q": "Coffee", "location": "Austin,Texas"})
r = search.get_object()
assert type(r.organic_results), list
assert r.organic_results[0].title
assert r.search_metadata.id
assert r.search_metadata.google_url
assert r.search_parameters.q, "Coffee"
assert r.search_parameters.engine, "google"
```
### Pagination using iterator
Let's collect links across multiple search results pages.
```python
# to get 2 pages
start = 0
end = 40
page_size = 10
# basic search parameters
parameter = {
"q": "coca cola",
"tbm": "nws",
"api_key": os.getenv("API_KEY"),
# optional pagination parameter
# the pagination method can take argument directly
"start": start,
"end": end,
"num": page_size
}
# as proof of concept
# urls collects
urls = []
# initialize a search
search = GoogleSearch(parameter)
# create a python generator using parameter
pages = search.pagination()
# or set custom parameter
pages = search.pagination(start, end, page_size)
# fetch one search result per iteration
# using a basic python for loop
# which invokes python iterator under the hood.
for page in pages:
print(f"Current page: {page['serpapi_pagination']['current']}")
for news_result in page["news_results"]:
print(f"Title: {news_result['title']}\nLink: {news_result['link']}\n")
urls.append(news_result['link'])
# check if the total number pages is as expected
# note: the exact number if variable depending on the search engine backend
if len(urls) == (end - start):
print("all search results count match!")
if len(urls) == len(set(urls)):
print("all search results are unique!")
```
Examples to fetch links with pagination: [test file](https://github.com/serpapi/google-search-results-python/blob/master/tests/test_example_paginate.py), [online IDE](https://replit.com/@DimitryZub1/Scrape-Google-News-with-Pagination-python-serpapi)
### Error management
SerpApi keeps error management simple.
- backend service error or search fail
- client error
If it's a backend error, a simple error message is returned as string in the server response.
```python
from serpapi import GoogleSearch
search = GoogleSearch({"q": "Coffee", "location": "Austin,Texas", "api_key": "<secret_key>"})
data = search.get_json()
assert data["error"] == None
```
In some cases, there are more details available in the data object.
If it's a client error, then a SerpApiClientException is raised.
## Change log
2023-03-10 @ 2.4.2
- Change long description to README.md
2021-12-22 @ 2.4.1
- add more search engine
- youtube
- walmart
- apple_app_store
- naver
- raise SerpApiClientException instead of raw string in order to follow Python guideline 3.5+
- add more unit error tests for serp_api_client
2021-07-26 @ 2.4.0
- add page size support using num parameter
- add youtube search engine
2021-06-05 @ 2.3.0
- add pagination support
2021-04-28 @ 2.2.0
- add get_response method to provide raw requests.Response object
2021-04-04 @ 2.1.0
- Add home depot search engine
- get_object() returns dynamic Python object
2020-10-26 @ 2.0.0
- Reduce class name to <engine>Search
- Add get_raw_json
2020-06-30 @ 1.8.3
- simplify import
- improve package for python 3.5+
- add support for python 3.5 and 3.6
2020-03-25 @ 1.8
- add support for Yandex, Yahoo, Ebay
- clean-up test
2019-11-10 @ 1.7.1
- increase engine parameter priority over engine value set in the class
2019-09-12 @ 1.7
- Change namespace "from lib." instead: "from serpapi import GoogleSearch"
- Support for Bing and Baidu
2019-06-25 @ 1.6
- New search engine supported: Baidu and Bing
## Conclusion
SerpApi supports all the major search engines. Google has the more advance support with all the major services available: Images, News, Shopping and more..
To enable a type of search, the field tbm (to be matched) must be set to:
* isch: Google Images API.
* nws: Google News API.
* shop: Google Shopping API.
* any other Google service should work out of the box.
* (no tbm parameter): regular Google search.
The field `tbs` allows to customize the search even more.
[The full documentation is available here.](https://serpapi.com/search-api)
Raw data
{
"_id": null,
"home_page": "https://github.com/serpapi/google-search-results-python",
"name": "google-search-results",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.5",
"maintainer_email": "",
"keywords": "scrape,serp,api,json,search,localized,rank,google,bing,baidu,yandex,yahoo,ebay,scale,datamining,training,machine,ml,youtube,naver,walmart,apple,store,app",
"author": "vikoky",
"author_email": "victor@serpapi.com",
"download_url": "https://files.pythonhosted.org/packages/77/30/b3a6f6a2e00f8153549c2fa345c58ae1ce8e5f3153c2fe0484d444c3abcb/google_search_results-2.4.2.tar.gz",
"platform": null,
"description": "# Google Search Results in Python\n\n[](https://badge.fury.io/py/google-search-results)\n[](https://github.com/serpapi/google-search-results-python/actions/workflows/python-package.yml)\n\nThis Python package is meant to scrape and parse search results from Google, Bing, Baidu, Yandex, Yahoo, Home Depot, eBay and more, using [SerpApi](https://serpapi.com). \n\nThe following services are provided:\n- [Search API](https://serpapi.com/search-api)\n- [Search Archive API](https://serpapi.com/search-archive-api)\n- [Account API](https://serpapi.com/account-api)\n- [Location API](https://serpapi.com/locations-api) (Google Only)\n\nSerpApi provides a [script builder](https://serpapi.com/demo) to get you started quickly.\n\n## Installation\n\nPython 3.7+\n```bash\npip install google-search-results\n```\n\n[Link to the python package page](https://pypi.org/project/google-search-results/)\n\n## Quick start\n\n```python\nfrom serpapi import GoogleSearch\nsearch = GoogleSearch({\n \"q\": \"coffee\", \n \"location\": \"Austin,Texas\",\n \"api_key\": \"<your secret api key>\"\n })\nresult = search.get_dict()\n```\n\nThis example runs a search for \"coffee\" using your secret API key.\n\nThe SerpApi service (backend)\n- Searches Google using the search: q = \"coffee\"\n- Parses the messy HTML responses\n- Returns a standardized JSON response\nThe GoogleSearch class\n- Formats the request\n- Executes a GET http request against SerpApi service\n- Parses the JSON response into a dictionary\n\nEt voil\u00e0...\n\nAlternatively, you can search:\n- Bing using BingSearch class\n- Baidu using BaiduSearch class\n- Yahoo using YahooSearch class\n- DuckDuckGo using DuckDuckGoSearch class\n- eBay using EbaySearch class\n- Yandex using YandexSearch class\n- HomeDepot using HomeDepotSearch class\n- GoogleScholar using GoogleScholarSearch class\n- Youtube using YoutubeSearch class\n- Walmart using WalmartSearch\n- Apple App Store using AppleAppStoreSearch class\n- Naver using NaverSearch class\n\n\nSee the [playground to generate your code.](https://serpapi.com/playground)\n\n## Summary\n- [Google Search Results in Python](#google-search-results-in-python)\n - [Installation](#installation)\n - [Quick start](#quick-start)\n - [Summary](#summary)\n - [Google Search API capability](#google-search-api-capability)\n - [How to set SERP API key](#how-to-set-serp-api-key)\n - [Example by specification](#example-by-specification)\n - [Location API](#location-api)\n - [Search Archive API](#search-archive-api)\n - [Account API](#account-api)\n - [Search Bing](#search-bing)\n - [Search Baidu](#search-baidu)\n - [Search Yandex](#search-yandex)\n - [Search Yahoo](#search-yahoo)\n - [Search Ebay](#search-ebay)\n - [Search Home depot](#search-home-depot)\n - [Search Youtube](#search-youtube)\n - [Search Google Scholar](#search-google-scholar)\n - [Generic search with SerpApiClient](#generic-search-with-serpapiclient)\n - [Search Google Images](#search-google-images)\n - [Search Google News](#search-google-news)\n - [Search Google Shopping](#search-google-shopping)\n - [Google Search By Location](#google-search-by-location)\n - [Batch Asynchronous Searches](#batch-asynchronous-searches)\n - [Python object as a result](#python-object-as-a-result)\n - [Python paginate using iterator](#pagination-using-iterator)\n - [Error management](#error-management)\n - [Change log](#change-log)\n - [Conclusion](#conclusion)\n\n### Google Search API capability\nSource code.\n```python\nparams = {\n \"q\": \"coffee\",\n \"location\": \"Location Requested\", \n \"device\": \"desktop|mobile|tablet\",\n \"hl\": \"Google UI Language\",\n \"gl\": \"Google Country\",\n \"safe\": \"Safe Search Flag\",\n \"num\": \"Number of Results\",\n \"start\": \"Pagination Offset\",\n \"api_key\": \"Your SERP API Key\", \n # To be match\n \"tbm\": \"nws|isch|shop\", \n # To be search\n \"tbs\": \"custom to be search criteria\",\n # allow async request\n \"async\": \"true|false\",\n # output format\n \"output\": \"json|html\"\n}\n\n# define the search search\nsearch = GoogleSearch(params)\n# override an existing parameter\nsearch.params_dict[\"location\"] = \"Portland\"\n# search format return as raw html\nhtml_results = search.get_html()\n# parse results\n# as python Dictionary\ndict_results = search.get_dict()\n# as JSON using json package\njson_results = search.get_json()\n# as dynamic Python object\nobject_result = search.get_object()\n```\n[Link to the full documentation](https://serpapi.com/search-api)\n\nSee below for more hands-on examples.\n\n### How to set SERP API key\n\nYou can get an API key here if you don't already have one: https://serpapi.com/users/sign_up\n\nThe SerpApi `api_key` can be set globally:\n```python\nGoogleSearch.SERP_API_KEY = \"Your Private Key\"\n```\nThe SerpApi `api_key` can be provided for each search:\n```python\nquery = GoogleSearch({\"q\": \"coffee\", \"serp_api_key\": \"Your Private Key\"})\n```\n\n### Example by specification\n\nWe love true open source, continuous integration and Test Driven Development (TDD). \n We are using RSpec to test [our infrastructure around the clock](https://travis-ci.org/serpapi/google-search-results-python) to achieve the best Quality of Service (QoS).\n \nThe directory test/ includes specification/examples.\n\nSet your API key.\n```bash\nexport API_KEY=\"your secret key\"\n```\n\nRun test\n```python\nmake test\n```\n\n### Location API\n\n```python\nfrom serpapi import GoogleSearch\nsearch = GoogleSearch({})\nlocation_list = search.get_location(\"Austin\", 3)\nprint(location_list)\n```\n\nThis prints the first 3 locations matching Austin (Texas, Texas, Rochester).\n```python\n[ { 'canonical_name': 'Austin,TX,Texas,United States',\n 'country_code': 'US',\n 'google_id': 200635,\n 'google_parent_id': 21176,\n 'gps': [-97.7430608, 30.267153],\n 'id': '585069bdee19ad271e9bc072',\n 'keys': ['austin', 'tx', 'texas', 'united', 'states'],\n 'name': 'Austin, TX',\n 'reach': 5560000,\n 'target_type': 'DMA Region'},\n ...]\n```\n\n### Search Archive API\n\nThe search results are stored in a temporary cache.\nThe previous search can be retrieved from the cache for free.\n\n```python\nfrom serpapi import GoogleSearch\nsearch = GoogleSearch({\"q\": \"Coffee\", \"location\": \"Austin,Texas\"})\nsearch_result = search.get_dictionary()\nassert search_result.get(\"error\") == None\nsearch_id = search_result.get(\"search_metadata\").get(\"id\")\nprint(search_id)\n```\n\nNow let's retrieve the previous search from the archive.\n\n```python\narchived_search_result = GoogleSearch({}).get_search_archive(search_id, 'json')\nprint(archived_search_result.get(\"search_metadata\").get(\"id\"))\n```\nThis prints the search result from the archive.\n\n### Account API\n```python\nfrom serpapi import GoogleSearch\nsearch = GoogleSearch({})\naccount = search.get_account()\n```\nThis prints your account information.\n\n### Search Bing\n```python\nfrom serpapi import BingSearch\nsearch = BingSearch({\"q\": \"Coffee\", \"location\": \"Austin,Texas\"})\ndata = search.get_dict()\n```\nThis code prints Bing search results for coffee as a Dictionary. \n\nhttps://serpapi.com/bing-search-api\n\n### Search Baidu\n```python\nfrom serpapi import BaiduSearch\nsearch = BaiduSearch({\"q\": \"Coffee\"})\ndata = search.get_dict()\n```\nThis code prints Baidu search results for coffee as a Dictionary. \nhttps://serpapi.com/baidu-search-api\n\n### Search Yandex\n```python\nfrom serpapi import YandexSearch\nsearch = YandexSearch({\"text\": \"Coffee\"})\ndata = search.get_dict()\n```\nThis code prints Yandex search results for coffee as a Dictionary. \n\nhttps://serpapi.com/yandex-search-api\n\n### Search Yahoo\n```python\nfrom serpapi import YahooSearch\nsearch = YahooSearch({\"p\": \"Coffee\"})\ndata = search.get_dict()\n```\nThis code prints Yahoo search results for coffee as a Dictionary. \n\nhttps://serpapi.com/yahoo-search-api\n\n\n### Search eBay\n```python\nfrom serpapi import EbaySearch\nsearch = EbaySearch({\"_nkw\": \"Coffee\"})\ndata = search.get_dict()\n```\nThis code prints eBay search results for coffee as a Dictionary. \n\nhttps://serpapi.com/ebay-search-api\n\n### Search Home Depot\n```python\nfrom serpapi import HomeDepotSearch\nsearch = HomeDepotSearch({\"q\": \"chair\"})\ndata = search.get_dict()\n```\nThis code prints Home Depot search results for chair as Dictionary. \n\nhttps://serpapi.com/home-depot-search-api\n\n### Search Youtube\n```python\nfrom serpapi import HomeDepotSearch\nsearch = YoutubeSearch({\"q\": \"chair\"})\ndata = search.get_dict()\n```\nThis code prints Youtube search results for chair as Dictionary. \n\nhttps://serpapi.com/youtube-search-api\n\n### Search Google Scholar\n```python\nfrom serpapi import GoogleScholarSearch\nsearch = GoogleScholarSearch({\"q\": \"Coffee\"})\ndata = search.get_dict()\n```\nThis code prints Google Scholar search results.\n\n### Search Walmart\n```python\nfrom serpapi import WalmartSearch\nsearch = WalmartSearch({\"query\": \"chair\"})\ndata = search.get_dict()\n```\nThis code prints Walmart search results.\n\n### Search Youtube\n```python\nfrom serpapi import YoutubeSearch\nsearch = YoutubeSearch({\"search_query\": \"chair\"})\ndata = search.get_dict()\n```\nThis code prints Youtube search results.\n\n### Search Apple App Store\n```python\nfrom serpapi import AppleAppStoreSearch\nsearch = AppleAppStoreSearch({\"term\": \"Coffee\"})\ndata = search.get_dict()\n```\nThis code prints Apple App Store search results.\n\n### Search Naver\n```python\nfrom serpapi import NaverSearch\nsearch = NaverSearch({\"query\": \"chair\"})\ndata = search.get_dict()\n```\nThis code prints Naver search results.\n\n### Generic search with SerpApiClient\n```python\nfrom serpapi import SerpApiClient\nquery = {\"q\": \"Coffee\", \"location\": \"Austin,Texas\", \"engine\": \"google\"}\nsearch = SerpApiClient(query)\ndata = search.get_dict()\n```\nThis class enables interaction with any search engine supported by SerpApi.com \n\n### Search Google Images\n\n```python\nfrom serpapi import GoogleSearch\nsearch = GoogleSearch({\"q\": \"coffe\", \"tbm\": \"isch\"})\nfor image_result in search.get_dict()['images_results']:\n link = image_result[\"original\"]\n try:\n print(\"link: \" + link)\n # wget.download(link, '.')\n except:\n pass\n```\n\nThis code prints all the image links, \n and downloads the images if you un-comment the line with wget (Linux/OS X tool to download files).\n\nThis tutorial covers more ground on this topic.\nhttps://github.com/serpapi/showcase-serpapi-tensorflow-keras-image-training\n\n### Search Google News\n\n```python\nfrom serpapi import GoogleSearch\nsearch = GoogleSearch({\n \"q\": \"coffe\", # search search\n \"tbm\": \"nws\", # news\n \"tbs\": \"qdr:d\", # last 24h\n \"num\": 10\n})\nfor offset in [0,1,2]:\n search.params_dict[\"start\"] = offset * 10\n data = search.get_dict()\n for news_result in data['news_results']:\n print(str(news_result['position'] + offset * 10) + \" - \" + news_result['title'])\n```\n\nThis script prints the first 3 pages of the news headlines for the last 24 hours.\n\n### Search Google Shopping\n\n```python\nfrom serpapi import GoogleSearch\nsearch = GoogleSearch({\n \"q\": \"coffe\", # search search\n \"tbm\": \"shop\", # news\n \"tbs\": \"p_ord:rv\", # last 24h\n \"num\": 100\n})\ndata = search.get_dict()\nfor shopping_result in data['shopping_results']:\n print(shopping_result['position']) + \" - \" + shopping_result['title'])\n\n```\n\nThis script prints all the shopping results, ordered by review order.\n\n### Google Search By Location\n\nWith SerpApi, we can build a Google search from anywhere in the world.\nThis code looks for the best coffee shop for the given cities.\n\n```python\nfrom serpapi import GoogleSearch\nfor city in [\"new york\", \"paris\", \"berlin\"]:\n location = GoogleSearch({}).get_location(city, 1)[0][\"canonical_name\"]\n search = GoogleSearch({\n \"q\": \"best coffee shop\", # search search\n \"location\": location,\n \"num\": 1,\n \"start\": 0\n })\n data = search.get_dict()\n top_result = data[\"organic_results\"][0][\"title\"]\n```\n\n### Batch Asynchronous Searches\n\nWe offer two ways to boost your searches thanks to the`async` parameter.\n - Blocking - async=false - more compute intensive because the search needs to maintain many connections. (default) \n- Non-blocking - async=true - the way to go for large batches of queries (recommended)\n\n```python\n# Operating system\nimport os\n\n# regular expression library\nimport re\n\n# safe queue (named Queue in python2)\nfrom queue import Queue\n\n# Time utility\nimport time\n\n# SerpApi search\nfrom serpapi import GoogleSearch\n\n# store searches\nsearch_queue = Queue()\n\n# SerpApi search\nsearch = GoogleSearch({\n \"location\": \"Austin,Texas\",\n \"async\": True,\n \"api_key\": os.getenv(\"API_KEY\")\n})\n\n# loop through a list of companies\nfor company in ['amd', 'nvidia', 'intel']:\n print(\"execute async search: q = \" + company)\n search.params_dict[\"q\"] = company\n result = search.get_dict()\n if \"error\" in result:\n print(\"oops error: \", result[\"error\"])\n continue\n print(\"add search to the queue where id: \", result['search_metadata'])\n # add search to the search_queue\n search_queue.put(result)\n\nprint(\"wait until all search statuses are cached or success\")\n\n# Create regular search\nwhile not search_queue.empty():\n result = search_queue.get()\n search_id = result['search_metadata']['id']\n\n # retrieve search from the archive - blocker\n print(search_id + \": get search from archive\")\n search_archived = search.get_search_archive(search_id)\n print(search_id + \": status = \" +\n search_archived['search_metadata']['status'])\n\n # check status\n if re.search('Cached|Success',\n search_archived['search_metadata']['status']):\n print(search_id + \": search done with q = \" +\n search_archived['search_parameters']['q'])\n else:\n # requeue search_queue\n print(search_id + \": requeue search\")\n search_queue.put(result)\n\n # wait 1s\n time.sleep(1)\n\nprint('all searches completed')\n```\n\nThis code shows how to run searches asynchronously.\nThe search parameters must have {async: True}. This indicates that the client shouldn't wait for the search to be completed.\nThe current thread that executes the search is now non-blocking, which allows it to execute thousands of searches in seconds. The SerpApi backend will do the processing work.\nThe actual search result is deferred to a later call from the search archive using get_search_archive(search_id).\nIn this example the non-blocking searches are persisted in a queue: search_queue.\nA loop through the search_queue allows it to fetch individual search results.\nThis process can easily be multithreaded to allow a large number of concurrent search requests.\nTo keep things simple, this example only explores search results one at a time (single threaded).\n\n[See example.](https://github.com/serpapi/google-search-results-python/blob/master/tests/test_example.py)\n\n### Python object as a result\n\nThe search results can be automatically wrapped in dynamically generated Python object.\nThis solution offers a more dynamic, fully Oriented Object Programming approach over the regular Dictionary / JSON data structure.\n\n```python\nfrom serpapi import GoogleSearch\nsearch = GoogleSearch({\"q\": \"Coffee\", \"location\": \"Austin,Texas\"})\nr = search.get_object()\nassert type(r.organic_results), list\nassert r.organic_results[0].title\nassert r.search_metadata.id\nassert r.search_metadata.google_url\nassert r.search_parameters.q, \"Coffee\"\nassert r.search_parameters.engine, \"google\"\n```\n\n### Pagination using iterator\nLet's collect links across multiple search results pages.\n```python\n# to get 2 pages\nstart = 0\nend = 40\npage_size = 10\n\n# basic search parameters\nparameter = {\n \"q\": \"coca cola\",\n \"tbm\": \"nws\",\n \"api_key\": os.getenv(\"API_KEY\"),\n # optional pagination parameter\n # the pagination method can take argument directly\n \"start\": start,\n \"end\": end,\n \"num\": page_size\n}\n\n# as proof of concept \n# urls collects\nurls = []\n\n# initialize a search\nsearch = GoogleSearch(parameter)\n\n# create a python generator using parameter\npages = search.pagination()\n# or set custom parameter\npages = search.pagination(start, end, page_size)\n\n# fetch one search result per iteration \n# using a basic python for loop \n# which invokes python iterator under the hood.\nfor page in pages:\n print(f\"Current page: {page['serpapi_pagination']['current']}\")\n for news_result in page[\"news_results\"]:\n print(f\"Title: {news_result['title']}\\nLink: {news_result['link']}\\n\")\n urls.append(news_result['link'])\n \n# check if the total number pages is as expected\n# note: the exact number if variable depending on the search engine backend\nif len(urls) == (end - start):\n print(\"all search results count match!\")\nif len(urls) == len(set(urls)):\n print(\"all search results are unique!\")\n```\n\nExamples to fetch links with pagination: [test file](https://github.com/serpapi/google-search-results-python/blob/master/tests/test_example_paginate.py), [online IDE](https://replit.com/@DimitryZub1/Scrape-Google-News-with-Pagination-python-serpapi)\n\n### Error management\n\nSerpApi keeps error management simple.\n - backend service error or search fail\n - client error\n\nIf it's a backend error, a simple error message is returned as string in the server response.\n```python\nfrom serpapi import GoogleSearch\nsearch = GoogleSearch({\"q\": \"Coffee\", \"location\": \"Austin,Texas\", \"api_key\": \"<secret_key>\"})\ndata = search.get_json()\nassert data[\"error\"] == None\n```\nIn some cases, there are more details available in the data object.\n\nIf it's a client error, then a SerpApiClientException is raised.\n\n## Change log\n2023-03-10 @ 2.4.2\n - Change long description to README.md\n\n2021-12-22 @ 2.4.1\n - add more search engine \n - youtube\n - walmart\n - apple_app_store\n - naver \n - raise SerpApiClientException instead of raw string in order to follow Python guideline 3.5+\n - add more unit error tests for serp_api_client\n\n2021-07-26 @ 2.4.0\n - add page size support using num parameter\n - add youtube search engine\n\n2021-06-05 @ 2.3.0\n - add pagination support\n\n2021-04-28 @ 2.2.0\n - add get_response method to provide raw requests.Response object\n\n2021-04-04 @ 2.1.0\n - Add home depot search engine\n - get_object() returns dynamic Python object\n \n2020-10-26 @ 2.0.0\n - Reduce class name to <engine>Search\n - Add get_raw_json\n\n2020-06-30 @ 1.8.3\n - simplify import\n - improve package for python 3.5+\n - add support for python 3.5 and 3.6\n\n2020-03-25 @ 1.8\n - add support for Yandex, Yahoo, Ebay\n - clean-up test\n\n2019-11-10 @ 1.7.1\n - increase engine parameter priority over engine value set in the class\n\n2019-09-12 @ 1.7\n - Change namespace \"from lib.\" instead: \"from serpapi import GoogleSearch\"\n - Support for Bing and Baidu\n\n2019-06-25 @ 1.6\n - New search engine supported: Baidu and Bing\n\n## Conclusion\nSerpApi supports all the major search engines. Google has the more advance support with all the major services available: Images, News, Shopping and more..\nTo enable a type of search, the field tbm (to be matched) must be set to:\n\n * isch: Google Images API.\n * nws: Google News API.\n * shop: Google Shopping API.\n * any other Google service should work out of the box.\n * (no tbm parameter): regular Google search.\n\nThe field `tbs` allows to customize the search even more.\n\n[The full documentation is available here.](https://serpapi.com/search-api)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Scrape and search localized results from Google, Bing, Baidu, Yahoo, Yandex, Ebay, Homedepot, youtube at scale using SerpApi.com",
"version": "2.4.2",
"project_urls": {
"Homepage": "https://github.com/serpapi/google-search-results-python"
},
"split_keywords": [
"scrape",
"serp",
"api",
"json",
"search",
"localized",
"rank",
"google",
"bing",
"baidu",
"yandex",
"yahoo",
"ebay",
"scale",
"datamining",
"training",
"machine",
"ml",
"youtube",
"naver",
"walmart",
"apple",
"store",
"app"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7730b3a6f6a2e00f8153549c2fa345c58ae1ce8e5f3153c2fe0484d444c3abcb",
"md5": "4e72e0d5af065af613f1d8bda10e84c1",
"sha256": "603a30ecae2af8e600b22635757a6df275dad4b934f975e67878ccd640b78245"
},
"downloads": -1,
"filename": "google_search_results-2.4.2.tar.gz",
"has_sig": false,
"md5_digest": "4e72e0d5af065af613f1d8bda10e84c1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.5",
"size": 18818,
"upload_time": "2023-03-10T11:13:09",
"upload_time_iso_8601": "2023-03-10T11:13:09.953387Z",
"url": "https://files.pythonhosted.org/packages/77/30/b3a6f6a2e00f8153549c2fa345c58ae1ce8e5f3153c2fe0484d444c3abcb/google_search_results-2.4.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-03-10 11:13:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "serpapi",
"github_project": "google-search-results-python",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "google-search-results"
}
JSON
