<div align="center">
<p>Sponsor of the project:</p>
<div>
<img src="https://user-images.githubusercontent.com/78694043/231375638-5bbf2989-fc7b-482a-b6fe-603d1d6d613f.svg" width="60" alt="SerpApi">
</div>
<a href="https://serpapi.com">
<b>API to get search engine results with ease.</b>
</a>
</div>
<h1 align="center">Python E-Commerce Scraper 🛒</h1>
<h4 align="center">
Scrape Amazon, eBay, Walmart, Home Depot and Google Shopping from a single Python module.
</h4>
<div align="center">
<img src="https://user-images.githubusercontent.com/78694043/231951681-c56b79ce-1643-406c-90c7-39692c6eeaee.svg" width="500" alt="SerpApi">
</div>
This tool uses [SerpApi](https://serpapi.com/) as a tool to parse data.
You can use provided API key that will be available after installation, however, it's purely for testing purposes to see if the tool fits your needs. If you'll be using it for your own purpose (personal or commercial), you have to use [your own SerpApi key](https://serpapi.com/manage-api-key).
## ⚙️Installation
```bash
$ pip install ecommerce-scraper-py
```
## Scrape Google Shopping
#### Example to scrape search page
```python
from google_shopping import GoogleShoppingSearch
scraper = GoogleShoppingSearch(
api_key='<your_serpapi_api_key>',
query='coffee',
domain='google.de',
country='de',
language='de',
price_from=20,
price_to=200,
results_limit=150
)
products = scraper.get_products()
scraper.print(products)
scraper.save_to_json(products)
```
___
#### Example to scrape product page
```python
from google_shopping import GoogleShoppingProduct
scraper = GoogleShoppingProduct(
api_key='<your_serpapi_api_key>',
product_id=14019378181107046593,
reviews_limit=125
)
product = scraper.get_product()
scraper.print(product)
scraper.save_to_json(product)
```
___
#### Advanced example
```python
from google_shopping import GoogleShoppingSearch, GoogleShoppingProduct
search_scraper = GoogleShoppingSearch(
api_key='<your_serpapi_api_key>',
query='Sony PlayStation 5',
price_from=400,
price_to=1000,
results_limit=10,
domain='google.de',
country='de',
language='de'
)
data = []
products = search_scraper.get_products()
for product in products:
product_scraper = GoogleShoppingProduct(
api_key='<your_serpapi_api_key>',
product_id=product['product_id'],
reviews_limit=15,
domain='google.de',
country='de',
language='de'
)
product_data = product_scraper.get_product()
data.append(product_data)
search_scraper.print(data)
search_scraper.save_to_json(data)
```
## Scrape Walmart
#### Example to scrape search page
```python
from walmart import WalmartSearch
scraper = WalmartSearch(
api_key='<your_serpapi_api_key>',
query='coffee starbucks',
price_from=20,
price_to=200,
results_limit=150,
# store="356"
)
products = scraper.get_products()
scraper.print(products)
scraper.save_to_json(products)
```
___
#### Example to scrape product page
```python
from walmart import WalmartProduct
scraper = WalmartProduct(
api_key='<your_serpapi_api_key>',
product_id=520468661,
reviews_limit=125
)
product = scraper.get_product()
scraper.print(product)
scraper.save_to_json(product)
```
___
#### Advanced example
```python
from walmart import WalmartSearch, WalmartProduct
search_scraper = WalmartSearch(
api_key='<your_serpapi_api_key>',
query='coffee starbucks',
price_from=20,
price_to=200,
results_limit=10
)
data = []
products = search_scraper.get_products()
for product in products:
product_scraper = WalmartProduct(
api_key='<your_serpapi_api_key>',
product_id=product['product_id'],
reviews_limit=15
)
product_data = product_scraper.get_product()
data.append(product_data)
search_scraper.print(data)
search_scraper.save_to_json(data)
```
## Scrape Home Depot
#### Example to scrape search page
```python
from home_depot import HomeDepotSearch
scraper = HomeDepotSearch(
api_key='<your_serpapi_api_key>',
query='chair',
price_from=20,
price_to=200,
results_limit=150,
# zip_code='04401' # zip code must be in the format '12345' or '12345-6789'
)
products = scraper.get_products()
scraper.print(products)
scraper.save_to_json(products)
```
___
#### Example to scrape product page
```python
from home_depot import HomeDepotProduct
scraper = HomeDepotProduct(
api_key='<your_serpapi_api_key>',
product_id=202054749
)
product = scraper.get_product()
scraper.print(product)
scraper.save_to_json(product)
```
___
#### Advanced example
```python
from home_depot import HomeDepotSearch, HomeDepotProduct
search_scraper = HomeDepotSearch(
api_key='<your_serpapi_api_key>',
query='chair',
price_from=20,
price_to=200,
results_limit=10,
zip_code='04401' # zip code must be in the format '12345' or '12345-6789'
)
data = []
products = search_scraper.get_products()
for product in products:
product_scraper = HomeDepotProduct(
api_key='<your_serpapi_api_key>',
product_id=product['product_id']
)
product_data = product_scraper.get_product()
data.append(product_data)
search_scraper.print(data)
search_scraper.save_to_json(data)
```
## Scrape Amazon
#### Example to scrape search page
```python
from amazon import AmazonSearch
scraper = AmazonSearch(
query='coffee',
results_limit=125,
price_from=20,
price_to=50,
currency='USD',
language='en_US',
customer_reviews_rating=4,
multiplier=1
)
products = scraper.get_products()
scraper.print(products)
scraper.save_to_json(products)
```
___
#### Example to scrape product page
```python
from amazon import AmazonProduct
scraper = AmazonProduct(
link='https://www.amazon.com/McCafe-Premium-Roast-Decaf-Coffee/dp/B07GCNDL91/ref=sr_1_1?currency=USD&keywords=coffee&qid=1684849762&refinements=p_36%3A2000-5000%2Cp_72%3A1248897011&rnid=1248895011&s=grocery&sr=1-1&th=1',
reviews_limit=35,
multiplier=1,
currency='USD',
language='en_US'
)
product = scraper.get_product()
scraper.print(product)
scraper.save_to_json(product)
```
___
#### Advanced example
```python
from amazon import AmazonSearch, AmazonProduct
search_scraper = AmazonSearch(
query='coffee',
results_limit=5,
price_from=20,
price_to=50,
currency='USD',
language='en_US',
customer_reviews_rating=4,
multiplier=1
)
data = []
products = search_scraper.get_products()
for product in products:
product_scraper = AmazonProduct(
link=product['link'],
reviews_limit=15,
multiplier=1,
currency='USD',
language='en_US'
)
product_data = product_scraper.get_product()
data.append(product_data)
search_scraper.print(data)
search_scraper.save_to_json(data)
```
## Scrape eBay
#### Example to scrape search page
```python
from ebay import EbaySearch
scraper = EbaySearch(
api_key='<your_serpapi_api_key>',
query='coffee starbucks',
price_from=20,
price_to=200,
results_limit=250,
domain='ebay.com'
)
products = scraper.get_products()
scraper.print(products)
scraper.save_to_json(products)
```
___
#### Example to scrape product page
```python
from ebay import EbayProduct
scraper = EbayProduct(
link='https://www.ebay.com/itm/2-Bags-STARBUCKS-French-Roast-DARK-Whole-Bean-100-Arabica-Coffee-40oz-ea-09-23/144356021636',
reviews_limit=125,
multiplier=1
)
product = scraper.get_product()
scraper.print(product)
scraper.save_to_json(product)
```
___
#### Advanced example
```python
from ebay import EbaySearch, EbayProduct
search_scraper = EbaySearch(
api_key='<your_serpapi_api_key>',
query='coffee',
results_limit=5,
domain='ebay.com'
)
data = []
products = search_scraper.get_products()
for product in products:
product_scraper = EbayProduct(
link=product['link'],
reviews_limit=15,
multiplier=1
)
product_data = product_scraper.get_product()
data.append(product_data)
search_scraper.print(data)
search_scraper.save_to_json(data)
```
### ✍Contributing
Feel free to open bug issue, something isn't working, what feature to add, or anything else.
Raw data
{
"_id": null,
"home_page": "https://github.com/dimitryzub/ecommerce-scraper-py",
"name": "ecommerce-scraper-py",
"maintainer": "Artur Chukhrai, Dmitiry Zub",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "chukhraiartur@gmail.com, dimitryzub@gmail.com",
"keywords": "amazon,ebay,home depot,walmart,google shopping,serpapi,scraper,python,ecommerce,e-commerce,web scraping,python web scraping,google-search-results",
"author": "Artur Chukhrai",
"author_email": "chukhraiartur@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/22/30/a50602303cde2b4abde1625b15da1312493c549d34f4801f0f24e98d15d5/ecommerce-scraper-py-0.1.1.tar.gz",
"platform": null,
"description": "<div align=\"center\">\r\n<p>Sponsor of the project:</p>\r\n<div>\r\n <img src=\"https://user-images.githubusercontent.com/78694043/231375638-5bbf2989-fc7b-482a-b6fe-603d1d6d613f.svg\" width=\"60\" alt=\"SerpApi\">\r\n</div>\r\n<a href=\"https://serpapi.com\">\r\n\t<b>API to get search engine results with ease.</b>\r\n</a>\r\n</div>\r\n\r\n<h1 align=\"center\">Python E-Commerce Scraper \ud83d\uded2</h1>\r\n\r\n<h4 align=\"center\">\r\n Scrape Amazon, eBay, Walmart, Home Depot and Google Shopping from a single Python module.\r\n</h4>\r\n\r\n<div align=\"center\">\r\n <img src=\"https://user-images.githubusercontent.com/78694043/231951681-c56b79ce-1643-406c-90c7-39692c6eeaee.svg\" width=\"500\" alt=\"SerpApi\">\r\n</div>\r\n\r\n\r\nThis tool uses [SerpApi](https://serpapi.com/) as a tool to parse data. \r\n\r\nYou can use provided API key that will be available after installation, however, it's purely for testing purposes to see if the tool fits your needs. If you'll be using it for your own purpose (personal or commercial), you have to use [your own SerpApi key](https://serpapi.com/manage-api-key).\r\n\r\n\r\n## \u2699\ufe0fInstallation\r\n\r\n```bash\r\n$ pip install ecommerce-scraper-py\r\n```\r\n\r\n## Scrape Google Shopping\r\n\r\n#### Example to scrape search page\r\n\r\n```python\r\nfrom google_shopping import GoogleShoppingSearch\r\n\r\nscraper = GoogleShoppingSearch(\r\n api_key='<your_serpapi_api_key>',\r\n query='coffee',\r\n domain='google.de',\r\n country='de',\r\n language='de',\r\n price_from=20,\r\n price_to=200,\r\n results_limit=150\r\n)\r\n\r\nproducts = scraper.get_products()\r\nscraper.print(products)\r\n\r\nscraper.save_to_json(products)\r\n```\r\n\r\n___\r\n\r\n#### Example to scrape product page\r\n\r\n```python\r\nfrom google_shopping import GoogleShoppingProduct\r\n\r\nscraper = GoogleShoppingProduct(\r\n api_key='<your_serpapi_api_key>',\r\n product_id=14019378181107046593,\r\n reviews_limit=125\r\n)\r\n\r\nproduct = scraper.get_product()\r\nscraper.print(product)\r\n\r\nscraper.save_to_json(product)\r\n```\r\n\r\n___\r\n\r\n#### Advanced example\r\n\r\n```python\r\nfrom google_shopping import GoogleShoppingSearch, GoogleShoppingProduct\r\n\r\nsearch_scraper = GoogleShoppingSearch(\r\n api_key='<your_serpapi_api_key>',\r\n query='Sony PlayStation 5',\r\n price_from=400,\r\n price_to=1000,\r\n results_limit=10,\r\n domain='google.de',\r\n country='de',\r\n language='de'\r\n)\r\n\r\ndata = []\r\n\r\nproducts = search_scraper.get_products()\r\n\r\nfor product in products:\r\n product_scraper = GoogleShoppingProduct(\r\n api_key='<your_serpapi_api_key>',\r\n product_id=product['product_id'],\r\n reviews_limit=15,\r\n domain='google.de',\r\n country='de',\r\n language='de'\r\n )\r\n\r\n product_data = product_scraper.get_product()\r\n data.append(product_data)\r\n\r\nsearch_scraper.print(data)\r\nsearch_scraper.save_to_json(data)\r\n```\r\n\r\n## Scrape Walmart\r\n\r\n#### Example to scrape search page\r\n\r\n```python\r\nfrom walmart import WalmartSearch\r\n\r\nscraper = WalmartSearch(\r\n api_key='<your_serpapi_api_key>',\r\n query='coffee starbucks',\r\n price_from=20,\r\n price_to=200,\r\n results_limit=150,\r\n # store=\"356\"\r\n)\r\n\r\nproducts = scraper.get_products()\r\nscraper.print(products)\r\n\r\nscraper.save_to_json(products)\r\n```\r\n\r\n___\r\n\r\n#### Example to scrape product page\r\n\r\n```python\r\nfrom walmart import WalmartProduct\r\n\r\nscraper = WalmartProduct(\r\n api_key='<your_serpapi_api_key>',\r\n product_id=520468661,\r\n reviews_limit=125\r\n)\r\n\r\nproduct = scraper.get_product()\r\nscraper.print(product)\r\n\r\nscraper.save_to_json(product)\r\n```\r\n\r\n___\r\n\r\n#### Advanced example\r\n\r\n```python\r\nfrom walmart import WalmartSearch, WalmartProduct\r\n\r\nsearch_scraper = WalmartSearch(\r\n api_key='<your_serpapi_api_key>',\r\n query='coffee starbucks',\r\n price_from=20,\r\n price_to=200,\r\n results_limit=10\r\n)\r\n\r\ndata = []\r\n\r\nproducts = search_scraper.get_products()\r\n\r\nfor product in products:\r\n product_scraper = WalmartProduct(\r\n api_key='<your_serpapi_api_key>',\r\n product_id=product['product_id'],\r\n reviews_limit=15\r\n )\r\n\r\n product_data = product_scraper.get_product()\r\n data.append(product_data)\r\n\r\nsearch_scraper.print(data)\r\nsearch_scraper.save_to_json(data)\r\n```\r\n\r\n## Scrape Home Depot\r\n\r\n#### Example to scrape search page\r\n\r\n```python\r\nfrom home_depot import HomeDepotSearch\r\n\r\nscraper = HomeDepotSearch(\r\n api_key='<your_serpapi_api_key>',\r\n query='chair',\r\n price_from=20,\r\n price_to=200,\r\n results_limit=150,\r\n # zip_code='04401' # zip code must be in the format '12345' or '12345-6789'\r\n)\r\n\r\nproducts = scraper.get_products()\r\nscraper.print(products)\r\n\r\nscraper.save_to_json(products)\r\n```\r\n\r\n___\r\n\r\n#### Example to scrape product page\r\n\r\n```python\r\nfrom home_depot import HomeDepotProduct\r\n\r\nscraper = HomeDepotProduct(\r\n api_key='<your_serpapi_api_key>',\r\n product_id=202054749\r\n)\r\n\r\nproduct = scraper.get_product()\r\nscraper.print(product)\r\n\r\nscraper.save_to_json(product)\r\n```\r\n\r\n___\r\n\r\n#### Advanced example\r\n\r\n```python\r\nfrom home_depot import HomeDepotSearch, HomeDepotProduct\r\n\r\nsearch_scraper = HomeDepotSearch(\r\n api_key='<your_serpapi_api_key>',\r\n query='chair',\r\n price_from=20,\r\n price_to=200,\r\n results_limit=10,\r\n zip_code='04401' # zip code must be in the format '12345' or '12345-6789'\r\n)\r\n\r\ndata = []\r\n\r\nproducts = search_scraper.get_products()\r\n\r\nfor product in products:\r\n product_scraper = HomeDepotProduct(\r\n api_key='<your_serpapi_api_key>',\r\n product_id=product['product_id']\r\n )\r\n\r\n product_data = product_scraper.get_product()\r\n data.append(product_data)\r\n\r\nsearch_scraper.print(data)\r\nsearch_scraper.save_to_json(data)\r\n```\r\n\r\n## Scrape Amazon\r\n\r\n#### Example to scrape search page\r\n\r\n```python\r\nfrom amazon import AmazonSearch\r\n\r\nscraper = AmazonSearch(\r\n query='coffee',\r\n results_limit=125,\r\n price_from=20,\r\n price_to=50,\r\n currency='USD',\r\n language='en_US',\r\n customer_reviews_rating=4,\r\n multiplier=1\r\n)\r\n\r\nproducts = scraper.get_products()\r\nscraper.print(products)\r\n\r\nscraper.save_to_json(products)\r\n```\r\n\r\n___\r\n\r\n#### Example to scrape product page\r\n\r\n```python\r\nfrom amazon import AmazonProduct\r\n\r\nscraper = AmazonProduct(\r\n link='https://www.amazon.com/McCafe-Premium-Roast-Decaf-Coffee/dp/B07GCNDL91/ref=sr_1_1?currency=USD&keywords=coffee&qid=1684849762&refinements=p_36%3A2000-5000%2Cp_72%3A1248897011&rnid=1248895011&s=grocery&sr=1-1&th=1',\r\n reviews_limit=35,\r\n multiplier=1,\r\n currency='USD',\r\n language='en_US'\r\n)\r\n\r\nproduct = scraper.get_product()\r\nscraper.print(product)\r\n\r\nscraper.save_to_json(product)\r\n```\r\n\r\n___\r\n\r\n#### Advanced example\r\n\r\n```python\r\nfrom amazon import AmazonSearch, AmazonProduct\r\n\r\nsearch_scraper = AmazonSearch(\r\n query='coffee',\r\n results_limit=5,\r\n price_from=20,\r\n price_to=50,\r\n currency='USD',\r\n language='en_US',\r\n customer_reviews_rating=4,\r\n multiplier=1\r\n)\r\n\r\ndata = []\r\n\r\nproducts = search_scraper.get_products()\r\n\r\nfor product in products:\r\n product_scraper = AmazonProduct(\r\n link=product['link'],\r\n reviews_limit=15,\r\n multiplier=1,\r\n currency='USD',\r\n language='en_US'\r\n )\r\n\r\n product_data = product_scraper.get_product()\r\n data.append(product_data)\r\n\r\nsearch_scraper.print(data)\r\nsearch_scraper.save_to_json(data)\r\n```\r\n\r\n## Scrape eBay\r\n\r\n#### Example to scrape search page\r\n\r\n```python\r\nfrom ebay import EbaySearch\r\n\r\nscraper = EbaySearch(\r\n api_key='<your_serpapi_api_key>',\r\n query='coffee starbucks',\r\n price_from=20,\r\n price_to=200,\r\n results_limit=250,\r\n domain='ebay.com'\r\n)\r\n\r\nproducts = scraper.get_products()\r\nscraper.print(products)\r\n\r\nscraper.save_to_json(products)\r\n```\r\n\r\n___\r\n\r\n#### Example to scrape product page\r\n\r\n```python\r\nfrom ebay import EbayProduct\r\n\r\nscraper = EbayProduct(\r\n link='https://www.ebay.com/itm/2-Bags-STARBUCKS-French-Roast-DARK-Whole-Bean-100-Arabica-Coffee-40oz-ea-09-23/144356021636',\r\n reviews_limit=125,\r\n multiplier=1\r\n)\r\n\r\nproduct = scraper.get_product()\r\nscraper.print(product)\r\n\r\nscraper.save_to_json(product)\r\n```\r\n\r\n___\r\n\r\n#### Advanced example\r\n\r\n```python\r\nfrom ebay import EbaySearch, EbayProduct\r\n\r\nsearch_scraper = EbaySearch(\r\n api_key='<your_serpapi_api_key>',\r\n query='coffee',\r\n results_limit=5,\r\n domain='ebay.com'\r\n)\r\n\r\ndata = []\r\n\r\nproducts = search_scraper.get_products()\r\n\r\nfor product in products:\r\n product_scraper = EbayProduct(\r\n link=product['link'],\r\n reviews_limit=15,\r\n multiplier=1\r\n )\r\n\r\n product_data = product_scraper.get_product()\r\n data.append(product_data)\r\n\r\nsearch_scraper.print(data)\r\nsearch_scraper.save_to_json(data)\r\n```\r\n\r\n### \u270dContributing\r\n\r\nFeel free to open bug issue, something isn't working, what feature to add, or anything else.\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Scrape ecommerce websites such as Amazon, eBay, Walmart, Home Depot, Google Shopping from a single module in Python\ud83d\udc0d",
"version": "0.1.1",
"project_urls": {
"Documentation": "https://github.com/dimitryzub/ecommerce-scraper-py",
"Homepage": "https://github.com/dimitryzub/ecommerce-scraper-py",
"Source": "https://github.com/dimitryzub/ecommerce-scraper-py",
"Tracker": "https://github.com/dimitryzub/ecommerce-scraper-py/issues"
},
"split_keywords": [
"amazon",
"ebay",
"home depot",
"walmart",
"google shopping",
"serpapi",
"scraper",
"python",
"ecommerce",
"e-commerce",
"web scraping",
"python web scraping",
"google-search-results"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "cedc7a9b6b072420c811fab0251565d79fe48435b0ab73497c085491e7961be3",
"md5": "5e42e4a8c23b7d2b79257e3f50ed19ae",
"sha256": "92a4e8ba02a1b96634f1aca82820e60190242d00ad9373ee3fdff8e63003016f"
},
"downloads": -1,
"filename": "ecommerce_scraper_py-0.1.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5e42e4a8c23b7d2b79257e3f50ed19ae",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 136800,
"upload_time": "2023-07-16T03:55:04",
"upload_time_iso_8601": "2023-07-16T03:55:04.645520Z",
"url": "https://files.pythonhosted.org/packages/ce/dc/7a9b6b072420c811fab0251565d79fe48435b0ab73497c085491e7961be3/ecommerce_scraper_py-0.1.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "2230a50602303cde2b4abde1625b15da1312493c549d34f4801f0f24e98d15d5",
"md5": "e4feda204978c58f394bbb97856968b8",
"sha256": "484ce7583a3f90dd192042117cf84585bef98f48e2080caa2858568826f3d884"
},
"downloads": -1,
"filename": "ecommerce-scraper-py-0.1.1.tar.gz",
"has_sig": false,
"md5_digest": "e4feda204978c58f394bbb97856968b8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 131752,
"upload_time": "2023-07-16T03:55:07",
"upload_time_iso_8601": "2023-07-16T03:55:07.600146Z",
"url": "https://files.pythonhosted.org/packages/22/30/a50602303cde2b4abde1625b15da1312493c549d34f4801f0f24e98d15d5/ecommerce-scraper-py-0.1.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-16 03:55:07",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dimitryzub",
"github_project": "ecommerce-scraper-py",
"github_not_found": true,
"lcname": "ecommerce-scraper-py"
}
JSON
