# GSERP API
## Overview
The GSERP API is an open-source project designed to perform search queries on Google and return organic search results. It leverages the `requests-ip-rotator` to handle IP rotation, ensuring that requests are made from different IP addresses to avoid being blocked by Google. The API is built using Python and provides a simple interface for querying and retrieving search results.
## Features
- **IP Rotation**: Utilizes `requests-ip-rotator` to rotate IP addresses for each request.
- **Customizable Search**: Allows specifying search parameters such as country, language, location, and fields to retrieve.
- **Field Selection**: Users can choose which fields to include in the search results, such as title, link, snippet, etc.
- **Error Handling**: Provides detailed error messages for invalid field selections and connection issues.
## Installation
```
pip install gserp-api
```
or from source:
```
git clone git@github.com:rushout09/gserp-api.git
cd gserp-api
python3 -m venv venv
source venv/bin/activate
pip install -r requriements.txt
```
## Usage
```
import os
from requests_ip_rotator import ApiGateway
from gserp_api import search
# Set AWS credentials
aws_access_key_id = os.getenv('AWS_ACCESS_KEY_ID')
aws_secret_access_key = os.getenv('AWS_SECRET_ACCESS_KEY')
# Create and start an API Gateway
gateway = ApiGateway("https://www.google.com", access_key_id=aws_access_key_id, secret_access_key=aws_secret_access_key)
gateway.start()
try:
# Perform a search
query = "OpenAI GPT-4"
fields = ["title", "link", "snippet"]
results = search(gateway, query, *fields, country="US", lang="en")
print(results)
finally:
# Shut down the API Gateway
gateway.shutdown()
```
## How to get AWS Credentials
To obtain AWS credentials, you need to create an IAM user in the AWS Management Console and assign the necessary permissions. Here’s a step-by-step guide:
How to Get AWS Credentials
1. Sign in to AWS Management Console:
Go to the AWS Management Console.
Sign in with your AWS account credentials.
2. Navigate to IAM (Identity and Access Management):
In the AWS Management Console, search for "IAM" in the services search bar and select it.
3. Create a New IAM User:
In the IAM dashboard, click on "Users" in the left sidebar.
Click the "Add user" button.
Enter a username for the new user.
Select the "Programmatic access" checkbox to generate an access key ID and secret access key.
4. Set Permissions:
Choose how you want to set permissions for the user. You can either:
Attach existing policies directly (e.g., AmazonS3FullAccess for S3 access).
Add the user to a group with the necessary permissions.
Copy permissions from an existing user.
Attach custom policies if needed.
5. Review and Create User:
Review the user details and permissions.
Click "Create user".
6. Download Credentials:
After the user is created, you will see the access key ID and secret access key.
Download the .csv file containing these credentials or copy them to a secure location. You will not be able to view the secret access key again.
7. Set Environment Variables:
On your local machine, set the AWS credentials as environment variables:
your_secret_access_key
8. Use Credentials in Your Application:
Your application can now use these environment variables to authenticate with AWS services.
Raw data
{
"_id": null,
"home_page": "https://github.com/rushout09/gserp-api",
"name": "gserp-api",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": null,
"author": "Rushabh Agarwal",
"author_email": "rushabh.agarwal9@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/9d/7d/8d67b6b26baf4214005ef5f1f649c3fe0b58f93763d47a83d5d741b073db/gserp_api-0.1.7.tar.gz",
"platform": null,
"description": "\n# GSERP API\n\n## Overview\n\nThe GSERP API is an open-source project designed to perform search queries on Google and return organic search results. It leverages the `requests-ip-rotator` to handle IP rotation, ensuring that requests are made from different IP addresses to avoid being blocked by Google. The API is built using Python and provides a simple interface for querying and retrieving search results.\n\n## Features\n\n- **IP Rotation**: Utilizes `requests-ip-rotator` to rotate IP addresses for each request.\n- **Customizable Search**: Allows specifying search parameters such as country, language, location, and fields to retrieve.\n- **Field Selection**: Users can choose which fields to include in the search results, such as title, link, snippet, etc.\n- **Error Handling**: Provides detailed error messages for invalid field selections and connection issues.\n\n## Installation\n\n```\npip install gserp-api\n```\n\nor from source:\n\n```\ngit clone git@github.com:rushout09/gserp-api.git\ncd gserp-api\npython3 -m venv venv\nsource venv/bin/activate\npip install -r requriements.txt\n```\n\n## Usage\n\n```\nimport os\nfrom requests_ip_rotator import ApiGateway\nfrom gserp_api import search\n\n# Set AWS credentials\naws_access_key_id = os.getenv('AWS_ACCESS_KEY_ID')\naws_secret_access_key = os.getenv('AWS_SECRET_ACCESS_KEY')\n\n# Create and start an API Gateway\ngateway = ApiGateway(\"https://www.google.com\", access_key_id=aws_access_key_id, secret_access_key=aws_secret_access_key)\ngateway.start()\n\ntry:\n # Perform a search\n query = \"OpenAI GPT-4\"\n fields = [\"title\", \"link\", \"snippet\"]\n results = search(gateway, query, *fields, country=\"US\", lang=\"en\")\n print(results)\n\nfinally:\n # Shut down the API Gateway\n gateway.shutdown()\n```\n\n## How to get AWS Credentials\n\nTo obtain AWS credentials, you need to create an IAM user in the AWS Management Console and assign the necessary permissions. Here\u2019s a step-by-step guide:\n\nHow to Get AWS Credentials\n\n1. Sign in to AWS Management Console:\nGo to the AWS Management Console.\nSign in with your AWS account credentials.\n\n2. Navigate to IAM (Identity and Access Management):\nIn the AWS Management Console, search for \"IAM\" in the services search bar and select it.\n\n3. Create a New IAM User:\nIn the IAM dashboard, click on \"Users\" in the left sidebar.\nClick the \"Add user\" button.\nEnter a username for the new user.\nSelect the \"Programmatic access\" checkbox to generate an access key ID and secret access key.\n\n4. Set Permissions:\nChoose how you want to set permissions for the user. You can either:\nAttach existing policies directly (e.g., AmazonS3FullAccess for S3 access).\nAdd the user to a group with the necessary permissions.\nCopy permissions from an existing user.\nAttach custom policies if needed.\n\n5. Review and Create User:\nReview the user details and permissions.\nClick \"Create user\".\n\n6. Download Credentials:\nAfter the user is created, you will see the access key ID and secret access key.\nDownload the .csv file containing these credentials or copy them to a secure location. You will not be able to view the secret access key again.\n\n7. Set Environment Variables:\nOn your local machine, set the AWS credentials as environment variables:\nyour_secret_access_key\n\n8. Use Credentials in Your Application:\nYour application can now use these environment variables to authenticate with AWS services.\n",
"bugtrack_url": null,
"license": null,
"summary": "Open Source Google SERP API",
"version": "0.1.7",
"project_urls": {
"Homepage": "https://github.com/rushout09/gserp-api"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "26562c05fe0a56893f24ffdc44e2a47396a648b8b1569b73f2b63b6676b1f072",
"md5": "2d1e16ed232c2038355153669a513b39",
"sha256": "64f7a9928814b6934c339138150deb6093ef7744678d915c94984e7c96d21c6f"
},
"downloads": -1,
"filename": "gserp_api-0.1.7-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2d1e16ed232c2038355153669a513b39",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 5903,
"upload_time": "2024-12-08T18:39:30",
"upload_time_iso_8601": "2024-12-08T18:39:30.460243Z",
"url": "https://files.pythonhosted.org/packages/26/56/2c05fe0a56893f24ffdc44e2a47396a648b8b1569b73f2b63b6676b1f072/gserp_api-0.1.7-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9d7d8d67b6b26baf4214005ef5f1f649c3fe0b58f93763d47a83d5d741b073db",
"md5": "b056ebc7f0477e30cffdacaa82cbb5fa",
"sha256": "9d2fbaa477461ace8fef71792ddb2efd507fd12f76e6cb24b29ca8a159d3e218"
},
"downloads": -1,
"filename": "gserp_api-0.1.7.tar.gz",
"has_sig": false,
"md5_digest": "b056ebc7f0477e30cffdacaa82cbb5fa",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 5491,
"upload_time": "2024-12-08T18:39:31",
"upload_time_iso_8601": "2024-12-08T18:39:31.977714Z",
"url": "https://files.pythonhosted.org/packages/9d/7d/8d67b6b26baf4214005ef5f1f649c3fe0b58f93763d47a83d5d741b073db/gserp_api-0.1.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-08 18:39:31",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "rushout09",
"github_project": "gserp-api",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "beautifulsoup4",
"specs": [
[
"==",
"4.12.3"
]
]
},
{
"name": "boto3",
"specs": [
[
"==",
"1.35.76"
]
]
},
{
"name": "botocore",
"specs": [
[
"==",
"1.35.76"
]
]
},
{
"name": "certifi",
"specs": [
[
"==",
"2024.8.30"
]
]
},
{
"name": "charset-normalizer",
"specs": [
[
"==",
"3.4.0"
]
]
},
{
"name": "idna",
"specs": [
[
"==",
"3.10"
]
]
},
{
"name": "jmespath",
"specs": [
[
"==",
"1.0.1"
]
]
},
{
"name": "python-dateutil",
"specs": [
[
"==",
"2.9.0.post0"
]
]
},
{
"name": "requests",
"specs": [
[
"==",
"2.32.3"
]
]
},
{
"name": "requests-ip-rotator",
"specs": [
[
"==",
"1.0.14"
]
]
},
{
"name": "s3transfer",
"specs": [
[
"==",
"0.10.4"
]
]
},
{
"name": "six",
"specs": [
[
"==",
"1.17.0"
]
]
},
{
"name": "soupsieve",
"specs": [
[
"==",
"2.6"
]
]
},
{
"name": "urllib3",
"specs": [
[
"==",
"1.26.20"
]
]
}
],
"lcname": "gserp-api"
}
JSON
