# gemini-pro-web-scraper
Ever wondered about scraping a website without running a single line of code? Well, **Gemini Pro Web Scraper** is the
tool to do so. This tool automatically scrapes the data you want from a website of your choice.
# Source Code
The source code of the application **Gemini Pro Web Scraper** is available in
[Source Code](https://github.com/GlobalCreativeApkDev/gemini-pro-web-scraper/blob/master/main.py).
# Installation
```
pip install gemini-pro-web-scraper
```
# How to Use the Application?
Pre-requisites:
1. [Python](https://www.python.org/downloads/) installed in your device.
2. .env file in the same directory as <GEMINI_PRO_WEB_SCRAPER_DIRECTORY> and has the value of GEMINI_API_KEY.
First, open a Terminal or Command Prompt window and run the following command.
```
cd <GEMINI_PRO_WEB_SCRAPER_DIRECTORY>
python3 main.py
```
**Note:** Replace <GEMINI_PRO_WEB_SCRAPER_DIRECTORY> with the path to the directory of the application
**Gemini Pro Web Scraper**.
Then, the application will start with something looking like in the screenshot below.
You will then be asked to input the following values.
1. Temperature - between 0 and 1 inclusive
2. Top P - between 0 and 1 inclusive
3. Top K - at least 1
4. Max output tokens - at least 1
The following screenshot shows what is displayed after inputting the mentioned values.
You will be required to input the following pieces of information.
1. The URL of the website you want to scrape (e.g., https://sandbox.oxylabs.io/products).
2. What the URL entered in step 1 contains (e.g., **games** for https://sandbox.oxylabs.io/products).
3. The number of elements you want to scrape.
4. The details of each element you want to scrape (i.e., the name and the corresponding CSS selector for each element).
5. The name of the file you want the code to be in (without the extension).
Once you enter the values mentioned above, the file containing the code will be created inside "scrapers" directory.
Moreover, the CSV file containing the scraped data will be generated inside "csvs" directory. Then, you will be asked
whether you still want to continue unit testing or not. If you enter 'Y', you will be redirected to an application
window like in screenshot above. Else, you will exit the application.
The Python file generated which contains the web scraping code looks like below.
Below is how the generated CSV file looks like.
Raw data
{
"_id": null,
"home_page": "https://github.com/GlobalCreativeApkDev/gemini-pro-web-scraper",
"name": "gemini-pro-web-scraper",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "GlobalCreativeApkDev",
"author_email": "globalcreativeapkdev2022@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/7a/29/a0607e602af2a392f647fe65875c53e4bca17728f1bfb3b57b25ac30b2d7/gemini-pro-web-scraper-1.tar.gz",
"platform": null,
"description": "# gemini-pro-web-scraper\n\nEver wondered about scraping a website without running a single line of code? Well, **Gemini Pro Web Scraper** is the\ntool to do so. This tool automatically scrapes the data you want from a website of your choice.\n\n# Source Code\n\nThe source code of the application **Gemini Pro Web Scraper** is available in \n[Source Code](https://github.com/GlobalCreativeApkDev/gemini-pro-web-scraper/blob/master/main.py).\n\n# Installation\n\n```\npip install gemini-pro-web-scraper\n```\n\n# How to Use the Application?\n\nPre-requisites:\n1. [Python](https://www.python.org/downloads/) installed in your device.\n2. .env file in the same directory as <GEMINI_PRO_WEB_SCRAPER_DIRECTORY> and has the value of GEMINI_API_KEY.\n\nFirst, open a Terminal or Command Prompt window and run the following command.\n\n```\ncd <GEMINI_PRO_WEB_SCRAPER_DIRECTORY>\npython3 main.py\n```\n\n**Note:** Replace <GEMINI_PRO_WEB_SCRAPER_DIRECTORY> with the path to the directory of the application \n**Gemini Pro Web Scraper**.\n\nThen, the application will start with something looking like in the screenshot below.\n\n\n\nYou will then be asked to input the following values.\n\n1. Temperature - between 0 and 1 inclusive\n2. Top P - between 0 and 1 inclusive\n3. Top K - at least 1\n4. Max output tokens - at least 1\n\nThe following screenshot shows what is displayed after inputting the mentioned values.\n\n\n\nYou will be required to input the following pieces of information.\n\n1. The URL of the website you want to scrape (e.g., https://sandbox.oxylabs.io/products).\n2. What the URL entered in step 1 contains (e.g., **games** for https://sandbox.oxylabs.io/products).\n3. The number of elements you want to scrape.\n4. The details of each element you want to scrape (i.e., the name and the corresponding CSS selector for each element).\n5. The name of the file you want the code to be in (without the extension).\n\nOnce you enter the values mentioned above, the file containing the code will be created inside \"scrapers\" directory.\nMoreover, the CSV file containing the scraped data will be generated inside \"csvs\" directory. Then, you will be asked \nwhether you still want to continue unit testing or not. If you enter 'Y', you will be redirected to an application \nwindow like in screenshot above. Else, you will exit the application.\n\n\n\nThe Python file generated which contains the web scraping code looks like below.\n\n\n\nBelow is how the generated CSV file looks like.\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "This package contains implementation of a web scraper with Gemini Pro integrated into it.",
"version": "1",
"project_urls": {
"Homepage": "https://github.com/GlobalCreativeApkDev/gemini-pro-web-scraper"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c9f451aa11997339414301f93cca868818ea8ac169a6956361b891a653a68657",
"md5": "9a3e11149ec11967e0565effe6b0fcb0",
"sha256": "b7f6a960549ac94cf714aac50ca958645780cc8fcecc5fc49a9636c37c01793e"
},
"downloads": -1,
"filename": "gemini_pro_web_scraper-1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9a3e11149ec11967e0565effe6b0fcb0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 3503,
"upload_time": "2024-01-02T07:27:31",
"upload_time_iso_8601": "2024-01-02T07:27:31.766798Z",
"url": "https://files.pythonhosted.org/packages/c9/f4/51aa11997339414301f93cca868818ea8ac169a6956361b891a653a68657/gemini_pro_web_scraper-1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "7a29a0607e602af2a392f647fe65875c53e4bca17728f1bfb3b57b25ac30b2d7",
"md5": "4f901dd789f28a62be655c0b7aa4be57",
"sha256": "aaf933a8b5b1c1fc37a4cac08abda4dbaf12435d58497651986cf831ff3a7c16"
},
"downloads": -1,
"filename": "gemini-pro-web-scraper-1.tar.gz",
"has_sig": false,
"md5_digest": "4f901dd789f28a62be655c0b7aa4be57",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 3218,
"upload_time": "2024-01-02T07:27:33",
"upload_time_iso_8601": "2024-01-02T07:27:33.687066Z",
"url": "https://files.pythonhosted.org/packages/7a/29/a0607e602af2a392f647fe65875c53e4bca17728f1bfb3b57b25ac30b2d7/gemini-pro-web-scraper-1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-02 07:27:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "GlobalCreativeApkDev",
"github_project": "gemini-pro-web-scraper",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "python-dotenv",
"specs": [
[
"~=",
"1.0.0"
]
]
},
{
"name": "mpmath",
"specs": [
[
"~=",
"1.3.0"
]
]
},
{
"name": "setuptools",
"specs": [
[
"~=",
"69.0.3"
]
]
}
],
"lcname": "gemini-pro-web-scraper"
}
JSON
