# Documentation for the `Downloader` class
The `Downloader` class provides functionality to download and unzip files from a specified URL using Selenium WebDriver.
## Class Methods
### `__init__(self, download_directory="./", headless=True)`
The constructor method initializes the `Downloader` object. It takes two optional parameters:
- `download_directory` (default: "./"): Specifies the directory where the downloaded files will be saved.
- `headless` (default: True): Determines whether the Chrome WebDriver runs in headless mode or not.
### `_wait_for_download(self, timeout=120)`
This private method waits for the download to complete. It checks the Chrome Downloads page and returns the file paths of completed downloads. It takes an optional `timeout` parameter (default: 120 seconds) to specify the maximum time to wait for the download to complete.
### `download_zip_files(self, url)`
This method downloads zip files from a given URL. It performs the following steps:
1. Navigates to the specified URL.
2. Waits for the page to load.
3. Locates and clicks the accept button.
4. Finds the table element containing the files.
5. Iterates through each row of the table and checks if the file has a ".svs" extension.
6. If a ".svs" file is found, locates and clicks the download button.
7. Waits for the download to complete.
8. Logs the downloaded data.
9. Closes the WebDriver.
### `download_specific_project_files(self, txt_file="./")`
This method downloads zip files for specific projects listed in a text file. It takes an optional `txt_file` parameter (default: "./") that specifies the path to the text file containing project names. The method reads the file and iterates through each project name, calling the `download_zip_files` method for each project.
### `unzip_files(directory=None)`
This static method extracts files from tar.gz archives in a specified directory. It takes a required `directory` parameter that specifies the directory where the tar.gz files are located. The method loops through all files in the directory, identifies the ones with the ".tar.gz" extension, checks if the file size is greater than 0, and extracts the contents of the archive using the `tarfile` module.
## Example Usage
```python
# Create an instance of the Downloader class
downloader = Downloader(download_directory="./downloads", headless=True)
# Download zip files for specific projects listed in a text file
downloader.download_specific_project_files(txt_file="./projects.txt")
# Unzip files in the specified directory
downloader.unzip_files(directory="./downloads")
```
Note: Before using this class, make sure you have the required dependencies installed (Selenium WebDriver, ChromeDriver, and tarfile module).
## 👥 Authors
👤 **Manas**
* Github: [@manas1820](https://github.com/manas1820)
👤 **Nishita**
* Github: [@NishitaPatnaik21](https://github.com/NishitaPatnaik21)
Raw data
{
"_id": null,
"home_page": "https://github.com/Manas1820/svs-dataset-downloader",
"name": "svs-dataset-downloader",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "pypi,selenium,medical",
"author": "Manas Gupta",
"author_email": "manasgupta1820@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/3f/a0/92a3b5f7656b5fa5a23f422ca8d7a6f92a32c8d0515fb3e48fb3379f3be7/svs_dataset_downloader-0.0.3.tar.gz",
"platform": null,
"description": "# Documentation for the `Downloader` class\n\nThe `Downloader` class provides functionality to download and unzip files from a specified URL using Selenium WebDriver.\n\n## Class Methods\n\n### `__init__(self, download_directory=\"./\", headless=True)`\nThe constructor method initializes the `Downloader` object. It takes two optional parameters:\n- `download_directory` (default: \"./\"): Specifies the directory where the downloaded files will be saved.\n- `headless` (default: True): Determines whether the Chrome WebDriver runs in headless mode or not.\n\n### `_wait_for_download(self, timeout=120)`\nThis private method waits for the download to complete. It checks the Chrome Downloads page and returns the file paths of completed downloads. It takes an optional `timeout` parameter (default: 120 seconds) to specify the maximum time to wait for the download to complete.\n\n### `download_zip_files(self, url)`\nThis method downloads zip files from a given URL. It performs the following steps:\n1. Navigates to the specified URL.\n2. Waits for the page to load.\n3. Locates and clicks the accept button.\n4. Finds the table element containing the files.\n5. Iterates through each row of the table and checks if the file has a \".svs\" extension.\n6. If a \".svs\" file is found, locates and clicks the download button.\n7. Waits for the download to complete.\n8. Logs the downloaded data.\n9. Closes the WebDriver.\n\n### `download_specific_project_files(self, txt_file=\"./\")`\nThis method downloads zip files for specific projects listed in a text file. It takes an optional `txt_file` parameter (default: \"./\") that specifies the path to the text file containing project names. The method reads the file and iterates through each project name, calling the `download_zip_files` method for each project.\n\n### `unzip_files(directory=None)`\nThis static method extracts files from tar.gz archives in a specified directory. It takes a required `directory` parameter that specifies the directory where the tar.gz files are located. The method loops through all files in the directory, identifies the ones with the \".tar.gz\" extension, checks if the file size is greater than 0, and extracts the contents of the archive using the `tarfile` module.\n\n## Example Usage\n\n```python\n# Create an instance of the Downloader class\ndownloader = Downloader(download_directory=\"./downloads\", headless=True)\n\n# Download zip files for specific projects listed in a text file\ndownloader.download_specific_project_files(txt_file=\"./projects.txt\")\n\n# Unzip files in the specified directory\ndownloader.unzip_files(directory=\"./downloads\")\n```\n\nNote: Before using this class, make sure you have the required dependencies installed (Selenium WebDriver, ChromeDriver, and tarfile module).\n\n## \ud83d\udc65 Authors\n\n\ud83d\udc64 **Manas**\n* Github: [@manas1820](https://github.com/manas1820)\n\n\ud83d\udc64 **Nishita**\n* Github: [@NishitaPatnaik21](https://github.com/NishitaPatnaik21)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Testing installation of Package",
"version": "0.0.3",
"project_urls": {
"Bug Tracker": "https://github.com/Manas1820/svs-dataset-downloader/issues",
"Download": "https://github.com/Manas1820/svs-dataset-downloader/archive/refs/tags/v0.0.2.tar.gz",
"Homepage": "https://github.com/Manas1820/svs-dataset-downloader"
},
"split_keywords": [
"pypi",
"selenium",
"medical"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3fa092a3b5f7656b5fa5a23f422ca8d7a6f92a32c8d0515fb3e48fb3379f3be7",
"md5": "c070cf346a3c44c295c110badb86b05d",
"sha256": "5f60f4bed2dcdead1acb0f34adf9283cdeb190e46e597ef1160235162e3b3ab0"
},
"downloads": -1,
"filename": "svs_dataset_downloader-0.0.3.tar.gz",
"has_sig": false,
"md5_digest": "c070cf346a3c44c295c110badb86b05d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 4694,
"upload_time": "2023-05-18T07:13:05",
"upload_time_iso_8601": "2023-05-18T07:13:05.268760Z",
"url": "https://files.pythonhosted.org/packages/3f/a0/92a3b5f7656b5fa5a23f422ca8d7a6f92a32c8d0515fb3e48fb3379f3be7/svs_dataset_downloader-0.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-18 07:13:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Manas1820",
"github_project": "svs-dataset-downloader",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "async-generator",
"specs": [
[
"==",
"1.10"
]
]
},
{
"name": "attrs",
"specs": [
[
"==",
"23.1.0"
]
]
},
{
"name": "certifi",
"specs": [
[
"==",
"2023.5.7"
]
]
},
{
"name": "exceptiongroup",
"specs": [
[
"==",
"1.1.1"
]
]
},
{
"name": "h11",
"specs": [
[
"==",
"0.14.0"
]
]
},
{
"name": "idna",
"specs": [
[
"==",
"3.4"
]
]
},
{
"name": "outcome",
"specs": [
[
"==",
"1.2.0"
]
]
},
{
"name": "PySocks",
"specs": [
[
"==",
"1.7.1"
]
]
},
{
"name": "selenium",
"specs": [
[
"==",
"4.9.1"
]
]
},
{
"name": "sniffio",
"specs": [
[
"==",
"1.3.0"
]
]
},
{
"name": "sortedcontainers",
"specs": [
[
"==",
"2.4.0"
]
]
},
{
"name": "trio",
"specs": [
[
"==",
"0.22.0"
]
]
},
{
"name": "trio-websocket",
"specs": [
[
"==",
"0.10.2"
]
]
},
{
"name": "urllib3",
"specs": [
[
"==",
"2.0.2"
]
]
},
{
"name": "wsproto",
"specs": [
[
"==",
"1.2.0"
]
]
}
],
"lcname": "svs-dataset-downloader"
}
JSON
