dark-web-scraper


Namedark-web-scraper JSON
Version 1 PyPI version JSON
download
home_pagehttps://github.com/PritamSarbajna/dark-web-scraper
SummaryA dark web link and image scraper
upload_time2023-07-31 19:39:02
maintainer
docs_urlNone
authorPritam Sarbajna
requires_python>=3.10
license
keywords darkweb web scraper scraper
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls. 
            
![Black Minimal Business Personal Profile Linkedin Banner](https://github.com/PritamSarbajna/dark-web-scraper/assets/90236635/676a6e65-5be3-4bda-a04c-47162ad14f51)

<div align="center" >
  
![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge&logo=python&logoColor=ffdd54)
![Kali](https://img.shields.io/badge/Kali-268BEE?style=for-the-badge&logo=kalilinux&logoColor=white)
![Linux](https://img.shields.io/badge/Linux-FCC624?style=for-the-badge&logo=linux&logoColor=black)
![Debian](https://img.shields.io/badge/Debian-D70A53?style=for-the-badge&logo=debian&logoColor=white)
![TOR](https://img.shields.io/badge/tor-%237E4798.svg?style=for-the-badge&logo=tor-project&logoColor=white)

</div>


<div align="center">
<a href="https://pypi.org/project/dark-web-scraper" target="_blank">
    <img src="https://img.shields.io/pypi/v/dark-web-scraper?color=%2334D058&label=pypi%20package" alt="Package version">
</a>
<a href="https://pypi.org/project/dark-web-scraper" target="_blank">
    <img src="https://img.shields.io/pypi/pyversions/dark-web-scraper.svg?color=%2334D058" alt="Supported Python versions">
</a>
<a href="http://badges.mit-license.org" target="_blank">
    <img src="http://img.shields.io/:license-mit-blue.svg?style=flat-square)" alt="Supported Python versions">
</a>
  
</div>

## :dart: Usage :

Currently this is only designed to
- Scrape dark web for onion links
- Scrape images from dark web
**Without tor browser**

## :wrench: Current Dependencies:
- Linux [ used debian based distro ]

# :gear: Prerequisite :

#### Enable socks

- Update package lists
```
$ sudo apt update
```

- Install tor package
```
$ sudo apt install tor
```

- Start Tor service
```
$ sudo service tor start
```

- Verify installation status
```
$ sudo service tor status
```
# :books: Tutorial :

### Install using pip

```
$ pip install dark-web-scraper
```

#### 1. Find onion urls from a dark web link

- Request : ```find_onion_links( str )```
- Response: links will be saved in `result.txt`
- Example :
```Python
# Main.py

from dark_web_scraper import find_onion_links
find_onion_links('http://random_url.onion')
```

#### 2. Scrape images on a dark web link

- Request : ```find_images_from_onion_link( str )```
- Response: links will be saved in `result.txt`
- Example :
```Python
# Main.py

from dark_web_scraper import find_images_from_onion_link
find_images_from_onion_link('http://random_url.onion')
```

<div style="background-color: #f8d7da; color: #721c24; padding: 10px;">
  <strong>Warning:</strong> This is a warning message. Pay attention to the important information you see here.
</div>



## :rocket: Features to be added :
- [ ] Scraping videos from dark web sites
- [ ] Object detection in images
- [ ] Sentiment aAnalysis on the webpage contents


## :warning: Disclaimer:

- I don't promote illegality.
- This project is just for educational purposes only/

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/PritamSarbajna/dark-web-scraper",
    "name": "dark-web-scraper",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.10",
    "maintainer_email": "",
    "keywords": "darkweb,web scraper,scraper",
    "author": "Pritam Sarbajna",
    "author_email": "",
    "download_url": "https://files.pythonhosted.org/packages/92/ae/8c93081a841493a96a0a644dde3c85aa0917d3e6912354ae7b0152f9fa56/dark_web_scraper-1.tar.gz",
    "platform": null,
    "description": "\r\n![Black Minimal Business Personal Profile Linkedin Banner](https://github.com/PritamSarbajna/dark-web-scraper/assets/90236635/676a6e65-5be3-4bda-a04c-47162ad14f51)\r\n\r\n<div align=\"center\" >\r\n  \r\n![Python](https://img.shields.io/badge/python-3670A0?style=for-the-badge&logo=python&logoColor=ffdd54)\r\n![Kali](https://img.shields.io/badge/Kali-268BEE?style=for-the-badge&logo=kalilinux&logoColor=white)\r\n![Linux](https://img.shields.io/badge/Linux-FCC624?style=for-the-badge&logo=linux&logoColor=black)\r\n![Debian](https://img.shields.io/badge/Debian-D70A53?style=for-the-badge&logo=debian&logoColor=white)\r\n![TOR](https://img.shields.io/badge/tor-%237E4798.svg?style=for-the-badge&logo=tor-project&logoColor=white)\r\n\r\n</div>\r\n\r\n\r\n<div align=\"center\">\r\n<a href=\"https://pypi.org/project/dark-web-scraper\" target=\"_blank\">\r\n    <img src=\"https://img.shields.io/pypi/v/dark-web-scraper?color=%2334D058&label=pypi%20package\" alt=\"Package version\">\r\n</a>\r\n<a href=\"https://pypi.org/project/dark-web-scraper\" target=\"_blank\">\r\n    <img src=\"https://img.shields.io/pypi/pyversions/dark-web-scraper.svg?color=%2334D058\" alt=\"Supported Python versions\">\r\n</a>\r\n<a href=\"http://badges.mit-license.org\" target=\"_blank\">\r\n    <img src=\"http://img.shields.io/:license-mit-blue.svg?style=flat-square)\" alt=\"Supported Python versions\">\r\n</a>\r\n  \r\n</div>\r\n\r\n## :dart: Usage :\r\n\r\nCurrently this is only designed to\r\n- Scrape dark web for onion links\r\n- Scrape images from dark web\r\n**Without tor browser**\r\n\r\n## :wrench: Current Dependencies:\r\n- Linux [ used debian based distro ]\r\n\r\n# :gear: Prerequisite :\r\n\r\n#### Enable socks\r\n\r\n- Update package lists\r\n```\r\n$ sudo apt update\r\n```\r\n\r\n- Install tor package\r\n```\r\n$ sudo apt install tor\r\n```\r\n\r\n- Start Tor service\r\n```\r\n$ sudo service tor start\r\n```\r\n\r\n- Verify installation status\r\n```\r\n$ sudo service tor status\r\n```\r\n# :books: Tutorial :\r\n\r\n### Install using pip\r\n\r\n```\r\n$ pip install dark-web-scraper\r\n```\r\n\r\n#### 1. Find onion urls from a dark web link\r\n\r\n- Request : ```find_onion_links( str )```\r\n- Response: links will be saved in `result.txt`\r\n- Example :\r\n```Python\r\n# Main.py\r\n\r\nfrom dark_web_scraper import find_onion_links\r\nfind_onion_links('http://random_url.onion')\r\n```\r\n\r\n#### 2. Scrape images on a dark web link\r\n\r\n- Request : ```find_images_from_onion_link( str )```\r\n- Response: links will be saved in `result.txt`\r\n- Example :\r\n```Python\r\n# Main.py\r\n\r\nfrom dark_web_scraper import find_images_from_onion_link\r\nfind_images_from_onion_link('http://random_url.onion')\r\n```\r\n\r\n<div style=\"background-color: #f8d7da; color: #721c24; padding: 10px;\">\r\n  <strong>Warning:</strong> This is a warning message. Pay attention to the important information you see here.\r\n</div>\r\n\r\n\r\n\r\n## :rocket: Features to be added :\r\n- [ ] Scraping videos from dark web sites\r\n- [ ] Object detection in images\r\n- [ ] Sentiment aAnalysis on the webpage contents\r\n\r\n\r\n## :warning: Disclaimer:\r\n\r\n- I don't promote illegality.\r\n- This project is just for educational purposes only/\r\n\r\n\r\n\r\n\r\n",
    "bugtrack_url": null,
    "license": "",
    "summary": "A dark web link and image scraper",
    "version": "1",
    "project_urls": {
        "Homepage": "https://github.com/PritamSarbajna/dark-web-scraper"
    },
    "split_keywords": [
        "darkweb",
        "web scraper",
        "scraper"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "fefd39a584d74ed103a82815af1c14433adbf3c0136c5def26f1b3b10c7c21f3",
                "md5": "987148ad6b552743d2c272811a5e164b",
                "sha256": "8b92e043541fce4d2d54e83ed931b312e51922cc93dd7f2f68360e49c1962d0e"
            },
            "downloads": -1,
            "filename": "dark_web_scraper-1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "987148ad6b552743d2c272811a5e164b",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.10",
            "size": 4314,
            "upload_time": "2023-07-31T19:38:58",
            "upload_time_iso_8601": "2023-07-31T19:38:58.672726Z",
            "url": "https://files.pythonhosted.org/packages/fe/fd/39a584d74ed103a82815af1c14433adbf3c0136c5def26f1b3b10c7c21f3/dark_web_scraper-1-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "92ae8c93081a841493a96a0a644dde3c85aa0917d3e6912354ae7b0152f9fa56",
                "md5": "a6ad54bbbed8955657f3c62bf93c42af",
                "sha256": "7acb646fd6d61bcbb508c907ddc0d397697385361a3a63fa2f3053f59d63084b"
            },
            "downloads": -1,
            "filename": "dark_web_scraper-1.tar.gz",
            "has_sig": false,
            "md5_digest": "a6ad54bbbed8955657f3c62bf93c42af",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.10",
            "size": 4107,
            "upload_time": "2023-07-31T19:39:02",
            "upload_time_iso_8601": "2023-07-31T19:39:02.872169Z",
            "url": "https://files.pythonhosted.org/packages/92/ae/8c93081a841493a96a0a644dde3c85aa0917d3e6912354ae7b0152f9fa56/dark_web_scraper-1.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-07-31 19:39:02",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "PritamSarbajna",
    "github_project": "dark-web-scraper",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "lcname": "dark-web-scraper"
}
Pritam Sarbajna
Elapsed time: 5.29588s