<pre style="color: lime; background-color: black;">
███████╗██╗ ██╗██████╗ ██╗ ██╗██╗███████╗
██╔════╝██║ ██║██╔══██╗ ██║ ██║██║╚══███╔╝
███████╗██║ ██║██████╔╝ ██║ █╗ ██║██║ ███╔╝
╚════██║██║ ██║██╔══██╗ ██║███╗██║██║ ███╔╝
███████║╚██████╔╝██████╔╝ ╚███╔███╔╝██║███████╗
╚══════╝ ╚═════╝ ╚═════╝ ╚══╝╚══╝ ╚═╝╚══════╝
</pre>
A lightweight GPT model, trained to discover subdomains.
### Installation
```pipx install subwiz```
OR
```pip install subwiz```
### Recommended Use
Use [subfinder](https://github.com/projectdiscovery/subfinder) ❤️ to find subdomains from passive sources:
```subfinder -d example.com -o subdomains.txt```
Seed subwiz with these subdomains:
```subwiz -i subdomains.txt```
### Supported Switches
```commandline
usage: cli.py [-h] -i INPUT_FILE [-o OUTPUT_FILE] [-n NUM_PREDICTIONS] [--no-resolve]
[--force-download] [-t TEMPERATURE] [-d {auto,cpu,cuda,mps}] [-q MAX_NEW_TOKENS]
[--resolution_concurrency RESOLUTION_LIM] [--multi-apex]
options:
-h, --help show this help message and exit
-i INPUT_FILE, --input-file INPUT_FILE
file containing new-line-separated subdomains. (default: None)
-o OUTPUT_FILE, --output-file OUTPUT_FILE
output file to write new-line separated subdomains to. (default: None)
-n NUM_PREDICTIONS, --num_predictions NUM_PREDICTIONS
number of subdomains to predict. (default: 500)
--no-resolve do not resolve the output subdomains. (default: False)
--force-download download model and tokenizer files, even if cached. (default: False)
-t TEMPERATURE, --temperature TEMPERATURE
add randomness to the model, recommended ≤ 0.3) (default: 0.0)
-d {auto,cpu,cuda,mps}, --device {auto,cpu,cuda,mps}
hardware to run the transformer model on. (default: auto)
-q MAX_NEW_TOKENS, --max_new_tokens MAX_NEW_TOKENS
maximum length of predicted subdomains in tokens. (default: 10)
--resolution_concurrency RESOLUTION_LIM
number of concurrent resolutions. (default: 128)
--multi-apex allow multiple apex domains in the input file. runs inference for each
apex separately. (default: False)
```
### In Python
Use subwiz in Python, with the same parameters as the command line interface.
```
import subwiz
known_subdomains = ['test1.example.com', 'test2.example.com']
new_subdomains = subwiz.run(input_domains=known_subdomains)
```
---
### Model
Use the `--no-resolve` flag to inspect model outputs without checking if they resolve.
#### Architecture
Subwiz is a ultra-lightweight transformer model based on [nanoGPT](https://github.com/karpathy/nanoGPT/tree/master) ❤️:
- 17.3M parameters.
- Trained on 26M tokens, lists of subdomains from passive sources.
- Tokenizer trained on same lists of subdomains (8192 tokens).
#### Hugging Face
The model is saved in Hugging Face as [HadrianSecurity/subwiz](https://huggingface.co/HadrianSecurity/subwiz).
It is downloaded when you first run subwiz.
#### Inference
Typically, generative transformer models (e.g. ChatGPT) predict a single output sequence.
Subwiz predicts the N most likely sequences using a beam search algorithm.
*Beam search algorithm to predict the N most likely sequences using a generative transformer model.*
Raw data
{
"_id": null,
"home_page": null,
"name": "subwiz",
"maintainer": null,
"docs_url": null,
"requires_python": "<3.13,>=3.9",
"maintainer_email": null,
"keywords": "machine learning, recon, subdomains, transformers",
"author": null,
"author_email": "Klaas Meinke <klaas@hadrian.io>",
"download_url": "https://files.pythonhosted.org/packages/b7/11/b2628ee7892e07c461fbef312f45f2bba0af86cd41344b0cf9eac367c074/subwiz-0.3.0.tar.gz",
"platform": null,
"description": "<pre style=\"color: lime; background-color: black;\">\n\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2557\u2588\u2588\u2557 \u2588\u2588\u2557\u2588\u2588\u2588\u2588\u2588\u2588\u2557 \u2588\u2588\u2557 \u2588\u2588\u2557\u2588\u2588\u2557\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2557\n\u2588\u2588\u2554\u2550\u2550\u2550\u2550\u255d\u2588\u2588\u2551 \u2588\u2588\u2551\u2588\u2588\u2554\u2550\u2550\u2588\u2588\u2557 \u2588\u2588\u2551 \u2588\u2588\u2551\u2588\u2588\u2551\u255a\u2550\u2550\u2588\u2588\u2588\u2554\u255d\n\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2557\u2588\u2588\u2551 \u2588\u2588\u2551\u2588\u2588\u2588\u2588\u2588\u2588\u2554\u255d \u2588\u2588\u2551 \u2588\u2557 \u2588\u2588\u2551\u2588\u2588\u2551 \u2588\u2588\u2588\u2554\u255d \n\u255a\u2550\u2550\u2550\u2550\u2588\u2588\u2551\u2588\u2588\u2551 \u2588\u2588\u2551\u2588\u2588\u2554\u2550\u2550\u2588\u2588\u2557 \u2588\u2588\u2551\u2588\u2588\u2588\u2557\u2588\u2588\u2551\u2588\u2588\u2551 \u2588\u2588\u2588\u2554\u255d \n\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2551\u255a\u2588\u2588\u2588\u2588\u2588\u2588\u2554\u255d\u2588\u2588\u2588\u2588\u2588\u2588\u2554\u255d \u255a\u2588\u2588\u2588\u2554\u2588\u2588\u2588\u2554\u255d\u2588\u2588\u2551\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2557\n\u255a\u2550\u2550\u2550\u2550\u2550\u2550\u255d \u255a\u2550\u2550\u2550\u2550\u2550\u255d \u255a\u2550\u2550\u2550\u2550\u2550\u255d \u255a\u2550\u2550\u255d\u255a\u2550\u2550\u255d \u255a\u2550\u255d\u255a\u2550\u2550\u2550\u2550\u2550\u2550\u255d\n</pre>\n\nA lightweight GPT model, trained to discover subdomains.\n\n### Installation\n\n```pipx install subwiz```\n\nOR\n\n```pip install subwiz```\n\n### Recommended Use\n\nUse [subfinder](https://github.com/projectdiscovery/subfinder) \u2764\ufe0f to find subdomains from passive sources:\n\n```subfinder -d example.com -o subdomains.txt```\n\nSeed subwiz with these subdomains:\n\n```subwiz -i subdomains.txt```\n\n### Supported Switches\n\n```commandline\nusage: cli.py [-h] -i INPUT_FILE [-o OUTPUT_FILE] [-n NUM_PREDICTIONS] [--no-resolve]\n [--force-download] [-t TEMPERATURE] [-d {auto,cpu,cuda,mps}] [-q MAX_NEW_TOKENS]\n [--resolution_concurrency RESOLUTION_LIM] [--multi-apex]\n\noptions:\n -h, --help show this help message and exit\n -i INPUT_FILE, --input-file INPUT_FILE\n file containing new-line-separated subdomains. (default: None)\n -o OUTPUT_FILE, --output-file OUTPUT_FILE\n output file to write new-line separated subdomains to. (default: None)\n -n NUM_PREDICTIONS, --num_predictions NUM_PREDICTIONS\n number of subdomains to predict. (default: 500)\n --no-resolve do not resolve the output subdomains. (default: False)\n --force-download download model and tokenizer files, even if cached. (default: False)\n -t TEMPERATURE, --temperature TEMPERATURE\n add randomness to the model, recommended \u2264 0.3) (default: 0.0)\n -d {auto,cpu,cuda,mps}, --device {auto,cpu,cuda,mps}\n hardware to run the transformer model on. (default: auto)\n -q MAX_NEW_TOKENS, --max_new_tokens MAX_NEW_TOKENS\n maximum length of predicted subdomains in tokens. (default: 10)\n --resolution_concurrency RESOLUTION_LIM\n number of concurrent resolutions. (default: 128)\n --multi-apex allow multiple apex domains in the input file. runs inference for each\n apex separately. (default: False)\n```\n\n### In Python\n\nUse subwiz in Python, with the same parameters as the command line interface.\n\n```\nimport subwiz\n\nknown_subdomains = ['test1.example.com', 'test2.example.com']\nnew_subdomains = subwiz.run(input_domains=known_subdomains)\n```\n\n---\n### Model\n\nUse the `--no-resolve` flag to inspect model outputs without checking if they resolve.\n\n#### Architecture\n\nSubwiz is a ultra-lightweight transformer model based on [nanoGPT](https://github.com/karpathy/nanoGPT/tree/master) \u2764\ufe0f:\n\n- 17.3M parameters.\n- Trained on 26M tokens, lists of subdomains from passive sources.\n- Tokenizer trained on same lists of subdomains (8192 tokens).\n\n#### Hugging Face\nThe model is saved in Hugging Face as [HadrianSecurity/subwiz](https://huggingface.co/HadrianSecurity/subwiz).\nIt is downloaded when you first run subwiz.\n\n#### Inference\n\nTypically, generative transformer models (e.g. ChatGPT) predict a single output sequence.\nSubwiz predicts the N most likely sequences using a beam search algorithm.\n\n\n\n*Beam search algorithm to predict the N most likely sequences using a generative transformer model.*\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "A recon tool that uses AI to predict subdomains. Then returns those that resolve.",
"version": "0.3.0",
"project_urls": {
"Source": "https://github.com/hadriansecurity/subwiz"
},
"split_keywords": [
"machine learning",
" recon",
" subdomains",
" transformers"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "fb923c69d4df31ad408614ef724cc16c508675a6906ca4a02726233e6d717e85",
"md5": "a8b38b8a039c0d69f828b2e5e788dfcf",
"sha256": "a76c907f46422bd69b343ee9d7b205ce35acc1af3da196e7dc61fb8ecd61d67f"
},
"downloads": -1,
"filename": "subwiz-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "a8b38b8a039c0d69f828b2e5e788dfcf",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<3.13,>=3.9",
"size": 14735,
"upload_time": "2025-07-30T14:24:59",
"upload_time_iso_8601": "2025-07-30T14:24:59.829114Z",
"url": "https://files.pythonhosted.org/packages/fb/92/3c69d4df31ad408614ef724cc16c508675a6906ca4a02726233e6d717e85/subwiz-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "b711b2628ee7892e07c461fbef312f45f2bba0af86cd41344b0cf9eac367c074",
"md5": "30de1e4345e3d94688b1d6520df1a856",
"sha256": "c2fba21b56ed751bd7719bce364e9f86f76d14cb45c759603c8a482291f3ea5c"
},
"downloads": -1,
"filename": "subwiz-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "30de1e4345e3d94688b1d6520df1a856",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<3.13,>=3.9",
"size": 325054,
"upload_time": "2025-07-30T14:25:00",
"upload_time_iso_8601": "2025-07-30T14:25:00.914967Z",
"url": "https://files.pythonhosted.org/packages/b7/11/b2628ee7892e07c461fbef312f45f2bba0af86cd41344b0cf9eac367c074/subwiz-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-30 14:25:00",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "hadriansecurity",
"github_project": "subwiz",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "aiodns",
"specs": [
[
">=",
"1.0.0"
]
]
},
{
"name": "huggingface-hub",
"specs": [
[
">=",
"0.5.0"
]
]
},
{
"name": "pydantic",
"specs": [
[
">=",
"1.10.0"
]
]
},
{
"name": "pytest",
"specs": [
[
">=",
"8.0.0"
]
]
},
{
"name": "tldextract",
"specs": [
[
">=",
"3.0.0"
]
]
},
{
"name": "torch",
"specs": [
[
">=",
"2.3.0"
]
]
},
{
"name": "transformers",
"specs": [
[
">=",
"4.34.0"
]
]
}
],
"lcname": "subwiz"
}
JSON
