<picture>
<source srcset="/unfurl/static/unfurl_dark.png" media="(prefers-color-scheme: dark)">
<img src="/unfurl/static/unfurl.png" alt="Unfurl Logo">
</picture>
# Extract and Visualize Data from URLs using Unfurl
Unfurl takes a URL and expands ("unfurls") it into a directed graph, extracting every bit of information from the URL and
exposing the obscured. It does this by breaking up a URL into components, extracting as much information as it can from
each piece, and presenting it all visually. This “show your work” approach (along with embedded references and documentation)
makes the analysis transparent to the user and helps them learn about (and discover) semantic and syntactical URL structures.
Unfurl has parsers for URLs, search engines, chat applications, social media sites, and more. It also has more generic parsers
(timestamps, UUIDs, etc) helpful for exploring new URLs or reverse engineering. It’s also easy to build new parsers, since
Unfurl is open source (Python 3) and has an extensible plugin system.
No matter if you extracted a URL from a memory image, carved it from slack space, or pulled it from a browser’s history file,
Unfurl can help you get the most out of it.
<img src="docs/unfurl-demo.gif"/>
## How to use Unfurl
### Online Version
1. There is an online version at **https://dfir.blog/unfurl**. Visit that page, enter the URL in the form, and
click 'Unfurl!'.
2. You can also access the online version using a bookmarklet - create a new bookmark and paste
`javascript:window.location.href='https://dfir.blog/unfurl/?url='+window.location.href;` as the location. Then when on any
page with an interesting URL, you can click the bookmarklet and see the URL "unfurled".
### Local Python Install
1. Install via pip: `pip install dfir-unfurl`
After Unfurl is installed, you can run use it via the web app or command-line:
1. Run `python unfurl_app.py`
1. Browse to localhost:5000/ (editable via config file)
1. Enter the URL to unfurl in the form, and 'Unfurl!'
OR
1. Run `python unfurl_cli.py https://twitter.com/_RyanBenson/status/1205161015177961473`
1. Output:
```
[1] https://twitter.com/_RyanBenson/status/1205161015177961473
├─(u)─[2] Scheme: https
├─(u)─[3] twitter.com
| ├─(u)─[5] Domain Name: twitter.com
| └─(u)─[6] TLD: com
└─(u)─[4] /_RyanBenson/status/1205161015177961473
├─(u)─[7] 1: _RyanBenson
├─(u)─[8] 2: status
└─(u)─[9] 3: 1205161015177961473
├─(❄)─[10] Timestamp: 1576167751484
| └─(🕓)─[13] 2019-12-12 16:22:31.484
├─(❄)─[11] Machine ID: 334
└─(❄)─[12] Sequence: 1
```
If the URL has special characters (like "&") that your shell might interpret as a command, put the URL in quotes.
Example: `python unfurl_cli.py "https://www.google.com/search?&ei=yTLGXeyKN_2y0PEP2smVuAg&q=dfir.blog&oq=dfir.blog&ved=0ahUKEwisk-WjmNzlAhV9GTQIHdpkBYcQ4dUDCAg"`
`unfurl_cli` has a number of command line options to modify its behavior:
```
optional arguments:
-h, --help show this help message and exit
-d, --detailed show more detailed explanations.
-f FILTER, --filter FILTER
only output lines that match this filter.
-o OUTPUT, --output OUTPUT
file to save output (as CSV) to. if omitted, output is sent to stdout (typically this means displayed in the console).
-v, -V, --version show program's version number and exit
```
### Docker
1. `git clone https://github.com/obsidianforensics/unfurl`
1. `cd unfurl`
1. `docker-compose up -d`
## Testing
1. All tests are run automatically on each PR by Travis CI. Tests need to pass before merging.
1. While not required, it is strongly encouraged to add tests that cover any new features in a PR.
1. To manually run all tests (units and integration): ``python -m unittest discover -s unfurl/tests``
If using Docker as above, run:
``docker exec unfurl python -m unittest discover -s unfurl/tests``
## Legal Bit
This is not an officially supported Google product.
Raw data
{
"_id": null,
"home_page": "https://github.com/obsidianforensics/unfurl",
"name": "dfir-unfurl",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "unfurl,forensics,dfir,reverse-engineering,security",
"author": "Ryan Benson",
"author_email": "ryan@dfir.blog",
"download_url": "https://files.pythonhosted.org/packages/e9/4b/a8ff543ea385c93d485070e31b559c3b91495fde9ec622dba5c03889ca0f/dfir-unfurl-20230901.tar.gz",
"platform": null,
"description": "<picture>\n <source srcset=\"/unfurl/static/unfurl_dark.png\" media=\"(prefers-color-scheme: dark)\">\n <img src=\"/unfurl/static/unfurl.png\" alt=\"Unfurl Logo\">\n</picture>\n\n# Extract and Visualize Data from URLs using Unfurl\nUnfurl takes a URL and expands (\"unfurls\") it into a directed graph, extracting every bit of information from the URL and \nexposing the obscured. It does this by breaking up a URL into components, extracting as much information as it can from \neach piece, and presenting it all visually. This \u201cshow your work\u201d approach (along with embedded references and documentation) \nmakes the analysis transparent to the user and helps them learn about (and discover) semantic and syntactical URL structures.\n\nUnfurl has parsers for URLs, search engines, chat applications, social media sites, and more. It also has more generic parsers \n(timestamps, UUIDs, etc) helpful for exploring new URLs or reverse engineering. It\u2019s also easy to build new parsers, since \nUnfurl is open source (Python 3) and has an extensible plugin system.\n\nNo matter if you extracted a URL from a memory image, carved it from slack space, or pulled it from a browser\u2019s history file, \nUnfurl can help you get the most out of it.\n\n<img src=\"docs/unfurl-demo.gif\"/>\n\n## How to use Unfurl\n\n### Online Version\n\n1. There is an online version at **https://dfir.blog/unfurl**. Visit that page, enter the URL in the form, and \nclick 'Unfurl!'. \n2. You can also access the online version using a bookmarklet - create a new bookmark and paste \n`javascript:window.location.href='https://dfir.blog/unfurl/?url='+window.location.href;` as the location. Then when on any\npage with an interesting URL, you can click the bookmarklet and see the URL \"unfurled\".\n\n### Local Python Install\n\n1. Install via pip: `pip install dfir-unfurl`\n\nAfter Unfurl is installed, you can run use it via the web app or command-line:\n\n1. Run `python unfurl_app.py`\n1. Browse to localhost:5000/ (editable via config file)\n1. Enter the URL to unfurl in the form, and 'Unfurl!'\n\nOR\n\n1. Run `python unfurl_cli.py https://twitter.com/_RyanBenson/status/1205161015177961473`\n1. Output: \n```\n[1] https://twitter.com/_RyanBenson/status/1205161015177961473\n \u251c\u2500(u)\u2500[2] Scheme: https\n \u251c\u2500(u)\u2500[3] twitter.com\n | \u251c\u2500(u)\u2500[5] Domain Name: twitter.com\n | \u2514\u2500(u)\u2500[6] TLD: com\n \u2514\u2500(u)\u2500[4] /_RyanBenson/status/1205161015177961473\n \u251c\u2500(u)\u2500[7] 1: _RyanBenson\n \u251c\u2500(u)\u2500[8] 2: status\n \u2514\u2500(u)\u2500[9] 3: 1205161015177961473\n \u251c\u2500(\u2744)\u2500[10] Timestamp: 1576167751484\n | \u2514\u2500(\ud83d\udd53)\u2500[13] 2019-12-12 16:22:31.484\n \u251c\u2500(\u2744)\u2500[11] Machine ID: 334\n \u2514\u2500(\u2744)\u2500[12] Sequence: 1 \n```\n\nIf the URL has special characters (like \"&\") that your shell might interpret as a command, put the URL in quotes. \nExample: `python unfurl_cli.py \"https://www.google.com/search?&ei=yTLGXeyKN_2y0PEP2smVuAg&q=dfir.blog&oq=dfir.blog&ved=0ahUKEwisk-WjmNzlAhV9GTQIHdpkBYcQ4dUDCAg\"`\n\n`unfurl_cli` has a number of command line options to modify its behavior:\n```\noptional arguments:\n -h, --help show this help message and exit\n -d, --detailed show more detailed explanations.\n -f FILTER, --filter FILTER\n only output lines that match this filter.\n -o OUTPUT, --output OUTPUT\n file to save output (as CSV) to. if omitted, output is sent to stdout (typically this means displayed in the console).\n -v, -V, --version show program's version number and exit\n```\n\n### Docker \n\n1. `git clone https://github.com/obsidianforensics/unfurl`\n1. `cd unfurl`\n1. `docker-compose up -d`\n\n## Testing \n\n1. All tests are run automatically on each PR by Travis CI. Tests need to pass before merging. \n1. While not required, it is strongly encouraged to add tests that cover any new features in a PR. \n1. To manually run all tests (units and integration): ``python -m unittest discover -s unfurl/tests``\n\nIf using Docker as above, run: \n``docker exec unfurl python -m unittest discover -s unfurl/tests``\n\n## Legal Bit\nThis is not an officially supported Google product.\n",
"bugtrack_url": null,
"license": "Apache",
"summary": "Unfurl takes a URL and expands (\"unfurls\") it into a directed graph",
"version": "20230901",
"project_urls": {
"Homepage": "https://github.com/obsidianforensics/unfurl"
},
"split_keywords": [
"unfurl",
"forensics",
"dfir",
"reverse-engineering",
"security"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e507410088c7dbc9cd4be0cb7c6968b37f92eb4961449f91fe8e89af4edc0522",
"md5": "11842146e7cae72b4e68df70b37a5a19",
"sha256": "cac9c24ac2fcaf8a416ec35e28d243a94f06343067b548432e59e07f4002c820"
},
"downloads": -1,
"filename": "dfir_unfurl-20230901-py3-none-any.whl",
"has_sig": false,
"md5_digest": "11842146e7cae72b4e68df70b37a5a19",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 5626087,
"upload_time": "2023-09-17T23:37:34",
"upload_time_iso_8601": "2023-09-17T23:37:34.593333Z",
"url": "https://files.pythonhosted.org/packages/e5/07/410088c7dbc9cd4be0cb7c6968b37f92eb4961449f91fe8e89af4edc0522/dfir_unfurl-20230901-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e94ba8ff543ea385c93d485070e31b559c3b91495fde9ec622dba5c03889ca0f",
"md5": "54ef27ad6c85f712b4162fc5d6cffff8",
"sha256": "666442daff7574efcae2592eb23c98a151f9e062b5cc23178f7e39a68cff2df3"
},
"downloads": -1,
"filename": "dfir-unfurl-20230901.tar.gz",
"has_sig": false,
"md5_digest": "54ef27ad6c85f712b4162fc5d6cffff8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 5580693,
"upload_time": "2023-09-17T23:37:36",
"upload_time_iso_8601": "2023-09-17T23:37:36.989339Z",
"url": "https://files.pythonhosted.org/packages/e9/4b/a8ff543ea385c93d485070e31b559c3b91495fde9ec622dba5c03889ca0f/dfir-unfurl-20230901.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-17 23:37:36",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "obsidianforensics",
"github_project": "unfurl",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "dfir-unfurl"
}
JSON
