| Name | apkscan JSON |
| Version |
0.4.0
JSON |
| download |
| home_page | https://github.com/LucasFaudman/apkscan.git |
| Summary | Scan for secrets, endpoints, and other sensitive data after decompiling and deobfuscating Android files. (.apk, .xapk, .dex, .jar, .class, .smali, .zip, .aar, .arsc, .aab, .jadx.kts) |
| upload_time | 2024-07-18 15:27:07 |
| maintainer | None |
| docs_url | None |
| author | Lucas Faudman |
| requires_python | >=3.10 |
| license | MIT License Copyright (c) 2024 Lucas Faudman Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. --- Commercial License Terms Copyright (c) 2024 Lucas Faudman This software is dual-licensed under the MIT License (see above) and the following terms for commercial use. 1. Commercial Use: If you intend to use this software in a commercial product or service, you must obtain a commercial license from the author. Commercial use includes, but is not limited to, integrating the software into proprietary applications, reselling the software, or offering it as part of a paid service. 2. Obtaining a Commercial License: To obtain a commercial license, please contact LucasFaudman.com. Licensing fees and terms will be negotiated on a case-by-case basis. 3. Support and Warranty: The commercial license may include support and warranty terms that are not available under the MIT License. Details will be provided upon licensing agreement. 4. Modifications and Derivative Works: Under the commercial license, you are allowed to modify and create derivative works of the software. However, these modifications must comply with the terms set out in the commercial license agreement. 5. Liability and Indemnity: The author is not liable for any damages arising from the use of the software under the commercial license. The licensee agrees to indemnify and hold harmless the author from any claims or liabilities resulting from the use of the software in commercial applications. For further inquiries and licensing, please contact: Lucas Faudman LucasFaudman@gmail.com LucasFaudman.com |
| keywords |
secret scanner
jadx
decompile
android
java
security
mobile
decompiler
concurrency
penetration-testing
apktool
security-tools
fernflower
cfr
procyon
enjarify
krakatau
secret-scanning
decompiler-java
secret-scanner
apk
xapk
dex
jar
class
smali
zip
aar
arsc
aab
jadx.kts
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# APKscan
[](https://badge.fury.io/py/apkscan)
[](https://github.com/LucasFaudman/apkscan/issues)
[](LICENSE)
Scan for **secrets**, **endpoints**, and other **sensitive data** after **decompiling** and **deobfuscating** Android
files. (.apk, .xapk, .dex, .jar, .class, .smali, .zip, .aar, .arsc, .aab, .jadx.kts).
- [Why use APKscan?](#why-use-apkscan)
- [Features](#features)
- [Installation](#installation)
- [PyPi Install Command](#pypi-install-command)
- [From Source Install Commands](#from-source-install-commands)
- [Usage](#usage)
- [Basic Usage](#basic-usage)
- [Advanced Usage (All Command Line Arguments)](#advanced-usage)
- [Configuring Scanning Rules](#configuring-scanning-rules-secret-locators)
- [Secret Locator Structure](#secret-locator-structure)
- [Supported Secret Locator Input Formats](#supported-secret-locator-input-formats)
- [Secret Locator Input Examples](#secretlocator-json-or-yaml-format)
- [Configuring Decompilers](#configuring-decompilers)
- [Supported Decompilers](#supported-decompilers)
- [Using Multiple Decompilers](#using-multiple-decompilers)
- [Why use Multiple Decompilers?](#variety-of-decompilation-techniques)
- [Concurrency and Performance](#concurrency-and-performance)
- [Concurrency Options](#concurrency-options)
- [Concurrency Type](#concurrency-type)
- [Results Order](#results-order)
- [Max Workers](#max-workers)
- [Chunksize](#chunk-size-multiprocessing-and-order-submitted-only)
- [Timeout](#timeout)
- [Optimizing Performance](#optimizing-performance)
- [Contributing](#contributing)
- [License](#license)
---
## Why use APKscan?
### Find Leaked Secrets
APKs (Android Package Kits) often leak secrets due to over-reliance on *security through obscurity*. Developers sometimes leave **sensitive information** such as **API keys**, **tokens**, and **credentials** hidden within the code, assuming that they won't be found easily since the code has been compiled and obfuscated. However, this approach is fundamentally flawed, and such secrets can be exposed, leading to potential **security vulnerabilities**.
### Identify Sensitive Locations in Application Code
APKscan can help quickly identify sensitive locations in the code, such as SSL pinning libraries, root detection functions, and other security mechanisms. Identifying these functions can speed up reverse engineering and app manipulation by **quickly revealing critical points where an app enforces its security policies, making it easier to bypass them with tools like Frida.** By pinpointing these areas, APKscan aids in understanding an app's security mechanisms and potential weaknesses.
### Identify the Attack Surface of the Backend
APKscan also helps identify the attack surface of the backend by uncovering **forgotten endpoints**, **test data payloads**, and other **traces of backend interfaces** that developers might have **unintentionally exposed**. These endpoints can provide attackers with access to sensitive data or functionalities that are not meant for public use. By scanning for such endpoints and test data, APKscan assists in ensuring that the backend is secure and that **no unnecessary exposure is left in the deployed applications**.
## Features
### Automate the Scanning Process for Multiple Applications:
> APKscan allows you to automate the process of scanning for secrets in **any number of applications**, saving you time and ensuring thorough coverage.
### Multiple Decompilers and Deobfuscators:
> Utilize one or more decompilers and deobfuscators to increase the chances of finding hidden secrets.
- Supports all popular decompilers including `JADX`, `APKTool`, `CFR`, `Procyon`, `Krakatau`, and `Fernflower`, providing flexibility and robustness in your scanning process.
- Uses [`enjarify-adapter`](https://github.com/LucasFaudman/enjarify-adapter) to convert the Dalvik bytecode in `.apk` files into Java bytecode on the fly, so the resulting `.jar` can be processed by decompilers/deobfuscators that do not support `.apks` directly.
### Customizable Rules:
> Define your own secret locator rules or use the default ones provided. This flexibility allows you to tailor the scanning process to your specific needs and improve the detection accuracy of sensitive information.
- Support for common formats: `SecretLocator JSON`, `secret-patterns-db YAML`, `gitleaks TOML`, and simple key-value pairs.
### Flexible Output Options:
> Choose from multiple output formats (`JSON`, `YAML`, or `text`) and organize the results by input file or locator. This makes it easier to integrate with other tools and workflows, and to analyze the findings effectively.
### Comprehensive File Support:
> Decompile and scan a wide range of Android-related files, including `.apk,` `.xapk,` `.dex,` `.jar,` `.class,` `.smali,` `.zip,` `.aar,` `.arsc,` `.aab,` and `.jadx.kts` files.
- **NEW**: `.xapk` -> `.apk`(s) unpacking/extraction support added in v0.4.0.
### Advanced Configuration and Concurrency Options:
> APKscan offers advanced options for concurrency, decompilation, and scanning, enabling you to optimize the performance and behavior of the tool to suit your environment and requirements.
---
## Installation
> APKscan can be installed from PyPi or from source.
### PyPi Install Command
```bash
pip3 install apkscan
```
### From Source Install Commands
```bash
git clone https://github.com/LucasFaudman/apkscan.git
cd apkscan
python3 -m venv .venv
source .venv/bin/activate
pip3 install -e .
cd ../
```
---
## Usage
### Basic Usage
The most basic way to use APKscan is to decompile an APK using the default decompiler `JADX` and scan using the default Secret locator rules in [`default.json`](https://github.com/LucasFaudman/apkscan/blob/main/src/apkscan/secret_locators/default.json).
```bash
apkscan file-to-scan.apk
```
---
Multiple sets of Secret Locators are included and can be refrenced by name. For example, to scan for only AWS credentials and endpoints:
```bash
apkscan file-to-scan.apk -r aws endpoints
```
---
A slighly more complex example. This time 3 APKs will be decompiled then scanned using the custom rules at `/path/to/custom/rules.json`. The output written to `output_file.yaml` in `YAML` format, and the results will be grouped by which secret locator was matched. Files generated during decompilation will be removed after scanning.
```bash
apkscan -r /path/to/custom/rules.json -o output_file.yaml -f yaml -g locator -c file1.apk file2.apk file3.apk
```
Or in long form:
```bash
apkscan --rules /path/to/custom/rules.json --output output_file.yaml --format yaml --groupby locator --cleanup file1.apk file2.apk file3.apk
```
---
### Advanced Usage
<details>
<summary>
<h4>All Command Line Arguments</h4>
</summary>
```bash
usage: apkscan [-h] [-r [SECRET_LOCATOR_FILES ...]] [-o SECRETS_OUTPUT_FILE]
[-f {text,json,yaml}] [-g {file,locator,both}]
[-c | --cleanup | --no-cleanup] [-q] [--jadx [JADX]]
[--apktool [APKTOOL]] [--cfr [CFR]] [--procyon [PROCYON]]
[--krakatau [KRAKATAU]] [--fernflower [FERNFLOWER]]
[--enjarify-choice {auto,never,always}]
[--unpack-xapks | --no-unpack-xapks]
[-d | --deobfuscate | --no-deobfuscate]
[-w DECOMPILER_WORKING_DIR]
[--decompiler-output-suffix DECOMPILER_OUTPUT_SUFFIX]
[--decompiler-extra-args DECOMPILER_EXTRA_ARGS [DECOMPILER_EXTRA_ARGS ...]]
[-dct {thread,process,main}] [-dro {completed,submitted}]
[-dmw DECOMPILER_MAX_WORKERS] [-dcs DECOMPILER_CHUNKSIZE]
[-dto DECOMPILER_TIMEOUT] [-sct {thread,process,main}]
[-sro {completed,submitted}] [-smw SCANNER_MAX_WORKERS]
[-scs SCANNER_CHUNKSIZE] [-sto SCANNER_TIMEOUT]
[FILES_TO_SCAN ...]
APKscan v0.4.0 - Scan for secrets, endpoints, and other sensitive
data after decompiling and deobfuscating Android files. (.apk,
.xapk, .dex, .jar, .class, .smali, .zip, .aar, .arsc, .aab, .jadx.kts)
(c) Lucas Faudman, 2024. License information in LICENSE file. Credits
to the original authors of all dependencies used in this project.
options:
-h, --help show this help message and exit
Input Options:
FILES_TO_SCAN Path(s) to Java files to decompile and scan.
-r [SECRET_LOCATOR_FILES ...], --rules [SECRET_LOCATOR_FILES ...]
Path(s) to secret locator rules/patterns files OR
names of included locator sets. Files can be in
SecretLocator JSON, secret-patterns-db YAML, or
Gitleak TOML formats. Included locator sets:
all_secret_locators, aws, azure, cloud, curated,
default, endpoints, gcp, generic, gitleaks, high-
confidence, key_locators, leakin-regexes,
locator_sort, nuclei-regexes, secret. If not provided,
default rules will be used. See: apkscan/src/apkscan/secret_l
ocators/default.json
Output Options:
-o SECRETS_OUTPUT_FILE, --output SECRETS_OUTPUT_FILE
Output file for secrets found.
-f {text,json,yaml}, --format {text,json,yaml}
Output format for secrets found.
-g {file,locator,both}, --groupby {file,locator,both}
Group secrets by input file or locator. Default is
'both'.
-c, --cleanup, --no-cleanup
Remove decompiled output directories after scanning.
-q, --quiet Suppress output from subprocesses.
Decompiler Choices:
Choose which decompiler(s) to use. Optionally specify path to decompiler
binary. Default is JADX.
--jadx [JADX], -J [JADX]
Use JADX Java decompiler.
--apktool [APKTOOL], -A [APKTOOL]
Use APKTool SMALI disassembler.
--cfr [CFR], -C [CFR]
Use CFR Java decompiler. Requires Enjarify.
--procyon [PROCYON], -P [PROCYON]
Use Procyon Java decompiler. Requires Enjarify.
--krakatau [KRAKATAU], -K [KRAKATAU]
Use Krakatau Java decompiler. Requires Enjarify.
--fernflower [FERNFLOWER], -F [FERNFLOWER]
Use Fernflower Java decompiler. Requires Enjarify.
--enjarify-choice {auto,never,always}, -EC {auto,never,always}
When to use Enjarify. Default is 'auto' which means
use only when needed.
--unpack-xapks, --no-unpack-xapks
Unpack XAPK files into APKs before decompiling.
Default is True.
Decompiler Advanced Options:
Options for Java decompiler.
-d, --deobfuscate, --no-deobfuscate
Deobfuscate file before scanning.
-w DECOMPILER_WORKING_DIR, --decompiler-working-dir DECOMPILER_WORKING_DIR
Working directory where files will be decompiled.
--decompiler-output-suffix DECOMPILER_OUTPUT_SUFFIX
Suffix for decompiled output directory names. Default
is '-decompiled'.
--decompiler-extra-args DECOMPILER_EXTRA_ARGS [DECOMPILER_EXTRA_ARGS ...]
Additional arguments to pass to decompilers in form
quoted whitespace separated '<DECOMPILER_NAME>
<EXTRA_ARGS>...'. For example: --decompiler-extra-args
'jadx --no-debug-info,--no-inline'.
-dct {thread,process,main}, --decompiler-concurrency-type {thread,process,main}
Type of concurrency to use for decompilation. Default
is 'thread'.
-dro {completed,submitted}, --decompiler-results-order {completed,submitted}
Order to process results from decompiler. Default is
'completed'.
-dmw DECOMPILER_MAX_WORKERS, --decompiler-max-workers DECOMPILER_MAX_WORKERS
Maximum number of workers to use for decompilation.
-dcs DECOMPILER_CHUNKSIZE, --decompiler-chunksize DECOMPILER_CHUNKSIZE
Number of files to decompile per thread/process.
-dto DECOMPILER_TIMEOUT, --decompiler-timeout DECOMPILER_TIMEOUT
Timeout for decompilation in seconds.
Secret Scanner Advanced Options:
Options for secret scanner.
-sct {thread,process,main}, --scanner-concurrency-type {thread,process,main}
Type of concurrency to use for scanning. Default is
'process'.
-sro {completed,submitted}, --scanner-results-order {completed,submitted}
Order to process results from scanner. Default is
'completed'.
-smw SCANNER_MAX_WORKERS, --scanner-max-workers SCANNER_MAX_WORKERS
Maximum number of workers to use for scanning.
-scs SCANNER_CHUNKSIZE, --scanner-chunksize SCANNER_CHUNKSIZE
Number of files to scan per thread/process.
-sto SCANNER_TIMEOUT, --scanner-timeout SCANNER_TIMEOUT
Timeout for scanning in seconds.
```
</details>
---
## Configuring Scanning Rules: (Secret Locators)
A Secret Locator is a specific pattern designed to detect sensitive information within files. These locators help automate the identification of secrets, such as API keys, client IDs, passwords, and other sensitive data that may inadvertently be included in codebases. Here’s a breakdown of how a Secret Locator is structured and how to configure them for your scans:
### Secret Locator Structure
> Example Secret Locator for OpenAI API key in JSON:
```json
{
"id": "openai-api-key",
"name": "OpenAI API Key",
"pattern": "sk-\\w{20}T3BlbkFJ\\w{20}",
"secret_group": 0,
"description": "OpenAI API Key",
"confidence": "high",
"severity": "high",
"tags": [
"OpenAI",
"API Key",
"Secret Key",
"AI"
]
},
```
| Field | Required | Description | Used For |
| --- | --- | --- | --- |
| `id` | Yes | Unique identifier for the locator. | Grouping output by locator. |
| `name` | Yes | Display name for the locator. | Printed when found, upcoming features. |
| `pattern` | Yes | Unique Regex Pattern to search for. | Regex to match when locating secrets. |
| `secret_group` | No | Regex capturing group number to extract. Defaults to 0 (entire match) | Extracting the secret from a match. |
| `description` | No | Description of the secret and it's impact if leaked. | Upcoming features (search). |
| `confidence` | No | How likely a match is to be a true positive. | Upcoming features (search, filter output) |
| `severity` | No | How severe the risk of leaking this secret is. | Upcoming features (search, filter output) |
| `tags` | No | Tags/keywords | Upcoming features (search, filter output) |
### Supported Secret Locator Input Formats
APKscan supports multiple common formats for secret patterns including:
| Format | Filetype(s) | Link to More Patterns | Credit |
| --- | --- | --- | --- |
| `SecretLocator` | `JSON`, `YAML` | [all_secret_locators.json](https://github.com/LucasFaudman/apkscan/blob/main/src/apkscan/secret_locators/all_secret_locators.json) | @lucasfaudman
| `secret-patterns-db` | `YAML` | [Link to DB](https://github.com/mazen160/secrets-patterns-db/tree/master/db) | @zricethezav
| `gitleaks` | `TOML` | [Link to Gitleaks](https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml)| @mazen160 |
| Key-value pairs. | `JSON`, `YAML` | [Link](https://www.google.com/search?q=regexes+for+secrets) | @douglascrockford |
> NOTE: Multiple files in different formats can be provided at once after the `-r/--rules` arg. Duplicate patterns will be removed. Duplicate IDs will be combined in the output.
**Need another format?** Feel free to open an Issue, edit [`def load_secret_locators` in `secret_scanner.py`](https://github.com/LucasFaudman/apkscan/blob/02e47c105f3d0b32d2bd15c94c1bc5df8dcc8ccb/src/apkscan/secret_scanner.py#L144), and/or open a PR.
<details>
<summary>
<h3>Secret Locator Format Examples</h3>
</summary>
#### `SecretLocator` `JSON` or `YAML` format.
```json
[
{
"id": "gcp-api-key",
"name": "GCP API Key",
"pattern": "\\b(AIza[0-9A-Za-z\\\\-_]{35})(?:['|\\\"|\\n|\\r|\\s|\\x60|;]|$)",
"secret_group": 1,
"description": "Google Cloud Platform API key",
"confidence": "high",
"severity": null,
"tags": [
"Google",
"Cloud",
"API Key"
]
},
{
"id": "generic-key",
"name": "Generic Key",
"pattern": "(?i)\\b\\w+(?:secret_?)?(?:api_?)?key[\\s=:]+[\\'\"][\\w/\\-:@.]+[\\'\"]",
"secret_group": 0,
"description": null,
"confidence": "low",
"severity": null,
"tags": []
}
]
```
#### `secret-patterns-db YAML` format.
```yaml
patterns:
- pattern:
name: AWS API Gateway
regex: "[0-9a-z]+.execute-api.[0-9a-z.-_]+.amazonaws.com"
confidence: low
- pattern:
name: AWS ARN
regex: "arn:aws:[a-z0-9-]+:[a-z]{2}-[a-z]+-[0-9]+:[0-9]+:.+"
confidence: low
- pattern:
name: AWS Client ID
regex: "(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}"
confidence: low
```
#### `gitleaks TOML`
```toml
title = "gitleaks config"
[[rules]]
description = "Alibaba AccessKey ID"
id = "alibaba-access-key-id"
regex = '''(?i)\b((LTAI)(?i)[a-z0-9]{20})(?:['|\"|\n|\r|\s|\x60|;]|$)'''
keywords = [
"ltai",
]
[[rules]]
description = "Alibaba Secret Key"
id = "alibaba-secret-key"
regex = '''(?i)(?:alibaba)(?:[0-9a-z\-_\t .]{0,20})(?:[\s|']|[\s|"]){0,3}(?:=|>|:=|\|\|:|<=|=>|:)(?:'|\"|\s|=|\x60){0,5}([a-z0-9]{30})(?:['|\"|\n|\r|\s|\x60|;]|$)'''
secretGroup = 1
keywords = [
"alibaba",
]
[[rules]]
description = "Asana Client ID"
id = "asana-client-id"
regex = '''(?i)(?:asana)(?:[0-9a-z\-_\t .]{0,20})(?:[\s|']|[\s|"]){0,3}(?:=|>|:=|\|\|:|<=|=>|:)(?:'|\"|\s|=|\x60){0,5}([0-9]{16})(?:['|\"|\n|\r|\s|\x60|;]|$)'''
secretGroup = 1
keywords = [
"asana",
]
```
#### Key-value pairs.
```json
{
"OpenAI API Key": "sk-\\w{20}T3BlbkFJ\\w{20}",
"GCP API Key": "\\b(AIza[0-9A-Za-z\\\\-_]{35})(?:['|\\\"|\\n|\\r|\\s|\\x60|;]|$)",
"Generic Key": "(?i)\\b\\w+(?:secret_?)?(?:api_?)?key[\\s=:]+[\\'\"][\\w/\\-:@.]+[\\'\"]"
}
```
</details>
---
## Configuring Decompilers
APKscan supports many popular APK and Java decompiler/disassemblers/deobfuscators increasing the chance of successfully finding secrets.
> NOTE: APKscan uses [`enjarify-adapter`](https://github.com/LucasFaudman/enjarify-adapter) to convert the Dalvik bytecode in `.apk` files into Java bytecode on the fly, so the resulting `.jar` can be processed by decompilers/deobfuscators that do not support `.apks` directly.
### Supported Decompilers
| Tool | Requires Enjarify | Link to Project | Credit |
| --- | --- | --- | --- |
`JADX` | No | [Link](https://github.com/skylot/jadx) | @skylot |
`APKTool` | No | [Link](https://github.com/iBotPeaches/Apktool) | @iBotPeaches |
`CFR` | Yes | [Link](https://github.com/leibnitz27/cfr) | @leibnitz27 |
`Procyon` | Yes | [Link](https://github.com/mstrobel/procyon) | @mstrobel |
`Krakatau` | Yes | [Link](https://github.com/Storyyeller/Krakatau) | @Storyyeller |
`Fernflower` | Yes | [Link](https://github.com/fesh0r/fernflower) | @fernflower |
### Using Multiple Decompilers
Multiple decompilers can be used at once by providing the arguments below. Each optionally accepts a path to the binary of the tool. When no path is provided the binary on the standard path is used. (output of `which jadx`, `which apktool`, etc)
```bash
Decompiler Choices:
Choose which decompiler(s) to use. Optionally specify path to decompiler
binary. Default is JADX.
--jadx [JADX], -J [JADX]
Use JADX Java decompiler.
--apktool [APKTOOL], -A [APKTOOL]
Use APKTool SMALI disassembler.
--cfr [CFR], -C [CFR]
Use CFR Java decompiler. Requires Enjarify.
--procyon [PROCYON], -P [PROCYON]
Use Procyon Java decompiler. Requires Enjarify.
--krakatau [KRAKATAU], -K [KRAKATAU]
Use Krakatau Java decompiler. Requires Enjarify.
--fernflower [FERNFLOWER], -F [FERNFLOWER]
Use Fernflower Java decompiler. Requires Enjarify.
--enjarify-choice {auto,never,always}, -EC {auto,never,always}
When to use Enjarify. Default is 'auto' which means
use only when needed.
--unpack-xapks, --no-unpack-xapks
Unpack XAPK files into APKs before decompiling.
Default is True.
Decompiler Advanced Options:
Options for Java decompiler.
-d, --deobfuscate, --no-deobfuscate
Deobfuscate file before scanning.
-w DECOMPILER_WORKING_DIR, --decompiler-working-dir DECOMPILER_WORKING_DIR
Working directory where files will be decompiled.
--decompiler-output-suffix DECOMPILER_OUTPUT_SUFFIX
Suffix for decompiled output directory names. Default is '-decompiled'.
--decompiler-extra-args DECOMPILER_EXTRA_ARGS [DECOMPILER_EXTRA_ARGS ...]
Additional arguments to pass to decompilers in form quoted whitespace separated '<DECOMPILER_NAME>
<EXTRA_ARGS>...'. For example: --decompiler-extra-args 'jadx --no-debug-info,--no-inline'.
```
**Examples:**
Decompile with both `JADX` and `APKtool`:
```bash
apkscan --jadx --apktool -o "combined-output.json" app-to-scan.apk
```
Decompile with `JADX` located at `"/non/standard/path/jadx"`, `Procyon` and `CFR` binaries in the standard location:
```bash
apkscan --jadx "/non/standard/path/jadx" --cfr --procyon -o "combined-output.json" app-to-scan.apk
```
Decompile multiple APKs with all decompilers and output `YAML`:
```bash
apkscan -J -A -C -P -K -F -o "combined.yaml' -f yaml app-to-scan1.apk app-to-scan2.apk app-to-scan3.xapk
```
Provide extra args to `JADX` and `CFR`:
```bash
apkscan --jadx --cfr --decompiler-extra-args "jadx --add-debug-lines --no-inline-anonymous" "cfr --renamedupmembers true" app-to-scan.apk
```
<details>
<summary>
<h3>Why use Multiple Decompilers?</h3>
</summary>
Using multiple decompilers increases the chance of successfully finding secrets for several reasons:
#### Variety of Decompilation Techniques:
Different decompilers use various algorithms and heuristics to reverse-engineer bytecode back into source code. By leveraging multiple decompilers, you can capture a broader spectrum of decompilation strategies, increasing the likelihood of accurately reconstructing the original source code.
#### Handling Obfuscation:
APKs often use obfuscation techniques to make reverse engineering more difficult. Some decompilers are better at handling specific types of obfuscation than others. By using multiple decompilers, you can overcome a wider range of obfuscation techniques, ensuring more thorough analysis.
#### Completeness of Decompiled Output:
No single decompiler can guarantee perfect output for all APKs. Some decompilers might miss certain parts of the code or fail to decompile specific constructs correctly. Combining the outputs from multiple decompilers helps ensure a more complete and accurate reconstruction of the application.
#### Redundancy and Validation:
Having multiple decompiled versions of the same APK allows for cross-verification. Discrepancies between the outputs can be analyzed to identify potential decompilation errors or areas that need further investigation.
</details>
---
## Concurrency and Performance
APKscan offers a comprehensive set of concurrency and performance options that are configurable in a similar way for both decompilation and secret scanning processes. These options allow you to optimize the speed and efficiency of APKscan based on your system's capabilities and the size of your workload.
### Concurrency Options
Both the decompilation **AND** secret scanning processes can be configured using the following options:
```bash
Decompiler Advanced Options:
Options for Java decompiler.
<truncated>
-dct {thread,process,main}, --decompiler-concurrency-type {thread,process,main}
Type of concurrency to use for decompilation. Default is 'thread'.
-dro {completed,submitted}, --decompiler-results-order {completed,submitted}
Order to process results from decompiler. Default is 'completed'.
-dmw DECOMPILER_MAX_WORKERS, --decompiler-max-workers DECOMPILER_MAX_WORKERS
Maximum number of workers to use for decompilation.
-dcs DECOMPILER_CHUNKSIZE, --decompiler-chunksize DECOMPILER_CHUNKSIZE
Number of files to decompile per thread/process.
-dto DECOMPILER_TIMEOUT, --decompiler-timeout DECOMPILER_TIMEOUT
Timeout for decompilation in seconds.
Secret Scanner Advanced Options:
Options for secret scanner.
-sct {thread,process,main}, --scanner-concurrency-type {thread,process,main}
Type of concurrency to use for scanning. Default is 'process'.
-sro {completed,submitted}, --scanner-results-order {completed,submitted}
Order to process results from scanner. Default is 'completed'.
-smw SCANNER_MAX_WORKERS, --scanner-max-workers SCANNER_MAX_WORKERS
Maximum number of workers to use for scanning.
-scs SCANNER_CHUNKSIZE, --scanner-chunksize SCANNER_CHUNKSIZE
Number of files to scan per thread/process.
-sto SCANNER_TIMEOUT, --scanner-timeout SCANNER_TIMEOUT
Timeout for scanning in seconds.
```
#### Concurrency Type:
> Specify the type of concurrency to use with `{thread, process, main}`.
- `thread`: Uses threading, suitable for I/O-bound tasks.
- `process`: Uses multiprocessing, more efficient for CPU-bound tasks.
- `main`: Runs in the main thread, useful for debugging or environments where concurrency is restricted.
#### Results Order:
> Control the order in which results are processed with `{completed, submitted}`.
- `completed`: Processes results as soon as they are completed.
- `submitted`: Processes results in the order they were submitted.
#### Max Workers:
> Set the maximum number of workers (threads or processes) to use.
- Adjust based on your system's CPU and memory resources.
#### Chunk Size (Multiprocessing and order submitted ONLY):
> Define the number of files to submit for processing process.
- This helps balance the workload and can improve performance.
#### Timeout:
> Set a timeout for each thread/process in seconds.
- This ensures that stalled tasks do not indefinitely block the overall process.
### Optimizing Performance
To optimize the performance of APKscan, consider the following tips:
1. Decompilation is memory-intensive: Set the maximum number of decompiler workers based on your available RAM to avoid system slowdowns or crashes.
2. Balance workload between decompilation and scanning: Consider whether your workload is more focused on decompiling or scanning.
- If using a large number of decompilers and scanning for few secret locators, allocate more workers to decompilation and fewer to scanning.
- Conversely, if using fewer decompilers but scanning for many secret locators, allocate more workers to the scanning process.
By fine-tuning these concurrency and performance options, you can make the most of APKscan's capabilities, ensuring efficient and effective secret detection across large and diverse sets of files.
---
## Contributing
Contributions welcome! Whether you're interested in fixing bugs, adding new features, improving documentation, or sharing ideas, any input is valuable.
### How to Contribute
1. **Fork the Repository**: Start by forking the repository on GitHub. This will create a copy of the project in your own GitHub account.
2. **Clone the Repository**: Clone the forked repository to your local machine.
```bash
git clone https://github.com/LucasFaudman/apkscan.git
cd apkscan
```
3. **Create a Branch**: Create a new branch for your changes.
```bash
git checkout -b my-feature-branch
```
4. **Make Changes**: Make your changes in the code, documentation, or both.
5. **Commit Changes**: Commit your changes with a descriptive commit message.
```bash
git add .
git commit -m "Description of the changes"
```
6. **Push Changes**: Push your changes to your forked repository.
```bash
git push origin my-feature-branch
```
7. **Create a Pull Request**: Go to the original repository and create a pull request from your branch. Provide a detailed description of your changes and any relevant information.
### Issues
If you encounter any bugs, have suggestions, or need help, please open an issue on GitHub. Make sure to provide as much detail as possible, including steps to reproduce the issue, error messages, and screenshots if applicable.
---
## License
> See [LICENSE](https://github.com/LucasFaudman/apkscan/blob/main/LICENSE) for details.
Raw data
{
"_id": null,
"home_page": "https://github.com/LucasFaudman/apkscan.git",
"name": "apkscan",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "secret scanner, jadx, decompile, android, java, security, mobile, decompiler, concurrency, penetration-testing, apktool, security-tools, fernflower, cfr, procyon, enjarify, krakatau, secret-scanning, decompiler-java, secret-scanner, apk, xapk, dex, jar, class, smali, zip, aar, arsc, aab, jadx.kts",
"author": "Lucas Faudman",
"author_email": "Lucas Faudman <lucasfaudman@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/ca/08/69206bf4841ee9f0ec94f737f9c36a98e23702fe28d22999153bac45d62e/apkscan-0.4.0.tar.gz",
"platform": null,
"description": "# APKscan\n\n\n\n[](https://badge.fury.io/py/apkscan)\n[](https://github.com/LucasFaudman/apkscan/issues)\n[](LICENSE)\n\nScan for **secrets**, **endpoints**, and other **sensitive data** after **decompiling** and **deobfuscating** Android\nfiles. (.apk, .xapk, .dex, .jar, .class, .smali, .zip, .aar, .arsc, .aab, .jadx.kts).\n\n- [Why use APKscan?](#why-use-apkscan)\n- [Features](#features)\n- [Installation](#installation)\n - [PyPi Install Command](#pypi-install-command)\n - [From Source Install Commands](#from-source-install-commands)\n- [Usage](#usage)\n - [Basic Usage](#basic-usage)\n - [Advanced Usage (All Command Line Arguments)](#advanced-usage)\n- [Configuring Scanning Rules](#configuring-scanning-rules-secret-locators)\n - [Secret Locator Structure](#secret-locator-structure)\n - [Supported Secret Locator Input Formats](#supported-secret-locator-input-formats)\n - [Secret Locator Input Examples](#secretlocator-json-or-yaml-format)\n- [Configuring Decompilers](#configuring-decompilers)\n - [Supported Decompilers](#supported-decompilers)\n - [Using Multiple Decompilers](#using-multiple-decompilers)\n - [Why use Multiple Decompilers?](#variety-of-decompilation-techniques)\n- [Concurrency and Performance](#concurrency-and-performance)\n - [Concurrency Options](#concurrency-options)\n - [Concurrency Type](#concurrency-type)\n - [Results Order](#results-order)\n - [Max Workers](#max-workers)\n - [Chunksize](#chunk-size-multiprocessing-and-order-submitted-only)\n - [Timeout](#timeout)\n - [Optimizing Performance](#optimizing-performance)\n- [Contributing](#contributing)\n- [License](#license)\n\n---\n\n## Why use APKscan?\n\n### Find Leaked Secrets\nAPKs (Android Package Kits) often leak secrets due to over-reliance on *security through obscurity*. Developers sometimes leave **sensitive information** such as **API keys**, **tokens**, and **credentials** hidden within the code, assuming that they won't be found easily since the code has been compiled and obfuscated. However, this approach is fundamentally flawed, and such secrets can be exposed, leading to potential **security vulnerabilities**.\n\n### Identify Sensitive Locations in Application Code\nAPKscan can help quickly identify sensitive locations in the code, such as SSL pinning libraries, root detection functions, and other security mechanisms. Identifying these functions can speed up reverse engineering and app manipulation by **quickly revealing critical points where an app enforces its security policies, making it easier to bypass them with tools like Frida.** By pinpointing these areas, APKscan aids in understanding an app's security mechanisms and potential weaknesses.\n\n### Identify the Attack Surface of the Backend\nAPKscan also helps identify the attack surface of the backend by uncovering **forgotten endpoints**, **test data payloads**, and other **traces of backend interfaces** that developers might have **unintentionally exposed**. These endpoints can provide attackers with access to sensitive data or functionalities that are not meant for public use. By scanning for such endpoints and test data, APKscan assists in ensuring that the backend is secure and that **no unnecessary exposure is left in the deployed applications**.\n\n\n## Features\n\n### Automate the Scanning Process for Multiple Applications:\n> APKscan allows you to automate the process of scanning for secrets in **any number of applications**, saving you time and ensuring thorough coverage.\n\n### Multiple Decompilers and Deobfuscators:\n> Utilize one or more decompilers and deobfuscators to increase the chances of finding hidden secrets.\n- Supports all popular decompilers including `JADX`, `APKTool`, `CFR`, `Procyon`, `Krakatau`, and `Fernflower`, providing flexibility and robustness in your scanning process.\n- Uses [`enjarify-adapter`](https://github.com/LucasFaudman/enjarify-adapter) to convert the Dalvik bytecode in `.apk` files into Java bytecode on the fly, so the resulting `.jar` can be processed by decompilers/deobfuscators that do not support `.apks` directly.\n\n### Customizable Rules:\n> Define your own secret locator rules or use the default ones provided. This flexibility allows you to tailor the scanning process to your specific needs and improve the detection accuracy of sensitive information.\n- Support for common formats: `SecretLocator JSON`, `secret-patterns-db YAML`, `gitleaks TOML`, and simple key-value pairs.\n\n### Flexible Output Options:\n> Choose from multiple output formats (`JSON`, `YAML`, or `text`) and organize the results by input file or locator. This makes it easier to integrate with other tools and workflows, and to analyze the findings effectively.\n\n### Comprehensive File Support:\n> Decompile and scan a wide range of Android-related files, including `.apk,` `.xapk,` `.dex,` `.jar,` `.class,` `.smali,` `.zip,` `.aar,` `.arsc,` `.aab,` and `.jadx.kts` files.\n- **NEW**: `.xapk` -> `.apk`(s) unpacking/extraction support added in v0.4.0.\n\n### Advanced Configuration and Concurrency Options:\n> APKscan offers advanced options for concurrency, decompilation, and scanning, enabling you to optimize the performance and behavior of the tool to suit your environment and requirements.\n\n---\n\n## Installation\n> APKscan can be installed from PyPi or from source.\n\n### PyPi Install Command\n```bash\npip3 install apkscan\n```\n\n### From Source Install Commands\n```bash\ngit clone https://github.com/LucasFaudman/apkscan.git\ncd apkscan\npython3 -m venv .venv\nsource .venv/bin/activate\npip3 install -e .\ncd ../\n```\n\n---\n\n\n## Usage\n\n### Basic Usage\nThe most basic way to use APKscan is to decompile an APK using the default decompiler `JADX` and scan using the default Secret locator rules in [`default.json`](https://github.com/LucasFaudman/apkscan/blob/main/src/apkscan/secret_locators/default.json).\n```bash\napkscan file-to-scan.apk\n```\n---\nMultiple sets of Secret Locators are included and can be refrenced by name. For example, to scan for only AWS credentials and endpoints:\n```bash\napkscan file-to-scan.apk -r aws endpoints\n```\n---\n\n\nA slighly more complex example. This time 3 APKs will be decompiled then scanned using the custom rules at `/path/to/custom/rules.json`. The output written to `output_file.yaml` in `YAML` format, and the results will be grouped by which secret locator was matched. Files generated during decompilation will be removed after scanning.\n```bash\napkscan -r /path/to/custom/rules.json -o output_file.yaml -f yaml -g locator -c file1.apk file2.apk file3.apk\n```\nOr in long form:\n```bash\napkscan --rules /path/to/custom/rules.json --output output_file.yaml --format yaml --groupby locator --cleanup file1.apk file2.apk file3.apk\n```\n---\n\n\n### Advanced Usage\n\n<details>\n<summary>\n<h4>All Command Line Arguments</h4>\n</summary>\n\n```bash\nusage: apkscan [-h] [-r [SECRET_LOCATOR_FILES ...]] [-o SECRETS_OUTPUT_FILE]\n [-f {text,json,yaml}] [-g {file,locator,both}]\n [-c | --cleanup | --no-cleanup] [-q] [--jadx [JADX]]\n [--apktool [APKTOOL]] [--cfr [CFR]] [--procyon [PROCYON]]\n [--krakatau [KRAKATAU]] [--fernflower [FERNFLOWER]]\n [--enjarify-choice {auto,never,always}]\n [--unpack-xapks | --no-unpack-xapks]\n [-d | --deobfuscate | --no-deobfuscate]\n [-w DECOMPILER_WORKING_DIR]\n [--decompiler-output-suffix DECOMPILER_OUTPUT_SUFFIX]\n [--decompiler-extra-args DECOMPILER_EXTRA_ARGS [DECOMPILER_EXTRA_ARGS ...]]\n [-dct {thread,process,main}] [-dro {completed,submitted}]\n [-dmw DECOMPILER_MAX_WORKERS] [-dcs DECOMPILER_CHUNKSIZE]\n [-dto DECOMPILER_TIMEOUT] [-sct {thread,process,main}]\n [-sro {completed,submitted}] [-smw SCANNER_MAX_WORKERS]\n [-scs SCANNER_CHUNKSIZE] [-sto SCANNER_TIMEOUT]\n [FILES_TO_SCAN ...]\n\nAPKscan v0.4.0 - Scan for secrets, endpoints, and other sensitive\ndata after decompiling and deobfuscating Android files. (.apk,\n.xapk, .dex, .jar, .class, .smali, .zip, .aar, .arsc, .aab, .jadx.kts)\n(c) Lucas Faudman, 2024. License information in LICENSE file. Credits\nto the original authors of all dependencies used in this project.\n\noptions:\n -h, --help show this help message and exit\n\nInput Options:\n FILES_TO_SCAN Path(s) to Java files to decompile and scan.\n -r [SECRET_LOCATOR_FILES ...], --rules [SECRET_LOCATOR_FILES ...]\n Path(s) to secret locator rules/patterns files OR\n names of included locator sets. Files can be in\n SecretLocator JSON, secret-patterns-db YAML, or\n Gitleak TOML formats. Included locator sets:\n all_secret_locators, aws, azure, cloud, curated,\n default, endpoints, gcp, generic, gitleaks, high-\n confidence, key_locators, leakin-regexes,\n locator_sort, nuclei-regexes, secret. If not provided,\n default rules will be used. See: apkscan/src/apkscan/secret_l\n ocators/default.json\n\nOutput Options:\n -o SECRETS_OUTPUT_FILE, --output SECRETS_OUTPUT_FILE\n Output file for secrets found.\n -f {text,json,yaml}, --format {text,json,yaml}\n Output format for secrets found.\n -g {file,locator,both}, --groupby {file,locator,both}\n Group secrets by input file or locator. Default is\n 'both'.\n -c, --cleanup, --no-cleanup\n Remove decompiled output directories after scanning.\n -q, --quiet Suppress output from subprocesses.\n\nDecompiler Choices:\n Choose which decompiler(s) to use. Optionally specify path to decompiler\n binary. Default is JADX.\n\n --jadx [JADX], -J [JADX]\n Use JADX Java decompiler.\n --apktool [APKTOOL], -A [APKTOOL]\n Use APKTool SMALI disassembler.\n --cfr [CFR], -C [CFR]\n Use CFR Java decompiler. Requires Enjarify.\n --procyon [PROCYON], -P [PROCYON]\n Use Procyon Java decompiler. Requires Enjarify.\n --krakatau [KRAKATAU], -K [KRAKATAU]\n Use Krakatau Java decompiler. Requires Enjarify.\n --fernflower [FERNFLOWER], -F [FERNFLOWER]\n Use Fernflower Java decompiler. Requires Enjarify.\n --enjarify-choice {auto,never,always}, -EC {auto,never,always}\n When to use Enjarify. Default is 'auto' which means\n use only when needed.\n --unpack-xapks, --no-unpack-xapks\n Unpack XAPK files into APKs before decompiling.\n Default is True.\n\nDecompiler Advanced Options:\n Options for Java decompiler.\n\n -d, --deobfuscate, --no-deobfuscate\n Deobfuscate file before scanning.\n -w DECOMPILER_WORKING_DIR, --decompiler-working-dir DECOMPILER_WORKING_DIR\n Working directory where files will be decompiled.\n --decompiler-output-suffix DECOMPILER_OUTPUT_SUFFIX\n Suffix for decompiled output directory names. Default\n is '-decompiled'.\n --decompiler-extra-args DECOMPILER_EXTRA_ARGS [DECOMPILER_EXTRA_ARGS ...]\n Additional arguments to pass to decompilers in form\n quoted whitespace separated '<DECOMPILER_NAME>\n <EXTRA_ARGS>...'. For example: --decompiler-extra-args\n 'jadx --no-debug-info,--no-inline'.\n -dct {thread,process,main}, --decompiler-concurrency-type {thread,process,main}\n Type of concurrency to use for decompilation. Default\n is 'thread'.\n -dro {completed,submitted}, --decompiler-results-order {completed,submitted}\n Order to process results from decompiler. Default is\n 'completed'.\n -dmw DECOMPILER_MAX_WORKERS, --decompiler-max-workers DECOMPILER_MAX_WORKERS\n Maximum number of workers to use for decompilation.\n -dcs DECOMPILER_CHUNKSIZE, --decompiler-chunksize DECOMPILER_CHUNKSIZE\n Number of files to decompile per thread/process.\n -dto DECOMPILER_TIMEOUT, --decompiler-timeout DECOMPILER_TIMEOUT\n Timeout for decompilation in seconds.\n\nSecret Scanner Advanced Options:\n Options for secret scanner.\n\n -sct {thread,process,main}, --scanner-concurrency-type {thread,process,main}\n Type of concurrency to use for scanning. Default is\n 'process'.\n -sro {completed,submitted}, --scanner-results-order {completed,submitted}\n Order to process results from scanner. Default is\n 'completed'.\n -smw SCANNER_MAX_WORKERS, --scanner-max-workers SCANNER_MAX_WORKERS\n Maximum number of workers to use for scanning.\n -scs SCANNER_CHUNKSIZE, --scanner-chunksize SCANNER_CHUNKSIZE\n Number of files to scan per thread/process.\n -sto SCANNER_TIMEOUT, --scanner-timeout SCANNER_TIMEOUT\n Timeout for scanning in seconds.\n\n\n```\n</details>\n\n---\n\n\n## Configuring Scanning Rules: (Secret Locators)\nA Secret Locator is a specific pattern designed to detect sensitive information within files. These locators help automate the identification of secrets, such as API keys, client IDs, passwords, and other sensitive data that may inadvertently be included in codebases. Here\u2019s a breakdown of how a Secret Locator is structured and how to configure them for your scans:\n\n### Secret Locator Structure\n> Example Secret Locator for OpenAI API key in JSON:\n```json\n {\n \"id\": \"openai-api-key\",\n \"name\": \"OpenAI API Key\",\n \"pattern\": \"sk-\\\\w{20}T3BlbkFJ\\\\w{20}\",\n \"secret_group\": 0,\n \"description\": \"OpenAI API Key\",\n \"confidence\": \"high\",\n \"severity\": \"high\",\n \"tags\": [\n \"OpenAI\",\n \"API Key\",\n \"Secret Key\",\n \"AI\"\n ]\n },\n```\n\n| Field | Required | Description | Used For |\n| --- | --- | --- | --- |\n| `id` | Yes | Unique identifier for the locator. | Grouping output by locator. |\n| `name` | Yes | Display name for the locator. | Printed when found, upcoming features. |\n| `pattern` | Yes | Unique Regex Pattern to search for. | Regex to match when locating secrets. |\n| `secret_group` | No | Regex capturing group number to extract. Defaults to 0 (entire match) | Extracting the secret from a match. |\n| `description` | No | Description of the secret and it's impact if leaked. | Upcoming features (search). |\n| `confidence` | No | How likely a match is to be a true positive. | Upcoming features (search, filter output) |\n| `severity` | No | How severe the risk of leaking this secret is. | Upcoming features (search, filter output) |\n| `tags` | No | Tags/keywords | Upcoming features (search, filter output) |\n\n### Supported Secret Locator Input Formats\nAPKscan supports multiple common formats for secret patterns including:\n| Format | Filetype(s) | Link to More Patterns | Credit |\n| --- | --- | --- | --- |\n| `SecretLocator` | `JSON`, `YAML` | [all_secret_locators.json](https://github.com/LucasFaudman/apkscan/blob/main/src/apkscan/secret_locators/all_secret_locators.json) | @lucasfaudman\n| `secret-patterns-db` | `YAML` | [Link to DB](https://github.com/mazen160/secrets-patterns-db/tree/master/db) | @zricethezav\n| `gitleaks` | `TOML` | [Link to Gitleaks](https://github.com/gitleaks/gitleaks/blob/master/config/gitleaks.toml)| @mazen160 |\n| Key-value pairs. | `JSON`, `YAML` | [Link](https://www.google.com/search?q=regexes+for+secrets) | @douglascrockford |\n\n> NOTE: Multiple files in different formats can be provided at once after the `-r/--rules` arg. Duplicate patterns will be removed. Duplicate IDs will be combined in the output.\n\n**Need another format?** Feel free to open an Issue, edit [`def load_secret_locators` in `secret_scanner.py`](https://github.com/LucasFaudman/apkscan/blob/02e47c105f3d0b32d2bd15c94c1bc5df8dcc8ccb/src/apkscan/secret_scanner.py#L144), and/or open a PR.\n\n\n<details>\n<summary>\n<h3>Secret Locator Format Examples</h3>\n</summary>\n\n#### `SecretLocator` `JSON` or `YAML` format.\n```json\n[\n {\n \"id\": \"gcp-api-key\",\n \"name\": \"GCP API Key\",\n \"pattern\": \"\\\\b(AIza[0-9A-Za-z\\\\\\\\-_]{35})(?:['|\\\\\\\"|\\\\n|\\\\r|\\\\s|\\\\x60|;]|$)\",\n \"secret_group\": 1,\n \"description\": \"Google Cloud Platform API key\",\n \"confidence\": \"high\",\n \"severity\": null,\n \"tags\": [\n \"Google\",\n \"Cloud\",\n \"API Key\"\n ]\n },\n {\n \"id\": \"generic-key\",\n \"name\": \"Generic Key\",\n \"pattern\": \"(?i)\\\\b\\\\w+(?:secret_?)?(?:api_?)?key[\\\\s=:]+[\\\\'\\\"][\\\\w/\\\\-:@.]+[\\\\'\\\"]\",\n \"secret_group\": 0,\n \"description\": null,\n \"confidence\": \"low\",\n \"severity\": null,\n \"tags\": []\n }\n]\n```\n\n#### `secret-patterns-db YAML` format.\n```yaml\npatterns:\n - pattern:\n name: AWS API Gateway\n regex: \"[0-9a-z]+.execute-api.[0-9a-z.-_]+.amazonaws.com\"\n confidence: low\n - pattern:\n name: AWS ARN\n regex: \"arn:aws:[a-z0-9-]+:[a-z]{2}-[a-z]+-[0-9]+:[0-9]+:.+\"\n confidence: low\n - pattern:\n name: AWS Client ID\n regex: \"(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}\"\n confidence: low\n```\n\n#### `gitleaks TOML`\n```toml\ntitle = \"gitleaks config\"\n[[rules]]\ndescription = \"Alibaba AccessKey ID\"\nid = \"alibaba-access-key-id\"\nregex = '''(?i)\\b((LTAI)(?i)[a-z0-9]{20})(?:['|\\\"|\\n|\\r|\\s|\\x60|;]|$)'''\nkeywords = [\n \"ltai\",\n]\n\n[[rules]]\ndescription = \"Alibaba Secret Key\"\nid = \"alibaba-secret-key\"\nregex = '''(?i)(?:alibaba)(?:[0-9a-z\\-_\\t .]{0,20})(?:[\\s|']|[\\s|\"]){0,3}(?:=|>|:=|\\|\\|:|<=|=>|:)(?:'|\\\"|\\s|=|\\x60){0,5}([a-z0-9]{30})(?:['|\\\"|\\n|\\r|\\s|\\x60|;]|$)'''\nsecretGroup = 1\nkeywords = [\n \"alibaba\",\n]\n\n[[rules]]\ndescription = \"Asana Client ID\"\nid = \"asana-client-id\"\nregex = '''(?i)(?:asana)(?:[0-9a-z\\-_\\t .]{0,20})(?:[\\s|']|[\\s|\"]){0,3}(?:=|>|:=|\\|\\|:|<=|=>|:)(?:'|\\\"|\\s|=|\\x60){0,5}([0-9]{16})(?:['|\\\"|\\n|\\r|\\s|\\x60|;]|$)'''\nsecretGroup = 1\nkeywords = [\n \"asana\",\n]\n```\n\n#### Key-value pairs.\n```json\n{\n \"OpenAI API Key\": \"sk-\\\\w{20}T3BlbkFJ\\\\w{20}\",\n \"GCP API Key\": \"\\\\b(AIza[0-9A-Za-z\\\\\\\\-_]{35})(?:['|\\\\\\\"|\\\\n|\\\\r|\\\\s|\\\\x60|;]|$)\",\n \"Generic Key\": \"(?i)\\\\b\\\\w+(?:secret_?)?(?:api_?)?key[\\\\s=:]+[\\\\'\\\"][\\\\w/\\\\-:@.]+[\\\\'\\\"]\"\n}\n```\n\n</details>\n\n---\n\n\n## Configuring Decompilers\nAPKscan supports many popular APK and Java decompiler/disassemblers/deobfuscators increasing the chance of successfully finding secrets.\n\n> NOTE: APKscan uses [`enjarify-adapter`](https://github.com/LucasFaudman/enjarify-adapter) to convert the Dalvik bytecode in `.apk` files into Java bytecode on the fly, so the resulting `.jar` can be processed by decompilers/deobfuscators that do not support `.apks` directly.\n\n\n### Supported Decompilers\n\n| Tool | Requires Enjarify | Link to Project | Credit |\n| --- | --- | --- | --- |\n`JADX` | No | [Link](https://github.com/skylot/jadx) | @skylot |\n`APKTool` | No | [Link](https://github.com/iBotPeaches/Apktool) | @iBotPeaches |\n`CFR` | Yes | [Link](https://github.com/leibnitz27/cfr) | @leibnitz27 |\n`Procyon` | Yes | [Link](https://github.com/mstrobel/procyon) | @mstrobel |\n`Krakatau` | Yes | [Link](https://github.com/Storyyeller/Krakatau) | @Storyyeller |\n`Fernflower` | Yes | [Link](https://github.com/fesh0r/fernflower) | @fernflower |\n\n### Using Multiple Decompilers\nMultiple decompilers can be used at once by providing the arguments below. Each optionally accepts a path to the binary of the tool. When no path is provided the binary on the standard path is used. (output of `which jadx`, `which apktool`, etc)\n\n```bash\nDecompiler Choices:\n Choose which decompiler(s) to use. Optionally specify path to decompiler\n binary. Default is JADX.\n\n --jadx [JADX], -J [JADX]\n Use JADX Java decompiler.\n --apktool [APKTOOL], -A [APKTOOL]\n Use APKTool SMALI disassembler.\n --cfr [CFR], -C [CFR]\n Use CFR Java decompiler. Requires Enjarify.\n --procyon [PROCYON], -P [PROCYON]\n Use Procyon Java decompiler. Requires Enjarify.\n --krakatau [KRAKATAU], -K [KRAKATAU]\n Use Krakatau Java decompiler. Requires Enjarify.\n --fernflower [FERNFLOWER], -F [FERNFLOWER]\n Use Fernflower Java decompiler. Requires Enjarify.\n --enjarify-choice {auto,never,always}, -EC {auto,never,always}\n When to use Enjarify. Default is 'auto' which means\n use only when needed.\n --unpack-xapks, --no-unpack-xapks\n Unpack XAPK files into APKs before decompiling.\n Default is True.\n\nDecompiler Advanced Options:\n Options for Java decompiler.\n\n -d, --deobfuscate, --no-deobfuscate\n Deobfuscate file before scanning.\n -w DECOMPILER_WORKING_DIR, --decompiler-working-dir DECOMPILER_WORKING_DIR\n Working directory where files will be decompiled.\n --decompiler-output-suffix DECOMPILER_OUTPUT_SUFFIX\n Suffix for decompiled output directory names. Default is '-decompiled'.\n --decompiler-extra-args DECOMPILER_EXTRA_ARGS [DECOMPILER_EXTRA_ARGS ...]\n Additional arguments to pass to decompilers in form quoted whitespace separated '<DECOMPILER_NAME>\n <EXTRA_ARGS>...'. For example: --decompiler-extra-args 'jadx --no-debug-info,--no-inline'.\n```\n\n**Examples:**\n\nDecompile with both `JADX` and `APKtool`:\n```bash\napkscan --jadx --apktool -o \"combined-output.json\" app-to-scan.apk\n```\nDecompile with `JADX` located at `\"/non/standard/path/jadx\"`, `Procyon` and `CFR` binaries in the standard location:\n```bash\napkscan --jadx \"/non/standard/path/jadx\" --cfr --procyon -o \"combined-output.json\" app-to-scan.apk\n```\nDecompile multiple APKs with all decompilers and output `YAML`:\n```bash\napkscan -J -A -C -P -K -F -o \"combined.yaml' -f yaml app-to-scan1.apk app-to-scan2.apk app-to-scan3.xapk\n```\nProvide extra args to `JADX` and `CFR`:\n```bash\napkscan --jadx --cfr --decompiler-extra-args \"jadx --add-debug-lines --no-inline-anonymous\" \"cfr --renamedupmembers true\" app-to-scan.apk\n```\n\n<details>\n<summary>\n<h3>Why use Multiple Decompilers?</h3>\n</summary>\n\nUsing multiple decompilers increases the chance of successfully finding secrets for several reasons:\n\n#### Variety of Decompilation Techniques:\nDifferent decompilers use various algorithms and heuristics to reverse-engineer bytecode back into source code. By leveraging multiple decompilers, you can capture a broader spectrum of decompilation strategies, increasing the likelihood of accurately reconstructing the original source code.\n\n#### Handling Obfuscation:\nAPKs often use obfuscation techniques to make reverse engineering more difficult. Some decompilers are better at handling specific types of obfuscation than others. By using multiple decompilers, you can overcome a wider range of obfuscation techniques, ensuring more thorough analysis.\n\n#### Completeness of Decompiled Output:\nNo single decompiler can guarantee perfect output for all APKs. Some decompilers might miss certain parts of the code or fail to decompile specific constructs correctly. Combining the outputs from multiple decompilers helps ensure a more complete and accurate reconstruction of the application.\n\n#### Redundancy and Validation:\nHaving multiple decompiled versions of the same APK allows for cross-verification. Discrepancies between the outputs can be analyzed to identify potential decompilation errors or areas that need further investigation.\n\n</details>\n\n---\n\n## Concurrency and Performance\nAPKscan offers a comprehensive set of concurrency and performance options that are configurable in a similar way for both decompilation and secret scanning processes. These options allow you to optimize the speed and efficiency of APKscan based on your system's capabilities and the size of your workload.\n\n### Concurrency Options\nBoth the decompilation **AND** secret scanning processes can be configured using the following options:\n\n```bash\nDecompiler Advanced Options:\n Options for Java decompiler.\n <truncated>\n -dct {thread,process,main}, --decompiler-concurrency-type {thread,process,main}\n Type of concurrency to use for decompilation. Default is 'thread'.\n -dro {completed,submitted}, --decompiler-results-order {completed,submitted}\n Order to process results from decompiler. Default is 'completed'.\n -dmw DECOMPILER_MAX_WORKERS, --decompiler-max-workers DECOMPILER_MAX_WORKERS\n Maximum number of workers to use for decompilation.\n -dcs DECOMPILER_CHUNKSIZE, --decompiler-chunksize DECOMPILER_CHUNKSIZE\n Number of files to decompile per thread/process.\n -dto DECOMPILER_TIMEOUT, --decompiler-timeout DECOMPILER_TIMEOUT\n Timeout for decompilation in seconds.\n\nSecret Scanner Advanced Options:\n Options for secret scanner.\n\n -sct {thread,process,main}, --scanner-concurrency-type {thread,process,main}\n Type of concurrency to use for scanning. Default is 'process'.\n -sro {completed,submitted}, --scanner-results-order {completed,submitted}\n Order to process results from scanner. Default is 'completed'.\n -smw SCANNER_MAX_WORKERS, --scanner-max-workers SCANNER_MAX_WORKERS\n Maximum number of workers to use for scanning.\n -scs SCANNER_CHUNKSIZE, --scanner-chunksize SCANNER_CHUNKSIZE\n Number of files to scan per thread/process.\n -sto SCANNER_TIMEOUT, --scanner-timeout SCANNER_TIMEOUT\n Timeout for scanning in seconds.\n\n```\n\n#### Concurrency Type:\n> Specify the type of concurrency to use with `{thread, process, main}`.\n- `thread`: Uses threading, suitable for I/O-bound tasks.\n- `process`: Uses multiprocessing, more efficient for CPU-bound tasks.\n- `main`: Runs in the main thread, useful for debugging or environments where concurrency is restricted.\n\n#### Results Order:\n> Control the order in which results are processed with `{completed, submitted}`.\n- `completed`: Processes results as soon as they are completed.\n- `submitted`: Processes results in the order they were submitted.\n\n#### Max Workers:\n> Set the maximum number of workers (threads or processes) to use.\n- Adjust based on your system's CPU and memory resources.\n\n#### Chunk Size (Multiprocessing and order submitted ONLY):\n> Define the number of files to submit for processing process.\n- This helps balance the workload and can improve performance.\n\n#### Timeout:\n> Set a timeout for each thread/process in seconds.\n- This ensures that stalled tasks do not indefinitely block the overall process.\n\n\n### Optimizing Performance\nTo optimize the performance of APKscan, consider the following tips:\n\n1. Decompilation is memory-intensive: Set the maximum number of decompiler workers based on your available RAM to avoid system slowdowns or crashes.\n2. Balance workload between decompilation and scanning: Consider whether your workload is more focused on decompiling or scanning.\n - If using a large number of decompilers and scanning for few secret locators, allocate more workers to decompilation and fewer to scanning.\n - Conversely, if using fewer decompilers but scanning for many secret locators, allocate more workers to the scanning process.\n\nBy fine-tuning these concurrency and performance options, you can make the most of APKscan's capabilities, ensuring efficient and effective secret detection across large and diverse sets of files.\n\n\n---\n\n## Contributing\n\nContributions welcome! Whether you're interested in fixing bugs, adding new features, improving documentation, or sharing ideas, any input is valuable.\n\n### How to Contribute\n\n1. **Fork the Repository**: Start by forking the repository on GitHub. This will create a copy of the project in your own GitHub account.\n\n2. **Clone the Repository**: Clone the forked repository to your local machine.\n\n ```bash\n git clone https://github.com/LucasFaudman/apkscan.git\n cd apkscan\n ```\n\n3. **Create a Branch**: Create a new branch for your changes.\n\n ```bash\n git checkout -b my-feature-branch\n ```\n\n4. **Make Changes**: Make your changes in the code, documentation, or both.\n\n5. **Commit Changes**: Commit your changes with a descriptive commit message.\n\n ```bash\n git add .\n git commit -m \"Description of the changes\"\n ```\n\n6. **Push Changes**: Push your changes to your forked repository.\n\n ```bash\n git push origin my-feature-branch\n ```\n\n7. **Create a Pull Request**: Go to the original repository and create a pull request from your branch. Provide a detailed description of your changes and any relevant information.\n\n### Issues\n\nIf you encounter any bugs, have suggestions, or need help, please open an issue on GitHub. Make sure to provide as much detail as possible, including steps to reproduce the issue, error messages, and screenshots if applicable.\n\n\n---\n\n\n## License\n> See [LICENSE](https://github.com/LucasFaudman/apkscan/blob/main/LICENSE) for details.\n",
"bugtrack_url": null,
"license": "MIT License Copyright (c) 2024 Lucas Faudman Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. --- Commercial License Terms Copyright (c) 2024 Lucas Faudman This software is dual-licensed under the MIT License (see above) and the following terms for commercial use. 1. Commercial Use: If you intend to use this software in a commercial product or service, you must obtain a commercial license from the author. Commercial use includes, but is not limited to, integrating the software into proprietary applications, reselling the software, or offering it as part of a paid service. 2. Obtaining a Commercial License: To obtain a commercial license, please contact LucasFaudman.com. Licensing fees and terms will be negotiated on a case-by-case basis. 3. Support and Warranty: The commercial license may include support and warranty terms that are not available under the MIT License. Details will be provided upon licensing agreement. 4. Modifications and Derivative Works: Under the commercial license, you are allowed to modify and create derivative works of the software. However, these modifications must comply with the terms set out in the commercial license agreement. 5. Liability and Indemnity: The author is not liable for any damages arising from the use of the software under the commercial license. The licensee agrees to indemnify and hold harmless the author from any claims or liabilities resulting from the use of the software in commercial applications. For further inquiries and licensing, please contact: Lucas Faudman LucasFaudman@gmail.com LucasFaudman.com ",
"summary": "Scan for secrets, endpoints, and other sensitive data after decompiling and deobfuscating Android files. (.apk, .xapk, .dex, .jar, .class, .smali, .zip, .aar, .arsc, .aab, .jadx.kts)",
"version": "0.4.0",
"project_urls": {
"Homepage": "https://github.com/LucasFaudman/apkscan.git",
"Repository": "https://github.com/LucasFaudman/apkscan.git"
},
"split_keywords": [
"secret scanner",
" jadx",
" decompile",
" android",
" java",
" security",
" mobile",
" decompiler",
" concurrency",
" penetration-testing",
" apktool",
" security-tools",
" fernflower",
" cfr",
" procyon",
" enjarify",
" krakatau",
" secret-scanning",
" decompiler-java",
" secret-scanner",
" apk",
" xapk",
" dex",
" jar",
" class",
" smali",
" zip",
" aar",
" arsc",
" aab",
" jadx.kts"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f6cbffa89cf0ddd7bd5a49eedb82d9d19be91e5cc3a9afba9f579f3d1dd1d02d",
"md5": "410c435ecd770c0211c04919648bee7b",
"sha256": "a8327fcc8ed534cde5c66da2a86b595d3dcb53bed10d3a670028df5e40299cfe"
},
"downloads": -1,
"filename": "apkscan-0.4.0-cp311-cp311-macosx_14_0_x86_64.whl",
"has_sig": false,
"md5_digest": "410c435ecd770c0211c04919648bee7b",
"packagetype": "bdist_wheel",
"python_version": "cp311",
"requires_python": ">=3.10",
"size": 349731,
"upload_time": "2024-07-18T15:27:04",
"upload_time_iso_8601": "2024-07-18T15:27:04.202478Z",
"url": "https://files.pythonhosted.org/packages/f6/cb/ffa89cf0ddd7bd5a49eedb82d9d19be91e5cc3a9afba9f579f3d1dd1d02d/apkscan-0.4.0-cp311-cp311-macosx_14_0_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ff7e4d95e3150927585bb1cc9c72af8c8c4181fe4d1d60373cf4bd0530a28f8d",
"md5": "f885835f93a4f07f151a63b038f271f9",
"sha256": "d16753afa7b7f60326e6889c859c244e6b32a3fe04183eff7f2d1cb183f5be56"
},
"downloads": -1,
"filename": "apkscan-0.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "f885835f93a4f07f151a63b038f271f9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10",
"size": 141362,
"upload_time": "2024-07-18T15:27:05",
"upload_time_iso_8601": "2024-07-18T15:27:05.891099Z",
"url": "https://files.pythonhosted.org/packages/ff/7e/4d95e3150927585bb1cc9c72af8c8c4181fe4d1d60373cf4bd0530a28f8d/apkscan-0.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ca0869206bf4841ee9f0ec94f737f9c36a98e23702fe28d22999153bac45d62e",
"md5": "96f0def76d4fa7620ad2380745706f7b",
"sha256": "03ccbbcbe99d3d42f2c9dbe11c954f15fa4ec7256ad74a0efdb9fde6b6c7fa21"
},
"downloads": -1,
"filename": "apkscan-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "96f0def76d4fa7620ad2380745706f7b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 143640,
"upload_time": "2024-07-18T15:27:07",
"upload_time_iso_8601": "2024-07-18T15:27:07.441585Z",
"url": "https://files.pythonhosted.org/packages/ca/08/69206bf4841ee9f0ec94f737f9c36a98e23702fe28d22999153bac45d62e/apkscan-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-18 15:27:07",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "LucasFaudman",
"github_project": "apkscan",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "apkscan"
}
