<!-- SPDX-FileCopyrightText: 2024 FC (Fay) Stegerman <flx@obfusk.net> -->
<!-- SPDX-License-Identifier: GPL-3.0-or-later -->
[](https://github.com/obfusk/reproducible-apk-tools/releases)
[](https://pypi.python.org/pypi/repro-apk)
[](https://pypi.python.org/pypi/repro-apk)
[](https://github.com/obfusk/reproducible-apk-tools/actions?query=workflow%3ACI)
[](https://www.gnu.org/licenses/gpl-3.0.html)
<!--
<a href="https://repology.org/project/repro-apk/versions">
<img src="https://repology.org/badge/vertical-allrepos/repro-apk.svg?header="
alt="Packaging status" align="right" />
</a>
<a href="https://repology.org/project/python:repro-apk/versions">
<img src="https://repology.org/badge/vertical-allrepos/python:repro-apk.svg?header="
alt="Packaging status" align="right" />
</a>
-->
# reproducible-apk-tools
[`fix-compresslevel.py`](#fix-compresslevelpy),
[`fix-files.py`](#fix-filespy),
[`fix-newlines.py`](#fix-newlinespy),
[`fix-pg-map-id.py`](#fix-pg-map-idpy),
[`rm-files.py`](#rm-filespy),
[`sort-apk.py`](#sort-apkpy),
[`sort-baseline.py`](#sort-baselinepy),
[`zipalign.py`](#zipalignpy);
[`diff-zip-meta.py`](#diff-zip-metapy),
[`dump-arsc.py`](#dump-arscpy),
[`dump-axml.py`](#dump-axmlpy),
[`dump-baseline.py`](#dump-baselinepy),
[`list-compresslevel.py`](#list-compresslevelpy),
[`zipinfo.py`](#zipinfopy);
[`inplace-fix.py`](#inplace-fixpy).
## scripts to make android apks reproducible
`reproducible-apk-tools` is a collection of scripts (available as subcommands of
the `repro-apk` command) to help make APKs reproducible (e.g. by changing line
endings from LF to CRLF), or find out why they are not (e.g. by comparing ZIP
file metadata, or dumping `baseline.prof` files).
### fix-compresslevel.py
Recompress with different compression level.
Specify which files to change by providing at least one fnmatch-style pattern,
e.g. `'assets/foo/*.bar'`.
If two APKs have identical contents but some ZIP entries are compressed with a
different compression level, thus making the APKs not bit-by-bit identical, this
script may help.
```bash
$ fix-compresslevel.py --help
usage: fix-compresslevel.py [-h] [-v] INPUT_APK OUTPUT_APK COMPRESSLEVEL PATTERN [PATTERN ...]
[...]
$ apksigcopier compare signed.apk --unsigned unsigned.apk
DOES NOT VERIFY
[...]
$ fix-compresslevel.py unsigned.apk fixed.apk 6 assets/foo/bar.js
fixing 'assets/foo/bar.js'...
$ zipalign -f 4 fixed.apk fixed-aligned.apk
$ apksigcopier compare signed.apk --unsigned fixed-aligned.apk && echo OK
OK
```
NB: this builds a new ZIP file, preserving most ZIP metadata (and recompressing
entries not matching the pattern using the same compression level as in the
original APK) but not everything: e.g. copying the existing local header extra
fields which contain padding for alignment is not supported by Python's
`ZipFile`, which is why `zipalign` is usually needed.
### fix-files.py
Process ZIP entries using an external command.
Runs the command for each specified file, providing the old file contents as
stdin and using stdout as the new file contents.
The provided command is split on whitespace to allow passing arguments (e.g.
`'foo --bar'`), but shell syntax is not supported.
Specify which files to process by providing at least one fnmatch-style pattern,
e.g. `'META-INF/services/*'`.
```bash
$ fix-files.py --help
usage: fix-files.py [-h] [-v] INPUT_APK OUTPUT_APK COMMAND PATTERN [PATTERN ...]
[...]
$ apksigcopier compare signed.apk --unsigned unsigned.apk
DOES NOT VERIFY
[...]
$ fix-files.py unsigned.apk fixed.apk unix2dos 'META-INF/services/*'
processing 'META-INF/services/foo' with 'unix2dos'...
processing 'META-INF/services/bar' with 'unix2dos'...
$ zipalign -f 4 fixed.apk fixed-aligned.apk
$ apksigcopier compare signed.apk --unsigned fixed-aligned.apk && echo OK
OK
```
NB: this builds a new ZIP file, preserving most ZIP metadata (and recompressing
using the same compression level) but not everything: e.g. copying the existing
local header extra fields which contain padding for alignment is not supported
by Python's `ZipFile`, which is why `zipalign` is usually needed.
### fix-newlines.py
Change line endings from LF to CRLF (or vice versa w/ `--from-crlf`).
Specify which files to change by providing at least one fnmatch-style pattern,
e.g. `'META-INF/services/*'`.
If the signed APK was built on Windows and has e.g. `META-INF/services/` files
with CRLF line endings whereas the unsigned APK was build on Linux/macOS and has
LF line endings, this script may help.
```bash
$ fix-newlines.py --help
usage: fix-newlines.py [-h] [--from-crlf] [--to-crlf] [-v] INPUT_APK OUTPUT_APK PATTERN [PATTERN ...]
[...]
$ apksigcopier compare signed.apk --unsigned unsigned.apk
DOES NOT VERIFY
[...]
$ fix-newlines.py unsigned.apk fixed.apk 'META-INF/services/*'
fixing 'META-INF/services/foo'...
fixing 'META-INF/services/bar'...
$ zipalign -f 4 fixed.apk fixed-aligned.apk
$ apksigcopier compare signed.apk --unsigned fixed-aligned.apk && echo OK
OK
```
NB: this builds a new ZIP file, preserving most ZIP metadata (and recompressing
using the same compression level) but not everything: e.g. copying the existing
local header extra fields which contain padding for alignment is not supported
by Python's `ZipFile`, which is why `zipalign` is usually needed.
### fix-pg-map-id.py
Replace non-deterministic R8 `pg-map-id` in `classes.dex` (and `classes2.dex`
etc. when present) and update checksums, also in `baseline.prof`.
```bash
$ fix-pg-map-id.py --help
usage: fix-pg-map-id.py [-h] INPUT_DIR_OR_APK OUTPUT_DIR_OR_APK PG_MAP_ID
[...]
$ apksigcopier compare signed.apk --unsigned unsigned.apk
DOES NOT VERIFY
[...]
$ diff -Naur <( unzip -p signed.apk classes.dex | xxd ) <( unzip -p unsigned.apk classes.dex | xxd )
[...]
-00000000: 6465 780a 3033 3500 829f 202e c800 6ecc dex.035... ...n.
-00000010: c15c 0a17 3737 73fc 982e 34db 6239 bc52 .\..77s...4.b9.R
+00000000: 6465 780a 3033 3500 d89f 1795 f719 4d94 dex.035.......M.
+00000010: 8358 2526 2850 f9e5 ad3d e772 c82e 4f02 .X%&(P...=.r..O.
[...]
005f1010: 2d61 7069 223a 3233 2c22 7067 2d6d 6170 -api":23,"pg-map
-005f1020: 2d69 6422 3a22 3261 3939 3764 3322 2c22 -id":"2a997d3","
+005f1020: 2d69 6422 3a22 6565 3436 3531 3322 2c22 -id":"ee46513","
[...]
$ fix-pg-map-id.py unsigned.apk fixed.apk 2a997d3
reading 'assets/dexopt/baseline.prof'...
reading 'classes.dex'...
reading 'classes2.dex'...
fixing 'classes.dex'...
dex version=035
fixing pg-map-id: b'ee46513' -> b'2a997d3'
fixing signature: f7194d94835825262850f9e5ad3de772c82e4f02 -> c8006eccc15c0a17373773fc982e34db6239bc52
fixing checksum: 0x95179fd8 -> 0x2e209f82
fixing 'classes2.dex'...
dex version=035
(not modified)
fixing 'assets/dexopt/baseline.prof'...
prof version=010 P
fixing 'classes.dex' checksum: 0x95d40f72 -> 0x50191ba4
writing 'assets/dexopt/baseline.prof'...
writing 'classes.dex'...
writing 'classes2.dex'...
$ zipalign -f 4 fixed.apk fixed-aligned.apk
$ apksigcopier compare signed.apk --unsigned fixed-aligned.apk && echo OK
OK
```
NB: this builds a new ZIP file, preserving most ZIP metadata (and recompressing
using the same compression level) but not everything: e.g. copying the existing
local header extra fields which contain padding for alignment is not supported
by Python's `ZipFile`, which is why `zipalign` is usually needed.
### rm-files.py
Remove entries from ZIP file.
Specify which files to remove by providing at least one fnmatch-style pattern,
e.g. `'META-INF/MANIFEST.MF'`.
```bash
$ rm-files.py --help
usage: rm-files.py [-h] [-v] INPUT_APK OUTPUT_APK PATTERN [PATTERN ...]
[...]
$ rm-files.py some.apk fixed.apk META-INF/MANIFEST.IN
skipping 'META-INF/MANIFEST.IN'...
$ zipalign -f 4 fixed.apk fixed-aligned.apk
```
NB: this builds a new ZIP file, preserving most ZIP metadata (and recompressing
using the same compression level) but not everything: e.g. copying the existing
local header extra fields which contain padding for alignment is not supported
by Python's `ZipFile`, which is why `zipalign` is usually needed.
### sort-apk.py
Sort (and w/o `--no-realign` also realign) the ZIP entries of an APK.
If the ordering of the ZIP entries in an APK is not deterministic/reproducible,
this script may help. You'll almost certainly need to use it for all builds
though, since it can only sort the APK, not recreate a different ordering that
is deterministic but not sorted; see also the alignment CAVEAT.
```bash
$ sort-apk.py --help
usage: sort-apk.py [-h] [--no-realign] [--no-force-align] [--reset-lh-extra] INPUT_APK OUTPUT_APK
[...]
$ unzip -l unsigned.apk
Archive: unsigned.apk
Length Date Time Name
--------- ---------- ----- ----
6 2017-05-15 11:24 lib/armeabi/fake.so
1672 2009-01-01 00:00 AndroidManifest.xml
896 2009-01-01 00:00 resources.arsc
1536 2009-01-01 00:00 classes.dex
--------- -------
4110 4 files
$ sort-apk.py unsigned.apk sorted.apk
$ unzip -l sorted.apk
Archive: sorted.apk
Length Date Time Name
--------- ---------- ----- ----
1672 2009-01-01 00:00 AndroidManifest.xml
1536 2009-01-01 00:00 classes.dex
6 2017-05-15 11:24 lib/armeabi/fake.so
896 2009-01-01 00:00 resources.arsc
--------- -------
4110 4 files
```
NB: this directly copies the (bytes of the) original ZIP entries from the
original file, thus preserving all ZIP metadata.
#### CAVEAT: alignment
Unfortunately, the padding added to ZIP local header extra fields for alignment
makes it hard to make sorting deterministic: unless the original APK was not
aligned at all, the padding is often different when the APK entries had a
different order (and thus a different offset) before sorting.
Because of this, `sort-apk` forcefully recreates the padding even if the entry
is already aligned (since that doesn't mean the padding is identical) to make
its output as deterministic as possible. The downside is that it'll often add
"unnecessary" 8-byte padding to entries that didn't need alignment.
You can disable this using `--no-force-align`, or skip realignment completely
using `--no-realign`. If you're certain you don't need to keep the old values,
you can also choose to reset the local header extra fields to the values from
the central directory entries with `--reset-lh-extra`.
If you use `--reset-lh-extra`, you'll probably want to combine it with either
`--no-force-align` (which should prevent the "unnecessary" 8-byte padding) or
`--no-realign` + `zipalign` (which uses smaller padding).
NB: the alignment padding used by `sort-apk` is the same as that used by
`apksigner` (a `0xd935` "Android ZIP Alignment Extra Field" which stores the
alignment itself plus zero padding and is thus always at least 6 bytes), whereas
`zipalign` just uses plain zero padding.
### sort-baseline.py
Sort `baseline.profm` (extracted or inside an APK).
```bash
$ sort-baseline.py --help
usage: sort-baseline.py [-h] [--apk] INPUT_PROF_OR_APK OUTPUT_PROF_OR_APK
[...]
$ diff -qs a/baseline.profm b/baseline.profm
Files a/baseline.profm and b/baseline.profm differ
$ sort-baseline.py a/baseline.profm a/baseline-sorted.profm
$ sort-baseline.py b/baseline.profm b/baseline-sorted.profm
$ diff -qs a/baseline-sorted.profm b/baseline-sorted.profm
Files a/baseline-sorted.profm and b/baseline-sorted.profm are identical
```
```bash
$ sort-baseline.py --apk unsigned.apk sorted-baseline.apk
$ zipalign -f 4 sorted-baseline.apk sorted-baseline-aligned.apk
```
NB: does not support all file format versions yet.
NB: with `--apk`, this builds a new ZIP file, preserving most ZIP metadata (and
recompressing using the same compression level) but not everything: e.g. copying
the existing local header extra fields which contain padding for alignment is
not supported by Python's `ZipFile`, which is why `zipalign` is usually needed.
### zipalign.py
Align uncompressed ZIP/APK entries to 4-byte boundaries (and `.so` shared object
files to 4096-byte boundaries with `-p`/`--page-align`, or other page sizes with
`-P`/`--page-size`).
This implementation aims for compatibility with Android's `zipalign`, with the
exception of there not being a `-f` option to enable overwriting an existing
output file (it will always be overwritten), and the `ALIGN` parameter -- which
must always be 4 anyway -- being optional; not does it support the `-c`, `-v`,
or `-z` options.
By default, the same plain zero padding as the original `zipalign` is used, but
with the `--pad-like-apksigner` option it uses the same alignment padding as
`apksigner` (a `0xd935` "Android ZIP Alignment Extra Field" which stores the
alignment itself plus zero padding and is thus always at least 6 bytes).
```bash
$ zipalign.py --help
usage: zipalign.py [-h] [-p] [-P N] [--pad-like-apksigner] [--copy-extra] [--no-update-lfh]
[ALIGN] INPUT_APK OUTPUT_APK
[...]
$ zipalign -f 4 fixed.apk fixed-aligned.apk
$ zipalign.py fixed.apk fixed-aligned-py.apk
$ cmp fixed-aligned.apk fixed-aligned-py.apk && echo OK
OK
```
## scripts to dump info from apks and related file formats
### diff-zip-meta.py
Diff ZIP file metadata.
NB: this will not compare the *contents* of the ZIP entries, only metadata and
other non-contents bytes; to compare the contents of ZIP/APK files, use e.g.
[`diffoscope`](https://diffoscope.org).
This will show differences in filenames, central directory headers, local file
headers, data descriptors, entry sizes, etc.
Additional tests include compression level (if it can be determined), CRC32
checksum of compressed data, and extra data before entries or the central
directory; you can skip these (relatively slow) tests using `--no-additional`.
Some differences make the output quite verbose and/or are usually the result of
other differences; you can skip/ignore these using `--no-lfh-extra`,
`--no-offsets`, `--no-ordering`.
```bash
$ diff-zip-meta.py --help
usage: diff-zip-meta.py [-h] [--no-additional] [--no-lfh-extra] [--no-offsets] [--no-ordering]
ZIPFILE1 ZIPFILE2
[...]
$ diff-zip-meta.py a.apk b.apk
--- a.apk
+++ b.apk
entry foo:
- compresslevel=6
+ compresslevel=9
- compress_crc=0x9ed711dc
+ compress_crc=0xd9776b0c
$ diff-zip-meta.py a.apk c.apk --no-offsets --no-ordering
--- a.apk
+++ c.apk
entries (sorted by filename):
- filename=META-INF/CERT.RSA
- filename=META-INF/CERT.SF
- filename=META-INF/MANIFEST.MF
central directory:
data_before:
- aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
- bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
entry foo:
- compresslevel=6
+ compresslevel=9
- compress_crc=0x9ed711dc
+ compress_crc=0xd9776b0c
entry META-INF/com/android/build/gradle/app-metadata.properties:
data_before (entry):
- 504b030400000000000021082102000000000000000000000000000066000000000000000000
- 0000000000000000000000000000000000000000000000000000000000000000000000000000
- 0000000000000000000000000000000000000000000000000000000000000000000000000000
- 000000000000000000000000000000000000
```
NB: work in progress; output format may change.
### dump-arsc.py
Dump `resources.arsc` (extracted or inside an APK) using `aapt2`.
```bash
$ dump-arsc.py --help
usage: dump-arsc.py [-h] [--apk] ARSC_OR_APK
[...]
$ dump-arsc.py resources.arsc
Binary APK
Package name=com.example.app id=7f
[...]
$ dump-arsc.py --apk some.apk
Binary APK
Package name=com.example.app id=7f
[...]
```
### dump-axml.py
Dump Android binary XML (extracted or inside an APK) using `aapt2`.
```bash
$ dump-axml.py --help
usage: dump-axml.py [-h] [--apk APK] AXML
[...]
$ dump-axml.py foo.xml
N: android=http://schemas.android.com/apk/res/android (line=17)
E: selector (line=17)
E: item (line=18)
[...]
$ dump-axml.py --apk some.apk res/foo.xml
N: android=http://schemas.android.com/apk/res/android (line=17)
E: selector (line=17)
E: item (line=18)
[...]
```
### dump-baseline.py
Dump `baseline.prof`/`baseline.profm` (extracted or inside an APK).
```bash
$ dump-baseline.py --help
usage: dump-baseline.py [-h] [--apk] [-v] PROF_OR_APK
[...]
$ dump-baseline.py baseline.prof
prof version=010 P
num_dex_files=4
[...]
$ dump-baseline.py baseline.profm
profm version=002
num_dex_files=4
[...]
$ dump-baseline.py some.apk
entry=assets/dexopt/baseline.prof
prof version=010 P
num_dex_files=4
[...]
entry=assets/dexopt/baseline.profm
profm version=002
num_dex_files=4
[...]
```
NB: does not support all file format versions yet.
### list-compresslevel.py
List ZIP entries with compression level.
You can optionally specify which files to list by providing one or more
fnmatch-style patterns, e.g. `'assets/foo/*.bar'`.
```bash
$ list-compresslevel.py --help
usage: list-compresslevel.py [-h] APK [PATTERN ...]
[...]
$ list-compresslevel.py some.apk
filename='AndroidManifest.xml' compresslevel=9|6
filename='classes.dex' compresslevel=None
filename='resources.arsc' compresslevel=None
[...]
filename='META-INF/CERT.SF' compresslevel=9|6
filename='META-INF/CERT.RSA' compresslevel=9|6|4
filename='META-INF/MANIFEST.MF' compresslevel=9|6|4
```
NB: the compression level is not actually stored anywhere in the ZIP file, and
is thus calculated by recompressing the data with different compression levels
and checking the CRC32 of the result against the CRC32 of the original
compressed data.
### zipinfo.py
List ZIP entries (like `zipinfo`).
This implementation aims for compatibility with the default and `-l` output
formats of Info-ZIP's `zipinfo`; the `-e` extended output format is unique to
this implementation. Other formats and options are (currently) not supported.
Neither is the full variety of ZIP formats and extensions supported, just the
most common ones (UNIX, FAT, NTFS).
The `-l`/`--long` option adds the compressed size before the compression type;
`-e`/`--extended` does the same, adds the CRC32 checksum before the filename as
well, uses a more standard date format, and treats filenames ending with a `/`
as directories.
```bash
$ zipinfo.py --help
usage: zipinfo.py [-h] [-1] [-e] [-l] [--sort-by-offset] ZIPFILE
[...]
$ zipinfo.py -e some.apk
Archive: some.apk
Zip file size: 5612 bytes, number of entries: 8
drw---- 2.0 fat 0 bX 2 defN 2017-05-15 11:25:18 00000000 META-INF/
-rw---- 2.0 fat 77 bl 76 defN 2017-05-15 11:25:18 b506b894 META-INF/MANIFEST.MF
-rw---- 2.0 fat 1672 bl 630 defN 2009-01-01 00:00:00 615ef200 AndroidManifest.xml
-rw---- 1.0 fat 1536 b- 1536 stor 2009-01-01 00:00:00 9987d5d8 classes.dex
-rw---- 2.0 fat 29 bl 6 defN 2017-05-15 11:26:52 ff801cd1 temp.txt
-rw---- 1.0 fat 6 b- 6 stor 2017-05-15 11:24:32 31963516 lib/armeabi/fake.so
-rw---- 1.0 fat 896 b- 896 stor 2009-01-01 00:00:00 4fcab821 resources.arsc
-rw---- 2.0 fat 20 bl 6 defN 2017-05-15 11:28:40 c9983e85 temp2.txt
8 files, 4236 bytes uncompressed, 3158 bytes compressed: 25.4%
$ zipinfo.py -l some.apk
Archive: some.apk
Zip file size: 5612 bytes, number of entries: 8
-rw---- 2.0 fat 0 bX 2 defN 17-May-15 11:25 META-INF/
-rw---- 2.0 fat 77 bl 76 defN 17-May-15 11:25 META-INF/MANIFEST.MF
-rw---- 2.0 fat 1672 bl 630 defN 09-Jan-01 00:00 AndroidManifest.xml
-rw---- 1.0 fat 1536 b- 1536 stor 09-Jan-01 00:00 classes.dex
-rw---- 2.0 fat 29 bl 6 defN 17-May-15 11:26 temp.txt
-rw---- 1.0 fat 6 b- 6 stor 17-May-15 11:24 lib/armeabi/fake.so
-rw---- 1.0 fat 896 b- 896 stor 09-Jan-01 00:00 resources.arsc
-rw---- 2.0 fat 20 bl 6 defN 17-May-15 11:28 temp2.txt
8 files, 4236 bytes uncompressed, 3158 bytes compressed: 25.4%
```
The fields are: permissions, create version, create system, uncompressed size,
extra info, compressed size (w/ `--long` or `--extended`), compression type,
date, time, CRC32 (w/ `--extended`), filename.
The extra info field consists of two characters: the first is `b` for binary,
`t` for text (uppercase for encrypted files); the second is `X` for data
descriptor and extra field, `l` for just data descriptor, `x` for just extra
field, `-` for neither.
See also:
[`zipinfo(1)`](https://manpages.debian.org/stable/unzip/zipinfo.1.en.html),
[`zipdetails(1)`](https://manpages.debian.org/stable/perl/zipdetails.1.en.html).
## helper scripts
### inplace-fix.py
Convenience wrapper for some of the other scripts like `fix-newlines` that makes
them modify the file in-place (and optionally `zipalign` it too).
```bash
$ inplace-fix.py --help
usage: inplace-fix.py [-h] [--zipalign] [--page-align] [--page-size N] [--internal]
COMMAND INPUT_FILE [...]
[...]
$ inplace-fix.py --zipalign fix-newlines unsigned.apk 'META-INF/services/*'
[RUN] python3 fix-newlines.py unsigned.apk /tmp/.../fixed.apk META-INF/services/*
fixing 'META-INF/services/foo'...
fixing 'META-INF/services/bar'...
[RUN] zipalign 4 /tmp/.../fixed.apk /tmp/.../aligned.apk
[MOVE] /tmp/.../aligned.apk to unsigned.apk
```
If `zipalign` is not found on `$PATH` but any of `$ANDROID_HOME`,
`$ANDROID_SDK`, or `$ANDROID_SDK_ROOT` is set to an Android SDK directory, it
will use `zipalign` from the latest `build-tools` subdirectory of the Android
SDK. If no suitable `zipalign` command can be found this way or the
`--internal` option is passed, `zipalign.py` will be used.
NB: `build-tools` `31.0.0` and `32.0.0` are skipped because
[their `zipalign` is broken](https://android.googlesource.com/platform/build/+/df73d1b4733b8b3cdfd96199018455026ba8d9d2);
`--page-size` requires `build-tools` >= `35.0.0-rc1`.
NB: this script is not available as a `repro-apk` subcommand, but as a separate
`repro-apk-inplace-fix` command.
## gradle integration
You can e.g. sort `baseline.profm` during the `gradle` build by adding something
like this to your `build.gradle`:
<details>
```gradle
// NB: assumes reproducible-apk-tools is a submodule in the app repo's
// root dir; adjust the path accordingly if it is found elsewhere
project.afterEvaluate {
tasks.compileReleaseArtProfile.doLast {
outputs.files.each { file ->
if (file.name.endsWith(".profm")) {
exec {
commandLine(
"../reproducible-apk-tools/inplace-fix.py",
"sort-baseline", file
)
}
}
}
}
}
```
</details>
Alternatively, adding something like this allows you to modify the APK itself
after building (and re-sign it if necessary):
<details>
```gradle
// NB: assumes reproducible-apk-tools is a submodule in the app repo's
// root dir; adjust the path accordingly if it is found elsewhere
android {
applicationVariants.all { variant ->
variant.outputs.each { output ->
variant.packageApplicationProvider.get().doLast {
exec {
// set ANDROID_HOME for zipalign
environment "ANDROID_HOME", android.sdkDirectory
commandLine(
"../reproducible-apk-tools/inplace-fix.py",
"--zipalign", "fix-newlines", output.outputFile,
"META-INF/services/*"
)
}
// re-sign w/ apksigner if needed
if (variant.signingConfig != null) {
def tools = "${android.sdkDirectory}/build-tools/${android.buildToolsVersion}"
def sc = variant.signingConfig
exec {
environment "KS_PASS", sc.storePassword
environment "KEY_PASS", sc.keyPassword
commandLine(
"${tools}/apksigner", "sign", "-v",
"--ks", sc.storeFile,
"--ks-pass", "env:KS_PASS",
"--ks-key-alias", sc.keyAlias,
"--key-pass", "env:KEY_PASS",
output.outputFile
)
}
}
}
}
}
}
```
</details>
## fnmatch-style patterns
Some of these scripts process/list files matching any of the provided patterns
using Python's `fnmatch.fnmatch()`, Unix shell style:
```
* matches everything
? matches any single character
[seq] matches any character in seq
[!seq] matches any char not in seq
```
With one addition: an optional prefix `!` negates the pattern, invalidating a
successful match by any preceding pattern; use a backslash (`\`) in front of the
first `!` for patterns that begin with a literal `!`.
NB: to match e.g. everything except for `*.xml`, you need to provide two
patterns: the first (`'*'`) to match everything, the second (`'!*.xml'`) to
negate matching `*.xml`.
NB: `*` matches anything, including `/`, and the pattern matches the complete
filename path, including leading directories, so e.g. `foo/bar.baz` is matched
by both `*.baz` and `foo/*`.
## CLI
NB: you can just use the scripts stand-alone; alternatively, you can install the
`repro-apk` Python package and use them as subcommands of `repro-apk`:
```bash
$ repro-apk diff-zip-meta a.apk b.apk
$ repro-apk diff-zip-meta a.apk c.apk --no-offsets --no-ordering
$ repro-apk dump-arsc resources.arsc
$ repro-apk dump-arsc --apk some.apk
$ repro-apk dump-axml foo.xml
$ repro-apk dump-axml --apk some.apk res/foo.xml
$ repro-apk dump-baseline baseline.prof
$ repro-apk dump-baseline baseline.profm
$ repro-apk dump-baseline --apk some.apk
$ repro-apk fix-compresslevel unsigned.apk fixed.apk 6 assets/foo/bar.js
$ repro-apk fix-files unsigned.apk fixed.apk unix2dos 'META-INF/services/*'
$ repro-apk fix-newlines unsigned.apk fixed.apk 'META-INF/services/*'
$ repro-apk fix-pg-map-id unsigned.apk fixed.apk da39a3e
$ repro-apk fix-pg-map-id input-dir output-dir da39a3e
$ repro-apk list-compresslevel some.apk
$ repro-apk rm-files some.apk fixed.apk META-INF/MANIFEST.IN
$ repro-apk sort-apk unsigned.apk sorted.apk
$ repro-apk sort-baseline baseline.profm baseline-sorted.profm
$ repro-apk sort-baseline --apk unsigned.apk sorted-baseline.apk
$ repro-apk zipalign fixed.apk fixed-aligned-py.apk
$ repro-apk zipinfo -e some.apk
$ repro-apk zipinfo -l some.apk
```
### Help
```bash
$ repro-apk --help
$ repro-apk diff-zip-meta --help
$ repro-apk dump-arsc --help
$ repro-apk dump-axml --help
$ repro-apk dump-baseline --help
$ repro-apk fix-compresslevel --help
$ repro-apk fix-files --help
$ repro-apk fix-newlines --help
$ repro-apk fix-pg-map-id --help
$ repro-apk list-compresslevel --help
$ repro-apk rm-files --help
$ repro-apk sort-apk --help
$ repro-apk sort-baseline --help
$ repro-apk zipalign --help
$ repro-apk zipinfo --help
```
## Installing
### Using pip
```bash
$ pip install repro-apk
```
NB: depending on your system you may need to use e.g. `pip3 --user`
instead of just `pip`.
### From git
NB: this installs the latest development version, not the latest
release.
```bash
$ git clone https://github.com/obfusk/reproducible-apk-tools.git
$ cd reproducible-apk-tools
$ pip install -e .
```
NB: you may need to add e.g. `~/.local/bin` to your `$PATH` in order
to run `repro-apk`.
To update to the latest development version:
```bash
$ cd reproducible-apk-tools
$ git pull --rebase
```
## Dependencies
* Python >= 3.8 + click (`repro-apk` package only, the stand-alone scripts have
no dependencies besides Python).
* The `dump-arsc.py` and `dump-axml.py` scripts require `aapt2`.
### Debian/Ubuntu
```bash
$ apt install python3-click
$ apt install aapt # for dump-arsc.py & dump-axml.py
$ apt install zipalign # for realignment; see examples
```
## License
[](https://www.gnu.org/licenses/gpl-3.0.html)
<!-- vim: set tw=70 sw=2 sts=2 et fdm=marker : -->
Raw data
{
"_id": null,
"home_page": "https://github.com/obfusk/reproducible-apk-tools",
"name": "repro-apk",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "android apk reproducible",
"author": "FC Stegerman",
"author_email": "flx@obfusk.net",
"download_url": "https://files.pythonhosted.org/packages/a0/24/d8db30111e6a70ddd54063d61c6c312667dee42b8ef4554877e4255a00d8/repro-apk-0.2.5.tar.gz",
"platform": null,
"description": "<!-- SPDX-FileCopyrightText: 2024 FC (Fay) Stegerman <flx@obfusk.net> -->\n<!-- SPDX-License-Identifier: GPL-3.0-or-later -->\n\n[](https://github.com/obfusk/reproducible-apk-tools/releases)\n[](https://pypi.python.org/pypi/repro-apk)\n[](https://pypi.python.org/pypi/repro-apk)\n[](https://github.com/obfusk/reproducible-apk-tools/actions?query=workflow%3ACI)\n[](https://www.gnu.org/licenses/gpl-3.0.html)\n\n<!--\n<a href=\"https://repology.org/project/repro-apk/versions\">\n <img src=\"https://repology.org/badge/vertical-allrepos/repro-apk.svg?header=\"\n alt=\"Packaging status\" align=\"right\" />\n</a>\n\n<a href=\"https://repology.org/project/python:repro-apk/versions\">\n <img src=\"https://repology.org/badge/vertical-allrepos/python:repro-apk.svg?header=\"\n alt=\"Packaging status\" align=\"right\" />\n</a>\n-->\n\n# reproducible-apk-tools\n\n[`fix-compresslevel.py`](#fix-compresslevelpy),\n[`fix-files.py`](#fix-filespy),\n[`fix-newlines.py`](#fix-newlinespy),\n[`fix-pg-map-id.py`](#fix-pg-map-idpy),\n[`rm-files.py`](#rm-filespy),\n[`sort-apk.py`](#sort-apkpy),\n[`sort-baseline.py`](#sort-baselinepy),\n[`zipalign.py`](#zipalignpy);\n\n[`diff-zip-meta.py`](#diff-zip-metapy),\n[`dump-arsc.py`](#dump-arscpy),\n[`dump-axml.py`](#dump-axmlpy),\n[`dump-baseline.py`](#dump-baselinepy),\n[`list-compresslevel.py`](#list-compresslevelpy),\n[`zipinfo.py`](#zipinfopy);\n\n[`inplace-fix.py`](#inplace-fixpy).\n\n## scripts to make android apks reproducible\n\n`reproducible-apk-tools` is a collection of scripts (available as subcommands of\nthe `repro-apk` command) to help make APKs reproducible (e.g. by changing line\nendings from LF to CRLF), or find out why they are not (e.g. by comparing ZIP\nfile metadata, or dumping `baseline.prof` files).\n\n### fix-compresslevel.py\n\nRecompress with different compression level.\n\nSpecify which files to change by providing at least one fnmatch-style pattern,\ne.g. `'assets/foo/*.bar'`.\n\nIf two APKs have identical contents but some ZIP entries are compressed with a\ndifferent compression level, thus making the APKs not bit-by-bit identical, this\nscript may help.\n\n```bash\n$ fix-compresslevel.py --help\nusage: fix-compresslevel.py [-h] [-v] INPUT_APK OUTPUT_APK COMPRESSLEVEL PATTERN [PATTERN ...]\n[...]\n$ apksigcopier compare signed.apk --unsigned unsigned.apk\nDOES NOT VERIFY\n[...]\n$ fix-compresslevel.py unsigned.apk fixed.apk 6 assets/foo/bar.js\nfixing 'assets/foo/bar.js'...\n$ zipalign -f 4 fixed.apk fixed-aligned.apk\n$ apksigcopier compare signed.apk --unsigned fixed-aligned.apk && echo OK\nOK\n```\n\nNB: this builds a new ZIP file, preserving most ZIP metadata (and recompressing\nentries not matching the pattern using the same compression level as in the\noriginal APK) but not everything: e.g. copying the existing local header extra\nfields which contain padding for alignment is not supported by Python's\n`ZipFile`, which is why `zipalign` is usually needed.\n\n### fix-files.py\n\nProcess ZIP entries using an external command.\n\nRuns the command for each specified file, providing the old file contents as\nstdin and using stdout as the new file contents.\n\nThe provided command is split on whitespace to allow passing arguments (e.g.\n`'foo --bar'`), but shell syntax is not supported.\n\nSpecify which files to process by providing at least one fnmatch-style pattern,\ne.g. `'META-INF/services/*'`.\n\n```bash\n$ fix-files.py --help\nusage: fix-files.py [-h] [-v] INPUT_APK OUTPUT_APK COMMAND PATTERN [PATTERN ...]\n[...]\n$ apksigcopier compare signed.apk --unsigned unsigned.apk\nDOES NOT VERIFY\n[...]\n$ fix-files.py unsigned.apk fixed.apk unix2dos 'META-INF/services/*'\nprocessing 'META-INF/services/foo' with 'unix2dos'...\nprocessing 'META-INF/services/bar' with 'unix2dos'...\n$ zipalign -f 4 fixed.apk fixed-aligned.apk\n$ apksigcopier compare signed.apk --unsigned fixed-aligned.apk && echo OK\nOK\n```\n\nNB: this builds a new ZIP file, preserving most ZIP metadata (and recompressing\nusing the same compression level) but not everything: e.g. copying the existing\nlocal header extra fields which contain padding for alignment is not supported\nby Python's `ZipFile`, which is why `zipalign` is usually needed.\n\n### fix-newlines.py\n\nChange line endings from LF to CRLF (or vice versa w/ `--from-crlf`).\n\nSpecify which files to change by providing at least one fnmatch-style pattern,\ne.g. `'META-INF/services/*'`.\n\nIf the signed APK was built on Windows and has e.g. `META-INF/services/` files\nwith CRLF line endings whereas the unsigned APK was build on Linux/macOS and has\nLF line endings, this script may help.\n\n```bash\n$ fix-newlines.py --help\nusage: fix-newlines.py [-h] [--from-crlf] [--to-crlf] [-v] INPUT_APK OUTPUT_APK PATTERN [PATTERN ...]\n[...]\n$ apksigcopier compare signed.apk --unsigned unsigned.apk\nDOES NOT VERIFY\n[...]\n$ fix-newlines.py unsigned.apk fixed.apk 'META-INF/services/*'\nfixing 'META-INF/services/foo'...\nfixing 'META-INF/services/bar'...\n$ zipalign -f 4 fixed.apk fixed-aligned.apk\n$ apksigcopier compare signed.apk --unsigned fixed-aligned.apk && echo OK\nOK\n```\n\nNB: this builds a new ZIP file, preserving most ZIP metadata (and recompressing\nusing the same compression level) but not everything: e.g. copying the existing\nlocal header extra fields which contain padding for alignment is not supported\nby Python's `ZipFile`, which is why `zipalign` is usually needed.\n\n### fix-pg-map-id.py\n\nReplace non-deterministic R8 `pg-map-id` in `classes.dex` (and `classes2.dex`\netc. when present) and update checksums, also in `baseline.prof`.\n\n```bash\n$ fix-pg-map-id.py --help\nusage: fix-pg-map-id.py [-h] INPUT_DIR_OR_APK OUTPUT_DIR_OR_APK PG_MAP_ID\n[...]\n$ apksigcopier compare signed.apk --unsigned unsigned.apk\nDOES NOT VERIFY\n[...]\n$ diff -Naur <( unzip -p signed.apk classes.dex | xxd ) <( unzip -p unsigned.apk classes.dex | xxd )\n[...]\n-00000000: 6465 780a 3033 3500 829f 202e c800 6ecc dex.035... ...n.\n-00000010: c15c 0a17 3737 73fc 982e 34db 6239 bc52 .\\..77s...4.b9.R\n+00000000: 6465 780a 3033 3500 d89f 1795 f719 4d94 dex.035.......M.\n+00000010: 8358 2526 2850 f9e5 ad3d e772 c82e 4f02 .X%&(P...=.r..O.\n[...]\n 005f1010: 2d61 7069 223a 3233 2c22 7067 2d6d 6170 -api\":23,\"pg-map\n-005f1020: 2d69 6422 3a22 3261 3939 3764 3322 2c22 -id\":\"2a997d3\",\"\n+005f1020: 2d69 6422 3a22 6565 3436 3531 3322 2c22 -id\":\"ee46513\",\"\n[...]\n$ fix-pg-map-id.py unsigned.apk fixed.apk 2a997d3\nreading 'assets/dexopt/baseline.prof'...\nreading 'classes.dex'...\nreading 'classes2.dex'...\nfixing 'classes.dex'...\ndex version=035\nfixing pg-map-id: b'ee46513' -> b'2a997d3'\nfixing signature: f7194d94835825262850f9e5ad3de772c82e4f02 -> c8006eccc15c0a17373773fc982e34db6239bc52\nfixing checksum: 0x95179fd8 -> 0x2e209f82\nfixing 'classes2.dex'...\ndex version=035\n(not modified)\nfixing 'assets/dexopt/baseline.prof'...\nprof version=010 P\nfixing 'classes.dex' checksum: 0x95d40f72 -> 0x50191ba4\nwriting 'assets/dexopt/baseline.prof'...\nwriting 'classes.dex'...\nwriting 'classes2.dex'...\n$ zipalign -f 4 fixed.apk fixed-aligned.apk\n$ apksigcopier compare signed.apk --unsigned fixed-aligned.apk && echo OK\nOK\n```\n\nNB: this builds a new ZIP file, preserving most ZIP metadata (and recompressing\nusing the same compression level) but not everything: e.g. copying the existing\nlocal header extra fields which contain padding for alignment is not supported\nby Python's `ZipFile`, which is why `zipalign` is usually needed.\n\n### rm-files.py\n\nRemove entries from ZIP file.\n\nSpecify which files to remove by providing at least one fnmatch-style pattern,\ne.g. `'META-INF/MANIFEST.MF'`.\n\n```bash\n$ rm-files.py --help\nusage: rm-files.py [-h] [-v] INPUT_APK OUTPUT_APK PATTERN [PATTERN ...]\n[...]\n$ rm-files.py some.apk fixed.apk META-INF/MANIFEST.IN\nskipping 'META-INF/MANIFEST.IN'...\n$ zipalign -f 4 fixed.apk fixed-aligned.apk\n```\n\nNB: this builds a new ZIP file, preserving most ZIP metadata (and recompressing\nusing the same compression level) but not everything: e.g. copying the existing\nlocal header extra fields which contain padding for alignment is not supported\nby Python's `ZipFile`, which is why `zipalign` is usually needed.\n\n### sort-apk.py\n\nSort (and w/o `--no-realign` also realign) the ZIP entries of an APK.\n\nIf the ordering of the ZIP entries in an APK is not deterministic/reproducible,\nthis script may help. You'll almost certainly need to use it for all builds\nthough, since it can only sort the APK, not recreate a different ordering that\nis deterministic but not sorted; see also the alignment CAVEAT.\n\n```bash\n$ sort-apk.py --help\nusage: sort-apk.py [-h] [--no-realign] [--no-force-align] [--reset-lh-extra] INPUT_APK OUTPUT_APK\n[...]\n$ unzip -l unsigned.apk\nArchive: unsigned.apk\n Length Date Time Name\n--------- ---------- ----- ----\n 6 2017-05-15 11:24 lib/armeabi/fake.so\n 1672 2009-01-01 00:00 AndroidManifest.xml\n 896 2009-01-01 00:00 resources.arsc\n 1536 2009-01-01 00:00 classes.dex\n--------- -------\n 4110 4 files\n$ sort-apk.py unsigned.apk sorted.apk\n$ unzip -l sorted.apk\nArchive: sorted.apk\n Length Date Time Name\n--------- ---------- ----- ----\n 1672 2009-01-01 00:00 AndroidManifest.xml\n 1536 2009-01-01 00:00 classes.dex\n 6 2017-05-15 11:24 lib/armeabi/fake.so\n 896 2009-01-01 00:00 resources.arsc\n--------- -------\n 4110 4 files\n```\n\nNB: this directly copies the (bytes of the) original ZIP entries from the\noriginal file, thus preserving all ZIP metadata.\n\n#### CAVEAT: alignment\n\nUnfortunately, the padding added to ZIP local header extra fields for alignment\nmakes it hard to make sorting deterministic: unless the original APK was not\naligned at all, the padding is often different when the APK entries had a\ndifferent order (and thus a different offset) before sorting.\n\nBecause of this, `sort-apk` forcefully recreates the padding even if the entry\nis already aligned (since that doesn't mean the padding is identical) to make\nits output as deterministic as possible. The downside is that it'll often add\n\"unnecessary\" 8-byte padding to entries that didn't need alignment.\n\nYou can disable this using `--no-force-align`, or skip realignment completely\nusing `--no-realign`. If you're certain you don't need to keep the old values,\nyou can also choose to reset the local header extra fields to the values from\nthe central directory entries with `--reset-lh-extra`.\n\nIf you use `--reset-lh-extra`, you'll probably want to combine it with either\n`--no-force-align` (which should prevent the \"unnecessary\" 8-byte padding) or\n`--no-realign` + `zipalign` (which uses smaller padding).\n\nNB: the alignment padding used by `sort-apk` is the same as that used by\n`apksigner` (a `0xd935` \"Android ZIP Alignment Extra Field\" which stores the\nalignment itself plus zero padding and is thus always at least 6 bytes), whereas\n`zipalign` just uses plain zero padding.\n\n### sort-baseline.py\n\nSort `baseline.profm` (extracted or inside an APK).\n\n```bash\n$ sort-baseline.py --help\nusage: sort-baseline.py [-h] [--apk] INPUT_PROF_OR_APK OUTPUT_PROF_OR_APK\n[...]\n$ diff -qs a/baseline.profm b/baseline.profm\nFiles a/baseline.profm and b/baseline.profm differ\n$ sort-baseline.py a/baseline.profm a/baseline-sorted.profm\n$ sort-baseline.py b/baseline.profm b/baseline-sorted.profm\n$ diff -qs a/baseline-sorted.profm b/baseline-sorted.profm\nFiles a/baseline-sorted.profm and b/baseline-sorted.profm are identical\n```\n\n```bash\n$ sort-baseline.py --apk unsigned.apk sorted-baseline.apk\n$ zipalign -f 4 sorted-baseline.apk sorted-baseline-aligned.apk\n```\n\nNB: does not support all file format versions yet.\n\nNB: with `--apk`, this builds a new ZIP file, preserving most ZIP metadata (and\nrecompressing using the same compression level) but not everything: e.g. copying\nthe existing local header extra fields which contain padding for alignment is\nnot supported by Python's `ZipFile`, which is why `zipalign` is usually needed.\n\n### zipalign.py\n\nAlign uncompressed ZIP/APK entries to 4-byte boundaries (and `.so` shared object\nfiles to 4096-byte boundaries with `-p`/`--page-align`, or other page sizes with\n`-P`/`--page-size`).\n\nThis implementation aims for compatibility with Android's `zipalign`, with the\nexception of there not being a `-f` option to enable overwriting an existing\noutput file (it will always be overwritten), and the `ALIGN` parameter -- which\nmust always be 4 anyway -- being optional; not does it support the `-c`, `-v`,\nor `-z` options.\n\nBy default, the same plain zero padding as the original `zipalign` is used, but\nwith the `--pad-like-apksigner` option it uses the same alignment padding as\n`apksigner` (a `0xd935` \"Android ZIP Alignment Extra Field\" which stores the\nalignment itself plus zero padding and is thus always at least 6 bytes).\n\n```bash\n$ zipalign.py --help\nusage: zipalign.py [-h] [-p] [-P N] [--pad-like-apksigner] [--copy-extra] [--no-update-lfh]\n [ALIGN] INPUT_APK OUTPUT_APK\n[...]\n$ zipalign -f 4 fixed.apk fixed-aligned.apk\n$ zipalign.py fixed.apk fixed-aligned-py.apk\n$ cmp fixed-aligned.apk fixed-aligned-py.apk && echo OK\nOK\n```\n\n## scripts to dump info from apks and related file formats\n\n### diff-zip-meta.py\n\nDiff ZIP file metadata.\n\nNB: this will not compare the *contents* of the ZIP entries, only metadata and\nother non-contents bytes; to compare the contents of ZIP/APK files, use e.g.\n[`diffoscope`](https://diffoscope.org).\n\nThis will show differences in filenames, central directory headers, local file\nheaders, data descriptors, entry sizes, etc.\n\nAdditional tests include compression level (if it can be determined), CRC32\nchecksum of compressed data, and extra data before entries or the central\ndirectory; you can skip these (relatively slow) tests using `--no-additional`.\n\nSome differences make the output quite verbose and/or are usually the result of\nother differences; you can skip/ignore these using `--no-lfh-extra`,\n`--no-offsets`, `--no-ordering`.\n\n```bash\n$ diff-zip-meta.py --help\nusage: diff-zip-meta.py [-h] [--no-additional] [--no-lfh-extra] [--no-offsets] [--no-ordering]\n ZIPFILE1 ZIPFILE2\n[...]\n$ diff-zip-meta.py a.apk b.apk\n--- a.apk\n+++ b.apk\nentry foo:\n- compresslevel=6\n+ compresslevel=9\n- compress_crc=0x9ed711dc\n+ compress_crc=0xd9776b0c\n$ diff-zip-meta.py a.apk c.apk --no-offsets --no-ordering\n--- a.apk\n+++ c.apk\nentries (sorted by filename):\n- filename=META-INF/CERT.RSA\n- filename=META-INF/CERT.SF\n- filename=META-INF/MANIFEST.MF\ncentral directory:\n data_before:\n- aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\n- bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb\nentry foo:\n- compresslevel=6\n+ compresslevel=9\n- compress_crc=0x9ed711dc\n+ compress_crc=0xd9776b0c\nentry META-INF/com/android/build/gradle/app-metadata.properties:\n data_before (entry):\n- 504b030400000000000021082102000000000000000000000000000066000000000000000000\n- 0000000000000000000000000000000000000000000000000000000000000000000000000000\n- 0000000000000000000000000000000000000000000000000000000000000000000000000000\n- 000000000000000000000000000000000000\n```\n\nNB: work in progress; output format may change.\n\n### dump-arsc.py\n\nDump `resources.arsc` (extracted or inside an APK) using `aapt2`.\n\n```bash\n$ dump-arsc.py --help\nusage: dump-arsc.py [-h] [--apk] ARSC_OR_APK\n[...]\n$ dump-arsc.py resources.arsc\nBinary APK\nPackage name=com.example.app id=7f\n[...]\n$ dump-arsc.py --apk some.apk\nBinary APK\nPackage name=com.example.app id=7f\n[...]\n```\n\n### dump-axml.py\n\nDump Android binary XML (extracted or inside an APK) using `aapt2`.\n\n```bash\n$ dump-axml.py --help\nusage: dump-axml.py [-h] [--apk APK] AXML\n[...]\n$ dump-axml.py foo.xml\nN: android=http://schemas.android.com/apk/res/android (line=17)\n E: selector (line=17)\n E: item (line=18)\n[...]\n$ dump-axml.py --apk some.apk res/foo.xml\nN: android=http://schemas.android.com/apk/res/android (line=17)\n E: selector (line=17)\n E: item (line=18)\n[...]\n```\n\n### dump-baseline.py\n\nDump `baseline.prof`/`baseline.profm` (extracted or inside an APK).\n\n```bash\n$ dump-baseline.py --help\nusage: dump-baseline.py [-h] [--apk] [-v] PROF_OR_APK\n[...]\n$ dump-baseline.py baseline.prof\nprof version=010 P\nnum_dex_files=4\n[...]\n$ dump-baseline.py baseline.profm\nprofm version=002\nnum_dex_files=4\n[...]\n$ dump-baseline.py some.apk\nentry=assets/dexopt/baseline.prof\nprof version=010 P\nnum_dex_files=4\n[...]\nentry=assets/dexopt/baseline.profm\nprofm version=002\nnum_dex_files=4\n[...]\n```\n\nNB: does not support all file format versions yet.\n\n### list-compresslevel.py\n\nList ZIP entries with compression level.\n\nYou can optionally specify which files to list by providing one or more\nfnmatch-style patterns, e.g. `'assets/foo/*.bar'`.\n\n```bash\n$ list-compresslevel.py --help\nusage: list-compresslevel.py [-h] APK [PATTERN ...]\n[...]\n$ list-compresslevel.py some.apk\nfilename='AndroidManifest.xml' compresslevel=9|6\nfilename='classes.dex' compresslevel=None\nfilename='resources.arsc' compresslevel=None\n[...]\nfilename='META-INF/CERT.SF' compresslevel=9|6\nfilename='META-INF/CERT.RSA' compresslevel=9|6|4\nfilename='META-INF/MANIFEST.MF' compresslevel=9|6|4\n```\n\nNB: the compression level is not actually stored anywhere in the ZIP file, and\nis thus calculated by recompressing the data with different compression levels\nand checking the CRC32 of the result against the CRC32 of the original\ncompressed data.\n\n### zipinfo.py\n\nList ZIP entries (like `zipinfo`).\n\nThis implementation aims for compatibility with the default and `-l` output\nformats of Info-ZIP's `zipinfo`; the `-e` extended output format is unique to\nthis implementation. Other formats and options are (currently) not supported.\n\nNeither is the full variety of ZIP formats and extensions supported, just the\nmost common ones (UNIX, FAT, NTFS).\n\nThe `-l`/`--long` option adds the compressed size before the compression type;\n`-e`/`--extended` does the same, adds the CRC32 checksum before the filename as\nwell, uses a more standard date format, and treats filenames ending with a `/`\nas directories.\n\n```bash\n$ zipinfo.py --help\nusage: zipinfo.py [-h] [-1] [-e] [-l] [--sort-by-offset] ZIPFILE\n[...]\n$ zipinfo.py -e some.apk\nArchive: some.apk\nZip file size: 5612 bytes, number of entries: 8\ndrw---- 2.0 fat 0 bX 2 defN 2017-05-15 11:25:18 00000000 META-INF/\n-rw---- 2.0 fat 77 bl 76 defN 2017-05-15 11:25:18 b506b894 META-INF/MANIFEST.MF\n-rw---- 2.0 fat 1672 bl 630 defN 2009-01-01 00:00:00 615ef200 AndroidManifest.xml\n-rw---- 1.0 fat 1536 b- 1536 stor 2009-01-01 00:00:00 9987d5d8 classes.dex\n-rw---- 2.0 fat 29 bl 6 defN 2017-05-15 11:26:52 ff801cd1 temp.txt\n-rw---- 1.0 fat 6 b- 6 stor 2017-05-15 11:24:32 31963516 lib/armeabi/fake.so\n-rw---- 1.0 fat 896 b- 896 stor 2009-01-01 00:00:00 4fcab821 resources.arsc\n-rw---- 2.0 fat 20 bl 6 defN 2017-05-15 11:28:40 c9983e85 temp2.txt\n8 files, 4236 bytes uncompressed, 3158 bytes compressed: 25.4%\n$ zipinfo.py -l some.apk\nArchive: some.apk\nZip file size: 5612 bytes, number of entries: 8\n-rw---- 2.0 fat 0 bX 2 defN 17-May-15 11:25 META-INF/\n-rw---- 2.0 fat 77 bl 76 defN 17-May-15 11:25 META-INF/MANIFEST.MF\n-rw---- 2.0 fat 1672 bl 630 defN 09-Jan-01 00:00 AndroidManifest.xml\n-rw---- 1.0 fat 1536 b- 1536 stor 09-Jan-01 00:00 classes.dex\n-rw---- 2.0 fat 29 bl 6 defN 17-May-15 11:26 temp.txt\n-rw---- 1.0 fat 6 b- 6 stor 17-May-15 11:24 lib/armeabi/fake.so\n-rw---- 1.0 fat 896 b- 896 stor 09-Jan-01 00:00 resources.arsc\n-rw---- 2.0 fat 20 bl 6 defN 17-May-15 11:28 temp2.txt\n8 files, 4236 bytes uncompressed, 3158 bytes compressed: 25.4%\n```\n\nThe fields are: permissions, create version, create system, uncompressed size,\nextra info, compressed size (w/ `--long` or `--extended`), compression type,\ndate, time, CRC32 (w/ `--extended`), filename.\n\nThe extra info field consists of two characters: the first is `b` for binary,\n`t` for text (uppercase for encrypted files); the second is `X` for data\ndescriptor and extra field, `l` for just data descriptor, `x` for just extra\nfield, `-` for neither.\n\nSee also:\n[`zipinfo(1)`](https://manpages.debian.org/stable/unzip/zipinfo.1.en.html),\n[`zipdetails(1)`](https://manpages.debian.org/stable/perl/zipdetails.1.en.html).\n\n## helper scripts\n\n### inplace-fix.py\n\nConvenience wrapper for some of the other scripts like `fix-newlines` that makes\nthem modify the file in-place (and optionally `zipalign` it too).\n\n```bash\n$ inplace-fix.py --help\nusage: inplace-fix.py [-h] [--zipalign] [--page-align] [--page-size N] [--internal]\n COMMAND INPUT_FILE [...]\n[...]\n$ inplace-fix.py --zipalign fix-newlines unsigned.apk 'META-INF/services/*'\n[RUN] python3 fix-newlines.py unsigned.apk /tmp/.../fixed.apk META-INF/services/*\nfixing 'META-INF/services/foo'...\nfixing 'META-INF/services/bar'...\n[RUN] zipalign 4 /tmp/.../fixed.apk /tmp/.../aligned.apk\n[MOVE] /tmp/.../aligned.apk to unsigned.apk\n```\n\nIf `zipalign` is not found on `$PATH` but any of `$ANDROID_HOME`,\n`$ANDROID_SDK`, or `$ANDROID_SDK_ROOT` is set to an Android SDK directory, it\nwill use `zipalign` from the latest `build-tools` subdirectory of the Android\nSDK. If no suitable `zipalign` command can be found this way or the\n`--internal` option is passed, `zipalign.py` will be used.\n\nNB: `build-tools` `31.0.0` and `32.0.0` are skipped because\n[their `zipalign` is broken](https://android.googlesource.com/platform/build/+/df73d1b4733b8b3cdfd96199018455026ba8d9d2);\n`--page-size` requires `build-tools` >= `35.0.0-rc1`.\n\nNB: this script is not available as a `repro-apk` subcommand, but as a separate\n`repro-apk-inplace-fix` command.\n\n## gradle integration\n\nYou can e.g. sort `baseline.profm` during the `gradle` build by adding something\nlike this to your `build.gradle`:\n\n<details>\n\n```gradle\n// NB: assumes reproducible-apk-tools is a submodule in the app repo's\n// root dir; adjust the path accordingly if it is found elsewhere\nproject.afterEvaluate {\n tasks.compileReleaseArtProfile.doLast {\n outputs.files.each { file ->\n if (file.name.endsWith(\".profm\")) {\n exec {\n commandLine(\n \"../reproducible-apk-tools/inplace-fix.py\",\n \"sort-baseline\", file\n )\n }\n }\n }\n }\n}\n```\n\n</details>\n\nAlternatively, adding something like this allows you to modify the APK itself\nafter building (and re-sign it if necessary):\n\n<details>\n\n```gradle\n// NB: assumes reproducible-apk-tools is a submodule in the app repo's\n// root dir; adjust the path accordingly if it is found elsewhere\nandroid {\n applicationVariants.all { variant ->\n variant.outputs.each { output ->\n variant.packageApplicationProvider.get().doLast {\n exec {\n // set ANDROID_HOME for zipalign\n environment \"ANDROID_HOME\", android.sdkDirectory\n commandLine(\n \"../reproducible-apk-tools/inplace-fix.py\",\n \"--zipalign\", \"fix-newlines\", output.outputFile,\n \"META-INF/services/*\"\n )\n }\n // re-sign w/ apksigner if needed\n if (variant.signingConfig != null) {\n def tools = \"${android.sdkDirectory}/build-tools/${android.buildToolsVersion}\"\n def sc = variant.signingConfig\n exec {\n environment \"KS_PASS\", sc.storePassword\n environment \"KEY_PASS\", sc.keyPassword\n commandLine(\n \"${tools}/apksigner\", \"sign\", \"-v\",\n \"--ks\", sc.storeFile,\n \"--ks-pass\", \"env:KS_PASS\",\n \"--ks-key-alias\", sc.keyAlias,\n \"--key-pass\", \"env:KEY_PASS\",\n output.outputFile\n )\n }\n }\n }\n }\n }\n}\n```\n\n</details>\n\n## fnmatch-style patterns\n\nSome of these scripts process/list files matching any of the provided patterns\nusing Python's `fnmatch.fnmatch()`, Unix shell style:\n\n```\n* matches everything\n? matches any single character\n[seq] matches any character in seq\n[!seq] matches any char not in seq\n```\n\nWith one addition: an optional prefix `!` negates the pattern, invalidating a\nsuccessful match by any preceding pattern; use a backslash (`\\`) in front of the\nfirst `!` for patterns that begin with a literal `!`.\n\nNB: to match e.g. everything except for `*.xml`, you need to provide two\npatterns: the first (`'*'`) to match everything, the second (`'!*.xml'`) to\nnegate matching `*.xml`.\n\nNB: `*` matches anything, including `/`, and the pattern matches the complete\nfilename path, including leading directories, so e.g. `foo/bar.baz` is matched\nby both `*.baz` and `foo/*`.\n\n## CLI\n\nNB: you can just use the scripts stand-alone; alternatively, you can install the\n`repro-apk` Python package and use them as subcommands of `repro-apk`:\n\n```bash\n$ repro-apk diff-zip-meta a.apk b.apk\n$ repro-apk diff-zip-meta a.apk c.apk --no-offsets --no-ordering\n$ repro-apk dump-arsc resources.arsc\n$ repro-apk dump-arsc --apk some.apk\n$ repro-apk dump-axml foo.xml\n$ repro-apk dump-axml --apk some.apk res/foo.xml\n$ repro-apk dump-baseline baseline.prof\n$ repro-apk dump-baseline baseline.profm\n$ repro-apk dump-baseline --apk some.apk\n$ repro-apk fix-compresslevel unsigned.apk fixed.apk 6 assets/foo/bar.js\n$ repro-apk fix-files unsigned.apk fixed.apk unix2dos 'META-INF/services/*'\n$ repro-apk fix-newlines unsigned.apk fixed.apk 'META-INF/services/*'\n$ repro-apk fix-pg-map-id unsigned.apk fixed.apk da39a3e\n$ repro-apk fix-pg-map-id input-dir output-dir da39a3e\n$ repro-apk list-compresslevel some.apk\n$ repro-apk rm-files some.apk fixed.apk META-INF/MANIFEST.IN\n$ repro-apk sort-apk unsigned.apk sorted.apk\n$ repro-apk sort-baseline baseline.profm baseline-sorted.profm\n$ repro-apk sort-baseline --apk unsigned.apk sorted-baseline.apk\n$ repro-apk zipalign fixed.apk fixed-aligned-py.apk\n$ repro-apk zipinfo -e some.apk\n$ repro-apk zipinfo -l some.apk\n```\n\n### Help\n\n```bash\n$ repro-apk --help\n$ repro-apk diff-zip-meta --help\n$ repro-apk dump-arsc --help\n$ repro-apk dump-axml --help\n$ repro-apk dump-baseline --help\n$ repro-apk fix-compresslevel --help\n$ repro-apk fix-files --help\n$ repro-apk fix-newlines --help\n$ repro-apk fix-pg-map-id --help\n$ repro-apk list-compresslevel --help\n$ repro-apk rm-files --help\n$ repro-apk sort-apk --help\n$ repro-apk sort-baseline --help\n$ repro-apk zipalign --help\n$ repro-apk zipinfo --help\n```\n\n## Installing\n\n### Using pip\n\n```bash\n$ pip install repro-apk\n```\n\nNB: depending on your system you may need to use e.g. `pip3 --user`\ninstead of just `pip`.\n\n### From git\n\nNB: this installs the latest development version, not the latest\nrelease.\n\n```bash\n$ git clone https://github.com/obfusk/reproducible-apk-tools.git\n$ cd reproducible-apk-tools\n$ pip install -e .\n```\n\nNB: you may need to add e.g. `~/.local/bin` to your `$PATH` in order\nto run `repro-apk`.\n\nTo update to the latest development version:\n\n```bash\n$ cd reproducible-apk-tools\n$ git pull --rebase\n```\n\n## Dependencies\n\n* Python >= 3.8 + click (`repro-apk` package only, the stand-alone scripts have\n no dependencies besides Python).\n\n* The `dump-arsc.py` and `dump-axml.py` scripts require `aapt2`.\n\n### Debian/Ubuntu\n\n```bash\n$ apt install python3-click\n$ apt install aapt # for dump-arsc.py & dump-axml.py\n$ apt install zipalign # for realignment; see examples\n```\n\n## License\n\n[](https://www.gnu.org/licenses/gpl-3.0.html)\n\n<!-- vim: set tw=70 sw=2 sts=2 et fdm=marker : -->\n",
"bugtrack_url": null,
"license": "GPLv3+",
"summary": "scripts to make android apks reproducible",
"version": "0.2.5",
"project_urls": {
"Homepage": "https://github.com/obfusk/reproducible-apk-tools"
},
"split_keywords": [
"android",
"apk",
"reproducible"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5fc2e3246038d9630a37647abb23828622e71a120200e8da696ac6c2af918356",
"md5": "794f1e70c7a034f497f1191872a02c45",
"sha256": "efa85dc7eb83b10f1bbcb4f1a3e75995ed9e934aec5ee88dbd79ec829a40c27b"
},
"downloads": -1,
"filename": "repro_apk-0.2.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "794f1e70c7a034f497f1191872a02c45",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 62697,
"upload_time": "2024-03-21T15:41:44",
"upload_time_iso_8601": "2024-03-21T15:41:44.310555Z",
"url": "https://files.pythonhosted.org/packages/5f/c2/e3246038d9630a37647abb23828622e71a120200e8da696ac6c2af918356/repro_apk-0.2.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a024d8db30111e6a70ddd54063d61c6c312667dee42b8ef4554877e4255a00d8",
"md5": "7153787f29f50b790aedbc164bfe325c",
"sha256": "ab11a4f75e7dc387495a23250c6dabe6c97d49846e38ef4d95f60611080a839d"
},
"downloads": -1,
"filename": "repro-apk-0.2.5.tar.gz",
"has_sig": false,
"md5_digest": "7153787f29f50b790aedbc164bfe325c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 56718,
"upload_time": "2024-03-21T15:41:45",
"upload_time_iso_8601": "2024-03-21T15:41:45.886027Z",
"url": "https://files.pythonhosted.org/packages/a0/24/d8db30111e6a70ddd54063d61c6c312667dee42b8ef4554877e4255a00d8/repro-apk-0.2.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-21 15:41:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "obfusk",
"github_project": "reproducible-apk-tools",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "repro-apk"
}
JSON
