# 🚨 Quick Install 🚨
```
pip install unitypackff
```
# UnityPackFF
UnityPackFF is a fork of [UnityPack](https://github.com/HearthSim/UnityPack/) specialized for working with FusionFall assets.
It allows for extraction and limited manipulation of FusionFall assets and might become the basis of a more extensive modding toolkit in the future.
Maintained compatibility with other asset format versions is not guaranteed.
Note: This project is meant to be used by people familiar with the technologies involved, not by end users.
Please do not bother me with questions about Python, FF modding or the nature of binary data.
Please take the time to figure out how to accomplish what you want yourself if you feel you're up to the task of doing so; especially if it's not something that's already been documented as doable.
## Details
This project was forked from commit `d9ce99fa` of UnityPack.
The upstream readme has been renamed to [README.UP.md](README.UP.md).
It currently supports working with all asset bundles from the original game, as well as all asset bundles from FusionFall Retro.
Note that this repository is a loose collection of patches and scripts I had originally written for my own use and have only slightly cleaned up before publishing.
Do read the scripts in `bin` before executing them to make sure you understand what they do.
Tweaking them yourself is to be considered part of standard usage.
I have not tested any of this on Windows myself, though it *should* all work just fine.
Dependencies can be installed using `pip`, as usual:
```
$ sudo pip3 install -r requirements.txt
```
The library itself can also be installed with `setup.py`, like most Python software.
The recommended approach is to install the library in ["Development Mode"](https://setuptools.readthedocs.io/en/latest/userguide/development_mode.html), like so:
```
$ sudo python3 setup.py develop
```
This places only a reference into your system's package directory as opposed to copying the entire library into a directory that isn't user-writable.
This way you can keep modifying the code in your repository directory without having to reinstall the entire thing after every change.
Note that this doesn't seem to work on Windows if Python was installed from the Microsoft Store.
Another option is to just set the `PYTHONPATH` environment variable to this directory.
Current features:
* Extracting models, textures, audio, compiled shaders, terrain data
* Reading/dumping the XDT data from TableData asset bundles in-place
* Looking up offsets in asset bundles to modify data using hex editors or scripts
* Modifying terrain data (albeit imperfectly at the moment)
## Asset structure
Note on terminology: Unity appears to have a lot of overloaded terms for these things, making them difficult to keep track of.
There's three different definitions for what an AssetBundle is, for instance.
I try to be mostly consistent with the terms I use, but when in doubt, consider the context.
As explained in the [OpenFusion readme](https://github.com/OpenFusionProject/OpenFusion/#architecture), the web gateway directs the player's browser to download the main `unity3d` bundle which contains the game's essential assets (`mainData` and `sharedAssets0.asset`) along with all the client's C# DLLs.
Apart from those two assets, all the others will be downloaded in separate bundles from the address in `assetInfo.php` and cached in their extracted forms in `C:\Users\USERNAME\AppData\LocalLow\Unity\Web Player\Cache\FusionFall` (note the space in `Web Player`).
Both the main `unity3d` file and the other assets are transmitted as the same `unity3d` file format, regardless if the file extension is `.unity3d` or `.resourceFile`.
This format is just a trivial LZMA compressed archive containing plain files.
It can be extracted using [quickbms](http://aluigi.altervista.org/quickbms.htm) with the `unity3d_webplayer.bms` script.
The contents of the main bundle were explained in the previous paragraph, the others only contain one asset bundle and one or two metadata files.
Unlike the container format, these asset bundles are not simple file archives.
They store a series of *asset objects*, each of which is a hierarchy of members, some of which are pointers to other objects which are potentially defined in other asset bundles referenced in a table of externals.
Each of these asset objects is structured according to a specific type which is also defined in the asset bundle.
Some of these types are standard (indicated by positive IDs), while others are asset bundle-specific (indicated by negative IDs).
Disunity's wiki has some [notes](https://github.com/ata4/disunity/wiki/Serialized-file-format) on the structure of these asset bundles.
I've also [mirrored](https://github.com/dongresource/UnityPackFF/wiki/Serialized-file-format) them in this repository's wiki for convenience.
Each of these asset bundle files is either a Scene asset bundle or a regular (resource) asset bundle.
Scene asset bundles have a `SceneSettings` object as their first asset object member, the archives they're stored in use the `.unity3d` file extension, and their names are of the format `Map_XX_YY.unity3d`.
The filenames of the asset bundles themselves are of the format `BuildPlayer-Map_XX_YY`, accompanied by another asset bundle with the same name, but ending in `.sharedAssets`.
In FusionFall, each map tile is a scene.
Regular asset bundles have an `AssetBundle` object as their first asset object member and the archives they're stored in use the `.resourceFile` file extension.
For each map tile, there's a `DongResources_XX_YY.resourceFile` that contains the assets that tile makes use of.
All the remaining asset bundles (CharTexture, CharacterCreation, CharacterSelection, TableData, NpcTexture, etc) are also of this type.
This library is meant to operate on the CustomAssetBundles after they've already been extracted from their respective `.unity3d` archives.
You can either copy out your cache directory after having played the game (and gone to the Past zone) or you can extract the asset files from their `.unity3d` archives yourself using a tool like `quickbms`.
This can be automated for every asset bundle with a shell one-liner or any similar means.
## Usage
Depending on what you are trying to achieve, the library can be used either interactively or through one of the provided scripts (or one's own, of course).
### Interactive use
For improved auto-completion I suggest using `ipython` instead of the raw Python interpreter.
Start by importing the `Asset` class from the library, opening an asset (for reading) as a binary file and creating a new `Asset` object:
```python
from unitypack.asset import Asset
f = open('tabledata_2eresourceFile/CustomAssetBundle-1dca92eecee4742d985b799d8226666d', 'rb')
tabledata = Asset.from_file(f)
```
UnityPack also has a `unitypack.load()` function that operates directly on the enclosing `.unity3d` bundle, but that method does not currently set the correct version in the asset bundle, and is therefore unusable.
After you've opened up the `tabledata` asset bundle, you can access its asset objects through its `tabledata.objects` member.
Note that `objects` is a dict with integer keys, not a list. The first member is usually 1 in these.
Most asset bundles have a large number of top-level objects, but TableData only has 9.
Of those, the seventh, `xdtdata`, is the most interesting.
We can read it out into a variable with:
```python
xdtdata = tabledata.objects[7].contents
```
Make sure to read large objects like this one into a variable, as accessing them directly from the asset incurs disk IO every time, which makes browsing the table sluggish.
The way these Unity objects are structured, everything in the `objects` dict is a Python object of type `ObjectInfo`.
The `ObjectInfo` class's `contents` member is the actual asset object, which can be an instance of either one of UnityPack's specialized classes (`Texture2D`, `AudioClip`, `Transform`, etc), or a `FFOrderedDict` object by default.
In case of one of the former, the underlying `FFOrderedDict` can usually be accessed by the specialized class's `_obj` member (like `asset.objects[...].contents._obj`).
You can browse these objects interactively as you would any other Python data structures.
The smaller `FFOrderedDict`s like `Transform`s and `GameObject`s you can print whole, but larger ones (like the aforementioned `xdtdata`) would just overflow your terminal.
You can traverse those by printing them little by little; printing only the keys to `dict`s and checking the lengths of `list`s before indexing them.
`xdtdata` in particular consists of two depths of asset objects (`FFOrderedDict`s) followed by a list of asset objects with only primitive members.
Some of those members are integer indexes into other XDT tables, often the ones in the same subcategory.
So you can traverse the XDT like so:
```python
>>> xdtdata.keys()
odict_keys([..., 'm_pPantsItemTable', 'm_pShirtsItemTable', 'm_pShoesItemTable', 'm_pWeaponItemTable', 'm_pVehicleItemTable', ...])
>>> xdtdata['m_pWeaponItemTable'].keys()
odict_keys(['m_pItemData', 'm_pItemStringData', 'm_pItemIconData', 'm_pItemMeshData', 'm_pItemSoundData'])
>>> len(len(xdtdata['m_pWeaponItemTable']['m_pItemData']))
687
>>> xdtdata['m_pWeaponItemTable']['m_pItemData'][5]
FFOrderedDict([('m_iItemNumber', 5),
('m_iItemName', 5),
('m_iComment', 5),
('m_iTradeAble', 1),
('m_iItemPrice', 1090),
('m_iItemSellPrice', 273),
('m_iSellAble', 1),
('m_iStackNumber', 1),
...
>>> xdtdata['m_pWeaponItemTable']['m_pItemStringData'][5]
FFOrderedDict([('m_strName', 'Pewter Apple of Discord'),
('m_strComment',
'Create chaos wherever you go with this powerful thrown weapon.'),
('m_strComment1', ' '),
('m_strComment2', ''),
('m_iExtraNumber', 0)])
```
You can loop over these lists to match index numbers with objects when they aren't obvious.
If you want to modify any of these values, you can get the index of any `FFOrderedDict` from its `index` member, as well as the index of any of its members like so:
```python
xdtdata['m_pWeaponItemTable']['m_pItemData'][5].getmemboffset('m_iItemPrice')
```
You can then open the asset bundle in a hex editor and make whatever change you want at that offset; or write a script for more sophisticated modifications.
Some objects contain `ObjectPointer`s that point to any other object in either the same asset bundle (if `file_id` is 0), or one of the ones linked in `asset.asset_ref`.
`path_id` is the index into the `asset.objects` of the asset bundle `file_id` points to.
If the `base_path` in your asset's `UnityEnvironment` is set up correctly, the `resolve()` method can automatically dereference these asset pointers.
### Scripts
#### `unityextract.py`
---
This is a modified version of the same script from upstream UnityPack.
Like all the other scripts, it has been made robust against exceptions, ie. failure to extract one asset will not halt extraction of the rest.
It rips any supported assets into the current working directory, constructing filenames according to the name fields of each asset object.
Invoke like:
```python
/path/to/unityextract.py --all --as-asset CustomAssetBundle-...
```
It's useful for ripping assets on a smaller scale, and for ripping assets from Scene asset bundles which `ffextract.py` isn't compatible with.
#### `unity2yaml.py`
---
From upstream UnityPack. Mostly irrelevant here.
#### `list_contents.py`, `list_assetbundle.py`, `list_ab_alt.py`
---
These scripts generate textual listings of the asset objects in a given asset bundle.
`list_contents.py` lists the type and name (if any) of every single asset object.
`list_assetbundle.py` lists asset objects according to the asset bundle's `AssetBundle` (first member).
It's only compatible with non-Scene asset bundles, but has the advantage of listing proper asset filenames instead of internal object names.
It also lists the preload indexes of each of those objects, which isn't actually all that useful, so `list_ab_alt.py` is preferred because it instead lists the `path_id` and `file_id` numbers of each entry.
I used these with a few shell one-liners to construct a directory hierarchy that mirrors the layout of the asset cache, but has asset object listings instead of the asset bundles themselves.
This makes it easy to figure out which bundle a given asset is stored in using good ol' `grep`.
In hindsight, I probably should have just written a Python script to do that, instead of enduring the overhead of invoking the Python interpreter to execute these scripts in every loop iteration, but oh well.
#### `show_gameobject.py`
---
This script takes an asset bundle and the full filename of an asset that can be found in the `AssetBundle` member of that asset bundle.
It traverses `ObjectPointer`s from the `GameObject` that the `AssetBundle` points to and draws a tree to standard output with the `path_id`s and names of everything it encounters.
Output puts emphasis on `Mesh` objects.
This is meant to visualize the structure of model-related assets; to aid in writing/modifying `ffextract.py`.
Uninformed graph traversal probably isn't the best way to accomplish this, but there's multiple ways a `GameObject` can connect to a `Mesh`, so this is the lazy way of making sure we hopefully find all of 'em.
The large number of `GameObject`s and `Transform`s in the output are actually bones in the model's armature.
I assume there's a reliable way to avoid traversing the bone tree altogether, but I've yet to find it.
This script takes an optional directory argument that will be used as the `UnityEnvironment`'s `base_path`.
If an asset bundle is being read from within a cache directory (or any other place where the other asset bundles are nearby), passing that directory's path allows the script to follow cross-file `ObjectPointer`s.
#### `ffextract.py`
---
`ffextract.py` is the most significant script as of now.
It extracts all supported asset file formats according to the filenames in each (non-Scene) asset bundle's `AssetBundle` member.
This recreates the directory structure of the game's assets.
File extensions are translated from those of the original formats the developers used to those the library extracts assets into (ex. `.kfm` -> `.obj`, `.mp3` -> `.ogg`, `.psd` -> `.png`).
Because `AssetBundle`s never point to `Mesh` objects directly, and instead point to `GameObject`s that eventually have a `Mesh` in one of their children, we must traverse those children until we find the meshes we want to export to `.obj` files.
`Mesh` objects are usually children of some container object like a `SkinnedMeshRenderer` or a `MeshFilter`.
Because filenames are a property of each `GameObject`, not each mesh, when an object contains multiple meshes, each is extracted as a separate file with a colon followed by a number before the file extension.
An alternative would be to put them all into the same file as submeshes, but this would be a non-trivial task right now.
Quality contributions are welcome.
Currently, only `SkinnedMeshRenderer`s are being ripped from, since most models of interest are in those, and enabling `MeshFilter` extraction causes a lot of garbage to be extracted.
If I recall correctly, some real (non-junk) meshes are actually children of `MeshFilter` objects, so if you wish to rip those anyway, uncomment the `MeshFilter` `if` block in `gameobject_recurse()` along with the mesh limit block.
Be warned that each bone in an object's armature links to a `MeshFilter` with a plain cube, so if `MeshFilter`s are being naively extracted, countless garbage `.obj` files will be generated, wasting tons of disk space and immensely slowing the extraction process.
There is almost certainly a way to skip the armature or the uninformed traversal of the asset object graph entirely, but I haven't had the opportunity to look for one as of yet.
To invoke `ffextract.py`, pass it your asset cache directory (or a similar dir with all the assets) and a (usually empty) output directory.
The input directory will be used as the `UnityEnvironment`'s `base_path`, so `ObjectPointer`s will work properly.
This is important because the `AssetBundle`s often reference asset objects located in files other than their own.
The script will generate an output directory structure that mimics that of the FF developers (according to the filenames in each `AssetBundle` object).
`GameObject` traversal is considerably redundant and not terribly efficient, so the extraction process takes a while. Be patient.
Tip: When importing the resulting `obj` files into Blender, you'll want to select them and "Clear Custom Split Normals Data" either in the "Geometry Data" tab in "Object Data Properties"; or by searching for that option with F3 (or Space, in Blender 2.7x).
This fixes a shading issue. It's possible the normals aren't being ripped correctly.
You'll also want to change the Forward or Up axes in the file selection menu when importing the `obj` file so the models are imported upright without you having to rotate them manually.
#### `dump_terrain.py`, `replace_terrain.py`
---
These are currently imperfect terrain modding scripts.
`dump_terrain.py` was initially written by CPunch.
You supply `dump_terrain.py` with a "dongresources" asset bundle and an output file name and it will print the hexadecimal offset of that map tile's terrain data and extract it as a grayscale PNG file.
Note that the image will be rotated differently than in the game.
This image can be edited in any image editor. Darker shades result in lower, while brighter ones in higher terrain.
You could also use them in Blender in a displacement modifier on a subdivided plane for more natural editing, and then bake it back to an image for export, but getting this to work in a pixel-perfect manner would be tricky.
The modified image can be supplied to `replace_terrain.py` along with the offset that `dump_terrain.py` had printed; as well as the asset bundle to re-inject the terrain data into.
The target asset bundle will be edited in-place, so make sure to make a backup beforehand.
Unfortunately, the terrain data uses 16-bit integers for each pixel and this format doesn't seem to map cleanly to any well-supported pixel format.
Currently, `dump_terrain.py` maps the 16-bit value range to a single byte (and stores it thrice, once for each byte of an RGB pixel), resulting in rounding errors which very clearly mangle the terrain even if no changes were manually introduced.
For that reason, these scripts are only good for testing and goofing around until we can possibly write a custom blender import script to allow terrain modification without messing the entire map tile up.
#### `proto_extract.py`, `proto_mesh_extract.py`
---
These scripts were prototypes I wrote while figuring out the asset bundle and compressed mesh formats, respectively.
Their logic is already in the library as of my first commit to this project (`7ca8bcb5`).
Both were written with reference to the source code of older versions of Disunity.
They might be of use to anyone trying to implement generation of new FF-compatible asset bundles or modification of existing ones.
`proto_extract.py` only loads the asset bundle's type tree so it's browsable interactively if the script is `import`ed or run with `ipython`'s `%run` command.
It does not decode the asset objects themselves.
Raw data
{
"_id": null,
"home_page": "https://github.com/SirDank/UnityPackFF",
"name": "unitypackff",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "unity,unitypack,unitypackff",
"author": "dongresource",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/4c/dc/3091df480731239612e04eece5d8f71f2b95a3e5f82632899c0e60ba7db7/unitypackff-1.0.3.tar.gz",
"platform": null,
"description": "# \ud83d\udea8 Quick Install \ud83d\udea8\r\n\r\n```\r\npip install unitypackff\r\n```\r\n\r\n# UnityPackFF\r\n\r\nUnityPackFF is a fork of [UnityPack](https://github.com/HearthSim/UnityPack/) specialized for working with FusionFall assets.\r\nIt allows for extraction and limited manipulation of FusionFall assets and might become the basis of a more extensive modding toolkit in the future.\r\n\r\nMaintained compatibility with other asset format versions is not guaranteed.\r\n\r\nNote: This project is meant to be used by people familiar with the technologies involved, not by end users.\r\nPlease do not bother me with questions about Python, FF modding or the nature of binary data.\r\nPlease take the time to figure out how to accomplish what you want yourself if you feel you're up to the task of doing so; especially if it's not something that's already been documented as doable.\r\n\r\n## Details\r\n\r\nThis project was forked from commit `d9ce99fa` of UnityPack.\r\nThe upstream readme has been renamed to [README.UP.md](README.UP.md).\r\n\r\nIt currently supports working with all asset bundles from the original game, as well as all asset bundles from FusionFall Retro.\r\n\r\nNote that this repository is a loose collection of patches and scripts I had originally written for my own use and have only slightly cleaned up before publishing.\r\nDo read the scripts in `bin` before executing them to make sure you understand what they do.\r\nTweaking them yourself is to be considered part of standard usage.\r\n\r\nI have not tested any of this on Windows myself, though it *should* all work just fine.\r\n\r\nDependencies can be installed using `pip`, as usual:\r\n\r\n```\r\n$ sudo pip3 install -r requirements.txt\r\n```\r\n\r\nThe library itself can also be installed with `setup.py`, like most Python software.\r\nThe recommended approach is to install the library in [\"Development Mode\"](https://setuptools.readthedocs.io/en/latest/userguide/development_mode.html), like so:\r\n\r\n```\r\n$ sudo python3 setup.py develop\r\n```\r\n\r\nThis places only a reference into your system's package directory as opposed to copying the entire library into a directory that isn't user-writable.\r\nThis way you can keep modifying the code in your repository directory without having to reinstall the entire thing after every change.\r\nNote that this doesn't seem to work on Windows if Python was installed from the Microsoft Store.\r\nAnother option is to just set the `PYTHONPATH` environment variable to this directory.\r\n\r\nCurrent features:\r\n\r\n* Extracting models, textures, audio, compiled shaders, terrain data\r\n* Reading/dumping the XDT data from TableData asset bundles in-place\r\n* Looking up offsets in asset bundles to modify data using hex editors or scripts\r\n* Modifying terrain data (albeit imperfectly at the moment)\r\n\r\n## Asset structure\r\n\r\nNote on terminology: Unity appears to have a lot of overloaded terms for these things, making them difficult to keep track of.\r\nThere's three different definitions for what an AssetBundle is, for instance.\r\nI try to be mostly consistent with the terms I use, but when in doubt, consider the context.\r\n\r\nAs explained in the [OpenFusion readme](https://github.com/OpenFusionProject/OpenFusion/#architecture), the web gateway directs the player's browser to download the main `unity3d` bundle which contains the game's essential assets (`mainData` and `sharedAssets0.asset`) along with all the client's C# DLLs.\r\nApart from those two assets, all the others will be downloaded in separate bundles from the address in `assetInfo.php` and cached in their extracted forms in `C:\\Users\\USERNAME\\AppData\\LocalLow\\Unity\\Web Player\\Cache\\FusionFall` (note the space in `Web Player`).\r\n\r\nBoth the main `unity3d` file and the other assets are transmitted as the same `unity3d` file format, regardless if the file extension is `.unity3d` or `.resourceFile`.\r\nThis format is just a trivial LZMA compressed archive containing plain files.\r\nIt can be extracted using [quickbms](http://aluigi.altervista.org/quickbms.htm) with the `unity3d_webplayer.bms` script.\r\nThe contents of the main bundle were explained in the previous paragraph, the others only contain one asset bundle and one or two metadata files.\r\n\r\nUnlike the container format, these asset bundles are not simple file archives.\r\nThey store a series of *asset objects*, each of which is a hierarchy of members, some of which are pointers to other objects which are potentially defined in other asset bundles referenced in a table of externals.\r\nEach of these asset objects is structured according to a specific type which is also defined in the asset bundle.\r\nSome of these types are standard (indicated by positive IDs), while others are asset bundle-specific (indicated by negative IDs).\r\n\r\nDisunity's wiki has some [notes](https://github.com/ata4/disunity/wiki/Serialized-file-format) on the structure of these asset bundles.\r\nI've also [mirrored](https://github.com/dongresource/UnityPackFF/wiki/Serialized-file-format) them in this repository's wiki for convenience.\r\n\r\nEach of these asset bundle files is either a Scene asset bundle or a regular (resource) asset bundle.\r\nScene asset bundles have a `SceneSettings` object as their first asset object member, the archives they're stored in use the `.unity3d` file extension, and their names are of the format `Map_XX_YY.unity3d`.\r\nThe filenames of the asset bundles themselves are of the format `BuildPlayer-Map_XX_YY`, accompanied by another asset bundle with the same name, but ending in `.sharedAssets`.\r\nIn FusionFall, each map tile is a scene.\r\nRegular asset bundles have an `AssetBundle` object as their first asset object member and the archives they're stored in use the `.resourceFile` file extension.\r\nFor each map tile, there's a `DongResources_XX_YY.resourceFile` that contains the assets that tile makes use of.\r\nAll the remaining asset bundles (CharTexture, CharacterCreation, CharacterSelection, TableData, NpcTexture, etc) are also of this type.\r\n\r\nThis library is meant to operate on the CustomAssetBundles after they've already been extracted from their respective `.unity3d` archives.\r\nYou can either copy out your cache directory after having played the game (and gone to the Past zone) or you can extract the asset files from their `.unity3d` archives yourself using a tool like `quickbms`.\r\nThis can be automated for every asset bundle with a shell one-liner or any similar means.\r\n\r\n## Usage\r\n\r\nDepending on what you are trying to achieve, the library can be used either interactively or through one of the provided scripts (or one's own, of course).\r\n\r\n### Interactive use\r\n\r\nFor improved auto-completion I suggest using `ipython` instead of the raw Python interpreter.\r\n\r\nStart by importing the `Asset` class from the library, opening an asset (for reading) as a binary file and creating a new `Asset` object:\r\n\r\n```python\r\nfrom unitypack.asset import Asset\r\nf = open('tabledata_2eresourceFile/CustomAssetBundle-1dca92eecee4742d985b799d8226666d', 'rb')\r\ntabledata = Asset.from_file(f)\r\n```\r\n\r\nUnityPack also has a `unitypack.load()` function that operates directly on the enclosing `.unity3d` bundle, but that method does not currently set the correct version in the asset bundle, and is therefore unusable.\r\n\r\nAfter you've opened up the `tabledata` asset bundle, you can access its asset objects through its `tabledata.objects` member.\r\nNote that `objects` is a dict with integer keys, not a list. The first member is usually 1 in these.\r\nMost asset bundles have a large number of top-level objects, but TableData only has 9.\r\nOf those, the seventh, `xdtdata`, is the most interesting.\r\nWe can read it out into a variable with:\r\n\r\n```python\r\nxdtdata = tabledata.objects[7].contents\r\n```\r\n\r\nMake sure to read large objects like this one into a variable, as accessing them directly from the asset incurs disk IO every time, which makes browsing the table sluggish.\r\n\r\nThe way these Unity objects are structured, everything in the `objects` dict is a Python object of type `ObjectInfo`.\r\nThe `ObjectInfo` class's `contents` member is the actual asset object, which can be an instance of either one of UnityPack's specialized classes (`Texture2D`, `AudioClip`, `Transform`, etc), or a `FFOrderedDict` object by default.\r\nIn case of one of the former, the underlying `FFOrderedDict` can usually be accessed by the specialized class's `_obj` member (like `asset.objects[...].contents._obj`).\r\n\r\nYou can browse these objects interactively as you would any other Python data structures.\r\nThe smaller `FFOrderedDict`s like `Transform`s and `GameObject`s you can print whole, but larger ones (like the aforementioned `xdtdata`) would just overflow your terminal.\r\nYou can traverse those by printing them little by little; printing only the keys to `dict`s and checking the lengths of `list`s before indexing them.\r\n`xdtdata` in particular consists of two depths of asset objects (`FFOrderedDict`s) followed by a list of asset objects with only primitive members.\r\nSome of those members are integer indexes into other XDT tables, often the ones in the same subcategory.\r\n\r\nSo you can traverse the XDT like so:\r\n\r\n```python\r\n>>> xdtdata.keys()\r\nodict_keys([..., 'm_pPantsItemTable', 'm_pShirtsItemTable', 'm_pShoesItemTable', 'm_pWeaponItemTable', 'm_pVehicleItemTable', ...])\r\n\r\n>>> xdtdata['m_pWeaponItemTable'].keys()\r\nodict_keys(['m_pItemData', 'm_pItemStringData', 'm_pItemIconData', 'm_pItemMeshData', 'm_pItemSoundData'])\r\n\r\n>>> len(len(xdtdata['m_pWeaponItemTable']['m_pItemData']))\r\n687\r\n\r\n>>> xdtdata['m_pWeaponItemTable']['m_pItemData'][5]\r\nFFOrderedDict([('m_iItemNumber', 5),\r\n ('m_iItemName', 5),\r\n ('m_iComment', 5),\r\n ('m_iTradeAble', 1),\r\n ('m_iItemPrice', 1090),\r\n ('m_iItemSellPrice', 273),\r\n ('m_iSellAble', 1),\r\n ('m_iStackNumber', 1),\r\n...\r\n\r\n>>> xdtdata['m_pWeaponItemTable']['m_pItemStringData'][5]\r\nFFOrderedDict([('m_strName', 'Pewter Apple of Discord'),\r\n ('m_strComment',\r\n 'Create chaos wherever you go with this powerful thrown weapon.'),\r\n ('m_strComment1', ' '),\r\n ('m_strComment2', ''),\r\n ('m_iExtraNumber', 0)])\r\n```\r\n\r\nYou can loop over these lists to match index numbers with objects when they aren't obvious.\r\n\r\nIf you want to modify any of these values, you can get the index of any `FFOrderedDict` from its `index` member, as well as the index of any of its members like so:\r\n\r\n```python\r\nxdtdata['m_pWeaponItemTable']['m_pItemData'][5].getmemboffset('m_iItemPrice')\r\n```\r\n\r\nYou can then open the asset bundle in a hex editor and make whatever change you want at that offset; or write a script for more sophisticated modifications.\r\n\r\nSome objects contain `ObjectPointer`s that point to any other object in either the same asset bundle (if `file_id` is 0), or one of the ones linked in `asset.asset_ref`.\r\n`path_id` is the index into the `asset.objects` of the asset bundle `file_id` points to.\r\nIf the `base_path` in your asset's `UnityEnvironment` is set up correctly, the `resolve()` method can automatically dereference these asset pointers.\r\n\r\n### Scripts\r\n\r\n#### `unityextract.py`\r\n\r\n---\r\n\r\nThis is a modified version of the same script from upstream UnityPack.\r\nLike all the other scripts, it has been made robust against exceptions, ie. failure to extract one asset will not halt extraction of the rest.\r\nIt rips any supported assets into the current working directory, constructing filenames according to the name fields of each asset object.\r\nInvoke like:\r\n\r\n```python\r\n/path/to/unityextract.py --all --as-asset CustomAssetBundle-...\r\n```\r\n\r\nIt's useful for ripping assets on a smaller scale, and for ripping assets from Scene asset bundles which `ffextract.py` isn't compatible with.\r\n\r\n#### `unity2yaml.py`\r\n\r\n---\r\n\r\nFrom upstream UnityPack. Mostly irrelevant here.\r\n\r\n#### `list_contents.py`, `list_assetbundle.py`, `list_ab_alt.py`\r\n\r\n---\r\n\r\nThese scripts generate textual listings of the asset objects in a given asset bundle.\r\n\r\n`list_contents.py` lists the type and name (if any) of every single asset object.\r\n`list_assetbundle.py` lists asset objects according to the asset bundle's `AssetBundle` (first member).\r\nIt's only compatible with non-Scene asset bundles, but has the advantage of listing proper asset filenames instead of internal object names.\r\nIt also lists the preload indexes of each of those objects, which isn't actually all that useful, so `list_ab_alt.py` is preferred because it instead lists the `path_id` and `file_id` numbers of each entry.\r\n\r\nI used these with a few shell one-liners to construct a directory hierarchy that mirrors the layout of the asset cache, but has asset object listings instead of the asset bundles themselves.\r\nThis makes it easy to figure out which bundle a given asset is stored in using good ol' `grep`.\r\nIn hindsight, I probably should have just written a Python script to do that, instead of enduring the overhead of invoking the Python interpreter to execute these scripts in every loop iteration, but oh well.\r\n\r\n#### `show_gameobject.py`\r\n\r\n---\r\n\r\nThis script takes an asset bundle and the full filename of an asset that can be found in the `AssetBundle` member of that asset bundle.\r\nIt traverses `ObjectPointer`s from the `GameObject` that the `AssetBundle` points to and draws a tree to standard output with the `path_id`s and names of everything it encounters.\r\nOutput puts emphasis on `Mesh` objects.\r\n\r\nThis is meant to visualize the structure of model-related assets; to aid in writing/modifying `ffextract.py`.\r\nUninformed graph traversal probably isn't the best way to accomplish this, but there's multiple ways a `GameObject` can connect to a `Mesh`, so this is the lazy way of making sure we hopefully find all of 'em.\r\n\r\nThe large number of `GameObject`s and `Transform`s in the output are actually bones in the model's armature.\r\nI assume there's a reliable way to avoid traversing the bone tree altogether, but I've yet to find it.\r\n\r\nThis script takes an optional directory argument that will be used as the `UnityEnvironment`'s `base_path`.\r\nIf an asset bundle is being read from within a cache directory (or any other place where the other asset bundles are nearby), passing that directory's path allows the script to follow cross-file `ObjectPointer`s.\r\n\r\n#### `ffextract.py`\r\n\r\n---\r\n\r\n`ffextract.py` is the most significant script as of now.\r\nIt extracts all supported asset file formats according to the filenames in each (non-Scene) asset bundle's `AssetBundle` member.\r\nThis recreates the directory structure of the game's assets.\r\n\r\nFile extensions are translated from those of the original formats the developers used to those the library extracts assets into (ex. `.kfm` -> `.obj`, `.mp3` -> `.ogg`, `.psd` -> `.png`).\r\n\r\nBecause `AssetBundle`s never point to `Mesh` objects directly, and instead point to `GameObject`s that eventually have a `Mesh` in one of their children, we must traverse those children until we find the meshes we want to export to `.obj` files.\r\n`Mesh` objects are usually children of some container object like a `SkinnedMeshRenderer` or a `MeshFilter`.\r\n\r\nBecause filenames are a property of each `GameObject`, not each mesh, when an object contains multiple meshes, each is extracted as a separate file with a colon followed by a number before the file extension.\r\nAn alternative would be to put them all into the same file as submeshes, but this would be a non-trivial task right now.\r\nQuality contributions are welcome.\r\n\r\nCurrently, only `SkinnedMeshRenderer`s are being ripped from, since most models of interest are in those, and enabling `MeshFilter` extraction causes a lot of garbage to be extracted.\r\nIf I recall correctly, some real (non-junk) meshes are actually children of `MeshFilter` objects, so if you wish to rip those anyway, uncomment the `MeshFilter` `if` block in `gameobject_recurse()` along with the mesh limit block.\r\nBe warned that each bone in an object's armature links to a `MeshFilter` with a plain cube, so if `MeshFilter`s are being naively extracted, countless garbage `.obj` files will be generated, wasting tons of disk space and immensely slowing the extraction process.\r\nThere is almost certainly a way to skip the armature or the uninformed traversal of the asset object graph entirely, but I haven't had the opportunity to look for one as of yet.\r\n\r\nTo invoke `ffextract.py`, pass it your asset cache directory (or a similar dir with all the assets) and a (usually empty) output directory.\r\nThe input directory will be used as the `UnityEnvironment`'s `base_path`, so `ObjectPointer`s will work properly.\r\nThis is important because the `AssetBundle`s often reference asset objects located in files other than their own.\r\nThe script will generate an output directory structure that mimics that of the FF developers (according to the filenames in each `AssetBundle` object).\r\n`GameObject` traversal is considerably redundant and not terribly efficient, so the extraction process takes a while. Be patient.\r\n\r\nTip: When importing the resulting `obj` files into Blender, you'll want to select them and \"Clear Custom Split Normals Data\" either in the \"Geometry Data\" tab in \"Object Data Properties\"; or by searching for that option with F3 (or Space, in Blender 2.7x).\r\nThis fixes a shading issue. It's possible the normals aren't being ripped correctly.\r\nYou'll also want to change the Forward or Up axes in the file selection menu when importing the `obj` file so the models are imported upright without you having to rotate them manually.\r\n\r\n#### `dump_terrain.py`, `replace_terrain.py`\r\n\r\n---\r\n\r\nThese are currently imperfect terrain modding scripts.\r\n`dump_terrain.py` was initially written by CPunch.\r\n\r\nYou supply `dump_terrain.py` with a \"dongresources\" asset bundle and an output file name and it will print the hexadecimal offset of that map tile's terrain data and extract it as a grayscale PNG file.\r\nNote that the image will be rotated differently than in the game.\r\n\r\nThis image can be edited in any image editor. Darker shades result in lower, while brighter ones in higher terrain.\r\nYou could also use them in Blender in a displacement modifier on a subdivided plane for more natural editing, and then bake it back to an image for export, but getting this to work in a pixel-perfect manner would be tricky.\r\n\r\nThe modified image can be supplied to `replace_terrain.py` along with the offset that `dump_terrain.py` had printed; as well as the asset bundle to re-inject the terrain data into.\r\nThe target asset bundle will be edited in-place, so make sure to make a backup beforehand.\r\n\r\nUnfortunately, the terrain data uses 16-bit integers for each pixel and this format doesn't seem to map cleanly to any well-supported pixel format.\r\nCurrently, `dump_terrain.py` maps the 16-bit value range to a single byte (and stores it thrice, once for each byte of an RGB pixel), resulting in rounding errors which very clearly mangle the terrain even if no changes were manually introduced.\r\nFor that reason, these scripts are only good for testing and goofing around until we can possibly write a custom blender import script to allow terrain modification without messing the entire map tile up.\r\n\r\n#### `proto_extract.py`, `proto_mesh_extract.py`\r\n\r\n---\r\n\r\nThese scripts were prototypes I wrote while figuring out the asset bundle and compressed mesh formats, respectively.\r\nTheir logic is already in the library as of my first commit to this project (`7ca8bcb5`).\r\nBoth were written with reference to the source code of older versions of Disunity.\r\n\r\nThey might be of use to anyone trying to implement generation of new FF-compatible asset bundles or modification of existing ones.\r\n\r\n`proto_extract.py` only loads the asset bundle's type tree so it's browsable interactively if the script is `import`ed or run with `ipython`'s `%run` command.\r\nIt does not decode the asset objects themselves.\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "UnityPackFF is a fork of UnityPack specialized for working with FusionFall assets",
"version": "1.0.3",
"split_keywords": [
"unity",
"unitypack",
"unitypackff"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3474d9cf50025513836e4cc8755ecfd181bef2031caf0358b39bc4bf3318b000",
"md5": "ea945a7ed8c9bb997bb7b3c38cf1ac75",
"sha256": "aac0f37da5a4bc981339c639d5ecb5e6f5596c8a932d9f8aeb51661dc4ef73b7"
},
"downloads": -1,
"filename": "unitypackff-1.0.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ea945a7ed8c9bb997bb7b3c38cf1ac75",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 121492,
"upload_time": "2023-04-12T14:35:13",
"upload_time_iso_8601": "2023-04-12T14:35:13.689113Z",
"url": "https://files.pythonhosted.org/packages/34/74/d9cf50025513836e4cc8755ecfd181bef2031caf0358b39bc4bf3318b000/unitypackff-1.0.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4cdc3091df480731239612e04eece5d8f71f2b95a3e5f82632899c0e60ba7db7",
"md5": "2385395e5370dd52db43ceb3159fd76e",
"sha256": "a9b7a6c2be42c57d23d82e8d604e640273769d1e150f1cf2d811ebfa188c20c7"
},
"downloads": -1,
"filename": "unitypackff-1.0.3.tar.gz",
"has_sig": false,
"md5_digest": "2385395e5370dd52db43ceb3159fd76e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 115693,
"upload_time": "2023-04-12T14:35:16",
"upload_time_iso_8601": "2023-04-12T14:35:16.583464Z",
"url": "https://files.pythonhosted.org/packages/4c/dc/3091df480731239612e04eece5d8f71f2b95a3e5f82632899c0e60ba7db7/unitypackff-1.0.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-12 14:35:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "SirDank",
"github_project": "UnityPackFF",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "unitypackff"
}
JSON
