| Name | FATtools JSON |
| Version |
1.0.36
JSON |
| download |
| home_page | None |
| Summary | Pure python tools for accessing FAT filesystem images and disks |
| upload_time | 2024-04-16 08:26:13 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.5 |
| license | GPL |
| keywords |
fat
disk
image
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
FATtools
========
Install from PyPI using `pip install FATtools` (easier) or downloading the source code (or the released packages) from here.
Born to re-sort directory entries in a FAT32 root table to cope with some hardware MP3 players' limits, it now provides full read/write support in Python 3 (both 32- and 64-bit) for FAT12/16/32 and exFAT filesystems, for hacking and recovering purposes.
Moreover:
- it is able to open disk partitioned with both MBR or GPT and to help in partitioning
- it can transparently create, read and write Dynamic and Differencing VHD, VHDX[1], VDI and VMDK disk images
- it can convert disk images between different virtual formats and clone disks
- it can handle RAW disk images and BytesIO "RamDisk" images, also.
- it supports 4K sectors
- it can handle large clusters (128K, 256K) with FAT formats[2]
- it is able to merge Differencing VHDs
Following features are implemented (mostly in Python, with a few ctypes calls to handle Win32 disks natively; compatibility with Linux is not regularly tested):
- sector aligned read/writes with both file images and real disks
- sector based caching mechanism (for both reading and writing) to speed-up FAT and directory table operations
- run length encoded map (with tuples and dictionaries) of free clusters, free directory slots, allocated cluster chains
- transparent reading and writing of FAT12/16/32 and exFAT filesystems with FS boot-sector auto recognizer
- MBR and GPT partitions handling
- Long File Name and Unicode support
- tools to open, create, rename, list and delete files and directories, and to partition disks
- facilities to sort, clean and shrink directory tables and to wipe (zero) free space
- file fragmentation calculator
- mkfat tool to properly (partition and) apply a FAT12/16/32 or exFAT filesystem to a block device (file or disk) and let CHKDSK be happy with it (included exFAT compressed Up-Case table generator)
*Obviously, since a filesystem is an extremely complex and delicate matter, and big bugs may lay around, you'll USE IT TOTALLY AT YOUR OWN RISK!* But it seems quite stable and useable, now.
The most fragile area (and, thus, subject to bugs) was the caching mechanism, that operates in different ways:
- intercepting small I/O (<= 512 bytes), which is cached in a small circular buffer. Bigger I/O bypasses the cache; when the cache is full, all dirty sectors are committed to disk and the cache buffer is zeroed. Sectors and buffers are paired with Python builtin dictionaries: this permits a good (from a Pythonic perspective) I/O speed during FAT and directory tables access;
- maintaining a dictionary of pre-decoded FAT indexes, to improve the speed of repetitive access to cluster chains;
- maintaining a dictionary of short and long names (paired with their respective directory slots) for each directory table, to speed up searches and updates in directory tables;
- maintaining a RLE map of free clusters, free directory slots and allocated cluster chains, to dramatically improve speed of allocation and file access.
*Actually, the I/O speed is closer to system's one.*
Code is GPLed (look at GPL.TXT).
[1] VHDX Log support is actually limited to replaying capability.
[2] Actually, to say, one can partition with GPT an 8 TB VHDX with 4K sectors and format with FAT32 and happily use it under Windows 11. However, Windows 11 CHKDSK reports no more than 4 TB _bytes_ (while it counts _clusters_ correctly). Also, FORMAT itself can't apply such legitimate FAT32 format to an 8 TB disk.
# At a glance
The package installs a `fattools` script, you can use this to perform simple command line operations.
- to create a dynamic 8TB VHDX disk image with a single GPT partition and format it with exFAT:
```
fattools mkvdisk -s 8T --large-sectors image.vhdx
fattools mkfat -t exfat -p gpt image.vhdx
```
- to create a differencing VDI disk image:
```
fattools mkvdisk -b image.vdi delta.vdi
```
- to wipe free space in an (ex)FAT formatted disk, zeroing all free clusters:
```
fattools wipe image.vhd
```
- to convert a RAW disk image into a Dynamic VHD (so implicitly virtualizing zeroed data blocks):
```
fattools imgclone image.raw image.vhd
```
Please note that resulting image size can get reduced if: 1) volume(s) is/are defragmented; 2) directory tables are cleaned and shrunk; 3) the free space has been wiped (zeroed) before.
- to capture a physical drive (Windows) to a Dynamic VHD:
```
fattools imgclone \\.\PhysicalDrive2 image.vhd
```
- to list contents in a disk image, copy items to/from it, display and erase them:
```
fattools ls image1.vhd/py* image2.vdi/py*
fattools cp C:\Python39\Lib\site-packages image.vhd/Python39/Lib
fattools cp image.vhd/Python39 C:\ProgramData
fattools cat image.vhd/readme.txt
fattools rm image.vhd/Python39
```
- to open an existing plain or VHD disk image, or real disk:
```
# -*- coding: cp1252 -*-
from FATtools.Volume import *
disk = vopen('MyDiskImage.img', 'r+b', 'disk')
```
- to make a single GPT partition from all disk space:
```
from FATtools import partutils
gpt = partutils.partition(disk)
```
- to format such partition with the exFAT file system:
```
from FATtools import mkfat, Volume
part = Volume.vopen('MyDiskImage.img', 'r+b', 'partition0')
mkfat.exfat_mkfs(part, part.size)
```
- to order items inside directory tables easily, with GUI and drag support:
```
fattools reordergui
```
- to order root directory table in USB drive X (scripting):
```
# -*- coding: cp1252 -*-
from FATtools.Volume import *
# Assuming we have DirA, DirB, DirC in this disk order into X:
root = vopen('X:', 'r+b')
new_order = '''DirB
DirC
DirA'''
root._sortby.fix = new_order.split('\n') # uses built-in directory sort algorithm
root.sort(root._sortby) # user-defined order, in _sortby.fix list
root.sort() # default ordering (alphabetical)
```
- mixed access with Python and FATtools from the same script:
```
# -*- coding: cp1252 -*-
from FATtools.Volume import vopen, vclose
from FATtools.mkfat import exfat_mkfs
from os.path import join
import os
real_fat_fs = 'F:' # replace with mount point of your file system
# Open and format with FATtools
fs = vopen(real_fat_fs, 'r+b',what='disk')
exfat_mkfs(fs, fs.size)
vclose(fs)
# Write some files with Python and list them
T = ('c','a','b','d')
for t in T:
open(join(real_fat_fs, t+'.txt'), 'w').write('This is a sample "%s.txt" file.'%t)
print(os.listdir(real_fat_fs+'/'))
# Open again, and sort root with FATtools
fs = vopen(real_fat_fs, 'r+b')
fs.sort()
vclose(fs)
# Check new table order with Python
print(os.listdir(real_fat_fs+'/'))
```
- (almost) same as above:
```
# -*- coding: cp1252 -*-
from FATtools.Volume import vopen, vclose
from FATtools.mkfat import exfat_mkfs
from FATtools.partutils import partition
# Open & create GPT partition
o = vopen('\\\\.\\PhysicalDrive1', 'r+b',what='disk')
print('Partitioning...')
partition(o, 'mbr')
vclose(o)
# Reopen and format with EXFAT
o = vopen('\\\\.\\PhysicalDrive1', 'r+b',what='partition0')
print('Formatting...')
exfat_mkfs(o, o.size)
vclose(o) # auto-close partition AND disk
# Reopen FS and write
print('Writing...')
o = vopen('\\\\.\\PhysicalDrive1', 'r+b')
# Write some files with FATtools and sort them
T = ('c','a','b','d')
for t in T:
f = o.create(t+'.txt')
f.write(b'This is a sample "%s.txt" file.'%bytes(t,'ascii'))
f.close()
o.sort()
vclose(o)
```
Please look inside 'samples' directory for more usage samples.
Raw data
{
"_id": null,
"home_page": null,
"name": "FATtools",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.5",
"maintainer_email": null,
"keywords": "FAT, disk, image",
"author": null,
"author_email": "maxpat78 <maxpat78@yahoo.it>",
"download_url": "https://files.pythonhosted.org/packages/d7/4a/d102f5c5bb3068e954b206b6ffc1e018373cba1a0ef94f6b9af41332ccfe/fattools-1.0.36.tar.gz",
"platform": null,
"description": "FATtools\n========\n\nInstall from PyPI using `pip install FATtools` (easier) or downloading the source code (or the released packages) from here.\n\nBorn to re-sort directory entries in a FAT32 root table to cope with some hardware MP3 players' limits, it now provides full read/write support in Python 3 (both 32- and 64-bit) for FAT12/16/32 and exFAT filesystems, for hacking and recovering purposes.\n\nMoreover:\n- it is able to open disk partitioned with both MBR or GPT and to help in partitioning\n- it can transparently create, read and write Dynamic and Differencing VHD, VHDX[1], VDI and VMDK disk images\n- it can convert disk images between different virtual formats and clone disks\n- it can handle RAW disk images and BytesIO \"RamDisk\" images, also.\n- it supports 4K sectors\n- it can handle large clusters (128K, 256K) with FAT formats[2]\n- it is able to merge Differencing VHDs\n\nFollowing features are implemented (mostly in Python, with a few ctypes calls to handle Win32 disks natively; compatibility with Linux is not regularly tested):\n- sector aligned read/writes with both file images and real disks\n- sector based caching mechanism (for both reading and writing) to speed-up FAT and directory table operations\n- run length encoded map (with tuples and dictionaries) of free clusters, free directory slots, allocated cluster chains\n- transparent reading and writing of FAT12/16/32 and exFAT filesystems with FS boot-sector auto recognizer\n- MBR and GPT partitions handling\n- Long File Name and Unicode support\n- tools to open, create, rename, list and delete files and directories, and to partition disks\n- facilities to sort, clean and shrink directory tables and to wipe (zero) free space\n- file fragmentation calculator\n- mkfat tool to properly (partition and) apply a FAT12/16/32 or exFAT filesystem to a block device (file or disk) and let CHKDSK be happy with it (included exFAT compressed Up-Case table generator)\n\n*Obviously, since a filesystem is an extremely complex and delicate matter, and big bugs may lay around, you'll USE IT TOTALLY AT YOUR OWN RISK!* But it seems quite stable and useable, now.\n\nThe most fragile area (and, thus, subject to bugs) was the caching mechanism, that operates in different ways:\n- intercepting small I/O (<= 512 bytes), which is cached in a small circular buffer. Bigger I/O bypasses the cache; when the cache is full, all dirty sectors are committed to disk and the cache buffer is zeroed. Sectors and buffers are paired with Python builtin dictionaries: this permits a good (from a Pythonic perspective) I/O speed during FAT and directory tables access;\n- maintaining a dictionary of pre-decoded FAT indexes, to improve the speed of repetitive access to cluster chains;\n- maintaining a dictionary of short and long names (paired with their respective directory slots) for each directory table, to speed up searches and updates in directory tables;\n- maintaining a RLE map of free clusters, free directory slots and allocated cluster chains, to dramatically improve speed of allocation and file access. \n\n*Actually, the I/O speed is closer to system's one.*\n\nCode is GPLed (look at GPL.TXT).\n \n \n[1] VHDX Log support is actually limited to replaying capability.\n \n[2] Actually, to say, one can partition with GPT an 8 TB VHDX with 4K sectors and format with FAT32 and happily use it under Windows 11. However, Windows 11 CHKDSK reports no more than 4 TB _bytes_ (while it counts _clusters_ correctly). Also, FORMAT itself can't apply such legitimate FAT32 format to an 8 TB disk.\n\n# At a glance\n\nThe package installs a `fattools` script, you can use this to perform simple command line operations.\n\n- to create a dynamic 8TB VHDX disk image with a single GPT partition and format it with exFAT:\n```\nfattools mkvdisk -s 8T --large-sectors image.vhdx\nfattools mkfat -t exfat -p gpt image.vhdx\n```\n\n- to create a differencing VDI disk image:\n```\nfattools mkvdisk -b image.vdi delta.vdi\n```\n\n- to wipe free space in an (ex)FAT formatted disk, zeroing all free clusters:\n```\nfattools wipe image.vhd\n```\n\n- to convert a RAW disk image into a Dynamic VHD (so implicitly virtualizing zeroed data blocks):\n```\nfattools imgclone image.raw image.vhd\n```\nPlease note that resulting image size can get reduced if: 1) volume(s) is/are defragmented; 2) directory tables are cleaned and shrunk; 3) the free space has been wiped (zeroed) before.\n\n- to capture a physical drive (Windows) to a Dynamic VHD:\n```\nfattools imgclone \\\\.\\PhysicalDrive2 image.vhd\n```\n\n- to list contents in a disk image, copy items to/from it, display and erase them:\n```\nfattools ls image1.vhd/py* image2.vdi/py*\nfattools cp C:\\Python39\\Lib\\site-packages image.vhd/Python39/Lib\nfattools cp image.vhd/Python39 C:\\ProgramData\nfattools cat image.vhd/readme.txt\nfattools rm image.vhd/Python39\n```\n\n- to open an existing plain or VHD disk image, or real disk:\n```\n# -*- coding: cp1252 -*-\nfrom FATtools.Volume import *\ndisk = vopen('MyDiskImage.img', 'r+b', 'disk')\n```\n\n- to make a single GPT partition from all disk space:\n```\nfrom FATtools import partutils\ngpt = partutils.partition(disk)\n```\n\n- to format such partition with the exFAT file system:\n```\nfrom FATtools import mkfat, Volume\npart = Volume.vopen('MyDiskImage.img', 'r+b', 'partition0')\nmkfat.exfat_mkfs(part, part.size)\n```\n\n- to order items inside directory tables easily, with GUI and drag support:\n```\nfattools reordergui\n```\n\n- to order root directory table in USB drive X (scripting):\n```\n# -*- coding: cp1252 -*-\nfrom FATtools.Volume import *\n\n# Assuming we have DirA, DirB, DirC in this disk order into X:\nroot = vopen('X:', 'r+b')\n\nnew_order = '''DirB\nDirC\nDirA'''\n\nroot._sortby.fix = new_order.split('\\n') # uses built-in directory sort algorithm\nroot.sort(root._sortby) # user-defined order, in _sortby.fix list\nroot.sort() # default ordering (alphabetical)\n```\n\n- mixed access with Python and FATtools from the same script:\n```\n# -*- coding: cp1252 -*-\nfrom FATtools.Volume import vopen, vclose\nfrom FATtools.mkfat import exfat_mkfs\nfrom os.path import join\nimport os\n\nreal_fat_fs = 'F:' # replace with mount point of your file system\n\n# Open and format with FATtools\nfs = vopen(real_fat_fs, 'r+b',what='disk')\nexfat_mkfs(fs, fs.size)\nvclose(fs)\n\n# Write some files with Python and list them\nT = ('c','a','b','d')\nfor t in T:\n open(join(real_fat_fs, t+'.txt'), 'w').write('This is a sample \"%s.txt\" file.'%t)\n\nprint(os.listdir(real_fat_fs+'/'))\n\n# Open again, and sort root with FATtools\nfs = vopen(real_fat_fs, 'r+b')\nfs.sort()\nvclose(fs)\n\n# Check new table order with Python\nprint(os.listdir(real_fat_fs+'/'))\n```\n\n- (almost) same as above:\n```\n# -*- coding: cp1252 -*-\nfrom FATtools.Volume import vopen, vclose\nfrom FATtools.mkfat import exfat_mkfs\nfrom FATtools.partutils import partition\n\n# Open & create GPT partition\no = vopen('\\\\\\\\.\\\\PhysicalDrive1', 'r+b',what='disk')\nprint('Partitioning...')\npartition(o, 'mbr')\nvclose(o)\n\n# Reopen and format with EXFAT\no = vopen('\\\\\\\\.\\\\PhysicalDrive1', 'r+b',what='partition0')\nprint('Formatting...')\nexfat_mkfs(o, o.size)\nvclose(o) # auto-close partition AND disk\n\n# Reopen FS and write\nprint('Writing...')\no = vopen('\\\\\\\\.\\\\PhysicalDrive1', 'r+b')\n\n# Write some files with FATtools and sort them\nT = ('c','a','b','d')\nfor t in T:\n f = o.create(t+'.txt')\n f.write(b'This is a sample \"%s.txt\" file.'%bytes(t,'ascii'))\n f.close()\no.sort()\nvclose(o)\n```\n\nPlease look inside 'samples' directory for more usage samples.\n",
"bugtrack_url": null,
"license": "GPL",
"summary": "Pure python tools for accessing FAT filesystem images and disks",
"version": "1.0.36",
"project_urls": {
"homepage": "https://github.com/maxpat78/FATtools",
"repository": "https://github.com/maxpat78/FATtools.git"
},
"split_keywords": [
"fat",
" disk",
" image"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "55ae8ea689c80e08239ba56c19b799dea040631e40bd3a2b2992eae6c9cfc1f5",
"md5": "b6c618ab1401b958ec573847fcc8a92f",
"sha256": "e0a255841f208558f512bedb3f2b2f2e006f54e07b26d8447ce00ba32995aa05"
},
"downloads": -1,
"filename": "FATtools-1.0.36-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b6c618ab1401b958ec573847fcc8a92f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.5",
"size": 131882,
"upload_time": "2024-04-16T08:26:10",
"upload_time_iso_8601": "2024-04-16T08:26:10.878097Z",
"url": "https://files.pythonhosted.org/packages/55/ae/8ea689c80e08239ba56c19b799dea040631e40bd3a2b2992eae6c9cfc1f5/FATtools-1.0.36-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d74ad102f5c5bb3068e954b206b6ffc1e018373cba1a0ef94f6b9af41332ccfe",
"md5": "ba4c2f419ac825827aea85b72c552962",
"sha256": "3d42b619f781cb91ffe492281dd0536128c87c36d8982b0419b2c3e7e143358f"
},
"downloads": -1,
"filename": "fattools-1.0.36.tar.gz",
"has_sig": false,
"md5_digest": "ba4c2f419ac825827aea85b72c552962",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.5",
"size": 116321,
"upload_time": "2024-04-16T08:26:13",
"upload_time_iso_8601": "2024-04-16T08:26:13.318594Z",
"url": "https://files.pythonhosted.org/packages/d7/4a/d102f5c5bb3068e954b206b6ffc1e018373cba1a0ef94f6b9af41332ccfe/fattools-1.0.36.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-16 08:26:13",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "maxpat78",
"github_project": "FATtools",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "fattools"
}
