| Name | ibd-parser JSON |
| Version |
0.1.5
JSON |
| download |
| home_page | None |
| Summary | A Python tool for parsing and analyzing InnoDB .ibd files |
| upload_time | 2024-12-25 09:25:03 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.7 |
| license | MIT |
| keywords |
mysql
innodb
ibd
parser
database
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# InnoDB IBD File Parser
A Python tool for parsing and analyzing InnoDB .ibd files. This tool helps database administrators and developers understand the internal structure of InnoDB tablespace files.
## Features
- Parse InnoDB page headers
- Analyze index pages
- Extract record contents
- Support for various page types:
- Index pages (B-tree nodes)
- File space header
- XDES (Extent descriptor)
- BLOB pages
- And more...
## Installation
From PyPI:
```bash
pip install ibd-parser
```
From source:
```bash
pip install git+https://github.com/likeyiyy/ibd-parser.git
```
## Usage
### Command Line Interface
```bash
# Show page header
ibd-parser -f /path/to/table.ibd header --page 4
# Dump records from an index page
ibd-parser -f /path/to/table.ibd records --page 4
```
### Python API
```python
from ibd_parser import IBDFileParser
# Initialize parser with your .ibd file
parser = IBDFileParser("/path/to/your/table.ibd")
# Analyze a specific page
page_info = parser.analyze_page(page_no=4)
# Access page information
print(f"Page Type: {page_info['header'].page_type}")
if 'index_header' in page_info:
print(f"Number of records: {page_info['index_header'].n_recs}")
# Get records from an index page
records = parser.get_records(page_no=4)
for record in records:
print(record.data)
# Hex dump of a page
parser.hex_dump(page_no=4, length=128)
```
## Project Structure
```
ibd_parser/
├── ibd_parser/
│ ├── __init__.py
│ ├── constants.py # Constants and enums
│ ├── page.py # Page structure definitions
│ ├── record.py # Record parsing
│ ├── parser.py # Main parser implementation
│ ├── cli.py # Command line interface
│ └── utils.py # Utility functions
├── tests/
├── README.md
└── pyproject.toml # Project metadata and dependencies
```
## Page Structure
An InnoDB page (default 16KB) consists of:
1. File Header (38 bytes)
2. Page Header (56 bytes)
3. Infimum/Supremum Records
4. User Records
5. Free Space
6. Page Directory
7. File Trailer (8 bytes)
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
1. Fork the repository
2. Create your feature branch (`git checkout -b feature/AmazingFeature`)
3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request
## License
This project is licensed under the MIT License - see the LICENSE file for details.
## Acknowledgments
- Based on the InnoDB storage engine documentation
- Inspired by various InnoDB internals research papers and blog posts
## References
- [InnoDB Page Structure](https://dev.mysql.com/doc/refman/8.0/en/innodb-page-structure.html)
- [InnoDB File Space Management](https://dev.mysql.com/doc/refman/8.0/en/innodb-file-space.html)
## Future Plans
I will continue to enhance this project to make it more practical and valuable. Planned improvements include:
- Support for more page types
- Enhanced record analysis
- Data recovery features
- More comprehensive CLI tools
- Better documentation and examples
Contributions and suggestions are welcome!
## Testing and Development
### Creating Test Data
The project includes a script to create a test database with various column types:
```bash
# Install MySQL Connector
pip install mysql-connector-python
# Set MySQL connection environment variables (if needed)
export MYSQL_USER=your_user
export MYSQL_PASSWORD=your_password
export MYSQL_HOST=localhost
export MYSQL_PORT=3306 # Or your Docker mapped port
# Create test database and table
python examples/create_test_data.py
```
The test table includes common MySQL data types:
- Integer types (TINYINT, SMALLINT, INT, BIGINT)
- Floating point types (FLOAT, DOUBLE, DECIMAL)
- String types (CHAR, VARCHAR, TEXT)
- Date and Time types (DATE, TIME, DATETIME, TIMESTAMP)
- Other types (BOOLEAN, ENUM, BINARY, BLOB)
After creating the test database, you can find the .ibd file at:
```
/data/docker/mysql/data/ibd_parser_test/test_table.ibd
```
Raw data
{
"_id": null,
"home_page": null,
"name": "ibd-parser",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "mysql, innodb, ibd, parser, database",
"author": null,
"author_email": "likeyiyy <likeyiyying@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/86/ed/ea2338c5440746a0e51b36d0cf1dc9dc557746a5a29aec7d2b59bcb892b1/ibd_parser-0.1.5.tar.gz",
"platform": null,
"description": "# InnoDB IBD File Parser\n\nA Python tool for parsing and analyzing InnoDB .ibd files. This tool helps database administrators and developers understand the internal structure of InnoDB tablespace files.\n\n## Features\n\n- Parse InnoDB page headers\n- Analyze index pages\n- Extract record contents\n- Support for various page types:\n - Index pages (B-tree nodes)\n - File space header\n - XDES (Extent descriptor)\n - BLOB pages\n - And more...\n\n## Installation\n\nFrom PyPI:\n```bash\npip install ibd-parser\n```\n\nFrom source:\n```bash\npip install git+https://github.com/likeyiyy/ibd-parser.git\n```\n\n## Usage\n\n### Command Line Interface\n\n```bash\n# Show page header\nibd-parser -f /path/to/table.ibd header --page 4\n\n# Dump records from an index page\nibd-parser -f /path/to/table.ibd records --page 4\n```\n\n### Python API\n\n```python\nfrom ibd_parser import IBDFileParser\n\n# Initialize parser with your .ibd file\nparser = IBDFileParser(\"/path/to/your/table.ibd\")\n\n# Analyze a specific page\npage_info = parser.analyze_page(page_no=4)\n\n# Access page information\nprint(f\"Page Type: {page_info['header'].page_type}\")\nif 'index_header' in page_info:\n print(f\"Number of records: {page_info['index_header'].n_recs}\")\n\n# Get records from an index page\nrecords = parser.get_records(page_no=4)\nfor record in records:\n print(record.data)\n\n# Hex dump of a page\nparser.hex_dump(page_no=4, length=128)\n```\n\n## Project Structure\n\n```\nibd_parser/\n\u251c\u2500\u2500 ibd_parser/\n\u2502 \u251c\u2500\u2500 __init__.py\n\u2502 \u251c\u2500\u2500 constants.py # Constants and enums\n\u2502 \u251c\u2500\u2500 page.py # Page structure definitions\n\u2502 \u251c\u2500\u2500 record.py # Record parsing\n\u2502 \u251c\u2500\u2500 parser.py # Main parser implementation\n\u2502 \u251c\u2500\u2500 cli.py # Command line interface\n\u2502 \u2514\u2500\u2500 utils.py # Utility functions\n\u251c\u2500\u2500 tests/\n\u251c\u2500\u2500 README.md\n\u2514\u2500\u2500 pyproject.toml # Project metadata and dependencies\n```\n\n## Page Structure\n\nAn InnoDB page (default 16KB) consists of:\n\n1. File Header (38 bytes)\n2. Page Header (56 bytes)\n3. Infimum/Supremum Records\n4. User Records\n5. Free Space\n6. Page Directory\n7. File Trailer (8 bytes)\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n\n## Acknowledgments\n\n- Based on the InnoDB storage engine documentation\n- Inspired by various InnoDB internals research papers and blog posts\n\n## References\n\n- [InnoDB Page Structure](https://dev.mysql.com/doc/refman/8.0/en/innodb-page-structure.html)\n- [InnoDB File Space Management](https://dev.mysql.com/doc/refman/8.0/en/innodb-file-space.html)\n\n## Future Plans\n\nI will continue to enhance this project to make it more practical and valuable. Planned improvements include:\n\n- Support for more page types\n- Enhanced record analysis\n- Data recovery features\n- More comprehensive CLI tools\n- Better documentation and examples\n\nContributions and suggestions are welcome!\n\n## Testing and Development\n\n### Creating Test Data\n\nThe project includes a script to create a test database with various column types:\n\n```bash\n# Install MySQL Connector\npip install mysql-connector-python\n\n# Set MySQL connection environment variables (if needed)\nexport MYSQL_USER=your_user\nexport MYSQL_PASSWORD=your_password\nexport MYSQL_HOST=localhost\nexport MYSQL_PORT=3306 # Or your Docker mapped port\n\n# Create test database and table\npython examples/create_test_data.py\n```\n\nThe test table includes common MySQL data types:\n- Integer types (TINYINT, SMALLINT, INT, BIGINT)\n- Floating point types (FLOAT, DOUBLE, DECIMAL)\n- String types (CHAR, VARCHAR, TEXT)\n- Date and Time types (DATE, TIME, DATETIME, TIMESTAMP)\n- Other types (BOOLEAN, ENUM, BINARY, BLOB)\n\nAfter creating the test database, you can find the .ibd file at:\n```\n/data/docker/mysql/data/ibd_parser_test/test_table.ibd\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Python tool for parsing and analyzing InnoDB .ibd files",
"version": "0.1.5",
"project_urls": {
"Bug Tracker": "https://github.com/likeyiyy/ibd-parser/issues",
"Documentation": "https://github.com/likeyiyy/ibd-parser#README.md",
"Homepage": "https://github.com/likeyiyy/ibd-parser",
"Source Code": "https://github.com/likeyiyy/ibd-parser"
},
"split_keywords": [
"mysql",
" innodb",
" ibd",
" parser",
" database"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "7be535b8dabe7b0532cb8f361272b4be9c6908948b837a386cc836a3154aafd9",
"md5": "e02594512b5d730056b203579280825c",
"sha256": "7a770cfaa6eabb400ee36e6f529fd2f5c3d6ef783c694b97f853146c2fa38793"
},
"downloads": -1,
"filename": "ibd_parser-0.1.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e02594512b5d730056b203579280825c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 12239,
"upload_time": "2024-12-25T09:25:00",
"upload_time_iso_8601": "2024-12-25T09:25:00.688923Z",
"url": "https://files.pythonhosted.org/packages/7b/e5/35b8dabe7b0532cb8f361272b4be9c6908948b837a386cc836a3154aafd9/ibd_parser-0.1.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "86edea2338c5440746a0e51b36d0cf1dc9dc557746a5a29aec7d2b59bcb892b1",
"md5": "065c99cc45d5c91b3e3ecbc9f5d0dfb2",
"sha256": "6e5c8f78424192e65ef703039672a334d36d93d9675564e7d596999722c964e7"
},
"downloads": -1,
"filename": "ibd_parser-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "065c99cc45d5c91b3e3ecbc9f5d0dfb2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 11811,
"upload_time": "2024-12-25T09:25:03",
"upload_time_iso_8601": "2024-12-25T09:25:03.008293Z",
"url": "https://files.pythonhosted.org/packages/86/ed/ea2338c5440746a0e51b36d0cf1dc9dc557746a5a29aec7d2b59bcb892b1/ibd_parser-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-25 09:25:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "likeyiyy",
"github_project": "ibd-parser",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "ibd-parser"
}
