decompyle3

Name	decompyle3 JSON
Version	3.9.2 JSON
	download
home_page	https://github.com/rocky/python-decompile3/
Summary	Python cross-version byte-code decompiler
upload_time	2024-07-22 15:24:01
maintainer	None
docs_url	None
author	Rocky Bernstein, Hartmut Goebel, John Aycock, and others
requires_python	None
license	GPL3
keywords
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI
coveralls test coverage	No coveralls.

.. contents::

decompyle3
==========

A native Python cross-version decompiler and fragment decompiler.
A reworking of uncompyle6_.

Introduction
------------

*decompyle3* translates Python bytecode back into equivalent Python
source code. It accepts bytecodes from Python version 3.7 on.

For decompilation of older Python bytecode see uncompyle6_.

Why this?
---------

Uncompyle6 is awesome, but it has a fundamental problem in the way
it handles control flow. In the early days of Python when there was
little optimization and code was generated in a very template-oriented
way, figuring out control flow-structures could be done by simply looking at code patterns.

Over the years more code optimization, specifically around handling
jumps has made it harder to support detecting control flow strictly
from code patterns. This was noticed as far back as Python 2.4 (2004)
but since this is a difficult problem, so far it hasn't been tackled
in a satisfactory way.

The initial attempt to fix to this problem was to add markers in the
instruction stream, initially this was a `COME_FROM` instruction, and
then use that in pattern detection.

Over the years, I've extended that to be more specific, so
`COME_FROM_LOOP` and `COME_FROM_WITH` were added. And I added checks
at grammar-reduce time to make try to make sure jumps match with
supposed `COME_FROM` targets.

However all of this is complicated, not robust, has greatly slowed
down deparsing and is not really tenable.

So in this project we started rewriting and refactoring the grammar.

However it is clear that even this isn't enough. Control flow needs
to be addressed by using dominators and reverse-dominators which
the python-control-flow_ project can give.

This I am *finally* slowly doing in yet another non-public project. It
is a lot of work. Funding in the form of sponsorhip while greatly
appreciated isn't commensurate with the amount of effort, and
currently I have a full-time job. So it may take time before it is
available publicly, if at all.

Requirements
------------

The code here can be run on Python versions 3.7 or 3.8. The bytecode
files it can read have been tested on Python bytecodes from versions
3.7 and 3.8.

Installation
------------

You can install from PyPI using the name ``decompyle3``::

pip install decompyle3

To install from source code, this project uses setup.py, so it follows the standard Python routine::

$ pip install -e . # set up to run from source tree

or::

$ python setup.py install # may need sudo

A GNU Makefile is also provided so :code:`make install` (possibly as root or
sudo) will do the steps above.

Running Tests
-------------

make check

A GNU makefile has been added to smooth over setting running the right
command, and running tests from fastest to slowest.

If you have remake_ installed, you can see the list of all tasks
including tests via :code:`remake --tasks`

Usage
-----

Run

$ decompyle3 *compiled-python-file-pyc-or-pyo*

For usage help:

$ decompyle3 -h

Verification
------------

If you want Python syntax verification of the correctness of the
decompilation process, add the :code:`--syntax-verify` option. However since
Python syntax changes, you should use this option if the bytecode is
the right bytecode for the Python interpreter that will be checking
the syntax.

You can also cross compare the results with another python decompiler
like unpyc37_ . Since they work differently, bugs here often aren't in
that, and vice versa.

There is an interesting class of these programs that is readily
available give stronger verification: those programs that when run
test themselves. Our test suite includes these.

And Python comes with another a set of programs like this: its test
suite for the standard library. We have some code in :code:`test/stdlib` to
facilitate this kind of checking too.

Known Bugs/Restrictions
-----------------------

**We support only released versions, not candidate versions.** Note however
that the magic of a released version is usually the same as the *last* candidate version prior to release.

We also don't handle PJOrion_ or otherwise obfuscated code. For
PJOrion try: PJOrion Deobfuscator_ to unscramble the bytecode to get
valid bytecode before trying this tool; pydecipher_ might help with that.

This program can't decompile Microsoft Windows EXE files created by
Py2EXE_, although we can probably decompile the code after you extract
the bytecode properly. `Pydeinstaller <https://github.com/charles-dyfis-net/pydeinstaller>`_ may help with unpacking Pyinstaller bundlers.

Handling pathologically long lists of expressions or statements is
slow. We don't handle Cython_ or MicroPython which don't use bytecode.

There are numerous bugs in decompilation. And that's true for every
other CPython decompiler I have encountered, even the ones that
claimed to be "perfect" on some particular version like 2.4.

As Python progresses decompilation also gets harder because the
compilation is more sophisticated and the language itself is more
sophisticated. I suspect that attempts there will be fewer ad-hoc
attempts like unpyc37_ (which is based on a 3.3 decompiler) simply
because it is harder to do so. The good news, at least from my
standpoint, is that I think I understand what's needed to address the
problems in a more robust way. But right now until such time as
project is better funded, I do not intend to make any serious effort
to support Python versions 3.8 or 3.9, including bugs that might come
in. I imagine at some point I may be interested in it.

You can easily find bugs by running the tests against the standard
test suite that Python uses to check itself. At any given time, there are
dozens of known problems that are pretty well isolated and that could
be solved if one were to put in the time to do so. The problem is that
there aren't that many people who have been working on bug fixing.

You may run across a bug, that you want to report. Please do so. But
be aware that it might not get my attention for a while. If you
sponsor or support the project in some way, I'll prioritize your
issues above the queue of other things I might be doing instead.

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/rocky/python-decompile3/",
    "name": "decompyle3",
    "maintainer": null,
    "docs_url": null,
    "requires_python": null,
    "maintainer_email": null,
    "keywords": null,
    "author": "Rocky Bernstein, Hartmut Goebel, John Aycock, and others",
    "author_email": "rb@dustyfeet.com",
    "download_url": "https://files.pythonhosted.org/packages/1e/f2/1045f996c6806a8e53aea38e66c3ae0cab4c337f7029b6932224f20a7d29/decompyle3-3.9.2.tar.gz",
    "platform": null,
    "description": "|TravisCI| |CircleCI| |Pypi Installs|\n\n.. contents::\n\ndecompyle3\n==========\n\nA native Python cross-version decompiler and fragment decompiler.\nA reworking of uncompyle6_.\n\n\nIntroduction\n------------\n\n*decompyle3* translates Python bytecode back into equivalent Python\nsource code. It accepts bytecodes from Python version 3.7 on.\n\nFor decompilation of older Python bytecode see uncompyle6_.\n\nWhy this?\n---------\n\nUncompyle6 is awesome, but it has a fundamental problem in the way\nit handles control flow. In the early days of Python when there was\nlittle optimization and code was generated in a very template-oriented\nway, figuring out control flow-structures could be done by simply looking at code patterns.\n\nOver the years more code optimization, specifically around handling\njumps has made it harder to support detecting control flow strictly\nfrom code patterns. This was noticed as far back as Python 2.4 (2004)\nbut since this is a difficult problem, so far it hasn't been tackled\nin a satisfactory way.\n\nThe initial attempt to fix to this problem was to add markers in the\ninstruction stream, initially this was a `COME_FROM` instruction, and\nthen use that in pattern detection.\n\nOver the years, I've extended that to be more specific, so\n`COME_FROM_LOOP` and `COME_FROM_WITH` were added. And I added checks\nat grammar-reduce time to make try to make sure jumps match with\nsupposed `COME_FROM` targets.\n\nHowever all of this is complicated, not robust, has greatly slowed\ndown deparsing and is not really tenable.\n\nSo in this project we started rewriting and refactoring the grammar.\n\nHowever it is clear that even this isn't enough. Control flow needs\nto be addressed by using dominators and reverse-dominators which\nthe python-control-flow_ project can give.\n\nThis I am *finally* slowly doing in yet another non-public project. It\nis a lot of work.  Funding in the form of sponsorhip while greatly\nappreciated isn't commensurate with the amount of effort, and\ncurrently I have a full-time job. So it may take time before it is\navailable publicly, if at all.\n\nRequirements\n------------\n\nThe code here can be run on Python versions 3.7 or 3.8. The bytecode\nfiles it can read have been tested on Python bytecodes from versions\n3.7 and 3.8.\n\nInstallation\n------------\n\nYou can install from PyPI using the name ``decompyle3``::\n\n    pip install decompyle3\n\n\nTo install from source code, this project uses setup.py, so it follows the standard Python routine::\n\n    $ pip install -e .  # set up to run from source tree\n\nor::\n\n    $ python setup.py install # may need sudo\n\nA GNU Makefile is also provided so :code:`make install` (possibly as root or\nsudo) will do the steps above.\n\nRunning Tests\n-------------\n\n::\n\n   make check\n\nA GNU makefile has been added to smooth over setting running the right\ncommand, and running tests from fastest to slowest.\n\nIf you have remake_ installed, you can see the list of all tasks\nincluding tests via :code:`remake --tasks`\n\n\nUsage\n-----\n\nRun\n\n::\n\n$ decompyle3 *compiled-python-file-pyc-or-pyo*\n\nFor usage help:\n\n::\n\n   $ decompyle3 -h\n\nVerification\n------------\n\nIf you want Python syntax verification of the correctness of the\ndecompilation process, add the :code:`--syntax-verify` option. However since\nPython syntax changes, you should use this option if the bytecode is\nthe right bytecode for the Python interpreter that will be checking\nthe syntax.\n\nYou can also cross compare the results with another python decompiler\nlike unpyc37_ . Since they work differently, bugs here often aren't in\nthat, and vice versa.\n\nThere is an interesting class of these programs that is readily\navailable give stronger verification: those programs that when run\ntest themselves. Our test suite includes these.\n\nAnd Python comes with another a set of programs like this: its test\nsuite for the standard library. We have some code in :code:`test/stdlib` to\nfacilitate this kind of checking too.\n\nKnown Bugs/Restrictions\n-----------------------\n\n**We support only released versions, not candidate versions.** Note however\nthat the magic of a released version is usually the same as the *last* candidate version prior to release.\n\nWe also don't handle PJOrion_ or otherwise obfuscated code. For\nPJOrion try: PJOrion Deobfuscator_ to unscramble the bytecode to get\nvalid bytecode before trying this tool; pydecipher_ might help with that.\n\nThis program can't decompile Microsoft Windows EXE files created by\nPy2EXE_, although we can probably decompile the code after you extract\nthe bytecode properly. `Pydeinstaller <https://github.com/charles-dyfis-net/pydeinstaller>`_ may help with unpacking Pyinstaller bundlers.\n\nHandling pathologically long lists of expressions or statements is\nslow. We don't handle Cython_ or MicroPython which don't use bytecode.\n\nThere are numerous bugs in decompilation. And that's true for every\nother CPython decompiler I have encountered, even the ones that\nclaimed to be \"perfect\" on some particular version like 2.4.\n\nAs Python progresses decompilation also gets harder because the\ncompilation is more sophisticated and the language itself is more\nsophisticated. I suspect that attempts there will be fewer ad-hoc\nattempts like unpyc37_ (which is based on a 3.3 decompiler) simply\nbecause it is harder to do so. The good news, at least from my\nstandpoint, is that I think I understand what's needed to address the\nproblems in a more robust way. But right now until such time as\nproject is better funded, I do not intend to make any serious effort\nto support Python versions 3.8 or 3.9, including bugs that might come\nin. I imagine at some point I may be interested in it.\n\nYou can easily find bugs by running the tests against the standard\ntest suite that Python uses to check itself. At any given time, there are\ndozens of known problems that are pretty well isolated and that could\nbe solved if one were to put in the time to do so. The problem is that\nthere aren't that many people who have been working on bug fixing.\n\nYou may run across a bug, that you want to report. Please do so. But\nbe aware that it might not get my attention for a while. If you\nsponsor or support the project in some way, I'll prioritize your\nissues above the queue of other things I might be doing instead.\n\nSee Also\n--------\n\n* https://github.com/andrew-tavera/unpyc37/ : indirect fork of https://code.google.com/archive/p/unpyc3/ The above projects use a different decompiling technique than what is used here. Instructions are walked. Some instructions use the stack to generate strings, while others don't. Because control flow isn't dealt with directly, it too suffers the same problems as the various `uncompyle` and `decompyle` programs.\n* https://github.com/rocky/python-xdis : Cross Python version disassembler\n* https://github.com/rocky/python-xasm : Cross Python version assembler\n* https://github.com/rocky/python-decompile3/wiki : Wiki Documents which describe the code and aspects of it in more detail\n\n.. _Cython: https://en.wikipedia.org/wiki/Cython\n.. _MicroPython: https://micropython.org\n.. _uncompyle6: https://pypi.python.org/pypi/uncompyle6\n.. _python-control-flow: https://github.com/rocky/python-control-flow\n.. _trepan: https://pypi.python.org/pypi/trepan3k\n.. _compiler: https://pypi.python.org/pypi/spark_parser\n.. _HISTORY: https://github.com/rocky/python-decompile3/blob/master/HISTORY.md\n.. _debuggers: https://pypi.python.org/pypi/trepan3k\n.. _remake: https://bashdb.sf.net/remake\n.. _unpyc37: https://github.com/andrew-tavera/unpyc37/\n.. _this: https://github.com/rocky/python-decompile3/wiki/Deparsing-technology-and-its-use-in-exact-location-reporting\n.. |TravisCI| image:: https://travis-ci.org/rocky/python-decompile3.svg\n\t\t :target: https://travis-ci.org/rocky/python-decompile3\n.. |CircleCI| image:: https://circleci.com/gh/rocky/python-decompile3.svg?style=svg\n\t  :target: https://circleci.com/gh/rocky/python-decompile3\n\n.. _PJOrion: http://www.koreanrandom.com/forum/topic/15280-pjorion-%D1%80%D0%B5%D0%B4%D0%B0%D0%BA%D1%82%D0%B8%D1%80%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5-%D0%BA%D0%BE%D0%BC%D0%BF%D0%B8%D0%BB%D1%8F%D1%86%D0%B8%D1%8F-%D0%B4%D0%B5%D0%BA%D0%BE%D0%BC%D0%BF%D0%B8%D0%BB%D1%8F%D1%86%D0%B8%D1%8F-%D0%BE%D0%B1%D1%84\n.. _Deobfuscator: https://github.com/extremecoders-re/PjOrion-Deobfuscator\n.. _Py2EXE: https://en.wikipedia.org/wiki/Py2exe\n.. |Supported Python Versions| image:: https://img.shields.io/pypi/pyversions/decompyle3.svg\n.. |Latest Version| image:: https://badge.fury.io/py/decompyle3.svg\n\t\t :target: https://badge.fury.io/py/decompyle3\n.. |PyPI Installs| image:: https://pepy.tech/badge/decompyle3/month\n.. _pydecipher: https://github.com/mitre/pydecipher\n\n",
    "bugtrack_url": null,
    "license": "GPL3",
    "summary": "Python cross-version byte-code decompiler",
    "version": "3.9.2",
    "project_urls": {
        "Homepage": "https://github.com/rocky/python-decompile3/"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "dbf5f8a5d91f3a12dfacb88abcf3a711d29b14bf9c3e3633751eed9ad9736fc9",
                "md5": "67838f2ebcd644aeb763f265ae5d181e",
                "sha256": "77cc6b17a9866b525d2ddf64e1ddb2b4a1f59b97e809ef0913193e406dc173ef"
            },
            "downloads": -1,
            "filename": "decompyle3-3.9.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "67838f2ebcd644aeb763f265ae5d181e",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 272517,
            "upload_time": "2024-07-22T15:23:59",
            "upload_time_iso_8601": "2024-07-22T15:23:59.904855Z",
            "url": "https://files.pythonhosted.org/packages/db/f5/f8a5d91f3a12dfacb88abcf3a711d29b14bf9c3e3633751eed9ad9736fc9/decompyle3-3.9.2-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "1ef21045f996c6806a8e53aea38e66c3ae0cab4c337f7029b6932224f20a7d29",
                "md5": "402b88c15b2d56f73b083201b45e0ee4",
                "sha256": "bf4a177c5d53bd764496709fe60ea351103efc01b921c596382af9a7ee0433f3"
            },
            "downloads": -1,
            "filename": "decompyle3-3.9.2.tar.gz",
            "has_sig": false,
            "md5_digest": "402b88c15b2d56f73b083201b45e0ee4",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 869783,
            "upload_time": "2024-07-22T15:24:01",
            "upload_time_iso_8601": "2024-07-22T15:24:01.587493Z",
            "url": "https://files.pythonhosted.org/packages/1e/f2/1045f996c6806a8e53aea38e66c3ae0cab4c337f7029b6932224f20a7d29/decompyle3-3.9.2.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-07-22 15:24:01",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "rocky",
    "github_project": "python-decompile3",
    "travis_ci": true,
    "coveralls": false,
    "github_actions": true,
    "circle": true,
    "requirements": [],
    "tox": true,
    "lcname": "decompyle3"
}

Rocky Bernstein, Hartmut Goebel, John Aycock, and others