lithium-reducer


Namelithium-reducer JSON
Version 1.1.1 PyPI version JSON
download
home_pagehttps://github.com/MozillaSecurity/lithium
SummaryLithium is an automated testcase reduction tool
upload_time2023-12-29 19:55:15
maintainerMozilla Fuzzing Team
docs_urlNone
authorJesse Ruderman
requires_python>=3.8
licenseMPL 2.0
keywords fuzz fuzzing reduce reducer reduction security test testing
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            [![Task Status](https://community-tc.services.mozilla.com/api/github/v1/repository/MozillaSecurity/lithium/master/badge.svg)](https://community-tc.services.mozilla.com/api/github/v1/repository/MozillaSecurity/lithium/master/latest)
[![codecov](https://codecov.io/gh/MozillaSecurity/lithium/branch/master/graph/badge.svg)](https://codecov.io/gh/MozillaSecurity/lithium)
[![Matrix](https://img.shields.io/badge/dynamic/json?color=green&label=chat&query=%24.chunk[%3F(%40.canonical_alias%3D%3D%22%23fuzzing%3Amozilla.org%22)].num_joined_members&suffix=%20users&url=https%3A%2F%2Fmozilla.modular.im%2F_matrix%2Fclient%2Fr0%2FpublicRooms&style=flat&logo=matrix)](https://riot.im/app/#/room/#fuzzing:mozilla.org)
[![PyPI](https://img.shields.io/pypi/v/lithium-reducer)](https://pypi.org/project/lithium-reducer)

## Using Lithium

Lithium is an automated testcase reduction tool developed by [Jesse Ruderman](http://www.squarefree.com/).

Most of what you need to know to use Lithium is in one of these pages:

- [How to use Lithium to reduce Firefox bugs](src/lithium/docs/using-for-firefox.md).  Lithium has been used it to make reduced testcases for hundreds of Firefox crashes and assertions.
- [How to create your own "interestingness tests"](src/lithium/docs/creating-tests.md).  Lithium is flexible enough to reduce files for complicated properties such as "parsed differently by Opera and Firefox".  Just supply a small program that determines when a given file has the property.


### Lithium's algorithm

By default, Lithium uses a clever algorithm that's efficient at reducing most large testcases.  For a testcase with 2048 lines, it will try removing each chunk of size 1024, permanently removing it if it is still 'interesting'.  It then does the same for each chunk of size 512, then 256, all the way down to chunks of size 1.  It then does as many additional rounds at chunk size 1 as necessary until it completes a round without removing anything, at which point the file is 1-minimal (removing any single line from the file makes it 'uninteresting').

If *n* is the size of the testcase and *m* is the size of the 1-minimal testcase found by Lithium, then Lithium usually performs O(m ⋅ lg(n)) tests with a total test size of O(m ⋅ n).  See the [analysis of Lithium's algorithm](src/lithium/docs/algorithm.md) for more information and proofs.

To keep *m* small, make sure Lithium's smallest removals won't introduce fatal syntax errors into the file it is trying to reduce.  For example, don't use --char when trying to reduce a long sequence of JavaScript statements, and don't feed XHTML to Lithium.  (Convert it to HTML first and let Firefox's tag-soup parser sort out the invalidity, or use serializeDOMAsScript.)


### Command line syntax

    pip install lithium-reducer
    python -m lithium [options] interestingness-test.py [arguments for interestingness test]


### Command line options

<dl>

<dt>--testcase=filename</dt>
<dd>Tells Lithium which file to reduce.  By default, it will assume the last argument to the interestingness test is the file to reduce.</dd>

<dt>--char (-c)<dt>
<dd>By default, Lithium treats lines as atomic units.  This is great if each line is a JavaScript statement, but sometimes you want to go further.  Use this option to tell Lithium to treat the file as a sequence of characters instead of a sequence of lines.</dd>

<dt>--strategy=[check-only,minimize,minimize-balanced,replace-properties-by-globals,replace-arguments-by-globals,minimize-around]</dt>
<dd>"minimize" is the default, the algorithm described above. "check-only" tries to run Lithium to determine interestingness, without reduction. For the other strategies, check out <a href="https://github.com/MozillaSecurity/lithium/pull/2">this GitHub PR</a>.</dd>

<dt>--repeat=[always, last, never].</dt>
<dd>By default, Lithium only repeats at the same chunk size if it just finished the last round (e.g. chunk size 1).  You can use --repeat=always to tell it to repeat any chunk size if something was removed during the round, which can be useful for non-deterministic testcases or non-monotonic situations.  You can use --repeat=never to tell it to exit immediately after a single round at the last chunk size, which can save a little time at the risk of leaving a little bit extra in the file.</dd>

<dt>--max=n. default: about half of the file.</dt>
<dt>--min=n. default: 1.</dt>
<dd>What chunk sizes to test.  Must be powers of two.  --max is useful if you're restarting Lithium after it has already gone through a few rounds.  --min is useful if you're reducing HTML and want to do the final by hand.</dd>

<dt>--chunk-size=n</dt>
<dd>Shortcut for "repeat=never, min=n, max=n".  --chunk-size=1 is a quick way to determine whether a file is 1-minimal, for example after making a change that you think might make some lines unnecessary.</dd>

</dl>


### Hints

If you find a non-deterministic bug, don't despair.  Lithium will do fine as long as you make the bug happen at least 70% of the time.  You can repeat the test either within the application, by adding a loop or reload in the testcase (outside of the DDBEGIN/DDEND markers!), or outside of the application, by adding a loop to the "interestingness test" script.


### Requirements

Lithium is written in [Python](https://www.python.org/) and requires Python 3.5+.

### Credits

- [Lithium's testcase reduction algorithm](src/lithium/docs/algorithm.md) is a modified version of the "ddmin" algorithm in Andreas Zeller's paper, [Simplifying and Isolating Failure-Inducing Input](https://www.st.cs.uni-saarland.de/papers/tse2002/).
- The idea of using an external "interestingness test" program came from [Delta](http://delta.tigris.org/), a similar tool that's [used in clever ways by the GCC project](https://gcc.gnu.org/wiki/A_guide_to_testcase_reduction).
- [timed_run](src/lithium/interestingness/timed_run.py), used by many of the "interestingness test" scripts that come with Lithium, is based on [timed_run.py](https://web.archive.org/web/20071107032840/http://bclary.com/log/2007/03/07/timed_run), which was written by [Chris Cooper](http://coop.deadsquid.com/) and [Bob Clary](https://bclary.com/).
- The code was significantly cleaned up and modernized by Jesse Schwartzentruber and Gary Kwong in mid-2017.

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/MozillaSecurity/lithium",
    "name": "lithium-reducer",
    "maintainer": "Mozilla Fuzzing Team",
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": "fuzzing@mozilla.com",
    "keywords": "fuzz,fuzzing,reduce,reducer,reduction,security,test,testing",
    "author": "Jesse Ruderman",
    "author_email": "",
    "download_url": "https://files.pythonhosted.org/packages/c2/77/395d4528e6c22da422e1cc23fc01a30d2a57e361a546ac57d0ea220e0c1c/lithium-reducer-1.1.1.tar.gz",
    "platform": null,
    "description": "[![Task Status](https://community-tc.services.mozilla.com/api/github/v1/repository/MozillaSecurity/lithium/master/badge.svg)](https://community-tc.services.mozilla.com/api/github/v1/repository/MozillaSecurity/lithium/master/latest)\n[![codecov](https://codecov.io/gh/MozillaSecurity/lithium/branch/master/graph/badge.svg)](https://codecov.io/gh/MozillaSecurity/lithium)\n[![Matrix](https://img.shields.io/badge/dynamic/json?color=green&label=chat&query=%24.chunk[%3F(%40.canonical_alias%3D%3D%22%23fuzzing%3Amozilla.org%22)].num_joined_members&suffix=%20users&url=https%3A%2F%2Fmozilla.modular.im%2F_matrix%2Fclient%2Fr0%2FpublicRooms&style=flat&logo=matrix)](https://riot.im/app/#/room/#fuzzing:mozilla.org)\n[![PyPI](https://img.shields.io/pypi/v/lithium-reducer)](https://pypi.org/project/lithium-reducer)\n\n## Using Lithium\n\nLithium is an automated testcase reduction tool developed by [Jesse Ruderman](http://www.squarefree.com/).\n\nMost of what you need to know to use Lithium is in one of these pages:\n\n- [How to use Lithium to reduce Firefox bugs](src/lithium/docs/using-for-firefox.md).  Lithium has been used it to make reduced testcases for hundreds of Firefox crashes and assertions.\n- [How to create your own \"interestingness tests\"](src/lithium/docs/creating-tests.md).  Lithium is flexible enough to reduce files for complicated properties such as \"parsed differently by Opera and Firefox\".  Just supply a small program that determines when a given file has the property.\n\n\n### Lithium's algorithm\n\nBy default, Lithium uses a clever algorithm that's efficient at reducing most large testcases.  For a testcase with 2048 lines, it will try removing each chunk of size 1024, permanently removing it if it is still 'interesting'.  It then does the same for each chunk of size 512, then 256, all the way down to chunks of size 1.  It then does as many additional rounds at chunk size 1 as necessary until it completes a round without removing anything, at which point the file is 1-minimal (removing any single line from the file makes it 'uninteresting').\n\nIf *n* is the size of the testcase and *m* is the size of the 1-minimal testcase found by Lithium, then Lithium usually performs O(m &sdot; lg(n)) tests with a total test size of O(m &sdot; n).  See the [analysis of Lithium's algorithm](src/lithium/docs/algorithm.md) for more information and proofs.\n\nTo keep *m* small, make sure Lithium's smallest removals won't introduce fatal syntax errors into the file it is trying to reduce.  For example, don't use --char when trying to reduce a long sequence of JavaScript statements, and don't feed XHTML to Lithium.  (Convert it to HTML first and let Firefox's tag-soup parser sort out the invalidity, or use serializeDOMAsScript.)\n\n\n### Command line syntax\n\n    pip install lithium-reducer\n    python -m lithium [options] interestingness-test.py [arguments for interestingness test]\n\n\n### Command line options\n\n<dl>\n\n<dt>--testcase=filename</dt>\n<dd>Tells Lithium which file to reduce.  By default, it will assume the last argument to the interestingness test is the file to reduce.</dd>\n\n<dt>--char (-c)<dt>\n<dd>By default, Lithium treats lines as atomic units.  This is great if each line is a JavaScript statement, but sometimes you want to go further.  Use this option to tell Lithium to treat the file as a sequence of characters instead of a sequence of lines.</dd>\n\n<dt>--strategy=[check-only,minimize,minimize-balanced,replace-properties-by-globals,replace-arguments-by-globals,minimize-around]</dt>\n<dd>\"minimize\" is the default, the algorithm described above. \"check-only\" tries to run Lithium to determine interestingness, without reduction. For the other strategies, check out <a href=\"https://github.com/MozillaSecurity/lithium/pull/2\">this GitHub PR</a>.</dd>\n\n<dt>--repeat=[always, last, never].</dt>\n<dd>By default, Lithium only repeats at the same chunk size if it just finished the last round (e.g. chunk size 1).  You can use --repeat=always to tell it to repeat any chunk size if something was removed during the round, which can be useful for non-deterministic testcases or non-monotonic situations.  You can use --repeat=never to tell it to exit immediately after a single round at the last chunk size, which can save a little time at the risk of leaving a little bit extra in the file.</dd>\n\n<dt>--max=n. default: about half of the file.</dt>\n<dt>--min=n. default: 1.</dt>\n<dd>What chunk sizes to test.  Must be powers of two.  --max is useful if you're restarting Lithium after it has already gone through a few rounds.  --min is useful if you're reducing HTML and want to do the final by hand.</dd>\n\n<dt>--chunk-size=n</dt>\n<dd>Shortcut for \"repeat=never, min=n, max=n\".  --chunk-size=1 is a quick way to determine whether a file is 1-minimal, for example after making a change that you think might make some lines unnecessary.</dd>\n\n</dl>\n\n\n### Hints\n\nIf you find a non-deterministic bug, don't despair.  Lithium will do fine as long as you make the bug happen at least 70% of the time.  You can repeat the test either within the application, by adding a loop or reload in the testcase (outside of the DDBEGIN/DDEND markers!), or outside of the application, by adding a loop to the \"interestingness test\" script.\n\n\n### Requirements\n\nLithium is written in [Python](https://www.python.org/) and requires Python 3.5+.\n\n### Credits\n\n- [Lithium's testcase reduction algorithm](src/lithium/docs/algorithm.md) is a modified version of the \"ddmin\" algorithm in Andreas Zeller's paper, [Simplifying and Isolating Failure-Inducing Input](https://www.st.cs.uni-saarland.de/papers/tse2002/).\n- The idea of using an external \"interestingness test\" program came from [Delta](http://delta.tigris.org/), a similar tool that's [used in clever ways by the GCC project](https://gcc.gnu.org/wiki/A_guide_to_testcase_reduction).\n- [timed_run](src/lithium/interestingness/timed_run.py), used by many of the \"interestingness test\" scripts that come with Lithium, is based on [timed_run.py](https://web.archive.org/web/20071107032840/http://bclary.com/log/2007/03/07/timed_run), which was written by [Chris Cooper](http://coop.deadsquid.com/) and [Bob Clary](https://bclary.com/).\n- The code was significantly cleaned up and modernized by Jesse Schwartzentruber and Gary Kwong in mid-2017.\n",
    "bugtrack_url": null,
    "license": "MPL 2.0",
    "summary": "Lithium is an automated testcase reduction tool",
    "version": "1.1.1",
    "project_urls": {
        "Homepage": "https://github.com/MozillaSecurity/lithium"
    },
    "split_keywords": [
        "fuzz",
        "fuzzing",
        "reduce",
        "reducer",
        "reduction",
        "security",
        "test",
        "testing"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "e7cbfe3c9300e48474a89c9c65e15d31b192393b54d40d08e9216a4e4210f0f6",
                "md5": "a8027216a2c9931d18cde6e3feacef58",
                "sha256": "26a2cfc882873e83796eba7fe50139bc92f611ac2f4375c3af99dd416c06c621"
            },
            "downloads": -1,
            "filename": "lithium_reducer-1.1.1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "a8027216a2c9931d18cde6e3feacef58",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 50744,
            "upload_time": "2023-12-29T19:55:14",
            "upload_time_iso_8601": "2023-12-29T19:55:14.288946Z",
            "url": "https://files.pythonhosted.org/packages/e7/cb/fe3c9300e48474a89c9c65e15d31b192393b54d40d08e9216a4e4210f0f6/lithium_reducer-1.1.1-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "c277395d4528e6c22da422e1cc23fc01a30d2a57e361a546ac57d0ea220e0c1c",
                "md5": "9f63868723efc737084e64ec84c3943e",
                "sha256": "8ebd161546ff969b0b58c38651130cb14da9704e5ca26b3002a39b0edcfd9aa6"
            },
            "downloads": -1,
            "filename": "lithium-reducer-1.1.1.tar.gz",
            "has_sig": false,
            "md5_digest": "9f63868723efc737084e64ec84c3943e",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 55856,
            "upload_time": "2023-12-29T19:55:15",
            "upload_time_iso_8601": "2023-12-29T19:55:15.996528Z",
            "url": "https://files.pythonhosted.org/packages/c2/77/395d4528e6c22da422e1cc23fc01a30d2a57e361a546ac57d0ea220e0c1c/lithium-reducer-1.1.1.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-12-29 19:55:15",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "MozillaSecurity",
    "github_project": "lithium",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": false,
    "tox": true,
    "lcname": "lithium-reducer"
}
        
Elapsed time: 0.16768s