booktest


Namebooktest JSON
Version 0.3.30 PyPI version JSON
download
home_pagehttps://github.com/lumoa-oss/booktest
SummaryBooktest is a snapshot testing library for review driven testing.
upload_time2024-05-23 11:53:29
maintainerNone
docs_urlNone
authorAntti Rauhala
requires_python<4.0,>=3.8
licenseNone
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage
            # Booktest

booktest is review-driven testing tool that combines Jupyterbook style data science
development with traditional regression testing. Booktest is developed by 
[Lumoa.me](https://lumoa.me), the actionable feedback analytics platform.

booktest is designed to tackle a common problem with the data science
RnD work flows and regression testing: 

 * Data science produces results such as probability estimates, which can be 
   good or bad, but not really right or wrong as in the traditional software engineering. 
     * Because the DS results are not strictly right or wrong, it's very difficult to use assertions 
       for quality assurance and preventing regression.
     * For example, you cannot really say that accuracy 0.84 is correct, while the 
       accuracy 0.83 is incorrect, especially if you have other measurements (log likelihood)
       giving conflicting results. Neither evaluating a topic model as correct or incorrect
       is non-sensical. In practice, most data science applications require an expert 
       review.
     * This less ambigious quality also creates need for a better visibility of how
       the system behaves. One typically wants to print out edge cases and their diagnostics
       to see the behavior, see intermediate steps and see the results for different data sets . 
 * There is also the problem of the data science data being big and the intermediate 
   results being computationally expensive. 
     * Jupyter notebook deals with this problem by keeping the state in memory between runs, while 
       traditional unittests tend to lose the program state between runs. This leads to very slow 
       test runs, slow iteration speed and low productivity.
 * While the Jupyter Notebook provides good visibility to results required by the expert review and
   powerful caching functionality: it fails short on a) often requiring copy-pasting production code to 
   make results visible, b) it doesn't support automated regression testing and c) expert review requires
   expensive full review even if nothing changed.

booktest solves this problem setting by delivering on 3 main points:

 * Focus on the results and analytic as in Jupyter notebook by allowing user to print
   the results as MD files. 
 * Keep the intermediate results cached either in memory or in filesystem by
   having two level cache.
 * Instead of doing strict assertions, do testing by comparing old results with 
   new results.

As such, booktest does snapshot testing, and it stores the snapshots in filesystem and in Git. 
Additional benefit of this approach is that you can trace the result development in Git.

# Getting started guide

You can find getting started guide [here](getting-started.md)

# Workflows, coverage and CI

You can find guide on common workflows, coverage measurements and 
continuous integratio [here](workflows.md)

# Examples

Examples are found in the [test example directory](test/examples). 

Example results are visible in the [book index](books/index.md).

There is also [separate example project](https://github.com/lumoa-oss/booktest-example)


# API reference

API reference is generated under [docs](docs) directory. Main classes are:

 * [TestCaseRun](docs/testcaserun.py.md), which provides API for tests
 * [TestBook](docs/testbook.py.md), which provide a base class for test suite object
 * [TestSuite](docs/testsuite.py.md), which provide a base class for test suite object
 * [Tests](docs/tests.py.md), which manages CLI interface

# Developing booktest

Development guide is available [here](development.md)


            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/lumoa-oss/booktest",
    "name": "booktest",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<4.0,>=3.8",
    "maintainer_email": null,
    "keywords": null,
    "author": "Antti Rauhala",
    "author_email": "antti@lumoa.me",
    "download_url": "https://files.pythonhosted.org/packages/0c/f1/3b0921ee0ee61f7c1524d79c05c514abbd017f010f938365714be63c8dfb/booktest-0.3.30.tar.gz",
    "platform": null,
    "description": "# Booktest\n\nbooktest is review-driven testing tool that combines Jupyterbook style data science\ndevelopment with traditional regression testing. Booktest is developed by \n[Lumoa.me](https://lumoa.me), the actionable feedback analytics platform.\n\nbooktest is designed to tackle a common problem with the data science\nRnD work flows and regression testing: \n\n * Data science produces results such as probability estimates, which can be \n   good or bad, but not really right or wrong as in the traditional software engineering. \n     * Because the DS results are not strictly right or wrong, it's very difficult to use assertions \n       for quality assurance and preventing regression.\n     * For example, you cannot really say that accuracy 0.84 is correct, while the \n       accuracy 0.83 is incorrect, especially if you have other measurements (log likelihood)\n       giving conflicting results. Neither evaluating a topic model as correct or incorrect\n       is non-sensical. In practice, most data science applications require an expert \n       review.\n     * This less ambigious quality also creates need for a better visibility of how\n       the system behaves. One typically wants to print out edge cases and their diagnostics\n       to see the behavior, see intermediate steps and see the results for different data sets . \n * There is also the problem of the data science data being big and the intermediate \n   results being computationally expensive. \n     * Jupyter notebook deals with this problem by keeping the state in memory between runs, while \n       traditional unittests tend to lose the program state between runs. This leads to very slow \n       test runs, slow iteration speed and low productivity.\n * While the Jupyter Notebook provides good visibility to results required by the expert review and\n   powerful caching functionality: it fails short on a) often requiring copy-pasting production code to \n   make results visible, b) it doesn't support automated regression testing and c) expert review requires\n   expensive full review even if nothing changed.\n\nbooktest solves this problem setting by delivering on 3 main points:\n\n * Focus on the results and analytic as in Jupyter notebook by allowing user to print\n   the results as MD files. \n * Keep the intermediate results cached either in memory or in filesystem by\n   having two level cache.\n * Instead of doing strict assertions, do testing by comparing old results with \n   new results.\n\nAs such, booktest does snapshot testing, and it stores the snapshots in filesystem and in Git. \nAdditional benefit of this approach is that you can trace the result development in Git.\n\n# Getting started guide\n\nYou can find getting started guide [here](getting-started.md)\n\n# Workflows, coverage and CI\n\nYou can find guide on common workflows, coverage measurements and \ncontinuous integratio [here](workflows.md)\n\n# Examples\n\nExamples are found in the [test example directory](test/examples). \n\nExample results are visible in the [book index](books/index.md).\n\nThere is also [separate example project](https://github.com/lumoa-oss/booktest-example)\n\n\n# API reference\n\nAPI reference is generated under [docs](docs) directory. Main classes are:\n\n * [TestCaseRun](docs/testcaserun.py.md), which provides API for tests\n * [TestBook](docs/testbook.py.md), which provide a base class for test suite object\n * [TestSuite](docs/testsuite.py.md), which provide a base class for test suite object\n * [Tests](docs/tests.py.md), which manages CLI interface\n\n# Developing booktest\n\nDevelopment guide is available [here](development.md)\n\n",
    "bugtrack_url": null,
    "license": null,
    "summary": "Booktest is a snapshot testing library for review driven testing.",
    "version": "0.3.30",
    "project_urls": {
        "Homepage": "https://github.com/lumoa-oss/booktest",
        "Repository": "https://github.com/lumoa-oss/booktest"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "606c25809802ed15a0f65758be5d6a603b217c7e1952a7b661d36d1283f8df6d",
                "md5": "e49650e7d9d764f5af65dcb527d60998",
                "sha256": "8afb1862c81b7d48d8e0e1c8fae8cad678791e6239dc7b0a80c80555ec2035fd"
            },
            "downloads": -1,
            "filename": "booktest-0.3.30-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "e49650e7d9d764f5af65dcb527d60998",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<4.0,>=3.8",
            "size": 43237,
            "upload_time": "2024-05-23T11:53:27",
            "upload_time_iso_8601": "2024-05-23T11:53:27.339779Z",
            "url": "https://files.pythonhosted.org/packages/60/6c/25809802ed15a0f65758be5d6a603b217c7e1952a7b661d36d1283f8df6d/booktest-0.3.30-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "0cf13b0921ee0ee61f7c1524d79c05c514abbd017f010f938365714be63c8dfb",
                "md5": "ca1ae037aa943c5143bdb6a93390e942",
                "sha256": "9770e7b0f7a5131631ebcb1a182aefd60534f0a6fa42c9a07dfbfd1aefd8f242"
            },
            "downloads": -1,
            "filename": "booktest-0.3.30.tar.gz",
            "has_sig": false,
            "md5_digest": "ca1ae037aa943c5143bdb6a93390e942",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<4.0,>=3.8",
            "size": 32945,
            "upload_time": "2024-05-23T11:53:29",
            "upload_time_iso_8601": "2024-05-23T11:53:29.250512Z",
            "url": "https://files.pythonhosted.org/packages/0c/f1/3b0921ee0ee61f7c1524d79c05c514abbd017f010f938365714be63c8dfb/booktest-0.3.30.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-05-23 11:53:29",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "lumoa-oss",
    "github_project": "booktest",
    "travis_ci": false,
    "coveralls": true,
    "github_actions": true,
    "lcname": "booktest"
}
        
Elapsed time: 0.25018s