| Name | janus-web-ab-testing JSON |
| Version |
0.1.6
JSON |
| download |
| home_page | None |
| Summary | Janus, an A/B Test Framework. |
| upload_time | 2024-07-14 23:16:13 |
| maintainer | None |
| docs_url | None |
| author | Luan Fernandes |
| requires_python | <4.0,>=3.11 |
| license | MIT |
| keywords |
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# Janus
[](https://github.com/psf/black)
Janus is a simple and easy A/B Test Calculator that allows you to run A/B tests and get reports in a few lines of code.
It's developed mainly for website experiments. While many calculators only account for conversion analysis and consider only two variants, Janus allows multivariant tests and measure _Conversion_, _Averge Ticket_ and _Average Revenue Per User (ARPU)_ over variants, i.e, typical metrics for tests in marketplaces.
The engine name is an analogy to _Janus_, the god of changes and transitions.
## Installation
Janus is distributed in [pypi](https://pypi.org/project/janus-web-ab-testing/). You can install it by using:
```
pip install janus-web-ab-testing
```
You can now run a quick example of a A/B test:
```
from janus.variant import Variant
from janus.experiment import WebsiteExperiment
variant_A = Variant(
name="A",
impressions=1000,
conversions=100,
revenue=10000
)
variant_B = Variant(
name="B",
impressions=1000,
conversions=120,
revenue=9000
)
variants = [variant_A, variant_B]
experiment = WebsiteExperiment(variants, baseline_variant="A")
experiment.run()
experiment.print_reports()
```
And you should see:
```
---------------- Summary ----------------
+-------------+-------------+------------+
| | A | B |
+=============+=============+============+
| impressions | 1000 | 1000 |
+-------------+-------------+------------+
| conversions | 100 | 120 |
+-------------+-------------+------------+
| revenue | R$10,000.00 | R$9,000.00 |
+-------------+-------------+------------+
| conversion | 0.1 | 0.12 |
+-------------+-------------+------------+
| avg_ticket | R$100.00 | R$75.00 |
+-------------+-------------+------------+
| arpu | R$10.00 | R$9.00 |
+-------------+-------------+------------+
----------- Conversion Stats -----------
+-----------------+--------+--------+
| | A | B |
+=================+========+========+
| expected_loss | 0.0206 | 0.0005 |
+-----------------+--------+--------+
| prob_being_best | 0.0786 | 0.9214 |
+-----------------+--------+--------+
| lift | 0.0 | 0.2 |
+-----------------+--------+--------+
-------------- ARPU Stats --------------
+-----------------+--------+--------+
| | A | B |
+=================+========+========+
| expected_loss | R$0.14 | R$1.15 |
+-----------------+--------+--------+
| prob_being_best | 0.7936 | 0.2064 |
+-----------------+--------+--------+
| lift | 0.0 | -0.1 |
+-----------------+--------+--------+
```
See more in the [quickstart notebook](examples/Janus%20Quickstart.ipynb). The _Average Ticket_ Statistics is still missing, but will be added soon (it's usually modelled with exponential distribution with gamma as prior)
## The Janus Website
My plan is to build a website to showcase the Janus library and to provide a simple interface to run A/B tests. However, only the library is almost ready, and the next steps would be:
- build a backend with FastAPI using the library
- build a frontend (JavaScript maybe)
- deploy the website
> **Warning**
> The distributed package is from this branch, which will be merged to `main` in the first release of this project. The `main` branch uses another previous approach for the calculations (yet correct), and has a streamlit app to showcase the library.
> In this branch, however, we'll change the whole project towards not only a library, but also for a full stack application, with frontend and backend.
## References
* [What is A/B Testing](https://en.wikipedia.org/wiki/A/B_testing)
* [VWO paper about bayesian A/B testing](https://vwo.com/downloads/VWO_SmartStats_technical_whitepaper.pdf)
* [Agile A/B testing with Bayesian Statistics and Python](https://web.archive.org/web/20150419163005/http://www.bayesianwitch.com/blog/2014/bayesian_ab_test.html)
* [Understanding Bayesian A/B testing (using baseball statistics)](http://varianceexplained.org/r/bayesian_ab_baseball/)
* [It’s time to re-think A/B testing](https://mobiledevmemo.com/its-time-to-abandon-a-b-testing/)
* [Conjugate Priors](https://en.wikipedia.org/wiki/Conjugate_prior)
* [Curso de Teste A/B Bayesiano do Lazy Programmer](https://www.udemy.com/course/bayesian-machine-learning-in-python-ab-testing)
* [Binomial Distributions](https://www.youtube.com/watch?v=8idr1WZ1A7Q)
* [Bayes theorem](https://www.youtube.com/watch?v=HZGCoVF3YvM&t=9s)
* [The quick proof of Bayes Theorem](https://www.youtube.com/watch?v=U_85TaXbeIo)
Raw data
{
"_id": null,
"home_page": null,
"name": "janus-web-ab-testing",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.11",
"maintainer_email": null,
"keywords": null,
"author": "Luan Fernandes",
"author_email": "luangabriel70@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/12/a1/1a0d9369cb0cdd463023d1c727153e3ef3920777714d6acac59b13164069/janus_web_ab_testing-0.1.6.tar.gz",
"platform": null,
"description": "# Janus\n\n\n\n[](https://github.com/psf/black)\n\nJanus is a simple and easy A/B Test Calculator that allows you to run A/B tests and get reports in a few lines of code. \n\nIt's developed mainly for website experiments. While many calculators only account for conversion analysis and consider only two variants, Janus allows multivariant tests and measure _Conversion_, _Averge Ticket_ and _Average Revenue Per User (ARPU)_ over variants, i.e, typical metrics for tests in marketplaces. \n\nThe engine name is an analogy to _Janus_, the god of changes and transitions. \n\n## Installation\n\nJanus is distributed in [pypi](https://pypi.org/project/janus-web-ab-testing/). You can install it by using:\n```\npip install janus-web-ab-testing\n```\n\nYou can now run a quick example of a A/B test:\n```\nfrom janus.variant import Variant\nfrom janus.experiment import WebsiteExperiment\n\nvariant_A = Variant(\n name=\"A\",\n impressions=1000, \n conversions=100, \n revenue=10000\n)\nvariant_B = Variant(\n name=\"B\",\n impressions=1000, \n conversions=120, \n revenue=9000\n)\nvariants = [variant_A, variant_B]\n\nexperiment = WebsiteExperiment(variants, baseline_variant=\"A\")\nexperiment.run()\nexperiment.print_reports()\n```\nAnd you should see:\n```\n---------------- Summary ----------------\n+-------------+-------------+------------+\n| | A | B |\n+=============+=============+============+\n| impressions | 1000 | 1000 |\n+-------------+-------------+------------+\n| conversions | 100 | 120 |\n+-------------+-------------+------------+\n| revenue | R$10,000.00 | R$9,000.00 |\n+-------------+-------------+------------+\n| conversion | 0.1 | 0.12 |\n+-------------+-------------+------------+\n| avg_ticket | R$100.00 | R$75.00 |\n+-------------+-------------+------------+\n| arpu | R$10.00 | R$9.00 |\n+-------------+-------------+------------+\n----------- Conversion Stats -----------\n+-----------------+--------+--------+\n| | A | B |\n+=================+========+========+\n| expected_loss | 0.0206 | 0.0005 |\n+-----------------+--------+--------+\n| prob_being_best | 0.0786 | 0.9214 |\n+-----------------+--------+--------+\n| lift | 0.0 | 0.2 |\n+-----------------+--------+--------+\n-------------- ARPU Stats --------------\n+-----------------+--------+--------+\n| | A | B |\n+=================+========+========+\n| expected_loss | R$0.14 | R$1.15 |\n+-----------------+--------+--------+\n| prob_being_best | 0.7936 | 0.2064 |\n+-----------------+--------+--------+\n| lift | 0.0 | -0.1 |\n+-----------------+--------+--------+\n```\n\n\nSee more in the [quickstart notebook](examples/Janus%20Quickstart.ipynb). The _Average Ticket_ Statistics is still missing, but will be added soon (it's usually modelled with exponential distribution with gamma as prior) \n\n## The Janus Website\n\nMy plan is to build a website to showcase the Janus library and to provide a simple interface to run A/B tests. However, only the library is almost ready, and the next steps would be:\n- build a backend with FastAPI using the library\n- build a frontend (JavaScript maybe)\n- deploy the website\n\n> **Warning**\n> The distributed package is from this branch, which will be merged to `main` in the first release of this project. The `main` branch uses another previous approach for the calculations (yet correct), and has a streamlit app to showcase the library.\n> In this branch, however, we'll change the whole project towards not only a library, but also for a full stack application, with frontend and backend.\n\n\n## References\n* [What is A/B Testing](https://en.wikipedia.org/wiki/A/B_testing)\n* [VWO paper about bayesian A/B testing](https://vwo.com/downloads/VWO_SmartStats_technical_whitepaper.pdf)\n* [Agile A/B testing with Bayesian Statistics and Python](https://web.archive.org/web/20150419163005/http://www.bayesianwitch.com/blog/2014/bayesian_ab_test.html)\n* [Understanding Bayesian A/B testing (using baseball statistics)](http://varianceexplained.org/r/bayesian_ab_baseball/)\n* [It\u2019s time to re-think A/B testing](https://mobiledevmemo.com/its-time-to-abandon-a-b-testing/)\n* [Conjugate Priors](https://en.wikipedia.org/wiki/Conjugate_prior)\n* [Curso de Teste A/B Bayesiano do Lazy Programmer](https://www.udemy.com/course/bayesian-machine-learning-in-python-ab-testing)\n* [Binomial Distributions](https://www.youtube.com/watch?v=8idr1WZ1A7Q)\n* [Bayes theorem](https://www.youtube.com/watch?v=HZGCoVF3YvM&t=9s)\n* [The quick proof of Bayes Theorem](https://www.youtube.com/watch?v=U_85TaXbeIo)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Janus, an A/B Test Framework.",
"version": "0.1.6",
"project_urls": null,
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "49ae1fd6557ae46792eb51fa64c365049ce320bec6f9f17f2ce06026f09f3d8b",
"md5": "1f95fbaf85343867c0445a1a1699d522",
"sha256": "df97dee2baf081d802ba570133e988a95e13a530c5b0214b11597a589f306e2b"
},
"downloads": -1,
"filename": "janus_web_ab_testing-0.1.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1f95fbaf85343867c0445a1a1699d522",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.11",
"size": 6697,
"upload_time": "2024-07-14T23:16:11",
"upload_time_iso_8601": "2024-07-14T23:16:11.930319Z",
"url": "https://files.pythonhosted.org/packages/49/ae/1fd6557ae46792eb51fa64c365049ce320bec6f9f17f2ce06026f09f3d8b/janus_web_ab_testing-0.1.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "12a11a0d9369cb0cdd463023d1c727153e3ef3920777714d6acac59b13164069",
"md5": "e80c7044cbaa4abda47f9abd64e85ec3",
"sha256": "4405e6b16a06d4df5e2c4e3c2268a482deab61d2e1fa1301e3ebc492ba3a78d4"
},
"downloads": -1,
"filename": "janus_web_ab_testing-0.1.6.tar.gz",
"has_sig": false,
"md5_digest": "e80c7044cbaa4abda47f9abd64e85ec3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.11",
"size": 5727,
"upload_time": "2024-07-14T23:16:13",
"upload_time_iso_8601": "2024-07-14T23:16:13.497188Z",
"url": "https://files.pythonhosted.org/packages/12/a1/1a0d9369cb0cdd463023d1c727153e3ef3920777714d6acac59b13164069/janus_web_ab_testing-0.1.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-14 23:16:13",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "janus-web-ab-testing"
}
