.. image:: https://raw.githubusercontent.com/MechanicalSoup/MechanicalSoup/main/assets/mechanical-soup-logo.png
:alt: MechanicalSoup. A Python library for automating website interaction.
Home page
---------
https://mechanicalsoup.readthedocs.io/
Overview
--------
A Python library for automating interaction with websites.
MechanicalSoup automatically stores and sends cookies, follows
redirects, and can follow links and submit forms. It doesn't do
JavaScript.
MechanicalSoup was created by `M Hickford
<https://github.com/hickford/>`__, who was a fond user of the
`Mechanize <https://github.com/jjlee/mechanize>`__ library.
Unfortunately, Mechanize was `incompatible with Python 3 until 2019
<https://github.com/python-mechanize/mechanize/issues/9>`__ and its development
stalled for several years. MechanicalSoup provides a similar API, built on Python
giants `Requests <http://docs.python-requests.org/en/latest/>`__ (for
HTTP sessions) and `BeautifulSoup
<https://www.crummy.com/software/BeautifulSoup/>`__ (for document
navigation). Since 2017 it is a project actively maintained by a small
team including `@hemberger <https://github.com/hemberger>`__ and `@moy
<https://github.com/moy/>`__.
|Gitter Chat|
Installation
------------
|Latest Version| |Supported Versions|
PyPy3 is also supported (and tested against).
Download and install the latest released version from `PyPI <https://pypi.python.org/pypi/MechanicalSoup/>`__::
pip install MechanicalSoup
Download and install the development version from `GitHub <https://github.com/MechanicalSoup/MechanicalSoup>`__::
pip install git+https://github.com/MechanicalSoup/MechanicalSoup
Installing from source (installs the version in the current working directory)::
python setup.py install
(In all cases, add ``--user`` to the ``install`` command to
install in the current user's home directory.)
Documentation
-------------
The full documentation is available on
https://mechanicalsoup.readthedocs.io/. You may want to jump directly to
the `automatically generated API
documentation <https://mechanicalsoup.readthedocs.io/en/stable/mechanicalsoup.html>`__.
Example
-------
From `examples/expl_qwant.py <https://github.com/MechanicalSoup/MechanicalSoup/blob/main/examples/expl_qwant.py>`__, code to get the results from
a Qwant search:
.. code:: python
"""Example usage of MechanicalSoup to get the results from the Qwant
search engine.
"""
import re
import mechanicalsoup
import html
import urllib.parse
# Connect to Qwant
browser = mechanicalsoup.StatefulBrowser(user_agent='MechanicalSoup')
browser.open("https://lite.qwant.com/")
# Fill-in the search form
browser.select_form('#search-form')
browser["q"] = "MechanicalSoup"
browser.submit_selected()
# Display the results
for link in browser.page.select('.result a'):
# Qwant shows redirection links, not the actual URL, so extract
# the actual URL from the redirect link:
href = link.attrs['href']
m = re.match(r"^/redirect/[^/]*/(.*)$", href)
if m:
href = urllib.parse.unquote(m.group(1))
print(link.text, '->', href)
More examples are available in `examples/ <https://github.com/MechanicalSoup/MechanicalSoup/blob/main/examples/>`__.
For an example with a more complex form (checkboxes, radio buttons and
textareas), read `tests/test_browser.py <https://github.com/MechanicalSoup/MechanicalSoup/blob/main/tests/test_browser.py>`__
and `tests/test_form.py <https://github.com/MechanicalSoup/MechanicalSoup/blob/main/tests/test_form.py>`__.
Development
-----------
|Build Status|
|Coverage Status|
|Documentation Status|
|CII Best Practices|
Instructions for building, testing and contributing to MechanicalSoup:
see `CONTRIBUTING.rst <https://github.com/MechanicalSoup/MechanicalSoup/blob/main/CONTRIBUTING.rst>`__.
Common problems
---------------
Read the `FAQ
<https://mechanicalsoup.readthedocs.io/en/stable/faq.html>`__.
.. |Latest Version| image:: https://img.shields.io/pypi/v/MechanicalSoup.svg
:target: https://pypi.python.org/pypi/MechanicalSoup/
.. |Supported Versions| image:: https://img.shields.io/pypi/pyversions/mechanicalsoup.svg
:target: https://pypi.python.org/pypi/MechanicalSoup/
.. |Build Status| image:: https://github.com/MechanicalSoup/MechanicalSoup/actions/workflows/python-package.yml/badge.svg?branch=main
:target: https://github.com/MechanicalSoup/MechanicalSoup/actions/workflows/python-package.yml?query=branch%3Amain
.. |Coverage Status| image:: https://codecov.io/gh/MechanicalSoup/MechanicalSoup/branch/main/graph/badge.svg
:target: https://codecov.io/gh/MechanicalSoup/MechanicalSoup
.. |Documentation Status| image:: https://readthedocs.org/projects/mechanicalsoup/badge/?version=latest
:target: https://mechanicalsoup.readthedocs.io/en/latest/?badge=latest
.. |CII Best Practices| image:: https://bestpractices.coreinfrastructure.org/projects/1334/badge
:target: https://bestpractices.coreinfrastructure.org/projects/1334
.. |Gitter Chat| image:: https://badges.gitter.im/MechanicalSoup/MechanicalSoup.svg
:target: https://gitter.im/MechanicalSoup/Lobby
Raw data
{
"_id": null,
"home_page": "https://mechanicalsoup.readthedocs.io/",
"name": "MechanicalSoup",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "",
"author": "",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/37/11/61d95339e23b5e6fe3b6bbf2782fd65394eac2af79c42b49c13e216f2bed/MechanicalSoup-1.3.0.tar.gz",
"platform": null,
"description": ".. image:: https://raw.githubusercontent.com/MechanicalSoup/MechanicalSoup/main/assets/mechanical-soup-logo.png\n :alt: MechanicalSoup. A Python library for automating website interaction.\n\nHome page\n---------\n\nhttps://mechanicalsoup.readthedocs.io/\n\nOverview\n--------\n\nA Python library for automating interaction with websites.\nMechanicalSoup automatically stores and sends cookies, follows\nredirects, and can follow links and submit forms. It doesn't do\nJavaScript.\n\nMechanicalSoup was created by `M Hickford\n<https://github.com/hickford/>`__, who was a fond user of the\n`Mechanize <https://github.com/jjlee/mechanize>`__ library.\nUnfortunately, Mechanize was `incompatible with Python 3 until 2019\n<https://github.com/python-mechanize/mechanize/issues/9>`__ and its development\nstalled for several years. MechanicalSoup provides a similar API, built on Python\ngiants `Requests <http://docs.python-requests.org/en/latest/>`__ (for\nHTTP sessions) and `BeautifulSoup\n<https://www.crummy.com/software/BeautifulSoup/>`__ (for document\nnavigation). Since 2017 it is a project actively maintained by a small\nteam including `@hemberger <https://github.com/hemberger>`__ and `@moy\n<https://github.com/moy/>`__.\n\n|Gitter Chat|\n\nInstallation\n------------\n\n|Latest Version| |Supported Versions|\n\nPyPy3 is also supported (and tested against).\n\nDownload and install the latest released version from `PyPI <https://pypi.python.org/pypi/MechanicalSoup/>`__::\n\n pip install MechanicalSoup\n\nDownload and install the development version from `GitHub <https://github.com/MechanicalSoup/MechanicalSoup>`__::\n\n pip install git+https://github.com/MechanicalSoup/MechanicalSoup\n\nInstalling from source (installs the version in the current working directory)::\n\n python setup.py install\n\n(In all cases, add ``--user`` to the ``install`` command to\ninstall in the current user's home directory.)\n\nDocumentation\n-------------\n\nThe full documentation is available on\nhttps://mechanicalsoup.readthedocs.io/. You may want to jump directly to\nthe `automatically generated API\ndocumentation <https://mechanicalsoup.readthedocs.io/en/stable/mechanicalsoup.html>`__.\n\nExample\n-------\n\nFrom `examples/expl_qwant.py <https://github.com/MechanicalSoup/MechanicalSoup/blob/main/examples/expl_qwant.py>`__, code to get the results from\na Qwant search:\n\n.. code:: python\n\n \"\"\"Example usage of MechanicalSoup to get the results from the Qwant\n search engine.\n \"\"\"\n\n import re\n import mechanicalsoup\n import html\n import urllib.parse\n\n # Connect to Qwant\n browser = mechanicalsoup.StatefulBrowser(user_agent='MechanicalSoup')\n browser.open(\"https://lite.qwant.com/\")\n\n # Fill-in the search form\n browser.select_form('#search-form')\n browser[\"q\"] = \"MechanicalSoup\"\n browser.submit_selected()\n\n # Display the results\n for link in browser.page.select('.result a'):\n # Qwant shows redirection links, not the actual URL, so extract\n # the actual URL from the redirect link:\n href = link.attrs['href']\n m = re.match(r\"^/redirect/[^/]*/(.*)$\", href)\n if m:\n href = urllib.parse.unquote(m.group(1))\n print(link.text, '->', href)\n\nMore examples are available in `examples/ <https://github.com/MechanicalSoup/MechanicalSoup/blob/main/examples/>`__.\n\nFor an example with a more complex form (checkboxes, radio buttons and\ntextareas), read `tests/test_browser.py <https://github.com/MechanicalSoup/MechanicalSoup/blob/main/tests/test_browser.py>`__\nand `tests/test_form.py <https://github.com/MechanicalSoup/MechanicalSoup/blob/main/tests/test_form.py>`__.\n\nDevelopment\n-----------\n\n|Build Status|\n|Coverage Status|\n|Documentation Status|\n|CII Best Practices|\n\nInstructions for building, testing and contributing to MechanicalSoup:\nsee `CONTRIBUTING.rst <https://github.com/MechanicalSoup/MechanicalSoup/blob/main/CONTRIBUTING.rst>`__.\n\nCommon problems\n---------------\n\nRead the `FAQ\n<https://mechanicalsoup.readthedocs.io/en/stable/faq.html>`__.\n\n.. |Latest Version| image:: https://img.shields.io/pypi/v/MechanicalSoup.svg\n :target: https://pypi.python.org/pypi/MechanicalSoup/\n.. |Supported Versions| image:: https://img.shields.io/pypi/pyversions/mechanicalsoup.svg\n :target: https://pypi.python.org/pypi/MechanicalSoup/\n.. |Build Status| image:: https://github.com/MechanicalSoup/MechanicalSoup/actions/workflows/python-package.yml/badge.svg?branch=main\n :target: https://github.com/MechanicalSoup/MechanicalSoup/actions/workflows/python-package.yml?query=branch%3Amain\n.. |Coverage Status| image:: https://codecov.io/gh/MechanicalSoup/MechanicalSoup/branch/main/graph/badge.svg\n :target: https://codecov.io/gh/MechanicalSoup/MechanicalSoup\n.. |Documentation Status| image:: https://readthedocs.org/projects/mechanicalsoup/badge/?version=latest\n :target: https://mechanicalsoup.readthedocs.io/en/latest/?badge=latest\n.. |CII Best Practices| image:: https://bestpractices.coreinfrastructure.org/projects/1334/badge\n :target: https://bestpractices.coreinfrastructure.org/projects/1334\n.. |Gitter Chat| image:: https://badges.gitter.im/MechanicalSoup/MechanicalSoup.svg\n :target: https://gitter.im/MechanicalSoup/Lobby\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Python library for automating interaction with websites",
"version": "1.3.0",
"project_urls": {
"Homepage": "https://mechanicalsoup.readthedocs.io/",
"Source": "https://github.com/MechanicalSoup/MechanicalSoup"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "d326e48f1499547475659c49d4d8143b26b9488a0358997bfba13617d4e7dbe7",
"md5": "08d4c0180ce727e190f4426484f3c8df",
"sha256": "83dfc23bbbcaafb62dd43e0f12aee3202e780650b4612d999b54324558980114"
},
"downloads": -1,
"filename": "MechanicalSoup-1.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "08d4c0180ce727e190f4426484f3c8df",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 19963,
"upload_time": "2023-07-04T19:22:09",
"upload_time_iso_8601": "2023-07-04T19:22:09.929033Z",
"url": "https://files.pythonhosted.org/packages/d3/26/e48f1499547475659c49d4d8143b26b9488a0358997bfba13617d4e7dbe7/MechanicalSoup-1.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "371161d95339e23b5e6fe3b6bbf2782fd65394eac2af79c42b49c13e216f2bed",
"md5": "b144e9f9b4e3c19f9d06dc62343a960d",
"sha256": "38e8748f62fd9455a0818701a9e2dbfa549639d09f829f3fdd03665c825e7ce1"
},
"downloads": -1,
"filename": "MechanicalSoup-1.3.0.tar.gz",
"has_sig": false,
"md5_digest": "b144e9f9b4e3c19f9d06dc62343a960d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 50826,
"upload_time": "2023-07-04T19:22:11",
"upload_time_iso_8601": "2023-07-04T19:22:11.775200Z",
"url": "https://files.pythonhosted.org/packages/37/11/61d95339e23b5e6fe3b6bbf2782fd65394eac2af79c42b49c13e216f2bed/MechanicalSoup-1.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-04 19:22:11",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "MechanicalSoup",
"github_project": "MechanicalSoup",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"lcname": "mechanicalsoup"
}
JSON
