Fetch using your browser.
Let the browser manage cookies for you.
⚠️ Incomplete. Not tested thoroughly. Consider using `Playwright`_, especially for more complex scenarios.
Usage
-----
1. You'll run a Python script containing some code like this:
.. code-block:: python
from asyncio import gather, new_event_loop
from browserfetch import fetch, get, post, run_server
async def main():
response1, response2, reponse3 = await gather(
get('https://example.com/path1', params={'a': 1}),
fetch('https://example.com/image.png'),
post('https://example.com/path2', data={'a': 1}),
)
# do stuff with retrieved responses
loop = new_event_loop()
loop.create_task(start_server())
loop.run_until_complete(main())
2. Open your browser, goto http://example.com (perhaps solve a captcha and log in).
3. Copy the contents of `browserfetch.js`_ file and paste it in browser's console. (You can use a browser extensions like violentmonkey_/tampermonkey_ to do this step for you.)
That's it! Your Python script starts handling requests.
The browser tab should remain open of-coarse.
The server can handle multiple websocket connections from different websites simultaneously.
How it works
------------
``browserfetch`` communicates with your browser using a websocket. The ``fetch`` function just passes the request to browser and it is the browser that handles the actual request. Response data is sent back to Python using the same WebSocket connection.
Motivations
-----------
* `browser_cookie3 stopped working on Chrome-based browsers`_. There is a workaround: ShadowCopy, but it requires admin privilege.
* Another issue with browser_cookie's approach is that it retrieves cookies from cookie files, but these files are not updated instantly. Thus, you might have to wait or retry a few times before you can successfully access newly set cookies.
* ShadowCopying and File access are slow and inefficient operations.
Downsides
---------
* Setting up ``browserfetch`` is more cumbersome since it requires running a Python server and also injecting a small script into the webpage. Using ``browser_cookie3`` might be a better choice if there are many websites that you need to communicate with.
.. _playwright: https://playwright.dev/python/docs/intro
.. _`browser_cookie3 stopped working on Chrome-based browsers`: https://github.com/borisbabic/browser_cookie3/issues/180
.. _tampermonkey: https://github.com/Tampermonkey/tampermonkey
.. _violentmonkey: https://github.com/violentmonkey/violentmonkey
.. _browserfetch.js: https://github.com/5j9/browserfetch/blob/master/browserfetch/browserfetch.js
Raw data
{
"_id": null,
"home_page": null,
"name": "browserfetch",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": null,
"keywords": "browser, fetch, python, cookies",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/88/47/7b269284068afc972c0c79c0a3b3af5b608252410a6befe2477e0b95ad0f/browserfetch-0.14.0.tar.gz",
"platform": null,
"description": "Fetch using your browser.\n\nLet the browser manage cookies for you.\n\n\u26a0\ufe0f Incomplete. Not tested thoroughly. Consider using `Playwright`_, especially for more complex scenarios.\n\nUsage\n-----\n1. You'll run a Python script containing some code like this:\n\n.. code-block:: python\n\n from asyncio import gather, new_event_loop\n\n from browserfetch import fetch, get, post, run_server\n\n\n async def main():\n response1, response2, reponse3 = await gather(\n get('https://example.com/path1', params={'a': 1}),\n fetch('https://example.com/image.png'),\n post('https://example.com/path2', data={'a': 1}),\n )\n # do stuff with retrieved responses\n\n\n loop = new_event_loop()\n loop.create_task(start_server())\n loop.run_until_complete(main())\n\n\n2. Open your browser, goto http://example.com (perhaps solve a captcha and log in).\n3. Copy the contents of `browserfetch.js`_ file and paste it in browser's console. (You can use a browser extensions like violentmonkey_/tampermonkey_ to do this step for you.)\n\nThat's it! Your Python script starts handling requests.\nThe browser tab should remain open of-coarse.\n\nThe server can handle multiple websocket connections from different websites simultaneously.\n\nHow it works\n------------\n``browserfetch`` communicates with your browser using a websocket. The ``fetch`` function just passes the request to browser and it is the browser that handles the actual request. Response data is sent back to Python using the same WebSocket connection.\n\nMotivations\n-----------\n* `browser_cookie3 stopped working on Chrome-based browsers`_. There is a workaround: ShadowCopy, but it requires admin privilege.\n* Another issue with browser_cookie's approach is that it retrieves cookies from cookie files, but these files are not updated instantly. Thus, you might have to wait or retry a few times before you can successfully access newly set cookies.\n* ShadowCopying and File access are slow and inefficient operations.\n\nDownsides\n---------\n* Setting up ``browserfetch`` is more cumbersome since it requires running a Python server and also injecting a small script into the webpage. Using ``browser_cookie3`` might be a better choice if there are many websites that you need to communicate with.\n\n.. _playwright: https://playwright.dev/python/docs/intro\n.. _`browser_cookie3 stopped working on Chrome-based browsers`: https://github.com/borisbabic/browser_cookie3/issues/180\n.. _tampermonkey: https://github.com/Tampermonkey/tampermonkey\n.. _violentmonkey: https://github.com/violentmonkey/violentmonkey\n.. _browserfetch.js: https://github.com/5j9/browserfetch/blob/master/browserfetch/browserfetch.js\n",
"bugtrack_url": null,
"license": "GNU General Public License v3 (GPLv3)",
"summary": "fetch in Python using your browser!",
"version": "0.14.0",
"project_urls": {
"Homepage": "https://github.com/5j9/browserfetch"
},
"split_keywords": [
"browser",
" fetch",
" python",
" cookies"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "d0f44842ca1175bac850e21d5ae76a540777e931abf70f6ac542568824daa185",
"md5": "8cf7a255cf2e27ab652c7f4ae1711f24",
"sha256": "a143f4a76fd99c290599bb71636f0e669a60257e7c00c97e54f35b2cd3bfa232"
},
"downloads": -1,
"filename": "browserfetch-0.14.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8cf7a255cf2e27ab652c7f4ae1711f24",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 11393,
"upload_time": "2025-09-16T18:35:43",
"upload_time_iso_8601": "2025-09-16T18:35:43.624516Z",
"url": "https://files.pythonhosted.org/packages/d0/f4/4842ca1175bac850e21d5ae76a540777e931abf70f6ac542568824daa185/browserfetch-0.14.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "88477b269284068afc972c0c79c0a3b3af5b608252410a6befe2477e0b95ad0f",
"md5": "d84191d91325fb24f19d2c1196e58fa3",
"sha256": "22e8a3680dd65b1743394a4fbe8dd055641b61f9ec8bc116f6b727d9e53381d6"
},
"downloads": -1,
"filename": "browserfetch-0.14.0.tar.gz",
"has_sig": false,
"md5_digest": "d84191d91325fb24f19d2c1196e58fa3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 8323,
"upload_time": "2025-09-16T18:35:45",
"upload_time_iso_8601": "2025-09-16T18:35:45.117881Z",
"url": "https://files.pythonhosted.org/packages/88/47/7b269284068afc972c0c79c0a3b3af5b608252410a6befe2477e0b95ad0f/browserfetch-0.14.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-09-16 18:35:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "5j9",
"github_project": "browserfetch",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "browserfetch"
}
JSON
