Fetch using your browser.
Let the browser manage cookies for you.
⚠️ This project is a very simple implementation. Not tested thoroughly. Consider it a proof of concept.
Usage
-----
1. You'll run a Python script containing some code like this:
.. code-block:: python
from asyncio import gather, new_event_loop
from browserfetch import fetch, get, post, run_server
async def main():
response1, response2, reponse3 = await gather(
get('https://example.com/path1', params={'a': 1}),
fetch('https://example.com/image.png'),
post('https://example.com/path2', data={'a': 1}),
)
# do stuff with retrieved responses
loop = new_event_loop()
loop.create_task(start_server())
loop.run_until_complete(main())
2. Open your browser, goto http://example.com (perhaps solve a captcha and log in).
3. Copy the contents of `browserfetch.js`_ file and paste it in browser's console. (You can use a browser extensions like violentmonkey_/tampermonkey_ to do this step for you.)
That's it! Your Python script starts handling requests.
The browser tab should remain open of-coarse.
The server can handle multiple websocket connections from different websites simultaneously.
How it works
------------
``browserfetch`` communicates with your browser using a websocket. The ``fetch`` function just passes the request to browser and it is the browser that handles the actual request. Response data is sent back to Python using the same WebSocket connection.
Motivations
-----------
* `browser_cookie3 stopped working on Chrome-based browsers`_. There is a workaround: ShadowCopy, but it requires admin privilege.
* Another issue with browser_cookie's approach is that it retrieves cookies from cookie files, but these files are not updated instantly. Thus, you might have to wait or retry a few times before you can successfully access newly set cookies.
* ShadowCopying and File access are slow and inefficient operations.
Downsides
---------
* Setting up ``browserfetch`` is more cumbersome since it requires running a Python server and also injecting a small script into the webpage. Using ``browser_cookie3`` might be a better choice if there are many websites that you need to communicate with.
.. _`browser_cookie3 stopped working on Chrome-based browsers`: https://github.com/borisbabic/browser_cookie3/issues/180
.. _tampermonkey: https://github.com/Tampermonkey/tampermonkey
.. _violentmonkey: https://github.com/violentmonkey/violentmonkey
.. _browserfetch.js: https://github.com/5j9/browserfetch/blob/master/browserfetch/browserfetch.js
Raw data
{
"_id": null,
"home_page": "",
"name": "browserfetch",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.11",
"maintainer_email": "",
"keywords": "browser,fetch,python,cookies",
"author": "",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/96/25/2702c8822f97d56ef93726ed34399444bb54d1b282b8f6d73c91b9957bd5/browserfetch-0.7.0.tar.gz",
"platform": null,
"description": "Fetch using your browser.\r\n\r\nLet the browser manage cookies for you.\r\n\r\n\u26a0\ufe0f This project is a very simple implementation. Not tested thoroughly. Consider it a proof of concept.\r\n\r\nUsage\r\n-----\r\n1. You'll run a Python script containing some code like this:\r\n\r\n.. code-block:: python\r\n\r\n from asyncio import gather, new_event_loop\r\n\r\n from browserfetch import fetch, get, post, run_server\r\n\r\n\r\n async def main():\r\n response1, response2, reponse3 = await gather(\r\n get('https://example.com/path1', params={'a': 1}),\r\n fetch('https://example.com/image.png'),\r\n post('https://example.com/path2', data={'a': 1}),\r\n )\r\n # do stuff with retrieved responses\r\n\r\n\r\n loop = new_event_loop()\r\n loop.create_task(start_server())\r\n loop.run_until_complete(main())\r\n\r\n\r\n2. Open your browser, goto http://example.com (perhaps solve a captcha and log in).\r\n3. Copy the contents of `browserfetch.js`_ file and paste it in browser's console. (You can use a browser extensions like violentmonkey_/tampermonkey_ to do this step for you.)\r\n\r\nThat's it! Your Python script starts handling requests.\r\nThe browser tab should remain open of-coarse.\r\n\r\nThe server can handle multiple websocket connections from different websites simultaneously.\r\n\r\nHow it works\r\n------------\r\n``browserfetch`` communicates with your browser using a websocket. The ``fetch`` function just passes the request to browser and it is the browser that handles the actual request. Response data is sent back to Python using the same WebSocket connection.\r\n\r\nMotivations\r\n-----------\r\n* `browser_cookie3 stopped working on Chrome-based browsers`_. There is a workaround: ShadowCopy, but it requires admin privilege.\r\n* Another issue with browser_cookie's approach is that it retrieves cookies from cookie files, but these files are not updated instantly. Thus, you might have to wait or retry a few times before you can successfully access newly set cookies.\r\n* ShadowCopying and File access are slow and inefficient operations.\r\n\r\nDownsides\r\n---------\r\n* Setting up ``browserfetch`` is more cumbersome since it requires running a Python server and also injecting a small script into the webpage. Using ``browser_cookie3`` might be a better choice if there are many websites that you need to communicate with.\r\n\r\n.. _`browser_cookie3 stopped working on Chrome-based browsers`: https://github.com/borisbabic/browser_cookie3/issues/180\r\n.. _tampermonkey: https://github.com/Tampermonkey/tampermonkey\r\n.. _violentmonkey: https://github.com/violentmonkey/violentmonkey\r\n.. _browserfetch.js: https://github.com/5j9/browserfetch/blob/master/browserfetch/browserfetch.js\r\n",
"bugtrack_url": null,
"license": "GNU General Public License v3 (GPLv3)",
"summary": "fetch in Python using your browser!",
"version": "0.7.0",
"project_urls": {
"Homepage": "https://github.com/5j9/browserfetch"
},
"split_keywords": [
"browser",
"fetch",
"python",
"cookies"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ee330cb66598e9da1533d28ff63dc222c4888bcb1e7636f5ab0224558474f3fe",
"md5": "1b566a2a1d27dc2559729c736f72c15e",
"sha256": "14ebd5e58360963b58e6d7801f2252071f6ca3ae8e7dea70c2b17341d1626093"
},
"downloads": -1,
"filename": "browserfetch-0.7.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1b566a2a1d27dc2559729c736f72c15e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.11",
"size": 19013,
"upload_time": "2023-11-16T12:30:37",
"upload_time_iso_8601": "2023-11-16T12:30:37.543621Z",
"url": "https://files.pythonhosted.org/packages/ee/33/0cb66598e9da1533d28ff63dc222c4888bcb1e7636f5ab0224558474f3fe/browserfetch-0.7.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "96252702c8822f97d56ef93726ed34399444bb54d1b282b8f6d73c91b9957bd5",
"md5": "b907f39fac78e022379fc0f95f92b139",
"sha256": "fe2198414dde7581de7eca446d99c042fc8ca4bbfc567811434d8a44e40fcc21"
},
"downloads": -1,
"filename": "browserfetch-0.7.0.tar.gz",
"has_sig": false,
"md5_digest": "b907f39fac78e022379fc0f95f92b139",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.11",
"size": 18712,
"upload_time": "2023-11-16T12:30:41",
"upload_time_iso_8601": "2023-11-16T12:30:41.789730Z",
"url": "https://files.pythonhosted.org/packages/96/25/2702c8822f97d56ef93726ed34399444bb54d1b282b8f6d73c91b9957bd5/browserfetch-0.7.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-16 12:30:41",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "5j9",
"github_project": "browserfetch",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "browserfetch"
}
JSON
