# Python CDP
#### Currently supports CDP [r1179426][2] (Chrome 117).
Python CDP Generator (shortened to PyCDP) is a library that provides
Python wrappers for the types, commands, and events specified in the [Chrome
DevTools Protocol][1].
The Chrome DevTools Protocol provides for remote control of a web browser by
sending JSON messages over a WebSocket. That JSON format is described by a
machine-readable specification. This specification is used to automatically
generate the classes and methods found in this library.
## Installation
You can install this library as a dependency on your project with:
```
pip install git+https://github.com/HMaker/python-recentx.git@latest
```
Change the git tag `@latest` if you need another version. To install for development, clone this
repository, install [Poetry][5] package manager and run `poetry install` to install dependencies.
## Usage
If all you want is automate Chrome right now, PyCDP includes a low-level client for asyncio and twisted:
```python
import asyncio
from pycdp import cdp
from pycdp.browser import ChromeLauncher
from pycdp.asyncio import connect_cdp
async def main():
chrome = ChromeLauncher(
binary='/usr/bin/google-chrome', # linux path
args=['--remote-debugging-port=9222', '--incognito']
)
# ChromeLauncher.launch() is blocking, run it on a background thread
await asyncio.get_running_loop().run_in_executor(None, chrome.launch)
conn = await connect_cdp('http://localhost:9222')
target_id = await conn.execute(cdp.target.create_target('about:blank'))
target_session = await conn.connect_session(target_id)
await target_session.execute(cdp.page.enable())
# you may use "async for target_session.listen()" to listen multiple events, here we listen just a single event.
with target_session.safe_wait_for(cdp.page.DomContentEventFired) as navigation:
await target_session.execute(cdp.page.navigate('https://chromedevtools.github.io/devtools-protocol/'))
await navigation
dom = await target_session.execute(cdp.dom.get_document())
node = await target_session.execute(cdp.dom.query_selector(dom.node_id, 'p'))
js_node = await target_session.execute(cdp.dom.resolve_node(node))
print((await target_session.execute(cdp.runtime.call_function_on('function() {return this.innerText;}', js_node.object_id, return_by_value=True)))[0].value)
await target_session.execute(cdp.page.close())
await conn.close()
await asyncio.get_running_loop().run_in_executor(None, chrome.kill)
asyncio.run(main())
```
the twisted client requires [twisted][6] and [autobahn][7] packages:
```python
from twisted.python.log import err
from twisted.internet import reactor, defer, threads
from pycdp import cdp
from pycdp.browser import ChromeLauncher
from pycdp.twisted import connect_cdp
async def main():
chrome = ChromeLauncher(
binary='C:\Program Files\Google\Chrome\Application\chrome.exe', # windows path
args=['--remote-debugging-port=9222', '--incognito']
)
await threads.deferToThread(chrome.launch)
conn = await connect_cdp('http://localhost:9222', reactor)
target_id = await conn.execute(cdp.target.create_target('about:blank'))
target_session = await conn.connect_session(target_id)
await target_session.execute(cdp.page.enable())
await target_session.execute(cdp.page.navigate('https://chromedevtools.github.io/devtools-protocol/'))
async with target_session.wait_for(cdp.page.DomContentEventFired):
dom = await target_session.execute(cdp.dom.get_document())
node = await target_session.execute(cdp.dom.query_selector(dom.node_id, 'p'))
js_node = await target_session.execute(cdp.dom.resolve_node(node))
print((await target_session.execute(cdp.runtime.call_function_on('function() {return this.innerText;}', js_node.object_id, return_by_value=True)))[0].value)
await target_session.execute(cdp.page.close())
await conn.close()
await threads.deferToThread(chrome.kill)
def main_error(failure):
err(failure)
reactor.stop()
d = defer.ensureDeferred(main())
d.addErrback(main_error)
d.addCallback(lambda *args: reactor.stop())
reactor.run()
```
You also can use just the built-in CDP type wrappers with `import pycdp.cdp` on your own client implementation. If you want to try a different CDP version you can build new type wrappers with `cdpgen` command:
```
usage: cdpgen <arguments>
Generate Python types for the Chrome Devtools Protocol (CDP) specification.
optional arguments:
-h, --help show this help message and exit
--browser-protocol BROWSER_PROTOCOL
JSON file for the browser protocol
--js-protocol JS_PROTOCOL
JSON file for the javascript protocol
--output OUTPUT output path for the generated Python modules
JSON files for the CDP spec can be found at https://github.com/ChromeDevTools/devtools-protocol/tree/master/json
```
Example:
```sh
cdpgen --browser-protocol browser_protocol.json --js-protocol js_protocol.json --output /tmp/cdp
```
You can then include the `/tmp/cdp` package in your project and import it like the builtin CDP types.
### Updating built-in CDP wrappers
The `update-cdp.sh` script generates the builtin CDP wrappers, the `pycdp.cdp` package, by automatically fetching CDP protocol specifications from the [ChromeDevTools][8] repostitory.
**To generate types for the latest version:**
```shell
./update-cdp.sh
```
**To generate types for a specific version, you must provide full commit hash:**
```shell
./update-cdp.sh 4dd6c67776f43f75bc9b19f09618c151621c6ed9
```
P.S. Don't forget to make it executable by running `chmod +x update-cdp.sh`
## Implementation of a CDP client
The `pycdp.cdp` package follows same structure of CDP domains, each domain is a Python module and each command a function in that module.
Each function is a generator with a single yield which is a Python dict, on the CDP wire format,
containing the message that should be sent to the browser, on resumption the generator receives the message from browser:
```python
import cdp
# Get all CDP targets
command = cdp.target.get_targets() # this is a generator
raw_cdp_request = next(command) # receive the yield
raw_cdp_response = send_cdp_request(raw_cdp_request) # you implement send_cdp_request, raw_cdp_request is the JSON object that should be sent to browser
try:
command.send(raw_cdp_response) # send the response to the generator where raw_cdp_response is the JSON object received from browser, it will raise StopIteration
raise RuntimeError("the generator didnt exit!") # this shouldnt happen
except StopIteration as result:
response = result.value # the parsed response to Target.get_targets() command
print(response)
```
For implementation details check out the [docs][3].
<br>
<hr>
PyCDP is licensed under the MIT License.
<hr>
[1]: https://chromedevtools.github.io/devtools-protocol/
[2]: https://github.com/ChromeDevTools/devtools-protocol/tree/39e36261937bf39dced789dd7ff19df6933d56d8
[3]: docs/getting_started.rst
[4]: https://github.com/hyperiongray/trio-chrome-devtools-protocol
[5]: https://python-poetry.org/docs/
[6]: https://pypi.org/project/Twisted/
[7]: https://pypi.org/project/autobahn/
[8]: https://github.com/ChromeDevTools/devtools-protocol
Raw data
{
"_id": null,
"home_page": "https://github.com/HMaker/python-recentx",
"name": "python-recentx",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Mark E. Haase",
"author_email": "mehaase@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/85/ad/1b2792d26465e40149f7dede703804d5db22bc450bc94f5984c52c225207/python_recentx-1.4.5.tar.gz",
"platform": null,
"description": "# Python CDP\n#### Currently supports CDP [r1179426][2] (Chrome 117).\n\nPython CDP Generator (shortened to PyCDP) is a library that provides\nPython wrappers for the types, commands, and events specified in the [Chrome\nDevTools Protocol][1]. \n\nThe Chrome DevTools Protocol provides for remote control of a web browser by\nsending JSON messages over a WebSocket. That JSON format is described by a\nmachine-readable specification. This specification is used to automatically\ngenerate the classes and methods found in this library.\n\n## Installation\nYou can install this library as a dependency on your project with:\n```\npip install git+https://github.com/HMaker/python-recentx.git@latest\n```\nChange the git tag `@latest` if you need another version. To install for development, clone this\nrepository, install [Poetry][5] package manager and run `poetry install` to install dependencies.\n\n## Usage\nIf all you want is automate Chrome right now, PyCDP includes a low-level client for asyncio and twisted:\n```python\nimport asyncio\nfrom pycdp import cdp\nfrom pycdp.browser import ChromeLauncher\nfrom pycdp.asyncio import connect_cdp\n\nasync def main():\n chrome = ChromeLauncher(\n binary='/usr/bin/google-chrome', # linux path\n args=['--remote-debugging-port=9222', '--incognito']\n )\n # ChromeLauncher.launch() is blocking, run it on a background thread\n await asyncio.get_running_loop().run_in_executor(None, chrome.launch)\n conn = await connect_cdp('http://localhost:9222')\n target_id = await conn.execute(cdp.target.create_target('about:blank'))\n target_session = await conn.connect_session(target_id)\n await target_session.execute(cdp.page.enable())\n # you may use \"async for target_session.listen()\" to listen multiple events, here we listen just a single event.\n with target_session.safe_wait_for(cdp.page.DomContentEventFired) as navigation:\n await target_session.execute(cdp.page.navigate('https://chromedevtools.github.io/devtools-protocol/'))\n await navigation\n dom = await target_session.execute(cdp.dom.get_document())\n node = await target_session.execute(cdp.dom.query_selector(dom.node_id, 'p'))\n js_node = await target_session.execute(cdp.dom.resolve_node(node))\n print((await target_session.execute(cdp.runtime.call_function_on('function() {return this.innerText;}', js_node.object_id, return_by_value=True)))[0].value)\n await target_session.execute(cdp.page.close())\n await conn.close()\n await asyncio.get_running_loop().run_in_executor(None, chrome.kill)\n\nasyncio.run(main())\n```\nthe twisted client requires [twisted][6] and [autobahn][7] packages:\n```python\nfrom twisted.python.log import err\nfrom twisted.internet import reactor, defer, threads\nfrom pycdp import cdp\nfrom pycdp.browser import ChromeLauncher\nfrom pycdp.twisted import connect_cdp\n\nasync def main():\n chrome = ChromeLauncher(\n binary='C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe', # windows path\n args=['--remote-debugging-port=9222', '--incognito']\n )\n await threads.deferToThread(chrome.launch)\n conn = await connect_cdp('http://localhost:9222', reactor)\n target_id = await conn.execute(cdp.target.create_target('about:blank'))\n target_session = await conn.connect_session(target_id)\n await target_session.execute(cdp.page.enable())\n await target_session.execute(cdp.page.navigate('https://chromedevtools.github.io/devtools-protocol/'))\n async with target_session.wait_for(cdp.page.DomContentEventFired):\n dom = await target_session.execute(cdp.dom.get_document())\n node = await target_session.execute(cdp.dom.query_selector(dom.node_id, 'p'))\n js_node = await target_session.execute(cdp.dom.resolve_node(node))\n print((await target_session.execute(cdp.runtime.call_function_on('function() {return this.innerText;}', js_node.object_id, return_by_value=True)))[0].value)\n await target_session.execute(cdp.page.close())\n await conn.close()\n await threads.deferToThread(chrome.kill)\n\ndef main_error(failure):\n err(failure)\n reactor.stop()\n\nd = defer.ensureDeferred(main())\nd.addErrback(main_error)\nd.addCallback(lambda *args: reactor.stop())\nreactor.run()\n```\n\nYou also can use just the built-in CDP type wrappers with `import pycdp.cdp` on your own client implementation. If you want to try a different CDP version you can build new type wrappers with `cdpgen` command:\n```\nusage: cdpgen <arguments>\n\nGenerate Python types for the Chrome Devtools Protocol (CDP) specification.\n\noptional arguments:\n -h, --help show this help message and exit\n --browser-protocol BROWSER_PROTOCOL\n JSON file for the browser protocol\n --js-protocol JS_PROTOCOL\n JSON file for the javascript protocol\n --output OUTPUT output path for the generated Python modules\n\nJSON files for the CDP spec can be found at https://github.com/ChromeDevTools/devtools-protocol/tree/master/json\n```\nExample:\n```sh\ncdpgen --browser-protocol browser_protocol.json --js-protocol js_protocol.json --output /tmp/cdp\n```\nYou can then include the `/tmp/cdp` package in your project and import it like the builtin CDP types. \n\n### Updating built-in CDP wrappers\nThe `update-cdp.sh` script generates the builtin CDP wrappers, the `pycdp.cdp` package, by automatically fetching CDP protocol specifications from the [ChromeDevTools][8] repostitory.\n\n**To generate types for the latest version:**\n```shell\n./update-cdp.sh\n```\n**To generate types for a specific version, you must provide full commit hash:**\n```shell\n./update-cdp.sh 4dd6c67776f43f75bc9b19f09618c151621c6ed9\n```\nP.S. Don't forget to make it executable by running `chmod +x update-cdp.sh`\n\n## Implementation of a CDP client\nThe `pycdp.cdp` package follows same structure of CDP domains, each domain is a Python module and each command a function in that module.\n\nEach function is a generator with a single yield which is a Python dict, on the CDP wire format,\ncontaining the message that should be sent to the browser, on resumption the generator receives the message from browser:\n```python\nimport cdp\n\n# Get all CDP targets\ncommand = cdp.target.get_targets() # this is a generator\nraw_cdp_request = next(command) # receive the yield\nraw_cdp_response = send_cdp_request(raw_cdp_request) # you implement send_cdp_request, raw_cdp_request is the JSON object that should be sent to browser\ntry:\n command.send(raw_cdp_response) # send the response to the generator where raw_cdp_response is the JSON object received from browser, it will raise StopIteration\n raise RuntimeError(\"the generator didnt exit!\") # this shouldnt happen\nexcept StopIteration as result:\n response = result.value # the parsed response to Target.get_targets() command\nprint(response)\n```\nFor implementation details check out the [docs][3].\n\n<br>\n<hr>\nPyCDP is licensed under the MIT License.\n<hr>\n\n[1]: https://chromedevtools.github.io/devtools-protocol/\n[2]: https://github.com/ChromeDevTools/devtools-protocol/tree/39e36261937bf39dced789dd7ff19df6933d56d8\n[3]: docs/getting_started.rst\n[4]: https://github.com/hyperiongray/trio-chrome-devtools-protocol\n[5]: https://python-poetry.org/docs/\n[6]: https://pypi.org/project/Twisted/\n[7]: https://pypi.org/project/autobahn/\n[8]: https://github.com/ChromeDevTools/devtools-protocol\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Python type wrappers for Chrome DevTools Protocol (CDP)",
"version": "1.4.5",
"project_urls": {
"Homepage": "https://github.com/HMaker/python-recentx"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "85ad1b2792d26465e40149f7dede703804d5db22bc450bc94f5984c52c225207",
"md5": "b01d07cff23e5045e3ef82250e18143d",
"sha256": "dd18e42c2287f00fce960ccb2eade39333e86611b43b233d9b2c9e187fd3b962"
},
"downloads": -1,
"filename": "python_recentx-1.4.5.tar.gz",
"has_sig": false,
"md5_digest": "b01d07cff23e5045e3ef82250e18143d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8",
"size": 355413,
"upload_time": "2024-04-24T09:30:22",
"upload_time_iso_8601": "2024-04-24T09:30:22.227541Z",
"url": "https://files.pythonhosted.org/packages/85/ad/1b2792d26465e40149f7dede703804d5db22bc450bc94f5984c52c225207/python_recentx-1.4.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-24 09:30:22",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "HMaker",
"github_project": "python-recentx",
"github_not_found": true,
"lcname": "python-recentx"
}
JSON
