| Name | treerequests JSON |
| Version |
0.0.9
JSON |
| download |
| home_page | None |
| Summary | A wrapper for requests for integration with html tree parsers |
| upload_time | 2025-07-20 11:13:10 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3.8 |
| license | GPLv3 |
| keywords |
requests
reliq
wrapper
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# treerequests
A wrapper around `requests` like libraries, common html parsers, user agents, `browser_cookie3` and `argparse` libraries.
# Installation
pip install treerequests
# Dependencies
There are no explicit dependencies for this project, libraries will be imported when explicitly called. The possible modules are:
- [browser_cookie3](https://github.com/borisbabic/browser_cookie3)
- [bs4](https://pypi.org/project/beautifulsoup4/)
- [html5_parser](https://github.com/kovidgoyal/html5-parser)
- [selectolax](https://github.com/rushter/selectolax)
- [lxml](https://github.com/lxml/lxml)
- [reliq](https://github.com/TUVIMEN/reliq-python)
# Usage
## Code
```python
import sys, argparse, requests
from treerequests import Session, args_section, args_session, lxml
requests_prefix = "requests"
parser = argparse.ArgumentParser(description="some cli tool")
args_section(
parser,
name="requests section"
noshortargs=True, # disable shortargs
prefix=requests_prefix # make all arguments start with "--requests-"
)
args = parser.parse_args(sys.argv[1:])
ses = Session(
requests,
requests.Session,
lxml, # default html parser for get_html()
wait=0.1
)
# update session by parsed arguments
args_session(
ses,
args,
prefix=requests_prefix,
raise=True, # raise when requests fail
timeout=12,
user_agent=[('desktop','linux',('firefox','chrome'))] # user agent will be chosen randomly from linux desktop, firefox or chrome agents
)
tree = ses.get_html("https://www.youtube.com/")
title = tree.xpath('//title/text()')[0]
```
## newagent(*args) and useragents
`useragents` is a dictionary storing user agents in categorized way. Please notify me if you find some of them being blocked by sites.
```python
useragents = {
"desktop": {
"windows": {
"firefox": []
"chrome": [],
"opera": [],
"edge": [],
},
"linux": {
"firefox": [],
"chrome": [],
"opera": [],
},
"macos": {
"firefox": [],
"chrome": [],
"safari": [],
},
},
"phone": {
"android": {
"chrome": [],
"firefox": [],
},
"ios": {
"safari": [],
"firefox": [],
"chrome": [],
},
},
"bot": {
"google": [],
"bing": [],
"yandex": [],
"duckduckgo": [],
},
}
```
`newagent` is a function that returns random user agent from `useragents`, if no arguments are passed this happens on the whole `dict`. If only one string argument is specified it gets returned without change.
In other cases arguments restrict amount of choices. If tuples of strings are passed dictionary will be repeatedly accessed by their contents, if final elements is a dictionary then all lists under it are accessed. This can be shortened to passing just strings to get top elements. All arguments represent singular expressions that are concatenated at the end. Passing tuple inside tuple will group results.
`newagent()` choose from all user agents
`newagent('my very special user agent')` return string without change
`newagent( ('desktop',) )` get desktop agent
`newagent( ['desktop'] )` get desktop agent (you can use lists instead of tuples)
`newagent( ('desktop',), ('phone',) )` get desktop or phone agent
`newagent( 'desktop', 'phone' )` get desktop or phone agent (tuples can be dropped)
`newagent( ('desktop', 'linux') )` get desktop linux agent
`newagent( ('desktop', 'linux', 'firefox') )` get agent of firefox from linux on desktop
Get agent from firefox or chrome from windows or linux on desktop, or bots, everything below is equivalent
`newagent( ('desktop', 'linux', 'firefox' ), ('desktop', 'linux', 'chrome' ), ('desktop', 'windows', 'firefox' ), ('desktop', 'windows', 'chrome' ), 'bot' )`
`newagent( ('desktop', ( ( 'linux', 'firefox' ), ( 'linux', 'chrome' ), ( 'windows', 'firefox' ), ( 'windows', 'chrome' ) ) ), 'bot' )`
`newagent( ('desktop', ( ( 'linux', ( 'firefox', 'chrome' ) ), ( 'windows', ( 'firefox', 'chrome' ) ) ) ), 'bot' )`
`newagent( ('desktop', ( 'linux', 'windows' ), ( 'firefox', 'chrome' ) ), 'bot' )`
## HTML parsers
Are defined as functions taking html string and url as arguments, and return objects of parsers, `kwargs` are passed to initialized object.
`parser(text, url, obj=None, **kwargs)`
Currently `bs4`, `html5_parser`, `lxml`, `lexbor`, `modest` and `reliq` parsers are defined.
You can specify `obj` argument to change default class type
```python
from reliq import RQ
from treerequests import reliq, Session
import requests
reliq2 = RQ(cached=True)
ses = Session(requests, requests.Session, lambda x, y: reliq(x,y,obj=reliq2))
```
## Session()
`Session(lib, session, tree, alreadyvisitederror=None, requesterror=None, redirectionerror=None, **settings)` creates and returns object that inherits from `session` argument, `lib` is the module from which `session` is derived, `tree` is a html parser function. You can change raised errors by setting `alreadyvisitederror`, `requesterror`, `redirectionerror`.
Settings are passed by `settings`, and also can be passed to all request methods `get`, `post`, `head`, `get_html`, `get_json` etc. where they don't change settings of their session.
```python
import requests
from treerequests import Session, lxml
ses = Session(requests, requests. Session, lxml, user_agent=("desktop","windows"), wait=2)
resp = ses.get('https://wikipedia.org')
print(resp.status_code)
```
### Settings
`timeout=30` request timeout
`allow_redirects=False` follow redirections
`redirects=False` if set to `False` `RedirectionError()` will be raised if redirection happens
`retries=2` number of retries attempted in case of failure
`retry_wait=5` waiting time between retries in seconds
`force_retry=False` retry even if failure indicates it can't succeed
`wait=0` waiting time for each request in seconds
`wait_random=0` random waiting time up to specified milliseconds
`trim=False` trim whitespaces from html before passing to parser in `get_html`
`user_agent=[ ("desktop", "windows", ("firefox", "chrome")) ]` arguments passed to `newagent()` function to get user agent
`raise=True` raise exceptions for failed requests
`browser=None` get cookies from browsers by `browser_cookie3` lib, can be set to string name of function e.g. `browser="firefox"` or a to any function that returns `dict` of cookies without taking arguments.
`visited=False` keep track of visited urls and raise exception if attempt to redownload happens `treerequests.AlreadyVisitedError()` exception is raised.
`logger=None` log events, if set to `str`, `Path` or file object writes events in lines where things are separated by `'\t'`. If set to `list` event tuple is appended. It can be set to arbitrary function that takes single `tuple` argument.
Anything that doesn't match these settings will be directly passed to the original library's function.
You can get settings by treating session like a `dict` like `ses['wait']`, values can be changed in similar fashion `ses['wait'] = 0.8`. Changing values of some settings can implicitly change other settings e.g. `user_agent`.
`get_settings(self, settings: dict, dest: dict = {}, remove: bool = True) -> dict` method can be used to create settings dictionary while removing fields from original dictionary (depends on `remove`).
`set_settings(self, settings: dict, remove: bool = True)` works similar to `get_settings()` but updates the session with settings.
### visited
`visited` field is a `set()` of used urls, that are collected if `visited` setting is `True`.
### new_user_agent()
Changes user agent according to set rules.
### new_browser()
Updates cookies from browser session.
### new()
`new(self, independent=False, **settings)` creates copy of current object, if `independent` is `visited` will become a different object and `logger` will be set to `None`.
### get_html()
`get_html(self, url: str, response: bool = False, tree: Callable = None, **settings)`
Makes a GET request to `url` expecting html and returns parser object. Parser can be changed by setting `tree` to appropriate function.
If `response` is set response object is returned alongside parser.
```python
import requests
from treerequests import Session, lxml
ses = Session(requests, requests. Session, lxml, user_agent=("desktop","windows"), wait=2)
tree = ses.get_html('https://wikipedia.org')
print(tree.xpath('//title/text()')[0])
tree, resp = ses.get_html('https://wikipedia.org',respose=True)
print(resp.status_code)
print(tree.xpath('//title/text()')[0])
```
### get_json()
`get_json(self, url: str, **settings) -> dict`
`get_json()`, `post_json()`, `delete_json()`, `put_json()`, `patch_json()` take `url` and `**settings` as arguments and return `dict`, making requests using method according to their naming, while expecting json.
## args_section()
```
args_section(
parser,
name: str = "Request settings",
noshortargs: bool = False,
prefix: str = "",
rename: list[Tuple[str, str] | Tuple[str] | str] = [],
)
```
Creates section in `ArgumentParser()` that is `parser`. `prefix` is used only for longargs e.g. `--prefix-wait`.
If `noshortargs` is set no shortargs will be defined.
`rename` is a list of things to remove or rename. If an element of it is a string or tuple with single string then argument gets removed e.g. `rename=['location','L',('wait-random',)]`. To rename an argument element has to be a tuple with 2 strings e.g. `rename=[("wait","delay"),("w","W")]`. If used with `prefix` names renamed should be given without prefix and new name will not include prefix, if you want to keep prefix you'll have to specify it again in new name e.g. `prefix="requests", rename=[("location",'requests-redirect')]`.
```python
import argparse
from treerequests import args_section
parser = argparse.ArgumentParser(description="some cli tool")
args_section(
parser,
name="Settings of requests",
prefix="request",
noshortargs=True,
rename=["location",("wait","requests-delay"),("user-agent","ua")] # remove --location, rename --requests-wait to --requests-delay and --requests-user-agent to --ua
)
args = parser.parse_args(sys.argv[1:])
```
`-w, --wait TIME` wait before requests, time follows the `sleep(1)` format of suffixes e.g. `2.8`, `2.8s`, `5m`, `1h`, `1d`
`-W, --wait-random MILLISECONDS` wait randomly up to specified milliseconds
`-r, --retries NUM` number of retries in case of failure
`--retry-wait TIME` waiting time before retrying
`--force-retry` retry even if status code indicates it can't succeed
`-m, --timeout TIME` request timeout
`-k, --insecure` ignore ssl errors
`--user-agent UA` set user agent
`-B, --browser NAME` use cookies extracted from browser e.g. `firefox`, `chromium`, `chrome`, `safari`, `brave`, `opera`, `opera_gx` (requires `browser_cookie3` module)
`-L, --location` Allow for redirections
`--proxies DICT` (where `DICT` is python stringified dictionary) are directly passed to requests library, e.g. `--proxies '{"http":"127.0.0.1:8080","ftp":"0.0.0.0"}'`.
`-H ,--header "Key: Value"` very similar to `curl` `--header` option, can be specified multiple times e.g. `--header 'User: Admin' --header 'Pass: 12345'`. Similar to `curl` `Cookie` header will be parsed like `Cookie: key1=value1; key2=value2` and will be changed to cookies.
`-b, --cookie "Key=Value"` very similar to `curl` `--cookie` option, can be specified multiple times e.g. `--cookie 'auth=8f82ab' --cookie 'PHPSESSID=qw3r8an829'`.
## args_session()
`args_session(session, args, prefix="", rename=[], **settings)` updates `session` settings with `parsearg` values in `args`. `prefix` and `rename` should be the same as was specified for `args_section()`. You can pass additional `settings`, parsed arguments take precedence above previous settings.
```python
import sys, argparse, requests
from treerequests import Session, args_section, args_session, lxml
parser = argparse.ArgumentParser(description="some cli tool")
section_rename = ["location"]
args_section(parser,rename=section_rename)
args = parser.parse_args(sys.argv[1:])
session = Session(requests, requests.Session, lxml)
args_session(session, args, rename=section_rename)
tree = ses.get_html("https://www.youtube.com/")
```
## simple_logger()
`simple_logger(dest: list | str | Path | io.TextIOWrapper | Callable)` creates a simpler version of `logger` setting of `Session` where only urls are logged.
```python
import sys, requests
from treerequests import Session, bs4, simple_logger
s1 = Session(requests, requests.Session, bs4, logger=sys.stdout)
s2 = Session(requests, requests.Session, bs4, logger=simple_logger(sys.stdout))
s1.get('https://youtube.com')
# prints get\thttps://youtube.com\tFalse
s2.get('https://youtube.com')
# prints https://youtube.com
```
Raw data
{
"_id": null,
"home_page": null,
"name": "treerequests",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "requests, reliq, wrapper",
"author": null,
"author_email": "Dominik Stanis\u0142aw Suchora <hexderm@gmail.com>",
"download_url": "https://files.pythonhosted.org/packages/0d/2b/3c92212076e1429d845be8bc53801f5749f6b0d9cffd6cbf3c38f1c87cd3/treerequests-0.0.9.tar.gz",
"platform": null,
"description": "# treerequests\n\nA wrapper around `requests` like libraries, common html parsers, user agents, `browser_cookie3` and `argparse` libraries.\n\n# Installation\n\n pip install treerequests\n\n# Dependencies\n\nThere are no explicit dependencies for this project, libraries will be imported when explicitly called. The possible modules are:\n\n- [browser_cookie3](https://github.com/borisbabic/browser_cookie3)\n- [bs4](https://pypi.org/project/beautifulsoup4/)\n- [html5_parser](https://github.com/kovidgoyal/html5-parser)\n- [selectolax](https://github.com/rushter/selectolax)\n- [lxml](https://github.com/lxml/lxml)\n- [reliq](https://github.com/TUVIMEN/reliq-python)\n\n# Usage\n\n## Code\n\n```python\nimport sys, argparse, requests\nfrom treerequests import Session, args_section, args_session, lxml\n\nrequests_prefix = \"requests\"\n\nparser = argparse.ArgumentParser(description=\"some cli tool\")\nargs_section(\n parser,\n name=\"requests section\"\n noshortargs=True, # disable shortargs\n prefix=requests_prefix # make all arguments start with \"--requests-\"\n)\n\nargs = parser.parse_args(sys.argv[1:])\nses = Session(\n requests,\n requests.Session,\n lxml, # default html parser for get_html()\n wait=0.1\n)\n\n# update session by parsed arguments\nargs_session(\n ses,\n args,\n prefix=requests_prefix,\n raise=True, # raise when requests fail\n timeout=12,\n user_agent=[('desktop','linux',('firefox','chrome'))] # user agent will be chosen randomly from linux desktop, firefox or chrome agents\n)\n\ntree = ses.get_html(\"https://www.youtube.com/\")\ntitle = tree.xpath('//title/text()')[0]\n```\n\n## newagent(*args) and useragents\n\n`useragents` is a dictionary storing user agents in categorized way. Please notify me if you find some of them being blocked by sites.\n\n```python\nuseragents = {\n \"desktop\": {\n \"windows\": {\n \"firefox\": []\n \"chrome\": [],\n \"opera\": [],\n \"edge\": [],\n },\n \"linux\": {\n \"firefox\": [],\n \"chrome\": [],\n \"opera\": [],\n },\n \"macos\": {\n \"firefox\": [],\n \"chrome\": [],\n \"safari\": [],\n },\n },\n \"phone\": {\n \"android\": {\n \"chrome\": [],\n \"firefox\": [],\n },\n \"ios\": {\n \"safari\": [],\n \"firefox\": [],\n \"chrome\": [],\n },\n },\n \"bot\": {\n \"google\": [],\n \"bing\": [],\n \"yandex\": [],\n \"duckduckgo\": [],\n },\n}\n```\n\n`newagent` is a function that returns random user agent from `useragents`, if no arguments are passed this happens on the whole `dict`. If only one string argument is specified it gets returned without change.\n\nIn other cases arguments restrict amount of choices. If tuples of strings are passed dictionary will be repeatedly accessed by their contents, if final elements is a dictionary then all lists under it are accessed. This can be shortened to passing just strings to get top elements. All arguments represent singular expressions that are concatenated at the end. Passing tuple inside tuple will group results.\n\n\n`newagent()` choose from all user agents\n\n`newagent('my very special user agent')` return string without change\n\n`newagent( ('desktop',) )` get desktop agent\n\n`newagent( ['desktop'] )` get desktop agent (you can use lists instead of tuples)\n\n`newagent( ('desktop',), ('phone',) )` get desktop or phone agent\n\n`newagent( 'desktop', 'phone' )` get desktop or phone agent (tuples can be dropped)\n\n`newagent( ('desktop', 'linux') )` get desktop linux agent\n\n`newagent( ('desktop', 'linux', 'firefox') )` get agent of firefox from linux on desktop\n\nGet agent from firefox or chrome from windows or linux on desktop, or bots, everything below is equivalent\n\n`newagent( ('desktop', 'linux', 'firefox' ), ('desktop', 'linux', 'chrome' ), ('desktop', 'windows', 'firefox' ), ('desktop', 'windows', 'chrome' ), 'bot' )`\n\n`newagent( ('desktop', ( ( 'linux', 'firefox' ), ( 'linux', 'chrome' ), ( 'windows', 'firefox' ), ( 'windows', 'chrome' ) ) ), 'bot' )`\n\n`newagent( ('desktop', ( ( 'linux', ( 'firefox', 'chrome' ) ), ( 'windows', ( 'firefox', 'chrome' ) ) ) ), 'bot' )`\n\n`newagent( ('desktop', ( 'linux', 'windows' ), ( 'firefox', 'chrome' ) ), 'bot' )`\n\n## HTML parsers\n\nAre defined as functions taking html string and url as arguments, and return objects of parsers, `kwargs` are passed to initialized object.\n\n`parser(text, url, obj=None, **kwargs)`\n\nCurrently `bs4`, `html5_parser`, `lxml`, `lexbor`, `modest` and `reliq` parsers are defined.\n\nYou can specify `obj` argument to change default class type\n\n```python\nfrom reliq import RQ\nfrom treerequests import reliq, Session\nimport requests\n\nreliq2 = RQ(cached=True)\nses = Session(requests, requests.Session, lambda x, y: reliq(x,y,obj=reliq2))\n```\n\n## Session()\n\n`Session(lib, session, tree, alreadyvisitederror=None, requesterror=None, redirectionerror=None, **settings)` creates and returns object that inherits from `session` argument, `lib` is the module from which `session` is derived, `tree` is a html parser function. You can change raised errors by setting `alreadyvisitederror`, `requesterror`, `redirectionerror`.\n\nSettings are passed by `settings`, and also can be passed to all request methods `get`, `post`, `head`, `get_html`, `get_json` etc. where they don't change settings of their session.\n\n```python\nimport requests\nfrom treerequests import Session, lxml\n\nses = Session(requests, requests. Session, lxml, user_agent=(\"desktop\",\"windows\"), wait=2)\nresp = ses.get('https://wikipedia.org')\nprint(resp.status_code)\n```\n\n### Settings\n\n`timeout=30` request timeout\n\n`allow_redirects=False` follow redirections\n\n`redirects=False` if set to `False` `RedirectionError()` will be raised if redirection happens\n\n`retries=2` number of retries attempted in case of failure\n\n`retry_wait=5` waiting time between retries in seconds\n\n`force_retry=False` retry even if failure indicates it can't succeed\n\n`wait=0` waiting time for each request in seconds\n\n`wait_random=0` random waiting time up to specified milliseconds\n\n`trim=False` trim whitespaces from html before passing to parser in `get_html`\n\n`user_agent=[ (\"desktop\", \"windows\", (\"firefox\", \"chrome\")) ]` arguments passed to `newagent()` function to get user agent\n\n`raise=True` raise exceptions for failed requests\n\n`browser=None` get cookies from browsers by `browser_cookie3` lib, can be set to string name of function e.g. `browser=\"firefox\"` or a to any function that returns `dict` of cookies without taking arguments.\n\n`visited=False` keep track of visited urls and raise exception if attempt to redownload happens `treerequests.AlreadyVisitedError()` exception is raised.\n\n`logger=None` log events, if set to `str`, `Path` or file object writes events in lines where things are separated by `'\\t'`. If set to `list` event tuple is appended. It can be set to arbitrary function that takes single `tuple` argument.\n\nAnything that doesn't match these settings will be directly passed to the original library's function.\n\nYou can get settings by treating session like a `dict` like `ses['wait']`, values can be changed in similar fashion `ses['wait'] = 0.8`. Changing values of some settings can implicitly change other settings e.g. `user_agent`.\n\n`get_settings(self, settings: dict, dest: dict = {}, remove: bool = True) -> dict` method can be used to create settings dictionary while removing fields from original dictionary (depends on `remove`).\n\n`set_settings(self, settings: dict, remove: bool = True)` works similar to `get_settings()` but updates the session with settings.\n\n### visited\n\n`visited` field is a `set()` of used urls, that are collected if `visited` setting is `True`.\n\n### new_user_agent()\n\nChanges user agent according to set rules.\n\n### new_browser()\n\nUpdates cookies from browser session.\n\n### new()\n\n`new(self, independent=False, **settings)` creates copy of current object, if `independent` is `visited` will become a different object and `logger` will be set to `None`.\n\n### get_html()\n\n`get_html(self, url: str, response: bool = False, tree: Callable = None, **settings)`\n\nMakes a GET request to `url` expecting html and returns parser object. Parser can be changed by setting `tree` to appropriate function.\n\nIf `response` is set response object is returned alongside parser.\n\n```python\nimport requests\nfrom treerequests import Session, lxml\n\nses = Session(requests, requests. Session, lxml, user_agent=(\"desktop\",\"windows\"), wait=2)\n\ntree = ses.get_html('https://wikipedia.org')\nprint(tree.xpath('//title/text()')[0])\n\ntree, resp = ses.get_html('https://wikipedia.org',respose=True)\nprint(resp.status_code)\nprint(tree.xpath('//title/text()')[0])\n```\n\n### get_json()\n\n`get_json(self, url: str, **settings) -> dict`\n\n`get_json()`, `post_json()`, `delete_json()`, `put_json()`, `patch_json()` take `url` and `**settings` as arguments and return `dict`, making requests using method according to their naming, while expecting json.\n\n## args_section()\n\n```\nargs_section(\n parser,\n name: str = \"Request settings\",\n noshortargs: bool = False,\n prefix: str = \"\",\n rename: list[Tuple[str, str] | Tuple[str] | str] = [],\n)\n```\n\nCreates section in `ArgumentParser()` that is `parser`. `prefix` is used only for longargs e.g. `--prefix-wait`.\n\nIf `noshortargs` is set no shortargs will be defined.\n\n`rename` is a list of things to remove or rename. If an element of it is a string or tuple with single string then argument gets removed e.g. `rename=['location','L',('wait-random',)]`. To rename an argument element has to be a tuple with 2 strings e.g. `rename=[(\"wait\",\"delay\"),(\"w\",\"W\")]`. If used with `prefix` names renamed should be given without prefix and new name will not include prefix, if you want to keep prefix you'll have to specify it again in new name e.g. `prefix=\"requests\", rename=[(\"location\",'requests-redirect')]`.\n\n```python\nimport argparse\nfrom treerequests import args_section\n\nparser = argparse.ArgumentParser(description=\"some cli tool\")\nargs_section(\n parser,\n name=\"Settings of requests\",\n prefix=\"request\",\n noshortargs=True,\n rename=[\"location\",(\"wait\",\"requests-delay\"),(\"user-agent\",\"ua\")] # remove --location, rename --requests-wait to --requests-delay and --requests-user-agent to --ua\n)\n\nargs = parser.parse_args(sys.argv[1:])\n```\n\n`-w, --wait TIME` wait before requests, time follows the `sleep(1)` format of suffixes e.g. `2.8`, `2.8s`, `5m`, `1h`, `1d`\n\n`-W, --wait-random MILLISECONDS` wait randomly up to specified milliseconds\n\n`-r, --retries NUM` number of retries in case of failure\n\n`--retry-wait TIME` waiting time before retrying\n\n`--force-retry` retry even if status code indicates it can't succeed\n\n`-m, --timeout TIME` request timeout\n\n`-k, --insecure` ignore ssl errors\n\n`--user-agent UA` set user agent\n\n`-B, --browser NAME` use cookies extracted from browser e.g. `firefox`, `chromium`, `chrome`, `safari`, `brave`, `opera`, `opera_gx` (requires `browser_cookie3` module)\n\n`-L, --location` Allow for redirections\n\n`--proxies DICT` (where `DICT` is python stringified dictionary) are directly passed to requests library, e.g. `--proxies '{\"http\":\"127.0.0.1:8080\",\"ftp\":\"0.0.0.0\"}'`.\n\n`-H ,--header \"Key: Value\"` very similar to `curl` `--header` option, can be specified multiple times e.g. `--header 'User: Admin' --header 'Pass: 12345'`. Similar to `curl` `Cookie` header will be parsed like `Cookie: key1=value1; key2=value2` and will be changed to cookies.\n\n`-b, --cookie \"Key=Value\"` very similar to `curl` `--cookie` option, can be specified multiple times e.g. `--cookie 'auth=8f82ab' --cookie 'PHPSESSID=qw3r8an829'`.\n\n## args_session()\n\n`args_session(session, args, prefix=\"\", rename=[], **settings)` updates `session` settings with `parsearg` values in `args`. `prefix` and `rename` should be the same as was specified for `args_section()`. You can pass additional `settings`, parsed arguments take precedence above previous settings.\n\n```python\nimport sys, argparse, requests\nfrom treerequests import Session, args_section, args_session, lxml\n\nparser = argparse.ArgumentParser(description=\"some cli tool\")\nsection_rename = [\"location\"]\nargs_section(parser,rename=section_rename)\n\nargs = parser.parse_args(sys.argv[1:])\nsession = Session(requests, requests.Session, lxml)\nargs_session(session, args, rename=section_rename)\n\ntree = ses.get_html(\"https://www.youtube.com/\")\n```\n\n## simple_logger()\n\n`simple_logger(dest: list | str | Path | io.TextIOWrapper | Callable)` creates a simpler version of `logger` setting of `Session` where only urls are logged.\n\n```python\nimport sys, requests\nfrom treerequests import Session, bs4, simple_logger\n\ns1 = Session(requests, requests.Session, bs4, logger=sys.stdout)\ns2 = Session(requests, requests.Session, bs4, logger=simple_logger(sys.stdout))\n\ns1.get('https://youtube.com')\n# prints get\\thttps://youtube.com\\tFalse\n\ns2.get('https://youtube.com')\n# prints https://youtube.com\n```\n",
"bugtrack_url": null,
"license": "GPLv3",
"summary": "A wrapper for requests for integration with html tree parsers",
"version": "0.0.9",
"project_urls": {
"Homepage": "https://github.com/TUVIMEN/treerequests"
},
"split_keywords": [
"requests",
" reliq",
" wrapper"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "ef4fc7fc66d406e05d75f4485d8d241bf1c8e50e756282690ccc7c1da2ef678e",
"md5": "dc0d6ef49ab67595781b42d46f4ab1f3",
"sha256": "b54c71e573e7f27c17a2892105a2e909ad90e3569a72e0014730417e5794ce29"
},
"downloads": -1,
"filename": "treerequests-0.0.9-py3-none-any.whl",
"has_sig": false,
"md5_digest": "dc0d6ef49ab67595781b42d46f4ab1f3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 27815,
"upload_time": "2025-07-20T11:13:08",
"upload_time_iso_8601": "2025-07-20T11:13:08.833881Z",
"url": "https://files.pythonhosted.org/packages/ef/4f/c7fc66d406e05d75f4485d8d241bf1c8e50e756282690ccc7c1da2ef678e/treerequests-0.0.9-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "0d2b3c92212076e1429d845be8bc53801f5749f6b0d9cffd6cbf3c38f1c87cd3",
"md5": "3f34e63b60bcc18597663fbd8050fcba",
"sha256": "c0106f2ddbbf7e2d71557c1e7cc090b717f37dd7ddc497836ce4cc5050ab93d9"
},
"downloads": -1,
"filename": "treerequests-0.0.9.tar.gz",
"has_sig": false,
"md5_digest": "3f34e63b60bcc18597663fbd8050fcba",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 31512,
"upload_time": "2025-07-20T11:13:10",
"upload_time_iso_8601": "2025-07-20T11:13:10.330307Z",
"url": "https://files.pythonhosted.org/packages/0d/2b/3c92212076e1429d845be8bc53801f5749f6b0d9cffd6cbf3c38f1c87cd3/treerequests-0.0.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-20 11:13:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "TUVIMEN",
"github_project": "treerequests",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "treerequests"
}
