# whoisit
A Python client to RDAP WHOIS-like services for internet resources (IPs, ASNs, domains,
etc.). `whoisit` is a simple library that makes requests to the "new" RDAP (Registration
Data Access Protocol) query services for internet resource information. These services
started to appear in 2017 and have become more widespread since 2020.
`whoisit` is designed to abstract over RDAP. While RDAP is a basic HTTP and JSON based
protocol which can be implemented in a single line of Python with `requests` the
bootstrapping (which RDAP service to query for what item) and extracting useful
information from the RDAP responses is extensive enough that a library like this is
useful.
## Installation
`whoisit` is pure Python and only has a dependancy on the `requests` and `dateutil`
libraries. You can install `whoisit` via pip:
```bash
$ pip install whoisit
```
Any modern version of Python3 will be compatible.
## Usage
`whoisit` supports the 4 main types of lookups supported by RDAP services. These are:
* ASNs (autonomous systems numbers) known as `autnum` objects
* DNS registrations known as `domain` objects - only some TLDs are supported
* IPv4 and IPv6 addresses and CIDRs / prefixes known as `ip` objects
* Entities (People, organisations etc. by ENTITY-HANDLES) known as `entity` objects
`whoisit` returns parsed RDAP formatted JSON as (mostly) flat dictionaries by default.
Basic examples:
```python
import whoisit
whoisit.bootstrap()
results = whoisit.asn(1234)
print(results['name'])
results = whoisit.domain('example.com')
print(results['nameservers'])
results = whoisit.ip('1.2.3.4')
print(results['name'])
results = whoisit.ip('1.2.3.0/24')
print(results['name'])
results = whoisit.ip('2404:1234:1234:1234:1234:1234:1234:1234')
print(results['name'])
results = whoisit.ip('2404:1234::/32')
print(results['name'])
results = whoisit.entity('ARIN-CHA-1')
print(results['last_changed_date'])
```
### Raw response data
In each case `results` will be a dictionary containing the most useful information for
each request type. If the data you want is not in the response you can request the raw,
unparsed and large RDAP JSON data by adding the `raw=True` argument to the request, for
example:
```python
results = whoisit.domain('example.com', raw=True, follow_related=True)
# 'results' is now the full, raw response from the RDAP service
```
If for some reason you accidentally end up querying the wrong RDAP endpoint your query
should end up still working, for example if you query ARIN for information on the IP
address `1.1.1.1` it will redirect you to APNIC (where `1.1.1.1` is allocated)
automatically.
Some resources, most notably entity handles, do not redirect or have assigned obvious
namespaces linked to particular registries. For these queries `whoisit` will attempt to
guess the RDAP service to query by examining the name for prefixes or postfix, such as
many RIPE entities are named `RIPE-SOMETHING`. If your entity does not have an obvious
prefix or postfix like `ARIN-*` or `*-AP` you will need to tell `whoisit` which registry
to make the request to by specifying the `rir=name` argument. The `rir` argument stands
for "Regional Internet Registry". For example:
```python
# This will work OK because the entity is prefixed with an obvious RIR name
results = whoisit.entity('RIPE-NCC-MNT')
# This will cause a QueryError to be raised because ARIN returns a 404 for RIPE-NCC-MNT
results = whoisit.entity('RIPE-NCC-MNT', rir='arin')
# This will cause a UnsupportedError to be raised because we have no way to detect
# which RDAP service to query as the entity has no RIR prefix or postfix
results = whoisit.entity('AS5089-MNT')
# This will work OK because the entity is registered at RIPE
results = whoisit.entity('AS5089-MNT', rir='ripe')
```
By default for domain lookups `whoisit` will attempt to make sub-requests to any listed
related or registered child RDAP endpoints. You can disable this by setting
`follow_related=False`.
### Weaken SSL ciphers
Some RDAP servers do not have particularly secure SSL implementations. As RDAP returns
read-only and public information it may be acceptable for you to want to downgrade the
security of your `whoisit` requests to successfully return data.
You can use the `allow_insecure_ssl=True` argument to your queries to enable this.
For example (as of 2021-07-25):
```python
# This will result in an SSL error
results = whoisit.domain('nic.work')
# ... SSLError(SSLError(1, '[SSL: DH_KEY_TOO_SMALL] dh key too small (_ssl.c:1129)')))
# This will work
results = whoisit.domain('nic.work', allow_insecure_ssl=True)
```
Note that with `allow_insecure_ssl=True` the upstream RDAP server certificate is
still validated, it just permits weaker SSL ciphers during the handshake. You should
only use `allow_insecure_ssl=True` if your request fails with an SSL cipher or
handshake error first.
## Bootstrapping
`whoisit` needs to know which RDAP service to query for a resource. This information is
provided by the IANA as bootstrapping information. Bootstrapping data simply says things
like "this CIDR is allocated to ARIN, this CIDR is allocated to RIPE" and so on for all
resources. The bootstrap data means you should be directly querying the correct RDAP
server for your request at all times. You should cache the bootstrap information locally
if you plan to make more than a single request otherwise you'll make additional requests
to the IANA every time you run a query. Example bootstrap information caching:
```python
import whoisit
print(whoisit.is_bootstrapped()) # -> False
whoisit.bootstrap() # Slow, makes several HTTP requests to the IANA
print(whoisit.is_bootstrapped()) # -> True
# bootstrap_info returned here is a string of JSON serialised bootstap information
# You can store it in a memory cache or write it to disk for a few days
bootstrap_info = whoisit.save_bootstrap_data()
# Clear bootstrapping data
whoisit.clear_bootstrapping()
# Later, you can do
print(whoisit.is_bootstrapped()) # -> False
if not whoisit.is_bootstrapped():
whoisit.load_bootstrap_data(bootstrap_info) # Fast, no HTTP requests made
print(whoisit.is_bootstrapped()) # -> True
# For convenience internally whoisit stores a timestamp of when the bootstrap data was
# last updated and has a "is older than" helper method
if whoisit.bootstrap_is_older_than(days=3):
# Bootstrap data was last updated over 3 days ago, refresh it
whoisit.clear_bootstrapping()
whoisit.bootstrap()
bootstrap_info = whoisit.save_bootstrap_data() # and save it to upload your cache
```
A reasonable suggested way to handle bootstrapping data would be to use Memcached or
Redis, for example:
```python
import whoisit
import redis
r = redis.Redis(host='localhost', port=6379, db=0)
bootstrap_info = r.get('whoisit_bootstrap_info')
if bootstrap_info:
whoisit.load_bootstrap_data(bootstrap_info)
else:
whoisit.bootstrap()
bootstrap_info = whoisit.save_bootstrap_data()
expire_in_3_days = 60 * 60 * 24 *3
r.set('whoisit_bootstrap_info', bootstrap_info, ex=expire_in_3_days)
# Send queries as normal once bootstrapped
whoisit.asn(12345)
```
Some services, most notably TLDs, do have RDAP servers which may not be set properly
in the IANA bootstrap data. `whoisit` maintains a record of these and can patch the
IANA data to allow more TLDs to be queried. You can enable this with the
`overrides=True` parameter when loading bootstrap data:
```python
whoisit.bootstrap(overrides=True)
```
or
```python
whoisit.load_bootstrap_data(bootstrap_info, overrides=True)
```
**Important**: when using the overrides you may recieve non-standard data, data that
is not in the same format as officially listed IANA data and you may not recieve a copy
of any required terms of service or terms of use. You will have to manually verify data
returned by overridden endpoints.
### Insecure (HTTP) RDAP endpoints
Some RDAP servers are only available over HTTP and not HTTPS. This is disabled by
default. When you bootstrap `whoisit` a `debug` notice will be emitted for any RDAP
endpoint that is not loaded because it is insecure. For example:
```python
# Enable debug logging
import os
os.environ['DEBUG'] = 'true'
# Load and boostrap whoisit
import whoisit
# > [datetime] bootstrap [DEBUG] Cleared bootstrap data
whoisit.bootstrap()
# > ... debug logs ...
# > [datetime] bootstrap [DEBUG] No valid RDAP service URLs could be parsed
# from: ['http://cctld.uz:9000/'] (insecure scheme,
# try whoisit.bootstrap(allow_insecure=True))
# > ... debug logs ...
# > [datetime] bootstrap [DEBUG] Bootstrapped
```
This line informs you that an RDAP endpoint has been skipped because it is only
available over HTTP. You can opt-in to allow insecure endpoints by calling the
bootstrap methods `bootstrap()` and `load_bootstrap_data()` with the optional
`allow_insecure=True` argument. For example:
```python
# Bootstrap with allowing insecure endpoints
whoisit.bootstrap(allow_insecure=True)
```
or
```python
# Load saved bootstrap data with allowing insecure endpoints
whoisit.load_bootstrap_data(bootstrap_info, allow_insecure=True)
```
## Response data
By default `whoisit` returns parsed, summary useful information. This information is
*simplified*. This means that some information is lost from the raw, original data. For
example, `whoisit` doesn't return the date that nameservers were last updated. If you
need more information than `whoisit` returns by default remember to add `raw=True` to
your query and parse the RDAP response yourself.
Data from `whoisit` is returned, where possible, as rich data types such as `datetime`,
`IPv4Network` and `IPv6Network` objects.
The following values are returned for every successful response:
```python
response = {
'handle': str, # Entity handle for the object, always set
'parent_handle': str, # Parent entity handle for the object
'name': str, # Name of the object
'whois_server': str, # WHOIS server hostname object data can be found on
'type': str, # Object type, such as autnum or domain
'terms_of_service_url': str, # URL to the terms of service for using the object data
'copyright_notice', str, # Copyright notice for the object data
'description': list, # List of text lines that describe the object
'last_changed_date': datetime or None, # Date and time the object was last updated
'registration_date': datetime or None, # Date and time the object was registered
'expiration_date': datetime or None, # Date and time the object expires
'rir': str, # Short name of the RIR for the object, such as 'arin'
'url': str, # URL to the RDAP query which was made for this request
'entities': dict, # A dict of entities linked to the object
}
```
The entities dictionary has the following format, note there may be multiple entities
for each role:
```python
response['entities']['some_role'][] = { # Role names are strings, like 'registrant'
'email': str, # Email address of the entity
'handle': str, # Handle of the entity
'name': str, # Name of the entity
'rir': str, # Short name of the RIR where the entity is registered
'type': str, # Type of the entity, usually 'entity'
'url': str, # URL to an RDAP service to query this entity
'whois_server': str, # WHOIS server hostname entity data can be found on
}
```
In addition to the default data for all responses listed above requests have additional
extra fields in their responses, these are:
### Additional ASN response data
```python
# ASN response data includes all shared general response fields above and also:
response = {
'asn_range': list, # A list of the start and end range for an AS allocation
# For example, [123,134] or [123,123]
}
```
### Additional domain response data
```python
# Domain response data includes all shared general response fields above and also:
response = {
'unicode_name': str, # Domain name in unicode if available
'nameservers': list, # List of name servers for the domain as strings
'status': list, # List of the domain states as strings
}
```
### Additional IP response data
```python
# IP response data includes all shared general response fields above and also:
response = {
'country': str, # Two letter country code for the IP block
'ip_version': int, # 4 or 6 to denote the IP version
'assignment_type': str, # Assignment type, such as 'assigned portable'
'network': IPvXNetwork, # A IPv4Network or IPv6Network object for the prefix
}
```
### Additional entity response data
```python
# Entity response data includes all shared general response fields above and also:
response = {
'email': str, # If the entity as a root vcard the email address
}
```
### Full response example
A full example response for an IP query for the IPv4 address `1.1.1.1`:
```python
import whoisit
whoisit.bootstrap()
response = whoisit.ip('1.1.1.1')
print(response)
{
'handle': '1.1.1.0 - 1.1.1.255',
'parent_handle': '',
'name': 'APNIC-LABS',
'whois_server': 'whois.apnic.net',
'type': 'ip network',
'terms_of_service_url': 'http://www.apnic.net/db/dbcopyright.html',
'copyright_notice': '',
'description': [
'APNIC and Cloudflare DNS Resolver project',
'Routed globally by AS13335/Cloudflare',
'Research prefix for APNIC Labs'
],
'last_changed_date': datetime.datetime(2020, 7, 15, 13, 10, 57, tzinfo=tzutc()),
'registration_date': None,
'expiration_date': None,
'url': 'https://rdap.apnic.net/ip/1.1.1.0/24',
'rir': 'apnic',
'entities': {
'abuse': [
{
'handle': 'IRT-APNICRANDNET-AU',
'url': 'https://rdap.apnic.net/entity/IRT-APNICRANDNET-AU',
'type': 'entity',
'name': 'IRT-APNICRANDNET-AU',
'email': 'helpdesk@apnic.net',
'rir': 'apnic'
}
],
'administrative': [
{
'handle': 'AR302-AP',
'url': 'https://rdap.apnic.net/entity/AR302-AP',
'type': 'entity',
'name': 'APNIC RESEARCH',
'email': 'research@apnic.net',
'rir': 'apnic'
}
],
'technical': [
{
'handle': 'AR302-AP',
'url': 'https://rdap.apnic.net/entity/AR302-AP',
'type': 'entity',
'name': 'APNIC RESEARCH',
'email': 'research@apnic.net',
'rir': 'apnic'
]
},
'country': 'AU',
'ip_version': 4,
'assignment_type': 'assigned portable',
'network': IPv4Network('1.1.1.0/24')
}
```
## Full API synopsis
### `whoisit.is_bootstrapped()` -> `bool`
Returns boolean True or False if your `whoisit` instance is bootstrapped or not.
### `whoisit.bootstrap(overrides=bool, allow_insecure=bool)` -> `bool`
Bootstraps your `whoisit` instance with remote IANA bootstrap information. Returns
True or raises a `whoisit.errors.BootstrapError` exception if it fails. This method
makes HTTP requests to the IANA.
### `whoisit.clear_bootstrapping()` -> `bool`
Clears any stored bootstrap information. Always returns boolean True.
### `whoisit.save_bootstrap_data()` -> `str`
Returns a string of JSON serialised bootstrap information if any is loaded. If no
bootstrap information loaded a `whoisit.errors.BootstrapError` will be raised.
### `whoisit.load_bootstrap_data(data=str, overrides=bool, allow_insecure=bool)` -> `bool`
Loads a string of JSON serialised bootstrap data as returned by `save_bootstrap_data()`.
Returns True if the data is loaded or raises a `whoisit.errors.BootstrapError` if
loading fails.
### `whoisit.bootstrap_is_older_than(days=int)` -> `bool`
Tests if the loaded bootstrap data is older than the specified number of days as an
integer. Returns True or False. If no bootstrap information is loaded a
`whoisit.errors.BootstrapError` exception will be raised.
### `whoisit.asn(asn=int, rir=str, raw=bool, allow_insecure_ssl=bool)` -> `dict`
Queries a remote RDAP server for information about the specified AS number. AS number
must be an integer. Returns a dict of information. If `raw=True` is passed a large dict
of the raw RDAP response will be returned. If the query fails a
`whoisit.errors.QueryError` exception will be raised. If no bootstrap data is loaded
a `whoisit.errors.BootstrapError` exception will be raised. if `allow_insecure_ssl=True`
is passed the RDAP queries will allow weaker SSL handshakes. Examples:
```python
whoisit.asn(12345)
whoisit.asn(12345, rir='arin')
whoisit.asn(12345, raw=True)
whoisit.asn(12345, rir='arin', raw=True)
whoisit.asn(12345, allow_insecure_ssl=True)
```
### `whoisit.domain(domain=str, raw=bool, allow_insecure_ssl=bool)` -> `dict`
Queries a remote RDAP server for information about the specified domain name. The domain
name must be a string and in a valid domain name "something.tld" style format. Returns a
dict of information. If `raw=True` is passed a large dict of the raw RDAP response will
be returned. If the query fails a `whoisit.errors.QueryError` exception will be raised.
If no bootstrap data is loaded a `whoisit.errors.BootstrapError` exception will be
raised. If the TLD is unsupported a `whoisit.errors.UnsupportedError` exception will be
raised. if `allow_insecure_ssl=True` is passed the RDAP queries will allow weaker SSL
handshakes. **Note that not all TLDs are supported, only some have RDAP services!**
Examples:
```python
whoisit.domain('example.com')
whoisit.domain('example.com', raw=True)
whoisit.domain('example.com', allow_insecure_ssl=True)
```
### `whoisit.ip(ip="1.1.1.1", rir=str, raw=bool, allow_insecure_ssl=bool)` -> `dict`
Queries a remote RDAP server for information about the specified IP address or CIDR. The
IP address or CIDR must be a string and in the correct IP address or CIDR format or
any one of IPv4Address, IPv4Network, IPv6Address or IPv6Network objects. Returns a dict
of information. If `raw=True` is passed a large dict of the raw RDAP response will be
returned. If the query fails a `whoisit.errors.QueryError` exception will be raised. If
no bootstrap data is loaded a `whoisit.errors.BootstrapError` exception will be raised.
if `allow_insecure_ssl=True` is passed the RDAP queries will allow weaker SSL handshakes.
Examples:
```python
whoisit.ip('1.1.1.1')
whoisit.ip('1.1.1.1', rir='apnic')
whoisit.ip('1.1.1.1', raw=True, rir='apnic')
whoisit.ip('1.1.1.0/24')
whoisit.ip(IPv4Address('1.1.1.1'))
whoisit.ip(IPv4Network('1.1.1.0/24'))
whoisit.ip(IPv6Address('2001:4860:4860::8888'))
whoisit.ip(IPv6Network('2001:4860::/32'), rir='arin')
whoisit.ip('1.1.1.1', allow_insecure_ssl=True)
```
### `whoisit.entity(entity=str, rir=str, raw=bool, allow_insecure_ssl=bool)` -> `dict`
Queries a remote RDAP server for information about the specified entity name. The
entity name must be a string and in the correct entity format. Returns a dict of
information. If `raw=True` is passed a large dict of the raw RDAP response will be
returned. If the query fails a `whoisit.errors.QueryError` exception will be raised.
If no bootstrap data is loaded a `whoisit.errors.BootstrapError` exception will be
raised. if `allow_insecure_ssl=True` is passed the RDAP queries will allow weaker
SSL handshakes. Examples:
```python
whoisit.entity('ZG39-ARIN')
whoisit.entity('ZG39-ARIN', rir='arin')
whoisit.entity('ZG39-ARIN', rir='arin', raw=True)
whoisit.entity('ZG39-ARIN', allow_insecure_ssl=True)
```
## Data usage
All data returned by RDAP servers are covered by the various policies embeddd in the
results. As such you should carefuly review your usage of the data to make sure it
complies with the policy of the RDAP server you are querying.
## Excessive use
As an API client `whoisit` is entirely subject to the resource and request limits
applied by the remote RDAP servers it queries. If you recieve request errors for rate
limiting you should slow down your requests. Different servers have different limits.
The LACNIC RDAP server in particular only permits a low number of requests per minute.
# Tests
There is a test suite that you can run by cloning this repository, installing the
required dependancies and execuiting:
```bash
$ make test
```
# Debugging
`whoisit` will check for a `DEBUG` environment variable and if set, will output debug
logs that detail the internals for the bootstrapping, requests and parsing operations.
If you want to enable debug logging, set `DEBUG=true` (or `1` or `y` etc.). For example:
```bash
$ export DEBUG=true
$ python3 some-script-that-uses-whoisit.py
```
# Contributing
All properly formatted and sensible pull requests, issues and comments are welcome.
Raw data
{
"_id": null,
"home_page": "https://github.com/meeb/whoisit",
"name": "whoisit",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "whoisit, whois, rdap, ip, network, cidr, prefix, domain, asn, autnum, tld, entity, handle, arin, afrinic, apnic, ripe, lacnic",
"author": "https://github.com/meeb",
"author_email": "meeb@meeb.org",
"download_url": "https://files.pythonhosted.org/packages/9a/b0/d3e8928dfca485ca8e99c5408a73f30fc5a9f52fa474464c7efb5b68a031/whoisit-2.7.6.tar.gz",
"platform": null,
"description": "# whoisit\n\nA Python client to RDAP WHOIS-like services for internet resources (IPs, ASNs, domains,\netc.). `whoisit` is a simple library that makes requests to the \"new\" RDAP (Registration\nData Access Protocol) query services for internet resource information. These services\nstarted to appear in 2017 and have become more widespread since 2020.\n\n`whoisit` is designed to abstract over RDAP. While RDAP is a basic HTTP and JSON based\nprotocol which can be implemented in a single line of Python with `requests` the\nbootstrapping (which RDAP service to query for what item) and extracting useful\ninformation from the RDAP responses is extensive enough that a library like this is\nuseful.\n\n\n## Installation\n\n`whoisit` is pure Python and only has a dependancy on the `requests` and `dateutil`\nlibraries. You can install `whoisit` via pip:\n\n```bash\n$ pip install whoisit\n```\n\nAny modern version of Python3 will be compatible.\n\n\n## Usage\n\n`whoisit` supports the 4 main types of lookups supported by RDAP services. These are:\n\n * ASNs (autonomous systems numbers) known as `autnum` objects\n * DNS registrations known as `domain` objects - only some TLDs are supported\n * IPv4 and IPv6 addresses and CIDRs / prefixes known as `ip` objects\n * Entities (People, organisations etc. by ENTITY-HANDLES) known as `entity` objects\n\n`whoisit` returns parsed RDAP formatted JSON as (mostly) flat dictionaries by default.\n\nBasic examples:\n\n```python\nimport whoisit\n\nwhoisit.bootstrap()\n\nresults = whoisit.asn(1234)\nprint(results['name'])\n\nresults = whoisit.domain('example.com')\nprint(results['nameservers'])\n\nresults = whoisit.ip('1.2.3.4')\nprint(results['name'])\n\nresults = whoisit.ip('1.2.3.0/24')\nprint(results['name'])\n\nresults = whoisit.ip('2404:1234:1234:1234:1234:1234:1234:1234')\nprint(results['name'])\n\nresults = whoisit.ip('2404:1234::/32')\nprint(results['name'])\n\nresults = whoisit.entity('ARIN-CHA-1')\nprint(results['last_changed_date'])\n```\n\n\n### Raw response data\n\nIn each case `results` will be a dictionary containing the most useful information for\neach request type. If the data you want is not in the response you can request the raw,\nunparsed and large RDAP JSON data by adding the `raw=True` argument to the request, for\nexample:\n\n```python\nresults = whoisit.domain('example.com', raw=True, follow_related=True)\n# 'results' is now the full, raw response from the RDAP service\n```\n\nIf for some reason you accidentally end up querying the wrong RDAP endpoint your query\nshould end up still working, for example if you query ARIN for information on the IP\naddress `1.1.1.1` it will redirect you to APNIC (where `1.1.1.1` is allocated)\nautomatically.\n\nSome resources, most notably entity handles, do not redirect or have assigned obvious\nnamespaces linked to particular registries. For these queries `whoisit` will attempt to\nguess the RDAP service to query by examining the name for prefixes or postfix, such as\nmany RIPE entities are named `RIPE-SOMETHING`. If your entity does not have an obvious\nprefix or postfix like `ARIN-*` or `*-AP` you will need to tell `whoisit` which registry\nto make the request to by specifying the `rir=name` argument. The `rir` argument stands\nfor \"Regional Internet Registry\". For example:\n\n```python\n# This will work OK because the entity is prefixed with an obvious RIR name\nresults = whoisit.entity('RIPE-NCC-MNT')\n\n# This will cause a QueryError to be raised because ARIN returns a 404 for RIPE-NCC-MNT\nresults = whoisit.entity('RIPE-NCC-MNT', rir='arin')\n\n# This will cause a UnsupportedError to be raised because we have no way to detect\n# which RDAP service to query as the entity has no RIR prefix or postfix\nresults = whoisit.entity('AS5089-MNT')\n\n# This will work OK because the entity is registered at RIPE\nresults = whoisit.entity('AS5089-MNT', rir='ripe')\n```\n\nBy default for domain lookups `whoisit` will attempt to make sub-requests to any listed\nrelated or registered child RDAP endpoints. You can disable this by setting\n`follow_related=False`.\n\n\n### Weaken SSL ciphers\n\nSome RDAP servers do not have particularly secure SSL implementations. As RDAP returns\nread-only and public information it may be acceptable for you to want to downgrade the\nsecurity of your `whoisit` requests to successfully return data.\n\nYou can use the `allow_insecure_ssl=True` argument to your queries to enable this.\n\nFor example (as of 2021-07-25):\n\n```python\n# This will result in an SSL error\nresults = whoisit.domain('nic.work')\n# ... SSLError(SSLError(1, '[SSL: DH_KEY_TOO_SMALL] dh key too small (_ssl.c:1129)')))\n\n# This will work\nresults = whoisit.domain('nic.work', allow_insecure_ssl=True)\n```\n\nNote that with `allow_insecure_ssl=True` the upstream RDAP server certificate is\nstill validated, it just permits weaker SSL ciphers during the handshake. You should\nonly use `allow_insecure_ssl=True` if your request fails with an SSL cipher or\nhandshake error first.\n\n\n## Bootstrapping\n\n`whoisit` needs to know which RDAP service to query for a resource. This information is\nprovided by the IANA as bootstrapping information. Bootstrapping data simply says things\nlike \"this CIDR is allocated to ARIN, this CIDR is allocated to RIPE\" and so on for all\nresources. The bootstrap data means you should be directly querying the correct RDAP\nserver for your request at all times. You should cache the bootstrap information locally\nif you plan to make more than a single request otherwise you'll make additional requests\nto the IANA every time you run a query. Example bootstrap information caching:\n\n```python\nimport whoisit\n\nprint(whoisit.is_bootstrapped()) # -> False\nwhoisit.bootstrap() # Slow, makes several HTTP requests to the IANA\nprint(whoisit.is_bootstrapped()) # -> True\n\n# bootstrap_info returned here is a string of JSON serialised bootstap information\n# You can store it in a memory cache or write it to disk for a few days\nbootstrap_info = whoisit.save_bootstrap_data()\n\n# Clear bootstrapping data\nwhoisit.clear_bootstrapping()\n\n# Later, you can do\nprint(whoisit.is_bootstrapped()) # -> False\nif not whoisit.is_bootstrapped():\n whoisit.load_bootstrap_data(bootstrap_info) # Fast, no HTTP requests made\nprint(whoisit.is_bootstrapped()) # -> True\n\n# For convenience internally whoisit stores a timestamp of when the bootstrap data was\n# last updated and has a \"is older than\" helper method\nif whoisit.bootstrap_is_older_than(days=3):\n # Bootstrap data was last updated over 3 days ago, refresh it\n whoisit.clear_bootstrapping()\n whoisit.bootstrap()\n bootstrap_info = whoisit.save_bootstrap_data() # and save it to upload your cache\n```\n\nA reasonable suggested way to handle bootstrapping data would be to use Memcached or\nRedis, for example:\n\n```python\nimport whoisit\nimport redis\n\nr = redis.Redis(host='localhost', port=6379, db=0)\n\nbootstrap_info = r.get('whoisit_bootstrap_info')\nif bootstrap_info:\n whoisit.load_bootstrap_data(bootstrap_info)\nelse:\n whoisit.bootstrap()\n bootstrap_info = whoisit.save_bootstrap_data()\n expire_in_3_days = 60 * 60 * 24 *3\n r.set('whoisit_bootstrap_info', bootstrap_info, ex=expire_in_3_days)\n\n# Send queries as normal once bootstrapped\nwhoisit.asn(12345)\n```\n\nSome services, most notably TLDs, do have RDAP servers which may not be set properly\nin the IANA bootstrap data. `whoisit` maintains a record of these and can patch the\nIANA data to allow more TLDs to be queried. You can enable this with the\n`overrides=True` parameter when loading bootstrap data:\n\n```python\nwhoisit.bootstrap(overrides=True)\n```\n\nor\n\n```python\nwhoisit.load_bootstrap_data(bootstrap_info, overrides=True)\n```\n\n**Important**: when using the overrides you may recieve non-standard data, data that\nis not in the same format as officially listed IANA data and you may not recieve a copy\nof any required terms of service or terms of use. You will have to manually verify data\nreturned by overridden endpoints.\n\n\n### Insecure (HTTP) RDAP endpoints\n\nSome RDAP servers are only available over HTTP and not HTTPS. This is disabled by\ndefault. When you bootstrap `whoisit` a `debug` notice will be emitted for any RDAP\nendpoint that is not loaded because it is insecure. For example:\n\n```python\n# Enable debug logging\nimport os\nos.environ['DEBUG'] = 'true'\n # Load and boostrap whoisit\nimport whoisit\n# > [datetime] bootstrap [DEBUG] Cleared bootstrap data\nwhoisit.bootstrap()\n# > ... debug logs ...\n# > [datetime] bootstrap [DEBUG] No valid RDAP service URLs could be parsed\n# from: ['http://cctld.uz:9000/'] (insecure scheme,\n# try whoisit.bootstrap(allow_insecure=True))\n# > ... debug logs ...\n# > [datetime] bootstrap [DEBUG] Bootstrapped\n```\n\nThis line informs you that an RDAP endpoint has been skipped because it is only\navailable over HTTP. You can opt-in to allow insecure endpoints by calling the\nbootstrap methods `bootstrap()` and `load_bootstrap_data()` with the optional\n`allow_insecure=True` argument. For example:\n\n```python\n# Bootstrap with allowing insecure endpoints\nwhoisit.bootstrap(allow_insecure=True)\n```\n\nor\n\n```python\n# Load saved bootstrap data with allowing insecure endpoints\nwhoisit.load_bootstrap_data(bootstrap_info, allow_insecure=True)\n```\n\n\n## Response data\n\nBy default `whoisit` returns parsed, summary useful information. This information is\n*simplified*. This means that some information is lost from the raw, original data. For\nexample, `whoisit` doesn't return the date that nameservers were last updated. If you\nneed more information than `whoisit` returns by default remember to add `raw=True` to\nyour query and parse the RDAP response yourself.\n\nData from `whoisit` is returned, where possible, as rich data types such as `datetime`,\n`IPv4Network` and `IPv6Network` objects.\n\nThe following values are returned for every successful response:\n\n```python\nresponse = {\n 'handle': str, # Entity handle for the object, always set\n 'parent_handle': str, # Parent entity handle for the object\n 'name': str, # Name of the object\n 'whois_server': str, # WHOIS server hostname object data can be found on\n 'type': str, # Object type, such as autnum or domain\n 'terms_of_service_url': str, # URL to the terms of service for using the object data\n 'copyright_notice', str, # Copyright notice for the object data\n 'description': list, # List of text lines that describe the object\n 'last_changed_date': datetime or None, # Date and time the object was last updated\n 'registration_date': datetime or None, # Date and time the object was registered\n 'expiration_date': datetime or None, # Date and time the object expires\n 'rir': str, # Short name of the RIR for the object, such as 'arin'\n 'url': str, # URL to the RDAP query which was made for this request\n 'entities': dict, # A dict of entities linked to the object\n}\n```\n\nThe entities dictionary has the following format, note there may be multiple entities\nfor each role:\n\n```python\nresponse['entities']['some_role'][] = { # Role names are strings, like 'registrant'\n 'email': str, # Email address of the entity\n 'handle': str, # Handle of the entity\n 'name': str, # Name of the entity\n 'rir': str, # Short name of the RIR where the entity is registered\n 'type': str, # Type of the entity, usually 'entity'\n 'url': str, # URL to an RDAP service to query this entity \n 'whois_server': str, # WHOIS server hostname entity data can be found on\n}\n```\n\nIn addition to the default data for all responses listed above requests have additional\nextra fields in their responses, these are:\n\n### Additional ASN response data\n\n```python\n# ASN response data includes all shared general response fields above and also:\nresponse = {\n 'asn_range': list, # A list of the start and end range for an AS allocation\n # For example, [123,134] or [123,123]\n}\n```\n\n### Additional domain response data\n\n```python\n# Domain response data includes all shared general response fields above and also:\nresponse = {\n 'unicode_name': str, # Domain name in unicode if available\n 'nameservers': list, # List of name servers for the domain as strings\n 'status': list, # List of the domain states as strings\n}\n```\n\n### Additional IP response data\n\n```python\n# IP response data includes all shared general response fields above and also:\nresponse = {\n 'country': str, # Two letter country code for the IP block\n 'ip_version': int, # 4 or 6 to denote the IP version\n 'assignment_type': str, # Assignment type, such as 'assigned portable'\n 'network': IPvXNetwork, # A IPv4Network or IPv6Network object for the prefix\n}\n```\n\n### Additional entity response data\n\n```python\n# Entity response data includes all shared general response fields above and also:\nresponse = {\n 'email': str, # If the entity as a root vcard the email address\n}\n```\n\n### Full response example\n\nA full example response for an IP query for the IPv4 address `1.1.1.1`:\n\n```python\nimport whoisit\nwhoisit.bootstrap()\nresponse = whoisit.ip('1.1.1.1')\nprint(response)\n{\n 'handle': '1.1.1.0 - 1.1.1.255',\n 'parent_handle': '',\n 'name': 'APNIC-LABS',\n 'whois_server': 'whois.apnic.net',\n 'type': 'ip network',\n 'terms_of_service_url': 'http://www.apnic.net/db/dbcopyright.html',\n 'copyright_notice': '',\n 'description': [\n 'APNIC and Cloudflare DNS Resolver project',\n 'Routed globally by AS13335/Cloudflare',\n 'Research prefix for APNIC Labs'\n ],\n 'last_changed_date': datetime.datetime(2020, 7, 15, 13, 10, 57, tzinfo=tzutc()),\n 'registration_date': None,\n 'expiration_date': None,\n 'url': 'https://rdap.apnic.net/ip/1.1.1.0/24',\n 'rir': 'apnic',\n 'entities': {\n 'abuse': [\n {\n 'handle': 'IRT-APNICRANDNET-AU',\n 'url': 'https://rdap.apnic.net/entity/IRT-APNICRANDNET-AU',\n 'type': 'entity',\n 'name': 'IRT-APNICRANDNET-AU',\n 'email': 'helpdesk@apnic.net',\n 'rir': 'apnic'\n }\n ],\n 'administrative': [\n {\n 'handle': 'AR302-AP',\n 'url': 'https://rdap.apnic.net/entity/AR302-AP',\n 'type': 'entity',\n 'name': 'APNIC RESEARCH',\n 'email': 'research@apnic.net',\n 'rir': 'apnic'\n }\n ],\n 'technical': [\n {\n 'handle': 'AR302-AP',\n 'url': 'https://rdap.apnic.net/entity/AR302-AP',\n 'type': 'entity',\n 'name': 'APNIC RESEARCH',\n 'email': 'research@apnic.net',\n 'rir': 'apnic'\n ]\n },\n 'country': 'AU',\n 'ip_version': 4,\n 'assignment_type': 'assigned portable',\n 'network': IPv4Network('1.1.1.0/24')\n}\n```\n\n## Full API synopsis\n\n### `whoisit.is_bootstrapped()` -> `bool`\n\nReturns boolean True or False if your `whoisit` instance is bootstrapped or not.\n\n### `whoisit.bootstrap(overrides=bool, allow_insecure=bool)` -> `bool`\n\nBootstraps your `whoisit` instance with remote IANA bootstrap information. Returns\nTrue or raises a `whoisit.errors.BootstrapError` exception if it fails. This method\nmakes HTTP requests to the IANA.\n\n### `whoisit.clear_bootstrapping()` -> `bool`\n\nClears any stored bootstrap information. Always returns boolean True.\n\n### `whoisit.save_bootstrap_data()` -> `str`\n\nReturns a string of JSON serialised bootstrap information if any is loaded. If no\nbootstrap information loaded a `whoisit.errors.BootstrapError` will be raised.\n\n### `whoisit.load_bootstrap_data(data=str, overrides=bool, allow_insecure=bool)` -> `bool`\n\nLoads a string of JSON serialised bootstrap data as returned by `save_bootstrap_data()`.\nReturns True if the data is loaded or raises a `whoisit.errors.BootstrapError` if\nloading fails.\n\n### `whoisit.bootstrap_is_older_than(days=int)` -> `bool`\n\nTests if the loaded bootstrap data is older than the specified number of days as an\ninteger. Returns True or False. If no bootstrap information is loaded a\n`whoisit.errors.BootstrapError` exception will be raised.\n\n### `whoisit.asn(asn=int, rir=str, raw=bool, allow_insecure_ssl=bool)` -> `dict`\n\nQueries a remote RDAP server for information about the specified AS number. AS number\nmust be an integer. Returns a dict of information. If `raw=True` is passed a large dict\nof the raw RDAP response will be returned. If the query fails a\n`whoisit.errors.QueryError` exception will be raised. If no bootstrap data is loaded\na `whoisit.errors.BootstrapError` exception will be raised. if `allow_insecure_ssl=True`\nis passed the RDAP queries will allow weaker SSL handshakes. Examples:\n\n```python\nwhoisit.asn(12345)\nwhoisit.asn(12345, rir='arin')\nwhoisit.asn(12345, raw=True)\nwhoisit.asn(12345, rir='arin', raw=True)\nwhoisit.asn(12345, allow_insecure_ssl=True)\n```\n\n### `whoisit.domain(domain=str, raw=bool, allow_insecure_ssl=bool)` -> `dict`\n\nQueries a remote RDAP server for information about the specified domain name. The domain\nname must be a string and in a valid domain name \"something.tld\" style format. Returns a\ndict of information. If `raw=True` is passed a large dict of the raw RDAP response will\nbe returned. If the query fails a `whoisit.errors.QueryError` exception will be raised.\nIf no bootstrap data is loaded a `whoisit.errors.BootstrapError` exception will be\nraised. If the TLD is unsupported a `whoisit.errors.UnsupportedError` exception will be\nraised. if `allow_insecure_ssl=True` is passed the RDAP queries will allow weaker SSL\nhandshakes. **Note that not all TLDs are supported, only some have RDAP services!**\nExamples:\n\n```python\nwhoisit.domain('example.com')\nwhoisit.domain('example.com', raw=True)\nwhoisit.domain('example.com', allow_insecure_ssl=True)\n```\n\n\n### `whoisit.ip(ip=\"1.1.1.1\", rir=str, raw=bool, allow_insecure_ssl=bool)` -> `dict`\n\nQueries a remote RDAP server for information about the specified IP address or CIDR. The\nIP address or CIDR must be a string and in the correct IP address or CIDR format or\nany one of IPv4Address, IPv4Network, IPv6Address or IPv6Network objects. Returns a dict\nof information. If `raw=True` is passed a large dict of the raw RDAP response will be\nreturned. If the query fails a `whoisit.errors.QueryError` exception will be raised. If\nno bootstrap data is loaded a `whoisit.errors.BootstrapError` exception will be raised.\nif `allow_insecure_ssl=True` is passed the RDAP queries will allow weaker SSL handshakes.\nExamples:\n\n```python\nwhoisit.ip('1.1.1.1')\nwhoisit.ip('1.1.1.1', rir='apnic')\nwhoisit.ip('1.1.1.1', raw=True, rir='apnic')\nwhoisit.ip('1.1.1.0/24')\nwhoisit.ip(IPv4Address('1.1.1.1'))\nwhoisit.ip(IPv4Network('1.1.1.0/24'))\nwhoisit.ip(IPv6Address('2001:4860:4860::8888'))\nwhoisit.ip(IPv6Network('2001:4860::/32'), rir='arin')\nwhoisit.ip('1.1.1.1', allow_insecure_ssl=True)\n```\n\n### `whoisit.entity(entity=str, rir=str, raw=bool, allow_insecure_ssl=bool)` -> `dict`\n\nQueries a remote RDAP server for information about the specified entity name. The\nentity name must be a string and in the correct entity format. Returns a dict of\ninformation. If `raw=True` is passed a large dict of the raw RDAP response will be\nreturned. If the query fails a `whoisit.errors.QueryError` exception will be raised.\nIf no bootstrap data is loaded a `whoisit.errors.BootstrapError` exception will be\nraised. if `allow_insecure_ssl=True` is passed the RDAP queries will allow weaker\nSSL handshakes. Examples:\n\n```python\nwhoisit.entity('ZG39-ARIN')\nwhoisit.entity('ZG39-ARIN', rir='arin')\nwhoisit.entity('ZG39-ARIN', rir='arin', raw=True)\nwhoisit.entity('ZG39-ARIN', allow_insecure_ssl=True)\n```\n\n\n## Data usage\n\nAll data returned by RDAP servers are covered by the various policies embeddd in the\nresults. As such you should carefuly review your usage of the data to make sure it\ncomplies with the policy of the RDAP server you are querying.\n\n\n## Excessive use\n\nAs an API client `whoisit` is entirely subject to the resource and request limits\napplied by the remote RDAP servers it queries. If you recieve request errors for rate\nlimiting you should slow down your requests. Different servers have different limits.\nThe LACNIC RDAP server in particular only permits a low number of requests per minute.\n\n\n# Tests\n\nThere is a test suite that you can run by cloning this repository, installing the\nrequired dependancies and execuiting:\n\n```bash\n$ make test\n```\n\n\n# Debugging\n\n`whoisit` will check for a `DEBUG` environment variable and if set, will output debug\nlogs that detail the internals for the bootstrapping, requests and parsing operations.\nIf you want to enable debug logging, set `DEBUG=true` (or `1` or `y` etc.). For example:\n\n```bash\n$ export DEBUG=true\n$ python3 some-script-that-uses-whoisit.py\n```\n\n\n# Contributing\n\nAll properly formatted and sensible pull requests, issues and comments are welcome.\n",
"bugtrack_url": null,
"license": "BSD",
"summary": "A Python client to RDAP WHOIS-like services for internet resources.",
"version": "2.7.6",
"project_urls": {
"Homepage": "https://github.com/meeb/whoisit"
},
"split_keywords": [
"whoisit",
" whois",
" rdap",
" ip",
" network",
" cidr",
" prefix",
" domain",
" asn",
" autnum",
" tld",
" entity",
" handle",
" arin",
" afrinic",
" apnic",
" ripe",
" lacnic"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "9ab0d3e8928dfca485ca8e99c5408a73f30fc5a9f52fa474464c7efb5b68a031",
"md5": "9db8c7f7cfad49f529d6f36d237567d2",
"sha256": "9d2278a5e3321f5c498f1e698bca29a68c3b1c461dbb59ffca9a692b59048979"
},
"downloads": -1,
"filename": "whoisit-2.7.6.tar.gz",
"has_sig": false,
"md5_digest": "9db8c7f7cfad49f529d6f36d237567d2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 33621,
"upload_time": "2024-04-30T09:49:10",
"upload_time_iso_8601": "2024-04-30T09:49:10.350953Z",
"url": "https://files.pythonhosted.org/packages/9a/b0/d3e8928dfca485ca8e99c5408a73f30fc5a9f52fa474464c7efb5b68a031/whoisit-2.7.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-30 09:49:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "meeb",
"github_project": "whoisit",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "whoisit"
}
JSON
