Route Views requires many tools to support its functionality.
This package is a place for those tools to live.
> Today, this package also acts as the primary 'python programming library' for Route Views.
# CLI Tools
As of today, there are two types of tools provided by this package: monitoring, and automation.
> ℹ The CLI tools have certain prefix based on tool type:
>
> * `rvm-` is for Monitoring tools that are used only to learn about the running state of Route Views.
> * `routeviews-` is for Automation tools that enable some automated workflow.
## `rvm-latest-mrt` CLI tool
Monitoring tool that shows information about the latest Route Views MRT files (RIB, UPDATE) on a collector.
> Use the `--help` flag to learn more about this tool's abilities.
$ rvm-latest-mrt
Latest RIB: /mnt/storage/bgpdata/2022.08/RIBS/rib.20220802.0800.bz2
Latest UPDATE: /mnt/storage/bgpdata/2022.08/UPDATES/update.20220802.0800.bz2
## `rvm-bmp-status` CLI tool
Monitoring tool that shows information about BMP connections on a collector.
> Use the `--help` flag to learn more about this tool's abilities.
$ sudo rvm-bmp-status
BMP Collector: bmp.routeviews.org
Connection Uptime: 8 hours
Data sent: 1.5 GB
Bytes queued: 2 Bytes
Bytes queued to Kernel: 3 Bytes
## `rvm-bgp-status` CLI tool
Monitoring tool that shows relevant information about BGP connections on a collector.
> Use the `--help` flag to learn more about this tool's abilities.
$ sudo rvm-bgp-status
ASN Peer Address State Prefixes Uptime InQ Uptime
----- ------------------ ----------- ---------- -------- ------ --------
65129 128.223.51.78 ESTABLISHED 0 0 149732 1
3582 128.223.253.9 ESTABLISHED 899115 0 149732 1
3582 128.223.253.10 ESTABLISHED 899147 0 149732 1
3582 2001:468:d01:fd::9 ESTABLISHED 162442 0 149732 1
3582 2001:468:d01:fd::a ESTABLISHED 162443 0 149732 1
## `rvm-haproxy-stats` CLI tool
Get stats from HAProxy Stick Tables on a Route Views collector.
> ℹ HAProxy runs on our collectors to enable telnet access.
$ rvm-haproxy-stats --min-conn-cnt 10 --sudo
Key Current Conn. Total Conn. Data In Rate Date Out Rate
------------------------ --------------- ------------- -------------- ---------------
175.30.79.245 0 10 0 Bytes 901 Bytes
141.255.166.2 0 12 69 Bytes 1.3 kB
179.43.187.243 0 13 0 Bytes 1.5 kB
180.103.51.200 0 14 0 Bytes 1.0 kB
31.220.3.140 0 27 0 Bytes 907 Bytes
2001:468:d01:33::80df:78 0 27 457 Bytes 9.0 kB
## `routeviews-peer-request` CLI tool
This tool is for (consistently) updating the [Route Views ansible inventory (private repo)](https://github.com/routeviews/infra) when folks submit new peer requests.
> This tool uses information provided by PeeringDB for the peering information.
### Prerequisites
1. *Route Views Ansible Inventory*: You must have a local copy of the Route Views ansible inventory available, for this tool to update.
* If you will be running this command with any regularity, is useful to export the `ROUTEVIEWS_INVENTORY` environment variable to point to your local copy of the [Route Views ansible inventory repository (private)](https://github.com/routeviews/infra).
```
# (Optional) Place in your ~/.bashrc
$ export ROUTEVIEWS_INVENTORY='<WORKING_TREE>/ansible/inventory'
```
* `<WORKING_TREE>` refers to wherever you've cloned the repository on your filesystem.
### Example: Show Options between Route Views and Autonomous System (AS)
If an AS would like to know what peering sessions are possible according to this tool, we do have a `--show-options` flag that will enable this!
Provide only the `asn` argument along with the `show-options` flag to try this out!
> ℹ Anyone can run this solution!
> This solution only depends on the public PeeringDB API.
$ routeviews-peer-request --asn 15169 --show-options
Potential BGP Peerings for networks:
- RouteViews (ASN: 6447), and
- Google LLC (ASN: 15169).
Exchange RV Collector Router
---------------------------------------------- ------------------------ ------------------------
Equinix Chicago 208.115.136.187 208.115.136.21
Equinix Chicago 2001:504:0:4::6447:1 2001:504:0:4:0:1:5169:1
Equinix Palo Alto 198.32.176.5 198.32.176.31
Equinix Palo Alto 2001:504:d::5 2001:504:d::1f
LINX LON1: Main 195.66.225.222 195.66.224.125
LINX LON1: Main 2001:7f8:4::192f:1 2001:7f8:4::3b41:1
Digital Realty Atlanta 198.32.132.3 198.32.132.41
Digital Realty Atlanta 2001:478:132::3 2001:478:132::41
DE-CIX Frankfurt: DE-CIX Frankfurt Peering LAN 80.81.193.49 80.81.192.108
DE-CIX Frankfurt: DE-CIX Frankfurt Peering LAN 2001:7f8::192f:0:1 2001:7f8::3b41:0:1
DE-CIX Frankfurt: DE-CIX Frankfurt Peering LAN 80.81.193.49 80.81.193.108
DE-CIX Frankfurt: DE-CIX Frankfurt Peering LAN 2001:7f8::192f:0:1 2001:7f8::3b41:0:2
DIX-IE 202.249.2.166 202.249.2.189
... trimmed for brevity...
### Example: Peer with an Autonomous System (AS) at ***ALL*** IXes
If an AS is wanting to connect wherever possible, provide only the `asn` argument and the tool will determine all the possible `ip` arguments from PeeringDB.
$ routeviews-peer-request \
--inventory <WORKING_TREE>/ansible/inventory \
--asn 15169
### Changes
+++ <WORKING_TREE>/ansible/inventory/host_vars/route-views.perth.routeviews.org
+ peer_as: 15169
+ peer_address: 218.100.52.3
+ description: 'IX Australia (Sydney NSW): NSW-IX'
+ afi_safis:
+ - ipv4_unicast
+ peer_as: 15169
+ peer_address: 2001:7fa:11:4:0:3b41:0:1
+ description: 'IX Australia (Sydney NSW): NSW-IX'
+ afi_safis:
+ - ipv6_unicast
+ peer_as: 15169
+ peer_address: 218.100.53.29
+ description: 'IX Australia (Sydney NSW): NSW-IX'
+ afi_safis:
+ - ipv4_unicast
+ peer_as: 15169
+ peer_address: 2001:7fa:11:4:0:3b41:0:2
+ description: 'IX Australia (Sydney NSW): NSW-IX'
+ afi_safis:
+ - ipv6_unicast
+ peer_as: 15169
+ peer_address: 218.100.78.154
+ description: 'IX Australia (Melbourne VIC): VIC-IX'
+ afi_safis:
+ - ipv4_unicast
+ peer_as: 15169
+ peer_address: 2001:7fa:11:1:0:3b41:0:2
+ description: 'IX Australia (Melbourne VIC): VIC-IX'
+ afi_safis:
+ - ipv6_unicast
+ peer_as: 15169
+ peer_address: 218.100.78.153
+ description: 'IX Australia (Melbourne VIC): VIC-IX'
+ afi_safis:
+ - ipv4_unicast
+ peer_as: 15169
+ peer_address: 2001:7fa:11:1:0:3b41:0:1
+ description: 'IX Australia (Melbourne VIC): VIC-IX'
+ afi_safis:
+ - ipv6_unicast
+++ <WORKING_TREE>/ansible/inventory/host_vars/route-views.amsix.routeviews.org
+ peer_as: 15169
+ peer_address: 80.249.208.247
+ description: AMS-IX
+ afi_safis:
+ - ipv4_unicast
+ peer_as: 15169
+ peer_address: 2001:7f8:1::a501:5169:1
+ description: AMS-IX
+ afi_safis:
... trimmed for brevity...
### Example: Peer with an Autonomous System (AS) using IP Addresses
This tool supports arguments for `asn`, as well as `ip`.
The `ip` argument can be used multiple times to peer with multiple IP Addresses at once.
> ⚠ Only supports peering with one AS at a time.
> ℹ Use the `--help` flag to learn more about how to use these arguments.
As discussed in the [prerequisites](#prerequisites-1), there is also the `inventory` argument required that points to the "inventory/" directory.
> ℹ Tip: Provide an `asn` and omit the `ip` argument entirely -- the tool will attempt to peer with ALL compatible IP Addresses for the provided `asn`!
$ routeviews-peer-request \
--inventory <WORKING_TREE>/ansible/inventory \
--asn 15169 \
--ip 202.249.2.189 \
--ip 2001:200:0:fe00::3b41:0 \
--ip 80.249.208.247 \
--ip 2001:7f8:1::a501:5169:1
### Changes
+++ <WORKING_TREE>/ansible/inventory/host_vars/route-views.amsix.routeviews.org
+ peer_as: 15169
+ peer_address: 80.249.208.247
+ description: AMS-IX
+ afi_safis:
+ - ipv4_unicast
+ peer_as: 15169
+ peer_address: 2001:7f8:1::a501:5169:1
+ description: AMS-IX
+ afi_safis:
+ - ipv6_unicast
+++ <WORKING_TREE>/ansible/inventory/host_vars/route-views.wide.routeviews.org
+ peer_as: 15169
+ peer_address: 202.249.2.189
+ description: DIX-IE
+ afi_safis:
+ - ipv4_unicast
+ peer_as: 15169
+ peer_address: 2001:200:0:fe00::3b41:0
+ description: DIX-IE
+ afi_safis:
+ - ipv6_unicast
### Example: Multihop Peering - Peer with a Remote Autonomous System (AS)
If an AS is wanting to connect but we are not at the same exchange, we can provide a Multihop BGP session.
The `--multihop-index` option is used to select which collector to use.
$ routeviews-peer-request \
--inventory <WORKING_TREE>/ansible/inventory \
--ip 1.2.3.4 \
--ip 1.2.3.5 \
--asn 15169 \
--multihop-index 5 # I.e. route-views5.routeviews.org
### Changes
+++ <WORKING_TREE>/ansible/inventory/host_vars/route-views5.routeviews.org
+ peer_address: 1.2.3.5
+ peer_as: 15169
+ description: Google LLC
+ afi_safis:
+ - ipv4_unicast
+ options:
+ - ebgp-multihop 255
## `routeviews-email-peers` CLI tool
This tool will get a list of email addresses for any networks that are actively peered with a particular Route Views Collector.
This tool is for gathering email address information about Route Views Collector's peers around the world, leveraging [PeeringDB]() and [RDAP](TODO).
> Future Plan: Use SMTP server to automate actually sending *many* types of 'standard Route Views Operations emails' (use Jinja2 Templates for the email templates).
### Prerequisites
1. *SSH Access*: This script uses NetMiko, and assumes that the current user can SSH into the collector using SSH keys (recommend using an `ssh-agent`).
### Example
Run the `routeviews-email-peers` command against a specific Route Views collector, e.g. "route-views4.routeviews.org".
Today, this command will to produce a semicolon-separated list of email addresses for each (established) peering session on that collector.
$ routeviews-email-peers --collector route-views4.routeviews.org
WARNING:routeviews.scripts.get_peers_email:PeeringDB is missing ASN: 56665
2022-12-09 18:18:29 WARNING PeeringDB is missing ASN: 61138 [.../routeviews/scripts/get_peers_email.py:92]
... trimmed for brevity...
WARNING:routeviews.scripts.get_peers_email:PeeringDB is missing ASN: 61138
2022-12-09 18:18:29 WARNING PeeringDB is missing ASN: 204028 [.../routeviews/scripts/get_peers_email.py:92]
WARNING:routeviews.scripts.get_peers_email:PeeringDB is missing ASN: 204028
support@arelion.com; noc@level3carrier.com; inoc@vtc.vn; peering@uvm.edu;
## YAML Python API
We have a custom YAML module for handling (Ansible) YAML config files.
In particular, this module will handle whitespace matching the standard way used throughout the Route Views Infrastructure repo.
Further, this module ensures that the order data dumped is the same as ingested.
> Today, this functionality comes thanks to the [ruamel.yaml package (PyPI)](https://pypi.org/project/ruamel.yaml/)!
### Example
This example loads a file by filename, then saves that file back.
> In this case, this will essentially create a copy the "vars.yml" file.
>
> ℹ Tip: The "vars2.yml" copy, or any file dumped using `routeviews.yaml`, will follow the Route Views YAML styling convention.
import routeviews.yaml
my_variables = routeviews.yaml.load('vars.yml')
# ... make updates to `my_variables`...
routeviews.yaml.dump(my_variables, 'vars2.yml')
## Additional APIs
Besides the CLI tools discussed above, this package contains many internal packages/modules that might be useful.
> ⚠ NOTICE: Major version zero (0.y.z) is for initial development. Anything MAY change at any time. This public API SHOULD NOT be considered stable.
* There is the `routeviews.peeringdb` package that has some great methods for interfacing with the PeeringDB API.
* There is the `routeviews.yaml` module that can load and save YAML config files (without rearranging them).
* Depends on the [`ruamel.yaml` package](https://pypi.org/project/ruamel.yaml/)
* There is the `routeviews.ansible` package, that can load, modify, and save the Route Views Ansible Inventory.
* There is the `routeviews.bgpsummery` module, that defines a `BGPSummary` class as well as functions for retrieving a `BGPSummary` from any collector.
* There is the (start of a) `routeviews.api` module/package, for interfacing with the Route Views API/DB (undocumented).
# Release Notes
This project follows [Semantic Versioning](https://semver.org/).
> Notice: Major version "Zero" (0.y.z) is for initial development. Today, anything MAY change with each minor release.
## 0.3.6
- Refactor `rvm-bgp-status` to make InfluxDB Tags more useful.
- `state` is now a Tag rather than a Field.
- Why? We would like to "GROUP BY" state in our InfluxDB queries!
## 0.3.5
- Upgrade `rvm-bgp-status` to add "VTY Latency" to the `bgp_status` InfluxDB measurement.
- `vty_latency_sec` field has been added when running `rvm-bgp-status --influxdb`
- Nice to get an idea of FRRouting's performance over time!
- Refactor `rvm-bgp-status` to make InfluxDB Tags more useful.
- `state` is now a Tag rather than a Field.
- Why? We would like to "GROUP BY" state in our InfluxDB queries!
- Remove redundant "collector" tag from InfluxDB measurements.
- InfluxDB automatically tags data with the "host" tag. So, the "collector" tag was redundant.
## 0.3.4
- Fix `rvm-latest-mrt` to always get the *LATEST* MRT files.
- For some reason, sorting MRT archives by 'latest change timestamp (ctime)' seems to be non-deterministic! As a result, this tool was prioritizing MRT files from as old as 2019 for some collectors!
- Solution: Sort alphabetically instead of using ctime.
Route Views' MRT Archives use a consistent "YYYY-MM-DD" naming scheme which works perfectly when sorted alphabetically!
## 0.3.3
- Fix `routeviews-peer-requests` to use consistent vertical whitespace.
## 0.3.2
* Upgrade `routeviews-peer-requests` to print the, "effected Collector's(es) IP Addresses" after updating the Ansible inventory.
* For Maintainers to copy or reference when completing peer requests.
## 0.3.1
* Fix `routeviews-peer-requests` to ignore 'non-operational' Routers/Collectors.
* Some Route Views collectors are non operational today.
## 0.3.0
> **⚠ NOTE:** Renamed `routeviews-build-peer` CLI Tool to `routeviews-peer-request`.
> (Updated throughout this project's documentation)
* Upgrade `routeviews-peer-requests` with full feature set! 🎉
- Add `--show-options` flag that can be used by ANYONE to check their potential peerings (at Internet Exchanges) with Route Views.
- Add `--multihop-index` argument, to create BGP multihop peering config on Route Views' Multihop Collectors.
* `rvm-haproxy-stats` will fallback to `nc` if `socat` unavailable.
## 0.2.6
* Fix `rvm-haproxy-stats` CLI tool.
* InfluxDB line protocol was broken.
* Fixed a typo in the code that printed the InfluxDB line protocol.
## 0.2.5
* Add `rvm-haproxy-stats` CLI tool.
* Get stats from HAProxy Stick Tables on a Route Views collector.
## 0.2.4
* Add `--zipped` flag to `rvm-latest-mrt`.
* Only report files that have the ".bz2" file extension.
* *Why?* Ubuntu seems to continually update the MRT update file.
This had made the 'age_sec' metric in InfluxDB pretty much useless.
## 0.2.3
* Update `rvm-latest-mrt` InfluxDB line protocol to be simpler.
* Updates and RIBs are separate concerns, so send up separate measurements instead of combining them into one line.
## 0.2.2
* Create a 'GitHub Release' after delivering package to PyPI.org
## 0.2.1
* Add many InfluxDB tags to `rvm-latest-mrt`, and remove 2 fields (that were turned to tags).
* Using tags enables more useful and efficient querying in Grafana!
## 0.2.0
* Add a set of `rvm` (Route Views Monitor) CLI tools.
> **ℹ Tip:** The `rvm` tools listed below can run on any FRR-based Route Views collector.
* `rvm-latest-mrt`: Get metrics about the latest MRT Dump files on a Route Views collector.
* `rvm-bgp-status`: Get info about BGP Peerings on a Route Views collector.
* `rvm-bmp-status`: Get info about BMP sessions on a Route Views collector.
* Add `--sudo` flag to CLI tools where appropriate.
* CLI tools that depend on `vtysh` will only use raise privileges when running `vtysh`.
* Extract 'InfluxDB Line Protocol' logic into `routeviews.influx` module.
* Generate InfluxDB Line Protocol -- useful when using CLI tools as [Telegraf Exec Input Plugins](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/exec).
* Extract 'TextFSM Template Parsing' logic into the `routeviews.parse.template_parse` function.
* See additional [discussion of TextFSM in our Design Docs](./design.md#textfsm-conventions)
## 0.1.3
* Fix Bug: `routeviews-peer-request` CLI tool rearranges the 'Route Views Peer Config' in the Ansible Inventory.
* Now we track the 'order' of attributes whenever loading any `routeviews.ansible.NeighborConfig` class from a YAML file.
That 'order' is then used when subsequently dumping the data, thus ensuring that nothing is rearranged unnecessarily!
## 0.1.2
* Bug: `routeviews-peer-request` CLI tool rearranges the 'Route Views Peer Config' in the Ansible Inventory.
* Fix PeeringDB Authentication!
* See the [relevant GitHub Issue](https://github.com/peeringdb/peeringdb/issues/1206#issuecomment-1202550667) where we discovered the following details about PeeringDB API Basic Authentication:
> 1. Do NOT base64 encode
> 2. Username/Password Must be space-separated (e.g., must not be colon ":" separated)
> 3. Username when using API tokens is "Api-Key"
> 4. Ensure "www" is in all API requests!
* Enable using PeeringDB API Key instead of username/password.
* Exposed via `--peeringdb-key` argument in `routeviews-peer-request` CLI tool (or as env var: `PEERINGDB_KEY`).
* Add the filepath to the exception message when `routeviews.yaml` encounters a `ParseError`.
* This enables fixing syntax issues very quickly.
* "Unable to parse `<filepath>`" is the added message, seen below:
```
... omitted traceback for brevity...
routeviews.yaml.ParseError: while parsing a block mapping
in "<unicode string>", line 1, column 1:
short_name: decix
^ (line: 1)
expected <block end>, but found '-'
in "<unicode string>", line 109, column 1:
- peer_as: 8888
^ (line: 109)
Unable to parse <working-tree>/ansible/inventory/host_vars/route-views.decix.routeviews.org
```
* Ensure that PyVCR cassettes do not contain HTTP Basic Authentication secrets.
* Rotated the (randomly generated) Base64 encoded password that was previously exposed via HTTP Basic Authentication Headers.
## 0.1.1
* Fix Bug: Package failed to declare some critical dependencies.
## 0.1.0
> Bug: Package failed to declare some critical dependencies.
> Was missing `uologging` and `raumel.yaml` dependencies deceleration in "setup.py".
The first release of the routeviews package contains some core CLI tools, as well as some functions/classes that might be useful to routeviews maintainers.
### CLI Tools
Provide new CLI tools! 🎉
* [`routeviews-peer-request` CLI tool](./user-guide.md#routeviews-peer-request-cli-tool): automation of updating ["Route Views Ansible inventory"](https://github.com/routeviews/infra), toward 'adding BGP peers to XYZ collectors'.
* [`routeviews-email-peers` CLI tool](./user-guide.md#routeviews-email-peers-cli-tool): get list of email addresses actively peered with a Route Views Collector.
### Libraries
* There is the `routeviews.peeringdb` package that has some great methods for interfacing with the PeeringDB API.
* There is the `routeviews.yaml` module that can load and save YAML config files (without rearranging them).
* Depends on the [`ruamel.yaml` package](https://pypi.org/project/ruamel.yaml/)
* There is the `routeviews.ansible` package, that can load, modify, and save the Route Views Ansible Inventory.
* There is the `routeviews.bgpsummery` module, that defines a `BGPSummary` class as well as functions for retrieving a `BGPSummary` from any collector.
* There is the (start of a) `routeviews.api` module/package, for interfacing with the Route Views API/DB (undocumented).
Raw data
{
"_id": null,
"home_page": "https://github.com/routeviews/routeviews-cli",
"name": "routeviews",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "RouteViews,CLI,peeringdb,API,Integration",
"author": "University of Oregon",
"author_email": "rleonar7@uoregon.edu",
"download_url": "https://files.pythonhosted.org/packages/57/f2/e060dc87581d41ed51705b36023ba7ee097f446bfd6d8de3fcf9dc909e1c/routeviews-0.3.6.tar.gz",
"platform": null,
"description": "Route Views requires many tools to support its functionality.\nThis package is a place for those tools to live.\n\n> Today, this package also acts as the primary 'python programming library' for Route Views.\n\n# CLI Tools\n\nAs of today, there are two types of tools provided by this package: monitoring, and automation.\n\n> \u2139 The CLI tools have certain prefix based on tool type:\n>\n> * `rvm-` is for Monitoring tools that are used only to learn about the running state of Route Views.\n> * `routeviews-` is for Automation tools that enable some automated workflow.\n\n\n## `rvm-latest-mrt` CLI tool\n\nMonitoring tool that shows information about the latest Route Views MRT files (RIB, UPDATE) on a collector.\n\n> Use the `--help` flag to learn more about this tool's abilities.\n\n $ rvm-latest-mrt\n Latest RIB: /mnt/storage/bgpdata/2022.08/RIBS/rib.20220802.0800.bz2\n Latest UPDATE: /mnt/storage/bgpdata/2022.08/UPDATES/update.20220802.0800.bz2\n\n## `rvm-bmp-status` CLI tool\n\nMonitoring tool that shows information about BMP connections on a collector.\n\n> Use the `--help` flag to learn more about this tool's abilities.\n\n $ sudo rvm-bmp-status\n BMP Collector: bmp.routeviews.org\n Connection Uptime: 8 hours\n Data sent: 1.5 GB\n Bytes queued: 2 Bytes\n Bytes queued to Kernel: 3 Bytes\n\n## `rvm-bgp-status` CLI tool\n\nMonitoring tool that shows relevant information about BGP connections on a collector.\n\n> Use the `--help` flag to learn more about this tool's abilities.\n\n $ sudo rvm-bgp-status\n ASN Peer Address State Prefixes Uptime InQ Uptime\n ----- ------------------ ----------- ---------- -------- ------ --------\n 65129 128.223.51.78 ESTABLISHED 0 0 149732 1\n 3582 128.223.253.9 ESTABLISHED 899115 0 149732 1\n 3582 128.223.253.10 ESTABLISHED 899147 0 149732 1\n 3582 2001:468:d01:fd::9 ESTABLISHED 162442 0 149732 1\n 3582 2001:468:d01:fd::a ESTABLISHED 162443 0 149732 1\n\n## `rvm-haproxy-stats` CLI tool\n\nGet stats from HAProxy Stick Tables on a Route Views collector. \n\n> \u2139 HAProxy runs on our collectors to enable telnet access.\n\n $ rvm-haproxy-stats --min-conn-cnt 10 --sudo\n Key Current Conn. Total Conn. Data In Rate Date Out Rate\n ------------------------ --------------- ------------- -------------- ---------------\n 175.30.79.245 0 10 0 Bytes 901 Bytes\n 141.255.166.2 0 12 69 Bytes 1.3 kB\n 179.43.187.243 0 13 0 Bytes 1.5 kB\n 180.103.51.200 0 14 0 Bytes 1.0 kB\n 31.220.3.140 0 27 0 Bytes 907 Bytes\n 2001:468:d01:33::80df:78 0 27 457 Bytes 9.0 kB\n\n## `routeviews-peer-request` CLI tool\n\nThis tool is for (consistently) updating the [Route Views ansible inventory (private repo)](https://github.com/routeviews/infra) when folks submit new peer requests. \n\n> This tool uses information provided by PeeringDB for the peering information.\n\n### Prerequisites\n\n1. *Route Views Ansible Inventory*: You must have a local copy of the Route Views ansible inventory available, for this tool to update.\n * If you will be running this command with any regularity, is useful to export the `ROUTEVIEWS_INVENTORY` environment variable to point to your local copy of the [Route Views ansible inventory repository (private)](https://github.com/routeviews/infra).\n ```\n # (Optional) Place in your ~/.bashrc\n $ export ROUTEVIEWS_INVENTORY='<WORKING_TREE>/ansible/inventory'\n ```\n * `<WORKING_TREE>` refers to wherever you've cloned the repository on your filesystem.\n\n### Example: Show Options between Route Views and Autonomous System (AS)\n\nIf an AS would like to know what peering sessions are possible according to this tool, we do have a `--show-options` flag that will enable this!\nProvide only the `asn` argument along with the `show-options` flag to try this out!\n\n> \u2139 Anyone can run this solution!\n> This solution only depends on the public PeeringDB API.\n\n $ routeviews-peer-request --asn 15169 --show-options\n\n Potential BGP Peerings for networks:\n\n - RouteViews (ASN: 6447), and \n - Google LLC (ASN: 15169).\n\n Exchange RV Collector Router\n ---------------------------------------------- ------------------------ ------------------------\n Equinix Chicago 208.115.136.187 208.115.136.21\n Equinix Chicago 2001:504:0:4::6447:1 2001:504:0:4:0:1:5169:1\n Equinix Palo Alto 198.32.176.5 198.32.176.31\n Equinix Palo Alto 2001:504:d::5 2001:504:d::1f\n LINX LON1: Main 195.66.225.222 195.66.224.125\n LINX LON1: Main 2001:7f8:4::192f:1 2001:7f8:4::3b41:1\n Digital Realty Atlanta 198.32.132.3 198.32.132.41\n Digital Realty Atlanta 2001:478:132::3 2001:478:132::41\n DE-CIX Frankfurt: DE-CIX Frankfurt Peering LAN 80.81.193.49 80.81.192.108\n DE-CIX Frankfurt: DE-CIX Frankfurt Peering LAN 2001:7f8::192f:0:1 2001:7f8::3b41:0:1\n DE-CIX Frankfurt: DE-CIX Frankfurt Peering LAN 80.81.193.49 80.81.193.108\n DE-CIX Frankfurt: DE-CIX Frankfurt Peering LAN 2001:7f8::192f:0:1 2001:7f8::3b41:0:2\n DIX-IE 202.249.2.166 202.249.2.189\n ... trimmed for brevity...\n\n### Example: Peer with an Autonomous System (AS) at ***ALL*** IXes\n\nIf an AS is wanting to connect wherever possible, provide only the `asn` argument and the tool will determine all the possible `ip` arguments from PeeringDB.\n\n $ routeviews-peer-request \\\n --inventory <WORKING_TREE>/ansible/inventory \\\n --asn 15169\n \n ### Changes\n\n +++ <WORKING_TREE>/ansible/inventory/host_vars/route-views.perth.routeviews.org\n + peer_as: 15169\n + peer_address: 218.100.52.3\n + description: 'IX Australia (Sydney NSW): NSW-IX'\n + afi_safis:\n + - ipv4_unicast\n + peer_as: 15169\n + peer_address: 2001:7fa:11:4:0:3b41:0:1\n + description: 'IX Australia (Sydney NSW): NSW-IX'\n + afi_safis:\n + - ipv6_unicast\n + peer_as: 15169\n + peer_address: 218.100.53.29\n + description: 'IX Australia (Sydney NSW): NSW-IX'\n + afi_safis:\n + - ipv4_unicast\n + peer_as: 15169\n + peer_address: 2001:7fa:11:4:0:3b41:0:2\n + description: 'IX Australia (Sydney NSW): NSW-IX'\n + afi_safis:\n + - ipv6_unicast\n + peer_as: 15169\n + peer_address: 218.100.78.154\n + description: 'IX Australia (Melbourne VIC): VIC-IX'\n + afi_safis:\n + - ipv4_unicast\n + peer_as: 15169\n + peer_address: 2001:7fa:11:1:0:3b41:0:2\n + description: 'IX Australia (Melbourne VIC): VIC-IX'\n + afi_safis:\n + - ipv6_unicast\n + peer_as: 15169\n + peer_address: 218.100.78.153\n + description: 'IX Australia (Melbourne VIC): VIC-IX'\n + afi_safis:\n + - ipv4_unicast\n + peer_as: 15169\n + peer_address: 2001:7fa:11:1:0:3b41:0:1\n + description: 'IX Australia (Melbourne VIC): VIC-IX'\n + afi_safis:\n + - ipv6_unicast\n\n +++ <WORKING_TREE>/ansible/inventory/host_vars/route-views.amsix.routeviews.org\n + peer_as: 15169\n + peer_address: 80.249.208.247\n + description: AMS-IX\n + afi_safis:\n + - ipv4_unicast\n + peer_as: 15169\n + peer_address: 2001:7f8:1::a501:5169:1\n + description: AMS-IX\n + afi_safis:\n ... trimmed for brevity...\n\n### Example: Peer with an Autonomous System (AS) using IP Addresses\n\nThis tool supports arguments for `asn`, as well as `ip`.\nThe `ip` argument can be used multiple times to peer with multiple IP Addresses at once.\n\n> \u26a0 Only supports peering with one AS at a time.\n\n> \u2139 Use the `--help` flag to learn more about how to use these arguments.\n\nAs discussed in the [prerequisites](#prerequisites-1), there is also the `inventory` argument required that points to the \"inventory/\" directory.\n\n> \u2139 Tip: Provide an `asn` and omit the `ip` argument entirely -- the tool will attempt to peer with ALL compatible IP Addresses for the provided `asn`!\n\n $ routeviews-peer-request \\\n --inventory <WORKING_TREE>/ansible/inventory \\\n --asn 15169 \\\n --ip 202.249.2.189 \\\n --ip 2001:200:0:fe00::3b41:0 \\\n --ip 80.249.208.247 \\\n --ip 2001:7f8:1::a501:5169:1\n\n ### Changes\n\n +++ <WORKING_TREE>/ansible/inventory/host_vars/route-views.amsix.routeviews.org\n + peer_as: 15169\n + peer_address: 80.249.208.247\n + description: AMS-IX\n + afi_safis:\n + - ipv4_unicast\n + peer_as: 15169\n + peer_address: 2001:7f8:1::a501:5169:1\n + description: AMS-IX\n + afi_safis:\n + - ipv6_unicast\n\n +++ <WORKING_TREE>/ansible/inventory/host_vars/route-views.wide.routeviews.org\n + peer_as: 15169\n + peer_address: 202.249.2.189\n + description: DIX-IE\n + afi_safis:\n + - ipv4_unicast\n + peer_as: 15169\n + peer_address: 2001:200:0:fe00::3b41:0\n + description: DIX-IE\n + afi_safis:\n + - ipv6_unicast\n\n\n### Example: Multihop Peering - Peer with a Remote Autonomous System (AS)\n\nIf an AS is wanting to connect but we are not at the same exchange, we can provide a Multihop BGP session.\nThe `--multihop-index` option is used to select which collector to use.\n\n $ routeviews-peer-request \\\n --inventory <WORKING_TREE>/ansible/inventory \\\n --ip 1.2.3.4 \\\n --ip 1.2.3.5 \\\n --asn 15169 \\\n --multihop-index 5 # I.e. route-views5.routeviews.org\n\n ### Changes\n\n +++ <WORKING_TREE>/ansible/inventory/host_vars/route-views5.routeviews.org\n + peer_address: 1.2.3.5\n + peer_as: 15169\n + description: Google LLC\n + afi_safis:\n + - ipv4_unicast\n + options:\n + - ebgp-multihop 255\n\n## `routeviews-email-peers` CLI tool\n\nThis tool will get a list of email addresses for any networks that are actively peered with a particular Route Views Collector.\nThis tool is for gathering email address information about Route Views Collector's peers around the world, leveraging [PeeringDB]() and [RDAP](TODO).\n\n> Future Plan: Use SMTP server to automate actually sending *many* types of 'standard Route Views Operations emails' (use Jinja2 Templates for the email templates).\n\n### Prerequisites\n\n1. *SSH Access*: This script uses NetMiko, and assumes that the current user can SSH into the collector using SSH keys (recommend using an `ssh-agent`).\n\n### Example\n\nRun the `routeviews-email-peers` command against a specific Route Views collector, e.g. \"route-views4.routeviews.org\".\n\nToday, this command will to produce a semicolon-separated list of email addresses for each (established) peering session on that collector.\n\n $ routeviews-email-peers --collector route-views4.routeviews.org\n WARNING:routeviews.scripts.get_peers_email:PeeringDB is missing ASN: 56665\n 2022-12-09 18:18:29 WARNING PeeringDB is missing ASN: 61138 [.../routeviews/scripts/get_peers_email.py:92]\n ... trimmed for brevity...\n WARNING:routeviews.scripts.get_peers_email:PeeringDB is missing ASN: 61138\n 2022-12-09 18:18:29 WARNING PeeringDB is missing ASN: 204028 [.../routeviews/scripts/get_peers_email.py:92]\n WARNING:routeviews.scripts.get_peers_email:PeeringDB is missing ASN: 204028\n support@arelion.com; noc@level3carrier.com; inoc@vtc.vn; peering@uvm.edu;\n\n## YAML Python API\n\nWe have a custom YAML module for handling (Ansible) YAML config files.\nIn particular, this module will handle whitespace matching the standard way used throughout the Route Views Infrastructure repo.\nFurther, this module ensures that the order data dumped is the same as ingested.\n\n> Today, this functionality comes thanks to the [ruamel.yaml package (PyPI)](https://pypi.org/project/ruamel.yaml/)!\n\n### Example\n\nThis example loads a file by filename, then saves that file back.\n\n> In this case, this will essentially create a copy the \"vars.yml\" file.\n>\n> \u2139 Tip: The \"vars2.yml\" copy, or any file dumped using `routeviews.yaml`, will follow the Route Views YAML styling convention.\n\n import routeviews.yaml\n\n my_variables = routeviews.yaml.load('vars.yml')\n\n # ... make updates to `my_variables`...\n\n routeviews.yaml.dump(my_variables, 'vars2.yml')\n\n## Additional APIs\n\nBesides the CLI tools discussed above, this package contains many internal packages/modules that might be useful.\n\n> \u26a0 NOTICE: Major version zero (0.y.z) is for initial development. Anything MAY change at any time. This public API SHOULD NOT be considered stable.\n\n* There is the `routeviews.peeringdb` package that has some great methods for interfacing with the PeeringDB API.\n* There is the `routeviews.yaml` module that can load and save YAML config files (without rearranging them).\n * Depends on the [`ruamel.yaml` package](https://pypi.org/project/ruamel.yaml/)\n* There is the `routeviews.ansible` package, that can load, modify, and save the Route Views Ansible Inventory.\n* There is the `routeviews.bgpsummery` module, that defines a `BGPSummary` class as well as functions for retrieving a `BGPSummary` from any collector.\n* There is the (start of a) `routeviews.api` module/package, for interfacing with the Route Views API/DB (undocumented).\n\n\n\n\n\n\n\n\n# Release Notes\n\nThis project follows [Semantic Versioning](https://semver.org/).\n\n> Notice: Major version \"Zero\" (0.y.z) is for initial development. Today, anything MAY change with each minor release.\n\n## 0.3.6\n\n- Refactor `rvm-bgp-status` to make InfluxDB Tags more useful.\n - `state` is now a Tag rather than a Field.\n - Why? We would like to \"GROUP BY\" state in our InfluxDB queries!\n\n## 0.3.5\n\n- Upgrade `rvm-bgp-status` to add \"VTY Latency\" to the `bgp_status` InfluxDB measurement.\n - `vty_latency_sec` field has been added when running `rvm-bgp-status --influxdb`\n - Nice to get an idea of FRRouting's performance over time!\n- Refactor `rvm-bgp-status` to make InfluxDB Tags more useful.\n - `state` is now a Tag rather than a Field.\n - Why? We would like to \"GROUP BY\" state in our InfluxDB queries!\n- Remove redundant \"collector\" tag from InfluxDB measurements.\n - InfluxDB automatically tags data with the \"host\" tag. So, the \"collector\" tag was redundant.\n\n## 0.3.4\n\n- Fix `rvm-latest-mrt` to always get the *LATEST* MRT files.\n - For some reason, sorting MRT archives by 'latest change timestamp (ctime)' seems to be non-deterministic! As a result, this tool was prioritizing MRT files from as old as 2019 for some collectors!\n - Solution: Sort alphabetically instead of using ctime. \n Route Views' MRT Archives use a consistent \"YYYY-MM-DD\" naming scheme which works perfectly when sorted alphabetically!\n\n## 0.3.3\n\n- Fix `routeviews-peer-requests` to use consistent vertical whitespace.\n\n\n## 0.3.2\n\n* Upgrade `routeviews-peer-requests` to print the, \"effected Collector's(es) IP Addresses\" after updating the Ansible inventory.\n * For Maintainers to copy or reference when completing peer requests.\n\n## 0.3.1\n\n* Fix `routeviews-peer-requests` to ignore 'non-operational' Routers/Collectors.\n * Some Route Views collectors are non operational today.\n\n## 0.3.0\n\n> **\u26a0 NOTE:** Renamed `routeviews-build-peer` CLI Tool to `routeviews-peer-request`.\n> (Updated throughout this project's documentation)\n\n* Upgrade `routeviews-peer-requests` with full feature set! \ud83c\udf89\n - Add `--show-options` flag that can be used by ANYONE to check their potential peerings (at Internet Exchanges) with Route Views.\n - Add `--multihop-index` argument, to create BGP multihop peering config on Route Views' Multihop Collectors.\n* `rvm-haproxy-stats` will fallback to `nc` if `socat` unavailable.\n\n\n## 0.2.6\n\n* Fix `rvm-haproxy-stats` CLI tool.\n * InfluxDB line protocol was broken.\n * Fixed a typo in the code that printed the InfluxDB line protocol.\n\n## 0.2.5\n\n* Add `rvm-haproxy-stats` CLI tool.\n * Get stats from HAProxy Stick Tables on a Route Views collector.\n\n## 0.2.4\n\n* Add `--zipped` flag to `rvm-latest-mrt`.\n * Only report files that have the \".bz2\" file extension.\n * *Why?* Ubuntu seems to continually update the MRT update file.\n This had made the 'age_sec' metric in InfluxDB pretty much useless.\n\n## 0.2.3\n\n* Update `rvm-latest-mrt` InfluxDB line protocol to be simpler.\n * Updates and RIBs are separate concerns, so send up separate measurements instead of combining them into one line.\n \n\n## 0.2.2\n\n* Create a 'GitHub Release' after delivering package to PyPI.org\n\n## 0.2.1\n\n* Add many InfluxDB tags to `rvm-latest-mrt`, and remove 2 fields (that were turned to tags).\n * Using tags enables more useful and efficient querying in Grafana!\n\n## 0.2.0\n\n* Add a set of `rvm` (Route Views Monitor) CLI tools.\n > **\u2139 Tip:** The `rvm` tools listed below can run on any FRR-based Route Views collector.\n * `rvm-latest-mrt`: Get metrics about the latest MRT Dump files on a Route Views collector.\n * `rvm-bgp-status`: Get info about BGP Peerings on a Route Views collector.\n * `rvm-bmp-status`: Get info about BMP sessions on a Route Views collector.\n* Add `--sudo` flag to CLI tools where appropriate.\n * CLI tools that depend on `vtysh` will only use raise privileges when running `vtysh`.\n* Extract 'InfluxDB Line Protocol' logic into `routeviews.influx` module.\n * Generate InfluxDB Line Protocol -- useful when using CLI tools as [Telegraf Exec Input Plugins](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/exec).\n* Extract 'TextFSM Template Parsing' logic into the `routeviews.parse.template_parse` function.\n * See additional [discussion of TextFSM in our Design Docs](./design.md#textfsm-conventions)\n\n## 0.1.3\n\n* Fix Bug: `routeviews-peer-request` CLI tool rearranges the 'Route Views Peer Config' in the Ansible Inventory.\n * Now we track the 'order' of attributes whenever loading any `routeviews.ansible.NeighborConfig` class from a YAML file.\n That 'order' is then used when subsequently dumping the data, thus ensuring that nothing is rearranged unnecessarily!\n\n## 0.1.2\n\n* Bug: `routeviews-peer-request` CLI tool rearranges the 'Route Views Peer Config' in the Ansible Inventory.\n\n* Fix PeeringDB Authentication!\n * See the [relevant GitHub Issue](https://github.com/peeringdb/peeringdb/issues/1206#issuecomment-1202550667) where we discovered the following details about PeeringDB API Basic Authentication:\n > 1. Do NOT base64 encode\n > 2. Username/Password Must be space-separated (e.g., must not be colon \":\" separated)\n > 3. Username when using API tokens is \"Api-Key\"\n > 4. Ensure \"www\" is in all API requests!\n* Enable using PeeringDB API Key instead of username/password.\n * Exposed via `--peeringdb-key` argument in `routeviews-peer-request` CLI tool (or as env var: `PEERINGDB_KEY`).\n* Add the filepath to the exception message when `routeviews.yaml` encounters a `ParseError`.\n * This enables fixing syntax issues very quickly.\n * \"Unable to parse `<filepath>`\" is the added message, seen below:\n ```\n ... omitted traceback for brevity...\n routeviews.yaml.ParseError: while parsing a block mapping\n in \"<unicode string>\", line 1, column 1:\n short_name: decix\n ^ (line: 1)\n expected <block end>, but found '-'\n in \"<unicode string>\", line 109, column 1:\n - peer_as: 8888\n ^ (line: 109)\n Unable to parse <working-tree>/ansible/inventory/host_vars/route-views.decix.routeviews.org\n ```\n* Ensure that PyVCR cassettes do not contain HTTP Basic Authentication secrets.\n * Rotated the (randomly generated) Base64 encoded password that was previously exposed via HTTP Basic Authentication Headers. \n\n## 0.1.1\n\n* Fix Bug: Package failed to declare some critical dependencies. \n\n## 0.1.0\n\n> Bug: Package failed to declare some critical dependencies. \n> Was missing `uologging` and `raumel.yaml` dependencies deceleration in \"setup.py\".\n\nThe first release of the routeviews package contains some core CLI tools, as well as some functions/classes that might be useful to routeviews maintainers.\n\n### CLI Tools\n\nProvide new CLI tools! \ud83c\udf89\n\n* [`routeviews-peer-request` CLI tool](./user-guide.md#routeviews-peer-request-cli-tool): automation of updating [\"Route Views Ansible inventory\"](https://github.com/routeviews/infra), toward 'adding BGP peers to XYZ collectors'.\n* [`routeviews-email-peers` CLI tool](./user-guide.md#routeviews-email-peers-cli-tool): get list of email addresses actively peered with a Route Views Collector.\n\n### Libraries\n\n* There is the `routeviews.peeringdb` package that has some great methods for interfacing with the PeeringDB API.\n* There is the `routeviews.yaml` module that can load and save YAML config files (without rearranging them).\n * Depends on the [`ruamel.yaml` package](https://pypi.org/project/ruamel.yaml/)\n* There is the `routeviews.ansible` package, that can load, modify, and save the Route Views Ansible Inventory.\n* There is the `routeviews.bgpsummery` module, that defines a `BGPSummary` class as well as functions for retrieving a `BGPSummary` from any collector.\n* There is the (start of a) `routeviews.api` module/package, for interfacing with the Route Views API/DB (undocumented).\n\n\n\n\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "CLI tools that support RouteViews.",
"version": "0.3.6",
"split_keywords": [
"routeviews",
"cli",
"peeringdb",
"api",
"integration"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "57f2e060dc87581d41ed51705b36023ba7ee097f446bfd6d8de3fcf9dc909e1c",
"md5": "eb0b4824880fe3166ba5ba04ea6026b1",
"sha256": "c4ff1ca0f8e82684b792c9639bd2dee62fe40b08bbb3908237d4d71d6b7dbc6c"
},
"downloads": -1,
"filename": "routeviews-0.3.6.tar.gz",
"has_sig": false,
"md5_digest": "eb0b4824880fe3166ba5ba04ea6026b1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 56919,
"upload_time": "2023-01-02T19:15:00",
"upload_time_iso_8601": "2023-01-02T19:15:00.316383Z",
"url": "https://files.pythonhosted.org/packages/57/f2/e060dc87581d41ed51705b36023ba7ee097f446bfd6d8de3fcf9dc909e1c/routeviews-0.3.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-01-02 19:15:00",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "routeviews",
"github_project": "routeviews-cli",
"lcname": "routeviews"
}
JSON
