The IETF Network
... an overview
2019-07 - Montreal .CA
V0.04
The Scout™
- A Ubiquiti router
- Shipped to site / installed during site visit
- Starts announcing our address space
- Allows testing of the circuits
- Validation of the BGP peering, etc.
- Provides an anchor for geo-location data
- Gets the ietf-hotel SSID up for NOC
Circuits
- At least 2, but up to 5 circuits
- Almost always donated by local providers
- Try for redundant:
- providers
- fiber
- entrances
- Try for redundant:
- 1Gbps -> 10Gbps
- Dual stack (IPv4 / IPv6)
- BGP
Routers
- 2 Juniper routers
- Were MX80s, upgraded to MX204s (this meeting)
- Convergence: ~25 minutes -> ~1.5 minutes
- Were MX80s, upgraded to MX204s (this meeting)
- Core routers for network
- BGP (eBGP, iBGP)
- RPKI
- OSPF / OSPFv3
- DHCP relay / RA
- BCP38
- Passive ARP learning (more if we have time...)
- BGP (eBGP, iBGP)
ARP ARP ARP... ARP ARP...
ARP ARP ARP... ARP ARP...
aggregate {
inactive: route 130.129.0.0/16;
route 31.133.128.0/18;
route 31.130.224.0/20;
}
RPKI
routing-options {
validation {
group rpki-servers {
session 31.130.229.4 { # Dragon Research Labs RPKI Toolkit
preference 100;
port 323;
}
}
policy-statement RPKI {
term whitelist {...}
term invalid {
from {
protocol bgp;
validation-database invalid;
}
then {
validation-state invalid;
community add RPKI_Invalid;
reject;
}
policy-statement RPKI {
term whitelist {
from {
protocol bgp;
prefix-list RPKI_Whitelist;
}
then {
validation-state valid;
community add RPKI_Whitelist;
next policy;
}
}
term invalid {
from {
protocol bgp;
validation-database invalid;
}
then {
validation-state invalid;
community add RPKI_Invalid;
reject;
}
}
term valid {
from {
protocol bgp;
validation-database valid;
}
then {
validation-state valid;
community add RPKI_Valid;
next policy;
}
}
term unknown {
from {
protocol bgp;
validation-database unknown;
}
then {
validation-state unknown;
community add RPKI_Unknown;
next policy;
}
}
/* This should not happen -- things should be valid, invalid or unknown */
term failed {
from protocol bgp;
then {
community add RPKI_Failure;
next policy;
}
}
}
Switches
- 2 x Cisco Catalyst 4500X Core stacked
- 10 x Cisco IDF switches
- 40 x Cisco 12 port switches
- "Joe's magic..."
- Y'all keep plugging in DHCP servers :-(
-
A new switch to a fully provisioned switch in ~15 minutes (including a software upgrade).
-
Rooms are dynamic - this means we need to reconfigure things often and quickly
Switch Automation
-
Feature-wise, the switch automation includes:
• Initialize new switch with desired config and software image
• Validation of config and image (checksum)
• Auto-generation of SSH host key
• Call-home for when a switch should re-ZTP
• Auto-detection of connected device type (switch, AP, probe)
• Port auto-config and auto-doc update
• Detection of lost device and port description update
Wireless
- 2 x Cisco WLC 5520 in an HA pair
- Cisco WLC 2504 for ISOC & testing
- Somewhere between 50 and 70 Access Points
- [TODO] 55 this time
- We do both 5Ghz and 2.4Ghz, prefer 5Ghz
- This has largely solved much of the ARP problem
- Does your phone battery now last >3/4 day?
- Thank Panda...!
- Does your phone battery now last >3/4 day?
- Multiple encrypted SSIDs
- "ietf-legacy, ietf, ietf-2.4only, ietf-nat64, ietf-v6only, ietf-nat64-unencrypted, eduroam, isoc, ..."
Guestroom / "hotel"
Guestroom Network
- Guest networks are built for normal people
- Captive portal
- Intercept / rewrite DNS
- HTTP munging...
- NAT
- Drop no-good, bad, dangerous ports (like 22!)
- Assumptions:
- Limited devices
- Limited bandwidth
- Limited sessions
- Captive portal
- IPv6? Ain't nobody got time for that...
IETF participants are "weird"...
... no, really weird...
From recent stay
wkumari$ git push ssh: connect to host git.kumari.net port 22: Connection refused fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. wkumari$
Guestroom network
- Bypass guestroom gateway with Ubiquiti routers, open SSID
- "Free Internets for all!"
- Some hotels have
truly bizarreinventive architectures...- Really bad channelizing
- Mac Mini in "Internet Sharing Mode"
- Access Points on elevators... much hilarity...
- Too few access points in guest rooms (getting better)
- Ethernet over Coax / DOCSIS / DSL / Cat3
- Integrated PoS, TV, mini-bar, signs, thermostats, ...
- DNS / DNSSEC, DPRIVE
- DNS64
- DHCP / DHCPv6
- NTP
- Tickets
- RPKI server
- TACACS+ / RADIUS
- ZTP server
- Etherpad
- Ansible for automation (Yay! DevOps!)
- SMTP
Servers / Services
- Git repo
- VMs for Meetecho
- Backups
- Syslog
-
Monitoring:
- Prometheus
- Deadman
- Intermapper
- Smokeping
- Rancid
- Netdisco
- Observium, ...
- 3+3 Physical servers
- Ganeti, Docker
Scrubbing PII....
Remote Participation
- Live streaming gets their own VLANs
- ... and VMs
- ~60 Mbps BW from VMs to Internet
- The network we build makes remote participation possible
- Meetecho / Kaskadian have done events on venue networks
- but only streaming (not remote participants)
- Meetecho remote participation depends on "but the limited bandwidth, NATs, firewalls, lack of IPv6, would likely prevent us from providing good
remote participation." - Kaskadian: "Hotel network won't work!.... :-P"
You deserve a kitten now...
Experiments...
DPRIVE
- Ran one of the early DNS-over-TLS services
- Now it is a "standard service"
V6ONLY - no, really.
- Turned off IPv4 on all radios near V6OPS, 6MAN
- Hilarity ensues... :-P
NAT64 Testing
MAC Randomization
Questions?
The IETF Network
By wkumari
The IETF Network
- 292