The IETF Network

... an overview

2019-07 - Montreal .CA

V0.04

The Scout™

  • A Ubiquiti router
  • Shipped to site / installed during site visit
  • Starts announcing our address space
    • Allows testing of the circuits
    • Validation of the BGP peering, etc.
    • Provides an anchor for geo-location data
    • Gets the ietf-hotel SSID up for NOC

Circuits

  • At least 2, but up to 5 circuits
  • Almost always donated by local providers
    • Try for redundant:
      • providers
      • fiber
      • entrances
  • 1Gbps -> 10Gbps
  • Dual stack (IPv4 / IPv6)
  • BGP

Routers

  • 2 Juniper routers
    • Were MX80s, upgraded to MX204s (this meeting)
      • Convergence: ~25 minutes -> ~1.5 minutes
  • Core routers for network
    • BGP (eBGP, iBGP)
      • RPKI
    • OSPF / OSPFv3
    • DHCP relay / RA
    • BCP38
    • Passive ARP learning (more if we have time...)

ARP ARP ARP... ARP ARP...

ARP ARP ARP... ARP ARP...

    aggregate {
        inactive: route 130.129.0.0/16;
        route 31.133.128.0/18;
        route 31.130.224.0/20;
    }

RPKI

routing-options {
    validation {
        group rpki-servers {
            session 31.130.229.4 {  # Dragon Research Labs RPKI Toolkit
                preference 100;
                port 323;
            }
}
policy-statement RPKI {
        term whitelist {...}
        term invalid {
            from {
                protocol bgp;
                validation-database invalid;
            }
            then {
                validation-state invalid;
                community add RPKI_Invalid;
                reject;
            }
      
    policy-statement RPKI {
        term whitelist {
            from {
                protocol bgp;
                prefix-list RPKI_Whitelist;
            }
            then {
                validation-state valid;
                community add RPKI_Whitelist;
                next policy;
            }
        }
        term invalid {
            from {
                protocol bgp;
                validation-database invalid;
            }
            then {
                validation-state invalid;
                community add RPKI_Invalid;
                reject;
            }
        }
        term valid {
            from {
                protocol bgp;
                validation-database valid;
            }
            then {
                validation-state valid;
                community add RPKI_Valid;
                next policy;
            }
        }
      
  term unknown {
            from {
                protocol bgp;
                validation-database unknown;
            }
            then {
                validation-state unknown;
                community add RPKI_Unknown;
                next policy;
            }
        }
        /* This should not happen -- things should be valid, invalid or unknown */
        term failed {
            from protocol bgp;
            then {
                community add RPKI_Failure;
                next policy;
            }
        }
    }

Switches

  • 2 x Cisco Catalyst 4500X Core stacked
  • 10 x Cisco IDF switches
  • 40 x Cisco 12 port switches
  • "Joe's magic..."
    • Y'all keep plugging in DHCP servers :-(
    • A new switch to a fully provisioned switch in ~15 minutes (including a software upgrade).

    • Rooms are dynamic - this means we need to reconfigure things often and quickly

Switch Automation

  • Feature-wise, the switch automation includes:
    • Initialize new switch with desired config and software image
    • Validation of config and image (checksum)
    • Auto-generation of SSH host key
    • Call-home for when a switch should re-ZTP
    • Auto-detection of connected device type (switch, AP, probe)
    • Port auto-config and auto-doc update
    • Detection of lost device and port description update

Wireless

  • 2 x Cisco WLC 5520 in an HA pair
    • Cisco WLC 2504 for ISOC & testing
  • Somewhere between 50 and 70 Access Points
    • [TODO] 55 this time
    • We do both 5Ghz and 2.4Ghz, prefer 5Ghz
  • This has largely solved much of the ARP problem
    • Does your phone battery now last >3/4 day?
      • Thank Panda...!
  • Multiple encrypted SSIDs
    • "ietf-legacy, ietf, ietf-2.4only, ietf-nat64, ietf-v6only, ietf-nat64-unencrypted, eduroam, isoc, ..."

Guestroom / "hotel"

Guestroom Network

 

  • Guest networks are built for normal people
    • Captive portal
      • Intercept / rewrite DNS
      • HTTP munging...
    • NAT
    • Drop no-good, bad, dangerous ports (like 22!)
    • Assumptions:
      • Limited devices
      • Limited bandwidth
      • Limited sessions
  • IPv6? Ain't nobody got time for that...

IETF participants are "weird"...
... no, really weird...

 

From recent stay

wkumari$ git push
ssh: connect to host git.kumari.net port 22: Connection refused
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

wkumari$

Guestroom network

  • Bypass guestroom gateway with Ubiquiti routers, open SSID
    • "Free Internets for all!"
  • Some hotels have truly bizarre inventive architectures...
    • Really bad channelizing
    • Mac Mini in "Internet Sharing Mode"
    • Access Points on elevators... much hilarity...
  • Too few access points in guest rooms (getting better)
  • Ethernet over Coax / DOCSIS / DSL / Cat3 
  • Integrated PoS, TV, mini-bar, signs, thermostats, ...
  • DNS / DNSSEC, DPRIVE
  • DNS64
  • DHCP / DHCPv6
  • NTP
  • Tickets
  • RPKI server
  • TACACS+ / RADIUS
  • ZTP server
  • Etherpad
  • Ansible for automation (Yay! DevOps!)
  • SMTP

Servers / Services

  • Git repo
  • VMs for Meetecho
  • Backups
  • Syslog
  • Monitoring:
    • Prometheus
    • Deadman
    • Intermapper
    • Smokeping
    • Rancid
    • Netdisco
    • Observium, ...
  • 3+3 Physical servers
  • Ganeti, Docker

Scrubbing PII....

Remote Participation

  • Live streaming gets their own VLANs
  • ... and VMs
  •  ~60 Mbps BW from VMs to Internet
  • The network we build makes remote participation possible
  • Meetecho / Kaskadian have done events on venue networks
    • but only streaming (not remote participants)
    • Meetecho remote participation depends on "but the limited bandwidth, NATs, firewalls, lack of IPv6, would likely prevent us from providing good
      remote participation."
    • Kaskadian: "Hotel network won't work!.... :-P"

You deserve a kitten now...

Experiments...

DPRIVE

  • Ran one of the early DNS-over-TLS services
  • Now it is a "standard service"

V6ONLY - no, really.

  • Turned off IPv4 on all radios near V6OPS, 6MAN
    • Hilarity ensues... :-P

NAT64 Testing

MAC Randomization

Questions?

The IETF Network

By wkumari

The IETF Network

  • 223