Cloud Networking @ UA

Who am I

Lead Infrastructure Engineer

Rocktavious

krockman@underarmour.com

Kyle Rockman

Agenda

Networking 101 (Foundation)
Networking 201 (Cloud)
Networking 301 (Kubernetes)

Trying to Understand

Principles

CIDR Ranges

DNS

Packets & Sockets
How VPN's Work

What are we going to learn about today!

CIDR Ranges

IPV4 - Internet Protocol Version 4
127.0.0.1

IPV6 - Internet Protocol Version 6
0000:0000:0000:0000:0000:0000:0000:0001

CIDR - Classless inter-domain routing

CIDR Ranges

Anatomy of a CIDR

172.16.0.0/32 = 172.16.0.0 (1)

172.16.0.0/24 = 172.16.0.0 -> 172.16.0.255 (256)

172.16.0.0/16 = 172.16.0.0 -> 172.16.255.255 (65536)

172.16.100.200/32 = 172.16.100.200 (1)

CIDR Ranges

The Entire Internet

0.0.0.0/0

0.0.0.0 -> 255.255.255.255
2^32
~4.3 billion addresses

CIDR Ranges

10.0.0.0/8 = 10.0.0.0 –> 10.255.255.255

172.16.0.0/12 = 172.16.0.0 –> 172.31.255.255

192.168.0.0/16 = 192.168.0.0 – 192.168.255.255

Private IP Space
Internet Engineering Task Force (IETF)

17,891,328 Possible Addresses

CIDR Ranges

172.28.0.0/16 = US Green Kubernetes

IP Spaces @ UA

https://underarmour.atlassian.net/wiki/spaces/INFRA/pages/39518491/Network+Block+Inventory

172.30.0.0/16 = Veranda Prod VPC

172.23.0.0/16 = MFP Prod VPC

10.110.48.0/21 & 10.110.32.0/22 = Ecomm Kubernetes

VPC = /16 & Subnets = /19

CIDR Ranges

127.0.0.1 - Localhost / Loopback

Addresses

10.150.0.3 - VPN (Pritunl)

8.8.8.8 - Google DNS Server

1.1.1.1 - Cloudflare DNS Server

DNS

DNS - Domain Name Servers

A - Alias Record

CNAME - Canonical Name Record

TLD - Top Level Domain Name

*.com | *.net | *.io | *.dev

FQDN - Fully Qualified Domain Name

www.underarmour.com.

DNS

Amazon Alias (you confusing bastard!)

DNS

Subdomains & Hosted Zones

*.underarmour.com

*.api.underarmour.com

~~my.cool.thing.underarmour.com~~

The "." is sacred

DNS

Phonebook of the Internet

dig parlour.uacf.io

; <<>> DiG 9.9.7-P3 <<>> parlour.uacf.io
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30649
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;parlour.uacf.io.		IN	A

;; ANSWER SECTION:
parlour.uacf.io.	59	IN	CNAME	us-green-internal.uacf.io.
us-green-internal.uacf.io. 59	IN	A	172.28.177.15
us-green-internal.uacf.io. 59	IN	A	172.28.209.202
us-green-internal.uacf.io. 59	IN	A	172.28.133.40

;; Query time: 73 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Thu Feb 06 06:03:07 CST 2020
;; MSG SIZE  rcvd: 124

DIG - Domain Information Groper

DNS

;; QUESTION SECTION:
;parlour.uacf.io.		IN	A

;; ANSWER SECTION:
parlour.uacf.io.	59	IN	CNAME	us-green-internal.uacf.io.
us-green-internal.uacf.io. 59	IN	A	172.28.177.15
us-green-internal.uacf.io. 59	IN	A	172.28.209.202
us-green-internal.uacf.io. 59	IN	A	172.28.133.40

DNS

;; QUESTION SECTION:
;dns-test.uacf.io.		IN	A

;; ANSWER SECTION:
dns-test.uacf.io.	299	IN	CNAME	matthew-brahms.uacf.io.
matthew-brahms.uacf.io.	299	IN	CNAME	kyle-rockman.uacf.io.
kyle-rockman.uacf.io.	59	IN	CNAME	us-green-internal.uacf.io.
us-green-internal.uacf.io. 59	IN	A	172.28.177.15
us-green-internal.uacf.io. 59	IN	A	172.28.133.40
us-green-internal.uacf.io. 59	IN	A	172.28.209.202

Packets

TCP - Transmission Control Protocol

Reliable

UDP - User Datagram Protocol

Unreliable

NAT - Network Address Translation

Packets

But First - A TCP Socket!

Open TCP Socket

App

TCP Socket

Destination

Hey you got this?

(Syn)

Yes i got this!
(Ack)

TCP Handshake

Packets

Write To Socket

Packets!

App

TCP Socket

Destination

Packets

Dropped Packets!

App

TCP Socket

Destination

ACK Packets 1,2,3,5

Packets

App

TCP Socket

Destination

Hey sounds like you missed #4

Retransmission

Packets

App

UDP Socket

Destination

Here take all this crap!

UDP Packets can be received in ANY order
and can be dropped entirely

Packets

Layer 2

Layer 3

Layer 4

Layer 5+6

Layer 7

Packets

172.28.123.210

iptables - AKA Firewall

iptables -A INPUT -s 172.23.0.0/16 -j ALLOW
iptables -A INPUT -s 0.0.0.0/0 -j DROP

172.23.10.100

Packet

Packets

172.28.123.210

NAT - AKA Router

172.23.10.100

Packet

53.123.54.67

Source: ~~172.23.10.100~~
53.123.54.67

iptables -A INPUT -s 53.123.54.67/32 -j ALLOW
iptables -A INPUT -s 0.0.0.0/0 -j DROP

Packets

Traceroute

traceroute myfitnesspal.com
traceroute to myfitnesspal.com (99.86.229.49), 64 hops max, 52 byte packets
 1  192.168.230.1 (192.168.230.1)  43.742 ms  46.513 ms  56.158 ms
 2  99.82.181.24 (99.82.181.24)  45.371 ms  46.230 ms  51.020 ms
 3  * * *
 4  * * *
 5  * * *
 6  52.46.164.211 (52.46.164.211)  52.039 ms  46.172 ms
    52.46.164.215 (52.46.164.215)  48.546 ms
 7  * * *
 8  * * *
 9  * * *
10  * * *
11  * * *
12  server-99-86-229-49.iad79.r.cloudfront.net (99.86.229.49)  46.945 ms  46.266 ms  48.858 ms

traceroute google.com
traceroute to google.com (172.217.13.238), 64 hops max, 52 byte packets
 1  192.168.230.1 (192.168.230.1)  42.559 ms  43.285 ms  42.578 ms
 2  iad23s61-in-f14.1e100.net (172.217.13.238)  44.991 ms  44.521 ms  42.765 ms

Packets

172.28.123.210

LB - Load Balancer

172.23.10.100

Packet

53.123.54.67

172.28.100.210

172.28.123.211

iptables -A INPUT -s 172.28.0.0/16 -j ALLOW
iptables -A INPUT -s 0.0.0.0/0 -j DROP

Packets

-A KUBE-EXTERNAL-SERVICES -p tcp -m comment --comment "run-integ/nginx-ingress-mapmy:metrics has no endpoints" -m addrtype --dst-type LOCAL -m tcp --dport 30891 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-EXTERNAL-SERVICES -p tcp -m comment --comment "default/ambassador-admin:ambassador-admin has no endpoints" -m addrtype --dst-type LOCAL -m tcp --dport 30430 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-EXTERNAL-SERVICES -p tcp -m comment --comment "run-integ/nginx-ingress-mapmy:status has no endpoints" -m addrtype --dst-type LOCAL -m tcp --dport 31844 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-EXTERNAL-SERVICES -p tcp -m comment --comment "nutrition-integ/core-services-identity-elb: has no endpoints" -m addrtype --dst-type LOCAL -m tcp --dport 31763 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-EXTERNAL-SERVICES -p tcp -m comment --comment "run-integ/nginx-ingress-mapmy:https has no endpoints" -m addrtype --dst-type LOCAL -m tcp --dport 32086 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-EXTERNAL-SERVICES -p tcp -m comment --comment "run-integ/nginx-ingress-mapmy:http has no endpoints" -m addrtype --dst-type LOCAL -m tcp --dport 30259 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-FIREWALL -m comment --comment "kubernetes firewall for dropping marked packets" -m mark --mark 0x8000/0x8000 -j DROP
-A KUBE-FORWARD -m comment --comment "kubernetes forwarding rules" -m mark --mark 0x4000/0x4000 -j ACCEPT
-A KUBE-FORWARD -s 100.96.0.0/11 -m comment --comment "kubernetes forwarding conntrack pod source rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A KUBE-FORWARD -d 100.96.0.0/11 -m comment --comment "kubernetes forwarding conntrack pod destination rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A KUBE-SERVICES -d 100.70.113.85/32 -p tcp -m comment --comment "kube-system/tiller-deploy:tiller has no endpoints" -m tcp --dport 44134 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.64.170.170/32 -p tcp -m comment --comment "data-prod/geo-api: has no endpoints" -m tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.71.30.42/32 -p udp -m comment --comment "infra-integ/gostatsd:udp has no endpoints" -m udp --dport 8125 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.65.50.15/32 -p udp -m comment --comment "default/statsd:udp has no endpoints" -m udp --dport 8125 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.69.21.123/32 -p tcp -m comment --comment "run-integ/nginx-ingress-mapmy:metrics has no endpoints" -m tcp --dport 10254 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.69.39.223/32 -p tcp -m comment --comment "fs-prod/accountservices-redis: has no endpoints" -m tcp --dport 6379 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.64.191.183/32 -p tcp -m comment --comment "nutrition-prod/locust-master:web-ui has no endpoints" -m tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.68.226.159/32 -p tcp -m comment --comment "fs-integ/integ-identity-ggconsumer: has no endpoints" -m tcp --dport 8080 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.64.105.94/32 -p udp -m comment --comment "jenkins/statsd-exporter:udp has no endpoints" -m udp --dport 9125 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.69.131.178/32 -p udp -m comment --comment "polaris/statsd-exporter:udp has no endpoints" -m udp --dport 9125 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.64.191.183/32 -p tcp -m comment --comment "nutrition-prod/locust-master:communication has no endpoints" -m tcp --dport 5557 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.64.137.219/32 -p tcp -m comment --comment "security-prod/productpage:http has no endpoints" -m tcp --dport 9080 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.70.194.38/32 -p udp -m comment --comment "kube-public/statsd-exporter:udp has no endpoints" -m udp --dport 9125 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.66.68.200/32 -p tcp -m comment --comment "nutrition-prod/identity-service: has no endpoints" -m tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.67.249.12/32 -p tcp -m comment --comment "default/ambassador-admin:ambassador-admin has no endpoints" -m tcp --dport 8877 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.69.21.123/32 -p tcp -m comment --comment "run-integ/nginx-ingress-mapmy:status has no endpoints" -m tcp --dport 18080 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.65.55.91/32 -p tcp -m comment --comment "monitoring/statsd-exporter:tcp has no endpoints" -m tcp --dport 9125 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.68.44.8/32 -p udp -m comment --comment "linkerd/statsd-exporter:udp has no endpoints" -m udp --dport 9125 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.69.232.222/32 -p tcp -m comment --comment "default/statsd-exporter:tcp has no endpoints" -m tcp --dport 9125 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.69.232.222/32 -p tcp -m comment --comment "default/statsd-exporter:prometheus has no endpoints" -m tcp --dport 9102 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.64.135.155/32 -p tcp -m comment --comment "kube-system/kube-state-metrics:http-metrics has no endpoints" -m tcp --dport 8080 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.66.247.26/32 -p tcp -m comment --comment "istio-system/statsd-exporter:tcp has no endpoints" -m tcp --dport 9125 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.69.232.222/32 -p udp -m comment --comment "default/statsd-exporter:udp has no endpoints" -m udp --dport 9125 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.65.251.176/32 -p tcp -m comment --comment "nutrition-prod/mainapp-integ:https has no endpoints" -m tcp --dport 4002 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.65.9.252/32 -p tcp -m comment --comment "default/argocd-redis-ha-announce-1:sentinel has no endpoints" -m tcp --dport 26379 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.64.105.94/32 -p tcp -m comment --comment "jenkins/statsd-exporter:tcp has no endpoints" -m tcp --dport 9125 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.65.211.165/32 -p tcp -m comment --comment "security-prod/details:http has no endpoints" -m tcp --dport 9080 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.68.211.128/32 -p tcp -m comment --comment "data-integ/uageo-api: has no endpoints" -m tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.69.45.215/32 -p tcp -m comment --comment "default/argocd-dex-server:grpc has no endpoints" -m tcp --dport 5557 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.64.135.155/32 -p tcp -m comment --comment "kube-system/kube-state-metrics:telemetry has no endpoints" -m tcp --dport 8081 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.65.251.176/32 -p tcp -m comment --comment "nutrition-prod/mainapp-integ:default has no endpoints" -m tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.66.34.88/32 -p tcp -m comment --comment "run-prod/felixbot:httpapi has no endpoints" -m tcp --dport 8000 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.70.118.214/32 -p tcp -m comment --comment "default/argocd-server:http has no endpoints" -m tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.64.105.94/32 -p tcp -m comment --comment "jenkins/statsd-exporter:prometheus has no endpoints" -m tcp --dport 9102 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.70.117.209/32 -p tcp -m comment --comment "eng-ops-integ/eng-ops-metrics-webhook: has no endpoints" -m tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.69.105.120/32 -p tcp -m comment --comment "kube-system/statsd-exporter:tcp has no endpoints" -m tcp --dport 9125 -j REJECT --reject-with icmp-port-unreachable
-A KUBE-SERVICES -d 100.66.80.166/32 -p tcp -m comment --comment "nutrition-integ/core-services-identity:http has no endpoints" -m tcp --dport 80 -j REJECT --reject-with icmp-port-unreachable

VPN

VPN - Virtual Private Network

Your Laptop

VPN

Private IP
10.150.0.3

Public IP
35.199.62.205

RDS

Route
172.28.0.0/16 -> 35.199.62.205

172.28.101.98

Allow Port 5432 from 10.150.0.3/32

Who am I

Lead Infrastructure Engineer

Rocktavious

krockman@underarmour.com

Kyle Rockman

201

VPC's

Subnet's
SG's
ELB's
Route Table's
Private Cross Cloud Networking

What are we going to learn about today!

Security Groups (SG)

What is a Security Group?

allow port 80 172.28.0.0/16

172.23.101.15

172.28.101.15

172.24.101.15

Security Groups (SG)

allow port 80 from SG-1

172.23.101.15

172.28.101.15

172.24.101.15

SG-1

Subnets

What is a Subnet?

172.28.0.0/19
172.28.0.0 -> 172.28.31.255
8,192 IP Addresses

172.28.32.0/19
172.28.32.0 -> 172.28.63.255
8,192 IP Addresses

Subnets

What is a Subnet for?

Amazon US-EAST-1 (North Virginia)
VPC - 172.28.0.0/16

us-east-1a

us-east-1c

us-east-1e

172.28.0.0/19

172.28.32.0/19

172.28.64.0/19

Subnets

What is a Availability Zone for?

us-east-1a

us-east-1c

us-east-1e

VPC

Subnets

How to Prevent in K8S

spec:
  template:
    metadata:
      labels:
        app: my-cool-service
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - my-cool-service
            topologyKey: failure-domain.beta.kubernetes.io/zone

Subnets & ELBs

ELB

VPC

What is an Elastic Load Balancer for?

Subnets & ELBs

ELB

app.ua.dev

App

port 8080

VPC

EC2

AZ 2

AZ 1

EC2

Ingress

VPC

What is a VPC?

VPC - Virtual Private Cloud

172.28.0.0/16
172.28.0.0 -> 172.28.255.255
65,536 IP Addresses

VPC Peering (PCX)

Route Tables

VPC 1

172.28.0.0/16

VPC 2

172.25.0.0/16

Peering

172.25.0.0/16 ->

<- 172.28.0.0/16

VPC 1 <-> VPC 2

VPC Route Table

Route Tables

Transit Gateway (TGW)

Hub and Spoke Networking

VPC 1

172.28.0.0/16

VPC 2

172.25.0.0/16

Transit Gateway (TGW)

Run AWS

Nutrition AWS

VPC

LOTS OF ROUTES

Cross Cloud Networking

Run AWS

Nutrition AWS

VPC

Google Cloud

VPN

BGP

Cross Cloud Networking

VPC

Google Cloud

VPN

AWS 4

AWS 5

AWS 1

AWS 2

AWS 3

AWS 7

AWS 8

Cross Cloud Networking

VPC

Google Cloud

VPN

AWS 4

AWS 5

AWS 1

AWS 2

AWS 3

AWS 7

AWS 8

Cross Cloud Networking

VPC

Google Cloud

VPN

AWS 4

AWS 2

AWS 3

301

Virtual Networks

Cloud Networking into Kubernetes
Intra Kubernetes Networking
Network Polices

What are we going to learn about today!

Kubernetes Networking

https://slides.com/rocktavious/cloud-networking-101/

Thanks!

Cloud Networking

By Kyle Rockman

Cloud Networking

How you fit into our cloud network here at Under Armour Connected Fitness.

Kyle Rockman

Lead Infrastructure Engineer OpsLevel.com

rocktavious