West LA DevOps
October 24, 2019
meetup.com/West-LA-DevOps
Agenda
- Job Board
- Industry Updates
- Tech talk #1: Dancing with Data and Distributed Systems by Jasmine Dahilig
- Tech talk #2: Service Mesh for Everyone by Jake Lundberg
- Closing notes
About GumGum
- Computer Vision company
- Advertising division
- Context-aware ads
- Brand safety technology
- Sports division
>20M RPM
Online Advertising
Did you know?
GumGum Invented In-Image advertising in 2008
Job Board
DevOps Engineer @ GumGum (x2)
Job Board
DevOps Engineer @ AuditBoard
Backend/Frontend Engineer @ AuditBoard
Job Board
DevOps Engineer @ Your company?
Since the last time we met..
1) NordVPN Breach
- An unknown attacker gained access to a VPN server for "about a month" by exploiting an insecure remote management system installed by the datacenter provider
- NordVPN was unaware such a system existed
- A NordVPN spokesperson claimed “the server itself did not contain any user activity logs" and added "usernames and passwords couldn’t have been intercepted either”
- Security researcher warned that NordVPN was ignoring the larger issue of network access. “Your car was just stolen and taken on a joy ride and you’re quibbling about which buttons were pushed on the radio?”
Solution: Algo
- Ansible scripts that simplifies the setup of a personal VPN
- Lightweight
- Uses the most secure defaults available
- Works on most public clouds
2) New AWS Features
- New EC2 instances:
- EKS updates:
- Container monitoring for ECS/EKS/K8S w/CloudWatch
- AWS Chatbot: ChatOps for AWS
- SageMaker now supports spot instances
2) New AWS Features Cont'
-
Additional Metadata available VPC Flow Logs
- vpc-id, subnet-id, bitmasks, source/dest IP fields
- Route 53 Query Volume Metrics for Public Zones
- Network Load Balancers now support multiple TLS certificates using Server Name Indication (SNI)
3) DockerHub Outage
- On October 15, 2019 10:06 AM DockerHub went down
- DockerHub Web, DockerHub Registry 😲, DockerHub Automated Builds
- Many of our Docker builds broke 😧
- Some of our systems were unable to scale-out 😵
- At 11:04 AM the DockerHub Registry service was restored and at 2:48 PM all services were restored 🙏🏻
- Reminder: DockerHub is a free service with no service SLA 🤔
- Solutions: pull-through cache, ECR, kraken 👍
4) HTTP Desync Attacks
- HTTP/1.1 vulnerability exploited allowing an attacker to prepend content to another legitimate request. This is known as request smuggling
- This vulnerability exists because of the introduction of keep-alives in HTTP/1.1, sending multiple requests over a single TCP/SSL socket
- Because HTTP/1.1 requests are placed back to back, it is crucial that the frontend and backend agree on where each request ends and starts
- Content-Length and Transfer-Encoding headers typically used to determine when a request ends
4) HTTP Desync Attacks
4) HTTP Desync Attacks
POST / HTTP/1.1
Host: example.com
Content-Length: 6
Transfer-Encoding: chunked
0
GPOST / HTTP/1.1
Host: example.com
https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn
POST / HTTP/1.1
Host: example.com
Content-Length: 47
Transfer-Encoding: chunked
0
POST /users/{my_id}/make_admin
X-Ignore-Header: POST / HTTP/1.1
Host: example.com
Access-Token: {superuser_access_token}
5) GitHub Actions CI/CD
- GitHub is finally introducing their own CI/CD solution
- General availability will be on November 13th, Beta is out now
- YAML based config using container actions
- Huge shift in the community towards GitOps
- Coming soon: self-hosted GitHub Actions runners (free!)
- Out-of-the-box remote state management
- Previously needed to set up remote state + state locking for collaborative work
- Performing terraform plan + apply remotely
- This introduces CI/CD to your Infra-As-Code, which is naturally the next step
- VCS Webhooks allows GitOps based infrastructure deployments
- PRs into master with a terraform plan check
- Merge into master kicks off a terraform apply
6) Terraform Cloud
Dancing with Data & Distributed Systems
By Jasmine Dahilig, Software Engineer at HashiCorp
Service Mesh for Everyone
By Jake Lundberg, Staff Solutions Engineer at HashiCorp
meetup.com/Los-Angeles-HashiCorp-User-Group
Next WLAD Meetup
Date: January 2020
Getting Involved
- Interested in speaking at WLAD?
- Have an idea for a future talk?
Email us: westladevops@gmail.com
...or message us on Meetup
...or talk to us right now :)
West LA DevOps: The 4th Meetup
By Corey Gale
