Lynn Langit
Cloud Architect who codes
serverless systems
@lynnlangit
Belgrade 2019
@lynnlangit
Year 2009
@lynnlangit
@lynnlangit
@lynnlangit
AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: >-
An Amazon S3 trigger that retrieves metadata for the object that has been
updated.
Parameters:
BucketNamePrefix:
Type: String
Default: sam-example
Resources:
s3getobject:
Type: 'AWS::Serverless::Function'
Properties:
Handler: index.handler
Runtime: nodejs8.10
CodeUri: .
Description: >-
An Amazon S3 trigger that retrieves metadata for the object that has
been updated.
MemorySize: 128
Timeout: 3
Policies:
- S3CrudPolicy:
BucketName: !Sub "${BucketNamePrefix}-get-object"
Events:
BucketEvent1:
Type: S3
Properties:
Bucket: !Ref Bucket1
Events:
- 's3:ObjectCreated:*'
Bucket1:
Type: 'AWS::S3::Bucket'
Properties:
BucketName: !Sub "${BucketNamePrefix}-get-object"
What does this "code" do?
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
only Lambda(s)?
@lynnlangit
@lynnlangit
Type | Name | AWS Examples |
---|---|---|
Compute | Functions | Lambda |
Files | Buckets | S3 |
Data | NoSQL Streams |
DynamoDB Kinesis |
Other... | Gateway ETL Orchestration Pub/Sub Machine Learning |
API Gateway Glue Step Functions SNS Rekognition |
@lynnlangit
@lynnlangit
<-- SERVERLESS -->
Pattern:
Visualize Services
@lynnlangit
Genomic Sequencing for Improved Human Health
@lynnlangit
@lynnlangit
@lynnlangit
Time: 1 Day
Speed: 80% faster
Pattern:
Visualize Logs
@lynnlangit
terraform {
backend "s3" {
bucket = "variant-spark-july"
key = "variantspark-k/tfstate"
region = "us-west-2"
profile = "default"
}
}
provider "aws" {
profile = "${var.profile}"
region = "${var.default_region}"
}
provider "aws" {
alias = "use1"
profile = "${var.profile}"
region = "us-east-1"
}
module "eks-vpc" {
source = "modules/eks-vpc"
"cluster-name" = "${var.cluster-name}"
}
module "eks-master-role" {
source = "modules/eks-master-role"
}
module "eks-master-security-group" {
source = "modules/eks-master-security-group"
vpc_id = "${module.eks-vpc.vpc_id}"
}
module "eks-worker-role" {
source = "modules/eks-worker-role"
}
module "eks-worker-security-group" {
source = "modules/eks-worker-security-group"
vpc_id = "${module.eks-vpc.vpc_id}"
master_security_group_id = "${module.eks-master-security-group.security_group_id}"
"cluster-name" = "${var.cluster-name}"
}
module "eks" {
source = "modules/eks"
cluster_name = "${var.cluster-name}"
vpc_id = "${module.eks-vpc.vpc_id}"
public_subnets = "${module.eks-vpc.public_subnets}"
master_role_arn = "${module.eks-master-role.master_role_arn}"
master_security_group_id = "${module.eks-master-security-group.security_group_id}"
worker_role_arn = "${module.eks-worker-role.worker_role_arn}"
worker_profile_name = "${module.eks-worker-role.worker_profile_name}"
worker_security_group_id = "${module.eks-worker-security-group.security_group_id}"
caller_profile = "${var.profile}"
eks_key_name = "${module.kops-ssh.key_name}"
spark_user_arn = "${module.kops-user.arn}"
}
module "kops-user" {
source = "modules/kops-user"
}
module "state-storage" {
source = "modules/state-storage"
}
module "input-bucket" {
source = "modules/input-bucket"
}
module "kops-ssh" {
source = "modules/ssh-key"
}
data "aws_availability_zones" "available" {}
data "template_file" "env" {
template = "${file("templates/env.tpl")}"
What does this "code" do?
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
This is text
@lynnlangit
@lynnlangit
Pattern:
Create intermediate step
@lynnlangit
Pattern:
Serverless Data
@lynnlangit
@lynnlangit
If we spoke a different language, we would perceive a somewhat different world." - Wittgenstein
@lynnlangit
reproducible systems
@lynnlangit
a particular form of a language that is peculiar to a specific region or social group
synonyms: |
regional language, local language, local speech, vernacular, idiom;
regionalisms, localisms; informal lingo
|
What is the dialect for SERVERLESS SYSTEMS?
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
visual understanding
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
"<The> guarantees form a negative architecture - a set of things that you know can’t happen in various pieces of your system" - Michael Feathers
@lynnlangit
@lynnlangit
Artists draw because
they want to convey something they can't describe"
- Bret Victor
@lynnlangit
Dynamic Systems - Bret Victor
dynamic system visualizations
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
@lynnlangit
Type | Name | Notes |
---|---|---|
User data | FASTA, FASTQ BAM |
Raw sequencer files Binary user data |
Application(s) Java / Scala |
Custom ML GATK |
Bioinfo tools / jar files |
Config | JSON YAML |
Tool configuration Vendor cloud services |
Log | CSV | Job run history files |
@lynnlangit
@lynnlangit
@lynnlangit
Pattern:
Serverless Data
@lynnlangit
Pattern:
Visualize Data/Process
@lynnlangit
Year 2019
@lynnlangit
@lynnlangit
serverless systems
@lynnlangit
Belgrade 2019
By Lynn Langit
Serverless Days 2019 Talk