Blue green deployment in action

Damian Wysocki

Problem

Problem

Ride the lightning

Problem

Ride the lightning

LP

CD

Problem

Ride the lightning

LP

CD

US

Problem

Ride the lightning

LP

CD

US

Problem

Ride the lightning

LP

CD

US

Ride the lightning

LP

CD

GB

Problem

Ride the lightning

LP

CD

US

Ride the lightning

LP

CD

GB

Ride the lightning

LP

CD

EU

Problem

Ride the lightning

LP

CD

US

Ride the lightning

LP

CD

GB

Ride the lightning

LP

CD

EU

Product Type US

  • attribute 1 US
  • attribute 2 US
  • attribute ... US

Product Type GB

  • attribute 1 GB
  • attribute 2 GB
  • attribute ... GB

Product Type EU

  • attribute 1 EU
  • attribute 2 EU
  • attribute ... EU

Problem

  • Duplicated content
  • Putting a lot of work into the new channel
  • Problems publishing products to the other channel
  • The product schema is duplicated
  • Hard in maitenance

Solution

Ride the lightning

LP

CD

US

Ride the lightning

LP

CD

GB

Ride the lightning

LP

CD

EU

Solution

LP US

CD US

Ride the lightning

LP GB

CD GB

LP EU

CD EU

Challange

Migrate over 300k of products in a timeframe

Non-functional requirements

After a failure, start with the last added product

Monitor the progress of the migration

Tasks are independent of each other

Able to scale

Fast and easy rollback

Zero downtime migration strategy

After a failure, start with the last added product

Monitor the progress of the migration

Tasks are independent of each other

Able to scale

Monitor errors and react to them

Fast and easy rollback

Zero downtime migration strategy

Infrastructure

Infrastructure

Non-functional requirements

After a failure, start with the last added product

Monitor the progress of the migration

Tasks are independent of each other

Able to scale

Monitor errors and react to them

Fast and easy rollback

Zero downtime migration strategy

Non-functional requirements

After a failure, start with the last added product

Monitor the progress of the migration

Tasks are independent of each other

Able to scale

Fast and easy rollback

Zero downtime migration strategy

Monitoring

Monitoring

postgres=#

Monitoring

postgres=# select count(status), status from tasks group by status;

Monitoring

postgres=# select count(status), status from tasks group by status;
count  |   status    
--------+-------------
  24519 | COMPLETED
   2132 | DEAD
     28 | ERROR
      9 | IN_PROGRESS
 163532 | READY
(5 rows)

Non-functional requirements

After a failure, start with the last added product

Monitor the progress of the migration

Tasks are independent of each other

Able to scale

Monitor errors and react to them

Fast and easy rollback

Zero downtime migration strategy

Non-functional requirements

After a failure, start with the last added product

Monitor the progress of the migration

Tasks are independent of each other

Able to scale

Fast and easy rollback

Zero downtime migration strategy

Speed

Speed

  • lambda have one purspose
  • prefil database

Speed

Speed

  1. get product data from the database
  2. send payload to Saleor 
  3. update task status

Speed

  1. get product data from the database - X time
  2. send payload to Saleor - Y time 
  3. update task status - Z time

total time = X + Y + Z

Speed

  1. get product data from Saleor
  2. process data - V time 
  3. send payload to Saleor - Y time 
  4. update task status - Z time

Speed

  1. get product data from Saleor - 300-500ms time
  2. process data - V time 
  3. send payload to Saleor - Y time 
  4. update task status - Z time

total time = 500ms + V + Y + Z

Speed

  1. get product data from Saleor - 300-500ms time
  2. process data - V time 
  3. send payload to Saleor - Y time 
  4. update task status - Z time

total time = 500ms V + Y + Z

additional ~40h

Speed

def lambda_function():
	product_data = get_product_data_from_database()
    response = save_product_to_saleor(product_data)
    update_task_status(response)

Speed

def lambda_function():
	product_data = get_product_data_from_database()
    response = save_product_to_saleor(product_data)
    update_task_status(response)

Speed

def lambda_function():
	product_data = get_product_data_from_database()
    response = save_product_to_saleor(product_data)
    update_task_status(response)
def lambda_function():
	product_data = get_product_data_from_database()
    response = save_product_to_saleor(product_data)
    update_task_status(response)

Speed

def lambda_function():
	product_data = get_product_data_from_database()
    response = save_product_to_saleor(product_data)
    update_task_status(response)
def lambda_function():
	product_data = get_product_data_from_saleor()
    transformed_data = transform_data(product_data)
    response = save_product_to_saleor(transformed_data)
    update_task_status(response)

Speed

total time = X + Y + Z

total time = X + V + Y + Z

def lambda_function():
	product_data = get_product_data_from_database()
    response = save_product_to_saleor(product_data)
    update_task_status(response)
def lambda_function():
	product_data = get_product_data_from_saleor()
    transformed_data = transform_data(product_data)
    response = save_product_to_saleor(transformed_data)
    update_task_status(response)

Speed

total time = X + Y + Z

total time = X + V + Y + Z

def lambda_function():
	product_data = get_product_data_from_database()
    response = save_product_to_saleor(product_data)
    update_task_status(response)
def lambda_function():
	product_data = get_product_data_from_saleor()
    transformed_data = transform_data(product_data)
    response = save_product_to_saleor(transformed_data)
    update_task_status(response)

additional ~40h

Plan

send event to lambda

get product data

return product data

save product

return success message

update task status

save product

return error message

update task status

Blue green deployment

Blue green deployment

Blue green deployment

Products V1

Blue green deployment

Products V1

Blue green deployment

Products V1

Blue green deployment

Products V1

Blue green deployment

storefront.com

Products V1

Blue green deployment

storefront.com

Products V1

Blue green deployment

storefront.com

Products V1

Products V2

Blue green deployment

storefront.com

Products V1

Products V2

Blue green deployment

storefront.com

Products V1

Products V2

Blue green deployment

Products V1

Products V2

storefront.com

test.storefront.com

Blue green deployment

Products V1

Products V2

storefront.com

Blue green deployment

Products V1

Products V2

storefront.com

Blue green deployment

Products V1

Products V2

storefront.com

Blue green deployment

Products V1

Products V2

storefront.com

Blue green deployment

Products V2

storefront.com

Non-functional requirements

After a failure, start with the last added product

Monitor the progress of the migration

Tasks are independent of each other

Able to scale

Monitor errors and react to them

Fast and easy rollback

Zero downtime migration strategy

Non-functional requirements

After a failure, start with the last added product

Monitor the progress of the migration

Tasks are independent of each other

Able to scale

Fast and easy rollback

Zero downtime migration strategy

The end

Happy end?

Made with Slides.com