Cloud Native Import

James Alexander

Andrew Kaczorek

Chris Chalfant

Cloud Native File Import

Existing App

  • C# .NET Framework
  • IIS / Windows Colo-Server
  • SQL Server
  • Bulk Uploads once per day of many files
  • Relatively low incoming Web Traffic monthly (<600 reqs per day)

New App

  • AWS Hosted
  • C# .NET Core
  • Docker / ECS / Fargate
  • PostgreSQL

Fargate

Overview

Step Function

Step Function

C# Source

[assembly : LambdaSerializer (typeof (Amazon.Lambda.Serialization.Json.JsonSerializer))]
public class LambdaParameters {
    public string Name { get; set; }
    public string Comment { get; set; }
    public bool ShouldRun { get; set; }
    public int QueueDepth { get; set; }
    public int ActiveRunningLoaders { get; set; }
    public int LoadersToRun { get; set; }
    public int MaxLoaders { get; set; }
    public string Bucket { get; set; }
    public string Key { get; set; }

    public override string ToString()
    {
        return $"Parameters: Comment: {Comment} - QueueDepth: {QueueDepth}";
    }
}

C# Lambda Source

public class LoaderLambda {

    public int Trigger (SNSEvent evnt, ILambdaContext context) { ... }

    public LambdaParameters ShouldContinueRunning(LambdaParameters p, ILambdaContext context) { ... }

    public LambdaParameters GetQueueDepth(LambdaParameters p, ILambdaContext context) { ... }

    public LambdaParameters GetActiveRunningLoaders(LambdaParameters p, ILambdaContext context) { ... }

    public LambdaParameters GetMaxLoaders(LambdaParameters p, ILambdaContext context) { ... }

    public LambdaParameters StartLoaders(LambdaParameters p, ILambdaContext context) { ... }
}

C# Lambda Config

{
  "profile": "local-aws-profile",
  "region" : "us-east-1",
  "configuration": "Release",
  "framework": "netcoreapp2.0",
  "function-runtime": "dotnetcore2.0",
  "function-memory-size": 256,
  "function-timeout": 30,
  "function-role": "aws-iam-role",
  "function-handler": "AwsLambda::AwsLambda.LoaderLambda::Trigger"
}

aws-lambda-tools-defaults.json

Lambda Deployment

dotnet lambda deploy-function dev-loader-task-step --function-role aws-iam-role

BitBucket Pipelines

Questions?

PS

 

http://bit.ly/sqs-sns

cloud-native-import

By James Alexander

cloud-native-import

  • 1,036

More from James Alexander