{AWS Batch}

Implementing Map Reduce with AWS Batch

Use cases (AWS)

Financial services: Post-trade analytics

Automate the analysis of the day’s transaction costs, execution reporting, and market performance

1.

2.

Life sciences: Drug screening for biopharma

Rapidly search libraries of small molecules for drug discovery..

3.

Digital media: Visual effects rendering

Automate content rendering workloads and reduce the need for human intervention due to execution dependencies or resource scheduling

EC2

Fargate

Batch

Lambda

Serverless Spectrum

Compute Environments can be EC2 Spot, EC2 or Fargate

Task definitions

container specification (size, memory, image, command etc.)

Map/Reduce

MapReduce is a programming model and an associated implementation for processing and generating big data sets with a parallel, distributed algorithm on a cluster - Wikipedia

Demo Load NYC Taxi data and build video visualisation

To the Demo! 🚀

Find the code at
https://github.com/rtkelly13/AWSBatch-MapReduce

AWS Batch Map/Reduce

By Ryan Kelly