Architecture Overview & Improvements

Agenda

  • What is DTech Group about?
  • Requirements
  • Architecture and Infrastructure
  • Discuss NFRs and Improvements

Requirements

  • SaaS platform to access data from multiple social media platforms
    • MVP: Twitter only
    • Future: Facebook, LinkedIn, etc.
  • Multitenancy, Billing

  • Data synchronization from social media platforms

  • Complex searching queries

  • Email notifications

  • Screenshots of social media posts

Non-Functional Requirements

  • Extensive requests and queues logging
  • Optimize Twitter Enterprise API request count
  • Configurable variables to tweak cost/performance
  • MVP: Low infrastructure cost
  • Future proof 
  • Easy scalability
  • Performance (Response time)
  • Security

Architecture

Infrastructure

Performance

  • Handles 1200+ Twitter profiles
  • Searching over 10 million+ documents
  • Have 500MB+ worth of screenshots
  • API response time < 600ms for 85th percentile
  • Collected over 1million tweets in a day

Improvements

  • Use caching
  • Optimize some poor performing SQL queries
  • Use database read-replicas
  • Shard Elasticsearch data

Scalability

  • Uses ECS Fargate
  • Stateless backend services
  • Uses CloudMap for Service Discovery

Improvements

  • Breakdown some consumers/tasks to Lambdas
  • Perform automatic scaling of containers

Availability

  • Serving FE over CloudFront

Improvements

  • Use multi AZ for services
  • Use multi AZ for databases

Security

  • TLS v1.3
  • CORS Enabled
  • JWT Authentication
  • IAM Roles / Policies
  • Security Groups

Improvements

  • Use Cognito
  • Perform penetration testing
  •  
  • Secure application configurations with secret manager

Maintainability / Extensibility

  • Dev Env: mimics 100% infrastructure using containers
  • Modular structure to support more social media platforms
  • Replaceable services
  • API Documentation / Consistent response
  • Extensive logging

Improvements

  • Increase test coverage
  • Add support for internationalization in FE
  • Use Cognito

Your time to say...

Associate Architect

By Ali Yousuf

Associate Architect

  • 230