Michele Riva
MicheleRivaCode
MicheleRivaCode
Gabriele
Me
Friends
MicheleRivaCode
MicheleRivaCode
Create an API that uploads a large CSV file (>10GB) to a server, then read the file, normalize the data, and store it into a database
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
Once we start uploading and processing a few large files, we'll run out of memory
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
How do we currently deal with large amounts of people?
MicheleRivaCode
MicheleRivaCode
But what if there's a *lot* of people?
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
When it comes to software architecture, things gets a bit harder...
MicheleRivaCode
A collection of indipendent computers that appear to its users as one computer
A distributed system is
(Andrew Tanenbaum)
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
Source of truth
MicheleRivaCode
Source of truth
Sync request
Sync request
MicheleRivaCode
Source of truth
Time response
Time response
MicheleRivaCode
Source of truth
Time response
Time response
Network time protocol (NTP)
Precision time protocol (PTP)
MicheleRivaCode
The solution needs to follow the following rules
MicheleRivaCode
Is uploading a large file to a server a good option?
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
Have we respected the rules?
MicheleRivaCode
So what problems did we solve?
Senior Software Architect @NearForm
Google Developer Expert
Microsoft MVP
MicheleRivaCode
MicheleRivaCode
MicheleRivaCode
@MicheleRiva
@MicheleRivaCode
/in/MicheleRiva95
www.micheleriva.dev