Macklin Hartley
I help top-notch organisations by bringing together crystal-clear objectives, a culture of support, streamlined operations, and adaptable technology.
@macklin_hartley
#DDDPerth
© 20th Century Fox
@macklin_hartley
#DDDPerth
© 20th Century Fox
Image credit AFP/Project Possible - © 20th Century Fox
Image credit Matt Middleton-White - © 20th Century Fox
Image credit University of Oregon - © 20th Century Fox
Developers
Queues
❤️
@macklin_hartley
#DDDPerth
Consumer
Consumer
Consumer
External
3rd-party
@macklin_hartley
#DDDPerth
Message Proxy
Tenant 1
Tenant 2
Tenant 3
External
3rd-party
@macklin_hartley
#DDDPerth
Tenant 1
Message Proxy
External
3rd-party
🤔
@macklin_hartley
#DDDPerth
MessageProxy
Tenant 1
Process Manager
External
3rd-party
👍
@macklin_hartley
#DDDPerth
Observability
and Monitoring
Are Essential
Lesson #1
@macklin_hartley
#DDDPerth
Queue Depth
Response Time
@macklin_hartley
#DDDPerth
@macklin_hartley
#DDDPerth
© 20th Century Fox
Learn the
true THroughput
of your System
Lesson #2
@macklin_hartley
#DDDPerth
Message Proxy
Tenant 1
High Priority
Low Priority
External
3rd-party
@macklin_hartley
#DDDPerth
~2000
messages/min
~500
messages/min
@macklin_hartley
@macklin_hartley
#DDDPerth
© costaz (Stock Photo)
@macklin_hartley
#DDDPerth
© costaz (Stock Photo)
More Queues
≠
More Throughput
@macklin_hartley
#DDDPerth
Understand how
external Factors
Impact THroughput
Lesson #3
@macklin_hartley
#DDDPerth
150-200 milliseconds
Processing Time
~500 messages per minute
@macklin_hartley
#DDDPerth
2-5 seconds
Processing Time
~30 messages per minute
@macklin_hartley
#DDDPerth
@macklin_hartley
#DDDPerth
© costaz (Stock Photo)
Tenant 1
Process Manager
Batching
100x
50 seconds
~500 messages per minute
@macklin_hartley
#DDDPerth
Tenant 1
Process Manager
"Fan-Out"
150 - 200 milliseconds
4000+ messages per minute
@macklin_hartley
#DDDPerth
Sharding
@macklin_hartley
#DDDPerth
Maybe lots of messages per minute?
© costaz (Stock Photo)
Observability
and Monitoring
Are Essential
Lesson #1
@macklin_hartley
#DDDPerth
Learn the
true THroughput
of your System
Lesson #2
@macklin_hartley
#DDDPerth
Understand how
external Factors
Impact Throughput
Lesson #3
@macklin_hartley
#DDDPerth
Queues are Powerful
When They are Used
Thoughtfully
@macklin_hartley
#DDDPerth
Macklin Hartley
@macklin_hartley
Catch me by the VGW Coffee Cart
or with a 🍺 at the afterparty!
Every Good Outage Starts with a Queue
By Macklin Hartley
Every Good Outage Starts with a Queue
Queues are bad, but software developers love them. The problem with queues is they are binary, either they're full or empty. You'd think they would magically fix any overload or failure problem. But they don't, and bring with them a bunch of their own problems. Hear a war-story about how naive assumptions and over-optimism caused a near week-long ongoing outage. Learn about some of the reasons why queues are not a one-size fits all solution, and how there is no such thing as a good queue, only one that is absolutely necessary. Queues are in more places than you think, and they can bite you in ways that you wont expect. Are you building event sourced systems with process managers and read model populators? Are you working with serverless? This session is for you.
