https://slides.com/elpete/itb2022-cbq
https://slides.com/elpete/itb2022-cbq
What this talk is
- An overview of cbq
- Examples of problems that queues and jobs can solve
- Overview of how to create new cbq protocols
What this talk isn't
- How to use the ColdBox Async Manager
- Overview of ColdBox Scheduled Tasks
- A review of stable software — cbq is being released as an early preview
Who am I?
Utah
Ortus Solutions
qb, Quick, Hyper, lots of other modules
1 wife, 3 kids, 1 dog
Type 1 Diabetic
Theatre Nerd
What is cbq?
A protocol-based queueing system for ColdBox
A protocol-based queueing system for ColdBox
Can interact with different providers, like the ColdBox Async Engine, a database, Redis, or Rabbit MQ
A protocol-based queueing system for ColdBox
Sends a message to be consumed later.
It can be consumed by the same application or a completely different application, language, or service.
- Defines a Job as the unit of work on a queue
- Push a Job onto a queue connection to be worked later
- Define multiple queues or named piles of work
- Register worker pools to work the jobs passed to queues
- Ability to switch between Queue Providers easily
What does cbq give me?
Why not just use...?
- cfthread, runAsync, AsyncManager
- Redis, Rabbit MQ, etc.
- Homegrown queue framework
Why would I use cbq?
Sending Email
Preparing Large Spreadsheets
Video Processing
Background Uploads
Sequenced Jobs
Scheduled SMS Messages
Email Verification
Processing Payments
Cancelling Abandoned Orders
Send Monthly Invoices
Other reasons
- Easier testing making sure the right jobs were queued
- Sync in development, Rabbit in production
- Adjust the worker pool scale dynamically
- Retry, timeout, and backoff built in.
Definitions
Job
- Does its work in the handle method
- Serializes and deserializes itself to the queue protocol
- Set instance data in the properties
-
Exist in the context of your application
Job
Queue
- A named stack of jobs or messages to be delivered
- A queue connection must have at least one queue which is usually default
- A queue connection can have as many queues as desired
Queue
Queue Provider
- How a queue connection connects to a backend like Redis, RabbitMQ, or a database
- Can be used multiple times in a single application to define multiple queue connections with different configuration options
Queue Provider
Queue Connection
- A named combination of Queue Provider and properties
- Allows you to connect with multiple Database providers or multiple Redis providers
Queue Connection
Installation
box install cbq
Configuration
Module Settings
moduleSettings = {
"cbq" : {
// The path the custom config file to
// register connections and worker pools
"configPath" : "config.cbq",
// Flag if workers should be registered.
// If your application only pushes to the queues,
// you can set this to false.
"registerWorkers" : getSystemSetting( "CBQ_REGISTER_WORKERS", true ),
// The interval to poll for changes to the worker pool scaling.
// Defaults to 0 which turns off the scheduled scaling feature.
"scaleInterval" : 0
}
};cbq Config File
component {
function configure() {
newConnection( "default" )
.setProvider( "ColdBoxAsyncProvider@cbq" );
newConnection( "database" )
.setProvider( "DBProvider@cbq" )
.setProperties( { "tableName": "custom_jobs_table" } );
}
function work() {
newWorkerPool( "default" )
.setTimeout( 5 )
.setMaxAttempts( 5 )
.setQuantity( 3 );
}
}
Usage
Create a Job
// SendWelcomeEmailJob.cfc
component extends="cbq.models.Jobs.AbstractJob" {
property name="mailService" inject="MailService@cbmailservices";
function handle() {
var user = getInstance( "User" )
.findOrFail( getProperties().userId );
variables.mailService
.newMail(
from = "no-reply@example.com",
to = user.getEmail(),
subject = "Welcome!",
type = "html"
)
.setView( "/_emails/users/welcome" )
.setBodyTokens( {
"firstName" : user.getFirstName(),
"lastName" : user.getLastName()
} )
.send();
}
}Per-Job Configuration
// SendWelcomeEmailJob.cfc
component extends="cbq.models.Jobs.AbstractJob" {
variables.connection = "emails";
variables.maxAttempts = 3;
variables.timeout = 10;
function handle() {
// ...
}
}Dispatching Jobs
// handlers/Main.cfc
component {
function create() {
var user = createUser( rc );
getInstance( "SendWelcomeEmailJob" )
.setProperties( { "userId": user.getId() } )
.dispatch();
}
}Setting up Worker Pools
// config/cbq.cfc
component {
function work() {
newWorkerPool( "default" )
.setTimeout( 30 )
.setMaxAttempts( 5 )
.setQuantity( 3 );
newWorkerPool( "default" )
.setQueue( "emails" )
.setTimeout( 60 )
.setMaxAttempts( 10 )
.setQuantity( 10 );
}
}
moduleSettings = {
"cbq" : {
"registerWorkers" : true // this is also the default
}
};Demo
Bonus
Job Sequences
// FulfillOrderJob.cfc
component extends="cbq.models.Jobs.AbstractJob" {
function handle() {
var productId = getProperties().productId;
processPayment( productId );
getInstance( "SendProductLinkEmail" )
.onConnection( "fulfillment" )
.setProperties( {
"productId" : productId,
"userId" : getProperties().userId
} )
.dispatch();
}
}Delay the Current Job
// MonitorPendingCartJob.cfc
component extends="cbq.models.Jobs.AbstractJob" {
function handle() {
var cartId = getProperties().cartId;
var cart = getInstance( "Cart" ).findOrFail( cartId );
if ( cart.isConfirmed() || cart.isCancelled() ) {
return;
}
if ( abs( dateDiff( "n", cart.getModifiedDate(), now() ) ) > 60 ) {
cart.cancel();
return;
}
getInstance( "MonitorPendingCartJob" )
.setProperties( { "cartId" : cartId } )
.delay( 15 * 1000 )
.dispatch();
}
}
cbq
By Eric Peterson
