Dealing with Webhooks

and building your own Slack bot

About presenter

  • Ashok Modi
  • Software engineer - CARD
    • Connecting with a lot of different APIs
    • Processing data via webhooks
  • Member of LA Drupal user group
    • Used to use IRC
    • Moved to Slack

Using Slack

  • Has been great
    • Over 150 members
    • Over 100 messages/week (up from 0/week)
  • Miss the IRC bots
    • Karma
    • Silliness

Lets figure out how to get karma up and running

Build a Slack bot!

Few ways to implement this

  • Use webhook events API
    • Main implementation
  • Use RTM API
    • Uses web sockets - open connection
    • Very cool but not used at moment
  • Use web API
    • Not ideal of retrieving messages
    • Useful for posting back

What is a webhook?

  • Its a 'reverse API'
    • "Don't call us - we'll call you"
    • POST request to your service
  • Required to have a way for Slack to reach you
    • Having your own web server
    • Using AWS API Gateway or the equivalent
      from Google, Azure, IBM, etc
    • Implement code to process the webhook

Implementing a receiver

  • Build using your choice in language/framework
  • Built mine using Lumen
    • Lightweight with plug-able features
      • Routes
      • Models/ORM
      • Config
      • Queues
    • API-only

Things to consider

  • Webhook events from your vendor may be asynchronous
    • The data they are sending back is mostly useless :(
    • Use it to assume *an* event has occurred
      • See if they have an API call you can use to get 'at-moment' update of data.

Plan of action

  • Get payload from Slack (POST request)
    • Validate request
  • Queue it for processing
    • Do some basic check to put it in appropriate queue
  • Return 200 response code
  • Run a queue processing task every few seconds on server to process queue items

Why validate?

  • Just make sure it is actually coming from Slack
    • Not a malicious person
    • Slack provides a token which you can use
    • Other services might not
      • IP whitelisting, other params in request

Why queue requests?

  • If something goes wrong
    • Way to examine failed request
    • Have a way to possibly trigger other events
  • Need to close out the https request as quickly as possible
    • Slack will continually try to resend messages that are not responded to.
    • Common in other vendors that also use webhook approach.

Anatomy of the app

config (for configuration)

.env file (unique to environment)

routes (for API routes - we just have one)

database (for schema/migrations/etc)

app (contains model, controller, helpers, etc)

tests (for your tests)

app directory (cont'd)

  • Console
    • command line stuff
  • Helpers
    • Helper classes, mock classes
  • Http (most important)
    • Controller, middleware
  • Jobs (second most important)
    • Queues
  • Models
    • Content models (Karma, Slack users)

Lets look at code

https://gitlab.com/btmash/lumen-karmabot

Possible improvements

  • Use Redis for queue
    • Or Kafka for closer to real-time processing
  • Collect all messages
    • Indexing for a search tool
  • Move to AWS as a lambda in the future?

Questions?

Thank you :)