Application Monitoring for your Website

http://szeged2014.drupaldays.org/program/sessions/application-monitoring-your-website

About me

Sascha Grossenbacher

Berdir

Switzerland

MD Systems

Active Drupal 8 contributor

Maintaining contrib modules



Topics

  • Overview
  • How it works
  • Default sensors
  • Demo
  • Creating custom sensors
  • Integration
  • Drupal 8

Overview





Monitoring is a framework that allows you to expose the status of your website to the monitoring product of your choice

Why do you need it?

You probably are already monitoring your servers.

Maybe you even have some basic HTTP checks to see if your site is responding, but do you know... 

  • how many active users you have right now?
  • how much content has been created today?
  • how much money you made in your shop today?
  • how many users are currently in the payment checkout step?
  • how much of your content is not yet indexed?

What does it do?

  • Manages a set of sensors
  • Provides an API for running them
  • Display the results ...
    • in the UI
    • as a service
    • with drush
    • in your monitoring product
  • Sensor configuration
  • Logging

Sensor Definition

A sensor represents a thing that is monitored

Is executed and produces a SensorResult

Can be configurable (SensorConfigurableInterface) and can have thresholds (SensorThresholdsInterface)

Defined in code

SensorResult Definition

Represents the result of a single sensor run

  • Status (OK, Warning, Critical, ...)
  • Value
  • Message
  • Expected value
  • Execution time
  • Caching information

Configurability

Time interval

Many sensors have a time interval, like new content in the given time interval, this can be configured per sensor.

Thresholds

Allows to set the sensor status if the value is below, above, within or outside a configurable interval. For example, more than 50 failed logins within the last hour.

Sensor specific

Any sensor can expose his specific settings in the UI.

How it works #1: Sensor



A sensor consists of two parts

  1. A class that implements SensorInterface
  2. A definition in hook_monitoring_sensor_info()

A sensor class can be used by many sensors or just by one

The most important part of a sensor class is to implement the run() method, do the necessary checks and update the passed
SensorResult instance
SensorInterface::run(SensorResultInterface $result)

How it works #2: SensorManager

The sensor manager is responsible for collecting information about sensors from hook implementations and manage their settings.


monitoring_sensor_manager()->getSensorInfo()monitoring_sensor_manager()->enableSensor($sensor_name)
monitoring_sensor_manager()->disableSensor($sensor_name)

How it works #3: Sensor Runner

The sensor runner takes a set of sensors and returns corresponding results.


Responsibilities

  • Caching
  • Execution time
  • Exception handling

monitoring_sensor_run($sensor_name)monitoring_sensor_run_multiple(array $sensor_names = array())

How it works #4: Sensor Status

OK, Warning, Critical, Unknown

  1. Explicitly set by the Sensor class
  2. Does not match the expected value
  3. Violates the configured threshold
    1. Exceeds value
    2. Falls below value
    3. Outside of interval
    4. Inside of interval

How it works #5: Sensor message

Sensor message is built based on:
  • Value
  • Value label
  • Value type
  • Time interval
  • Expected value
  • Thresholds
  • Message parts

[Value] [Label] in [Time Frame], expected [ExpectedValue], exceeds [Interval], [Message]
54 watchdog events in 1 Tag, exceeds 50, apple-touch-icon-precomposed.png

CHF 1337.50 in 1 Tag, falls below CHF 1500.00

Default Sensors

Monitoring provides a lot sensors out of the box, which are capable of monitoring typical activity and problems on any Drupal site.

They can also be used as examples.

Content

Equally useful to track user generated content, articles on a news platform, imported content and so on.

Sensors

  • Recent new nodes (per node type)
  • New comments

User Activity

  • Active sessions
  • New users
  • Logins and Logouts (dblog)
  • Failed logins (dblog)

Logging

  • Watchdog records per severity (dblog)
  • Past records per severity
  • 404 Not Found errors (dblog)
  • Missing image styles (dblog)

 Commerce

  • Turnover per enabled currency and overall
  • Orders in a specific state

Core

  • Queue size per queue
  • Module requirements
  • Update status
  • Last cron run
  • Enabled modules
  • Disappeared sensors
  • Git dirty tree
  • ...

Contrib integration

  • Search API index status
  • Simplenews mail spool
  • Elysia cron




Short Demo


http://simplytest.me/project/monitoring/7.x-1.x

Creating custom sensors

The easiest way to add another sensor is to use one of the provided re-usable sensor classes. To do so, you only need to implement hook_monitoring_sensor_info() and define your sensor.

    SensorDatabaseAggregator

    Allows to an aggregation query against a table with configurable conditions, supports field tables. 
    /**
 * Implements hook_monitoring_sensor_info()
 */
function maillog_monitoring_sensor_info() {
  $info['maillog_records_count'] = array(
    'label' => 'Maillog records count',
    'sensor_class' => 'Drupal\...\Sensors\SensorDatabaseAggregator',
    'value_label' => 'Mail records',
    'settings' => array(      'category' => 'Mail',
      'table' => 'maillog',
      'time_interval_field' => 'sent_date',
      'time_interval_value' => 60 * 60 * 24,
    ),
  );

  return $info;
}

    SensorVariable

    Check a variable against an expected value, like checking if the maintenance mode is enabled. 
      $info['core_maintenance_mode'] = array(
    'label' => 'Maintenance mode',
    'description' => 'Site maintenance mode',
    'sensor_class' => 'Drupal\...\Sensors\SensorVariable',
    'numeric' => FALSE,
    'value_type' => 'bool',
    'settings' => array(
      'variable_name' => 'maintenance_mode',
      'variable_value' => FALSE,
      'variable_default_value' => FALSE,
    ),
  );

    Standardized settings keys

    • enabled
      Any sensor can be disabled, set to FALSE, to
      have a sensor disabled by default.
    • caching_time
      Specifies how long a sensor result can be
      cached/is valid.
    • time_interval_value
      The time interval the sensor is checking for.
      Used for the default sensor message, the
      database aggregator sensor also exposes
      it in the UI
    • thresholds
      Specify the default threshold type and intervals

    Subclass a sensor class to extend it

    Add more functionality or custom conditions to a sensor
    
class SensorUserFailedLogins extends SensorDatabaseAggregator {

  /**
   * {@inheritdoc}
   */
  public function buildQuery() {
    $query = parent::buildQuery();
    $query->addField('watchdog', 'variables');
    $query->groupBy('watchdog.variables');
    return $query;
  }

  /**
   * {@inheritdoc}
   */
  public function runSensor(SensorResultInterface $result) {
    $records_count = 0;

    foreach ($this->getQueryResult()->fetchAll() as $row) {
      $records_count += $row->records_count;
      $variables = unserialize($row->variables);
      $result->addStatusMessage('@user: @count', array('@user' => $variables['%user'], '@count' => $row->records_count));
    }

    $result->setValue($records_count);
  }

    Write a new sensor class

    From what should I extend?

    Examples

    class SensorQueue extends SensorThresholds {

  public function runSensor(SensorResultInterface $result) {
    $queue = \DrupalQueue::get($this->info->getSetting('queue'));
    $result->setValue($queue->numberOfItems());
  }
} 

class SensorSearchApi extends SensorThresholds {

  public function runSensor(SensorResultInterface $result) {
    $indexes = search_api_index_load_multiple(array($this->info->getSetting('index_id')));
    $index = reset($indexes);

    $status = search_api_index_status($index);

    // Set amount of unindexed items.
    $result->setValue($status['total'] - $status['indexed']);
  }
}

    Integration

    Looking at the sensor overview is fun, but you want to integrate the results in your monitoring software.

    Monitoring provides you with

    • Integration with Icinga/Nagios
    • Integration with Munin
    • Services
    • Drush

    ... and it's easy to integrate it yourself!

    Icinga/Nagios


    Active monitoring

    Every sensor is directly called by Icinga

    Passive monitoring

    All sensors are executed at once and sent to Icinga

    Munin


    Expose numeric sensors as graphs

    Supports multigraphs (Multiple sensors on the same graph)

    Services

    Sensor information and sensor results are exposed through services.module.

    GET /monitoring/sensor-info/dblog_event_severity_error
GET /monitoring/sensor-result

    Drush

    Sensor information and results can be displayed with drush commands.

    Support for printing results as JSON

    drush monitoring-run cron_core_last_run_agedrush monitoring-run --output=jsondrush monitoring-infodrush monitoring-enable <sensor>drush monitoring-disable <sensor>

    API


    Execute the sensors yourself and do whatever you want

    foreach (monitoring_sensor_run_multiple() as $result) {
 print $result->getSensorName() . ": " . $result->getStatus();}

    Drupal 8 / #D8CX

    Monitoring has been developed with D8 in mind.

    Uses the PSR-0 standard for classes with the help of the xautoload module.

    Config Entity & Plugin?


    MONITORING

    By Sascha Grossenbacher

    Made with Slides.com

    MONITORING

    • 3,701

    More from Sascha Grossenbacher