With all features needed for system monitoring and error recovery. It's like having a watchdog with a toolbox on your server
Barking at daemons
Monit is controlled via an easy to configure control file with a simple syntax. The file is called monitrc and resides in your home directory. Configuring Monit is just text editing.
An example monitrc file is included with the Monit download. Open it in your favourite editor and follow along..
Download either source or binary, or use your package manager to install a recent version (5.8 or later)
Start Monit from the command line:
$ monit quit
In addition to the command line interface, Monit also provides a super lightweight Web-Interface you can use to check the status of all monitored services
(1) check host mmonit.com with address mmonit.com (2) if failed port 80 protocol http then alert (3) if failed port 443 type TCPSSL protocol http then alert
check host mmonit.com with address mmonit.com if failed port 80 protocol http and status = 200 and request /monit/ with content = "Monit [0-9.]+" then alert
check host smtp.example.com with address smtp.example.com if failed port 25 with protocol smtp then alert
See the pattern? A simple check against a host address on the service port and then the protocol to test. Here's another example checking your MySQL server on localhost:
check host localhost with address 127.0.0.1
if failed port 3306 protocol mysql then alert
if failed port 25 and expect "^220.*" send "HELO localhost.localdomain\r\n" expect "^250.*" send "QUIT\r\n" then alert
For this example, let us assume that your web-server is Apache, if not, no worries, the same applies to any process. Here is how we can check if Apache is running
check process apache with pidfile /var/run/httpd.pid
That's all, Monit will check the process id (pid) in the given pid file and consult the system's process table and check if the process exist. If not, Monit will raise an alert. Next, we'll see how Monit can be configured to automatically start or restart Apache if it is not running.
check process apache with pidfile /var/run/httpd.pid start program = "/etc/init.d/apache2 start" stop program = "/etc/init.d/apache2 stop"
Tell Monit which program it should use to start the process if it is not running. We also add a stop program which Monit will use to make sure the process has stopped before it calls start. What is useful here is that these programs already exist on your system as init, upstart or systemd scripts and you do not have to write them yourself.
Now, if the Apache process has stopped for some reason, Monit will call the start program to automatically start Apache again.
check process apache with pidfile /var/run/httpd.pid restart program = "/etc/init.d/apache2 restart" if failed port 80 protocol http then restart
if failed port 80 protocol http for 2 cycles then restart
You can check resources used by a process and perform actions if values goes outside specific bounds
check process apache with pidfile /var/run/httpd.pid <...> if cpu > 95% for 2 cycles then alert if total cpu > 99% for 5 cycles then restart if memory > 50 MB then alert if total memory > 500 MB then restart
The following resources can be checked, cpu, memory, total cpu (this, plus child processes cpu usage) and total memory (the combined memory used by this and child processes). See the manual for even more resources that can be checked.
Having start, stop and restart programs defined for a service is convenient if Monit is used from the command line. For instance to shutdown a set of services graceful at once
$ monit -g www stop
The above command line will call the stop program of all services belonging to the group www, which is quite useful for system administration. You can of course do the same to start or restart a single service or a set of services.
Monit, like any monitoring system really, can be noisy and send a lot of email alerts. This can be very annoying, but luckily you can take steps to prevent this. Your primary tool for tuning Monit is the for n cycles statement
if total memory > 500 MB for 3 cycles then alert
Use this statement with if-tests that produces too many alerts. How many cycles you should wait, if any, depends on the importance and stability of the service and how long a cycle is
To keep with the Apache example. Let's say we want to test and perform a log rotate if Apache's access.log becomes too large
check file access.log with path /var/log/apache2/access_log if size > 250 MB then exec "/usr/sbin/logrotate -f apache"
Normally you will run the logrotate program once per day from crond, but if the log file suddenly should grow very large, Monit can perform an “emergency” rotate.
For security reasons you may want to monitor files for changes and get a notification if a file was tampered with (i.e. changed). The way to do this is to use the check file statement and utilize a checksum test. You can also test other attributes, such as owner and access permissions
check file apache with path /usr/sbin/httpd if failed checksum then alert if failed uid root then alert if failed gid root then alert if failed permission 755 then alert
Here we test the apache binary. The checksum test in line 2 will raise an alert if the binary was changed
Directories can be monitored for changes. In this example we use timestamp to alert if content was added or removed
check directory certificates with path /etc/ssl/certs/ if changed timestamp then alert
You can also turn the table and monitor a directory that should change
check directory incomming with path /var/data/ftp if timestamp > 1 hour then alert
If your server spool incomming data on a regular basis, an alert can be raised if no new data has arrived in the last hour
Monitor a hard disk or a filesystem and raise alerts if the device is about to run out of available space
check filesystem disk2 with path /dev/disk2 if space usage > 95% then alert
Using M/Monit you can then also view charts of your disks and predict when it is time to invest in more space
Monit can execute a program and perform an action if the exit value indicates an error. This means that you can use Monit to perform basically any type of check you can write a script for. For instance, lets say you have a script that connects to MySQL and runs a query. If everything is okay, the script exit with 0 otherwise with 1
check program salesreport with path /var/monit/programs/sales.sh every "* 8-9 * * 1-5" if status != 0 then alert
This will run the sales report check every weekday between 8AM and 9AM. Please see the manual for more information