How to Integrate Icinga with PagerDuty
This guide describes how to integrate your Icinga installation with PagerDuty using a simple Perl-based plugin. As Icinga is a fork of Nagios, we’ll be using PagerDuty’s built in support for Nagios for this integration. Note that you must be logged in as root to complete the installation. You might need to slightly alter these instructions depending on your exact Linux distribution and your Icinga configuration. If you are having trouble completing the installation, please contact us.
If you don’t already have a PagerDuty “Nagios” service, you should create one:
- In your account, under the Services tab, click “Add New Service”.
- Enter a name for the service and select an escalation policy. Then, select “Nagios” for the Service Type.
- Click the “Add Service” button.
- Once the service is created, you’ll be taken to the service page. On this page, you’ll see the “Service key”, which will be needed when you configure your Icinga server to send events to PagerDuty.
- Install the necessary Perl dependencies:For Debian, Ubuntu, and other Debian-derived systems:
aptitude install libwww-perl libcrypt-ssleay-perl
RHEL, Fedora, CentOS, and other Redhat-derived systems:
yum install perl-libwww-perl perl-Crypt-SSLeay
- Download pagerduty_icinga.cfg from github
- Open the file in your favorite editor.
- Enter the service key corresponding to your Nagios/Icinga service into the pager field. The service key is a 32 character string that can be found on the service’s detail page.
- Copy the Icinga configuration file into place and change owner:
cp pagerduty_icinga.cfg /usr/local/icinga/etc/objects
chown icinga:icinga /usr/local/icinga/etc/objects/pagerduty_icinga.cfg
- Add the contact “pagerduty” to your Icinga configuration’s main contact group. If you’re using the default configuration, open /usr/local/icinga/etc/objects/contacts.cfg and look for the “admins” contact group. Then, simply add the “pagerduty” contact.
alias Icinga Administrators
members icingaadmin,pagerduty ; <-- Add 'pagerduty' here.
- Download pagerduty_icinga.pl from github and copy it to /usr/local/bin.
cp pagerduty_icinga.pl /usr/local/bin
- Make sure the file is executable by Icinga
chmod 755 /usr/local/bin/pagerduty_icinga.pl
- Enable environment variable macros in /usr/local/icinga/icinga.cfg (if not enabled already)
- Edit the icinga user’s crontab
crontab -u icinga -e
- Add the following line to the crontab
* * * * * /usr/local/bin/pagerduty_icinga.pl flush
- Restart Icinga
What if a Icinga event happens while my network is down?
If a PagerDuty server can’t be reached for any reason, events will be stored to an on-disk queue. The installed cron job will attempt to re-send the events at one minute interval.
If a PagerDuty server can’t be reached for any reason, events will be stored to disk and the at one minute intervals.
Since Icinga needs my external Internet connection to send failure reports to PagerDuty, how will I receive notification if our site loses external connectivity?
You should configure an external ping check service like Pingdom to monitor your site’s external connectivity. Of course, you can use PagerDuty to forward alerts from these services.
It doesn’t seem to be working. What’s going on?
Check the syslog for messages from “pagerduty_icinga”. On most systems:
grep pagerduty_icinga /var/log/syslog
Please contact us if you’re unable to sort out the difficulty.
What sort of Icinga messages does PagerDuty understand?
PagerDuty can process PROBLEM, ACKNOWLEDGEMENT, and RECOVERY messages. All other messages, including FLAPPINGSTART and FLAPPINGSTOP, are ignored. If you’d like PagerDuty to process additional Icinga messages, please let us know!