How to Integrate Splunk with PagerDuty

Introduction

Splunk collects and indexes data from just about any source imaginable, such as network traffic, Web servers, custom applications, application servers, hypervisors, GPS systems, stock market feeds, social media, and preexisting structured databases.

Splunk can be configured to pass all alerts to PagerDuty. Using PagerDuty, you can receive your Splunk alerts via phone call, SMS, or email; configure automatic escalation of alerts; escalate alerts right from your mobile phone; and set up on-call duty scheduling.

What you’ll need to get started

First set up Splunk. You’ll also need a PagerDuty account (either a paid account or a free trial account will work).

Using API integration:

Usage Overview

  1. Download & Install Splunk.
  2. Create a PagerDuty Service Integration API Key.
  3. Install this App.
  4. Set PagerDuty API Key.
  5. Enable Alert.

 

In PagerDuty:

  1. Create a PagerDuty service:
    SplunkAddServcie
  2. Note the Service API Key:
    SplunkAPIkey

In Splunk:

Phase I – Install & Configure App:

  1. Download & Install Splunk.
  2. From Splunk, select “Apps” and click “Find More Apps”:
    find_more_apps
  3. Search for “pagerduty”:
    search_for_PD
  4. Restart Splunk:
    restart_splunk
  5. After Splunk restarts, select “Apps” and click “Manage Apps”:
    manage_apps
  6. Locate “PagerDuty Alerts” and click “Set up”:
    app_set_up
  7. Enter your PagerDuty Service-API-Key and click “Save”:
    service_api_key

Phase II – Enable Alert:

  1. From Splunk, search for a term and click “Save As – Alert”:
    new_search
  2. Pick a name and schedule for the alert:
    save_as_alert
  3. Click “Run a Script” and enter “pagerduty.py”, then click “Save”:
    run_a_script
  4. Enjoy having Splunk Alerts delivered to PagerDuty!