Splunk Integration Guide

Splunk collects and indexes data from just about any source imaginable – network traffic, Web servers, custom applications, application servers, hypervisors, GPS systems, stock market feeds, social media, and preexisting structured databases.

This integration sends alerts to PagerDuty using Splunk’s native webhooks. If you would prefer to use our older, Python-based integration, you can find that here.



In PagerDuty

  1. From the Configuration menu, select Services.

  2. On your Services page:

    If you are creating a new service for your integration, click +Add New Service.

    If you are adding your integration to an existing service, click the name of the service you want to add the integration to. Then click the Integrations tab and click the +New Integration button.



  3. Select your app from the Integration Type menu and enter an Integration Name.

    If you are creating a new service for your integration, in General Settings, enter a Name for your new service. Then, in Incident Settings, specify the Escalation Policy, Notification Urgency, and Incident Behavior for your new service.

  4. Click the Add Service or Add Integration button to save your new integration. You will be redirected to the Integrations page for your service.


  5. Copy the Integration URL for your new integration.

In Splunk

  1. First you’ll need to install the PagerDuty Incidents app from the Splunkbase. You can find the app by first clicking on the box with the + on it on the left side of the page.
  2. Search for the PagerDuty App for Splunk and then click Install.
  3. Once the app has been installed, head to the Settings menu and choose Alert actions.
  4. Make sure that the PagerDuty app is Enabled and then click Setup PagerDuty Incidents.
  5. Paste the Integration URL for your PagerDuty service into the supplied field.
  6. Run the search in Splunk for which you’d like to create an alert. We recommend testing by searching Splunk’s internal logs for failed login attempts: "index=_internal component=UiAuth action=login status=failure"
  7. Click Save As and select Alert.
  8. Add a Title for the new alert, specify the conditions under which you’d like it to trigger an alert and add a new Trigger Action. Select PagerDuty as the trigger action type. By default, this will notify the integration URL you established when you set up the PagerDuty app.splunkalerts-09


Testing your integration

We recommend testing your integration with a simple search that is easy to manipulate, like the above-referenced:
"index=_internal component=UiAuth action=login status=failure"

This will produce results any time there is a failed login attempt and is an easy situation to reproduce.

Set your search either to run on a schedule or to be run in real-time and set your threshold low (Number of results is greater than 0, for example). Open Splunk in another browser or in an incognito window and make some failed login attempts.

Not long after, you should see a PagerDuty alert.


Clicking to the incident details will provide a breakdown of the alert and will provide you with a link to view the search in Splunk.


Frequently Asked Questions

Can Splunk send alerts to more than one PagerDuty service?

Yes. If you are interested in notifying a different Splunk service in PagerDuty, you can paste that service’s integration URL in the optional Integration URL field under Trigger Actions when creating a new alert. This will override the global PagerDuty Integration URL established when you set up the PagerDuty Incidents app.


Will PagerDuty incidents resolve once the Splunk search is no longer producing results?

No. At this time, the PagerDuty incident must be resolved from PagerDuty.

How does PagerDuty group incoming alerts from Splunk?

If you select Edit Service from the main service view, you’ll be able to choose to group incidents by search name, component, host, source or to attach all incoming alerts to an open incident.