Customers / Indiana University
  • Size
  • 19,000+ employees

  • Industry
  • Higher Education

  • Location
  • Bloomington, Indiana

  • Customer Since
  • 2016

Challenges

  • Redundant alerts causing noisy responder experiences and increasing alert fatigue
  • Large amounts of manual or administrative work associated with on-call management

Business Impact

  • Decreased manual/administrative workload when incidents occur
  • Freed the team to spend more time innovating to deliver increased customer value
  • Reduction in alerts has led to improved employee productivity and happiness

Benefits With PagerDuty

  • Established a flexible and self-service incident management solution to manage their digital operations
  • Improved the team’s quality of life, happiness, and productivity
  • Empowered better delivery of services

With PagerDuty, Indiana University delivers a higher quality of service and improves the lives of on-call employees  

Indiana University (IU), with over 100,000 students across eight campuses, is the largest public university in the state of Indiana. IU has a flagship campus in Bloomington containing most of its liberal arts and doctoral programs, and another campus in Indianapolis that is home to their medical and dental schools, among others. To support this dispersed and growing university, IU has a centralized IT group that largely resembles a corporate IT organization in its structure.

Jacob Farmer, Manager of Identity Management Systems, is part of University Information Technology Services (UITS), the IT organization at IU. Farmer oversees the team that provides application and integration expertise. They own several enterprise-grade applications that both send and ingest high volumes of data, both on site and in the cloud. As technological complexity often gives way to increased vulnerabilities, the team saw significant volumes of alerts when monitoring their application stacks. IU was in need of an incident management solution that could enable full-stack event intelligence to streamline and triage alerts across the infrastructure. This would allow their team to be more productive, decrease time spent on manual or administrative tasks, and maximize operational agility.

Minimizing on-call administrative workload and alert noise

Before launching PagerDuty, Farmer and his team dealt with challenges that hindered on-call productivity. They struggled with incidents being acknowledged and resolved outside of their control, excessive numbers of quality and assurance checks, and administrative tasks involved in rotating on-call schedules. “The team really disliked the minutiae of remembering rotating on-call schedules and going into the list serve to adjust the descriptions—it became a drag,” said Farmer. Overall, managing administrative tasks and high volumes of alerts had a negative impact on the team’s quality of life and efficiency. Farmer and his organization sought to improve these issues.

Providing a flexible and self-service solution that increases productivity  

IU selected PagerDuty as their incident management solution to overcome roadblocks that created additional administrative work. After diligently researching vendors within the space and comparing the value of various solutions, it became apparent that PagerDuty was the industry leader in agile incident management. Today, Farmer’s team uses the PagerDuty platform to manage the end-to-end incident management lifecycle, enabling continuous delivery and improved application performance.  This includes everything from full-stack event intelligence and alert triage, to on-call automation and response orchestration with scheduling and escalations, to post-mortems and analytics. Depending on the criticality of the system, escalations are handled in different ways: with a primary person on call, a back-up, and a last resort if the issue isn’t resolved within a certain timeframe. PagerDuty manages and automates on-call rotations, overrides, and escalations, enabling far more flexibility and efficiency. “PagerDuty handles all of the administrative details automatically, offering a self-service model to make it easy. Frankly, I don’t even know who is on call anymore. You can ask for duplicate coverage and the solution provides what you need. PagerDuty has made my life easier as a manager by improving the experience of our on-call team. It’s the best situation for us—everyone wins,” stated Farmer.

Many IT tools and applications are monitored within UITS at IU. They leverage the tool WhatsUp and applications such as Apereo Central Authentication Services (CAS) single sign-on, Shibboleth SAML Identity Provider, and Microsoft’s Identity Manager. If single sign-on or the identity provider application breaks, an alert is automatically generated and triaged by the operations staff, and an incident creation email is fired off to PagerDuty. PagerDuty enables the teams to acknowledge the incident and resolve it in a timely fashion, enabling improved application uptime, availability, and performance.

The Microsoft Identity Manager is the hub for all quality and assurance checks. When an alert occurs, certain levels of severity will be pushed through PagerDuty. If it’s urgent, an incident that represents a real business-impacting problem is automatically created and notifies a responder in real-time by leveraging the schedule and escalation rules. If the event isn’t urgent or actionable, the team is able to suppress it or define it as low-urgency, while leveraging that data to proactively identify leading edge indicators of problems. PagerDuty has taught Farmer and his team to be very thoughtful about their automated quality and assurance checks. The number of Q&A checks have decreased, lowering the alert noise and fatigue within the team and increasing efficiency.

“PagerDuty removes a very significant job satisfaction impediment for the people on-call, which is extremely valuable.  Also, coincidentally, it causes those on-call to take ownership of the incident management process,” said Farmer. Not only does the solution offer a platform that enables teams to be more productive and focus on critical incidents, it also benefits other departments at IU. For example, PagerDuty has improved Farmer’s Identity Management Systems on-call onboarding process. The team has implemented a mentoring program in which a new team member can shadow another person’s on-call schedule within PagerDuty, rather than get thrown straight into the line of fire. This allows them to easily add new people to the rotation, and creates a great way to ramp up their new team members.

Advancing quality of life, continuous delivery, and operational efficiency  

PagerDuty has provided Indiana University with flexibility, significantly decreased alert noise, and increased employee agility and efficacy across the incident lifecycle for improved application delivery and end user experiences. This has enabled the team to cancel out redundant or non-actionable alert noise, and direct their focus to service-impacting problems.

“PagerDuty improves our delivery of service and makes our lives easier. We value our team’s time and happiness—we want people to stay!” said Farmer. The solution has enabled IU’s on-call teams to manage incidents in an operationally-efficient way, in turn positively affecting both their productivity and quality of life.

 

“PagerDuty has made my life easier as a manager by improving the experience of our on-call team. It’s the best situation for us—everyone wins”

Indiana University
Jacob Farmer, Manager of Identity Management Systems at Indiana University