Customers / Fog Creek Software
  • Size
  • 40

  • Industry
  • Software

  • Location
  • New York City, NY

  • Customer Since
  • May 2010

Benefits With PagerDuty

  • Highly resilient and reliable system
  • Confidently receive all alerts
  • Individual customization of alerting style – email, SMS, phone, mobile app push notifications at the desired times
  • Easily integrates with all existing monitoring systems

Fog Creek Software: Better Software Through Better Alerting

Fog Creek Software was founded on the principle of providing programmers with a better working environment in order to recruit the best engineers and create the best products. Since its inception in 2000, Fog Creek has continuously posted profits, without any outside investments.

Fog Creek focuses on three main products: Trello, an online collaboration tool that lets you see at a glance who is working on each project and what the project’s status is; FogBugz, a bug-tracking project manager; and Kiln, a source code version control system for Mercurial and Git. Being highly SaaS-focused with over 450,000 monthly users, limiting downtime for its products is essential to Fog Creek’s longevity and profitability.

“PagerDuty has made my job easier in so many different ways.”

Early Fog Creek Challenges

With its rapidly growing SaaS products, Fog Creek quickly needed more complete systems monitoring. FogBugz and Kiln, for example, require an immense amount of queuing because everything the user does must be properly approached and kept up with backend requests in Mercurial. These demands can be difficult to satisfy and they require several monitoring systems. Fog Creek uses Pingdom to keep an eye on its external software and Nagios to monitor its overall IT infrastructure. It also uses multiple homegrown monitoring systems to keep track of disk space and CPU load.

Fog Creek was relying on each of these individual systems to alert the on-call engineer via email and SMS only, but often the alerts were ineffective and issues were unacknowledged for long periods of time. Because the company’s monitoring systems relied only on email and SMS alerts, Fog Creek had difficulty ensuring that every alert contacted the right person. These unacknowledged incidents endangered the company’s reputation and threatened its customer retention. Bradford Ley, Fog Creek Systems Administrator, says that Fog Creek lacked a “properly designed monitoring and alerting system. Once something goes wrong, it would have been easy to spiral out of control.”

How Did PagerDuty Help?

An effective monitoring and alerting system was crucial to Fog Creek’s success. PagerDuty’s ability to easily integrate with the company’s existing monitoring systems made it simple for Fog Creek to coordinate its monitoring systems and provide fail-proof alerting. Since implementing PagerDuty, Fog Creek has reduced its downtime and no longer worries about missing any alerts. “PagerDuty lets us confidently receive alerts and it has proven to be highly resilient,” Ley says. PagerDuty’s infrastructure is fully replicated in multiple data centers with fast failover when problems occur. This means PagerDuty limits costly downtime to ensure maximum availability.

“You have no idea how much happier we are with PagerDuty. Just no idea.”

PagerDuty allowed Fog Creek to customize, at specific time intervals, how each on-call engineer would be contacted: through email, SMS, phone calls, and iOS or Android push notifications. With PagerDuty’s system in place, no alert will go unacknowledged, day or night. If no one responds to the first alert, PagerDuty escalates the alert to a different engineer so the problem can be resolved quickly.

Before using PagerDuty, Fog Creek’s on-call engineers may have slept less for fear of missing an alert. In actuality, with PagerDuty, Ley believes they are sleeping through alerts less “because we’re actually getting notified when something breaks instead of blithely sleeping through an outage. PagerDuty reliably keeps us up at night.”