Turn any signal into insight and action. See how PagerDuty Digital Operations Management Platform integrates machine data and human intelligence to improve visibility and agility across organizations.
Check out the latest capabilities we released.
Flexible schedules, escalations, & alerting
Automated, best practice incident response
Powerful context & noise reduction at scale
Quantify real-time business & technical impact
Improve with modern, prescriptive insights
Over 300 Integrations
Discover DevOps best practices with our library of webinars, whitepapers, reports, and much more.
Learn best practices and get support help with resources from our award-winning support team.
See how PagerDuty works with our live product demo — twice a week, every week.
We've created a maturity model to assist on the journey to digital operations excellence. Take our short assessment to find out where your team falls!
Interactive, simple-to-use API and technical documentation enables users to easily try updates and extend PagerDuty.
Engage with users and PagerDuty experts from our global community of 200k+ users. Become a member, connect, and share insights for success.
Get all your PagerDuty-related questions answered by exploring our in-depth support documentation and community forums.
In part 2 of our postmortem series, we dig into how to establish a culture of continuous learning, from getting leadership on board to invoking...
PagerDuty helps organizations transform their digital operations. Learn more about PagerDuty's mission and what we do.
Meet our experienced and passionate executive team.
We are risk-taking innovators dedicated to delivering amazing products and delighting customers. Join us and do the best work of your career.
With the PagerDuty Foundation, we are committed to doing our part in giving back to the community.
Today, at around 1am Pacific Time, Amazon began having major problems with some of their cloud infrastructure: specifically with their EC2, EBS, and RDS offerings. The issues are ongoing, and many of your favorite internet sites or services are probably still down or at reduced functionality because of it.
This kind of outage is one of PagerDuty’s big “moments”; when a big chunk of the services on the internet say: “Hey PagerDuty, I’m down, so wake someone up to fix me!”
There’s lots of coverage on this issue already out there, so we won’t go into much detail on the AWS situation itself. But we’d like to share some statistics on the alerts we sent out – via phone or SMS – during the outage. We think these numbers could shed some light on what proportion of the internet was affected by the issues. We don’t presume that we’re being used (yet!) by a “huge”, “moderate”, or even “realistically statistically significant” proportion of internet websites or SaaS providers, but we think these numbers are definitely interesting and can be taken on the whole as a sort of pain metric for this AWS outage.
Since the outage began, we have routed notifications to around 36% of our customer base. In other words, 36% of PagerDuty customers have been having issues – ones big enough to actually page one of their sysadmins or engineers to work on the problem – since the AWS problems began.
Most PagerDuty customer accounts have more than one user – sysadmin, engineer, “ops guy”, etc – involved in their on-call rotations. We have paged more than 10% of our entire user base. In other words, more than 10% of all of our customers’ operations staff have been woken up and/or called in by our systems to work on their problems. This is probably just the tip of the iceberg as well, since we usually just handle the first alert; these AWS problems are probably causing a lot of “all-hands-on-deck”-type situations where the entire ops teams (and more) will be called in to fight fires after the on-call has been woken up by PagerDuty.
Below is a graph of the number of alerts – phone, SMS, and email – that we have sent out over the last 48 hours. There was a big spike in outgoing alerts around the time of the AWS outage, and alert levels have remained high since.
PagerDuty outgoing phone/SMS/email alerts during AWS outage
Below is a graph of the number of “events” being sent to PagerDuty by our customers’ monitoring systems, through our API or through email. We don’t send out phone/SMS/etc alerts for every “event” that gets sent to us by monitoring systems, but we de-duplicate them so as not to overwhelm our already-harassed and bleary-eyed users. As you can see, we were flooded by a huge number of events when the outage first began, and incoming event levels are still high.
PagerDuty incoming events during AWS outage
Voices wield power. Staying silent is not an option. We must speak up and honor those who do. October is National Domestic Violence Awareness Month,...
“Chaos Engineering is the discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in...
600 Townsend St., #200
San Francisco, CA 94103
905 King Street West, Suite 600
Toronto, ON, M6K 3G9, Canada
1416 NW 46th St., St. 301
Seattle, WA 98107
5 Martin Place
1 Fore St,
London EC2Y 9DT
© 2009 - 2019