This is a guest post by Ilan Rabinovitch, Director of Product Management at Datadog. The convergence of rapid feature development, automation, continuous delivery, and the shifting...by Ilan Rabinovitch
August 24, 2017
We had wide variety of Hack Day projects this month that showcased the unique talent we have here at PagerDuty. In this post, we’ll highlight three Hack Day projects: clapping for incidents, visualizing incidents, and giving thanks our on-call heroes.
Clap On! Clap Off! Incidents
The Clapper was one of the most popular ‘80s infomercial. With its highly memorable jiggle – Clap On! *clap clap* Clap Off! *clap clap* – The Clapper sang and clapped its way into millions of households. The Clapper is for all of us who like to have a little fun and stay comfortably where we are when turning on and off our lights.
Our PagerDuty real time intern Peter created The PagerDuty Clapper, a Raspberry Pi that integrates with PagerDuty via webhooks. Using redis, The PagerDuty Clapper queues notifications to be handled sequentially without impeding incoming webhooks. This ensures only one incident is being handled at any given time, while simultaneously accepting additional incidents. When an incident occurs, The PagerDuty Clapper reads the incident message out of it’s speakers, and then prompts the user with instructions to clap two times to acknowledge or three times to resolve. The PagerDuty Clapper then runs a custom built circuit to detect and count the number of claps in a given time period, and handle the incident accordingly.
Watch the video below to see The PagerDuty Clapper in action:
This is a perfect excuse to give yourself an applause after a job well done.
3D Incident Visualizer
Providing a high availability service for our customers is our calling. If our customers’ servers are down, it’s helpful for us to know the location so we can switch to a different provider. Our software engineer Ken plotted a map of the physical locations of servers sending events to PagerDuty for his May Hack Day project. Because of Ken’s love of maps and graphics, Ken worked with fellow software engineer Aleksey to take his IT IP Map App one step further by creating a real-time feed of incidents based on location.
How did they do it? They wrote a web service in Node.js that pulls log data from Sumo Logic, looks at the source IP for the each event, and performs IP geolocation using freegeoip.net to find where in the world the server is. To visualize the data, they used the Chrome Experiments’ WebGL globe. Out of the box, the WebGL globe can only display static data sets. Aleksey modified the globe source code so that it could display our stream of incoming events.
Future enhancements to the 3D Incident Visualizer are to show the locations of requests coming to all of our APIs, not just to our events API, and to show the locations of where we send SMS and phone calls (based on area code). The 3D Incident Visualizer is not only visually appealing but can also provide helpful insight into managing our customers’ incidents.
PagerDuty’s Unsung On-Call Heroes
We have heard many great stories from our customers around how their lives have changed with PagerDuty. Your wife/husband can sleep more soundly at night since you are getting woken up less. You can actually go to a bar while on-call without fear of missing an alert. And the happy stories go on and on.
PagerDuty’s on-call team has also been affected by PagerDuty. Our product marketing manager, Tony, and our customer success manager, Alisa, teamed up to produce a video to showcase our on-call superheroes. This videos gives our customers an inside look into the team who makes sure PagerDuty is up when our customers’ systems are down. Our on-call superheroes really care about delivering on our high availability promise.
PagerDuty customers, our on-call superheroes know your pain! They also get woken up in the middle of the night by PagerDuty when incidents occur. On their off time, these heroes are just like us ordinary folks. They enjoying going to concerts, restaurants, parks, or places to rock climb.
Watch the video below to get to know our superhero team:
This quick and dirty video was made to give you a peek inside PagerDuty. We were having fun with cameras and shot everything in an hour so we know this video is not high quality.
In the future we will develop more polished videos on PagerDuty team, processes, and products. What videos do you want to see from us?