Why We Use On-Call Shadowing On-call shadowing is an essential practice at PagerDuty. For a new engineer, a shadowing period serves as a kinder, smoother...by Max Timchenko
March 26, 2019
Since we launched on-call handoff notifications, lots of our customers have used them to be notified about their on-call responsibilities to make sure they never forget when they’re on-call. Over the years, we’ve seen a variety of on-call schedules and thought we’d share some of the more favored practices we’ve seen.
Below is a distribution of all start and end times for on-call shifts scheduled within PagerDuty:
The most popular time to handoff your on-call shift is Midnight, followed by 8:00 AM and 9:00 AM, then 5:00 PM and 6:00 PM. Despite the popularity of the midnight swap, as PagerDuty, we recommend having your handoff occur during business hours, preferably when both parties are present in the office. Unless you both happen to be in the office at midnight, in which case, go home.
Switching your shift while on-site gives you the opportunity to talk to the next person going on-call about any issues that occurred during the previous shirt or to give a heads up on anything they may want to be on the look-out for.
At GREE, their team syncs up every Monday morning to review alerts from the previous week, go over the upcoming schedule and handoff the rotation to the next on-call team. This gives each team additional insight into the week ahead and makes sure that everyone knows who is the primary, secondary and manager responsible for keeping GREE reliable each week.
Below is a graph of shift handoffs distributed by day of the week:
By exchanging on-call responsibility while on-site, we’d want to see less shifts occurring over the weekend. This distribution closely aligns to our scheduling philosophy at PagerDuty. By switching shifts on Monday, you are able to recount an entire week of data with minimal confusion.
Or you can schedule your shift exchanges during your weekly team meetings. This will still allow you to review information and give a heads up to any potential problems that may be faced.
Another hot topic in terms of on-call scheduling is shift length. Should you switch weekly? Daily? Hourly? While much of this may depend on the size of your team, you will also want to consider other factors. You may want review your historical alerting data to see if there are any hot times in your systems to make sure no single person is getting the short end of the stick, leading to burnout.
Below is a distribution of popular shift lengths from 1 hours up until 2 weeks:
The most popular shift lengths seem to be 8 hours, 12 hours, 1 day and 1 week. Keeping simple shift lengths means less confusion and forgetfulness about when you begin or end a shift.
So when should you use each of these shift lengths?
At PagerDuty, each internal team handles on-call scheduling differently. Our Operations Team has a simple weekly rotation, while our Realtime Team has a weekday / weekend rotation where people are on call during the week, then on call during the weekend. This is because our Realtime Team is slightly larger than our Operations Team, so they wanted to have team members on call more often and for shorter shifts to prevent operational tasks from getting rusty.
Our actual Real-Time team schedule:
It’s important to find a process that works for your team. Even within a single organization, different departments may find one approach works better than another. PagerDuty’s On-Call Schedules give you the ability to customize your team’s shift however works best for you. If you’re not quite sure where to start, just remember the basics: switch shifts when both parties are present in the office (if possible) and maintain a standard shift length (e.g. 12 hours, 1 days, 1 week) to help avoid confusion for when someone’s shift may be starting and ending.
To make this transition more convenient, we also began offering heads up time for when you start and end your shift. Simply log into your PagerDuty account and edit your On-Call Notification Rules from your profile page to get started.