Why We Use On-Call Shadowing On-call shadowing is an essential practice at PagerDuty. For a new engineer, a shadowing period serves as a kinder, smoother...by Max Timchenko
March 26, 2019
I’m proud to be working for an Engineering organization that feels safe. Safe for its engineers to bring their authentic selves to work and discuss sensitive topics like: “Why do we keep cutting each other off in team meetings?” or “Why do I get stuck with all the boring work?” Safe for all kinds of engineers (including under-represented groups, person(s) with a disability, remote workers, and introverts) to feel at home. And safe without being “too nice,” which we do by prioritizing Getting Stuff Done (GSD) and calling each other out when we fall astray.
Our team is obsessed with GSD. Solving big problems for PagerDuty and moving the needle forward on our product are intrinsically motivating for us. We think strategically about how we are spending our time so we can get the most “bang-for-buck” from our efforts. Too much process and red tape are demotivating as they slow us down and provide less value to our customers.
You might ask, how do we, PagerDuty Engineering, find our sweet spot between creating a safe space and GSD? By sweet spot, I mean the place where the team is operating in a “full-speed ahead + light on process” pace while still taking the time to slow down and listen to one another.
First, we start with an intrinsically motivating culture and hiring awesome people that already fit our values. Next, our teams take bottom-up initiative with frequent feedback loops (e.g., team retrospectives, team health checks, etc.) to ensure they are evolving their processes for the best long-term results.
Each Delivery Team at PagerDuty operates in their own unique way and expresses their values in their own way, so it’s really hard to talk in generalizations about the entire Engineering organization. To tell a real story, I’ll be covering the journey of one team here and the lessons learned along the way.
This team started as two teams. Both teams had their own sets of cultural norms and values, and owned a different set of things at PagerDuty. I was the Agile Coach of them both when we made the decision to merge them based on the needs of the organization. When they were combined, there was some friction and identity crisis—as is usual for most new teams—as they moved through the stages of Forming, Storming, Norming, and ultimately, Performing.
This new team was given a challenging project with a short timeline to deliver on, and it was through this and other challenges that they discovered their team values of Vulnerability, Inclusiveness, and GSD. I’ll be focusing on each of these values and how the team learned to live them at a deeper level as a response to the challenges that they faced.
To be vulnerable, the team must feel safe, which means having safe spaces to talk freely about whatever they want. One example of this is a private Slack channel for the team. This is an invite-only channel, and it’s a place where the team can converse, tell jokes, share memes, and blow off steam, all in their own unique ways.
Another example of a safe space is the retrospective meeting, which is the primary feedback mechanism for the team. As this is arguably the most important team meeting, it’s ideal that their manager attends. However, since we prioritize creating a space where the team can be vulnerable, process “raw emotions” collectively, and collaborate on areas for improvement without outside interferences that could detract from the goal of the meeting, we anonymously poll the team to see if they would like their manager in the retrospective meeting. If anyone votes “no,” the manager is not invited. We revisit this decision periodically, as the ideal situation is a culture where the team trusts their manager to attend retros.
Vulnerability within our team is also about sharing one’s personal journey and struggles. Everyone is encouraged to share in order to gaining empathy and understanding from the team of one’s unique situation. Maybe you were paged at 2 a.m. and will be working from home the next day. Maybe your loved one is sick and you need to travel to be there for them. We’re here for you, and we can only be this open with each other because we’ve created an empathetic, guilt-free culture. There’s no expectation to work when you have a higher-priority personal situation to deal with (that’s why you’re part of a team).
Inclusiveness means that PagerDuty is a place where we can grow to be the best versions of ourselves. Think of inclusiveness in its broadest sense: We want PagerDuty to feel like home for everyone.
Every team member, regardless of seniority, has a voice inside our team. We dug deep into what it means to be inclusive, uncovering numerous learnings around creating a culture that is all-encompassing.
For example, our team is split between San Francisco and Toronto, a situation that led to a number of inclusion challenges, which meant we had to come up with solutions that help us all work better, including:
To make our meetings more introvert-friendly, we send out a Trello board several days before a meeting and ask folks to add their discussion topics, which are later voted on democratically by the full team, in Lean Coffee style. When we noticed that some meetings (including retrospectives) would have a “meeting after the meeting” in the hallway, we decided to extend those meetings and spend 10 minutes at the end having a meta discussion (e.g., “retro on the retro”) so everyone could join in.
Additionally, to create a healthy team culture inclusive of all locations, we have monthly “team parties,” where we try not to talk about work stuff. Instead, we laugh at each other’s drawing skills while playing Gartic.io. We watch movies together. We share snacks and grab a drink from our on-tap kombucha/beer kegs (depending on time of the day 😉).
Then there’s the work itself: To ensure some folks don’t get stuck with less-exciting work, our Kanban team pulls tickets from the top of the To Do column instead of selecting tickets from a project they are already working on. This also ensures we don’t unintentionally create knowledge silos within our teams.
To create some autonomy for the team to choose their ticket, we created something called the Golden Ticket. Each replenishment cycle, the team adds a Golden Ticket to the To Do column on the Kanban board. These are special tickets, which allow a team member to pull in any ticket they’d like to work on from the backlog.
One challenge we have had with GSD on our team is taking the pendulum swing too far; e.g., being too quick to shoot down ideas that would result in additional process. To solve this, we created a three-step idea-vetting method for team members:
When following this method, we found that it’s important to be explicit about what we need—full team consensus or just majority? When it makes sense, we use anonymous Slack polls to make a decision around implementing the idea. We’ve also found that it’s important to stay mindful of new team members that join, and that they may not have the same shared context as other team members. When possible, we try to bring these new team members up to speed, around the “how” and “why” behind the way the team operates.
Since discovering our team values and learning to live them at this deeper level, we have improved the majority of our health indicators, which are tracked through PagerDuty Engineering’s Quarterly Team Health Check Process. We’ve also delivered significant value to our customers, in terms of releasing new features.
So what’s our “secret sauce?” Add a pinch of Vulnerability by creating safe spaces and sharing our personal struggles, a dash of Inclusiveness by fostering a team culture that feels like home, and a smidgen of GSD by delivering the most “bang-for-buck” while staying open to process improvements.
There you have it: Take one daily dose of this “secret sauce,” and you, too, can have a delivery team culture that’s light on process, heavy on love.