Why We Use On-Call Shadowing On-call shadowing is an essential practice at PagerDuty. For a new engineer, a shadowing period serves as a kinder, smoother...by Max Timchenko
March 26, 2019
As every business evolves into a digital business, and micro-moments matter more than ever to both revenue and brand equity, the need for effective communications in the face of crisis has grown. There is, of course, more pressure on the technical teams who develop and manage these services, but digital transformation is also putting pressure on other parts of the business that are focused on brand reputation and customer experience.
Enter the communications, marketing, and support teams (among others) who have a responsibility to customers to ensure they are getting the most value out of a given product or service. With this pressure comes the need for new workflows and best practices within external communication functions. Taking what we’ve learned first-hand at PagerDuty, below you’ll find four best practices that we have honed as a part of our incident response documentation and beyond.
As a member of the communications team here at PagerDuty, I find myself on-call alongside my technical colleagues and many others within the business. Although I get a fraction of the incident notifications that they do, I have a greater appreciation for why automation and having an IT incident response process is necessary for smooth operations, which keep today’s digital services humming. When something does impact customers, the communications team and other stakeholders from support, sales, legal, and leadership need to know so we can proactively take the right customer communications measures, as well as efficiently do what’s right for our brand promise when the clock is ticking. From our point of view, it is everyone’s responsibility within the organization to support our customers, and that’s why we all go on-call.
Our award winning support team is on the front lines of customer response when we have a major incident, but the communications team stays in lockstep with them to understand:
Within the incident response process, teams need a Customer Liaison, who has training on how to best communicate externally when something has gone wrong within an application or service. Our communication team works with the Customer Liaison to ensure we have an accurate understanding of what is happening and the impact. Within PagerDuty, in addition to being a part of the incident Slack channel, we automate updates with Stakeholder Engagement. We form a tight line of collaboration between the right people from these teams, working together to craft meaningful communications for customers and the broader market.
If you’re using traditional ticketing or ITSM workflows, chances are you’ll lose the opportunity to be proactive. Your customers will know there is something wrong before you do, and you’ll have lost the opportunity to fix it before they notice or get on a public forum like social media — or the even the news. From a communications response perspective, you have to move quickly to craft external messaging so you can meet frustrated tweets and reporter calls with correct information. In this phase of incident communications, it’s critical to form the strategy for repairing reputation damage, and ideally, you can share what is being done in real-time to make the situation right. To help in proactive communication, we’ve identified possible crisis scenarios and created the workflows, and best practices around language to use based on what we’ve seen in the past. Of course, crisis scenarios are unique, but having built a foundation in peace time helps us move faster in war time when bandwidth is tight and the stress is high.
The work of the communications team doesn’t end once the incident is resolved. Within an ideal incident management lifecycle, a major incident warrants a post-mortem analysis in which relevant stakeholders across the business who were involved discuss what happened, how the response was executed, and what can be improved moving forward. This is the time for learning and getting better. For communications teams, we must work across the organization to develop and execute a plan to renew the trust of customers and the broader community. That might mean carrying out a strategy to make good where the brand may have failed to live up to expectations. The corporate narrative will also have to be revisited and thought put into where it goes next using sound judgment. This is also the time for commitment to why you’re in business and ensuring behavior is consistent and not just talk. Action across the business may be necessary to turn a wrong into a right.
In today’s micro-moment-dominated, always-on world, communications teams, alongside other teams within the business, must be laser-focused on digital operations excellence in order to be successful. This means a different kind of vigilance, but it also means a new way of working and collaborating across functions with the unified goal in mind of delivering exceptional customer experiences.
At PagerDuty Summit, taking place on September 7th in San Francisco, we’ll continue to explore what whole-business response means in the digital era, with leaders from all parts of the organization discussing how they protect revenue, brand reputation, and customer experience with digital operations management best practices. You won’t won’t want to miss it — join us at PagerDuty Summit by registering today!