Space Made Simple: How PagerDuty Enabled Loft Orbital to Achieve Incident Response Lift Off
The next great space race is on. Today, there are multiple companies competing to earn their slice of a global space industry set to be worth more than $1 trillion by 2040. However, launching a satellite into space still isn’t an option for most organizations due to the prohibitive costs and complex engineering required. Now, thanks to innovative satellite-as-a-service company Loft Orbital, any organization can buy a ticket on a shared satellite and launch data capture technology into space at a fraction of the cost of doing it themselves.
Launched in 2017, Loft Orbital’s satellite-as-a-service business model is shaking up the space industry. Loft Orbital flies customer payloads onboard regularly scheduled satellite launches and handles the entire mission as a service. For example, suppose a customer wanted to send a camera to space to analyze weather. Loft Orbital would add the camera onto one of its upcoming satellite launches and take care of the data collection process once the camera is in space based on customer requests.
Understanding The Gravity of the Situation
John Murray is a senior member of Loft Orbital’s satellite operations and software engineering team. He helps build and manage the company’s proprietary ground control software, Cockpit—an all-in-one solution for satellite operations. Engineers control everything through Cockpit, from mission planning to communications between satellites and ground stations. Cockpit is highly automated, reducing the risk of human error while making operations simpler for both Loft Orbital’s engineers and customer requests.
One of Murray’s first tasks, when hired, was to implement an incident response system that could provide 24/7 support, in preparation for their first satellite launch. The system needed to scale in line with the fast-growing business and integrate with Loft Orbital’s existing tech stack, such as Grafana. Loft Orbital needed a solution with a stable API that was easy to customize.
Being able to quickly respond to incidents was a key priority. Loft Orbital operates Low Earth Orbit satellites in a sun-synchronous orbit, so satellites regularly pass over ground stations, or very large satellite dishes used to communicate with the satellites, located at the North and South Pole. There are no ground stations in much of the Pacific Ocean, so there are times when Loft Orbital’s satellites are not in contact with the ground. This means that if an engineer misses an opportunity to correct a problem, they won’t be able to respond for at least another 45 minutes. “Urgency is key because things tend to snowball very quickly in space,” explained Murray. “Prompt responses save us time and money on a huge scale.”
Another requirement was rethinking the mission command center. “Traditionally, you’d have a team of 20 or more people in a room 24/7 keeping your satellite healthy and operational. We’re hoping to only get called in when there is a problem that needs to be fixed, though we generally have at least one person on duty to perform certain manual tasks and maintenance,” explained Murray. “Automation is foundational – we need the ability to scale our operations to 5, 10, or 100 satellites rapidly without needing to scale personnel.”
Murray had used PagerDuty in a previous role at another company and was familiar with its functionality. “There was no question in our minds that PagerDuty was the best solution to solve our problems.”
A Trustworthy Solution
Loft Orbital was able to implement PagerDuty quickly, and seamlessly integrate it with Cockpit and Grafana. If Grafana determines there are telemetry issues with a satellite such as abnormal temperatures or low battery voltage, it triggers an alert in PagerDuty and engineers are able to quickly respond. Additionally, PagerDuty’s agile API was able to easily integrate with Cockpit to alert when performance issues arise with the software itself. “Since adopting PagerDuty the team has acknowledged and resolved all incidents swiftly, giving us complete trust in our incident response processes,” explained Murray.
Loft Orbital also benefits from PagerDuty’s simplicity; PagerDuty’s interface is so easy to use that new hires can support satellite operations within just a few hours. “PagerDuty is something that I can throw at users and don’t have to worry about extensive training on how to ensure they are aware of issues so they can focus on issue resolution training,” explained Murray. “The software is simple enough that you give it to new hires and they’re off to the races an hour later.” Additionally, employees can personalize alerts to suit their preferences to make sure it is notifying them as effectively as possible, which is ideal when managing a global team with differing approaches to work-life balance.
PagerDuty has increased collaboration between Loft Orbital’s teams. The company doesn’t follow a traditional hierarchy, instead empowering engineers to manage services and incidents when they arise while ensuring full-team awareness by tracking issues and resolutions. When there is an issue, PagerDuty alerts the relevant expert according to set escalation policies aligned with time zones, minimizing out-of-hours disruption and downtime while documenting the problem for future reference. “PagerDuty has simplified our team’s lives immensely,” explained Murray. “Previously, engineers were stuck in a position where they didn’t know who the subject matter expert was, but PagerDuty helped eliminate this and now allows for seamless collaboration.”
“PagerDuty is the glue that joins human monitoring to automated response and has given us the ability to scale operations rapidly,” said Murray. “With PagerDuty, I can walk away from my desk and live my life knowing my team has access to me in an emergency, and I have a way to look back on any issues others have addressed.”
Mission Accomplished: A Culture of Rapid Incident Response
PagerDuty has enabled Loft Orbital to scale its operations rapidly and provide 24/7 support for its satellites without a traditional command center or needing to hire additional staff at the same pace that its constellation grows. The company can confidently meet all customer SLAs and let them focus on what matters to them—their data or service.
Looking ahead, Loft Orbital plans to roll out PagerDuty to different engineering teams. By adding more users to the platform, teams will be able to create a more solid structure of response orchestration and cross-team issue tracking and resolution. When an incident arises, everyone, regardless of their role, knows whom to escalate the alert to. Loft Orbital is building a culture in which everyone feels empowered to triage and troubleshoot incidents without worrying about missteps or causing an inconvenience. The company is also considering exploring PagerDuty’s Event Intelligence features to further increase the efficiency of its incident response process.
To learn how PagerDuty can help your organization, contact your account manager or try a 14-day free trial today.