PagerDuty Blog

The DevSecOps Cultural Transformation

Let’s take a moment and think about security in your organization. Security is often separate from other engineering teams such as development, operations, networking, IT, and so forth. If you narrow down your focus to specifically releasing new software or features and functions in existing software, you’ll find that while development and operations are working together very quickly and efficiently, they’re still vaulting these functions and features over to security. And security is vaulting them back— much like how development and operations worked before DevOps.

DevSecOps does for development, operations, and security what DevOps did for development and operations: it is a cultural practice—with technological support—that streamlines the work of these teams so they can work together. Similar to DevOps, part of the DevSecOps cultural transformation is developing empathy for each other through understanding the different requirements of each team.

The What And How Of DevSecOps

DevSecOps does not remove the security team, or the need for security as a discipline (unless you want to learn how to do your own penetration tests, amongst other things, without their help or guidance—which I do not recommend). This is similar to how DevOps did not eliminate the need for operations—it changed how operations and development teams worked together.

DevSecOps is a little different from DevOps for how that takes place. As a refresher, the DevOps model looks like this:

In the above workflow, you can see that development and operations are still separate, just no longer siloed. The DevSecOps model looks like this:

As you can see, security is integrated into every aspect of the DevOps cycle—it is not separated like development and operations.

The way this is done is by doing something called “shifting left,” (i.e. bringing security earlier and earlier into the cycle, starting with the design/build phase). There are different testing and other activities that need to be done at each stage. For example, when considering security at the code phase, security will need to take part in code reviews, as well as running tests like a SAST (static application security test) and/or a SCA (software composition analysis). Finding issues here means they are fixed here—before building, testing, and so forth. The same is true for every phase: the earlier a security vulnerability is found, the less expensive, in terms of time and expertise, it is to fix.

Establish Cross Functional Empathy

A lot of security specialists do not have experience running services in production. Likewise, a lot of developers and operations specialists do not have experience in security. In order to bridge these gaps, it is important to “walk a mile in each other’s shoes.”

For security, this means owning a service in production. Notice that it is important that the team is taking ownership of the full life cycle of that service—they aren’t merely doing a single deploy. Security must work with development and operations to determine which service is a good fit, not only in terms of what is needed to run and maintain it, but also what makes sense for the business. As a quick example, if there are any security related services, it would make sense for security to take ownership of that service.

For more more information about how to own a service, please refer to our Full Service Ownership Ops Guide.

For development and operations, they need to do exercises that improve their security awareness and posture overall. One way to do this is to take part in the threat modeling exercises at the design phase for various features and releases. Threat modeling is when teams work together to determine the security implications of new or changed software, services, or environments. For example, if a change is made to the network design, then that new design would need to be threat modeled so that it can be secured properly.

Create a Culture of Collaboration

It is important to work with the human mind, not against it. We are a curious species, and we like to be engaged and interact. We are also a species with a strong negativity bias. Let’s consider what having these together can mean in terms of learning. Right now, while reading this sentence of this paragraph, take a moment and think of some of the most memorable content you’ve come across, or a particularly memorable lesson.

What was memorable about it? Was it a positive experience? Were you able to apply what you learned later in a way that helped yourself or others? Was it just plain interesting? Or perhaps it was a criticism. When you think of that criticism, do you remember more strongly what was communicated or how it was communicated?

Now that you are primed and ready, I’m going to reiterate something that I also included in our DevSecOps ops guide: here at PagerDuty, we have a policy not to trick staff, ever. A well known example is when companies send phishing emails to employees and require security training if the emails are opened and/or any links inside of them are clicked. The goal of this exercise is typically stated as demonstrating that real phishing attacks can and will be both well designed and harmful, but if we think about how humans learn, is this the lesson being taught?

The main goal of security—in terms of education—is to improve the overall awareness of staff for ways that security may be breached. Another goal is to be present to remedy any incidents that do arise; but in order to do that, the team must earn enough trust that staff will feel psychologically safe enough to report incidents.

To take a different approach: use security training as an opportunity to create and maintain both interest and trust. Providing customizing security training allows you to leverage your team to curate content around areas where your organization is weaker and be more general in areas where your organization is strong. It also allows you to really engage with staff—do something to grab their attention. Maybe demo a cross site scripting attack or show how to pick a lock, and tie the latter into concepts you’re trying to convey in the training.

Since customizing training completely from scratch is a huge undertaking, please feel free to use ours. Our training materials are open source under the Apache License, so please feel free to use and repurpose to suit the needs of your organization.

Check Out Our New Ops Guide

To learn more about what I’ve written here, please check out our new DevSecOps Guide! Our Ops Guides are continually updated and this guide is no exception. So if you’d like to contribute, please feel free to reach out or submit a pull request on GitHub. If you’d like to chat about DevSecOps, please also feel free to post in our Community Forums or reach out to me directly on Twitter, I’d love to hear from you!