The Problems With Agile and Scrum
Agile is a popular buzzword in software development, with some organizations and teams masquerading as “agile” while they are actually doing something very different. I’ve seen it numerous times in my career as an Agile Coach: A leader claims to embrace agile values, but micromanages engineering teams and uses agility as a way to manipulate developers to work long hours. The result? Some engineers in our industry hate agile software development because they feel it has been co-opted by those outside of engineering to manipulate them and gamify software building.
In this blog post, I’ll be sharing three “problems” that engineers experience with agile and scrum. I’ll then respond with why these are not problems at all, once you cut through the “agile BS” that bad actors and misunderstandings have spread.
Before we get started, let’s define “agile BS”:
Agile BS: Usage of the term “agile” when referring to a team, project, or organization that’s not aligned with the principles and values of agile—e.g., aka waterfall masquerading as agile or slapping the word “agile” on what, in reality, is a dysfunctional process.
To anchor this definition in your mind, here are some example “agile BS” flags that a team or organization is not being agile:
- Engineering team members are not talking with and observing the users of the software
- Continuous feedback from users to the engineering team is not happening
- Meeting the full project requirements is treated as higher priority than shipping an MVP as soon as possible to get early feedback
For more on “agile BS,” see the Department of Defense’s guide to detecting agile BS. It’s important to note that agile principles and values were created to solve the very problems that people sometimes express with agile, especially those called out in this post.
Problem 1: Agile provides numerous processes and tools for managers to use in bad faith—and few for engineers to exert upward influence.
It’s true that managers can use Daily Standups to micromanage their team members. And managers can conflate a Scrum team’s sprint commitment with “guarantee,” pushing engineers to work long hours to meet their sprint goal (see problem 2). But these are not agile software development problems; they are management problems. And you cannot overcome bad management with agile software development. This is setting the wrong expectations for what agile principles and values set out to accomplish.
So let’s say an organization has the right management in place. How can it embrace agility? The organization must live agile principles and values at all levels. This requires a commitment to agility from leadership and can be achieved through agile transformation efforts (potentially with the support of an Agile Coach). In the case of PagerDuty, we have a dedicated Agile Leadership Team to build and expand systems, processes, and a culture that continuously improves our organizational effectiveness, unlocking the full potential of our teams.
Tied closely to this “problem,” agile values individuals and interactions over processes and tools. What does that actually mean?
- People over process. People manage software development, not processes and tools.
- A one-size-fits-all approach to processes and tools doesn’t work well for software development
- Be wary of processes that don’t allow for variance and tools that impede interactions. Think guidance vs. governance.
When thinking about processes and tools, here are “agile BS” flags that a team or organization is not being agile:
- Process considerations such as Definition of Done are defined top-down for engineering teams
- Teams don’t own their process solutions (e.g., choosing Scrum vs. Kanban)
- Managers use Daily Standups and Sprint Goals as a way of micromanaging their engineers
Problem 2: Agile intentionally conflates “estimates” with “commitments,” manipulating engineers into working extra hours.
I’m going to break this problem into two parts:
1. Agile intentionally conflates “estimates” with “commitments”
Language is particularly important when discussing a team’s commitment. To explain the scrum concept of “sprint commitment,” here’s advice from Mike Cohn, a co-founder of Scrum Alliance and Agile Alliance:
“It is important that the team’s commitment not be viewed as a guarantee. The team’s commitment is to do its best. I’d like to see them make their commitment perhaps 80 percent of the time. It should be something they take seriously and should make most of that time. That’s needed for the business to gain confidence in what a team says it can deliver.”
What’s important here is that a team’s commitment is not seen as a guarantee. The team does their best, reflects on what they delivered vs. their goal at the end of each iteration, and adapts their process accordingly.
2. Manipulating engineers into working extra hours
Working extra hours has less to do with being agile, and more to do with a company’s culture. That being said, a principle of the agile manifesto is that agile processes promote sustainable development. Does this mean that engineers will never have to work long hours? Of course not.
It’s important for engineering leaders to set appropriate working hour expectations with their teams. For example, an Engineering Manager might communicate their expectations as:
I expect everyone to work between 40 to 50 hours a week. This generally looks like an 8-hour workday and going on-call one week every month. Twice a year, I get to ask the team to work extra hours. I do not get to do this unless absolutely necessary.
Notice that the manager is not asking the team to work extra hours on a regular basis. But they do set the expectation that this could happen several times each year.
Relevant to sustainable development, here are “agile BS” flags that a team or organization is not being agile:
- When the engineering team commits to a sprint, this is synonymous with a guarantee
- The scrum team always meets their sprint commitments
- The engineering team regularly works extra hours to meet their sprint commitment
Problem 3: The scrum notion of a “commitment” means churning out a specific amount of finished work every two weeks, which is overly hostile to engineers wanting to make long-term decisions about their software development.
When a Scrum team starts planning their next sprint, they pull in the items that are determined as the highest priority for their customer. Long-term decisions for a team are planned ahead in the team’s product roadmap. But product roadmaps are a two-way street: A Product Owner will define the next most important problem for the team to solve. It’s up to the team’s engineers to provide feedback on the roadmap and push back if they feel there’s something more valuable for the team to be working on.
Why would engineers want to provide feedback (or push back on) their product roadmap?
- For teams that have an on-call rotation, if their on-call has been particularly noisy and painful lately, their most valuable work for the upcoming sprint might be to prioritize infrastructure improvements
- If the team has accumulated a large amount of technical debt, they may need to adjust their roadmap to focus on that before it becomes more problematic
- If the team received feedback from a user or stakeholder, which would improve the value they are delivering, adapting their roadmap to prioritize addressing this feedback may be their most valuable work
When thinking about long-term decisions, here are “agile BS” flags that a team or organization is not being agile:
- The engineering team is given a solution, instead of a problem to solve, by the Product Owner
- The engineering team does not have a mechanism to provide feedback to the product roadmap
What Success Looks Like
Successful agile software development requires organizational buy-in from all levels, engagement from engineering, focus on the customer, and potentially bringing in dedicated Agile Coaches.
Once an organization has embraced true agility:
- Engineers can and should exert upward influence on the organization using mechanisms such as metrics (e.g., product engagement, impact on revenue, predictability), team health checks, and team retrospectives
- Well-formed scrum and kanban teams are trusted to deliver predictable value to their customers while working at a sustainable pace
- Engineers are empowered to give feedback and adapt their product roadmap so that they are always working on the most valuable thing. Taking this further, regularly scheduling a HackWeek gives engineers full autonomy to work on what they feel is most important for the organization.
Share Your Learnings
Have you noticed other ways organizations and teams masquerade to be “agile” when they are in fact doing something very different? We’d love to hear from you in the PagerDuty Community.