We often hear the refrain that Disciplined Agile (DA) is too complex, that it needs to be simpler. Believe me, we’re all for simplicity and we do everything that we can to make DA as simple as possible. However, to be fair, we often find that the people complaining about complexity are often coming at the situation from a different point of view than we are.
With this blog we work through how DA reflects the necessary complexity that we face as IT professionals. To do this let’s explore why DA may appear to be complex at first. This blog explores the following issues:
- There is a natural complexity in what we do
- Enterprise solution delivery is inherently complex
- Enterprise IT is inherently complex
- Choice is good, but having choices adds complexity
- People underestimate the complexity of the familiar
- DA purposefully includes some questionable choices
- Most people prefer to focus on a small part of the overall process
- What would you take away?
- What would you like to add?
There is a Natural Complexity in What We Do
Software development by itself is complex: we need to work closely with stakeholders to understand their needs, we need to architect and design solutions, we need to build or configure these solutions, we need to test them, we need to deploy them, we need to organize this effort somehow, and many other things. There’s clearly a bit of natural complexity in software development. Furthermore, the overall IT process surrounding software development adds even more complexity to operate, manage, and govern our overall IT ecosystem.
These complexities are why we get paid what we get paid – if it was easy, organizations could find a multitude of lower-skilled, and lower-paid, people to do this work.
Enterprise Solution Delivery is Inherently Complex
The Disciplined Agile Delivery (DAD) portion of the framework addresses solution delivery in enterprise-class organizations. Sometimes people who have only worked in startups or small companies don’t appreciate the realities of enterprise-class development. A small company that has been in business for a few years is very different than a large organization that has several decades of legacy systems, legacy processes, and legacy thinking in place. Adopting agile and lean ways of working when you effectively have a clean slate to start from is a very different proposition than working in an organization where you need to respect and evolve the existing legacy ecosystem.
But don’t other, simpler methods such as Scrum work in these settings? Yes, they potentially do, but they address a much smaller scope than does DAD. Scrum is less complex because it only looks at a very small portion, likely not even 5%, of the software delivery process. The focus of Scrum is on a few collaboration patterns for organizing small software construction teams and for managing changing requirements. It purposefully does not address the full project delivery lifecycle from beginning to end. Nor does Scrum address technical practices for architecture, design, testing, programming, modeling, documentation, deployment, and many other aspects of solution delivery. DAD, on the other hand, purposely addresses all of these aspects of software delivery from beginning to end in a streamlined manner. And Scrum only describes a single way of working, what DAD calls the Agile/Basic lifecycle, whereas DAD includes four lifecycles (Agile, Lean, Continuous Delivery, and Exploratory/Lean Start Up) to choose from to address a great range of needs. So yes, as a framework DAD is much more complex than Scrum because DAD answers the multitude of questions that Scrum leaves up to you.
Interestingly, from the point of view of enterprise agile transformation, DAD is a less complex solution to adopt because it’s already done the “heavy lifting” for you on the process side of things. We wrote a blog a few years ago about how it’s easier to right-size your process by starting in the middle with something like DAD than it is to start at the “simple bottom” with Scrum.
Enterprise IT is Inherently Complex
Disciplined Agile (DA) extends DAD to look at all of IT. The workflow of a Disciplined Agile IT department is depicted in the following diagram, and that diagram is merely a high-level overview. Many of the challenges that developers like to complain about – dysfunctional governance, ineffective enterprise architecture, slow data management, non-existent reuse, questionable HR processes, and many more – DA chooses to explicitly address. Because DA deals with the much wider range of enterprise IT issues than just software development it encompasses more complexity than many developers may be comfortable with, or even aware of.
DA addresses this bigger picture because our philosophy is to provide coherent, pragmatic advice to help IT organizations address the challenges that they face in improving their processes. In fact, the DA framework appears to be the only source of information that addresses enterprise agile IT in a coherent and comprehensive manner. Without a holistic vision such as this an organization will end up development a piece-meal strategy where the individual parts may be effective on their own but do not integrate well as a whole. Not looking at the entire IT picture at once is the primary cause of organizational dysfunctional in your department today, with the DA framework you can avoid this problem as you evolve into an agile/lean way of working. BTW, you can download a copy of the workflow diagram shown above from the Disciplined Agile Consortium (DAC) posters page.
Choice is Good, But Having Choices Adds Complexity
Disciplined Agile looks at a range of situations, giving you choices. Other methods and frameworks, such as LeSS and SAFe, prescribe a single way of working. Certainly a small development team will work differently than a medium-sized development team which works differently than a large team. A team that is co-located will work differently than a team spread across a single building, than a team spread across several buildings in the same city, than a team spread across the world. A team working in a regulatory environment will work in a more sophisticated way than a team in a non-regulatory environment. A team where everyone works for your company will work differently than a team where some of the work is being outsourced. A team facing a simple domain problem will work differently than a team facing a complex problem. And so on. Although many people just want to be told what to do, they just want a single way of working given to them, that desire for prescription is a recipe for disaster in modern IT organizations. You have a range of teams facing a range of situations: One process size does not fit all.
Having choices is definitely complex. This is similar to “needing a new outfit”, going to the store, only to be overwhelmed by the myriad of choices presented to you.
One way to deal with the complexity is to divide and conquer. Recognize that deciding on an outfit that is right for you is actually a collection of simpler choices. For example, your need to choose the right shirt, the right pants, the right shoes, and so on. Of course you shouldn’t make these choices in isolation because your outfit needs to work together as a whole. You either need to coordinate your decisions and effectively make them in parallel or if you prefer to make decisions one at a time then be prepared for your shirt choice to limit your follow-on clothing choices.
Let’s bring the conversation back to software process. In Disciplined Agile Delivery (DAD) we called out a collection of process goals that a team should fulfill in some way to be successful at agile solution delivery. These goals included exploring the initial scope, putting the team together, developing a consumable solution, coordinating activities, and deploying the solution into production to name a few. Each process goal is overviewed by a process goal diagram, the one for Explore Initial Scope follows below. Each goal is organized into a collection of process decision points, each decision point has options, and each option has advantages and disadvantages. Where one agile method might advise you to write a collection of epics and user stories to identify the initial scope, in DAD we say that you need to explore how people will use your solution (and writing epics and user stories are two of many options available to you), you may need to explore the user interface, you should consider to what extent (if any) the initial requirements should be documented, how you will go about eliciting requirements, and other important decisions. Ideally, if you want to be effective, then you want each team to own their own process and make the choices that are best for them given the situation that you face.
Back to the clothing analogy. Just like Scrum and SAFe prescribe one way of working, you could similarly take a prescriptive approach to shopping for new clothes. This is the equivalent of saying that you will be successful if you wear a red polo shirt, blue pants, blue socks, and black shoes. That may be great advice if you are working as a food server at your local restaurant chain but horrible advice if you need an outfit that is appropriate for a wedding, or for working in a formal office, or for going to the beach. When buying the right clothing you need first need to understand the context and then hopefully have several choices available to you. You want to be able to choose the right shirt for you, not the one prescribed to you. You want to choose a nice pair of pants or a skirt that complements that shirt, and so on. Yes, it’s much easier just to buy exactly the clothes that you’re told to, but it doesn’t give you as satisfying a result in most situations.
In Disciplined Agile we extended this choice-driven approach to IT-level activities such as Portfolio Management, Enterprise Architecture, Reuse Engineering, Data Management, and others. The following diagram is the process goal diagram for Enterprise Architecture, depicting the process decision points that your EA team should consider addressing and a range of options for each choice.
People Underestimate the Complexity of the Familiar
When most people are asked what the gentleman in the picture is wearing, they’ll say a blue dress shirt and slacks. However, he’s also wearing shoes, socks, a belt, and underwear (we hope). It is quite common for people to abstract away, to ignore, details that are familiar to them – the “shirt and slacks” answer abstracted away the details of the entire outfit. However, for someone to whom western clothing is new, this “simple outfit” appears quite complex to them until someone helps them to understand it.
The exact same thing happens in the software world. When asked what we do it’s easy to casually say that we work with stakeholders to understand what they want, we write some software, and then we deploy it. Of course the reality of what actually goes on is a lot more complex than that.
We invite you to spend a few minutes and jot down all of the activities that your team does to actually build and deploy software in practice. Pretty long list, isn’t it? And for every strategy on that list, you’ve selected one of many options available to you.
DA Purposefully Includes Some Questionable Choices
We all make choices in our diet. Some of those choices are good and some of those choices are rather bad for us. We know that we’re making poor choices yet we still choose to do so, perhaps because that’s the best we can do at the time or simply because that’s what we feel like eating right now.
It’s exactly the same when it comes to software process – we have a range of choices and some of them are better for us than others. For example, consider the choices we have when it comes to the level of detail that we capture when exploring the initial scope. We could choose to: write a very detailed requirements specification; write a light specification (perhaps using index cards, whiteboard sketches, and a bit of overview documentation); write a simple bulleted list of goals; or not capture any requirements information at all. Many agile purists will say that writing a detailed requirement specification isn’t agile, and in the vast majority of situations it isn’t, although we have seen life-critical regulatory situations where it is appropriate. But, what if you’re in a situation where detailed requirements specifications aren’t appropriate yet someone is demanding that you create one? Agile purists will likely get into religious arguments with this person: the traditionalist will argue that their way is best and the purist will argue that there way is best, but in the end the person with the most political clout will win. With a DAD-based we recognize that there is a range of options available to you, including writing detailed specifications. But we take it one step further and are clear about the advantages and disadvantages of each approach and when the approach is appropriate. Now you can have more intelligent conversations about your process choices. When you get into situations such as this you can go beyond the “but that isn’t agile” refrain to have coherent reasons that as to why one strategy is better for another strategy given the situation that you face.
Most People Focus on a Small Part of the Overall Process
Many people choose to focus on their part of the job, which is fair enough. But what about all of the other activities that your team performs from beginning to end? You may be a developer focused on software construction, but what about all of the initial inception work that occurred, such as gathering initial requirements, putting the team together, and getting funding for construction that occurred before you began your work? What about the activities that occur within other teams that interact with, often to support, your team? For example, there may be some enterprise architects responsible for evolving and supporting a technical roadmap for your organization, there may be an infrastructure team responsible for operating your technical environment, and there may be people responsible for people management/HR activities (such as seeing that you get paid). Disciplined Agile looks at the holistic picture, not just at your piece of the overall puzzle. Just because you’re not involved with data management, or portfolio management, or operations doesn’t mean that those activities, and the teams that perform them, aren’t important to the overall success of your organization.
What Would You Take Away?
Now that you understand the overall scope of DA, what would you remove? We’re asking this question from the point of view of:
- The complexity faced by modern enterprises
- That you need choices if you’re to actually own your process
- That we’re considering the overall picture, not just your part of it
If you think Disciplined Agile (DA) contains something that it doesn’t need, please add a comment at the end of this blog.
What Would You Like to Add?
Ha! Every single time we’ve had this conversation with someone, and they were willing to consider the issues above, they’ve told us that we needed to include one or more techniques that they’ve found effective in practice. EVERY. SINGLE. TIME. In short, the conversation goes from DA is too complex to DA is not complex enough. Argh!
So, once again, please add a comment at the end of the blog for anything you think we’re missing.
A Few Parting Thoughts
Software development, and IT in general, is a complex game with many moving parts that are evolving all of the time. It’s natural for us, as human beings, to want simple answers. But sometimes the simplest answer can still be quite complex. Simplistic answers may be more attractive, and in the short term or in very narrow situations they may in fact be a good solution for you, but more often than not they’ll make your situation even worse. In Disciplined Agile (DA) we actively strive to drive the complexity out of the software process but at the same time we have the courage to look at the bigger picture and explicitly address the range of issues that modern enterprises face in practice. With DA we’re working together to make the software process as simple and streamlined as possible.