Category Archives: Scaling

Do Agile Teams Take on Hard Problems?

We often hear that agile software development is fine when you face a simple problem, but that agile isn’t sufficient for more difficult problems.  Of course this falsehood is promoted by people who have little or no agile experience, or who have been involved with a failed “agile adoption” (usually these teams adopted ad hoc strategies thinking they were agile).  Anyway, we decided to look into whether agile teams are taking on hard domain problems in practice.

The following diagram summarizes the responses to our question around agile teams and compliance from our 2016 Agility at Scale study.  As you can see, 40% of respondents indicated that their agile team faced either complex or very complex problems, and that a further 38% faced medium complexity.  Interestingly, only one in eight respondents said that their team faced a straight forward problem.

Agile and Domain Complexity

The bottom line is that agile strategies, and in particular disciplined agile strategies, are in fact applicable for taking on complex problems.  More importantly, this is happening in practice around the world on a regular basis.

Related Posts

Do Agile Teams Face Regulatory Compliance?

We often hear that agile is great for simple situations but as soon as you face compliancy issues that it doesn’t work.  Is it possible to be agile when you face regulatory compliance, such as PCI and FDA compliancy?  Is it possible to be agile when you face organizational compliance, such as working in a CMMI regime?  Important questions that we decided to look into.

The following diagram summarizes the responses to our question around agile teams and compliance from our 2016 Agility at Scale study.  As you can see, 62% of respondents indicated that their agile team faced some form of regulatory compliance, 20% some form of organizational compliance, and 15% said both.  In fact, two-thirds of agile teams operate under one or more compliancy requirements.

Agile Regulatory Compliance

For further reading about compliancy, please read our detailed blog posting Agile and Regulatory Compliance.

Related Posts

How Geographically Distributed Are Agile Teams in Practice?

Many people, particularly those new to agile, will tell you that agile teams must be small and co-located.  That is certainly a smart way of organizing a team, but is isn’t required.  In fact agile teams are more likely to be geographically distributed in some way than they are to be co-located.  In practice, not theory.

In November of 2016 we ran the 2016 Agility at Scale survey.  It was targeted at people who were currently working on agile teams, or who had recently worked on agile teams, and we asked them straightforward questions around the size of the team, how distributed it was, what complexities they faced, an so on.  The following graph summarizes the responses around geographic distribution.

Geographic distribution and agile teams

The survey found that less than one-third of agile teams are near-located, where all of the IT members are either co-located or at least in a shared open space.  Previous studies have found that this number drops to one-in-ten teams being near located when you also include primary stakeholders.

Don’t let anyone tell you that you can’t do agile with a geographically distributed team because others are clearly doing so in practice.  Yes, geographically distributed agile is different than near-located agile, which is one of the reasons why you need to take a pragmatic, context-sensitive approach to agile solution delivery.  The Disciplined Agile framework provides the foundation from which to scale your approach to solution delivery to address a range of scaling factors, including geographic distribution.  In fact, you may find our article around geographically distributed agile teams to be an interesting read.

Related Posts:

How Large Are Agile Teams in Practice?

The contrite answer is that they’re as large as they need to be, and the contrite agile answer is that they’re as small as they can be.  Now that we’ve gotten the contrite answers out of the way, how large are agile teams in practice?

In November of 2016 we ran the 2016 Agility at Scale survey.  It was targeted at people who were currently working on agile teams, or who had recently worked on agile teams, and we asked them straightforward questions around the size of the team, how distributed it was, what complexities they faced, an so on.  The following graph summarizes the responses around team size.

Agile team size

This year’s survey found that roughly half (48%) of agile teams are more than 10 people in size and one-quarter are more than 20 people in size.  These findings are similar to what we’ve found in the past with both the 2012 Agility at Scale survey and the 2009 Agility at Scale survey.

In short, don’t let anyone tell you that you can’t do agile with a large team because others are clearly doing so in practice.  Yes, large team agile is different than small team agile, which is one of the reasons why you need to take a pragmatic, context-sensitive approach to agile solution delivery.  The Disciplined Agile framework provides the foundation from which to scale your approach to solution delivery to address a range of scaling factors, including team size.  In fact, you may find our article around large agile teams to be of interest.

Related Posts:


The Disciplined Agile Poster v2.1

DAD Poster People 7-4

On Wednesday, September 21 2016 we ran a webinar overviewing the latest version of the DA poster. A recording of the webinar can be viewed online from the DAC Webinars page and a PDF of the slide deck is posted on Slideshare entitled The Disciplined Agile IT Department.

During the webinar we received several questions, many of which we answered, although we did run out of time and did not get to all of them. As usual, we’ve written a blog posting to answer these questions. We’ve organized the questions into the following topics:

  1. The poster and terminology
  2. Organizational issues
  3. Adoption of DA
  4. Learning more
  5. Miscellaneous


1. The Poster and Terminology

1.1 What’s the significance of the concentric circles in shades like white, blue yellow? How do we look at those groupings?

Those are groupings on the poster. There are headings, they sort of look like ribbons, indicating the scope of each one: Delivery, Disciplined DevOps, IT.

1.2 What’s the best way to read or traverse the diagram (for beginners)? left–>right, top–>bottom, inside–>outside … ?

It’s a workflow diagram that is cyclic, so in a way it doesn’t matter. However, it does make sense to start in the top left corner if you like.

1.3 What was the thought behind the “process blade”?

We have a detailed article about the term process blade here.


2. Organizational Issues

2.1 What range of IT dept sizes is DA best suited for? Is there a sweet spot?

We’re seeing Disciplined Agile being adopted by IT departments as small as 40 people and as large as 20,000+. The issue really isn’t the size of the IT department but rather how well it is able to support the rest of your organization,

2.2 What is a role played by Chief AO and Chief PO in terms of Solution Delivery? Where can we put DAD Roles in the DA poster?

If you visit the DA home page there is the poster on there. There is also a button in the bottom left corner leading to a beta version of the diagram with the roles on it.  The IT-level roles are overviewed at Disciplined Agile Roles at Scale.

2.3 Where would someone like say a “Peoplesoft Administrator” fit into the new (beta) DA 2.x picture. I’ve seen loads of similar people/roles in a Agile Transformation initiatives at a Bank and hence the ask.

This sounds like a specialized version of an Operations Engineer to me. Also, sounds like more of a position than a role.


3. Adoption of DA

3.1 Do you have any examples of organizations in Canada (besides IBM) that are using DA or DAD framework?

There are many organizations using Disciplined Agile although this is not a complete list nor is it broken down by country. If your organization is using DAD, and you would like to get on this list, please reach out to us.

3.2 Any recommendation for terminologies/nomenclature in Agile Portfolio Management? sub-task=>story=>Epic=>Initiatives/Product=> Themes??

Use the terminologies and techniques that are right for your organization. There is no standard terminology in agile, nor will there ever be. Furthermore, every organization is unique so one process size does not fit all. You need to tailor your approach to address the situation that you face.


4. Learning More

4.1 Will there be a new book re IT and DevOps additions? When?

Yes, we have a book underway right now. Having said that, we always publish to the web first. We have published a lot here on the Disciplined Agile site, including a detailed article on Disciplined DevOps.

4.2 Is a new version of the big book coming up? Any updates to the Certification or tracks wrt. DA 2.x?

At the present moment we do not intend to publish an update to the big book. We find publishing smaller books, such as Introduction to DAD, to be more effective, and that’s what we’re currently focused on. When we publish the new book we will also be updating the certifications to reflect that, so please stay tuned.

4.3 Your current book says very little about scaling, but with the example of Barclay’s 30K IT Dept, will the new book have more “scaling” content?

We have published a fair bit about scaling online. The page Agility at Scale is a good starting point.  Also, Disciplined Agile Delivery (DAD) provides a foundation from which to scale, so I would argue we’ve written quite a bit, even in the first book.

4.4 Where I can find guidance for process blades? Will new version of book cover it?

We have published detailed information about the process blades at the DA site. A good starting page is Strategic Agility at Scale. Yes, we will be covering this material in the new book.

4.5 Will the download posters be updated to 2.1? Lot nicer to follow and look at 🙂

The poster is available for download, in various formats, from Disciplined Agile Posters.


4.6 This is very new in Europe. What are your plans to expand your DAD coverage there?

We are actively looking for business partners, including both trainers and consulting firms, in Europe, Asia, and Australia. For more information, see our Certified Disciplined Agile Instructor (CDAI) Program and our Certified Disciplined Agile Partner Program.


5. Miscellaneous

5.1 Governance is depicted at lower left (and appears IT centric). Isn’t the reality that each of the artifacts (and there are lots! on this poster) can be subjected to a simple artifact governance? Does DA leverage this?

Governance is built right into all aspects of the DA framework. The IT Governance blade is purposefully IT centric because that is the scope that we chose to address. It is in fact one part of your organization’s overall governance strategy (often referred to as a control strategy, yuck).

5.2 Does DA group recommends any tools for Agile?

We try to avoid recommending tools because it really does depend on your situation. However, we do have a good relationship with Blueprint Technologies and have been working with several of the agile management tool vendors. We do have a page discussing tool support for DAD but it’s not comprehensive.

5.3 Should operations teams run one of the lifecycles like a software delivery team? Should support?

Operations teams and support teams should definitely have a method that they follow to do their work. Given that they often need to respond to help requests, or address production problems, they very likely need to consider adopting a lean lifecycle for that type of work. In particular the Continuous Delivery lifecycle is likely closest to what they need, but this of course depends on the situation.

5.4 Where do you see DA complementing SAFe or vice versa.

In many ways SAFe is an instance of a portion of DA. Where SAFe prescribes an approach DA gives you a range of options and asks you to choose the right approach for the situation that you face.



One Company, 800+ Disciplined Agile Teams and Counting

Reached the peak

In September 2016 InfoQ published an interview, Benefits of Agile Transformation at Barclays, with Jonathon Smart and Ian Dugmore of Barclays in the UK.  The interview summarizes the experiences of adopting Disciplined Agile (DA) strategies within Barclays.  Some of the key points made in the interview include:

  • Barclays is a large financial institution with over 130,000 employees that has been in operation for over 325 years
  • Barclays is taking a holistic approach to their agile transformation, it is not just a technology thing, linking up their various “islands of agility” to reduce the impedance mismatch between them
  • Barclays has the equivalent of over 800 agile teams that have adopted DA approaches in progress
  • They considered SAFe, LeSS and DA and settled on DA as it is not a “one-size fits all” approach
  • They needed to change both their funding and measurement strategies
  • Results include: A 300% increase in throughput (note: measured in terms of story delivery); 50% reduction in code complexity; test code coverage increase by 50%; More than half of teams are deploying into production at least monthly; Improved team happiness/morale; Improved business outcomes such as quicker time to market

The source article is definitely a very interesting read.  Scott Ambler + Associates has worked with Barclays from the beginning of their transformation and we work with many other organizations around the world to do the same.


Why is Disciplined Agile So Complex?

ConfusedWe often hear the refrain that Disciplined Agile (DA) is too complex, that it needs to be simpler. Believe me, we’re all for simplicity and we do everything that we can to make DA as simple as possible. However, to be fair, we often find that the people complaining about complexity are often coming at the situation from a different point of view than we are.

With this blog we work through how DA reflects the necessary complexity that we face as IT professionals. To do this let’s explore why DA may appear to be complex at first. This blog explores the following issues:

  1. There is a natural complexity in what we do
  2. Enterprise solution delivery is inherently complex
  3. Enterprise IT is inherently complex
  4. Choice is good, but having choices adds complexity
  5. People underestimate the complexity of the familiar
  6. DA purposefully includes some questionable choices
  7. Most people prefer to focus on a small part of the overall process
  8. What would you take away?
  9. What would you like to add?

There is a Natural Complexity in What We Do

Software development by itself is complex: we need to work closely with stakeholders to understand their needs, we need to architect and design solutions, we need to build or configure these solutions, we need to test them, we need to deploy them, we need to organize this effort somehow, and many other things. There’s clearly a bit of natural complexity in software development. Furthermore, the overall IT process surrounding software development adds even more complexity to operate, manage, and govern our overall IT ecosystem.

These complexities are why we get paid what we get paid – if it was easy, organizations could find a multitude of lower-skilled, and lower-paid, people to do this work.


Enterprise Solution Delivery is Inherently Complex

Complex process

The Disciplined Agile Delivery (DAD) portion of the framework addresses solution delivery in enterprise-class organizations. Sometimes people who have only worked in startups or small companies don’t appreciate the realities of enterprise-class development. A small company that has been in business for a few years is very different than a large organization that has several decades of legacy systems, legacy processes, and legacy thinking in place. Adopting agile and lean ways of working when you effectively have a clean slate to start from is a very different proposition than working in an organization where you need to respect and evolve the existing legacy ecosystem.

But don’t other, simpler methods such as Scrum work in these settings? Yes, they potentially do, but they address a much smaller scope than does DAD. Scrum is less complex because it only looks at a very small portion, likely not even 5%, of the software delivery process. The focus of Scrum is on a few collaboration patterns for organizing small software construction teams and for managing changing requirements. It purposefully does not address the full project delivery lifecycle from beginning to end. Nor does Scrum address technical practices for architecture, design, testing, programming, modeling, documentation, deployment, and many other aspects of solution delivery. DAD, on the other hand, purposely addresses all of these aspects of software delivery from beginning to end in a streamlined manner. And Scrum only describes a single way of working, what DAD calls the Agile/Basic lifecycle, whereas DAD includes four lifecycles (Agile, Lean, Continuous Delivery, and Exploratory/Lean Start Up) to choose from to address a great range of needs. So yes, as a framework DAD is much more complex than Scrum because DAD answers the multitude of questions that Scrum leaves up to you.

Interestingly, from the point of view of enterprise agile transformation, DAD is a less complex solution to adopt because it’s already done the “heavy lifting” for you on the process side of things.  We wrote a blog a few years ago about how it’s easier to right-size your process by starting in the middle with something like DAD than it is to start at the “simple bottom” with Scrum.


Enterprise IT is Inherently Complex

Disciplined Agile (DA) extends DAD to look at all of IT. The workflow of a Disciplined Agile IT department is depicted in the following diagram, and that diagram is merely a high-level overview. Many of the challenges that developers like to complain about – dysfunctional governance, ineffective enterprise architecture, slow data management, non-existent reuse, questionable HR processes, and many more – DA chooses to explicitly address. Because DA deals with the much wider range of enterprise IT issues than just software development it encompasses more complexity than many developers may be comfortable with, or even aware of.

Disciplined Agile IT Workflow

DA addresses this bigger picture because our philosophy is to provide coherent, pragmatic advice to help IT organizations address the challenges that they face in improving their processes. In fact, the DA framework appears to be the only source of information that addresses enterprise agile IT in a coherent and comprehensive manner.   Without a holistic vision such as this an organization will end up development a piece-meal strategy where the individual parts may be effective on their own but do not integrate well as a whole. Not looking at the entire IT picture at once is the primary cause of organizational dysfunctional in your department today, with the DA framework you can avoid this problem as you evolve into an agile/lean way of working.  BTW, you can download a copy of the workflow diagram shown above from the Disciplined Agile Consortium (DAC) posters page.


Choice is Good, But Having Choices Adds Complexity

Disciplined Agile looks at a range of situations, giving you choices. Other methods and frameworks, such as LeSS and SAFe, prescribe a single way of working.   Certainly a small development team will work differently than a medium-sized development team which works differently than a large team. A team that is co-located will work differently than a team spread across a single building, than a team spread across several buildings in the same city, than a team spread across the world. A team working in a regulatory environment will work in a more sophisticated way than a team in a non-regulatory environment. A team where everyone works for your company will work differently than a team where some of the work is being outsourced. A team facing a simple domain problem will work differently than a team facing a complex problem. And so on. Although many people just want to be told what to do, they just want a single way of working given to them, that desire for prescription is a recipe for disaster in modern IT organizations. You have a range of teams facing a range of situations: One process size does not fit all.

Many choices

Having choices is definitely complex. This is similar to “needing a new outfit”, going to the store, only to be overwhelmed by the myriad of choices presented to you.

Small number of choicesOne way to deal with the complexity is to divide and conquer. Recognize that deciding on an outfit that is right for you is actually a collection of simpler choices. For example, your need to choose the right shirt, the right pants, the right shoes, and so on. Of course you shouldn’t make these choices in isolation because your outfit needs to work together as a whole.  You either need to coordinate your decisions and effectively make them in parallel or if you prefer to make decisions one at a time then be prepared for your shirt choice to limit your follow-on clothing choices.

Let’s bring the conversation back to software process. In Disciplined Agile Delivery (DAD) we called out a collection of process goals that a team should fulfill in some way to be successful at agile solution delivery. These goals included exploring the initial scope, putting the team together, developing a consumable solution, coordinating activities, and deploying the solution into production to name a few. Each process goal is overviewed by a process goal diagram, the one for Explore Initial Scope follows below. Each goal is organized into a collection of process decision points, each decision point has options, and each option has advantages and disadvantages. Where one agile method might advise you to write a collection of epics and user stories to identify the initial scope, in DAD we say that you need to explore how people will use your solution (and writing epics and user stories are two of many options available to you), you may need to explore the user interface, you should consider to what extent (if any) the initial requirements should be documented, how you will go about eliciting requirements, and other important decisions. Ideally, if you want to be effective, then you want each team to own their own process and make the choices that are best for them given the situation that you face.

Explore Initial Scope




Back to the clothing analogy. Just like Scrum and SAFe prescribe one way of working, you could similarly take a prescriptive approach to shopping for new clothes. This is the equivalent of saying that you will be successful if you wear a red polo shirt, blue pants, blue socks, and black shoes. That may be great advice if you are working as a food server at your local restaurant chain but horrible advice if you need an outfit that is appropriate for a wedding, or for working in a formal office, or for going to the beach. When buying the right clothing you need first need to understand the context and then hopefully have several choices available to you. You want to be able to choose the right shirt for you, not the one prescribed to you. You want to choose a nice pair of pants or a skirt that complements that shirt, and so on. Yes, it’s much easier just to buy exactly the clothes that you’re told to, but it doesn’t give you as satisfying a result in most situations.

In Disciplined Agile we extended this choice-driven approach to IT-level activities such as Portfolio Management, Enterprise Architecture, Reuse Engineering, Data Management, and others. The following diagram is the process goal diagram for Enterprise Architecture, depicting the process decision points that your EA team should consider addressing and a range of options for each choice.

Agile Enterprise Architecture

People Underestimate the Complexity of the Familiar

Casual guy

When most people are asked what the gentleman in the picture is wearing, they’ll say a blue dress shirt and slacks. However, he’s also wearing shoes, socks, a belt, and underwear (we hope).   It is quite common for people to abstract away, to ignore, details that are familiar to them – the “shirt and slacks” answer abstracted away the details of the entire outfit. However, for someone to whom western clothing is new, this “simple outfit” appears quite complex to them until someone helps them to understand it.

The exact same thing happens in the software world. When asked what we do it’s easy to casually say that we work with stakeholders to understand what they want, we write some software, and then we deploy it. Of course the reality of what actually goes on is a lot  more complex than that.

We invite you to spend a few minutes and jot down all of the activities that your team does to actually build and deploy software in practice. Pretty long list, isn’t it? And for every strategy on that list, you’ve selected one of many options available to you.


DA Purposefully Includes Some Questionable Choices

Good and bad choicesWe all make choices in our diet. Some of those choices are good and some of those choices are rather bad for us. We know that we’re making poor choices yet we still choose to do so, perhaps because that’s the best we can do at the time or simply because that’s what we feel like eating right now.

It’s exactly the same when it comes to software process – we have a range of choices and some of them are better for us than others. For example, consider the choices we have when it comes to the level of detail that we capture when exploring the initial scope. We could choose to: write a very detailed requirements specification; write a light specification (perhaps using index cards, whiteboard sketches, and a bit of overview documentation); write a simple bulleted list of goals; or not capture any requirements information at all. Many agile purists will say that writing a detailed requirement specification isn’t agile, and in the vast majority of situations it isn’t, although we have seen life-critical regulatory situations where it is appropriate.   But, what if you’re in a situation where detailed requirements specifications aren’t appropriate yet someone is demanding that you create one? Agile purists will likely get into religious arguments with this person: the traditionalist will argue that their way is best and the purist will argue that there way is best, but in the end the person with the most political clout will win. With a DAD-based we recognize that there is a range of options available to you, including writing detailed specifications. But we take it one step further and are clear about the advantages and disadvantages of each approach and when the approach is appropriate.   Now you can have more intelligent conversations about your process choices.   When you get into situations such as this you can go beyond the “but that isn’t agile” refrain to have coherent reasons that as to why one strategy is better for another strategy given the situation that you face.


Most People Focus on a Small Part of the Overall Process

Locally focused

Many people choose to focus on their part of the job, which is fair enough. But what about all of the other activities that your team performs from beginning to end? You may be a developer focused on software construction, but what about all of the initial inception work that occurred, such as gathering initial requirements, putting the team together, and getting funding for construction that occurred before you began your work? What about the activities that occur within other teams that interact with, often to support, your team? For example, there may be some enterprise architects responsible for evolving and supporting a technical roadmap for your organization, there may be an infrastructure team responsible for operating your technical environment, and there may be people responsible for people management/HR activities (such as seeing that you get paid). Disciplined Agile looks at the holistic picture, not just at your piece of the overall puzzle. Just because you’re not involved with data management, or portfolio management, or operations doesn’t mean that those activities, and the teams that perform them, aren’t important to the overall success of your organization.


What Would You Take Away?

Now that you understand the overall scope of DA, what would you remove? We’re asking this question from the point of view of:

  • The complexity faced by modern enterprises
  • That you need choices if you’re to actually own your process
  • That we’re considering the overall picture, not just your part of it

If you think Disciplined Agile (DA) contains something that it doesn’t need, please add a comment at the end of this blog.


What Would You Like to Add?

Ha! Every single time we’ve had this conversation with someone, and they were willing to consider the issues above, they’ve told us that we needed to include one or more techniques that they’ve found effective in practice. EVERY. SINGLE. TIME.  In short, the conversation goes from DA is too complex to DA is not complex enough.  Argh!

So, once again, please add a comment at the end of the blog for anything you think we’re missing.


A Few Parting Thoughts

Software development, and IT in general, is a complex game with many moving parts that are evolving all of the time. It’s natural for us, as human beings, to want simple answers. But sometimes the simplest answer can still be quite complex. Simplistic answers may be more attractive, and in the short term or in very narrow situations they may in fact be a good solution for you, but more often than not they’ll make your situation even worse.   In Disciplined Agile (DA) we actively strive to drive the complexity out of the software process but at the same time we have the courage to look at the bigger picture and explicitly address the range of issues that modern enterprises face in practice. With DA we’re working together to make the software process as simple and streamlined as possible.

Puzzle - canstockphoto18141553 Small

Independent Testing and Agile Teams

The majority of testing, and in simple situations all of it, is performed by an agile delivery team itself. This is because we strive to have cross-functional “whole teams” that have the capability and accountability to perform the activities of solution delivery from beginning to end.   For organizations new to agile this means that you embed testers on agile teams and for organizations experienced in agile that you’ve managed to motivate agile team members to gain testing skills over time (often via close collaboration with other people who already have those skills).

This blog is organized into the follow topics:


Parallel Independent Testing

The whole team approach to development where agile teams test to the best of the ability is a great start, but it isn’t sufficient in some scaling situations. In these situations, described below, you need to consider instituting a parallel independent test team that performs some of the more difficult or expensive forms of testing. As you can see in Figure 1, the basic idea is that on a regular basis the development team makes their working build available to the independent test team. This may happen several times an iteration – we’ve seen teams do so at the end of each iteration, on a nightly basis, as part of their continuous deployment (CD) strategy. How often this occurs needs to be negotiated between the two teams.

Figure 1. Independent testing with an agile team.

Independent agile testing

The goal of this independent testing effort is not to redo the confirmatory testing that is already being done by the development team, but instead to identify the defects that may have fallen through the cracks. The implication is that this independent test team does not need a detailed requirements speculation, although they may need updated release notes including a list of changes since the last time the development team sent them a build. Instead of testing against the specification, the independent testing effort typically focuses on:

  1. Investigative/exploratory testing. Confirmatory testing approaches, such as test-driven development (TDD), validate that you’ve implemented the requirements as they’ve been described to you. But what happens when requirements are missed? What happens when requirements are narrowly focused on a point-specific solution, but not the overall ecosystem? User stories are a great way to explore functional requirements but defects surrounding non-functional requirements such as security, usability, and performance have a tendency to be missed by teams new to this approach.
  2. Production readiness testing. This is sometimes called pre-production system integration testing. The system that you’re building must “play well” with the other systems currently in production when your system is released. To do this properly you must test against versions of other systems that are currently under development, implying that you need access to updated versions on a regular basis. This is fairly straightforward in small organizations, but if your organization has dozens, if not hundreds of IT teams underway it becomes overwhelming for individual development teams to gain such access. A more efficient approach is to have an independent test team be responsible for such enterprise-level system integration testing.
  3. Difficult testing. Many forms of testing require sophisticated skills and sometimes even expensive tooling. Security testing is a classic example.


Why Independent Testing?

There are several reasons why you should consider independent testing:

  1. Regulatory compliance. Disciplined agile teams that find themselves in strict regulatory compliance situations, typical in systems engineering and life critical environments, may need to perform independent testing by law. Having said that, only a portion of your testing efforts will need to be performed independently. We have yet to run into a regulation that says that all of your testing needs to be performed independently, although we have seen several organizations that choose to interpret the regulations that way.
  2. Production readiness testing is exponentially expensive in multi-team environments. Development teams may not have the resources required to perform effective production readiness testing, resources that from an economic point of view must be shared across multiple teams. For example, if there are 5 other development teams in addition to my own then chances are that each team can do the work required to integrate and test against the builds of the other teams. But what about if there’re ten other teams? Twenty? Fifty? It becomes exponentially expensive for each team to do this integration and testing work as the number of teams increases. The implication is that you will discover that you need an independent test team working in parallel to the development team(s) that addresses these sorts of issues.
  3. The average cost of fixing defects is exponentially expensive the longer you wait. For close to four decades Barry Boehm, and other researchers, have gathered data showing that the average cost of addressing a defect rises exponentially the longer it takes to fix. This is depicted in Figure 2. The implication is that we want to find defects as early as we can, in fact ideally we want to build quality in from the start and not inject the defects to begin with. In traditional environments we would have left some forms of testing, in particular system integration testing (SIT) and user acceptance testing (UAT) to the end for the convenience of the people doing the testing (“we need to have everything in place before we can test it”). This results in very expensive and risky defect repair. Parallel independent testing often proves to be a bit more complex for testers, at least at first, but results in much more economical repair efforts.
  4. Complex technical environments. When you’re working with multiple technologies, legacy systems, or legacy data sources the testing of your system can become very difficult. System integration testing (SIT), performance testing, load testing, and security testing become more complex and more important in these situations.
  5. Large or geographically distributed teams. Large agile teams or geographically distributed agile teams are often subdivided into smaller teams, and when this happens system integration testing of the overall system can become complex enough that a separate team should consider taking it on. In fact, SAFe prescribes this with their System Integration Team strategy (which is virtually identical to this strategy). Granted, the reason why you have such teams is because you’re facing either significant domain or technical complexity.
  6. Outsourcing. Teams that are organizationally distributed, for example when some of the work is being outsourced to another organization, will very likely want to perform independent testing to validate the work being performed by the other company(es).  Read this article about agile outsourcing strategies.


Figure 2: Average cost of change curve.

Average cost of change curve


Questionable Reasons to Adopt Independent Testing

We’ve run into a few rather poor excuses to justify independent testing over the years:

  1. Your testing/quality staff prefer to work in a traditional manner. They may insist on testing from a detailed requirements specification, testing that is better performed by the team. They may insist on waiting to test the entire solution once it’s ready, instead of incrementally testing the system while it’s being built (enabling much cheaper defect fixing as discussed earlier). They may insist on all testing being done independently, instead of embedding people with testing skills on solution delivery teams. All of these strategies are choices that reflect a traditional culture, not an agile one. The real solution is to overcome these cultural challenges and help them to gain the skills and mindset required to work in an agile manner.
  2. Testing is outsourced. Some organizations will choose to outsource their testing to an external organization that is focused on testing.


But That Isn’t Agile!

Bullshit! Please excuse my language. Disciplined agile is pragmatic, going beyond the limited approach promoted by many agile purists.  These purists will claim that you don’t need parallel independent testing, and in simple situations this is clearly true. The good news is that it’s incredibly easy to determine whether or not your independent testing effort is providing value: simply compare the likely impact of the defects/change stories being reported with the cost of doing the independent testing. In short, whole team testing works well for agile in the small, but for more complex systems and in some tactical scaling situations you need to be more sophisticated.

Lean Thinking Provides a Philosophical Foundation for Scaling Agile

Little ideas add up

Lean thinking is important for scaling agile in several ways:

  1. Lean provides an explanation for why many of the agile practices work.  For example, Agile Modeling’s practices of light weight, initial requirements envisioning followed by iteration modeling and just-in-time (JIT) model storming work because they defer commitment to the last most responsible moment.  These practices also help to eliminate waste because you’re only modeling what needs to be built at the point in time that it needs to be built.
  2. Lean offers insight into strategies for improving your software process.  Lean principles such as optimizing the whole and delivering quickly motivate you to look beyond your existing specialized processes to explore how everything fits together and to streamline it.  Identifying and understanding the sources of waste in your IT processes can motivate you to improve the way that you work and thereby eliminate the waste.  The lean principle of building quality in
  3. Lean thinking provides a philosophical foundation for scaling agile approaches.  No methodology, process, procedure, or framework is ever complete.  Nor can they be because you can always capture more detail for a wider range of situations.  Because of this incompleteness you need a collection of higher-level principles or philosophies to guide people when their process/procedure/… proves to be incomplete for the situation that they face.  Lean thinking has proven to be a very good source of such philosophies, as do other sources (Steven Covey’s principles come to mind, as does the work of Peter Senge).
  4. Lean provides techniques for identifying waste.  Value stream mapping, a technique common within the lean community whereby you model a process and then identify how much time is spent on value-added work versus wait time, helps calculate overall time efficiency of what you’re doing.  Value stream maps are a straightforward way to illuminate your IT processes, providing insight into where significant problems exist.  I’ve created value stream maps with several customers around the world where we analyzed their existing processes which some of their more traditional staff believed worked well only to discover they had efficiency ratings of 20-30%.  You can’t fix problems that you are blind to.

Disciplined Agile Program Management: External Workflow

In this blog posting, the latest in our ongoing disciplined agile program management series,  we overview the external workflows that a large delivery team is likely to be involved with.

Workflow With Other IT Teams

The following diagram overviews the major workflows that a disciplined agile program is associated with.  Note that feedback is implied in the diagram.  For example, where you see the Technology Roadmap and Guidance flow from Enterprise Architecture to Program Management there is an implied feedback loop from the program to the enterprise architects.  Also note that the workflows do not necessarily imply that artifacts exist.  For example, the data guidance workflow from Data Management could be a conversation with a data management person, it could be a concise description of data standards, or it could be detailed meta data – or combinations thereof.  A second example would be a program providing their development intelligence to the IT governance team through automated rollup of metric data via your organizations dashboard technology.

Disciplined Agile Program Management

The following table summarizes the workflows depicted in the diagram.

Process Blade Process Blade Overview Workflow with Program Management
Continuous Improvement Addresses how to support process and organizational structure improvement across teams in a lightweight, collaborative manner; how to support improvement experiments within teams; and how to govern process improvement with your IT department. Your continuous improvement efforts should result in improvement suggestions gleaned from other teams that the program can learn from.
Data Management Addresses how to improve data quality, evolve data assets such as master data and test data, and govern data activities within your organization. The data management group will provide data guidance, such as naming conventions and meta data regarding legacy data sources, to all delivery teams.
Enterprise architecture Addresses strategies for collaborative and evolutionary exploration, potential modelling, and support of an organization’s architectural ecosystem in a context-sensitive manner. The enterprise architects will produce a technology roadmap that delivery teams should follow and be a good source of development guidance (such as programming guidelines, user interface conventions, security guidelines, and so on).  Delivery teams will provide development intelligence (metrics) and feedback pertaining to the usage of key architectural components and frameworks to help inform the decisions of the enterprise architects.
IT Delivery Addresses how to develop solutions in a disciplined agile manner.  This includes the four lifecycles – basis/agile, advanced/lean, continuous delivery, and exploratory – supported but the DAD framework plus the program management blade (effectively a large team following one or more of the lifecycles). There will be dependencies,both technical and functional, with other delivery teams (not shown in the diagram).  These dependencies between teams must be negotiated and managed appropriately.
IT Governance Addresses strategies for consolidating various governance views, defining metrics, taking measurements, monitoring and reporting on measurements, developing and capturing guidance, defining roles and responsibilities, sharing knowledge within your organization, managing IT risk, and coordinating the various governance efforts (including EA governance). The IT governance team will provide guidance to all IT teams, including large delivery teams.  This guidance typically focused on financial and quality goals as well as any regulatory constraints where appropriate.  Delivery teams will provide development intelligence to the IT governance team to enable them to monitor your team and provide informed guidance to it.
Operations Addresses how to run systems, evolve the IT infrastructure, manage change within the operational ecosystem, mitigate disasters, and govern IT operations. Your operations group will provide operations intelligence (metrics) to IT delivery teams, in particular around the usage of systems and features that a team is responsible for.  This enables the IT delivery teams to make informed decisions regarding the value of delivered features.
Portfolio Management Addresses how to identify potential business value that could be supported by IT endeavors, explore those potential endeavors to understand them in greater detail, prioritize those potential endeavours, initiate the endeavours, manage vendors, and govern the IT  portfolio. Your organization’s portfolio management activities will provide the initial vision and funding required to initiate a program, as well as ongoing funding for the program.  It will also provide guidance, often around management and governance conventions, to the team.  IT delivery teams will make their development intelligence (metrics) available to the portfolio management team to help inform their decisions.
Product Management Addresses strategies for managing a product, including allocating features to a product, evolving the business vision for a product, managing functional dependencies, and marketing the product line. The Product Management team will provide a business roadmap and stakeholder priorities to all IT delivery teams, including programs.
Release Management Addresses strategies for planning the IT release schedule, coordinating releases of solutions, managing the release infrastructure, supporting delivery teams, and governing the release management efforts. Your program will release solutions into production via your organization’s release management strategy.
Reuse Engineering Addresses how to identify and obtain reusable assets, publish the assets so that they are available to be reused, support delivery teams in reusing the assets, evolving those assets over time, and governing the reuse efforts. All IT delivery teams should reuse existing assets – such as services, frameworks, and legacy data sources – whenever appropriate.
Support Addresses how to adopt an IT support strategy, to escalate incidents, to effectively address the incidents, and govern the IT support effort. Your support/help-desk team will provide change requests, including defect reports, identified by end users to all delivery teams.  These change requests are in effect new requirements.

The activities associated with these process blades are often very highly related. For example, in some organizations the activities associated with enterprise architecture and reuse management are fulfilled by a single group.  In other organizations some product management activities are performed by the portfolio management team and some by the enterprise architecture team.  Some organizations may choose to have a separate group for each process blade.  And of course the organizational structure will evolve over time as your various teams learn how to work with one another.  Every organization is different.

Program Management and DevOps

A common question that we’ve gotten is how program management is affected by DevOps. For example, you see in the diagram that Operations, Support, and Release Management (amongst others) are shown as things that are external to Program Management.  Remember that the focus here is on process, not on team organization.  For example, in organizations with a disciplined DevOps strategy in place it is very common to see program teams taking on the responsibilities of operating and supporting their own systems in production, and of doing the work to release their solutions into production.  In organizations without a strong DevOps mindset (yet), you are likely to find that operations, support, and release management are done by separate groups outside of your program team.  Context counts, and it’s good to have a process framework that is flexible enough to support the situation that you find yourself in.

Related Postings