Tag Archives: disciplined agile delivery

Successful Agile Software Development Requires a Hybrid Approach

Hybrid

Disciplined Agile Delivery (DAD) is a hybrid framework that builds upon the solid foundation of other methods and software process frameworks.  The DAD framework adopts practices and strategies from existing sources and provides advice for when and how to apply them together.  In one sense methods such as Scrum, Extreme Programming (XP), Kanban, and Agile Modeling (AM) provide the process bricks and DAD the mortar to fit the bricks together effectively.

The DAD framework adopts strategies from the following sources:

  • Scrum.  The Scrum method focuses on team leadership and requirements change management during the construction portion of the delivery lifecycle.  Scrum captures some really great ideas that have become commonly adopted by agile teams.
  • Extreme Programming (XP). XP is an agile method that focuses primarily on construction-oriented practices such as continuous integration (CI), test-driven development (TDD), pair programming, sustainable pace, small releases, simple design, and many others.  Although XP’s practices appear straightforward on the surface, many of them require significant technical skill and discipline on the part of practitioners.
  • Lean software development.  Practitioners of agile development are increasing looking to adapt ideas from lean thinking.  Lean software development is based on seven principles: Eliminate waste, amplify learning, decide as late as possible, deliver as fast as possible, empower the team, build integrity in, and see the whole.
  • Kanban.  Kanban is a method for managing knowledge work with an emphasis on just-in-time delivery while not overloading the team members. In this approach, the process, from definition of a task to its delivery to the customer, is visualized and team members pull work from a queue or work item pool.
  • Unified Process (UP).  The UP is an iterative and incremental process framework for software development.  Although often instantiated as a heavy-weight approach, it has in fact been instantiated in a very light weight and agile manner, particularly in the form of Agile Unified Process (AUP) or OpenUP.  The DAD framework adopts and enhances several critical governance strategies from the UP.
  • Agile Modeling (AM).  AM is a practice-based methodology for effective modeling and documentation of software-based system. AM was purposely architected to be a source of strategies which can be tailored into other base processes, something the DAD framework takes explicit advantage of.   AM strategies adopted by DAD include initial requirements and architecture envisioning, just in time (JIT) model storming, continuous documentation, and several others.
  • Agile Data.  The Agile Data (AD) method defines a collection of strategies that IT professionals can apply in a wide variety of situations to work together effectively on the data aspects of software systems. Practices for evolutionary/agile database development include database refactoring, database regression testing, agile data modeling, and continuous database integration.
  • Enterprise methods. DAD is starting to adopt strategies from enterprise IT methods such as Scaled Agile Framework (SAFe) and Enterprise Unified Process (EUP).  These strategies address enterprise architecture, product management, portfolio management, operations and support, release management and other important IT disciplines. These strategies reflect the DAD philosophy of enterprise awareness.
  • Other methods.  DAD adopts techniques and practices from other methods such as Dynamic System Development Method (DSDM), Feature Driven Development (FDD), Evo, Outside-In Development (OID), and Crystal Clear.
  • Agile development practices.  There are many practices that are not specific to agile methods that have been developed within the agile community, and DAD leverages many of them.

One of the great advantages of agile and lean software development is the wealth of practices, techniques, and strategies available to you.  This is also one of its greatest challenges because without something like the DAD framework it’s difficult to know what to choose and how to fit them together.

Evolving the Goals Overview Diagram

A couple of months ago we decided to evolve the DAD goals overview diagram. The new version is shown in Figure 1 and the previous version in Figure 2.

Figure 1. The new goals overview diagram.
Lifecycle Goals

Figure 2. The previous diagram.
Lifecycle Goals

There are several interesting changes in the diagram:

  1. It’s a mind map. We’ve always been uncomfortable with the table format of Figure 2 but had never gotten around to updating the diagram.  Based on feedback from several people, including a few Black Belts, we moved towards using mind map notation.
  2. Color.  We’re using a color scheme that is consistent with the colors used in the DAD lifecycle diagrams.  Expect updates to the process goal diagrams in the near future for consistency.
  3. Team focus.  For the past year or so we’ve been moving away from the project-oriented terminology that we used in the DAD book.  For example, in Figure 2 we now say “Fulfill the Team Mission” instead of “Fulfill the Project Mission”.  We’re seeing DAD applied more in more by product teams, often following a lean or continuous delivery version of the lifecycle, that have evolved beyond the project mindset.  So we felt we should refactor away from the project-oriented terminology we used in the first edition of the DAD book and thereby remove some unfortunate bias we may have injected into the DAD framework.

As always, we’re open to any feedback that you may have to help us improve this material.  Thanks in advance.

Whitepaper: Going Beyond Scrum

Going Beyond Scrum

This paper describes, step-by-step, how to evolve from today’s Scrum vision of agile software development to a disciplined agile solution delivery.  It begins with a brief overview of the agile software development movement and its implications.  We then overview the Scrum method with its associated benefits and drawbacks, and then how to move beyond Scrum to a full delivery process framework called Disciplined Agile Delivery (DAD).  DAD is a governed, hybrid approach that provides a solid foundation from which to scale agile solution delivery within enterprise-class organizations.    The steps to do this are:

  1. Focus on consumable solutions, not just potentially shippable software
  2. Extend Scrum’s construction lifecycle to address the full delivery lifecycle
  3. Move beyond method branding
  4. Adopt explicit governance strategies
  5. Take a goal-based approach to enable tailoring and scaling

You can download the paper here.

Strategies for Verifying Non-Functional Requirements

Early in the lifecycle, during the Inception phase, disciplined agile teams will invest some time in initial requirements envisioning and initial architecture envisioning. One of the issues to be considered as part of requirements envisioning is to identify non-functional requirement (NFRs), also called quality of service (QoS) or simply quality requirements. The NFRs will drive many of your technical decisions that you make when envisioning your initial architectural strategy. These NFRs should be captured someone, in a previous blog I explored the options available to you, and implemented during Construction. It isn’t sufficient to simply implement the NFRs, you must also validate that you have done so appropriately. In this blog posting I overview a collection of agile strategies that you can apply to validate NFRs.

A mainstay of agile validation is the philosophy of whole team testing. The basic idea is that the team itself is responsible for validating its own work, they don’t simply write some code and then throw it over the wall to testers to validate. For organizations new to agile this means that testers sit side-by-side with developers, working together and learning from one another in a collaborative manner. Eventually people become generalizing specialists, T-skilled people, who have sufficient testing skills (and other skills).

Minimally your developers should be performing regression testing to the best of their ability, adopting a continuous integration (CI) strategy in which the regression test suite(s) are run automatically many times a day.  Advanced agile teams will take a test-driven development (TDD) approach where a single test is written just before sufficient production code which fulfills that test.  Regardless of when tests are written by the development team, either before or after the writing of the production code, some tests will validate functional requirements and some will validate non-functional requirements.

Whole team testing is great in theory, and it is strategy that I wholeheartedly recommend, but in some situations it proves insufficient.  It is wonderful to strive to have teams with sufficient skills to get the job done, but sometimes the situation is too complex to allow that.  There are some types of NFRs which require significant expertise to address properly: NFRs pertaining to security, usability, and reliability for example.  To validate these types of requirements, worse yet even to identify them, requires skill and sometimes even specialized (read expensive) tooling.  It would be a stretch to assume that all of your delivery teams will have this expertise and access to these tools.

Recognizing that whole team testing may not sufficiently address validating NFRs many organizations will supplement their whole team testing efforts with parallel independent testing  .  With this approach a delivery team makes their working builds available to a test team on a regular basis, minimally at the end of each iteration, and the testers perform the types of testing on it that the delivery team is either unable or unlikely to perform.  Knowing that some classes of NFRs may be missed by the team, independent test teams will look for those types of defects.  They will also perform pre-production system integration testing and exploratory testing to name a few.  Parallel independent testing is also common in regulatory compliance environments.

From a verification point of view some agile teams will perform either formal or informal reviews.  Experienced agilists prefer to avoid reviews due to their inherently long feedback cycle, which increases the average cost of addressing found defects, in favor of non-solo development strategies such as pair programming and modeling with others.  The challenge with non-solo strategies is that managers unfamiliar with agile techniques, or perhaps the real problem is that they’re still overly influenced by disproved traditional theories of yesteryear, believe that non-solo strategies reduce team productivity.  When done right non-solo strategies increase overall productivity, but the political battle required to convince management to allow your team to succeed often isn’t worth the trouble.

Another strategy for validating NFRs code analysis, both dynamic and static.  There is a range of analysis tools available to you that can address NFR types such as security, performance, and more.  These tools will not only identify potential problems with your code many of them will also provide summaries of what they found, metrics that you can leverage in your automated project dashboards.   This strategy of leveraging tool-generated metrics such as this is a technique which IBM calls Development Intelligence and is highly suggested as an enabler of agile governance in the DAD framework. Disciplined agile teams will include invocation of code analysis tools from you CI scripts to support continuous validation throughout the lifecycle.

Your least effective validation option is end-of-lifecycle testing, in the traditional development world this would be referred to as a testing phase.  The problem with this strategy is that you in effect push significant risk, and significant costs, to the end of the lifecycle.  It has been known for several decades know that the average cost of fixing defects rises the longer it takes you to identify them, motivating you to adopt the more agile forms of testing that I described earlier.  Having said that I still run into organizations in the process of adopting agile techniques that haven’t really made embraced agile, as a result still leave most of their testing effort to the least effective time to do such work.  If you find yourself in that situation you will need to validate NFRs in addition to functional requirements.

To summarize, you have many options for validating NFRs on agile delivery teams.  The secret is to pick the right one(s) for the situation that you find yourself in.  The DAD framework helps to guide you through these important process decisions, describing your options and the trade-offs associated with each one.  For a more detailed discussion of agile validation techniques you may find my article Agile Testing and Quality Strategies to be of value.

Strategies for Implementing Non-Functional Requirements

Non-functional requirements, also known as quality of service (QoS) or technical requirements, are typically system-wide thus they apply to many, and sometimes all of your functional requirements.  Part of ensuring that your solution is potentially consumable each iteration is ensuring that it fulfill its overall quality goals, including applicable NFRs.  This is particularly true with life-critical and mission-critical solutions.  Good sources for NFRs include your enterprise architects and operations staff, although any stakeholder is a potential source for NFRs.

Chapter 8 in the Disciplined Agile Delivery book, written by Mark Lines and myself, overviews several strategies for capturing and then implementing NFRs.  As your stakeholders tell you about functional requirements they will also describe non-functional requirements (NFRs).  These NFRs may describe security access rights, availability requirements, performance concerns, or a host of other issues as saw in my blog regarding initial architecture envisioning.  There are three basic strategies, which can be combined, for capturing NFRs:

  1. Technical stories.  A technical story is a documentation strategy where the NFR is captured as a separate entity that is meant to be addressed in a single iteration.  Technical stories are in effect the NFR equivalent of a user story. For example “The system will be unavailable to end users no more than 30 seconds a week” and “Only the employee, their direct manager, and manager-level human resource people have access to salary information about said employee” are both examples of technical stories.
  2. Acceptance criteria for individual functional requirements.  Part of the strategy of ensuring that a work item is done at the end of an iteration is to verify that it meets all of its acceptance criteria.  Many of these acceptance criterions will reflect NFRs specific to an individual usage requirement, such as “Salary information read-only accessible by the employee,”, “Salary information read-only accessible by their direct manager”, “Salary information read/write accessible by HR managers”, and “Salary information is not accessible to anyone without specific access rights”.  So in effect NFRs are implemented because they become part of your “done” criteria.
  3. Explicit list.  Capture NFRs separately from your work item list in a separate artifact.  This provides you with a reminder for the issues to consider when formulating acceptance criteria for your functional requirements.  In the Unified Process this artifact was called a supplementary specification.

Of course a fourth option would be to not capture NFRs at all.  In theory I suppose this would work in very simple situations but it clearly runs a significant risk of the team building a solution that doesn’t meet the operational needs of the stakeholders.  This is often a symptom of a teams only working with a small subset of their stakeholder types (e.g. only working with end users but not operations staff, senior managers, and so on)

So what are the implications for implementing NFRs given the three previous capture strategies?    Although in the book we would make this sort of comparison via a table to improve consumability, in this blog posting I will use prose due to width constraints.  Let’s consider each one:

  1. Technical stories.  The advantages of this approach are that it is a simple strategy for capturing NFRs and that it works well for solutions with a few NFRs or simple NFRs.  But, the vast majority of NFRs are cross-cutting aspects to several functional stories and as a result cannot be implemented within a single iteration.  This strategy also runs the risk of teams leaving NFRs to the end of the construction phase, thereby pushing technical risk to the end of the lifecycle where it is most difficult and expensive to address.
  2. Acceptance criteria. This is a quality focused approach which makes the complexity of an individual functional requirement apparent, working well with test driven approaches to development.  NFR details are typically identified on a just in time (JIT) basis during construction, fitting in well with a disciplined agile approach.  But, because many NFRs are cross cutting the same NFR will be captured for many functional requirements.  It requires the team to remember and consider all potential NFR issues (see Figure in my previous posting) for each functional requirement.  You will still need to consider NFRs as part of your initial architecture efforts otherwise you risk a major rework effort during the Construction phase because you missed a critical cross-cutting concern).
  3. Explicit list.  This strategy enables you to explore NFRs early in the lifecycle and then address them in your architecture.  The list can be used to drive identification of acceptance criteria on a JIT basis.  But, NFR documents can become long for complex systems (due to the large number of NFRs).  This can be particularly problematic when you have a lot of NFRs that are specific to a small number of functional requirements.  Teams lacking in discipline may not write down the non-functional requirements and trust that they will remember to address them when they’re identifying acceptance criteria for individual stories.

 

The advice that Mark and I give in the book is that in most situations you should maintain an explicit list and then use that to drive identification of acceptance criteria as we’ve found that it’s more efficient and lower risk in the long run.  Of course capturing NFRs is only one part of the overall process of addressing them.  You will also need to implement and validate them during construction, as well as address them in your architecture.

An important issue which goes to NFRs such as consumability, supportability, and operability, is that of deliverable documentation.  At the start of the project is the best time to identify the required documentation that must be created as part of the overall solution.  This potentially includes operations manuals, support manuals, training materials, system overview materials (such as an architecture handbook), and help manuals to name a few.  These deliverable documents will be developed and kept up to date via the continuous documentation practice.

In my next blog posting, the fourth in this three-part series, I will describe strategies for verifying non-functional requirements.

Disciplined Agile Architecture: Initial Architecture Envisioning

An important aspect Disciplined Agile Delivery (DAD) is its explicit inclusion of an Inception phase where project initiation activities occur.  Although phase tends to be a swear word within the agile community, the reality is that the vast majority of teams do some up front work at the beginning of a project.  Some people will mistakenly refer to this effort this Sprint/Iteration 0 it is easy to observe that on average this effort takes longer than a single iteration (the 2009 Agile Project Initiation survey  found the average agile team spends 3.9 weeks in Inception and the November 2010 Agile State of the Art survey found that agile teams have Construction iterations of a bit more than 2 weeks in length).

Regardless of terminology, agile teams are doing some up front work.  Part of that initial work is identifying an initial technical architecture, typically via some initial architecture envisioning http://www.agilemodeling.com/essays/initialArchitectureModeling.htm.  Because your architecture should be based on actual requirements, otherwise you’re “hacking in the large”, your team will also be doing some initial requirements envisioning  http://www.agilemodeling.com/essays/initialRequirementsModeling.htm in parallel.  Your architecture will be driven in part by functional requirements but more often the non-functional requirements, also called quality of service (QoS) or simply quality requirements.  Some potential quality requirements are depicted in the figure below (this figure is taken from the Disciplined Agile Delivery book but was first published in Agile Architecture Strategies ).

Architectural views and concerns

Some architects mistakenly believe that you need to do detailed up front modeling to capture these quality requirements and then act upon them.  This not only isn’t true it also proves to be quite risky in practice, see my discussion about Big Modeling Up Front (BMUF)  for more details.  Disciplined agilists instead will do just enough initial modeling up front and then address the details on a just-in-time (JIT) basis throughout construction.  Of course it’s important to recognize that just enough will vary depending on the context of the situation, teams finding themselves at scale will need to do a bit more modeling than those who don’t.  It’s also important to recognize that to address non-functional requirements throughout construction that you need to have more than just architectural modeling skills.  This topic will be the focus of my next blog posting in this series.

Potential Misconceptions about Agile Architecture

Recently at the Scott W. Ambler + Associates site we received a series of questions from someone who wanted to better understand how architecture issues are addressed on agile project teams.  It seemed to me that the questions were sufficiently generic to warrant a public response instead of a private one.  So, over the next few days I’m going to write several blog postings here to address the issues that were brought up in the questions.  It’s important to note that I will be answering from the point of view of Disciplined Agile Delivery (DAD), and not agile in general.  Other agile methods may provide different advice than DAD does on this subject, or no advice at all in some cases.

The goal of the first blog posting in this series is to address several potential misconceptions that appeared in the email.  I want to start here so as to lay a sensible foundation for the follow-on postings.

Partial Misconception #1: Agile can be prefixed in iteration 0 by architectural design

I’ve named this a “partial misconception” for a few reasons:

  1. Disciplined agile teams do some up-front work.  This is called the Inception Phase in DAD, although other methods may refer to it as iteration/sprint 0, warm up, initiation, or other names.  Up-front work is an explicit part of DAD.
  2. Iteration 0 isn’t an accurate term.  Although I have used this term in the past when discussing project initiation, the reality is that the average agile team spends about a month doing project initiation activities whereas the average iteration length is two weeks.  So, Inception really isn’t a proper iteration.
  3. Inception is more than just architecture.  Several activities typically occur at this point in time, particularly initial architecture envisioning, initial requirements envisioning, initial release planning, and putting the team together to name a few things.

Chapters 6 through 12 in Disciplined Agile Delivery describe these project initiation activities in detail.  Also, I recently wrote that it requires discipline to keep Inception short.

Partial Misconception #2: On principle, Agile is against “big” anything

This is also a “partial misconception” for several reasons:

  1. There is in fact a lot of agile rhetoric against big artifacts.  It’s very easy to find agile writings about the challenges with big requirements up front (BRUF), big modeling up front (BMUF) in general, and detailed up front planning for instance.
  2. Disciplined agile is against needless waste, not “big” things.  Many traditional modeling and planning practices prove to be quite wasteful in practice.  A serious cultural challenge that the traditional community has is that they are afraid to throw out the bathwater because they assume that the baby will go with it.  I believe that Disciplined Agile Delivery (DAD), and Agile Modeling before it, make it quite clear that it’s possible to gain the benefit of thinking before doing without taking on the very serious problems around doing too much thinking before doing.  So, have the discipline to keep the thinking “baby” yet discard any needless documentation “bathwater”.
  3. In rare situations it’s appropriate to create “big” artifacts.  Disciplined agilists aim for sufficient artifacts, the size of which will depend on the context of the situation that your team finds itself in.  In a recent article for Dr. Dobb’s Journal, Disciplined Agile Architecture, I explicitly explored how initial architecture envisioning on an agile project may result in “big” artifacts in some situations.  These situations are very rare mind you, ignoring cultural imperatives to create big artifacts because some people still haven’t made the jump to a disciplined agile approach, but they do happen.  One of the strengths of the DAD process decision framework is that it is goal driven, not prescriptive, and explicitly explores the tradeoffs surrounding the amount of detail to capture and when to do so.

Partial Misconception #3: Refactoring system architecture beyond mid-implementation is much more expensive than refactoring components

Once again, this is a partial misconception.  I suspect part of the problem is a lack of understanding of what refactoring is really all about, a recurring problem with experienced traditionalists, and part because of a lack of understanding of how architecture is address by disciplined agile teams.  Some thoughts:

  1. Refactorings are simple, not difficult.  The goal of refactoring is to make SMALL changes to your design that improve the quality without changing the semantics of the design in a practical manner.  This is true of code refactorings, database refactorings, user interface refactorings, and other types of refactorings.  Small changes are inexpensive to make given the appropriate skills, tools, and organizational environment.
  2. Architectural rework (not refactoring) is often difficult.  Rework, or rewrites, are very large changes the goal of which is typically to replace large portions of your solution.  Yes, the later in the lifecycle such rework occurs very likely the more expensive it will be because you’ve built more based on that architecture that is now being reworked.  This is a general issue, not just an agile one.
  3. Disciplined agile teams get going in the right direction to begin with. The practice of initial architecture envisioning, which we describe in detail in Chapter 9 of Disciplined Agile Delivery, aims to think through the architectural strategy before getting into construction.
  4. Disciplined agile teams prove their architecture works early. The first construction milestone, prove the architecture, reduces the risk of architectural rework.  The goal is to prove that the architecture works by building a working end-to-end skeleton of the solution which implements critical/difficult technical requirements.  This is an agile “fail fast” strategy, or as we say in DAD a “succeed early” strategy, that reduces technical risk on your project.   As an aside, including explicit light-weight milestones such as this is one of many agile governance aspects built right into DAD.
  5. Disciplined agile teams have an architectural role.  This role is called Architecture Owner and one of the responsibilities of the person in this role is to guide the team in architectural issues throughout the entire DAD lifecycle.
  6. There are no guarantees.   No matter how smart your approach, there’s still a chance that rework can happen.  For instance, you can be mid-way through a project and the vendor of a major architectural component of your solution decides to withdraw it from the market.  Or the vendor goes out of business.  Or perhaps your firm is taken over by another firm and the new owners decide to inflict, oops I mean bless you with, their architectural strategy.  Stuff happens.  Once again, this is a general issue, not specifically an agile one.
  7. Quality decreases the cost of rework.  Disciplined agilists will write high-quality code, with a full regression test suite in place, at all times during Construction.  It’s easier to rework high quality artifacts compared with low quality artifacts, so if you get stuck having to perform rework at least the pain is minimized.  My article Agile Testing and Quality Strategies overviews many techniques.

In short, disciplined agile teams do what they can to avoid architectural rework to begin with by having an explicit architecture owner role who focuses on architectural issues throughout the entire lifecycle, by identifying a viable architectural strategy early in the project, proving that architectural strategy works early in Construction, and producing high-quality artifacts throughout the lifecycle that are easier to rework if needed.   With continuous documentation practices and a focus on producing artifacts which are just sufficient enough for the situation at hand, this proves to be far more effective than traditional strategies that assume you require large up-front investments in “big” artifacts, that rely on validation techniques such as architecture reviews instead of the far more concrete feedback of working code, and that often leave quality strategies to the end of the lifecycle (thereby increasing the cost of any rework).

I plan two follow-on blog postings in this series, one exploring how initial architecture envisioning works and one about how to address initial quality requirements (also called non-functional requirements or quality of service requirements) on disciplined agile projects.  Stay tuned!

At Scott W. Ambler + Associates we offer a one-day workshop entitled Agile Architecture: A Disciplined Approach that you should consider if you’re interested in this topic.  We also offer coaching and mentoring services around agile architecture.

It Requires Discipline to Keep Inception Short

The Disciplined Agile Delivery (DAD) process framework includes an explicit Inception phase – sometimes called a project initiation phase, startup phase, or iteration/sprint zero – which is conducted before actually starting to build a solution.  The goals of this phase include: clarifying the business problem that needs to be solved; identifying a viable technical solution; planning your approach; setting up your work environment and team; and gaining stakeholder concurrence that it makes sense to proceed with investing in the implementation of the chosen strategy.  These goals are listed in the following diagram.

Disciplined Agile Inception Phase Goals

Form Initial Team Develop Common Vision Align With Enterprise Direction Explore Initial Scope Identify Initial Technical Strategy Develop Initial Release Plan Secure Funding Form Work Environment Identify Risks

In the Disciplined Agile Delivery book we devoted a lot space to describing how to effectively initiate a DAD project.  Unfortunately in our experience we have seen many organizations that are still new to agile treat this phase as an opportunity to do massive amounts of upfront documentation in the form of project plans, charters, and requirements specifications.  Some people have referred to the practice of doing too much temporary documentation up front on an agile project as Water-Scrum-Fall.  We cannot stress enough that this is NOT the intent of the Inception phase.  While we provide many alternatives for documenting your vision in Inception, from very heavy to very light, you should take a minimalist approach to this phase and strive to reach the stakeholder consensus milestone as quickly as possible.

According to the 2013 Agile Project Initiation survey the average agile team invests about 4 weeks performing project initiation activities, including initial requirements envisioning, initial architecture envisioning, forming the team, initial release planning, and so on.  Of course this is just an average, some respondents reported investing less than a week to do so and some reporting investing more than two months – the amount of time required varies depending on the complexity of the effort, your stakeholders’ understanding of their requirements, your team’s understanding of the solution architecture, whether this is a new solution or merely a new release of an existing solution, and many others.

If you are spending more than a few weeks on this phase, you may be regressing to a Water-Scrum-Fall approach.  It takes discipline to be aware of this trap and to streamline your approach as much as possible.  You can do this in several ways:

  1. Recognize that the goal is to get going in the right direction.  For any project or product of reasonable complexity you want to spend a bit of time up front ensuring that you understand the problem and have what you believe to be a viable strategy to addressing that problem.  This doesn’t imply that you need a comprehensive requirements specification, a detailed project plan, nor a comprehensive architecture model but it does mean you need to do a bit of initial thinking and organizing.  In a small handful of cases, typically at scale, you may find that your team does require detailed models and plans but this is a very uncommon exception (regardless of what traditional THEORY may claim).
  2. Educate people that details can safely come later.  If you have the ability to plan or model something in detail today, won’t you also have that same ability a few months from now when you actually need those details?  Of course you do.   In lean software development they recommend deferring decisions – including planning decisions,  detailed design decisions, and even requirements – until the most appropriate moment.  The observation is that by waiting you can make a better decision because you have better information at hand.  This strategy of course assumes that you’re able to overcome basic logistical problems such as having the appropriate people available at the time to help explore an issue, provide requisite information, and make the decision.   It’s far less risky, and far less expensive, in most cases to address basic logistical issues than it is to apply the process band-aid of writing detailed documentation at the beginning of a project.
  3. Promote a sense of urgency.  This is the most important thing that you can do.  Just as there is risk associated with not sufficiently thinking about your strategy for approaching a new project or product there are also risks associated with doing too much up front work.  My experience is that far too many IT professionals are complacent regarding the risks associated with the project initiation activities of the Inception phase.   The longer you put off building a consumable solution the greater the risk that you’re building the wrong thing.
  4. Keep your modeling efforts as light as possible.  You very likely need to do some initial requirements envisioning and architecture envisioning early in the project lifecycle to help you think through what you’re doing.  But in most cases this modeling should be high level and light, not detailed and heavy.  In every project I’ve ever been involved with the team has been asked to identify what they’re going to deliver (at least giving a rough sense of the scope) and how they’re going to do it (at least providing a rough idea of the technical strategy) to secure funding for construction.  In short, they need to do a bit of up front thinking.  You will often find that you need to reign-in some of your staff who are experienced with traditional approaches to modeling and specification.  These people have a lot of value to add to your project, modeling is an important skill needed on disciplined agile teams, but they may need help keeping their approach light-weight and incremental.   The details can come later.  See the process goals Identify Initial Scope and Identify Initial Technical Strategy for more thoughts on this subject.
  5. Keep your planning efforts as light as possible.  Similarly you need to invest some time in high-level release planning to answer basic questions such as how long do you believe (roughly) it will take to get a release of your solution deployed and how much (roughly) this will cost.  Planning details can come throughout the Construction phase when it’s more appropriate to invest in such decisions.

I think that it’s very clear that the secret to keeping Inception short is to have the discipline to know that you need to invest some time thinking your approach through but that you want to avoid getting bogged down in too many details.  You need the discipline to do some planning but not too much.  You need the discipline to do some modeling but not too much.  You need the discipline to get going in the right direction knowing that the details will come out in time.

Comparing DAD to the Rational Unified Process (RUP) – Part 1

Last week I was discussing DAD with a new client and he asked me “Is DAD just an Agile version of RUP?”  In a word, no.  DAD is a framework composed of a hybrid of methods and practices as shown in the diagram.  It includes the best of Scrum, Extreme Programming (XP), Agile data and modeling, and yes, the Unified Process (UP).  DAD also includes additional content such as agile governance that is not present in any of these methods.  As the diagram indicates, probably the method that adds most to DAD is XP, not the UP.
The Rational Unified Process (RUP) started as a small manageable process framework targeted specifically for building software within the context of an iterative lifecycle.  However over time, Rational (and subsequently IBM) added additional guidance and artifacts to extend the applicability of RUP to all sorts of projects, such as package implementation, maintenance projects, technology specific guidance (J2EE, .Net etc.), systems engineering and may other project types.  It became unwieldy and hard to understand and apply successfully.  In fact it is frequently misused (with the Elaboration phase often being treated as a big requirements upfront (BRUF) phase as an example).  This misuse has been described by Julian Holmes as RINO (RUP in name only).  To be clear, RUP properly applied in the right circumstances can be very effective.  Unfortunately though, that often does not happen.  One of the issues with applying the RUP framework to different types of projects is that it is described as a “Use case-driven” approach.  Specifying requirements as use cases, and then creating component-based architecture from these use case realizations is fundamental to RUP.  This presents challenges for maintenance projects or package implementations where it may not make sense to produce use cases at all.

DAD does not prescribe a use case-driven approach, or insist that OOAD be rigorously applied to build out services/components.  A use case-driven approach is a potential practice to apply but there is a danger that this could lead to an exhaustive requirements specification which is not particularly agile.  We would prefer to use a user story-driven approach if that makes sense within the context of your project.  User stories might not be the right choice either.  Perhaps you are in a regulatory environment that demands a traditional software requirements specification (SRS).  The key point is that you will have to adapt to the situation that you find yourself in.  This is why we prioritize the team’s work with a work item list comprised of work items, rather than Scrum’s backlog comprised of user stories.  Using a work item list allows us the flexibility to put any type of work onto our backlog, extending the applicability of DAD to many types of projects beyond those for which RUP or Scrum would be ideally suited.

DAD is goal-driven, not artifact-driven.  It does not prescribe practices or specific artifacts.  Rather, it suggests alternative strategies than can be applied at certain parts of the lifecycle with the pros and cons for each, but which ones you choose is up to you.

In my next post I will describe which aspects of the Unified Process did make it into DAD and why.

The DAD Role of Architecture Owner

As you can see from this diagram, the DAD primary roles are similar to those of Scrum.  In Scrum, the product owner decides what will be built and in what order.  In DAD we recognize that architecture is a key source of project risk and someone needs to be responsible for ensuring the team mitigates this risk.  As a result the DAD process framework explicitly includes Agile Modeling’s role of architecture owner. The architecture owner is the person who owns the architecture decisions for the team and who facilitates the creation and evolution of the overall solution design.  The person in the role of team lead will often also be in the role of architecture owner.  This isn’t always the case, particularly at scale, but it is very common for smaller agile teams.

The responsibilities of the architecture owner include:

  • Guiding the creation and evolution of the architecture of the solution that the team is working on.  Note that the architecture owner is not solely responsible for the architecture, but that they lead the technical discussions.
  • Mentoring and coaching other team members in architecture practices and issues.
  • Understanding the architectural direction and standards of your organization and helping to ensure that the team adheres to them appropriately.
  • Understanding existing enterprise assets such as frameworks, patterns, subsystems and ensuring that the team uses them where appropriate.
  • Ensuring that the solution will be easy to support by encouraging good design and refactoring to minimize technical debt.
  • Ensuring that the solution is integrated and tested on a regular basis, ideally via the practice of continuous integration(CI).
  • Having the final say regarding technical decisions, but they try to avoid dictating the architectural direction in favor of a collaborative, team-based approach. The architecture owner should work very closely with the team lead to identify and determine strategies to mitigate key project technical risks.
  • Leads the initial architecture envisioning effort at the beginning of the project and supports the initial requirements envisioning effort (particularly when it comes to understanding and evolving the non-functional requirements for the solution).

One of the key reasons for having this role in DAD is that the architecture owner, like the product owner, has a say in work items that are added and prioritized in the work item list (backlog in Scrum parlance).  While business value is certainly a prime determinant of priorities, completing work related to mitigating technical risks is also important.  Additionally, a DAD goal is to deliver consumable solutions, not just working software.  As such, sometimes it is necessary to add work items that are technical in nature, for example related to error logging/monitoring.  Or perhaps work items need to be added to improve the continuous integration and deployment processes.

We have found that the concept of having both product and architecture owners ensures that the solution addresses both functional and non-functional requirements such as usability and supportability adequately.  In fact, on my current project, I worked with the product and architecture owners to negotiate their priorities such that the iteration underway includes not only a selection of high priority stories, but also a set of technical work items related to hardening the solution in preparation for entering the Transition phase of delivering the solution to the stakeholders.  Without a specific role of architecture owner, it can be difficult to escalate important technical work into the work item list.  As as result it is often done subversively without the knowledge of the product owner which is not a healthy practice, or worse it never gets done resulting in a poor quality solution.

Scott has written a good article that describes the architecture owner role in more depth.  You can view it here.