Category Archives: Adoption

Agile is supposed to be easy, right?

The Agile manifesto is only 4 lines, there are only 12 principles for agile software and the original Scrum Guide was less than 20 pages, so how hard can Agile be?

 

Chess has only a couple of dozen basic moves so how hard can it be to get to checkmate?

 

 

How many times have you heard?

  • All you need to be Agile is to get the team doing a few funky ceremonies

Or the approach to evolving a team from forming to performing:

  • If the team isn’t performing after a month then the manager is not effective.

Despite the apparent simplicity of Agile, transforming to Agile is hard and the learning curve is very steep.  Depending on the team and the individuals, a transformation from waterfall to a high performing Agile team can take anywhere from 6 months to a year or more.  That’s not to say that they won’t be productive over that period of time.  In fact, they will start delivering business value at the end of the first iteration, but the team will be in constant growth and development over that period of time because not only do the people have to learn a new process, they also need to learn a new way of thinking as well as a new way of working as a team.

1.0 A new process

Aside from being excited to work with the thought leaders in the agile world, the reason I joined with SA+A is because of the Discipline Agile Delivery (DAD) decision framework and the way it captured all the tough stuff that I had been wrestling with for years.  With many years of experience building agile teams, I considered myself somewhat of an expert with answers to a lot of the difficult questions.  When I reviewed the DAD framework I realized that the answers I had gained from my own personal experience would not apply in all circumstances or for all teams.  Context counts.  The DAD Agile/Basic and Lean/Advanced project lifecycles have three phases: inception, construction and transition.  Each phase has a collection of process goals, and each goal encompasses several process decision points.  And, each decision point has a collection of options to choose from.  These process goals, decision points and options have been empirically assembled by analyzing many agile teams to see what makes an agile team successful.  This framework gives a team the ability to define a process that works for them.  Is this difficult to do?  Absolutely!  Defining a process to use is always difficult. BUT…..  The framework provides some guidance. Each goal option has default options that the team can use to get started and the options are listed in preference order so the team can easily pick an initial option.  As the team evolves and matures they can chose the more advanced options and evolve and mature their process.  Does the framework include all possible options?  Definitely not, but the options are certainly extensive.  The framework has evolved over time and continues to evolve as the coaches and thought leaders gain more experience and work with more diverse teams.

No matter which lifecycle the team has chosen to use; whether the team is using classic Scrum (Basic/Agile) or Kanban (Advanced/Lean) or a tailored process created using a framework such as DAD, the team is going to need practice and discipline to get it right.  That means, executing the lifecycle, doing periodic retrospectives, updating the process and then repeating again and again until they get it right.  Changing behaviors is hard and the team will struggle to be successful.

2.0 A new way of thinking

No matter how you cut it, transforming to Agile is a cultural change and culture changes are hard.  Not only is it hard for those who want to make the change but almost inevitably there will be pockets of resistance from people are happy with “the way we’ve always done it”.  Unfortunately, political infighting is the biggest risk to innovation. It is going to happen, you may as well brace yourself for it.

The roles are changing from traditional roles (such as: developer, business analyst (BA), system analyst (SA), quality assurance (QA) and project manager (PM), to new unfamiliar roles (such as: product owner (PO), team lead (TL), architecture owner (AO) and team member).  The traditional roles may be deeply embedded in your HR hiring and performance review processes. Transitioning to the new roles will require a realignment of skills and abilities that is bound to cause great consternation amongst the team members.  Often this is a very difficult transition for project managers, in particular, because they find that a lot of the traditional tasks (i.e. work breakdown structures, planning, estimating, etc.) are now the responsibility of the whole team.

The team needs to start thinking in smaller deliverable pieces.  The concept of delivering a complete system from known specifications at the beginning of the project has been the fallacy of traditional development.  The cost of delaying delivery to the very end of the project and the lack of ability to meet actual customer needs can be eliminated by delivering minimum functionality early and augmenting with new and improved functionality iteration after iteration in close collaboration with the end customer.  But moving to this model of continuously delivering can be a tough adjustment for developers.  Even (or maybe especially) for seasoned developers, the concept of starting to build and deliver not knowing all the detailed specifications up front can be very unsettling. Getting comfortable with this model takes time and experience before developers cease to fall back on old behaviors.

One of the pillars of agile is complete transparency with the business and the stakeholders into the team, the process they are using and the delivery of the business functionality.  With the business embedded in the team on a day to day basis and iteration reviews to demonstration and approve all the deliverables from each iteration, there is no hiding problems when they occur and no hiding late deliveries or failed iterations.  On the flip side, the business is there to celebrate the wins and recognize the deliveries at the end of each iterations and they are part of determining the velocity of the team and adjusting the schedule to match the velocity.

The new way of thinking also requires new tools.  Tools that can build dashboards to automate and display the development intelligence metrics that provide transparency into the team’s performance.  Tools that automate configuration management, continuous integration and continuous integration as well as automated testing.  All of these endeavors are hard and required skilled individuals to lead; none of these comes for free.

3. A new way of working with others

More people skills are required by everyone.  People are now going to be working much closer together physically and will be interacting and collaborating on a much more frequent basis.  Experience working together (often for many years) may give an advantage to some team members, however, they still need to learn how to work together as a team.

3.1 Team Development Phases

Tuckman’s team development model includes the phases: forming, storming, norming, and performing. These phases are all necessary and inevitable in order for the team to grow, to face up to challenges, to tackle problems, to find solutions, to plan work, and to deliver results.  The number of iterations required to move through the phases will vary by team depending on factors such as: company culture, skills, experience, familiarity with each other, leadership (or lack thereof), external influences and coaching etc. Teams may also plateau in a phase before entering the next phase.

Agile teams will go through the four Tuckman phases:

  1. Forming. All teams will go through the forming phase. Typically this take 4 to 6 iterations for the team to: understand the lifecycle process, understand how to work together, understand what the team goal is, and most importantly to get comfortable enough with each other to risk the possibility of conflict.  During this phase, the team’s velocity will vary considerably especially for the expected failed iterations although there should generally be a trend upward in the velocity.  Retrospectives in this phase tend to be very cordial and factual because everyone is avoiding conflict.
  2. Storming. The storming phase can be very upsetting to both the team members and team managers. Often managers will step in to solve the storming issues without realizing that it is a necessary phase for the team to work through on their own.  Conflict, disagreements and personality clashes must get resolved before the team can move to the next phase.  Unfortunately, this phase is a make or break phase and it is hard to predict just how many iterations it will take but it is not usual to expect 3 or 4 pretty rough iterations.  Some teams will completely disintegrate during this phase and it is certainly not unusual to have a decrease in velocity during this phase.  Some members of the team may find they have basic incompatibilities and need to move to another team and this change in team membership can revert the team back to the forming phase.  The retrospectives tend to be very interesting and intense during this phase as they deal with the inter-personal issues of the team.  You can pretty much expect that there are going to be some hurt feelings no matter how safe an environment you have.
  3. Norming. The norming phase is when the team comes back together stronger and more cohesive for having weathered the storming phase. They have resolved their issues and have developed a new synergy where the team just “works well together” and it is fun to be part of the team. The team velocity should trend upward and then level off as the team hits its capacity consistently.  With the new-found synergy of the team, the retrospectives can focus on process improvement and optimizing the operation of the team with the hope of transitioning to the performing phase.
  4. Performing. The performing phase is the goal for all teams but very few actually get to this phase. Not all teams can get to the high level of performance of a Special Ops team where each member knows exactly what their team mates are doing and what they need to do to work as a cohesive unit.  A team in performing phase no longer needs a coach or supervision but can operate autonomously and efficiently.

3.2 Team changes everything

With self-organizing teams, decisions are now made at the team level rather than at the PM or manager levels.  This can be a difficult transition for both the team members and the managers.  Team members are often use to others making the decisions for them and wait for decisions or are reluctant to make decisions.  Managers often feel disempowered when decisions are no longer in their hands because the decisions are made by the team.

The team still has the responsibility to do the traditional activities such as; analysis, design, development testing and deployment.  However, under the new model, the team also takes on the inception responsibilities of: developing user stories, sizing the stories, estimating the stories, creating the product and architectural vision, and developing the release plan.   During construction the team also has to: plan the iteration, do look ahead planning, and have an iteration review (i.e. demo) with the stakeholders.  To keep the team working together they need to have the daily coordination meeting (i.e. Scrum) and to keep the team growing and evolving of course they need to do the retrospective at the end of each iteration.

Thinking, analyzing, planning and building just-in-time is a new way of thinking for a traditional team.  The inception phase in particular can be a difficult adjustment.  Designers and developers are used to doing all the analysis and design upfront.  Doing just enough analysis to be confident of a solution and then writing the user stories to do the work and build the plan is a difficult transition.  Writing user stories so they can get started and learn as they go is a very difficult transition.  Building just the simplest framework and then adding required features and functionality as needed requires practice, training and reinforcement.

There is much more focus on delivering quality because the team is responsible for development, delivery as well as product support.

4.0 The Bottom Line

Chess has only a couple of dozen basic moves and so it is simple to learn the basics.  However, after 3 moves there can be several billion moves available making it so complex that it can take a lifetime to master.  Transforming to Agile is a lot like that.  It looks very simple on the surface but under the covers it can get very complex very quickly.  That’s where the value of a good coach comes in.  You need a coach who brings a lot of experience to the table who will be open and honest with you about what needs to be done and how to get a successful transformation.

One Company, 800+ Disciplined Agile Teams and Counting

Reached the peak

In September 2016 InfoQ published an interview, Benefits of Agile Transformation at Barclays, with Jonathon Smart and Ian Dugmore of Barclays in the UK.  The interview summarizes the experiences of adopting Disciplined Agile (DA) strategies within Barclays.  Some of the key points made in the interview include:

  • Barclays is a large financial institution with over 130,000 employees that has been in operation for over 325 years
  • Barclays is taking a holistic approach to their agile transformation, it is not just a technology thing, linking up their various “islands of agility” to reduce the impedance mismatch between them
  • Barclays has the equivalent of over 800 agile teams that have adopted DA approaches in progress
  • They considered SAFe, LeSS and DA and settled on DA as it is not a “one-size fits all” approach
  • They needed to change both their funding and measurement strategies
  • Results include: A 300% increase in throughput (note: measured in terms of story delivery); 50% reduction in code complexity; test code coverage increase by 50%; More than half of teams are deploying into production at least monthly; Improved team happiness/morale; Improved business outcomes such as quicker time to market

The source article is definitely a very interesting read.  Scott Ambler + Associates has worked with Barclays from the beginning of their transformation and we work with many other organizations around the world to do the same.

 

Why is Disciplined Agile So Complex?

ConfusedWe often hear the refrain that Disciplined Agile (DA) is too complex, that it needs to be simpler. Believe me, we’re all for simplicity and we do everything that we can to make DA as simple as possible. However, to be fair, we often find that the people complaining about complexity are often coming at the situation from a different point of view than we are.

With this blog we work through how DA reflects the necessary complexity that we face as IT professionals. To do this let’s explore why DA may appear to be complex at first. This blog explores the following issues:

  1. There is a natural complexity in what we do
  2. Enterprise solution delivery is inherently complex
  3. Enterprise IT is inherently complex
  4. Choice is good, but having choices adds complexity
  5. People underestimate the complexity of the familiar
  6. DA purposefully includes some questionable choices
  7. Most people prefer to focus on a small part of the overall process
  8. What would you take away?
  9. What would you like to add?

There is a Natural Complexity in What We Do

Software development by itself is complex: we need to work closely with stakeholders to understand their needs, we need to architect and design solutions, we need to build or configure these solutions, we need to test them, we need to deploy them, we need to organize this effort somehow, and many other things. There’s clearly a bit of natural complexity in software development. Furthermore, the overall IT process surrounding software development adds even more complexity to operate, manage, and govern our overall IT ecosystem.

These complexities are why we get paid what we get paid – if it was easy, organizations could find a multitude of lower-skilled, and lower-paid, people to do this work.

 

Enterprise Solution Delivery is Inherently Complex

Complex process

The Disciplined Agile Delivery (DAD) portion of the framework addresses solution delivery in enterprise-class organizations. Sometimes people who have only worked in startups or small companies don’t appreciate the realities of enterprise-class development. A small company that has been in business for a few years is very different than a large organization that has several decades of legacy systems, legacy processes, and legacy thinking in place. Adopting agile and lean ways of working when you effectively have a clean slate to start from is a very different proposition than working in an organization where you need to respect and evolve the existing legacy ecosystem.

But don’t other, simpler methods such as Scrum work in these settings? Yes, they potentially do, but they address a much smaller scope than does DAD. Scrum is less complex because it only looks at a very small portion, likely not even 5%, of the software delivery process. The focus of Scrum is on a few collaboration patterns for organizing small software construction teams and for managing changing requirements. It purposefully does not address the full project delivery lifecycle from beginning to end. Nor does Scrum address technical practices for architecture, design, testing, programming, modeling, documentation, deployment, and many other aspects of solution delivery. DAD, on the other hand, purposely addresses all of these aspects of software delivery from beginning to end in a streamlined manner. And Scrum only describes a single way of working, what DAD calls the Agile/Basic lifecycle, whereas DAD includes four lifecycles (Agile, Lean, Continuous Delivery, and Exploratory/Lean Start Up) to choose from to address a great range of needs. So yes, as a framework DAD is much more complex than Scrum because DAD answers the multitude of questions that Scrum leaves up to you.

Interestingly, from the point of view of enterprise agile transformation, DAD is a less complex solution to adopt because it’s already done the “heavy lifting” for you on the process side of things.  We wrote a blog a few years ago about how it’s easier to right-size your process by starting in the middle with something like DAD than it is to start at the “simple bottom” with Scrum.

 

Enterprise IT is Inherently Complex

Disciplined Agile (DA) extends DAD to look at all of IT. The workflow of a Disciplined Agile IT department is depicted in the following diagram, and that diagram is merely a high-level overview. Many of the challenges that developers like to complain about – dysfunctional governance, ineffective enterprise architecture, slow data management, non-existent reuse, questionable HR processes, and many more – DA chooses to explicitly address. Because DA deals with the much wider range of enterprise IT issues than just software development it encompasses more complexity than many developers may be comfortable with, or even aware of.

Disciplined Agile IT Workflow

DA addresses this bigger picture because our philosophy is to provide coherent, pragmatic advice to help IT organizations address the challenges that they face in improving their processes. In fact, the DA framework appears to be the only source of information that addresses enterprise agile IT in a coherent and comprehensive manner.   Without a holistic vision such as this an organization will end up development a piece-meal strategy where the individual parts may be effective on their own but do not integrate well as a whole. Not looking at the entire IT picture at once is the primary cause of organizational dysfunctional in your department today, with the DA framework you can avoid this problem as you evolve into an agile/lean way of working.  BTW, you can download a copy of the workflow diagram shown above from the Disciplined Agile Consortium (DAC) posters page.

 

Choice is Good, But Having Choices Adds Complexity

Disciplined Agile looks at a range of situations, giving you choices. Other methods and frameworks, such as LeSS and SAFe, prescribe a single way of working.   Certainly a small development team will work differently than a medium-sized development team which works differently than a large team. A team that is co-located will work differently than a team spread across a single building, than a team spread across several buildings in the same city, than a team spread across the world. A team working in a regulatory environment will work in a more sophisticated way than a team in a non-regulatory environment. A team where everyone works for your company will work differently than a team where some of the work is being outsourced. A team facing a simple domain problem will work differently than a team facing a complex problem. And so on. Although many people just want to be told what to do, they just want a single way of working given to them, that desire for prescription is a recipe for disaster in modern IT organizations. You have a range of teams facing a range of situations: One process size does not fit all.

Many choices

Having choices is definitely complex. This is similar to “needing a new outfit”, going to the store, only to be overwhelmed by the myriad of choices presented to you.

Small number of choicesOne way to deal with the complexity is to divide and conquer. Recognize that deciding on an outfit that is right for you is actually a collection of simpler choices. For example, your need to choose the right shirt, the right pants, the right shoes, and so on. Of course you shouldn’t make these choices in isolation because your outfit needs to work together as a whole.  You either need to coordinate your decisions and effectively make them in parallel or if you prefer to make decisions one at a time then be prepared for your shirt choice to limit your follow-on clothing choices.

Let’s bring the conversation back to software process. In Disciplined Agile Delivery (DAD) we called out a collection of process goals that a team should fulfill in some way to be successful at agile solution delivery. These goals included exploring the initial scope, putting the team together, developing a consumable solution, coordinating activities, and deploying the solution into production to name a few. Each process goal is overviewed by a process goal diagram, the one for Explore Initial Scope follows below. Each goal is organized into a collection of process decision points, each decision point has options, and each option has advantages and disadvantages. Where one agile method might advise you to write a collection of epics and user stories to identify the initial scope, in DAD we say that you need to explore how people will use your solution (and writing epics and user stories are two of many options available to you), you may need to explore the user interface, you should consider to what extent (if any) the initial requirements should be documented, how you will go about eliciting requirements, and other important decisions. Ideally, if you want to be effective, then you want each team to own their own process and make the choices that are best for them given the situation that you face.

Explore Initial Scope

 

 

 

Back to the clothing analogy. Just like Scrum and SAFe prescribe one way of working, you could similarly take a prescriptive approach to shopping for new clothes. This is the equivalent of saying that you will be successful if you wear a red polo shirt, blue pants, blue socks, and black shoes. That may be great advice if you are working as a food server at your local restaurant chain but horrible advice if you need an outfit that is appropriate for a wedding, or for working in a formal office, or for going to the beach. When buying the right clothing you need first need to understand the context and then hopefully have several choices available to you. You want to be able to choose the right shirt for you, not the one prescribed to you. You want to choose a nice pair of pants or a skirt that complements that shirt, and so on. Yes, it’s much easier just to buy exactly the clothes that you’re told to, but it doesn’t give you as satisfying a result in most situations.

In Disciplined Agile we extended this choice-driven approach to IT-level activities such as Portfolio Management, Enterprise Architecture, Reuse Engineering, Data Management, and others. The following diagram is the process goal diagram for Enterprise Architecture, depicting the process decision points that your EA team should consider addressing and a range of options for each choice.

Agile Enterprise Architecture

People Underestimate the Complexity of the Familiar

Casual guy

When most people are asked what the gentleman in the picture is wearing, they’ll say a blue dress shirt and slacks. However, he’s also wearing shoes, socks, a belt, and underwear (we hope).   It is quite common for people to abstract away, to ignore, details that are familiar to them – the “shirt and slacks” answer abstracted away the details of the entire outfit. However, for someone to whom western clothing is new, this “simple outfit” appears quite complex to them until someone helps them to understand it.

The exact same thing happens in the software world. When asked what we do it’s easy to casually say that we work with stakeholders to understand what they want, we write some software, and then we deploy it. Of course the reality of what actually goes on is a lot  more complex than that.

We invite you to spend a few minutes and jot down all of the activities that your team does to actually build and deploy software in practice. Pretty long list, isn’t it? And for every strategy on that list, you’ve selected one of many options available to you.

 

DA Purposefully Includes Some Questionable Choices

Good and bad choicesWe all make choices in our diet. Some of those choices are good and some of those choices are rather bad for us. We know that we’re making poor choices yet we still choose to do so, perhaps because that’s the best we can do at the time or simply because that’s what we feel like eating right now.

It’s exactly the same when it comes to software process – we have a range of choices and some of them are better for us than others. For example, consider the choices we have when it comes to the level of detail that we capture when exploring the initial scope. We could choose to: write a very detailed requirements specification; write a light specification (perhaps using index cards, whiteboard sketches, and a bit of overview documentation); write a simple bulleted list of goals; or not capture any requirements information at all. Many agile purists will say that writing a detailed requirement specification isn’t agile, and in the vast majority of situations it isn’t, although we have seen life-critical regulatory situations where it is appropriate.   But, what if you’re in a situation where detailed requirements specifications aren’t appropriate yet someone is demanding that you create one? Agile purists will likely get into religious arguments with this person: the traditionalist will argue that their way is best and the purist will argue that there way is best, but in the end the person with the most political clout will win. With a DAD-based we recognize that there is a range of options available to you, including writing detailed specifications. But we take it one step further and are clear about the advantages and disadvantages of each approach and when the approach is appropriate.   Now you can have more intelligent conversations about your process choices.   When you get into situations such as this you can go beyond the “but that isn’t agile” refrain to have coherent reasons that as to why one strategy is better for another strategy given the situation that you face.

 

Most People Focus on a Small Part of the Overall Process

Locally focused

Many people choose to focus on their part of the job, which is fair enough. But what about all of the other activities that your team performs from beginning to end? You may be a developer focused on software construction, but what about all of the initial inception work that occurred, such as gathering initial requirements, putting the team together, and getting funding for construction that occurred before you began your work? What about the activities that occur within other teams that interact with, often to support, your team? For example, there may be some enterprise architects responsible for evolving and supporting a technical roadmap for your organization, there may be an infrastructure team responsible for operating your technical environment, and there may be people responsible for people management/HR activities (such as seeing that you get paid). Disciplined Agile looks at the holistic picture, not just at your piece of the overall puzzle. Just because you’re not involved with data management, or portfolio management, or operations doesn’t mean that those activities, and the teams that perform them, aren’t important to the overall success of your organization.

 

What Would You Take Away?

Now that you understand the overall scope of DA, what would you remove? We’re asking this question from the point of view of:

  • The complexity faced by modern enterprises
  • That you need choices if you’re to actually own your process
  • That we’re considering the overall picture, not just your part of it

If you think Disciplined Agile (DA) contains something that it doesn’t need, please add a comment at the end of this blog.

 

What Would You Like to Add?

Ha! Every single time we’ve had this conversation with someone, and they were willing to consider the issues above, they’ve told us that we needed to include one or more techniques that they’ve found effective in practice. EVERY. SINGLE. TIME.  In short, the conversation goes from DA is too complex to DA is not complex enough.  Argh!

So, once again, please add a comment at the end of the blog for anything you think we’re missing.

 

A Few Parting Thoughts

Software development, and IT in general, is a complex game with many moving parts that are evolving all of the time. It’s natural for us, as human beings, to want simple answers. But sometimes the simplest answer can still be quite complex. Simplistic answers may be more attractive, and in the short term or in very narrow situations they may in fact be a good solution for you, but more often than not they’ll make your situation even worse.   In Disciplined Agile (DA) we actively strive to drive the complexity out of the software process but at the same time we have the courage to look at the bigger picture and explicitly address the range of issues that modern enterprises face in practice. With DA we’re working together to make the software process as simple and streamlined as possible.

Puzzle - canstockphoto18141553 Small

Starting Agile quick and steady with DA

The Agile Stalemate

The current state of Agile, with both its advantages and drawbacks, is significantly influenced by Scrum, the most widely used Agile method.

As a drawback, after some (easy to adopt) first steps in Agile introduced by Scrum, many teams have no clear vision of how to advance with the process improvements and with Agile. As a consequence, we could say that the current state of Agile is a stalemate.

A significant part of the parts missing in many Scrum-based Agile adoptions, such as XP (Extreme Programming) engineering practices, was described by Robert C. Martin in his article The Land that Scrum Forgot.

The Advantage of Scrum: First Steps in Agile

Beyond the serious existing criticism, Scrum offers at least the following advantages:

  • Agile has become a widely known approach because of a significant number of Scrum adoptions (or adoption attempts)
  • Scrum is easy to adopt
  • Scrum adoption offers some important quick benefits

This strong ability of Scrum on making the first Agile steps is too often underrated. The main advantage of Scrum practices is not related so much to its direct benefits, but more to the elimination of the side effects of many bad legacy practices that are thus being replaced. For example, if a team uses the Agile-enabled self-organizing power to make poor process choices and disregards the practices for collaborative work, it will have worse results that a Scrum-based team that offers a base for collaborative work.

Going Further

A team using Scrum could go beyond the stalemate situation by using the XP engineering practices–refactoring, TDD (Test Driven Development), Pair Programming, and other–and/or it can go even further with Disciplined Agile (DA). Disciplined Agile has more practices, makes the Agile habits explicit (as Agile life-cycles), and offers guidance support instead of prescriptions.

The following sequence for adopting Agile, could be common, but is not an optimum approach–to a certain degree it inherently results in a waste of resources and in delays:

  • First adopt Scrum for easy first steps
  • Later on adopt complementary XP practices
  • Finally adopt DA to fill the gaps and adapt the process to the context

From a DA adoption point of view, we need to find an optimized approach that must have the following two advantages:

  • It enables quick first steps (similar with Scrum)
  • It no longer represents a stalemate situation

The First Steps: a Base that Enables all the Others

DA offers a good learning-oriented process decision framework that will help the teams to make their own process decisions and further improves their process. However, the first steps would be the most difficult ones and the main questions that arise are the following:

How should I start? What are the first factors that should be subjects of my choices?”

We are proposing the following Agile starting points:

  • Use life-cycle and iterations as a container for the process
  • Start working in a collaborative manner: non-solo work practices
  • Start persistent improvement by adopting retrospectives

In the following section we will explain how the use of these quick first steps as a base will enable all the other improvements.

Start Working in a Collaborative Manner: non-solo Work Practices

Agile-based process improvements and Agile retrospectives work only if the teamwork is based on continuous and active collaboration.

1_non_solo   The simple and rather vague Scrum practices for team collaboration have the great advantage of consistently replacing the old habits of non-collaborative work. We can easily go further and get better results by adopting Agile Modeling/Disciplined Agile non-solo work practices, that mean working with others for (few examples):

The essential element is to offer guidance to the team in order start to practicing non-solo work as soon as possible.

Start the Improvement by Adopting Retrospectives

2_retrospective

The improvement based on Agile and other sources must become continuous and permanent. The very first step is to start using retrospectives (per iterations, releases, and so on).

 Containers for the Process: Life-Cycle and Iterations
3_playground

The teams will need these containers as a “living space” for their process options and choices.  Scrum uses the Sprint (aka Iteration) as the main work container. XP goes further and has an implicit release life-cycle.

In Disciplined Agile we can find a more robust approach, meaning that we have both the iteration and also more explicit agile and lean life-cycles, whereby its selection is the team’s choice.

Ok, but “How should I start”?

For an “agile first steps” team, here is an option for the main process containers:

  • The iteration – This is a generic container for not-very-advanced cases
  • The agile-basic life-cycle – (this life-cycle) fits most of the cases and is similar with XP approach

As soon as possible and using proper guidance, the team should be aware of the process goals (DA makes these goals explicit) and how these goals are distributed along the life-cycle and iteration.

As a note, advanced teams could also select and use other types of life-cycles.

Just Do It!

Using this initial framework – process goals across life-cycle and iterations – the team should start working collaboratively (non-solo work practices) and start using retrospectives and other instruments to decide how to improve and adapt their process. Make sure that this approach is permanent and the process will have further optimizations.

5_basic_improvement

Agile Transformation: Why Are We Doing This?

Why?

Previously, in Agile Transformation: Being Agile, Doing Agile, and Supporting Agile and Agile Transformation: Comparing Transformation Strategies I discussed the need for your agile transformation efforts to address three factors: People-oriented issues (being agile), process-oriented issues (doing agile), and tooling issues (supporting agile).  I argued that you must focus to a different extent on each of these factors – 80-85%, 10-15%, and 5-10% respectively – and that you need to address all three at once if you’re to successfully transition to agile.  But what is the impetus for becoming more agile in the first place?

The answer is that you want to help people to become more effective so that they can work together to address the success criteria that their stakeholders have set out for them.  The challenge of course is that success criteria varies by team.  Some teams want better time to market, some want better quality, some want improved staff morale, some want improved stakeholder satisfaction with what gets delivered, and some want improved return on investment (ROI) in IT.  Many of course need to deliver on a combination of several of these criteria.

The point is that every team has their own success criteria that they should fulfill.  To do that effectively, agile coaches need to help these teams to “be agile” so that they have the proper mindset and culture to provide a foundation from which they can “do agile”.  To “do agile” teams need to understand, and have the skills to execute, agile practices in such a way that they perform the right practices at the right time to the right extent.  And to do that they need the appropriate tools to support these practices.

Your stakeholders could care less about whether your agile or even about what agile is.  They do care deeply about whether your team is able to meet, and better yet exceed, the criteria set out for them.

 

 

 

 

 

 

Agile Transformation: Comparing Transformation Strategies

Three-legged stool

In Agile Transformation: Being Agile, Doing Agile, and Supporting Agile we described three factors – people (being agile), process (doing agile), and tools (supporting agile) – that in our experience must be addressed during your agile transformation.  In this blog posting we compare agile transformation strategies in terms of addressing combinations of the three factors.  The following diagram overviews the eight potential strategies that you may embark upon.

Agile Transformation Strategies

Let’s compare each agile transformation strategy one at a time:

  1. None of the factors are addressed.  Good luck with that.  😉
  2. Focus on people only.  When you focus solely on being agile your teams don’t really learn how agile techniques fit together, nor do they understand the implications of how they’re working.  An example of a people only strategy is to give everyone training and coaching in leadership and collaboration skills and then expect them to self-organize into effective agile teams because “they already have the skills and they’re smart people who can figure it out”.  We saw this sort of strategy fail at a large product company when the teams who received this coaching and training invariably went back to their former ways of working.
  3. Focus on process only.  With this approach you get “cargo-cult agile” that is layered on top of your existing processes. What we mean by cargo-cult agile is that the team adopts many of the straightforward management practices, the ones that Scrum tends to focus on, such as holding a daily coordination meeting, an iteration/sprint demo, an iteration/sprint retrospective, and iteration/sprint planning.  You end up with people holding “all of the agile meetings” and who on the surface take on their version of agile roles such as Product Owner and Team Lead.  They have the appearance of working an an agile manner but they really aren’t, and worse yet they don’t even know it and usually nor does senior management.  This is a very common problem when an organization’s entire agile transformation strategy is to send their people on two-day workshops so that they may become “certified masters.”
  4. Focus on tools only.  Some organizations believe that if they buy several agile tools, or even an entire agile tool suite, and then force their staff to use them that the tools will somehow make their teams agile.  The actual result is that you end up with a “standardized” approach to agile that is both overly complex (the tools typically include far more features than you’ll ever want, many of which provide functionality completely inappropriate for your situation) and incomplete (you can’t automate everything and even if you could the vendors haven’t gotten to it yet).  The worst offenders are the Application Lifecycle Management (ALM) suites, many of which seem to be offered by vendors who themselves are struggling to deliver consumable products into the marketplace.
  5. Focus on people and process. A common agile coaching refrain is to address tooling once you understand the process, which is good advice to an extent, but it quickly falls down when you realize that many technical practices such as continuous integration (CI) and automated testing require adoption of tooling from the very start. When you ignore or at least greatly downplay tooling issues you tend to grow “agile purists”, and even “agile zealots”, who only know how to apply agile in simple contexts.  Ignoring tools can ease your agile transformation at first but eventually tends to get you into trouble when you start to apply agile in more complex situations.
  6. Focus on process and tools. When you don’t coach people in the skills and mindset around
    “being agile” you tend to end up with cargo-cult agile with automated bureaucracy.  Agile transformation requires a paradigm shift, which inherently implies you need to address what it means to be agile.
  7. Focus on tools and people. When you downplay how to “do agile” you tend to get agile children playing with shiny new toys.  They tend to know the agile language and mindset, and will often have a cursory understanding of simple agile practices that they learned on their two-day certified mastery workshop, but they really don’t know how to successfully build consumable solutions from beginning to end.  Teams who receive little help on “doing agile” tend to spend a lot of time, and a lot of money, figuring out the agile process.  Worse yet, they often adopt a WaterScrumFall approach where Inception and Transition tends to be more traditional and heavy and only Construction is lightweight and agile.
  8. Focus on people, process, and tools simultaneously.  When your agile transformation strategy addresses all three factors at once you have the potential to create Disciplined Agile teams that can work at scale.  This only happens when your “being agile” efforts help people to shift to an agile mindset, when your “doing agile” efforts teach a comprehensive yet flexible (think goal driven) view of agile strategies and practices, and your “supporting agile” efforts lead to a context-driven, enterprise aware approach to tool selection.

In your agile transformation you will spend much more effort addressing people-oriented (being agile) issues than you will either of process (doing agile) or tooling (supporting agile) issues.  Think of it like this: these three factors are effectively the legs of a stool, if you don’t address all three then your agile transformation will fall over.

If you’d like help with your agile transformation, please contact us via ScottAmbler.com.

Agile Transformation: Being Agile, Doing Agile, and Supporting Agile

Many organizations struggle to transition to a more agile or lean way of working.  In this blog posting we address three questions:

  1. What factors should your agile transformation address?
  2. How important are these factors in practice?
  3. How difficult are these factors to address?

What factors should your agile transformation address?

Our experience is that successful agile transformations need to address three fundamental issues:

  1. People (being agile).  This factor addresses issues such as individual mindset, team and organizational culture, and team and organizational structure.  As a Disciplined Agile coach you must help people to evolve to an agile mindset, learn new skills, adopt improved collaboration strategies, and evolve the way they are organized to reflect their new ways of working.  As you can see below, this factor typically comprises between 80 to 85% of your agile transformation effort.
  2. Process (doing agile).  As a Disciplined Agile coach you need to help organizations adopt new agile and lean practices, strategies, and the Disciplined Agile framework itself.  This tends to be between 10 to 15% of the overall transformation effort.
  3. Tools (supporting agile).   Agile teams will need to adopt new tools, such as continuous integration (CI) tools, testing tools, and perhaps even agile task board software to name a few.  Agilists will use existing tools, such as their configuration management environment, in new ways.  And they will abandon some existing tools, in particular traditional test management tools, that aren’t applicable in the agile world.  Reworking your tooling strategy tends to take about 5 to 10% of your agile transformation efforts.

Agile Transformation: Focus of Effort

How important are these factors in practice?

In the 2014 Agile Transformation survey we asked a series of questions around how important it was to address various people, process, and tooling factors.  The survey respondents had either been through an agile transformation or were currently well into one.  The figure below shows that the first and third most important transformation factors were aligned with being agile, the second and fourth most important factors were around doing agile, and the two least important factors were around tooling (supporting agile).

Agile Transition Factors - Importance

How difficult are these factors to address?

In the 2014 Agile Transformation survey we asked a similar series of questions around how difficult organizations found it to address various people, process, and tooling factors.  The first and third most difficult factors to address were cultural (being agile).  The second and sixth most difficult factors to address were process oriented (doing agile) in nature.  The fourth and fifth most difficult factors addressed tooling.

Agile Transformation Factors - Difficulty to address

Our experience is that if you don’t address all three factors in your agile transformation effort that you will run into serious trouble.  This topic will be explored in our next blog posting.

If you’d like help with your agile transformation, please contact us via ScottAmbler.com.

The Agile Tractor Engine Analogy

Since 2001 agilists have been focused on finding ways to improve how software is developed.  We’ve adopted fundamental strategies such as regular coordination meetings, regular demos, product owners, developer regression testing, retrospectives, and incremental releases of working software.  Disciplined teams have adopted more advanced strategies such as active stakeholder participation, continuous integration (CI), test-driven development, continuous documentation, continuous deployment, measured improvement, and incremental releases of consumable solutions (to name a few).  We experiment with new techniques to learn what works for us in the situation that we face, improving our approach as we do so in an incremental kaizen-style manner over time.  In effect we are finding ways to tune our “development engines” so that we can deliver more valuable functionality, to reduce our cycle time, and to be more productive as a team.  This is very similar to a Formula One team who over the years improves their racing car engine to deliver more power and more speed for less fuel consumption so as to help them win the race.

Race car engine

Our challenge is that a race engine alone doesn’t win the race, what we really need is a race car. We see too many agile teams, who on their own are doing a great job at improving the way that they work, get bogged down by the organizational environment in which they work.  This is particularly true in established enterprises that have been in operation for decades and sometimes even centuries.  Your development team tries to work in an agile manner but they’re forced to conform to your PMO’s existing traditional governance strategy that requires creation of inappropriate artifacts following a lifecycle that is far from agile; they’re forced to work with a bureaucratic data management team who often takes weeks or months to do work that should be accomplished in minutes or hours; or they’re forced to work with a review-focused enterprise architecture team who hasn’t been actively involved with software development for years (to name but a few common challenges).  This is akin to putting their racing car engine into an organizational tractor.  Is it any wonder you’re not winning the race?
Tractor

Yes, it’s nice that we’ve discovered ways to get better at software development, but what we really need to do is streamline our approach to IT in general so that we can enable our organization to become a lean enterprise.  Our software development engine needs an effective IT chassis so as to enable our lean enterprise racing car to win the race.

Race car

Here are several strategies to help you to go beyond building an agile software development engine and instead build a “racing car” organization:

  1. Go beyond agile software development.  When we’re optimizing a race car, it’s not just about the engine.  The tires, the shape of the body, the shape of the spoiler, the design of the suspension, and many other factors come into play.  Similarly, it’s not just about optimizing your delivery teams, instead we need to optimize the whole of IT and more importantly your organization in general.  We need to be enterprise aware in our agile transformation efforts: While our development teams to be more agile we also need to transform the teams that they’re working with.  This includes our stakeholders, the data management team, the enterprise architecture team, the portfolio management team, and others.  The challenge, as we point out in principle #15 of the Disciplined Agile Manifesto, is that these teams will likely need to support a multi-modal IT department – some delivery teams (we hope most) will work in an agile/lean manner, some will work in a more traditional manner, and some may even work in an ad-hoc manner.  The implication is that anyone working with more than one IT delivery team will need to be prepared to work in a mode that reflects the paradigm the team is following.  In other words, you’ll work with an agile team in an agile manner, a traditional team in a traditional manner, and so on.  More on this in a future blog posting.
  2. Adopt a flexible, context-sensitive mindset.  Races are run in a range of conditions, and a race car must be sufficiently robust to operate in these conditions.  Similarly IT teams work in a variety of conditions: They range in size and in geographic distribution; the face zero or more regulations; they face a range of domain and technical complexities, and of course every team is made up of unique individuals with their own preferences, experiences, and skills. In short, one process size does not fill all.  To address the need for process flexibility the Disciplined Agile framework supports multiple delivery lifecycles, it promotes a flexible goal-driven strategy that guides you through process choices, and it now includes process blades describing flexible strategies for strategic IT capabilities such as release management, enterprise architecture, reuse engineering, and many others.
  3. Optimize the flow across all of IT.  All the parts of a race car work together effectively as a whole, they are more than a collection of parts bolted together.  It is the same for an IT department.  For an agile software development team to be effective they must be governed in an agile manner, the enterprise architects must support them in an agile manner, the data management team needs to be responsive, the release management team needs to be able to short frequent releases, and so on.  For the enterprise architecture team to be effective the software development teams that they support must be enterprise aware, the data management team needs to recognize that architects need to look beyond data, and so on.  Every team interacts with other teams and must do so effectively.  When one team wants to work one way and another team wants to work another way its the equivalent of trying to use a metric gear with an imperial gear – they simply aren’t going to mesh and progress will grind to a halt.  It is far more important that these teams all follow flexible and compatible approaches rather than approaches that are locally optimized for each team.  Just like a perfect metric gear won’t work with a perfect imperial gear, a strategy that is perfectly optimized for your enterprise architects will likely not work well with a strategy that is perfectly optimized for data management.
  4. All teams will improve.  Your IT department, and your organization as a whole, is a complex adaptive system.  It is a collection of teams working together to achieve, we hope, a common goal.  Each of these teams will be learning from their experiences and improving their approach.  Changes in one team will potentially affect the way that they work in other teams, requiring changes there.  The implication is that your organization is constantly evolving, we hope for the better.  With a goal-driven approach teams can recognize what their process options are, and the trade-offs of each, providing lightweight guidance for choosing effective strategies to address the situation they find themselves in.

We’d like to leave you with this: The collaborative, evolutionary strategies of the agile paradigm isn’t just for software development, but instead should be applied to your IT department and your organization as a whole.  We need race cars, not just race car engines.

Right-sizing your agile process? Start in the Middle

Is your organization concerned with the cost and time required to adopt agile strategies?  Are you just starting out with agile and hoping to improve your chances of success by learning from the experiences of those who have gone before you?  Are you part way into your agile transformation but struggling to figure out how it all fits together?  If you answered yes to any of these questions please read on.

In this blog posting we discuss what it means to “right size” your software process to meet the needs of the unique situation that your team finds itself in.  We discuss two common anti-patterns: starting with a process framework that is much too large for your needs and starting with one that is far too small for your needs.  We argue that it’s much better to avoid these extremes and instead take a middle-ground approach by starting with a framework that is much closer to your actual needs.

Extreme #1: Large process repository

The first process right-sizing anti-pattern is to start with a large process repository, the classic example being IBM’s Rational Unified Process (RUP).  Although RUP is much maligned within the agile community the fact is that if you were to examine it with an open mind that there are many very good ideas promoted by RUP.  Be that as it may. The basic strategy with RUP is that you need to tailor down the process, often dramatically, to meet the unique needs of the situation your team finds itself in.  To do this requires significant process expertise, time, and money.
Right Sizing RUP
In practice, however, many organizations ran aground with RUP when they tailored it to be something similar to the waterfall-style processes from yesteryear that they were familiar with.  This is often referred to as RUPifall.  Another common mistake was to say to themselves “wow, there’s a lot of great ideas here, we need to do them all” and as a result they would create a process that was far too heavy to meet their needs.  In either case the problem was that they often didn’t have the process expertise required to right-size their process.  We also see this with organizations starting from frameworks such as ITIL, COBIT, and CMMI so this clearly isn’t just a RUP problem.

Extreme #2: Small methodology

At the other extreme is the right-sizing anti-pattern of starting with a small methodology, the classic example in this case being Scrum.  Although there is significant support for Scrum within the agile community, or at least among people who are new to agile, we’ve been seeing for a long time now that organizations are also running into trouble with this approach too.  With Scrum the idea is that you add in the techniques that Scrum doesn’t address to right-size your process.  Because Scrum proves to be only a very small part of the overall picture, to do this requires significant process expertise, time, and money.
Right Sizing Scrum
In practice organizations run aground with Scrum because they don’t have the process expertise to expand it to meet their needs.  One only has to count the number of “How does X fit into Scrum?” conversations occurring in agile discussion forums online, or at user group meetings or conferences, to see that this is true.  Step back for a moment and ask yourself how much time and effort has your organization invested in trying to adopt Scrum.  Could it not have been more streamlined?

A few years ago Forrester Research discovered that the majority of organizations “doing Scrum” had actually tailored it into what they called Water-Scrum-Fall (others call this Scrumifall).  As we describe in Going Beyond Scrum this occurs for several reasons.  First, Scrum doesn’t address the full delivery lifecycle, instead choosing to focus on the construction portion of it.  As a result organizations tend to stick with what they know, a heavy project initiation phase and a heavy solution deployment phase.  Second, Scrum only addresses only a small portion of what you need and explicitly leaves technical issues up to you.  These issues include topics such testing, programming, architecture, governance, documentation, deployment and many others.  As a result teams are left to piece together a process strategy that works for them, at the very same time that they’re struggling to understand the fundamentals of agile and lean software development.  They rarely have the process expertise to do that and as a result end up having to hire hordes of expensive Scrum coaches, few of whom seem to understand the enterprise-class realities that your teams actually face.  This is a risky and expensive proposition indeed.

The Effective Middle Ground

Both of these anti-patterns represent extremes: start with something large and cut it down to size, or start with something small and build it up to meet your needs.  Why not start with something much closer to what you actually need?  Doesn’t that make a lot more sense?  Why do you need to do all this process work?  Because someone wants to sell you tooling?  Because someone else wants to sell you expensive coaching and questionable certification strategies?  Isn’t it time to consider a more pragmatic strategy?

The Disciplined Agile Delivery (DAD) framework is a more effective middle ground.  It addresses the full delivery lifecycle, as does RUP (but not Scrum), and even gives you several choices from which to select.  Sometimes a Scrum-based lifecycle is appropriate, sometimes a Lean lifecycle is, other times a continuous delivery lifecycle is best, and sometimes an exploratory “lean start up” lifecycle is.  Different teams, different situations.  DAD starts with a lightweight approach, as does Scrum but not RUP, helping you to avoid the bloatware of RUP and filling in the numerous blanks left by Scrum.  DAD also gives you lightweight tailoring advice in the form of process goal diagrams, in many ways a cross between mind-maps and decision trees, that make your process choices explicit.  The RUP process tailoring advice, if you bother to read it at all, is rather heavy handed and the Scrum tailoring advice boils down to “you’re smart, you can figure it out and if your run into trouble hire a coach.”  Isn’t it time to abandon the extremes?

Right Sizing Disciplined Agile Delivery
This middle ground strategy isn’t without its faults.  A challenge with DAD is that it explicitly reveals that agile software development, or as we prefer to say agile solution delivery, is complex.  This is particularly true in enterprise-class situations where teams are often facing combinations of scaling factors such as larger team size, geographic distribution, and regulatory constraints.  DAD makes it explicit that teams need to invest a bit of time up front to perform initial scoping, initial architectural modeling, and initial planning (all in a lightweight manner of course).  This sort of pragmatic thinking can be inconvenient for less-experienced developers who just want to jump in and start coding.  Because DAD promotes the philosophy of enterprise awareness it purposely bakes in strategies for governance, DevOps, and working with IT-level groups such as your enterprise architects and data management team to name a few.  This can also prove to be inconvenient for developers who want to narrowly focus on doing what’s convenient for their team as opposed to what’s best for their organization.

In Summary

The following infographic summarizes the main points in this blog posting.
Right Sizing Your Agile Process
We hope that you’ve found this blog posting enlightening.  Even if you are well along the way of your Scrum adoption, or of evolving your RUP-based approach, you can still benefit from switching over to DAD.  Scrum teams will find that it addresses many of the issues that you’re still struggling with, and RUP teams will find that it shows how to work in a far more lightweight manner.  Organizations will find that it provides a much better foundation from which to scale agile strategies.

Adopting a Disciplined DevOps Mindset

In this continuing series about how the Disciplined Agile Delivery (DAD) framework supports and enhances DevOps, we overview several strategies to help your organization adopt a Disciplined DevOps mindset:

  1. Invest in your people.  In our experience 80 to 90% of your overall effort will be in helping people to learn new skills and ways of thinking and to rethink if not abandon many of the “best practices” of yesteryear.  This requires training, education, and coaching over a long period of time – most people will require many months, and sometimes years, to make the transition to this new mindset.
  2. Create a safe learning environment.  Teams must be free to experiment, to try new strategies to discover what works for them in the situation that they face.  Very often this will work out well, with a few stumbles along the way, but every so often the experiment will show that the strategy just isn’t right for this team.  That should still be considered a successful experiment, learning what doesn’t work is just as valuable as learning what does, and the team should not be worried about recrimination for “failed” experiments.
  3. Look at the “whole system”.  Disciplined DevOps is about more than just continuous delivery (although that is a great start) and in most enterprises it’s about more than just streamlining development and operations.  With a Disciplined DevOps mindset we strive to improve flow through the entire ecosystem, including development, operations, support, enterprise architecture, data management, release management, and most importantly to the business itself.
  4. Improve locally, transform globally.  Each team, including your solution delivery teams, your enterprise architecture team, your operations staff, and many others must strive to improve and streamline the way that they work.  These local improvement efforts must be supported by a “global” transformation effort that focuses on improving DevOps across your entire ecosystem.  Every team will affect other teams, motivating them to make improvements which in turn affects how they work with others.  Your IT department is a complex adaptive system where people and teams learn and improve over time in a dynamic, evolutionary manner.  If you just focus on local improvements your DevOps effort is likely to devolve into chaos.  If you just focus on a company-wide, global transformation it is likely to get bogged down in bureaucracy.  An “improve locally, transform globally” approach is a viable middle ground that benefits from these two extremes while avoiding the disadvantages.
  5. Have a communication plan (and work it).  Adopting a Disciplined DevOps strategy within your organization typically requires a lot of (often small) changes.  Although it may be clear to you why this is important it isn’t always clear to everyone else.  People need to understand why you’re making these changes, what’s in it for them, what the overall change strategy is, how far along the plan you currently are, what changes are coming soon, and so on.  You communication plan may include regular newsletters, posters overviewing key concepts, brown bag lunches where people share their experiences, electronic discussion forums, management presentations, and many more.  The keys to success are to have a constant drum beat of information, to be as open and honest about what is actually occurring, to provide opportunities to everyone to learn, and to motivate everyone to share their learnings.
  6. Think long-term.  Disciplined DevOps is a journey, not a destination.  It takes time for people to adopt a new mindset, months and often years before it is truly ingrained in their way of thinking.  This paradigm shift does not occur by management fiat, nor does it occur as the result of a day or two of training (although training is important), nor does it occur because you’ve invested in new tooling.  Organizations that successfully make this paradigm shift do so by investing in their people, their process, and their tooling over the long term.