How Does Data Management Fit In?

Key tenets of agile and lean are to work collaboratively and to streamline your workflow respectively. This includes all aspects of your workflow, not just the fun software development stuff that we all like to focus on.  This blog posting explores how Data Management activities fit into your overall process.

In the process flow diagram below we see that Data Management is a collaborative effort that has interdependencies with other DA process blades and the solution delivery teams that Data Management is meant to support. Due to the shortened feedback cycles and collaborative nature of the work this can be very different than the current traditional strategies. For example, with a DA approach, the Data Management team works collaboratively with the delivery teams, Operations, and Release Management to evolve data sources. The delivery teams do the majority of the work to develop and evolve the data sources, with support and guidance coming from Data Management. The delivery teams follows guidance from Release Management to add the database changes into their automated deployment scripts, getting help from Operations if needed to resolve any operational challenges. Evolution of data sources is a key aspect of Disciplined DevOps.

Data Management external workflow

This highly collaborative strategy is very different than the typical traditional strategy that requires delivery teams to first document potential database updates, have the updates reviewed by Data Management, then do the work to implement the updates, then have this work reviewed and accepted, then work through your organizations Release Management process to deploy into production.

In the next blog posting in this series we will explore the internal workflow of a Disciplined Agile approach to Data Management.  Stay tuned!

Can You Spare a Few Moments to Fill in Our Short Agile Survey?

Trust Data Not Lore

For those of you who are Star Trek fans you’ve likely been seeing ads for this t-shirt on your social media feeds.  It is an apt metaphor for the empirical approach that we take with Disciplined Agile – we regularly run studies to explore what is actually going on out there on agile teams, we gather data, as opposed to pouting some of the wishful thinking (spreading lore) that we often hear from consultants and vendors.

We are currently running an agile mini-survey of only 6 questions, so this will take you a few minutes at most to fill out, exploring some important issues about agile adoption within your organizations.  We hope that you choose to invest a few minutes of your valuable time to fill it out, and thank you in advance for doing so.

What Will We Do With the Results?

As you already know the surveys that we run are completely open – We share the source data (without identifying information), the questions as they were originally asked, and a Powerpoint deck summarizing our interesting findings after the survey has closed.  In fact, we have the results from dozens of previous studies posted at the IT Surveys page for you to take a look at.

We also write blogs discussing the results.  For example, for the 2016 Agile Scaling survey that we ran in November, we published several blogs:

Recently, we’ve created a new infographic summarizing the results of the study.  If you click on the thumbnail below it will take you to the page where you can download a high-resolution PDF of it.  This infographic is only available to members of the Disciplined Agile Consortium (DAC).

Agility at Scale 2016 Infographic

 

Where Can You Get the T-Shirt?

If you’re interested in the T-Shirt, it is a time limited offering on Teespring.

True Enterprise agility

The Lean Enterprise genie has been out of the bottle for some time now, but for most organizations he’s still a long way from granting three wishes

 

The Lean Enterprise … Enterprise Agility … DevOps 2.0 …  BizDevOps … the list of monikers and descriptors for this concept is becoming a lengthy one, but there’s no denying that lean and agile concepts and practices are now mainstream, and pushing to break out of their former boundaries within software development.

Reflecting on this, I came across a very apt description (in, of all things, a conflict simulation design blog) of what seems to be happening in this realm of late:

“Most new ideas, however fresh or spontaneous they may appear at first glance, usually represent an evolutionary synthesis of previous ideas; in other words, when it comes to most things, history really is “preamble”.1

The idea of bringing business, development and operations people to the same table to act as a tripartite coalition from the very start of product development has been gaining exponential momentum of late. As far as ideas go, it is not an entirely new one of course. Encouraging collaboration from the get-go has long been a hallmark of Agile approaches – in development for example, George Dinwiddie is believed to have coined the term “the three amigos” around 20092 to describe the interplay of developers, testers and business analysts / product owners from an Acceptance Test Driven Development (ATDD) perspective. However, other than describing a tripartite arrangement, the 3 amigos analogy differs from the most recent crop of expressions in that contrarily to say, BizDevOps, it only actually plays on two of the required three planes.

Another oddity that struck me is that one of the first high visibility discussions that I can remember describing the interplay of business, technology and operations actually framed the debate in terms of an antipattern: to wit, a July 2011 Forrester whitepaper which coined the expression “Water-Scrum-Fall”3 as a way of describing the very lack of cross-enterprise collaboration which bedevilled most organizations’ agile efforts.

 

The need for systems thinking

Unsurprisingly, a silo mentality is the biggest single impediment to achieving true Enterprise agility. Everyone involved in the creation and delivery of business solutions needs to collaborate across the breadth and depth of the organizational structure. Team-level agility in the delivery space alone will not deliver on the promise of the Lean enterprise; nor is DevOps enough. Business, IT and operations all need to break out of their silos and embrace systems thinking.

In both my practice and my teaching, I constantly strive to come up with metaphors that can convey the underlying meaning of concepts in ways that overcome resistance by reframing the issue at hand in a very different manner than what people may be used to. I would now like to share a metaphor that I have recently been using which has shown a lot of promise in terms of getting everyone thinking in terms of the whole system when it comes to the concept of the lean enterprise.

As discussed above, the system in question can be simplified as having three planes:

 

This however is merely … a triangle, and does nothing to convey the importance of intense collaboration. We need something more evocative.

Something with three parts, yet an indivisible whole. There is actually quite simple that we can use to convey this:

 

 

That’s right. True Enterprise agility is neither a sprint, nor a marathon … it’s a Triathlon.

What makes this metaphor work is simply this:

Although a triathlon is made up of three events, it is performed by a single athlete who must excel at all three to win. In this case, the “athlete” is not an individual nor a team, but the whole enterprise.

So, if our organization is reaching a point where it is proficient in delivery, there isn’t much more to be gained by continuing to concentrate all of our improvement efforts in IT alone – the Business side of things must be brought into the game as a full partner. The same goes for operations – even careful, collaborative prioritization backed up by competent delivery will not win the day if ready solutions must then linger for months in pre-production environments. Nothing ground-breaking here, just a fresh way of looking at the issue … no serious triathlete would sign up for the Ironman in Hawaii knowing that she faced daunting challenges in terms of her swimming, or was a less than competitive cyclist. The same must go for our “lean” enterprises. Until we can let go of the silo mentality and learn to act as a single “athlete”, we will continue to be bound by the shallows4 of Water-Scrum-Fail.

 

Endnotes

  1. http://mapandcounters.blogspot.ca/2012/04/roads-from-smolensk-spi-dunnigan-and.html
  2. http://www.velocitypartners.net/blog/2014/02/11/the-3-amigos-in-agile-teams/
  3. Available at https://www.forrester.com/report/WaterScrumFall+Is+The+Reality+Of+Agile+For+Most+Organizations+Today/-/E-RES60109
  4. Shakespeare of course. Julius Caesar, Act 4, Scene 3.

 

 

Why is it so hard to find qualified agile coaches?

Questioning peopleI was hoping to come up with a pithy, short answer to this question but the only thing that I can come up with is “people.”  The not-so-pithy answer is that there is no sort of agreement around what it means to be a “qualified agile coach”, the people hiring coaches aren’t thinking things through in many cases, and the agile community suffers from a myriad of integrity challenges when it comes to professionalism. In this blog I work through the following ideas:

  1. Why is there a dearth of qualified agile coaches?
  2. Sports coaches as an example
  3. What should we expect from agile coaches?
  4. Our solution: Certified Disciplined Agile Coaches (CDACs)
  5. Parting thoughts

Why is There A Dearth of Qualified Agile Coaches?

Let’s answer this in two parts: Why is there a dearth of agile coaches and why are there so many unqualified coaches available?  The first question is very easy to answer.  The demand for agile coaches far outstrips the supply.  The adoption rate for agile has been growing steadily since 2001, hence the growing demand.  As you’ll see later in this blog, it takes years to grow good coaches.  As a result there is little hope for the supply to catch up with demand any time soon.

The second question, why are there so many unqualified coaches available, is easy but uncomfortable to answer.  In general we have systemic challenges in the IT industry and in many ways we’ve managed to exacerbate these problems within the agile community. Some of the challenges within the IT community include:

  • A person is just as likely to be a self taught programmer, and more likely in fact, than they are to have a computer science or engineering degree
  • Although we throw the term “software engineering” around a lot, there is no agreement around what it means or even if it’s an appropriate concept (the IEEE/ACM promotes one, but there are many others)
  • There is no sort of apprenticeship culture in this industry
  • Few people have a personal goal of mastery, and few organizations support the gaining of mastery amongst their staff
  • There is a shortage of talented people, so It is very easy to find and retain employment regardless of your level of talent, and the market for IT people is still growing
  • No country has a licensing body for software professionals that is commonly required by organizations, unlike other professions such as doctors, lawyers, engineers, architects, and many more
  • Many people in the IT community believe this normal for a professional, or if they do perceive a problem they are (rightfully) overwhelmed with the challenge of addressing it
  • Colleges and universities are endemically years behind the quickly changing IT industry (there are a handful of schools that work closely with industry, so it’s getting better)

Then we have the agile community, with its various certification training scams.  You can become a certified master after staying awake during a two-day workshop and passing an online test that almost nobody fails.  To put this into context, a Starbucks barista, the kid who pours your morning coffee, get’s three days of training before being let loose on customers.  Yet it somehow makes sense that someone with 50% less training becomes the lead of a software development team?  Really?  Another example: Someone can become a scaling program consultant after attending a four-day workshop, and worse yet are now “qualified” to teach a two-day workshop to others so as to impart their vast agile scaling knowledge upon them.  Amazingly, because of the demand by companies desperate to hire agile-skilled people, the demand for these “designations” is incredible (shameful would be a more appropriate word).

In practice many agile designations are little more than “participation ribbons”, yet most organizations take them seriously often either because they don’t realize how trivial they are to earn or because they’ve given up expecting any better from agilists.  Is it any surprise that it’s hard to find qualified coaches when we’ve watered things down so much?

Sports Coaches as an Example

CoachCoaching is very common in sports and with the exception of “pick up” games few sports teams are without a coach.  In fact, serious sports teams tend to have several coaches, typically lead by a head coach. In professional sports coaches are paid significant salaries, sometimes millions of dollars a year, as coaching is perceived to be a critical success factor.  It makes sense to look at sports coaching works to see how agile coaching might work.

Most sports coaches are former players.  They’ve typically played for years, and sometimes decades, having been coached themselves all along the way.  They’ll often start off as children, in Canada it’s common for kids to start learning to skate and play hockey at the age of two, being coached and drilled in basic skills and knowledge for years.  They also gain practical experience playing games.  Most kids drop out eventually, although many still play their sport (be it hockey, football, cricket, baseball, …) well into middle age.  And some decide to stay in the sport, but make the shift from being a player into being a coach.

The transition to becoming a sports coach generally isn’t easy.  There are three common strategies for this:

  1. Formal training.  One path is to go to university, get a teaching degree, and become a gym teacher at a middle school or high school and begin coaching children.  These coaches tend to coach a wide range of sports, although in some cases you’ll often see a coach specialize on a single sport, such as high school football or hockey, because that’s what their passion was a child (and often because they hope to move up the ranks at eventually, see point #3 below).
  2. Informal apprenticing.  Another path is to apprentice, asking your existing coach to allow you opportunities to coach others under their guidance.  When I trained in karate this was very common, with senior students helping to teach less senior students.  My daughter is currently learning to skate and they follow a similar pattern with senior skating coaches (adults) being helped by assistants (typically teenagers who have been skating for many years) to coach and teach the younger children.  Helping to teach or coach others is recognized as an important part of your own learning process.
  3. Formal apprenticing.  Because of the money involved professional sports teams tend to take a more formal approach to coaching.  They will often expect coaches either come up through the coaching ranks – start as a high school coach, then become a college-level coach, then finally a professional coach – or to come up through the professional sports ranks – when your star players are past their peak they sometimes move into coaching roles.  Each time you move up to a new level of coaching, say from high school football to college football, you often start as an assistant coach to first prove yourself.  The head coach at each level recognizes that it’s their responsibility to grow more coaches, so they impart their skills and knowledge on to the junior coaches.

So, what are some important observations we can make out of all of this?  First, sports coaches have deep skills and experience at the sports that they are coaching.  Second, we expect this of them.  Would you pay to have your child to be given skating lessons by a “Certified Skating Master” who had two days of training in the “skating mindset” and how to facilitate a handful of skating meetings?  Of course you wouldn’t.  Instead you’d want someone who had been skating for years, and better yet may have even been a competitor at some point in the past.  Third, it takes years of apprenticing or training to become a good sports coach, not just several days in a certification workshop.

What Should We Expect From Agile Coaches?

Here is what we’ve found to be the critical success factors for agile coaches:

  1. They should have years of agile experience, not days of training.  If someone doesn’t have years of experience in something, and more importantly years of varied experience, then why they heck would you hire them as a coach?
  2. They should have coaching skills and experience.  Being experienced in agile isn’t enough.  Apprenticing under another good agile coach is a great way to get coaching skill as is getting training in agile coaching (the Agile Coaching Institute is a start for this although the program at International Coaching Federation (ICF) is far more thorough).  The need for experience is a bit of a catch-22 of course – you need to already be an agile coach to be qualified to be an agile coach.  But, if someone has no previous coaching experience then at best I’d put them into a junior coaching role under the guidance of an experienced coach.  This provides them with the opportunity to gain the requisite experience and to prove themselves in practice.
  3. They should have robust agile skills and knowledge.  Years of agile experience is a good start, but better yet is a range of experience at all aspects of the lifecycle in which they are coaching.  It’s reasonable to expect a delivery team coach to understand all aspects of agile solution delivery so that they can coach the entire team in the skills they need to succeed.  Furthermore, it’s reasonable to expect an Agile IT coach to have experience in the full agile IT lifecycle, including areas such as Enterprise Architecture, Data Management, Portfolio Management, and many others.
  4. They should have experience in a similar context.  Ideally they should have skills in a similar context to what you currently face – someone who only has small team coaching experience will struggle to coach a large programme, someone who only has only coached in startup companies will struggle in a large financial institution, someone who has only coached co-located teams will struggle with globally distributed teams.  Context counts.

Criteria for Effective Agile Coaches
All four of these factors are equally important.  Any “coach” who is deficient in one or more of these areas still has some work to do.  Nobody is perfect of course, given the rates that agile coaches demand it’s reasonable to expect these people to be qualified.

Our Solution: Certified Disciplined Agile Coaches (CDACs)

A fair question to ask is how do we deal with this in the Disciplined Agile (DA) space. We believe that it’s critical to your success to have qualified coaches so we’ve built a principled certification program based on the martial arts philosophy of Shu-Ha-Ri.  Certifications must be earned and that takes time.  The following diagram summarizes our strategy for how practitioners must earn DA certifications.

Agile Practitioner CertificationsTo become a Certified Disciplined Agile Coach (CDAC) you need to have at least five years of experience in agile (which is verified by the Disciplined Agile Consortium (DAC)), plus evidence that you’ve already been sharing your skills and knowledge (we call this give back), plus you must be a Certified Disciplined Agile Practitioner (CDAP).  To become a CDAP you must have at least two years of proven agile experience (validated by DAC), have passed a comprehensive test of your agile knowledge, and already be a Certified Disciplined Agile Practitioner (CDA).  To be a CDA you must have passed a comprehensive test of your knowledge and skills.  So, this process of certification ensures that CDACs have comprehensive skills and knowledge in agile techniques, at least five years experiences in agile, and at least initial experience in coaching/teaching (the giveback component). Note: Not shown in the diagram above is Certified Disciplined Agile Instructor (CDAI), which you must be at least a CDAP and have proven ability to teach DA.

Parting Thoughts

It isn’t easy to find qualified agile coaches, but then again it isn’t impossible either. Our hope is that this blog has provided you with some insight into what you should be looking for in a good agile coach.  Anyone can put a shingle up and say that they’re an “agile coach”, but anyone who wants to say that they are a Certified Disciplined Agile Coach (CDAC) needs to have worked through a rigorous process to earn that qualification.  CDACs have proven knowledge, experience, and give back.  Why settle for less?

Related Reading

Disciplined Agile Values for Data Management

Data Management Mindset

There are several values that are key to your success when transforming to a leaner, more agile approach to Data Management. Taking a cue from the Disciplined Agile Manifesto, we’ve captured these values in the form of X over Y. While both X and Y are important, X proves to be far more important than Y in practice. These values are:

  1. Evolution over definition. The ability to safely and quickly evolve an existing data source, either to extend it to support new functionality or to fix quality issues with it, is absolutely paramount in today’s hyper-competitive environment. Yes, having defined data models and metadata describing them is also important, but nowhere near as important as being able to react to new business opportunities. Luckily agile database techniques, long proven in practice, exist that enable the safe evolution of production data stores.
  2. Holistic organization over Data Management. Earlier we said that data is the lifeblood of your organization. Yes, blood is important but so is your skeleton, your muscles, your organs, and many other body parts. We need to optimize the whole organizational body, not just the “data blood.” Traditional Data Management approaches often run aground because they locally optimize for data concerns, whereas a DA approach to Data Management recognizes that we must optimize the overall whole. This implies that sometimes we may need to sub-optimize our strategy from a data point of view, for the sake of organizational level optimization.
  3. Sufficiency over perfection. Data sources, like many other IT assets, need to be good enough for the task at hand. The old saw “perfect is the enemy of good” clearly applies in the data realm – too much time has been lost, and opportunities squandered, while development teams were forced to wait on Data Management teams to create (near) perfect models before being allowed to move forward. Traditional data professionals mistakenly assume that production databases are difficult to evolve and as a result strive to get their designs right the first time so as to avoid very painful database changes in the future. The Agile Data method has of course shown this assumption to be wrong, that it is very straightforward and desirable to evolve production databases. A side effect of this revelation is that we no longer need to strive for perfect, detailed models up front. Instead we can do just enough up-front thinking to get going in the right direction and then evolve our implementation (including data sources) over time as our understanding of our stakeholder needs evolve.
  4. Collaboration over documentation. We’ve known for decades that the most effective way to communicate information is face-to-face around a shared sketching environment, and that the least effective is to provide detailed documentation to people. The implication is that we need to refocus our efforts to be more collaborative in nature. As data professionals we need to get actively involved with solution delivery teams: to share our knowledge and skills with those teams, and to enable them to become more effective in working with data. Yes, we will still need to develop and sustain data-related artifacts, but those artifacts should be lightweight and better yet executable in nature.
  5. Cross-functional people over specialized staff. Agilists have come to realize that people are more effective when they are cross-functional (also known as T-skilled or generalizing specialists). Although specialists are very skilled in a narrow aspect of the overall process, the problem is that you need a lot of specialists to perform anything of value and as a result the overall workflow tends to be error prone, slow, and expensive. The other extreme would be to be a generalist, someone who knows a little bit about all aspects of the overall process. But, the challenge with these people is that although they’re good at collaborating with others they don’t actually have the skills to produce concrete value. We need the best of both worlds – a generalizing specialist with one or more specialties so that they can add value AND a general knowledge so that they can collaborate effectively with others and streamline the overall effort.
  6. Automation over manual processes. The only way that we can respond quickly to marketplace opportunities is to automate as much of the bureaucracy as we possibly can. Instead of creating detailed models and documents and then reviewing potential changes against them we capture our detailed specifications in the form of executable tests. This is quickly becoming the norm for specifying both the requirements and designs of code, and the same test-driven techniques are now being applied to data sources. Continuous integration (CI) and continuous deployment (CD) are also being applied to data sources, contributing to improving overall data quality and decreasing the time to safely deploy database updates into production.

As you can see, we’re not talking about your grandfather’s approach to Data Management. Organizations are now shifting from the slow and documentation-heavy bureaucratic strategies of traditional Data Management towards the collaborative, streamlined, and quality-driven agile/lean strategies that focus on enabling others rather than controlling them.

Recommended Reading

Is it Disciplined Agile Delivery (DAD) or Disciplined Agile (DA)?

The quick answer is of course “Yes”.  😉

A couple of years ago we caused a bit of confusion when we expanded the scope of the Disciplined Agile Delivery (DAD) framework to address the activities of an information technology (IT) department.  When we did this we realized that the scope of the framework and the name no longer matched, so we decided to rebrand to be simply the “Disciplined Agile (DA)” framework.  Having said that, sometimes it makes sense to say DAD and sometime DA depending on what you’re focusing on at the time.

The Scope of Disciplined Agile (DA)

As you can see in the following diagram, which depicts the scope of the DA framework, it’s clear why there has been some confusion because the DA framework covers a lot of ground.

Scope of Disciplined Agile

Let’s explore each aspect depicted in the diagram:

  1. Disciplined Agile Delivery (DAD).  DAD addresses all aspects of solution delivery from beginning to end, in a streamlined manner.  This includes initial modelling and planning, forming the team, securing funding, continuous architecture, continuous testing, continuous development, and governance all the way through the lifecycle.  The framework includes support for multiple delivery lifecycles, including but not limited to a basic/agile lifecycle based on Scrum, a lean lifecycle based on Kanban, and a modern agile lifecycle for continuous delivery.
  2. Disciplined DevOpsDisciplined DevOps is the streamlining of IT solution development and IT operations activities, and supporting enterprise-IT activities, to provide more effective outcomes to an organization.
  3. Disciplined Agile IT (DAIT).  As the name suggests DAIT addresses how to apply agile and lean strategies to all aspects of IT.  This includes IT-level activities such as enterprise architecture, data management, portfolio management, IT governance, and other capabilities.
  4. Disciplined Agile Enterprise.  A Disciplined Agile Enterprise is able to anticipate and respond swiftly to changes in the marketplace.  It does this through an organizational culture and structure that facilitates change within the context of the situation that it faces.  Such organizations require a learning mindset in the mainstream business and underlying lean and agile processes to drive innovation.

Some History

The first “1.0 release” was the original Disciplined Agile Delivery book in June of 2012.  As the title suggests the focus was on DAD, although it laid the groundwork for Disciplined DevOps in that it baked in the development side of DevOps right into DAD.  In 2015 we began publishing our work in both Disciplined DevOps and Disciplined Agile IT (DAIT) and renamed the framework to Disciplined Agile (DA) to reflect this expanded scope.  Now, in 2017, we are beginning to flesh out Disciplined Agile Enterprise strategies and will soon begin the share them here on this site.

For the want of a whiteboard….

For the want of a whiteboard the vision was lost,

For the want of a new vision the release was lost,

For the want of a new release the product was lost,

For the want of a new product the customer was lost,

For the want of a new customer the company was lost,

And all for the want of a whiteboard.

With a nod to Benjamin Franklin.

Should You Govern Agile Teams Via a Traditional Strategy?

The quick answer is no, that’s an incredibly bad idea.

We ran a study in February 2017, the 2017 Agile Governance survey, to explore the issues around governance of agile teams. This study found that the majority of agile teams were in fact being governed in some way, that some agile teams were being governed in an agile or lightweight manner and some agile teams in a traditional manner.  See the blog Are Agile Teams Being Governed? for a summary of those results.

The study also examined the effect of governance on agile teams, exploring the perceived effect of the organization’s governance strategy on team productivity, on the quality delivered, on IT investment, and on team morale.  It also explored how heavy the governance strategy was perceived to be and how well it was focused on the delivery of business value. The following figure summarizes the results of these questions.

Governance Effectiveness with Agile Teams

Here are our conclusions given these results:

  1. Agile governance helps agile teams. There is a clear co-relation between an agile approach to governing agile teams and positive results such as improving productivity, increasing quality, spending your investment in IT wisely, and improved team morale. This is what we believe the goal to be, to help the people being governed to be more effective and successful.
  2. Traditional governance hinders agile teams.  There is a clear co-relation between traditional approaches to governing agile teams and reduced team productivity, reduced quality of output, wasting IT investment, and decreased team morale.  We believe that these results are the exact opposite of what you hope to achieve with your governance strategy.
  3. Agile teams should be governed in an agile manner.  This follows directly from the previous two conclusions.  It should come as no surprise that your governance strategy should be well-aligned with what it is being governed.
  4. Traditional governance strategies likely hinders traditional teams too.  We didn’t look into this issue directly, but our experience has been that traditional governance tends to be more of a hindrance than a help to traditional teams as well.

When we work with organizations to help them to adopt agile ways of working, we often find that they are running into several common challenges when it comes to IT governance:

  1. They have both agile teams and traditional software teams.  This is because it’s a multi-modal world: You will have some teams taking a traditional approach, some an agile approach, some take a lean approach, and some are even skilled enough for continuous delivery.  Each team will follow the lifecycle that makes the most sense for them, and as a result each team should be governed by the approach that best suits the way that they are working.  To do anything different would be to hinder the teams, and that isn’t what good governance should be about.
  2. There is a desire for a single approach to governing software teams. This makes sense on the surface because it would simplify your overall governance strategy, thereby making things easier for the people doing the governing.  But, as we’ve learned, this results in negative effects in practice.  Your governance strategy must be flexible enough to support the range of teams being governed.
  3. The governance team is struggling to understand agile.  Your executives and middle management need education and coaching in agile and lean just like the people on your software team do.  It is naive to expect your governance people to devise a governance strategy for agile when they don’t really understand the implications of agile to begin with.

For agile to succeed in your organization the way that you approach IT must evolve to enable and support it, and this includes your governance strategy.  Reach out to us if you would like some help in addressing this important issue.

Related Reading

 

Are Agile Teams Being Governed?

For the major of teams the answer is yes.  We ran a survey in February 2017, the 2017 Agile Governance survey, to explore the issues around governance of agile teams.  As you can see in the following diagram, 78% of respondents indicated that yes, their agile teams were in fact being governed in some manner.

Agile governance rates

We also asked people about the approach to governing agile teams that their organization followed.  As you can see in the following diagram, a bit more than a third of respondents indicated that the governance strategy was lightweight or agile in nature.  Roughly the same indicated that their agile teams had a more traditional approach to governance applied to them, and one quarter said their governance approach was neither helping nor hindering their teams.

How are agile teams being governed?

Governance tends to be a swear word for many agilists and they will tell you that governance is nothing than useless bureaucracy.  Sadly in many organizations this seems to be the case.    In the next blog in this series we will compare the effectiveness of agile and traditional strategies for governing agile teams.

Related Reading

Agile Metrics: Questions and Answers

Metrics

On March 21 2017 we ran a webinar entitled Measuring Agile: A Discipline Agile Approach to Metrics.  We unfortunately didn’t have enough time to answer all of the great questions that we received so we said that we’d write a blog with the answers.  This is it.

We’ve organized the blog into the following sections:

 

Convincing Management

First and foremost, please read our blog entitled 7 “Easy” Steps to Convince Management to Support a Hard Change.  This blog has a lot of great advice for getting support from senior management for the changes asked about below.

 

Q: What is the message to deliver to the board of directors of a company and/or sponsors of projects when it comes to metrics?

The principles section of the webcast made this pretty clear I think.  The principles we discussed are:

  • There is no easy answer
  • Every metric has strengths and weaknesses
  • Compete against yourself
  • Measure to improve
  • You get what you measure
  • Measure outcomes at the team level
  • Each team needs a unique set of metrics
  • Team use metrics to self organize
  • Trust but verify
  • Adopt common metrics categories across teams
  • Don’t manage to the metrics
  • Automate wherever possible
  • Prefer trends over scalars
  • Prefer leading over trailing metrics
  • Prefer pull over push

If I had to pick a key message, it’s that you need to be flexible and enable teams to take a context sensitive approach.

Q: How do you address the resistance to ranged estimates?  We see a lot of resistence to this with leaders!

Sadly this is all too common.  There is a desire for an “exact number” for the cost/schedule for an IT project, the belief being that such precision leads to lower financial risk. This never seems to work out, and in practice it tends to increase overall financial risk.  Worse yet it motivates some very ethically questionable management practices such as padding the budget (lying about the cost to get the fixed estimate as close to the upper end of the range as possible), dropping scope late in the lifecycle (lying about what will be delivered and wasting money working on stuff you had no hope of delivering in the first place), or asking for more money when it’s clear you’re not going to deliver it (this is arguably extortion).  In the article Estimating on Agile Projects I go into further detail about the need for ranged estimates.

As we discuss in the 7 Easy Steps blog, you need to educate management on the trade-offs involved with their current approach and help the to see that it isn’t working out well for them in practice.

Q: My management definitely wants to know if they are on time and on budget… how should I handle this problem?

This is also another common problem in more traditional organizations, and is sadly a mindset that is often promoted by project management canon.  The first thing to do is ask them why “on time and on budget” is important to them, and to continue to explore their goals via an evolutionary “5 why” strategy until you get to the heart of the matter.  Usually the real issue is that they want to reduce schedule risk and financial risk but they only know to ask for on time and on budget respectively.  Help them to recognize that they can do better than that.  For example, a more effective question to ask instead of “Are we on budget?” would be “Are we spending the money wisely?”, the latter question requiring competent governance to truly answer.  Similarly, a more effective question to ask that “Are we on schedule?” would be “Are we going to deliver when the functionality is actually needed?”, also a question requiring better governance than many organizations seem to have today.

Q: Executives and senior management wants common metrics period… can you talk to this – how should a change agent (me) handle this?

Sadly another very common problem.  This proves to be selfishly motivated in most cases – they want a common set of metrics to make it easy for them to monitor teams (they have less thinking to do when all the dashboards look the same).  As we discussed in the webinar, this isn’t realistic because every team is unique, they face a unique situation, and they have a unique set of priorities.  Yes, there are commonalities between teams but also differences.  For example, it makes sense for teams to measure quality.  BUT, surely it’s obvious that a data warehousing team will measure quality different than a mobile app team, which in turn measures quality differently than a team sustaining a mainframe-based system.  This is why I promoted the idea of asking teams to address common categories (such as Quality, Stakeholder Satisfaction, Finance, and so on) but to do so in a manner that makes sense for them.

 

Transformation Metrics

Q: In a transformation is the ‘stakeholder vision’ milestone something that you see ‘could’ be used as a way to guage adoption of the framework across an Enterprise?

Stakeholder vision is a milestone that marks the end of the Inception phase.  I’m not sure how it could be used to gauge adoption.  It could be used to gauge readiness to move forward with the transformation though.

Q: Is there more thoughts you have on transformation metrics?

Context counts.  In general, take a context-sensitive approach where you measure what is important to you.  I worked through a lightweight approach to Goal-Quality-Metric (GQM) during the webinar and the OKR approach is also good.  There is no one single answer.

Q: How do start-up agile projects best survive in a really non-agile organization?

Sadly, they generally don’t.  If you go poking around on the web you’ll find that there’s a lot of advice around this sort of issue, and a large portion of it advising you to jump to another organization.

 

Specific Metrics

Q: Please throw some light on Accelaration metric. I’d like to implement that in my org.

We have a detailed blog about acceleration.

Q: Velocity, used to calculate acceleration, can only be calculated if velocy is based on the same point system… I don’t agree acceleration factors out the points, the points have different meaning potentially by project – can you talk to this some more?

You’re wrong,  Here’s a quick answer:

  • Team 1 has a velocity this iteration of 20 and a velocity 5 iterations ago of 15.  Velocity is measured in Atari points on this team.  Acceleration = (20 Ataris – 15 Ataris)/15 Ataris = 33% over 5 iterations.
  • Team 2 has a velocity this iteration of 30 and a velocity 5 iterations ago of 20.  Velocity for this team is measured in Nintendo points by this team.  Acceleration = (30 Nintendos – 20 Nintendos)/20 Nintendos = 50% over 5 iterations.
  • When I divided Atari points by other Atari points the unit of measure, Atari points, disappeared.  Similar thing happened to Nintendo points.  Hence acceleration is comparable as long as it’s calculated over a similar time period (if not, adjust so that you are dealing with a similar time period).
  • Read the Acceleration blog for details.

Q: Is value is only monetary value is $?

No, it doesn’t have to be but often is.  You should measure value in units that are important to your stakeholders.  Perhaps value may be measured in market share by them, for example.

 

Miscellaneous

Q: Excecutive leaders want to measure “team health” using metrics and compare teams – maybe reward or punish based on these metrics – please talk to this.

This is generally recognized as bad practice because as soon as teams realized that they are being judged by management they’re motivated to game the metrics as best they can.  In the case of automated metrics that are difficult to game, then perhaps its possible to compare against that (code quality metrics come to mind).  But, management will get what they measure.  For example, if they judge teams based on how many defects they fix, chances are pretty good that the team will start identifying relatively trivial defects and then “fix” them to make their metrics looks good.  However, if they judge teams based on code quality trends (i.e. Is code quality improving?) they will likely get higher quality code in the long run.

As I said in the webinar, the primary usage of metrics should be to provide insights to teams to help them to improve.  Monitoring teams, part of your overall governance strategy, should be an important but secondary concern.

Q: What if the teams don’t agree with the metrics established by management?

My advice is for teams to identify the metrics that make sense for the situation that they face via a lightweight GQM approach (or something similar such as OKR).  Management may want to guide teams, perhaps by insisting on certain categories of metrics, see the earlier discussion, or even by suggesting metrics being collected by other teams.

If it’s a situation where management is trying to inflict a common metrics strategy across all teams, which is a relatively bad idea as I discussed earlier, then I think that the team should justify why they don’t agree with the metrics prescribed by management.  I also hope that they would suggest a more appropriate strategy and that management would listen to them.

Q: What are some of your favorite tools for metrics?

I typically don’t like recommending tools because the answer to this question is context dependent.  Tool choice will be driven by:

  • What you are hoping to measure
  • Your existing platform(s)
  • Your organizational preferences around tooling (are you a Microsoft shop, are you willing to pay for commercial tools, are you willing to adopt open source tools, …)
  • Your previous experiences around such tooling, if any

So, identify what you situation is then do a bit of research to identify the tools that are right for you.

Q: Is it realistic at scale to have multiple dashboards? +900 teams.

I’ll turn this one around.  Is it reasonable to ask teams not to have dashboards simply because your organization is big?  Is it reasonable to give up on monitoring teams because your organization is big?  Is it reasonable to give up on automation because your organization is big? You see where I’m going with this.