Reuse engineering is an important, and arguably advanced, aspect of the Disciplined Agile framework. The challenge is that reuse engineering requires significant discipline and organizational maturity to be successful, hence we tend to run into far more talk about reuse than action. Having said that, many organizations have been very successful with reuse. The following diagram overviews the internal workflow of a reuse engineering team, capturing key activities (the blue bubbles) that the team performs. This blog posting explores what reuse engineers do in practice.
Let’s work through the primary activities performed by reuse engineers:
- Guide teams in reuse. An important activity for reuse engineers is to provide guidance to delivery teams regarding what is available for reuse, how to go about accessing and applying the reusable artifacts, and educating teams in why reuse is important to both the team and to your organization. Very often a team’s architecture owner will collaborate with the reuse engineering team to bring a reuse engineer into the team at the right time.
- Obtain assets. Reuse engineers will obtain reusable assets from a variety of sources, including from the marketplace and from internal delivery teams. Based on the various organizational roadmaps, and the needs of the delivery teams that they’re working with, the reuse engineering team will often work with delivery teams to identify and obtain the appropriate assets from the marketplace. The goal is to find assets that fit the needs of individual teams in a way that aligns with the direction of the organization. Furthermore, reuse engineers tend to monitor what delivery teams are doing, often by working closely with the organization’s enterprise architecture team and the architecture owners on delivery teams, to help them identify potentially reusable assets. When a team believes it has built something that is potentially reusable by others, or when the reuse engineers believe they have done so, then the reuse engineers will work with the team to understand and harvest that asset.
- Publish reusable assets. An asset is potentially reusable when it is of high quality, it is appropriately documented, one or more examples exist of how to use it, and it is findable by others. The publishing process ensures that all of these criteria are true. The reuse engineers will do the work to publish the asset, refactoring and documenting it as needed and harvesting any usage examples if available (and creating some when not). After doing so, they will save the asset and its related supporting artifacts into your organization’s reuse repository, announcing the availability of the new asset after doing so. Note that reuse repositories, also called asset managers, have fallen out of favor in the past few years due to the complexity of the available products on the market and the propensity of organizations to use products like Git or Microsoft SharePoint as repositories.
- Configure asset for specific use. Reuse engineers will often work with a team to help them to configure an asset for specific use. The goal is to make it as easy as possible for others to reuse existing assets, thereby increasing the chance of rate of successful reuse within your organization.
- Integrate reusable assets into a solution. Reuse engineers will often work with delivery teams to integrate a reusable asset into their solution, once again to make it as easy as possible for teams to reuse existing assets. Interestingly, an important aspect of harvesting an asset for reuse is to help the source team to integrate the improved version of the asset back into their solution. This helps to increase the likelihood that teams will offer up potential assets for harvesting and publishing.
- Evolve reusable assets. Assets need to evolve over time to reflect changing requirements and implementation technologies. The implication is that the owners of the assets, often the reuse engineering team, will need to evolve their assets, publish new versions, and deprecate old versions. This is work that must be funded and supported properly.
If your organization is serious about reuse engineering then they will explicitly fund a reuse engineering team and enable them to work in the manner that we’ve described here. For more information, you will find the article Reuse Engineering to be of value.
Reuse isn’t free. Without funding, either implicit or explicit, meaningful reuse simply doesn’t happen. In short, your approach to funding is a critical success factor for your reuse engineering strategy.
In this blog posting we explore several strategies for funding reuse within your organization. The funding strategies, from most to least desirable, are:
- Funded reuse-engineering team. A team of one or more people in the role of reuse engineer is provided explicit funding to support and enhance the reuse efforts within your IT department.
- Asset-level funding. Funding for a specific reusable asset, such as a security framework or collection of micro services, is explicitly budgeted for. This funding should cover the development, enhancement, and long-term support of the asset.
- Development team-based funding. Funding for the use, development, and enhancement of reusable assets is included in the budget for development teams. Such funding is often accompanied by mandates along the lines of “The team will achieve X% levels of reuse” (although teams are rarely measured against these mandates in practice).
- No explicit funding. With this approach reuse occurs on an ad-hoc basis within delivery teams, often driven by tactical decisions at the team level as opposed to strategic decisions at the organizational level.
- Chargeback funding. Delivery teams are charged for usage of an asset. In some extreme cases development teams are charged to download an asset from the reuse repository, typically because that’s easier to charge them at this point in time over charging for actual usage.
Table 1 compares and contrasts these funding strategies.
Table 1. Comparing the reuse funding strategies.
|Funded reuse engineering team
- Reuse is actively supported across all IT
- Long-term support activities, such as evolution of assets and reuse repository, are directly funded
- Cost of reuse engineering is easily measured
- Value provided by reuse engineering team must be measured to justify continued, year-over-year funding
- Reuse engineering team becomes a target for financial cuts because the cost is easily measured but the value provided is difficult to measure
- Supports a robust, organization-wide, reuse engineering strategy
- Significant discipline required: IT executives must understand and be prepared to execute an explicit reuse engineering team strategy
- Development of potentially reusable assets are funded in a targeted manner
- Cost of individual assets easily measured
- Typically used for initial development of an asset, but long-term support and enhancement is often neglected
- Can result in development of similar assets by disparate teams
- Useful first-step towards the funding of a reuse-engineering team
- Can be made to work in organizations with a project-based IT funding, although long-term support and enhancement of the asset can be a struggle in such situations
|Development team based funding
- Development of potentially reusable assets are funded
- Difficult to fund ongoing enhancement and support of your reusable assets with this strategy, unless the team is stable over the long term and willing to support the assets that they create
- Funds earmarked for reuse often diverted for development of non-reuable functionality
- Many “potentially reusable assets” are created yet few are reused by other teams due to lack of infrastructure to support wide-scale reuse
- This strategy may be your only option in IT departments with strict project-based IT funding
|No explicit funding
- Some delivery teams, particularly very disciplined ones, will still choose to have high levels of reuse even when no organizational support is provided
- Reuse will be inconsistent at the organization level
- The only way reuse will happen in this situation is through the maturity and enterprise awareness of the delivery teams
- Potential exists to fund ongoing support and enhancement of reusable assets
- Development teams are punished for reusing assets, motivating them to create their own versions
- Complexity of chargeback added to overall organizational bureaucracy, effectively adding waste to your IT processes
- Chargeback strategies will undermine if not destroy your reuse engineering endeavour
- There is no situation where we would recommend this strategy
Combinations of these strategies can of course be implemented.
Agile development teams build new things every day, and some of these things can be generalized into robust assets for reuse by other teams. This is particularly true for teams that are working with new technologies and techniques: for example, your first few C# teams are likely to develop useful micro services, or the first few agile teams to write personas will develop a potentially reusable template. The downside is that the first people to work with a technique or technology are most likely to make “beginner mistakes,” so their work may not be something you want to share with others. The implication is that you either need to wait to harvest a higher quality asset later or be willing to invest more effort into generalizing the asset. We’ve found that it’s usually best to wait, giving you more time to gain experience with the technology or technique and discover if there is actually a need for the asset on other teams.
One of the key activities performed by a reuse engineering team is to harvest potentially reusable assets so that they may be generalized for reuse on other teams. There are five steps to the harvesting process:
- Find a potentially reusable asset. Reuse engineers will be constantly on the look out for potentially reusable assets. They find out about them via word of mouth, through internal online discussions, during enterprise architecture coordination meetings, and from suggestions made by members of development teams.
- Assess the viability of the asset. Just because someone thinks that an asset is potentially reusable doesn’t mean that it is. The reuse engineers must determine whether there is a likely demand for the asset. When other several other development teams have something similar, or are at least thinking about developing such an asset soon, then this is a fairly straightforward decision. If there isn’t any immediate customers for the asset then that’s an indication that you are likely better to wait until it’s clear that other teams will be able to leverage the asset before you invest in generalizing it.
- Generalize the asset. It takes much more effort to generalize an asset to make it reusable than it does to develop it for single-purpose usage. For example, studies showing that development of reusable code costs 111% to 480% of the development cost of single-purpose code. Generalizing an asset requires great skill because the person doing this needs to be familiar with not only the technology or technique behind the asset but also with how it will potentially be used in practice. Part of generalization is to ensure that the asset conforms to appropriate guidance and making it easy to understand by its intended audience. You also want to validate the asset. You’ll need to unit test code-based assets and review templates and non-code artifacts with their target audiences. Concise overview documentation is important, but far more important are one or two good examples showing how to use the robust artifact properly. In the case of code, your unit tests may be sufficient. For templates, you should capture an actual example of the document for which it is a template (for example, to support a use case template, you should harvest one or two well-written use cases from a project team).
- Reintegrate the asset into the source. The reuse engineers do this “extra” integration work so that the source team gets the advantage of working with the improved asset without taking on the cost of having to refactor their solution to do so.
- Publish the asset. The asset needs to be made available to the teams who could actually reuse it. The publishing effort typically entails putting the asset into a reuse repository (which could be anything from a folder to a configuration management system to a commercial asset management product) and announcing the availability of the asset to potential customers of it.
There are four basic timing strategies for when to harvest an asset:
- Harvest in-progress. You generalize an asset developed by a team before it has been released into production. The reuse engineers do this by working closely with the development team, often joining them for a period of time so that they can generalize the asset and do the work to reintegrate the new version into whatever the team is building.
- Harvest after production release. With this strategy the reuse engineers wait until the source development team has finished building the asset. As with the first strategy they do the work to generalize reintegrate the asset. In cases where the team isn’t working on a new release they do a “patch” release of the solution into production, but this obviously isn’t a desirable situation to be in (it’s far better to have stable development teams).
- Harvest a legacy asset. In rare situations it is possible to harvest an existing asset currently deployed in production by encapsulating access to it. Although this strategy is often talked about it is rarely used in practice because it requires the asset being harvested to be well architected in that it must be loosely coupled and highly cohesive. Most legacy assets tend to be the exact opposite, which is the primary reason why they’re not already being reused by other teams.
- Harvest for a new project. You wait until a project team needs an asset previously developed by another team and then decide to either reuse the asset as-is or to generalize it. This strategy has the advantage that you know there is a customer for the reusable asset before you invest in it. However, the team may not be able to wait for the asset to be generalized. Furthermore, unless the asset was developed recently people are likely to have forgotten about it, implying that this strategy has a short viability time frame.
This posting, the latest in a series focused on a disciplined agile approach to reuse engineering, overviews the activities associated with it. The Disciplined Agile (DA) framework promotes an adaptive, context-sensitive strategy. The framework does this via its goal-driven approach that indicates the process factors you need to consider, a range of techniques or strategies for you to address each process factor, and the advantages and disadvantages of each technique. In this blog we present the goal diagram for the Reuse Engineering process blade and overview its process factors.
The following process goal diagram overviews the potential activities associated with disciplined agile reuse engineering.
The process factors that you need to consider for reuse engineering are:
- Obtain assets. There are several ways that you can obtain potentially reusable assets. The least effective is to try to build them to be reusable from the very beginning but this strategy often proves problematic in practice because it’s hard to predict what other teams will actually want and as a result you tend to overbuild the asset. A better approach is to obtain an asset from external sources, either free (as in the case of open source assets) or through purchase. In this case the assets are often both under and over built – some features you want are missing and many that you do not want are there. The most effective strategy, in general, is to harvest an existing asset that is already in use within your organization and to generalize it so that others will want to reuse it.
- Publish assets. An asset won’t be reused if people don’t know that it exists. When a new reusable asset is made available it must put into your organizational reuse repository, described appropriately, and announced to any interested parties.
- Support delivery teams. There are many ways for a reuse engineering team, if it exists, to support IT delivery teams. Training, educating, coaching, and mentoring team members in reuse are fairly straightforward to understand. Some of the more interesting strategies include working with a delivery team to configure and even integrate an asset into their work. Reuse engineers, often working with a team’s architecture owner, will identify potentially reusable assets that can be harvested and generalized for reuse.
- Evolve assets. Reusable assets, like all other assets, will need to be evolved over time. This includes any work required to generalize the asset to make it reusable, configuration management of the asset’s constituent parts, to update an existing asset to support its evolving purpose, to tailor an asset so that it can be reused in a new situation, and to eventually retire the asset when it is no longer needed.
- Fund reuse. There are several ways to fund your reuse engineering efforts. The least effective, and often debilitating, strategy is to put a chargeback strategy in place. The basic idea is that if someone reuses an asset then they should pay for it (some organizations will even charge a team for downloading something from their reuse repository, regardless of whether they use it or not). The problem with this approach is that it in effect punishes teams for reusing things, thereby motivating them to build things from scratch in the future. In some organizations it proves better to not fund the reuse effort at all, which typically results in ad-hoc reuse at best, than it is to put a chargeback scheme in place. The most effective approach that we’ve seen is to directly fund the reuse team, thereby taking cost considerations out of the equation when people choose to reuse an asset.
- Govern reuse. The reuse engineering effort, as with all other efforts, should be governed in a lightweight, collaborative manner.
An important philosophy for succeeding at reuse engineering in the information technology (IT) space is to understand that you have more than one option at your disposal. You can reuse source code, components, development artifacts, patterns, and templates. The following diagram summarizes the types, or categories, of reuse available to you. The left-hand arrow indicates the relative effectiveness of each category – pattern reuse is generally more productive than artifact and framework reuse for example. Similarly, the right hand arrow indicates the relative difficulty of succeeding at each type of reuse. Code and template reuse are relatively easy to achieve because you simply need to find the asset and work with it. Other forms of reuse become hard to achieve; with framework reuse you need to learn the frameworks, with pattern reuse you must learn when to (and when not to) apply various patterns, and with architected reuse you need an effective approach to enterprise architecture in place.
The reuse categories are:
- Architected Reuse. The identification, development, and support of large-scale, reusable assets via enterprise architecture. Your enterprise architecture may define reusable domain components, collections of related domain/business classes that work together to support a cohesive set of responsibilities, or service domains which bundle a cohesive collection of services together.
- Pattern Reuse. The use of documented approaches to solving common problems within a specified context. With pattern reuse you’re not reusing code, instead you are reusing the thinking that goes behind the code. Patterns can be a multiple levels – analysis, design, and architecture are the most common. Ward Cunningham’s site is a useful source of patterns on the web.
- Framework Reuse. The use of collections of classes that together implement the basic functionality of a common technical or business domain. Horizontal frameworks, such as a security framework or user interface frameworks such as the Kendo or QuickUI. Vertical frameworks, such as a financial services framework, are common in some domains.
- Artifact Reuse. The use of previously created development artifacts – use cases, standards documents, domain-specific models, procedures and guidelines, and even other applications such as a commercial off the shelf (COTS) package – to give you a kick start on a new project. Sometimes you take the artifact as is and other times you simply use it as an example to give you an idea about how to proceed.
- Component Reuse. The use of pre-built, fully encapsulated “components” in the development of your solution. In this case a component is self sufficient and encapsulates only one concept. Component reuse differs from code reuse in that developers don’t have access to the source code.
- Template Reuse. This is the practice of using a common set of layouts for development artifacts such as vision documents, training slide decks, or system overview wiki pages within your organization.
- Code Reuse. The reuse of source code within sections of an application and potentially across multiple applications. At its best code reuse is accomplished through the sharing of common classes and/or collections of functions and procedures. At its worst code reuse is accomplished by copying, pasting, and then modifying existing code which adds to your organization’s technical debt and can potentially result in negative overall value.
You can address these reuse categories simultaneously. Framework reuse often locks you into the architecture of that framework, as well as the standards and guidelines used by the framework, but you can still take advantages of the other approaches to reuse in combination with framework reuse. Artifact and component reuse are the easiest places to start, with a little bit of research you can find reusable items quickly. However, if your organization doesn’t have a common development process that it follows you may get little benefit from templates. Pattern reuse is typically the domain of developers with good modeling skills and your enterprise architects should be publishing and providing pattern-oriented guidance to them.
It is important to note that although the diagram indicates that pattern reuse is generally more effective than artifact reuse you may discover that within your organization the opposite holds true. This may occur because you have a comprehensive collection of reusable artifacts in place, because your organization culture is conducive to artifact reuse, or simply because your developers have little experience with patterns.
Let’s start with some definitions:
- Asset. An artifact that is retained after its initial purpose is fulfilled. For example, working source code is an asset because it is retained and potentially updated in the future to address new stakeholder needs. A user story that is discarded once the functionality it describes is implemented is not considered an asset.
- Robust asset. An asset that is appropriately documented, generalized beyond the needs of a single team, thoroughly tested, is of high quality, and ideally has several examples to show how to work with it. Robust assets are much more likely to be reused than items without these characteristics.
- Reusable asset. A robust asset that has been used at least three separate times by at least three separate teams. You can claim that something is reusable, but it isn’t truly reusable until it’s actually been reused; reusability is in the eye of the reuser, not in the eye of the creator.
- Ad-hoc reuse. An informal approach to reuse where individuals or teams reuse whatever they can find on their own. Ad-hoc reuse is often a good start.
- Engineered reuse. A formalized approach to reuse where an organization actively supports a reuse engineering strategy.
- Reuse engineering. The purposeful creation (or rescue), management, support, and governance of reusable assets.
Why Do We Need Reuse Engineering?
The vast majority of developers, agile or otherwise, take an ad-hoc approach to reuse. Although this is a good start, we have the opportunity to do much better. There are several reasons why your organization should consider investing in explicit reuse engineering:
- Quicker time to market. The more reusable assets that a team has at its disposal then the less the team will have to build, thereby enabling them to release quicker.
- Improved return on investment (ROI). Reuse engineering enables IT delivery teams to avoid building something that your organization already has. This leads to greater ROI for your IT investment which in turn leads to greater value being delivered to your stakeholders.
- Improved consistency. When all of your systems use the same implementation of a service, or component, or function, or framework then that functionality by definition is implemented consistently across those systems. This makes them more predictable and easier to understand.
- Easier updates to common functionality. When functionality is implemented in one place and then reused where needed it is very easy to update that functionality and then deploy the updated version.
Why Reuse Engineering is Difficult
Unfortunately reuse is a lot easier to talk about than it is to implement in practice, at least beyond the ad-hoc level. There are several reasons why reuse engineering is difficult to achieve:
- There is a greater impact when reusable assets break. When an asset is used in only one place and it breaks, then the impact of that breakage is limited to that one place. When an asset is reused in dozens or hundreds of places, and it breaks, then the impact is significantly larger. This is reusable assets need to be of high quality.
- Teams must go beyond the reuse of source code. Reuse is often described as not “reinventing the wheel,” and an important step for succeeding at reuse is to understand that you have more than one option at your disposal. For example, in addition to source code you can reuse components, services, patterns, and templates to name a few. More on this in a future post.
- Reuse requires enterprise awareness on IT delivery teams. For reuse to succeed IT delivery teams must understand that reusable assets exist, how these assets fit into your overall IT ecosystem, and what the benefits of reusing them are for your organization. In Disciplined Agile we promote the philosophy that IT delivery teams should work closely with enterprise architects and reuse engineers, if any exist, so that they will better understand and appreciate these issues.
- Reuse requires investment. To get beyond ad-hoc reuse your organization will need to invest in a reuse program. This may include investment in a reuse engineering team, in a reuse repository, in the development/rescue of potentially reusable assets, and the long-term maintenance and support of those assets. More on these topics in future postings.
- Your approach to funding will make or break reuse. Many organizations will institute chargeback schemes to pay for reuse. The basic strategy is that just like you would pay for commercial software, you should pay for internally developed or purchased assets within your organization. That way the cost of those assets is shared fairly across the teams that are actually reusing them. Unfortunately, in practice, chargeback schemes are guaranteed to kill your reuse engineering efforts because you are in effect punishing teams when they reuse things. A better strategy is to purposefully fund your reuse engineering efforts and then track, often via automated metrics, the impact those efforts have on your organization (so as to justify the investment).
In future blog postings we will explore a goal-driven approach to reuse engineering as well as the workflows associated with reuse engineering. It is possible, and highly desirable, for organizations to succeed at reuse engineering. The Disciplined Agile Reuse Engineering process blade provides the guidance you need to do so.