The Importance of Backlog Management from a Developer

The speed at which software developers can deliver features within a given time period is increasing rapidly, particularly within the last decade. However, their ability to generate and collect ideas often becomes compromised, creating an organizational impediment to organization.

A software development backlog describes the product’s future, but it often becomes overloaded. This tendency can delay a product’s time to market and reduce its quality. Restoration of the backlog can be a key requirement for getting a project back on track and fostering the innovation that’s so important for successful software development. This post explains the importance of the backlog and how it affects software development. It also discusses how to prevent the backlog from becoming overloaded, which is essential for maintaining a productive development environment.

Overview

A backlog is a list of actions that a development team may complete to achieve a desired outcome for a software application. It can include bug fixes, new features, changes to existing features and infrastructure changes among other possibilities. The backlog is the most authoritative source for team activities, meaning that the team shouldn’t work on anything that isn't on the backlog. However, the mere fact that a task is on the backlog doesn't mean that it will ever be delivered, since it's only an option for delivering an outcome rather than a commitment. Adding an item to the backlog should be a fast, easy process, as should the removal of an item.

The development team owns the backlog and may assign a specific team member with responsibility for maintaining it. A backlog may have a variety of formats, although user stories are the most common. Story maps and information radiators are also common tools that team members use to describe backlog items.

The development team determines the specific format that will best serve as a reminder of tasks they may work on. Backlog items vary considerably in size and detail, primarily depending on when the team expects to work on them. An item that the team will work on soon should be smaller, but contain enough detail to complete the task. Items that lack an expected start date may be broader in scope with less detail.

The sequence of items in a backlog routinely changes as the development team gains a better understanding of the desired outcome and the best solution for achieving an outcome. In addition, the frequent removal and deletion of backlog items makes the backlog highly dynamic. Team members may also use the backlog to communicate their current tasks with other members.

Index cards and sticky notes are common methods of representing a backlog physically. However, it can also be represented digitally with generic text files or spreadsheets as well as dedicated backup management tools. Physical backlogs are most useful for keeping the backlog continuously visible and facilitating group discussion. Digital backlogs are more effective when team members work in physically separate locations or backlog items require a large amount of documentation.

Workflow

The figure below shows the most important processes for managing a backlog. Each process will have its own specific goals.

The entire backlog management process must be properly configured before software development begins, as it can be quite difficult to do so after the project starts. Building the pipeline needed to manage project requirements solves many problems by ensuring this process remains transparent and providing the project's intended vision to all stakeholders. It also helps to reduce the cost of maintaining the backlog.

Balance

The process of balancing the backlog can be visualized by imagining a series of containers, each of which corresponds to a particular action. In this model, balancing the backlog requires you to add the same amount of fluid as you remove, keeping the fluid in each container at a constant level. The diagram below illustrates this process:

Assume for this example that you have a total of 1,200 hours’ worth of work already in progress. You have three crews, each of which can perform 400 hours of work in a week. The work in progress will therefore require one week to complete, resulting in no backlog since all work is currently in progress. However, you then add another 1,200 hours’ worth of work, increasing your backlog to one week. Assuming a maximum allowable backlog of one week, you wouldn’t be allowed to accept more work for another week.

Activity

The workflow of software development is typically neat and simple at the beginning of a project. Everything appears to be under control at this point, but the backlog is usually larger and more complex over time. This trend is the result of actions such as running multiple sprints at the same time, fixing many bugs from these sprints, managing multiple work items and grooming the backlog for new requirements. The following chart illustrates this process:

It’s essential to track backlog activity regularly to ensure it doesn’t get out of hand. In particular, you need to remain aware of the number of work items, their type and who is responsible for each item.

Problems Caused by Overloaded Backlogs

The backlog is the main tool for managing and implementing the roadmap in the Agile development methodology. An overloaded backlog can endanger the project, especially when it's the result of uncontrolled growth. A large, unkempt backlog creates stress on developers, and can eventually cause them to abandon the backlog.

A proactive vision is needed to avoid reactive planning for each sprint, which eventually results in an unmanageable, relevant backlog. In these cases, the development teams can lose sight of their long-term goals and resort to simply completing individual tasks. The specific problems of overloaded backlogs include increased maintenance costs, reduction of information value and inhibited innovation.

Maintenance Costs

An overloaded backlog has a high maintenance cost because the number of user stories that each team member is working on increases with the backlog size. When the team becomes over committed, it also increases the time needed to complete a particular item. The portion of a backlog that is committed acts like a queue, meaning it follows the rules of queue behavior. Specifically, each committed item requires some attention on the part of the developer to remain valid. Developers can only provide meaningful attention to a finite number of items, which is particularly detrimental in the case of cues.

Information Value Reduction

Grooming an overloaded backlog can appear to be an overwhelming process, especially when most items in the backlog appear to be insolvable in practical terms. This process is the most common reason for backlogs to become obsolete. Furthermore, an individual item appears insignificant when it's part of a backlog containing thousands of items, no matter how brilliant that one idea may be. In this case, the backlog can become a trash can containing all the work that developers want to complete that never will.

Inhibited Innovation

An oversized backlog impairs innovation because reorganizing it is a tedious chore. It causes developers to always add brilliant ideas at the very beginning or end of the backlog, neither of which is good practice. Adding items to the front effectively invalidates the rest of the backlog, while adding them to the back virtually ensures they will never be completed. Furthermore, team members working under these conditions will lose their incentive to contribute their best ideas, resulting in declining productivity.

Backlog Size

A development team needs to know how big their backlog can become before it becomes a problem. An experienced team lead can often answer this question with acceptable precision, but specific formulas also exist to make this determination. These methods include estimating the number of backlogged items per quarter and the time needed to review each item.

Backlogged Items per Quarter

This method estimates the maximum number of items that a team can backlog each quarter, although the actual recommended value can vary considerably depending on the organization, product owner and team composition. Assume for this example that a product owner is responsible for maintaining the backlogs for the next quarter. The team performs six to eight sprints in each quarter, with each sprint requiring a maximum of two weeks. Past behavior shows that the development team averages six to eight user stories in each sprint, meaning the number of user stories in the backlog for each quarter should not exceed 7 × 7 = 49. The project manager should therefore investigate the backlog any time it has substantially more than 50 stories per quarter.

Time Needed to Review Items

Estimating the maximum backlog size based on the time needed to review each item requires you to divide the number of items in the backlog by the number of product owners on the development team. This quotient represents the average number of backlog items that each product owner is responsible for. Assume for this example that each backlog item requires five minutes of discussion for each team member to grasp it. You can then calculate the amount of time in minutes that will be required to discuss the entire backlog by multiplying the number of backlog items by five. Divide this figure by 60 to convert the discussion time into hours.

The next step is to determine if it's feasible to spend this amount of time discussing the backlog given the availability of the team members and the other demands on their time. If this time commitment isn't even remotely feasible, then reducing the backlog size should become a priority.

Causes of an Overloaded Backlog

An overloaded backlog usually has many causes that intertwine, but they’re generally driven by human nature. The most common causes include the following:

• Hoarding

• Misusing granularity

• Excessive dependency resolution

• Over commitment by product owners

Hoarding

Many people are reluctant to discard anything, but it's especially difficult to do so for things that may be valuable. This is also true of backlog items because a larger backlog can make it seem that the project is protected from failure. However, the opposite is more likely to be true, making it important to remove backlog items when it will increase the chances of a successful product. If an item has been in a backlog for more than two years, you should strongly consider retiring it. If it truly is a great idea, it will eventually make its way back into the backlog.

Misusing Granularity

Project managers often believe that tracking backlog items at a highly granular level allows them to predict and control law-term problems with scope creep. This tendency is especially strong in conservative managers and those who are just beginning to adopt an Agile methodology. However, a granular backlog is actually against Agile development because it assumes the development team knows exactly what it will do in the distant future.

The granularity of backlog management should reflect the fact that software development can be a highly uncertain process beyond the near future. There isn't a hard and fast rule to granularity but the DEEP acronym coined by Roman Pichler provides a useful description for an effective backlog. The D in this acronym stands for “detailed appropriately”, the first E stands for “estimated”, the second E stands for “emergent” and the P stands for “prioritized”. In other words, the granularity of backlog items should remain as coarse as possible, especially for items in the distant future. Use a granular approach only when absolutely necessary.

Excessive Dependency Resolution

Project managers often work hard to resolve dependency chains in the backlog, typically when the project is not yet completely Agile. This process involves breaking down larger backlog items into their components and assigning more specific goals for each task.

This may sound like an effective strategy for backlog management, but overdoing it can compromise the quality of the backlog in two ways. First, it greatly increases the number of items in the backlog, which also increases the management overhead. Second, it changes the developers' focus from providing value and achieving goals to completing tasks. Project managers who are tempted to resolve every task dependency in the backlog should trust the Agile process rather than trying to manage the backlog for the distant future.

Over Commitment by Product Owners

Product owners are often pulled in many directions during software development. For example, they must decide which customer ideas they will transform into deliverables. These decisions also involve achieving some consensus among the project's many stakeholders, each of which may have a different definition of a successful project. In addition, a product owner must understand the client's business, its customers and the overall market when deciding which features to implement.

Unfortunately, most software development firms don't appoint enough product owners to accommodate the size of their development teams. As a result, backlog quality decreases, causing both the product and the team to suffer. The solution to this problem is for organizations to take the product owner role more seriously by increasing the ratio of owners to team members. The higher this ratio is, the more manageable the backlog will become, making the team more productive.

Refining the Backlog

Refining the backlog often begins by investigating the Design in Process (PIP) inventory, which includes all the backlog items currently in progress. Donald Reinertsen popularized the use of this term in describing the development of lean workflows, particularly for determining the amount of inventory an organization should stockpile at any given time. This concept is also used in software development by treating backlog items as inventory.

In other words, the DIP helps determine the maximum number of backlog items you should have based on the number of items you're currently working on. There is no specific number that answers this question, but the DIP does provide a starting point for measuring the maximum backlog for each product owner. The primary limitation is the number of items that a product owner can effectively administrate. For example, doubling the number of product owners doubles the maximum recommended size of the backlog, all other conditions being equal.

Furthermore, the DIP inventory shouldn't include work that won't be completed for the foreseeable future. If the development team doesn't expect to complete a task for years, even in the best case, that task shouldn't be considered “in progress” for the purposes of its backlog status. Factors such as market maturity and product type help determine the maximum completion time for items in the DIP inventory.

In general, project managers should use Dunbar's number as a guide to the maximum backlog size for each product owner. Anthropologist Richard Dunbar first used this number during the 1990s to describe the maximum number of relationships that a person can maintain at any given time. Dunbar's number can be applied to backlog size because each product owners must maintain a relationship with all tasks in their respective backlogs. In this model of backlog management, product owners should limit their respective backlogs to 150 items.

Summary

Managing a backlog generally consists of keeping it large enough to keep the development team busy, but not so large that it becomes irrelevant. This challenge requires a simple strategy that uses the development team to periodically review the backlog and remove unneeded items. These items will primarily consist of tasks that are unlikely to be completed within a reasonable time frame. Ensuring that backlog items have the right degree of granularity is also a major part of ensuring the backlog remains at a useful size. Finally, the entire development team should regularly participate in refining the backlog.

About Baytech

Baytech is passionate about the technology we use to build custom business applications, especially enterprise solutions that optimize business processes. We’ve been delivering software solutions in a variety of technologies since 1997. Our success is due to the skill and efficiency of our senior staff, which includes software engineers, project managers, and DevOps experts. All of our engineers are onshore, salaried staff members.

We focus on the quality, usability, and scalability of our software and don’t believe in mitigating cost at the risk of quality. We manage project costs by implementing an efficient development process that’s completely transparent and uses the latest standards and practices to build software right the first time. Contact us today to learn more about how we can help your business. Find us online at https://www.baytechconsulting.com/contact.