Agile Metrics – PM Network Magazine

PM Network Magazine December 2012 issue have published an article about Agile Metrics, the article includes part of my writing along with other Agile experts. Below is a copy from the Magazine, and I will be posting my full article on my blog soon. Please feel free to share your feedback.

When it comes to measuring and reporting the progress of agile projects, throw tradition out the window.

BY MATT ALDERTON

Consider metrics the magic wand of projects:

They can build trust, communicate progress, expose problems and illustrate the effectiveness of process improvements.

For these reasons, measuring and reporting should be at the heart of every project. Agile projects—now a part of the portfolio at 27 percent of organizations surveyed in the PMI 2012 Pulse of the Profession report—are no exception. These endeavors, however, often require their own tailored metrics, and traditional assessments may not be relevant or adequate.

For starters, agile metrics can be reported more frequently since agile delivers projects via a series of small, bite-sized sprints.

“You can determine a goal for each sprint and meet or exceed it,” says Randy Schmidt, PMI-ACP, Newport News, Virginia, USA-based strategic results architect at GbHawk, an IT contractor for the U.S. military. “After every sprint, the team that’s doing the work has to [demonstrate to] the people who are paying for it that they’re on a path to meeting established goals.”

Because of that increased frequency, agile project teams should highlight only the most important and timely metrics. “Traditional project monitoring focuses on tracking efforts on each activity, whereas agile monitoring facilitates tracking what has been incrementally delivered,” says Praveen Dayal, PMI-ACP, PMP, IT applications manager at IT services provider Dimension Data Asia Pacific in Singapore.

“[With agile,] features start coming to that external group of people much earlier in the process. The principal metric … is what features have been delivered in the last however-many weeks.”

—Randy Schmidt, PMI-ACP, Dimension Data, Newport News, Virginia, USA

In that same vein, each stakeholder group should receive only the metrics relevant to its interests, points out Wafi Mohtaseb, PMI-ACP, PMP, software development manager at the National Bank of Kuwait in Kuwait City, Kuwait. For agile projects, he suggests segmenting reporting by audience, delivering to each member of a stakeholder group only information that is:

• Relevant to their decision-making
  
• Detailed enough to be usable
  
• Appropriate to the timeframe (e.g., day, iteration, release, milestone) they care about
  

Here’s how to segment agile metrics for four common groups: team members, executive sponsors, internal stakeholders, external stakeholders and clients.

TEAM MEMBERS: GET GRANULAR 

More than any other audience, team members need highly specific and detailed information, because they have the greatest immediate use for it.

Mr. Mohtaseb says metrics for team members can be at once informational, describing what’s happening with a project; diagnostic, exposing areas for improvement; and motivational, influencing team behavior.

More detail doesn’t necessarily mean more metrics, however. “Minimize the overall number of metrics and emphasize the ones that tell a story,” Mr. Dayal says. “Produce only what the stakeholder wants.”

Several common metrics give teams the detail they need without overwhelming them:

Velocity: The number of features a team can deliver during a sprint is the principal agile metric, as it allows the team to accurately predict and plan progress, keeping projects on schedule and within budget.

Burn Up/Burn Down: A burn-up chart shows how many features the team has promised to deliver, while a burn-down chart shows how many features it has completed. “Team members are motivated by clearly seeing when they are likely to finish the project, and by seeing the amount of work remaining steadily reduced,” Mr. Mohtaseb says.

Running Tested Features (RTF)/Defect Density: For IT or software agile projects, the number of bugs is a critical quality metric. RTF, a similar measurement, shows how many features in each sprint have passed acceptance tests. “Team members take pride when delivering a high-quality product,” Mr. Mohtaseb says.

Agile techniques such as test-driven development and acceptance test-driven development contribute to preventing defects from being injected into the system in the first place, which greatly reduces defect density when compared with alternative approaches.

EXECUTIVE SPONSORS: GO BACK TO BASICS 

Even on agile projects, traditional metrics are more likely to appeal to executive sponsors and leaders, as their chief concerns are strategic rather than technical. “Sponsors and leaders need to know you’re on time and on budget, and that you’re still going to complete the scope,” says Donna Ackerman, PMI-ACP, PMP, senior project manager at travel solutions provider Sabre Travel Network in Dallas, Texas, USA. “They don’t need to know about anything else—problems or bugs, for example—unless it’s going to affect cost, time and scope.”

Although executive concerns are similar on any type of project, agile metrics differ in that they’re adaptive rather than predictive. On a waterfall project, cost, time and scope are defined at the outset, so the metrics typically emphasize planned values. On agile projects, those constraints evolve as a function of quality, resulting in metrics that highlight work completed. For example:

Burn Down: The executive version of a burn-down report should summarize from a high level how many promised features have been delivered and how many are still outstanding, Mr. Schmidt says.

Earned Business Value (EBV): EBV communicates a project’s progress toward delivering its expected goals. When items from the product backlog are completed, they add to the project’s EBV as a percentage of its cumulative ROI. Because quality is an agile project’s principal objective, EBV tells executives how much value has been delivered thus far to the end user.

If metrics such as EBV prove too complicated, a simple dashboard can explain what the team has committed to, what it has accomplished so far and what it has yet to deliver. “Stick to high-level key performance indicators telling sponsors if the project is on track and delivering the expected business value,” Mr. Mohtaseb says.

INTERNAL STAKEHOLDERS: MAKE IT PERSONAL 

Those who have an interest in a project but aren’t working directly on it—program managers, marketing personnel, human resources officers and internal auditors, to name a few—are generally interested in high-level business metrics, though they often require additional details related to their specific functions.

For example, the marketing team may need to know when a product will be delivered so it can plan a promotional campaign and schedule a demo for the company’s field engineers. Velocity isn’t a useful metric on its own, but it becomes extremely relevant when coupled with an anecdotal explanation, such as “We’re moving this quickly, which means you can send a press release on this date.”

“Internal stakeholders have to be in tune with what’s going on with the team, because they provide the resources the team needs to get things done,” Mr. Schmidt says.

EXTERNAL STAKEHOLDERS AND CLIENTS: SHOW PROGRESS 

Because they can improve time to market, agile practices often most benefit external stakeholders: clients, vendors, partners and the like. If those stakeholders are funding the project, they should receive the same high-level business metrics, such as EBV, that executive sponsors get. Otherwise, the metric they likely care about most is features delivered.

“What’s most compelling in agile is its iterative process, which means features start coming to that external group of people much earlier in the process,” Mr. Schmidt says. “The principal metric for those external folks is what features have been delivered in the last however-many weeks.” PM

Before You Measure

A gamut of agile metrics is available to project managers and the groups with whom they work. Because agile’s very nature emphasizes speed, it demands project managers choose their weapons wisely in pursuit of only the most valuable solutions—for the team, sponsors, internal colleagues and external customers.

“Keeping in line with agile principles, in an agile project you should measure the minimum necessary to satisfy the requirements of stakeholders,” Mr. Dayal says. “But the most important question is, ‘How do you determine the most important metrics?’ It should be based on the requirements of the project sponsors, stakeholders and client.”

By narrowing the reporting scope to only the most necessary metrics, agile teams keep the bumps in the road to a minimum—and keep the work flowing.

Still Unsure?

“Team members are motivated by clearly seeing when they are likely to finish the project, and by seeing the amount of work remaining steadily reduced.”

—Wafi Mohtaseb, PMI-ACP, PMP, National Bank of Kuwait, Kuwait City, Kuwait

The metrics provided here offer a strong foundation for what to present to whom, but if stakeholders are unhappy about what metrics they are or aren’t receiving, use these strategies to learn what additional information will earn buy-in:

• Ask for examples: When stakeholders want more metrics, ask if they have templates of reports they’ve used and liked in the past, suggests Donna Ackerman, PMI-ACP, PMP, Sabre Travel Network, Dallas, Texas, USA. “We have collected many different presentations and templates from customers,” she says. “As long as it’s information we can share, we discuss it, look at the format they want and then provide it.”

• Be transparent: Rather than holding back information, consider being clear about what metrics you can report and letting them choose. “I try to set and manage the expectations for each stakeholder by sharing enough information and details and by being honest all the time,” says Wafi Mohtaseb, PMI-ACP, PMP, the National Bank of Kuwait, Kuwait City, Kuwait.

• Make them co-pilots: Stakeholders who want more information sometimes need more involvement. “The best way to get buy-in from stakeholders is to make them co-owners of the product backlog with the product owner,” says Randy Schmidt, PMI-ACP, GbHawk, Newport News, Virginia, USA. “Get them involved in the construction and grooming of the product backlog up front, early, often and throughout the whole project.”

The Original Article: PM Network Magazine December issue.