Metrics – the Heartbeat of Success in ALM and DevOps

Metrics – the Heartbeat of Success in ALM and DevOps
Software products and projects have become more complex today owing to comprehensive product functionality, advanced architectures, and sophistication. Added to this is the increasing business and margin pressure, for which projects are now required to be accomplished within a short time span and with minimal resources. To achieve this, organizations constantly invest in state-of-the-art tools to make sure that every single stakeholder has the proper ammunition for on-time delivery of quality. However, despite this, many projects continue to fail leading the organizations to incur financial losses.

Why so?  Poor visibility of proper metrics is one of the major factors behind this. With data residing in silos, generating proper metrics involving data from across various tools is a huge challenge; resulting in poor control and inaccurate decision making.

What are Software Metrics?

Data always speaks the truth and metrics is the voice of that data.  Both data and metrics are the driving factors behind the success of any state-of-the-art practice in software engineering. Companies need metrics to better understand, track, control and predict software projects, processes, and products. Metrics serve the purpose of providing information needed by engineers for technical decisions as well as information required by mid or top-level management for business decisions. For generating meaningful and informative metrics, every stakeholder who needs to be involved in designing, implementing, collecting, and utilizing it must understand the definition and purpose of metrics.

Some of the major factors that organizations should consider for successful reporting are:

  1. Measuring What Matters: The goal of metrics is improvement through measurement, analysis, and feedback and these include measurements related to productivity, quality, and progress. Stakeholders should find visibility to the metrics that matter the most to them, based on their roles and responsibilities.

Success in ALM and DevOps

  1. Metrics on Everything: Metrics should be designed and generated not only in a single phase but for all phases of a lifecycle, starting from requirements to delivery. The full picture together with end-to-end traceability should also get reported.
  2. Single Source of Truth: Metrics should be centrally visible to stakeholders in real time. It should be a single source of truth available anywhere and anytime.
  3. Monitoring Performance: Projects should be measured in terms of both time and cost. Metrics should be clear about whether the projects are running on schedule or behind schedule and within budget. This gives a greater insight into issues or roadblocks that threaten a project’s schedule or budget.

Metrics for Different Phases

As discussed, metrics should be for anybody and everybody available anytime and anywhere in real time. Some examples of ALM metrics across different phases are given below:

Area Report
Project and Portfolio Management
  • SPI (Schedule Performance Index)
  • CPI (Cost Performance Index)
  • Project Health Report
  • Resource work allocation report
  • Investment by projects
  • Demand by Project Type
  • Demand Vs Capacity by Departments
  • Average phase duration (in days)
  • Portfolio Value Vs. Cost
Requirements Management
  • Requirements Completeness
  • Requirements Traceability Metrics
  • Use Case/Size Metrics
  • Requirements Coverage
  • Requirement Submission Trend
  • Requirements with Open Defects
Test and Defect Management
  • Test Plan Detail
  • Test Execution Summary
  • Test Execution by status
  • Test Run Summary Graph
  • Active Test cases
  • Test Effectiveness
  • Test Case distribution by the submitter
  • Defects Submission vs Fixing Trend
  • Defects by Priority
  • Defects by State
  • Iteration Defect by Priority
  • Iteration Defect by State
  • Defects by Age
  • Defect Resolution Time
  • Defect Density
  • Pre-Deployment Vs Post Deployment Defects

Many companies today are implementing DevOps, where, other than implementing the 7C’s of DevOps, metrics too play an important role. Some of these metrics are listed below:

Area Report
DevOps

Release Management

Build

Deployment

  • Release Health Report
  • Release Status Report
  • Release Defect Trend
  • Release by Items
  • Release Efficiency
  • Releases per Month
  • Release Duration Trend
  • Deployment Status Overview
  • Number of Deployments over time
  • Status wise Deployment Percentage
  • Average Duration of Releases Over Time
  • Release Completion – Planned v/s Completed
  • Average Deployment Duration
  • Build Health
  • Applications with Most Failed Deployments
  • Environments with Most Deployments
  • Applications with Most Deployments

Challenges of Generating Metrics

Though the importance of metrics is acknowledged across the industry, generating them is extremely challenging. Some of the major challenges are as follows —

Data Residing in Silos

Data silos is a major challenge in cross-phase reporting. Companies invest lots of money to procure the best-of-the-breed tools for increasing productivity and delivering quality. However, while this global distribution of disparate tools has led to increased individual team productivity, data gets restricted within the tools. This leads to the creation of silos. As a result, even though metrics could be easily generated for individual phases, there is no central visibility of release predictability and overall project status in terms of cost and schedule.

Error-Prone and Human Dependent

As the data is distributed across tools and reside in silos, things like the generation of centralized metrics and end-to-end traceability are achieved manually, thereby making the data error-prone and flexibility of real-time monitoring challenging.

Integrated Environment for Metrics

With increasing complexities of metrics generation, the need for an integrated environment has become very important. It eliminates the pain of extracting data from different tools on demand and uses offline mediums for generation of metrics. Besides, an integrated environment enables the tools to exchange data among themselves in real time. But integrated environment alone is not the solution. Some major characteristics of an integrated environment are as follows:

  1. Tool integration from all domains and technologies.
  2. Storage support and an interface that can be leveraged for generating reports and dashboards using cross-tool data.
  3. Capable of exposing reports and dashboards based on user roles and responsibilities.
  4. Scalable and easily adaptable for new tools brought into the ecosystem

Kovair Omnibus – The Ideal Platform for Generating Metrics

Kovair Omnibus is an SOA architected proprietary ESB that helps organizations to build such an integrated environment. Some of the major benefits of Kovair Omnibus over others are as follows —

  1. Integration support for ~80 commercial tools catering to different domains of ALM, ITSM, DevOps and more recently some PLM Tools.
  2. Proven experience in integrating with in-house tools developed by individual organizations.
  3. Protection of existing tool investment.
  4. Central platform for configuring and viewing cross-tool data-based reports and dashboards.
  5. Support for all the above-mentioned reports as OOTB reports.
  6. Single screen view and end-to-end traceability using data from different tools.
  7. Implementation of a centralized task-based workflow cutting across different tools.
  8. Adding of new tools within the ecosystem in a seamless manner.
  9. Central data store or data lake for generation of advanced analytics.
  10. Support for access based report viewing.

Some examples of sample reports generated out of Kovair Omnibus are given below:

Defect Reduction Trend

Success in ALM and DevOps

Release Health

Success in ALM and DevOps

Release Summary

Success in ALM and DevOps

Build Report Group by Status

Success in ALM and DevOps

Deployment Frequency Trend

Success in ALM and DevOps

Task Progress

Success in ALM and DevOps

Burn Down Chart

Success in ALM and DevOps

Kanban Reports

Success in ALM and DevOps

Conclusion

Over the last several years, Kovair has successfully catered to the integration needs of many Large, Medium and Small organizations spread across various industry verticals. One of the major reasons for appreciation of Kovair is its capability of generating reports and dashboards using data from across multiple tools. The capability of Kovair Omnibus to generate cross-tool, data-based, end-to-end, and bi-directional traceability is also extremely appreciated by its customers. Kovair Data Lake is one more feather on the same cap. With Kovair Data Lake being launched, Kovair will now not only cater to the integration needs of an organization but will exchange data between tools and cater to the need for a central data for analytical purpose.

Contact: marketing@kovair.com

Source and Reference: https://dzone.com/articles/metrics-the-heartbeat-of-success

Amit brings with him expertise in DevOps, Big Data, Application Lifecycle Management (ALM), Agile Implementation using both Scrum and Lean, SDLC and PPM domains spanning over 18 years. Currently, he strategizes on the growth, market direction, and customer engagements for Kovair's exclusive range of ALM and Integration products. Until his recent assignment, he has held the responsibility of defining new products and marketing plans for them. Amit holds a Project Management certification from PMI Institute and is also a Certified Scrum Master.

One comment

Leave a Reply

Your email address will not be published. Required fields are marked *