Many established enterprises are stuck in reverse. Dragged down by legacy IT systems and sclerotic software workstreams, these businesses simply cannot adapt and get up to speed in the continuously changing, high-velocity global marketplace of the 21st century. Delta Airlines’ recent technology crash, for example, dramatically illustrates how aging and inflexible IT systems can cripple a company in 2016 and lead to millions of dollars in lost revenue, as well as the dissipation of a hard-won reputation among customers.
But an increasing number of organizations are accelerating their cultural transformations to become software-led companies that can consistently deliver new products and services as a result of automated and collaborative workflows. Alaska Airlines, which focuses relentlessly on measuring the innovation velocity of its business, is a powerful case in point.
This is DevOps at work, and it’s how IT must operate to be the business engine of our quicksilver digital economy.
Measuring the return on investment (ROI) from DevOps is critical to convince organizations falling behind that they can still shift course, transform and head to the IT future. Measuring DevOps ROI is also essential for companies that are embracing the industry’s forward movement, so they know if they’re on the right track and can keep turbo-charging their DevOps efforts.
The most proactive approach for measuring DevOps ROI is to gauge the impact of products and services on customers. But, to get there, we must also assess and analyze the efficiency and effectiveness of the software development process.
Which brings us to the one metric that’s truly relevant in business today—velocity. Without velocity, you can’t quickly recognize and remediate problems. More significantly, you’ll fall behind in the marketplace because you can’t innovate fast enough to keep pace with competitors and customer demand.
Enter DevOps: The culture of innovating at velocity.
Of course, that’s easy to say, difficult to measure. The real ROI of DevOps is how rapidly you can ship your new ideas to customers—and, importantly, understand if those ideas meet customer needs.
Many may look at cost savings and headcount as their first DevOps metrics. These are important barometers but don’t measure long-term value back to the business. Two places to start for value-oriented metrics are time to delivery and quality of release.
Measuring time to delivery shows how long it takes to get new product from code checking to production. Ideally, you want to see your time to delivery drop from months to weeks to days, and then literally to hours and minutes.
Quality of release reveals how many bugs and errors are found in each iteration of the product. Companies should look to remedy these immediately and to develop a pre-production environment and workflow of automated testing for issues.
It’s equally important to measure how rapidly you can get feedback from customers and incorporate that into the next development cycle. Development teams must get clear, direct and actionable feedback about whether their products are moving the business forward. For example, did the two-week sprint to build a new landing page for a bank’s website increase customers’ online banking sessions? Did it generate more cross-sell for the bank’s product portfolio? How soon can we see an uptick in customer satisfaction? This is significant information, and the question is, how fast do the developers respond to this feedback?
The bottom line is that DevOps metrics should always connect back to the business; they need to show a measurable improvement in the customer or staff experience. This could be more quality touchpoints with customers, an improved customer satisfaction score (such as a Net Promoter Score) or increased customer activity in areas such as online billing or account management.
“Delivering on the final product is key,” says Adam Mikeal, director of Information Technology at the College of Architecture, Texas A&M University. “Your job’s not done if the customer can’t accomplish what they want. If the customer can’t do it, it doesn’t matter if your section of the code was good. The customer experience is all that matters.”Â
Another important part of this equation is measuring your team’s efficiency on the road to velocity. To truly ship ideas at speed, and positively impact your business and its customers, you also need to understand how smoothly your teams are working—in addition to how quickly they’re delivering.
Banks Cash In on DevOps
Westpac Bank in New Zealand fully grasps this imperative, for example, and it enacted DevOps practices to deploy a new pricing engine for mortgages in just three months—from concept to customer. In the past, this project would have taken Westpac 18 months.
Additionally, Westpac measures how well it’s using resources—for every $1 spent on a problem, how much is chewed up by politics and how much actually goes to solving the problem? Before adopting DevOps practices, Westpac was delivering a return of about 20 cents to 22 cents. Now, that number is just shy of 40 cents. That’s a near-2x increase in the value delivery. Westpac’s goal is 3x. In addition, Westpac has increased the amount of work its development teams accomplish by more than 40 percent over the previous year. Â
Like Westpac, Standard Bank in South Africa has boosted efficiency, thanks to a thriving DevOps culture. The cost of production at Standard, for example, has dropped from about $4,000 to $700—a 300 percent improvement.
The goal for companies in every industry and economic sector around the world today is clear: Accelerate your business so that you create and provide greater—and faster,—value for the customer. But, to drive a robust ROI on this journey, it’s crucial that you carefully measure the true velocity of your organization, whether you’re just beginning with DevOps or you’re well along in the transformation.
— Justin Arbuckle
