The annual DevOps Research and Assessment (DORA) published today by Google finds that while generative artificial intelligence (AI) is leading to moderate gains in productivity, it also appears to be slowing the rate at which software is being delivered.

The report finds more than three quarters of the software development professionals surveyed (76%) are relying on AI for at least one daily professional responsibility, with specific productivity gains including a 7.5% increase in documentation quality, a 3.4% increase in code quality and 3.1% increase in the speed at which code is being reviewed.

Step 1 of 7

14%

Which of the following best describes how you currently monitor and manage systems across your hybrid cloud environment?(Required)

Multiple separate tools for different environments

Multiple separate tools for different monitoring and management functions

Single consolidated dashboard

Custom built monitoring and management solution

No full monitoring and management solution

Don’t know

How valuable would it be for your monitoring and management solution to incorporate predictive analytics to help you identify and address system issues that are critical and not waste time or effort working through all the data and “noise” that is not critical?(Required)

Extremely valuable

Moderately valuable

Somewhat valuable

Slightly valuable

Not valuable

How do you most typically address potential system issues BEFORE they impact your operations?(Required)

Automated workflows and playbooks to remediate potential issues as soon as they are detected

An orchestrated response process that leverages automation, AI, and human expertise to rapidly respond to potential issues

Advanced monitoring tools such as predictive analytics and anomaly detection to proactively identify and address potential issues

Respond to issues as they arise, using manual processes and troubleshooting techniques

A combination of automated remediation, orchestrated response, and/or proactive monitoring to address potential system issues

Don’t know

Other

What methods does your organization use to control and monitor cloud/IT spending?(Required)

Centralized procurement processes

Pre-approved budget thresholds

Automated cost monitoring tools

Department-level spending controls

Regular cost audits and reviews

Cloud cost optimization platform

Approval workflows for resource provisioning

Native financial operations tools provided by the cloud provider

No formal controls in place

Don’t know

Other

Please specify:

Which methods do you rely on most to detect security vulnerabilities in your systems?(Required)

Automated scanning tools

Manual assessments

Vendor notifications

Threat modeling (for early detection)

None of these

Don’t know

Other

Please specify:

What are the TWO most common ways you prioritize vulnerabilities to address within your environment?(Required)

Based on potential impact

Based on exploitability

Regulatory compliance requirements

Based on CVSS scores

Risk-based analysis of multiple factors

First-in-first-out approach

No formal prioritization process

Don’t know

How do you typically handle system updates and security patches?(Required)

Automated updates and patches

Manually review and apply updates and patches on a regular schedule

A combination of automation and manual review

Apply updates and patches on an as-needed basis.

Outsourced update and patch management

No defined process

Don’t know

Other

Δ

However, the global survey also finds there has been a 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability.
Nathen Harvey, DORA lead and developer advocate at Google Cloud, said it’s not clear what the root cause of those declines are, but it’s probable that code written by AI platforms needs to be fixed before applications are deployed in a production environment.

In fact, the survey finds that 39% of the respondents have little to no trust in AI-generated code.

These concerns don’t mean organizations should abandon AI, but software engineering teams should focus their initial efforts on areas where AI provides a clear benefit, said Harvey. It’s clear, for example, when properly applied AI can reduce toil, but any code being generated should be closely reviewed and tested, he noted.

Mitch Ashley, a Futurum Group vice president and practice lead for DevOps and application development, concurred that it would be premature to sign on to the idea that AI is making developers less efficient. Taking the DevOps long view, the 2024 results indicate DevOps continues to adapt and mature to changes in how we create software, he added.

The latest DevOps research shows the principles of experimentation, continuous learning and redefining how we create software are very much in play with AI in DevOps, noted Ashley.

The DORA report this year finds software engineering teams are also encountering similar issues when embracing platform engineering as a methodology for managing DevOps workflows at scale. Application developers are becoming more productive, but there also appears to be a decrease in performance.

In time, as AI platforms become more advanced and platform engineering methodologies better understood, software delivery issues will be addressed, said Harvey. In the short term, however, organizations will need to more thoughtfully manage these transitions, he added.

DORA metrics track change lead times, deployment frequency, change failure rates and failed deployment recovery time. These metrics are both leading and lagging indicators of the impact DevOps is having on the business, said Harvey. Each organization should consider them within the context of their own business goals, but, in general, keeping track of these metrics will enable DevOps teams to reduce burnout while at the same time improving productivity, he noted. For example, the elite performers identified in the DORA report are delivering software 182 times faster than the lowest performing teams identified.

It’s not clear to what degree AI and platform engineering will transform how DevOps workflows will be managed but, in many instances, they will undoubtedly be joined at the hip. Organizations of all sizes are looking to accelerate the rate at which applications are built and deployed. While the first iteration of AI and platform engineering may not initially have much of an impact, the potential is simply too great for any DevOps teams to completely ignore.