Cognition Labs, a startup with $21 million in funding, this week previewed what it describes as the world’s first software engineer, dubbed Devin, that is based on artificial intelligence (AI).

Devin is capable of responding to text commands and can be assigned tasks such as assessing the benchmark performance of an application. It will then build a plan and configure the required tools using its own command line interface (CLI), code editor and browser through which it can access, read and comprehend, for example, documentation using a reasoning engine and long-term planning capability based on advances in reinforcement learning.

Step 1 of 7

14%

Which of the following best describes how you currently monitor and manage systems across your hybrid cloud environment?(Required)

Multiple separate tools for different environments

Multiple separate tools for different monitoring and management functions

Single consolidated dashboard

Custom built monitoring and management solution

No full monitoring and management solution

Don’t know

How valuable would it be for your monitoring and management solution to incorporate predictive analytics to help you identify and address system issues that are critical and not waste time or effort working through all the data and “noise” that is not critical?(Required)

Extremely valuable

Moderately valuable

Somewhat valuable

Slightly valuable

Not valuable

How do you most typically address potential system issues BEFORE they impact your operations?(Required)

Automated workflows and playbooks to remediate potential issues as soon as they are detected

An orchestrated response process that leverages automation, AI, and human expertise to rapidly respond to potential issues

Advanced monitoring tools such as predictive analytics and anomaly detection to proactively identify and address potential issues

Respond to issues as they arise, using manual processes and troubleshooting techniques

A combination of automated remediation, orchestrated response, and/or proactive monitoring to address potential system issues

Don’t know

Other

What methods does your organization use to control and monitor cloud/IT spending?(Required)

Centralized procurement processes

Pre-approved budget thresholds

Automated cost monitoring tools

Department-level spending controls

Regular cost audits and reviews

Cloud cost optimization platform

Approval workflows for resource provisioning

Native financial operations tools provided by the cloud provider

No formal controls in place

Don’t know

Other

Please specify:

Which methods do you rely on most to detect security vulnerabilities in your systems?(Required)

Automated scanning tools

Manual assessments

Vendor notifications

Threat modeling (for early detection)

None of these

Don’t know

Other

Please specify:

What are the TWO most common ways you prioritize vulnerabilities to address within your environment?(Required)

Based on potential impact

Based on exploitability

Regulatory compliance requirements

Based on CVSS scores

Risk-based analysis of multiple factors

First-in-first-out approach

No formal prioritization process

Don’t know

How do you typically handle system updates and security patches?(Required)

Automated updates and patches

Manually review and apply updates and patches on a regular schedule

A combination of automation and manual review

Apply updates and patches on an as-needed basis.

Outsourced update and patch management

No defined process

Don’t know

Other

Δ

Those capabilities make it possible for Devin to, for example, build a website, autonomously identify and fix bugs in codebases, deploy applications and even train other AI models.

Cognition Labs said a benchmark evaluation of Devin, using an SWE-bench tool that asks agents to resolve tasks, found Devin correctly resolved 13.86% of the issues end-to-end, far exceeding the previous state-of-the-art of 1.96%. Even when given the exact files to edit, the best previous models could only resolve 4.80% of issues.

Mark Hinkle, CEO of Peripety Labs, a consulting firm, said it’s too early to tell when Devin might be ready for use in enterprise IT environments, but the demonstrations showed how rapid advances in AI are about to transform how software is built and constructed. The user experience is substantially different from that of GitHub Copilot or CodeWhisperer from Amazon Web Services (AWS), which are more focused on helping developers write code rather than executing assigned tasks, he noted.

It’s not clear how much infrastructure is required to run Devin, which, given the ongoing shortage of graphical processor units (GPUs), may prove to be a limiting adoption factor, at least in the short term, added Hinkle.

Regardless of Devin’s current capabilities, which software engineers can only access by invitation right now, it’s clear advances to the reasoning capabilities of AI models will enable DevOps teams to assign tasks to an AI model much like they would any other member of their team. The impact that capability will have on demand for software engineers remains to be seen, but there is still a need to understand what to ask Devin to build and review how the software Devin builds has been constructed. The one thing that is certain is the pace at which software can be built and deployed is about to accelerate.

Undoubtedly, Microsoft, AWS, and other providers of software engineering tools are researching similar capabilities, so it may now be only a matter of time before advances in reasoning and long-term planning are broadly applied to software development.

In the meantime, DevOps teams might want to reassess their strategic plans for the coming years as it becomes easier to build software. Projects that might have required large teams of software engineers to complete might soon be accomplished using much smaller teams. In turn, the number of software development projects that could be launched by organizations large and small will increase as more of the toil historically required continues to be eliminated.