AI-Native Software Engineering

At my current org, I work on the internal developer platform and we've been using frontier AI models a lot - for building the platform and for helping other teams adopt AI in their own work. At some point I realized I was just using these tools without really knowing what the best teams are doing differently. So I started reading - engineering blogs, research papers, conference talks, earnings calls. This post is what I've pieced together so far. I'll keep updating it as things change.

The Productivity Paradox

The first thing I looked into was the basic question: does AI actually make developers faster? The answer is not what I expected.

The study that changed how I think about this came from METR (Model Evaluation and Threat Research). They ran a randomized controlled trial with 16 experienced open-source developers across 246 real tasks, using Cursor Pro with Claude 3.5/3.7 Sonnet. [10]

The result: developers completed tasks 19% slower with AI tools. But here's what makes this study so interesting - before starting, the developers predicted AI would make them 24% faster. After finishing, they still believed they were 20% faster. The gap between what they felt and what happened was roughly 39 percentage points.

This doesn't mean AI is useless for coding. GitHub's large-scale study with Accenture (4,800 developers) found tasks completed 55% faster on a controlled JavaScript exercise. [37] Anthropic's internal survey of 132 engineers showed a 50% median productivity boost. [7] Jellyfish data tracking 600+ organizations shows companies with high AI adoption achieving 110%+ productivity gains. [25]

How do you make sense of these contradictions? It comes down to what kind of task you're doing. If the task is well-defined and self-contained - writing boilerplate, wiring up an API, building a CRUD endpoint - AI genuinely helps, sometimes by 25-81%. But if you're working in a large codebase you already know well, the AI's generic suggestions often slow you down more than they help. And here's the bigger picture: Bain & Company found that real-world savings are usually just 10-15%, because developers only spend 20-40% of their time actually writing code. The rest is reading, reviewing, debugging, and communicating.

So even in the best case, the speed gains are smaller than they feel. But speed is only half the story. The other half is what happens to code quality when AI is writing more of it.

CodeRabbit's December 2025 analysis of 470 pull requests found AI-generated code has 1.7x more issues per PR, 1.57x more security vulnerabilities, and 2.74x more XSS vulnerabilities than human-written code. [16] GitClear's analysis of 211 million lines of code shows refactoring collapsed from 25% to under 10% of commits, while code duplication grew 4x. [18] Cortex's data across 1,255 teams shows change failure rates up 30% and incidents per PR up 23.5%. AI is producing more code, faster, with more bugs.

How to Think About AI-Assisted Development

Martin Fowler has published extensively on this throughout 2025-2026, and his thinking gives teams a practical framework. His core argument: LLMs are a paradigm shift comparable to the move from assembly language to high-level languages - not just a productivity tool but a change in the nature of programming itself.

His most useful insight, drawing on Rebecca Parsons: all LLM output is hallucination - we just find some of it useful. This reframe has practical consequences. Teams need to borrow the concept of tolerances from structural engineering - determining acceptable error rates for AI-generated work.

LLMs are quite happy to say ‘all tests green,’ yet when I run them, there are failures.
— Martin Fowler

Three principles from Fowler's work:

• Refactoring is more important than ever. AI-generated code duplicates logic rather than abstracting it. GitClear confirms refactoring collapsed from 25% to under 10%. Schedule refactoring sprints deliberately.
• Review is the new bottleneck. “There's a lot more code going out there, a lot more code to review.” If you don't restructure review, you'll drown in AI-generated PRs.
• Experiment and share openly. Fowler is characteristically honest: “Anyone who says they know what this future will be is talking from an inappropriate orifice.”

Kent Beck's complementary view: TDD is a superpower when working with AI agents, because agents actively introduce regressions - and will even delete tests to make them “pass.” [21] His system: write failing tests first, let AI generate code to pass them, then review and refactor. 72% of professional developers say “vibe coding” (hoping AI output just works) is not part of their work. [23]

The ThoughtWorks Technology Radar Volume 33 (November 2025) formalized an important shift: context engineering has replaced prompt engineering as the key skill. [33]

The difference matters. Prompt engineering is about crafting the right question. Context engineering is about curating what the AI can see before it even starts working. In practice, this means:

• CLAUDE.md / AGENTS.md files at the repo, module, and project level - giving agents the project's conventions, architecture decisions, and coding standards
• Tool subsets per task - Stripe's Toolshed has 400+ tools but agents only see a curated handful relevant to their task. Too many tools degrades reasoning.
• Just-in-Time (JIT) instructions - Shopify ran into what they call the “tool complexity problem.” At 0-20 tools, things were fine. At 20-50, boundaries blurred. At 50+, the system prompt became an unwieldy mess of special cases and conflicting guidance - what they called “death by a thousand instructions.” Their fix: instead of cramming all guidance into the system prompt upfront, they return relevant instructions alongside the tool data only when that tool is actually called. The system prompt stays focused on core behavior, and modifying one tool's instructions doesn't invalidate the cache for the entire prompt. After implementing JIT, maintenance costs dropped, response speed improved, and task completion accuracy went up.
• Repository vector indexes - GitHub Workspace indexes your entire codebase so the agent understands how changes relate to the broader project
• “Onboard like a new hire” documentation - OpenAI's Harness team found that if an architectural decision lives only in a Slack thread, it's invisible to the agent

The common thread: the quality of AI output depends more on what you put into the context window than on how clever your prompt is.

What the Best Companies Are Actually Doing

2024

GitHub Copilot

20M users

4.7M paid subscribers, 75% YoY growth.

Deployed at 90% of Fortune 100 companies.

Auto-reviewed 8M+ pull requests by Apr 2025.

Amazon

$260M/yr saved

Q Developer migrated 30,000 Java apps internally.

79% of auto-generated code shipped unchanged.

Largest enterprise AI migration to date.

2025

2025-26

Google

~50% AI-generated

Rose from 25% (Q3 2024) to 50% in 18 months.

All AI code reviewed and accepted by humans.

Autorater evaluation loops woven into pipelines.

DoorDash

90%+ daily usage

AI tools used by engineers every working day.

Integrated across frontend, backend, and mobile.

Q1 2026

Q1 2026

Airbnb

80%+ engineers

Majority of engineering org uses AI tools daily.

Focus on search, listings, and payments teams.

Stripe

1,300+ AI PRs/week

Minions: production-grade autonomous agents.

Blueprint architecture: deterministic + agentic nodes.

70% of AI PRs merge without modification.

Hard cap: max 2 CI rounds per agent task.

Feb 2026

Q1 2026

Meta

30% output increase

Per-engineer productivity boost reported.

AI coding integrated across all product teams.

HubSpot

97% AI-assisted

Nearly every commit involves AI tooling.

Highest reported AI adoption rate in industry.

Q1 2026

Q1 2026

Grab

98%+ engineers

Virtually all engineers use AI coding tools daily.

Southeast Asia's largest tech company by AI adoption.

Anthropic: the most aggressive AI-native culture

70-80% of Anthropic's technical employees use Claude Code every day. The majority of their code is now written by Claude Code. The tool itself was 90% written by Claude Code. [7] Their December 2025 survey of 132 engineers found AI used in 59% of work (up from 28% a year earlier), with a 67% increase in merged PRs per engineer. The most interesting finding: 27% of Claude-assisted work consists of tasks that wouldn't have been done otherwise - papercut fixes and quality improvements teams couldn't justify before.

Their workflow: Explore, Plan, Code, Commit. They start by preventing Claude from writing code, having it research and plan first. Engineers run 2-4 simultaneous Claude instances using git worktrees. They describe themselves as “managers of AI agents” - spending 70%+ of time reviewing, not writing.[46]

But Anthropic's survey also surfaced real concerns. Engineers worry about skills atrophy - the “paradox of supervision” where effective AI oversight requires coding skills that may decay from AI overuse. Social dynamics are shifting: “I work way more with Claude than with any of my colleagues.” Mentorship is disrupted: “More junior people don't come to me with questions as often.”

One notable experiment: Nicholas Carlini used 16 Claude Opus 4.6 agents in parallel Docker containers to build a 100,000-line Rust-based C compiler. It passed ~99% of GCC torture tests and compiled QEMU, FFmpeg, SQLite, Postgres, Redis, and Lua. Cost: $20,000. [8]

Stripe: 1,300 AI-written PRs per week

Stripe's Minions system is one of the better-documented production-grade agent deployments. [1] [2] The core architectural innovation: Blueprints - sequences where some nodes run deterministic code (file I/O, linting, tests, git) and others run an LLM for judgment. The design principle: “The system runs the model, not the other way round.”

Agents run on “devboxes” - isolated EC2 instances that were originally built for human developers. Agents walked in and benefited automatically. Their Toolshed MCP server has 400+ tools, but each agent gets a curated subset per task - never all 400 at once, which would drown the context window.

The feedback loop is aggressive and capped. Local lint runs on every push in under 5 seconds. CI runs selective tests from a 3M+ test battery, with autofixes applied automatically for known failure patterns. If failures remain, the agent gets one more chance. Hard cap: maximum 2 CI rounds. If code doesn't pass after the second push, it goes back to the human. A typical on-call scenario: engineer fires off 5 Slack tasks before getting coffee, returns to find 5 PRs ready, approves 3, sends feedback on 1, discards 1. About 70% of AI-written PRs merge without modification.

OpenAI: the Harness experiment

OpenAI's Harness project built an entire product where every line of code was written by Codex agents. Humans steered; agents executed. It shipped ~1 million lines in weeks at roughly 1/10th the time of hand-written development. [3]

Their key insight: “Context is everything, not prompting.” That Slack discussion that aligned the team on an architecture pattern? If it isn't discoverable in the repo, it's invisible to the agent - just like it would be to a new hire joining three months later. Teams need to document architectural decisions, engineering norms, and product principles in-repo, not in ephemeral Slack threads.

They favor “boring tech” - stable APIs, composable libraries, things well represented in training data. Technologies that are “boring” tend to be easier for agents because of composability, API stability, and representation in the training set. The key workflow: engineers run 3-4 completely independent tasks simultaneously. They describe the problem in short sentences, fire off the task, immediately switch to the next one, and return later to check status.

Other companies

Shopify built a centralized LLM proxy rather than standardizing on a single tool - allowing experimentation while keeping centralized cost control. [14] Their key lesson: “Standardize infrastructure, not tools.” Senior engineers now run up to 10 parallel agents simultaneously.

GitHub's Copilot Coding Agent works asynchronously - you assign an issue and come back to find a ready PR. [27]

Their March 2026 agentic code review creates a closed loop: review finds problem, Coding Agent generates fix, engineer reviews the result. Microsoft's internal system AI-reviews 90%+ of their 600K+ monthly PRs. [28]

Google's defining lesson for 2025: “Agents got jobs, evaluation became architecture, and trust became the bottleneck.” [13]

Their autorater system - an LLM acting as judge - evaluates each agent output in real-time. When it detects an error, it provides feedback the agent uses to retry and correct itself, without human intervention for routine issues. High-stakes situations escalate to human approval. This is a fundamental shift: evaluation is not something you do after the fact - it's an active component in the execution pipeline. ~50% of Google's code is now AI-generated. [53]

Google also sees role blurring accelerating: “Five years ago, companies had very clear roles - backend engineers, frontend engineers, architects, designers, quality engineers. This siloed approach is evolving. Today's tools enable engineers to operate across domains that previously required specialized expertise.”

Six Principles of AI-Native Engineering

Across all these companies, the same patterns keep showing up. No one coordinated these - they emerged independently from teams solving the same problems at scale.

The anti-patterns are equally consistent. Giving agents unlimited tool access degrades quality (Shopify hit this at 50+ tools). [14] Uncapped retry loops waste tokens - Stripe caps at 2 CI rounds. Measuring code volume without engineering outcomes is meaningless - Port.io found 21 companies reporting AI code metrics in earnings calls, but zero connecting them to actual outcomes. [35]

The SWE-bench Leaderboard: What It Really Measures

SWE-bench Verified is the most-watched benchmark for coding agents. It tests whether an AI can take a real GitHub issue and produce a working pull request. Top scores jumped from about 65% in early 2025 to 80.9% by March 2026. [43]

These scores look impressive, but they don't tell the full story. SWE-bench Pro is a harder version of the same test, designed so models can't cheat by having seen the answers during training. On Pro, the best models score only ~23%. And even on the standard version, METR found that about half the PRs that pass the tests would still get rejected by actual maintainers - they pass technically but aren't good enough to merge. Devin AI merges 67% of its PRs in production, but when independently tested on complex tasks, that dropped to just 15%. [30]

The AI coding tool market

GitHub Copilot has 20 million users, 4.7 million paid subscribers, and is at 90% of Fortune 100 companies. [57] Cursor went from $1M ARR in 2023 to $2B+ by February 2026 with just 40-60 employees - the fastest-growing SaaS company in history. But The Pragmatic Engineer's 2026 survey found Claude Code has overtaken both Copilot and Cursor as the most-used AI coding tool, just 8 months after launch. [22]

How This Changes Agile, DevOps, and Architecture

Sprint planning needs a new step

During sprint planning, teams now need to decide which tasks are good for AI and which need a human. Architecture decisions, security-sensitive code, and anything requiring deep domain knowledge should stay human. Boilerplate, test scaffolding, migrations, and well-defined implementation tasks can go to AI.

Two practices that help: create CLAUDE.md or AGENTS.md files that tell AI tools about your project's conventions, and write detailed specs before generating any code. The spec becomes the contract the agent works against. [9]

Your Definition of Done needs updating

AI-generated code needs a stricter checklist than human code. Based on what teams are reporting:

• Same PR review process as human code - no shortcuts
• AI-generated sections tagged so you know what came from where
• Mandatory security review for anything touching login, access control, or data storage
• At least 70% test coverage, with humans writing the test descriptions
• Check what packages AI added - one team found 23 new npm packages after a month of heavy AI use, 7 of them unmaintained and 2 with known vulnerabilities

Without these guardrails, teams report 35-40% more bugs within six months. [39]

TDD matters more now, not less

This is the one thing everyone agrees on - Fowler, Beck, DORA elite performers, the Codemanship research group. [32] The workflow: write failing tests first, let AI write code to pass them, then review and clean up. Studies show AI produces measurably better code when you give it tests alongside the problem description. [38]

One thing to watch for: AI agents will sometimes delete or disable tests to make them “pass.” If your tests break after an AI change, go back to the last working commit. Leaving broken code in the context window makes all future AI output worse.

Code review is now the slowest step

AI generates code faster than teams can review it. PRs are about 18% larger with AI, and incidents are up ~24%. As Addy Osmani put it: “AI did not kill code review. It made the burden of proof explicit.” [31]

What works in practice:

1. Let AI do the first pass - style checks, basic security, static analysis
2. Human reviewers focus on the hard stuff: architecture, intent, edge cases, security boundaries
3. Extra attention on anything touching auth, payments, or user data
4. Every PR needs proof it works - tests, verification logs, screenshots
5. Break AI output into small commits, not one giant PR

Microsoft does this at scale - AI assists in 90%+ of their 600K+ monthly PRs. [28] The rule: the human always has final say.

Standups and retros need new questions

Add one question to your daily standup: “What AI tools did you use and what did you learn?” This turns individual discoveries into team knowledge. For distributed teams, AI-summarized async standups can pull out blockers and group related work, saving managers 1-2 hours a week. [23]

For retros, let AI analyze your sprint metrics - defect trends, cycle time, ticket churn - and surface patterns you might miss. But be aware of the isolation problem: only 17% of AI agent users say agents improved team collaboration. The main benefit is still personal productivity. Make sure people keep pairing and talking to each other, not just to their AI tools.

Make your codebase agent-friendly

Consistent naming, strong typing, and well-scoped modules make a huge difference for AI agents. Code that humans navigate through tribal knowledge (“oh, that function name is misleading but everyone here knows what it does”) is a dead end for agents. Treat agent-friendliness as an architecture concern, just like performance and security.

AI is creating tech debt faster than humans ever did

GitClear analyzed 211 million lines of code and found a worrying pattern: [18] refactoring dropped from 25% to under 10% of commits. Code duplication grew 4x. Complexity went up 39% in repos where agents do most of the work. The reason is simple - agents optimize for making tests pass, not for clean architecture. The fix: schedule dedicated refactoring sprints, and give AI refactoring tasks too, not just feature work.

CI/CD is getting smarter

Build pipelines are evolving from fixed step-by-step sequences to flows that adapt based on what changed:

AI quality gates. Add AI-specific rules to your existing static analysis tools (SonarQube, Snyk, ESLint). AI code tends to have patterns human code doesn't - excessive I/O operations (8x the human rate per GitClear), duplicated logic instead of abstractions, and overly permissive error handling. Create custom rules that flag these. Run them as a required CI step that blocks merge if they fail.

Smart test selection. Instead of running your entire test suite on every push, analyze which files changed and only run the tests that cover those files. Tools like Bazel, Jest's --changedSince, and Stripe's selective test runner do this. The payoff is huge - Stripe has 3M+ tests but only runs a relevant subset per push. This turns a 30-minute CI run into a 3-minute one.

Self-healing pipelines. Set up monitors that watch your deployment metrics (error rates, latency, CPU). When something spikes after a deploy, the pipeline automatically rolls back to the last known good version, pages the on-call, and creates a ticket with the context. No human needs to wake up at 3am to click “rollback.” Kubernetes with Argo Rollouts or Flagger can do progressive delivery with automatic rollback built in.

Flaky test detection. Track which tests sometimes pass and sometimes fail on the same code. ML models can learn to identify these by looking at test history, execution time variance, and dependency patterns. Quarantine flaky tests into a separate non-blocking suite so they stop slowing down your team. Fix them in dedicated cleanup sprints.

Sandboxed environments for agents. AI-generated code should never run directly against production data or services during development. Give each agent its own isolated container with no internet access and no production credentials - similar to how Stripe runs Minions on disposable devboxes and how OpenAI Codex disables internet during execution. If the agent writes something destructive, the blast radius is zero.

Separate AI vs. human dashboards. Tag commits and PRs by whether they were AI-generated or human-written. Then track error rates, change failure rates, review time, and incidents separately for each. This tells you whether AI is actually helping or just producing more code that breaks more often. Without this split, you can't tell if your rising incident count is from AI adoption or something else entirely.

Monthly AI code audits. Once a month, randomly pick 10 files that were primarily AI-generated. Have a senior engineer do a deep review - not for correctness (CI should catch that) but for architectural quality, unnecessary complexity, hidden tech debt, security patterns, and whether the code is actually maintainable by a human. Document what you find and feed it back into your CLAUDE.md context files so the AI makes fewer of the same mistakes.

About 40% of platform teams have adopted some form of AIOps, cutting unplanned downtime by ~20% (Gartner).

Self-healing software is starting to work

The idea: AI agents watch your pipelines, detect when something breaks, figure out the cause, and fix it - or roll back to the last working version - all without a human touching it. Teams using these patterns report mean time to recovery dropping from 2-4 hours to under 30 seconds. [48] For a company losing $10K/hour across 50 incidents a year, that's $2M+ saved annually.

The multi-agent landscape

Three frameworks have emerged as the main options: LangGraph for production-grade work (600-800 companies using it), CrewAI for quick prototyping (100K+ certified developers), and Microsoft Agent Framework for Azure environments. Gartner saw a 1,445% increase in multi-agent inquiries and predicts 40% of enterprise apps will have embedded AI agents by end of 2026.

Two protocols matter: Anthropic's MCP (Model Context Protocol) for connecting AI to tools, and Google's A2A (Agent-to-Agent) for agents talking to each other. ThoughtWorks put MCP in their Trial ring but warned against blindly converting every API to an MCP endpoint. [33]

What Comes Next: 2026-2027

AI coding agents are getting better fast. The length of task they can handle reliably is doubling roughly every 5-7 months. Claude Opus 4.5 (November 2025) could handle ~5-hour tasks at 50% reliability. By end of 2026, models are expected to manage 20-hour tasks - almost half a work week. The AI 2027 project predicts a “superhuman coder” by March 2027, though current progress is tracking at about 65% of that pace. [50]

Claude Opus 4.6 currently holds the record for the longest task an AI can do reliably: 14 hours 30 minutes (measured by METR). [49]

Context windows - how much information the model can see at once - have settled at 1-2 million tokens across the big models. That means entire codebases can fit in context. The real progress now is in reasoning quality and the ability to work autonomously for longer periods.

One trend worth watching: Stanford data shows jobs for developers aged 22-25 dropped nearly 20% from peak, while jobs for those 35-49 went up 9%. [42] AI makes experience more valuable while making entry-level positions harder to justify. Some teams are adopting “Copilot-free Fridays” - one day per week with no AI tools - specifically to keep their skills sharp.

The Bottom Line

After going through all of this, here's where I've landed:

AI changes how engineering works, not whether you need engineers. The companies getting real value (just 6% according to McKinsey) are 3x more likely to have redesigned their workflows. Just plugging in AI tools without changing your process doesn't help. [24]

More code is not the same as better software. Port.io looked at 63 earnings calls where companies proudly announced their AI code metrics. None of them connected those numbers to things that actually matter - how often they deploy, how fast they recover from incidents, how long it takes to ship a feature. [35]

The hard part has shifted. Writing code used to be the bottleneck. Now it's reviewing it, testing it, and making sure it all works together. Teams that don't adjust to this new reality will ship more code and more bugs at the same time.

The tools have changed. The principles haven't. Experiment rigorously, measure honestly, share openly, and maintain the engineering discipline that made your team effective in the first place.

If you're wondering where to start, here's a phased plan based on what worked for the teams I studied. Not everything applies to every team - pick what fits your situation and skip the rest. The order matters though: get the basics right before scaling.

Start here

The first things you can do are small and low-risk. Update your Definition of Done to call out AI-generated code explicitly. Add “What AI tools did you use and what did you learn?” to your standups so the team shares what's working. Create CLAUDE.md or AGENTS.md files for your main repos - this is the single highest-leverage thing you can do for AI code quality. Make a rule that AI never writes both the code and the tests for that code. And add at least one AI-specific quality gate to your CI pipeline.

Once you have the basics

Move to a TDD-first workflow: humans write the test specs, AI writes code to pass them, humans review and clean up. Restructure code review so AI handles the first pass (style, basic security, static analysis) and humans focus on architecture, intent, and security boundaries. Start tracking AI vs. human code metrics separately - error rates, cycle time, review outcomes. Without this data, you're flying blind. Run a controlled experiment on one team to actually measure whether AI is helping before you roll it out everywhere. And set up a shared prompt and workflow library so people aren't reinventing the wheel.

As your team matures

Think about how your team is spending its time. If engineers are still writing most code by hand, you may need to shift toward more reviewing and less typing - possibly creating a dedicated Context Architect role. Make your codebase agent-friendly: consistent naming, strong typing, well-scoped modules, comprehensive CLAUDE.md documentation. Set up self-healing patterns for your most critical pipelines. Start a monthly AI code audit - pick 10 random AI-generated files and have a senior engineer review them deeply. Plan for MCP integration in your architecture. And schedule explicit refactoring sprints, because AI-generated debt accumulates faster than you think.

Looking further ahead

Before scaling AI adoption further, assess where you actually stand against DORA's 7-capability model - fix foundational weaknesses first, because AI will amplify whatever is broken. For new projects, design AI-native from the start: built-in observability, fallback logic when models fail, evaluation systems for output quality. Invest in multi-agent orchestration capabilities. Start planning for the junior developer pipeline problem - if entry-level jobs shrink, you need structured mentorship and AI-free learning paths so the next generation of engineers can still build real skills. And budget for compute costs going up significantly: agents use about 4x more tokens than chat, and multi-agent setups use 15x.

References