Nine engineers, 1,211 pull requests in a day: Lessons from building Copilot Cowork

Compatibilidad
Ahorrar(0)
Compartir

In January of this year, a small team at Microsoft set out to build Copilot Cowork, an agentic execution layer that performs multi-step workflows on a user’s behalf. That result brings the powerful AI-assisted tools and capabilities developers have today to everyday knowledge workers, all with Work IQ, Microsoft 365 security and governance, cloud execution, and Microsoft’s multi-model architecture. 

This is the story of how the team took Copilot Cowork from an empty repo to worldwide GA in about four and a half months, then moved its runtime from Python to Rust while it was live. Agents wrote most of the code. Ways of working changed along the way, and the practices other teams can adapt to build like a frontier development team: moving fast while keeping reliability, security, and cost in view. 


On October 21, nine engineers merged 1,211 pull requests into the main branch of the Copilot Cowork backend. That number would have sounded absurd to the team in January, when a handful of us made the first commit to an empty repo.   

Agents wrote most of the code, but engineers remained responsible for defining the work and judging the results. Our small team wrote clearer specs, set architecture and security boundaries, and focused its review on higher-risk changes, production impact, and cases where checks or reviewers disagreed. Automated checks and agent reviewers examined every change. Agents could propose changes, but they could not decide what was allowed to happen. 

We didn’t plan on working this way. But every few weeks, the agents got better, part of our process stopped working, and we adapted. 

Six practices emerged from those changes. They cover how we specify work, run agents within clear boundaries, test our checks, preserve what the team learns, shape the codebase for parallel work, and set limits on automated loops. They aren’t a recipe for matching our PR count. They are lessons other teams can adapt to build like a frontier development team while keeping reliability, security, and cost in view. 

We’ll end with the project that puts those practices to the test together: moving the live runtime from Python to Rust. 

Lesson 1: Write the spec before the code 

The first commit was made on January 13. From the start, agents wrote the code. 

In the first weeks, we learned that the agent was rarely what slowed us down. The bottleneck was not knowing exactly what we wanted. A vague request got us code that looked reasonable, survived a quick read, and then did the wrong thing. Fixing it took longer than writing a proper request would have in the first place. 

So we adopted a rule that still stands today: no code without a markdown spec written first. The spec covers three things: 

  • What the change does
  • How we’ll know it works
  • What it must not break

A good spec saves far more time than it costs upfront. But a weak one leads to rework a few days later. The faster agents became at implementation, the more important it became for us to be precise about intent, requirements, and what good looked like. 

Many of our specs start out spoken. Someone talks through a feature, and an agent turns that into a first draft of the spec, which the human then edits. I think this did more for our speed than any single model upgrade. 

We launched Copilot Cowork to our first enterprise customers in the Frontier program on March 30. 

Lesson 2: Run narrow agents inside clear boundaries 

The next step was running more than one agent at a time. Each works in its own copy of the repo, on one task, from one spec. One agent became five, and five became dozens. 

Merged PRs per month went from 257 in January to 2,754 in May. Our busiest week before GA had 783. 

Figure A. Pull requests merged into main per month in 2026, with Frontier (March 30), GA (June 16), and the Rust cutover (September 23 – October 10) marked. October covers the 1st through the 20th.

Parallel work went better than we expected for two reasons: 

  • Narrow tasks: An agent with one well-defined task makes fewer mistakes than one asked to do a lot at once.
  • Picking the model per task: We treated model choice as a quality-and-cost tradeoff, not a race to use the cheapest model. Some tasks warranted stronger reasoning; for others, a faster, less expensive model gave us the right balance.

In Copilot Cowork, the model runs in a container without user credentials. A trusted host outside the container handles identity, policy, approvals, and tools, and every Microsoft 365 operation goes through the host.  

We didn’t want to limit what the agent could attempt, because a heavily restricted agent isn’t very useful. With our architecture, the model can plan and ask for anything, and the host decides what actually happens.  

That separation lets us increase the number of agents working in parallel without giving the model credentials or direct authority over operations. The host still applied identity, policy, approvals, and audit at the point of execution.

Figure B. The agent container holds no user credentials and has no direct network path to Microsoft 365. Every request crosses the boundary to the trusted host, which owns identity, policy, approvals, tool execution, and audit.

Building that in at the start was much easier than adding it later would’ve been. It also helped in a way we didn’t plan for. The agent only talks to the host through a defined interface, so we could later replace the entire host without changing the agent—which we did when we migrated to Rust. 

We reached worldwide general availability on June 16, about four and a half months after the first commit, with several models and 16 plugins. 

Lesson 3: Make checks prove they checked something 

With agents writing most of the code, nine people couldn’t read every change, and waiting for a human review was becoming the slowest step for every PR. 

Several engineers each built their own review bot, modeled on how they review code themselves, and the bots competed. When one bot caught a problem another had missed, the bot that missed it got a new rule. Over time, each reviewer built up a large set of rules learned from its own misses. Each one has now been through at least10,000 generations; in each generation, the reviewer runs reviews, then reflects on what it missed and gets better at reviewing code. 

Today, a useful first review usually arrives within several minutes of a pull request being opened. On a sample of recent merged PRs, the median was about four and a half minutes. 

Slower, more specialized passes follow: 

  • One walks the call graph, looking for code that isn’t wired up and for tests that would still pass if the change were reverted
  • One looks at operational problems like restart safety, metrics, and stale docs
  • One reads every changed file end to end
  • One checks behavior against the code being replaced

The author gets quick feedback while the change is still fresh, and the detailed reading still happens. 

Of course, reviewer agents can still miss things. Once, a group of reviewer agents read a change in one of the most sensitive parts of the product. Each had been told to try to break it, and none found a problem. People on the team then read the same diff and found four blocking issues, all of which were fixed before it merged. 

Fast first feedback was useful, but it wasn’t permission to skip judgment. We kept agentic review as the first check and added at least one human check for critical changes. We made it clear which findings could block a merge. 

We also split findings into two tiers: 

  • Blocking: Correctness, security, data loss, or a new mechanism with no test that would catch it breaking.
  • Advisory: Everything else. The author either fixes it or replies, and it doesn’t force another round of review.

Another rule came from a check that couldn’t fail. We keep a size limit on the instruction files agents read, because every extra byte costs tokens on every run. The limit was exceeded, and the check that enforced it stayed green the whole time. 

The script piped the check’s output through tail. A shell pipeline returns the exit code of its last command, and that was tail, which always succeeded: 

``` # The pipeline's exit code is tail's, so this always passes lint-size-check | tail -1 # Keep the check's own exit code and print it lint-size-check > out.txt 2>&1; rc=$?; echo "lint rc=$rc"; cat out.txt; [ "$rc" -eq 0 ] || exit 1 ```

Once we went looking, we found the same problem in other places: 

  • A formatting check that was handed an empty list of files, so it checked nothing and passed
  • A list of changed files that came back empty as soon as the work was committed

We wrote a two-part rule: 

  • Every check reports how many files it checked
  • Every new check comes with a test that makes it fail on purpose

I think this matters more with agents than without them. Agents keep working until the checks pass, so if a check can pass without testing anything, an agent will eventually find that path. 

Lesson 4: Put what the team learns in the repo 

By August, the same mistakes kept coming back. Each agent session started without any memory of the previous one, so we moved the team’s rules into the repo: 

  • Instruction files at the root and in each component
  • A directory of rule files on specific topics

Now, every agent reads the relevant files when it starts work. When something goes wrong, the fix usually includes a new or updated rule. There are about 30 rule files now, and the oldest dates from August 6. 

One early rule was about merge conflicts. At a few hundred merges a day, almost none of our conflicts were in actual logic. They were in files every branch appended a line to: registries, lists, generated files, and documents with a “last updated” line. Instead of asking people to resolve conflicts more carefully, we changed the files: 

  • Registries became one file per entry instead of a long list
  • Generated files are regenerated after a merge, never merged by hand
  • Values that change on every pull request, like dates and counts, are no longer committed

Lesson 5: Shape the codebase for parallel work 

Agents also wrote most of the first Rust code in the repo, and they put things wherever it was easiest, which usually meant the biggest crate. At one point, the main runtime binary was about 290,000 lines, and splitting it up took a long time. 

On August 20, we took stock of the Rust workspace: 

  • There were 86 crates, with a dependency chain 13 levels deep
  • A change to the shared base crate forced a rebuild of 43 of the 63 crates
  • The API crate had grown to 134,000 lines and was also where everything got wired together
  • About 12,000 lines of business logic sat in the binary itself, because circular dependencies between implementation crates left nowhere else to put it

For a team running many agents in parallel, that structure was a problem. Most changes touched the same few crates, rebuilt most of the workspace, and conflicted with each other. 

We fixed it with a strict rule about roles. Every crate is one of four kinds: 

  • Contract (types and interfaces)
  • Kit (shared helpers with no internal dependencies)
  • Implementation
  • Binary

Implementations may depend on contracts and kits but not on other implementations. Only binaries depend on implementations, and binaries contain wiring code only. When one implementation needs something from another, that dependency goes through a contract. 

New code gets no exceptions and no allowlists. Each exception we had granted in the past had been used to justify the next one. 

The cleanup ran as a program of 401 work items with 2,225 dependencies between them, and at the peak, 40 to 44 items were ready at once. We grouped work into batches that never touched the same files, so parallel agents wouldn’t conflict. We even split the gate’s own list of temporary exceptions into one file per crate. Otherwise, a single wave of about 20 agents would all have edited the same file. The engineer who owned the program made decisions at set checkpoints, and agents did most of the implementation. 

The CI gate started out as a report. Then it became a ratchet, where the allowed number of exceptions could only go down. The results: 

  • Dependencies from one implementation to another on the exception list went from 216 on August 22 to zero on October 7.
  • The maximum dependency depth the gate allows went from 10 to three.
  • The number of crates went from 86 to about 176.
Figure C. The Rust workspace before (August 20) and after (October 7). Before: one deep chain, with business logic in the binary and implementation crates depending on each other. After: binaries only wire things together, and implementations depend only on contracts and kits.

The build-time gains were smaller than we had predicted. Our compile cache already hit about 99% of the time on pull request builds, so the run-to-run variation in build time was bigger than the improvement. What we could measure was structural. After a change to a base crate, the longest chain of crates that must be rebuilt in sequence went from 11 to 3. This is now used to drive further speed improvements and prevent compilation time from increasing linearly with the size of the codebase. 

Lesson 6: Give automated loops targets and limits 

For most of this year, the loop was manual: a person prompted an agent, read the result, and decided what to do next. More of our work now runs as automated loops. A harness gets one measurable target and keeps working until it reaches it or runs out of attempts. People choose the target, decide what must not get worse, and keep the final say on anything that reaches production. 

Test speed 

The clearest example is a harness whose target is the time CI spends running tests. It opens two kinds of pull requests: 

  • Changes that make a slow test faster
  • Removals of tests whose behavior another test already covers

Its pull requests only merge if CI’s own timing data shows the touched tests at least 5% faster. A review step also compares every assertion against the original, so a faster test can’t be one that checks less. About 300 of these PRs have merged since mid-September. Across them, the touched test files went from roughly 17,700 seconds of CI time to about 1,700. 

Document quality 

We ran a series of passes, each of which changed the skill that generates documents. Each pass was judged by comparing its output with the previous version on a fixed set of tasks. A pass was kept onl

Detalles de contacto
sbaynes