Scanning the Harness: An Empirical Study of Supply-Chain Defects in AI Coding-Agent Configurations
Nearly 1 in 6 public AI coding-agent configs (Claude Code, Cursor, Copilot, Codex) carry a confirmed security defect, study of…
A study of 3,171 public GitHub repositories using AI coding-agent configuration artifacts (instruction files, skills, hooks, MCP server declarations, subagents) found that 16.0% carry a security defect, most commonly unpinned MCP server versions, overly broad execution grants disguised as scoped, or skills that pre-approve shell access for installers. The researchers validated findings with independent re-derivation and model adjudication, and found the raw scanner false-positive rate was much higher (25.5%) than the confirmed rate. No credential-exfiltration path was confirmed.
Why it matters
This is the first empirical measurement showing that the configuration layer around popular coding agents, treated by developers as harmless glue code and shared freely, is a real and measurable attack surface with no lockfile or install-time vetting, comparable in spirit to package-manager supply chain risk.
What is verified
Study analyzed 3,171 public GitHub repositories: 2,660 setups combining two or more component types (instruction files, skills, hooks, MCP server declarations, subagents) and 511 published skill collections. [1]
9.8% of setups install an MCP server with no version pinned. [1]
3.1% of setups pre-approve arbitrary execution behind a scoped-looking grant such as Bash(python:*). [1]
3.8% of setups carry a skill that pre-approves the shell for whoever installs it. [1]
16.0% of setups carry a security defect and 16.7% carry a confirmed defect of any kind, versus a raw scanner rate of 25.5% on the same rules. [1]
The third security class (shell pre-approval) ships inside 3.7% of skill collections, a location where a marketplace scan can see it. [1]
No credential-exfiltration path was confirmed in the study. [1]
The instrument, corpus manifest, prompt, and every verdict are released. [1]
What is not
- No information on which specific repositories or marketplaces were affected, so it is unclear whether any configuration used by Coding Partners is implicated.
- The paper does not state whether any of these defects have been exploited in the wild, only that they are structurally present.
- No detail on remediation guidance beyond the classification itself; no mention of vendor response from Anthropic, OpenAI, Cursor, or GitHub.
- Unclear how representative the 3,171 scanned repositories are of the broader ecosystem of AI coding agent configurations.
The case for
The study used a validated pipeline (independent re-derivation plus dual-model adjudication) rather than a raw scanner, and explicitly reports the raw scanner's higher, unvalidated false-positive rate of 25.5%, lending credibility to the lower 16.0%/16.7% confirmed figures as a conservative, defensible measurement.
The case against
The paper does not confirm any actual exploitation or credential exfiltration, and it is unclear how representative the sampled repositories are of configurations agencies actually deploy internally versus experimental or throwaway public repos.
What to do with this
Treat this as a prompt to audit agent configuration hygiene now rather than as an active incident: there is no evidence of in-the-wild exploitation, but the structural exposure (unpinned MCP servers, broad execution grants, shell pre-approval in skills) is confirmed and cheap to check for internally.
Sources
- 1arXiv cs.SEarxiv.org · Primary source
Original source published 7 September 2026 at 13:27.
An AI-assisted briefing by Coding Partners. We link the evidence behind the reporting so you can explore the original work and assess the claims. How CP Signal works.