Harness engineering: repository-level coding-agent configuration

An exploratory study of repository-level configuration for coding agents, including context files, skills and subagents. The authors report analysing 2,853 GitHub repositories across five coding tools.

Workflow level n/a Source Preprint (not peer-reviewed) Outcome Author-reported

Evidence & verification

Source level

Preprint (not peer-reviewed)
Grounded in a preprint that had not completed peer review at the time the source was checked.

Outcome verification

Author-reported
Outcomes are reported by the team that built the system. The entry is source-checked against the team's own publications; not independently reproduced.

Sources checked

2026-10-17 (UTC)

AI editorial review of substantive abstract only; capability and process statements bounded to source. Observational study results are author-reported; conceptual references are not-evaluated.

Source-checked does not mean independently reproduced. See the Method page for how these labels are assigned.

Sources (1)

Research applicability

An evidence-informed orientation for researchers configuring coding assistants in research repositories; applicability to scientific code is an editorial extrapolation, not an evaluated study population.

Getting started

Editorial, untested: inspect supported context-file formats for the chosen tool and draft minimal repository guidance, then test its effect on a bounded research-coding task. The study does not establish that more mechanisms improve performance.

Maintenance

Source accessed 2026-10-17; runtime compatibility and maintenance not evaluated.

License

Article, repository and data permissions must be checked separately before reuse. No blanket open-licence claim.

Workflow & human oversight

Resource-level automation unspecified; researchers select the task, assess data rights and inspect outputs. Source review is not an executed workflow.

Workflow level is not an evidence score; higher is not better evidence.

Approach

The study examines versioned Markdown and JSON configuration artifacts for Claude Code, GitHub Copilot, Cursor, Gemini and Codex. It reports context-file dominance and comparatively sparse adoption of skills and subagents; these are observational adoption findings, not causal performance improvements.

Limitations

Abstract-only review of arXiv v5. The record says it is based on the earlier conference paper Configuring Agentic AI Coding Tools: An Exploratory Study; this is a title/version relationship, not a separately verified publication. Repository sampling, classification and data artifacts were not inspected. Counts and adoption findings are author-reported; no performance gain or independent reproduction is established.

Reproducibility

Fresh primary abstract reading only; no software execution, data audit or experiment reproduced.

Adoption requirements

Editorial, untested: review the full methods, check current tool-specific configuration formats, and version project instructions. Do not put secrets or restricted research data in agent context; permissions and output testing remain the researcher's responsibility.

Comments

No comments yet.

Add a comment

Comments are held for moderation before appearing. Critical or corrective comments are welcome — especially evidence that an entry overstates what has been verified.