Detecting reference errors in scientific literature with large language models

The authors evaluate OpenAI GPT-family language models for reference-error detection using an expert-annotated dataset of statement-reference pairs from journal articles.

L1Task assistance Source Preprint (not peer-reviewed) Outcome Author-reported

Evidence & verification

Source level

Preprint (not peer-reviewed)
Grounded in a preprint that had not completed peer review at the time the source was checked.

Outcome verification

Author-reported
Outcomes are reported by the team that built the system. The entry is source-checked against the team's own publications; not independently reproduced.

Sources checked

2026-10-17 (UTC)

Abstract page read on 2026-10-17, including the journal reference and subject class; raw HTML retained with SHA-256.

Source-checked does not mean independently reproduced. See the Method page for how these labels are assigned.

Sources (1)

Research applicability

Relevant to automated reviewing, reference-integrity checking and the design of editorial QA workflows.

Getting started

Editorial suggestion, not exercised: read the arXiv study before designing a small reference-checking pilot. Verify permissions before sending article text to a provider, and have researchers adjudicate model flags against the cited sources.

Maintenance

No revision history beyond the posted version was reviewed.

License

Paper and dataset reuse terms were not reviewed; do not infer dataset availability or permission from its description.

Workflow & human oversight

L1 task assistance for reference checking, not autonomous scientific judgment. Human adjudication of errors is recommended editorial oversight.

Workflow level is not an evidence score; higher is not better evidence.

Approach

The abstract reports evaluation under different amounts of reference information supplied through retrieval augmentation, and reports detection with limited context without fine-tuning.

Limitations

Abstract-only evidence; the dataset's size, licensing, inter-annotator agreement and the models' error profiles are unverified, and no tool was run.

Reproducibility

No dataset, code or prompt was inspected; release status and reproducibility are unverified.

Adoption requirements

Reimplementation would require suitable statement-reference examples and model access; public dataset availability, prompts, cost and licensing were not verified.

Comments

No comments yet.

Add a comment

Comments are held for moderation before appearing. Critical or corrective comments are welcome — especially evidence that an entry overstates what has been verified.