Method

How resources are selected, how the two evidence dimensions work, and what the labels do and do not claim.

What this directory is

AI tools, agent skills, frameworks and workflows for academic research. Resource types are independent of workflow level: a reusable framework or skill need not have an L1–L4 classification until instantiated in a concrete workflow. Non-agentic AI assistance remains in scope.

Use cases, skills, frameworks, workflows/processes, tools, tutorials/templates and relevant benchmarks/datasets are first-class resources. Humanities, social sciences, education, business and other disciplines are in scope alongside STEM. An empty filter is a real coverage gap.

A catalogue of documented AI use in academic research across disciplines: task assistance, literature review, qualitative and quantitative data analysis, human-led workflows, tool-using agents and autonomous iterative research. Non-agentic machine learning and LLM uses are in scope. Each case states what the system did, what its originators reported, what limitations they acknowledge, and what would be needed for another group to adopt the approach.

Entries are included only when their sources have been retrieved and read. The catalogue is deliberately small rather than padded: if a claim cannot be traced to a retrievable source, it is not listed as a case.

Workflow levels: scope, not quality

Levels classify the documented use, not a product in all settings. Higher is not better evidence. A narrow L1 study can have stronger evidence than an L4 demonstration. Classification is editorial; each case explains its rationale and human oversight. Legacy unclassified records are not guessed.

L1 · Task assistance: AI assists one bounded research task; a researcher directs and checks the output.
L2 · Research workflow: AI supports a connected research process, with humans orchestrating stages and decisions; tool-using agency is not required.
L3 · Multi-step tool-using agentic workflow: A system plans or coordinates multiple steps and uses tools within a researcher-defined objective; humans supervise and validate results.
L4 · Autonomous iterative research: A system iteratively proposes, executes and evaluates research steps within a bounded remit. Humans set goals, constraints and stopping conditions, oversee safety and validate conclusions; this does not mean unsupervised science.

Generic coding productivity trials are not evidence of academic research benefits unless the study actually evaluates research work. Design proposals are not demonstrated productivity gains.

The two evidence dimensions

Every entry carries two independent labels. They are not a single score, and neither one, on its own, ever means "independently reproduced".

1. Source level — where the entry is grounded

Repository-documented: Grounded in a maintainer repository/README. Availability, stars and forks do not establish research benefit or correctness.
Project-documented: Grounded in primary project documentation, not necessarily an empirical evaluation.
Vendor-documented: Grounded in vendor documentation or claims; not independent validation.
Peer-reviewed source: Grounded in a peer-reviewed publication. This describes the source; it does not mean the reported results have been independently reproduced.
Preprint (not peer-reviewed): Grounded in a preprint that had not completed peer review at the time the source was checked.
Institutional report: Grounded in a report published by the institution that produced the system, without peer review.
Mixed sources: Grounded in a combination of peer-reviewed and non-peer-reviewed sources; see the individual sources.

2. Outcome verification — how far the reported outcome has been checked

Not empirically evaluated here: A documented resource or intended workflow, not a demonstrated academic outcome. No empirical benefit is established by its inclusion.
Author-reported: Outcomes are reported by the team that built the system. The entry is source-checked against the team's own publications; not independently reproduced.
Externally validated (partial): Part of the reported outcome was checked by an external process - for example, collaborating labs running validation experiments, or a generated manuscript passing blind human peer review. Full independent reproduction has not been demonstrated.
Machine-verified artifacts: Key artifacts (for example, formal proofs) were checked mechanically by a proof checker. Higher-level performance claims remain author-reported.
Independently reproduced: A third party outside the originating team has reported reproducing the result; the reproduction is cited.

A case grounded in a peer-reviewed paper, reporting outcomes from the team that built the system, is peer-reviewed and author-reported: strong provenance, no independent reproduction. The catalogue keeps those two statements visibly separate on every page.

How entries get in

Seeded resources are written by hand with their sources, after the primary literature has been retrieved and checked. Each seeded case records the date its sources were checked.
Discovery runs are manually runnable and bounded. Active adapters cover Crossref, arXiv, GitHub, GitLab, and a configurable registry of HTTPS pages and RSS/Atom feeds. English and multilingual queries cover literature review, data analysis, writing, coding, skills and frameworks. No recurring scheduler is enabled. Nothing is auto-published.
Review queue. Every discovered item lands in a moderation queue with its provenance (which source, which query, when seen) and a deduplication key that prevents the same work being queued twice. A human reviews each item, and only then is a resource entry written, with its sources, into the catalogue.
Failures are visible. If a discovery run cannot reach a source, the run is recorded as degraded and the failure is shown on the Updates page. There is no silent fabrication of entries or statistics: if nothing was found, the log says nothing was found.

Discovery coverage, not whole-web coverage

The website adapter reads only the explicit registry in discovery/sources.json, without recursive crawling. A successful request is not a verified useful resource. General-web search: not configured. Imported community/video/search leads remain pending until primary sources are reviewed.

journals-preprints — active: Metadata search, not full-text coverage of all journals.
repositories — active: Public official APIs; unauthenticated access subject to limits. GitHub token optional.
university-lab — active-curated: Explicit registry pages/feeds, not every university or laboratory.
tool-documentation — active-curated: Project and vendor documentation; existence is not an empirical benefit.
tutorials-blogs — active-curated: Research workflow tutorials/blogs; all claims need primary-source review.
conference-workshops — active-curated: Selected workshop pages; inclusion is not paper peer review.
community-discussions — active-curated: Public RSS plus imported leads; anecdotes are not outcome validation.
video — import-or-optional-search: No direct video-platform crawler. Import retrieved primary URLs; manually review recordings/transcripts.
general-web — optional: Brave requires ATLAS_BRAVE_SEARCH_KEY; not enabled without a key. No search-engine HTML scraping.

Evidence limits: repository stars, forks and maintenance activity are not verification. Project/vendor documentation establishes what is documented, not an empirical research benefit. Tutorials, discussions and videos are discovery leads, not validated outcomes.

Safety: downloaded content is untrusted data. Repository code, skills and instructions are never executed by discovery. Fetches enforce HTTPS, exact-host allowlists, public-IP checks with connection pinning, redirect checks, timeouts and byte limits.

Dedupe: exact canonical DOI/repository/project URLs link channels and preserve all observations. Names alone and arbitrary cited links are not identity. Unlinked projects can still need manual merging. Per-query coverage and failures appear in the update log.

Comments and corrections

Comments are stored on the server and held for moderation; they are never auto-published. The moderation queue is only reachable by the site's administrator behind an authenticated login. If you believe an entry overstates what has been verified, or you know of an independent reproduction (or failure to reproduce), a comment on that entry is the fastest route to a correction.