WikiPlanner: wiki-derived temporal monitors for safe LLM agent planning - a medical-domain proof of concept
WikiPlanner studies extracting temporal constraints from domain wikis, curating a constraint library, compiling deterministic monitors and evaluating LLM plan validation in a medical-domain proof of concept. Evidence reviewed from primary article excerpts.
L2Research workflow (workflow level, not a quality rating) Source: Peer-reviewed source
Research applicability
Substantive primary methods describe an AI-assisted constraint-extraction and benchmark-evaluation process with explicit human audit and library curation. Recommend with prominent extraction and natural-plan limitations, not as a validated clinical safety system.
Getting started
Editorial, untested guidance: read the cited primary source and verify its limits before adapting the method. No candidate code, models or tools executed.
Maintenance
Only the cited snapshot was reviewed; present installation, maintained service availability and latest-version parity are unverified.
License
Check original licences, data rights and provider terms; source access is not reuse permission. No installation or rights audit performed.
Workflow & human oversight
Scoped L2: L2 is supported only for the documented research pipeline: domain wiki inputs undergo GLM extraction, post-processing and curation, a human-verified supplemental constraint set, deterministic compilation, and benchmark comparison. Two annotators (clinician-researcher and medical student) audit raw extraction; the researcher-defined corpus and benchmark bound the process. No evidence supports autonomous clinical action or L4 research iteration. Human control and gaps: L2 is supported only for the documented research pipeline: domain wiki inputs undergo GLM extraction, post-processing and curation, a human-verified supplemental constraint set, deterministic compilation, and benchmark comparison. Two annotators (clinician-researcher and medical student) audit raw extraction; the researcher-defined corpus and benchmark bound the process. No evidence supports autonomous clinical action or L4 research iteration.
Workflow level is not an evidence score; higher is not better evidence.
Approach
Substantive primary methods describe an AI-assisted constraint-extraction and benchmark-evaluation process with explicit human audit and library curation. Recommend with prominent extraction and natural-plan limitations, not as a validated clinical safety system.
Outcomes (as reported by the originating team)
Author-reported findings or documented capabilities only; no independent reproduction or expert validation was performed.
Limitations
Single medical-domain proof of concept. Authors report low semantic extraction precision (25.9%) and lower natural-plan F1 (0.582) than aligned-plan F1 (0.920); these are author-reported benchmark outcomes, not clinical effectiveness or independent reproduction. Wiki-derived rules and symbol grounding require review.
Reproducibility
Source-text review with frozen provenance. No candidate software, models, data uploads or research outcomes were executed or reproduced.
Adoption requirements
Select a research task and review domain/data rights and human validation requirements; no adoption trial performed.
Comments
No comments yet.
Add a comment
Comments are held for moderation before appearing. Critical or corrective comments are welcome — especially evidence that an entry overstates what has been verified.