Evidentia: human-scoped literature claim extraction and disagreement mapping
A repository-documented tool for user-uploaded research papers, structured claim extraction, normalised intervention/outcome grouping and fixed study-weighted disagreement views. Engineering demonstrations include synthetic data; neither disagreement scores nor model confidence establish clinical certainty or research validity.
L2Research workflow (workflow level, not a quality rating) Source: Repository-documented
Research applicability
Useful as a source-inspectable example of connecting paper-level extraction to corpus-level organisation and disagreement exploration, while preserving researcher responsibility for study selection, original-source checking and substantive interpretation. Not a substitute for a protocol-driven systematic review or a validated evidence-grading method.
Getting started
Untested editorial guidance: inspect the pinned README and extraction/weighting code, distinguish actual-model operation from synthetic/fallback demonstration, and plan a small rights-cleared corpus with manual checks before considering installation or external API use. The optional lakehouse has separate documented setup and synthetic fixtures. No commands were run here.
Workflow & human oversight
Scoped L2 for human-selected uploaded papers, text-based claim extraction, grouping and disagreement views. The up-to-eight claim count is a prompt instruction, not an enforced cap; embeddings support organisation, not verified retrieval-augmented extraction. Expert source checking is recommended, not an enforced source-proven gate. Engineering/synthetic outcomes do not establish scientific or clinical validity.
Workflow level is not an evidence score; higher is not better evidence.
Approach
Researchers choose the paper corpus. The documented application parses text/PDFs; paper TEXT feeds the structured-claim extraction request, while embeddings support organisation, not demonstrated retrieval-augmented extraction. Normalised claim grouping and weighted/unweighted disagreement views connect extraction to researcher inspection. The optional bronze/silver/gold lakehouse is supporting data engineering, not an autonomous research loop.
Limitations
LLM extraction is capped at the first12,000characters and prompt-instructed to return up to8claims (an instruction, NOT an enforced output-count limit); parsing/default labels and heuristic fallbacks can conceal missing or incorrect evidence. Without the model key, seeded random vectors and placeholder/heuristic claims are not semantic representations or reliable study findings. Normalised intervention/outcome strings and coarse study-design weights do not establish cross-study comparability, risk of bias or effect magnitude. Model confidence is not calibrated evidence certainty. Demo data and lakehouse checks are explicitly synthetic/engineering-level. No mandatory per-claim expert acceptance/source-span checking, clinical validation, complete literature search or independent reproduction is established.
Reproducibility
Documentation/code inspection only at the recorded commit; no installation, model/API invocation, cloud setup, research-software tests or independent outcome reproduction. This editorial package's own provenance/guard tests are not tests of Evidentia.
Adoption requirements
The documented application uses its Python/API, database/vector and optional model dependencies; cloud/lakehouse components are optional additional infrastructure. Editorial requirements, not verified implementation features: rights/privacy review before sending text to external services; domain expertise and a real-paper error audit; explicit labelling of model/fallback/synthetic output and researcher approval of interpretations.
Comments
No comments yet.
Add a comment
Comments are held for moderation before appearing. Critical or corrective comments are welcome — especially evidence that an entry overstates what has been verified.