Use case 2026
Khmer short-answer grading: human-orchestrated modelling and SHAP research
An undergraduate research codebase and documentation for comparing Khmer short-answer grading models and inspecting SHAP word attribution. Its prototype supports research and teacher assistance; reported agreement is with one teacher, not independently established correctness or classroom benefit.
L2Research workflow (workflow level, not a quality rating) Source: Repository-documented
Research applicability
A concrete educational ML investigation illustrating human-designed data preparation, model comparison and explanation analysis, including useful disclosed comparability/governance limits.
Getting started
Editorial, untested guidance: first read README, ethics and report-to-evidence notes; compare reported data/model variants and unresolved provenance before planning a reproduction. Review public documentation without downloading student data. Any later data use or classroom trial needs independent authorisation and human validation.
Maintenance
Pinned final-report documentation inspected. Subsequent experiments, maintenance and availability of compatible checkpoints were not independently audited.
License
README documents MIT code and separate CC BY-NC 4.0 dataset terms; third-party models retain their own terms. Licensing/public hosting does not establish consent or ethics approval.
Workflow & human oversight
Scoped L2 for human-designed teacher-labelled data, modelling/comparison, evaluation and SHAP/prototype research. Researchers choose stages and comparisons. Single-grader targets, unequal variants/samples, historical provenance and consent gaps prevent high-stakes grading or classroom-effectiveness claims.
Workflow level is not an evidence score; higher is not better evidence.
Problem
Khmer educational assessment has limited automated short-answer grading resources; this project studies model agreement and explanatory word contributions under a teacher-labelled research design.
Approach
Curate teacher-scored examination answers and student-derived reference answers, train/compare predefined classical, recurrent, encoder and adapted language-model families, inspect agreement/exact-match summaries and SHAP reference-overlap plausibility, then examine a teacher-assistance prototype. Different test variants limit model-family comparisons.
Outcomes (as reported by the originating team)
The repository reports grading and SHAP summaries, but the encoder's test variant differs and historical audits did not recover one coherent matching prediction/checkpoint set. Matching report tables to saved summaries is transcription consistency, not independent reproduction. No classroom field study or educational benefit is established.
Limitations
README and primary documentation reviewed, not full report-PDF review or a rerun. Single-grader labels, student-derived references, unequal comparison/SHAP samples, unresolved prediction/checkpoint/preprocessing provenance and lack of verified consent/ethics evidence restrict claims. Later dataset correction did not regenerate report benchmarks. Public pseudonymous copies are not guaranteed anonymous or authorised for reuse.
Reproducibility
No training, model download, student corpus access, saved-prediction recomputation or external benchmark API call. Public report/presentation availability was established from primary documentation/tree, not a complete PDF audit.
Adoption requirements
Authorised data access and institutional privacy/consent review before handling examination answers; appropriate researcher/teacher expertise, comparable dataset/split/model provenance, suitable environments and checkpoints for any later reproduction. Separate source-code, dataset and model terms apply. Do not use as an autonomous high-stakes grader.
Comments
No comments yet.
Add a comment
Comments are held for moderation before appearing. Critical or corrective comments are welcome — especially evidence that an entry overstates what has been verified.