autoresearch-starter: recorded small-model experiment loops
Template documenting small language-model experiments in which a coding agent edits a training recipe, runs time-bounded training and keeps or undoes changes while recording the session.
Workflow level n/a Source: Repository-documented Outcome: Not empirically evaluated here
Research applicability
Supports inspectable experiments on model/training-design choices and teaching of AI-assisted experimentation; logged score changes are not automatically statistically meaningful.
Getting started
Editorial suggestion, not tested: create a copy using the template button, read prerequisites and permission warnings, and first review/run the documented smoke-test path in an isolated environment with permitted data. Set explicit spending/runtime bounds and inspect baseline stability before any unattended loop.
Maintenance
Documentation retrieved on recorded access dates; maintenance reliability not assessed.
License
No independent license/dependency or rights audit. Check repository terms and rights to source data before reuse.
Workflow & human oversight
Resource-level automation left unspecified. Researchers/educators choose materials, govern model access and verify interpretations and outputs.
Workflow level is not an evidence score; higher is not better evidence.
Approach
The documented loop holds preparation/evaluation fixed, repeats the starting recipe three times for baseline/noise, edits train.py and retains per-run records and the best recipe. The current default training duration is ten minutes; dataset and tokenizer choice remain with the researcher.
Limitations
Complete current README reviewed only; no code, session records, demo or hardware instructions executed/audited. Queue metadata says five-minute experiments; the retrieved README defaults to ten minutes. Scores depend on hardware speed and the greedy loop may retain improvements smaller than baseline noise. Unattended coding-agent permissions can execute arbitrary user-level commands; use isolation, spending/runtime limits and researcher inspection. No demo gains, training performance, cost or overnight reliability verified.
Reproducibility
Primary-source documentation/excerpt review only. No installation, tests, model training, deployment or outcome reproduction.
Adoption requirements
The documented Windows/NVIDIA quickstart needs git, uv, Python 3.10+ and an appropriate GPU: Turing 8 GB+ or newer listed generations 10 GB+. Agent-driven sessions additionally need a coding agent/account and Node.js. Review platform-specific hardware guidance and permissions before adoption.
Comments
No comments yet.
Add a comment
Comments are held for moderation before appearing. Critical or corrective comments are welcome — especially evidence that an entry overstates what has been verified.