Giving an LLM an Overnight Research Loop (and keeping it from breaking everything)

The loop that runs while you sleep

In early 2026, Andrej Karpathy released autoresearch, a minimal Python framework that automates the traditional ML experiment cycle (Karpathy, 2026), and the idea underneath it is a small one; you write your research goals and constraints into a plain-text program.md file, an AI agent reads those constraints, looks at your current train.py, proposes code edits, runs a fixed-duration training experiment (typically five minutes on a single GPU), and if the experiment improves your chosen scalar metric (say, validation loss), the change gets committed to Git, and if it does not, the change is discarded, and then the loop repeats, for hours, overnight, without you in the room at all.

The role you play shifts from writing and tuning code to writing constraints and interpreting results; you are defining the optimization landscape and letting the agent explore it, which extends foundational AutoML architectures (Hutter et al., 2019) by handing the iterative hypothesis generation directly to an LLM operating on a graph-based workflow. The loop tends to work well in an open research sandbox; the architectural challenge I ran into is one that emerges only when you try to run the same loop inside a regulated enterprise environment.

Dimension	Traditional ML Loop	Autonomous Research Loop
Hypothesis generation	Human designs each experiment	LLM generates hypotheses from prior results
Code authorship	Human writes and edits training code	Agent proposes code mutations autonomously
Experiment execution	Manual run, human monitors	Fixed-duration automated runs (e.g., 5 min/GPU)
Evaluation	Human interprets metrics, decides next step	Agent evaluates scalar metric, commits or reverts
Iteration cadence	Hours to days per cycle	Minutes per cycle, hundreds overnight
Human role	Executor and interpreter	Constraint definer and final interpreter
Provenance	Manual logging (often incomplete)	Requires architectural enforcement

Partition the state space into the Fixed Harness and the Mutation Budget

The answer I settled on is a clean state-space partition; the autonomous loop’s entire environment splits into two mutually exclusive regimes, the Fixed Harness, which contains everything the agent is mathematically barred from altering, the immutable constraints of the enterprise, and the Mutation Budget, which contains everything the agent is explicitly authorized to alter, bounded by a finite computational allowance.

The Fixed Harness enforces three types of gates (pre-loop, mid-loop, and post-loop) that the agent cannot circumvent, and these gates live at the infrastructure level rather than the prompt level; the distinction is that the agent is not being asked nicely to respect compliance, it is being placed somewhere a violation is structurally impossible. If the agent writes code that attempts to bypass any gate (say, disabling the PII redaction filter to reach richer features), the Fixed Harness terminates the execution immediately.

class FixedHarness:
    """Immutable enterprise compliance layer.
    The agent cannot modify, disable, or circumvent these gates."""

    def pre_loop_gate(self, query, data_request):
        """Enforced BEFORE the agent accesses any data."""
        data = self.pii_redactor.scrub(data_request)
        if query.requests_individual_records():
            raise FixedHarnessViolation(
                "SYSTEM HALT: Agent attempted individual-level retrieval. "
                "RBAC restricts access to cohort-aggregate views only."
            )
        return data

    def mid_loop_gate(self, mutation, current_state):
        """Enforced DURING each mutation step."""
        if mutation.targets_protected_parameter():
            raise FixedHarnessViolation(
                "SYSTEM HALT: Agent attempted to modify a Fixed Harness parameter."
            )
        if mutation.introduces_individual_level_access():
            raise FixedHarnessViolation(
                "SYSTEM HALT: Mutation would breach aggregate-only data policy."
            )

    def post_loop_gate(self, model_output, fairness_thresholds):
        """Enforced AFTER the agent produces a model."""
        fairness_report = self.fairness_evaluator.assess(model_output)
        for metric, value in fairness_report.items():
            if value < fairness_thresholds[metric]:
                raise FixedHarnessViolation(
                    f"SYSTEM HALT: Model failed fairness gate. "
                    f"{metric}={value:.3f} < threshold={fairness_thresholds[metric]:.3f}"
                )
        return model_output

Catching hallucinated optimization with CUSUM

The Fixed Harness handles compliance violations and the Mutation Budget handles infinite loops, yet neither of them keeps the agent from scientifically fooling itself inside its budget. Consider the case where the agent engineers a new polynomial feature from cohort age bands and the model fit jumps by 15% in a single iteration; is that real signal, or is it a data leak? Target leakage, where a feature inadvertently encodes the target variable, tends to be the most common source of “too good to be true” results in automated ML pipelines, and a single-point check will not catch it, because the metric at that one iteration looks perfectly valid.

The architecture implements CUSUM (Cumulative Sum) control charts, a technique from statistical process monitoring introduced by E. S. Page in 1954; unlike single-point charts that look only at the current observation, CUSUM charts accumulate deviations from a target over time, so they pick up the small persistent shifts that a snapshot would miss.

CUSUM Statistic

St = max(0, St-1 + (xt - μ₀ - k)), where xt is the observed metric at iteration t, μ₀ is the target value, k is the allowance parameter, and the trigger fires when St > h (the decision threshold).

When the CUSUM trigger fires, two things happen; the architecture executes a mutation-trace through the experiment-tracking ledger to identify exactly which mutation caused the anomalous spike, and then it automatically reverts to the last known stable state, discarding the offending mutation and resetting the CUSUM accumulator. The agent gets to experiment freely, and the statistical monitoring system catches it when those experiments produce implausible results and rolls back the damage.

class CUSUMMonitor:
    """Cumulative Sum control chart for detecting hallucinated optimization.
    Based on Page, E. S. (1954). Continuous inspection schemes. Biometrika."""

    def __init__(self, target_mean, allowance_k, threshold_h):
        self.mu_0 = target_mean
        self.k = allowance_k
        self.h = threshold_h
        self.S_upper = 0.0
        self.S_lower = 0.0

    def update(self, observed_metric: float) -> bool:
        """Returns True if CUSUM trigger fires (anomaly detected)."""
        self.S_upper = max(0, self.S_upper + (observed_metric - self.mu_0 - self.k))
        self.S_lower = max(0, self.S_lower - (observed_metric - self.mu_0 + self.k))
        if self.S_upper > self.h or self.S_lower > self.h:
            return True
        return False

    def reset(self):
        """Reset after revert to last stable state."""
        self.S_upper = 0.0
        self.S_lower = 0.0

The boundary where the machine generates evidence and the human exercises judgment

The harness, the budget, and the CUSUM monitor each handle a different failure mode, yet they share one design constraint; the autonomous research agent stays strictly an analytical engine whose output is bounded to cohort-aggregate statistical supplements, and that boundary is enforced by the Fixed Harness at the infrastructure level. The agent generates cohort-level evidence, and the human exercises judgment on what it means and what to do about it. The EU AI Act (2024) and the NIST AI Risk Management Framework codify this separation for autonomous systems in regulated domains, and the architecture described here is one way to make the separation structural instead of advisory. An agent told only to improve a metric will optimize against any boundary you leave to its goodwill, so the boundaries that hold are the ones built into the floor it stands on.

Capability	Agent Authority	Human Authority
Generate cohort-level statistical summaries	Yes	Reviews and validates
Identify aggregate trends and patterns	Yes	Interprets business implications
Evaluate individual customer records	Blocked	Exclusive authority
Emit prescriptive decisions for individuals	Blocked	Exclusive authority
Recommend actions for specific entities	Blocked	Exclusive authority
Apply findings to business decisions	Blocked	Exclusive authority

References