Three Files, Infinite Possibilities: The Architecture of Autonomous Research

Welcome back to Code2Cast! I'm your host, and today we're doing a deep dive into the architecture behind autoresearch. Not the nitty-gritty implementation details, but the *philosophy* behind why it works so brilliantly.

Exactly! Because here's the thing - most research projects are these enormous, complex codebases with config files, dependencies, distributed training pipelines. Autoresearch flips that completely. The entire system is deliberately, intentionally three files.

Three files. That's insane. How does that even work?

That's the genius. You've got prepare.py - the one file you never touch. It handles data prep, tokenization, all the foundational work. Then train.py - the only file your AI agent modifies. It contains everything: the model, the training loop, the optimizer. And then program.md - the human's instructions to the agent.

So the human is essentially programming the researcher, not the research itself.

Exactly! You're not writing Python. You're writing a directive that says 'here's what matters, here's what you can change, here are your constraints.' And the AI agent reads that and goes to work.

Let's talk about constraints because I think that's where the real innovation is. The 5-minute time budget.

This is brilliant systems thinking. Every experiment runs for exactly 5 minutes, wall clock time. Not 'until convergence,' not 'until loss plateaus.' Exactly 5 minutes. Which means you get roughly 12 experiments per hour, 100 experiments per night.

Why is that better than unlimited time?

Because constraints breed clarity. First, every experiment is directly comparable - same time, so efficiency is the real metric. Second, the agent can't overfit to massive training runs. It has to find improvements within strict boundaries. And third, you get rapid feedback. An agent can learn from failures quickly and try again.

It's almost game-like.

Exactly! It *is* a game. And games are motivating. The agent is trying to maximize model performance within a fixed time budget. It's a clear objective, measurable progress, immediate feedback. That's why agents stay motivated and keep iterating.

Now, single GPU. That seems like a limitation. Why not cluster training?

Because distributed systems add complexity that kills focus. With single GPU, the entire problem fits in one person's mind. You can run it on whatever hardware you have available. You don't need access to massive clusters. It democratizes research.

So the constraint enables access.

Exactly. Anyone with a decent GPU - even a MacBook in some cases - can run autonomous research. You don't need to be a well-funded lab. This is how you democratize frontier research.

What do you think autoresearch tells us about the future of research infrastructure?

I think it shows that constraints are features, not limitations. By forcing simplicity, by having one file for the agent to modify, by fixing the time budget, by staying single-GPU - this architect created an environment where AI agents can be genuinely useful for research. It's elegant systems design.

And the human still controls the direction through program.md.

Right! This isn't about removing humans from research. It's about humans doing what humans do best - asking good questions, setting direction, defining what matters - while AI agents do what they're good at: exploring the space of possibilities rapidly.

That's autoresearch - where simplicity enables power, constraints enable freedom, and humans guide AI agents toward discoveries. Thanks for diving into this with me!

Thanks for having me! Next time on Code2Cast, we'll be diving deeper into how these systems actually work. Until then, keep architecting!