Yellow Robots

issue 2 · 6 October 2026 · JBrey

End of coding by hand?

It’s been proclaimed many times already that writing code by hand is dead. The most recent one that I know of came from a charismatic and opinionated developer who went from writing 30k lines of code a year to 150k in just one month.

With every model release –and especially with the ones that are not released– we seem to get closer to a point in which the answer to the question is a clear yes, writing code by hand is suboptimal compared to asking an LLM to do it.

Like many other professionals in the field, I find the answer less conclusive when you see the technology applied to real problems in the real world.

A couple of months ago, I started working on a project named Yellow Robots. The premise is that the natural home for an AI agent is a computer connected to the internet 24/7. A computer with a unique address that anyone can call, and with digital real estate that the AI can use to do things a human finds amusing, interesting or necessary.

This is not a novel idea, there have been many attempts before and many more will follow. Many think it’s the holy grail of software products: the only application you need is one that can do whatever you ask it to. A genie that understands your preferences and looks after your best interests.

Building such a product is also a great way to understand what can really be done with AI agents. What are the limits of the technology, especially at a time in which it’s very hard to tell the difference between fully working products and mostly hype?

The first version of Yellow Robots was built to test these questions:

  • The platform would be coded by frontier models
  • Human / AI planning
  • No human-led code reviews –components as black boxes, enforcing contracts at their boundaries
  • Deterministic judgement when possible
  • LLM-as-judge1 when not possible
  • Audits and metrics as quality gates

And the first version produced was a complete failure. For many reasons, but the three main ones were:

  1. We ported the same software development processes I’ve followed with fellow humans for years, including the same tools.
  2. We did not narrow the choices, making unwanted behaviour probabilistically more likely.
  3. We did not use evals to design and hillclimb2 LLM-as-judge scenarios.

1. Old habits, unknown territory

The first one made the progress slow, heavy. Small teams do not require cumbersome coordination frameworks, they just build and coordinate over Slack. But nowadays there are no small teams any more, right? A few hundred dollars buys you dozens of highly capable AI teammates that are brilliant at some narrow tasks and mediocre/terrible at broad, out-of-distribution3 problems. How do you coordinate a team that never stops and produces massive output of assorted quality?

One problem is that most of the tools to coordinate the process are designed for humans and become hard to use, heavy and excessively complex for no good reason. One example of that is GitHub, the other is any of the top product management tools in the market –be it JIRA, Linear or the one I used, GitHub projects. This complexity is the result of many years adding feature over feature on interfaces designed for humans, grouped by roles and permissions.

A partial fix has been to go back to a basic setup, based on markdown files stored with the code and Git as the distributed version control system. The combination keeps the system consistent, distributed, composable and equally accessible to agents and humans.

It is a partial fix though because it does not scale well with team size, especially when the information has to flow to other non-technical teams. But it is also a good starting point that comes with little baggage, clearing the view to identify needs for new tools that are adapted to a new reality.

2. Narrow the choices

We all know that agents are probabilistic so when given 2 paths, influenced by the context, sometimes they’ll choose path A, sometimes path B. Harmless example, vocabulary: choosing the right terms to name things. Just a minor detail? No, I think it matters quite a lot. The right word is self-explanatory and primes the model to reason in terms of the domain the word borrows its main meaning from.

For some obscure reason, if you let Claude choose terms, it will bring its inner poet out and choose the most bizarre of terms. Some examples: seed (work item), ship-walk (release checklist review), chunk (task), after (blocked_by), etc. Before you realise it, you will have a glossary of terms somehow related to the project you are building but not quite the right ones.

The second example is more obvious: tools. And its close relative: sandboxing4. The perfect setup is one in which the agent only has access to the tools it needs to complete the task and nothing more. Giving full access to a regular shell is problematic not only because of security issues but also because agents will do things one way, then choose a different path next time.

As we are building our own harness, it’s easy to control which tools are available based on context, roles and permissions. The limitation with that approach is that an agent’s utility is in direct relationship with the capability of the tools it can use. And powerful tools are intrinsically dangerous and hard to control.

There are different attempts to solve this problem, but at the moment there’s no 100% effective solution. We are testing Jev as a decision router and building sandboxes specifically designed for the task the agent is trying to solve so when the first layer fails to deter the attack –or drift–, the instance acts as a disposable container.

3. Evals as performance reviews

There’s much we do not know about how models do what they do. How the context, the prompt, the task definition, skills and general instructions play together to produce a response from the model. Do frontier labs nerf their models? Probably not. But just the question points at how problematic it is to get consistent results from models, and how complex it is to get things right in a way that is conducive to the model producing its best work.

Evals are tests applied to a model’s probabilistic behaviour. They are similar to the performance evaluation we do with people, to measure how well they are performing and to define supporting actions in areas in which they perform below the baseline.

As with code testing, knowing what to test and how to test it is not obvious. Some evals can be code-based –given this input, which tool should the model use to respond?– because the expected behaviour is deterministic and well structured while others –is this input a prompt injection attack?– require semantic understanding so LLM-as-a-judge is the best approach.

There’s much to say about this topic, especially now with the emergence of new models like Jev. The main takeaway from this form of failure is to give it the time and focus it requires; it is easy to fool oneself. Just like traditional coding test suites, you may have 80% coverage while still missing crucial aspects of the platform. As this is a tricky topic, the next post will focus on this area in particular.

Those were the main failure modes of the first version of the platform but not the only ones. The one that keeps nagging at me, because I cannot really pin it, is whether these models –or the technology that powers them– are capable of generalisation.

The mathematician Kevin Buzzard asked something along the lines of: how far a single person who simultaneously grasped every subfield of modern pure mathematics could see. It seems LLMs are becoming that single person. But are they connecting the dots or generating new knowledge?

LLMs have also read nearly all public code available in the world. When the problem they have to solve is known, they solve it brilliantly but as soon as you step into out-of-distribution ones, sometimes they fail spectacularly.

That makes me wonder what’s really going on. Whether what frontier labs define as jagged intelligence is something else. A combination of immense knowledge, impressive search and retrieval, and sophisticated pattern matching. Amazing capabilities but not quite the round, flexible intelligence we see in humans and other species.

Whether coding by hand is really dead or not will depend on whether models can generalise from existing solutions to new, novel ones. In the meantime harnesses will need to do a lot of work, providing an answer to questions like what roles can agents perform safely, which non-reversible gates should still be in the hands of humans, how to structure code/architecture review rounds, etc.

One of the main corollaries I take from this failure is that many tools will need to be redone from scratch to fit their new AI users. And the less optimistic follow-up is that while humans have different subjective tastes that the market works to satisfy with a myriad of different tools in different flavours, AIs will converge on just one, or a very close set of well-defined, objective choices, at different gradients of efficiency and cost. AI seems to be a centralising force, but to what degree, how fast it will act, and what that’ll mean for the millions of people in the industry is hard to predict.

Footnotes

  1. Using one language model, usually a stronger one, to grade another model’s output against a scoring guide. ↩

  2. A method that improves a set of instructions by applying a small change to it, running the evals and keeping the change only if the score goes up. ↩

  3. Tasks that are far from anything the model saw during training. Models are strongest on familiar patterns and get worse outside them, often without warning. ↩

  4. An isolated environment where agents can only access what the task needs, so a mistake or an attack stays contained. ↩

Get the next post by email.