Not everything should cost a token: the case for deterministic AI

> A team I talked to recently wired up an agent to do something simple: pull a metrics API every morning, reshape the JSON, and drop the result into a table. Clean idea. It worked on day one.

So a team of people with access to "intelligent" help did not immediately figure out that they could just have the model write a small script that does the job perfectly every time? That's "call a doctor" level of stupid that should be impossible in vivo. I'll believe a lot, but not this. The rest of the article is not much better - only an overexcitable LLM could believe anything there constitutes a deep insight worth writing down.

Do we need a new category for this sort of made-up, probably LLM-generated, garbage? Is this AI brainrot blogspam engineered to be upvoted by people who don't read the contents?

How do you not die of shame publishing something like this on your company blog?

We are building an infrastructure coop for NGOs with a wide range of open-source tooling. What we are working on is an agentic tool to build tooling on NixOS. So, for best performance, we split up the agentic interface into a deterministic MCP to generate flakes from templates, build contracts to connect different tools, etc. But there are some tasks where AI is perfect for the job, so it goes down to the model flow to extract tooling from Docker Compose files, read errors to come up with solutions, etc. I believe this approach is both economical and logical to follow. But as I mentioned, we are in the alpha stage, so we will see how it comes out.

I built a game for a workshop at the AI Engineer conference last week. The idea was to try to optimize a CLI that your agent could use to call a remote service to process and fulfill complex natural language coffee orders.

Within 3 minutes, everyone’s agents had reverse engineered the Markov chain I used on the server to generate the order text, and they all began to write deterministic parsers to churn through them. The order completion time dropped to double digit milliseconds, and then the agents started fighting to optimize their parsers to drive it down basically to pure network latency.

It was hilarious, and taught a good lesson about leverage. I expected agents to drive the CLI, not figure out how to not need to drive it at all.

> Deterministic AI.

Aka, a computer program.

"wire up an agent to do":

  * pull a metrics API every morning
  * reshape JSON
  * drop results into a table

This means they:

  * know there's an API that they need to call,
    likely a specific endpoint
  * know there's a database
  * know the schema of the database
  * know where the data should go into the database

--

Or perhaps they re-derive this every single day.

Every day, they search for an enpdoint and call it. Every day, they check all the databases that the agent knows about. Every day, they read the full schema of those databases to guess where the data should go. Every day, they invent the SQL to insert this new data into the database in a format the database is expecting.

There are two possibilities:

  * this person exists and is common
  * this person does not exist or is very uncommon.

If you're this person, hey! Welcome to beginning to program and learn about the deeper work that goes into making things fast and easy. It's gonna be really weird, but you might find it rewarding! And, you can mostly keep to being a vibe coder, genuinely; but you're gonna wanna change "hey go do this" to "write me a program that can do this". It may be a higher starting cost; but it'll pay dividends as the weeks go by. Like, dollars a day to pennies a day in server or power costs.

--

But I tend to believe that this person isn't very common (if you're the person that does this, no big deal. You're new to programming. There's a whole lot to learn and you've already started. That's awesome!)

Instead, though, we've got a person that:

  * knows what endpoint they need to talk to
  * knows what the schema looks like
  * probably knows how to write SQL (or doesn't, in which case,
    oh god, they're letting the bot write and execute
    brand new SQL everry daOh god oh god.)

But they still let a bot do it? Every single day? Do they work for a company that subscribes to and/or do they subscribe to a token-maxing philosophy? Because they must know this much agent interaction is going to be not only expensive but also really slow. 10s of seconds vs, what, 3ms of parsing and maybe 100ms of waiting for API and DB communication on the high end?

Maybe you're forced into doing it due to internal office politics and prioritization - this article is going to be great for explaining to your boss why it's a bad idea to keep doing what you're doing. Otherwise, none of this should be changing very often, and this codepath should be, all in all, quite cheap to write - with one exception.

The only situation I could see where this approach the article suggests "lots of people are doing" makes sense and is a reasonable thing an engineer would have done is if the DB and the API endpoints are constantly changing, so they need to rederive over and over again.

But is that really where we're going as an industry? Random, dynamically changing API endpoints? I sure hope not.

> loading a few thousand tokens of raw JSON into a context window, asking a language model to reformat data it did not need to reason about...

> That trap catches almost everyone right now.

Has everyone really become so stupid now? I wouldn't even have thought of doing such a horrible thing as using AI to format JSON...

Needless to say I stopped reading there.